Announcement

Collapse
No announcement yet.

Upload text files to web server using gzip compression?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Upload text files to web server using gzip compression?

    I often need to upload and download large text files to my web server. Uploading the raw text files takes a really long time, so I currently compress one or more text files into a ZIP archive, then upload that to the server and decompress back to the original files on the server using PHP. That works, but it requires some extra steps and I get occasional random errors from the PHP decompression (Still trying to track that down).

    Is it possible to upload/download text files using on-the-fly gzip compression (preferably in memory) the way browsers do with web pages? Anyone have an example in Power BASIC?
    Anthony Watson, Mountain Software
    www.mountainsoftware.com

  • #2
    Something I did with a pho webserver.
    We have faxes come in the server and it creates tif files.
    A copy old tgevtif file goes to web server page.
    I weite a php script that looks for all the tif files and converts them to PDF files and at the end of the php scripts, it sends the browser back to the webpage location wity the files sorted by date. The script runs fast even on multiplagevtif files that might be 100 pages long.
    I use wger to up load some files to some php scripts as well that handles all the username and passwords.
    The server is not worked hard.
    By a similar method. You could leave your files in a compress form as well for easy down loading.
    I am not aware of any browser/server compression. But usually browsers and servers can be adjusted in various setting that allow for speedy transfers.
    2 of these severs are old windows xp servers on chest as you can get laptops
    p purvis

    Comment


    • #3
      I don't mind sharing the php script.as an attached file.
      p purvis

      Comment


      • #4
        Anthony, here's a general tip that helped me over the years: if you are looking to solve a specific task in PB, but can't find some source code samples, try searching for VB Classic (aka "VB6" or "VBA") code, Most of the time there's something out there. Given the similarity, it often is possible to convert it easily to PB and tweak it to take advantage of PB's capabilities.

        In this specfific case, searching for "vba gzip" turned up http://www.freevbcode.com/ShowCode.asp?ID=966 amongst others. Which in turn tells you that it's based on the https://www.zlib.net/ library.

        Comment


        • #5
          Originally posted by Knuth Konrad View Post
          Anthony, here's a general tip that helped me over the years: if you are looking to solve a specific task in PB, but can't find some source code samples, try searching for VB Classic (aka "VB6" or "VBA") code, Most of the time there's something out there. Given the similarity, it often is possible to convert it easily to PB and tweak it to take advantage of PB's capabilities.

          In this specfific case, searching for "vba gzip" turned up http://www.freevbcode.com/ShowCode.asp?ID=966 amongst others. Which in turn tells you that it's based on the https://www.zlib.net/ library.
          I don't think that works for the original question about gzip

          Comment


          • #6
            Don Dickinson's ZLib wrapper (from 2001) has the necessary gzopen/gzread/gzwrite/gzclose declarations and gzcompress/gzdecompress functions:

            https://forum.powerbasic.com/forum/u...2-zlib-wrapper

            Comment


            • #7
              Originally posted by Paul Purvis View Post
              ..
              I am not aware of any browser/server compression. But usually browsers and servers can be adjusted in various setting that allow for speedy transfers.
              2 of these severs are old windows xp servers on chest as you can get laptops
              All modern browsers can request and decompress gzip content on the fly.
              All good web servers can compress them in response to that request..

              Your browser should be sending a Header with the URL request, something like: Accept-Encoding: gzip, deflate.


              You can configure a web server to compress pages when they get that Header in several ways. For Apache, the easiest way is with a .htaccess file.
              You can also get PHP pages to do it internally with:
              <?php if (substr_count($_SERVER[‘HTTP_ACCEPT_ENCODING’], ‘gzip’)) ob_start(“ob_gzhandler”); else ob_start(); ?>

              Every time you Google something, you will be getting the results sent to you compressed, Same with Facebook etc.

              Added: Just checked in Firefox Developer Tools, this page is 155KB, but it was sent gzipped so only 22KB of data were transferred.

              Comment


              • #8
                Originally posted by Stuart McLachlan View Post

                I don't think that works for the original question about gzip
                And I don't think you bothered to look at zlib's side.

                zlib was written by Jean-loup Gailly (compression) and Mark Adler (decompression). Jean-loup is also the primary author of gzip
                And from gzip's homepage:
                gzip also refers to the associated compressed data format used by the utility.
                And that (=the data format) is what HTTP compression is about:
                The most common compression schemes include gzip and Deflate
                Code:
                HTTP/1.1 200 OK
                Date: mon, 26 June 2016 22:38:34 GMT
                Server: Apache/1.3.3.7 (Unix)  (Red-Hat/Linux)
                Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
                Accept-Ranges: bytes
                Content-Length: 438
                Connection: close
                Content-Type: text/html; charset=UTF-8
                Content-Encoding: gzip
                Content-Encoding tokens: gzip – GNU zip format (described in RFC 1952). Uses the deflate algorithm for compression, but the data format and the checksum algorithm differ from the "deflate" content-encoding. This method is the most broadly supported as of March 2011.

                Comment


                • #9
                  Originally posted by Knuth Konrad View Post

                  And I don't think you bothered to look at zlib's side.
                  .
                  I didn't need to look at the "zlib's side".

                  I am quite familiar with that DLL and its capabilities - I use it frequently, including with Don Dickinsoon's wrapper, which I linked to in my subsequent post, to implement gzip..

                  Neither the zlib site site nor the freevbcode you linked to provide a simple answer to the OP's question. The freevbcode doesn't use gzip at all and the zlib site doesn't contain any relevant code samples either.That's why I provided the link to Don Dickinson's wrapper in my subsequent post

                  Maybe a slight misunderstanding. When I said "I don't think that works for the original question about gzip", I was referring to the freevbcode, not zlib.dll itself

                  Comment


                  • #10
                    Ah, I see.

                    I guess we both mised each other's point, hence the miscommunication. Remember, I started with a recommendation that helped me over time: search for equivalent VB6/VBA code, if no PB code example seems to be available. And I demonstrated what the results in this case are.

                    The tasks at hand are:
                    1) Identify what "gzip" means in the context of HTTP communication. The answer: in that context it refers to a compression scheme
                    2) Look for a) source code or b) library (preferably a widely used one), which does that. The answer to this: zlib.dll
                    3) If 2) comes up with b), search for source code that uses b). The answer to that: the FreeVBCode snippet.*)
                    4) Convert to PB and adapt code to your needs.

                    *) And that's where I implied, but didn't mention "let's pretend there's no PB source code example available", which in this case fortunately is, causing our "disagreement".

                    Comment


                    • #11
                      I did not know about the text compression of some web server code or web browsers.
                      After viewing some large text files over the web some years ago, it seems i am not the only misinformed.
                      Some of my text files grow constantly like active log files. I will have to look into this later.
                      But for static text files
                      Their is good reason to keep a compressed file of the text file and maybe even a chopped text file to preview text large files.
                      p purvis

                      Comment


                      • #12
                        After a bit of experimentation, I discovered larger Zip files were failing during decompression because the process timed out. Apparently the server aborts the PHP script when the process is taking too long. The solution was to upload the files in a single ZIP file as I had been, but extract the files in the Zip one at a time, instead of trying to unZip the entire archive at once.

                        This solves the problem I was having so I don't need on-the-fly gzip compression after all. Thanks for the input!
                        Anthony Watson, Mountain Software
                        www.mountainsoftware.com

                        Comment


                        • #13
                          create a phpinfo.php file to view your current php settings
                          an output from phpinfo.php on one of my servers
                          Attached Files
                          p purvis

                          Comment

                          Working...
                          X