Announcement

Collapse
No announcement yet.

Upload client

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Upload client

    I am currently writing an web upload client and I have run into a problem and need some help. The basic client works with text files no problem. But when I try word files (.doc) I only get about 6 bytes of the file. This due to the fact that there is (1A character) End of file marker in the header of the file.

    The problem is in STDIN LINE Temp, it does not handle binary files. It stops
    reading at the End of file marker.

    What I need is a way to read binary files from stdin that ignores the end of file marker.


    This is the code

    '------------------------------------------------------------------
    ' This the web based document manager version 1.0
    ' by M C Draper Advanced Business Concepts
    '
    ' Changes 14/3/2009 Initial Version
    '------------------------------------------------------------------

    #COMPILE EXE
    #DIM ALL

    $INCLUDE "\pbcc40\file.inc"
    $INCLUDE "\pbcc40\PBCGI.INC"
    $INCLUDE "\pbcc40\CGIlib.inc"

    SUB LoadPage(HtmlPage AS STRING)
    '
    ' thie routine will check if the files exits, then open and read the file
    ' and output throught cgi to the client
    '
    DIM filehandle AS LONG
    DIM Buffer AS STRING
    '
    IF FileExists(HtmlPage) = 0 THEN EXIT SUB
    '
    ' Open the file
    Filehandle = FREEFILE
    OPEN Htmlpage FOR INPUT AS Filehandle
    '
    ' read the contents of the file
    WHILE NOT EOF(FileHandle)
    LINE INPUT# Filehandle, Buffer
    WriteCGI buffer
    WEND

    CLOSE FileHandle
    END SUB

    SUB ListFiles
    LoadPage("menupage.html")
    END SUB

    SUB UploadFile
    ON ERROR RESUME NEXT

    LOCAL Temp AS STRING
    LOCAL Buffer AS STRING
    LOCAL Boundary AS STRING
    LOCAL Content AS STRING
    LOCAL Context AS STRING
    LOCAL Filename AS STRING
    LOCAL FileNameStart AS INTEGER
    LOCAL File AS STRING

    LOCAL Filehandle AS LONG

    STDIN LINE Boundary
    writecgi "Boundary = "+Boundary + "<br>"

    STDIN LINE Content
    writecgi "Content = "+Content + "<br>"

    STDIN LINE Context
    writecgi "Context = "+Context + "<br>"


    Buffer = ""

    WHILE INSTR(Temp,Boundary) = 0
    STDIN LINE Temp
    '
    ' don't add the boundaty to the file
    IF INSTR(Temp,Boundary) = 0 THEN Buffer = Buffer + Temp + $CRLF
    'writecgi temp
    WEND

    writecgi "File = " + Buffer + "<br>"
    FileNameStart = INSTR(Content,"filename=") + 10

    FileName = MID$(Content,FileNameStart,LEN(Content) - FileNameStart)
    writecgi Filename
    File = RIGHT$(Filename,LEN(filename) - INSTR(-1,Filename,"\"))
    writecgi ">"+file+"<"
    '
    '
    ' Save the file to disk
    ' You cam savein the web directory as it is write protected. You havew to a separte directory
    ' for the uploaded files.
    '
    Filehandle = FREEFILE
    OPEN "c:\" +File FOR OUTPUT AS Filehandle
    PRINT# Filehandle, Buffer
    CLOSE Filehandle
    writecgi "error = " + STR$(ERR)

    END SUB

    FUNCTION PBMAIN () AS LONG

    DIM UserLine AS STRING


    IF COMMAND$ = "ListFiles" THEN ListFiles:EXIT FUNCTION
    IF COMMAND$ = "UploadFile" THEN UploadFile:EXIT FUNCTION

    END FUNCTION
    Attached Files

  • #2
    Code:
    Is this 1 program?  
    Would the cgi program read the file?
    If so, then within the cgi program:
     
      hFile = FREEFILE
      DO
        ERRCLEAR
        OPEN sFileName FOR BINARY LOCK WRITE AS #hFile
        IF ERR THEN WriteCGI  "<LI>Error opening "  + STR$(ERR)
        INCR attempt
      LOOP UNTIL ERR = 0 OR attempt > 10  'don't write if somebody else is at same time.
      IF attempt > 10 THEN
         WriteCGI "<LI>Site too busy, please try later"
         EXIT FUNCTION
      END IF
      GET$ #hFile, LOF(hFile), sBuffer
      CLOSE #hFile
      'Modify sBuffer and write out new file
    END FUNCTION

    Comment


    • #3
      Thanks Mike but that is not the problem.

      The upload from the web client works just fine which is evidenced by the fact that a text file does not have a problem. The problem comes when I upload a .doc file which contains a EOF characters '1A'. The problem, I think, is the 'STDIN LINE' function, it stops when it reads the EOF character.

      When uploading a binary file you have to ignore the characters and continue till you get to the physical end of the file. Just same way you read a binary file but from the stdin.

      I have tried reading by ignoring stdineof and kept reading until I get the second boundary at the end of the file. That did not work and resulted in loop.
      Last edited by Martin Draper; 14 Mar 2009, 07:15 PM.

      Comment


      • #4
        STDIN is for reading text from the standard input device..
        http://www.powerbasic.com/support/pb...ad.php?t=24345

        Plus, you do not need to read the file and then use writecgi with text files.
        Just point browser to the file with STDOUT "LOCATION: " + URL + $CRLF
        Last edited by Mike Doty; 14 Mar 2009, 08:24 PM.

        Comment


        • #5
          I had a thought, when I looked up the STDIN LINE function there was also a function to get the handle for stdin which I read could be used to to treat the stdin as file stream using open handle.

          Sp I changed the code to use this code to read
          Filehandle = FREEFILE
          OPEN HANDLE GETSTDIN FOR INPUT AS Filehandle LEN = 4096

          WHILE NOT EOF(Filehandle)
          LINE INPUT #Filehandle, temp
          writecgi temp
          Buffer = Buffer + Temp
          WEND
          Now this works fine with text files but does not work with the .doc files due to the EOF markers in the file. So I wrote a simple program to test reading a .doc file as a binary file and this works just fine.

          Filehandle = FREEFILE
          OPEN "C:\healthjob.doc" FOR BINARY AS Filehandle

          WHILE NOT EOF(Filehandle)
          GET$ #Filehandle,256, Temp
          Buffer = Buffer + Temp
          WEND
          90 CLOSE Filehandle
          So I modified the getstdin function above use the binary file mode the same way as the test program.

          [CODE]
          Filehandle = FREEFILE
          OPEN HANDLE GETSTDIN FOR Binary AS Filehandle

          WHILE NOT EOF(Filehandle)
          GET$ #Filehandle,256, Temp
          writecgi temp
          Buffer = Buffer + Temp
          WEND
          [/QUOTE]

          Now this should work but NOPE it just sits there. I must be missing something
          any ideas PLEASE !
          Last edited by Martin Draper; 17 Mar 2009, 02:13 AM.

          Comment


          • #6
            .DOC files can not be treated as text files. They have fonts, etc.
            Any binary character could be at any position.

            If you want to display a .DOC file to a browser, point the browser
            to the .doc file using the LOCATION statement.

            Plus, the browser does not know what type of file is in memory. Reading 256 characters of a binary file and adding $CRLF will not work.
            To display .DOC files requires that the client has software associated to the .DOC extension and you point the browser to the file.
            Last edited by Mike Doty; 14 Mar 2009, 08:39 PM.

            Comment


            • #7
              Mile you are missing the point, I am trying to upload a .doc file to a web based server to be stored which happens everyday with php, perl, vb and every other web language.

              With the common gateway interface there are three ways data is sent to a web server GET, POST and Command line interface. The command line interface is out because you are limited to command line buffer size which 256characters (depending on the web server).

              With GET method you are limited to the maxsize of the QueryString which is no where big enough to upload files. That only leaves the STDIN which all the webservers used to to transfer files. So by defination the STDIN has to be able handle any data so it has to be able to handle Binary.

              GETSTDIN allows you to get the current handle for the standard input and use
              the open handle to read the data from the stream. The open handle allows the stream to be opened as binary to read binary data which is what I am currently trying and it can't get to work.

              I am not trying to display the a doc file, the writecgi is for diagnositic purposes to see what is happening. I an try to get file to server so it can be cataloged and stored as part of web document mangement system.

              Comment


              • #8
                Code:
                Filehandle = FREEFILE
                OPEN "C:\healthjob.doc" FOR BINARY AS Filehandle
                 
                WHILE NOT EOF(Filehandle)
                   GET$ #Filehandle,256, Temp
                    Buffer = Buffer + Temp
                WEND
                90 CLOSE Filehandle
                A cgi program only has access to files on the server.
                This is attempting to read a file on the client.

                Maybe someone has some javascript to send files. http://javascript.internet.com/forms/upload-filter.html
                Does this look like what you are trying to do?
                The cgi side of this javascript to write to the server should do it. Can't work on this right now. Hope this helped.
                Last edited by Mike Doty; 15 Mar 2009, 09:52 AM.

                Comment


                • #9
                  This could be used as a template to send to a cgi program:
                  Reposted here so you get an email message.
                  http://javascript.internet.com/forms/upload-filter.html

                  Comment


                  • #10
                    You can try reading STDIN as a regular disk file with something like..
                    Code:
                      hSys = GETSTDIN 
                      hFile = FREEFILE 
                      OPEN HANDLE hSys FOR BINARY AS hFIle 
                      nBYte = LOF (hFile) 
                      Buff$   =  STRING$(nByte, $NUL) 
                      GET$      hFile, BUff$
                      CLOSE     hFile
                    If STDIN is complete (that is, the LOF() is the complete byte count), that will get you the file data as a string and you may do what you will with it.

                    I don't know how STDIN and STDOUT work well enough to know exactly how the "size" of STDIN is determined, but this has got to be a head start of sorts.

                    Another thought...

                    If that LOF() thing does not work out, you can get the data a byte at a time until you hit (logical) end-of-file
                    Code:
                      hSys = GETSTDIN 
                      hFile = FREEFILE 
                      OPEN HANDLE hSys FOR BINARY AS hFIle 
                      DO 
                          GET    hFile, , BYteVar?
                          IF ISFALSE ERR THEN            ' did not hit EOF 
                                Buff  =  Buff & CHR$(ByteVar)
                          ELSE
                                EXIT DO 
                          END IF
                      LOOP 
                      CLOSE     hFile




                    MCM
                    Last edited by Michael Mattias; 15 Mar 2009, 10:18 AM.
                    Michael Mattias
                    Tal Systems (retired)
                    Port Washington WI USA
                    [email protected]
                    http://www.talsystems.com

                    Comment


                    • #11
                      this is what I use:

                      HTML
                      Code:
                      <html>
                      <body>
                      
                      	<form enctype="multipart/form-data" 
                      		    method="post" 
                      		    name="uFile" 
                      		    action="http://www.mydomain.com/cgi/uploadDoc.exe">
                      		    
                      	      DOC file : <input type="FILE" name="docFile" id="docFile" size="40"><p>
                      	
                      				<input type="submit">
                      	</form>  
                      	
                      </body>	
                      </html>
                      and the PB source:

                      Code:
                      ' upload a word document from a web page
                      
                      #Include "win32api.inc"     
                      Declare Function webdataStdIn() As String         
                      Declare Sub WriteCGI (ByVal st As String)          
                      
                      Function PbMain()    
                          Local fNumber As Long     
                          Local fName As String
                          Local sParams As String    
                          Local inFile As String  
                          Local outImage As String  
                          Local tempBuf As String  
                          LOCAL filen AS STRING    
                          Local errMsg As String       
                          Local fileID As String   
                          Local sp1,sp2 As Long       
                        
                        ' error trap  
                          On Error GoTo errOut                 
                          
                        ' write the start of the returning html  
                          writeCGI "<html><body>"    
                          
                        ' get file from stdin                
                          sParams=webdataStdIn()                     
                       
                        ' make sure it is a word document  
                          If InStr(sParams,"Content-Type: application/msword")=0 Then   
                              errMsg="error: not a word document"
                              GoTo errOut
                          End If
                          
                        ' the actual data of the file is the remainder of the file after /msword + 4 more characters
                          inFile=Trim$(Remain$(sParams,"Content-Type: application/msword"))   
                          inFile=Right$(infile,Len(infile)-4)  
                          
                         ' get the actual file name       
                          sp1=InStr(sParams,"fileID")+7
                          sp2=InStr(sp1,sParams,"-")
                          fileID=Trim$(Mid$(sParams,sp1,sp2-sp1))    
                          Replace Chr$(0) With "" In fileID
                          Replace Chr$(10) With "" In fileID
                          Replace Chr$(13) With "" In fileID
                         
                         ' write the image to local disk
                           fNumber = FreeFile      
                           outImage="edocUploads\myDoc.doc"
                           Open outImage For Binary As #fNumber
                           Put$ #fNumber, inFile
                           Close #fNumber           
                           
                         ' give user response
                           writeCGI "done"   
                           writeCGI "</body></html>"   
                           
                      Exit Function
                          errOut:
                           writeCGI errMsg
                           writeCGI "</body></html>"     
                           
                      End Function
                      
                      ' use for binary files             
                      ' I think this is by Dave N and Don D.
                      Function webdataStdIn() As String
                         Dim hInput As Long
                         Dim iToRead As Long
                         Dim iRead As Long
                         Dim iResult As Long
                         Dim sBuffer As String
                         Dim sOutBuffer As String       
                      
                         iToRead = Val(Environ$("CONTENT_LENGTH"))
                         hInput = GetStdHandle(%STD_INPUT_HANDLE)
                         If hInput Then
                            Do
                               If iToRead > 32000 Then
                                  sBuffer = Space$(32000)
                               ElseIf iToRead = 0 Then
                                  Exit Do
                               Else
                                  sBuffer = Space$(iToRead)
                               End If
                      
                               iResult = ReadFile(hInput, ByVal StrPtr(sBuffer), _
                                                  Len(sBuffer), iRead, ByVal %Null)
                      
                               ' If there was an error, return nothing
                               If iResult = 0 Then
                                  Exit Do
                      
                               ' We're done if iRead is 0
                               ElseIf iRead < 1 Then
                                  Exit Do
                      
                               ' Otherwise, accumulate the buffer
                               Else
                                  sOutBuffer = sOutBuffer + Left$(sBuffer, iRead)
                                  If Len(sOutBuffer) >= iToRead Then Exit Do
                      
                               End If
                            Loop
                         End If
                      
                         Function = sOutBuffer
                      
                      End Function
                      
                      
                      '------------------------------------------------------------------------------       
                      ' taken from pbcgi.inc
                      ' WriteCGI writes an HTML string to the web server. It automatically takes care
                      ' of the necessary header.       '
                      Sub WriteCGI (ByVal st As String) 
                          Static header As Long      
                          If IsFalse header Then
                              STDOUT "Content-type: text/html"
                              STDOUT
                              header = -1
                          End If  
                          Replace "''" With $Dq In st 
                          STDOUT st     
                      End Sub
                      
                      '------------------------------------------------------------------------------
                      ' CgiParam parses raw CGI data to return the parameter you specify. The target
                      ' parameter name is not case-sensitive.
                      '
                      Function CgiParam (ByVal sParmList As String, ByVal sTarget As String) As String
                      
                          Local ix    As Long
                          Local sParm As String
                      
                          sTarget = UCase$(sTarget) + "="
                      
                          For ix = 1 To ParseCount(sParmList, "&")
                              sParm = Parse$(sParmList, "&", ix)
                              If UCase$(Left$(sParm, Len(sTarget))) = sTarget Then
                                  Function = DecodeCGI(Mid$(sParm, Len(sTarget) + 1))
                                  Exit For
                              End If
                          Next
                      
                      End Function
                      
                      '------------------------------------------------------------------------------
                      ' DecodeCGI decodes the special characters in a CGI string.
                      '
                      Function DecodeCGI (ByVal sInput As String) As String
                      
                          Local pbInput   As Byte Ptr
                          Local pbOutput  As Byte Ptr
                          Local ncbInput  As Long
                          Local ncbOutput As Long
                          Local ncHex     As Long
                      
                          ncbInput = Len(sInput)
                          If ncbInput = 0 Then Exit Function
                      
                          pbInput = StrPtr(sInput)
                          pbOutput = pbInput
                      
                          Do
                              Select Case Const @pbInput
                      
                              Case 37   ' "&"
                                  Incr pbInput
                                  Decr ncbInput
                                  ncHex = Min(2, ncbInput)
                                  If ncHex Then
                                      @pbOutput = Val("&H" + Peek$(pbInput, ncHex))
                                      pbInput = pbInput + ncHex
                                      ncbInput = ncbInput - ncHex
                                  End If
                      
                              Case 43   ' "+"
                                  @pbOutput = 32   ' $SPC
                                  Incr pbInput
                                  Decr ncbInput
                      
                              Case Else
                                  @pbOutput = @pbInput
                                  Incr pbInput
                                  Decr ncbInput
                              End Select
                      
                              Incr pbOutput
                              Incr ncbOutput
                      
                          Loop While ncbInput
                      
                          Function = Left$(sInput, ncbOutput)
                      
                      End Function
                      
                      Macro cgiDecode = DecodeCGI
                      Last edited by Shawn Anderson; 16 Mar 2009, 06:16 PM.

                      Comment


                      • #12
                        First of all, please edit your prior posts' code tags. You have a couple up there where you open a "[ code]" section and end it with an "end quote" "[ /quote]" tag.

                        I could not find some of the stuff you had already tried, which included the OPEN HANDLE of STDIN.

                        Now this code here:
                        Code:
                        Filehandle = FREEFILE
                        OPEN HANDLE GETSTDIN FOR Binary AS Filehandle 
                        WHILE NOT EOF(Filehandle)
                           GET$ #Filehandle,256, Temp 
                           writecgi temp
                           Buffer = Buffer + Temp
                        WEND
                        .. if it just sits there, it's because EOF() is not returning TRUE. EOF() is funky when files opened FOR BINARY are involved: it does not return TRUE until you actually read PAST the EOF.

                        FWIW, the GET$() function may also be sensitive to 0x1A (EOF marker), as GET$ is a "text" function.

                        Try the byte-at-a-time code "as byte" I suggested above and testing for EOF after the GET and see what happens.


                        MCM
                        Michael Mattias
                        Tal Systems (retired)
                        Port Washington WI USA
                        [email protected]
                        http://www.talsystems.com

                        Comment


                        • #13
                          Nice job, Shawn.
                          I was thinking about embedding some javascript, your method is better.

                          Comment


                          • #14
                            Or,mayb GET$() with a byte count of 256 is waiting...

                            PB/WIn 9.01 help:
                            Remarks
                            GET$ reads Count& characters from file number filenum&, and assigns them to sVar$. File filenum& must have been opened in binary mode. Characters are read starting at the current file pointer position, which can be set with the SEEK statement. When the file is first opened, the pointer is at the beginning of the file (unless the LEN clause is used in the corresponding OPEN statement, position 1 is used by default). After GET$, the file pointer position will have been advanced by Count& bytes.
                            Nowhere on this help page does it say what happens if there are less than Count& bytes available from current SEEK() to EOF.

                            I shall write to support to find out. Maybe it's waiting until 256 bytes are available.

                            MCM
                            Michael Mattias
                            Tal Systems (retired)
                            Port Washington WI USA
                            [email protected]
                            http://www.talsystems.com

                            Comment


                            • #15
                              Michael,
                              That code is incorrect. The server would have to be reading a file from itself.

                              Comment


                              • #16
                                Hi Michael,
                                I tried the lof and it returns 0 for opening the file "for Binary" or "For Input".

                                I tried your second idea and it does not read any data. The len of the
                                data being return is always zero. To make things simplier I tested the upload client with a simple text file ( The Source code this the client)

                                Filehandle = FREEFILE
                                OPEN HANDLE GETSTDIN FOR BINARY AS Filehandle
                                DO
                                GET$ #Filehandle,256, Temp
                                Buffer = Buffer + Temp
                                writecgi STR$(LEN(temp))
                                IF ISFALSE ERR THEN EXIT DO
                                LOOP
                                I trapped the error in this case and it returns error 52.

                                I changed the code to use the handle as input instead of binary and it works but it does not find the end of file

                                Filehandle = FREEFILE
                                OPEN HANDLE GETSTDIN FOR INPUT AS Filehandle

                                DO
                                'GET$ #Filehandle,256, Temp
                                LINE INPUT #Filehandle, temp
                                Buffer = Buffer + Temp
                                writecgi STR$(LEN(temp))
                                writecgi "error = "+STR$(ERR)
                                IF ISTRUE ERR THEN EXIT DO
                                writecgi temp +"<br>"
                                'IF INSTR(temp,Boundary) > 0 THEN EXIT LOOP
                                LOOP
                                Last edited by Martin Draper; 17 Mar 2009, 03:17 AM.

                                Comment


                                • #17
                                  You sure the OPEN ...
                                  Code:
                                  OPEN HANDLE GETSTDIN FOR BINARY AS Filehandle
                                  .. is succeeding? No, you are not because you are not testing ERR.

                                  Oops, need to test that myself to see if that bug was fixed (OPEN HANDLE was not returning ERR on failure). Don't see an entry in history.txt file.
                                  Michael Mattias
                                  Tal Systems (retired)
                                  Port Washington WI USA
                                  [email protected]
                                  http://www.talsystems.com

                                  Comment


                                  • #18
                                    Well, the GET$ thing was a thought but...
                                    When you specify a count parameter in the GET$ statement that is greater
                                    than the remaining number of bytes in the file, only the remaining bytes
                                    are read and copied to the string parameter. I have logged a request
                                    that this be added to the help file.
                                    MCM
                                    Michael Mattias
                                    Tal Systems (retired)
                                    Port Washington WI USA
                                    [email protected]
                                    http://www.talsystems.com

                                    Comment


                                    • #19
                                      Michael,
                                      I changed the code to test err on opening the handle and it fails.

                                      Filehandle = FREEFILE
                                      OPEN HANDLE GETSTDIN FOR BINARY AS Filehandle
                                      IF ISTRUE ERR THEN writecgi " Open Failed": EXIT SUB
                                      It Looks like that bug has not been fixed in 4.03 of PBCC40.

                                      Comment


                                      • #20
                                        If you got a non-zero ERR, that means it *is* fixed.

                                        Now, why it won't open, that's a good question.

                                        Hmm... trying to open STDIN in an output-capable mode (the default for FOR BINARY) can't be right.

                                        It can't hurt to try opening it 'FOR BINARY ACCESS READ' and maybe LOCK SHARED, too.

                                        MCM
                                        Michael Mattias
                                        Tal Systems (retired)
                                        Port Washington WI USA
                                        [email protected]
                                        http://www.talsystems.com

                                        Comment

                                        Working...
                                        X