Upload client

[Martin Draper]

Hi,
I have tried opening the handle with 'FOR BINARY ACCESS READ' and with
LOCK SHARED and still no good. The boys have fixed the err but they have not fixed the problem. You have to be able to read binary data from stdin
when you upload binary files from a browser. If it is not allowed it means you can't upload gif's, jpg's, pdf's, word, excel and other files.

What is needed is an input function that reads until the end of the data from stdin from the current position in the data. This way you could use "line input" to read the text headers from the file then "binary input" to the read the rest of the data.

So I going to try the webdataStdIn() code by Shawn.

[Mike Doty]

That is the way it is done.

[Michael Mattias]

. You have to be able to read binary data from stdin
when you upload binary files from a browser. If it is not allowed it means you can't upload gif's, jpg's, pdf's, word, excel and other files.

The is no way LINE INPUT or STDIN LINE can be used to read non-text data. Period. Those functions were designed to treat CRLF as 'end of record' and ^Z as end of file marker.

For whatever it's worth, once you get a fileNAME (from STDIN or from anywhere else) , you can always upload it, either by specifying the name in some CGI/HTML code, or by using InternetOpen() and InternetWriteFile().

(Do applications really send non-text data to other applications via STDIN? That sounds wrong.)

MCM

[Mike Doty]

More than a file name is required.
CGI does not have access to files on the client.
Notice that Shawns code requires html running on the client to send the file to the cgi running on the server.

[Martin Draper]

Thanks Shawn your webdataStdIn() works beautifully.
It took me a while to get my head around it but now I understand. The upload is a chunked transfer the same as many web servers use for downloading. I does not return equal size blocks so you can't look for a block smaller than buffer size as last block. You have to know the size of the transfer or a buffer size of zero or error to indicate the end of the transfer.

The new version works with all file types and there seems to be a limit of a about one 1M which I think is to do with a limit in the browser or a timeout.
Which is a problem for another day.

The routine reads the data from the browser through the current console handle which is the STDIN (hInput = GetStdHandle(%STD_INPUT_HANDLE).

The problem is that Powerbasic does not do it because it has not been written to do so. Same problem with OPEN HANDLE GETSTDIN FOR BINARY AS Filehandle.

Future feature Please !!!

[Marco Pontello]

The new version works with all file types and there seems to be a limit of a about one 1M which I think is to do with a limit in the browser or a timeout.

That may depend on the web server also. Apache, for example, usually default to a limit of 4MB for a multipart / form upload.

Bye!

[Mike Doty]

cgiUpload to get around limitations

Here is another way to upload files.

If you can get clients to download a program then browser limitations can be eliminated.
The program they download can act as a TCP server and this cgi will receive any files from them.

This cgi would run on a remote server since ISP won't allow running a server on their server.
If you are running your own server then cgi isn't necessary. Just TCP between the two machines.

'cgiUpload.bas
'Bytes received must match BufferLength on every request
'STDOUT statements are only for testing
 
#COMPILE EXE
#DIM ALL
%FALSE = 0
%TRUE  = 1
$Server = "255.255.255.255"   'IP of the client running the server
%PortNumber = 12345            'Server port on client machine
 
FUNCTION PBMAIN AS LONG  
  STDOUT "Content-type: text/html" & $CRLF  'required by some servers
  STDOUT "<PRE>"
  LOCAL nSocket      AS LONG
  LOCAL result       AS LONG
  LOCAL BufferLength AS LONG    'server must send this exact number of bytes each time
  LOCAL sFileName    AS STRING  'file name to be uploaded to us
  LOCAL sFileSize    AS STRING  'length of file to be uploaded to us
  LOCAL sData        AS STRING  'data uploaded
 
  nSocket = FREEFILE
  TCP OPEN PORT %PortNumber AT $Server AS nSocket TIMEOUT 5000
  IF ERR THEN STDOUT "Unable to run" + STR$(ERRCLEAR):EXIT FUNCTION
 
  BufferLength = 32   'each response must be 32-bytes
 
  TCP SEND #nSocket, "FILENAME:"   'request file name for client
  result = tcpSafeReceive(nSocket, BufferLength,sFileName)
  IF result = %FALSE THEN
     STDOUT "Incorrect file name"
  ELSE
    sFileName = RTRIM$(sFileName)
    STDOUT "Filename: " + sFileName
 
    TCP SEND #nSocket, "FILESIZE:"  'request file size from client
    result = tcpSafeReceive(nSocket, BufferLength,sFileSize)
    IF result = %FALSE THEN
       STDOUT "Incorrect FILESIZE:"
    ELSE
      sFileSize = RTRIM$(sFileSize)
      BufferLength = VAL(sFileSize)
      TCP SEND nSocket, "UPLOAD:"   'tell server to start sending
      STDOUT "Filesize: " + sFileSize
      STDOUT "Bytes to receive:" + STR$(BufferLength)
      result = tcpSafeReceive(nSocket,BufferLength, sData) 'receive file
      IF result = 0 THEN
        STDOUT "Unable to receive file"
      ELSE
        STDOUT "Bytes received:" + STR$(LEN(sData))
      END IF
    END IF
  END IF
  TCP CLOSE #nSocket
END FUNCTION
FUNCTION tcpSafeReceive(BYVAL hSocket AS LONG, BYVAL iBufferLen AS LONG, _
                        recBuff AS STRING) AS LONG
   DIM iLeft AS LONG
   DIM sBuffer AS STRING
   recBuff = ""
   iLeft = iBufferLen
   DO
      sBuffer = SPACE$(iLeft)
      ON ERROR RESUME NEXT
      sBuffer = SPACE$(iBufferLen)
      TCP RECV hSocket, iLeft, sBuffer
      IF ERR THEN
         FUNCTION = %False
         EXIT FUNCTION
      END IF
      recBuff = recBuff + sBuffer
      IF LEN(recBuff) >= iBufferLen THEN
         EXIT DO
      END IF
      iLeft = iBufferLen - LEN(recBuff)
      SLEEP 5
   LOOP
   FUNCTION = %True
END FUNCTION

'Some server code the client might download

#PBFORMS CREATED V1.51
'TcpServer.Bas
'------------------------------------------------------------------------------
%PortNumber = 1234
$cgiProgram = "[URL="http://www.yourserver.com/cgi-bin/cgiprogram.exe"]www.yourserver.com/cgi-bin/cgiprogram.exe[/URL]"
#COMPILE EXE
#DIM ALL
#PBFORMS BEGIN INCLUDES
#IF NOT %DEF(%WINAPI)
    #INCLUDE "WIN32API.INC"
#ENDIF
#PBFORMS END INCLUDES
'------------------------------------------------------------------------------
#INCLUDE "WS2_32.INC"
'------------------------------------------------------------------------------
'   ** Constants **
'------------------------------------------------------------------------------
#PBFORMS BEGIN CONSTANTS
%IDR_IMGFILE1 =  102    '*
%IDD_DIALOG1  =  101
%IDM_CLEAR    = 4097    '*
%IDC_LABEL1   = 4098
%IDC_BUTTON1  = 4099
#PBFORMS END CONSTANTS
%INVALID_SOCKET = &HFFFFFFFF???
'------------------------------------------------------------------------------
%TCP_ACCEPT = %WM_USER + 4093  ' Any value larger than %WM_USER + 500
%TCP_ECHO   = %WM_USER + 4094  ' Any value larger than %WM_USER + 500
'------------------------------------------------------------------------------
'   ** Declarations **
'------------------------------------------------------------------------------
DECLARE CALLBACK FUNCTION ShowDIALOG1Proc()
DECLARE FUNCTION ShowDIALOG1(BYVAL hParent AS DWORD) AS LONG
DECLARE FUNCTION AttachMENU1(BYVAL hDlg AS DWORD) AS DWORD
#PBFORMS DECLARATIONS
DECLARE SUB LogEvent (hwnd AS LONG, BYVAL Buffer AS STRING)
DECLARE SUB PlayWave (zWav AS ASCIIZ * 128)
DECLARE SUB InitialHeading(hwnd AS LONG)
'------------------------------------------------------------------------------
'------------------------------------------------------------------------------
'   ** Main Application Entry Point **
'------------------------------------------------------------------------------
FUNCTION PBMAIN()
    ShowDIALOG1 %HWND_DESKTOP
END FUNCTION
'------------------------------------------------------------------------------
'------------------------------------------------------------------------------
'   ** CallBacks **
'------------------------------------------------------------------------------
CALLBACK FUNCTION ShowDIALOG1Proc()
    LOCAL PortNumber AS LONG
    LOCAL s AS STRING
    LOCAL sBuffer AS STRING
    LOCAL sPacket AS STRING
    STATIC Counter_STATIC AS LONG
    STATIC hServer_STATIC AS LONG
    STATIC hEcho_STATIC AS LONG
    LOCAL sSendBack AS STRING
    LOCAL zText AS ASCIIZ * 128
    SELECT CASE AS LONG CBMSG
        CASE %WM_INITDIALOG
            ' Initialization handler
          hServer_STATIC = FREEFILE
          PortNumber = %PortNumber
          DIM hPort AS LONG
          hport = FREEFILE
          TCP OPEN SERVER PORT PortNumber AS hServer_STATIC
          IF ERR THEN
              'Logevent
              hServer_STATIC = 0
              DIALOG POST CBHNDL, %WM_USER + 999, 0,0  'don't use dialog end in initdialog per manual
          ELSE
              TCP NOTIFY hServer_STATIC, ACCEPT TO CBHNDL AS %TCP_ACCEPT
          END IF
          hEcho_STATIC = %INVALID_SOCKET    'why invalid here, so doesn't go through CASE statement?
          FUNCTION = 1
        CASE %WM_USER + 999
            DIALOG END CBHNDL
            FUNCTION =1
        CASE %TCP_ACCEPT
             'each time an html page is loaded a cgi program will send request through here
             SELECT CASE CBLPARAM
              CASE %FD_ACCEPT
                 hEcho_STATIC = FREEFILE
                 TCP ACCEPT hServer_STATIC AS hEcho_STATIC
                 TCP NOTIFY hEcho_STATIC, RECV CLOSE TO CBHNDL AS %TCP_ECHO
               END SELECT
          FUNCTION = 1
        CASE %TCP_ECHO
          SELECT CASE CBLPARAM
            CASE %FD_READ
                IF hEcho_STATIC <> %INVALID_SOCKET THEN
                    sBuffer = ""
                    sPacket = ""
                    DO
                      TCP RECV hEcho_STATIC, 1024, sBuffer
                      sPacket = sPacket & sBuffer
                    LOOP UNTIL sBuffer = "" OR ISTRUE EOF(hEcho_STATIC) OR ISTRUE ERR
 
                   IF LEN(sBuffer) THEN        'return something to cgi program
                       INCR Counter_STATIC
                       s = sPacket
 
                       'Filename and size to send (replaced with your input)
                       IF s = "FILENAME:" THEN
                         sSendBack = SPACE$(32) 'match bufferlength in cgiprogram
                         LSET sSendBack = "MyFileNameHere"
                         TCP SEND hEcho_STATIC, sSendBack
                       ELSEIF s = "FILESIZE:" THEN
                         sSendBack = SPACE$(32) 'match bufferlength in cgiprogram
                         LSET sSendBack = "12345" '123 byte bufferlength for file
                         TCP SEND hEcho_STATIC, sSendBack
                       ELSEIF s = "UPLOAD:" THEN
                         sSendBack = STRING$(12345,"X") 'bufferlength must match filesize
                         TCP SEND hEcho_STATIC, sSendBack
                       END IF
                       'LogEvent CBHNDL,"Sending back " + s
                   END IF
                ELSE
                    'LogEvent
                END IF
             END SELECT
        CASE %WM_NCACTIVATE
            STATIC hWndSaveFocus AS DWORD
            IF ISFALSE CBWPARAM THEN
                ' Save control focus
                hWndSaveFocus = GetFocus()
            ELSEIF hWndSaveFocus THEN
                ' Restore control focus
                SetFocus(hWndSaveFocus)
                hWndSaveFocus = 0
            END IF
 
        CASE %WM_COMMAND
            ' Process control notifications
            SELECT CASE AS LONG CBCTL
                ' /* Inserted by PB/Forms 04-19-2009 09:31:53
                CASE %IDC_BUTTON1
                    IF CBCTLMSG = %BN_CLICKED OR CBCTLMSG = 1 THEN
                       'call cgiupload
                       zText = $cgiProgram
                       ShellExecute %NULL, "OPEN", zText, BYVAL %NULL, CURDIR$, %SW_SHOWNORMAL
                    END IF
 
            END SELECT
        CASE %WM_CLOSE
        CASE %WM_QUERYENDSESSION
            PostQuitMessage 0
            'do stuff
            FUNCTION = 1
    END SELECT
END FUNCTION
'------------------------------------------------------------------------------
'------------------------------------------------------------------------------
'   ** Dialogs **
'------------------------------------------------------------------------------
FUNCTION ShowDIALOG1(BYVAL hParent AS DWORD) AS LONG
    LOCAL lRslt AS LONG
#PBFORMS BEGIN DIALOG %IDD_DIALOG1->->
    LOCAL hDlg  AS DWORD
    DIALOG NEW hParent, "Client's upload program", 191, 135, 188, 79, %WS_POPUP OR _
        %WS_THICKFRAME OR %WS_CAPTION OR %WS_SYSMENU OR %WS_MINIMIZEBOX OR _
        %WS_CLIPCHILDREN OR %WS_VISIBLE OR %DS_CENTER OR %DS_3DLOOK OR _
        %DS_NOFAILCREATE OR %DS_SETFONT, %WS_EX_CONTROLPARENT OR %WS_EX_LEFT _
        OR %WS_EX_LTRREADING OR %WS_EX_RIGHTSCROLLBAR, TO hDlg
    CONTROL ADD LABEL,  hDlg, %IDC_LABEL1, "This server should contact " + _
        "cgiprogram.exe", 25, 10, 155, 25
    CONTROL ADD BUTTON, hDlg, %IDC_BUTTON1, "Call cgiprogram.exe", 35, 45, 95, _
        15
#PBFORMS END DIALOG
    DIALOG SHOW MODAL hDlg, CALL ShowDIALOG1Proc TO lRslt
#PBFORMS BEGIN CLEANUP %IDD_DIALOG1
#PBFORMS END CLEANUP
    FUNCTION = lRslt
END FUNCTION

[Michael Mattias]

Inspired by success at http://www.powerbasic.com/support/pb...ad.php?t=40331 .....

'CLPIPE2.BAS
' test commandline piping with binary input
t=40331
' 4/19/09  MCM
' PB/CC  5.0.1

#COMPILE EXE
#DIM ALL

#INCLUDE "WIN32API.INC"
FUNCTION PBMAIN () AS LONG
     CALL StdinBinary ()
END FUNCTION
'  ----------------------------------------------------------------------------
' PIPING BINARY STDIN TO STDOUT
' command line used:
' Type clpipe.exe | clipipe.exe > clpipeso.exe
' then to check: fc clpipe.exe clpipeso.exe ==>  FC: No differences encountered
' ------------------------------------------------------------------------------

FUNCTION StdInBinary () AS LONG
     
  LOCAL hPBI AS LONG, hPBO AS LONG, hSys AS LONG
  LOCAL inBYte AS BYTE

  hSys  = GetStdHandle (%STD_INPUT_HANDLE)
  hPBI  = FREEFILE
  OPEN HANDLE hSys FOR BINARY ACCESS READ AS hPBI
  hSys = GetStdHandle (%STD_OUTPUT_HANDLE)
  hPBO = FREEFILE
  OPEN   HANDLE hSys FOR OUTPUT AS hPBO
  ' EIF() for BINARY is funny... you
  ' have to test AFTER a failed read...
  DO
     GET #hPBI,,inByte
     IF ERR THEN
           PRINT USING$ ("error # & on get of next byte", ERR, ERROR$(ERRCLEAR))
           ' PRINT was broke in 5.0.0, doing nothing if STDOUT was redirected
           ' let's see if that is fixed in 5.0.1... it is, it is! Works perfectly
          ' this results in ERR 75 path file access error at EOF
          ' FC reports input = output, unless you do other STDOUT stff
          EXIT DO
     ELSE
     'PRINT #hPBO, UCASE$(a$)
         PRINT #hPBO, CHR$(inbyte);
     END IF
  LOOP
  CLOSE hPBI
  CLOSE hPBO
END FUNCTION

' ** END OF PROGRAM **

Should be reasonably adaptable to "whatever" circumstance....

(Note: PRINT was broke in help file; it apparently always worked like this... PRINT is NOT redirectable and requires #CONSOLE ON, which is pretty much what 5.0.1 help file says, albeit somewhat indirectly)


MCM

[Shawn Anderson]

The new version works with all file types and there seems to be a limit of a about one 1M which I think is to do with a limit in the browser or a timeout.

This is a timeout problem when using HTTP for upload (or download for that matter). HTTP only allows the connection to be open for so long before it closes it. If your file is large and/or upload speed it slow, you won't make it. This can be helped by a setting on the web server. I think the default is 2 minutes in IIS.

For very large files I'd use another protocol like FTP or a client like in Mike's example.

[Michael Mattias]

>For very large files I'd use another protocol like FTP or a client like in Mike's example.

This demo....
WinInet FTP Upload (Overwrite or Append) Demo 3-10-08

... is an FTP upload. In the middle of that is some code you could adapt to use the WinInet "InternetWriteFile()" function instead of messing around with STDOUT or HTML script.

Just reviewing this thread, I have become confused as to the problem: is it "getting" non-text data into the program via STDIN? Or is it writing data you already have to a url? Or maybe, both?