Using http to download binary files

[Scott Turchin]

So I wrote that function that grabs a web page, but when I attempt to use it I get an extra 291 bytes in the file and it's not useable....
I want to attempt to download a .zip or .exe file from the web using http (usually a bad idea but there is a reason for this (Firewall))

Is there anything different downloading a binary file?
I suspect the header will show it as binary, I'll put a sniffer on tonight and check it out. Meanwhile if anyone has done this I'd appreciate some insight...

Scott



-------------
Scott
mailto:[email protected][email protected]</A>

[Semen Matusovski]

Scott --
There are no problems to download binary files from HTTP.
Of course, you should cut title.

   #Compile Exe
   #Register None
   #Dim All
   #Include "WIN32API.INC"
   %Port = 80
   $WebDir = "ltr.perevozki.ru/Ltr32/"
   $BinFile = "Ltr32Upd.Exe"
   $LocDir = "C:\"

   Sub CPrint (SOut As String)
      Static hConsole As Long, cWritten As Long
      If hConsole = 0 Then AllocConsole: hConsole = GetStdHandle(-11&)
      WriteConsole hConsole, ByCopy sOut + $CrLf, Len(sOut) + 2, cWritten, ByVal 0&
   End Sub
   
   Function PbMain
      Dim Buffer As String, kBuffer As Long, i As Long
      Tcp Open Port %Port At Left$($WebDir, Instr($WebDir, "/") - 1) As #1 TimeOut 5000
      Tcp Print #1, "GET <A HREF="http://"" TARGET=_blank>http://"</A>  + $WebDir + $BinFile + " HTTP/1.0"
      Tcp Print #1, ""
      kBuffer = 4096: Buffer$ = Space$(kBuffer)
      CPrint "Load " + $WebDir + $BinFile
      Open $LocDir + $BinFile For Output As #2
      Do
         Tcp Recv #1, kBuffer, Buffer
         If Len(Buffer) = 0 Then Exit Do
         CPrint "Received" + Str$(Len(Buffer)) + " bytes"
         Print #2, Buffer;
      Loop
      Close #2
      Tcp Close #1
      Open $LocDir + $BinFile For Binary As #2
      CPrint "Total" + Str$(Lof(2)) + " bytes"
      Get$ #2, Lof(2), Buffer: Close #2
      i = Instr(Buffer, $CrLf + $CrLf)
      If UCase$(Left$(Buffer, 5)) = "HTTP/" And i Then _
         Open $LocDir + $BinFile For Output As #2: _
         CPrint "Title includes" + Str$(i + 3) + " bytes": _
         Print #2, Mid$(Buffer, i + 4);: Close #2
      MsgBox "Done"
   End Function

------------------

[Scott Turchin]

Actually, I just modified my version of thesource to WRITE binarily and walah, problemsolved, now it will save ANY type to disk.

This code is cut and pasted from the DLL and from the .EXE That calls it, but it works.
This is the header that proceeds all html transactions:
'HTTP/1.1 200 OK
'Date: Mon, 08 May 2000 15:39:16 GMT
'Server: Apache/1.3.6 (Unix) PHP/3.0.7 mod_perl/1.21 mod_frontpage/3.0.4.3
'Last-Modified: Sun, 23 Apr 2000 03:47:23 GMT
'ETag: "6828-21a0-3902724b"
'Accept-Ranges: bytes
'Content-Length: 8608
'Keep-Alive: timeout=65, Max=200
'Connection: Keep-Alive
'Datatype, html/text

DataType is the key, it is application/zip in this case, makes sense

EXE CODE
Function PbMain () As Long
Local lResult As Long
Local lPort   As Long
Local wwwPage As String
Local Referrer As String
Local AppName  As String
Local LocalFile As String

lPort   = 80
AppName = "Winlog For Windows 98"
Referrer = "Application: " + AppName
wwwPage = "http://www.tngbbs.com/ccs/files/winlog98.zip"
LocalFile = CurDir$ + "\winlog98.zip"
'Header shows: application/zip for binary files
lResult = GetWebPage(lPort, _
                     ByVal wwwPage,_
                     AppName, _
                     Referrer,_
                     LocalFile)
End Function


DLL CODE:
'------------------------------------------------------------------------------------------------------------------------
Function GetWebPage(wwwPort As Long, _
                    ByVal wwwPage As String, _
                    AppName As String, _
                    Referrer As String, _
                    LocalFile As String) Export As Long

Local hFile         As Long
Local hTCP          As Long
Local lCount        As Long
Local lLoop         As Long
Local ErrType       As Long

Local TmpFile       As String
Local TempDir       As Asciiz * 255
Local wwwRequest    As String

Local lHostName     As String
Local LastModDate   As String
Local AcceptRange   As String
Local buffer        As String
Local WebIn         As String
Local sInArray      As String
Local lByteCount    As Long

'******************************
'Extract three things:
'Host , ie www.tngbbs.com 
'Webpage, ie /winlog/help.html
wwwPage = Remove$(wwwPage,"http://")
lHostName = Left$(wwwPage,Instr(wwwPage,"/") - 1)
wwwPage = Remove$(wwwPage,lHostName)
'******************************
'We will use the temp directory, not a cache directory
GetTempPath SizeOf(TempDir),TempDir
TmpFile = TempDir + Mid$(LocalFile, Instr(-1, LocalFile, "\") + 1)
'Build the HTTP header
wwwRequest = "GET " + wwwPage + " HTTP/1.1" + $CRLF
wwwRequest = wwwRequest + "Accept: */*" + $CRLF
wwwRequest = wwwRequest + "Referer: " + Referrer + $CRLF
wwwRequest = wwwRequest + "Accept-Language: en-us" + $CRLF
wwwRequest = wwwRequest + "Accept-Encoding: gzip, deflate" + $CRLF
wwwRequest = wwwRequest + "User-Agent: Mozilla/4.0 (compatable; " + AppName + "; " + GetWindowsVersion + ")" + $CRLF
wwwRequest = wwwRequest + "Host: " + lHostName + $CRLF
wwwRequest = wwwRequest + "Connection: Keep-Alive" + $CRLF
wwwRequest = wwwRequest + "" + $CRLF

'Safety precaution
If wwwPort = 0 Then wwwPort = 80
'Open the port, Send the request

Tcp Open Port wwwPort At lHostName As hTcp TimeOut 10
ErrType = GetLastError
If ErrType Then
   MsgBox "Could not connect to " + wwwPage,%MB_ICONSTOP, "Connection Error!"
   Function = ErrType
   Exit Function 'Required for when connection not available, GPF otherwise
End If

Tcp Print hTcp, wwwRequest

Do
  Tcp Recv #hTcp,4096, buffer
  WebIn = WebIn + buffer
Loop While Len(buffer)
Tcp Close hTcp

lCount = Tally(WebIn,$CR)
'Test to see if you got enough minimum data
If lCount < 11 Then Exit Function

For lLoop = 1 To 10
    sInArray = Parse$(WebIn,$CR,lLoop)
    Select Case lLoop
           Case 4
             LastModDate = Right$(sInArray,Len(sInArray) - 15)
           Case 6
             AcceptRange = Trim$(Right$(sInArray,Len(sInArray) - 15))'almost ALWAYS "bytes"
           Case 7
             'lByteCount contains the EXPECTED length of the page, minus header
             If AcceptRange = "bytes" Then
                 lByteCount = Val(Right$(sInArray,Len(sInArray) - 17))
             End If
    End Select
Next
'Remove the header here
If Len(WebIn) > lByteCount Then WebIn = Right$(WebIn,lByteCount)

hFile = FreeFile
Open TmpFile For Binary As #hFile
Put$ #hFile, WebIn
Close #hFile
If IsTrue Exist(TmpFile) Then
    FileCopy TmpFile, LocalFile
    If IsTrue Exist(LocalFile) Then Function = %TRUE
    Kill TmpFile
End If
End Function

------------------
Scott
mailto:[email protected][email protected]</A>