Password protected ascii file

[Ralph Brereton]

Does anyone know how to download a p'word protected ascii file (csv) ?
Protocal in use seems to be http1.1
I logged on successfully with
TCP PRINT "GET " + filename + " HTTP/1.1"
TCP PRINT "Host: " + "wwwaddress"
TCP PRINT "Authorisation: Basic " + encode64(uname+pword)
Then recd message telling me "302 Found"
But same message tells me Location (of my file) is at an http:// address
What am I supposed to do now ?
Close connection; start again at new location ? That can't be right.
So I tried
TCP PRINT "GET " + filename + "HTTP/1.1"
TCP PRINT "Host: " + httpaddress (no mention of uname/pword)
Was told "Bad Request" and connection was closed.
I suspect I don't know how to phrase that last get.
Any ideas from you clever people out there ?

[Mike Stefanik]

If you're getting a 302 result code from the server, it means that the location of the resource has moved. The response header will contain a Location: header value that specifies the new location, and you must go to that new location to retrieve the file. Keep in mind that the new location may also redirect you to yet another location (it's rare, but technically possible). So you want code that looks something like:

do
   connect to the server
   request the resource (file)
   if result code is 301 or 302
       get new location from Location: header field
       parse the location URL
   else
       exit loop
   end if
loop

You want to check for both 301 (permanently moved) and 302 (temporarily moved) result codes. One other thing, not related to this. Because you're specifying HTTP 1.1, make sure that your code that reads the data returned by the server is written to be able to handle chunking correctly; if you don't, it's likely that you could get data that looks like it has "garbage" scattered throughout it. To avoid the whole chunking issue, you may want to specify HTTP 1.0 instead.

[Ralph Brereton]

Thanks, Mike. I guessed as much. But what do you mean by "you must go to the new location". Do you mean I must close my first connection, then open a new one, with all the bother of uname and pword a secdond time ? Also, thank you for the tip about http1.0. I'll try that. Anything for simplicity.

[Mike Stefanik]

Thanks, Mike. I guessed as much. But what do you mean by "you must go to the new location". Do you mean I must close my first connection, then open a new one, with all the bother of uname and pword a secdond time ? Also, thank you for the tip about http1.0. I'll try that. Anything for simplicity.

Generally speaking, yes, you should close the first connection and open a new one (with the same authentication, etc.). While it's common that the location of the "moved" resource is on the same system, that's not always the case. The resource http://www.olddomain.com/foo.txt could be redirected to http://www.newdomain/some/nested/folder/bar.txt for example. Never make the assumption that the new URL references a resource on the system that you've just connected to previously.

EDIT: Keep in mind that it's technically possible that the new location URL not even specify an HTTP URL. For example, http://www.olddomain.com/foo.txt could be redirected to ftp://ftp.newdomain.com/files/bar.txt or something along those lines. That's even less common than cross-domain redirection, but in that case, you'd actually need to switch protocols in order to retrieve the resource. If your program is designed to only support HTTP, not HTTPS, FTP, etc. then you'll also want to make a check that the redirected URL begins with "http://"

[Ralph Brereton]

Thank you, again, Mike. It is complicated isn't it ? Maybe I should buy the Marshallsoft serial comm package and stop trying to do things beyond my ability.

[Ralph Brereton]

Mike, I just clicked on your web address and discovered "Catalyst File Transfer Control" ! At first it looked expensive, but it handles FTP as well as http, and I need both. I'd have to buy two Marshallsoft products to get that functionality.

[Mike Stefanik]

The File Transfer Control handles both standard FTP and HTTP, as well as secure FTPS (FTP+SSL) and HTTPS. However, it's an ActiveX control and so it has the associated overhead and registration requirements that a Windows DLL does not. Generally speaking, we recommend our SocketTools Library Edition to PowerBASIC developers, but it is possible to use the ActiveX control. Here's an example of what the code would look like:

#Compile Exe
#Dim All

#Include "WIN32API.INC"

Interface Dispatch IFileTransfer
    Member Call About<&HFFFFFDD8>()
    Member Let Enabled<&HFFFFFDFE>()
    Member Get Enabled<&HFFFFFDFE>() As Long
    Member Get Account<&H00000001>() As String
    Member Let Account<&H00000001>()
    Member Get ActivePort<&H00000002>(In Index As Long<&H00000000>) As Long
    Member Let ActivePort<&H00000002>(In Index As Long<&H00000000>)
    Member Get AppendFile<&H00000003>() As Long
    Member Let AppendFile<&H00000003>()
    Member Get CertificateExpires<&H00000004>() As String
    Member Get CertificateIssued<&H00000005>() As String
    Member Get CertificateIssuer<&H00000006>() As String
    Member Get CertificateName<&H00000007>() As String
    Member Let CertificateName<&H00000007>()
    Member Get CertificateStatus<&H00000008>() As Long
    Member Get CertificateStore<&H00000009>() As String
    Member Let CertificateStore<&H00000009>()
    Member Get CertificateSubject<&H0000000A>() As String
    Member Get ChannelMode<&H0000000B>(In Index As Long<&H00000000>) As Long
    Member Let ChannelMode<&H0000000B>(In Index As Long<&H00000000>)
    Member Get CipherStrength<&H0000000C>() As Long
    Member Get DirectoryFormat<&H0000000D>() As Long
    Member Let DirectoryFormat<&H0000000D>()
    Member Get Features<&H0000000E>() As Long
    Member Let Features<&H0000000E>()
    Member Get FileType<&H0000000F>() As Long
    Member Let FileType<&H0000000F>()
    Member Get HashStrength<&H00000010>() As Long
    Member Get KeepAlive<&H00000011>() As Long
    Member Let KeepAlive<&H00000011>()
    Member Get LastError<&H00000012>() As Long
    Member Let LastError<&H00000012>()
    Member Get LastErrorString<&H00000013>() As String
    Member Get Library<&H00000014>() As String
    Member Let Library<&H00000014>()
    Member Get LocalAddress<&H00000015>() As String
    Member Get LocalName<&H00000016>() As String
    Member Get Options<&H00000017>() As Long
    Member Let Options<&H00000017>()
    Member Get Passive<&H00000018>() As Long
    Member Let Passive<&H00000018>()
    Member Get Password<&H00000019>() As String
    Member Let Password<&H00000019>()
    Member Get Priority<&H0000001A>() As Long
    Member Let Priority<&H0000001A>()
    Member Get ProtocolVersion<&H0000001B>() As String
    Member Let ProtocolVersion<&H0000001B>()
    Member Get ProxyPassword<&H0000001C>() As String
    Member Let ProxyPassword<&H0000001C>()
    Member Get ProxyPort<&H0000001D>() As Long
    Member Let ProxyPort<&H0000001D>()
    Member Get ProxyServer<&H0000001E>() As String
    Member Let ProxyServer<&H0000001E>()
    Member Get ProxyType<&H0000001F>() As Long
    Member Let ProxyType<&H0000001F>()
    Member Get ProxyUser<&H00000020>() As String
    Member Let ProxyUser<&H00000020>()
    Member Get Resource<&H00000021>() As String
    Member Let Resource<&H00000021>()
    Member Get ResultCode<&H00000022>() As Long
    Member Get ResultString<&H00000023>() As String
    Member Get Secure<&H00000024>() As Long
    Member Let Secure<&H00000024>()
    Member Get SecureCipher<&H00000025>() As Long
    Member Get SecureHash<&H00000026>() As Long
    Member Get SecureKeyExchange<&H00000027>() As Long
    Member Get SecureProtocol<&H00000028>() As Long
    Member Let SecureProtocol<&H00000028>()
    Member Get ServerDirectory<&H00000029>() As String
    Member Let ServerDirectory<&H00000029>()
    Member Get ServerName<&H0000002A>() As String
    Member Let ServerName<&H0000002A>()
    Member Get ServerPort<&H0000002B>() As Long
    Member Let ServerPort<&H0000002B>()
    Member Get ServerType<&H0000002C>() As Long
    Member Let ServerType<&H0000002C>()
    Member Get System<&H0000002D>() As String
    Member Get ThrowError<&H0000002E>() As Long
    Member Let ThrowError<&H0000002E>()
    Member Get TimeOut<&H0000002F>() As Long
    Member Let TimeOut<&H0000002F>()
    Member Get Trace<&H00000030>() As Long
    Member Let Trace<&H00000030>()
    Member Get TraceFile<&H00000031>() As String
    Member Let TraceFile<&H00000031>()
    Member Get TraceFlags<&H00000032>() As Long
    Member Let TraceFlags<&H00000032>()
    Member Get TransferBytes<&H00000033>() As Long
    Member Get TransferTime<&H00000034>() As Long
    Member Get URL<&H00000035>() As String
    Member Let URL<&H00000035>()
    Member Get UserName<&H00000036>() As String
    Member Let UserName<&H00000036>()
    Member Get Version<&H00000037>() As String
    Member Call Cancel<&H00000038>() As Variant
    Member Call ChangeDirectory<&H00000039>(In DirName As Variant<&H00000000>) As Variant
    Member Call CloseDirectory<&H0000003A>() As Variant
    Member Call Command<&H0000003B>(In Command As Variant<&H00000000>) As Variant
    Member Call Connect<&H0000003C>(Optional In ServerName As Variant<&H00000000>, Optional In ServerPort As Variant<&H00000001>, Optional In UserName As Variant<&H00000002>, Optional In Password As Variant<&H00000003>, _
        Optional In TimeOut As Variant<&H00000004>, Optional In Options As Variant<&H00000005>) As Variant
    Member Call DeleteFile<&H0000003D>(In FileName As Variant<&H00000000>) As Variant
    Member Call Disconnect<&H0000003E>() As Variant
    Member Call FileStatus<&H0000003F>(In FileName As Variant<&H00000000>, Optional InOut FileLength As Variant<&H00000001>, Optional InOut FileDate As Variant<&H00000002>, Optional InOut FileOwner As Variant<&H00000003>, _
        Optional InOut FileGroup As Variant<&H00000004>, Optional InOut FilePerms As Variant<&H00000005>, Optional InOut IsDirectory As Variant<&H00000006>) As Variant
    Member Call GetFile<&H00000040>(In LocalFile As Variant<&H00000000>, Optional In RemoteFile As Variant<&H00000001>, Optional In RestartOffset As Variant<&H00000002>) As Variant
    Member Call GetFileStatus<&H00000041>(In FileName As Variant<&H00000000>, Optional InOut FileLength As Variant<&H00000001>, Optional InOut FileDate As Variant<&H00000002>, Optional InOut FileOwner As Variant<&H00000003>, _
        Optional InOut FileGroup As Variant<&H00000004>, Optional InOut FilePerms As Variant<&H00000005>, Optional InOut IsDirectory As Variant<&H00000006>) As Variant
    Member Call GetMultipleFiles<&H00000042>(In LocalDir As Variant<&H00000000>, In RemoteDir As Variant<&H00000001>, In FileMask As Variant<&H00000002>) As Variant
    Member Call GetResource<&H00000043>(In LocalFile As Variant<&H00000000>, In Resource As Variant<&H00000001>) As Variant
    Member Call Initialize<&H00000044>(Optional In LicenseKey As Variant<&H00000000>, Optional In ModuleName As Variant<&H00000001>, Optional In Options As Variant<&H00000002>, Optional In Reserved As Variant<&H00000003>) As Variant
    Member Call MakeDirectory<&H00000045>(In DirName As Variant<&H00000000>) As Variant
    Member Call OpenDirectory<&H00000046>(Optional In DirName As Variant<&H00000000>, Optional In ParseList As Variant<&H00000001>) As Variant
    Member Call PostFile<&H00000047>(In LocalFile As Variant<&H00000000>, Optional In RemoteFile As Variant<&H00000001>, Optional In FieldName As Variant<&H00000002>, Optional In Options As Variant<&H00000003>) As Variant
    Member Call PutFile<&H00000048>(In LocalFile As Variant<&H00000000>, Optional In RemoteFile As Variant<&H00000001>, Optional In RestartOffset As Variant<&H00000002>) As Variant
    Member Call PutMultipleFiles<&H00000049>(In LocalDir As Variant<&H00000000>, In RemoteDir As Variant<&H00000001>, In FileMask As Variant<&H00000002>) As Variant
    Member Call PutResource<&H0000004A>(In LocalFile As Variant<&H00000000>, In Resource As Variant<&H00000001>) As Variant
    Member Call ReadDirectory<&H0000004B>(InOut FileName As Variant<&H00000000>, Optional InOut FileLength As Variant<&H00000001>, Optional InOut FileDate As Variant<&H00000002>, Optional InOut FileOwner As Variant<&H00000003>, _
        Optional InOut FileGroup As Variant<&H00000004>, Optional InOut FilePerms As Variant<&H00000005>, Optional InOut IsDirectory As Variant<&H00000006>) As Variant
    Member Call RemoveDirectory<&H0000004C>(In DirName As Variant<&H00000000>) As Variant
    Member Call RenameFile<&H0000004D>(In OldFileName As Variant<&H00000000>, In NewFileName As Variant<&H00000001>) As Variant
    Member Call Reset<&H0000004E>()
    Member Call Uninitialize<&H0000004F>()
End Interface

Function PBMain () As Long
    Dim oFileTransfer As IFileTransfer
    Dim varLocalFile As Variant
    Dim varRemoteFile As Variant
    Dim varError As Variant

    Let oFileTransfer = New IFileTransfer In "SocketTools.FileTransfer.5"

    varLocalFile = "c:\filename.txt"
    varRemoteFile = "http://server.domain.com/filename.txt"

    Object Call oFileTransfer.GetFile(varLocalFile, varRemoteFile) To varError
    If Variant#(varError) > 0 Then
        Object Get oFileTransfer.LastErrorString To varError
        MsgBox Variant$(varError), %MB_ICONEXCLAMATION, "Error"
    Else
        MsgBox "Download completed successfully", %MB_OK, "File Transfer"
    End If

    Let oFileTransfer = Nothing
End Function

The equivalent code using the Library Edition to download a file from an HTTP server would look like this:

#Compile Exe
#Dim All

#Include "WIN32API.INC"
#Include "CSTOOLS5.INC"

Function PBMain () As Long
    Dim szLocalFile As Asciiz * %MAX_PATH
    Dim szRemoteFile As Asciiz * %MAX_PATH
    Dim httpStatus As HTTPTRANSFERSTATUS
    Dim szError As Asciiz * 128
    Dim bResult As Integer
    
    szLocalFile = "c:\filename.txt"
    szRemoteFile = "http://server.domain.com/filename.txt"

    HttpInitialize($CSTOOLS5_LICENSE_KEY, %NULL)

    bResult = HttpDownloadFile(szLocalFile, _
                               szRemoteFile, _
                               %HTTP_TIMEOUT, _
                               %HTTP_OPTION_NONE, _
                               httpStatus, _
                               %NULL, 0)

    If IsTrue(bResult) Then
        MsgBox "Download completed successfully", %MB_OK, "SocketTools"
    Else
        HttpGetErrorString(HttpGetLastError(), szError, 128)
        MsgBox szError, %MB_ICONEXCLAMATION, "Error"
    End If
       
    HttpUninitialize()

End Function

At its core, the code is fairly similar. The example using the ActiveX control requires the interface definition (which can be generated using the COM Browser included with PowerBASIC), and requires that you use variants along with the "Object Call", "Object Set", "Object Get" syntax which can appear to be a bit strange.

The Library Edition is the straight-forward API style programming that most PowerBASIC developers are more familiar and comfortable with. We support both our ActiveX controls and DLLs with PowerBASIC, so the one that you choose is largely a matter of personal preference.