URLOpenBlockingStream

[Gary Beene]

I was looking for an API alternative to URLDownloadToFile (but, to download to memory, not a file) and ran across URLOpenBlockingStream in this thread ...

https://forum.powerbasic.com/forum/u...-api-functions

The code works on a web page from my site, but not from the PowerBASIC site. Why would that be?

#Compile Exe           'Jose's Headers
#Debug Display On
#Debug Error On

#Include "WIN32API.INC"
#Include "urlmon.inc"
Global gTemp$
Function PBMain()
   Local stream As IStream, hr As Long
   Local buffer As AsciiZ * 500, bytesRead As Dword
   'hr = URLOpenBlockingStream(Nothing, "https://forum.powerbasic.com/forum/user-to-user-discussions/powerbasic-for-dos/148-bug-in-on-line-help", stream, 0, Nothing)   'does not work
   hr = URLOpenBlockingStream(Nothing, "http://www.garybeene.com", stream, 0, Nothing)   'works
   While (stream.Read(VarPtr(buffer), 500, bytesRead) = %S_Ok) And (bytesRead >< 0)
     gTemp$ += buffer
     bytesRead = 0
   Wend
   stream = Nothing
   ? gTemp$
End Function

[Jassim Mohammad Kareem]

Could it be because the POWERBSIC site is SECURE (https://forum.powerbasic.com) and your site is not (http://www.garybeene.com) and it needs an extra permission?

[Gary Beene]

Howdy, Jassim!

It works with https://www.google.com.

[Stuart McLachlan]

Try this, you may be surprised:

#COMPILE EXE           'Jose's Headers
#DEBUG DISPLAY ON
#DEBUG ERROR ON

#INCLUDE "WIN32API.INC"
#INCLUDE "urlmon.inc"
GLOBAL gTemp$
FUNCTION PBMAIN()
   LOCAL stream AS IStream, hr AS LONG
   LOCAL buffer AS ASCIIZ * 500, bytesRead AS DWORD
   hr = URLOpenBlockingStream(NOTHING, "https://forum.powerbasic.com/forum/user-to-user-discussions/powerbasic-for-dos/148-bug-in-on-line-help", stream, 0, NOTHING)   'does not work
   'hr = URLOpenBlockingStream(Nothing, "http://www.garybeene.com", stream, 0, Nothing)   'works
   OPEN "test.log" FOR OUTPUT AS #1
    WHILE (stream.Read(VARPTR(buffer), 500, bytesRead) = %S_OK) AND (bytesRead >< 0)
     gTemp$ += left$(buffer,bytesRead)
     PRINT #1, left$(buffer,bytesread)
     bytesRead = 0
   WEND
   CLOSE #1
   stream = NOTHING
   ? gTemp$
END FUNCTION

Edited: Changed Buffer to Left$(Buffer,BytesRead)

[Stuart McLachlan]

It's the "? gTemp$ " that is causing the application to hang!

Added: All sorts of strange things are happening if you try to msgbox any large string after the above code.

Added2: Now even if I try a large string BEFORE the above code after a fresh compile.
I suspect that the URLOpenBlockingStream has not released something has done something really strange to my system. Guess I'll have to reboot.

[Gary Beene]

Howdy, Stuart!

Yes, very surprised!

The "test.log" file can be opened and read just fine in NotePad.

The gTemp$ variable contains bytes that MsgBox cannot display?

What's the story?

[Gary Beene]

On a quick test, this allows the ? gTemp$ to work ...

Replace $Cr With $Spc In gTemp$
Replace $Lf With $Spc In gTemp$
Replace Chr$(0) With $Spc In gTemp$
Replace Chr$(1) With $Spc In gTemp$
? gTemp$

need to narrow it down ...

[Gary Beene]

Well, that is mighty weird. Only these two are needed to allow it to display ...

Replace $Cr With $Spc In gTemp$
Replace $Lf With $Spc In gTemp$

I've seen ? not display content before but don't recall the parameters that make that happen.

Looking at the log file in a hex viewer, I see some TAB characters. I don't know that would give ? a problem. But interestingly, the 09 always seems to precede 0D0A.

Clues, but I still can't explain it. A ? limitation of some kind.

[Gary Beene]

Just testing ... this msgbox displays just fine...

Function PBMain() As Long
? "a" + Chr$(09) + "b"
End Function

[Gary Beene]

This allows display as well ...

gTemp$ = Retain$(gTemp$, Any Chr$(14 To 255))

If I open the LOG file in NotePad and use Save-As, notepad says the file format is UTF-8.

[Stuart McLachlan]

This allows display as well ...

gTemp$ = Retain$(gTemp$, Any Chr$(14 To 255))

If I open the LOG file in NotePad and use Save-As, notepad says the file format is UTF-8.

Yep, tt's typical - a best guess.

What happens if you put StrT=STRING$(60000,A"):? strT anywhere. After running various modifications on the earlier code, that simple instruction doesn't work properly

[Stuart McLachlan]

Hmmm, Rebooted and same result. I've never tried strings that big before in a messagebox. Looks like there's a limit to the contents of a messagebox - but haven't been able to find a reference.

[Stuart McLachlan]

Incdentally, that stream IS UTF8. At location 57956,7 there is a C2 A9 hex pair which is the UTF8 encoding of the Copyright character.
You need to run gTemp through UTF8TOCHR$().

In Notepad++ go to there (with cursor at start of file hit Ctrl+G and enter 57956)
You will see the copyright sign.
Select Encoding -ANSI from the menu and you will see it change to two characters!

[Mike Doty]

'

1 Not reading using ISTREAM
2 Check out Bob Carvers alternative from your link in post #1 reading the PowerBASIC site.

'Jose's Headers
#COMPILE EXE
%UNICODE = 1
#INCLUDE "WIN32API.INC"
#INCLUDE "urlmon.inc"

FUNCTION PBMAIN()

GOTO Alternatively

LOCAL stream AS IStream
LOCAL hr AS LONG
hr = URLOpenBlockingStream(NOTHING, "https://www.google.com", stream, 0, NOTHING)

LOCAL buffer AS ASCIIZ * 500
LOCAL bytesRead AS DWORD

WHILE (stream.Read(VARPTR(buffer), 500, bytesRead) = %S_OK) AND (bytesRead >< 0)
   ' do whatever with buffer

   bytesRead = 0
WEND
stream = NOTHING

Alternatively:
'
 LOCAL cacheFile AS WSTRINGZ * %MAX_PATH
 hr = URLDownloadToCacheFile( _
    NOTHING, _
    "https://forum.powerbasic.com/forum/user-to-user-discussions/powerbasic-for-dos/148-bug-in-on-line-help",_
    BYVAL VARPTR(cacheFile), _
    %MAX_PATH, _
    0, _
    NOTHING _
 )
' Here, use cacheFile with the normal PB file functions (OPEN et al)
' then KILL to delete it from the cache
'hr = URLOpenBlockingStream(Nothing, "https://forum.powerbasic.com/forum/user-to-user-discussions/powerbasic-for-dos/148-bug-in-on-line-help", stream, 0, Nothing)   'does not work
'
'
ShellExecute (%NULL, "OPEN", cachefile, BYVAL %NULL, CURDIR$, %SW_SHOWNORMAL)
END FUNCTION'

[Gary Beene]

Howdy, Mike!

I've used URLDownloadToFile. How is DownloadToCacheFile different?

I can see that with the first, you name the file and with the second, it tells you the file name.

Other than wanting to pick the file name myself, why chose one of those API over the other?

Do you know if one has any better performance?

[Gary Beene]

Stuart,

You need to run gTemp through UTF8TOCHR$().

? UTF8ToChr$(gTemp$)

also fails to display.

... removed some comments...

[Stuart McLachlan]

Stuart,

? UTF8ToChr$(gTemp$)

also fails to display.

... removed some comments...

Yep, still the same problem with apparent limits to Msgbox, but the UTF8 content is a different matter.
If you subsequently use the downloaded data in PB in any way without passing it through UTF8ToCHR$(), the C2 A9 will not represent ©, (copyright symbol) it will represent © (Capital A circumflex followed by Copyright symbol)