Array of Bytes instead of Asciiz

[Roger Rines]

This information continues to expand my understanding of how to work with memory. I’ve played extensively with each of the sample code items by using them as presented and then by making changes to them to try things so I could see what happens. I like this style of learning as it help me put things in context.

Some answers to the questions:

Are the older Global/Local Alloc functions still worthwhile using? I used to use GlobalAlloc() quite a bit but several years ago I stopped using it because my MSDN documentation from around 1998 (which I still use) even at that early date said these functions were only for compatibility with 16 bit windows...

Because I’ve not had to deal any of this to create some really successful data tools, I’ve not been aware of how things changed. My long experience with PDS 7.1 and its need to play memory games, has probably kept me from wanting to go back down that complicated road. However, the explanations in this thread are helping me understand the process of dealing with memory is much easier in Windows and especially with PB.

Since I and likely many here are not familiar with how the "series of Asciiz string addresses" is coming to the DLL "from TradeStation", it would enlighten further if you could clarify if these are single ASCIIZ strings sent one after another as separate strings; or a composite of several ASCIIZ values sent as a single string or as a packet of variable, but specified length, or ?

A dynamic string can contain embedded nuls. but one has to know that they are expeced and deal with them in the code. It's a simple parse to get the values that you need to retrieve the actual text strings. Then the question is whether those text strings would have any embedded nuls as well.

For the most part it is a simple passing of a pointer to where the Asciiz string information is stored. However, if the string data is passed as part of a string array, then there is a need to use the TS API interface DLL to find string information.

How this is passed to a PB DLL is done through function calls within TradeStation’s Easy Language. Easy Language is a scripting language designed for creating automated trading signals, and its syntax looks like a Pascal-Basic hybrid with its Basic like reserved words that use BEGIN END boundaries for compound statements and almost every statement is terminated with a semicolon.

There is a file named PB-Bones.Zip available in the PB Forum’s file storage area where a Word file and a skeleton DLL structure contains the translated C++ headers for accessing TradeStation’s variables using the TS API DLL. That zip pack was created to promote PB with the enormous customer base using TradeStation everyday. When I get a better understand of how to work with their COM interface, I’ll create a PB-COM approach and publish that here and in the TradeStation forum.

FWIW you don't need controls to hold data associated with a window....

A. SetWindowLong %GWL_USERDATA, value (retrieve with GetWindowLong)
B. SetProp hWnd, $PROPERTY_NAME, value (retrieve with GetProp)

Value is an integer; so you can make it a pointer or a handle.

Michael, I didn’t know that, although I’ve seen code where it seemed like this was the case. I’ll try to understand this more because I think there is a growing need to create DLLs and COM interfaces for third party programs. With a better understanding of how to handle memory, that information can help me to be more creative. Besides it provides a better footing for my coding

Roger, it turns out it is somewhat more complex to handle the memory for arrays using DIM...AT.
[snip]

Your example and then my playing around with some changes to it brought that message home clearly. Since toying around with a couple of versions of your sample code, I find I like the POKE$ approach better because the code acts the same way as it would had it been passed using the CONTROL SEND statement shown above.

I liked how much information was shared in this thread, and especially the samples. By playing around with the samples and then reading a lot of help information on some of the items, I now have a few more tools in my coding-box to use for dealing with memory. I also have a longer list of items to chase down to see if I can understand how to use them, and why.

Thanks again to all who helped.

[John Gleason]

it seemed like I could clear the memory space with an ERASE byBuffer() call before the function doing this work terminated. Is my assumption correct that ERASE is all I’ll need to clear the buffer space?

Roger, it turns out it is somewhat more complex to handle the memory for arrays using DIM...AT.

From the PB docs it says:
Absolute arrays (those created by DIM...AT) are handled differently by ERASE. An explicit ERASE will release the individual elements of an absolute array, if needed, but the full data block is left as-is because no assumptions can be made as to its origin. It is the programmer's responsibility to ensure that the memory block overlaid by the absolute array is handled correctly. In an implied ERASE (Sub/Function exit) of a LOCAL absolute array, the internal array descriptor is deactivated, but no changes of any kind are made to the individual data elements or the full block. RESET may be used to set arrays to zeroes or empty strings without releasing the data block.

Here's an expanded example to hopefully shed some light.

#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

   DIM lpBuffer AS ASCIIZ PTR
   LOCAL msgPtr, nLength AS LONG
   LOCAL msgTxt AS STRING

   msgTxt = STRING$(300, 232) & $NUL
   msgPtr = STRPTR(msgTxt)
   nLength = LEN(msgTxt) - 1
   REDIM byBuffer(nLength + 1) AS GLOBAL BYTE AT msgPtr

   lpBuffer = VARPTR(byBuffer(0))
   ? $DQ & @lpBuffer & $DQ
   ERASE byBuffer()         'releases elements of byBuffer()
   RESET msgTxt             'releases "origin" msgTxt dynamic string. Now zero length.
   ? $DQ & msgTxt & $DQ
   ? $DQ & @lpBuffer & $DQ  '<< this shows that nothing happened to the data in the memory block itself. But since this
                            'memory is now unallocated (via the above ERASE and RESET) and free to be used by other processes,
                            'future use of @lpBuffer without reassigning the lpBuffer address could result, I believe, in a GPF.
                            'I only used it here to perhaps increase understanding of how the memory block is being handled.

END FUNCTION

[Richard Angell]

Here's a version that returns the full monte even if there are embedded NULs, CHR$(0)s. It does not pass a terminating NUL,, but neither would a typical ASCIIZ fetch. If for some reason you want to have a terminating NUL just concatentate one to (at) the FUNCTION return with the following change: FUNCTION = PEEK$(@SomePtr,PEEK(DWORD,@SomePtr - 4)+1)

The second version of the DLL:

#COMPILE DLL "AP2DS2.DLL"
#DIM ALL
%USEMACROS = 1
#INCLUDE "Win32API.inc"
GLOBAL ghInstance AS DWORD
GLOBAL pTextBoxString AS DWORD PTR
GLOBAL TextBoxString AS STRING

FUNCTION GetTextPtr ALIAS "GetTextPtr" ()EXPORT AS DWORD
    'Here you would actually need to get the string from the real Edit Control
    'We are just spoofing it here for simplicity of demonstration.
    'Assumes the caller only cares for the stripped string.
    'In memory all dynamic strings have the NULL appended as a last character
    '(see In-Line Assembler topic "Passing dynamic strings")
    TextBoxString = CHR$(0)  'initialize the global here in the DLL
    pTextBoxString = VARPTR(TextBoxString)  'point to the handle
    FUNCTION = pTextBoxString
END FUNCTION
'following function just wants to see a 32 bit value, so changed to DWORD PTR
FUNCTION GetTextByPtr ALIAS "GetTextByPtr" (BYVAL SomePtr AS DWORD PTR)EXPORT AS STRING
    STATIC swstr AS LONG
    IF SomePtr <> pTextBoxString OR TextBoxString = "" THEN
        FUNCTION = ""
    ELSE
        'in the real DLL this is where to code the real string reader
        IF swstr = 0 THEN
            swstr = 1
            TextBoxString = "This is a string"+CHR$(0)+" of variable size"
        ELSE
            swstr = 0
            TextBoxString = "This is a longer string that was assigned to the dynamic string in the DLL"
        END IF
        FUNCTION = PEEK$(@SomePtr,PEEK(DWORD,@SomePtr - 4))  
    END IF
END FUNCTION

FUNCTION LIBMAIN (BYVAL hInstance   AS LONG, _
                  BYVAL fwdReason   AS LONG, _
                  BYVAL lpvReserved AS LONG) AS LONG
    SELECT CASE fwdReason
        CASE %DLL_PROCESS_ATTACH
            ghInstance = hInstance
            FUNCTION = 1   'success!
        CASE %DLL_PROCESS_DETACH
            FUNCTION = 1   'success!
        CASE %DLL_THREAD_ATTACH
            FUNCTION = 1   'success!
        CASE %DLL_THREAD_DETACH
            FUNCTION = 1   'success!
    END SELECT
END FUNCTION
[code]
and a changed test program to show tthat he emmbs come back as well

#COMPILE EXE
#DIM ALL
DECLARE FUNCTION GetTextPtr LIB "AP2DS2.DLL" ALIAS "GetTextPtr"() AS DWORD
DECLARE FUNCTION GetTextByPtr LIB "AP2DS2.DLL" ALIAS "GetTextByPtr"(BYVAL SomePtr AS DWORD PTR) AS STRING
FUNCTION PBMAIN () AS LONG
LOCAL MyAPTR AS ASCIIZ PTR
LOCAL a$
MyAPTR = GetTextPtr()
a$ = GetTextByPtr(MyAPTR) : GOSUB MakeDisplayString
a$ = GetTextByPtr(MyAPTR) : GOSUB MakeDisplayString
' WAITKEY$ 'uncomment for use with PB/CC
EXIT FUNCTION
MakeDisplayString: 'just to show what's coming back
REPLACE CHR$(0) WITH " *NUL*" IN a$
? a$
RETURN
END FUNCTION
[/code]

[Richard Angell]

My project will need to accept a series of Asciiz string addresses from TradeStation, use TradeStation’s API DLL to get the string information at those locations and then pass that data to another DLL where it will be processed.

Since I and likely many here are not familiar with how the "series of Asciiz string addresses" is coming to the DLL "from TradeStation", it would enlighten further if you could clarify if these are single ASCIIZ strings sent one after another as separate strings; or a composite of several ASCIIZ values sent as a single string or as a packet of variable, but specified length, or ?

A dynamic string can contain embedded nuls. but one has to know that they are expeced and deal with them in the code. It's a simple parse to get the values that you need to retrieve the actual text strings. Then the question is whether those text strings would have any embedded nuls as well.

[Michael Mattias]

FWIW you don't need controls to hold data associated with a window....

A. SetWindowLong %GWL_USERDATA, value (retrieve with GetWindowLong)
B. SetProp hWnd, $PROPERTY_NAME, value (retrieve with GetProp)

Value is an integer; so you can make it a pointer or a handle.

[Fred Harris]

GlobalAlloc

Are the older Global/Local Alloc functions still worthwhile using? I used to use GlobalAlloc() quite a bit but several years ago I stopped using it because my MSDN documentation from around 1998 (which I still use) even at that early date said these functions were only for compatibility with 16 bit windows...

Global and Local Functions

The global and local functions are the 16-bit Windows heap functions. Win32 memory management 
supports these functions for porting from 16-bit Windows, or maintaining source code 
compatibility with 16-bit Windows. The global and local functions are slower than the new memory
management functions and do not provide as many features. Therefore, new applications should not 
use these functions. 

A process can use the GlobalAlloc and LocalAlloc functions to allocate memory. Win32 memory 
management does not provide a separate local heap and global heap, as 16-bit Windows does. As a 
result, there is no difference between the memory objects allocated by these functions. In addition,
the change from a 16-bit segmented memory model to a 32-bit virtual memory model has made some of 
the related global and local functions and their options unnecessary or meaningless. For example, 
there are no longer near and far pointers, because both local and global allocations return 32-bit 
virtual addresses.

[Richard Angell]

Ms

Not exactly a red flag, Roger.

PB's dynamic stings can be reurned 2 ways, as explained nicely in the referenced in-line assembler section. Each dynamic string in memory is preceeeded by a 4 byte (LONG/DWORD) value that is the total length. Using that value one can code to return any string with a wee bit more code.

[Roger Rines]

I always learn from these types of responses especially when the postings provide extended information about why things work the way to do. I also like the variety of the responses because they give breadth to the solutions possible and that always gives me more options.

Thank you all for responding.

Fred & Richard I appreciate the extra effort you gave with your extended information because it filled in some holes I have about how to go beyond the simple memory handling methods that dominate most of the tools I create, and the information also generated questions about how to cleanup after I want things to stop.

From a task perspective, I selected John’s approach because it seemed so elegant and closest to how the sample dialog code was intending when a control not was available, and because it seemed like I could clear the memory space with an ERASE byBuffer() call before the function doing this work terminated. Is my assumption correct that ERASE is all I’ll need to clear the buffer space?

A little more information:
I can see I wasn’t as clear enough when I said there wouldn’t be any controls available to handle the process. This posting is part of a project to create an intermediary DLL that will serve as the process handler between a third-party program (TradeStation) and a DLL that will provide a better approach to one of TradeStation’s current capabilities. My project will need to accept a series of Asciiz string addresses from TradeStation, use TradeStation’s API DLL to get the string information at those locations and then pass that data to another DLL where it will be processed.

Right now I’m finding that John’s solution is working well, but my understanding of how to cleanup memory issues that need more than an ERASE Array() call is lacking. Knowing more about what else is needed for cleaning up memory after I’m done would certainly be appreciated and adopted.

In Richard’s DLL process example, which I enjoyed understanding and executing, he mentions

This method does not allow for any NULs embedded in the string. For a normal edit box this is not likely to be a problem. There are other methods for strings with embedded nuls that require a wee bit more code.

Given what I mention above, Richard’s caveat is now a big red flag blowing over this work. Do I need to do more than strip the $Nul character out when the Asciiz string arrives? Or, were you just talking about what is being passed to a text control?

[Richard Angell]

For reference, I've tried passing the address handle of the variable length text data padded with a $Nul terminator to the DLL by using VarPtr to pass the handle of the padded string, but that didn't work.

It is easy to use a dynamic string in the DLL and return a pointer with VARPTR which points to the string handle. However you need to treat it as a pointer to a pointer. This is important here because perhaps your edit control's text can change between queries. Depending on how the length changes the string engine may place the data in a new memory location and update the string handle's value with the new address to reflect the change.

Manage your own memory block if you want, but this will do the job just as well for how it seems to be described. If you have a lot of controls, it should be fairly easy to see how this concept can be extended to work with arrays of dymanic strings as well.

First a small DLL that shows the principle

#COMPILE DLL "AP2DS.DLL"
#DIM ALL

%USEMACROS = 1
#INCLUDE "Win32API.inc"

GLOBAL ghInstance AS DWORD
GLOBAL pTextBoxString AS DWORD PTR
GLOBAL TextBoxString AS STRING

FUNCTION GetTextPtr ALIAS "GetTextPtr" ()EXPORT AS DWORD
    'Here you would actually need to get the string from the real Edit Control
    'We are just spoofing it here for simplicity of demonstration.
    'Assumes the caller only cares for the stripped string.
    'In memory all dynamic strings have the NUL [CHR$(0)] appended as a last character
    '(see In-Line Assembler topic "Passing dynamic strings")
    TextBoxString = CHR$(0)  'initialize the global here in the DLL
    pTextBoxString = VARPTR(TextBoxString)  'point to the handle
    FUNCTION = pTextBoxString
END FUNCTION
FUNCTION GetTextByPtr ALIAS "GetTextByPtr" (BYVAL SomePtr AS ASCIIZ PTR)EXPORT AS STRING
    STATIC swstr AS LONG
    IF SomePtr <> pTextBoxString OR TextBoxString = "" THEN
        FUNCTION = ""
    ELSE
        'in the real DLL this is where code:
        'CONTROL GET TEXT hDlg,%ID_EditCtrl to TextBoxString
        IF swstr = 0 THEN
            swstr = 1
            TextBoxString = "This is a string of variable size"
        ELSE
            swstr = 0
            TextBoxString = "This is a longer string that was assigned to the dynamic string in the DLL"
        END IF
        FUNCTION = @@SomePtr
    END IF
END FUNCTION

FUNCTION LIBMAIN (BYVAL hInstance   AS LONG, _
                  BYVAL fwdReason   AS LONG, _
                  BYVAL lpvReserved AS LONG) AS LONG
    SELECT CASE fwdReason
        CASE %DLL_PROCESS_ATTACH
            ghInstance = hInstance
            FUNCTION = 1   'success!
        CASE %DLL_PROCESS_DETACH
            FUNCTION = 1   'success!
        CASE %DLL_THREAD_ATTACH
            FUNCTION = 1   'success!
        CASE %DLL_THREAD_DETACH
            FUNCTION = 1   'success!
    END SELECT
END FUNCTION

and a test program to see the action

#COMPILE EXE
#DIM ALL
DECLARE FUNCTION GetTextPtr LIB "AP2DS.DLL" ALIAS "GetTextPtr"() AS DWORD
DECLARE FUNCTION GetTextByPtr LIB "AP2DS.DLL" ALIAS "GetTextByPtr"(BYVAL SomePtr AS ASCIIZ PTR) AS STRING
FUNCTION PBMAIN () AS LONG
    LOCAL MyAPTR AS ASCIIZ PTR
    MyAPTR = GetTextPtr()
    ? GetTextByPtr(MyAPTR) ' simulating getting text that is changed
    ? GetTextByPtr(MyAPTR) ' at various times over the life of the edit box
    ? GetTextByPtr(MyAPTR) ' which may be your real world situation
    ' WAITKEY$             'uncomment for use with PB/CC
END FUNCTION

Added the following caveat:
This method does not allow for any NULs embedded in the string. For a normal edit box this is not likely to be a problem. There are other methods for strings with embedded nuls that require a wee bit more code.

[Michael Mattias]

>a trade off between simplicity vs. speed...

.. and resource utilization... and features.....and maintainability.....and lead time... and cost.....

All applications involve tradeoffs.

This is the kind of thing I want to get into when I move away from applications development. I really, truly believe the modern development tools have lulled the average developer into a belief these things don't matter... or at least do not have to be considered very carefully when designing.

True, faster processors, cheaper memory and more expensive programmers have mitigated against spending a lot of time thinking about these things, but as far as I'm concerned the 'tradeoff decision-making process' is a fundmental skill and developers should be well-grounded in all fundmentals.

[Kev Peel]

Yes, the handle option is more efficient but then you need to "GlobalLock" to use it.
Depends on the application as always - a trade off between simplicity vs. speed.

[Michael Mattias]

> I prefer GlobalAlloc(%GPTR, <strsize>+1).

Using the GPTR option fixes the location of the block in your process' virtual address space. Using GHND instead (plus GlobalLock/Unlock when access is needed) allows Windows to relocate the memory as required and results in a more efficient use of your virtual memory.

Not that the typical allocations made with GlobalAlloc (4 Mb or less) are all that big that this will become a real performance factor, but it's worth thinking about, at least from time to time. ....

[added]
oh yeah... you can't use a GPTR-allocated block to transfer to the clipboard... so that might be an additional consideration....

[Kev Peel]

There are plenty of ways to allocate variable memory. I prefer GlobalAlloc(%GPTR, <strsize>+1). It's simple, effective, and can be passed around the program without going out of scope. Just remember to GlobalFree it afterwards

[Chris Boss]

Use a variable length string for the string buffer.

LOCAL nLength AS LONG, TextBuffer$
 CONTROL SEND hDlg, %IDC_MESSAGE, %WM_GETTEXTLENGTH, 0, 0 TO nLength
 IF nLength>0 THEN
      TextBuffer$=STRING$(nLength+1,CHR$(0))
      ' Pass the buffer handle to the string buffer
      lpBuffer = STRPTR(TextBuffer$)
      CONTROL SEND hDlg, %IDC_MESSAGE, %WM_GETTEXT, nLength + 1, lpBuffer
      TextBuffer$=LEFT$(TextBuffer$,nLength)
 ELSE
      TextBuffer$=""
 END IF

[Michael Mattias]

>Tried Poke$ byBuffer(0), MsgTxt$, ... but that GPFs nicely.

Of course it does.

==> POKE$ address, stringvar

ByBuffer(0) is not an address: it's the ASC() of the first character of the string.

[John Gleason]

I think an efficient way to do it if you have msgTxt$ already $NUL padded might be:

   Dim lpBuffer As Asciiz Ptr
   LOCAL msgPtr AS LONG

   msgPtr = STRPTR(msgTxt$)
   REDIM byBuffer(nLength + 1) As Global Byte AT msgPtr

   lpBuffer = VarPtr(byBuffer(0))

Now your string is in the byBuffer() array.

[Fred Harris]

pointers maybe???

Roger,

Memory to hold a string of characters, bytes, or data in any form can be always had through memory allocation calls, unless all memory is used up, of course, but that is rather uncommon in 32 bit Windows (certainly wasn't uncommon in DOS though!). Once memory is obtained the bytes have to be moved there, as I believe you are trying to do with Poke$. Backing up a bit though, the return value from a memory allocation call is always the first byte address of a linear address space.

There are a number of memory allocation functions, but lets take HeapAlloc() as an example. Lets suppose you have a string of 35 bytes that are in a text box and you need to arrange temporary storage for them before sending them or a pointer to them to some external function. It would look something like this...

Local szBuffer As Asciiz*128
Call GetWindowText(hTextBox,szBuffer,128)

assume szBuffer holds your text, and the string it contains is 35 chars long

Local hHeap As Dword
Local pszStr As Asciiz Ptr

hHeap=GetProcessHeap()
pszStr=HeapAlloc(hHeap,%HEAP_NO_SERIALIZE,36)
Poke$ Asciiz, pszStr, szBuffer

After the Poke$ your string of 35 bytes will have been copied to the address held in pszStr, and that pointer could be passed 'as is' to an api function requiring a pointer to a string. However, be aware that an api function needing a pointer to a string wants the address of the first byte of the string and because the standard parameter passing mechanism of BASIC is to take the address of a variable put in a parameter list in a function, you need to put a Byval in front of the pointer or the address of the pointer will be passed, not the address of the starting address of your string.

But you know, in the case above, the function that wants that string would be just as happy with this...

Varptr(szBuffer)

as that's the buffer you moved the text out of the text box to. Am I answering your question???

[Edwin Knoppert]

Control Get TEXT ??

[Roger Rines]

I have a code sample showing how to retrieve text from an edit control that is then sized for length of the text, plus 1 byte. With the length determined by “text-length+1” that value is then used to size an Array of Global Byte(s) created as data container for the text in the edit control.

An Asciiz Pointer is then declared and given the handle of the Array of Bytes so that after the text is moved into the Array of Bytes the handle can then be passed to an external DLL as an Asciiz Pointer where text is retrieved.

I've never had to move text into an Array of Bytes, and while I could easily pass text into an Asciiz string, that won't work because the compiler defines the size of the Asciiz String at compile time and the DLL getting the text wants the message text trimmed before passing.

We can only dimension Asciiz variables at compile, but arrays can be sized during execution so it seems like a neat way to get around the fixed length variable sizing when the potential size of a message text length could be large or tiny and always is unknown at compile.

Here is the offered sample section of code that works great in a Dialog process:

   Local nLength As Long
 
   '  Determine the Size of the Message being sent by getting the
   '  length of the message text that was entered by the user
   Control Send hDlg, %IDC_MESSAGE, %WM_GETTEXTLENGTH, 0, 0 To nLength
 
   '  Allocate a temporary buffer that is large enough to contain the
   '  message text and a pointer to that buffer
   Dim byBuffer(nLength + 1) As Global Byte
   Dim lpBuffer As Asciiz Ptr
 
   ' Pass the buffer handle to the Asciiz Ptr
   lpBuffer = VarPtr(byBuffer(0))
 
   '  Put the Message Text into the lpBuffer
   Control Send hDlg, %IDC_MESSAGE, %WM_GETTEXT, nLength + 1, lpBuffer

For reference, I've tried passing the address handle of the variable length text data padded with a $Nul terminator to the DLL by using VarPtr to pass the handle of the padded string, but that didn't work. Tried Poke$ byBuffer(0), MsgTxt$, and lpBuffer = VarPtr(byBuffer(0)), but that GPFs nicely.

How do I solve this kind of an issue when I don't have a Control to act as an intermediary in the work?