Accessing a Windows global memory buffer

[John Harvill]

Dear Sirs,

I'm attempting to access a "Windows global memory buffer" created by a third-party library for managing data being transfered from quickly sampling analog-to-digital I/O cards. The documentation for this library supports C/C++, Visual Basic, and Delphi. The third-party library is called the "Universal Library" from Measurement Computing for supporting their I/O cards, and is presently (at this writing) a free download.

According to the documentation, we initiate the global Windows memory buffer with a function call, which returns a handle to the buffer. Then we pass the handle to other functions which manage the transfer of data from the I/O cards to the Windows global memory buffer. To access the buffer, if we are using Visual Basic we use another function to copy the contents of the buffer to an array, or if we are using C++ we can cast the handle of the buffer a pointer of the appropriate type (such as unsigned short, which is a Word in PowerBasic) with statements as follows:

unsigned short *DataArray=Null
DataArray = (unsigned short*)MemBuffer

My question is whether I can somehow directly access the Windows Buffer in PowerBasic. For example, can I perform a similar pointer cast as done in C++, or maybe overlay a PowerBasic Array over the Windows global memory buffer. What do I do with that handle? :thinking:

I've been working on converting the header file for the Universal Library to support PowerBasic 8.04 syntax macros and function definitions, and have had some success using the library for functions which do not involve arrays or buffers.

Sincerely,
John Harvill

[Fred Harris]

The 'handle' to a "Global Memory Buffer" is, I'm assuming, the return from one of the Global memory allocation functions, which, far as I know, is a typical virtual memory address, just like any memory address we have access to in PowerBASIC (or C or C++ or anything else, for that matter).

Of course, with PB you don't need casting, as its just a 32 bit unsigned number. If you know you have an 8K buffer containing 2K 32 bit quantities, it can be accessed like so...

Local ptrDword As Dword Ptr

ptrDword= Global Function Return (a handle in C parlance)

To access the fifth zero based element that would be...

a_Dword_var = @ptrDword[5]

Here is a link to using pointers with PowerBASIC that might be useful...

http://www.powerbasic.com/support/pbforums/showthread.php?t=25165&highlight=Fred+Harris

[Cliff Nichols]

Hi John,
On occasion I have used the "Universal Library" from Measurement Computing, so I would be interested if/when you complete the port that it get posted in the source code forum.

I think the post from Fred Harris is a good place for starters

[Michael Mattias]

with a function call, which returns a handle to the buffer. Then we pass the handle to other functions ..

As a general rule, when a library returns a "handle" it means you must treat that handle opaquely and do all operations on that handle's underlying object (in this case a block of memory) using only the functions provided for this purpose.

However...since this library (apparently) has a function to return a pointer to the memory, PTR variables are probably the easiest way, as shown by Mr. Harris.

Unless I misread your post, and what is available is a copy of, or a pointer to a copy of, that data... if it's a pointer to a copy, PTR variables are good; if it's a copy of the memory, you can store it in a string, an array, or in a block of memory you allocate yourself.

[Fred Harris]

...one final thought. If you prefer working with indexed array notation instead of pointer notation off of a base pointer (@ptrNum[x]), you can use PowerBASIC's absolute arrays to overlay an array over the memory returned from a function. Then you have a situation where Varptr(arr(0))=ptrReturnedMemory and...

arr(0)[email protected][0]

also...

Mr. Harris thinks we're getting terribly formal around here. What do you think Mr. Nichols?

[John Harvill]

Thankyou everybody for the valuable insights. I need to work on this more with the guidance you've provided me.

What concerns me is that the documentation refers to the address as a "handle" which I understand to be defined in Windows as a pointer into a table of pointers which is manipulated by the Windows memory management system. I don't know whether C++ would know about that distinction when casting the memory handle to a pointer.

The documentation says the prototype for the function which allocates the Windows global memory buffer for C++ is

int cbWinBufAlloc( long NumPoints )

while the header file for C++ says

HGLOBAL EXTCCONV cbWinBufAlloc (long NumPoints)

The documentation says for Visual Basic
Function cbWinBufAlloc( ByVal NumPoints&) as Long

while the header file for Visual Basic says
Declare Function cbWinBufAlloc Lib "cbw32.dll" (ByVal NumPoints&) As Long

Which I've translated for PowerBasic as
DECLARE FUNCTION cbWinBufAlloc LIB "cbw32.dll" ALIAS "cbWinBufAlloc" ( BYVAL NumPoints& ) AS LONG

Maybe I should change the return type from "LONG" to "DWORD"

Sincerely,
John Harvill

[Scott Hauser]

Maybe I should change the return type from "LONG" to "DWORD"

Should be OK just as it is: DECLARE FUNCTION cbWinBufAlloc LIB "cbw32.dll" ALIAS "cbWinBufAlloc"(BYVAL NumPoints&) AS LONG

See: Unlike most other functions in the library, this function does not return an error code. It returns a Windows global memory handle which can then be passed to the scan functions in the library. If an error occurs the handle will come back as 0 to indicate that the buffer was not allocated.

[John Harvill]

Update: I went on a wild-goose-chase thru MSDN about the HGLOBAL return type in the header file for C++ programs, and after experimenting with API functions which work on memory allocated with the HGLOBAL type, I've come to the conclusion that the memory Handle is probably not an actual HGLOBAL handle, but instead a simple pointer as suggested by Mr Harris. One clue is when the API function GlobalFlags( MemHandle ) returned "GMEM_INVALID_HANDLE" (0x8000), which means the handle was not a valid HGLOBAL handle. Another clue is when GlobalSize( MemHandle ) returned zero bytes allocated. Finally, I took Mr. Harris advise and set a Word_Pointer with the returned value from the allocation function, and then commanded the ADC board (which I'm experimenting with) to take samples and write to the allocated buffer. Then, I printed out the buffer using the syntax suggested by Mr Harris, @prtWord[ iCount ] wherein I discovered real measurement data.

Real cool. Thanks alot.

About the converted header file, it still requires more refinement to be correct. I need to work more on the array parameters in the function declaration before it should be provided for general use. I'm happy to share what I have so far with the understanding that it is not yet perfected.

Sincerely Yours,
John Harvill

[Michael Mattias]

refers to the address as a "handle" which I understand to be defined in Windows as a pointer

That's a bad assumption or a misunderstanding on your part.

"Handle" is an important concept to understand in the Windows' programming environment. It is - as I stated earlier - a reference which must be treated opaquely.

The hGLOBAL casting "suggests" it is the return of the GlobalAlloc function, except I work with one application where the documentation show the return as a "hGlobal" but the text goes on to explain that the numeric value is in fact a pointer. (a phone call learned it was created using the GlobalAllocPtr() macro, which equals GlobalAlloc() + GlobalLock())

Based on your results.e.g. the value of the handle appears to be a pointer but does not respond well to the GlobalAlloc() family of functions, added to the fact the library was created with 'C' and/or C++ language tells me the memory was probably allocated by the c intrinsic 'malloc' which is implemented .. however the compiler chooses to do it... but I'd guess they did it the same way I did in malloc, realloc, free for PB/Win32 September 20, 2002

However, the general rule still applies: never try to use a "handle" except in the manner described by the publisher. One good reason is, whatever internal meaning that publisher ascribed to the value may change without notice in a future release of the application or library.


MCM

[Fred Harris]

This whole issue somewhat intrigues me, and I've been struggling to remember a good description of the issue I have in one of my Windows CE programming tutorials I have at the office. Right now and for the past week I've been sick as a dog at home with the flu but tomorrow I'll somehow manage to make it back to the office and I'll look that up. If I find anything interesting I'll post it.

Using pointers or using absolute arrays to access the data should be equivalent, but perhaps you might want to look at that. Absolute arrays are an interesting feature of PowerBASIC that I've only played with for a few minutes and have never used an any real apps, but they look interesting.

[John Harvill]

I would normally leave Handles in places meant for Handles. It was the recommendation in the documentation to read this buffer by casting its handle to a pointer to an unsigned short (ie. WORD) which I thought could be implemented in PowerBasic. I'm betting that the Universal Library is committed to this implementation, because if a future revision breaks the code which performs this casting as recommended in the current documentation, then many customers who followed the documentation will be adversily affected.

Sincerely,
John Harvill

[Michael Mattias]

> then many customers who followed the documentation will be adversily affected.

The publisher of this library is 100% in the right even if he DOES change the "meaning" of a "handle value" - as long as all the documented functions continue to 'work as advertised.'

That one or more users 'assume' some specific meaning to the value of a handle is, well, what those users deserve for not treating the handle opaquely.

MCM

[John Harvill]

I disaggree. I am not assuming the meaning of this particular memory handle when I (the user) am following the documentation like a cook-book. Let me quote the notes section for the function cbWinBufToArray() which copies the contents of the Windows global memory buffer to an array, which appears to be intended for Visual Basic users who don't have direct access to the memory buffer. The documentation says:

Although this function is available to both Windows C and Delphi programs, it is not necessary, since it is possible to manipulate the memory buffer directly by casting the MemHandle returned from cbWinBufAlloc() to the appropriate type. This method avoids having to copy the data from the memory buffer to an array.

Refer to the following example:

/*declare and initialize the variables*/
long Count=1000;
unsigned short *DataArray=NULL;
int MemHandle=0;

/*allocate the buffer and cast it to a pointer to an unsigned short*/
MemHandle = cbWinBufAlloc(Count);
DataArray = (unsigned short*)MemHandle;

/*Scan the waveform data*/
cbAInScan(......,MemHandle,...);

/*print the results*/
for(int i=0; i<Count; ++i)
printf("Data[%d]=%d\n", i, DataArray[i])

/*free the buffer and NULL the pointer*/
cbWinBufFree(MemHandle);
DataArray = NULL;

In conclusion, those users who follow the directions to cast the memory handle into a WORD pointer are simply following directions. New revisions of the Universal Library should be backward compatible with these directions or many users will be adversily affected.

Sincerely,
John Harvill

[Michael Mattias]

Your link doesn't work; it is translated as

mk:@MSITStore:K:\MCC\ULHelp.chm::/ Function_Reference/Windows_Memory_Management_Functions/cbWinBufAlloc().htm

.. so I can only "assume" the argument passed is "size wanted."

If the documentation says you may cast the return of that function as a pointer, then the function is behaving as advertised and it "should" always work.

However, you need to bear in mind that there are at least three ways I can think off the top of my head as to how that function may be implemented 'under the hood':

hMem = GlobalLock (GlobalAlloc (size, flags)
hMem = HeapAlloc (hHeap, size, flags)
S = SPACE$(size): hMem = STRPTR (S) (SysallocstringByteLen underneath this)

While this does not affect what "hMem" means, it DOES have an impact on both querying the size of the buffer and how that buffer must be de-allocated when you're done with it... which is why the library writer is responsible for creating functions to do those things and all the user (you) need to pass is the "handle."

Here's another example. More common example of a "handle"

  hFile = FREEFILE

When you get this value from the PB compiler, what does it mean to you? Answer: Nothing, except "hfile" is a number you must use to do anything with a file opened using that handle.

I'm sure internally the compiler writers have some kind of "meaning" assigned to this value, but that value has meaning ONLY to the runtime library and you, the programmer, must treat this handle opaquely.

I know I am beating up on the semantics of the word "handle" here, but "handle" is such a significant concept when programming in Windows that it deserves some repetition and reinforcement.

MCM

[Fred Harris]

It does appear that the allocation functions that return a HGLOBAL or HLOCAL are cast to those value before being returned by the high level allovation functions - meaning they were originally something else - as Mr. Mattias keeps alluding to 'handles'. At a lower level these allocation functions no doubt must call the virtual memory allocation functions which return 'blocks' or 'pages' of bytes, not individual bytes. Beneath that level the Operating system must make the translation between virtual and physical
addresses. Thank God application programmers don't have to deal with that!

In your above example I'm not sure why MemHandle is even necessary. I'd just try...

DataArray=(unsigned short*)cbWinBufAlloc(...)

But then if the function is defined as returning a long that might not work. But I don't know why they did it that way.

[Michael Mattias]

>HGLOBAL or HLOCAL

It's semantics:

HGLOBAL, HLOCAL : C language :: LONG, DWORD, anything PTR : PowerBASIC language

"What" the function returns is important, always; how the programmer accesses the what varies with source language and application.

In this case, "what" the cbWinBufAlloc function returns is "the address of a block of memory of the requested size." How you choose to use that value is up to you.

(I'm assuming that is how the function is documented).

MCM

[John Harvill]

I guess the example in the documentation sets the memHandle before the DataArray so the program can call cbWinBufFree(MemHandle) after finishing with the memory buffer.

This whole issue is exploring the requirements for passing arrays to library routines written in C/C++. Going beyond the particular handle in question and looking at the general requirements for passing arrays between PowerBasic and the library routines which work with arrays, I notice that the on-line documentation for PowerBasic says that an array is passed by placing the array "descriptor handle" on the stack, while C passes a pointer to the first element of the array. Obviously these are not compatible. It appears we are forced to access the arrays with pointers when those arrays are passed back and forth to routines written in C/C++. In PowerBasic, we must either,

1. Declare a function, where an array parameter is established by an explicit pointer BYVAL (BYVAL PointerVariable AS type), and don't forget the explicit BYVAL PointerVariable in the function call, and then process the array in code with pointers, or

2. Declare the function with a scalar variable BYREF and call the routine with a scalar variable followed by setting a PointerVariable = VARPTR( ScalarVariable ), or

3. Maybe there is another possibility --
Declare the function with a scalar variable BYREF while calling the function using the first element of an array BYREF as follows:

DECLARE FUNCTION libfunction LIB "a.lib" ALIAS "libfun" ( BYREF ScalarVariable )

libfunction ( v(1) )

where v was previously dimensioned and will be later processed using standard array syntax. For example:

DIM V(1 TO 100) AS INTEGER

libfunction ( v(1) )

FOR i = 1 TO 100
something = v(i)
NEXT i


I really appreciate this discussion about memory handles and pointers. Its helping me interface memory buffers and arrays between PowerBasic and routines most likely written in C/C++.

Sorry for the defective link. Oops. I copied the help file on my disk drive, and the link was pointing to another page in help file. I corrected the problem by removing the link from my reply.

Sincerely,
John Harvill

[Michael Mattias]

An "array" (small a) is not the same as a PowerBASIC "Array."

When you read about 'arrays' in a non-PB context, it means "one or more discrete values stored end-to-end in an ordered fashion within a contiguous block of memory."

Array descriptors are not used except when communicating between two or more PB-created procedures. (At which time it is a darn sight handier than the alternatives).

Typically, 'arrays' are passed by passing the number of elements and the address of the first element.

[Scott Hauser]

I guess the example in the documentation sets the memHandle before the DataArray so the program can call cbWinBufFree(MemHandle) after finishing with the memory buffer.

Part of what has led to the semantics here, is due to it being used as both a pointer and handle. From the helpfile:

Unlike most other functions in the library, this function does not return an error code. It returns a Windows global memory handle which can then be passed to the scan functions in the library. If an error occurs the handle will come back as 0 to indicate that the buffer was not allocated.

The scan functions apply to A/D boards only. The handle is then passed to the appropriate cbFunctXX() that apply to specific A/D boards. Unless I am mis-reading the help file, you would pass the memHandle ByVal to these functions. IOW, you use it as a handle even though the cbFUnctS() use it as a pointer to board data addresses (databuffer). After you have completed all of the cbFunc() calls, you must call cbWinBufFree(MemHandle) to free the global memory that was in use.

Scott

[Michael Mattias]

IOW, you use it as a handle even though the cbFUnctS() use it as a pointer to board data addresses (databuffer). ....

Sounds exactly like my situation... memory allocated with GlobalAllocPtr, alhtough it could have been allocated with GlobalAlloc with the GMEM_FIXED flag, because that returns a pointer, too.

... After you have completed all of the cbFunc() calls, you must call cbWinBufFree(MemHandle) to free the global memory that was in use.

That's the key. The library has to provide the de-allocation function, because only the library knows exactly how the memory was allocated.

In my application I have to simulate what the 'original' calling application does... since I know it was allocated with GlobalAllocPtr, I know to de-allocate it I have do GlobalUnlock, GlobalHandle, GlobalFree in that order.

I guess it just comes down to - as it always does - "RTFM."

MCM