UDT Array Subscripts

[Cliff Nichols]

According to the docs

The total size of a UDT must be known at compile-time, so items like dynamic strings, which vary in size, cannot be part of a TYPE structure. A STRING PTR can, however, since a pointer is implemented as a DWORD.

Does that mean I can somehow have a pointer to an array so I can dynamically add/delete/sort array elements in the UDT?????

Reason I ask, is I have 2 situations that a UDT that has elements that can be an array of elements would be PERFECTLY suited for. However, I will not possibly know how many would be in the array until run-time.

Is it possible? and if so, how would I declare the array element of the UDT????

[Michael Mattias]

The number of elements in a table (what an "array" in a UDT should be called) is fixed at compile time, period.

What you CAN do is..

TYPE FOO 
   myarray AS STRING PTR 
  .....
END TYPE 

FUNCTION PBMAIN() 
 
  LOCAL S AS STRING , F AS FOO 
  LOCAL iARray() AS LONG 

   S = "" 
   ' initialize Foo:
   ....
   Foo.myarray  =  VARPTR (S) 
  ' fill the array in Foo with some number of sorted integers 
   CALL MyProc (Foo) 
   NumEl = LEN([email protected])\4 
    
   REDIM iArray(NumEl-1) AT STRPTR ([email protected]) 

   FOR Z = 0 TO NumEl-1 
        PRINT iArray (Z) 
   NEXT
  ' free the memory 
   [email protected] = ""
  'since iARray is now pointing at nothing best get that cleaned up, too
   ERASE iArray() 

.....

FUNCTION MyProc (F AS foo) AS LONG 

  LOCAL  iarray() AS LONG    ' local array 

  [email protected] =   STRING$(numElements * 4, 0)   ' allocate memory 
  REDIM  iArray (numelements -1) AT STRPTR ([email protected]) 
  Fill iarray with random integers 
  Array Sort iArray ()    ' sort'em 

...

Instead of a string / string PTR like this, it might be easier with the CC 5x/Win 9x compilers to use something like...

TYPE FOO
     myArray  AS LONG 
    ....
END TYPE 

...   
   GLOBAL MEM ALLOC   numelements *4 TO FOO.myArray   ' save the handle in the UDT

.. and then use the GLOBAL MEM LOCK, UNLOCK and SIZE statements to access your data.

I don't see a GLOBAL MEM REALLOC, so I guess you'll have to FREE and ALLOC if you want to change the size. (Strange there is no REALLOC, isn't it?)





MCM

[jcfuller]

If possible use a class instead of a udt. I have found a class much more flexible than a udt.

James

[Cliff Nichols]

Where am I going wrong?

I tried following suit with MCM's pseudo code but decided to try something simpler and go with a long ptr. but things were out of whack with "Dim At"

So I broke things down even more (no UDT's or anything till I can understand what I am doing wrong???)

Below is Demo Code of how to replicate my problems, but from reading the docs, I do not see why I get the results that I do? and why the docs show both a "Preserve" and "AT" but the compiler gives me an error if I try?

#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG
     LOCAL MemoryLocation AS LONG
     LOCAL MemoryLocationPtr AS LONG PTR
     
     DIM MyArray() AS LONG
     DIM MyArray2() AS LONG AT VARPTR(MemoryLocation)
     MemoryLocationPtr = VARPTR(MemoryLocation)
'*** Works as expected
     REDIM MyArray(0)
     REDIM MyArray2(1)
     MSGBOX "After Redim" + $CR + $TAB _
               + "Expected MyArray UBOUND = 0, Actual = " + STR$(UBOUND(MyArray)) + $CR + $TAB _
               + "Expected MyArray2 UBOUND = 1, Actual = " + STR$(UBOUND(MyArray2)) + $CR _
               + "At Memory Location " + STR$(MemoryLocationPtr)
               
     MyArray(0) = 5
     MyArray2(0) = 10
'*** Works as expected
     REDIM PRESERVE MyArray(UBOUND(MyArray) + 1)
     REDIM PRESERVE MyArray2(UBOUND(MyArray2) + 1)
     MSGBOX "After Redim Preserve" + $CR + $TAB _
               + "Expected MyArray UBOUND = 1, Actual = " + STR$(UBOUND(MyArray)) + $CR + $TAB _
               + "Expected MyArray2 UBOUND = 2, Actual = " + STR$(UBOUND(MyArray2)) + $CR _
               + "At Memory Location " + STR$(MemoryLocationPtr)

'*** What the heck is going on? Ubound becomes 0 and I can not use preserve with an absolute array???
     REDIM PRESERVE MyArray(UBOUND(MyArray) + 1)
     REDIM MyArray2(UBOUND(MyArray2) + 1) AT VARPTR(MemoryLocation)
'     REDIM Preserve MyArray2(UBOUND(MyArray2) + 1) at varptr(MemoryLocation)
     MSGBOX "After Redim Preserve at memory location" + $CR + $TAB _
               + "Expected MyArray UBOUND = 2, Actual = " + STR$(UBOUND(MyArray)) + $CR + $TAB _
               + "Expected MyArray2 UBOUND = 3, Actual = " + STR$(UBOUND(MyArray2)) + $CR _
               + "At Memory Location " + STR$(MemoryLocationPtr) + $CR + $TAB _
               + $CR + $TAB _
               + "Also notice that I can not use keywords 'PRESERVE' and 'AT' for some reason???"

END FUNCTION

What sort of am I doing this time???? The more I the more I can not see the tree from the forest

[Michael Mattias]

This statement here...

DIM MyArray2() AS LONG AT VARPTR(MemoryLocation)

.. should not even be allowed. There is no element count! I'd call that a compiler error.

"PRESERVE" and "AT" in the same REDIM statement makes no sense, since "AT" means YOU are managing the memory yourself and there is nothing for the compiler to PRESERVE.


Pray, "What" are you trying to do? Save a variable amount of data in a UDT?

In that case, I'd go with the "membername AS LONG" and have it retain the handle to a block of memory (GLOBAL MEM ALLOC).

The resize could be tricky if you want to preserve, but very doable.

You could also do it with PTRs, but then you'd probably want to use malloc, realloc, free for PB/Win32 September 20, 2002 to do your allocations directly to a pointer. I think I put a realloc in there, but if I didn't it's not hard , just call HeapRealloc passing the ptr)

WARNING: if you use malloc et all, SIZE is not going to tell you the number of elements in your array, since HeapAlloc works in a granular fashion (I think it's in multiples of 16 bytes), so you'll want to add a "valid item count" member to your UDT to keep track of how much of your "pArrayData" (a LONG PTR) is valid.

MCM

[Paul Squires]

We have had debates about dynamic tables/arrays in TYPEs before. Personally, I vote 100% in favor of them. There is zero downside for the programmer other than if the programmer wants to save the data to a disk file (and in those cases the programmer would have re-thought the decision to use the dynamic element of TYPE tables/arrays anyway). If you are not saving the data to disk then the TYPE becomes no different than any other abstract data type, albeit a hell of lot more flexible.

With the release of PB9, I simply gave up using MemAlloc hacks for dynamic elements in TYPEs and simply use CLASSes now. They are much easier to implement and maintain and the compiler handles all of the messy details of allocating/deallocating memory and scoping issues.

[Michael Mattias]

is zero downside for the programmer.....

The "downside" is the loss of the compile-time optimization which can be done PRECISELY BECAUSE UDTs are a fixed size. That 'size' value as well as the known offset of each member are used many, many times by the compiler.

You want dynamic data? Use a VARIANT, or a FIELD'ed buffer. Or manage memory yourself as shown here. For that matter, maybe you can have a UDT with...

TYPE FOO
    v  AS VARIANT
END TYPE

(I am going to have to try that one of these days. But even if it doesn't work, I know you could use "AS VARIANTAPI" or "TAGVARIANT", whatever is in the Win32API.INC file for the variant data structure).

The user-defined type as implemented by PowerBASIC is what it is and does what it does and does not do what it does not do. If that does not fit your application, a UDT is probably the wrong choice.

MCM

[Michael Mattias]

FWIW:Referring to tables in UDTs as "arrays" was still IMO a Big No-No.

One thinks of arrays as resizeable whereas one does not think of "tables" as resizeable.

[Cliff Nichols]

Code:
DIM MyArray2() AS LONG AT VARPTR(MemoryLocation)
.. should not even be allowed. There is no element count! I'd call that a compiler error.

I don't think so. Since in this case according to the docs

when an absolute array is Dimensioned (at a specific location in memory using the AT address syntax), PowerBASIC does not initialize the memory occupied by the array. Further, when an absolute array is erased, the memory is not released either.

Which means, I create, I dimension, I destroy......its not up to PB to tell me I may have made a mistake.

Now "DIM MyArray() AS LONG" on the other hand I would think would be a compiler error, but then again maybe not if I think of it as what's happening is I am getting a pointer to a memory location where my array will be. Or extrapolated from the docs, "It is declared, but not dimensioned yet" (Testing would show LBOUND(MyArray) would = 0, and UBOUND(MyArray) would = -1 (Which are checks I use in other projects to see if the array is usable or not)

Pray, "What" are you trying to do? Save a variable amount of data in a UDT?

Thats what I thought I said in my original post, but I guess its not as obvious as I thought I had said it. But Yes, an element of the UDT would be an array of unknown dimensions until runtime, so my 2 choices are use a huge value and hope I never exceed that amout, or use a pointer like you would a string pointer for dynamic strings.

Since I have never used "Dim AT" nor an array in a UDT, I did what every good programmer does when debugging a problem (especially when they are just learning the concepts) that 1st you nail down the basic concept and possible errors, then you move on to the next concept (and its errors) if they still exist.

That way you can find what may be "A comedy of errors" that can be caused by multiple concepts being mis-used when you really do not know what you are doing yet.

So I have gone back to basics to understand the differences of DIM and "DIM AT"

From the docs (and my comment after)

DIM
var[(subscripts)] 'Variable or Array
[AS [GLOBAL | INSTANCE | LOCAL | STATIC | THREADED]type] 'Data Type
[PTR | POINTER] 'Pointer
[AT address] 'At a specific memory location
[, ...] '<--- No Idea, maybe a ", TO var" type of thing????

I found no mention of not being allowed to intermix the optionals

ReDim on the other hand

REDIM
[PRESERVE] 'Save the data already in the array
array[(subscripts)] 'Array Dimensions
[AS type] 'Data Type
[AT address] 'At a specific memory location
[, ...] '<--- No Idea, maybe a ", TO var" type of thing????

Which clearly shows "PRESERVE" and "AT" in the same statement as optional. However I found another post from "Edwin Knoppert" that made mention to "PRESERVE" and "AT" are NOT allowed. and MCM made mention that it should be a compiler error. (But I was at work at the time, and now I can not find the posts to finish reading)

I will take a look at the Alloc functions, but my "Gibbs Gut" is telling me that 1st I need to understand the array part, and why errors where I would not expect them to be.

I am leaning toward emailing suppt about the docs perhaps being in error, but not ready to until I am sure its either a misdocumentation, or a "issue" (<--- Notice I did NOT use the "B" word ) and not something of my own doing.

Meanwhile if it is my own doing could someone PLEASE explain why if I

"DIM MyArray2() AS LONG AT VARPTR(MemoryLocation)"

so that I am in charge of "I create, I resize, I populate, I destroy" and I later

"REDIM MyArray2(UBOUND(MyArray2) + 1) AT VARPTR(MemoryLocation)"

that my array becomes one of zero elements?

Or does the word "AT" throw EVERYTHING out the window, and all keywords "REDIM, PRESERVE, LBOUND, UBOUND.....etc" are no longer valid to redimension or get the size of or etc....????

Whew.....what a lot to digest, but you see why all my confusion

[Cliff Nichols]

Wow...more posts, while I was in the midst of posting.

MCM brought up a nice point to research once I got the basics down.

You want dynamic data? Use a VARIANT, or a FIELD'ed buffer. Or manage memory yourself as shown here. For that matter, maybe you can have a UDT with...
Code:
TYPE FOO
v AS VARIANT
END TYPE
(I am going to have to try that one of these days. But even if it doesn't work, I know you could use "AS VARIANTAPI" or "TAGVARIANT", whatever is in the Win32API.INC file for the variant data structure).

I took a grasp at straws to understand, and found "Variant" is not allowed, but "VariantApi" is, and "TagVariant" does not exist (or at least as far as the "Win32Api.inc" file

(Just to save you some time, but if you want to test for yourself )
Best part is, that it may answer question(s) I had on a completely different subject and could not figure out, so I put on the "Back-Burner" till it either overboils, or makes a mess somewhere

The user-defined type as implemented by PowerBASIC is what it is and does what it does and does not do what it does not do. If that does not fit your application, a UDT is probably the wrong choice.

So true.....I may be attempting to do something that it was not meant for. (like using a screwdriver handle to bang in a nail), but if I do not understand what it was meant for and "HOW" it works and may try anyways just to find ("Everyone Else knew that by "Common Knowledge" what the heck made you try to use it otherwise???")

[Paul Squires]

The "downside" is the loss of the compile-time optimization which can be done PRECISELY BECAUSE UDTs are a fixed size. That 'size' value as well as the known offset of each member are used many, many times by the compiler.

Wrong. If I need a dynamic structure then I would need to have a fixed UDT and a dynamic array dimensioned and maintained separately. Whatever compiler advantage I win with the UDT, I lose with the separate array. I would hypothesize that a UDT that can handle dynamic elements would have greater programmer side optimization gains (better code maintenance and reusability) than what is gained from a few microseconds of compile time optimization.

You want dynamic data? Use a VARIANT, or a FIELD'ed buffer. Or manage memory yourself as shown here. For that matter, maybe you can have a UDT with...

TYPE FOO
    v  AS VARIANT
END TYPE

How is that any easier than using MemAlloc routines to handle the memory allocations/deallocations myself via a pointer in the UDT? It still doesn't allow me to save the UDT to disk which is the only benefit of a fixed size UDT.

The user-defined type as implemented by PowerBASIC is what it is and does what it does and does not do what it does not do. If that does not fit your application, a UDT is probably the wrong choice.

That's an obvious observation. It is not what I was saying. Everyone knows how the UDT in PB is implemented.... what I'm saying is that it can be enhanced to make it better. Just view my post as a new feature suggestion - one that I have previously sent to PB Support. Asking for dynamic elements in a UDT is no different than asking for the INSTR function to search from the end of a string using a negative offset, but yet somehow asking to tamper with the concept of a "UDT" seems sacrilegious. I could care less that a UDT has always been what it has been. That doesn't mean that it can evolve into something better. Visual Basic implemented the feature and I don't remember the Internet melting from any outrage over breaking any "rules" about what a UDT should be. Just because it is what is doesn't mean it needs to be what it is. PB5 was what it was, but yet PB9 is better because we allowed and embraced change.

Now, how's that for a Canadian taking a stand?

[Paul Squires]

FWIW:Referring to tables in UDTs as "arrays" was still IMO a Big No-No.

One thinks of arrays as resizeable whereas one does not think of "tables" as resizeable.

One thinks of dynamic arrays as resizable. An array to me can be fixed size or variable. It's no different than a string can be fixed size or variable. An ASCIIZ string is no more than an array of characters.

FWIW, I always think of Tables as resizable. I equate a Table to database terms: One that grows and shrinks as rows are added and removed.

A sematic game can be played on the terms Array, List, Table, etc... all day long.

Oh, and BTW, the TYPE structure needs to be extended to allow dynamic string elements as well. Every PB variable should be able to be used in a UDT. I'd love to have UDT elements such as MyString As String.

That's just my opinion of course.

[Martin Draper]

For my 2 cents worth, not having dynamic strings in UDT whether used in files or not, is a total pain. The oldest basic complier CB86 supported it, visual basic supports it and there are others. I have done very extensive work with B+Trees for database indexing. To do develop the technology in power basic I had to develop a dynamic file system.

Basically to overcome the problem you have to have field delimiter that defines the end of the field. While it does add an additional character for each field, space is not wasted with empty fields or partially filled fields. Which is what happens with database engines ( Varchar ).

The best example of this is to create a B+tree index on any field/s in records in a database file. You have to be able to handle any size field for the keys in the tree. If you have a simple tree with 2 keys per node is one thing but wihen you have 32 keys for node which you need for very large databases it is a real problem.

[Michael Mattias]

Isn't the bottom line simply....

"PowerBASIC's UDTs are always a size fixed at compile time. Deal with it."

FWIW, I am POSITIVE there are multiple posts for "arrays in UDTs" in the Source Code Forum which, yes, "deal with it."

Also FWIW, here's a UDT containing multiple "arrays" for my current project:

TYPE JWindowParam
  bnew  AS LONG                        ' if true this is a NEW job (else change assumed)
  ControlConnect  AS OLEDBCONNECTION   ' info to connect to the tsi_xxx tables maintained here
  ' next four are basically the info we need to either maintain the jobs
  ' or execute them.
  ' the control bp, bppt and bppd tables are nothing more than the bp, bppd and bppt tables
  ' with a jobID added.
  ' the bh (batch header) will have our control info in it.

  hdr             AS BATCH_HEADER                ' includes %JOB_TYPE_XXX and szName
  bp              AS BATCH_PROC
  nbppt           AS LONG                        ' number of BPPT in array
  BPPT            AS BATCH_PROC_PARM_TYPE PTR    ' malloc'd ptr
  nBPPD           AS LONG                        ' number of BPPD in array
  pBPPD           AS BATCH_PROC_PARM_DTL  PTR    ' malloc'd ptr
END TYPE

For both the BPPT and BPPD members , I access these in called procedures by passing the JWindowParam to that procedure and then using an absolute array on top of it...

FUNCTION Kit_CreateKitWindowControl (BYVAL hWnd AS LONG, JWP AS jWIndowParam) AS LONG

LOCAL BPPT() AS BATCH_PROC_PARM_TYPE, _
         BPPD() AS BATCH_PROC_PARM_DTL  

     REDIM  BPPT (JWP.nBPPT-1) AT JWP.pBPPT
     REDIM BPPD (JWP.nBPPD-1) AT  JWP.nBPPD 
....

     'HERE I CAN 'do array stuff"   with both BPPT() and BPPD().

When I have to resize, I use Realloc() to get a new pointer at pBPPT or pBPPD and change the element counts in either nBPPT or nBPPD

Works for me.

MCM