DLL Problem

[Michael Mattias]

>not fully understanding the subtlties of DIM and REDIM between the two languages.

Not to mention the difference between PowerBASIC and PowerBASIC... where DIM is only executed once regardless of how many times it is encountered.

I'm sure there is a good reason for this but darned if I can figure out what it is.

REDIM on the other hand is executed every time it is encountered.

[Ken German]

Some final comments: There was never a problem in the DLL with the 2 dimensional absolute arrays created in VB. The PB DLL worked equivalently using DIM...AT or alternatively REDIM...AT. These absolute arrays were never resized once they were created in VB, they were simply read and then refilled with data computed in the PB DLL. I could have made this clearer if I had posted the complete code, but I didn't think anyone would want to wade through something that was over 800 lines long.

My comment regarding using the ERASE command on the absolute array was in the nature of an aside. Using it, or not using it, had no effect on DLL performance. As a VB programmer it merely surprised me initially that ERASE didn't zero the absolute array .

The only problem with the DLL was the with global 2 dimensional arrays in PB that were used to do the computations. About 1 second after I posted the code outline, it occured to me that maybe I should have tried using only REDIM on the global 2 dimensional arrays created in PB. As it turned out this was a much better solution. So if I use only REDIM to set and change the size of the PB globals it is not necessary to ever erase them and the DLL can be called repeatedly without failure. As I did it initially using DIM the first time on the globals and REDIM everytime after that I found it necessary to erase the PB global before returning to VB if I wanted to call the DLL again in the same session. In essence it was my mistake at not fully understanding the subtlties of DIM and REDIM between the two languages.

The final result is that the optimized PB DLL works between 2.3 and 2.7 times faster than the VB compiled code depending on the image. A very significant improvement.

Ken

[John Petty]

Sorry Michael can't understand one line of that code, too much wine.
How did it resize a safe array that can be passed back to VB?

[Michael Mattias]

Whoa, so that's what happened when my server timed out whilst 'replying'.

Let's fix this up later, shall we?

MCM

[John Petty]

Here I am agreeing with you again Michael in fact posted exactly the same suggestion with a snippet of working code to his last question.

http://www.powerbasic.com/support/pbforums/showthread.php?t=38858

PB natively only supports fast pointers for 2 dimensional arrays and Ken has a 3 dimensional array, much discussion on this on another thread. Actually this is not a weakness on the part of PB because if I remember my Assembler correctly the X86 addressing modes are only optimised for up to 2 dimension arrays and so useing pointers for 3 or more could be slower.

As for resizing, agree big problem. If using Absolute arrays the the array can be REDIMMED smaller but never bigger as it does not allocate memory (as the manual says the underlying memory is the responsability of the programmer) so sometime later usually a GPF will occur.

Similarly if using arrays dirctly passed from VB the they are Safe Arrays and you need to use the relevant Windows API functions to resize them.

[Michael Mattias]

You don't even need to use arrays, you can use a pointer variable..

FUNCTION Foo  (Z() AS SOMETHING) ....

  LOCAL pZ  AS SOMETHING PTR , value as SOMETHING 

  pZ  =   VARPTR (Z (0,0)) 
  nEl  =   ARRAYATTR (Z(), 4)  ' get toal number of elements 

  FOR i =  1 TO nEl 
       value =  @pZ
       do something here 
       INCR   pZ 
  NEXT

With two-dimensional arrays you'd have to figure out when you INCR if that's the next row or the next column, and when you'd have to "wrap" to the next column or row, but it's an option.

Alternately, you can use use offset pointers...

 ...
      pz      = VARPTR (Z(0,0)) 
      value =  @pZ [row, column]   ' or maybe it's [column, row] 
      ' assumes zero=based row and column

Resizing (to return the array resized) is going entail using something else, since the 'source' of the data are not another PB-created array, so no PB descriptor is available to the function.

MCM

[John Petty]

As this is a DLL being called from VB then it is also important you show how the data is being passed as VB uses safe arrays.
For once I have to agree with Michael that this could actually be a problem of using Globals. If the arrays were Local then PB would be cleaning up properly before returning to VB. If the original sub/function is calling other sub/functions then there would be no speed degradation passing them ByRef to those sub/functions.
Globals should only ever be used in DLL's if you need to preserve the data between calls from the calling program, which would not appear to be the case here.
In fact if you are doing DIM AT then there is no need to predefine the arrays just use
DIM array AS SINGLE AT address
and by default it will be LOCAL and the code should then be reentrant.

[Michael Mattias]

DIM small(xNew\32,yNew\32) 'initial size for global swap array Small()
DIM vector(8) 'permanent size

These lines do not create absolute arrays. Absolute arrays are created by using DIM/REDIM AT address

>'Within the program Master() and Small() are resized a total of 20 times

Compiler version not shown, but IIRC there was a problem up thru 7x with memory not being released if using REDIM PRESERVE on a PASSED array. There's a FAQ on this in the FAQ section.

[Michael Mattias]

Given my druthers (which I hightly doubt is forthcoming), on ERASE of ABSOLUTE array I'd skip the RESET. (I'd do the RESET on non-absolute arrays).

Theory: "programmer is responsible for all underlying data management when using absolute arrays."

But as I said, now that I know what it's doing, I can work with it.

MCM

[Ken German]

Just to be clear, I didn't say that using ERASE on absolute arrays did anything. I merely started trying it because the manual implied it couldn't hurt, and indeed it didn't hurt anything.

The VB crashing problem was solved by explicitly erasing global dynamic arrays as outlined in the following excerpts:

'The initial declaration
#COMPILE DLL "ACCELERATE"
#DIM ALL
GLOBAL vector() AS SINGLE
GLOBAL Master() AS SINGLE
GLOBAL Small() AS SINGLE
'####################

'The initial dimensioning within a sub

DIM Master(xNew,yNew) 'initial size for global swap array Master()
DIM small(xNew\32,yNew\32) 'initial size for global swap array Small()
DIM vector(8) 'permanent size for global array vector()

'Within the program Master() and Small() are resized a total of 20 times
'----------------------------------------
'The final line before ending the DLL and returning to VB that enabled the DLL to be called more than once

ERASE Small(),Master()

'It was not necessary to explicitly erase vector()
'_______________________________

The DLL used no strings so nothing can be inferred regarding their behavior. My final point was that there was enough diversity in the responses that the manual needs a detailed explicit discussion of absolute arrays.

Ken German

[Bob Zale]

RESET(): Set elements to zero or null and release strings.

ERASE(): Do a RESET(), plus: Disassociate the array name from the data, and release the containing data block. The sole exception to the rule: with Absolute arrays, the containing data block can not be released because no assumptions could possibly be made about the method used to allocate the memory.

This is logical, symmetrical, and intuitive. Be calm... Say "Ommmmmm......." a few times... You'll learn to be at peace with these functions.

[Michael Mattias]

Ideologically I'm not sure I agree with that approach, but now that I know that's what it's doing.....

1. I can work with it
2. It COULD explain the OP's 'solution' , assuming he is using some kind of dynamic string array (code not shown).
3. It MIGHT explain the OP's 'solution' , assuming he is using an array of some other data type (code not shown).

[Bob Zale]

Yes.

[Michael Mattias]

I see what you mean, I think: the strings themselves are released, but the block of memory which holds the strings is not freed?

That is, ERASE(dynamic string absolute array) basically does a RESET() PLUS a "mark descriptor un-dim'ed?


MCM

[Bob Zale]

Yes, when you execute an ERASE, individual elements are erased, if necessary. For example, when they are dynamic strings. The "containing" block of memory is left unchanged for absolute arrays because no assumptions can be made about its origin.

What if you don't want the individual elements released? My first suggestion would be: Don't execute an ERASE statement.

Bob Zale
PowerBASIC Inc.

[Michael Mattias]

ERASE on the absolute arrays before returning to VB- a step which is a bit counterintuitive to a VB programmer.

That's counterintutive to this non-VB programmer, too.

ERASE on an absolute array should do nothing more than modify the array descriptor to indicate "array not dimensioned and no information or data are available" ... which is something the VB program should not notice at all.

[later]

I just looked at the 9x help again under ERASE

Absolute arrays (those created by DIM...AT) are handled differently by ERASE. An explicit ERASE will release the individual elements of an absolute array, if needed, but the full data block is left as-is because no assumptions can be made as to its origin. It is the programmer's responsibility to ensure that the memory block overlaid by the absolute array is handled correctly.

There is a troubling portion of that statement here:

...explicit ERASE will release the individual elements of an absolute array, if needed,

If needed? When can this be needed? Especially when the statement goes on to state the programmer is responsible for the correct handling of the memory block?

I remember a couple of releases ago I ran into some kind of problem with RESET of absolute arrays (string?), but reset should do what it says and no more... it should set numerics to zero and dynamic strings to null... but I can't see why ERASE would do that, since RESET would seem to be the programmer deliberately - as required - managing the memory block; whereas ERASE is just saying "I don't need to view this block of memory as a PB array anymore."

Anyone got any ideas as to when the compiler "releasing" memory from the memory block associated with an absolute array is needed? Or do I need to ask support to document when "is needed" is true?

MCM

[Ken German]

Thanks for all of the replies! I tried switching from DIM...AT to REDIM..AT and the crashing behavior was unchanged. After some additional reading of the Help I tried using ERASE on the global arrays created in the PB DLL and the problem was solved even with the original DIM...AT statements! It also worked when I used ERASE on the absolute arrays before returning to VB- a step which is a bit counterintuitive to a VB programmer.

Currently, using the PB DLL the specific processing operation runs about twice as fast as in the compiled all-VB version! Now I can concentrate on optimizing the PB code.

As to what is clear in the manual, my feeling is that the manual is like trying to read a novel when given a randomized collection of chapters. The absolute array is an incredibly useful tool. It deserves its own detailed entry complete with examples of what works and what doesn't.

Ken German

[Michael Mattias]

Yes, it DOES include a GLOBAL.

'Twould hardly do to provide a personally-biased example minus the most common scope used, would it?

MCM

[John Petty]

I agree with you, never have been able to work out the purpose of a DIM unless to protect against bad coding.
PS Micheal that example includes a global!!

[Michael Mattias]

The PB Manual is very clear, if you DIM an array that is already DIMed then it will ignore the new DIM

I solved that problem by never using DIM at all. I use only REDIM.

GLOBAL A() AS LONG

FUNCTION Foo
 Static B() AS SINGLE
 LOCAL C() AS CUR 

      ....
         REDIM   A(number) and/or B(number) and/or C(number)

MCM