array descriptors

[Michael Mattias]

What's with all the "timing" stuff jumping in here?

First, you make it work; only then do you make it work better.

Let's see, my function's "some code goes here" consists entirely of
OPEN
GET$
CLOSE
REDIM
POKE$

Hmmm... going a little deeper (if you want to call five statements "deep"), trying to make that single function work for "any" array datatype is going to require more code than that.

So, to make it work before making it work better, I think I'd start by passing an array of the correct data type with the correct number of dimensions.


MCM

[Richard Angell]

timing referring to:
1. using PB BASIC array statements, functions and reference norms,
versus
2. total roll your own using C.H..'s "portable" database concept, "blob" allocation, load and then, with home grown code manage it ... some avoiding rebuilding "the array descriptor". See Chris's very first post in this thread.

In both 1 and 2 each approach would be looked at separately, improved as needed, and then they would be compared.

@Chris:
"Is there anything in V9 which will help with this - I'm taking V9 features on a need-to-know basis, wish I had time to do otherwise!"

See GLOBALMEM in the help file if you have not already looked at it.

[John Petty]

Chris
I think the compiler actually does much of what you are asking already (8.03)
from the manual.
"When PUT is used on a binary file, the entire array specified by Arr() is written to the file. With dynamic strings, the file is written in the PowerBASIC and/or VB packed string format. If the string is shorter than 65535 bytes, a 2-byte length Word is followed by the string data. Otherwise, a 2-byte value of 65535 is followed by a length Double-word (DWORD), then finally the string data."
and
"When reading an array from the disk file, GET assigns data from the file into each element in the array, starting at the arrays LBOUND subscript. GET attempts to read the number of elements specified by rcds in the RECORDS option, or the number of elements in the array, whichever is smaller. The actual number of elements read is assigned to the variable count specified in the optional TO clause.

With a dynamic string array, it is assumed the file was written in the PowerBASIC and/or VB packed string format using PUT of an entire string array. If a string is shorter than 65535 bytes, a 2-byte length WORD is followed by the string data. Otherwise, a 2-byte value of 65535 is followed by a length Double-word (DWORD), then finally the string data.
With other array arrays types, the entire data area is read as a single block. In either case, it is presumed the file was created with the complementary PUT Array statement."

[John Petty]

Micheal
You still keep missing the purpose of memory mapped files. If the data actually needs to be written to the disk and are only being accessed by one application at a time then they have no extra functionality over a file opened in binary mode and can be slower (according to MS)

[Michael Mattias]

Timing was not the question: Portability was.

So which do you want?

What is they say? Features, performance, cost: pick any two? (something like that, anyway).

In this particular application, the actual requirements and/or goals are not stated explicitly: "copy array descriptor" is a how, not a what. And since "use Array descriptor across processes by directly loading it from media" is not an option, we are all guessing at what the goal is here.

My guess was, "save variable number of dimensiosn, variable maximum subscripts program Array (capitalized) data for Array (ditto) use in another process." So that's what I came up with.

FWIW, the save and load can be done even more simply than the way I had first thought of: after saving or reading your file header (YOUR 'array descriptor'), "PUT hFile, Array()" and "GET hFile, Array()" look like they might come in handy.

MCM

[Michael Mattias]

You still keep missing the purpose of memory mapped files. If the data actually needs to be written to the disk and are only being accessed by one application at a time then they have no extra functionality over a file opened in binary mode and can be slower (according to MS)

Oh, "according to Microsoft", huh?

Really?

I don't think you've taken a close look at the demo link I posted earlier in this thread:
Memory Mapped Files instead of RANDOM disk file access 5-8-04

I think that's some functionality of which Microsoft did not conceive, given they are not in the business of finding ways to use the PowerBASIC compiler's "absolute array" feature with Windows' system objects.

And slower than opening a file in "binary mode?" News Flash: There is no such thing as opening a file in "binary" mode using the Windows' API. Files are opened, period; there is no "access mode" (you may "RTFM" under CreateFile() if the details have temporarily escaped you).

If you are going to make things up, you'll have to do better than this if you want to fool us.

MCM

[Chris Holbrook]

No chance that I will write a function to handle all variable types, it would take me too long to test, but I will post the source code for string and UDT arrays when I have it working. It won't be this week, for reasons beyond my control.

The more I think about this the more I am convinced that getting data into and out of a database is exactly the sort of thing that the compiler should be facilitating for all it is worth.

[Michael Mattias]

The more I think about this the more I am convinced that getting data into and out of a database is exactly the sort of thing that the compiler should be facilitating for all it is worth.

Oh, come on!

If Mattiasaurus here is seriously considering dropping his long-time love affair with ODBC because using ADO with the compiler's COM interface is so much simpler, I think it's fair to say the compiler already does facilitate "getting data into and out of a database!"

(And that's only with 4x/8x!)

MCM

[John Petty]

Michael
I will answer your comments slightly out of order to make some sense of them.

News Flash: There is no such thing as opening a file in "binary" mode using the Windows' API. Files are opened, period; there is no "access mode" (you may "RTFM" under CreateFile() if the details have temporarily escaped you). News Flash: There is no such thing as opening a file in "binary" mode using the Windows' API. Files are opened, period; there is no "access mode" (you may "RTFM" under CreateFile() if the details have temporarily escaped you).

News Flash: This is a Power Basic forum and PB does have an Open Binary mode statement but does not mention Create File perhaps you RT the correct FM.

And slower than opening a file in "binary mode?

I know American is getting further and further away from English but I though you still used the basics, I never said that I said “OPENED” which puts it in a totally different concept, which brings us to your first criticism.

Oh, "according to Microsoft", huh?
Really?

Yes and without wasting the time to find the reference but surely such a great exponent of Memory Mapped Files should know the reason. MMF’s only read a file in 4K blocks as needed and rely on page faults to switch previously 4K blocks in and out of memory as needed whereas with PB GET$ I can load an entire file in one go and the use Absolute Arrays and all the other features you claim for MMF’s without (if one has sufficient memory) another disk access to either the pageswap file or the original file. I have programs that run for days with 500MB files loaded and never need to do a disk access after initial loads.

I don't think you've taken a close look at the demo link I posted earlier in this thread:
Memory Mapped Files instead of RANDOM disk file access 5-8-04

Well yes I have and many other of your regularly quoted old examples (to paraphrase you real programmers don’t use RANDOM files).
I agree most describe important concepts particularly about objects UNFORTUNATELY IN MANY SITUATIONS THEY NO LONGER WORK AS YOU DESCRIBE THEM
For instance recently there was a post about 2 users on the same computer starting the same program, I quoted using the MS preferred method of using a locking file. You as usual disagreed and continued on with all your usual wait for object methods which in many cases no longer work.
To be specific you are the one who needs to RTF Microsoft M. The Namespaces concept started changing in Windows 2000 Server Service Pack 4 (SP4), Windows Server 2003, Windows XP SP2 and has changed even more in Vista.
John

[Cliff Nichols]

Now now there Palin vs Biden

(sorry, I could not help myself)
I can see both sides, but boy do I wanna put on an Adam Sandler attitude

[Chris Holbrook]

content withdrawn

[Michael Mattias]

Actually I am quite proud of my demos. However, I have not received a 'bug report' in about three years.

I'm afraid you will have to "show failing code" if you want to claim they don't work.

You can send message here using messaging facility, or directly to address below.

For that matter you may post here, in which case any response (and I do mean ANY response) will appear here.

[John Petty]

Michael
I have no problem with posting ingormation here
Please read the following

http://blogs.msdn.com/maartenb/archive/2008/05/07/memory-mapped-files-from-xp-to-vista.aspx

of course this user was using them for what they were truelly intended.
The same issues obviously apply for using objects like mutexes to control the functions of different applications or multiple copies of the same application with different users.
John

[Michael Mattias]

Well, that's a good tip, assuming it tests out, of course.

Blogs are like "wikis" in that the words of our fortieth President, Ronald Wilson Reagan, should be applied: "Trust but Verify"

However, there was no such thing as Vista when those demos were created, nor is anyone paying me annual maintenance on same to keep up with new operating systems.


MCM

[John Petty]

Michael
Of course it checks out, to quote you RTFM

However, there was no such thing as Vista when those demos were created, nor is anyone paying me annual maintenance on same to keep up with new operating systems

True but your MMF demo is dated 2004 and the seperation of namespaces Global and Local (though a little easier to handle) was in Server 2003 and XP SP2 (though can't remember the exact release date).
John

[Michael Mattias]

The code is public domain, sir.

Please....feel free to use whatever I've done as a starting point to design, develop, test, debug, document and post a 'correction.'

I'm sure it would be a wonderful first contribution to the Source Code Forum from you.

MCM