array descriptors

[Michael Mattias]

The code is public domain, sir.

Please....feel free to use whatever I've done as a starting point to design, develop, test, debug, document and post a 'correction.'

I'm sure it would be a wonderful first contribution to the Source Code Forum from you.

MCM

[John Petty]

Michael
Of course it checks out, to quote you RTFM

However, there was no such thing as Vista when those demos were created, nor is anyone paying me annual maintenance on same to keep up with new operating systems

True but your MMF demo is dated 2004 and the seperation of namespaces Global and Local (though a little easier to handle) was in Server 2003 and XP SP2 (though can't remember the exact release date).
John

[Michael Mattias]

Well, that's a good tip, assuming it tests out, of course.

Blogs are like "wikis" in that the words of our fortieth President, Ronald Wilson Reagan, should be applied: "Trust but Verify"

However, there was no such thing as Vista when those demos were created, nor is anyone paying me annual maintenance on same to keep up with new operating systems.


MCM

[John Petty]

Michael
I have no problem with posting ingormation here
Please read the following

http://blogs.msdn.com/maartenb/archive/2008/05/07/memory-mapped-files-from-xp-to-vista.aspx

of course this user was using them for what they were truelly intended.
The same issues obviously apply for using objects like mutexes to control the functions of different applications or multiple copies of the same application with different users.
John

[Michael Mattias]

Actually I am quite proud of my demos. However, I have not received a 'bug report' in about three years.

I'm afraid you will have to "show failing code" if you want to claim they don't work.

You can send message here using messaging facility, or directly to address below.

For that matter you may post here, in which case any response (and I do mean ANY response) will appear here.

[Chris Holbrook]

content withdrawn

[Cliff Nichols]

Now now there Palin vs Biden

(sorry, I could not help myself)
I can see both sides, but boy do I wanna put on an Adam Sandler attitude

[John Petty]

Michael
I will answer your comments slightly out of order to make some sense of them.

News Flash: There is no such thing as opening a file in "binary" mode using the Windows' API. Files are opened, period; there is no "access mode" (you may "RTFM" under CreateFile() if the details have temporarily escaped you). News Flash: There is no such thing as opening a file in "binary" mode using the Windows' API. Files are opened, period; there is no "access mode" (you may "RTFM" under CreateFile() if the details have temporarily escaped you).

News Flash: This is a Power Basic forum and PB does have an Open Binary mode statement but does not mention Create File perhaps you RT the correct FM.

And slower than opening a file in "binary mode?

I know American is getting further and further away from English but I though you still used the basics, I never said that I said “OPENED” which puts it in a totally different concept, which brings us to your first criticism.

Oh, "according to Microsoft", huh?
Really?

Yes and without wasting the time to find the reference but surely such a great exponent of Memory Mapped Files should know the reason. MMF’s only read a file in 4K blocks as needed and rely on page faults to switch previously 4K blocks in and out of memory as needed whereas with PB GET$ I can load an entire file in one go and the use Absolute Arrays and all the other features you claim for MMF’s without (if one has sufficient memory) another disk access to either the pageswap file or the original file. I have programs that run for days with 500MB files loaded and never need to do a disk access after initial loads.

I don't think you've taken a close look at the demo link I posted earlier in this thread:
Memory Mapped Files instead of RANDOM disk file access 5-8-04

Well yes I have and many other of your regularly quoted old examples (to paraphrase you real programmers don’t use RANDOM files).
I agree most describe important concepts particularly about objects UNFORTUNATELY IN MANY SITUATIONS THEY NO LONGER WORK AS YOU DESCRIBE THEM
For instance recently there was a post about 2 users on the same computer starting the same program, I quoted using the MS preferred method of using a locking file. You as usual disagreed and continued on with all your usual wait for object methods which in many cases no longer work.
To be specific you are the one who needs to RTF Microsoft M. The Namespaces concept started changing in Windows 2000 Server Service Pack 4 (SP4), Windows Server 2003, Windows XP SP2 and has changed even more in Vista.
John

[Michael Mattias]

The more I think about this the more I am convinced that getting data into and out of a database is exactly the sort of thing that the compiler should be facilitating for all it is worth.

Oh, come on!

If Mattiasaurus here is seriously considering dropping his long-time love affair with ODBC because using ADO with the compiler's COM interface is so much simpler, I think it's fair to say the compiler already does facilitate "getting data into and out of a database!"

(And that's only with 4x/8x!)

MCM

[Chris Holbrook]

No chance that I will write a function to handle all variable types, it would take me too long to test, but I will post the source code for string and UDT arrays when I have it working. It won't be this week, for reasons beyond my control.

The more I think about this the more I am convinced that getting data into and out of a database is exactly the sort of thing that the compiler should be facilitating for all it is worth.

[Michael Mattias]

You still keep missing the purpose of memory mapped files. If the data actually needs to be written to the disk and are only being accessed by one application at a time then they have no extra functionality over a file opened in binary mode and can be slower (according to MS)

Oh, "according to Microsoft", huh?

Really?

I don't think you've taken a close look at the demo link I posted earlier in this thread:
Memory Mapped Files instead of RANDOM disk file access 5-8-04

I think that's some functionality of which Microsoft did not conceive, given they are not in the business of finding ways to use the PowerBASIC compiler's "absolute array" feature with Windows' system objects.

And slower than opening a file in "binary mode?" News Flash: There is no such thing as opening a file in "binary" mode using the Windows' API. Files are opened, period; there is no "access mode" (you may "RTFM" under CreateFile() if the details have temporarily escaped you).

If you are going to make things up, you'll have to do better than this if you want to fool us.

MCM

[Michael Mattias]

Timing was not the question: Portability was.

So which do you want?

What is they say? Features, performance, cost: pick any two? (something like that, anyway).

In this particular application, the actual requirements and/or goals are not stated explicitly: "copy array descriptor" is a how, not a what. And since "use Array descriptor across processes by directly loading it from media" is not an option, we are all guessing at what the goal is here.

My guess was, "save variable number of dimensiosn, variable maximum subscripts program Array (capitalized) data for Array (ditto) use in another process." So that's what I came up with.

FWIW, the save and load can be done even more simply than the way I had first thought of: after saving or reading your file header (YOUR 'array descriptor'), "PUT hFile, Array()" and "GET hFile, Array()" look like they might come in handy.

MCM

[John Petty]

Micheal
You still keep missing the purpose of memory mapped files. If the data actually needs to be written to the disk and are only being accessed by one application at a time then they have no extra functionality over a file opened in binary mode and can be slower (according to MS)

[John Petty]

Chris
I think the compiler actually does much of what you are asking already (8.03)
from the manual.
"When PUT is used on a binary file, the entire array specified by Arr() is written to the file. With dynamic strings, the file is written in the PowerBASIC and/or VB packed string format. If the string is shorter than 65535 bytes, a 2-byte length Word is followed by the string data. Otherwise, a 2-byte value of 65535 is followed by a length Double-word (DWORD), then finally the string data."
and
"When reading an array from the disk file, GET assigns data from the file into each element in the array, starting at the arrays LBOUND subscript. GET attempts to read the number of elements specified by rcds in the RECORDS option, or the number of elements in the array, whichever is smaller. The actual number of elements read is assigned to the variable count specified in the optional TO clause.

With a dynamic string array, it is assumed the file was written in the PowerBASIC and/or VB packed string format using PUT of an entire string array. If a string is shorter than 65535 bytes, a 2-byte length WORD is followed by the string data. Otherwise, a 2-byte value of 65535 is followed by a length Double-word (DWORD), then finally the string data.
With other array arrays types, the entire data area is read as a single block. In either case, it is presumed the file was created with the complementary PUT Array statement."

[Richard Angell]

timing referring to:
1. using PB BASIC array statements, functions and reference norms,
versus
2. total roll your own using C.H..'s "portable" database concept, "blob" allocation, load and then, with home grown code manage it ... some avoiding rebuilding "the array descriptor". See Chris's very first post in this thread.

In both 1 and 2 each approach would be looked at separately, improved as needed, and then they would be compared.

@Chris:
"Is there anything in V9 which will help with this - I'm taking V9 features on a need-to-know basis, wish I had time to do otherwise!"

See GLOBALMEM in the help file if you have not already looked at it.

[Michael Mattias]

What's with all the "timing" stuff jumping in here?

First, you make it work; only then do you make it work better.

Let's see, my function's "some code goes here" consists entirely of
OPEN
GET$
CLOSE
REDIM
POKE$

Hmmm... going a little deeper (if you want to call five statements "deep"), trying to make that single function work for "any" array datatype is going to require more code than that.

So, to make it work before making it work better, I think I'd start by passing an array of the correct data type with the correct number of dimensions.


MCM

[Chris Holbrook]

Time one array load and access cycle using only PB's commands to load and set-up nd access the array -vs- datafile with an array info prefix of your construction being doing the same thing using your own scheme.

OK, see what you mean. I have no need to do this, as a) I have no choice as to how I code the array to blob load and save, b) in this instance, performance is if little concern.

Dynamic String arrays management keeps used space to a minimum as changes are made to the strings stored in it.

... which is why I leave memory allocation to the string engine and locate my UDT arrays inside strings.[/QUOTE]

[Richard Angell]

Try what?

Time one array load and access cycle using only PB's commands to load and set-up nd access the array -vs- datafile with an array info prefix of your construction being doing the same thing using your own scheme.

[quote Sorry, I have no idea what you mean.[/quote]
Dynamic String arrays management keeps used space to a minimum as changes are made to the strings stored in it. So if a change is a shorer sring it can reclaim the unused space, if longer, maybe move one string to unused space freeing a larger group of bytes for the changed string to be moved into. This is what a string engine does for arrays so as not to have to allocate more memory when it is not necessary, IOW real-time memory optimization

[Chris Holbrook]

Why not try both ways and time it with the CPU's performance counter, then post it, (I think some others tried this in the last few years.)

Try what?

Make sure your data can handle all the potential memory management issues the compiled library code already provides for, as well as dynamic strings that do change their length and number in the program. IOW, a real test.

Sorry, I have no idea what you mean.

[Richard Angell]

Michael,

I purposely left it out knowing you would put it on the table . Figured it was a better than 9:1 odds.

Chris,

Why not try both ways and time it with the CPU's performance counter, then post it, (I think some others tried this in the last few years.) Make sure your data can handle all the potential memory management issues the compiled library code already provides for, as well as dynamic strings that do change their length and number in the program. IOW, a real test.