globalmem lock

[Bernhard Fomm]

GLOBALMEM LOCK prevents the memory will paged into the pagefile.sys?
If not, how can I prevent that the memory is paged?

#COMPILE EXE
#DIM ALL


%MEMSIZE = 1024*1024*1000   ' 1GB


FUNCTION PBMAIN () AS LONG
LOCAL dAdr  AS DWORD
LOCAL hPtr  AS DWORD
LOCAL lRes  AS LONG
LOCAL i     AS BYTE PTR

    GLOBALMEM ALLOC %MEMSIZE TO hPtr
    IF hPtr THEN
        GLOBALMEM LOCK hPtr TO dAdr

        FOR i = dAdr TO dAdr + %MEMSIZE
            @i = 65
        NEXT i

        MSGBOX "Address: " + HEX$(dAdr,8)
        
        GLOBALMEM UNLOCK hPtr TO lRes
            
        GLOBALMEM FREE hPtr TO lRes
    END IF
    
END FUNCTION

[Michael Mattias]

> GLOBALMEM LOCK prevents the memory will paged into the pagefile.sys?

I suspect that might be the case with SMALL blocks of memory, but I would not bet the farm on it. LOCK *does* guarantee the virtual address of the data will remain unchanged until the block is unlocked, however.

>If not, how can I prevent that the memory is paged

Don't worry about it. Paging under Windows is exceptionally efficient.

Frankly, I doubt you can control it anyway.

MCM

[Tom Hanlin]

Virtual memory has low-level hardware support. I'd expect that any data is subject to being paged. There is presumably some way to get around this for device drivers, but I don't know if the technique is available to applications.

It's possible to disable the page file completely, if that's critical. That's a system-wide setting for the entire computer, so it's not the sort of thing you'd do lightly.

[John Petty]

GLOBALMEM LOCK prevents the memory will paged into the pagefile.sys?
If not, how can I prevent that the memory is paged?

While paging may be efficient it is of course horribly slow. Only answer I know that can help is to have minimum hardware requirement for your program of 2 - 4GB RAM.

[Bernhard Fomm]

Can I with VirtualLock (WinAPI) protect any area of memory?

[Michael Mattias]

>Can I with VirtualLock (WinAPI) protect any area of memory?

Depends what you mean by "protect."

VirtualLock

The VirtualLock function locks the specified region of the process's virtual address space into physical memory, ensuring that subsequent access to the region will not incur a page fault.

MCM

[Tom Hanlin]

That does sound like it'll do the trick.

[Michael Mattias]

I'm curious what kind of application needs (or is it "needs"?) 1 GB unpaged VM.

[John Petty]

Microsoft use it in MSSQL, it needs very carefully implementation. For 1GB I think he needs to use SetProcessWorkingSetSize first. It can seriously degrade performance of other applications and will return an error if there is insufficient physical memory for this request totalled with prior requests of other programs.
Like I posted before if you ensure the computer has ample physical memory then page faults are rare. I have an application that needs 750MB that way but with 2GB RAM I can browse the net at the same time and not get significant page faults.

[Michael Mattias]

>It can seriously degrade performance of other applications and

My whole perpective on applications is Susie User sitting at her desk running a typical suite of office applications such as an ERP system, probably email and a word processor, maybe a spreadsheet and some other stuff... and my program.

"Degrading the performance of other applications" is not a good way to score points with Susie.... or with Susie's boss.. you know, that guy/gal who signs off on your invoices?

That's why I'm interested in what these "Gimme RAM" applications do: I just don't know and have trouble even imagining.

MCM

[John Petty]

Virtual memory has low-level hardware support. I'd expect that any data is subject to being paged. There is presumably some way to get around this for device drivers, but I don't know if the technique is available to applications.

It's possible to disable the page file completely, if that's critical. That's a system-wide setting for the entire computer, so it's not the sort of thing you'd do lightly.

I believe device drivers use a different approach in that they use memory that avoids the page table all together but uses true physical RAM addresses. On 32bit systems this is limited to a total for all devices of 256MB.

[John Petty]

>It can seriously degrade performance of other applications and

My whole perpective on applications is Susie User sitting at her desk running a typical suite of office applications such as an ERP system, probably email and a word processor, maybe a spreadsheet and some other stuff... and my program.

"Degrading the performance of other applications" is not a good way to score points with Susie.... or with Susie's boss.. you know, that guy/gal who signs off on your invoices?

That's why I'm interested in what these "Gimme RAM" applications do: I just don't know and have trouble even imagining.

MCM

I agree but the OP may be writing a server application. I don't lock memory for my ap and it is quite surprising just how much else can run at the same time before pageing effects speed (yes I have email running at the same time with my own spam program checking the mail server every 2 minutes and directly deleteing spam) particularly as the O/S can use up to 1GB for disk cache if there is sufficient memory. This was actually an advantage (and weakness) of 98 over NT4 which had a poor disk cache default, it was fixed in 2000. The only problem in 98 was that once the O/S grabbed the memory for disk cache it didn't like to give it back

[Bernhard Fomm]

I need a very large memory for processing huge amounts of data.
It would be best if I can use the entire 32-bit space (0 to +2GB).

How does a RAM disk? A RAM disk will not paged in pagefile.sys. Or?

[Michael Mattias]

need a very large memory for processing huge amounts of data.
It would be best if I can use the entire 32-bit space (0 to +2GB).

"Need," sir, is unlikely.. at least in applications I can imagine.

Why can't you process it one record/row at a time? How about in 100Mb Chunks? Randomly as the application needs a particular subset of that data?

Surely you cannot expect me to believe you have to look at the same 2 Gb of data more than once in this this application.

But... I really do have an open mind. Please describe this application; either I will agree with you or I will suggest 'another way' - a less resource-hungry way.

MCM

[Bernhard Fomm]

I'm working on very specific simulations. My English skills are not enough for the description. I could simplify the tasks, if I could work with 2GB RAM.

-

How does a RAM disk? A RAM disk will not paged in pagefile.sys. Or?

[Michael Mattias]

> I could simplify the tasks, if I could work with 2GB RAM.

Couldn't we all.

[Michael Mattias]

> How does a RAM disk? A RAM disk will not paged in pagefile.sys. Or?

All virtual memory is subject to paging. Except, it would appear, that which is locked with VirtualLock()

[John Petty]

"Need," sir, is unlikely.. at least in applications I can imagine.

Why can't you process it one record/row at a time? How about in 100Mb Chunks? Randomly as the application needs a particular subset of that data?

Surely you cannot expect me to believe you have to look at the same 2 Gb of data more than once in this this application.

But... I really do have an open mind. Please describe this application; either I will agree with you or I will suggest 'another way' - a less resource-hungry way.

MCM

Actually, the application I quoted with 750MB reads the entire amount of that data between 50 and 100 times, any chunk method would be too slow.
To be able to have 2GB of data loaded all the time you need a computer with 4GB of RAM, start it with the 3/GB switch, compile in 9.01 and edit the PE header of the compiled program to set the Large Address Flag (there are post in forums decribing how to do that). If it is the only application running then it is unlikely that any portion of the data will be paged out, without need for for VirtualLock.

[Michael Mattias]

> quoted with 750MB reads the entire amount of that data between 50 and 100 times

Each and every one of the 750,000,000,000 bytes has to be read between 50 and 100 times?

I could name that tune in a LOT fewer notes....

MCM

[Gösta H. Lovgren-2]

>
Each and every one of the 750,000,000,000 bytes has to be read between 50 and 100 times?

mmmm.... Shouldn't that be 805,306,368,000.00 bytes? (courtesy of Easy Tape {grin}.

I could name that tune in a LOT fewer notes....
MCM

You only think you can because if ET is correct, you would be out of tune. {snark}

======================
Different values make
for a richer world.
Unknown
======================