Announcement

Collapse
No announcement yet.

Memory Allocation

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Memory Allocation

    Can anyone offer explanations on the following with this sample code. My machine as 3 GB memory and the /3GB switch has been set. However, I can only allocate around 0.8 GB. On a similar machine, I can allocate about 1.7 GB. However, I can never allocate above that without failure. When you run this, open taskmgr and observe the MEM allocation to see the drop off.
    Code:
    #COMPILE EXE
    #DIM ALL
    
    FUNCTION PBMAIN () AS LONG
    
        DIM N           AS LONG
    
        DO
            N = VAL(INPUTBOX$("Array size"))
            REDIM x(N)      AS DOUBLE
            MSGBOX "DOUBLE array, size " + STR$(N) + $CRLF + "MEM (MB) Predicted to use:  " + STR$(08 * N / 1024 / 1000), , "MEM Test"
            ERASE x()
    
        LOOP UNTIL N = 00
    
    END FUNCTION

  • #2
    Check swap file size.

    Also check ERR after the DIM/REDIM. If the REDIM fails the rest of your results are totally and absolutely meaningless.

    Someone had this same issue about 3 weeks ago. They made the same mistake, not querying ERR. Basically the upshot of that thread was, his posted program ran perfectly on my machine with the same compiler and operating system, so it must have been a 'settings' difference, which is where I suggested checking the swap file.

    IIRC the followup discussion, I seggested the design was terrible, with one PB array never changing, just sitting there eating up memory... that data should have been stored for program use in a random access disk file and read when needed (which was not all that often).


    MCM
    Michael Mattias
    Tal Systems (retired)
    Port Washington WI USA
    [email protected]stems.com
    http://www.talsystems.com

    Comment


    • #3
      Duuno if ERROR checking helps out here or not.

      '
      Code:
      #Compile Exe
      #Dim All
                         
      Function PBMain () As Long
          Local N, Element_size           As Long
          Local s$
          Do                                     
             ErrClear
              N = Val(InputBox$("Array size")) 
              If n = 0 Then Exit Loop
              If n < 0 Then
                 ? Using$(" # way too big", n),," Grando Mucho Numberaro"
                 Iterate Loop
              End If
              Element_size = 32
              ReDim x(N) As Quad 'eat up memory faster
              
              If Err Then
                 ? Error$(Err) ,, "Error" & Str$(Err)
                 'Exit Loop
              End If
              s$ = "String array size with #, Elements " & $CrLf & $CrLf & _
                   " Predicted to use: #, MB "
              MsgBox Using$(s$, N, Element_size * N / 1024 / 1024), , "MEM Test"
              Erase x()
          Loop Until N = 00
      End Function
      '
      ==============================================
      "Thank you for sending me a copy of your book
      I'll waste no time reading it."
      Moses Hadas (1900-1966)
      ==============================================
      It's a pretty day. I hope you enjoy it.

      Gösta

      JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
      LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

      Comment


      • #4
        >Element_size = 32
        > ReDim x(N) As Quad

        QUADs now require 32 bytes?

        Darned, yet another undocumented change across compiler versions!!
        Michael Mattias
        Tal Systems (retired)
        Port Washington WI USA
        [email protected]
        http://www.talsystems.com

        Comment


        • #5
          thanks all

          ok ERR can be used to infer that the allocation is in error.

          But I still do not understand if the boot switch is set to /3GB and I have 4GB of memory, why I cannot get access to this memory. Any ideas ?

          Comment


          • #6
            >Any ideas ?

            What about the swap file size? Did you check that?
            Michael Mattias
            Tal Systems (retired)
            Port Washington WI USA
            [email protected]
            http://www.talsystems.com

            Comment


            • #7
              Dean,
              which compiler are you using? I think only the latest (PBWin9/PBCC5) can access to more than 2GB.

              I can get a 1,716,000,000 byte array on my PC but it's very slow because the swapfile is being used a lot as I only(!) have 512MB RAM installed.

              Paul.

              Comment


              • #8
                Originally posted by Michael Mattias View Post
                >Element_size = 32
                > ReDim x(N) As Quad

                QUADs now require 32 bytes?
                They do in my head, which be may roomy than your {grin}.

                Actually I was thinking 32 bit processor.

                ===================================
                "Once is happenstance.
                Twice is coincidence.
                Three times is enemy action."
                Auric Goldfinger, in "Goldfinger"
                by Ian L. Fleming (1908-1964)
                ===================================
                It's a pretty day. I hope you enjoy it.

                Gösta

                JWAM: (Quit Smoking): http://www.SwedesDock.com/smoking
                LDN - A Miracle Drug: http://www.SwedesDock.com/LDN/

                Comment


                • #9
                  yes, swap file size had no affect. Is that the case, that is, PB 9.0 will compile over the 1.8 GB limit? Bob Zale, can you address this if possible or if someone can, test the compilation using 9.0?

                  For those interested, as an aside, this level of memory usage is very typical for computational analyzes. I find the 3GB limit addressed very frequently on other BBs. They all seem to point to a ~ 1.8 GB shut off point. What gives?

                  thanks all for helping ...

                  Comment


                  • #10
                    Dean,
                    They all seem to point to a ~ 1.8 GB shut off point. What gives?
                    many applications, including PB before CC5/Win9, had a 2GB limit on total memory that any single process could handle. Originally, the OS occupied 2GB of address space and the program occupied the remaining 2GB from the 4GB available.

                    I think lots of folk complained and Microsoft listened .. a little, and later versions of the OS could be resticted to 1GB (still more than enough!) leaving 3GB to applications that could use it.
                    But, many applications were written with the original 2GB limit in mind and can not take advantage of the extra GB.

                    The 1.8GB is about right. It allows 200MB for your program code, all the stacks and bits an pieces of other stuff and the remaining 1.8GB is useable for data.
                    if someone can, test the compilation using 9.0?
                    I've run it with PBWin but my OS doesn't start with the /3GB flag so I still only have 2GB to play with, hence the cut off is still just under 2GB.
                    I remember trying the 3GB flag once and it caused lots of problems so I'm reluctant to try it again.

                    Paul.

                    Comment


                    • #11
                      PS maybe this thread may help:
                      http://www.powerbasic.com/support/pb...ad.php?t=17095

                      Comment


                      • #12
                        Paul

                        thanks for digging that up!. Excellent resource. Application runs on XP Pro but now the following needs to be worked:

                        "No APIs are required to support application memory tuning. However, it would be ineffective to automatically provide every application with a 3 GB address space. Executables that can use the 3 GB address space are required to have the bit IMAGE_FILE_LARGE_ADDRESS_AWARE set in their image header. If you are the developer of the executable, you can specify a linker flag:

                        /LARGEADDRESSAWARE


                        I will do some searching but does anyone know if it is possible to set this LARGEADDRESSAWARE switch in a PB application?

                        thanks all!

                        Comment


                        • #13
                          Dean,
                          looking at the .exe files produced by PBWin9, it doesn't appear that the large address aware flag is set.
                          You could try setting it yourself to see if it works.

                          If you look at the .exe file in a hex editor, the PE header appears to start at location &h80.
                          The header starts with the letters PE so you can find it easily if it's not at that exact location.
                          22 bytes beyond the P in PE is a word of value &h030E. The bits of this word are defined here:
                          http://msdn.microsoft.com/en-us/library/ms680313.aspx

                          If you were to set the 6th LSB bit (change &h030E to &h032E) then the application should have the large address aware flag set.

                          I haven't tried it but it maybe worth giving it a go.

                          Paul.

                          Comment


                          • #14
                            The fuddy-duddy speaks: instead of editing compiled executables, maybe the program should be re-engineered to NOT require that much virtual memory.

                            Or, it should be rewritten using a compiler which supports this kind of RAM requirement.
                            Michael Mattias
                            Tal Systems (retired)
                            Port Washington WI USA
                            [email protected]
                            http://www.talsystems.com

                            Comment


                            • #15
                              Paul

                              Excellent! Will give this a try. Any hex editor you can recommend? I think this is a common posting but perhaps you can suggest an easy to use favorite.

                              As for the apparency of mem bloat, this really isn't the case especially with computational simulations. This is just the new reality of computational physics. 64 bit will buy time but only time. CFD simulations are approaching 1 billion elements.

                              Comment


                              • #16
                                MCM I seem to recall that the initial announcements of 9 and and 5 claimed they handled 4GB of memory (they no longer do) and have posted in several discussions regarding this feature including a few days after their release but nobody from PB has ever confirmed or denied the capability or presence of the large address aware flag.
                                Dean merely editing in the flag without PB confirmation is extremely risky as the compiler would need to be doing all its address calculations using DWORDS or QUADS not LONGS.
                                If it is capapable, then running on a 64 bit CPU and 64 bit OS (32 bit program) with 6 GB of memory will actually allow you to to use the full 4GB of memory for your program.
                                All my comps are 64 bit capable and I own a copy of XP 64 bit Pro that I have never used ( nor have I yet upgraded to 9) while i wait for clarification.

                                Comment


                                • #17
                                  Dean,
                                  any hex editor would do. I use UltraEdit.

                                  Paul.

                                  Comment


                                  • #18
                                    John,
                                    the latest help file for PBWin9 lists as a feature:
                                    Use of all available memory (up to 4 Gb) for arrays and dynamic strings
                                    Paul.

                                    Comment


                                    • #19
                                      MCM,
                                      some tasks benefit hugely from that much memory just as tasks of 15 years ago benefitted by moving to 16MB of Windows 3.0 from the 640k of DOS.
                                      Also, it's not virtual memory Dean's after, it's real memory. He has 3GB installed and wishes to make use of it.

                                      Paul.

                                      Comment


                                      • #20
                                        ... actually I have 4GB installed but would be satisified with accessing 3 GB

                                        Paul, can you look at this map and tell me what to change? I am totally clueless here in the hex edit mode?

                                        thanks very much, regards, Dean
                                        Attached Files

                                        Comment

                                        Working...
                                        X