Announcement

Collapse
No announcement yet.

Mystery of Shrinking Resources

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Mystery of Shrinking Resources

    Hi,

    I should preface this by saying that I'm a civilian, not a
    software engineer, but have been successful so far porting a huge
    Wall Street financial simulation I wrote years ago in PB-DOS to
    PB/DLL over the last 8 months, but I'm noticing that if I run
    the program for several hours as users will do), my "Windows
    Resources" steadily dwindles from about 84% shortly after I start
    the program, to about 20% (and still falling) after running it
    about 3 hours.

    Eventually, if I run it long enough, I suspect Windows will
    issue "Dangerously Low on Resources" messages.

    My .EXE is about 700k, and does use some fairly large arrays,
    but those are all set at the start of a session, and randomly
    filled up with data, so the data in the arrays merely changes
    during the game -- no new or larger arrays are created.

    I was considering having the program periodically save its data
    to disk, and then shell to a small secondary .EXE to display
    some important announcement in the course of the game, and
    close the main program, while the small program, after a few
    seconds, reloads the main, but I see now that even AFTER exiting
    the program entirely, Windows is still showing only about 26%
    resources free, with no programs loaded. This is on a system
    with 128 megs of RAM, and it does about the same on an old Win
    95 laptop I've test it on, with only 16 megs, so the amount of
    RAM is apparently not a significant factor.

    Any thoughts on why the resources steadily diminish and what,
    if anything can be done to prevent this from happening? It's
    a great little game, and I have a lot of users of the DOS
    version eagerly waiting for the Windows version, but it looks
    like it's not going to be commercially viable if I can't stop it
    from eating up Windows "resource."

    ------------------

  • #2
    Mike, do you know which type of resources are dwindling? Is it GDI?

    My next question is: does the app do any graphics/drawing (screen or printer)? If so, I suspect that your problem is a memory leak cause by a buildup of GDI objects such as Fonts, Brushes, Pens, Bitmaps, etc.

    However, since we cannot see the code, this is just a quess...!

    If you can try running it under Window 2000, you can use TaskManager to watch the amount of GDI Objects and how much memory is being used in real time - very helpful.

    The next method would be to use a tool like "MemCheck" to look for leaks. These kind of tools are very exxpensive, but you may be able to obtain a 14-day trial edition to play with. Search the BBS for "memcheck" and you should find some discussions and links that may help you.

    I hope this helps!

    PS: if you need any help with finding the correct strategy to create and delete GDI objects, just ask!

    ------------------
    Lance
    PowerBASIC Support
    mailto:[email protected][email protected]</A>
    Lance
    mailto:[email protected]

    Comment


    • #3
      Mike;

      The Resource leak isn't likely anything having to do with generic
      PB code, like arrays. It is most likely caused by API functions calls
      that create and use Resources.

      The most common mistakes that will produce the affects you are speaking
      are failing to Destroy GDI objects, such as Brushes and pens (and
      others like fonts, Bitmaps, etc.).

      Brushes are often a culprit, since you don't think they use much GDI
      Resource memory, but actually they do. Failing to destroy a Brush
      over and over again (also creating over and over again), can have
      devastating effects.

      When you use Resources, such as brushes and pens, you should :

      (1) Create the Brush or Pen
      (2) Select it into the DC (save the old Brush or Pen handle)
      (3) Draw with it
      (4) Select the old Brush or Pen back into the DC
      (5) Destroy the created Brush or Pen.

      If you don't follow these steps and your app draws many , many times
      , your app will eat up GDI resources very fast.

      Resources that may need to stay in memory for longer periods of time
      should be created in the WM_CREATE message for the Dialog (or WM_INITDIALOG)
      and then be destroyed in the Dialogs WM_DESTROY message.

      If your app lowers the GDI resources by more than 10% then it is
      a Resource hog. 10% of GDI resources is quite a bit. Now in some instances
      this may be necessary, but apps should be written so they use resources
      efficiently. If an app progressively uses more and more of the GDI resources
      over time, then it has a serious Resource leak.




      ------------------
      Chris Boss
      Computer Workshop
      Developer of "EZGUI"
      http://cwsof.com
      http://twitter.com/EZGUIProGuy

      Comment


      • #4
        Thanks Lance and Chris.

        I'm not sure where to find out what kind of Windows resources are
        dwindling--GDI or other. I simply happened to go to the System
        Performance tab in Control Panel (Win 98), and Win 98 tells me
        "System resources: 47% free" with no further details.

        My program is mainly "text-intensive," although I used
        EZGUI to create the half-dozen dialog screens used in the
        program, since I didn't have a clue about how to use
        brushes to create colored backgrounds. Since I didn't
        dare tinker with the EZGUI code, I doubt if that is the
        problem area, unless the code EZGUI generates is defective
        in that it fails to destroy the created brush or pen it uses
        to color the background of dialog screens. (Which I doubt,
        since EZGUI seems to be very well done.)

        What I did note this afternoon, was that when "system resources"
        got down to 47% and when I exited the program, resources remained
        at 47% after. I then did a Win 98 Ctrl-Alt-Delete and noted that
        it showed that a "ghost" version of my program was still in
        memory, despite my having exited the program.

        I clicked on it to delete it and got the usual delayed response
        from Windows that the "Program is not responding... Do you want
        to terminate it?" and killed it, whereupon my System Resources
        went back to 83%, which is normal on my system. Hmmm.

        Any chance that some of the dialogs I terminate with "DIALOG END"
        commands remain in memory, if I send update messages to the
        screens after they have been destroyed? I have a stock ticker
        that runs simultaneously on the main screen and on two child
        screens when they are visible, and I don't check to see if
        either of the two child screens are still "alive" when sending
        data to the 3 stock tickers, since doing so did not appear
        to affect the functioning of the program.

        But perhaps that is the leakage that is occurring? (The
        data sent is simply sent via CONTROL SET TEXT to a label
        control on each of the 3 dialogs, sent several times a
        second, to create the illusion of a moving stock ticker.) Is
        it necessary to recreate a dialog (and perhaps hide it) before
        sending it update data, as with CONTROL SET TEXT?

        Thanks for any further thoughts on this little mystery....



        ------------------

        Comment


        • #5
          Mike;

          Use the "Resource Meter" accessory that comes with Windows to
          test the Resource levels during execution of your app.

          The Resources are broken down into three catagories, System, User
          and GDI.

          Run your application (with the Resource Meter displayed) and see
          "when" during the programs execution the Resources start dropping
          (ie. which dialog, which action, etc.). This will give you a clue
          of where in your program the leak is occuring.

          If your program and the resource meter can't fit on the screen at
          the same time, change your Resolution to the next higher one, so
          you can see both apps.

          The code generated by my Freeware DDT tools shouldn't produce the
          kind of resource leak you are reporting.

          It is also possible the leak is not GDI related, even though GDI
          leaks are the most common.



          ------------------
          Chris Boss
          Computer Workshop
          Developer of "EZGUI"
          http://cwsof.com
          http://twitter.com/EZGUIProGuy

          Comment


          • #6
            I'm not sure where to find out what kind of Windows resources are dwindling--GDI or other. I simply happened to go to the System Performance tab in Control Panel (Win 98), and Win 98 tells me "System resources: 47% free" with no further details.
            That figure comprises memory & GDI object usage... it is too unspecific to provide an exact clue.

            Since I didn't dare tinker with the EZGUI code, I doubt if that is the problem area, unless the code EZGUI generates is defective in that it fails to destroy the created brush or pen it uses to color the background of dialog screens. (Which I doubt, since EZGUI seems to be very well done.)
            While that is an honorable position, if you want to track the cause of the problem, you'll have to consider the possibility that the EZGUI portion of your code could actually be the cause of (or contribute to) the problem... you won't know until you locate the exact cause of resource-hogging.

            What I did note this afternoon, was that when "system resources" got down to 47% and when I exited the program, resources remained at 47% after. I then did a Win 98 Ctrl-Alt-Delete and noted that it showed that a "ghost" version of my program was still in memory, despite my having exited the program.
            This suggests that while your code is appearing to terminate, some portion is still running. It could be a message pump in your primary thread, or some other thread could still be running... without being able to review your code, it is only possible to guess. However, I think this clue is definitely worth pursuing!

            I clicked on it to delete it and got the usual delayed response from Windows that the "Program is not responding... Do you want to terminate it?" and killed it, whereupon my System Resources went back to 83%, which is normal on my system. Hmmm.
            This tends to confirm my hypothesis above.

            Any chance that some of the dialogs I terminate with "DIALOG END" commands remain in memory, if I send update messages to the screens after they have been destroyed? I have a stock ticker that runs simultaneously on the main screen and on two child screens when they are visible, and I don't check to see if either of the two child screens are still "alive" when sending data to the 3 stock tickers, since doing so did not appear to affect the functioning of the program.
            It is possible, but a DIALOG END by itself would not be the cause... it could be that you also need to use PostQuitMessage() cakk in the %WM_DESTROY handler in your main dialog callback... however, it will completely depend on how the GUI code is constructed. Are you using the SDK or DDT version of EZGUI?

            But perhaps that is the leakage that is occurring? (The data sent is simply sent via CONTROL SET TEXT to a label control on each of the 3 dialogs, sent several times a second, to create the illusion of a moving stock ticker.) Is it necessary to recreate a dialog (and perhaps hide it) before sending it update data, as with CONTROL SET TEXT?
            CONTROL SET TEXT itself wont be causing the leak. You should never need to destroy a dialog or window just to change the text in a control. If a STATIC (LABEL) control is using a transparent background, then changing the text can cause an unusual display in the control unless the control is invalidated correctly. I would have assumed that EZGUI would have generated code to take care of this "visual" aspect. That said, that is a visual problem, and not a memory leak problem.

            I'd suggest that you first ask the EZGUI folks to take a look over your code - they may spot something in the message pump or callback processing. If that fails, then start stripping parts out of the code (say, using #IF 0 & #ENDIF wrappers) and rerunning the code until the problem disappears.

            Please let us know if any of this helps, especially when you find the real solution.

            ------------------
            Lance
            PowerBASIC Support
            mailto:[email protected][email protected]</A>
            Lance
            mailto:[email protected]

            Comment


            • #7
              Mike;

              Also it is important to note that sometimes Resource leaks can be
              caused by the Windows control themselves and by executing an action
              on a windows control that produces such a leak, over and over again
              can produce a serious resource leak in an application.

              One example is the Toolbar control. When you use the default bitmaps
              for the toolbar that are built into the Common control DLL, there is
              a resource leak produced by it.

              Also when the data in a control is changed constantly, the control will
              be reallocating Global memory (using the API memory functions)
              every time you change the data (such as window text). Some controls
              may handle this repeated change over and over again better than others.
              Some controls may not take well to repeated changes and it is possible
              for the control to produce a memory leak in such instances.



              ------------------
              Chris Boss
              Computer Workshop
              Developer of "EZGUI"
              http://cwsof.com
              http://twitter.com/EZGUIProGuy

              Comment


              • #8
                True Chris, but Mike says the app does not appear to terminate correctly, so this tends to discount a resource leak problem to some degree.

                However, to dwindling resource level is something that needs investigating.

                It seems more likely that he could be encountering a memory fragmentation problem.

                Again, unless we can see the problem and review the code, we are just guessing.

                It is easier to debug a problem that can be actually seen <wide cheeky grin>

                ------------------
                Lance
                PowerBASIC Support
                mailto:[email protected][email protected]</A>
                Lance
                mailto:[email protected]

                Comment


                • #9
                  Originally posted by Chris Boss:


                  One example is the Toolbar control. When you use the default bitmaps
                  for the toolbar that are built into the Common control DLL, there is
                  a resource leak produced by it.

                  Also when the data in a control is changed constantly, the control will
                  be reallocating Global memory (using the API memory functions)
                  every time you change the data (such as window text). Some controls
                  may handle this repeated change over and over again better than others.
                  Some controls may not take well to repeated changes and it is possible
                  for the control to produce a memory leak in such instances.


                  Interesting. Do you have the knowledge base Id's for these.

                  James




                  ------------------

                  Comment


                  • #10
                    Mike;

                    If you email me a copy of the actual program (EXE, not source), I
                    will test it out for you.

                    mailto:[email protected][email protected]</A>

                    I doubt the code generated by the EZGUI Freeware Designer is the cause
                    since it simply generates DDT Dialog and Control creation commands
                    and thats it. Also the Designers code creates brushes upon startup
                    and releases brushs upon termination of the app. They aren't created
                    during execution.

                    Also your problem may be more than one problem, so the app failing
                    to terminate may not even be associated with the resource leak.

                    When you look at the code in PBMAIN generated by my Visual Designer
                    you can see that it creates resource during startup and then releases
                    them after the messageloop terminates :

                    Code:
                    FUNCTION PBMAIN
                        LOCAL Count&
                        LOCAL CC1 AS INIT_COMMON_CONTROLSEX
                    
                        CC1.dwSize=SIZEOF(CC1)
                        CC1.dwICC=%ICC_WIN95_CLASSES
                        InitCommonControlsEX CC1
                        EZLIB_DefColors
                        EZLIB_InitFonts
                    
                        ShowDialog_Form1 0
                        DO
                            DIALOG DOEVENTS TO Count&
                        LOOP UNTIL Count&=0
                    
                    
                        EZLIB_DeleteBrushes
                        EZLIB_DeleteFonts
                    END FUNCTION
                    The Do Loop in PBMain, I believe is the proper way to test for
                    the applications termination.

                    The code above assumes your first DIalog is Modeless, which it should
                    be. The first Dialog shouldn't be Modal.




                    ------------------
                    Chris Boss
                    Computer Workshop
                    Developer of "EZGUI"
                    http://cwsof.com
                    http://twitter.com/EZGUIProGuy

                    Comment


                    • #11
                      Gentlemen,

                      I encountered a similar situation early on in my learning curve,
                      perhaps Mike is making the same mistake I made.

                      Mike, check your code for the following situation:

                      1) You have a SUB or FUNCTION (procedure)
                      2) That procedure creates a dialog, assigns a dialog callback
                      3) dialog callback terminates dialog (DIALOG END)
                      4) procedure does NOT realize dialog has ended and runs
                      in infinite loop.

                      Just because a dialog ends doesn't mean that your SUB or
                      FUNCTION which created the dialog will automatically terminate.



                      ------------------
                      Bernard Ertl
                      Bernard Ertl
                      InterPlan Systems

                      Comment


                      • #12
                        James;

                        I did extensive testing of the Common Control Toolbars and I found
                        the Resource leak when using the builtin bitmaps. I haven't seen
                        anything in print about it on the MS web site, but that doesn't mean
                        it doesn't exist.

                        The reason most programmers would fail to notice it, is because
                        it only loses a small amount of GDI resource memory and is easily
                        missed. The reason I came across it, is because I build Visual
                        Designers and it is possible to create and destroy dozens of toolbars
                        over and over again. When the Toolbar is created and destroyed multiple
                        times, you will then notice the leak begins to build until you run
                        out of Resource memory.

                        I have sufficiently proved this to myself and I have no doubts
                        about it. When I use a external Bitmap for the Toolbars and
                        create and destroy them over and over again, the resource leak
                        does not exist.

                        Let me just say, that I do exhaustive tests when building software.

                        Rule of Thumb : The Operating System is not bullet proof !


                        ------------------


                        [This message has been edited by Chris Boss (edited June 04, 2001).]
                        Chris Boss
                        Computer Workshop
                        Developer of "EZGUI"
                        http://cwsof.com
                        http://twitter.com/EZGUIProGuy

                        Comment


                        • #13
                          Tests I have done on the toolbar control support what Chris said. Rapidly
                          creating and destroying toolbars that use the system bitmaps in commctrl.dll
                          produces a nasty leak in GDI resources. To make matters worse the resources are
                          not restored even after the application terminates. Toolbars that do not use the
                          system bitmaps do not exhibit this problem.
                          I cannot think of a single common control supplied by Microsoft that does not
                          have a plethora of bugs.

                          ------------------
                          Dominic Mitchell
                          Dominic Mitchell
                          Phoenix Visual Designer
                          http://www.phnxthunder.com

                          Comment


                          • #14
                            Mike,

                            A long time ago when I was learning GDI coding I found a technique
                            for tracking down dead BMP resources that were not released properly,
                            I used a toy from the SDK called "heapwalker" and opened up the list
                            and went looking for the dea remains of bitmaps.

                            As Lance mentioned there are some nice toys for finding leaks but
                            you have to buy them and learn to fly them as well. It may be worth
                            testing and code that is being looped and if possible, try commenting
                            out bits of it to see if you can isolate the area of code where the
                            leak comes from.

                            GDI can be one source of memory leaks but there can be others as well,
                            allocated memory that is not released properly can really add up
                            as a program runs for any length of time. Just make sure you are
                            creating and destroying the controls you are using in the correct manner
                            as this may be one of the problems.

                            I am sure Chris will help you as you are using his EZGUI as it is likely
                            to be a very simple thing that just happens to be in loop code somewhere
                            within the application.

                            Good Luck with it.

                            [email protected]

                            ------------------
                            hutch at movsd dot com
                            The MASM Forum

                            www.masm32.com

                            Comment


                            • #15
                              Chris / Dominic,
                              What ver(s) of commctrl.dll were tested??

                              James


                              ------------------

                              Comment


                              • #16
                                I had a similar problem one time when I was making a DIALOG END
                                call within a message handler under a custom (MY_MESSAGE = WM_USER + x)
                                message ID. I found that by ending the program there, that it stayed
                                in memory so what I had to do is post another message that would re-direct
                                the call to WM_COMMAND and process the DIALOG END from there. I don't
                                know why it did that but by re-directing the message the problem was cured.


                                Scott





                                ------------------
                                Scott Slater
                                Summit Computer Networks, Inc.
                                www.summitcn.com

                                Comment


                                • #17
                                  Scott, that sounds like you were SENDing a message to the dialog rather than POSTing it... depending on the context of the message that triggered the SendMessage, you can get these kind of problems.



                                  ------------------
                                  Lance
                                  PowerBASIC Support
                                  mailto:[email protected][email protected]</A>
                                  Lance
                                  mailto:[email protected]

                                  Comment


                                  • #18
                                    I ran Mikes program on Windows 95 (OSR1) and I sent him the results.

                                    My impression is that there is no Resource leak in his code. I think he
                                    is pushing the Label (or textbox) control which displays the Ticker Tape
                                    beyond its limits and that Windows may have a problem in keeping up
                                    with the rapid changes over a long period of time.

                                    No leaks show up in 5 or 10 minutes, so I doubt there is a serious
                                    leak produced by his code. It is possible the leak is produced over
                                    time as the control that handles the Ticker Tape may eventually begin
                                    to produce leaks with allocating Global Windows memory for the text
                                    string.


                                    My suggestion is to replace the Window control he uses for the Ticker Tape
                                    with a custom window class and to do the drawing himself.

                                    Now, if the leak does appear quickly on other operating systems , then
                                    it is possible it is an issue with the new OS (ME, 2000).

                                    Simply put, the culprit is most likely the control used for the
                                    Ticker Tape display.



                                    ------------------
                                    Chris Boss
                                    Computer Workshop
                                    Developer of "EZGUI"
                                    http://cwsof.com
                                    http://twitter.com/EZGUIProGuy

                                    Comment


                                    • #19
                                      Thanks for all the feedback, everyone. I'm going to try several
                                      of the suggested approaches, and if one of them solves the
                                      problem, I'll post the results here on the Forum, if it was
                                      something wrong with my code, rather than a problem inherent
                                      in Windows.

                                      ------------------

                                      Comment


                                      • #20
                                        Mike,

                                        I had one of those problems where sometimes when I closed the program
                                        it would disappear from the desktop, but actually still be in memory.
                                        I am using a windows timer to do do some background routines, and
                                        forgot that you should shut that down when you initiate your end
                                        program procedures. So now I turn off that timer first thing and
                                        it closes properly every time.

                                        Russ Srole

                                        ------------------
                                        "There are two novels that can change a bookish fourteen-year old's life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs." - John Rogers

                                        Comment

                                        Working...
                                        X