Announcement

Collapse
No announcement yet.

Optimal DLL Location

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Optimal DLL Location

    As I understand it the original concept of a DLL was that no matter how many concurrent programs were using it, there only needed to be one copy of it loaded in memory. This gave advantages in both program load speed, efficient use of physical memory and thus faster operation due to less page faults.
    There now seem to be appearing numerous Micrsoft articles that the DLL concept has been a failure and that they would rather revert to the Unix concept that a program is a complete EXE with no DLL's except system calls.
    The early indications of this were that Visual Studio 6 now by default places user DLL's in the same directory as the program. This implies, as windows looks for the DLL in the program directory first, that a new copy of the DLL is loaded into memory for each program that is using it.
    It seems that there are two main problems, the first is DLL version incompatabilty, for which they have even written a program called DLLHELL, which should not be a problem for careful PBDLL writers, however it does raise the question of is there any way to include the version number of a PBDLL such that it will appear on the properties page for a DLL (would be nice to be able to show copyright info too).
    The main problem appears to be that upon loading, the EXE expects to find the DLL at a specific memory location (or the DLL expects to load at a specific memory location) in the programs virtual memory space and finds that another DLL is already there. This causes the loader to have to do significant work relocating the second loaded DLL and load a copy which is no longer shared with other programs.
    If I write programs that use more than one PBDLL DLL is there any way for me to specify the load address of each DLL such that this conflict does not occur? Is the default location of a PBDLL out of the range normally used by Microsoft?
    As each running program is given its own 4 Gig flat address space there appears to be ample room for the programmer of a suite of programs to minimise this address conflict.
    Also is there any way in PBDLL to use ordinal addressing rather than name addressing for exported subs and functions (yes I know the version problems but it is much faster)?

  • #2
    John,

    There is a tool in VC 6 called REBASE that works on PBDLL DLLs.
    The only problem I know about that forces extra copies of a DLL
    to be loaded is when the DLL is compressed with an EXE
    compressor, apart from that, they normally increment the count
    of instances as documented.

    Preferred load adressing is not without its problems, unless you
    can control every DLL in the system, you will end up with
    relocations, even if the preferred load address has been set with
    REBASE or similar because the address range is finite. As far as
    the speed issue goes, it may in fact show on a 486 or similar but
    I suggest that it would in fact be hard to benchmark and trivial
    if you succeeded.

    For a statically loaded DLL, its loaded at startup and there are
    many other things that effect load time apart from loading a DLL.
    If you use LoadLibrary() and GetProcAddress() these considerations
    simply do not matter so I suggest that there is very little to
    gain by doing it. The operating system has priority with load
    addresses so you will always have to fit around them.

    Regards,

    [email protected]

    ------------------


    [This message has been edited by Steve Hutchesson (edited May 13, 2000).]
    hutch at movsd dot com
    The MASM Forum

    www.masm32.com

    Comment


    • #3
      Hutch
      Quote from http://msdn.microsoft.com/library/te...dlldanger1.htm
      ________________________________________________________
      When a DLL is created, a base address is specified by the linker suggesting where Windows should load it in a process's 32-bit address space. The default for DLLs created with Visual C++ is 0x10000000. Consider a shared DLL (call it abc.dll) that specifies 0x20000000 for the requested base address. Application a.exe is currently running with abc.dll loaded at 0x20000000. Application b.exe also uses the same abc.dll but has already loaded def.dll at 0x20000000. The OS must then change the base address (referred to as rebasing) of abc.dll to a unique address in process b. Because process b has rebased abc.dll, the OS cannot share abc.dll with the a.exe process; therefore, in this case the abc.dll provides no memory savings over statically linking with abc.dll.
      ______________________________________________________________
      This article also gives very good reasons for why we should have a method for putting easily readable version numbers in our DLL's
      and hopefully not have to resort to too many "Private DLL's"

      Regards

      ------------------

      Comment


      • #4
        however it does raise the question of is there any way to include the version number of a PBDLL such that it will appear on the properties page for a DLL (would be nice to be able to show copyright info too).
        Sure... create a resource script (if you don't already have one for the DLL and add a version table to it). You can dynamically check the version number at runtime using VerQueryValue() etc.

        Example version text from a .RC file:
        Code:
        VS_VERSION_INFO VERSIONINFO
        FILEVERSION 6, 0, 0, 0
        PRODUCTVERSION 6, 0, 0, 0
        FILEOS VOS__WINDOWS32
        FILETYPE VFT_APP
        BEGIN
          BLOCK "StringFileInfo"
          BEGIN
            BLOCK "040904E4"
            BEGIN
              VALUE "CompanyName",      "PowerBASIC, Inc.\0"
              VALUE "FileDescription",  "Application\0"
              VALUE "FileVersion",      "Version 6.0\0"
              VALUE "InternalName",     "PBNOTE\0"
              VALUE "OriginalFilename", "PBNOTE.EXE\0"
              VALUE "LegalCopyright",   "Copyright \251 1997,1999 PowerBASIC, Inc.\0"
              VALUE "LegalTrademarks",  "PowerBASIC is a trademark of PowerBASIC, Inc.\0"
              VALUE "ProductName",      "PB/DLL Compiler for Windows\0"
              VALUE "ProductVersion",   "Version 6.0\0"
              VALUE "Comments",         "Simple Editor Example.\0"
            END
          END
        END


        ------------------
        Lance
        PowerBASIC Support
        mailto:[email protected][email protected]</A>
        Lance
        mailto:[email protected]

        Comment


        • #5
          I put version information into *ALL* of my DLL's...That's how I track which version I am up to, etc...

          Now the question would be how to load it, but I think there's got to be a way to use LoadString to pull the data out:


          Code:
          VS_VERSION_INFO VERSIONINFO
          FILEVERSION 1, 4, 0, 0
          PRODUCTVERSION 1, 4, 0, 0
          FILEOS VOS_WINDOWS32
          FILETYPE VFT_APP
          BEGIN
            BLOCK "StringFileInfo"
            BEGIN
              BLOCK "040904E4"
              BEGIN
                VALUE "CompanyName",      "Computer Creations Software\000"
                VALUE "FileDescription",  "CCS-Common DLL\000"
                VALUE "FileVersion",      "01.40\000"
                VALUE "InternalName",     "CCS-COMMON\000"
                VALUE "OriginalFilename", "CCS.DLL\000"
                VALUE "LegalCopyright",   "Copyright (c) 1997-2000 Computer Creations Software\000"
                VALUE "LegalTrademarks",  "This product is licensed for Computer Creations Software only!\000"
                VALUE "ProductName",      "CCS-Common DLL\000"
                VALUE "ProductVersion",   "01.40\000"
              END
            END
          END

          ------------------
          Scott
          mailto:[email protected][email protected]</A>
          Scott Turchin
          MCSE, MCP+I
          http://www.tngbbs.com
          ----------------------
          True Karate-do is this: that in daily life, one's mind and body be trained and developed in a spirit of humility; and that in critical times, one be devoted utterly to the cause of justice. -Gichin Funakoshi

          Comment


          • #6
            Lance
            Thanks for the info
            As I have always used PBDLL for super fast DLL's for other programs (and am old and short on time), I had never used RC files. Really quite simple and effecive. I was pleased to see that contrary to a complaint in another subject in this forum, compiling the RC from the IDE produced both .RES and .PBR files.

            Scott
            Thanks also, re reading the data I han't checked yet but think Lance gave the pointer to this in his first Para.
            Regards


            ------------------

            Comment


            • #7
              John,

              Pretty much the same problem, rebasing a DLL will not fix the overlap
              of preferred loading addresses unless you can control every DLL in the
              system which of course you can't.

              The operating system does the rebasing on a needs basis. If you loaded
              10 different DLLs for any single application and rebased all of them so
              that their addresses were unique, it would work OK if and only if no
              other DLL in the whole system worked in that address range taken up by
              the 10 loaded DLLs.

              What I have suggested is that rebasing DLLs is a waste of time because
              even if you get it to work on one version of windows, you cannot assume
              that the preferred load addresses on another version of windows will be
              the same. This can be further messed up by different instances of the
              same version of Windows with different DLL loaded.

              One of your clients may have a different set of rebased DLLs loaded from
              another program that just mess up the base addressing that your rebased
              DLLs require. This is why the operating system does its rebasing on the
              fly.

              Now its not that Microsoft don't have an axe to grind here either, as they
              are the main exponent of the versioning problems associated with DLLs
              through badly designed redistributable DLL licencing, most of the problems
              come from installations that re-install an older version over a newer one.

              Another is the incompatibility of one DLL version to its predecessor, NT4
              service pack users can testify to the folly of the Microsoft system here.

              One of the tricks around the DLL problems from Microsoft in win9x is to
              keep the latest system upgrades on disk so that if an installation messes
              up a DLL, you can re-run the upgrade to get the latest version.

              Regards,

              [email protected]

              ------------------
              hutch at movsd dot com
              The MASM Forum

              www.masm32.com

              Comment


              • #8
                Concerning rebasing:

                on the other hand I can imagine a situation where an application suite written in PowerBASIC consisting of several PB dlls and several PB executables all loaded at startup would slow down the load time since the system would need to rebase all dlls successively.

                In this type of situation I would argue that it would be an advantage to choose an arbitrary load base for each DLL in the suite instead of letting them all try to load from the default &H00400000 location.

                Cheers

                Florent

                ------------------

                Comment


                • #9
                  Hutch
                  Couldn't agree with you more about Microsoft being the major culprit. Anyone who has been involved with the debacle called ADO will testify, I already have a comment on another forum showing the left hand doesn't know what the right hand is doing in Microsoft (re dates).
                  However I don't believe that means we should throw our hands up in the air and say there is nothing we can do about it! For me PB is all about speed (especially for programmers short on time).
                  Whilst as you point out there is no certain way to avoid address conflict, in the normal situation I believe there is a way to improve it.
                  Firstly of the 4 gig address space allocated to each running program then depending on the op sys (NT, W9X or 2000) 1 to 2 gigs is reserved for the op sys. This I hope means I won't write a DLL in PB that conflicts with any op sys DLL's.
                  Secondly as stated previously the default address for Microsoft's Visual C++ is 0x10000000, and I would hope that PB's compiler is well away from that (thanks Florent I think you have answered that).
                  However in the absence of any published information by PB nor the ability to specify an address when compiling a PB DLL I must assume that I will positively generate a conflict if I write 2 PB DLL's that are called from the same program!
                  At this stage let me correct a couple of errors I have made, first the Microsoft program to help sought out the problem is not called DLLHELL rather it is to solve that problem, second as stated in another subject I have not been using PB since PBDLL 1.0 only PBDLL 1.1.
                  My point is that as an old user I have used it for it's originally advertised purpose, to write fast DLL add ins to VB and have never been disapointed with the performace gains I have achieved. Whilst when I have time I am now experimenting with the newer features of the product my main usage mode has not yet changed. I'm sure that somewhere on this computer I have REBASE as I have full Visual Studio 6 and SDK 2000, the problem is I don't have time to either find it or learn to use it. Thats why I bought PB instead of just using MCASM as I did in the old days.
                  I routinely have my VB programs using several PD DLL's of which I expect my latest to compile at over 1 mb (no bloat). The problem I'm trying to solve is not so much load speed but page faults when every program in a suite has its own copy of the DLL's due to loader the rebasing my own DLL's.
                  Interestingly no one has commented on my original question of ordinal rather than name addressing, yet there are questions in other forums as to how you hide the avaiable exports in a DLL.

                  Regards


                  ------------------

                  Comment


                  • #10
                    John,

                    with my install of vc 6, it is in the
                    c:\Program Files\Microsoft Visual Studio\VC98\BIN dir
                    REBASE.EXE 59664 bytes.

                    It is not a hard utility to use and I think I understand the
                    problem with the size of the DLLs you are loading so it may be
                    worth a try to see if you can get the memory usage down. I
                    normally check for multiple copies of what is loaded by the 3
                    finger salute method and have a look at the list to see what is
                    there.

                    Now I had another suggestion if that does not help you with the
                    memory usage. If I remember rightly, Visual Basic cannot use
                    the true dynamic capacity of DLLs and has to load any DLL
                    statically at startup. What I had in mind, if somewhat messy in
                    its implementation is loading a DLL at startup into VB and having
                    calling functions in that DLL that use LoadLibrary() and
                    GetProcAddress()so that you only loaded the big DLLs dynamically.

                    The idea here is to keep memory usage to transient usage so that
                    the total of clashing load addresses does not generate memory
                    problems. Its a messy solution but if you do track it down to
                    load address problems, it may help some.

                    Regards,

                    [email protected]

                    ------------------


                    [This message has been edited by Steve Hutchesson (edited May 17, 2000).]
                    hutch at movsd dot com
                    The MASM Forum

                    www.masm32.com

                    Comment


                    • #11
                      Hello John,
                      I think you are correct about the compiler supporting ordinal export values, you could hide named functions and use the ordinal index instead.

                      Is this on the wish list yet? It would be very useful, something like:

                      Code:
                      Function MyFunction Ordinal 5 () Export As Long 
                      
                       ' This function is located at ordinal index number 5.
                      
                      End Function
                      What do you guys think?

                      Regards,



                      ------------------
                      Kev G Peel
                      KGP Software
                      Bridgwater, UK.
                      mailto:[email protected]e.co.uk[email protected]</A>

                      Comment


                      • #12
                        The option to select a DLL base is on the "wish list". I'd have to note, though, that to the best of my knowledge, the Windows overhead required to "re-base" a DLL at runtime is completely insignificant. I don't suppose anyone has benchmarks...?

                        The notion of exporting ordinals has been added to The List.


                        ------------------
                        Tom Hanlin
                        PowerBASIC Staff

                        Comment


                        • #13
                          Tom..

                          As previously addressed in another thread, the below article by
                          Matt Pietrek and his benchmarks agree with you regarding speed
                          loading DLL's.
                          http://msdn.microsoft.com/library/pe...0/hood0500.htm

                          An interesting tidbit about API ordinals..

                          http://msdn.microsoft.com/library/de...lprocedure.htm

                          "Important: The ordinal number for a specific API will be different with different operating systems. For example, the ordinal value for GetWindowsDirectory is 432 under Windows 95 (or later), but changes to 338 under Window NT 4.0. In sum, if you expect your applications to be run under different operating systems, don't use ordinal numbers to identify API procedures. This approach can still be useful when used with procedures that are not APIs, or when used in applications that have a very controlled distribution."



                          ------------------
                          Jim..
                          [email protected]
                          Jim..

                          Comment


                          • #14
                            Tom
                            I agree that the time it takes the windows loader to rebase a DLL is probably insignificant as stated by Microsft themselves. The real loss is that once rebased the DLL code is not used by any other concurrently running application that uses the same DLL, thus requiring multiple copies using up memory and creating page faults. This behaviour is quite logical as windows only maps a DLL into the application space, thus as rebasing changes internal referances all concurrently running applications must have the same base address for the DLL to ba able to use the same copy.
                            Jim
                            Your comments about the care needed with ordinal addressing in DLL's is well taken as the calling program needs to exactly match the called DLL. The operating system used i.e. NT, 95, 98 or 2000 does not effect the ordinal numbers of non Microsoft DLL's, rather the quote you have given is merely pointing that Microsfot may have a different number of callable functions, or they may be placed in a different order in the various OS's. I raised this point as a result of a question in another forum as to how Microsoft hides some of its DLL functions from general use. I believe they do it by telling the compiler to generate ordinal references and not load the name table. This is a standard function of their commercially available C compiler and it would not take too big a strech of the imagination to conclude that the in house compiler can do this selectively by exported function.
                            Regards

                            ------------------

                            Comment


                            • #15
                              Code:
                              The December 2000 issue of Windows Developer's Journal contains 
                              an excellent article about DLL rebasing and includes C code for a 
                              rebasing utility.  The code also can be downloaded from the WDJ web 
                              site: 
                                 
                               http://www.wdj.com/

                              ------------------
                              -- Greg
                              [email protected]

                              Comment


                              • #16
                                alternatively i posted some source code a little while ago in the source code forum doing just that
                                at http://www.powerbasic.com/support/pb...ad.php?t=22797

                                cheers

                                florent


                                [this message has been edited by florent heyworth (edited november 22, 2000).]

                                Comment

                                Working...
                                X