Announcement

Collapse
No announcement yet.

Limit on size of PBDLL programs?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Bern Ertl
    replied
    Mike (Jenkins - too many Mikes!),

    It *IS* possible for your .EXE and .DLLs to share the
    same GLOBAL variables. Arrays in the DLL can be mapped
    (overlayed) to the same memory space as the GLOBALS in
    the EXE. Scalar variables will either need to be copied
    to corresponding GLOBALs in the DLL or you could just use
    a pointer with indirect referencing.

    .EXE -->

    #INCLUDE "GLOBALS.INC"
    ...
    DIM GlobalVarAddresses( %MaxNumGlobalVarsAndArrays) AS DWORD
    GlobalVarAddresses( 0) = VARPTR( FirstGlobalVar&)
    GlobalVarAddresses( 1) = VARPTR( FirstGlobalArray&( 0))
    etc...
    CALL InitDLL( VARPTR( GlobalVarAddresses( 0)))

    .DLL -->

    #INCLUDE "GLOBALS.INC"
    ...
    SUB InitDLL( BYVAL ArrayAddress AS DWORD PTR)

    FirstGlobalVar& = @ArrayAddress 'or just use @ArrayAddress
    DIM FirstGlobalArray&( %FirstGlobalArraySize) AT @ArrayAddress[1]
    etc...



    ------------------
    Bernard Ertl

    Leave a comment:


  • Bernard Chapman
    replied
    Eric,

    You are dead right. All I want is to be able to compile larger apps. However, I understood from what I have been told that we were up against an impenetrable barrier - a bit like the Second Law of Thermodynamics.

    I also understood that the only way to overcome it was to go to a 32 bit compiler that I was told is already on the wish list.

    But you are right, any solution at all would be most gratefully and enthusiastically received.

    Regards,

    Bern

    PS Just out of interest, the compile speed listed in my previous message seems slower than others have been reporting.

    Even on 1300MHz Pentium 4 it increases to about 550 000 lines per minute. Nowhere near the 1 000 000 plus that others seem to achieve. Maybe it has to do with line length.

    This compile speed is excellent for my purposes and so is not an issue for me as it is not a rate limiting step.

    ------------------
    Bern

    www.insighttrading.com.au

    Leave a comment:


  • Michael Mattias
    replied
    .. I can't figure out why so many people are focused on the perceived issue of whether or not the compiler itself is a 16- or 32-bit app.
    I think I said earlier in this thread that I don't care if the compiler itself is a 32-bit product, but that seemed the easiest route to get to what the current compiler has as its biggest problems/bugs/deficiencies:

    1. Compiler GPFs when attempting to allocate too much memory. The 16-bit fix is to "Check ERRCLEAR" after each allocation and handle by using application-managed virtual (disk) memory, or at least politely terminating with a message that the program is too large to compile.

    2. No long filename support for #INCLUDE, #RESOURCE or #COMPILE. ( I don't know of a 16-bit fix for that).

    3. The third major problem is not sensitive to the OS: The documentation is full of errors and long on omissions; and the help file itself is user-hostile.

    Should PB deal with these, I'll be a happy camper.

    MCM

    Leave a comment:


  • Guest's Avatar
    Guest replied
    Originally posted by Paul Dwyer:
    Mike,

    I'm a little uncertain about your reluctance toward DLLs! I'm
    not much of a windows coder either but it strikes me that if
    you can break your code up into exe's then your're 99% of the
    way there!

    Once you've broken things up sucsessfully then your globals
    blah blah issues are gone. just remove the "Compile exe" from
    the start, add the export the word the functions you want to
    use, declare the functions in your exe bas's and call them like
    they are sitting in an include file.
    Paul, what you say is probably quite true in most cases.

    However, in the particular financial simulation I've been
    porting to Windows ("Wall Street Raider"), most of my SUBs or
    functions do a large number of complex calculations, drawing
    from many different parts of a GLOBAL database, and calling
    several other similarly complex functions/subs that do likewise.
    I generally need to pass only one or two parameters to a
    function in the main program, and let it massage the numbers by
    drawing from the many GLOBAL data arrays as needed (various
    asset accounts, tax and accounting records, reported earnings,
    profitability factors, strategies, etc., for each of 1600
    companies in the simulation).

    I don't know, but strongly suspect I would quickly run into other
    compiler or Win limitations if I tried to pass fifty or a
    hundred or so large arrays, and quite a few non-array variable,
    as parameters, to each called function....

    Unless DLL's have access to the huge number of global arrays
    and other globals of the main EXE (which I don't think is the
    case -- please correct me, someone, if I'm wrong, as that would
    be great good news in my present predicament!), it seems that in
    my case it won't make much difference whether I use multiple
    EXEs or DLLs, if I'm going to have to write the entire global
    database (or most of it) to disk anyway, each time I need to
    shell to the other EXE or call a DLL, and then have it update
    data in all the arrays it has modified.

    I have only a few, tiny functions that don't need to access
    globals, so I could put those in DLLs, but since it looks like
    I can probably solve my problem by simply breaking the EXE into
    2 large EXEs, it's simpler to just duplicate a few such small
    utility functions in the two EXEs, and keep the program down to
    2 executables and a couple of permanent data files, easy to
    install and use, even by the unwashed masses.

    If I had known last year that I would lose the convenience of
    having access to globals, by eventually having to break the
    program up into multiple modules, I might have broken each
    lengthy bit of processing each SUB or FUNCTION does into a
    dozen or so partial calculation steps, so that I would have
    had thousands, not a couple of hundred, of such subs/functions,
    most of which could be DLLs, each drawing on only one or two
    arrays.

    Next time I start on a huge program, I'll know better, but for
    now it looks like the less elegant solution is the only way out,
    unless I want to add another year to my development time....

    Hence my reluctance to use DLLs in the instant case, except
    in a few cases where they will still make sense at this late
    stage of development. (I went thru this exact same problem
    with my original Wall Street Raider, using Turbo Basic, many
    years ago, and wound up having to do about 8 .EXEs, but it's
    always worked like a charm, and stills sells fairly well, one
    of the last non-graphic DOS programs still out there. Kludgy,
    but problem-free.)

    The only other solution I can see with an unusual program like
    mine would be to maintain the database entirely on disk, but as
    my simulation is constantly updating all parts of the database,
    somewhat randomly, with thousands of calcs every second, most
    users would probably not like to see their hard disk grinding
    away constantly the whole time they are using the program (which
    is hours at a stretch, the serious addicts tell me).

    ------------------

    Leave a comment:


  • Russ Srole
    replied
    Eric,

    Your comments really hit home with me, probably because we come
    from a similar background (both really TV guys). I have found that
    the nature of the television business creates a results orientation.
    That is because, if we don't get those shows on the air then we
    don't have a job. The computer world seems very process oriented.
    So lots of folks focus on languages, OS's, OOPS, COM and all the
    other stuff that doesn't directly relate to the end result. It's
    really odd to me, but I've seen it time & time again. The only
    solution I know of is to give the customer two prices; my way costs
    $x, your way costs $x * 100 (or 1000). If he makes it worth your
    while then so be it.

    Russ Srole

    ------------------

    Leave a comment:


  • Paul Dwyer
    replied
    Originally posted by Mike Jenkins:
    Methinks you are right, Iain. Either that, or I'll just use
    brute force, breaking my large program into two or three .EXEs
    and writing the data to disk each time the user needs to go to
    a different part of the program that's in the other .EXE.

    Thank God for fast disk drives these days.... Or I'd be forced to
    learn how to do DLLs.

    Mike,

    I'm a little uncertain about your reluctance toward DLLs! I'm not much of a windows coder either but it strikes me that if you can break your code up into exe's then your're 99% of the way there!

    Once you've broken things up sucsessfully then your globals blah blah issues are gone. just remove the "Compile exe" from the start, add the export the word the functions you want to use, declare the functions in your exe bas's and call them like they are sitting in an include file.

    It's good programming style, in the direction your going anyway and best of all it looks to solve your problem. I've been following this thread with interest because while I've never hit this problem it's about the 10th time I've seen it in this forum so it's definately a problem that needs fixing and with zero update announcements from PB by policy it's something we all need to work around by ourselves to secure our investment in programming time



    ------------------
    Paul Dwyer
    Network Engineer
    Aussie in Tokyo

    Leave a comment:


  • Eric Pearson
    replied
    Just a thought...

    For the last couple of weeks I have been dealing with a certain customer who wants me to address a software problem at his site in a particular way. Frankly he is a semi-technical know-it-all non-programmer who thinks he understands computers because... Sorry, I digress. Let's just say that he doesn't understand several very important issues. His problem is, at least in part, a prejudice against BASIC because "everybody knows that C is better".

    The solution that he is focusing on is extreme overkill, and would involve the re-writing of large amounts of time-tested code. I have a simple solution that I have shown to work perfectly, but he still wants me to do it his way.

    My point is that IMO it is a mistake to focus too much on one perceived solution to any given problem. In other words, phrases like...

    > the 16 bit compiler limit

    ...and...

    > I am taking a punt on PB coming out
    > with 32 bit compiler shortly

    ...are a little like saying "I hope my doctor prescribes (name of drug) to cure my problem".

    Do you really care what the doctor prescribes? It's more productive to say "I hope my doctor prescribes an effective (drug, or whatever) to cure my problem".

    I am not trying to pick on you specifically, Bern. Phrases like that are pretty common on the BBS these days, in several different threads, and I can't figure out why so many people are focused on the perceived issue of whether or not the compiler itself is a 16- or 32-bit app.

    If PB comes out with a version of PB/DLL that can compile larger programs, do you really care?

    -- Eric


    ------------------
    Perfect Sync Development Tools
    Perfect Sync Web Site
    Contact Us: mailto:[email protected][email protected]</A>



    [This message has been edited by Eric Pearson (edited July 12, 2001).]

    Leave a comment:


  • Bernard Chapman
    replied
    I know that I am operating very close to the 16 bit compiler limit as when I overstep the mark I get a message "Out of main memory".

    So far I have been able to step back from the brink each time but I am always very close to the yawning chasm. Rather than adding yet another DLL with all its parameter transfer complications I am taking a punt on PB coming out with 32 bit compiler shortly. I know it is risky to count on such an outcome and I respect PB's "no vapourware" policy but I am used to risk in the industry that I am in.

    If it doesn't eventuate then I will have to take alternative measures.

    For your interest below is the compile report of my main EXE of almost 50,000 lines of source code. I am not sure which part of it is actually pushing the limit.

    Code:
    PowerBasic DLL/32 Compiler
    PB/DLL  Version 6.0 
    Copyright (c) 1996-99 by PowerBasic Inc.
    Carmel, California, USA
    All Rights Reserved 
    
    Primary source:  WINIT32.BAS   {49884 total lines}
    Compile time:  10.5 seconds, at 285051 lines/minute
     
    533920 bytes compiled code, 26399 bytes RTLibrary,
    21116 bytes string literals, and 62520 bytes dgroup.
    Executable stack size:  1048576 bytes.
    Disk image: 1203712 bytes   Memory image: 110035 bytes.
    Regards,

    Bern

    ------------------
    Bern

    www.insighttrading.com.au

    Leave a comment:


  • Guest's Avatar
    Guest replied
    Thanks, Russ, and Michael, and everyone else who's contributed
    to this thread, for your help. This forum is invaluable for
    Windows Virgins like meself.

    ------------------

    Leave a comment:


  • Russ Srole
    replied
    Mike,

    Like a lot of folks, I've hit that same wall & at first there was a
    very large sinking feeling, but like Iain, I got over it & now the
    dll thing is just not that big of a deal. In fact I put almost all
    my design code in the dll & have made the constants and include file
    so I don't have to worry about screwing it up. I have one rule and
    that is that every morning I do a full compile of all dlls and the
    exe, so that any changes to the common includes are accounted for.
    Same thing goes if I make a change to a common include. You'll be
    surprise at how much stuff you can put in a dll without much trouble
    at all. Start with the routines that don't use your globals. Also
    start with short routines. Later on you can write new routines in
    your exe and when you've debugged it, then move it to the dll.

    Good luck,
    Russ Srole



    ------------------

    Leave a comment:


  • Guest's Avatar
    Guest replied
    Originally posted by Iain Johnstone:
    I hit the same problem last year with a limit at 820K EXE. Since
    then, all the new code goes into DLLs. These are not so bad once
    you get used to them - I use a skeleton program to avoid rewriting
    the initial code. DLLs are the only way out of this problem ( I think!!)
    Iain Johnstone

    Methinks you are right, Iain. Either that, or I'll just use
    brute force, breaking my large program into two or three .EXEs
    and writing the data to disk each time the user needs to go to
    a different part of the program that's in the other .EXE.

    Thank God for fast disk drives these days.... Or I'd be forced to
    learn how to do DLLs.



    ------------------

    Leave a comment:


  • Eric Pearson
    replied
    Mike --

    You might want to read this...
    http://www.powerbasic.com/support/technote/dlls.asp

    -- Eric

    ------------------
    Perfect Sync Development Tools
    Perfect Sync Web Site
    Contact Us: mailto:[email protected][email protected]</A>

    Leave a comment:


  • Guest's Avatar
    Guest replied
    <<
    >don't really have a clue about manipulating data thru
    pointers <

    That's not always necessary. In fact in my experience it is
    rarely necessary.
    >>

    Hmmm... Thanks, Eric. That's very useful to know. Most of
    my SUBs/FUNCTIONs are pretty large and draw on GLOBALs or
    other SUBs/FUNCTIONs, but I may be able to at least move a few
    of the self-contained ones to DLLs, as you suggested. I had
    avoided DLL's before, since it seemed beyond my meager knowledge
    base, and there's only a brief mention in the PBDLL docs.


    ------------------

    Leave a comment:


  • Iain Johnstone
    replied
    I hit the same problem last year with a limit at 820K EXE. Since
    then, all the new code goes into DLLs. These are not so bad once
    you get used to them - I use a skeleton program to avoid rewriting
    the initial code. DLLs are the only way out of this problem ( I think!!)
    Iain Johnstone

    ------------------

    Leave a comment:


  • Brent Boshart
    replied
    I have a program with over 41,000 lines of codes with modules over 10,000 lines and it compiles fine to a single exe about 582k in size.

    Brent Boshart

    ------------------

    Leave a comment:


  • jeroen brouwers
    replied
    I can confirm the same problems to some extend.

    Mike, I don't know your code but another limitation is "there can be only so many 'if elseif' constructions in a sub or function". Breaking them up into several subs / functions does work OK.
    I can also agree with the idea of having seperate Dlls for common functions. You can just declare the dlls or load them when needed and pass the info needed back and forth from your main app (dll or exe) to these dlls. From a point of maintenance, this is also very advisable.

    Good luck with it.

    Greetings
    Jeroen


    ------------------


    [This message has been edited by jeroen brouwers (edited July 10, 2001).]

    Leave a comment:


  • Bern Ertl
    replied
    i have been reading this thread with some interest as i have already
    experienced most of these problems myself.

    mike jenkins wrote:

    no longer able to compile it. i keep getting a "destination
    file write error, line 10111" error message.
    i've seen that error message before when a copy of my program
    was still in memory, but that isn't the case this time.
    both times this has happened to me, i've split off a ton of code into
    a dll and everything worked again. check out the following for more
    help on passing globals between the exe & dll:

    http://www.powerbasic.com/support/pb...ead.php?t=3825


    mike jenkins wrote:

    oddly, i tried removing about 60k of source code, which i'll
    try to make into a separate exe, and found that only reduced the
    size of my main .exe by about 24k, not a big reduction, and hard
    to understand.
    not really. there is not a one to one correspondence between source
    code and executable (compiled) code. consider that :

    Code:
    areallyreallyreallyreallylongvariablename& = anotherlongvariablename&
    occupies roughly 70 bytes of source code. this should compile into
    just a few bytes of executable code while:

    Code:
    a& = ceil((b&^3+95)/23)*450
    occupies roughly 30 bytes of source code and should compile into many
    more bytes than the previous variable assignment.

    mike jenkins wrote:

    a half-dozen .exes, to be on the safe side, as i'm still getting
    odd errors, even after removing 60k of source code, wherever i
    put in msgboxes. example from one sub:

    a$="blah, blah, blah...."
    x=1030
    testcode$="100"
    msgbox a$
    testcode$="101"

    after the msgbox appears, and i click on ok, i get a run-time
    error in my errortrap routine, of error #9, testcode$="100", so
    i started a thread a while back when i found a similar problem in my code:

    http://www.powerbasic.com/support/pb...ead.php?t=3850


    i would strongly suggest checking your code for out of bounds array
    assignments. sometimes your app will continue running for a while
    before the error becomes noticable/problematic.




    ------------------
    bernard ertl

    Leave a comment:


  • Michael Mattias
    replied
    If you find yourself in a situation where you cannot conveniently eliminate GLOBAL variables when moving functions to a DLL, you can always write a function in the DLL which does noting but SET or GET global variables from the other module.

    That is

    Code:
    #COMPILE DLL
    
    
    ...
    
    GLOBAL a, b, c, d, e     ' will be GLOBAL in the DLL
    
    FUNCTION SetGlobal (VarNumber, Value) AS LONG
    
       IF VarNumber = %Set_A THEN
             a = Value
       ELSEIF VarNumber = %Set_B THEN
             b = Value
        etc etc
    END FUNCTION
    
     =======
    
    #COMPILE EXE
    
    ....
    
    DECLARE FUNCTION SetGlobal LIB "MYDLL.DLL" ALIAS "SETGLOBAL" (Varnum, Value)
    ....
    
    '(change global variable A, and need to notify the DLL)
      CALL SetGlobal (%Set_A, newAValue)

    Nope, I do not recommend this if you are starting from a clean sheet of paper, but it could get you past the current situation.

    MCM


    Leave a comment:


  • Guest's Avatar
    Guest replied
    <<Originally posted by Eric Pearson:
    Mike --

    Are you using the IDE? If so, it protects you from using source
    code lines that are longer than 255 characters. But if you are
    using UltraEdit or something else, you have to watch that limit
    yourself. No lines -- even comments or lines in $IF 0 blocks --
    can be longer than 255 characters, or you can get unexpected and
    hard-to-find errors.

    Just a thought...

    -- Eric
    >>

    Eric --

    Yup, I've written everything but my data files with the IDE.
    Thanks, anyway. I know what you mean. I have run into that
    problem where I tried piecing together a string of over 256
    characters, to display in a MSGBOX, which created some
    unpredictable results. (Learned that lesson the hard way, too.)

    -- Mike

    ------------------

    Leave a comment:


  • Eric Pearson
    replied
    > don't really have a clue about manipulating data thru pointers

    That's not always necessary. In fact in my experience it is rarely necessary.

    Do you have any subs or functions that do not use any GLOBAL variables and that do not call other functions? Or can you create a "group" of subs/functions that is self-contained from a GLOBAL and cross-calling standpoint? If so, that code can be moved 100% painlessly to a DLL.

    Things like self-contained parsing functions, utility functions... lots of things can be moved to a DLL without using any advanced techniques at all.

    -- Eric


    ------------------
    Perfect Sync Development Tools
    Perfect Sync Web Site
    Contact Us: mailto:[email protected][email protected]</A>

    [This message has been edited by Eric Pearson (edited July 09, 2001).]

    Leave a comment:

Working...
X