Limit on size of PBDLL programs?

[Mike Jenkins]

<<
>don't really have a clue about manipulating data thru
pointers <

That's not always necessary. In fact in my experience it is
rarely necessary.
>>

Hmmm... Thanks, Eric. That's very useful to know. Most of
my SUBs/FUNCTIONs are pretty large and draw on GLOBALs or
other SUBs/FUNCTIONs, but I may be able to at least move a few
of the self-contained ones to DLLs, as you suggested. I had
avoided DLL's before, since it seemed beyond my meager knowledge
base, and there's only a brief mention in the PBDLL docs.


------------------

[Eric Pearson]

Mike --

You might want to read this...
http://www.powerbasic.com/support/technote/dlls.asp

-- Eric

------------------
Perfect Sync Development Tools
Perfect Sync Web Site
Contact Us: mailto:[email protected][email protected]</A>

[Mike Jenkins]

Originally posted by Iain Johnstone:
I hit the same problem last year with a limit at 820K EXE. Since
then, all the new code goes into DLLs. These are not so bad once
you get used to them - I use a skeleton program to avoid rewriting
the initial code. DLLs are the only way out of this problem ( I think!!)
Iain Johnstone

Methinks you are right, Iain. Either that, or I'll just use
brute force, breaking my large program into two or three .EXEs
and writing the data to disk each time the user needs to go to
a different part of the program that's in the other .EXE.

Thank God for fast disk drives these days.... Or I'd be forced to
learn how to do DLLs.



------------------

[Russ Srole]

Mike,

Like a lot of folks, I've hit that same wall & at first there was a
very large sinking feeling, but like Iain, I got over it & now the
dll thing is just not that big of a deal. In fact I put almost all
my design code in the dll & have made the constants and include file
so I don't have to worry about screwing it up. I have one rule and
that is that every morning I do a full compile of all dlls and the
exe, so that any changes to the common includes are accounted for.
Same thing goes if I make a change to a common include. You'll be
surprise at how much stuff you can put in a dll without much trouble
at all. Start with the routines that don't use your globals. Also
start with short routines. Later on you can write new routines in
your exe and when you've debugged it, then move it to the dll.

Good luck,
Russ Srole



------------------

[Mike Jenkins]

Thanks, Russ, and Michael, and everyone else who's contributed
to this thread, for your help. This forum is invaluable for
Windows Virgins like meself.

------------------

[Bernard Chapman]

I know that I am operating very close to the 16 bit compiler limit as when I overstep the mark I get a message "Out of main memory".

So far I have been able to step back from the brink each time but I am always very close to the yawning chasm. Rather than adding yet another DLL with all its parameter transfer complications I am taking a punt on PB coming out with 32 bit compiler shortly. I know it is risky to count on such an outcome and I respect PB's "no vapourware" policy but I am used to risk in the industry that I am in.

If it doesn't eventuate then I will have to take alternative measures.

For your interest below is the compile report of my main EXE of almost 50,000 lines of source code. I am not sure which part of it is actually pushing the limit.

PowerBasic DLL/32 Compiler
PB/DLL  Version 6.0 
Copyright (c) 1996-99 by PowerBasic Inc.
Carmel, California, USA
All Rights Reserved 

Primary source:  WINIT32.BAS   {49884 total lines}
Compile time:  10.5 seconds, at 285051 lines/minute
 
533920 bytes compiled code, 26399 bytes RTLibrary,
21116 bytes string literals, and 62520 bytes dgroup.
Executable stack size:  1048576 bytes.
Disk image: 1203712 bytes   Memory image: 110035 bytes.

Regards,

Bern

------------------
Bern

www.insighttrading.com.au

[Eric Pearson]

Just a thought...

For the last couple of weeks I have been dealing with a certain customer who wants me to address a software problem at his site in a particular way. Frankly he is a semi-technical know-it-all non-programmer who thinks he understands computers because... Sorry, I digress. Let's just say that he doesn't understand several very important issues. His problem is, at least in part, a prejudice against BASIC because "everybody knows that C is better".

The solution that he is focusing on is extreme overkill, and would involve the re-writing of large amounts of time-tested code. I have a simple solution that I have shown to work perfectly, but he still wants me to do it his way.

My point is that IMO it is a mistake to focus too much on one perceived solution to any given problem. In other words, phrases like...

> the 16 bit compiler limit

...and...

> I am taking a punt on PB coming out
> with 32 bit compiler shortly

...are a little like saying "I hope my doctor prescribes (name of drug) to cure my problem".

Do you really care what the doctor prescribes? It's more productive to say "I hope my doctor prescribes an effective (drug, or whatever) to cure my problem".

I am not trying to pick on you specifically, Bern. Phrases like that are pretty common on the BBS these days, in several different threads, and I can't figure out why so many people are focused on the perceived issue of whether or not the compiler itself is a 16- or 32-bit app.

If PB comes out with a version of PB/DLL that can compile larger programs, do you really care?

-- Eric


------------------
Perfect Sync Development Tools
Perfect Sync Web Site
Contact Us: mailto:[email protected][email protected]</A>



[This message has been edited by Eric Pearson (edited July 12, 2001).]

[Paul Dwyer]

Originally posted by Mike Jenkins:
Methinks you are right, Iain. Either that, or I'll just use
brute force, breaking my large program into two or three .EXEs
and writing the data to disk each time the user needs to go to
a different part of the program that's in the other .EXE.

Thank God for fast disk drives these days.... Or I'd be forced to
learn how to do DLLs.

Mike,

I'm a little uncertain about your reluctance toward DLLs! I'm not much of a windows coder either but it strikes me that if you can break your code up into exe's then your're 99% of the way there!

Once you've broken things up sucsessfully then your globals blah blah issues are gone. just remove the "Compile exe" from the start, add the export the word the functions you want to use, declare the functions in your exe bas's and call them like they are sitting in an include file.

It's good programming style, in the direction your going anyway and best of all it looks to solve your problem. I've been following this thread with interest because while I've never hit this problem it's about the 10th time I've seen it in this forum so it's definately a problem that needs fixing and with zero update announcements from PB by policy it's something we all need to work around by ourselves to secure our investment in programming time



------------------
Paul Dwyer
Network Engineer
Aussie in Tokyo

[Russ Srole]

Eric,

Your comments really hit home with me, probably because we come
from a similar background (both really TV guys). I have found that
the nature of the television business creates a results orientation.
That is because, if we don't get those shows on the air then we
don't have a job. The computer world seems very process oriented.
So lots of folks focus on languages, OS's, OOPS, COM and all the
other stuff that doesn't directly relate to the end result. It's
really odd to me, but I've seen it time & time again. The only
solution I know of is to give the customer two prices; my way costs
$x, your way costs $x * 100 (or 1000). If he makes it worth your
while then so be it.

Russ Srole

------------------

[Mike Jenkins]

Originally posted by Paul Dwyer:
Mike,

I'm a little uncertain about your reluctance toward DLLs! I'm
not much of a windows coder either but it strikes me that if
you can break your code up into exe's then your're 99% of the
way there!

Once you've broken things up sucsessfully then your globals
blah blah issues are gone. just remove the "Compile exe" from
the start, add the export the word the functions you want to
use, declare the functions in your exe bas's and call them like
they are sitting in an include file.

Paul, what you say is probably quite true in most cases.

However, in the particular financial simulation I've been
porting to Windows ("Wall Street Raider"), most of my SUBs or
functions do a large number of complex calculations, drawing
from many different parts of a GLOBAL database, and calling
several other similarly complex functions/subs that do likewise.
I generally need to pass only one or two parameters to a
function in the main program, and let it massage the numbers by
drawing from the many GLOBAL data arrays as needed (various
asset accounts, tax and accounting records, reported earnings,
profitability factors, strategies, etc., for each of 1600
companies in the simulation).

I don't know, but strongly suspect I would quickly run into other
compiler or Win limitations if I tried to pass fifty or a
hundred or so large arrays, and quite a few non-array variable,
as parameters, to each called function....

Unless DLL's have access to the huge number of global arrays
and other globals of the main EXE (which I don't think is the
case -- please correct me, someone, if I'm wrong, as that would
be great good news in my present predicament!), it seems that in
my case it won't make much difference whether I use multiple
EXEs or DLLs, if I'm going to have to write the entire global
database (or most of it) to disk anyway, each time I need to
shell to the other EXE or call a DLL, and then have it update
data in all the arrays it has modified.

I have only a few, tiny functions that don't need to access
globals, so I could put those in DLLs, but since it looks like
I can probably solve my problem by simply breaking the EXE into
2 large EXEs, it's simpler to just duplicate a few such small
utility functions in the two EXEs, and keep the program down to
2 executables and a couple of permanent data files, easy to
install and use, even by the unwashed masses.

If I had known last year that I would lose the convenience of
having access to globals, by eventually having to break the
program up into multiple modules, I might have broken each
lengthy bit of processing each SUB or FUNCTION does into a
dozen or so partial calculation steps, so that I would have
had thousands, not a couple of hundred, of such subs/functions,
most of which could be DLLs, each drawing on only one or two
arrays.

Next time I start on a huge program, I'll know better, but for
now it looks like the less elegant solution is the only way out,
unless I want to add another year to my development time....

Hence my reluctance to use DLLs in the instant case, except
in a few cases where they will still make sense at this late
stage of development. (I went thru this exact same problem
with my original Wall Street Raider, using Turbo Basic, many
years ago, and wound up having to do about 8 .EXEs, but it's
always worked like a charm, and stills sells fairly well, one
of the last non-graphic DOS programs still out there. Kludgy,
but problem-free.)

The only other solution I can see with an unusual program like
mine would be to maintain the database entirely on disk, but as
my simulation is constantly updating all parts of the database,
somewhat randomly, with thousands of calcs every second, most
users would probably not like to see their hard disk grinding
away constantly the whole time they are using the program (which
is hours at a stretch, the serious addicts tell me).

------------------

[Michael Mattias]

.. I can't figure out why so many people are focused on the perceived issue of whether or not the compiler itself is a 16- or 32-bit app.

I think I said earlier in this thread that I don't care if the compiler itself is a 32-bit product, but that seemed the easiest route to get to what the current compiler has as its biggest problems/bugs/deficiencies:

1. Compiler GPFs when attempting to allocate too much memory. The 16-bit fix is to "Check ERRCLEAR" after each allocation and handle by using application-managed virtual (disk) memory, or at least politely terminating with a message that the program is too large to compile.

2. No long filename support for #INCLUDE, #RESOURCE or #COMPILE. ( I don't know of a 16-bit fix for that).

3. The third major problem is not sensitive to the OS: The documentation is full of errors and long on omissions; and the help file itself is user-hostile.

Should PB deal with these, I'll be a happy camper.

MCM

[Bernard Chapman]

Eric,

You are dead right. All I want is to be able to compile larger apps. However, I understood from what I have been told that we were up against an impenetrable barrier - a bit like the Second Law of Thermodynamics.

I also understood that the only way to overcome it was to go to a 32 bit compiler that I was told is already on the wish list.

But you are right, any solution at all would be most gratefully and enthusiastically received.

Regards,

Bern

PS Just out of interest, the compile speed listed in my previous message seems slower than others have been reporting.

Even on 1300MHz Pentium 4 it increases to about 550 000 lines per minute. Nowhere near the 1 000 000 plus that others seem to achieve. Maybe it has to do with line length.

This compile speed is excellent for my purposes and so is not an issue for me as it is not a rate limiting step.

------------------
Bern

www.insighttrading.com.au

[Bern Ertl]

Mike (Jenkins - too many Mikes!),

It *IS* possible for your .EXE and .DLLs to share the
same GLOBAL variables. Arrays in the DLL can be mapped
(overlayed) to the same memory space as the GLOBALS in
the EXE. Scalar variables will either need to be copied
to corresponding GLOBALs in the DLL or you could just use
a pointer with indirect referencing.

.EXE -->

#INCLUDE "GLOBALS.INC"
...
DIM GlobalVarAddresses( %MaxNumGlobalVarsAndArrays) AS DWORD
GlobalVarAddresses( 0) = VARPTR( FirstGlobalVar&)
GlobalVarAddresses( 1) = VARPTR( FirstGlobalArray&( 0))
etc...
CALL InitDLL( VARPTR( GlobalVarAddresses( 0)))

.DLL -->

#INCLUDE "GLOBALS.INC"
...
SUB InitDLL( BYVAL ArrayAddress AS DWORD PTR)

FirstGlobalVar& = @ArrayAddress 'or just use @ArrayAddress
DIM FirstGlobalArray&( %FirstGlobalArraySize) AT @ArrayAddress[1]
etc...



------------------
Bernard Ertl