Some little sum will find you some day errors!

[Mike Luther]

With appologies to the original poet .. who was talking about alchohol
and liver problems ...

Some little bug will find you some day.
Some little bug will get behind you some day.
With a nervous little quiver, the numbers won't deliver!
Some little bug will find you some day.

The below bug took down a site today and hasn't been seen until now in
a long time of using the comparisons where double precision numbers
are in use. It's a late Intel Pentium, not the one with the math bug
in the chip. The compiler is set to FLOATING POINT EMULATION. So,
just how does one ever become a real good behind the scenes bookkeeper
or mission control programmer under the following bug I'll illustrate?

Sure I can put in a conditional value that excludes number results
so small as to be infinitesimal in EVERY calculation! But that
would mean re-writing all the code for double precision numbers
everywhere and that is a *LOT* of work... gloom.

If I do that here in this instance to trap what we just found, and
let it post, how do I keep the infintesimal residual from destroying
a complete fully balanced accounting system or servo control operation
for heavy equipment, for example?

I offer for show and tell:

10 ' JUNK.BAS failure example for PB 3.5 after a site crash 11-Nov-2001
CLS
PRINT "Let's examine double precision for real world tasks? OK?
PRINT
' Play like we read from a text data file on a disk or COMM stream.
P1$ = "456.48" '                        Use string from disk file
P2$ = "-30.00" '                        Use string from disk file
P3$ = "-152.16" '                       Use string from disk file
P4$ = "-84.38" '                        Use string from disk file
P5$ = "-189.94" '                       Use string from disk file
' Compare the values to compute the AMOUNT vs SUM OF PAYOFF'ss
QMT# = VAL(P1$) '                       We need enough to pay this amount
PRINT " We need to pay "; QMT#; "w/pmts of 30.00/ 152.16/ 84.38/ 189.94
QMV# = QMT# '                           Creat a step variable watch
QMP1# = VAL(P2$) '                      Payment #1
QMP2# = VAL(P3$) '                      Payment #2
QMP3# = VAL(P4$) '                      Payment #3
QMP4# = VAL(P5$) '                      Payment #4
QMM# = QMM# + QMP1# '                   Add a payment to holding variable
QMV# = QMV# + QMP1# '                   Decrement step variable
PRINT " After step 1 "; QMV# '          Print step variable
QMM# = QMM# + QMP2# '                   Add a payment to holding variable
QMV# = QMV# + QMP2#'                    Decrement step variable
PRINT " After step 2 ";QMV#'            Print step variable
QMM# = QMM# + QMP3# '                   Add a payment to holding variable
QMV# = QMV# + QMP3#'                    Decrement step variable
PRINT " After step 3 ";QMV#'            Print step variable
QMM# = QMM# + QMP4# '                   Add a payment to holding variable
QMV# = QMV# + QMP4#'                    Decrement step variable
PRINT " After step 4 ";QMV#'            Print step variable
PRINT " Our debt was      "; QMT# '     Print debt
PRINT " Our payments were "; QMM# '     Print payoff total
' We have now enough money to pay this invoice
PRINT " Amount needed for payoff = "; QMT# + QMM#
PRINT " So much for double entry bookeeping in PowerBASIC?"
PRINT
PRINT " Now let's do this in array work too!"
' Now dimension double precision arrays and do this in them
DIM QAMT#(350) '                        Dimension double precision array
DIM QAMP#(350) '                        Dimension double precision array
DIM QAMM#(350) '                        Dimension double precision array
QAMT#(1) = VAL(P1$) '                   We need enough to pay this amount
QAMP#(1) = VAL(P2$) '                   Payment #1
QAMP#(2) = VAL(P3$) '                   Payment #2
QAMP#(3) = VAL(P4$) '                   Payment #3
QAMP#(4) = VAL(P5$) '                   Payment #4
QAMM#(1) = QAMM#(1) + QAMP#(1) '        Add a payment to holding variable
QAMM#(1) = QAMM#(1) + QAMP#(2) '        Add a payment to holding variable
QAMM#(1) = QAMM#(1) + QAMP#(3) '        Add a payment to holding variable
QAMM#(1) = QAMM#(1) + QAMP#(4) '        Add a payment to holding variable
PRINT " Our debt was      "; QAMT#(1)'  Print debt
PRINT " Our payments were "; QAMM#(1)'  Print payoof total
PRINT " Amount needed for payoff = "; QAMT#(1) + QAMM#(1)
PRINT " So much for double entry bookeeping in PowerBASIC?"
PRINT
PRINT "   Just a thought .... "
PRINT
PRINT " If this were course corrections, and we fired that "
PRINT "  booster rocket until we had 'er right on track ..
PRINT "     so, grin, where did Mariner X go?
PRINT
PRINT " But just in case you think the answer is simple, <CR>:";
ZZ0$ = INPUT$(1)
CLS
PRINT
PRINT " Maybe not!  Compare just the bottom increment!"
PRINT
P6$ = " 189.94" '                       Use string from disk file
P7$ = "-189.94" '                       Use string from disk file
QMT# = VAL(P6$) '                       Convert this to number variable
QMN# = VAL(P7$) '                       Convert this to number variable
PRINT " String to compare is: "; P6$ '  Show actual string
PRINT " String accumulate is: "; P7$ '  Show actual string
PRINT " Amount to compare is: "; QMT# ' Show number variable
PRINT " Amount to compare is: "; QMN# ' Show number variable
PRINT
PRINT " Amount needed for payoff = "; QMT# + QMN# '     Print comparator

Got a suggestion as to live with this deal. True, a bottle in front of
me is better than a frontal lobotomy, but gee folks ...






------------------
Mike Luther
[email protected]

[Mike Burns]

A global search and replace of # with @ fixes the problem ..

It changes the calculation to use BCD variables. You will always
have small amounts using floating point calculations. BCD variables
are excellent for use with money.

There are numbers that can not be expressed exactly in Base 2 just as
there are in Base 10 (1/3 for example).

Mike



------------------

[Lance Edmonds]

Mike Burns is dead right here... your use of double precision variables is causing very small rounding errors due to the problem of accurately storing values that are not an exact power of two (see the documentation for more information on floating point rounding errors).

In summary, floating point values can only accurately represent a finite number of real numbers - all other values are rounded to the closest value that can be represented.

You can see some of the effects of this with slight changes to your code like so:

PRINT " After step 1 "; STR$(QMV#,18) '          Print step variable
...
PRINT " After step 2 "; STR$(QMV#,18) '          Print step variable

The solution is to use the right data type for the job - in this case fixed-point BCD.

------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Mike Luther]

Suggestion taken and the financial side of this is fixed.

However, that does *NOT* solve the non-financial side of the issue
for equipment control as to null positioning needs. I'm going back
and taking another look at that issue elsewhere, as well as critical
other processes which do not use the financial side of life.

It also has some other ramifications. I'm going to have to go back
and think out the entire PB to C++ translation operation that is
working now for a substantial portion of my discrete source. That
added variable type and how to handle the translation for this into
the C++ realm has to be thought about.

This brings up another issue!

How do we postulate interface of the old Microsoft file stored variables
for their format for Double Precision in all this? Reason being is that
I do use and have no trouble with the provided PowerBASIC functions
which do this between it and the old stuff. But what if I want to
bring out an old M/S Double Precision value from a 15 year old disk file
and use it in BCD, then restore the file to what is needed for it?

I really do have files and things which go back all this time quite
happily doing its job for other than financial thing - - and those
thingees as well!

Advice for the data-lorn?






------------------
Mike Luther
[email protected]

[David J Walker]

[Mount soapbox]
The use of floating point computation in mission-critical
applications is not permitted. Ever. Period.
This is written in blood.
Unfortunately, a lot of people have not realised this yet.
[Fall off soapbox]

------------------

[Mike Luther]

Thanks David!

Well up 'til now, it ain't been critical .. but I can sure see
your point!

Again though .. How do we keep the Sins of the Father from being
visited on the Sons?

GDRFC!

Assuming we have a file with Microsoft Binary data stored on it in
double precision. We want to maintain that for compatibility and
it is *NOT* material as to the null. Yet internal stuff that may
be done in the operating portion of the code does depend on knowing
whether to do something or not ... which can be still stored that
way afteer all the logic is done and the tiny differences in the actual
final file value really don't matter.

What does PowerBasic do with something like [email protected] = QTY# or [email protected] = CVDM(R$),
where R$ was a file field element that was and is still to be stored in the
Microsoft format?

What does it do if you go back the other way and convert back from the
BCD format to the Microsoft format for storage and compatibility
purposes?

I think the whole issue will be reasonably moot for me and the conversion
to BCD for the code portions involved will be a reasonably easy
matter .. but not the final storage issue..

Inquiring mind wants to know!

By the way! One of the most interesting of these NULL deals in
telecommunications engineering work I've ever heard about was the
issue at Continental Electronics up in Richardson, Texas! They are
one of only a handful of folks in the megawatt transmitter business,
for such things as the Voice of America, Radio Free Europe, the
multi-megawatt VLF stuff that communicates with submarines under
water from 20 mile long antennas strung between hill tops over a
valley .. the Distant Early Warning DEW line over-the-horizon radar
systems and so on!

In the HF multi-megawatt transmitters, my cousin described that they
use a CAVITY tuned tank circuit in the 3Mhz to 30Mhz units! They
can shift the frequency to any point in that band in some two
seconds or so! It's done by taking a long hall-like whole room and
making one whole wall on the end into a massive plunger like a
piston in an engine! They cover the entire 20 plus feet with that
moving wall driven by a hydraulic ram .. controlled by a computer
driven servo system. Yep .. two seconds .. any place in the whole
range to tune the tank circuit .. wham bam; thank you mam!

Bobby noted that it took some real predictive algorithms to be able
to begin slaming on the brakes for that wall as it moves into
reasonance for the thing. It has to stop EXACTLY on the money at
exactly the right instant. Hence they have to apply both predictive
smoothing to the pressure waveform they create, at the same time
they also have to be dead certain that when it does stop they know
where ZERO is in the equation!

I don't think you'd want to be in the way of the thing as it slams
home up there near the fully plunged 30 Megahertz end!




------------------
Mike Luther
[email protected]

[Lance Edmonds]

This is all for money values, right?

Any "rounding" effects that will occur when translating from Double to fixed-point BCD is likely to have no effect since the rounding errors prone to the double precision values will be at a much greater number of decimal places than the 2 places you will be using with BCD.

For example, if we read the value 1.1000000000008754# (representing, say, $1.10), and you convert that to BCD ([email protected] = b#), then [email protected] will contain [email protected]

IOW, the rounding error amounts are so low that they should be of no consequence at 2 decimal places.

Likewise, you will introduce rounding errors when you translate back from BCD to double precision, but since the precision of the BCD is exact, the rounding effects will be significantly lower (almost non-existant).

Of course, you could always use the ROUND() function in your data conversions code...


------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Mike Luther]

No Lance ..

It's not just about money matters. (But of course money does! Grin!)

I've used double precision for many things inside my work where the
single presicion number system we started with in old M/S basic was
too small to handle the load. If I'd started into this game with
the wonder that is PowerBASIC, I'm sure my perspective would be all
different about these things!

I try not to mind admitting my mistakes, lack of knowlege or understanding
while noting that crow really doesn't taste very good ... chuckle.

I actually looked at the ROUND function for the first time when you
called my attention to it!! Long ago and very far away, the little Heathkit
monthly magazine was being printed for us and I had my first computer -- a
Heathkit H89 serial #659 which I still own and still works.

Some kind soul solved the rounding problem with number juggling that I
wrote down from the contributor's stuff. It became what is now
the SUB PK50060 (QZ#) in my own libraries, over time .. into the world
of double precision. Changing the cloth is a hard habit to break.

Ask any nun. Strictly in humor, I assure you and no offense ever
intended toward the wonderful work they do... or any other charitable
contributor, for that matter ...






------------------
Mike Luther
[email protected]

[David J Walker]

Floating point can really nail you in iterative/recursive situations
where the error can rapidly overcome the product. Aerodynamics,
matrix calculations used in stress work, etc. Bridges fall down,
aircraft crash, etc., etc. If you need REAL accuracy, you need
(dare I say it!) FORTRAN. There are some very accurate (and expensive)
mathematical libraries available. Of course! Power Fortran!
And totally off topic, with all the cheap Sun Sparc kit sloshing
about at the moment, how about PB Linux for Sparc? 4 64-bit CPUs
with 4MB caches, wow!

------------------

[Mike Luther]

Yes, David ..

I'm learning .. I'm learning!

However, given the circumstances, I wonder if a surplus AS-400 wouldn't
be a better choice than the Sun deal.

Here in our area, you used to see this curious bumper stickers on
this or that wreck .. "My other car is a 300SL.."

I guess it would be fun to use a tag line..

"My other computer is an AS-400.."

But even if you got one, how would you pay for the Op system site
rights? Kinda makes the whole arguement pointless about paying M/S
for a $300 upgrade for XP, or a two year subscription to ACP/MCP
from IBM for OS/2 for $195 sorta pointless, right?

One of the reasons the whole null game didn't bother me as much early on
was in the theory behind some of the what/if work we wanted to do
financially. Originally, our concept model was built on on a combined
income statement and balance sheet in the reverse display order from which
you commonly see them. It is the income statement which drives the balance
sheet, if you think about it.

The most complex one we did for research purposes was a massive work which
took some 100 odd variables in the Multi-Media Distribution System rural TV
finanacial analysis world. It carried these some 100 variables through a
complete four year total display cycle. You could change any variable in
the mix from number of bucket trucks, to weather delays, to crew induced
productivity and market variables. But it also took in the cost of capital,
amount borrowed, and project cash short or long-fall. What it then did was
to do a circular rec-calculation regression on the entire model, auto-borrowing
more or less money, until it then optimized the capital injection required, while
feeding back the entire checkbook overdraft so that it minimized over-run cost
charges for each month in the burn, out through the whole four years.

The combined output for the print run for this particular client run consulting
work spawned a printed paper output matrix over six feet tall by over eight
feet long. It had a complete calculated cell for every single entity for every
single month, through the entire life of the project!

The question that was originally posed was, "Will you give us the whole family farm
as a shareholder for working captital? In four years under the following 100 odd
parameters, we'll give you back your entire roughly $900,000 capital investment,
and about 33% return on your money in those same four years. After that the sun
will shine on your children forever." Well, not exactly in those words, but close
enough for investor solicitation, or so the promotors claimed.

Truth was, that instead of requiring some $1,000,000, the actual cash needed peak,
even for the best case scenario, occured at just over $8,500,000 in the 26th
month of the burn. By the end of the 48 months, the project still hadn't broken
even at all. Subsequent actual project failures all over with these MMDS deals
proved up that we were pretty close with out forecast!

No, at two tons, that elephant doesn't jump very high folks!

In that geographic coordination of operating sites that could optimize operations
performance was also involved, this whole game of null for solution first came
home to me big time! Assuming, for example, that your trip-range for installer
teams gains or loses you effectiveness on how far they have to roam in a day's
work, both the number of them,, as well as where they may be stationed in the
geographic area are significant. You, as well, as everything else, can move the
'pins', in the computing map, and redo the whole sheet to see how much money you
make or lose over four years for doing this or that.

The solution (A solution!) appears when moving the pin, for a given site, causes
the expenses and the cost of the project to go up. The true solve for the problem -
is at a ZERO incremental cost.

My real indoctrination into this was when the final model would not "solve", because
for the entire model, there was no ZERO solution, at all .. mathematically for the
reasons we are discussing here. The silly program just kept going and going and
going .. long before there was ever and Eveready Bunny! Of course I don't control,
nor even knew, exactly what the functions really are in the spreadsheet first runs
at this! But when time came to hard-code the concept, and plan for converting
all of this into BASIC, I then DID have some control over the math tools which
were and had to become a part of the core process, and the data points which
would be stored for maybe years. It takes years for clients, in many cases, to
even begin to reach the point of understanding in which they can use the tools
that you provide to them.

I find it no accident for example, in studying the Mormon 'proof' of this or that
in South America, that the Priest's children are to have had wheeled toys to play with
while those pyramids were still being build by sliding blocks into place!! There
are just some things which we are not ready to behold until we are ready to learn
them. That is why parabalistic equations .. and learning .. are there!

So, for example, we still carry the actual geographic coordinates in all our
demographic layout for all 'name' entities! The reason being that as part of the
learning process, that if I can, for example, know the precise coordinates at which
money has arrived (or been spent) in relation to a facility, we have the necessary
vector geometry to which we can score an account as to how well we are either
located in relation to it .. or it to us! I call that Revenue Vector Analasis work!
We can attempt to 'cure' deficiencies we may discover about our accounts we learn
from this, by a number of ways, in practice, including, the actual, more than just
a best hunch, of giving certain 'poison pill' accounts to our dearly beloved
competitors, as may be desirable!

Of course skilled professionals have been giving bad seeds to someone else for a
long time in life, based on intuition. But that's another issue ll in itself ..
until you can prove that it has been done! Wide grin.

And yes, aircraft crash. Yes, all kinds of things happen when the answer floats
around but never is sure.

But even I was surprised to see it come down to just not being able to pay
off a client's account because in this particular unique incidence, zero was
so little away from zero!

I really don't wan't to write code for facility management for embedded use
with bedside manners, that needs an AS-400 and FORTRAN, just to make sure that
everything out of the feeding tube was used! And don't you laugh! Turns
out that one of the things the nursing home audit crew in the State here
checks, is just exactly how good your math is at making sure that every
CC of tube feed goes where it is supposed to go and .. is either in the
resident ... or charted as thrown away. And over the month they will take
away payment for that can or so .. and you, the home, wind up paying for
the residual can of what they comp you for, even though it *WAS* used!

All because you may have missed a CC or so in the claim you made to them
for what you were 'feeding', vs. what cans you were buying! I kid you not!

Yes, financial data IS involved. But so are a lot of math items which
figure into this which are not such data. They count too!

It's getting rough out there, feller!


------------------
Mike Luther
[email protected]



[This message has been edited by Mike Luther (edited November 13, 2001).]

[Criss French]

At least in PowerBasic there is a chance to get on top of the accuracy problems without having to actually solve the problems. Interval Arithmetic is the way to actually solve the problems, and is the right way to do the mission critical stuff.

Check out this link to Sun:
http://www.sun.com/forte/cplusplus/interval/index.html

If the crew at PowerBasic want to score bigtime on the 'serious market', add an interval type to PowerBasic. Based on ## to best use FPU. Could be very good.

I don't understand how programmers came along one (dawn of the PC) and managed to forget (most) everything the old slide rule boys knew. We now live mostly in the shadow-standards of thier youthfully exhuberant delinquency. Every calculation carries an accuracy hit. The way it used to be done on paper, people actually used to have to think, once for every calculation. You have to add up the inaccuracy as well as the answer, lest your answer become inaccuracy. I don't think that using a computer absolves us from the duty of thinking. It may allow us to think once, then get a free repeat * (very large number). Obviously, you better have thought very well, or you'll get (slop) * (very large number).

I may be anal. I'm the type that won't use a DEFINT or DEFanything. I fully type-qualify every variable in all of my code. I read all the manuals cover to cover. Even still, QB4 would give me different answers depending on whether I was compiled or interpreted (pull hair out here), and act as though #double was a !single. PI = 3.crap. Well, maybe not that bad, but I had some very bad times handling survey data. I had to be extremely careful not to do the math in the wrong order. But I know I was just fighting chaos (I actually blame Bill, I figure for $50billion, he's earned the blame ). I am very happy that in PowerBasic I get the ##, @, and @@ to work with. Even if I later store the finished results of a big chain of ## based calcs to a much smaller number of decimal places. I'm now so happy. So very very happy. And thanks to to PowerBasic manual author(s) for a whole chapter of detail on number types. After ten years of QB I feel a huge weight and darkness lifted.

Have fun!



------------------
What can go wrong will go wrong.
Anything can go wrong.
What hasn't?!?!

[David J Walker]

Sounds about right, Criss.
I still keep my old guessing stick within easy reach! Useful for checking up on these new-fangled gadgets.

------------------

[Michael Mattias]

I solved one problem like this (lots of significant digits) by saving "before the decimal" and "after the decimal" amounts separately, in long integers. That results in 18 significant digits without floating-point rounding.

Of course, you have to do your own "carry" and "borrow" if you need to do arthimetic.

MCM