No announcement yet.

Some little sum will find you some day errors!

  • Filter
  • Time
  • Show
Clear All
new posts

  • Some little sum will find you some day errors!

    With appologies to the original poet .. who was talking about alchohol
    and liver problems ...

    Some little bug will find you some day.
    Some little bug will get behind you some day.
    With a nervous little quiver, the numbers won't deliver!
    Some little bug will find you some day.
    The below bug took down a site today and hasn't been seen until now in
    a long time of using the comparisons where double precision numbers
    are in use. It's a late Intel Pentium, not the one with the math bug
    in the chip. The compiler is set to FLOATING POINT EMULATION. So,
    just how does one ever become a real good behind the scenes bookkeeper
    or mission control programmer under the following bug I'll illustrate?

    Sure I can put in a conditional value that excludes number results
    so small as to be infinitesimal in EVERY calculation! But that
    would mean re-writing all the code for double precision numbers
    everywhere and that is a *LOT* of work... gloom.

    If I do that here in this instance to trap what we just found, and
    let it post, how do I keep the infintesimal residual from destroying
    a complete fully balanced accounting system or servo control operation
    for heavy equipment, for example?

    I offer for show and tell:

    10 ' JUNK.BAS failure example for PB 3.5 after a site crash 11-Nov-2001
    PRINT "Let's examine double precision for real world tasks? OK?
    ' Play like we read from a text data file on a disk or COMM stream.
    P1$ = "456.48" '                        Use string from disk file
    P2$ = "-30.00" '                        Use string from disk file
    P3$ = "-152.16" '                       Use string from disk file
    P4$ = "-84.38" '                        Use string from disk file
    P5$ = "-189.94" '                       Use string from disk file
    ' Compare the values to compute the AMOUNT vs SUM OF PAYOFF'ss
    QMT# = VAL(P1$) '                       We need enough to pay this amount
    PRINT " We need to pay "; QMT#; "w/pmts of 30.00/ 152.16/ 84.38/ 189.94
    QMV# = QMT# '                           Creat a step variable watch
    QMP1# = VAL(P2$) '                      Payment #1
    QMP2# = VAL(P3$) '                      Payment #2
    QMP3# = VAL(P4$) '                      Payment #3
    QMP4# = VAL(P5$) '                      Payment #4
    QMM# = QMM# + QMP1# '                   Add a payment to holding variable
    QMV# = QMV# + QMP1# '                   Decrement step variable
    PRINT " After step 1 "; QMV# '          Print step variable
    QMM# = QMM# + QMP2# '                   Add a payment to holding variable
    QMV# = QMV# + QMP2#'                    Decrement step variable
    PRINT " After step 2 ";QMV#'            Print step variable
    QMM# = QMM# + QMP3# '                   Add a payment to holding variable
    QMV# = QMV# + QMP3#'                    Decrement step variable
    PRINT " After step 3 ";QMV#'            Print step variable
    QMM# = QMM# + QMP4# '                   Add a payment to holding variable
    QMV# = QMV# + QMP4#'                    Decrement step variable
    PRINT " After step 4 ";QMV#'            Print step variable
    PRINT " Our debt was      "; QMT# '     Print debt
    PRINT " Our payments were "; QMM# '     Print payoff total
    ' We have now enough money to pay this invoice
    PRINT " Amount needed for payoff = "; QMT# + QMM#
    PRINT " So much for double entry bookeeping in PowerBASIC?"
    PRINT " Now let's do this in array work too!"
    ' Now dimension double precision arrays and do this in them
    DIM QAMT#(350) '                        Dimension double precision array
    DIM QAMP#(350) '                        Dimension double precision array
    DIM QAMM#(350) '                        Dimension double precision array
    QAMT#(1) = VAL(P1$) '                   We need enough to pay this amount
    QAMP#(1) = VAL(P2$) '                   Payment #1
    QAMP#(2) = VAL(P3$) '                   Payment #2
    QAMP#(3) = VAL(P4$) '                   Payment #3
    QAMP#(4) = VAL(P5$) '                   Payment #4
    QAMM#(1) = QAMM#(1) + QAMP#(1) '        Add a payment to holding variable
    QAMM#(1) = QAMM#(1) + QAMP#(2) '        Add a payment to holding variable
    QAMM#(1) = QAMM#(1) + QAMP#(3) '        Add a payment to holding variable
    QAMM#(1) = QAMM#(1) + QAMP#(4) '        Add a payment to holding variable
    PRINT " Our debt was      "; QAMT#(1)'  Print debt
    PRINT " Our payments were "; QAMM#(1)'  Print payoof total
    PRINT " Amount needed for payoff = "; QAMT#(1) + QAMM#(1)
    PRINT " So much for double entry bookeeping in PowerBASIC?"
    PRINT "   Just a thought .... "
    PRINT " If this were course corrections, and we fired that "
    PRINT "  booster rocket until we had 'er right on track ..
    PRINT "     so, grin, where did Mariner X go?
    PRINT " But just in case you think the answer is simple, <CR>:";
    ZZ0$ = INPUT$(1)
    PRINT " Maybe not!  Compare just the bottom increment!"
    P6$ = " 189.94" '                       Use string from disk file
    P7$ = "-189.94" '                       Use string from disk file
    QMT# = VAL(P6$) '                       Convert this to number variable
    QMN# = VAL(P7$) '                       Convert this to number variable
    PRINT " String to compare is: "; P6$ '  Show actual string
    PRINT " String accumulate is: "; P7$ '  Show actual string
    PRINT " Amount to compare is: "; QMT# ' Show number variable
    PRINT " Amount to compare is: "; QMN# ' Show number variable
    PRINT " Amount needed for payoff = "; QMT# + QMN# '     Print comparator
    Got a suggestion as to live with this deal. True, a bottle in front of
    me is better than a frontal lobotomy, but gee folks ...

    Mike Luther
    [email protected]
    Mike Luther
    [email protected]

  • #2
    A global search and replace of # with @ fixes the problem ..

    It changes the calculation to use BCD variables. You will always
    have small amounts using floating point calculations. BCD variables
    are excellent for use with money.

    There are numbers that can not be expressed exactly in Base 2 just as
    there are in Base 10 (1/3 for example).




    • #3
      Mike Burns is dead right here... your use of double precision variables is causing very small rounding errors due to the problem of accurately storing values that are not an exact power of two (see the documentation for more information on floating point rounding errors).

      In summary, floating point values can only accurately represent a finite number of real numbers - all other values are rounded to the closest value that can be represented.

      You can see some of the effects of this with slight changes to your code like so:
      PRINT " After step 1 "; STR$(QMV#,18) '          Print step variable
      PRINT " After step 2 "; STR$(QMV#,18) '          Print step variable
      The solution is to use the right data type for the job - in this case fixed-point BCD.

      PowerBASIC Support
      mailto:[email protected][email protected]</A>
      mailto:[email protected]


      • #4
        Suggestion taken and the financial side of this is fixed.

        However, that does *NOT* solve the non-financial side of the issue
        for equipment control as to null positioning needs. I'm going back
        and taking another look at that issue elsewhere, as well as critical
        other processes which do not use the financial side of life.

        It also has some other ramifications. I'm going to have to go back
        and think out the entire PB to C++ translation operation that is
        working now for a substantial portion of my discrete source. That
        added variable type and how to handle the translation for this into
        the C++ realm has to be thought about.

        This brings up another issue!

        How do we postulate interface of the old Microsoft file stored variables
        for their format for Double Precision in all this? Reason being is that
        I do use and have no trouble with the provided PowerBASIC functions
        which do this between it and the old stuff. But what if I want to
        bring out an old M/S Double Precision value from a 15 year old disk file
        and use it in BCD, then restore the file to what is needed for it?

        I really do have files and things which go back all this time quite
        happily doing its job for other than financial thing - - and those
        thingees as well!

        Advice for the data-lorn?

        Mike Luther
        [email protected]
        Mike Luther
        [email protected]


        • #5
          [Mount soapbox]
          The use of floating point computation in mission-critical
          applications is not permitted. Ever. Period.
          This is written in blood.
          Unfortunately, a lot of people have not realised this yet.
          [Fall off soapbox]



          • #6
            Thanks David!

            Well up 'til now, it ain't been critical .. but I can sure see
            your point!

            Again though .. How do we keep the Sins of the Father from being
            visited on the Sons?


            Assuming we have a file with Microsoft Binary data stored on it in
            double precision. We want to maintain that for compatibility and
            it is *NOT* material as to the null. Yet internal stuff that may
            be done in the operating portion of the code does depend on knowing
            whether to do something or not ... which can be still stored that
            way afteer all the logic is done and the tiny differences in the actual
            final file value really don't matter.

            What does PowerBasic do with something like [email protected] = QTY# or [email protected] = CVDM(R$),
            where R$ was a file field element that was and is still to be stored in the
            Microsoft format?

            What does it do if you go back the other way and convert back from the
            BCD format to the Microsoft format for storage and compatibility

            I think the whole issue will be reasonably moot for me and the conversion
            to BCD for the code portions involved will be a reasonably easy
            matter .. but not the final storage issue..

            Inquiring mind wants to know!

            By the way! One of the most interesting of these NULL deals in
            telecommunications engineering work I've ever heard about was the
            issue at Continental Electronics up in Richardson, Texas! They are
            one of only a handful of folks in the megawatt transmitter business,
            for such things as the Voice of America, Radio Free Europe, the
            multi-megawatt VLF stuff that communicates with submarines under
            water from 20 mile long antennas strung between hill tops over a
            valley .. the Distant Early Warning DEW line over-the-horizon radar
            systems and so on!

            In the HF multi-megawatt transmitters, my cousin described that they
            use a CAVITY tuned tank circuit in the 3Mhz to 30Mhz units! They
            can shift the frequency to any point in that band in some two
            seconds or so! It's done by taking a long hall-like whole room and
            making one whole wall on the end into a massive plunger like a
            piston in an engine! They cover the entire 20 plus feet with that
            moving wall driven by a hydraulic ram .. controlled by a computer
            driven servo system. Yep .. two seconds .. any place in the whole
            range to tune the tank circuit .. wham bam; thank you mam!

            Bobby noted that it took some real predictive algorithms to be able
            to begin slaming on the brakes for that wall as it moves into
            reasonance for the thing. It has to stop EXACTLY on the money at
            exactly the right instant. Hence they have to apply both predictive
            smoothing to the pressure waveform they create, at the same time
            they also have to be dead certain that when it does stop they know
            where ZERO is in the equation!

            I don't think you'd want to be in the way of the thing as it slams
            home up there near the fully plunged 30 Megahertz end!

            Mike Luther
            [email protected]
            Mike Luther
            [email protected]


            • #7
              This is all for money values, right?

              Any "rounding" effects that will occur when translating from Double to fixed-point BCD is likely to have no effect since the rounding errors prone to the double precision values will be at a much greater number of decimal places than the 2 places you will be using with BCD.

              For example, if we read the value 1.1000000000008754# (representing, say, $1.10), and you convert that to BCD ([email protected] = b#), then [email protected] will contain [email protected]

              IOW, the rounding error amounts are so low that they should be of no consequence at 2 decimal places.

              Likewise, you will introduce rounding errors when you translate back from BCD to double precision, but since the precision of the BCD is exact, the rounding effects will be significantly lower (almost non-existant).

              Of course, you could always use the ROUND() function in your data conversions code...

              PowerBASIC Support
              mailto:[email protected][email protected]</A>
              mailto:lanceedmo[email protected]


              • #8
                No Lance ..

                It's not just about money matters. (But of course money does! Grin!)

                I've used double precision for many things inside my work where the
                single presicion number system we started with in old M/S basic was
                too small to handle the load. If I'd started into this game with
                the wonder that is PowerBASIC, I'm sure my perspective would be all
                different about these things!

                I try not to mind admitting my mistakes, lack of knowlege or understanding
                while noting that crow really doesn't taste very good ... chuckle.

                I actually looked at the ROUND function for the first time when you
                called my attention to it!! Long ago and very far away, the little Heathkit
                monthly magazine was being printed for us and I had my first computer -- a
                Heathkit H89 serial #659 which I still own and still works.

                Some kind soul solved the rounding problem with number juggling that I
                wrote down from the contributor's stuff. It became what is now
                the SUB PK50060 (QZ#) in my own libraries, over time .. into the world
                of double precision. Changing the cloth is a hard habit to break.

                Ask any nun. Strictly in humor, I assure you and no offense ever
                intended toward the wonderful work they do... or any other charitable
                contributor, for that matter ...

                Mike Luther
                [email protected]
                Mike Luther
                [email protected]


                • #9
                  Floating point can really nail you in iterative/recursive situations
                  where the error can rapidly overcome the product. Aerodynamics,
                  matrix calculations used in stress work, etc. Bridges fall down,
                  aircraft crash, etc., etc. If you need REAL accuracy, you need
                  (dare I say it!) FORTRAN. There are some very accurate (and expensive)
                  mathematical libraries available. Of course! Power Fortran!
                  And totally off topic, with all the cheap Sun Sparc kit sloshing
                  about at the moment, how about PB Linux for Sparc? 4 64-bit CPUs
                  with 4MB caches, wow!



                  • #10
                    Yes, David ..

                    I'm learning .. I'm learning!

                    However, given the circumstances, I wonder if a surplus AS-400 wouldn't
                    be a better choice than the Sun deal.

                    Here in our area, you used to see this curious bumper stickers on
                    this or that wreck .. "My other car is a 300SL.."

                    I guess it would be fun to use a tag line..

                    "My other computer is an AS-400.."

                    But even if you got one, how would you pay for the Op system site
                    rights? Kinda makes the whole arguement pointless about paying M/S
                    for a $300 upgrade for XP, or a two year subscription to ACP/MCP
                    from IBM for OS/2 for $195 sorta pointless, right?

                    One of the reasons the whole null game didn't bother me as much early on
                    was in the theory behind some of the what/if work we wanted to do
                    financially. Originally, our concept model was built on on a combined
                    income statement and balance sheet in the reverse display order from which
                    you commonly see them. It is the income statement which drives the balance
                    sheet, if you think about it.

                    The most complex one we did for research purposes was a massive work which
                    took some 100 odd variables in the Multi-Media Distribution System rural TV
                    finanacial analysis world. It carried these some 100 variables through a
                    complete four year total display cycle. You could change any variable in
                    the mix from number of bucket trucks, to weather delays, to crew induced
                    productivity and market variables. But it also took in the cost of capital,
                    amount borrowed, and project cash short or long-fall. What it then did was
                    to do a circular rec-calculation regression on the entire model, auto-borrowing
                    more or less money, until it then optimized the capital injection required, while
                    feeding back the entire checkbook overdraft so that it minimized over-run cost
                    charges for each month in the burn, out through the whole four years.

                    The combined output for the print run for this particular client run consulting
                    work spawned a printed paper output matrix over six feet tall by over eight
                    feet long. It had a complete calculated cell for every single entity for every
                    single month, through the entire life of the project!

                    The question that was originally posed was, "Will you give us the whole family farm
                    as a shareholder for working captital? In four years under the following 100 odd
                    parameters, we'll give you back your entire roughly $900,000 capital investment,
                    and about 33% return on your money in those same four years. After that the sun
                    will shine on your children forever." Well, not exactly in those words, but close
                    enough for investor solicitation, or so the promotors claimed.

                    Truth was, that instead of requiring some $1,000,000, the actual cash needed peak,
                    even for the best case scenario, occured at just over $8,500,000 in the 26th
                    month of the burn. By the end of the 48 months, the project still hadn't broken
                    even at all. Subsequent actual project failures all over with these MMDS deals
                    proved up that we were pretty close with out forecast!

                    No, at two tons, that elephant doesn't jump very high folks!

                    In that geographic coordination of operating sites that could optimize operations
                    performance was also involved, this whole game of null for solution first came
                    home to me big time! Assuming, for example, that your trip-range for installer
                    teams gains or loses you effectiveness on how far they have to roam in a day's
                    work, both the number of them,, as well as where they may be stationed in the
                    geographic area are significant. You, as well, as everything else, can move the
                    'pins', in the computing map, and redo the whole sheet to see how much money you
                    make or lose over four years for doing this or that.

                    The solution (A solution!) appears when moving the pin, for a given site, causes
                    the expenses and the cost of the project to go up. The true solve for the problem -
                    is at a ZERO incremental cost.

                    My real indoctrination into this was when the final model would not "solve", because
                    for the entire model, there was no ZERO solution, at all .. mathematically for the
                    reasons we are discussing here. The silly program just kept going and going and
                    going .. long before there was ever and Eveready Bunny! Of course I don't control,
                    nor even knew, exactly what the functions really are in the spreadsheet first runs
                    at this! But when time came to hard-code the concept, and plan for converting
                    all of this into BASIC, I then DID have some control over the math tools which
                    were and had to become a part of the core process, and the data points which
                    would be stored for maybe years. It takes years for clients, in many cases, to
                    even begin to reach the point of understanding in which they can use the tools
                    that you provide to them.

                    I find it no accident for example, in studying the Mormon 'proof' of this or that
                    in South America, that the Priest's children are to have had wheeled toys to play with
                    while those pyramids were still being build by sliding blocks into place!! There
                    are just some things which we are not ready to behold until we are ready to learn
                    them. That is why parabalistic equations .. and learning .. are there!

                    So, for example, we still carry the actual geographic coordinates in all our
                    demographic layout for all 'name' entities! The reason being that as part of the
                    learning process, that if I can, for example, know the precise coordinates at which
                    money has arrived (or been spent) in relation to a facility, we have the necessary
                    vector geometry to which we can score an account as to how well we are either
                    located in relation to it .. or it to us! I call that Revenue Vector Analasis work!
                    We can attempt to 'cure' deficiencies we may discover about our accounts we learn
                    from this, by a number of ways, in practice, including, the actual, more than just
                    a best hunch, of giving certain 'poison pill' accounts to our dearly beloved
                    competitors, as may be desirable!

                    Of course skilled professionals have been giving bad seeds to someone else for a
                    long time in life, based on intuition. But that's another issue ll in itself ..
                    until you can prove that it has been done! Wide grin.

                    And yes, aircraft crash. Yes, all kinds of things happen when the answer floats
                    around but never is sure.

                    But even I was surprised to see it come down to just not being able to pay
                    off a client's account because in this particular unique incidence, zero was
                    so little away from zero!

                    I really don't wan't to write code for facility management for embedded use
                    with bedside manners, that needs an AS-400 and FORTRAN, just to make sure that
                    everything out of the feeding tube was used! And don't you laugh! Turns
                    out that one of the things the nursing home audit crew in the State here
                    checks, is just exactly how good your math is at making sure that every
                    CC of tube feed goes where it is supposed to go and .. is either in the
                    resident ... or charted as thrown away. And over the month they will take
                    away payment for that can or so .. and you, the home, wind up paying for
                    the residual can of what they comp you for, even though it *WAS* used!

                    All because you may have missed a CC or so in the claim you made to them
                    for what you were 'feeding', vs. what cans you were buying! I kid you not!

                    Yes, financial data IS involved. But so are a lot of math items which
                    figure into this which are not such data. They count too!

                    It's getting rough out there, feller!

                    Mike Luther
                    [email protected]

                    [This message has been edited by Mike Luther (edited November 13, 2001).]
                    Mike Luther
                    [email protected]


                    • #11
                      At least in PowerBasic there is a chance to get on top of the accuracy problems without having to actually solve the problems. Interval Arithmetic is the way to actually solve the problems, and is the right way to do the mission critical stuff.

                      Check out this link to Sun:

                      If the crew at PowerBasic want to score bigtime on the 'serious market', add an interval type to PowerBasic. Based on ## to best use FPU. Could be very good.

                      I don't understand how programmers came along one (dawn of the PC) and managed to forget (most) everything the old slide rule boys knew. We now live mostly in the shadow-standards of thier youthfully exhuberant delinquency. Every calculation carries an accuracy hit. The way it used to be done on paper, people actually used to have to think, once for every calculation. You have to add up the inaccuracy as well as the answer, lest your answer become inaccuracy. I don't think that using a computer absolves us from the duty of thinking. It may allow us to think once, then get a free repeat * (very large number). Obviously, you better have thought very well, or you'll get (slop) * (very large number).

                      I may be anal. I'm the type that won't use a DEFINT or DEFanything. I fully type-qualify every variable in all of my code. I read all the manuals cover to cover. Even still, QB4 would give me different answers depending on whether I was compiled or interpreted (pull hair out here), and act as though #double was a !single. PI = 3.crap. Well, maybe not that bad, but I had some very bad times handling survey data. I had to be extremely careful not to do the math in the wrong order. But I know I was just fighting chaos (I actually blame Bill, I figure for $50billion, he's earned the blame ). I am very happy that in PowerBasic I get the ##, @, and @@ to work with. Even if I later store the finished results of a big chain of ## based calcs to a much smaller number of decimal places. I'm now so happy. So very very happy. And thanks to to PowerBasic manual author(s) for a whole chapter of detail on number types. After ten years of QB I feel a huge weight and darkness lifted.

                      Have fun!

                      What can go wrong will go wrong.
                      Anything can go wrong.
                      What hasn't?!?!
                      What can go wrong will go wrong.
                      Anything can go wrong.
                      What hasn't?!?!


                      • #12
                        Sounds about right, Criss.
                        I still keep my old guessing stick within easy reach! Useful for checking up on these new-fangled gadgets.



                        • #13
                          I solved one problem like this (lots of significant digits) by saving "before the decimal" and "after the decimal" amounts separately, in long integers. That results in 18 significant digits without floating-point rounding.

                          Of course, you have to do your own "carry" and "borrow" if you need to do arthimetic.

                          Michael Mattias
                          Tal Systems Inc. (retired)
                          Racine WI USA
                          [email protected]