Announcement

Collapse
No announcement yet.

Calculation error, caused by ??

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Calculation error, caused by ??

    I was writing some code and run in to the next error, could you
    please confirm the following:
    When running the next code, the program miscalculates the sequence:
    -1.18935 + -039645 as .7928998 instead of .7929.

    Please try the next code to confirm my outcome !

    FUNCTION PBMAIN () AS LONG
    LOCAL count AS SINGLE
    LOCAL n AS LONG
    LOCAL temp AS STRING
    count=-1.982250
    LOCAL scale AS LONG
    DIM lstbxalc_list(20) AS LOCAL STRING
    scale=8
    ' count=-1.5858
    ' count=-0.39645
    PRINT STR$(-0.39645*3)
    PRINT
    PRINT STR$(-0.39645*3)+ " calc check 3 * -0.39645!"
    PRINT
    PRINT STR$(0.39645+0.39645)+ " calc 3 check 0.39645+0.39645!"
    PRINT STR$(0.39645+0.39645+0.39645)+ " calc 3 check 0.39645+0.39645+0.39645!"

    FOR N&=0 TO scale
    count=count+0.39645
    temp=STR$(count)
    IF count=0 THEN temp="0"
    IF count > -3 AND count < 1 THEN
    PRINT temp + " calc check result" + STR$(count)
    END IF
    LSTBXALC_List(N&)=temp
    ' LISTBOX ADD hmaindialog&,%MAINDIALOG_lstbxalc,temp
    NEXT N&
    PRINT STR$(1.18935/3)+ " 1.18935/3"
    PRINT STR$(0-0.39645-0.39645)+ " / 0-0.39645-0.39645"
    PRINT STR$(0-0.39645-0.39645-0.39645)+ " / 0-0.39645-0.39645-0.39645"
    END FUNCTION

    When I run this code, I get the next results back:

    -1.18935

    -1.18935 calc check 3 * -0.39645!

    .7929 calc 3 check 0.39645+0.39645!
    1.18935 calc 3 check 0.39645+0.39645+0.39645!
    -1.5858 calc check result-1.5858
    -1.18935 calc check result-1.18935 (last good)
    -.7928998 calc check result-.7928998 (Inbetween here the error occurs)
    -.3964498 calc check result-.3964498
    1.788139E-7 calc check result 1.788139E-7 (Here it should be zero)
    .3964502 calc check result .3964502
    .7929002 calc check result .7929002
    .39645 1.18935/3
    -.7929 / 0-0.39645-0.39645
    -1.18935 / 0-0.39645-0.39645-0.39645

    Do we have a another buggy Pentium, or is it something with PB, I got
    this result first in pbdll, so it's the same in both compilers.
    Further more, I have tested the executable on a Pentium PII-266 and
    on a Pentium Celeron with the same results.
    Can anyone confirm this ??!!

    Herman Kieskamp


    ------------------
    You gotta run, and don't loop back.

  • #2
    Sorry, I'm very often quiet chaotic. My first message is very confusing, so let met retry that.
    I need a list that starts at -1.98225. I add 0.39645 to it in a sequence so I should get then next result:
    -1.5858
    -1.18935
    -.7929
    -.39645
    0
    .39645
    .7929
    1.18935
    1.5858

    But what I get is the next :
    -1.5858
    -1.18935 }What goes wrong here ?
    -.7928998 }
    -.3964498
    1.788139
    .3964502
    .7929002
    1.18935
    1.5858

    Here's the "clear" snippet, the one above was used by me for some control calcalations.

    FUNCTION PBMAIN () AS LONG
    LOCAL count AS SINGLE
    LOCAL n AS LONG
    LOCAL temp AS STRING
    LOCAL scale AS LONG
    count=-1.982250 'start of list
    scale=8
    FOR N&=0 TO scale
    count=count+0.39645
    temp=STR$(count)
    PRINT temp + " calc check result" + STR$(count)
    NEXT N&
    END FUNCTION

    Can anyone confirm this ?
    The result is the same in pb-cc 2.0 and pb-dll 6.0.
    It also showed the same result on 2 different computers,
    a pII-266 and a Pentium Celeron 366.
    I Hope this message is less confusing as the first one is.

    Please confirm my findings, Am I crazy or what ?

    Herman Kieskamp


    ------------------
    You gotta run, and don't loop back.

    Comment


    • #3
      Herman --
      Nothing strange. Floating-point numbers are stored in IEEE format.
      During conversion to hexadecimal system last bit of mantissa is automatically rounded.
      Calculations decreases accuracy again.

      If you need finance calculations, use integer values (long, quad) and scale by yourself.
      For scientific calculations, like rule, double supplies enough accuracy.






      ------------------
      E-MAIL: [email protected]

      Comment


      • #4
        Rather than using SINGLE here, you might try more precise floating point
        representations, such as DOUBLE or EXT.

        ------------------
        Tom Hanlin
        PowerBASIC Staff

        Comment


        • #5
          To add to Tom's suggestion, be sure to include a type specifier to "cast" hard coded floating point values to the precision you desire.

          For example, if you are working with double-precision, then you would cast the numeric to double-precision with a trailing specifier:
          Code:
          DIM A AS DOUBLE
          ...
          A# = A# + 0.39645#

          ------------------
          Lance
          PowerBASIC Support
          mailto:[email protected][email protected]</A>
          Lance
          mailto:[email protected]

          Comment

          Working...
          X