Calculation error, caused by ??

[Herman Kieskamp]

I was writing some code and run in to the next error, could you
please confirm the following:
When running the next code, the program miscalculates the sequence:
-1.18935 + -039645 as .7928998 instead of .7929.

Please try the next code to confirm my outcome !

FUNCTION PBMAIN () AS LONG
LOCAL count AS SINGLE
LOCAL n AS LONG
LOCAL temp AS STRING
count=-1.982250
LOCAL scale AS LONG
DIM lstbxalc_list(20) AS LOCAL STRING
scale=8
' count=-1.5858
' count=-0.39645
PRINT STR$(-0.39645*3)
PRINT
PRINT STR$(-0.39645*3)+ " calc check 3 * -0.39645!"
PRINT
PRINT STR$(0.39645+0.39645)+ " calc 3 check 0.39645+0.39645!"
PRINT STR$(0.39645+0.39645+0.39645)+ " calc 3 check 0.39645+0.39645+0.39645!"

FOR N&=0 TO scale
count=count+0.39645
temp=STR$(count)
IF count=0 THEN temp="0"
IF count > -3 AND count < 1 THEN
PRINT temp + " calc check result" + STR$(count)
END IF
LSTBXALC_List(N&)=temp
' LISTBOX ADD hmaindialog&,%MAINDIALOG_lstbxalc,temp
NEXT N&
PRINT STR$(1.18935/3)+ " 1.18935/3"
PRINT STR$(0-0.39645-0.39645)+ " / 0-0.39645-0.39645"
PRINT STR$(0-0.39645-0.39645-0.39645)+ " / 0-0.39645-0.39645-0.39645"
END FUNCTION

When I run this code, I get the next results back:

-1.18935

-1.18935 calc check 3 * -0.39645!

.7929 calc 3 check 0.39645+0.39645!
1.18935 calc 3 check 0.39645+0.39645+0.39645!
-1.5858 calc check result-1.5858
-1.18935 calc check result-1.18935 (last good)
-.7928998 calc check result-.7928998 (Inbetween here the error occurs)
-.3964498 calc check result-.3964498
1.788139E-7 calc check result 1.788139E-7 (Here it should be zero)
.3964502 calc check result .3964502
.7929002 calc check result .7929002
.39645 1.18935/3
-.7929 / 0-0.39645-0.39645
-1.18935 / 0-0.39645-0.39645-0.39645

Do we have a another buggy Pentium, or is it something with PB, I got
this result first in pbdll, so it's the same in both compilers.
Further more, I have tested the executable on a Pentium PII-266 and
on a Pentium Celeron with the same results.
Can anyone confirm this ??!!

Herman Kieskamp


------------------

[Herman Kieskamp]

Sorry, I'm very often quiet chaotic. My first message is very confusing, so let met retry that.
I need a list that starts at -1.98225. I add 0.39645 to it in a sequence so I should get then next result:
-1.5858
-1.18935
-.7929
-.39645
0
.39645
.7929
1.18935
1.5858

But what I get is the next :
-1.5858
-1.18935 }What goes wrong here ?
-.7928998 }
-.3964498
1.788139
.3964502
.7929002
1.18935
1.5858

Here's the "clear" snippet, the one above was used by me for some control calcalations.

FUNCTION PBMAIN () AS LONG
LOCAL count AS SINGLE
LOCAL n AS LONG
LOCAL temp AS STRING
LOCAL scale AS LONG
count=-1.982250 'start of list
scale=8
FOR N&=0 TO scale
count=count+0.39645
temp=STR$(count)
PRINT temp + " calc check result" + STR$(count)
NEXT N&
END FUNCTION

Can anyone confirm this ?
The result is the same in pb-cc 2.0 and pb-dll 6.0.
It also showed the same result on 2 different computers,
a pII-266 and a Pentium Celeron 366.
I Hope this message is less confusing as the first one is.

Please confirm my findings, Am I crazy or what ?

Herman Kieskamp


------------------

[Semen Matusovski]

Herman --
Nothing strange. Floating-point numbers are stored in IEEE format.
During conversion to hexadecimal system last bit of mantissa is automatically rounded.
Calculations decreases accuracy again.

If you need finance calculations, use integer values (long, quad) and scale by yourself.
For scientific calculations, like rule, double supplies enough accuracy.






------------------
E-MAIL: [email protected]

[Tom Hanlin]

Rather than using SINGLE here, you might try more precise floating point
representations, such as DOUBLE or EXT.

------------------
Tom Hanlin
PowerBASIC Staff

[Lance Edmonds]

To add to Tom's suggestion, be sure to include a type specifier to "cast" hard coded floating point values to the precision you desire.

For example, if you are working with double-precision, then you would cast the numeric to double-precision with a trailing specifier:

DIM A AS DOUBLE
...
A# = A# + 0.39645#

------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>