Tweet
Originally posted by Steve Rossell
View Post
In the meantime, however, the following code illustrates a simple method for obtaining higher precision (18 decimal digits) for your floating point numeric literals. It involves using the VAL function to convert your literals to extended precision instead of casting them to extended precision; i.e, using VAL("literal") instead of literal##.
In using this method, note the following:
(1) If you have a floating point literal which has more than 18 significant digits, round the literal to 18 digits before using it in the string argument of VAL. If you don’t, VAL will truncate any digits beyond the 18 digits (instead of rounding to 18 digits), resulting in less precision.
(2) You do not have to add zeroes to a floating point literal with fewer than 18 significant digits. (In other words -- with VAL -- using 0.1 will give you the same result as 0.100000000000000000.
(3) If calculation time matters, do not use the VAL function within a loop with many iterations. It does have considerable overhead. Instead, assign the value of VAL(“literal”) to an extended precision variable and use the variable within the loop.
(4) If you have any doubt about the value of your literal as obtained from VAL(“literal”), check it out with STR$(VAL(“literal”), 18).
(5) This procedure has not been fully tested (or vetted), so please use good judgment in its use.
Any constructive comments relative to this post will be greatly appreciated.
--WH
Code:
#COMPILE EXE #DIM ALL FUNCTION PBMAIN() AS LONG DIM z AS LOCAL EXT, y AS LOCAL EXT, counter AS LOCAL LONG, timecheck AS LOCAL DOUBLE PRINT "Assigning 0.1 to z with 16 digits of precision" z = 0.1## PRINT:PRINT "z = 0.1## = ";STR$(z, 18) PRINT:PRINT:PRINT "Assigning 0.1 to z with 18 digits of precision" z = VAL("0.1") PRINT:PRINT "z = VAL(""0.1"") = ";STR$(z, 18) PRINT:PRINT:PRINT "Calculation of 0.1 - 1/10. (Exact result = 0)" z = 0.1## - 1/10 PRINT:PRINT "z = 0.1## - 1/10 = "; STR$(z,18) PRINT:PRINT:PRINT "Calculation of 0.1 - 1/10. (Exact result = 0)" z = VAL("0.1") - 1/10 PRINT:PRINT "z = VAL(""0.1"") - 1/10 = "; STR$(z,18) PRINT:PRINT:PRINT "Iteration loop (1 million iterations) using VAL(""0.1"") within the loop" timecheck = TIMER FOR counter = 1 TO 1000000 z = VAL("0.1") - 1/10 NEXT counter PRINT "Result: "; STR$(z, 18) PRINT "Elapsed time (seconds): ";ROUND(TIMER - timecheck, 4) PRINT:PRINT:PRINT "Iteration loop (1 million iterations) using EXT y (equal to VAL(""0.1"")) within the loop" timecheck = TIMER y = VAL("0.1") FOR counter = 1 TO 1000000 z = y - 1/10 NEXT counter PRINT "Result: "; STR$(z, 18) PRINT "Elapsed time (seconds): ";ROUND(TIMER - timecheck, 4) WAITKEY$ END FUNCTION
Comment