Announcement

**Paul Purvis** · 1 Jul 2008, 02:48 AM

sorry one better
this might show a way to avoid using the VAL possibly for input as literal and the first code may be a way to better assign a EXT from a string input
but does not solve rounding
if input is a string, maybe best to split string into two integer values and divide by number of decimal places used in input string

fire at will

Code:

'spit input into j and i
'the string is "16.585"  then split string into integers

j=16585
i=3    
if i>0 then
n=j/10^i 
else
n=j
end if

code to show using literals

Code:

#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG
    LOCAL j AS LONG
    LOCAL m AS EXT
    LOCAL n AS EXT
    j=16585        'ok
    n=j/1000       'ok

    'above delivers 16.585 into EXT variable n
    

    
    m=16.585       'bad input
   'm=16.585##   'bad input as well
    PRINT m
    PRINT STR$(m,18)
    PRINT n
    PRINT STR$(n,18)
    IF m=n THEN PRINT "match"
    IF m<>n THEN PRINT "no match"
  
    m=VAL("16.585")  'ok probably and should be slower than other ways
    PRINT m
    PRINT STR$(m,18)
    PRINT n
    PRINT STR$(n,18)
    IF m=n THEN PRINT "match"
    IF m<>n THEN PRINT "no match"

    m=16585##/1000##   'ok
    PRINT m
    PRINT STR$(m,18)
    PRINT n
    PRINT STR$(n,18)
    IF m=n THEN PRINT "match"
    IF m<>n THEN PRINT "no match"
     
    m=16585/1000   'ok
    PRINT m
    PRINT STR$(m,18)
    PRINT n
    PRINT STR$(n,18)
    IF m=n THEN PRINT "match"
    IF m<>n THEN PRINT "no match"


    m=16585   'ok
   m=m/1000  'ok
    PRINT m
    PRINT STR$(m,18)
    PRINT n
    PRINT STR$(n,18)
    IF m=n THEN PRINT "match"
    IF m<>n THEN PRINT "no match"

      WAITKEY$
END FUNCTION

**Walter Henn** · 1 Jul 2008, 04:56 AM

Originally posted by paul d purvis View Post

Code:

 (1)
'spit input into j and i
'the string is "16.585" then split string into integers
 
j=16585
i=3 
if i>0 then
n=j/10^i 
else
n=j
end if

Code:

 (2)
...
m=VAL("16.585") 'ok probably and should be slower than other ways
PRINT m
PRINT STR$(m,18)
PRINT n
PRINT STR$(n,18)
IF m=n THEN PRINT "match"
IF m<>n THEN PRINT "no match"
 
m=16585##/1000## 'ok
PRINT m
PRINT STR$(m,18)
PRINT n
PRINT STR$(n,18)
IF m=n THEN PRINT "match"
IF m<>n THEN PRINT "no match"
...

You might be interested in knowing that VAL ("16.585") in code snippet (2) above stores exactly the same value in m## as 16585/1000 (or 16585##/1000##). However, VAL is a function and takes approx. four times as long.

As far as code snippet (1) above for inputting EXT values, the statement, INPUT m (where m is an EXT variable), produces the same result as using VAL("value of m"). There is no loss in precision (e.g., EXT to DOUBLE) when you input values to EXT variables. Only EXT literals within the program lose precision because the maximum precision for numeric literals is DOUBLE. If,for some reason, you're using string input, use VAL(string variable) to assign to an EXT variable. Obviously, the speed penalty for converting input strings to EXT by using VAL is not a consideration.

**John Gleason** · 1 Jul 2008, 10:03 AM

Paul Purvis said: ...16585/1000

Interesting technique Paul, and like Walter's, my test loops I showed it to produce exactly the same value and be over EIGHT times faster on my machine. That's an efficient workaround.

As far as the rounding differences observed in the various routines, this is what I think is happening:
*Paul Purvis is looking for a round up on 5 based on "visual rounding" and this is what I solely based my function roundUp and roundUpV2 on, as specified in the IEEC post.
*Walter is looking for a round function that rounds up on 5, but only if the base2 representation of the "rounding 5" actually is equal to or exceeds 5 internally. Let's call this, "base2 round up on 5", and is what Paul Dixon's code has been based on I believe. Maybe we should make an IEEC spec for this too!
*roundUp failed, as Walter pointed out, because it rounded up .46 - .499... tails, and eventually roundUpV2 replaced it. So far, roundUpV2 has shown perfect results in all tests to 18 digits to perform "visual round up on 5".
*Interest in a "visual round down on 5" has been expressed (a "base2 round down on 5" might be possible too).

Probably all can be done, but are best done one at a time to avoid confusion.

So, 1) Paul P., how does roundUpV2 look to you? Do you see any problems or performance issues? It matches your (what I called) roundMe3 string version thru 15 or 16 digits, and roundMe4 with the noted exceptions so far in testing.
2) Walter, how are asmRoundUp/round_dixon looking? Any problems or exceptions there?

**Paul Purvis** · 1 Jul 2008, 10:41 AM

hi John,

i did not see anybody express the literals as i did recently with the integer assignments then division or simpler, an assignment with an integer being divided by the number of decimals needed.
i want to test later today those assignments against that database i produced. it is going to be a long day on other things, but i will test tonight.
i ran a test on a small sampling to spot check the integer/decimals assignment and everything seemed ok. so as it stands to continue to be ok, any program rounding on 5 up is going to get my support.

on a additional thought from what i have seen recently
a word of caution might be worthy when adding .5 or .05 or .005
it might be wise to assign a variable then use the variable in a expression or inside the expression use (5/10), (5/100), or (5/1000)
this might give a more precise number for rounding, but i have not tested it, but my gut feels right about it.

and on a much bigger plane(i am going to place myself in the firing lane on this one)
i would suggest anytime you have a expression that involves a decimal literal, you use the integer/decimal structure instead of decimal literal.

**John Gleason** · 1 Jul 2008, 12:54 PM

Originally posted by paul d purvis View Post

i did not see anybody express the literals as i did recently with the integer assignments then division or simpler, an assignment with an integer being divided by the number of decimals needed.

I didn't either so thought I'd mention it; good thing you came up with it and thanks for mentioning that I did not.

j/k. Any time you have code that is 8x faster (and correct hehheh), feel free to post a snippet.

Originally posted by paul d purvis View Post

and on a much bigger plane(i am going to place myself in the firing lane on this one)
i would suggest anytime you have a expression that involves a decimal literal, you use the integer/decimal structure instead of decimal literal.

I plan on taking that advice assuming the tests work out, which I'm pretty sure they will. I'll test a bunch against VAL("string representation of #")

**John Gleason** · 1 Jul 2008, 03:24 PM

Here's a program that creates a .bas which tests over 6,400 literals at a time. You can change the STEP value to most anything to vary the test, but probably prime numbers are best.

Code:

#COMPILE EXE
#DIM ALL
MACRO ifErr = IF xs <> xs2 THEN ? "error" & STR$(xs,18) & STR$(xs2,18): WAITKEY$

FUNCTION PBMAIN () AS LONG

    LOCAL ii AS EXT, xs, xs2 AS STRING
    LOCAL t AS DOUBLE
    OPEN "c:\pbLiteralTest.bas" FOR OUTPUT AS #1
    PRINT #1, "#COMPILE EXE"
    PRINT #1, "#DIM ALL"
    PRINT #1, "MACRO ifErr = IF xs <> xs2 THEN ? ""error"" & STR$(xs,18) & STR$(xs2,18): WAITKEY$"
    PRINT #1, " "
    PRINT #1, "FUNCTION PBMAIN () AS LONG"
    PRINT #1, "LOCAL xs, xs2 AS EXT"
    FOR ii = 1 TO 11000000000 STEP 1712467
          xs = "xs = val(""" & FORMAT$(ii / 1000000000, "0.000000000") & """)"
          xs2 = "xs2 = " & FORMAT$(ii) & " / 1000000000"
          PRINT #1, xs & ":" & xs2 & ":" & "ifErr"
    NEXT
    PRINT #1, "? ""done"" : WAITKEY$"
    PRINT #1, "end function"
    EXIT FUNCTION

END FUNCTION

Here is a little sample of what it creates for the test:

Code:

#COMPILE EXE
#DIM ALL
MACRO ifErr = IF xs <> xs2 THEN ? "error" & STR$(xs,18) & STR$(xs2,18): WAITKEY$
 
FUNCTION PBMAIN () AS LONG
LOCAL xs, xs2 AS EXT
xs = VAL("0.000000001"):xs2 = 1 / 1000000000:ifErr
xs = VAL("0.001712468"):xs2 = 1712468 / 1000000000:ifErr
xs = VAL("0.003424935"):xs2 = 3424935 / 1000000000:ifErr
xs = VAL("0.005137402"):xs2 = 5137402 / 1000000000:ifErr
xs = VAL("0.006849869"):xs2 = 6849869 / 1000000000:ifErr
xs = VAL("0.008562336"):xs2 = 8562336 / 1000000000:ifErr
xs = VAL("0.010274803"):xs2 = 10274803 / 1000000000:ifErr
xs = VAL("0.011987270"):xs2 = 11987270 / 1000000000:ifErr
xs = VAL("0.013699737"):xs2 = 13699737 / 1000000000:ifErr
xs = VAL("0.015412204"):xs2 = 15412204 / 1000000000:ifErr
xs = VAL("0.017124671"):xs2 = 17124671 / 1000000000:ifErr
xs = VAL("0.018837138"):xs2 = 18837138 / 1000000000:ifErr
xs = VAL("0.020549605"):xs2 = 20549605 / 1000000000:ifErr
xs = VAL("0.022262072"):xs2 = 22262072 / 1000000000:ifErr
xs = VAL("0.023974539"):xs2 = 23974539 / 1000000000:ifErr
xs = VAL("0.025687006"):xs2 = 25687006 / 1000000000:ifErr
xs = VAL("0.027399473"):xs2 = 27399473 / 1000000000:ifErr
xs = VAL("0.029111940"):xs2 = 29111940 / 1000000000:ifErr
xs = VAL("0.030824407"):xs2 = 30824407 / 1000000000:ifErr
xs = VAL("0.032536874"):xs2 = 32536874 / 1000000000:ifErr
xs = VAL("0.034249341"):xs2 = 34249341 / 1000000000:ifErr
xs = VAL("0.035961808"):xs2 = 35961808 / 1000000000:ifErr
xs = VAL("0.037674275"):xs2 = 37674275 / 1000000000:ifErr
xs = VAL("0.039386742"):xs2 = 39386742 / 1000000000:ifErr
xs = VAL("0.041099209"):xs2 = 41099209 / 1000000000:ifErr
xs = VAL("0.042811676"):xs2 = 42811676 / 1000000000:ifErr
xs = VAL("0.044524143"):xs2 = 44524143 / 1000000000:ifErr
xs = VAL("0.046236610"):xs2 = 46236610 / 1000000000:ifErr
xs = VAL("0.047949077"):xs2 = 47949077 / 1000000000:ifErr
? "done" : WAITKEY$
END FUNCTION

**Paul Purvis** · 1 Jul 2008, 03:57 PM

unless i read wrong on the internet,
the assigning of literals and rounding problems has plagued other compilers as well.
i will probably standardize on the decimal assignments with the integer/decimals on all other programs that i use and will use in the future, it is a reasonable workaround although i tried integer*decimals(.1,.01,.001,etc) to keep from getting a division by zero error, and i was also believed that multiplication is faster than division.

a few years ago, i had a very similar problem using cbasic86 to match values with decimals and i could not place my finger on the problem, it happened rarely, and i did not have a resource such as this forum to help me with my problems, but now i can see where my problem with matching of values where probably the same problems that where faced in this thread.

**David Roberts** · 1 Jul 2008, 04:39 PM

John

Seven significant figures: 1712467?

Try stressing the idea a bit and consider 15 significant figures. Not so rosy now I'm afraid.

**Paul Purvis** · 1 Jul 2008, 07:01 PM

David
do you have some specific code to share

**John Gleason** · 1 Jul 2008, 07:11 PM

I'll check that Dave, tho the sad face is looking ominous, portending doom...

**John Gleason** · 1 Jul 2008, 07:27 PM

Hey, good news, 6500 of these below worked. I'll test some more, but looking good so far.

Code:

#COMPILE EXE
#DIM ALL
MACRO ifErr = IF xs <> xs2 THEN ? "error" & STR$(xs,18) & STR$(xs2,18): WAITKEY$
 
FUNCTION PBMAIN () AS LONG
LOCAL xs, xs2 AS EXT
xs = VAL("10.0000000000000000"):xs2 = 100000000000000000 / 10000000000000000:ifErr
xs = VAL("10.0000017124673981"):xs2 = 100000017124673981 / 10000000000000000:ifErr
xs = VAL("10.0000034249347962"):xs2 = 100000034249347962 / 10000000000000000:ifErr
xs = VAL("10.0000051374021943"):xs2 = 100000051374021943 / 10000000000000000:ifErr
xs = VAL("10.0000068498695924"):xs2 = 100000068498695924 / 10000000000000000:ifErr
xs = VAL("10.0000085623369905"):xs2 = 100000085623369905 / 10000000000000000:ifErr
xs = VAL("10.0000102748043886"):xs2 = 100000102748043886 / 10000000000000000:ifErr
xs = VAL("10.0000119872717867"):xs2 = 100000119872717867 / 10000000000000000:ifErr
xs = VAL("10.0000136997391848"):xs2 = 100000136997391848 / 10000000000000000:ifErr
xs = VAL("10.0000154122065829"):xs2 = 100000154122065829 / 10000000000000000:ifErr
xs = VAL("10.0000171246739810"):xs2 = 100000171246739810 / 10000000000000000:ifErr
xs = VAL("10.0000188371413791"):xs2 = 100000188371413791 / 10000000000000000:ifErr
xs = VAL("10.0000205496087772"):xs2 = 100000205496087772 / 10000000000000000:ifErr
xs = VAL("10.0000222620761753"):xs2 = 100000222620761753 / 10000000000000000:ifErr
xs = VAL("10.0000239745435734"):xs2 = 100000239745435734 / 10000000000000000:ifErr
xs = VAL("10.0000256870109715"):xs2 = 100000256870109715 / 10000000000000000:ifErr
xs = VAL("10.0000273994783696"):xs2 = 100000273994783696 / 10000000000000000:ifErr
xs = VAL("10.0000291119457677"):xs2 = 100000291119457677 / 10000000000000000:ifErr
xs = VAL("10.0000308244131658"):xs2 = 100000308244131658 / 10000000000000000:ifErr
xs = VAL("10.0000325368805639"):xs2 = 100000325368805639 / 10000000000000000:ifErr
xs = VAL("10.0000342493479620"):xs2 = 100000342493479620 / 10000000000000000:ifErr
xs = VAL("10.0000359618153601"):xs2 = 100000359618153601 / 10000000000000000:ifErr
xs = VAL("10.0000376742827582"):xs2 = 100000376742827582 / 10000000000000000:ifErr
xs = VAL("10.0000393867501563"):xs2 = 100000393867501563 / 10000000000000000:ifErr
xs = VAL("10.0000410992175544"):xs2 = 100000410992175544 / 10000000000000000:ifErr
xs = VAL("10.0000428116849525"):xs2 = 100000428116849525 / 10000000000000000:ifErr
xs = VAL("10.0000445241523506"):xs2 = 100000445241523506 / 10000000000000000:ifErr
xs = VAL("10.0000462366197487"):xs2 = 100000462366197487 / 10000000000000000:ifErr
xs = VAL("10.0000479490871468"):xs2 = 100000479490871468 / 10000000000000000:ifErr
xs = VAL("10.0000496615545449"):xs2 = 100000496615545449 / 10000000000000000:ifErr
xs = VAL("10.0000513740219430"):xs2 = 100000513740219430 / 10000000000000000:ifErr
xs = VAL("10.0000530864893411"):xs2 = 100000530864893411 / 10000000000000000:ifErr
xs = VAL("10.0000547989567392"):xs2 = 100000547989567392 / 10000000000000000:ifErr
xs = VAL("10.0000565114241373"):xs2 = 100000565114241373 / 10000000000000000:ifErr
xs = VAL("10.0000582238915354"):xs2 = 100000582238915354 / 10000000000000000:ifErr
xs = VAL("10.0000599363589335"):xs2 = 100000599363589335 / 10000000000000000:ifErr
xs = VAL("10.0000616488263316"):xs2 = 100000616488263316 / 10000000000000000:ifErr
xs = VAL("10.0000633612937297"):xs2 = 100000633612937297 / 10000000000000000:ifErr
xs = VAL("10.0000650737611278"):xs2 = 100000650737611278 / 10000000000000000:ifErr
xs = VAL("10.0000667862285259"):xs2 = 100000667862285259 / 10000000000000000:ifErr
xs = VAL("10.0000684986959240"):xs2 = 100000684986959240 / 10000000000000000:ifErr
xs = VAL("10.0000702111633221"):xs2 = 100000702111633221 / 10000000000000000:ifErr
xs = VAL("10.0000719236307202"):xs2 = 100000719236307202 / 10000000000000000:ifErr
xs = VAL("10.0000736360981183"):xs2 = 100000736360981183 / 10000000000000000:ifErr
xs = VAL("10.0000753485655164"):xs2 = 100000753485655164 / 10000000000000000:ifErr
xs = VAL("10.0000770610329145"):xs2 = 100000770610329145 / 10000000000000000:ifErr
xs = VAL("10.0000787735003126"):xs2 = 100000787735003126 / 10000000000000000:ifErr
xs = VAL("10.0000804859677107"):xs2 = 100000804859677107 / 10000000000000000:ifErr
xs = VAL("10.0000821984351088"):xs2 = 100000821984351088 / 10000000000000000:ifErr
xs = VAL("10.0000839109025069"):xs2 = 100000839109025069 / 10000000000000000:ifErr
xs = VAL("10.0000856233699050"):xs2 = 100000856233699050 / 10000000000000000:ifErr
xs = VAL("10.0000873358373031"):xs2 = 100000873358373031 / 10000000000000000:ifErr
xs = VAL("10.0000890483047012"):xs2 = 100000890483047012 / 10000000000000000:ifErr
xs = VAL("10.0000907607720993"):xs2 = 100000907607720993 / 10000000000000000:ifErr
xs = VAL("10.0000924732394974"):xs2 = 100000924732394974 / 10000000000000000:ifErr
xs = VAL("10.0000941857068955"):xs2 = 100000941857068955 / 10000000000000000:ifErr
xs = VAL("10.0000958981742936"):xs2 = 100000958981742936 / 10000000000000000:ifErr
xs = VAL("10.0000976106416917"):xs2 = 100000976106416917 / 10000000000000000:ifErr
xs = VAL("10.0000993231090898"):xs2 = 100000993231090898 / 10000000000000000:ifErr
xs = VAL("10.0001010355764879"):xs2 = 100001010355764879 / 10000000000000000:ifErr

**Paul Purvis** · 1 Jul 2008, 11:49 PM

a poor test was removed

**David Roberts** · 2 Jul 2008, 05:40 AM

The fraction method vs VAL is OK. I made an error in the formatting.

I should add that there is a danger of gettng preoccupied with decimals.

If two binary representations are the same then their decimal counterparts will be the same. However, it does not follow that if two decimals are the same then their binary representations whence they came are the same. In other words it is possible for two binary representations to map into the same decimal space. We have then a collision in the same sense as in Hashing algorithms.

What I've been looking at is when a decimal counterpart is rounded up causing a cascade to the chosen round off point which in turn is rounded up by our rounding algorithm. The result being an erroneous round up. The cascade can be avoided if the decimal counterpart is a truncated value but I'm not sure how to do that. This is Paul Dixon's domain for sure.

**Walter Henn** · 2 Jul 2008, 07:39 AM

Code:

 
'Validation of Paul Purvis' integer/divide technique for assigning
'literal values to EXT variables by comparing his results against VAL(string).
 
#COMPILE EXE
#DIM ALL
 
FUNCTION PBMAIN () AS LONG
 
LOCAL number1, number2, random_number AS EXT
LOCAL divisor_digits, decimal_point AS LONG
LOCAL counter AS QUAD
LOCAL string1, string2 AS STRING
LOCAL exponent, exponent_position AS LONG
 
FOR counter = 1 TO 1000000000
    random_number = RND * 10 ' random numbers >= 0 and less than 10
    string1 = STR$(random_number,18)'convert random number to string; accuracy of conversion is not critical.
    number1 = VAL(string1) 'assign VAL(string1) to EXT number1
    exponent_position = INSTR(string1, ANY "eE")
    IF exponent_position <> 0 THEN
        exponent = VAL(MID$(string1, exponent_position +1))
        string2 = LEFT$(string1, exponent_position -1) 'get rid of exponent
    ELSE
        string2 = string1
        exponent = 0
    END IF
    decimal_point = INSTR(string2, ".")
    IF decimal_point = 0 THEN decimal_point = LEN(string2)
    divisor_digits = LEN(string2) - decimal_point - exponent
    string2 = REMOVE$(string2, ".") ' get rid of any decimal point
    number2 = VAL(string2)/10^divisor_digits 'string2 and 10^divisor_digits are whole numbers
    IF number1 <> number2 THEN
        PRINT "error"
        PRINT "VAL(string1): "; STR$(VAL(string1), 18)
        IF divisor_digits => 0 THEN
            PRINT "Int/Dec:      "; string2 +  "/ " + "1" + REPEAT$(divisor_digits, "0")
        ELSE
            PRINT "Int/Dec:      "; string2 + " * " + "1" + REPEAT$(-divisor_digits, "0")
        END IF
        PRINT "number1:      "; STR$(number1,18)
        PRINT "number2:      "; STR$(number2, 18)
        PRINT "diff: "; STR$(number1- number2, 18)
        PRINT REPEAT$(20, "-")
    END IF
NEXT counter
PRINT "End  of test!"
WAITKEY$
END FUNCTION

The code above can be used to test Paul Purvis' technique to get EXT precision from numeric literals by using the integer/divide approach instead of the VAL(string) approach. The program generates random numbers and can easily be used to verify billions of test values. Paul's technique, which is quite clever, appears to give exactly the same value as the VAL(string) approach, but is much faster by a factor of approx. 4-8 times.

**Paul Purvis** · 2 Jul 2008, 11:12 AM

message removed the results to a poor test

**Paul Purvis** · 2 Jul 2008, 02:27 PM

LET statement, get acquainted with it

i never read in manuals some documentation.

not reading the section on the LET statement and understanding it fully,
it is easy to miss that assignments(not math expressions) of an extended precision(EXTENDED OR EXT) and an extended currency variable(CURRENCYX OR CUX) to a variable are probably not supported in versions of pbcc 4.04 because it is not specifically written in the manual and easily over looked or like i said just misread.

the documentation gives information on assigning VARIANT variables.
i have only used a VARIANT variables from coping somebody else's code.
VARIANTS are stored with 128 bits, huh.
maybe i need to get to know a little more about using VARIANT variables.

the manual and online documentation states this

When assigning values to variants, PowerBASIC automatically analyzes the expression and chooses an appropriate or string data type for the internal representation of the Variant. However, you can specify a particular preferred format by adding the optional AS vartype clause. For example:
LET xyz = 21.34 AS LONG
In the above example, the Variant would actually contain the value 21, as the programmer forced it to Long-integer representation. vartype may be one of the following keywords: BYTE, WORD, DWORD, INTEGER, LONG, QUAD, SINGLE, DOUBLE, CURRENCY, STRING.

i would guess the compiler also automatically analyzes the expression and chooses an appropriate or string data type for other variables as well.

added: maybe powerbasic would interested in creating a section called manuals or something similar where they could add, change, or remove documentation as needed in just one single place for official statements concerning the use of their products.

**Walter Henn** · 2 Jul 2008, 04:25 PM

Originally posted by paul d purvis View Post

i never read in manuals some documentation.

not reading the section on the LET statement and understanding it fully,
it is easy to miss that assignments(not math expressions) of an extended precision(EXTENDED OR EXT) and an extended currency variable(CURRENCYX OR CUX) to a variable are probably not supported in versions of pbcc 4.04 because it is not specifically written in the manual and easily over looked or like i said just misread.

the documentation gives information on assigning VARIANT variables.
i have only used a VARIANT variables from coping somebody else's code.
VARIANTS are stored with 128 bits, huh.
maybe i need to get to know a little more about using VARIANT variables...

You're losing me on this post, Paul. Does the LET statement or variant variables have anything, at all, to do with the topic of rounding or VAL in this thread? If not, I respectfully request that you start a new thread.

**Walter Henn** · 2 Jul 2008, 05:52 PM

Originally posted by paul d purvis View Post

will this program suffice as a test

Code:

 
#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
LOCAL m AS EXT
LOCAL s AS STRING
LOCAL ss AS STRING
LOCAL i AS QUAD
LOCAL count AS QUAD
 
FOR i =90000000000000000 TO 99999999999999999 STEP 1712467
s=TRIM$(STR$(i,18))
s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)-1)
trimzeroout:
IF RIGHT$(s,1)="0"  THEN s=LEFT$(s,LEN(s)-1):GOTO trimzeroout
...
m=i/10000000000000000
ss=TRIM$(STR$(m,18))
IF ss<>s THEN
...

Actually, Paul, this test does not prove what it is supposed to. Correct me if I'm wrong, but I believe the intent was to prove that your integer/decimal technique was equivalent in accuracy to the VAL(string) technique.

The coding above only demonstrates that when you do an integer/decimal computation, your results, when rounded by the STR$ function, are exactly the same as that obtained my manipulating the string value of i to get the value, i/10000000000000000.

**Paul Purvis** · 2 Jul 2008, 05:55 PM

Walter
Unless i am convinced otherwise, yes i do feel it maybe relevant.

what is the difference between
local m as ext
m=16.585##
and
local m as ext
let m=16.585##

none i do believe

while it is true that rounding routines are not affected in either way

the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
that is why the integer/decimal is giving us a more precise values to round from.
so understanding the LET statement is critical in my view.

i was trying to make an effort to show why we were getting the values of something similar to 16.5849999something as a value to round from or even possibly to use the variable in a calculation of another expression that probably will make a difference in returned results.

i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.

also not show but in another test i have done with a DOUBLE variables today, i appears i may want to stick with EXTENDED variables totally for many of my math calculations where there is a decimal involved because of the 18 digit precision powerbasic keeps.

here is code that make me believe that.
notice that m when returned in a STR$ with 18 digits can produce other values than STR$ alone, thereby giving me an impression that the value may be different than what i want stored.
i have faith in pb using the correct values to do math with, but i can have faith and security blank too.

this is just for observation purposes only
like Walter said, maybe discussion for another day,
but i am very happy it seems we resolved the EXTENDED assignment deli ma.

Code:

#COMPILE EXE
#DIM ALL
FUNCTION PBMAIN () AS LONG
LOCAL m AS EXT
LOCAL mm AS DOUBLE
LOCAL s AS STRING
LOCAL ss AS STRING
LOCAL sss AS STRING
LOCAL i AS QUAD
LOCAL count AS QUAD

FOR i =900000000000000&& TO 999999999999999&& STEP 1712467&&

s=TRIM$(STR$(i,18))
s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)-1)
trimzeroout:
IF RIGHT$(s,1)="0"  THEN s=LEFT$(s,LEN(s)-1):GOTO trimzeroout

m=i/100000000000000
mm=i/100000000000000
ss=TRIM$(STR$(m,18))
sss=TRIM$(STR$(mm,18))

'IF ss<>s THEN
IF m<>mm THEN
    PRINT i
    PRINT s
    PRINT ss
    PRINT sss
    PRINT "diff found press any key to continue"
    WAITKEY$
END IF

INCR count

IF count MOD 500000 =0 THEN
    PRINT I," > "+S+ " > "+SS+STR$(COUNT,18)
END IF
NEXT i


PRINT "done"
WAITKEY$

END FUNCTION

i made a change to this program after Walter mentioned the STR$ rounding behavior

**Paul Purvis** · 2 Jul 2008, 06:01 PM

i was not aware of the STR$ doing any rounding, maybe i missed that or just forgot during the last few days, let us see what i lose over the 4th of July weekend.

Announcement

VAL rounds a variable

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment