> I'll look into that.
asm .803475219846933572 and num=Val(".803475219846933572")
may have different binary representations.
I talked about decimal domain collisions earlier.
Announcement
Collapse
No announcement yet.
VAL rounds a variable
Collapse
X

Something else...
Code:asm .803475219846933572 .8034752199 10 rV3 .8034752198 190661 random
Code:num=Val(".803475219846933572") digits = 10 Print ASMRoundUp(num , digits ); Round(num,10)
I'm getting a lot of them  my 8000 plus would be a hell of a lot less otherwise.
I'll look into that.
How many disagreements are you getting, John?
Leave a comment:

Here is an example of what I was calling one of the expected regions, where the 499999... rounds down in asmRoundUp but up in roundUpV3:
Code:asm 4.98934999999999141E13 .00000000000049893 17 rV3 .00000000000049894 498935 sequential
Leave a comment:

Hi John
I see that you removed the use of count. I put it back in and am getting about 8000 plus disagreements on a run of 2100000.
> but it matches the other posted algos so far everywhere but the expected regions.
What do you mean by "expected regions"?
There is bug in asm
Code:asm 7.4760000000000004E15 .000000000001 12 rV3 0 7476 sequential
Leave a comment:

My dern internet connection lately has been "sketchy" at best. There's nothing like keeping current by posting once every 2 or 3 days.
Thanks for the comments Dave, and be assured I view your posts with top shelf regard, especially those eased back to my level of understanding. Like you and Walter said, we may have moved into the theoretical, but usually that precedes finding some practicality.
Speaking of which (optimistically hoping it's practical) I worked a bit more and tested extensively the "visual rounding" algo below. I think it's value is that it rounds as you would expect by just looking at the decimal numbers, as if the computer were a decimal computer. And now with all the testing techniques that have been posted and I've used, I'm way more confident of its accuracy, and that I've tested all classes of input. This doesn't prove it correctindeed, I'd tested my original incorrect roundUp algo with billions of values and still missed a whole class of numbersbut it matches the other posted algos so far everywhere but the expected regions.
Code:'compiled with pbcc 4.04 'roundUpV3.bas #COMPILE EXE #DIM ALL #REGISTER NONE FUNCTION ASMRoundUp(num AS EXT, digits AS LONG) AS EXT '################################################################ '# This is an ASM version of: '# temp = 10^number_rounding_digits '# FUNCTION = SGN(number)*INT(ABS(number*temp) + 0.5##)/temp '# '# note: Parameters are passed by reference as it's faster. '# if this is changed then the code must be altered to match '################################################################ LOCAL OldControlWord AS INTEGER !jmp skip 'jump over the following lookup tables table: 'this is just a lookup table of all the powers of 10 from 0 to 18 !dd &h1,0 '1 !dd &hA,0 '10 !dd &h64,0 '100 !dd &h3E8,0 '1000 !dd &h2710,0 '10000 !dd &h186A0,0 '100000 !dd &hF4240,0 '1000000 !dd &h989680,0 '10000000 !dd &h5F5E100,0 '100000000 !dd &h3B9ACA00,0 '1000000000 !dd &h540BE400,&h2 '10000000000 !dd &h4876E800,&h17 '100000000000 !dd &hD4A51000,&hE8 '1000000000000 !dd &h4E72A000,&h918 '10000000000000 !dd &h107A4000,&h5AF3 '100000000000000 !dd &hA4C68000,&h38D7E '1000000000000000 !dd &h6FC10000,&h2386F2 '10000000000000000 !dd &h5D8A0000,&h1634578 '100000000000000000 !dd &hA7640000,&hDE0B6B3 '1000000000000000000 !dd &h89E80000,&h8AC72304 '10000000000000000000 !dd &h63100000,&h6BC75E2D '100000000000000000000 half: !dd &h3f000000 'the number 0.5 in single format FPControlWordRoundToZeroEXT: !dw &h01F3F 'control word to set FPU to round toward zero with extended precision skip: !mov eax,digits 'look up the scale required for this many digits !mov eax,[eax] 'must do this as digits is a function parameter so I get the address, not the value !fild qword ptr table[eax*8] 'get the scale factor !mov eax,num## 'again, got the address of num## as it's a function parameter !fld tbyte ptr [eax] 'load num## !fmul st(0),st(1) 'num## * temp !fld dword ptr half 'get the number 0.5 !mov ecx,79 !bt [eax],ecx 'check the sign of num## !jnc skp !fchs 'if num ve then make 0.5 ve too skp: !faddp st(1),st(0) 'num## * temp + 0.5 !fstcw OldControlWord 'save original control word !fldcw FPControlWordRoundToZeroEXT 'change FPU control word to round toward zero !frndint 'round to nearest integer > INT(num## * temp + 0.5) !fldcw OldControlWord 'restore original control word !fdivrp st(1),st(0) 'INT(num## * temp + 0.5) / temp !fstp function 'save answer END FUNCTION MACRO FUNCTION mfRandomLong() !mov eax, mwcSoph !mov ecx, mwcRandom !mul ecx !add eax, mwcCarry !adc edx, 0 !mov mwcRandom, eax !mov mwcCarry, edx END MACRO = mwcRandom FUNCTION extRnd() AS EXT ' 'creates an EXT pseudornd test value from 0.9999... to +0.9999... where each 'of the 18 digits are statistically random. Period is ~2^61 (2*10^18) EXT values 'before repeatingmany years. ' STATIC x, p5 AS EXT, xp, oneTime AS LONG STATIC mwcSoph, mwcRandom, mwcCarry AS LONG IF oneTime = 0 THEN oneTime = 1 'this section only needed once per program run. x = 999999999999999999 / 1000000000000000000 p5 = 5 / 10 xp = VARPTR(x) mwcSoph = &h68131E4B 'fixed sgprime constant. Leave intact mwcRandom = &ha5b218da 'user seed1, can be any LONG > 1 mwcCarry = &h3fe8700c 'user seed2, any positive LONG less than &h68131E4B (mwcSoph) and > 1 !dw &h310f ;time stamp counter to vary seed1 !add mwcRandom, eax ;vary seed1 mwcCarry = mwcCarry + TIMER * 18 'and vary seed2 a bit. END IF POKE LONG, xp, mfRandomLong POKE LONG, xp+4, mfRandomLong OR &h080000000 IF mfRandomLong < 0 THEN IF mfRandomLong < 0 THEN FUNCTION = (x  p5) ELSE FUNCTION = (x  p5) END IF ELSE IF mfRandomLong < 0 THEN FUNCTION = x ELSE FUNCTION = x END IF END IF EXIT FUNCTION END FUNCTION FUNCTION roundme4(BYREF X AS EXT, BYREF Factor AS LONG) AS EXT LOCAL MM AS EXT LOCAL additional AS LONG additional=factor+2& mm=x+VAL(MID$("."+REPEAT$(additional+3,"0"),1,additional+3)+"1") FUNCTION=ROUND(mm,factor) END FUNCTION FUNCTION roundUpV3(a AS EXT, b AS LONG) AS EXT ' 'rounds "a" to "b" digits past decimal point, but up on 5 instead of using 'banker's rounding. It rounds as you would expect by just looking at the decimal numbers, 'as if the computer were a decimal computer. eg. roundUpV3(123.4545, 3) becomes 123.455. ' STATIC x AS EXT STATIC qa AS QUAD STATIC fillOnce, sizeA AS LONG DIM ten(18) AS STATIC QUAD IF a < 0 THEN 'handle negatives x = ROUND(a, b) 'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding IF x <= a THEN 'rounded up, so no need to go thru hoops. FUNCTION = x EXIT FUNCTION END IF ELSE x = ROUND(a, b) 'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding IF x >= a THEN 'rounded up FUNCTION = x EXIT FUNCTION END IF END IF 'make powers of 10 to avoid exponent calcs IF fillOnce = 0 THEN ten(00) = 1 ten(01) = 10 ten(02) = 100 ten(03) = 1000 ten(04) = 10000 ten(05) = 100000 ten(06) = 1000000 ten(07) = 10000000 ten(08) = 100000000 ten(09) = 1000000000 ten(10) = 10000000000 ten(11) = 100000000000 ten(12) = 1000000000000 ten(13) = 10000000000000 ten(14) = 100000000000000 ten(15) = 1000000000000000 ten(16) = 10000000000000000 ten(17) = 100000000000000000 ten(18) = 1000000000000000000 fillOnce = 1 END IF 'how big is a? SELECT CASE ABS(a) CASE < ten(00): sizeA = 01: qa = a * ten(18) CASE < ten(01): sizeA = 02: qa = a * ten(17) CASE < ten(02): sizeA = 03: qa = a * ten(16) CASE < ten(03): sizeA = 04: qa = a * ten(15) CASE < ten(04): sizeA = 05: qa = a * ten(14) CASE < ten(05): sizeA = 06: qa = a * ten(13) CASE < ten(06): sizeA = 07: qa = a * ten(12) CASE < ten(07): sizeA = 08: qa = a * ten(11) CASE < ten(08): sizeA = 09: qa = a * ten(10) CASE < ten(09): sizeA = 10: qa = a * ten(09) CASE < ten(10): sizeA = 11: qa = a * ten(08) CASE < ten(11): sizeA = 12: qa = a * ten(07) CASE < ten(12): sizeA = 13: qa = a * ten(06) CASE < ten(13): sizeA = 14: qa = a * ten(05) CASE < ten(14): sizeA = 15: qa = a * ten(04) CASE < ten(15): sizeA = 16: qa = a * ten(03) CASE < ten(16): sizeA = 17: qa = a * ten(02) CASE < ten(17): sizeA = 18: qa = a * ten(01) CASE ELSE FUNCTION = a EXIT FUNCTION 'exit because a is too big to round END SELECT IF sizeA + b > 18 THEN FUNCTION = a EXIT FUNCTION 'exit because round digits too big END IF qa = qa \ ten(18  (sizeA + b)) 'make whole EXT effectively an integer sizeA = qa MOD 10 IF sizeA = 5 THEN 'is last digit a 5? FUNCTION = ((qa + 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position ELSEIF sizeA = 5 THEN 'or 5 FUNCTION = ((qa  10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position ELSE FUNCTION = x END IF END FUNCTION FUNCTION PBMAIN () AS LONG ' 'this prints out differences between asmRoundUp and roundUpV3 but stops if a difference is 'found between roundMe4 and roundUpV3, which should always match but for the known "49999" 'difference. It tests both random and sequential series. ' LOCAL M, n, m2, incrTest AS EXT LOCAL roundto, numDig, count, ii, tLoop AS LONG OPEN "c:\RoundV3asmDiff.txt" FOR APPEND AS #1 incrTest = 1 / 1000000000000000000 ? "rounding..." FOR ii = 1 TO 2100000'000 n = extRnd roundTo = ABS(extRnd * 1000000000000000000 MOD 17) 'round from 0 to 17 digits IF (ii AND &h07ffff) = 0 THEN PRINT ii IF asmRoundUp(n, roundTo) <> roundUpV3(n, roundTo) THEN PRINT #1, "asm" & STR$(n, 18), STR$(asmRoundUp(n, roundTo), 18) & STR$(roundTo) PRINT #1, "rV3" & " ", STR$(roundUpV3(n, roundTo), 18) & STR$(ii) & " random" END IF m = m + incrTest roundTo = ABS(extRnd * 1000000000 AND 7) + 10 'round from 10 to 17 digits m2 = roundUpV3(m, roundTo) IF asmRoundUp(m, roundTo) <> m2 THEN PRINT #1, "asm" & STR$(m, 18), STR$(asmRoundUp(m, roundTo), 18) & STR$(roundTo) PRINT #1, "rV3" & " ", STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential" END IF IF RoundMe4(m, roundTo) <> m2 AND INSTR(STR$(m, 18), "49999") = 0 _ AND INSTR(STR$(m, 18), "4.9999") = 0 THEN ? STR$(m, 18) & STR$(RoundMe4(m, roundTo), 18) & STR$(roundTo) ? STR$(m, 18) & STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential" WAITKEY$ END IF NEXT ? "done" WAITKEY$ END FUNCTION
Leave a comment:

frankly, I'm perfectly satisfied with banker's rounding
...but who knows, one day we just might want a roundup and Paul Dixon has provided us with that or we just might want a numeric literal to go the distance and until PB give it back to us we have Paul Purvis to thank for his integer division method.
With over 4700 views of this thread I'd reckon one or two are using one or both of the above and have been for some days.
Leave a comment:

Originally posted by Walter Henn View PostI think it's also important to keep the speed of conversion from literal to EXT in perspective...Originally posted by David Roberts View PostMost threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered...Last edited by Walter Henn; 7 Jul 2008, 05:05 AM.
Leave a comment:

I think it's also important to keep the speed of conversion from literal to EXT in perspective.
John used the phrase "uber resolution" with regard the asm data statements.
We may have constants such as sqr(2) and prefer to use a numeric literal.
Rounding up to 18 digits we have 1.41421356237309505. The binary representation of 141421356237309505/100000000000000000 is 3FFFB504F333F9DE648F. The extended binary representation of sqr(2) ends with ....6484.
sqr(2) * sqr(2)  2 = 0 where sqr(2) is computed with infinite precision.
With
Code:#Compile Exe #Dim All Function PBMain () As Long Local x, y As Ext x = 141421356237309505/100000000000000000 ' 18 digits 'x = 14142135623730950488016887242097/10000000000000000000000000000000 ' 32 digits GoTo pastYdata extendedYdata: !dw &h6484, &hF9DE, &hF333, &hB504, &h3FFF pastYdata: y = Peek(Ext, CodePtr(extendedYdata)) ? Str$(Sqr(2)*Sqr(2)  2,18) ? Str$(y*y  2,18) ? Str$(x*x  2,18) WAITKEY$ End Function
1.08420217248550443E19
1.08420217248550443E19
3.2526065174565133E18
With 32 digits, letting the system do its best I get
1.08420217248550443E19
1.08420217248550443E19
5.63785129692462306E18
Pedantic? This is well 'over the top' for anything that I do and may be so for most but perhaps one or more want that last ounce of precision.
It is worth remembering that we may be looking for 6 significant figure accuracy, say, in our final results but require many more significant figure accuracy for interim calculations because of accumulation error. I reckon that some of the jobs done by super computers where they are on their knees for days on end take 'so long' not just because of the amount of data being crunched but also because of the accuracy demanded of the interim calculations.
Added: Of course, we could use x = sqr(2)  a bad choice. How about pi, e and such or even a computation done elsewhere where we have access to the binary representation before it got 'corrupted' in its transition to the decimal domain.Last edited by David Roberts; 6 Jul 2008, 10:47 AM.
Leave a comment:

Originally posted by John Gleason View Post
... so we see division is fast too, some 30 times faster than VAL for this application, and is much easier to implement than the asm data statements...
I think it's also important to keep the speed of conversion from literal to EXT in perspective. Most programs have a limited number of literals that they employ. So, even if the conversion from literal to EXT is on the order of 1 usec, the speed of conversion would have little impact on the time to run the program. The exception, of course, would be the use of a literal in a loop with many iterations or a routine with many calls. However, in such cases, an EXT variable (equal to the value of the literal) should be placed in the loop or routine in place of the literal. This applies whether you're using VAL OR integer division.
Finally, I'd like to see Paul's integer division approach be placed also in the thread, "Increase the Precision of Your Floating Point Literals to 18 Decimal Digits."Last edited by Walter Henn; 6 Jul 2008, 07:21 PM.
Leave a comment:

Later I thought that probably a function example would be a good idea. So here is some code to test below:
Code:#COMPILE EXE #DIM ALL FUNCTION speedyExtAssign() AS LONG STATIC a,b,c,d,e AS EXT STATIC ii AS LONG, t AS SINGLE GOTO pastData 'skip over asm data operations. extendedAdata: !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D ;<< copy data here. This represents a in memory extendedBdata: !dw &hBB21, &h3F35, &h05A4, &h932C, &h401D ;b in memory extendedCdata: !dw &hBC21, &h3F35, &h05A4, &h932C, &h401D ;c extendedDdata: !dw &hBD21, &h3F35, &h05A4, &h932C, &h401D ;d extendedEdata: !dw &hBE21, &h3F35, &h05A4, &h932C, &h401D ;e pastData: t = TIMER FOR ii = 1 TO 100000000 '100,000,000 loops a = PEEK(EXT, CODEPTR(extendedAdata)) b = PEEK(EXT, CODEPTR(extendedBdata)) c = PEEK(EXT, CODEPTR(extendedCdata)) d = PEEK(EXT, CODEPTR(extendedDdata)) e = PEEK(EXT, CODEPTR(extendedEdata)) ' ? STR$(a,18) & STR$(b,18) & STR$(c,18) & STR$(d,18) & STR$(e,18): waitkey$ 'uncomment to look at the values NEXT t = TIMER  t ? "a half billion assignments done in" & STR$(t, 4) & " sec" ? "which is " & FORMAT$(500000000 / t, "0,") & " per sec" ? "" ? "and now using VAL:" t = TIMER FOR ii = 1 TO 3000000 '3,000,000 loops a = VAL("1234567890.12345678") NEXT t = TIMER  t ? "3 million assignments done in" & STR$(t, 4) & " sec" ? "which is " & FORMAT$(3000000 / t, "0,") & " per sec" ? "" ? "and finally division:" t = TIMER FOR ii = 1 TO 20000000 '20,000,000 loops a = 123456789012345678 / 100000000 b = 123456789012345681 / 100000000 c = 123456789012345684 / 100000000 d = 123456789012345687 / 100000000 e = 123456789012345690 / 100000000 NEXT t = TIMER  t ? "100 million assignments done in" & STR$(t, 4) & " sec" ? "which is " & FORMAT$(100000000 / t, "0,") & " per sec" ? "" ? "so we see division is fast too, some 30 times faster" ? "than VAL for this application, and is much easier to" ? "implement than the asm data statements." ? "" ? "However, one added feature of the asm method is that it" ? "allows entry of literals at ""uber resolution ;)"" of the" ? "full 18.95 (courtesy Dave Roberts) digits available." WAITKEY$ END FUNCTION FUNCTION PBMAIN () AS LONG speedyExtAssign END FUNCTION
Last edited by John Gleason; 5 Jul 2008, 02:18 PM.
Leave a comment:

Nice one, John. PEEKing an inline data line had not occurred to me  something that I've only done a few times.
Just out of curiosity I then looked at 'Poke Ext, y, 1234567890.12345678##' and found that, here to, the double precision evaluation got POKEd.
Leave a comment:

Dave, here's a way to do it with just a two tick deficit. I would think hardly anyone would notice two ticks on a 2GHz box, even IN a tight loop.
Code:#COMPILE EXE #DIM ALL FUNCTION PBMAIN () AS LONG LOCAL x, y, z AS EXT LOCAL wp AS LONG 'ptr to the actual ext stored data of x x = VAL("1234567890.12345678") 'your full 18 digits into x wp = VARPTR(x) 'ptr to the 5 words of x (10 bytes) OPEN "c:\hexExtendedPrecision1a.txt" FOR OUTPUT AS #1 PRINT #1, "!dw &h" & HEX$(PEEK(WORD, wp ), 4) & ", &h" & HEX$(PEEK(WORD, wp + 2), 4) & ", &h" & _ HEX$(PEEK(WORD, wp + 4), 4) & ", &h" & HEX$(PEEK(WORD, wp + 6), 4) & ", &h" & _ HEX$(PEEK(WORD, wp + 8), 4) ? "ok, copy the line in c:\hexExtendedPrecision1a.txt and put it in the program." GOTO pastYdata 'can't let program read thru the x data as if valid operations. extendedYdata: !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D ;<< copy data here. This represents x (and next, y) in memory pastYdata: y = PEEK(EXT, CODEPTR(extendedYdata)) '<< this is now approx 2 ticks slower than... z = 1234567890.12345678## '<< this, but y will now always be the full EXT IF x = y THEN ? "x =" & STR$(x, 18) ? "y =" & STR$(y, 18) ELSE ? "x does not yet = y" END IF WAITKEY$ END FUNCTION
Leave a comment:

From post #51 here Steve Rossell said "However, evaluation of numeric literals in your compiled programs are currently limited to double precision."
What isn't said is that the storage allocated is limited to 64 bits as well. Looking at a OllyDBug output on running an exe it seems to me, unless I am mistaken, that the storage allocation for extended precision numeric literals is still 80 bits but the resolution of a numeric literal is clearly only 64 bits.
So, if we declare 'Local y as Ext' this will give us our 80 bits of storage at address VarPtr(y).
I am not sufficiently aquainted with FPU assembly to go any further but I do know a man that is.
Can we fill this storage with our evaluation, via FPU instructions as opposed to Paul P's fractional method which is currently in the #1 slot, such that ref to 'y' is a ref to a genuine ext and not one that has had its tail cut off  well, rounded actually.
Added: Obviously, we would not assign 'y' via 'y =' but perhaps read a string at run time. Yeah, I know, VAL does that but a specific VAL bereft of any bells and whistles.Last edited by David Roberts; 5 Jul 2008, 04:47 AM.
Leave a comment:

Originally posted by paul d purvis View PostUnless i am convinced otherwise, yes i do feel it maybe relevant.
what is the difference between
local m as ext
m=16.585##
and
local m as ext
let m=16.585##
none i do believe
while it is true that rounding routines are not affected in either way
the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
that is why the integer/decimal is giving us a more precise values to round from.
so understanding the LET statement is critical in my view.
i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.
Internally, Currency and Extendedcurrency numbers are stored as Quadintegers with an implied decimal point (at 4 places for Currency, and at 2 places for Extendedcurrency). This approach ensures that all of the digits of the variables can be represented exactly.
Leave a comment:

Walter,
you are correct, i was not aware of the fact STR$ rounded, i just always used only str$(x) or str$(x,18).
thanks for pointing that out.
so yes, the test you pointed would fail to show differences.
i going to remove that test.
Leave a comment:

Originally posted by John Gleason View Post...
...
Walter is looking for a round function that rounds up on 5, but only if the base2 representation of the "rounding 5" actually is equal to or exceeds 5 internally. Let's call this, "base2 round up on 5", and is what Paul Dixon's code has been based on I believe...
Walter, how are asmRoundUp/round_dixon looking? Any problems or exceptions there?
The asmRoundUp and round_dixon routines have been found to be correct for all test values (in the bilions). Paul Dixon is a remarkable programmer.Last edited by Walter Henn; 2 Jul 2008, 07:09 PM.
Leave a comment:

i was not aware of the STR$ doing any rounding, maybe i missed that or just forgot during the last few days, let us see what i lose over the 4th of July weekend.
Leave a comment:

Walter
Unless i am convinced otherwise, yes i do feel it maybe relevant.
what is the difference between
local m as ext
m=16.585##
and
local m as ext
let m=16.585##
none i do believe
while it is true that rounding routines are not affected in either way
the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
that is why the integer/decimal is giving us a more precise values to round from.
so understanding the LET statement is critical in my view.
i was trying to make an effort to show why we were getting the values of something similar to 16.5849999something as a value to round from or even possibly to use the variable in a calculation of another expression that probably will make a difference in returned results.
i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.
also not show but in another test i have done with a DOUBLE variables today, i appears i may want to stick with EXTENDED variables totally for many of my math calculations where there is a decimal involved because of the 18 digit precision powerbasic keeps.
here is code that make me believe that.
notice that m when returned in a STR$ with 18 digits can produce other values than STR$ alone, thereby giving me an impression that the value may be different than what i want stored.
i have faith in pb using the correct values to do math with, but i can have faith and security blank too.
this is just for observation purposes only
like Walter said, maybe discussion for another day,
but i am very happy it seems we resolved the EXTENDED assignment deli ma.
Code:#COMPILE EXE #DIM ALL FUNCTION PBMAIN () AS LONG LOCAL m AS EXT LOCAL mm AS DOUBLE LOCAL s AS STRING LOCAL ss AS STRING LOCAL sss AS STRING LOCAL i AS QUAD LOCAL count AS QUAD FOR i =900000000000000&& TO 999999999999999&& STEP 1712467&& s=TRIM$(STR$(i,18)) s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)1) trimzeroout: IF RIGHT$(s,1)="0" THEN s=LEFT$(s,LEN(s)1):GOTO trimzeroout m=i/100000000000000 mm=i/100000000000000 ss=TRIM$(STR$(m,18)) sss=TRIM$(STR$(mm,18)) 'IF ss<>s THEN IF m<>mm THEN PRINT i PRINT s PRINT ss PRINT sss PRINT "diff found press any key to continue" WAITKEY$ END IF INCR count IF count MOD 500000 =0 THEN PRINT I," > "+S+ " > "+SS+STR$(COUNT,18) END IF NEXT i PRINT "done" WAITKEY$ END FUNCTION
Last edited by Paul Purvis; 2 Jul 2008, 06:32 PM.
Leave a comment:

Originally posted by paul d purvis View Postwill this program suffice as a test
Code:#COMPILE EXE #DIM ALL FUNCTION PBMAIN () AS LONG LOCAL m AS EXT LOCAL s AS STRING LOCAL ss AS STRING LOCAL i AS QUAD LOCAL count AS QUAD FOR i =90000000000000000 TO 99999999999999999 STEP 1712467 s=TRIM$(STR$(i,18)) s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)1) trimzeroout: IF RIGHT$(s,1)="0" THEN s=LEFT$(s,LEN(s)1):GOTO trimzeroout ... m=i/10000000000000000 ss=TRIM$(STR$(m,18)) IF ss<>s THEN ...
The coding above only demonstrates that when you do an integer/decimal computation, your results, when rounded by the STR$ function, are exactly the same as that obtained my manipulating the string value of i to get the value, i/10000000000000000.
Leave a comment:
Leave a comment: