VAL rounds a variable

[Walter Henn]

...
...
Walter is looking for a round function that rounds up on 5, but only if the base2 representation of the "rounding 5" actually is equal to or exceeds 5 internally. Let's call this, "base2 round up on 5", and is what Paul Dixon's code has been based on I believe...

Walter, how are asmRoundUp/round_dixon looking? Any problems or exceptions there?

Your statement above regarding what I'm looking for in a roundup on 5 routine and what Paul Dixon's code is based on is correct.

The asmRoundUp and round_dixon routines have been found to be correct for all test values (in the bilions). Paul Dixon is a remarkable programmer.

[Paul Purvis]

Walter,
you are correct, i was not aware of the fact STR$ rounded, i just always used only str$(x) or str$(x,18).
thanks for pointing that out.
so yes, the test you pointed would fail to show differences.
i going to remove that test.

[Walter Henn]

Unless i am convinced otherwise, yes i do feel it maybe relevant.

what is the difference between
local m as ext
m=16.585##
and
local m as ext
let m=16.585##

none i do believe

while it is true that rounding routines are not affected in either way

the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
that is why the integer/decimal is giving us a more precise values to round from.
so understanding the LET statement is critical in my view.

If the use of the LET statement does not result in any difference when assigning a value to a variable, why would understanding the LET statement be critical??? I believe that this is really muddying the water.

i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.

Extracted from PBCC 4.04 Hep File:
Internally, Currency and Extended-currency numbers are stored as Quad-integers with an implied decimal point (at 4 places for Currency, and at 2 places for Extended-currency). This approach ensures that all of the digits of the variables can be represented exactly.

[Paul Purvis]

Walter,
good to go on the quad integers

[David Roberts]

From post #51 here Steve Rossell said "However, evaluation of numeric literals in your compiled programs are currently limited to double precision."

What isn't said is that the storage allocated is limited to 64 bits as well. Looking at a OllyDBug output on running an exe it seems to me, unless I am mistaken, that the storage allocation for extended precision numeric literals is still 80 bits but the resolution of a numeric literal is clearly only 64 bits.

So, if we declare 'Local y as Ext' this will give us our 80 bits of storage at address VarPtr(y).

I am not sufficiently aquainted with FPU assembly to go any further but I do know a man that is.

Can we fill this storage with our evaluation, via FPU instructions as opposed to Paul P's fractional method which is currently in the #1 slot, such that ref to 'y' is a ref to a genuine ext and not one that has had its tail cut off - well, rounded actually.

Added: Obviously, we would not assign 'y' via 'y =' but perhaps read a string at run time. Yeah, I know, VAL does that but a specific VAL bereft of any bells and whistles.

[John Gleason]

Dave, here's a way to do it with just a two tick deficit. I would think hardly anyone would notice two ticks on a 2GHz box, even IN a tight loop.

#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

    LOCAL x, y, z AS EXT
    LOCAL wp AS LONG               'ptr to the actual ext stored data of x
    x = VAL("1234567890.12345678") 'your full 18 digits into x
    wp = VARPTR(x)                 'ptr to the 5 words of x (10 bytes)
    OPEN "c:\hexExtendedPrecision1a.txt" FOR OUTPUT AS #1

    PRINT #1, "!dw &h" &  HEX$(PEEK(WORD, wp    ), 4) & ", &h" & HEX$(PEEK(WORD, wp + 2), 4) & ", &h" & _
                          HEX$(PEEK(WORD, wp + 4), 4) & ", &h" & HEX$(PEEK(WORD, wp + 6), 4) & ", &h" & _
                          HEX$(PEEK(WORD, wp + 8), 4)
    ? "ok, copy the line in c:\hexExtendedPrecision1a.txt and put it in the program."
    GOTO pastYdata                 'can't let program read thru the x data as if valid operations.
    extendedYdata:
    !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D  ;<< copy data here. This represents x (and next, y) in memory
    pastYdata:

    y = PEEK(EXT, CODEPTR(extendedYdata))   '<< this is now approx 2 ticks slower than...
    z = 1234567890.12345678##               '<< this, but y will now always be the full EXT

    IF x = y THEN
       ? "x =" & STR$(x, 18)
       ? "y =" & STR$(y, 18)
    ELSE
       ? "x does not yet = y"
    END IF
    WAITKEY$
END FUNCTION

[David Roberts]

Nice one, John. PEEKing an inline data line had not occurred to me - something that I've only done a few times.

Just out of curiosity I then looked at 'Poke Ext, y, 1234567890.12345678##' and found that, here to, the double precision evaluation got POKEd.

[John Gleason]

Later I thought that probably a function example would be a good idea. So here is some code to test below:

#COMPILE EXE
#DIM ALL

FUNCTION speedyExtAssign() AS LONG
   STATIC a,b,c,d,e AS EXT
   STATIC ii AS LONG, t AS SINGLE
    GOTO pastData                 'skip over asm data operations.
    extendedAdata:
    !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D  ;<< copy data here. This represents a in memory
    extendedBdata:
    !dw &hBB21, &h3F35, &h05A4, &h932C, &h401D  ;b in memory
    extendedCdata:
    !dw &hBC21, &h3F35, &h05A4, &h932C, &h401D  ;c
    extendedDdata:
    !dw &hBD21, &h3F35, &h05A4, &h932C, &h401D  ;d
    extendedEdata:
    !dw &hBE21, &h3F35, &h05A4, &h932C, &h401D  ;e
    pastData:

    t = TIMER
    FOR ii = 1 TO 100000000       '100,000,000 loops
       a = PEEK(EXT, CODEPTR(extendedAdata))
       b = PEEK(EXT, CODEPTR(extendedBdata))
       c = PEEK(EXT, CODEPTR(extendedCdata))
       d = PEEK(EXT, CODEPTR(extendedDdata))
       e = PEEK(EXT, CODEPTR(extendedEdata))
'      ? STR$(a,18) & STR$(b,18) & STR$(c,18) & STR$(d,18) & STR$(e,18): waitkey$ 'uncomment to look at the values
    NEXT
    t = TIMER - t
    ? "a half billion assignments done in" & STR$(t, 4) & " sec"
    ? "which is " & FORMAT$(500000000 / t, "0,") & " per sec"
    ? ""
    ? "and now using VAL:"

    t = TIMER
    FOR ii = 1 TO 3000000         '3,000,000 loops
       a = VAL("1234567890.12345678")
    NEXT
    t = TIMER - t
    ? "3 million assignments done in" & STR$(t, 4) & " sec"
    ? "which is " & FORMAT$(3000000 / t, "0,") & " per sec"
    ? ""
    ? "and finally division:"

    t = TIMER
    FOR ii = 1 TO 20000000        '20,000,000 loops
       a = 123456789012345678 / 100000000
       b = 123456789012345681 / 100000000
       c = 123456789012345684 / 100000000
       d = 123456789012345687 / 100000000
       e = 123456789012345690 / 100000000
    NEXT
    t = TIMER - t
    ? "100 million assignments done in" & STR$(t, 4) & " sec"
    ? "which is " & FORMAT$(100000000 / t, "0,") & " per sec"
    ? ""
    ? "so we see division is fast too, some 30 times faster"
    ? "than VAL for this application, and is much easier to"
    ? "implement than the asm data statements."
    ? ""
    ? "However, one added feature of the asm method is that it"
    ? "allows entry of literals at ""uber resolution ;)"" of the"
    ? "full 18.95 (courtesy Dave Roberts) digits available."
    WAITKEY$
    

END FUNCTION

FUNCTION PBMAIN () AS LONG
    speedyExtAssign
END FUNCTION

[Walter Henn]

... so we see division is fast too, some 30 times faster than VAL for this application, and is much easier to implement than the asm data statements...

I'd like to make a few comments on the integer division approach for literal assignment to EXT variables by Paul Purvis. First of all, John, you stated in your code that this approach is some 30 times faster than the VAL method. I think it's important to point out that this speed increase only applies to the specific test values used in your code (values with 18 significant digits). Often literals as simple as .001 need to be used in code (far more often than 18-digit literals). The speed increase for integer division over VAL for such simple values is on the order of ten times.

I think it's also important to keep the speed of conversion from literal to EXT in perspective. Most programs have a limited number of literals that they employ. So, even if the conversion from literal to EXT is on the order of 1 usec, the speed of conversion would have little impact on the time to run the program. The exception, of course, would be the use of a literal in a loop with many iterations or a routine with many calls. However, in such cases, an EXT variable (equal to the value of the literal) should be placed in the loop or routine in place of the literal. This applies whether you're using VAL OR integer division.

Finally, I'd like to see Paul's integer division approach be placed also in the thread, "Increase the Precision of Your Floating Point Literals to 18 Decimal Digits."

[David Roberts]

I think it's also important to keep the speed of conversion from literal to EXT in perspective.

Most threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered. For me John Gleason has a wonderful habit of making me think, if not write, "had not occurred to me" when he is posting code to 'squeeze the last ounce'.

John used the phrase "uber resolution" with regard the asm data statements.

We may have constants such as sqr(2) and prefer to use a numeric literal.

Rounding up to 18 digits we have 1.41421356237309505. The binary representation of 141421356237309505/100000000000000000 is 3FFFB504F333F9DE648F. The extended binary representation of sqr(2) ends with ....6484.

sqr(2) * sqr(2) - 2 = 0 where sqr(2) is computed with infinite precision.

With

#Compile Exe
#Dim All
 
Function PBMain () As Long
 
  Local x, y As Ext
 
  x = 141421356237309505/100000000000000000 ' 18 digits
  'x = 14142135623730950488016887242097/10000000000000000000000000000000 ' 32 digits
 
  GoTo pastYdata
  extendedYdata:
  !dw &h6484, &hF9DE, &hF333, &hB504, &h3FFF
  pastYdata:
 
  y = Peek(Ext, CodePtr(extendedYdata))
 
  ? Str$(Sqr(2)*Sqr(2) - 2,18)
  ? Str$(y*y - 2,18)
  ? Str$(x*x - 2,18)
 
  WAITKEY$
 
End Function

I get

-1.08420217248550443E-19
-1.08420217248550443E-19
3.2526065174565133E-18

With 32 digits, letting the system do its best I get

-1.08420217248550443E-19
-1.08420217248550443E-19
-5.63785129692462306E-18

Pedantic? This is well 'over the top' for anything that I do and may be so for most but perhaps one or more want that last ounce of precision.

It is worth remembering that we may be looking for 6 significant figure accuracy, say, in our final results but require many more significant figure accuracy for interim calculations because of accumulation error. I reckon that some of the jobs done by super computers where they are on their knees for days on end take 'so long' not just because of the amount of data being crunched but also because of the accuracy demanded of the interim calculations.

Added: Of course, we could use x = sqr(2) - a bad choice. How about pi, e and such or even a computation done elsewhere where we have access to the binary representation before it got 'corrupted' in its transition to the decimal domain.

[Walter Henn]

I think it's also important to keep the speed of conversion from literal to EXT in perspective...

Most threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered...

I couldn't agree with you more, David. I have participated extensively in this thread not because of the importance to me of rounding (frankly, I'm perfectly satisfied with banker's rounding), but because I have learned valuable new concepts (visual rounding is not one of them) or been refreshed with old concepts.

[David Roberts]

frankly, I'm perfectly satisfied with banker's rounding

So am I.

...but who knows, one day we just might want a round-up and Paul Dixon has provided us with that or we just might want a numeric literal to go the distance and until PB give it back to us we have Paul Purvis to thank for his integer division method.

With over 4700 views of this thread I'd reckon one or two are using one or both of the above and have been for some days.

[John Gleason]

My dern internet connection lately has been "sketchy" at best. There's nothing like keeping current by posting once every 2 or 3 days.

Thanks for the comments Dave, and be assured I view your posts with top shelf regard, especially those eased back to my level of understanding. Like you and Walter said, we may have moved into the theoretical, but usually that precedes finding some practicality.

Speaking of which (optimistically hoping it's practical) I worked a bit more and tested extensively the "visual rounding" algo below. I think it's value is that it rounds as you would expect by just looking at the decimal numbers, as if the computer were a decimal computer. And now with all the testing techniques that have been posted and I've used, I'm way more confident of its accuracy, and that I've tested all classes of input. This doesn't prove it correct--indeed, I'd tested my original incorrect roundUp algo with billions of values and still missed a whole class of numbers--but it matches the other posted algos so far everywhere but the expected regions.

'compiled with pbcc 4.04
'roundUpV3.bas

#COMPILE EXE
#DIM ALL
#REGISTER NONE

FUNCTION ASMRoundUp(num AS EXT, digits AS LONG) AS EXT
'################################################################
'# This is an ASM version of:
'# temp = 10^number_rounding_digits
'# FUNCTION = SGN(number)*INT(ABS(number*temp) + 0.5##)/temp
'#
'# note: Parameters are passed by reference as it's faster.
'# if this is changed then the code must be altered to match
'################################################################

LOCAL OldControlWord AS INTEGER

!jmp skip                  'jump over the following lookup tables
table:
'this is just a lookup table of all the powers of 10 from 0 to 18
!dd &h1,0                  '1
!dd &hA,0                  '10
!dd &h64,0                 '100
!dd &h3E8,0                '1000
!dd &h2710,0               '10000
!dd &h186A0,0              '100000
!dd &hF4240,0              '1000000
!dd &h989680,0             '10000000
!dd &h5F5E100,0            '100000000
!dd &h3B9ACA00,0           '1000000000
!dd &h540BE400,&h2         '10000000000
!dd &h4876E800,&h17        '100000000000
!dd &hD4A51000,&hE8        '1000000000000
!dd &h4E72A000,&h918       '10000000000000
!dd &h107A4000,&h5AF3      '100000000000000
!dd &hA4C68000,&h38D7E     '1000000000000000
!dd &h6FC10000,&h2386F2    '10000000000000000
!dd &h5D8A0000,&h1634578   '100000000000000000
!dd &hA7640000,&hDE0B6B3   '1000000000000000000
!dd &h89E80000,&h8AC72304  '10000000000000000000
!dd &h63100000,&h6BC75E2D  '100000000000000000000
half:
!dd &h3f000000             'the number 0.5 in single format
FPControlWordRoundToZeroEXT:
!dw &h01F3F                'control word to set FPU to round toward zero with extended precision


skip:
!mov eax,digits                     'look up the scale required for this many digits
!mov eax,[eax]                      'must do this as digits is a function parameter so I get the address, not the value
!fild qword ptr table[eax*8]        'get the scale factor
!mov eax,num##                      'again, got the address of num## as it's a function parameter
!fld tbyte ptr [eax]                'load num##
!fmul st(0),st(1)                   'num## * temp
!fld dword ptr half                 'get the number 0.5
!mov ecx,79
!bt [eax],ecx                       'check the sign of num##
!jnc skp
!fchs                               'if num -ve then make 0.5 -ve too
skp:
!faddp st(1),st(0)                  'num## * temp  + 0.5
!fstcw OldControlWord               'save original control word
!fldcw FPControlWordRoundToZeroEXT  'change FPU control word to round toward zero
!frndint                            'round to nearest integer -> INT(num## * temp  + 0.5)
!fldcw OldControlWord               'restore original control word
!fdivrp st(1),st(0)                 'INT(num## * temp  + 0.5) / temp
!fstp function                      'save answer

END FUNCTION

MACRO FUNCTION mfRandomLong()
    !mov eax, mwcSoph
    !mov ecx, mwcRandom
    !mul ecx
    !add eax, mwcCarry
    !adc edx, 0
    !mov mwcRandom, eax
    !mov mwcCarry,  edx
END MACRO = mwcRandom

FUNCTION extRnd() AS EXT
'---------------------------------------------------------------------------
'creates an EXT pseudo-rnd test value from -0.9999... to +0.9999... where each
'of the 18 digits are statistically random. Period is ~2^61 (2*10^18) EXT values
'before repeating--many years.
'---------------------------------------------------------------------------
  STATIC x, p5 AS EXT, xp, oneTime AS LONG
  STATIC mwcSoph, mwcRandom, mwcCarry AS LONG

  IF oneTime = 0 THEN
     oneTime = 1             'this section only needed once per program run.
     x = 999999999999999999 / 1000000000000000000
     p5 = 5 / 10
     xp = VARPTR(x)
     mwcSoph   = &h68131E4B  'fixed sg-prime constant. Leave intact
     mwcRandom = &ha5b218da  'user seed1, can be any LONG > 1
     mwcCarry  = &h3fe8700c  'user seed2, any positive LONG less than &h68131E4B (mwcSoph) and > 1
     !dw &h310f              ;time stamp counter to vary seed1
     !add mwcRandom, eax     ;vary seed1
     mwcCarry = mwcCarry + TIMER * 18 'and vary seed2 a bit.
  END IF

     POKE LONG, xp, mfRandomLong
     POKE LONG, xp+4, mfRandomLong OR &h080000000
     IF mfRandomLong < 0  THEN
        IF mfRandomLong < 0 THEN
           FUNCTION = -(x - p5)
        ELSE
           FUNCTION =  (x - p5)
        END IF
     ELSE
        IF mfRandomLong < 0 THEN
           FUNCTION = -x
        ELSE
           FUNCTION =  x
        END IF
     END IF
     EXIT FUNCTION

END FUNCTION

FUNCTION roundme4(BYREF X AS EXT, BYREF Factor AS LONG) AS EXT
LOCAL MM AS EXT
LOCAL additional AS LONG
additional=factor+2&
mm=x+VAL(MID$("."+REPEAT$(additional+3,"0"),1,additional+3)+"1")
FUNCTION=ROUND(mm,factor)
END FUNCTION

FUNCTION roundUpV3(a AS EXT, b AS LONG) AS EXT
'---------------------------------------------------------------------------------------
'rounds "a" to "b" digits past decimal point, but up on 5 instead of using
'banker's rounding. It rounds as you would expect by just looking at the decimal numbers,
'as if the computer were a decimal computer. eg. roundUpV3(123.4545, 3) becomes 123.455.
'---------------------------------------------------------------------------------------
   STATIC x AS EXT
   STATIC qa AS QUAD
   STATIC fillOnce, sizeA AS LONG
   DIM ten(18) AS STATIC QUAD

   IF a < 0 THEN         'handle negatives
      x = ROUND(a, b)    'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding
      IF x <= a THEN     'rounded up, so no need to go thru hoops.
         FUNCTION = x
         EXIT FUNCTION
      END IF
   ELSE
      x = ROUND(a, b)    'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding
      IF x >= a THEN     'rounded up
         FUNCTION = x
         EXIT FUNCTION
      END IF
   END IF
                         'make powers of 10 to avoid exponent calcs
   IF fillOnce = 0 THEN
   ten(00) = 1
   ten(01) = 10
   ten(02) = 100
   ten(03) = 1000
   ten(04) = 10000
   ten(05) = 100000
   ten(06) = 1000000
   ten(07) = 10000000
   ten(08) = 100000000
   ten(09) = 1000000000
   ten(10) = 10000000000
   ten(11) = 100000000000
   ten(12) = 1000000000000
   ten(13) = 10000000000000
   ten(14) = 100000000000000
   ten(15) = 1000000000000000
   ten(16) = 10000000000000000
   ten(17) = 100000000000000000
   ten(18) = 1000000000000000000
   fillOnce = 1
   END IF
                        'how big is a?
   SELECT CASE ABS(a)
      CASE < ten(00): sizeA = 01: qa = a * ten(18)
      CASE < ten(01): sizeA = 02: qa = a * ten(17)
      CASE < ten(02): sizeA = 03: qa = a * ten(16)
      CASE < ten(03): sizeA = 04: qa = a * ten(15)
      CASE < ten(04): sizeA = 05: qa = a * ten(14)
      CASE < ten(05): sizeA = 06: qa = a * ten(13)
      CASE < ten(06): sizeA = 07: qa = a * ten(12)
      CASE < ten(07): sizeA = 08: qa = a * ten(11)
      CASE < ten(08): sizeA = 09: qa = a * ten(10)
      CASE < ten(09): sizeA = 10: qa = a * ten(09)
      CASE < ten(10): sizeA = 11: qa = a * ten(08)
      CASE < ten(11): sizeA = 12: qa = a * ten(07)
      CASE < ten(12): sizeA = 13: qa = a * ten(06)
      CASE < ten(13): sizeA = 14: qa = a * ten(05)
      CASE < ten(14): sizeA = 15: qa = a * ten(04)
      CASE < ten(15): sizeA = 16: qa = a * ten(03)
      CASE < ten(16): sizeA = 17: qa = a * ten(02)
      CASE < ten(17): sizeA = 18: qa = a * ten(01)
      CASE ELSE
         FUNCTION = a
         EXIT FUNCTION             'exit because a is too big to round
   END SELECT
   IF sizeA + b > 18 THEN
      FUNCTION = a
      EXIT FUNCTION                'exit because round digits too big
   END IF

   qa = qa \ ten(18 - (sizeA + b)) 'make whole EXT effectively an integer
   sizeA = qa MOD 10
   IF sizeA = 5 THEN               'is last digit a 5?
      FUNCTION = ((qa + 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position
   ELSEIF sizeA = -5 THEN          'or -5
      FUNCTION = ((qa - 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position
   ELSE
      FUNCTION = x
   END IF

END FUNCTION

FUNCTION PBMAIN () AS LONG
'---------------------------------------------------------------------------------------
'this prints out differences between asmRoundUp and roundUpV3 but stops if a difference is
'found between roundMe4 and roundUpV3, which should always match but for the known "49999"
'difference. It tests both random and sequential series.
'---------------------------------------------------------------------------------------
LOCAL M, n, m2, incrTest AS EXT
LOCAL roundto, numDig, count, ii, tLoop AS LONG
  OPEN "c:\RoundV3asmDiff.txt" FOR APPEND AS #1

  incrTest = 1 / 1000000000000000000
  ? "rounding..."
  FOR ii = 1 TO 2100000'000
   n = extRnd
   roundTo = ABS(extRnd * 1000000000000000000 MOD 17) 'round from 0 to 17 digits
   IF (ii AND &h07ffff) = 0 THEN PRINT ii
   IF asmRoundUp(n, roundTo) <> roundUpV3(n, roundTo) THEN
      PRINT #1, "asm" & STR$(n, 18), STR$(asmRoundUp(n, roundTo), 18) & STR$(roundTo)
      PRINT #1, "rV3" & "                       ", STR$(roundUpV3(n, roundTo), 18) & STR$(ii) & " random"
   END IF
  m = m + incrTest
  roundTo = ABS(extRnd * 1000000000 AND 7) + 10       'round from 10 to 17 digits
  m2 = roundUpV3(m, roundTo)
   IF asmRoundUp(m, roundTo) <> m2 THEN
      PRINT #1, "asm" & STR$(m, 18), STR$(asmRoundUp(m, roundTo), 18) & STR$(roundTo)
      PRINT #1, "rV3" & "                       ", STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential"
   END IF

   IF RoundMe4(m, roundTo) <> m2 AND INSTR(STR$(m, 18), "49999") = 0 _
                                 AND INSTR(STR$(m, 18), "4.9999") = 0 THEN
      ? STR$(m, 18) & STR$(RoundMe4(m, roundTo), 18) & STR$(roundTo)
      ? STR$(m, 18) & STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential"
      WAITKEY$
   END IF
  NEXT
  ? "done"
  WAITKEY$
END FUNCTION

[David Roberts]

Hi John

I see that you removed the use of count. I put it back in and am getting about 8000 plus disagreements on a run of 2100000.

> but it matches the other posted algos so far everywhere but the expected regions.

What do you mean by "expected regions"?

There is bug in asm

asm 7.4760000000000004E-15   .000000000001 12
rV3                          0 7476 sequential

[John Gleason]

Here is an example of what I was calling one of the expected regions, where the 499999... rounds down in asmRoundUp but up in roundUpV3:

asm 4.98934999999999141E-13  .00000000000049893 17
rV3                          .00000000000049894 498935 sequential

But I haven't seen one yet like your example posted.

[David Roberts]

Something else...

asm .803475219846933572      .8034752199 10
rV3                          .8034752198 190661 random

With

num=Val(".803475219846933572")
digits = 10
Print ASMRoundUp(num , digits ); Round(num,10)

where 'asm' is in a separate program I get asm & .8034752198 which agrees with rV3.

I'm getting a lot of them - my 8000 plus would be a hell of a lot less otherwise.

I'll look into that.

How many disagreements are you getting, John?

[David Roberts]

> I'll look into that.

asm .803475219846933572 and num=Val(".803475219846933572")

may have different binary representations.

I talked about decimal domain collisions earlier.

[David Roberts]

Had some questions, John, but I've answered them myself.

[Walter Henn]

> I'll look into that.

asm .803475219846933572 and num=Val(".803475219846933572")

may have different binary representations.

I talked about decimal domain collisions earlier.

Even if they did have different binary representations, they would at most differ in the last decimal digit. Even if they differed in the last six digits, both rounded to 10 digits would have to be .8034752198.

[David Roberts]

> Even if they did have different binary representations, they would at most differ in the last decimal digit.

When I talk about collisions, Walter, I'm referring to differing binary representations producing the same decimal representation.

Anyway,

> both rounded to 10 digits would have to be .8034752198.

Yes, they would - collisions would not be occuring at that level.

Unfortunately, on my machine the ASMRoundUp is not behaving in the same way with John's last code as it is with a stand alone written when Paul Dixon posted it. If it did the number of disagreements would plummet aiding a better concentration on them. John has not come back with how many disagreements he is having, if any.