Announcement

Collapse
No announcement yet.

VAL rounds a variable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by John Gleason View Post
    ...
    ...
    Walter is looking for a round function that rounds up on 5, but only if the base2 representation of the "rounding 5" actually is equal to or exceeds 5 internally. Let's call this, "base2 round up on 5", and is what Paul Dixon's code has been based on I believe...

    Walter, how are asmRoundUp/round_dixon looking? Any problems or exceptions there?
    Your statement above regarding what I'm looking for in a roundup on 5 routine and what Paul Dixon's code is based on is correct.

    The asmRoundUp and round_dixon routines have been found to be correct for all test values (in the bilions). Paul Dixon is a remarkable programmer.
    Last edited by Walter Henn; 2 Jul 2008, 07:09 PM.

    Comment


    • Walter,
      you are correct, i was not aware of the fact STR$ rounded, i just always used only str$(x) or str$(x,18).
      thanks for pointing that out.
      so yes, the test you pointed would fail to show differences.
      i going to remove that test.
      p purvis

      Comment


      • Originally posted by paul d purvis View Post
        Unless i am convinced otherwise, yes i do feel it maybe relevant.

        what is the difference between
        local m as ext
        m=16.585##
        and
        local m as ext
        let m=16.585##

        none i do believe

        while it is true that rounding routines are not affected in either way

        the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
        that is why the integer/decimal is giving us a more precise values to round from.
        so understanding the LET statement is critical in my view.
        If the use of the LET statement does not result in any difference when assigning a value to a variable, why would understanding the LET statement be critical??? I believe that this is really muddying the water.

        i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.
        Extracted from PBCC 4.04 Hep File:
        Internally, Currency and Extended-currency numbers are stored as Quad-integers with an implied decimal point (at 4 places for Currency, and at 2 places for Extended-currency). This approach ensures that all of the digits of the variables can be represented exactly.

        Comment


        • Walter,
          good to go on the quad integers
          p purvis

          Comment


          • From post #51 here Steve Rossell said "However, evaluation of numeric literals in your compiled programs are currently limited to double precision."

            What isn't said is that the storage allocated is limited to 64 bits as well. Looking at a OllyDBug output on running an exe it seems to me, unless I am mistaken, that the storage allocation for extended precision numeric literals is still 80 bits but the resolution of a numeric literal is clearly only 64 bits.

            So, if we declare 'Local y as Ext' this will give us our 80 bits of storage at address VarPtr(y).

            I am not sufficiently aquainted with FPU assembly to go any further but I do know a man that is.

            Can we fill this storage with our evaluation, via FPU instructions as opposed to Paul P's fractional method which is currently in the #1 slot, such that ref to 'y' is a ref to a genuine ext and not one that has had its tail cut off - well, rounded actually.

            Added: Obviously, we would not assign 'y' via 'y =' but perhaps read a string at run time. Yeah, I know, VAL does that but a specific VAL bereft of any bells and whistles.
            Last edited by David Roberts; 5 Jul 2008, 04:47 AM.

            Comment


            • Dave, here's a way to do it with just a two tick deficit. I would think hardly anyone would notice two ticks on a 2GHz box, even IN a tight loop.
              Code:
              #COMPILE EXE
              #DIM ALL
              
              FUNCTION PBMAIN () AS LONG
              
                  LOCAL x, y, z AS EXT
                  LOCAL wp AS LONG               'ptr to the actual ext stored data of x
                  x = VAL("1234567890.12345678") 'your full 18 digits into x
                  wp = VARPTR(x)                 'ptr to the 5 words of x (10 bytes)
                  OPEN "c:\hexExtendedPrecision1a.txt" FOR OUTPUT AS #1
              
                  PRINT #1, "!dw &h" &  HEX$(PEEK(WORD, wp    ), 4) & ", &h" & HEX$(PEEK(WORD, wp + 2), 4) & ", &h" & _
                                        HEX$(PEEK(WORD, wp + 4), 4) & ", &h" & HEX$(PEEK(WORD, wp + 6), 4) & ", &h" & _
                                        HEX$(PEEK(WORD, wp + 8), 4)
                  ? "ok, copy the line in c:\hexExtendedPrecision1a.txt and put it in the program."
                  GOTO pastYdata                 'can't let program read thru the x data as if valid operations.
                  extendedYdata:
                  !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D  ;<< copy data here. This represents x (and next, y) in memory
                  pastYdata:
              
                  y = PEEK(EXT, CODEPTR(extendedYdata))   '<< this is now approx 2 ticks slower than...
                  z = 1234567890.12345678##               '<< this, but y will now always be the full EXT
              
                  IF x = y THEN
                     ? "x =" & STR$(x, 18)
                     ? "y =" & STR$(y, 18)
                  ELSE
                     ? "x does not yet = y"
                  END IF
                  WAITKEY$
              END FUNCTION
              Last edited by John Gleason; 5 Jul 2008, 10:10 AM. Reason: filename edit

              Comment


              • Nice one, John. PEEKing an inline data line had not occurred to me - something that I've only done a few times.

                Just out of curiosity I then looked at 'Poke Ext, y, 1234567890.12345678##' and found that, here to, the double precision evaluation got POKEd.

                Comment


                • Later I thought that probably a function example would be a good idea. So here is some code to test below:
                  Code:
                  #COMPILE EXE
                  #DIM ALL
                  
                  FUNCTION speedyExtAssign() AS LONG
                     STATIC a,b,c,d,e AS EXT
                     STATIC ii AS LONG, t AS SINGLE
                      GOTO pastData                 'skip over asm data operations.
                      extendedAdata:
                      !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D  ;<< copy data here. This represents a in memory
                      extendedBdata:
                      !dw &hBB21, &h3F35, &h05A4, &h932C, &h401D  ;b in memory
                      extendedCdata:
                      !dw &hBC21, &h3F35, &h05A4, &h932C, &h401D  ;c
                      extendedDdata:
                      !dw &hBD21, &h3F35, &h05A4, &h932C, &h401D  ;d
                      extendedEdata:
                      !dw &hBE21, &h3F35, &h05A4, &h932C, &h401D  ;e
                      pastData:
                  
                      t = TIMER
                      FOR ii = 1 TO 100000000       '100,000,000 loops
                         a = PEEK(EXT, CODEPTR(extendedAdata))
                         b = PEEK(EXT, CODEPTR(extendedBdata))
                         c = PEEK(EXT, CODEPTR(extendedCdata))
                         d = PEEK(EXT, CODEPTR(extendedDdata))
                         e = PEEK(EXT, CODEPTR(extendedEdata))
                  '      ? STR$(a,18) & STR$(b,18) & STR$(c,18) & STR$(d,18) & STR$(e,18): waitkey$ 'uncomment to look at the values
                      NEXT
                      t = TIMER - t
                      ? "a half billion assignments done in" & STR$(t, 4) & " sec"
                      ? "which is " & FORMAT$(500000000 / t, "0,") & " per sec"
                      ? ""
                      ? "and now using VAL:"
                  
                      t = TIMER
                      FOR ii = 1 TO 3000000         '3,000,000 loops
                         a = VAL("1234567890.12345678")
                      NEXT
                      t = TIMER - t
                      ? "3 million assignments done in" & STR$(t, 4) & " sec"
                      ? "which is " & FORMAT$(3000000 / t, "0,") & " per sec"
                      ? ""
                      ? "and finally division:"
                  
                      t = TIMER
                      FOR ii = 1 TO 20000000        '20,000,000 loops
                         a = 123456789012345678 / 100000000
                         b = 123456789012345681 / 100000000
                         c = 123456789012345684 / 100000000
                         d = 123456789012345687 / 100000000
                         e = 123456789012345690 / 100000000
                      NEXT
                      t = TIMER - t
                      ? "100 million assignments done in" & STR$(t, 4) & " sec"
                      ? "which is " & FORMAT$(100000000 / t, "0,") & " per sec"
                      ? ""
                      ? "so we see division is fast too, some 30 times faster"
                      ? "than VAL for this application, and is much easier to"
                      ? "implement than the asm data statements."
                      ? ""
                      ? "However, one added feature of the asm method is that it"
                      ? "allows entry of literals at ""uber resolution ;)"" of the"
                      ? "full 18.95 (courtesy Dave Roberts) digits available."
                      WAITKEY$
                      
                  
                  END FUNCTION
                  
                  FUNCTION PBMAIN () AS LONG
                      speedyExtAssign
                  END FUNCTION
                  Last edited by John Gleason; 5 Jul 2008, 02:18 PM.

                  Comment


                  • Originally posted by John Gleason View Post

                    ... so we see division is fast too, some 30 times faster than VAL for this application, and is much easier to implement than the asm data statements...
                    I'd like to make a few comments on the integer division approach for literal assignment to EXT variables by Paul Purvis. First of all, John, you stated in your code that this approach is some 30 times faster than the VAL method. I think it's important to point out that this speed increase only applies to the specific test values used in your code (values with 18 significant digits). Often literals as simple as .001 need to be used in code (far more often than 18-digit literals). The speed increase for integer division over VAL for such simple values is on the order of ten times.

                    I think it's also important to keep the speed of conversion from literal to EXT in perspective. Most programs have a limited number of literals that they employ. So, even if the conversion from literal to EXT is on the order of 1 usec, the speed of conversion would have little impact on the time to run the program. The exception, of course, would be the use of a literal in a loop with many iterations or a routine with many calls. However, in such cases, an EXT variable (equal to the value of the literal) should be placed in the loop or routine in place of the literal. This applies whether you're using VAL OR integer division.

                    Finally, I'd like to see Paul's integer division approach be placed also in the thread, "Increase the Precision of Your Floating Point Literals to 18 Decimal Digits."
                    Last edited by Walter Henn; 6 Jul 2008, 07:21 PM.

                    Comment


                    • I think it's also important to keep the speed of conversion from literal to EXT in perspective.
                      Most threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered. For me John Gleason has a wonderful habit of making me think, if not write, "had not occurred to me" when he is posting code to 'squeeze the last ounce'.

                      John used the phrase "uber resolution" with regard the asm data statements.

                      We may have constants such as sqr(2) and prefer to use a numeric literal.

                      Rounding up to 18 digits we have 1.41421356237309505. The binary representation of 141421356237309505/100000000000000000 is 3FFFB504F333F9DE648F. The extended binary representation of sqr(2) ends with ....6484.

                      sqr(2) * sqr(2) - 2 = 0 where sqr(2) is computed with infinite precision.

                      With

                      Code:
                      #Compile Exe
                      #Dim All
                       
                      Function PBMain () As Long
                       
                        Local x, y As Ext
                       
                        x = 141421356237309505/100000000000000000 ' 18 digits
                        'x = 14142135623730950488016887242097/10000000000000000000000000000000 ' 32 digits
                       
                        GoTo pastYdata
                        extendedYdata:
                        !dw &h6484, &hF9DE, &hF333, &hB504, &h3FFF
                        pastYdata:
                       
                        y = Peek(Ext, CodePtr(extendedYdata))
                       
                        ? Str$(Sqr(2)*Sqr(2) - 2,18)
                        ? Str$(y*y - 2,18)
                        ? Str$(x*x - 2,18)
                       
                        WAITKEY$
                       
                      End Function
                      I get

                      -1.08420217248550443E-19
                      -1.08420217248550443E-19
                      3.2526065174565133E-18

                      With 32 digits, letting the system do its best I get

                      -1.08420217248550443E-19
                      -1.08420217248550443E-19
                      -5.63785129692462306E-18

                      Pedantic? This is well 'over the top' for anything that I do and may be so for most but perhaps one or more want that last ounce of precision.

                      It is worth remembering that we may be looking for 6 significant figure accuracy, say, in our final results but require many more significant figure accuracy for interim calculations because of accumulation error. I reckon that some of the jobs done by super computers where they are on their knees for days on end take 'so long' not just because of the amount of data being crunched but also because of the accuracy demanded of the interim calculations.

                      Added: Of course, we could use x = sqr(2) - a bad choice. How about pi, e and such or even a computation done elsewhere where we have access to the binary representation before it got 'corrupted' in its transition to the decimal domain.
                      Last edited by David Roberts; 6 Jul 2008, 10:47 AM.

                      Comment


                      • Originally posted by Walter Henn View Post
                        I think it's also important to keep the speed of conversion from literal to EXT in perspective...
                        Originally posted by David Roberts View Post
                        Most threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered...
                        I couldn't agree with you more, David. I have participated extensively in this thread not because of the importance to me of rounding (frankly, I'm perfectly satisfied with banker's rounding), but because I have learned valuable new concepts (visual rounding is not one of them) or been refreshed with old concepts.
                        Last edited by Walter Henn; 7 Jul 2008, 05:05 AM.

                        Comment


                        • frankly, I'm perfectly satisfied with banker's rounding
                          So am I.

                          ...but who knows, one day we just might want a round-up and Paul Dixon has provided us with that or we just might want a numeric literal to go the distance and until PB give it back to us we have Paul Purvis to thank for his integer division method.

                          With over 4700 views of this thread I'd reckon one or two are using one or both of the above and have been for some days.

                          Comment


                          • My dern internet connection lately has been "sketchy" at best. There's nothing like keeping current by posting once every 2 or 3 days.

                            Thanks for the comments Dave, and be assured I view your posts with top shelf regard, especially those eased back to my level of understanding. Like you and Walter said, we may have moved into the theoretical, but usually that precedes finding some practicality.

                            Speaking of which (optimistically hoping it's practical) I worked a bit more and tested extensively the "visual rounding" algo below. I think it's value is that it rounds as you would expect by just looking at the decimal numbers, as if the computer were a decimal computer. And now with all the testing techniques that have been posted and I've used, I'm way more confident of its accuracy, and that I've tested all classes of input. This doesn't prove it correct--indeed, I'd tested my original incorrect roundUp algo with billions of values and still missed a whole class of numbers--but it matches the other posted algos so far everywhere but the expected regions.

                            Code:
                            'compiled with pbcc 4.04
                            'roundUpV3.bas
                            
                            #COMPILE EXE
                            #DIM ALL
                            #REGISTER NONE
                            
                            FUNCTION ASMRoundUp(num AS EXT, digits AS LONG) AS EXT
                            '################################################################
                            '# This is an ASM version of:
                            '# temp = 10^number_rounding_digits
                            '# FUNCTION = SGN(number)*INT(ABS(number*temp) + 0.5##)/temp
                            '#
                            '# note: Parameters are passed by reference as it's faster.
                            '# if this is changed then the code must be altered to match
                            '################################################################
                            
                            LOCAL OldControlWord AS INTEGER
                            
                            !jmp skip                  'jump over the following lookup tables
                            table:
                            'this is just a lookup table of all the powers of 10 from 0 to 18
                            !dd &h1,0                  '1
                            !dd &hA,0                  '10
                            !dd &h64,0                 '100
                            !dd &h3E8,0                '1000
                            !dd &h2710,0               '10000
                            !dd &h186A0,0              '100000
                            !dd &hF4240,0              '1000000
                            !dd &h989680,0             '10000000
                            !dd &h5F5E100,0            '100000000
                            !dd &h3B9ACA00,0           '1000000000
                            !dd &h540BE400,&h2         '10000000000
                            !dd &h4876E800,&h17        '100000000000
                            !dd &hD4A51000,&hE8        '1000000000000
                            !dd &h4E72A000,&h918       '10000000000000
                            !dd &h107A4000,&h5AF3      '100000000000000
                            !dd &hA4C68000,&h38D7E     '1000000000000000
                            !dd &h6FC10000,&h2386F2    '10000000000000000
                            !dd &h5D8A0000,&h1634578   '100000000000000000
                            !dd &hA7640000,&hDE0B6B3   '1000000000000000000
                            !dd &h89E80000,&h8AC72304  '10000000000000000000
                            !dd &h63100000,&h6BC75E2D  '100000000000000000000
                            half:
                            !dd &h3f000000             'the number 0.5 in single format
                            FPControlWordRoundToZeroEXT:
                            !dw &h01F3F                'control word to set FPU to round toward zero with extended precision
                            
                            
                            skip:
                            !mov eax,digits                     'look up the scale required for this many digits
                            !mov eax,[eax]                      'must do this as digits is a function parameter so I get the address, not the value
                            !fild qword ptr table[eax*8]        'get the scale factor
                            !mov eax,num##                      'again, got the address of num## as it's a function parameter
                            !fld tbyte ptr [eax]                'load num##
                            !fmul st(0),st(1)                   'num## * temp
                            !fld dword ptr half                 'get the number 0.5
                            !mov ecx,79
                            !bt [eax],ecx                       'check the sign of num##
                            !jnc skp
                            !fchs                               'if num -ve then make 0.5 -ve too
                            skp:
                            !faddp st(1),st(0)                  'num## * temp  + 0.5
                            !fstcw OldControlWord               'save original control word
                            !fldcw FPControlWordRoundToZeroEXT  'change FPU control word to round toward zero
                            !frndint                            'round to nearest integer -> INT(num## * temp  + 0.5)
                            !fldcw OldControlWord               'restore original control word
                            !fdivrp st(1),st(0)                 'INT(num## * temp  + 0.5) / temp
                            !fstp function                      'save answer
                            
                            END FUNCTION
                            
                            MACRO FUNCTION mfRandomLong()
                                !mov eax, mwcSoph
                                !mov ecx, mwcRandom
                                !mul ecx
                                !add eax, mwcCarry
                                !adc edx, 0
                                !mov mwcRandom, eax
                                !mov mwcCarry,  edx
                            END MACRO = mwcRandom
                            
                            FUNCTION extRnd() AS EXT
                            '---------------------------------------------------------------------------
                            'creates an EXT pseudo-rnd test value from -0.9999... to +0.9999... where each
                            'of the 18 digits are statistically random. Period is ~2^61 (2*10^18) EXT values
                            'before repeating--many years.
                            '---------------------------------------------------------------------------
                              STATIC x, p5 AS EXT, xp, oneTime AS LONG
                              STATIC mwcSoph, mwcRandom, mwcCarry AS LONG
                            
                              IF oneTime = 0 THEN
                                 oneTime = 1             'this section only needed once per program run.
                                 x = 999999999999999999 / 1000000000000000000
                                 p5 = 5 / 10
                                 xp = VARPTR(x)
                                 mwcSoph   = &h68131E4B  'fixed sg-prime constant. Leave intact
                                 mwcRandom = &ha5b218da  'user seed1, can be any LONG > 1
                                 mwcCarry  = &h3fe8700c  'user seed2, any positive LONG less than &h68131E4B (mwcSoph) and > 1
                                 !dw &h310f              ;time stamp counter to vary seed1
                                 !add mwcRandom, eax     ;vary seed1
                                 mwcCarry = mwcCarry + TIMER * 18 'and vary seed2 a bit.
                              END IF
                            
                                 POKE LONG, xp, mfRandomLong
                                 POKE LONG, xp+4, mfRandomLong OR &h080000000
                                 IF mfRandomLong < 0  THEN
                                    IF mfRandomLong < 0 THEN
                                       FUNCTION = -(x - p5)
                                    ELSE
                                       FUNCTION =  (x - p5)
                                    END IF
                                 ELSE
                                    IF mfRandomLong < 0 THEN
                                       FUNCTION = -x
                                    ELSE
                                       FUNCTION =  x
                                    END IF
                                 END IF
                                 EXIT FUNCTION
                            
                            END FUNCTION
                            
                            FUNCTION roundme4(BYREF X AS EXT, BYREF Factor AS LONG) AS EXT
                            LOCAL MM AS EXT
                            LOCAL additional AS LONG
                            additional=factor+2&
                            mm=x+VAL(MID$("."+REPEAT$(additional+3,"0"),1,additional+3)+"1")
                            FUNCTION=ROUND(mm,factor)
                            END FUNCTION
                            
                            FUNCTION roundUpV3(a AS EXT, b AS LONG) AS EXT
                            '---------------------------------------------------------------------------------------
                            'rounds "a" to "b" digits past decimal point, but up on 5 instead of using
                            'banker's rounding. It rounds as you would expect by just looking at the decimal numbers,
                            'as if the computer were a decimal computer. eg. roundUpV3(123.4545, 3) becomes 123.455.
                            '---------------------------------------------------------------------------------------
                               STATIC x AS EXT
                               STATIC qa AS QUAD
                               STATIC fillOnce, sizeA AS LONG
                               DIM ten(18) AS STATIC QUAD
                            
                               IF a < 0 THEN         'handle negatives
                                  x = ROUND(a, b)    'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding
                                  IF x <= a THEN     'rounded up, so no need to go thru hoops.
                                     FUNCTION = x
                                     EXIT FUNCTION
                                  END IF
                               ELSE
                                  x = ROUND(a, b)    'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding
                                  IF x >= a THEN     'rounded up
                                     FUNCTION = x
                                     EXIT FUNCTION
                                  END IF
                               END IF
                                                     'make powers of 10 to avoid exponent calcs
                               IF fillOnce = 0 THEN
                               ten(00) = 1
                               ten(01) = 10
                               ten(02) = 100
                               ten(03) = 1000
                               ten(04) = 10000
                               ten(05) = 100000
                               ten(06) = 1000000
                               ten(07) = 10000000
                               ten(08) = 100000000
                               ten(09) = 1000000000
                               ten(10) = 10000000000
                               ten(11) = 100000000000
                               ten(12) = 1000000000000
                               ten(13) = 10000000000000
                               ten(14) = 100000000000000
                               ten(15) = 1000000000000000
                               ten(16) = 10000000000000000
                               ten(17) = 100000000000000000
                               ten(18) = 1000000000000000000
                               fillOnce = 1
                               END IF
                                                    'how big is a?
                               SELECT CASE ABS(a)
                                  CASE < ten(00): sizeA = 01: qa = a * ten(18)
                                  CASE < ten(01): sizeA = 02: qa = a * ten(17)
                                  CASE < ten(02): sizeA = 03: qa = a * ten(16)
                                  CASE < ten(03): sizeA = 04: qa = a * ten(15)
                                  CASE < ten(04): sizeA = 05: qa = a * ten(14)
                                  CASE < ten(05): sizeA = 06: qa = a * ten(13)
                                  CASE < ten(06): sizeA = 07: qa = a * ten(12)
                                  CASE < ten(07): sizeA = 08: qa = a * ten(11)
                                  CASE < ten(08): sizeA = 09: qa = a * ten(10)
                                  CASE < ten(09): sizeA = 10: qa = a * ten(09)
                                  CASE < ten(10): sizeA = 11: qa = a * ten(08)
                                  CASE < ten(11): sizeA = 12: qa = a * ten(07)
                                  CASE < ten(12): sizeA = 13: qa = a * ten(06)
                                  CASE < ten(13): sizeA = 14: qa = a * ten(05)
                                  CASE < ten(14): sizeA = 15: qa = a * ten(04)
                                  CASE < ten(15): sizeA = 16: qa = a * ten(03)
                                  CASE < ten(16): sizeA = 17: qa = a * ten(02)
                                  CASE < ten(17): sizeA = 18: qa = a * ten(01)
                                  CASE ELSE
                                     FUNCTION = a
                                     EXIT FUNCTION             'exit because a is too big to round
                               END SELECT
                               IF sizeA + b > 18 THEN
                                  FUNCTION = a
                                  EXIT FUNCTION                'exit because round digits too big
                               END IF
                            
                               qa = qa \ ten(18 - (sizeA + b)) 'make whole EXT effectively an integer
                               sizeA = qa MOD 10
                               IF sizeA = 5 THEN               'is last digit a 5?
                                  FUNCTION = ((qa + 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position
                               ELSEIF sizeA = -5 THEN          'or -5
                                  FUNCTION = ((qa - 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position
                               ELSE
                                  FUNCTION = x
                               END IF
                            
                            END FUNCTION
                            
                            FUNCTION PBMAIN () AS LONG
                            '---------------------------------------------------------------------------------------
                            'this prints out differences between asmRoundUp and roundUpV3 but stops if a difference is
                            'found between roundMe4 and roundUpV3, which should always match but for the known "49999"
                            'difference. It tests both random and sequential series.
                            '---------------------------------------------------------------------------------------
                            LOCAL M, n, m2, incrTest AS EXT
                            LOCAL roundto, numDig, count, ii, tLoop AS LONG
                              OPEN "c:\RoundV3asmDiff.txt" FOR APPEND AS #1
                            
                              incrTest = 1 / 1000000000000000000
                              ? "rounding..."
                              FOR ii = 1 TO 2100000'000
                               n = extRnd
                               roundTo = ABS(extRnd * 1000000000000000000 MOD 17) 'round from 0 to 17 digits
                               IF (ii AND &h07ffff) = 0 THEN PRINT ii
                               IF asmRoundUp(n, roundTo) <> roundUpV3(n, roundTo) THEN
                                  PRINT #1, "asm" & STR$(n, 18), STR$(asmRoundUp(n, roundTo), 18) & STR$(roundTo)
                                  PRINT #1, "rV3" & "                       ", STR$(roundUpV3(n, roundTo), 18) & STR$(ii) & " random"
                               END IF
                              m = m + incrTest
                              roundTo = ABS(extRnd * 1000000000 AND 7) + 10       'round from 10 to 17 digits
                              m2 = roundUpV3(m, roundTo)
                               IF asmRoundUp(m, roundTo) <> m2 THEN
                                  PRINT #1, "asm" & STR$(m, 18), STR$(asmRoundUp(m, roundTo), 18) & STR$(roundTo)
                                  PRINT #1, "rV3" & "                       ", STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential"
                               END IF
                            
                               IF RoundMe4(m, roundTo) <> m2 AND INSTR(STR$(m, 18), "49999") = 0 _
                                                             AND INSTR(STR$(m, 18), "4.9999") = 0 THEN
                                  ? STR$(m, 18) & STR$(RoundMe4(m, roundTo), 18) & STR$(roundTo)
                                  ? STR$(m, 18) & STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential"
                                  WAITKEY$
                               END IF
                              NEXT
                              ? "done"
                              WAITKEY$
                            END FUNCTION

                            Comment


                            • Hi John

                              I see that you removed the use of count. I put it back in and am getting about 8000 plus disagreements on a run of 2100000.

                              > but it matches the other posted algos so far everywhere but the expected regions.

                              What do you mean by "expected regions"?

                              There is bug in asm
                              Code:
                              asm 7.4760000000000004E-15   .000000000001 12
                              rV3                          0 7476 sequential

                              Comment


                              • Here is an example of what I was calling one of the expected regions, where the 499999... rounds down in asmRoundUp but up in roundUpV3:
                                Code:
                                asm 4.98934999999999141E-13  .00000000000049893 17
                                rV3                          .00000000000049894 498935 sequential
                                But I haven't seen one yet like your example posted.

                                Comment


                                • Something else...

                                  Code:
                                  asm .803475219846933572      .8034752199 10
                                  rV3                          .8034752198 190661 random
                                  With
                                  Code:
                                  num=Val(".803475219846933572")
                                  digits = 10
                                  Print ASMRoundUp(num , digits ); Round(num,10)
                                  where 'asm' is in a separate program I get asm & .8034752198 which agrees with rV3.

                                  I'm getting a lot of them - my 8000 plus would be a hell of a lot less otherwise.

                                  I'll look into that.

                                  How many disagreements are you getting, John?

                                  Comment


                                  • > I'll look into that.

                                    asm .803475219846933572 and num=Val(".803475219846933572")

                                    may have different binary representations.

                                    I talked about decimal domain collisions earlier.

                                    Comment


                                    • Had some questions, John, but I've answered them myself.
                                      Last edited by David Roberts; 7 Jul 2008, 07:08 PM.

                                      Comment


                                      • Originally posted by David Roberts View Post
                                        > I'll look into that.

                                        asm .803475219846933572 and num=Val(".803475219846933572")

                                        may have different binary representations.

                                        I talked about decimal domain collisions earlier.
                                        Even if they did have different binary representations, they would at most differ in the last decimal digit. Even if they differed in the last six digits, both rounded to 10 digits would have to be .8034752198.
                                        Last edited by Walter Henn; 7 Jul 2008, 10:03 PM.

                                        Comment


                                        • > Even if they did have different binary representations, they would at most differ in the last decimal digit.

                                          When I talk about collisions, Walter, I'm referring to differing binary representations producing the same decimal representation.

                                          Anyway,

                                          > both rounded to 10 digits would have to be .8034752198.

                                          Yes, they would - collisions would not be occuring at that level.

                                          Unfortunately, on my machine the ASMRoundUp is not behaving in the same way with John's last code as it is with a stand alone written when Paul Dixon posted it. If it did the number of disagreements would plummet aiding a better concentration on them. John has not come back with how many disagreements he is having, if any.

                                          Comment

                                          Working...
                                          X