Announcement

Collapse
No announcement yet.

VAL rounds a variable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • David Roberts
    replied
    > I'll look into that.

    asm .803475219846933572 and num=Val(".803475219846933572")

    may have different binary representations.

    I talked about decimal domain collisions earlier.

    Leave a comment:


  • David Roberts
    replied
    Something else...

    Code:
    asm .803475219846933572      .8034752199 10
    rV3                          .8034752198 190661 random
    With
    Code:
    num=Val(".803475219846933572")
    digits = 10
    Print ASMRoundUp(num , digits ); Round(num,10)
    where 'asm' is in a separate program I get asm & .8034752198 which agrees with rV3.

    I'm getting a lot of them - my 8000 plus would be a hell of a lot less otherwise.

    I'll look into that.

    How many disagreements are you getting, John?

    Leave a comment:


  • John Gleason
    replied
    Here is an example of what I was calling one of the expected regions, where the 499999... rounds down in asmRoundUp but up in roundUpV3:
    Code:
    asm 4.98934999999999141E-13  .00000000000049893 17
    rV3                          .00000000000049894 498935 sequential
    But I haven't seen one yet like your example posted.

    Leave a comment:


  • David Roberts
    replied
    Hi John

    I see that you removed the use of count. I put it back in and am getting about 8000 plus disagreements on a run of 2100000.

    > but it matches the other posted algos so far everywhere but the expected regions.

    What do you mean by "expected regions"?

    There is bug in asm
    Code:
    asm 7.4760000000000004E-15   .000000000001 12
    rV3                          0 7476 sequential

    Leave a comment:


  • John Gleason
    replied
    My dern internet connection lately has been "sketchy" at best. There's nothing like keeping current by posting once every 2 or 3 days.

    Thanks for the comments Dave, and be assured I view your posts with top shelf regard, especially those eased back to my level of understanding. Like you and Walter said, we may have moved into the theoretical, but usually that precedes finding some practicality.

    Speaking of which (optimistically hoping it's practical) I worked a bit more and tested extensively the "visual rounding" algo below. I think it's value is that it rounds as you would expect by just looking at the decimal numbers, as if the computer were a decimal computer. And now with all the testing techniques that have been posted and I've used, I'm way more confident of its accuracy, and that I've tested all classes of input. This doesn't prove it correct--indeed, I'd tested my original incorrect roundUp algo with billions of values and still missed a whole class of numbers--but it matches the other posted algos so far everywhere but the expected regions.

    Code:
    'compiled with pbcc 4.04
    'roundUpV3.bas
    
    #COMPILE EXE
    #DIM ALL
    #REGISTER NONE
    
    FUNCTION ASMRoundUp(num AS EXT, digits AS LONG) AS EXT
    '################################################################
    '# This is an ASM version of:
    '# temp = 10^number_rounding_digits
    '# FUNCTION = SGN(number)*INT(ABS(number*temp) + 0.5##)/temp
    '#
    '# note: Parameters are passed by reference as it's faster.
    '# if this is changed then the code must be altered to match
    '################################################################
    
    LOCAL OldControlWord AS INTEGER
    
    !jmp skip                  'jump over the following lookup tables
    table:
    'this is just a lookup table of all the powers of 10 from 0 to 18
    !dd &h1,0                  '1
    !dd &hA,0                  '10
    !dd &h64,0                 '100
    !dd &h3E8,0                '1000
    !dd &h2710,0               '10000
    !dd &h186A0,0              '100000
    !dd &hF4240,0              '1000000
    !dd &h989680,0             '10000000
    !dd &h5F5E100,0            '100000000
    !dd &h3B9ACA00,0           '1000000000
    !dd &h540BE400,&h2         '10000000000
    !dd &h4876E800,&h17        '100000000000
    !dd &hD4A51000,&hE8        '1000000000000
    !dd &h4E72A000,&h918       '10000000000000
    !dd &h107A4000,&h5AF3      '100000000000000
    !dd &hA4C68000,&h38D7E     '1000000000000000
    !dd &h6FC10000,&h2386F2    '10000000000000000
    !dd &h5D8A0000,&h1634578   '100000000000000000
    !dd &hA7640000,&hDE0B6B3   '1000000000000000000
    !dd &h89E80000,&h8AC72304  '10000000000000000000
    !dd &h63100000,&h6BC75E2D  '100000000000000000000
    half:
    !dd &h3f000000             'the number 0.5 in single format
    FPControlWordRoundToZeroEXT:
    !dw &h01F3F                'control word to set FPU to round toward zero with extended precision
    
    
    skip:
    !mov eax,digits                     'look up the scale required for this many digits
    !mov eax,[eax]                      'must do this as digits is a function parameter so I get the address, not the value
    !fild qword ptr table[eax*8]        'get the scale factor
    !mov eax,num##                      'again, got the address of num## as it's a function parameter
    !fld tbyte ptr [eax]                'load num##
    !fmul st(0),st(1)                   'num## * temp
    !fld dword ptr half                 'get the number 0.5
    !mov ecx,79
    !bt [eax],ecx                       'check the sign of num##
    !jnc skp
    !fchs                               'if num -ve then make 0.5 -ve too
    skp:
    !faddp st(1),st(0)                  'num## * temp  + 0.5
    !fstcw OldControlWord               'save original control word
    !fldcw FPControlWordRoundToZeroEXT  'change FPU control word to round toward zero
    !frndint                            'round to nearest integer -> INT(num## * temp  + 0.5)
    !fldcw OldControlWord               'restore original control word
    !fdivrp st(1),st(0)                 'INT(num## * temp  + 0.5) / temp
    !fstp function                      'save answer
    
    END FUNCTION
    
    MACRO FUNCTION mfRandomLong()
        !mov eax, mwcSoph
        !mov ecx, mwcRandom
        !mul ecx
        !add eax, mwcCarry
        !adc edx, 0
        !mov mwcRandom, eax
        !mov mwcCarry,  edx
    END MACRO = mwcRandom
    
    FUNCTION extRnd() AS EXT
    '---------------------------------------------------------------------------
    'creates an EXT pseudo-rnd test value from -0.9999... to +0.9999... where each
    'of the 18 digits are statistically random. Period is ~2^61 (2*10^18) EXT values
    'before repeating--many years.
    '---------------------------------------------------------------------------
      STATIC x, p5 AS EXT, xp, oneTime AS LONG
      STATIC mwcSoph, mwcRandom, mwcCarry AS LONG
    
      IF oneTime = 0 THEN
         oneTime = 1             'this section only needed once per program run.
         x = 999999999999999999 / 1000000000000000000
         p5 = 5 / 10
         xp = VARPTR(x)
         mwcSoph   = &h68131E4B  'fixed sg-prime constant. Leave intact
         mwcRandom = &ha5b218da  'user seed1, can be any LONG > 1
         mwcCarry  = &h3fe8700c  'user seed2, any positive LONG less than &h68131E4B (mwcSoph) and > 1
         !dw &h310f              ;time stamp counter to vary seed1
         !add mwcRandom, eax     ;vary seed1
         mwcCarry = mwcCarry + TIMER * 18 'and vary seed2 a bit.
      END IF
    
         POKE LONG, xp, mfRandomLong
         POKE LONG, xp+4, mfRandomLong OR &h080000000
         IF mfRandomLong < 0  THEN
            IF mfRandomLong < 0 THEN
               FUNCTION = -(x - p5)
            ELSE
               FUNCTION =  (x - p5)
            END IF
         ELSE
            IF mfRandomLong < 0 THEN
               FUNCTION = -x
            ELSE
               FUNCTION =  x
            END IF
         END IF
         EXIT FUNCTION
    
    END FUNCTION
    
    FUNCTION roundme4(BYREF X AS EXT, BYREF Factor AS LONG) AS EXT
    LOCAL MM AS EXT
    LOCAL additional AS LONG
    additional=factor+2&
    mm=x+VAL(MID$("."+REPEAT$(additional+3,"0"),1,additional+3)+"1")
    FUNCTION=ROUND(mm,factor)
    END FUNCTION
    
    FUNCTION roundUpV3(a AS EXT, b AS LONG) AS EXT
    '---------------------------------------------------------------------------------------
    'rounds "a" to "b" digits past decimal point, but up on 5 instead of using
    'banker's rounding. It rounds as you would expect by just looking at the decimal numbers,
    'as if the computer were a decimal computer. eg. roundUpV3(123.4545, 3) becomes 123.455.
    '---------------------------------------------------------------------------------------
       STATIC x AS EXT
       STATIC qa AS QUAD
       STATIC fillOnce, sizeA AS LONG
       DIM ten(18) AS STATIC QUAD
    
       IF a < 0 THEN         'handle negatives
          x = ROUND(a, b)    'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding
          IF x <= a THEN     'rounded up, so no need to go thru hoops.
             FUNCTION = x
             EXIT FUNCTION
          END IF
       ELSE
          x = ROUND(a, b)    'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding
          IF x >= a THEN     'rounded up
             FUNCTION = x
             EXIT FUNCTION
          END IF
       END IF
                             'make powers of 10 to avoid exponent calcs
       IF fillOnce = 0 THEN
       ten(00) = 1
       ten(01) = 10
       ten(02) = 100
       ten(03) = 1000
       ten(04) = 10000
       ten(05) = 100000
       ten(06) = 1000000
       ten(07) = 10000000
       ten(08) = 100000000
       ten(09) = 1000000000
       ten(10) = 10000000000
       ten(11) = 100000000000
       ten(12) = 1000000000000
       ten(13) = 10000000000000
       ten(14) = 100000000000000
       ten(15) = 1000000000000000
       ten(16) = 10000000000000000
       ten(17) = 100000000000000000
       ten(18) = 1000000000000000000
       fillOnce = 1
       END IF
                            'how big is a?
       SELECT CASE ABS(a)
          CASE < ten(00): sizeA = 01: qa = a * ten(18)
          CASE < ten(01): sizeA = 02: qa = a * ten(17)
          CASE < ten(02): sizeA = 03: qa = a * ten(16)
          CASE < ten(03): sizeA = 04: qa = a * ten(15)
          CASE < ten(04): sizeA = 05: qa = a * ten(14)
          CASE < ten(05): sizeA = 06: qa = a * ten(13)
          CASE < ten(06): sizeA = 07: qa = a * ten(12)
          CASE < ten(07): sizeA = 08: qa = a * ten(11)
          CASE < ten(08): sizeA = 09: qa = a * ten(10)
          CASE < ten(09): sizeA = 10: qa = a * ten(09)
          CASE < ten(10): sizeA = 11: qa = a * ten(08)
          CASE < ten(11): sizeA = 12: qa = a * ten(07)
          CASE < ten(12): sizeA = 13: qa = a * ten(06)
          CASE < ten(13): sizeA = 14: qa = a * ten(05)
          CASE < ten(14): sizeA = 15: qa = a * ten(04)
          CASE < ten(15): sizeA = 16: qa = a * ten(03)
          CASE < ten(16): sizeA = 17: qa = a * ten(02)
          CASE < ten(17): sizeA = 18: qa = a * ten(01)
          CASE ELSE
             FUNCTION = a
             EXIT FUNCTION             'exit because a is too big to round
       END SELECT
       IF sizeA + b > 18 THEN
          FUNCTION = a
          EXIT FUNCTION                'exit because round digits too big
       END IF
    
       qa = qa \ ten(18 - (sizeA + b)) 'make whole EXT effectively an integer
       sizeA = qa MOD 10
       IF sizeA = 5 THEN               'is last digit a 5?
          FUNCTION = ((qa + 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position
       ELSEIF sizeA = -5 THEN          'or -5
          FUNCTION = ((qa - 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position
       ELSE
          FUNCTION = x
       END IF
    
    END FUNCTION
    
    FUNCTION PBMAIN () AS LONG
    '---------------------------------------------------------------------------------------
    'this prints out differences between asmRoundUp and roundUpV3 but stops if a difference is
    'found between roundMe4 and roundUpV3, which should always match but for the known "49999"
    'difference. It tests both random and sequential series.
    '---------------------------------------------------------------------------------------
    LOCAL M, n, m2, incrTest AS EXT
    LOCAL roundto, numDig, count, ii, tLoop AS LONG
      OPEN "c:\RoundV3asmDiff.txt" FOR APPEND AS #1
    
      incrTest = 1 / 1000000000000000000
      ? "rounding..."
      FOR ii = 1 TO 2100000'000
       n = extRnd
       roundTo = ABS(extRnd * 1000000000000000000 MOD 17) 'round from 0 to 17 digits
       IF (ii AND &h07ffff) = 0 THEN PRINT ii
       IF asmRoundUp(n, roundTo) <> roundUpV3(n, roundTo) THEN
          PRINT #1, "asm" & STR$(n, 18), STR$(asmRoundUp(n, roundTo), 18) & STR$(roundTo)
          PRINT #1, "rV3" & "                       ", STR$(roundUpV3(n, roundTo), 18) & STR$(ii) & " random"
       END IF
      m = m + incrTest
      roundTo = ABS(extRnd * 1000000000 AND 7) + 10       'round from 10 to 17 digits
      m2 = roundUpV3(m, roundTo)
       IF asmRoundUp(m, roundTo) <> m2 THEN
          PRINT #1, "asm" & STR$(m, 18), STR$(asmRoundUp(m, roundTo), 18) & STR$(roundTo)
          PRINT #1, "rV3" & "                       ", STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential"
       END IF
    
       IF RoundMe4(m, roundTo) <> m2 AND INSTR(STR$(m, 18), "49999") = 0 _
                                     AND INSTR(STR$(m, 18), "4.9999") = 0 THEN
          ? STR$(m, 18) & STR$(RoundMe4(m, roundTo), 18) & STR$(roundTo)
          ? STR$(m, 18) & STR$(roundUpV3(m, roundTo), 18) & STR$(ii) & " sequential"
          WAITKEY$
       END IF
      NEXT
      ? "done"
      WAITKEY$
    END FUNCTION

    Leave a comment:


  • David Roberts
    replied
    frankly, I'm perfectly satisfied with banker's rounding
    So am I.

    ...but who knows, one day we just might want a round-up and Paul Dixon has provided us with that or we just might want a numeric literal to go the distance and until PB give it back to us we have Paul Purvis to thank for his integer division method.

    With over 4700 views of this thread I'd reckon one or two are using one or both of the above and have been for some days.

    Leave a comment:


  • Walter Henn
    replied
    Originally posted by Walter Henn View Post
    I think it's also important to keep the speed of conversion from literal to EXT in perspective...
    Originally posted by David Roberts View Post
    Most threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered...
    I couldn't agree with you more, David. I have participated extensively in this thread not because of the importance to me of rounding (frankly, I'm perfectly satisfied with banker's rounding), but because I have learned valuable new concepts (visual rounding is not one of them) or been refreshed with old concepts.
    Last edited by Walter Henn; 7 Jul 2008, 05:05 AM.

    Leave a comment:


  • David Roberts
    replied
    I think it's also important to keep the speed of conversion from literal to EXT in perspective.
    Most threads of this nature usually end with such a comment and many such threads have eventually entered into the domain of diminishing returns. I think that most of us are mindful of this but what I often find is the techniques used to 'squeeze the last ounce' are useful in different contexts that I may not have considered. For me John Gleason has a wonderful habit of making me think, if not write, "had not occurred to me" when he is posting code to 'squeeze the last ounce'.

    John used the phrase "uber resolution" with regard the asm data statements.

    We may have constants such as sqr(2) and prefer to use a numeric literal.

    Rounding up to 18 digits we have 1.41421356237309505. The binary representation of 141421356237309505/100000000000000000 is 3FFFB504F333F9DE648F. The extended binary representation of sqr(2) ends with ....6484.

    sqr(2) * sqr(2) - 2 = 0 where sqr(2) is computed with infinite precision.

    With

    Code:
    #Compile Exe
    #Dim All
     
    Function PBMain () As Long
     
      Local x, y As Ext
     
      x = 141421356237309505/100000000000000000 ' 18 digits
      'x = 14142135623730950488016887242097/10000000000000000000000000000000 ' 32 digits
     
      GoTo pastYdata
      extendedYdata:
      !dw &h6484, &hF9DE, &hF333, &hB504, &h3FFF
      pastYdata:
     
      y = Peek(Ext, CodePtr(extendedYdata))
     
      ? Str$(Sqr(2)*Sqr(2) - 2,18)
      ? Str$(y*y - 2,18)
      ? Str$(x*x - 2,18)
     
      WAITKEY$
     
    End Function
    I get

    -1.08420217248550443E-19
    -1.08420217248550443E-19
    3.2526065174565133E-18

    With 32 digits, letting the system do its best I get

    -1.08420217248550443E-19
    -1.08420217248550443E-19
    -5.63785129692462306E-18

    Pedantic? This is well 'over the top' for anything that I do and may be so for most but perhaps one or more want that last ounce of precision.

    It is worth remembering that we may be looking for 6 significant figure accuracy, say, in our final results but require many more significant figure accuracy for interim calculations because of accumulation error. I reckon that some of the jobs done by super computers where they are on their knees for days on end take 'so long' not just because of the amount of data being crunched but also because of the accuracy demanded of the interim calculations.

    Added: Of course, we could use x = sqr(2) - a bad choice. How about pi, e and such or even a computation done elsewhere where we have access to the binary representation before it got 'corrupted' in its transition to the decimal domain.
    Last edited by David Roberts; 6 Jul 2008, 10:47 AM.

    Leave a comment:


  • Walter Henn
    replied
    Originally posted by John Gleason View Post

    ... so we see division is fast too, some 30 times faster than VAL for this application, and is much easier to implement than the asm data statements...
    I'd like to make a few comments on the integer division approach for literal assignment to EXT variables by Paul Purvis. First of all, John, you stated in your code that this approach is some 30 times faster than the VAL method. I think it's important to point out that this speed increase only applies to the specific test values used in your code (values with 18 significant digits). Often literals as simple as .001 need to be used in code (far more often than 18-digit literals). The speed increase for integer division over VAL for such simple values is on the order of ten times.

    I think it's also important to keep the speed of conversion from literal to EXT in perspective. Most programs have a limited number of literals that they employ. So, even if the conversion from literal to EXT is on the order of 1 usec, the speed of conversion would have little impact on the time to run the program. The exception, of course, would be the use of a literal in a loop with many iterations or a routine with many calls. However, in such cases, an EXT variable (equal to the value of the literal) should be placed in the loop or routine in place of the literal. This applies whether you're using VAL OR integer division.

    Finally, I'd like to see Paul's integer division approach be placed also in the thread, "Increase the Precision of Your Floating Point Literals to 18 Decimal Digits."
    Last edited by Walter Henn; 6 Jul 2008, 07:21 PM.

    Leave a comment:


  • John Gleason
    replied
    Later I thought that probably a function example would be a good idea. So here is some code to test below:
    Code:
    #COMPILE EXE
    #DIM ALL
    
    FUNCTION speedyExtAssign() AS LONG
       STATIC a,b,c,d,e AS EXT
       STATIC ii AS LONG, t AS SINGLE
        GOTO pastData                 'skip over asm data operations.
        extendedAdata:
        !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D  ;<< copy data here. This represents a in memory
        extendedBdata:
        !dw &hBB21, &h3F35, &h05A4, &h932C, &h401D  ;b in memory
        extendedCdata:
        !dw &hBC21, &h3F35, &h05A4, &h932C, &h401D  ;c
        extendedDdata:
        !dw &hBD21, &h3F35, &h05A4, &h932C, &h401D  ;d
        extendedEdata:
        !dw &hBE21, &h3F35, &h05A4, &h932C, &h401D  ;e
        pastData:
    
        t = TIMER
        FOR ii = 1 TO 100000000       '100,000,000 loops
           a = PEEK(EXT, CODEPTR(extendedAdata))
           b = PEEK(EXT, CODEPTR(extendedBdata))
           c = PEEK(EXT, CODEPTR(extendedCdata))
           d = PEEK(EXT, CODEPTR(extendedDdata))
           e = PEEK(EXT, CODEPTR(extendedEdata))
    '      ? STR$(a,18) & STR$(b,18) & STR$(c,18) & STR$(d,18) & STR$(e,18): waitkey$ 'uncomment to look at the values
        NEXT
        t = TIMER - t
        ? "a half billion assignments done in" & STR$(t, 4) & " sec"
        ? "which is " & FORMAT$(500000000 / t, "0,") & " per sec"
        ? ""
        ? "and now using VAL:"
    
        t = TIMER
        FOR ii = 1 TO 3000000         '3,000,000 loops
           a = VAL("1234567890.12345678")
        NEXT
        t = TIMER - t
        ? "3 million assignments done in" & STR$(t, 4) & " sec"
        ? "which is " & FORMAT$(3000000 / t, "0,") & " per sec"
        ? ""
        ? "and finally division:"
    
        t = TIMER
        FOR ii = 1 TO 20000000        '20,000,000 loops
           a = 123456789012345678 / 100000000
           b = 123456789012345681 / 100000000
           c = 123456789012345684 / 100000000
           d = 123456789012345687 / 100000000
           e = 123456789012345690 / 100000000
        NEXT
        t = TIMER - t
        ? "100 million assignments done in" & STR$(t, 4) & " sec"
        ? "which is " & FORMAT$(100000000 / t, "0,") & " per sec"
        ? ""
        ? "so we see division is fast too, some 30 times faster"
        ? "than VAL for this application, and is much easier to"
        ? "implement than the asm data statements."
        ? ""
        ? "However, one added feature of the asm method is that it"
        ? "allows entry of literals at ""uber resolution ;)"" of the"
        ? "full 18.95 (courtesy Dave Roberts) digits available."
        WAITKEY$
        
    
    END FUNCTION
    
    FUNCTION PBMAIN () AS LONG
        speedyExtAssign
    END FUNCTION
    Last edited by John Gleason; 5 Jul 2008, 02:18 PM.

    Leave a comment:


  • David Roberts
    replied
    Nice one, John. PEEKing an inline data line had not occurred to me - something that I've only done a few times.

    Just out of curiosity I then looked at 'Poke Ext, y, 1234567890.12345678##' and found that, here to, the double precision evaluation got POKEd.

    Leave a comment:


  • John Gleason
    replied
    Dave, here's a way to do it with just a two tick deficit. I would think hardly anyone would notice two ticks on a 2GHz box, even IN a tight loop.
    Code:
    #COMPILE EXE
    #DIM ALL
    
    FUNCTION PBMAIN () AS LONG
    
        LOCAL x, y, z AS EXT
        LOCAL wp AS LONG               'ptr to the actual ext stored data of x
        x = VAL("1234567890.12345678") 'your full 18 digits into x
        wp = VARPTR(x)                 'ptr to the 5 words of x (10 bytes)
        OPEN "c:\hexExtendedPrecision1a.txt" FOR OUTPUT AS #1
    
        PRINT #1, "!dw &h" &  HEX$(PEEK(WORD, wp    ), 4) & ", &h" & HEX$(PEEK(WORD, wp + 2), 4) & ", &h" & _
                              HEX$(PEEK(WORD, wp + 4), 4) & ", &h" & HEX$(PEEK(WORD, wp + 6), 4) & ", &h" & _
                              HEX$(PEEK(WORD, wp + 8), 4)
        ? "ok, copy the line in c:\hexExtendedPrecision1a.txt and put it in the program."
        GOTO pastYdata                 'can't let program read thru the x data as if valid operations.
        extendedYdata:
        !dw &hBA21, &h3F35, &h05A4, &h932C, &h401D  ;<< copy data here. This represents x (and next, y) in memory
        pastYdata:
    
        y = PEEK(EXT, CODEPTR(extendedYdata))   '<< this is now approx 2 ticks slower than...
        z = 1234567890.12345678##               '<< this, but y will now always be the full EXT
    
        IF x = y THEN
           ? "x =" & STR$(x, 18)
           ? "y =" & STR$(y, 18)
        ELSE
           ? "x does not yet = y"
        END IF
        WAITKEY$
    END FUNCTION
    Last edited by John Gleason; 5 Jul 2008, 10:10 AM. Reason: filename edit

    Leave a comment:


  • David Roberts
    replied
    From post #51 here Steve Rossell said "However, evaluation of numeric literals in your compiled programs are currently limited to double precision."

    What isn't said is that the storage allocated is limited to 64 bits as well. Looking at a OllyDBug output on running an exe it seems to me, unless I am mistaken, that the storage allocation for extended precision numeric literals is still 80 bits but the resolution of a numeric literal is clearly only 64 bits.

    So, if we declare 'Local y as Ext' this will give us our 80 bits of storage at address VarPtr(y).

    I am not sufficiently aquainted with FPU assembly to go any further but I do know a man that is.

    Can we fill this storage with our evaluation, via FPU instructions as opposed to Paul P's fractional method which is currently in the #1 slot, such that ref to 'y' is a ref to a genuine ext and not one that has had its tail cut off - well, rounded actually.

    Added: Obviously, we would not assign 'y' via 'y =' but perhaps read a string at run time. Yeah, I know, VAL does that but a specific VAL bereft of any bells and whistles.
    Last edited by David Roberts; 5 Jul 2008, 04:47 AM.

    Leave a comment:


  • Paul Purvis
    replied
    Walter,
    good to go on the quad integers

    Leave a comment:


  • Walter Henn
    replied
    Originally posted by paul d purvis View Post
    Unless i am convinced otherwise, yes i do feel it maybe relevant.

    what is the difference between
    local m as ext
    m=16.585##
    and
    local m as ext
    let m=16.585##

    none i do believe

    while it is true that rounding routines are not affected in either way

    the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
    that is why the integer/decimal is giving us a more precise values to round from.
    so understanding the LET statement is critical in my view.
    If the use of the LET statement does not result in any difference when assigning a value to a variable, why would understanding the LET statement be critical??? I believe that this is really muddying the water.

    i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.
    Extracted from PBCC 4.04 Hep File:
    Internally, Currency and Extended-currency numbers are stored as Quad-integers with an implied decimal point (at 4 places for Currency, and at 2 places for Extended-currency). This approach ensures that all of the digits of the variables can be represented exactly.

    Leave a comment:


  • Paul Purvis
    replied
    Walter,
    you are correct, i was not aware of the fact STR$ rounded, i just always used only str$(x) or str$(x,18).
    thanks for pointing that out.
    so yes, the test you pointed would fail to show differences.
    i going to remove that test.

    Leave a comment:


  • Walter Henn
    replied
    Originally posted by John Gleason View Post
    ...
    ...
    Walter is looking for a round function that rounds up on 5, but only if the base2 representation of the "rounding 5" actually is equal to or exceeds 5 internally. Let's call this, "base2 round up on 5", and is what Paul Dixon's code has been based on I believe...

    Walter, how are asmRoundUp/round_dixon looking? Any problems or exceptions there?
    Your statement above regarding what I'm looking for in a roundup on 5 routine and what Paul Dixon's code is based on is correct.

    The asmRoundUp and round_dixon routines have been found to be correct for all test values (in the bilions). Paul Dixon is a remarkable programmer.
    Last edited by Walter Henn; 2 Jul 2008, 07:09 PM.

    Leave a comment:


  • Paul Purvis
    replied
    i was not aware of the STR$ doing any rounding, maybe i missed that or just forgot during the last few days, let us see what i lose over the 4th of July weekend.

    Leave a comment:


  • Paul Purvis
    replied
    Walter
    Unless i am convinced otherwise, yes i do feel it maybe relevant.

    what is the difference between
    local m as ext
    m=16.585##
    and
    local m as ext
    let m=16.585##

    none i do believe

    while it is true that rounding routines are not affected in either way

    the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
    that is why the integer/decimal is giving us a more precise values to round from.
    so understanding the LET statement is critical in my view.

    i was trying to make an effort to show why we were getting the values of something similar to 16.5849999something as a value to round from or even possibly to use the variable in a calculation of another expression that probably will make a difference in returned results.

    i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.

    also not show but in another test i have done with a DOUBLE variables today, i appears i may want to stick with EXTENDED variables totally for many of my math calculations where there is a decimal involved because of the 18 digit precision powerbasic keeps.

    here is code that make me believe that.
    notice that m when returned in a STR$ with 18 digits can produce other values than STR$ alone, thereby giving me an impression that the value may be different than what i want stored.
    i have faith in pb using the correct values to do math with, but i can have faith and security blank too.


    this is just for observation purposes only
    like Walter said, maybe discussion for another day,
    but i am very happy it seems we resolved the EXTENDED assignment deli ma.


    Code:
    #COMPILE EXE
    #DIM ALL
    FUNCTION PBMAIN () AS LONG
    LOCAL m AS EXT
    LOCAL mm AS DOUBLE
    LOCAL s AS STRING
    LOCAL ss AS STRING
    LOCAL sss AS STRING
    LOCAL i AS QUAD
    LOCAL count AS QUAD
    
    FOR i =900000000000000&& TO 999999999999999&& STEP 1712467&&
    
    s=TRIM$(STR$(i,18))
    s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)-1)
    trimzeroout:
    IF RIGHT$(s,1)="0"  THEN s=LEFT$(s,LEN(s)-1):GOTO trimzeroout
    
    m=i/100000000000000
    mm=i/100000000000000
    ss=TRIM$(STR$(m,18))
    sss=TRIM$(STR$(mm,18))
    
    'IF ss<>s THEN
    IF m<>mm THEN
        PRINT i
        PRINT s
        PRINT ss
        PRINT sss
        PRINT "diff found press any key to continue"
        WAITKEY$
    END IF
    
    INCR count
    
    IF count MOD 500000 =0 THEN
        PRINT I," > "+S+ " > "+SS+STR$(COUNT,18)
    END IF
    NEXT i
    
    
    PRINT "done"
    WAITKEY$
    
    END FUNCTION
    i made a change to this program after Walter mentioned the STR$ rounding behavior
    Last edited by Paul Purvis; 2 Jul 2008, 06:32 PM.

    Leave a comment:


  • Walter Henn
    replied
    Originally posted by paul d purvis View Post
    will this program suffice as a test


    Code:
     
    #COMPILE EXE
    #DIM ALL
    FUNCTION PBMAIN () AS LONG
    LOCAL m AS EXT
    LOCAL s AS STRING
    LOCAL ss AS STRING
    LOCAL i AS QUAD
    LOCAL count AS QUAD
     
    FOR i =90000000000000000 TO 99999999999999999 STEP 1712467
    s=TRIM$(STR$(i,18))
    s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)-1)
    trimzeroout:
    IF RIGHT$(s,1)="0"  THEN s=LEFT$(s,LEN(s)-1):GOTO trimzeroout
    ...
    m=i/10000000000000000
    ss=TRIM$(STR$(m,18))
    IF ss<>s THEN
    ...
    Actually, Paul, this test does not prove what it is supposed to. Correct me if I'm wrong, but I believe the intent was to prove that your integer/decimal technique was equivalent in accuracy to the VAL(string) technique.

    The coding above only demonstrates that when you do an integer/decimal computation, your results, when rounded by the STR$ function, are exactly the same as that obtained my manipulating the string value of i to get the value, i/10000000000000000.

    Leave a comment:

Working...
X