Announcement

Collapse
No announcement yet.

Quads: displaying all 19 digits

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Eric Pearson
    replied
    this code shows that the 18-digit limitation is in format$, not the quad math operation. and it demonstrates an easy workaround for using format$ with huge values...

    <font face="courier new, courier" size="3"><pre>
    $register none
    $compile exe
    function pbmain
    dim xyz as local integer
    dim q1 as local quad
    dim q2 as local quad
    'apply 80-bit precision patch
    xyz = 0
    asm fstcw xyz
    asm or xyz, &b0000001100000000%
    asm fldcw xyz
    'perform the test
    q1=&h7fffffffffffffff???
    q2=&h7ffffffffffffffe???
    print "direct:"
    print format$(q1,"#")
    print format$(q2,"#")
    print
    print "difference between values = ";q1-q2
    print
    print "workaround:"
    print format$(q1\100000);format$(q1 mod 100000)
    print format$(q2\100000);format$(q2 mod 100000)
    waitkey$
    end function
    [/CODE]

    here is a link to the 80-bit faq:

    http://www.powerbasic.com/support/pb...hread.php?t=45

    -- eric
    ------------------
    perfect sync: perfect sync development tools
    email: mailto:[email protected][email protected]</a>



    [this message has been edited by eric pearson (edited may 17, 2000).]

    Leave a comment:


  • Eric Pearson
    replied
    Interesing. I applied the 80-bit patch from the FAQ forum to that program and I still get incorrect results.

    Code:
    9223372036854775810
    9223372036854775810
    9223372036854775800
    -- Eric


    ------------------
    Perfect Sync: Perfect Sync Development Tools
    Email: mailto:[email protected][email protected]</A>

    Leave a comment:


  • Francois Roy
    Guest replied
    That's what I assumed too, but running this simple test
    told me that some rounding is taking place somewhere.

    You would expect the three values to be different on output:

    FUNCTION PBMAIN
    DIM Q AS QUAD
    Q=&H7FFFFFFFFFFFFFFF
    PRINT FORMAT$(Q,"#")
    Q=&H7FFFFFFFFFFFFFFE
    PRINT FORMAT$(Q,"#")
    Q=&H7FFFFFFFFFFFFFFD
    PRINT FORMAT$(Q,"#")
    END FUNCTION

    but here is the output I get:

    9223372036854776320
    9223372036854776320
    9223372036854776320

    Leave a comment:


  • Lance Edmonds
    replied
    STR$() only works up to 18 digits. FORMAT$(QuadVar, ",") works up to 19 digits ok, and the alternative format string with a single pound symbol "#" works too.

    Here is a small PB/CC example to get you going:

    Code:
    FUNCTION PBMAIN
        DIM a AS QUAD
        a = &H7FFFFFFFFFFFFFFF
        PRINT a
        PRINT STR$(a,18)
    '    PRINT STR$(a,19)
        PRINT FORMAT$(a, ",")
        PRINT FORMAT$(a, "#")
        WAITKEY$
    END FUNCTION
    On a related topic, be sure to read the FAQ on 80 bit floating point precision - see the FAQ forum for details.

    ------------------
    Lance
    PowerBASIC Support
    mailto:[email protected][email protected]</A>

    Leave a comment:


  • Francois Roy
    Guest started a topic Quads: displaying all 19 digits

    Quads: displaying all 19 digits

    I have noticed the limitation of Str$() to display only 18
    digits of precision, but no such mention is made for Format$(),
    although I can't seem to get an accurate 19-digit display of
    large Quad values, even with a "##################0" mask.

    Is there any way at all to display the full precision of a
    large Quad value?
Working...
X