Announcement

Collapse
No announcement yet.

Quads: displaying all 19 digits

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Quads: displaying all 19 digits

    I have noticed the limitation of Str$() to display only 18
    digits of precision, but no such mention is made for Format$(),
    although I can't seem to get an accurate 19-digit display of
    large Quad values, even with a "##################0" mask.

    Is there any way at all to display the full precision of a
    large Quad value?

  • #2
    STR$() only works up to 18 digits. FORMAT$(QuadVar, ",") works up to 19 digits ok, and the alternative format string with a single pound symbol "#" works too.

    Here is a small PB/CC example to get you going:

    Code:
    FUNCTION PBMAIN
        DIM a AS QUAD
        a = &H7FFFFFFFFFFFFFFF
        PRINT a
        PRINT STR$(a,18)
    '    PRINT STR$(a,19)
        PRINT FORMAT$(a, ",")
        PRINT FORMAT$(a, "#")
        WAITKEY$
    END FUNCTION
    On a related topic, be sure to read the FAQ on 80 bit floating point precision - see the FAQ forum for details.

    ------------------
    Lance
    PowerBASIC Support
    mailto:[email protected][email protected]</A>
    Lance
    mailto:[email protected]

    Comment


    • #3
      That's what I assumed too, but running this simple test
      told me that some rounding is taking place somewhere.

      You would expect the three values to be different on output:

      FUNCTION PBMAIN
      DIM Q AS QUAD
      Q=&H7FFFFFFFFFFFFFFF
      PRINT FORMAT$(Q,"#")
      Q=&H7FFFFFFFFFFFFFFE
      PRINT FORMAT$(Q,"#")
      Q=&H7FFFFFFFFFFFFFFD
      PRINT FORMAT$(Q,"#")
      END FUNCTION

      but here is the output I get:

      9223372036854776320
      9223372036854776320
      9223372036854776320

      Comment


      • #4
        Interesing. I applied the 80-bit patch from the FAQ forum to that program and I still get incorrect results.

        Code:
        9223372036854775810
        9223372036854775810
        9223372036854775800
        -- Eric


        ------------------
        Perfect Sync: Perfect Sync Development Tools
        Email: mailto:[email protected][email protected]</A>

        "Not my circus, not my monkeys."

        Comment


        • #5
          this code shows that the 18-digit limitation is in format$, not the quad math operation. and it demonstrates an easy workaround for using format$ with huge values...

          <font face="courier new, courier" size="3"><pre>
          $register none
          $compile exe
          function pbmain
          dim xyz as local integer
          dim q1 as local quad
          dim q2 as local quad
          'apply 80-bit precision patch
          xyz = 0
          asm fstcw xyz
          asm or xyz, &b0000001100000000%
          asm fldcw xyz
          'perform the test
          q1=&h7fffffffffffffff???
          q2=&h7ffffffffffffffe???
          print "direct:"
          print format$(q1,"#")
          print format$(q2,"#")
          print
          print "difference between values = ";q1-q2
          print
          print "workaround:"
          print format$(q1\100000);format$(q1 mod 100000)
          print format$(q2\100000);format$(q2 mod 100000)
          waitkey$
          end function
          [/CODE]

          here is a link to the 80-bit faq:

          http://www.powerbasic.com/support/pb...hread.php?t=45

          -- eric
          ------------------
          perfect sync: perfect sync development tools
          email: mailto:[email protected][email protected]rfectsync.com</a>



          [this message has been edited by eric pearson (edited may 17, 2000).]
          "Not my circus, not my monkeys."

          Comment

          Working...
          X