Announcement

Collapse
No announcement yet.

Why is this giving me a GPF?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Lance Edmonds
    replied
    No it does not, but you *are* limited to 18 decimal places with FORMAT$(). Moreover, SINGLE precision has an accuracy limitation of 7 digits, so displaying more than 7 digits is "wasteful" as best, inaccurate at worst.

    Also, be sure to read the FPU 80-bit precision note in the FAQ forum - it may be useful if you really do need to use 80-bit precision in your floating point calculations.



    ------------------
    Lance
    PowerBASIC Support
    mailto:[email protected][email protected]</A>

    Leave a comment:


  • mark smit
    Guest replied
    Thanks Chris!


    I did notice that when I reduce the amount of "#" in the format
    string the GPF goes away but this still seems very strange to me.


    This might be a dumb question but does the "." count as a digit?


    ------------------
    Cheers

    Leave a comment:


  • Chris Boss
    replied
    I just noticed another thing.

    You aren't using #Register None in your code.

    Try adding #Register None and see if the GPF disappears.
    It shouldn't make a difference since you are using a Single,
    but you never know.




    ------------------

    Leave a comment:


  • Chris Boss
    replied
    Mark;

    You are pushing the Single limit to its maximum value !

    Maybe you are causing some kind of overflow with the variable
    and the GPF isn't produced until you use the single in the Format$
    function.

    Try setting the value a little smaller and see if it still GPF's.

    If it doesn't GPF with a smaller value, then it likely is an
    overflow problem. The value you used is the max value according to
    the PB docs. Maybe the actual max value is slightly smaller.



    ------------------

    Leave a comment:


  • mark smit
    Guest started a topic Why is this giving me a GPF?

    Why is this giving me a GPF?

    Code:
    #compile exe
    
    function pbmain
        local n as single
    
    
        n = 3.37*10^38
    
        msgbox format$(n,"#.####################") '*** GPF right here ***
    end function


    ------------------
    Cheers
Working...
X