Announcement

Collapse
No announcement yet.

Precision used when printing a double

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Precision used when printing a double

    All,

    I am using PowerBASIC to translate a QBasic program for one of our customers. It is working well. The one item I have not found a solution for is the following.

    In the QBasic program that I have, none of the variables are declared. Consequently, there is code like the following:
    R = 0.5
    PRINT R

    this produces
    0.5
    in the output

    Now, I have declared all variables, so I have something like
    DIM R AS DOUBLE
    R = 0.5
    PRINT R

    this produces something like
    0.5000000000000123

    The client seems to think there is a command that can limit the precision of a variable. If not, I will have to change all of the PRINT statements to reflect the precision he is looking for.

    My questions:
    1. Is this the best way to declare these variables (all variants before)
    2. Is there a global way to change the precision printed of a variable type?

    All help is greatly appreciated.

  • #2
    Format$

    Hi Scott!

    Have you looked at the Format$() built in function? The second parameter to that function is a string mask where you can specify how many digits to the left or right of the decimal you want.
    Fred
    "fharris"+Chr$(64)+"evenlink"+Chr$(46)+"com"

    Comment


    • #3
      Precision

      Scott,

      Have you tried the Currency variable ?
      [email protected]@=0.5 will give you 0.50 and bye bye round off errors
      Old QB45 Programmer

      Comment


      • #4
        example

        Just thought I'd add this...

        Code:
        #Compile Exe
        #Dim All
        
        Function PBMain() As Long
          Local r As Double
          
          r=0.54321
          Print "r="r
          Print "r="Format$(r,"#.#")
          Waitkey$
            
          PBMain=0
        End Function
        Gives you this...

        r= .543210029602051
        r=0.5
        Fred
        "fharris"+Chr$(64)+"evenlink"+Chr$(46)+"com"

        Comment


        • #5
          TYPE YOUR LITERALS WHEN ASSIGNING, ALWAYS!
          Code:
          Local r As Double
          WRONG: 
            r=0.5
          RIGHT: 
            r = 0.5[SIZE="5"][b]#[/b][/SIZE]
          HOWEVER... SINGLE , DOUBLE and EXT are all floating-point, meaning they are always approximations, so your problem is NOT going to go away completely even if you do type all your literals.

          If you need 4 or fewer decimal places, use CURRENCY. You can use extended currency (CUX) if you only need two decimals.

          MCM
          Michael Mattias
          Tal Systems (retired)
          Port Washington WI USA
          [email protected]
          http://www.talsystems.com

          Comment


          • #6
            Or, kind of a long way of going about it: Convert the number to a string then parse it out.
            There are no atheists in a fox hole or the morning of a math test.
            If my flag offends you, I'll help you pack.

            Comment


            • #7
              If I didn't already know better, I'd deduce from reading this thread that trying to do a verb-for-verb, function-for-function conversion from one source code language to another was a stone loser, even when both languages are BASIC dialects.
              Michael Mattias
              Tal Systems (retired)
              Port Washington WI USA
              [email protected]
              http://www.talsystems.com

              Comment


              • #8
                Originally posted by Scott Butler View Post
                All,

                I am using PowerBASIC to translate a QBasic program for one of our customers. It is working well. The one item I have not found a solution for is the following.

                In the QBasic program that I have, none of the variables are declared. Consequently, there is code like the following:
                R = 0.5
                PRINT R

                this produces
                0.5
                in the output

                Now, I have declared all variables, so I have something like
                DIM R AS DOUBLE
                R = 0.5
                PRINT R

                this produces something like
                0.5000000000000123
                The default type in QBasic was a single not a double and never a variant, so the following code of your example shows the difference
                Code:
                #COMPILE EXE
                '#DIM ALL
                
                FUNCTION PBMAIN () AS LONG
                
                    a#=0.5000000000000123
                    b! =0.5000000000000123
                    ? FORMAT$(a#) + "  " + FORMAT$(b!)
                
                END FUNCTION
                However I agree with other posters that you should be choosing better types depending on the use
                Last edited by John Petty; 24 May 2008, 10:58 AM.

                Comment


                • #9
                  Originally posted by Michael Mattias View Post
                  TYPE YOUR LITERALS WHEN ASSIGNING, ALWAYS!
                  Code:
                  Local r As Double
                  WRONG: 
                    r=0.5
                  RIGHT: 
                    r = 0.5[SIZE="5"][b]#[/b][/SIZE]
                  Michael not sure what you are getting at. How can one be wrong and one be right when they both give the same result? The compiler has very strict rules as to how it stores a constant, AFIK always use the smallest possible type to store the constant (though not sure about integers maybe always as longs).
                  The only two reasons I know to type a literal are firstly for speed ie if you know the number 100 is going to be used in a floating calc then typing it saves a transformation or if storing in hex and you might be worried about the sign bit.

                  Comment


                  • #10
                    0.5 or 0.5# are bad examples because as luck would have it it is one of the decimal fractions which can be represented exactly in binary. ( 0.5 (d) = &B0.1)
                    Michael Mattias
                    Tal Systems (retired)
                    Port Washington WI USA
                    [email protected]
                    http://www.talsystems.com

                    Comment


                    • #11
                      Having dealt with monetary values in QB for years, the best conversions in PBCC:
                      My favorite is using the currency type as mentioned before. Of course this requires either a dim or adding the @@ identifier at the end of the variable. perhaps a defcux a-z

                      The second best as also discussed previously is using Format(amt,[mask$])

                      PBcc now has print using like QB so that is a possible use for you.

                      Then there is round(amt,digits)
                      Whether round will help im not sure
                      r=round(r,2)
                      print r
                      Last edited by Fred Buffington; 24 May 2008, 10:08 PM.
                      Client Writeup for the CPA

                      buffs.proboards2.com

                      Links Page

                      Comment


                      • #12
                        INTEL uses 80 bit registers to do floating-point math. (This is extended precision.) It then rounds the values to the precision for the variable. Depending on the code it may keep a variable in the 80 bit register for more calculations or save to memory. Results will vary depending on the problem. Look at the PB HELP files for SIN() it uses EXTENDED and then returns the value to the precision of the variable.

                        When using a numeric value, PB will convert the constant to the lowest precision it can fit the number. So in your example the constant 0.5 was rounded to fit SINGLE precision and then converted and rounded to fit DOUBLE precision. If any math was done later in your program then it was most likely done in EXTENDED precision so the values were rounded in the converting to EXTENDED, rounded after calcuations and then rounded to fit back into the DOUBLE precision variable. The conversion of binnary to decimal number requires rounding. The likelyhood of having a small number added or subtracted like 0.0000000000000123 to your original number is very high. This is the nature of floating-point math.

                        If you can, I would use EXTENDED precision from the start. If you use PRINT STR$(R,5) then PB will round the value to 5 digits.
                        0.5000000000000123 => 0.5000

                        A crude way to display less digits globally is to convert from DOUBLE precision to a temporary SINGLE precision variable right before printing/displaying the temporary rounded value.

                        Comment


                        • #13
                          I think the main point of the question has been missed here. Scott is trying to do a quick conversion from Qbasic. As the variables were not typed he assumed them to be variants which did not exist until MS 32 bit compilers (I think VB4 was the first basic version) instead were the all the default of singles.
                          His client is actually correct, if Scott had typed all the variables as singles rather than doubles then it would limit the precision of the variables and the PB PRINT should give the same answers as Qbasic.
                          Scott does not say this is a financial package.

                          Comment


                          • #14
                            QB conversion

                            As somebody that did a lot of that kind of conversion, the quick and dirty way is :

                            DEFSNG a-z

                            This will make PBCC like QB so most of the old code can be ported as is. Also, all the hard disk data is 100% compatible.

                            But once you have tasted the wonder of using CURRENCY you will start using it all the time.
                            Old QB45 Programmer

                            Comment


                            • #15
                              All,

                              Thanks for all of the answers and suggestions. I am used to formatting all of my output variables as most of our work is in C# and C++. I was hoping to find a magic bullet to easily make the output look the same for the QBasic and PowerBASIC programs.

                              The Currency suggestion is great however the code does some scientific calculations and I am not sure the round off error would be acceptable.

                              Thanks again!

                              Comment


                              • #16
                                The Currency suggestion is great however the code does some scientific calculations and I am not sure the round off error would be acceptable
                                As long as you stay in the documented range of values, CURRENCY datatypes have no rounding, since they are scaled binary rather than floating point types.

                                SINGLE, DOUBLE and EXT will always involve rounding, since all are floating point types.
                                Michael Mattias
                                Tal Systems (retired)
                                Port Washington WI USA
                                [email protected]
                                http://www.talsystems.com

                                Comment


                                • #17
                                  Originally posted by Michael Mattias View Post
                                  CURRENCY datatypes have no rounding, since they are scaled binary rather than floating point types.

                                  SINGLE, DOUBLE and EXT will always involve rounding, since all are floating point types.
                                  This some new form of mathematics Michael? Of course there can be rounding errors in the currency type. If you divide 2 by 3 or 1 by 3 where do all the extra 66666's and 333333's go? Currency is perfect for + - * but can give rounding errors for other operations.
                                  Perhaps Scott should have used the word "precision" as a single gives 6 significant decimal places

                                  Comment


                                  • #18
                                    1/3 and 2/3 are out of range, which is four decimal places. That's what I was thinking anyway.. but I never do anything with decimals except unit and extended prices.

                                    Pretty apparent my 'math world' is limited, isn't it?

                                    I never have to divide, except integer divisions to get the number of standard packs some integer quantity requires.

                                    And if I have to multiply for some fractional thing like sales tax, I just round off to nearest penny anyway since that's what can get paid.
                                    Michael Mattias
                                    Tal Systems (retired)
                                    Port Washington WI USA
                                    [email protected]
                                    http://www.talsystems.com

                                    Comment

                                    Working...
                                    X