Announcement

Collapse
No announcement yet.

Thoughts in rounding, yes again on floating numbers

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Thoughts in rounding, yes again on floating numbers

    I forgot how many decimal places can be stored in a higher precision value.
    I was thinking of something along the line of removing any whole part of a floating number any way I can and using shift with the fractional parts of a number even if a loop had to be in it for high number of decimal places.
    The real I am putting this post out is because I am not intimate with the effects and I wanted to maintain as much accuracy to the original floating point number.
    Maybe my thoughts are way off.
    We had some goood code before but I believe it might of used strings.
    It would seem better to shift as many decimal places as can be shifted then chop off any whole number part to fit the decimal placed wanted for a result. In other words over reach your decimal places wanted then work out the chopping code of the whole numbers.
    The last math part might be to multiple the whole number by 1 in decimal format such as 170 x .01 = 1.70 there by providing a result to 2 decimal places.
    p purvis

  • #2
    I forgot how many decimal places can be stored in a higher precision value.
    18 digits in extended floating point.
    Dale

    Comment


    • #3
      Many numbers cannot be represented exactly as a Single,Double or Ext

      If you want a fixed number of decimal places, you can often use Currency/CurrencyEx numeric types or your own" scaled integers"

      As far as rounding is concerned, it's generally better to use the maximum precision available throughout your calculations and only round for display purposes.
      --
      [URL="http://www.camcopng.com"]CAMCo - Applications Development & ICT Consultancy[/URL][URL="http://www.hostingpng.com"]
      PNG Domain Hosting[/URL]

      Comment


      • #4
        Dale. Is that the maximum Powerbasic can hold in a value or just present.
        p purvis

        Comment


        • #5
          Stuart. Horshoe tossing will not do for regulatory reasons in my calculations or even displaying.
          I guess I will dig further.
          p purvis

          Comment


          • #6
            This might be of interest... RoundHalfAwayFromZero

            Comment


            • #7
              I guess it safe to say that if you cannot store a value on a disk as a one off variable value then any decimal places beyond that should be considered insignificant.
              p purvis

              Comment


              • #8
                Originally posted by Paul Purvis View Post
                I guess it safe to say that if you cannot store a value on a disk as a one off variable value then any decimal places beyond that should be considered insignificant.
                Given sufficient storage, you can store any value on a disk as a one off variable You just have to use enough bits
                --
                [URL="http://www.camcopng.com"]CAMCo - Applications Development & ICT Consultancy[/URL][URL="http://www.hostingpng.com"]
                PNG Domain Hosting[/URL]

                Comment


                • #9
                  Paul,

                  > Horshoe tossing will not do for regulatory reasons in my calculations or even displaying.

                  I think the starting point for you is to find out what the regulators requirements are first, once you have that you can design a method that will come as close to that as possible given that there is a limit to the precision available on computers. If it is an issue of currency, the obvious is that non computer people dealing with money don't want 99.99999999999999999999, they want 100 and the choice of rounding will be dictated by the regulatory environment that your software will be used in.

                  The old C standard was truncation so 99.99999999999 ended up as 99, a split at 50% of the fraction can accumulate errors over a large number of cases like the payroll of a very large corporation and I have seen long ago a payroll system where the rounding was always down and the rounding amount was put in a separate account until it was large enough to give all of the employees an extra cent to empty the temporary account.

                  For precision the technique that Stuart suggested is probably the most accurate as the errors in 18 digit range become very small but again, see what your regulators require.
                  hutch at movsd dot com
                  The MASM Forum

                  www.masm32.com

                  Comment

                  Working...
                  X