Announcement

Collapse
No announcement yet.

VAL rounds a variable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • It is worth noting that I wasn't considering 315/10000.

    I was concerned about a sequence like 49999999 being rounded up to a 5 on entering the quad mode. The 5 then caused the round up code to round up our value AGAIN. This is clearly a bug.

    Here is another one and shows that we don't even need to end on a '9'. A '5' will cause a cascade.

    num = Val("0.0123456784999999995")

    Before adding snippet.

    roundUpV3(num, 9) => .012345679

    We have then a round up on '...5' algoritm rounding up on '...4999999995'.

    After adding snippet.

    roundUpV3(num, 9) => .012345678

    Comment


    • Having read through 400 posts I come to the conclusion that if I want individual numbers to visually correct, that is, in compliance with the rounding method taught in grade schools, use one of the ones developed here in this thread.

      But if I want to use a number that yields more accurate results on average when used in calculations, use PB's ROUND(var,place)

      Rod
      Rod
      In some future era, dark matter and dark energy will only be found in Astronomy's Dark Ages.

      Comment


      • I think that I've mentioned that I have no use for what is developing here and PB's Round suits me especially when adding a column of results.

        However, a scientific or manufacturing requirement may be such that 2.5, say, rounds to 3 and not 2.

        It boils down to "Does it matter and if it does then decide which to choose and if not then PB's Round".

        Comment


        • David

          thanks for reporting that returned error

          it might be worth while to concentrate on values with 499??

          i see what you mean on that Val("0.0123456784999999995")
          what snippet are you referring too.

          i noticed you did not mention the results of roundme6c which does comes up with the correct answer.
          p purvis

          Comment


          • Paul

            what snippet are you referring too.

            i noticed you did not mention the results of roundme6c which does comes up with the correct answer.
            Post #392 and I mentioned roundme6c a short while after posting the snippet.

            PS Paul Dixon's ASM routine is OK too.
            Last edited by David Roberts; 20 Jul 2008, 03:42 AM.

            Comment


            • Originally posted by David Roberts View Post
              Ok but we wouldn't be expecting a user of roundupV3 or 4 to do that and what if, which is likely, the value is the result of a numerical computation?
              num = Val("0.0000000000314999999999999999")
              Before adding snippet...
              Given the very tiny number represented and the rarity of probable need for rounding accuracy to 19 digits or greater on that number, my thought is that it's reasonable to pre-process it, round it using 18-digit accuracy, then post-process it. Adding the snippet to the function alters all "0.0*" rounding.

              My current huge concern is, "what if, which is likely, the value is the result of a numerical computation?" If possible, I'd like a way to feed or preferably have the functions accept computed EXT values in a reliable way.

              Comment


              • Adding the snippet to the function alters all "0.0*" rounding.
                Would you give me an example, John?

                Comment


                • David,
                  i think you missed a post.
                  it was mentioned sometime back that these routines do not round any 18 digits but only up to 18 digits to the right of the decimal place.
                  is was late last night when i read your post and i even forgot about that.
                  Last edited by Paul Purvis; 20 Jul 2008, 12:38 PM.
                  p purvis

                  Comment


                  • Found some, John.

                    The rounding aspect seems fine, it is just the decimal point that is adrift. There is a conflict with your code later on depending upon the exit path. I'll work on it.

                    Comment


                    • i said i resolved myself to finalizing the roundme routines, but i unresolved myself.
                      ever get the feeling some answers to your problems are reaching out and slapping you in the face, that is the feeling i have been having and i am just staring back
                      there should be a speed up and maybe even possibility of getting over the 18 decimal place with some new thoughts.

                      John,
                      i think we can use the ROUND statement once again to our advantage.
                      lets focus on positive numbers for simplicity

                      in the roundme6c function, the ROUND statement is used to get a quick answer if the a result is larger than the value give.
                      let us do the same but after that piece of code of where we know round is less than the value let us do this

                      z being a value of something like 4999*something

                      if round(x,y)+z >value then result is equal to round(x,y)
                      p purvis

                      Comment


                      • >> Would you give me an example, John?
                        Code:
                        -.05191759539913762 << original EXT
                        rV4-.51917595399137619      -.519175953991 12
                        rV3                         -.519175953991 9 random
                        Here's an example, tho it's not really a rounding alteration, rather it is a shift of the decimal. I'm pretty sure I could fix it either by passing the EXT param BYREF, adding a copy of the param, or maybe redividing by adjust afterward. My preference tho for this special situation would be the pre-shift; round function; post-shift method.

                        Comment


                        • Ah, I see you noticed it Dave. I'm pretty quick eh?

                          Comment


                          • My preference tho for this special situation would be the pre-shift; round function; post-shift method.
                            If I wore a commercial hat, John, my preference would be for a seamless solution - less telephone calls.

                            Comment


                            • Paul, I sort of see what ya mean there. I'd say work on it but limit it to 18 digits at least at first. I'm trying to work on the "result of a numerical computation?" thang.

                              Comment


                              • Unless I'm mistaken a slight tweak was all that was required. The problem that was solved could only exit one way.

                                Code:
                                Local adjust, oldb As Long
                                adjust = [COLOR="Red"]Fix[/COLOR](Log10(Abs(a))) ' From Int to Fix
                                oldb = b    [COLOR="Red"]<= Tweak[/COLOR]
                                If adjust < [COLOR="Red"]0[/COLOR] Then ' From -1 to 0
                                  adjust = Abs( adjust )
                                  [COLOR="Red"]'Decr adjust[/COLOR] ' and remove this
                                  a = a*10^adjust
                                  b = b - adjust
                                End If
                                and replace b with oldb when we check |sizeA| = 5.
                                Last edited by David Roberts; 20 Jul 2008, 11:18 PM. Reason: A 'Print' was left over from debugging

                                Comment


                                • Post deleted
                                  Last edited by Walter Henn; 21 Jul 2008, 05:59 AM.

                                  Comment


                                  • John , I've just done 500,000,000 RND/100 with rounupV3 with the revised snippet and no leading zeros messed up. Incidentally, it wasn't just '0.*' - I found some '0.0*'.

                                    Anyway, for my money this now puts you back in the ring with the two Pauls' code which did not have the above problem.

                                    Added: Just edited the snippet, Int to Fix and removed 'Decr adjust'
                                    Added more: Whoops - the condition should now be 0 and not -1
                                    Last edited by David Roberts; 20 Jul 2008, 11:19 PM. Reason: Condition

                                    Comment

                                    Working...
                                    X