Announcement

Collapse
No announcement yet.

VAL rounds a variable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • David Roberts
    replied
    John , I've just done 500,000,000 RND/100 with rounupV3 with the revised snippet and no leading zeros messed up. Incidentally, it wasn't just '0.*' - I found some '0.0*'.

    Anyway, for my money this now puts you back in the ring with the two Pauls' code which did not have the above problem.

    Added: Just edited the snippet, Int to Fix and removed 'Decr adjust'
    Added more: Whoops - the condition should now be 0 and not -1
    Last edited by David Roberts; 20 Jul 2008, 11:19 PM. Reason: Condition

    Leave a comment:


  • Walter Henn
    replied
    Post deleted
    Last edited by Walter Henn; 21 Jul 2008, 05:59 AM.

    Leave a comment:


  • David Roberts
    replied
    Unless I'm mistaken a slight tweak was all that was required. The problem that was solved could only exit one way.

    Code:
    Local adjust, oldb As Long
    adjust = [COLOR="Red"]Fix[/COLOR](Log10(Abs(a))) ' From Int to Fix
    oldb = b    [COLOR="Red"]<= Tweak[/COLOR]
    If adjust < [COLOR="Red"]0[/COLOR] Then ' From -1 to 0
      adjust = Abs( adjust )
      [COLOR="Red"]'Decr adjust[/COLOR] ' and remove this
      a = a*10^adjust
      b = b - adjust
    End If
    and replace b with oldb when we check |sizeA| = 5.
    Last edited by David Roberts; 20 Jul 2008, 11:18 PM. Reason: A 'Print' was left over from debugging

    Leave a comment:


  • John Gleason
    replied
    Paul, I sort of see what ya mean there. I'd say work on it but limit it to 18 digits at least at first. I'm trying to work on the "result of a numerical computation?" thang.

    Leave a comment:


  • David Roberts
    replied
    My preference tho for this special situation would be the pre-shift; round function; post-shift method.
    If I wore a commercial hat, John, my preference would be for a seamless solution - less telephone calls.

    Leave a comment:


  • John Gleason
    replied
    Ah, I see you noticed it Dave. I'm pretty quick eh?

    Leave a comment:


  • John Gleason
    replied
    >> Would you give me an example, John?
    Code:
    -.05191759539913762 << original EXT
    rV4-.51917595399137619      -.519175953991 12
    rV3                         -.519175953991 9 random
    Here's an example, tho it's not really a rounding alteration, rather it is a shift of the decimal. I'm pretty sure I could fix it either by passing the EXT param BYREF, adding a copy of the param, or maybe redividing by adjust afterward. My preference tho for this special situation would be the pre-shift; round function; post-shift method.

    Leave a comment:


  • Paul Purvis
    replied
    i said i resolved myself to finalizing the roundme routines, but i unresolved myself.
    ever get the feeling some answers to your problems are reaching out and slapping you in the face, that is the feeling i have been having and i am just staring back
    there should be a speed up and maybe even possibility of getting over the 18 decimal place with some new thoughts.

    John,
    i think we can use the ROUND statement once again to our advantage.
    lets focus on positive numbers for simplicity

    in the roundme6c function, the ROUND statement is used to get a quick answer if the a result is larger than the value give.
    let us do the same but after that piece of code of where we know round is less than the value let us do this

    z being a value of something like 4999*something

    if round(x,y)+z >value then result is equal to round(x,y)

    Leave a comment:


  • David Roberts
    replied
    Found some, John.

    The rounding aspect seems fine, it is just the decimal point that is adrift. There is a conflict with your code later on depending upon the exit path. I'll work on it.

    Leave a comment:


  • Paul Purvis
    replied
    David,
    i think you missed a post.
    it was mentioned sometime back that these routines do not round any 18 digits but only up to 18 digits to the right of the decimal place.
    is was late last night when i read your post and i even forgot about that.
    Last edited by Paul Purvis; 20 Jul 2008, 12:38 PM.

    Leave a comment:


  • David Roberts
    replied
    Adding the snippet to the function alters all "0.0*" rounding.
    Would you give me an example, John?

    Leave a comment:


  • John Gleason
    replied
    Originally posted by David Roberts View Post
    Ok but we wouldn't be expecting a user of roundupV3 or 4 to do that and what if, which is likely, the value is the result of a numerical computation?
    num = Val("0.0000000000314999999999999999")
    Before adding snippet...
    Given the very tiny number represented and the rarity of probable need for rounding accuracy to 19 digits or greater on that number, my thought is that it's reasonable to pre-process it, round it using 18-digit accuracy, then post-process it. Adding the snippet to the function alters all "0.0*" rounding.

    My current huge concern is, "what if, which is likely, the value is the result of a numerical computation?" If possible, I'd like a way to feed or preferably have the functions accept computed EXT values in a reliable way.

    Leave a comment:


  • David Roberts
    replied
    Paul

    what snippet are you referring too.

    i noticed you did not mention the results of roundme6c which does comes up with the correct answer.
    Post #392 and I mentioned roundme6c a short while after posting the snippet.

    PS Paul Dixon's ASM routine is OK too.
    Last edited by David Roberts; 20 Jul 2008, 03:42 AM.

    Leave a comment:


  • Paul Purvis
    replied
    David

    thanks for reporting that returned error

    it might be worth while to concentrate on values with 499??

    i see what you mean on that Val("0.0123456784999999995")
    what snippet are you referring too.

    i noticed you did not mention the results of roundme6c which does comes up with the correct answer.

    Leave a comment:


  • David Roberts
    replied
    I think that I've mentioned that I have no use for what is developing here and PB's Round suits me especially when adding a column of results.

    However, a scientific or manufacturing requirement may be such that 2.5, say, rounds to 3 and not 2.

    It boils down to "Does it matter and if it does then decide which to choose and if not then PB's Round".

    Leave a comment:


  • Rodney Hicks
    replied
    Having read through 400 posts I come to the conclusion that if I want individual numbers to visually correct, that is, in compliance with the rounding method taught in grade schools, use one of the ones developed here in this thread.

    But if I want to use a number that yields more accurate results on average when used in calculations, use PB's ROUND(var,place)

    Rod

    Leave a comment:


  • David Roberts
    replied
    It is worth noting that I wasn't considering 315/10000.

    I was concerned about a sequence like 49999999 being rounded up to a 5 on entering the quad mode. The 5 then caused the round up code to round up our value AGAIN. This is clearly a bug.

    Here is another one and shows that we don't even need to end on a '9'. A '5' will cause a cascade.

    num = Val("0.0123456784999999995")

    Before adding snippet.

    roundUpV3(num, 9) => .012345679

    We have then a round up on '...5' algoritm rounding up on '...4999999995'.

    After adding snippet.

    roundUpV3(num, 9) => .012345678

    Leave a comment:


  • Walter Henn
    replied
    Post deleted.
    Last edited by Walter Henn; 20 Jul 2008, 03:36 AM.

    Leave a comment:


  • Walter Henn
    replied
    Post deleted
    Last edited by Walter Henn; 21 Jul 2008, 06:12 AM. Reason: The quote from a previous post of mine has been slightly modified to provide greater clarity.

    Leave a comment:


  • Paul Purvis
    replied
    removed for the purposes of not confusing anybody
    Last edited by Paul Purvis; 19 Jul 2008, 11:54 PM.

    Leave a comment:

Working...
X