If you write
x = 3.045##
STR$(x, 18) return exactly 3.045
x = 3.045##
STR$(x, 18) return exactly 3.045
Floats are often approximations.
This particular scenario is just begging for:
Code:
LOCAL X AS CUR X = [email protected]
MCM
LOCAL X AS CUR X = [email protected]
#COMPILE EXE #DIM ALL FUNCTION PBMAIN () AS LONG LOCAL x AS EXT x = VAL("3.045") x = ROUND(x, 2) ? STR$(x, 18) END FUNCTION
ext val = 305836372081947.55 round digits = 1 roundCommon = 305836372081947.5 roundUpV3 = 305836372081947.6 ext val = 5052478737236737.55 round digits = 1 roundCommon = 5052478737236737.5 roundUpV3 = 5052478737236737.6 ext val = 9830868917420.82595 round digits = 4 roundCommon = 9830868917420.8259 roundUpV3 = 9830868917420.826 ext val = 10937049177568.15 round digits = 1 roundCommon = 10937049177568.1 roundUpV3 = 10937049177568.2 ext val = 5273843846123.20805 round digits = 4 roundCommon = 5273843846123.208 roundUpV3 = 5273843846123.2081 ext val = 89655156198343.4155 round digits = 3 roundCommon = 89655156198343.415 roundUpV3 = 89655156198343.416 ext val = 87225388707924.7075 round digits = 3 roundCommon = 87225388707924.707 roundUpV3 = 87225388707924.708 ext val = 4419172709427.19405 round digits = 4 roundCommon = 4419172709427.194 roundUpV3 = 4419172709427.1941 ext val = 2647177321133404.15 round digits = 1 roundCommon = 2647177321133404.1 roundUpV3 = 2647177321133404.2 ext val = 49036207327374.0515 round digits = 3 roundCommon = 49036207327374.051 roundUpV3 = 49036207327374.052 ext val = 4933067151369.03745 round digits = 4 roundCommon = 4933067151369.0374 roundUpV3 = 4933067151369.0375 ext val = 6186623783298.8815 round digits = 3 roundCommon = 6186623783298.881 roundUpV3 = 6186623783298.882 ext val = 1087885000.888785 round digits = 5 roundCommon = 1087885000.88878 roundUpV3 = 1087885000.88879 ext val = 5656671723178.33375 round digits = 4 roundCommon = 5656671723178.3337
FUNCTION roundUpV3(a AS EXT, b AS LONG) AS EXT '--------------------------------------------------------------------------------------- 'rounds "a" to "b" digits past decimal point, but up on 5 instead of using 'banker's rounding. It rounds as you would expect by just looking at the decimal numbers, 'as if the computer were a decimal computer. eg. roundUpV3(123.4545, 3) becomes 123.455. '--------------------------------------------------------------------------------------- STATIC x AS EXT STATIC qa AS QUAD STATIC fillOnce, sizeA AS LONG DIM ten(18) AS STATIC QUAD IF a < 0 THEN 'handle negatives x = ROUND(a, b) 'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding IF x <= a THEN 'rounded up, so no need to go thru hoops. FUNCTION = x EXIT FUNCTION END IF ELSE x = ROUND(a, b) 'in 8.04 (and probably most recent CC ver.) round to even, or banker's rounding IF x >= a THEN 'rounded up FUNCTION = x EXIT FUNCTION END IF END IF 'make powers of 10 to avoid exponent calcs IF fillOnce = 0 THEN ten(00) = 1 ten(01) = 10 ten(02) = 100 ten(03) = 1000 ten(04) = 10000 ten(05) = 100000 ten(06) = 1000000 ten(07) = 10000000 ten(08) = 100000000 ten(09) = 1000000000 ten(10) = 10000000000 ten(11) = 100000000000 ten(12) = 1000000000000 ten(13) = 10000000000000 ten(14) = 100000000000000 ten(15) = 1000000000000000 ten(16) = 10000000000000000 ten(17) = 100000000000000000 ten(18) = 1000000000000000000 fillOnce = 1 END IF 'how big is a? SELECT CASE ABS(a) CASE < ten(00): sizeA = 01: qa = a * ten(18) CASE < ten(01): sizeA = 02: qa = a * ten(17) CASE < ten(02): sizeA = 03: qa = a * ten(16) CASE < ten(03): sizeA = 04: qa = a * ten(15) CASE < ten(04): sizeA = 05: qa = a * ten(14) CASE < ten(05): sizeA = 06: qa = a * ten(13) CASE < ten(06): sizeA = 07: qa = a * ten(12) CASE < ten(07): sizeA = 08: qa = a * ten(11) CASE < ten(08): sizeA = 09: qa = a * ten(10) CASE < ten(09): sizeA = 10: qa = a * ten(09) CASE < ten(10): sizeA = 11: qa = a * ten(08) CASE < ten(11): sizeA = 12: qa = a * ten(07) CASE < ten(12): sizeA = 13: qa = a * ten(06) CASE < ten(13): sizeA = 14: qa = a * ten(05) CASE < ten(14): sizeA = 15: qa = a * ten(04) CASE < ten(15): sizeA = 16: qa = a * ten(03) CASE < ten(16): sizeA = 17: qa = a * ten(02) CASE < ten(17): sizeA = 18: qa = a * ten(01) CASE ELSE FUNCTION = a EXIT FUNCTION 'exit because a is too big to round END SELECT IF sizeA + b > 18 THEN FUNCTION = a EXIT FUNCTION 'exit because round digits too big END IF qa = qa \ ten(18 - (sizeA + b)) 'make whole EXT effectively an integer sizeA = qa MOD 10 IF sizeA = 5 THEN 'is last digit a 5? FUNCTION = ((qa + 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position ELSEIF sizeA = -5 THEN 'or -5 FUNCTION = ((qa - 10) \ 10) / ten(b)'if so round up prev. digits and replace decimal point in correct position ELSE FUNCTION = x END IF END FUNCTION
if digit >= 5 then UP else DOWN endif
if digit > 5 then UP elseif digit < 5 then DOWN else chouse the next evens ... 5.0000000... DOWN 5.0000001... UP endif
FUNCTION RoundCommon##(rdVal##, BYVAL rlDec&) LOCAL lShift&, lSign&, qCalc&& lShift& = CHOOSE&(rlDec&, 100, 1000, 10000, 100000, 1000000) lSign& = SGN(rdVal##) qCalc&& = CQUD(FIX(rdVal## * lShift&)) qCalc&& = ABS(qCalc&&) qCalc&& = qCalc&& + 5 qCalc&& = qCalc&& * lSign& qCalc&& = qCalc&& \ 10 lShift& = CHOOSE&(rlDec&, 10, 100, 1000, 10000, 100000) FUNCTION = CEXT(qCalc&& / lShift&) END FUNCTION
We process personal data about users of our site, through the use of cookies and other technologies, to deliver our services, and to analyze site activity. For additional details, refer to our Privacy Policy.
By clicking "I AGREE" below, you agree to our Privacy Policy and our personal data processing and cookie practices as described therein. You also acknowledge that this forum may be hosted outside your country and you consent to the collection, storage, and processing of your data in the country where this forum is hosted.
Leave a comment: