Announcement

Collapse
No announcement yet.

VAL rounds a variable

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • sorry one better
    this might show a way to avoid using the VAL possibly for input as literal and the first code may be a way to better assign a EXT from a string input
    but does not solve rounding
    if input is a string, maybe best to split string into two integer values and divide by number of decimal places used in input string

    fire at will

    Code:
    'spit input into j and i
    'the string is "16.585"  then split string into integers
    
    j=16585
    i=3    
    if i>0 then
    n=j/10^i 
    else
    n=j
    end if
    code to show using literals

    Code:
    #COMPILE EXE
    #DIM ALL
    
    FUNCTION PBMAIN () AS LONG
        LOCAL j AS LONG
        LOCAL m AS EXT
        LOCAL n AS EXT
        j=16585        'ok
        n=j/1000       'ok
    
        'above delivers 16.585 into EXT variable n
        
    
        
        m=16.585       'bad input
       'm=16.585##   'bad input as well
        PRINT m
        PRINT STR$(m,18)
        PRINT n
        PRINT STR$(n,18)
        IF m=n THEN PRINT "match"
        IF m<>n THEN PRINT "no match"
      
        m=VAL("16.585")  'ok probably and should be slower than other ways
        PRINT m
        PRINT STR$(m,18)
        PRINT n
        PRINT STR$(n,18)
        IF m=n THEN PRINT "match"
        IF m<>n THEN PRINT "no match"
    
        m=16585##/1000##   'ok
        PRINT m
        PRINT STR$(m,18)
        PRINT n
        PRINT STR$(n,18)
        IF m=n THEN PRINT "match"
        IF m<>n THEN PRINT "no match"
         
        m=16585/1000   'ok
        PRINT m
        PRINT STR$(m,18)
        PRINT n
        PRINT STR$(n,18)
        IF m=n THEN PRINT "match"
        IF m<>n THEN PRINT "no match"
    
    
        m=16585   'ok
       m=m/1000  'ok
        PRINT m
        PRINT STR$(m,18)
        PRINT n
        PRINT STR$(n,18)
        IF m=n THEN PRINT "match"
        IF m<>n THEN PRINT "no match"
    
          WAITKEY$
    END FUNCTION
    Last edited by Paul Purvis; 1 Jul 2008, 03:37 AM.
    p purvis

    Comment


    • Originally posted by paul d purvis View Post
      Code:
       (1)
      'spit input into j and i
      'the string is "16.585" then split string into integers
       
      j=16585
      i=3 
      if i>0 then
      n=j/10^i 
      else
      n=j
      end if
      Code:
       (2)
      ...
      m=VAL("16.585") 'ok probably and should be slower than other ways
      PRINT m
      PRINT STR$(m,18)
      PRINT n
      PRINT STR$(n,18)
      IF m=n THEN PRINT "match"
      IF m<>n THEN PRINT "no match"
       
      m=16585##/1000## 'ok
      PRINT m
      PRINT STR$(m,18)
      PRINT n
      PRINT STR$(n,18)
      IF m=n THEN PRINT "match"
      IF m<>n THEN PRINT "no match"
      ...

      You might be interested in knowing that VAL ("16.585") in code snippet (2) above stores exactly the same value in m## as 16585/1000 (or 16585##/1000##). However, VAL is a function and takes approx. four times as long.

      As far as code snippet (1) above for inputting EXT values, the statement, INPUT m (where m is an EXT variable), produces the same result as using VAL("value of m"). There is no loss in precision (e.g., EXT to DOUBLE) when you input values to EXT variables. Only EXT literals within the program lose precision because the maximum precision for numeric literals is DOUBLE. If,for some reason, you're using string input, use VAL(string variable) to assign to an EXT variable. Obviously, the speed penalty for converting input strings to EXT by using VAL is not a consideration.
      Last edited by Walter Henn; 1 Jul 2008, 05:37 AM.

      Comment


      • Paul Purvis said: ...16585/1000
        Interesting technique Paul, and like Walter's, my test loops I showed it to produce exactly the same value and be over EIGHT times faster on my machine. That's an efficient workaround.

        As far as the rounding differences observed in the various routines, this is what I think is happening:
        *Paul Purvis is looking for a round up on 5 based on "visual rounding" and this is what I solely based my function roundUp and roundUpV2 on, as specified in the IEEC post.
        *Walter is looking for a round function that rounds up on 5, but only if the base2 representation of the "rounding 5" actually is equal to or exceeds 5 internally. Let's call this, "base2 round up on 5", and is what Paul Dixon's code has been based on I believe. Maybe we should make an IEEC spec for this too!
        *roundUp failed, as Walter pointed out, because it rounded up .46 - .499... tails, and eventually roundUpV2 replaced it. So far, roundUpV2 has shown perfect results in all tests to 18 digits to perform "visual round up on 5".
        *Interest in a "visual round down on 5" has been expressed (a "base2 round down on 5" might be possible too).

        Probably all can be done, but are best done one at a time to avoid confusion.

        So, 1) Paul P., how does roundUpV2 look to you? Do you see any problems or performance issues? It matches your (what I called) roundMe3 string version thru 15 or 16 digits, and roundMe4 with the noted exceptions so far in testing.
        2) Walter, how are asmRoundUp/round_dixon looking? Any problems or exceptions there?

        Comment


        • hi John,

          i did not see anybody express the literals as i did recently with the integer assignments then division or simpler, an assignment with an integer being divided by the number of decimals needed.
          i want to test later today those assignments against that database i produced. it is going to be a long day on other things, but i will test tonight.
          i ran a test on a small sampling to spot check the integer/decimals assignment and everything seemed ok. so as it stands to continue to be ok, any program rounding on 5 up is going to get my support.

          on a additional thought from what i have seen recently
          a word of caution might be worthy when adding .5 or .05 or .005
          it might be wise to assign a variable then use the variable in a expression or inside the expression use (5/10), (5/100), or (5/1000)
          this might give a more precise number for rounding, but i have not tested it, but my gut feels right about it.

          and on a much bigger plane(i am going to place myself in the firing lane on this one)
          i would suggest anytime you have a expression that involves a decimal literal, you use the integer/decimal structure instead of decimal literal.

          Last edited by Paul Purvis; 1 Jul 2008, 10:55 AM.
          p purvis

          Comment


          • Originally posted by paul d purvis View Post
            i did not see anybody express the literals as i did recently with the integer assignments then division or simpler, an assignment with an integer being divided by the number of decimals needed.
            I didn't either so thought I'd mention it; good thing you came up with it and thanks for mentioning that I did not. j/k. Any time you have code that is 8x faster (and correct hehheh), feel free to post a snippet.

            Originally posted by paul d purvis View Post
            and on a much bigger plane(i am going to place myself in the firing lane on this one)
            i would suggest anytime you have a expression that involves a decimal literal, you use the integer/decimal structure instead of decimal literal.
            I plan on taking that advice assuming the tests work out, which I'm pretty sure they will. I'll test a bunch against VAL("string representation of #")

            Comment


            • Here's a program that creates a .bas which tests over 6,400 literals at a time. You can change the STEP value to most anything to vary the test, but probably prime numbers are best.
              Code:
              #COMPILE EXE
              #DIM ALL
              MACRO ifErr = IF xs <> xs2 THEN ? "error" & STR$(xs,18) & STR$(xs2,18): WAITKEY$
              
              FUNCTION PBMAIN () AS LONG
              
                  LOCAL ii AS EXT, xs, xs2 AS STRING
                  LOCAL t AS DOUBLE
                  OPEN "c:\pbLiteralTest.bas" FOR OUTPUT AS #1
                  PRINT #1, "#COMPILE EXE"
                  PRINT #1, "#DIM ALL"
                  PRINT #1, "MACRO ifErr = IF xs <> xs2 THEN ? ""error"" & STR$(xs,18) & STR$(xs2,18): WAITKEY$"
                  PRINT #1, " "
                  PRINT #1, "FUNCTION PBMAIN () AS LONG"
                  PRINT #1, "LOCAL xs, xs2 AS EXT"
                  FOR ii = 1 TO 11000000000 STEP 1712467
                        xs = "xs = val(""" & FORMAT$(ii / 1000000000, "0.000000000") & """)"
                        xs2 = "xs2 = " & FORMAT$(ii) & " / 1000000000"
                        PRINT #1, xs & ":" & xs2 & ":" & "ifErr"
                  NEXT
                  PRINT #1, "? ""done"" : WAITKEY$"
                  PRINT #1, "end function"
                  EXIT FUNCTION
              
              END FUNCTION
              Here is a little sample of what it creates for the test:
              Code:
              #COMPILE EXE
              #DIM ALL
              MACRO ifErr = IF xs <> xs2 THEN ? "error" & STR$(xs,18) & STR$(xs2,18): WAITKEY$
               
              FUNCTION PBMAIN () AS LONG
              LOCAL xs, xs2 AS EXT
              xs = VAL("0.000000001"):xs2 = 1 / 1000000000:ifErr
              xs = VAL("0.001712468"):xs2 = 1712468 / 1000000000:ifErr
              xs = VAL("0.003424935"):xs2 = 3424935 / 1000000000:ifErr
              xs = VAL("0.005137402"):xs2 = 5137402 / 1000000000:ifErr
              xs = VAL("0.006849869"):xs2 = 6849869 / 1000000000:ifErr
              xs = VAL("0.008562336"):xs2 = 8562336 / 1000000000:ifErr
              xs = VAL("0.010274803"):xs2 = 10274803 / 1000000000:ifErr
              xs = VAL("0.011987270"):xs2 = 11987270 / 1000000000:ifErr
              xs = VAL("0.013699737"):xs2 = 13699737 / 1000000000:ifErr
              xs = VAL("0.015412204"):xs2 = 15412204 / 1000000000:ifErr
              xs = VAL("0.017124671"):xs2 = 17124671 / 1000000000:ifErr
              xs = VAL("0.018837138"):xs2 = 18837138 / 1000000000:ifErr
              xs = VAL("0.020549605"):xs2 = 20549605 / 1000000000:ifErr
              xs = VAL("0.022262072"):xs2 = 22262072 / 1000000000:ifErr
              xs = VAL("0.023974539"):xs2 = 23974539 / 1000000000:ifErr
              xs = VAL("0.025687006"):xs2 = 25687006 / 1000000000:ifErr
              xs = VAL("0.027399473"):xs2 = 27399473 / 1000000000:ifErr
              xs = VAL("0.029111940"):xs2 = 29111940 / 1000000000:ifErr
              xs = VAL("0.030824407"):xs2 = 30824407 / 1000000000:ifErr
              xs = VAL("0.032536874"):xs2 = 32536874 / 1000000000:ifErr
              xs = VAL("0.034249341"):xs2 = 34249341 / 1000000000:ifErr
              xs = VAL("0.035961808"):xs2 = 35961808 / 1000000000:ifErr
              xs = VAL("0.037674275"):xs2 = 37674275 / 1000000000:ifErr
              xs = VAL("0.039386742"):xs2 = 39386742 / 1000000000:ifErr
              xs = VAL("0.041099209"):xs2 = 41099209 / 1000000000:ifErr
              xs = VAL("0.042811676"):xs2 = 42811676 / 1000000000:ifErr
              xs = VAL("0.044524143"):xs2 = 44524143 / 1000000000:ifErr
              xs = VAL("0.046236610"):xs2 = 46236610 / 1000000000:ifErr
              xs = VAL("0.047949077"):xs2 = 47949077 / 1000000000:ifErr
              ? "done" : WAITKEY$
              END FUNCTION

              Comment


              • unless i read wrong on the internet,
                the assigning of literals and rounding problems has plagued other compilers as well.
                i will probably standardize on the decimal assignments with the integer/decimals on all other programs that i use and will use in the future, it is a reasonable workaround although i tried integer*decimals(.1,.01,.001,etc) to keep from getting a division by zero error, and i was also believed that multiplication is faster than division.

                a few years ago, i had a very similar problem using cbasic86 to match values with decimals and i could not place my finger on the problem, it happened rarely, and i did not have a resource such as this forum to help me with my problems, but now i can see where my problem with matching of values where probably the same problems that where faced in this thread.
                Last edited by Paul Purvis; 1 Jul 2008, 03:59 PM.
                p purvis

                Comment


                • John

                  Seven significant figures: 1712467?

                  Try stressing the idea a bit and consider 15 significant figures. Not so rosy now I'm afraid.

                  Comment


                  • David
                    do you have some specific code to share
                    p purvis

                    Comment


                    • I'll check that Dave, tho the sad face is looking ominous, portending doom...

                      Comment


                      • Hey, good news, 6500 of these below worked. I'll test some more, but looking good so far.
                        Code:
                        #COMPILE EXE
                        #DIM ALL
                        MACRO ifErr = IF xs <> xs2 THEN ? "error" & STR$(xs,18) & STR$(xs2,18): WAITKEY$
                         
                        FUNCTION PBMAIN () AS LONG
                        LOCAL xs, xs2 AS EXT
                        xs = VAL("10.0000000000000000"):xs2 = 100000000000000000 / 10000000000000000:ifErr
                        xs = VAL("10.0000017124673981"):xs2 = 100000017124673981 / 10000000000000000:ifErr
                        xs = VAL("10.0000034249347962"):xs2 = 100000034249347962 / 10000000000000000:ifErr
                        xs = VAL("10.0000051374021943"):xs2 = 100000051374021943 / 10000000000000000:ifErr
                        xs = VAL("10.0000068498695924"):xs2 = 100000068498695924 / 10000000000000000:ifErr
                        xs = VAL("10.0000085623369905"):xs2 = 100000085623369905 / 10000000000000000:ifErr
                        xs = VAL("10.0000102748043886"):xs2 = 100000102748043886 / 10000000000000000:ifErr
                        xs = VAL("10.0000119872717867"):xs2 = 100000119872717867 / 10000000000000000:ifErr
                        xs = VAL("10.0000136997391848"):xs2 = 100000136997391848 / 10000000000000000:ifErr
                        xs = VAL("10.0000154122065829"):xs2 = 100000154122065829 / 10000000000000000:ifErr
                        xs = VAL("10.0000171246739810"):xs2 = 100000171246739810 / 10000000000000000:ifErr
                        xs = VAL("10.0000188371413791"):xs2 = 100000188371413791 / 10000000000000000:ifErr
                        xs = VAL("10.0000205496087772"):xs2 = 100000205496087772 / 10000000000000000:ifErr
                        xs = VAL("10.0000222620761753"):xs2 = 100000222620761753 / 10000000000000000:ifErr
                        xs = VAL("10.0000239745435734"):xs2 = 100000239745435734 / 10000000000000000:ifErr
                        xs = VAL("10.0000256870109715"):xs2 = 100000256870109715 / 10000000000000000:ifErr
                        xs = VAL("10.0000273994783696"):xs2 = 100000273994783696 / 10000000000000000:ifErr
                        xs = VAL("10.0000291119457677"):xs2 = 100000291119457677 / 10000000000000000:ifErr
                        xs = VAL("10.0000308244131658"):xs2 = 100000308244131658 / 10000000000000000:ifErr
                        xs = VAL("10.0000325368805639"):xs2 = 100000325368805639 / 10000000000000000:ifErr
                        xs = VAL("10.0000342493479620"):xs2 = 100000342493479620 / 10000000000000000:ifErr
                        xs = VAL("10.0000359618153601"):xs2 = 100000359618153601 / 10000000000000000:ifErr
                        xs = VAL("10.0000376742827582"):xs2 = 100000376742827582 / 10000000000000000:ifErr
                        xs = VAL("10.0000393867501563"):xs2 = 100000393867501563 / 10000000000000000:ifErr
                        xs = VAL("10.0000410992175544"):xs2 = 100000410992175544 / 10000000000000000:ifErr
                        xs = VAL("10.0000428116849525"):xs2 = 100000428116849525 / 10000000000000000:ifErr
                        xs = VAL("10.0000445241523506"):xs2 = 100000445241523506 / 10000000000000000:ifErr
                        xs = VAL("10.0000462366197487"):xs2 = 100000462366197487 / 10000000000000000:ifErr
                        xs = VAL("10.0000479490871468"):xs2 = 100000479490871468 / 10000000000000000:ifErr
                        xs = VAL("10.0000496615545449"):xs2 = 100000496615545449 / 10000000000000000:ifErr
                        xs = VAL("10.0000513740219430"):xs2 = 100000513740219430 / 10000000000000000:ifErr
                        xs = VAL("10.0000530864893411"):xs2 = 100000530864893411 / 10000000000000000:ifErr
                        xs = VAL("10.0000547989567392"):xs2 = 100000547989567392 / 10000000000000000:ifErr
                        xs = VAL("10.0000565114241373"):xs2 = 100000565114241373 / 10000000000000000:ifErr
                        xs = VAL("10.0000582238915354"):xs2 = 100000582238915354 / 10000000000000000:ifErr
                        xs = VAL("10.0000599363589335"):xs2 = 100000599363589335 / 10000000000000000:ifErr
                        xs = VAL("10.0000616488263316"):xs2 = 100000616488263316 / 10000000000000000:ifErr
                        xs = VAL("10.0000633612937297"):xs2 = 100000633612937297 / 10000000000000000:ifErr
                        xs = VAL("10.0000650737611278"):xs2 = 100000650737611278 / 10000000000000000:ifErr
                        xs = VAL("10.0000667862285259"):xs2 = 100000667862285259 / 10000000000000000:ifErr
                        xs = VAL("10.0000684986959240"):xs2 = 100000684986959240 / 10000000000000000:ifErr
                        xs = VAL("10.0000702111633221"):xs2 = 100000702111633221 / 10000000000000000:ifErr
                        xs = VAL("10.0000719236307202"):xs2 = 100000719236307202 / 10000000000000000:ifErr
                        xs = VAL("10.0000736360981183"):xs2 = 100000736360981183 / 10000000000000000:ifErr
                        xs = VAL("10.0000753485655164"):xs2 = 100000753485655164 / 10000000000000000:ifErr
                        xs = VAL("10.0000770610329145"):xs2 = 100000770610329145 / 10000000000000000:ifErr
                        xs = VAL("10.0000787735003126"):xs2 = 100000787735003126 / 10000000000000000:ifErr
                        xs = VAL("10.0000804859677107"):xs2 = 100000804859677107 / 10000000000000000:ifErr
                        xs = VAL("10.0000821984351088"):xs2 = 100000821984351088 / 10000000000000000:ifErr
                        xs = VAL("10.0000839109025069"):xs2 = 100000839109025069 / 10000000000000000:ifErr
                        xs = VAL("10.0000856233699050"):xs2 = 100000856233699050 / 10000000000000000:ifErr
                        xs = VAL("10.0000873358373031"):xs2 = 100000873358373031 / 10000000000000000:ifErr
                        xs = VAL("10.0000890483047012"):xs2 = 100000890483047012 / 10000000000000000:ifErr
                        xs = VAL("10.0000907607720993"):xs2 = 100000907607720993 / 10000000000000000:ifErr
                        xs = VAL("10.0000924732394974"):xs2 = 100000924732394974 / 10000000000000000:ifErr
                        xs = VAL("10.0000941857068955"):xs2 = 100000941857068955 / 10000000000000000:ifErr
                        xs = VAL("10.0000958981742936"):xs2 = 100000958981742936 / 10000000000000000:ifErr
                        xs = VAL("10.0000976106416917"):xs2 = 100000976106416917 / 10000000000000000:ifErr
                        xs = VAL("10.0000993231090898"):xs2 = 100000993231090898 / 10000000000000000:ifErr
                        xs = VAL("10.0001010355764879"):xs2 = 100001010355764879 / 10000000000000000:ifErr

                        Comment


                        • a poor test was removed
                          Last edited by Paul Purvis; 2 Jul 2008, 06:25 PM.
                          p purvis

                          Comment


                          • The fraction method vs VAL is OK. I made an error in the formatting.

                            I should add that there is a danger of gettng preoccupied with decimals.

                            If two binary representations are the same then their decimal counterparts will be the same. However, it does not follow that if two decimals are the same then their binary representations whence they came are the same. In other words it is possible for two binary representations to map into the same decimal space. We have then a collision in the same sense as in Hashing algorithms.

                            What I've been looking at is when a decimal counterpart is rounded up causing a cascade to the chosen round off point which in turn is rounded up by our rounding algorithm. The result being an erroneous round up. The cascade can be avoided if the decimal counterpart is a truncated value but I'm not sure how to do that. This is Paul Dixon's domain for sure.

                            Comment


                            • Code:
                               
                              'Validation of Paul Purvis' integer/divide technique for assigning
                              'literal values to EXT variables by comparing his results against VAL(string).
                               
                              #COMPILE EXE
                              #DIM ALL
                               
                              FUNCTION PBMAIN () AS LONG
                               
                              LOCAL number1, number2, random_number AS EXT
                              LOCAL divisor_digits, decimal_point AS LONG
                              LOCAL counter AS QUAD
                              LOCAL string1, string2 AS STRING
                              LOCAL exponent, exponent_position AS LONG
                               
                              FOR counter = 1 TO 1000000000
                                  random_number = RND * 10 ' random numbers >= 0 and less than 10
                                  string1 = STR$(random_number,18)'convert random number to string; accuracy of conversion is not critical.
                                  number1 = VAL(string1) 'assign VAL(string1) to EXT number1
                                  exponent_position = INSTR(string1, ANY "eE")
                                  IF exponent_position <> 0 THEN
                                      exponent = VAL(MID$(string1, exponent_position +1))
                                      string2 = LEFT$(string1, exponent_position -1) 'get rid of exponent
                                  ELSE
                                      string2 = string1
                                      exponent = 0
                                  END IF
                                  decimal_point = INSTR(string2, ".")
                                  IF decimal_point = 0 THEN decimal_point = LEN(string2)
                                  divisor_digits = LEN(string2) - decimal_point - exponent
                                  string2 = REMOVE$(string2, ".") ' get rid of any decimal point
                                  number2 = VAL(string2)/10^divisor_digits 'string2 and 10^divisor_digits are whole numbers
                                  IF number1 <> number2 THEN
                                      PRINT "error"
                                      PRINT "VAL(string1): "; STR$(VAL(string1), 18)
                                      IF divisor_digits => 0 THEN
                                          PRINT "Int/Dec:      "; string2 +  "/ " + "1" + REPEAT$(divisor_digits, "0")
                                      ELSE
                                          PRINT "Int/Dec:      "; string2 + " * " + "1" + REPEAT$(-divisor_digits, "0")
                                      END IF
                                      PRINT "number1:      "; STR$(number1,18)
                                      PRINT "number2:      "; STR$(number2, 18)
                                      PRINT "diff: "; STR$(number1- number2, 18)
                                      PRINT REPEAT$(20, "-")
                                  END IF
                              NEXT counter
                              PRINT "End  of test!"
                              WAITKEY$
                              END FUNCTION

                              The code above can be used to test Paul Purvis' technique to get EXT precision from numeric literals by using the integer/divide approach instead of the VAL(string) approach. The program generates random numbers and can easily be used to verify billions of test values. Paul's technique, which is quite clever, appears to give exactly the same value as the VAL(string) approach, but is much faster by a factor of approx. 4-8 times.
                              Last edited by Walter Henn; 2 Jul 2008, 07:52 PM.

                              Comment


                              • message removed the results to a poor test
                                Last edited by Paul Purvis; 2 Jul 2008, 06:24 PM.
                                p purvis

                                Comment


                                • LET statement, get acquainted with it

                                  i never read in manuals some documentation.

                                  not reading the section on the LET statement and understanding it fully,
                                  it is easy to miss that assignments(not math expressions) of an extended precision(EXTENDED OR EXT) and an extended currency variable(CURRENCYX OR CUX) to a variable are probably not supported in versions of pbcc 4.04 because it is not specifically written in the manual and easily over looked or like i said just misread.

                                  the documentation gives information on assigning VARIANT variables.
                                  i have only used a VARIANT variables from coping somebody else's code.
                                  VARIANTS are stored with 128 bits, huh.
                                  maybe i need to get to know a little more about using VARIANT variables.

                                  the manual and online documentation states this
                                  When assigning values to variants, PowerBASIC automatically analyzes the expression and chooses an appropriate or string data type for the internal representation of the Variant. However, you can specify a particular preferred format by adding the optional AS vartype clause. For example:
                                  LET xyz = 21.34 AS LONG
                                  In the above example, the Variant would actually contain the value 21, as the programmer forced it to Long-integer representation. vartype may be one of the following keywords: BYTE, WORD, DWORD, INTEGER, LONG, QUAD, SINGLE, DOUBLE, CURRENCY, STRING.
                                  i would guess the compiler also automatically analyzes the expression and chooses an appropriate or string data type for other variables as well.

                                  added: maybe powerbasic would interested in creating a section called manuals or something similar where they could add, change, or remove documentation as needed in just one single place for official statements concerning the use of their products.
                                  Last edited by Paul Purvis; 2 Jul 2008, 04:03 PM.
                                  p purvis

                                  Comment


                                  • Originally posted by paul d purvis View Post
                                    i never read in manuals some documentation.

                                    not reading the section on the LET statement and understanding it fully,
                                    it is easy to miss that assignments(not math expressions) of an extended precision(EXTENDED OR EXT) and an extended currency variable(CURRENCYX OR CUX) to a variable are probably not supported in versions of pbcc 4.04 because it is not specifically written in the manual and easily over looked or like i said just misread.

                                    the documentation gives information on assigning VARIANT variables.
                                    i have only used a VARIANT variables from coping somebody else's code.
                                    VARIANTS are stored with 128 bits, huh.
                                    maybe i need to get to know a little more about using VARIANT variables...
                                    You're losing me on this post, Paul. Does the LET statement or variant variables have anything, at all, to do with the topic of rounding or VAL in this thread? If not, I respectfully request that you start a new thread.

                                    Comment


                                    • Originally posted by paul d purvis View Post
                                      will this program suffice as a test


                                      Code:
                                       
                                      #COMPILE EXE
                                      #DIM ALL
                                      FUNCTION PBMAIN () AS LONG
                                      LOCAL m AS EXT
                                      LOCAL s AS STRING
                                      LOCAL ss AS STRING
                                      LOCAL i AS QUAD
                                      LOCAL count AS QUAD
                                       
                                      FOR i =90000000000000000 TO 99999999999999999 STEP 1712467
                                      s=TRIM$(STR$(i,18))
                                      s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)-1)
                                      trimzeroout:
                                      IF RIGHT$(s,1)="0"  THEN s=LEFT$(s,LEN(s)-1):GOTO trimzeroout
                                      ...
                                      m=i/10000000000000000
                                      ss=TRIM$(STR$(m,18))
                                      IF ss<>s THEN
                                      ...
                                      Actually, Paul, this test does not prove what it is supposed to. Correct me if I'm wrong, but I believe the intent was to prove that your integer/decimal technique was equivalent in accuracy to the VAL(string) technique.

                                      The coding above only demonstrates that when you do an integer/decimal computation, your results, when rounded by the STR$ function, are exactly the same as that obtained my manipulating the string value of i to get the value, i/10000000000000000.

                                      Comment


                                      • Walter
                                        Unless i am convinced otherwise, yes i do feel it maybe relevant.

                                        what is the difference between
                                        local m as ext
                                        m=16.585##
                                        and
                                        local m as ext
                                        let m=16.585##

                                        none i do believe

                                        while it is true that rounding routines are not affected in either way

                                        the value stored to round from is possibly different if you do not use a expression to make a assignment to the extended variable m.
                                        that is why the integer/decimal is giving us a more precise values to round from.
                                        so understanding the LET statement is critical in my view.

                                        i was trying to make an effort to show why we were getting the values of something similar to 16.5849999something as a value to round from or even possibly to use the variable in a calculation of another expression that probably will make a difference in returned results.

                                        i also was trying to make a statement that CURRENCYX may have the same similar behavior, i have not tested that, but my gut feeling is the CURRENCYX may act just the same as the EXTENDED variable.

                                        also not show but in another test i have done with a DOUBLE variables today, i appears i may want to stick with EXTENDED variables totally for many of my math calculations where there is a decimal involved because of the 18 digit precision powerbasic keeps.

                                        here is code that make me believe that.
                                        notice that m when returned in a STR$ with 18 digits can produce other values than STR$ alone, thereby giving me an impression that the value may be different than what i want stored.
                                        i have faith in pb using the correct values to do math with, but i can have faith and security blank too.


                                        this is just for observation purposes only
                                        like Walter said, maybe discussion for another day,
                                        but i am very happy it seems we resolved the EXTENDED assignment deli ma.


                                        Code:
                                        #COMPILE EXE
                                        #DIM ALL
                                        FUNCTION PBMAIN () AS LONG
                                        LOCAL m AS EXT
                                        LOCAL mm AS DOUBLE
                                        LOCAL s AS STRING
                                        LOCAL ss AS STRING
                                        LOCAL sss AS STRING
                                        LOCAL i AS QUAD
                                        LOCAL count AS QUAD
                                        
                                        FOR i =900000000000000&& TO 999999999999999&& STEP 1712467&&
                                        
                                        s=TRIM$(STR$(i,18))
                                        s=LEFT$(s,1)+"."+RIGHT$(s,LEN(s)-1)
                                        trimzeroout:
                                        IF RIGHT$(s,1)="0"  THEN s=LEFT$(s,LEN(s)-1):GOTO trimzeroout
                                        
                                        m=i/100000000000000
                                        mm=i/100000000000000
                                        ss=TRIM$(STR$(m,18))
                                        sss=TRIM$(STR$(mm,18))
                                        
                                        'IF ss<>s THEN
                                        IF m<>mm THEN
                                            PRINT i
                                            PRINT s
                                            PRINT ss
                                            PRINT sss
                                            PRINT "diff found press any key to continue"
                                            WAITKEY$
                                        END IF
                                        
                                        INCR count
                                        
                                        IF count MOD 500000 =0 THEN
                                            PRINT I," > "+S+ " > "+SS+STR$(COUNT,18)
                                        END IF
                                        NEXT i
                                        
                                        
                                        PRINT "done"
                                        WAITKEY$
                                        
                                        END FUNCTION
                                        i made a change to this program after Walter mentioned the STR$ rounding behavior
                                        Last edited by Paul Purvis; 2 Jul 2008, 06:32 PM.
                                        p purvis

                                        Comment


                                        • i was not aware of the STR$ doing any rounding, maybe i missed that or just forgot during the last few days, let us see what i lose over the 4th of July weekend.
                                          p purvis

                                          Comment

                                          Working...
                                          X