Announcement

Collapse
No announcement yet.

PB VAL() - Problem ... maybe

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • PB VAL() - Problem ... maybe

    PB VAL() - Problem ... maybe ?!

    ---- String S ="7, ... ... ... "
    ---- String len: 1,212,415
    VAL( s ) = 7
    VAL( extract$( s, "," ) ) = 7
    VAL( parse$( s, 1 ) ) = 7

    ---- String len: 1,212,416
    VAL( s ) = 0 - for PB ver.9.05/10.04
    VAL( extract$( s, "," ) ) = 7
    VAL( parse$( s, 1 ) ) = 7

    Test:
    Code:
    FUNCTION PBMAIN() AS LONG
    
    	LOCAL s, t, h AS STRING
    
    	s = "7," + STRING$( 1212413, $SPC )
    	GOSUB MakeT: h = t
    
    	s = s + $SPC ' OR >
    	GOSUB MakeT
    
    	? "S = """ + LEFT$( s, 50 ) + "... """ + $CR + $CR + h + $CR + t, _
    		%MB_ICONWARNING, "PB VAL() - Problem ?!"
    
    	EXIT FUNCTION
    
    MakeT:
    	t = $TAB + "S len: " + FORMAT$( LEN( s ), "#," ) + $CR + _
    		"VAL( s )" + $TAB + $TAB + $TAB + "=" + STR$( VAL( s )) + IIF$( VAL( s ), "", " - PB ver" + HEX$( %PB_REVISION ) + " ?!" ) + $CR + _
    		"VAL( extract$( s, "","" ) )" + $TAB + "=" + STR$( VAL( EXTRACT$( s, "," ))) + $CR + _
    		"VAL( parse$( s, 1 ) )" + $TAB + "=" + STR$( VAL( PARSE$( s, 1 ))) + $CR
    RETURN
    
    END FUNCTION

  • #2
    Interesting. Offhand, I haven't a clue why this is happening.
    Rod
    In some future era, dark matter and dark energy will only be found in Astronomy's Dark Ages.

    Comment


    • #3
      First thought is that you are actually using all of your available memory.

      Difficult to know without a little hardware, resident software info, and configuration stuff, but your string is 2.36 GIG. If you have 1 byte left and add a 2 byte string, the function will fail. In a larger application you may have corrupted other parts of memory. I would not be surprised by a GPF somewhere down the line.

      Second thought is that your formatting function has dropped a format character into the string t. That is a rather complex concatenation (14 literals).

      FTM: "Format characters are not allowed, and will cause termination of the evaluation."

      The EXTRACT$ and PARSE$ functions work because they both return STRING values while VAL() returns numeric. Once the required info is found no further evaluation of the string will occur, unlike like MS basics which evaluate the entire string in almost all cases. This might work in MS basics because the default variable type is VARIANT. In PB 9+ there is no default variable type.

      But that's just a quick look. Hope it helps.
      Last edited by StanHelton; 12 Feb 2015, 06:01 AM. Reason: Clarification of EXTRACT$ and PARSE$ - fix numbers
      Do not go quiet into that good night,
      ... Rage, rage against the dark.

      Comment


      • #4
        The behavior is the same way back in PBCC3 btw

        I guess it's no coincidence that your string of "7," + 1212413 spaces = 0x127FFF characters in length ... one more and it clicks over one of those many magic "x7F half boundaries" and goes to 0x128000.

        Here's the Length dword that sits in front of every PB dynamic-size string ...
        FF 7F 12 00 <-- "3" & STRING$( 1212414, $SPC ), which VAL calculates correctly
        00 80 12 00 <-- "3" & STRING$( 1212415, $SPC ), which VAL calculates incorrectly

        So as it was failing on an interesting range I threw together this proggy to test to see if more ranges are involved... which is the case
        Code:
        #COMPILE EXE
        FUNCTION PBMAIN() AS LONG
        LOCAL i AS LONG, x AS LONG, s AS STRING
        i = 1
        DO
         s = "7" & STRING$(i, $SPC)
         IF VAL(s) = 7 THEN
          IF x = 1 THEN
             ? "Good from " & STR$(LEN(s)) & "/0x" & HEX$(LEN(s))
             x = 0
          END IF
         ELSE
          IF x = 0 THEN
             ? "Fail from " & STR$(LEN(s)) & "/0x" & HEX$(LEN(s))
             x = 1
          END IF
         END IF
         INCR i
        LOOP
        END FUNCTION
         
        "7[I]{n spaces}[/I]" (total length up to 32767/0x7FFF) works fine... then...
        [b]OUTPUT:[/b]
        Fail from  32768/0x8000
        Good from  65537/0x10001
        Fail from  98304/0x18000
        Good from  131073/0x20001
        Fail from  163840/0x28000
        Good from  196609/0x30001
        Fail from  229376/0x38000
        Good from  262145/0x40001
        Fail from  294912/0x48000    [I](i stopped running it here)[/I]
        there's a fair bit of code under the hood to VAL() as it's general nature and processing numbers as ASCII chars like "7", "." etc, plus "e" and "n" and all the code that comes along with exponentiation/scientific notation, but we can also figure out what's going on just from various runtime tests like this one, so i encourage others to submit some, but I'm on the verge of some serious Z for tonight!

        Plamen,
        btw your demo uses "7," ... the doco for VAL says "Format characters (like commas) are not allowed, and will cause early termination of the evaluation."
        (The issue still remains without the comma, however)
        Last edited by Wayne Diamond; 12 Feb 2015, 07:43 AM.
        -

        Comment


        • #5
          The largest integer type is quad at 64 bits. The most character intensive string format for numbers is binary. It takes 66 characters in a string to completely define a quad in binary. An "&", and a "B", followed by 64 one and/or zero characters.

          The fact that the VAL() accepts over a million garbage characters in the string to be converted to a numeric type is amazing, not a problem. If the posibility exists for a string that long to be fed to VAL() , then some validity checking should be performed first.

          While it is a curiousity that strings longer than &H127FFF fail, it is not a problem, just a limitation to be aware of.

          Cheers,
          Dale

          Comment


          • #6
            Dale Yarker
            While it is a curiousity that strings longer than &H127FFF fail, it is not a problem, just a limitation to be aware of.
            Cheers, - ??? !!!
            See: Wayne Diamond test: (String len: 32,768 - VAL( s ) = 0

            Comment


            • #7
              OK, 32K!
              Dale

              Comment


              • #8
                One possibility to test for is that 32k is the intended limit (and 32k digits is still a ridiculously massive number which should cater for most!), and that it's only by chance and the simple test params (ie. string of all spaces) that's leading to "false corrects" ... easy enough to test for things like that with specific values >32k in the Lo and Hi bits etc, but thats me done for tonight
                -

                Comment


                • #9
                  Code:
                  TRY 
                      X = VAL (something)
                   CATCH 
                       Message "Oops"
                  END TRY
                  Or use ON ERROR GOTO.

                  If something bad is happening you SHOULD get ERR set to something, unless it's numeric overflow, which is not caught.

                  Re
                  >... but your string is 2.36 GIG

                  ???

                  String len: 1,212,415
                  And for sure any string operation which uses up all available memory returns error 7 (Out of Memory).

                  Best guess any problem here is either numeric overflow or overrun of some internal buffer not resulting in a protection fault (eg request for additional stack).

                  The ERR test should be included in any code "demonstrating the problem" since if ERR is set the programmer should know (PowerBASIC 101) the result is not to be trusted.

                  The ambitious may wish to use Structured Exception Handling, which WILL catch numeric overflow. But the better fix would be "don't do that" as VAL(digit[s] comma digit[s] comma) is totally silly and has no meaning anyway, unless you are trying to avoid the use of PARSE$ or EXTRACT$. And we all know what I think of "cute code tricks" designed to "beat the system."

                  MCM
                  Michael Mattias
                  Tal Systems (retired)
                  Port Washington WI USA
                  [email protected]
                  http://www.talsystems.com

                  Comment


                  • #10
                    The problem is that in determining VAL() the compiler internally uses 16 bit values, not 32 bit.
                    When the length of the string exceeds 15 bits, VAL() sees it as a negative length (within the 16 bit number), acts as though it has reached the end of the string when it hasn't, and returns zero.

                    It's a bug.
                    I suspect it's left over from the DOS days when strings couldn't be any bigger than 32k.

                    No error is set because the compiler doesn't notice it.

                    There is no numeric overflow internally so exception handling won't catch it either.

                    Having a comma is OK, it's just seen as a non-numeric character and ends the process, as would a "p" or a "W".

                    The comment in the help file with regards to commas in the string is intended to point out that VAL("1,000") will not return 1000, but will return 1.

                    Paul.

                    Comment


                    • #11
                      >I suspect it's left over from the DOS days when strings couldn't be any bigger than 32k.

                      You mean sometimes code gets ported from one O/S to another and something gets MISSED? Gee whiz, I've never seen that before!
                      Michael Mattias
                      Tal Systems (retired)
                      Port Washington WI USA
                      [email protected]
                      http://www.talsystems.com

                      Comment


                      • #12
                        'A way if you really need a long.
                        'Accept only 0123456789-
                        Code:
                         FUNCTION PBMAIN () AS LONG
                          ? STR$(VAL("1,000"))         + $CRLF + _  ' 1
                            STR$(string2Long("1,000")) + $CRLF + _  '1000
                            STR$(string2LONG("1,-3B"))              '-13
                        END FUNCTION
                         
                        FUNCTION string2long (sNum AS STRING) AS LONG
                        IF LEN(sNum) = 0 THEN EXIT FUNCTION 'added 9/3/2011
                        !mov edi,sNum                   ;get the pointer to the string information (not the string)
                        !mov edi,[edi]                  ;get the point to the string contents  (added this)
                        '--------------------------------------------------------------------------------------------
                        !xor edx,edx                    ;the sum =0
                        !xor ecx,ecx                    ;the neg flag =0
                        !movzx eax,byte ptr [edi]       ;get the first character
                        lp:
                        !cmp eax,"0"                    ;is character < a "0"?
                        !jl  lessthan0                  ;yes, skip to the non-digit check
                        !cmp eax,"9"                    ;is it greater than a "9"?
                        !jg  nextcharacter              ;yes, get next char.
                        'to get here it must be a digit
                        !imul edx,10                    ;sum=sum*10 ..
                        !add edx,eax                    ;    + digit
                        !sub edx,48                     ;    - 48
                        !jmp nextcharacter
                        lessthan0:
                        REM !cmp eax,"."                ;is it a "."           'rem to terminate on decimal
                        REM !je done                    ;yes, then exit loop   'rem to terminate on decimal
                        !cmp eax,"-"                    ;is it a "-"
                        !jne nextcharacter              ;no, get next character
                        !mov ecx,1                      ;set the neg flag
                        nextcharacter:
                        !inc edi                        ;increment the string pointer
                        !movzx eax,byte ptr [edi]       ;get next character
                        !or eax,eax                     ;test if zero
                        !jnz lp                         ;not zero, go back and do next character
                        done:
                        !cmp ecx,0                      ;is neg flag set?
                        !je skip                        ;no, skip next instruction
                        !neg edx                        ;yes, negate answer
                        skip:
                        !mov function,edx               ;write answer to function
                        END FUNCTION

                        Comment


                        • #13
                          C'mon Mike. A real programmer would..

                          Code:
                            strIn = "123456.........1234567"  ' ultralong string 
                            decpos = INSTR (strin, ".") 
                            REDIM  digits(LEN (StrIn)-1) AS BYTE AT STRPTR (strin)
                            theValue## = 0 
                            Multiplier## = 1 
                            FOR Z =  decPos -1 TO 1 STEP -1
                             digitval = digits(Z) 
                             theValue += digitVal * Multiplier 
                             Multiplier *=10 
                            NEXT
                            Multiplier= .1##
                            FOR Z = decPos+1 TO LEN(StrIn)
                               digitval = digit (Z) 
                               thevalue## +=  Digitval * Multiplier
                               Multiplier /= 10 
                            NEXT
                          
                            thevalue## contains the answer now.
                          Handling binary, octal, hex and scientific notation left to users as exercise.

                          MCM
                          Michael Mattias
                          Tal Systems (retired)
                          Port Washington WI USA
                          [email protected]
                          http://www.talsystems.com

                          Comment


                          • #14
                            sometimes code gets ported from one O/S to another and something gets MISSED?
                            but, what if it wasn't... :hmmm:

                            Comment


                            • #15
                              Correcting left to user

                              > C'mon Mike. A real programmer would..


                              C'mon Michael. A real programmer would post compilable function/macro that was tested.
                              You funny guy. He is using floating point. I am using LONG.
                              Code:
                              #DIM ALL
                              
                              FUNCTION PBMAIN () AS LONG
                                LOCAL s1,s2 AS STRING
                                s1 = "1234.........5678"  ' ultralong string
                                s2 = "1,000"
                               
                                ? STR$(MMVal(s1))       + $CR +_   '55671 wrong
                                  STR$(MMVAL(s2))                  '5  wrong
                               
                                 ?STR$(string2long(s1)) + $CR +_   '12345678
                                  STR$(STRING2Long(s2))            '1000
                              END FUNCTION
                              '
                              FUNCTION MMVAL(StrIn AS STRING) AS LONG
                                LOCAL theValue, Multiplier,the AS EXT
                                LOCAL z, decpos,digitval,digit AS LONG
                                 decpos = INSTR (Strin, ".")
                                REDIM  digits(LEN (StrIn)-1) AS BYTE AT STRPTR (strin)
                                theValue## = 0
                                Multiplier## = 1
                                FOR Z =  decPos -1 TO 1 STEP -1
                                 digitval = digits(Z)
                                 theValue += digitVal * Multiplier
                                 Multiplier *=10
                                NEXT
                                Multiplier= .1##
                                FOR Z = decPos+1 TO LEN(StrIn)
                                   'digitval = digit (Z)  ' digit  (Z) undefinied
                                   digitval = digits(Z)   ' digits (Z)
                                   thevalue## +=  Digitval * Multiplier
                                   Multiplier /= 10
                                NEXT
                                FUNCTION = theValue## 'contains the answer now.
                              END FUNCTION
                               
                              FUNCTION string2long (sNum AS STRING) AS LONG
                              IF LEN(sNum) = 0 THEN EXIT FUNCTION 'added 9/3/2011
                              !mov edi,sNum                   ;get the pointer to the string information (not the string)
                              !mov edi,[edi]                  ;get the point to the string contents  (added this)
                              '--------------------------------------------------------------------------------------------
                              !xor edx,edx                    ;the sum =0
                              !xor ecx,ecx                    ;the neg flag =0
                              !movzx eax,byte ptr [edi]       ;get the first character
                              lp:
                              !cmp eax,"0"                    ;is character < a "0"?
                              !jl  lessthan0                  ;yes, skip to the non-digit check
                              !cmp eax,"9"                    ;is it greater than a "9"?
                              !jg  nextcharacter              ;yes, get next char.
                              'to get here it must be a digit
                              !imul edx,10                    ;sum=sum*10 ..
                              !add edx,eax                    ;    + digit
                              !sub edx,48                     ;    - 48
                              !jmp nextcharacter
                              lessthan0:
                              REM !cmp eax,"."                ;is it a "."           'rem to terminate on decimal
                              REM !je done                    ;yes, then exit loop   'rem to terminate on decimal
                              !cmp eax,"-"                    ;is it a "-"
                              !jne nextcharacter              ;no, get next character
                              !mov ecx,1                      ;set the neg flag
                              nextcharacter:
                              !inc edi                        ;increment the string pointer
                              !movzx eax,byte ptr [edi]       ;get next character
                              !or eax,eax                     ;test if zero
                              !jnz lp                         ;not zero, go back and do next character
                              done:
                              !cmp ecx,0                      ;is neg flag set?
                              !je skip                        ;no, skip next instruction
                              !neg edx                        ;yes, negate answer
                              skip:
                              !mov function,edx               ;write answer to function
                              END FUNCTION
                              Last edited by Mike Doty; 13 Feb 2015, 10:52 AM.

                              Comment


                              • #16
                                I've just seen too many "ports" go bad on Ye Olde "Oops, I missed one."

                                We had a Rule of Thumb we used when I was GM of a VAR firm:

                                "If we make an error which would have cost one dollar to avoid during design, it will cost ten dollars to fix after it's programmed and one hundred dollars to fix once it's installed."

                                Most of the code shown here for "peer support" is compiled without #DEBUG ERROR ON, meaning you would have no structured way to catch the null pointer error here... OR, programmers test on their systems, which are typically atypical of user systems, where you DO have "unshared" printers.

                                You do get the benefit of #DEBUG ERROR ON if you test using the stepping debugger, but that's a separate compilation and I really don't like to test EXEs which are not compiled EXACTLY as tested.

                                That's the "one dollar, ten dollars, one hundred dollars" thing talking to me.. and I'm not even putting a cost on the loss of user goodwill: "These guys can't change one thing without breaking another" or "They never get the first shipment of my changes right the first time."

                                Mr. Chobanov did well to catch this problem now, but that only happened because he paid attention to the system ERR variable. He has set a good example for others.

                                MCM
                                Last edited by Michael Mattias; 13 Feb 2015, 11:06 AM. Reason: correct some less-than-high-school quality grammar.
                                Michael Mattias
                                Tal Systems (retired)
                                Port Washington WI USA
                                [email protected]
                                http://www.talsystems.com

                                Comment


                                • #17
                                  C'mon Michael. A real programmer would post compilable function/macro that was tested.
                                  You funny guy. He is using floating point. I am using LONG.
                                  I do big ideas. Details are left to staff.

                                  All I see "corrected" BTW is one lousy misspelled variable. What did you expect from something written online?

                                  Besides, I did floats too.

                                  More importantly - to me anyway - I used nothing but pure PowerBASIC intrinsics... showing, too, there is more than one way to skin a cat.


                                  MCM
                                  Real Men Do It Using BASIC.
                                  Last edited by Michael Mattias; 13 Feb 2015, 11:05 AM.
                                  Michael Mattias
                                  Tal Systems (retired)
                                  Port Washington WI USA
                                  [email protected]
                                  http://www.talsystems.com

                                  Comment


                                  • #18
                                    I liked the way you were doing it to avoid assembly language.
                                    Assembler may have to be changed for 64-bit in the future.

                                    The FUNCTION/MACRO has more than a misspelled variable.
                                    It does not produce the correct results 1,000 = 5.

                                    The routine would be worth finishing.

                                    Comment


                                    • #19
                                      >1,000 = [erroneously returns] 5.

                                      Well, edit your string or something.

                                      Besides, I ain't gonna use it. VAL() is good, so it ain't worth finishing either.
                                      Michael Mattias
                                      Tal Systems (retired)
                                      Port Washington WI USA
                                      [email protected]
                                      http://www.talsystems.com

                                      Comment


                                      • #20
                                        You are the user (it is your code.)
                                        Code:
                                          strIn = "123456.........1234567"  ' ultralong string 
                                          decpos = INSTR (strin, ".") 
                                          REDIM  digits(LEN (StrIn)-1) AS BYTE AT STRPTR (strin)
                                          theValue## = 0 
                                          Multiplier## = 1 
                                          FOR Z =  decPos -1 TO 1 STEP -1
                                           digitval = digits(Z) 
                                           theValue += digitVal * Multiplier 
                                           Multiplier *=10 
                                          NEXT
                                          Multiplier= .1##
                                          FOR Z = decPos+1 TO LEN(StrIn)
                                             digitval = digit (Z) 
                                             thevalue## +=  Digitval * Multiplier
                                             Multiplier /= 10 
                                          NEXT
                                        
                                          thevalue## contains the answer now.

                                        Comment

                                        Working...
                                        X