Announcement

Collapse
No announcement yet.

One more try, then I promise I'll stop...

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • One more try, then I promise I'll stop...

    OK, I am still trying to understand why my variant assignment statement causes subsequent POKE$ operations to silently fail. I think my first program might have been too long to expect people to look at it. I've tried again with the simplest possible program that demonstrates what I am now suspecting is a bug in PBCC 5.0 with variant array assignment.

    BTW, Yes - I *know* variant assignment is slow and not a good way to copy arrays, but I believe this code demonstrates a problem with its implementation. Such a problem could easily affect legitimate uses of it, causing programs to silently fail, producing incorrect results that are very difficult to trace in production code.

    Please, could someone try running this program on their system and see if they also always get to the line indicating "Arrays not equal."? Thanks.

    Code:
    #DIM ALL
    
    %LAST = 98664440
        
    GLOBAL gArray1() AS LONG, gArray2() AS LONG, gVar AS VARIANT
    
    FUNCTION PBMAIN() AS LONG
    
           DIM i AS LONG
    
           REDIM gArray1(%LAST), gArray2(%LAST)
    
           FOR i = 0 TO %LAST
               gArray1(i) = RND(1,999)
           NEXT i
    
           'NOTE: Once this line executes, the POKE$ silently fails.
           gVar = gArray1() : gArray2() = gVar
    
           REDIM gArray2(%LAST)
    
           'NOTE: With %LAST >= 98664440, this line will silently fail.
           POKE$ VARPTR(gArray2(0)), PEEK$(VARPTR(gArray1(0)), 4 * (%LAST+1))
    
           FOR i = 0 TO %LAST
    
               IF gArray1(i) <> gArray2(i) THEN
                   STDOUT "Arrays not equal.  Press any key..."
                   WAITKEY$
                   EXIT FUNCTION
               END IF
    
           NEXT i
    
           STDOUT "Arrays were equal.  Press any key..."
           WAITKEY$
           
    END FUNCTION

  • #2
    IWH (it works here)

    Code run:
    Code:
    ' FILE: ARRAY_VT.BAS
    ' 01/10/09
    
    #DIM ALL
    
    %LAST = 98664440
    
    GLOBAL gArray1() AS LONG, gArray2() AS LONG, gVar AS VARIANT
    
    FUNCTION PBMAIN() AS LONG
     LOCAL Serr  AS STRING
    
    
           DIM i AS LONG
           ON ERROR GOTO ErrorTrap
           
           sErr = "REDIMing gArrary1"
           
           STDOUT sErr
           REDIM gArray1(%LAST)
           sErr =  "REDIMing GArray2
           STDOUT sErr
           REDIM gArray2(%LAST)
           
           sErr = "Filling GArray1"
           STDOUT sErr
           FOR i = 0 TO %LAST
               gArray1(i) = RND(1,999)
           NEXT i
    
           'NOTE: Once this line executes, the POKE$ silently fails.
           sErr = "Assigning GArray1 TO gVar"
           STDOUT sErr
           gVar = gArray1()
           sErr =  "Assigning Garray2 from gvar"
           STDOUT sErr
           gArray2() = gVar
           
           sErr = "REDIMing GArray2"
           STDOUT sErr
           REDIM gArray2(%LAST)   ' This voids the content of gArray2
    
           'NOTE: With %LAST >= 98664440, this line will silently fail.
           
           sErr = "POKE'ing new values into Garray2"
           STDOUT sErr
           POKE$ VARPTR(gArray2(0)), PEEK$(VARPTR(gArray1(0)), 4 * (%LAST+1))
           
           sErr = "Comparing Arrays"
           STDOUT sErr
           FOR i = 0 TO %LAST
               IF gArray1(i) <> gArray2(i) THEN
                   STDOUT "Arrays not equal.  Press any key..."
                   WAITKEY$
                   EXIT FUNCTION
               END IF
    
           NEXT i
    
           STDOUT "Arrays were equal.  Press any key..."
    ExitPoint:
           WAITKEY$
           EXIT FUNCTION
           
    ErrorTrap:
     STDOUT "Entered error trap on Error " & FORMAT$(ERR)
     RESUME  ExitPoint
            
    
    END FUNCTION
    OUTPUT:
    Code:
    REDIMing gArrary1
    REDIMing GArray2
    Filling GArray1
    Assigning GArray1 TO gVar
    Assigning Garray2 from gvar
    REDIMing GArray2
    POKE'ing new values into Garray2
    Comparing Arrays
    Arrays were equal.  Press any key...
    PB/CC 5.0.0, Win XP/Pro SP3 all updates.

    MCM
    Michael Mattias
    Tal Systems (retired)
    Port Washington WI USA
    [email protected]
    http://www.talsystems.com

    Comment


    • #3
      BTW, I think you need to set the variant to NOTHING (EMPTY?) when you are done with it. Otherwise it just sucks up memory. True, it's released when the process ends, but not erasing "work" variables which are eating up this much memory when your program has more to do is IMO quite careless.
      Michael Mattias
      Tal Systems (retired)
      Port Washington WI USA
      [email protected]
      http://www.talsystems.com

      Comment


      • #4
        And frankly, in your other demo program, I wonder if you are running out of memory, since you don't clear out your workspace. But we wouldn't know since you don't have any error trapping in that code.

        Here I used ON ERROR, but simply testing the system ERR variable is plenty good enough.

        FWIW, Error 7 is returned if a DIM/REDIM or any other allication/reallocation (eg, the temporary string created by PEEK$) fails for want of memory.

        The compiler has so many tools... or maybe I should say so many unused tools.
        Michael Mattias
        Tal Systems (retired)
        Port Washington WI USA
        [email protected]com
        http://www.talsystems.com

        Comment


        • #5
          FWIW.. although I did not see anywhere it could possibly come into play, I recompiled and ran with the #DEBUG ERROR ON directive.

          Same output.

          And just in case you have not guessed yet... five bucks says your program "failures" are the result of willy-nilly proceeding after an error you failed to trap.

          MCM
          Last edited by Michael Mattias; 10 Jan 2009, 11:29 AM.
          Michael Mattias
          Tal Systems (retired)
          Port Washington WI USA
          [email protected]
          http://www.talsystems.com

          Comment


          • #6
            On my laptop Win XP with 3Gigs memory arrays are equal

            On VM Win 2000 set for 256 MB memory arrays are not equal

            Seems the number of elements is directly related to the amount of memory available.


            Originally posted by Patrick Mills View Post
            OK, I am still trying to understand why my variant assignment statement causes subsequent POKE$ operations to silently fail. I think my first program might have been too long to expect people to look at it. I've tried again with the simplest possible program that demonstrates what I am now suspecting is a bug in PBCC 5.0 with variant array assignment.

            BTW, Yes - I *know* variant assignment is slow and not a good way to copy arrays, but I believe this code demonstrates a problem with its implementation. Such a problem could easily affect legitimate uses of it, causing programs to silently fail, producing incorrect results that are very difficult to trace in production code.

            Please, could someone try running this program on their system and see if they also always get to the line indicating "Arrays not equal."? Thanks.

            Code:
            #DIM ALL
            
            %LAST = 98664440
                
            GLOBAL gArray1() AS LONG, gArray2() AS LONG, gVar AS VARIANT
            
            FUNCTION PBMAIN() AS LONG
            
                   DIM i AS LONG
            
                   REDIM gArray1(%LAST), gArray2(%LAST)
            
                   FOR i = 0 TO %LAST
                       gArray1(i) = RND(1,999)
                   NEXT i
            
                   'NOTE: Once this line executes, the POKE$ silently fails.
                   gVar = gArray1() : gArray2() = gVar
            
                   REDIM gArray2(%LAST)
            
                   'NOTE: With %LAST >= 98664440, this line will silently fail.
                   POKE$ VARPTR(gArray2(0)), PEEK$(VARPTR(gArray1(0)), 4 * (%LAST+1))
            
                   FOR i = 0 TO %LAST
            
                       IF gArray1(i) <> gArray2(i) THEN
                           STDOUT "Arrays not equal.  Press any key..."
                           WAITKEY$
                           EXIT FUNCTION
                       END IF
            
                   NEXT i
            
                   STDOUT "Arrays were equal.  Press any key..."
                   WAITKEY$
                   
            END FUNCTION

            Comment


            • #7
              Originally posted by Michael Mattias View Post
              FWIW.. although I did not see anywhere it could possibly come into play, I recompiled and ran with the #DEBUG ERROR ON directive.

              Same output.

              And just in case you have not guessed yet... five bucks says your program "failures" are the result of willy-nilly proceeding after an error you failed to trap.

              MCM
              Michael,

              I'm afraid I got a bit hasty with my sample code and didn't error trap. I should obviously have done that. Oops.

              I pasted your error-trapped code into my PBCC window and ran it. Here is the output:

              REDIMing gArrary1
              REDIMing GArray2
              Filling GArray1
              Assigning GArray1 TO gVar
              Assigning Garray2 from gvar
              REDIMing GArray2
              POKE'ing new values into Garray2
              Comparing Arrays
              Arrays not equal. Press any key...

              Bingo - silent fail of POKE, no error triggered.

              I'm stumped as to why I'm experiencing this problem.

              We have different OS's, and, apparently, different versions of PBCC.

              I'm running Vista Ultimate SP1 (all updates) and PBCC 5.00.0085.

              EDIT:

              I tried running this code under Windows XP SP3 (running as a guest OS in Vista, using VPC 2007), and it ran without errors. The failure does seem to be related to the OS, and not to the version of PBCC.

              Is anyone running Vista SP1 willing to try Michael's error-trapped version of the code and report their results back to this thread?
              Last edited by Patrick Mills; 10 Jan 2009, 01:22 PM.

              Comment


              • #8
                Originally posted by Brian Chirgwin View Post
                On my laptop Win XP with 3Gigs memory arrays are equal

                On VM Win 2000 set for 256 MB memory arrays are not equal

                Seems the number of elements is directly related to the amount of memory available.
                @Brian,

                I tried running Michael's error-trapped version on my Vista SP1 (4 gigs) machine and still get the problem with arrays not being equal, although no errors are detected by the error trap.

                On my VPC Win XP Pro SP3 (1 gig) the code runs correctly.

                Therefore, I remain stumped. We have my copy of Vista (4 gigs) failing, Michael's copy of XP SP3 (memory unspecified) working, your copy of XP (3 gigs) working, your copy of virtual Win2000 (256 megs) failing, and my copy of virtual XP (1 gig) working.

                The two failures are on Vista (4 gigs) and 2000 (265 megs) - not much similarity there. If you were running my original non-error-trapped code, your windows 2000 failure could certainly be due to an untrapped memory allocation error, but that isn't what's happening on my 4-gig version of Vista using Michael's error-trapped version.

                Comment


                • #9
                  >Therefore, I remain stumped

                  One thing to look at: adequate swap space. I had one of these with a client a little over a year ago. Never any error codes from anything, but seemingly random failures, about three times per week, usuaully on "RPC server not responding."

                  We upped the swap space (had been set at 4 Gb max), never had the problem again.

                  Methinks there's probably lots of software out there that says, "this allocation can't fail because this program is only using <X> of our allowed 2 Gb" - but that allocation WILL fail if inadequate swap space is available.

                  In this program, however, because you are asking the compiler to make an allocation (as opposed to the compiler doing things behind the scenes for you), you SHOULD be getting a memory ERR value. In my experience the PB compilers have always been very good about that.

                  If swap space is not the problem ===> [email protected]

                  MCM
                  Michael Mattias
                  Tal Systems (retired)
                  Port Washington WI USA
                  [email protected]
                  http://www.talsystems.com

                  Comment


                  • #10
                    >Bingo - silent fail of POKE, no error triggered

                    BTW, this is a dangerous assumption on your part. As your code is written, true, POKE may have failed; however, it's also possible the PEEK failed; or that your source data array may have been corrupted BEFORE it was PEEK'ed.

                    Besides, neither PEEK nor POKE will never set an application error (the system ERR value). All these functions know to do is read or write "N" bytes at addresss "Y" and it does as it is told. They assume the programmer owns the N bytes at address Y, and there really is no (effficient) way to add some kind of automatic checking.

                    If you want to detect attempted bad accesses of memory DURING one of these operations you will have to use something like structured exception handling. Search here, I know there are examples.
                    Michael Mattias
                    Tal Systems (retired)
                    Port Washington WI USA
                    [email protected]
                    http://www.talsystems.com

                    Comment


                    • #11
                      What's the purpose of this code?
                      Why use the variants, to proof a 'bug'?

                      Variants being set as you do become safearray handles.
                      Memory created not similar as ordinary arrays.
                      Pokeing to an array element as you seem to try is imo a serious thing to avoid.

                      Using a variants having a safearray is usually used for data exchange.
                      Not really 'work memory'.
                      hellobasic

                      Comment


                      • #12
                        @Edwin,

                        The sample code I've posted has no "purpose" other than to demonstrate the point I'm asking about.

                        I originally was not (and am still not) at all interested in using variants as an array copying method. The possibility simply occured to me as I was testing other methods, so I gave it a try since it was trivial to code. When I noticed that another technique I was trying at the time (the Poke/Peek method) began to fail after I added the variant assignment method, I tried to code a simple example of the problem to post to the forum.

                        Clearly, using proper error traps and assuring that resources are released when not needed would be something I (hopefully) not neglect in production code, but these samples are not production code. The general error trap added by Michael was something I should have done before posting, since an allocation error could certainly have resulted in some of the problems I was posting about, and my code as posted wouldn't have caught it. Mea Culpa. However, as we've seen, error trapping still does not catch or prevent what was happening in my code.

                        You can question whether using the variant to proof a "bug" is legitimate, but if there is a problem with PBCC's implemention of variants, what assurance would we have that it won't show up when we're not copying arrays, but exchanging data with COM objects? Perhaps I should have coded up a COM object data exchange example using variants that behaves unpredictably, but I'm not working with COM right now - I'm working with moving large arrays about, and just happened onto the possibility of a variant oddity in PB.

                        As far as Poke'ing to an array, yes it's a dangerous thing to do (as is calling the MoveMemory api from your code, and for the same reasons). But my application needs to quickly copy large arrays, or portions of them, back and forth, and I needed to find a faster way to do it than looping through them with indices, or even looping through them with pointers. The MoveMemory APIwill meet my needs speedwise, but the dangers of altering arrays outside the safety of the compiler's bounds checking remains. I must either accept the danger or put up with slower performance, as an alternative to moving memory around doesn't exist for my application. Someone suggested memory-mapped files, but I need to reference the copied arrays with the same algorithms as the original arrays (and to compare them, as well), so storing them in a different form is not really a good answer, either.

                        Comment


                        • #13
                          >As far as Poke'ing to an array, yes it's a dangerous thing to do (as is calling the MoveMemory api from your code, and for the same reasons)

                          In this case there is a difference.
                          Wrong calls on the variant i mean, go low level for safearrays.
                          But then, there now better ways to pass data back and forth by not using late bound com interfaces and not using variants.
                          hellobasic

                          Comment


                          • #14
                            Originally posted by Michael Mattias View Post
                            >Bingo - silent fail of POKE, no error triggered

                            BTW, this is a dangerous assumption on your part.
                            I suppose so, but I actually did realize that the problem wasn't necessarily the POKE (even though that *is* what I wrote :-) As far as the source array being corrupted - even though my posted code didn't show this, I have other versions which allow me to see the before/after contents of each array. After the variant assignment completes, not only is array1 equal to array2, but they are both still equal to an untouched third array with the same data. After the peek/poke failure, array1 and the (unshown in the posted code) third array are still equal and untouched, and array2 is in its initialized state from the REDIM. Not a single byte of it is changed by the PEEK/POKE code. As far as my program is concerned, the PEEK/POKE line effectively becomes a NOOP (at least with respect to the running code - perhaps 400 million bytes of *something* *somewhere* are being changed, but there is no GPF or other visible failure apparent).

                            The odd thing is that I can add other Peek/Poke code to the example and it continues to work, right alongside the code that fails.

                            If I add a loop that uses Peek/Poke to move data 4-bytes at a time from the first array to the second, that Peek/Poke code succeeds right after the bulk copy peek/poke fails. I can begin increasing the sizes of the moves from 4 bytes to 8 to 16, etc. and things still work. By process of elimination, I could probably come up with the point at which it stops working, but I'm losing interest in doing that, since this is just experimental code.

                            Anyway, thanks for everyone's indulgence. I apologize if you all think this was a waste of time. If at some point in the future I really need to do a large variant assignment followed by an Peek/Poke that mysteriously fails, and I can't think of any other way to do what I need, I guess I'll contact PB support. Otherwise, I think my need to understand this problem has reached its limit, due to lack of any practical impact.
                            Last edited by Patrick Mills; 10 Jan 2009, 07:09 PM.

                            Comment


                            • #15
                              Three quick points:
                              • Test code which does not employ all the error-testing mechanisms available is not really test code.
                              • I have over the years encountered 'flaky' performance with ALL software - not just software I have created with the PB compilers - when operating with "really big" data sets or "near" some documented limit.
                              • PB's support department is where you should go if it appears the compiler is doing something naughty, including the naughtiness of not catching and reporting an error condition. Surely you would like your users to report difficulties to you, regardless if they post them in some peer support forum or not.


                              MCM
                              Michael Mattias
                              Tal Systems (retired)
                              Port Washington WI USA
                              [email protected]
                              http://www.talsystems.com

                              Comment


                              • #16
                                > Not a single byte of it is changed by the PEEK/POKE code

                                That's the key. One of those two statements (the PEEK or the POKE) failed.

                                Since they are in an expression now, you can't separate what's happening.

                                Try:

                                Instead of ...
                                Code:
                                       POKE$  address, PEEK$(address,size)
                                Change to two lines:
                                Code:
                                 S = PEEK$ (address, size )  
                                 POKE$  address, S 
                                 ' should add at this point: 
                                 S= ""   ' deallocate string
                                If that fails to set an error code, you can analyze the string S to see if it "PEEKed" correctly. If S is incorrect, you have found a bug in the compiler.

                                (Yes, it IS the same code. When you use an expression like this, the compiler is creating "S" anyway, it just does it behind the scenes).

                                But take a look at my suggested design in your other thread. That uses a LOT less system resources.

                                I was thinking about that some more and now realize I only went halfway. There's no reason you could not also use an MMF for the "working" array. That you can map only PART of the file suggests that by working in pieces you can accomodate even larger sets of data without breaking the memory bank.

                                MCM
                                Michael Mattias
                                Tal Systems (retired)
                                Port Washington WI USA
                                [email protected]
                                http://www.talsystems.com

                                Comment

                                Working...
                                X