Announcement

Collapse
No announcement yet.

retrieving column-major ordered arrays

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • retrieving column-major ordered arrays

    Dear All

    Please help me; I am fairly desperate because of my unability to retrieve multidimensional arrays (from 3 dimensions upwards) containing integers. The problem can be illustrated by the following example:

    Given the 3-dimensional table "little%", i.e. an array declared with

    DIM little%(2,3,2)
    and stored (successfully, I checked that with a plain text editor) on disk with
    OPEN "little_table.tab" FOR OUTPUT AS #1
    PRINT #1, little%()
    CLOSE #1

    PB seemingly can not read the array out again. Any attempt (or rather umpteen attempts in many imaginable variations) to retrieve it with something like:

    OPEN "little_table.tab" FOR INPUT AS #1
    INPUT #1, little%(2,3,2)
    CLOSE #1

    invariably result in a non-return voyage to never-never-land, complete with Microsoft's farewell kiss "sorry for the inconvenience" etc...

    Variants like <INPUT #1, little%()> triggered either an immediate complaint by the compiler, or the same crash. What went wrong? It seems difficult to imagine that PB has no procedure for reading in arrays which it itself stored in the column-major order !

    - Please note that before annoying the community with what may be a beginners problem with PB (although I was an old hand at Turbo-Pascal ages ago, until Version 6 or 7 if I remember rightly), I checked the Forum back as far as the last Great Ice Age (MANY thanks to Steve Rossell for his invaluable, brand-new Boolean search tool which makes life so much easier !). This included for instance reading Aldo Vitagliano's suggestions back in the year 2000 and many others, but I found no answer to the actual question above. Maybe I searched inaccurately...

    - The often proposed nested "for I=1 to ..." loops will scarcely do, if only because these become too cumbersome for handling certain later calculations which involve a large number of dimensions. I did read the numerous apt warnings about too many dimensions, in the Manual and elsewhere, but in the present case they are practically unavoidable. Actually, any odd language can handle arrays with an arbitrary number of dimensions through the use of nested "for I=1 to ...", but a main reason why I recently chose PB - besides its many obvious qualities and its considerate support - was its ability to gracefully handle up to 8 dimensions if the need arose, or so I thought.

    - Storage in binary form is inappropriate. The arrays represent statistical states of physical systems, and the stored files must be not be limited to use with PB, but written in a standard form simply recoverable by a motley crowd of physicists and statisticians. PB's column-major order is ideal for the purpose. I am fully aware that this kind of reading necessitates stating the number of dimensions of the array and the extent of each. ¿Must this be done in a way which differs from the explicit use of the identification <little%(2,3,2)> (in the present example)?

    I hope this will not give too much trouble, and sincerely thank in advance for any forthcoming help.

    Lucien Preuss (
    Switzerland)


  • #2
    Input requires separate variables for each item that is to be read.
    You can try
    INPUT #1, little%(1,1,1), little%(1,1,2), little%(1,2,1), little%(1,2,2)....
    or something along those lines.

    Comment


    • #3
      retrieving column-major ordered arrays

      Search the help file for GET and PUT in binary files.

      To write the array to a file:
      Code:
      DIM little%(2,3,2) 
      OPEN "little_table.tab" FOR BINARY AS #1
      PUT #1, ,little%()
      CLOSE #1

      To reload the array from a file:
      Code:
      DIM little%(2,3,2) 
      OPEN "little_table.tab" FOR BINARY AS #1
      GET #1, ,little%()
      CLOSE #1
      P.S. Why you have posted in "Using the forums" instead of "PowerBASIC for Windows" or "PowerBASIC Console Compiler"?
      Last edited by Tiziano Magni; 26 Oct 2009, 03:29 AM.

      Comment


      • #4
        Sincere Thanks to Pagani and Magni for their answers.
        However, the problem is more difficult than it looks. Pagani’s suggestion seems unpractical when the array has hundreds of elements. As for Magni’s proposal to use PUT and GET, I suppose he is on the right track, but his solution does not work in its present form, and I was unable to improve it in spite of many attempts. Maybe the suggested simple PUT procedure actually works, but I doubt it. Anyhow, this can not be checked before the GET procedure is in proper order, because only PB and VB can read (and hence check) the proprietary “packed string format” of the binary file (Manual p.381). A kind of Catch 22.

        The GET procedure probably requires using the FILESCAN statement, but I could not find out how this must be done for a multidimensional array. The Manual is cryptic about it, as it concentrate on strings (which have a meaningful overall length), and on individual records of lists (which have a precise location), so that I am at a loss about how to include into the GET procedure both the number of dimensions and the extent of each dimension in a multidimensional array. Maybe even the PUT procedure must contain an explicit specification of these parameters.

        Has anybody an idea? Otherwise I must, as a last resort, ask somebody from the inner circle.

        PS. The simple answer to Magni’s question is: because I am a tyro in these matters – I never wrote a single line in a Forum or the like before. I hope you will pardon me and I promise to amend.

        Comment


        • #5
          Lucien:

          This is the general form to store and retrieve a multidimensional array in binary format. You have to calculate the size of the full array, having into consideration that the number of elements is the dimension plus 1, and the size of the variables you use (short integer in this case=2: 2*101*4*5=4040)

          Best regards,


          Code:
           
          #COMPILE EXE
          #DIM ALL
          FUNCTION PBMAIN () AS LONG
           LOCAL LT%(),II%,IJ%,KK%
           DIM LT%(100,3,4)
           FOR II%=1 TO 100
            FOR IJ%=1 TO 3
             FOR KK%=1 TO 4
              LT%(II%,IJ%,KK%)=II%*IJ%*KK%
             NEXT
            NEXT
           NEXT
           OPEN "ARR.BIN" FOR BINARY AS #1
           PUT #1,4040,LT%()
           CLOSE 1
           
           REDIM LT%(100,3,4)   'clear the values in memory 
           PRINT LT%(38,3,4)     'verify
           PRINT LT%(67,2,3)
           
           
           OPEN "ARR.BIN" FOR BINARY AS #1
            GET #1,4040,LT%()              'retrieve the full array
           CLOSE 1
           PRINT LT%(38,3,4)               ' verify
           PRINT LT%(67,2,3)
           PRINT LT%(100,3,4)
           WAITKEY$
          END FUNCTION
          Last edited by Manuel Valdes; 29 Oct 2009, 01:54 PM.

          Comment


          • #6
            Code:
            FOR II%=1 TO 100
              FOR IJ%=1 TO 3
               FOR KK%=1 TO 4
                LT%(II%,IJ%,KK%)=II%*IJ%*KK%
               NEXT
              NEXT
             NEXT
            Wow.

            Why don't you get lazy?
            Code:
            Storage =   ARRAYATTR(array(), 4) * ARRAYATTR(array(),5) 
            '                -----------------------   ------------------------
            '                 total elements               size of element
            MCM
            Michael Mattias
            Tal Systems (retired)
            Port Washington WI USA
            [email protected]
            http://www.talsystems.com

            Comment


            • #7
              Lucien,

              Maybe I didn't understand what you try to do

              If you want to write and then read the array values, the following example in PBCC 5.02 does it using PUT and GET.
              Note that the dimensions of the array are fixed.

              Code:
              #COMPILE EXE
              #DIM ALL
              
              FUNCTION PBMAIN () AS LONG
                  ' Write an array to a file with PUT instruction then read
                  ' the saved values into another array with GET instruction
                  '
                  DIM littlewrite(2,3,2) AS INTEGER, littleread(2,3,2) AS INTEGER
                  DIM i AS INTEGER, j AS INTEGER, k AS INTEGER
                  '
                  PRINT "Assign values 101 - 136 to the array..."
                  ARRAY ASSIGN littlewrite() = 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, _
                                          113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, _
                                          125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136
                  FOR i = LBOUND(littlewrite, 1) TO UBOUND(littlewrite, 1)
                    FOR j = LBOUND(littlewrite, 2) TO UBOUND(littlewrite, 2)
                      FOR k = LBOUND(littlewrite, 3) TO UBOUND(littlewrite, 3)
                        PRINT littlewrite(i, j, k); $SPC;
                      NEXT k
                      PRINT
                    NEXT j
                    PRINT
                  NEXT i
                  PRINT
                  ' write array values to file
                  OPEN "little_table.tab" FOR BINARY AS #1
                  PUT #1, ,littlewrite()
                  CLOSE #1
                  ' read array values from file
                  OPEN "little_table.tab" FOR BINARY AS #1
                  GET #1, , littleread()
                  CLOSE #1
                  '
                  PRINT
                  PRINT "Values from file..."
                  FOR i = LBOUND(littleread, 1) TO UBOUND(littleread, 1)
                    FOR j = LBOUND(littleread, 2) TO UBOUND(littleread, 2)
                      FOR k = LBOUND(littleread, 3) TO UBOUND(littleread, 3)
                        PRINT littleread(i, j, k); $SPC;
                      NEXT k
                      PRINT
                    NEXT j
                    PRINT
                  NEXT i
                  '
                  WAITKEY$
                  '
              END FUNCTION
              P.S. The FILESCAN instruction works only with variable length string data!

              Quoting the help file:
              FILESCAN statement
              PURPOSE - Rapidly scan a file opened for INPUT or BINARY mode, in order to obtain size information about variable length string data
              RESTRICTIONS - If FILESCAN is applied to other file formats, the results are undefined.


              Tiziano Magni
              Last edited by Tiziano Magni; 29 Oct 2009, 05:12 PM.

              Comment


              • #8
                Huge thanks to all, what a wonderful and fast as lightning crowd you are at PowerBasic.

                Manuel’s listing is an outstanding mixture of knowledge and commonsense. It worked immediately. Apart from offering a solution for a very disturbing problem, it made me ashamed because I missed the automatic method which he uses to fill up an array through multiplications at each trial. This is sooo much more comfortable than my own little trial program which forced me to enter check numbers through the keyboard at each run! I will use Manuel’s method for many future checks.

                Michael’s suggestion to use ARRAYATTR for evaluating the size of the array seems valuable too. I did not try it out yet, but it looks very useful, not so much for lazyness as for flexibility when an application requires arrays of different sizes. It seems regrettable that the manual does not mention ARRAYATTR under the heading “see also…” at the end of the entry for FILESCAN.

                Sincere thanks again, and two final questions:
                ¿ why does Manuel prefer REDIM to RESET which I used hitherto for emptying arrays ?
                ¿ Is PB’s binary format some kind of standard, or particular to PB and VB ? This is of interest in cases where a user wishes to enter an array of empirical data into an existing PB application. It is fairly easy to tell a physicist or a statistician how to fill a “column-major” array with his or her data, but much more of a problem to explain the transformation of the data into some obscure, not readily available “binary format”.

                Comment


                • #9
                  Save/Load 'n-dimension' array of anything to/from disk

                  Pseudo-code only

                  Given: MyArray(), nDimensions, some datatype

                  Code:
                  SaveME: 
                  
                      OPEN  "myFile" fOR BINARY AS hFile 
                      PUT    hFile, ARRAYATTR (myarray(), 1)  ' save datatype
                      PUT    hFile, ARRAYATTR( myArray(), 5)  ' save element size 
                      nDimension = ARRAYATTR(myarray(),  3)
                      PUT    hFile, , nDimension    ' save number of dimensions
                      FOR Z = 1 TO ndimension
                         PUT    hFile,,LBOUND(myArray,Z)      'save LBOUND/UBOUND of each dimension
                         PUT    hFile, UBOUND (myarray, Z)
                     NEXT 
                     PUT hFile,,myArray()    ' save array data 
                  
                  
                  LoadMe: 
                  
                     OPEN "myFile" FOR BINARY as hFile 
                     GET   hFile, , DataType
                     Get   hFile,,  ElSize
                     GET   hFile,,  nDimension 
                     REDIM lb(nDimension), ub(Dimension) 
                     FOR Z = 1 TO nDimension 
                         GET  hFile,, LB(Z)
                         GEt  hFiie,, UB(Z) 
                     NEXT 
                    ' Here is where it gets really ugly with a variable number 
                    ' of dimensions, since you need a separate REDIM statement 
                    ' for each possible number of dimensions and datatype 
                  
                     eg  
                     SELECT CASE nDimension 
                           CASE 1 
                              REDIM mArray( lb(1) TO ub(1) AS BYTE
                           CASE 2 
                             REDIM myArray (Lb(1) TO ub(1), lb(2) TO Ub(2)) 
                            ....
                            CASE n   
                               REDIM myArray (Lb(1) TO ub(1), lb(2) TO Ub(2)...., lb(n) TO ub(n)
                  
                     END SELECT 
                     GET   hFile,,MyArray()   ' load the data
                  I have never tried it, but instead of doing all possible combinations of datatype and number of dimensions, it might be possible to use the technique shown here...
                  Single Function to process any array type (CC3/Win7+) 12-17-03
                  .. to effect a REDIM "as a different datatype"

                  The doc says you can't change the number of dimensions, but does not say you can't change the datatype.

                  I think in the real world however, applications are NOT going to be dealing with a variable number of dimensions or "any" datatype, so you could cut down this code to support the number of dimensions and datatypes for this application.

                  Heck, it's a starting point.
                  MCM
                  Michael Mattias
                  Tal Systems (retired)
                  Port Washington WI USA
                  [email protected]
                  http://www.talsystems.com

                  Comment


                  • #10
                    Lucien:

                    I am very glad being able to help you, as I am very grateful for what I can learn every day here in PB Forums.

                    REDIM is a better choice since ERASE deallocate your previous DIMming so following an ERASE you have to DIM it again. If you simply want to clear the content of the array without changing it's dimensions then you can use RESET instead. REDIM can also be used with the PRESERVE option, so you can change the dimensioning of the ARRAY preserving it's content.

                    As far as I know every Basic compiler have the capacity of handling files as Binary.

                    Best regards,

                    Comment


                    • #11
                      >As far as I know every Basic compiler have the capacity of handling files as Binary

                      And if it doesn't....

                      Code:
                        OPEN "file" FOR RANDOM as #762  [COLOR="Red"][b]LEN=1[/b][/COLOR]
                      MCM
                      Michael Mattias
                      Tal Systems (retired)
                      Port Washington WI USA
                      [email protected]
                      http://www.talsystems.com

                      Comment


                      • #12
                        ¿ Is PB’s binary format some kind of standard, or..
                        PB stores Array Data in Column Major order.
                        Using PUT with Binary files, string array data is written to disk with a size value before each dynamic string element.
                        See this thread for more info on how to Save / Read arrays of string data using either PUT / GET or PRINT / LINE INPUT..
                        http://www.powerbasic.com/support/pb...ad.php?t=41125

                        To explore how integer values are handled, I've modified Manuel's original sample code a little below.
                        The test uses a small 3-dimensional array to see how the data is written to disk..
                        Code:
                        ' PBWin or PBCC
                        #COMPILE EXE
                        #DIM ALL
                        #INCLUDE "win32API.inc"                     ' For MessageBox()
                         
                        FUNCTION PBMAIN () AS LONG
                         LOCAL LT%(), II%, IJ%, IK%, Temp?()
                         DIM LT%(1,2,2) 
                         LOCAL FileLen&&, Count&, Elements&
                         
                          FOR II% = 0 TO 1
                           FOR IJ% = 0 TO 2
                            FOR IK% = 0 TO 2
                             LT%(II%,IJ%,IK%) = II%*100 + IJ%*10 + IK%   ' 0,1,2,10,11,12,20,21,22,100,101,102 etc
                            NEXT
                           NEXT
                          NEXT
                         
                          KILL "ARR.BIN"                            ' Fresh file each time
                          OPEN "ARR.BIN" FOR BINARY AS #1
                          ' PUT [#] fNum&, [RecPos], Arr()          ' Note - RecPos is optional 
                            PUT #1,,LT%()                           ' start writing at first position in file if not used.
                          CLOSE #1
                         
                          OPEN "ARR.BIN" FOR BINARY AS #1
                            FileLen&& = LOF(#1) : MessageBox 0, "File Length:" + str$(FileLen&&) + " Bytes","LOF from file",0 
                            REDIM Temp?(FileLen&&)                  ' For test - uses one dimensional array to retrieve data
                          ' GET [#] filenum&, [RecPos], Arr() [RECORDS rcds] [TO count]
                            GET #1,,Temp?() TO Count&               ' retrieve the full array data and count the Bytes read
                                                                    ' (FileLen should = Count, the number of bytes in file)
                          CLOSE #1
                         
                          RESET LT%()                               ' Clear array data
                          Elements& = ARRAYATTR(LT%(), 4)           ' With mods array params could be saved with & retrieved from file data?
                          REDIM LT%(1,2,2) AT VarPtr(Temp?(0))      ' Overlay an Integer array with appropriate dimensions.
                                                                    ' NB: Number and size of array dimensions must be correct !!
                          MessageBox 0, "Data from file." + $CR + _
                                        "Array Val: (1,1,2) =" + str$(LT%(1,1,2)), "Array Elements:" + str$(Elements&), 0 ' verify
                        END FUNCTION
                        '------------------/
                        #IF 0
                         Data derived from nested for-next loops:
                         0,1,2,10,11,12,20,21,22,100,101,102,110,111,112,120,121,122
                         
                         Integers are stored in memory as two bytes per value in 'LSB, MSB' order.
                         Array values are stored in memory and PUT to file in Column Major order:
                         (Using hex editor to read ARR.BIN..)
                          000000  00 00 64 00 0a 00 6e 00 14 00 78 00   0,100, 10,110, 20,120
                          00000c  01 00 65 00 0b 00 6f 00 15 00 79 00   1,101, 11,111, 21,121
                          000018  02 00 66 00 0c 00 70 00 16 00 7a 00   2,102, 12,112, 22,122
                         
                         PB's GET statement is used to rewrite file data to memory.
                        #ENDIF
                        Rgds, Dave

                        Comment

                        Working...
                        X