Announcement

Collapse
No announcement yet.

storing a dynamic string array

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • storing a dynamic string array

    Hi there,

    is there a better way of saving a dynamic string array to disk as
    looping all through it and save it line by line ?

    Are there any documents about pb's internal array header ?

    Best rgds
    Ralph

  • #2
    Internally, PowerBASIC uses OLE string engine for dynamic strings. No official documentation has been released on the array headers or handles.

    If performance is your reason for this question, maybe you need to revise the way you are holding the data in memory? ie, use a single large "buffer" (and use pointers to manipulate the contents) instead of an array? What exactly is the problem you wish to solve?

    ------------------
    Lance
    PowerBASIC Support
    mailto:[email protected][email protected]</A>
    Lance
    mailto:[email protected]

    Comment


    • #3
      Writing each element to disk can take a long time, but if you use
      a string buffer when you save the array, it can speed up the process
      a lot. The following example is written from memory, so it may need
      some adjustment, but just to show you the general idea:

      Code:
        LOCAL I AS LONG, bPos AS LONG, ub AS LONG, Buffer AS STRING
       
        ub = UBOUND(Array): bPos = 1: Buffer = SPACE$(32000)
       
        FOR I = 0 TO ub - 1
          MID$(Buffer, bPos) = Array(I) & $CRLF
          bPos = bPos + LEN(Array(I)) + 2
          IF bPos > 30000 OR I = ub - 1 THEN  'dump it to disk
            'the PRINT statement automatically adds a CRLF, so..
            PRINT# 1, LEFT$(Buffer, bPos - 3)
            bPos = 1: Buffer = SPACE$(32000)
          END IF
        NEXT
      ------------------

      Comment


      • #4
        I want to join to Borje - buffering is very important - first of all, for writing, but not less for reading.
        I tested following code on my PC (550Mhz, UW SCSI)
        Code:
        #Compile Exe
        #Register None
        #Dim All
        #Include "WIN32API.INC"
        
        Function PbMain
          %MaxBufferSize = 8192
          Dim strAr(100000 : 900000) As String
          Local i As Long, l As Long, lb As Long, ub As Long, bPos As Long
          lb = LBound(strAr): ub = UBound(strAr)
          For i = lb To ub: strAr(i) = String$(115, "*") + " No." + Str$(i): Next
        
          ' Variant 1
          Local Buffer As String * %MaxBufferSize
          ' Variant 2 (approx. eq.1)
          'Local Buffer As String: Buffer = Space$(%MaxBufferSize)
          ' Variant 3 (not effective)
          'Local Buffer As Asciiz * (%MaxBufferSize + 1): Buffer = Space$(%MaxBufferSize)
        
          Local t1 As Double: t1 = Timer
          Open "~1" For Output As #1
          For i = lb To ub
            l = Len(strAr(i)) + 2
            If (%MaxBufferSize - bPos) < l Then _
                Print# 1, Left$(Buffer, bPos);: bPos = 0
            Mid$(Buffer, bPos + 1, l) = strAr(i) + $CRLF
            bPos = bPos + l
          Next
          Print# 1, Left$(Buffer, bPos);
          Close #1
          MsgBox "Writing time is " + Str$(Timer - t1)
          ReDim strAr(100000 : 900000) As String
          t1# = Timer
          Open "~1" For Input As #1
          For i = lb To ub
            Line Input #1, strAr(i)
          Next
          Close #1
          MsgBox "Reading time is " + Str$(Timer - t1)
        End Function
        To create 100 mb file is necessary about 8 sec (not bad).
        Reading time (w/o buffering takes about 36 sec).
        With size of buffer not all clear (at least, on my PC, because SCSI uses own cash). I feel difference when decrease size less than 1 Kb. But for ordinary PC, I think, it's necessary to take 1,5-2 * cluster size. Perhaps, 8 Kb is enough.

        ------------------


        [This message has been edited by Semen Matusovski (edited February 26, 2000).]

        Comment


        • #5
          Ralph, I use a pretty easy/fast method that goes like this: When you save your dynamic strings collate them into one big string and save each strings length in a byte/int/long array (depending on the maximum length of your strings) at the same time. Save the index array and the string array as binary files. To retrieve them just open the index file and the string file and load them into memory in one shot. The number of strings in your file is the length of the index file divided by 1,2 or 4 (depending on what index size you used). To access an element, simply add up the indexes up to that index number-1 and this'll give you the start character of the string. The length of the string is in index number. So, to get element 5....x$ = mid$(BigString, Index(1)+Index(2)+Index(3)+Index(4), Index(5)). If these things are getting too big for memory you can do the same thing with get$ statements. Obviously it has it's limitations, especially if you need to edit the existing records. You can toy around with the general idea to make it work for you..ie after loading the two files into memory, put them into a dynamic array and destroy the string and index array to free memory. My goal was to make fast disk accesses across a network. Anyway, it's just another idea.

          ------------------

          Comment


          • #6
            Lance, I think you comments re OLE strings apply only to 32-bit PB/DLL. Yes, PB still ships and supports a 16-bit compiler; and yes, 16-bit applications are still in use. In fact, the U.S. Government's HCFA (Health Card Financing Administration) mandates that software provided to Medicare providers by authorized TPAs work on 16-bit systems!

            Anyway, assuming this is a 32-bit question, why do we need to save the string array to disk anyway? You can handle 2Gb of user data in the 32-bit environment in memory/Windows-managed virtual memory; are we trying to manage more than 2GB?



            ------------------
            Michael Mattias
            Racine WI USA
            [email protected]
            Michael Mattias
            Tal Systems Inc. (retired)
            Racine WI USA
            [email protected]
            http://www.talsystems.com

            Comment


            • #7
              Thanks for all the replies. I tried to use a string array to
              keep message texts of different size in memory. Found a much
              smarter solution for my problem. I'll use a "structured file".
              Pls look at my posting in programming - topic "in file - file system".

              The aproach is a little bit like wyman's suggestion. But instead
              of an index array a "in file file system" is more flexible.
              The GNU CPP source i use for this is part of the Sinclair QL
              emulator for linux. Shall i post it ?

              Best rgds
              Ralph

              ------------------

              Comment


              • #8
                Ralph

                yes please: I'd love to look at the CPP source. If it's too long to post my e-mail address is mailto:[email protected][email protected]</A>

                Thanks

                Florent



                [This message has been edited by Florent Heyworth (edited February 26, 2000).]

                Comment

                Working...
                X