Announcement

Collapse
No announcement yet.

Multi-Threaded Split Command

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multi-Threaded Split Command

    I am writing a dll function that is going to be performing
    duty cycle tests on a square wave. I am using a program
    called test point which is VERY slow for mathematical
    operations so I am going to make this.

    It first loads up a file called "c:\greg.dat" which is
    tab delimited and CRLF after each row. There are
    16 rows and 350 lines so it loads that up and what I'm trying
    to do is populate an array matrix [Rows,Cols] the downfall is when
    I use Dave's Split command, it seems that is takes very long
    to get done with the split procedure. So at first I thought
    maybe if I put it in a thread, well it still seems to take longer
    then it should. What I am trying to accomplish is something
    that takes only a second or so because after I put it in a [1,0]
    array then I have to sort it out and put it in the [rows,cols]
    array and then do all the math stuff. Any ideas? I was thinking
    maybe if there was a may to actually thread the split command?

    ah another idea I just thought of was splitting the buffer up in
    multiple parts and then reassembling them and calling multiple
    threads. I'll try and but any other ideas are welcome.

    here is the code, to create the greg.dat file open up excel
    and type random numbers in a column with 7 decimal places and
    have it go down to 350 rows and then just copy over to fill
    all 16 columns and then save as a TAB delimited file. Or
    email me at [email protected] and I'll send you the file.

    Code:
    COMPILE EXE
    GLOBAL StringArray() AS STRING
    GLOBAL Taskbuffer AS STRING
    
    FUNCTION GetRows(InBuffer AS STRING) AS LONG
    END FUNCTION
    
    SUB Split(BYVAL expression AS STRING, BYVAL delimeter AS STRING, StrArray() AS STRING)
      LOCAL c AS LONG
      LOCAL x AS LONG
      IF LEN(delimeter) = 0 THEN
        delimeter = " "
      END IF
      c = MAX(PARSECOUNT(expression, delimeter), 1)
      REDIM StrArray(0 TO c - 1) AS STRING
      FOR x = 1 TO c
        StrArray(x - 1) = PARSE$(expression, delimeter, x)
      NEXT
    END SUB
    
    FUNCTION MakeArray(InputId AS LONG) EXPORT AS LONG
       'MSGBOX sbuffer
       Split TaskBuffer, $TAB, StringArray()
    '   MSGBOX STR$(UBOUND(StringArray()))
    END FUNCTION
    
    
    FUNCTION OutputDuty(InFilename AS STRING) AS LONG
        '//open InFilename and pump into buffer
    DIM sbuffer AS STRING
        OPEN InFilename FOR BINARY AS #1
        sbuffer = STRING$(LOF(1), 0)
        GET 1,,sbuffer
        CLOSE #1
        '//Make temporary buffer for THREADED array function
        TaskBuffer = sbuffer
        '//Replace all the CRLF with tabs for easier importing
        REPLACE $CRLF WITH $TAB IN TaskBuffer
        '//Launch THREAD for [1,0] array
    DIM ThreadID AS LONG
        THREAD CREATE MakeArray(0) TO ThreadID
        '//find first CRLF
    DIM FirstCRLF AS LONG
        FirstCRLF = INSTR(1, sbuffer, $CRLF)
        '//put first line in buffer
    DIM linebuffer AS STRING
        linebuffer = LEFT$(sbuffer, FirstCRLF)
        '//count how many fields are present and ADD + for CRLF
    DIM NumOfCols AS LONG
        NumOfCols = TALLY(linebuffer, $TAB) + 1
        '//clear line buffer
        linebuffer = ""
        '//Get total number CRLF
    DIM NumOfRows AS LONG
        NumOfRows = TALLY(sbuffer, $CRLF)
        '//Number of Records
    DIM NumOfRecords AS LONG
        NumOfRecords = NumOfRows * NumOfCols
        '//Assemble a Matrix on [NumRows X NumCols]
    DIM Matrix(NumofRows,NumOfCols) AS STRING
        '//wait for matrix to finish
    DIM ThreadResult AS LONG
        DO
            THREAD STATUS MakeArray(0) TO ThreadResult
        LOOP WHILE ThreadResult
    
    
    MSGBOX STR$(UBOUND(StringArray()))
    END FUNCTION
    
    
    FUNCTION PBMAIN() AS LONG
    MSGBOX STR$(OutPutDuty("C:\greg.dat"))
    'MSGBOX STR$(UBOUND(StringArray()))
    END FUNCTION
    ------------------
    -Greg

    [This message has been edited by Gregery D Engle (edited July 12, 2000).]
    -Greg
    [email protected]
    MCP,MCSA,MCSE,MCSD

  • #2
    I found where the problem was when I used the command
    THREAD STATUS it would take an extra 5 seconds or so to return a
    value so instead I just check the ubound on the array and if
    its > 0 then it exits the loop

    Thanks anyways.

    ------------------
    -Greg
    -Greg
    [email protected]
    MCP,MCSA,MCSE,MCSD

    Comment


    • #3
      Greg --
      What do you name "slowly" ?
      Sure, that on average PC it possible to convert tab-delimitted array 1-2 Mb to Asciiz array during 100-200ms maximum.
      I don't know Dave's algorithm; I mean a method, posted above.

      I cut a fragment from one utility (for internal usage; so, there is no deep control) and tested on my real array
      (Rows = 8741; Columns = 9; SizeofFile = 535145 bytes)

      Results (Win2000, P-III-550, fast SCSI HDD) - 20ms.

      Code:
         #Compile Exe
         #Register None
         #Dim All
         #Include "Win32Api.Inc"
      
         %maxEl = 100000
         $FileNm = "nsiesr.txt"
            
         Function PbMain
            Dim t1 As Single, t2 As Single
            t1 = Timer
            
            Dim Info As String, SizeOfFile As Long
            Dim k1 As Byte Ptr, k2 As Byte Ptr, kk As Byte Ptr
            Dim nRow As Long, nCol As Long, nEl As Long
            Open $FileNm For Binary As #1
            SizeOfFile = Lof(1): Get$ #1, SizeOfFile, Info
            Close #1
        
            ReDim ArTmp(%maxEl) As Asciiz Ptr
            k1 = StrPtr(Info): kk = k1: k2 = k1 + SizeOfFile
            Do
               If k1 >= k2 Then Exit Do
               If @k1 = 9 Then
                  ArTmp(nEl) = kk: kk = k1 + 1: @k1 = 0: Incr nEl
               ElseIf @k1 = 13 Then
                  ArTmp(nEl) = kk: kk = k1 + 2: @k1 = 0: Incr nEl
                  If nCol = 0 Then nCol = nEl
                  Incr nRow: If (nRow * nCol) <> nEl Then Beep: Exit Function
               End If
               Incr k1
            Loop
            If (nRow * nCol) <> nEl Then Beep: Exit Function
        
            ReDim Ar(1 To nCol, 1 To nRow) As Asciiz Ptr At VarPtr(ArTmp(0))
            
            t2 = Timer
            MsgBox "nRow =" + Str$(nRow) + "  " + "  nCol =" + Str$(nCol) + _
               " SizeOfFile =" + Str$(SizeOfFile), , "Time = " + Format$(t2 -t1, "##.### ms")
      
            ' Usage @Ar(Column, Row) !!!
            MsgBox @Ar(2, 2)
      
         End Function
      [This message has been edited by Semen Matusovski (edited July 12, 2000).]

      Comment


      • #4
        Semen, Greg's code is using a thread in a very inefficient way... his preimary thread is locked in a loop waiting for the second thread to finish... in this instance it would be more efficient to place all the code in the main thread.

        However, if Greg wishes to retain the multi-threaded approach for other reasons, he may do well to consider releasing time slices in his "wait for thread to finish" loop so as to cede processor time to the other thread... ie:
        Code:
        ...
        DO        
          THREAD STATUS MakeArray(0) TO ThreadResult    
          SLEEP 100  ' cede approx 100mSec to other threads & processes.
        LOOP WHILE ThreadResult
        ...


        ------------------
        Lance
        PowerBASIC Support
        mailto:[email protected][email protected]</A>
        Lance
        mailto:[email protected]

        Comment


        • #5
          lance -
          how greg uses threads is separate question.
          when i wrote previous message, i didn't know dave's algorithm.
          just now i found it.
          like algorithm it's absolutelly ineffective
          i guess, that dave wanted to demonstration pb functions only.

          ------------------

          Comment

          Working...
          X