Announcement

Collapse
No announcement yet.

File Reading - VB Smokes PBDLL

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • File Reading - VB Smokes PBDLL

    All,

    I have tried the following code in both VB and PBDLL.
    Much to my dismay, VB reads my file in 3 seconds and PBDLL takes between 11 and 13 seconds. Is there something I am missing?

    Thanks In Advance,

    Rich
    ''***********************************************************
    FUNCTION LoadTextFile(BYVAL sFileName AS STRING) AS LONG


    DIM iFreeFile AS INTEGER
    DIM strTemp AS STRING

    DIM sStart AS STRING
    DIM sEndTime AS STRING

    sStart = TIME$
    iFreeFile = FREEFILE

    OPEN sFilename FOR INPUT AS #iFreeFile
    WHILE NOT EOF(iFreeFile)
    LINE INPUT #iFreeFile, strTemp

    WEND
    CLOSE #iFreeFile

    sEndTime = TIME$


    END FUNCTION

    _____________________________________________________________________________________
    It's hard when you're up to your armpits in alligators to remember you came here to drain the swamp.
    President Reagan
    February 10, 1982

  • #2
    At a guess, VB's using larger default file buffers. You can change the settings in PB with the LEN= option of the OPEN statement.

    ------------------
    Tom Hanlin
    PowerBASIC Staff

    Comment


    • #3
      Tom;

      You may want to update the PB Help file so it mentions the use of LEN for INPUT mode in the OPEN statement. The PB docs only mention using it for RANDOM files record size and not any other modes. I would never have known LEN could be used for INPUT mode if you hadn't mentioned it here and that it affected the buffer size.



      ------------------
      Chris Boss
      Computer Workshop
      Developer of "EZGUI"
      http://cwsof.com
      http://twitter.com/EZGUIProGuy

      Comment


      • #4
        On a 66 MB file:

        VB 43.672 Secs and 41.33919 Secs

        PB/CC 23.50731 Secs and 23.78144 Secs

        Code:
        PowerBASIC Code
        Function PbMain
          Dim t As Single
          Dim a$
          t = Timer
          Open "D:\PB_Stuff\DBArchive\Apr17\pagerMessages.out" For Input As #1 Len = 16384
          Do While Not EOF(1)
            Line Input #1, a$
          Loop
          Close 1
          t = Timer - t
          StdOut "PB Elapsed:" & str$(t)
        End Function
         
         
        VB Code
        Option Explicit
         
        Private Sub Command1_Click()
          Dim t As Single
          Dim a$
          t = Timer
          Open "D:\PB_Stuff\DBArchive\Apr17\pagerMessages.out" For Input As #1 Len = 16384
          Do While Not EOF(1)
            Line Input #1, a$
          Loop
          Close 1
          t = Timer - t
          Text1.Text = "VB Elapsed:" & t
        End Sub
        Ron

        Comment


        • #5
          I appreciate everyone's input. I'm going to research the LEN = option. This not a homogeneous file. It has some lines that are five characters and others in the thousands of characters.

          It will be interesting...

          Thanks,

          Rich



          ------------------
          _____________________________________________________________________________________
          It's hard when you're up to your armpits in alligators to remember you came here to drain the swamp.
          President Reagan
          February 10, 1982

          Comment


          • #6
            Interesting !

            I checked my VB docs and it does say that LEN is the buffer size for INPUT mode (LEN is meaningless for BINARY mode though).

            I assume then that OPEN in PB must work exactly the same as it does in VB (in this particular case about LEN).

            Thanks RON for the test code. Once again PB wins out over VB.


            ------------------
            Chris Boss
            Computer Workshop
            Developer of "EZGUI"
            http://cwsof.com
            http://twitter.com/EZGUIProGuy

            Comment


            • #7
              Rick,

              I have a solution that allows for fast reading from a input sequential file (using CR+LF, CR only delimiter or even any other delimiter). It is very fast because it uses directly the core API file I/O.
              If you want to try it, let me know.


              ------------------
              Patrice Terrier
              mailto[email protected][email protected]</A>

              [This message has been edited by Patrice Terrier (edited April 19, 2000).]
              Patrice Terrier
              www.zapsolution.com
              www.objreader.com
              Addons: GDImage.DLL 32/64-bit (Graphic library), WinLIFT.DLL 32/64-bit (Skin Engine).

              Comment


              • #8
                Guys --
                I don't like Line Input, because of dynamic strings.
                If file is not gigant (not more 1-2 Mb), I prefer to open as Binary, to read whole by Get$ #f, Lof(f), s.
                Then to search $CRLF. Not bad way - Redim Ar(0 To Lof(f) - 1) As Byte At StrPtr(s).

                PS. To my surprise, my workarounds is faster a little only (25-30%). Big LEN in PB gives enough good results.

                [This message has been edited by Semen Matusovski (edited April 19, 2000).]

                Comment


                • #9
                  I have experimented with the Len = option. On my test file, it decreases the time to five seconds. This is still two seconds slower than VB.

                  Patrice - I appreciate your offer. I look forward to trying out your code on the test file.

                  Rich




                  ------------------
                  _____________________________________________________________________________________
                  It's hard when you're up to your armpits in alligators to remember you came here to drain the swamp.
                  President Reagan
                  February 10, 1982

                  Comment


                  • #10
                    Rich, I use VB a lot and I have not found VB's file reading to
                    top PB's file reading in any file mode. File reading is one VB does quite well.
                    Would you mind posting the code for VB and PB which you used as a test.
                    Are you calling a PB/DLL from VB? How large is the file you are reading and how many lines are in it?

                    Knowing the number of lines and total bytes would allow a similar test to be performed.

                    Thanks.

                    Ron

                    Comment


                    • #11
                      I compared VB6 (compiled) and PBDLL6.
                      Test file - copy of Windows-98 registry file (.reg), 7Mb
                      And received following (Win2000, P-III-550, fast SCSI-3 HDD):
                      Code:
                                                               VB         PB
                      If to read whole file                  300 ms       60 ms 
                      (without dividing to lines)
                      Line Input (Len = 16384)               1140 ms      320 ms
                      Rich --
                      Which OS do you use ?
                      If Windows 9x, you started PB program first, then VB ?
                      (there are some jokes with buffers; I have impression that Windows remembers previously read file and really doesn't read it again; it's very well visible with floppy)



                      [This message has been edited by Semen Matusovski (edited April 19, 2000).]

                      Comment


                      • #12
                        I have posted a fast and easy to use line input replacement in the source code section.

                        Regards


                        ------------------
                        Patrice Terrier
                        mailto[email protected][email protected]</A>
                        Patrice Terrier
                        www.zapsolution.com
                        www.objreader.com
                        Addons: GDImage.DLL 32/64-bit (Graphic library), WinLIFT.DLL 32/64-bit (Skin Engine).

                        Comment


                        • #13
                          Semen - I am using a P III with NT 4.0 SP 5 and 256 MB of memory.

                          ' VB Code **************************************************
                          Option Explicit


                          Private Sub cmdTime_Click()
                          Dim sngTime As Single
                          Dim sTempInput As String
                          Dim iFreeFile As Integer

                          iFreeFile = FreeFile

                          sngTime = Timer
                          Open "c:\test\mytest.txt" For Input As #iFreeFile
                          Do While Not EOF(iFreeFile)
                          Line Input #iFreeFile, sTempInput
                          Loop
                          Close iFreeFile
                          sngTime = Timer - sngTime
                          txtTime.Text = "VB Elapsed:" & sngTime

                          End Sub


                          ' PB Code **************************************************

                          FUNCTION LoadTextFile(BYVAL sFileName AS STRING) AS LONG


                          DIM iFreeFile AS INTEGER
                          DIM sTemp AS STRING
                          DIM sngTime AS SINGLE


                          iFreeFile = FREEFILE

                          sngTime = TIMER
                          OPEN sFileName FOR INPUT AS #iFreeFile
                          DO WHILE NOT EOF(iFreeFile)
                          LINE INPUT #iFreeFile, sTemp
                          LOOP
                          CLOSE iFreeFile
                          sngTime = TIMER - sngTime

                          END FUNCTION

                          '**************************************************************

                          I ran both in the debugger. The PB version I stop on the "End Function" to check the time.

                          The file size is 18,911 KB.


                          Rich




                          ------------------
                          _____________________________________________________________________________________
                          It's hard when you're up to your armpits in alligators to remember you came here to drain the swamp.
                          President Reagan
                          February 10, 1982

                          Comment


                          • #14
                            Patrice,

                            Thanks for the code!!

                            Rich

                            ------------------
                            _____________________________________________________________________________________
                            It's hard when you're up to your armpits in alligators to remember you came here to drain the swamp.
                            President Reagan
                            February 10, 1982

                            Comment


                            • #15
                              I tried this code in VB, it's not even CLOSE to comparing to PB, PB smokes VB on my P2/450 w/256 megs of ram on Windows 2000 server....
                              If I recall PB did the 37 meg file in less than 45 seconds, the RC4NcryptString uses pointers so has virtualy no overhead...

                              If it helps:

                              Code:
                              Function InitThread(ByVal x As Long) Export As Long
                              Local ErrType   As Integer
                              Local FilePos   As Long
                              Local FileSize  As Long
                              Local MaxBuff   As Long
                              Local InData    As String
                              Local Buff      As String
                              Local zText     As Asciiz * 255
                              g_lngInitThreadHandle = %TRUE
                              On Error GoTo ERRTRAP
                              'Set Mouse busy
                              MousePtr 11
                              If Exist(FileSpec) Then
                                  F = FreeFile
                                  Open FileSpec For Binary Access Read Lock Shared As #F
                                  G = FreeFile
                                  Open TmpFile For Binary Access Write Lock Shared As #G
                              Else
                                  MsgBox FileSpec + " does not exist!", %MB_ICONSTOP, Mine
                                  Exit Function
                              End If
                              
                              FileSize = Lof(F)
                              BytesToRead& = FileSize
                              FilePos& = 1
                              MaxBuff& = 4096
                              
                              Control Send hDlg, %PROGRESS, %PBM_SETRANGE, 0 ,MakLng(0,FileSize&\MaxBuff&)
                              Control Send hDlg, %PROGRESS, %PBM_SETSTEP, 1 , 0  'MakLng(0,MaxBuff&)
                              
                              
                              While IsTrue BytesToRead&
                                    BuffSize& = Min(BytesToRead&, MaxBuff&)
                                    BytesToRead& = BytesToRead& - BuffSize&
                                    Get$ F, BuffSize&, InData
                                    Buff = RC4NcryptString(ByVal InData, KeySt, DLL_KeyNum,DLL_CipherStrength)
                              '      Buff = rc4(InData)
                                    Control Send hDlg, %PROGRESS, %PBM_STEPIT , 0, 0
                                    Put$ G,Buff
                                    FilePos = FilePos + BuffSize&
                                    Seek F, FilePos         'Continue file operations
                              Wend
                              Close F
                              Close G  
                              nKill FileSpec
                              Name TmpFile As FileSpec
                              'Set Global Flag to False here
                              g_lngInitThreadHandle = %FALSE
                              'Reset mouse pointer
                              MousePtr 0
                              
                              Dialog End hDlg, 1
                              Dialog End tDlg, 1
                              
                              Exit Function
                              ERRTRAP:
                              ErrType = Err
                              
                              Result = LoadString(hInst, LoWrd(ErrType), zText,SizeOf(zText))
                              MsgBox "Error " + Trim$(Str$(ErrType)) + zText, %MB_ICONSTOP, Mine
                              End Function
                              ------------------
                              Scott
                              mailto:[email protected][email protected]</A>

                              [This message has been edited by Scott Turchin (edited April 19, 2000).]
                              Scott Turchin
                              MCSE, MCP+I
                              http://www.tngbbs.com
                              ----------------------
                              True Karate-do is this: that in daily life, one's mind and body be trained and developed in a spirit of humility; and that in critical times, one be devoted utterly to the cause of justice. -Gichin Funakoshi

                              Comment


                              • #16
                                Originally posted by Rich Brockway:
                                I ran both in the debugger.
                                Rich --
                                It's incorrect test.
                                Compiled and IDE VB program executes (for this sample) approx. the same time.
                                But not PB.
                                On my PC: PB (compiled) - 320 ms. In debugger - 28 seconds !!!

                                ------------------

                                Comment


                                • #17
                                  Rich;

                                  Likely when you run the program from within the PB Debugger, PB is creating a "temporary" version of the program compiled with "Extra" debugger code after "every single command" in your code. This extra over head makes it possible to track things that normally could not be tracked in a normally compile program.

                                  This "extra" code is what is making the program so slow.

                                  VB is uniquely different than most compilers, so their is little difference between the compiled version and running it in the VB IDE. VB isn't really a true compiler, since it uses large runtimes to do most things and little native code is really created when compiled. PB on the other hand compiles to native code and has very little runtime code (embeded in your EXE). PB "smokes" compared to VB, but obviously there is a downside to using the debugger.


                                  ------------------
                                  Chris Boss
                                  Computer Workshop
                                  Developer of "EZGUI"
                                  http://cwsof.com
                                  http://twitter.com/EZGUIProGuy

                                  Comment


                                  • #18
                                    Rich, in Vb's IDE code the running code is threaded 'P' code.
                                    It sometimes outperforms VB's so-called "Native Code" exe's.

                                    The PowerBASIC debugger is not going to give you any measure of its performance whatsoever.

                                    Compile the PowerBasic code and then perform your benchmark agains the Vb code compiled in
                                    both output code models ('P' code and 'native' code).

                                    Havagud1,
                                    Ron

                                    Comment


                                    • #19
                                      Final verdict from compiled exes with my file:

                                      VB 2.3
                                      PB 2.7

                                      Rich

                                      ------------------
                                      _____________________________________________________________________________________
                                      It's hard when you're up to your armpits in alligators to remember you came here to drain the swamp.
                                      President Reagan
                                      February 10, 1982

                                      Comment


                                      • #20
                                        Have a look at Patrice Terrier's fast line input replacement
                                        BUFIN.bas in the "Source code Forum":

                                        measured on my 400 MHz 256 MB computer:

                                        tested on a 110 MB file

                                        PB:
                                        -------------------------------------------------
                                        15.46 seconds

                                        compiled VB with all optimisations and RecLen=32767
                                        --------------------------------------------------
                                        34.42 seconds

                                        What more can be said?

                                        Cheers

                                        Florent

                                        ------------------

                                        Comment

                                        Working...
                                        X