File Reading - VB Smokes PBDLL

[Rich Brockway]

All,

I have tried the following code in both VB and PBDLL.
Much to my dismay, VB reads my file in 3 seconds and PBDLL takes between 11 and 13 seconds. Is there something I am missing?

Thanks In Advance,

Rich
''***********************************************************
FUNCTION LoadTextFile(BYVAL sFileName AS STRING) AS LONG


DIM iFreeFile AS INTEGER
DIM strTemp AS STRING

DIM sStart AS STRING
DIM sEndTime AS STRING

sStart = TIME$
iFreeFile = FREEFILE

OPEN sFilename FOR INPUT AS #iFreeFile
WHILE NOT EOF(iFreeFile)
LINE INPUT #iFreeFile, strTemp

WEND
CLOSE #iFreeFile

sEndTime = TIME$


END FUNCTION

[Tom Hanlin]

At a guess, VB's using larger default file buffers. You can change the settings in PB with the LEN= option of the OPEN statement.

------------------
Tom Hanlin
PowerBASIC Staff

[Chris Boss]

Tom;

You may want to update the PB Help file so it mentions the use of LEN for INPUT mode in the OPEN statement. The PB docs only mention using it for RANDOM files record size and not any other modes. I would never have known LEN could be used for INPUT mode if you hadn't mentioned it here and that it affected the buffer size.



------------------

[Guest]

On a 66 MB file:

VB 43.672 Secs and 41.33919 Secs

PB/CC 23.50731 Secs and 23.78144 Secs

[b]PowerBASIC Code[/b]
Function PbMain
  Dim t As Single
  Dim a$
  t = Timer
  Open "D:\PB_Stuff\DBArchive\Apr17\pagerMessages.out" For Input As #1 Len = 16384
  Do While Not EOF(1)
    Line Input #1, a$
  Loop
  Close 1
  t = Timer - t
  StdOut "PB Elapsed:" & str$(t)
End Function
 
 
[b]VB Code[/b]
Option Explicit
 
Private Sub Command1_Click()
  Dim t As Single
  Dim a$
  t = Timer
  Open "D:\PB_Stuff\DBArchive\Apr17\pagerMessages.out" For Input As #1 Len = 16384
  Do While Not EOF(1)
    Line Input #1, a$
  Loop
  Close 1
  t = Timer - t
  Text1.Text = "VB Elapsed:" & t
End Sub

Ron

[Rich Brockway]

I appreciate everyone's input. I'm going to research the LEN = option. This not a homogeneous file. It has some lines that are five characters and others in the thousands of characters.

It will be interesting...

Thanks,

Rich



------------------

[Chris Boss]

Interesting !

I checked my VB docs and it does say that LEN is the buffer size for INPUT mode (LEN is meaningless for BINARY mode though).

I assume then that OPEN in PB must work exactly the same as it does in VB (in this particular case about LEN).

Thanks RON for the test code. Once again PB wins out over VB.


------------------

[Patrice Terrier]

Rick,

I have a solution that allows for fast reading from a input sequential file (using CR+LF, CR only delimiter or even any other delimiter). It is very fast because it uses directly the core API file I/O.
If you want to try it, let me know.


------------------
Patrice Terrier
mailto[email protected][email protected]</A>

[This message has been edited by Patrice Terrier (edited April 19, 2000).]

[Semen Matusovski]

Guys --
I don't like Line Input, because of dynamic strings.
If file is not gigant (not more 1-2 Mb), I prefer to open as Binary, to read whole by Get$ #f, Lof(f), s.
Then to search $CRLF. Not bad way - Redim Ar(0 To Lof(f) - 1) As Byte At StrPtr(s).

PS. To my surprise, my workarounds is faster a little only (25-30%). Big LEN in PB gives enough good results.

[This message has been edited by Semen Matusovski (edited April 19, 2000).]

[Rich Brockway]

I have experimented with the Len = option. On my test file, it decreases the time to five seconds. This is still two seconds slower than VB.

Patrice - I appreciate your offer. I look forward to trying out your code on the test file.

Rich




------------------

[Guest]

Rich, I use VB a lot and I have not found VB's file reading to
top PB's file reading in any file mode. File reading is one VB does quite well.
Would you mind posting the code for VB and PB which you used as a test.
Are you calling a PB/DLL from VB? How large is the file you are reading and how many lines are in it?

Knowing the number of lines and total bytes would allow a similar test to be performed.

Thanks.

Ron

[Semen Matusovski]

I compared VB6 (compiled) and PBDLL6.
Test file - copy of Windows-98 registry file (.reg), 7Mb
And received following (Win2000, P-III-550, fast SCSI-3 HDD):

                                         VB         PB
If to read whole file                  300 ms       60 ms 
(without dividing to lines)
Line Input (Len = 16384)               1140 ms      320 ms

Rich --
Which OS do you use ?
If Windows 9x, you started PB program first, then VB ?
(there are some jokes with buffers; I have impression that Windows remembers previously read file and really doesn't read it again; it's very well visible with floppy)



[This message has been edited by Semen Matusovski (edited April 19, 2000).]

[Patrice Terrier]

I have posted a fast and easy to use line input replacement in the source code section.

Regards


------------------
Patrice Terrier
mailto[email protected][email protected]</A>

[Rich Brockway]

Semen - I am using a P III with NT 4.0 SP 5 and 256 MB of memory.

' VB Code **************************************************
Option Explicit


Private Sub cmdTime_Click()
Dim sngTime As Single
Dim sTempInput As String
Dim iFreeFile As Integer

iFreeFile = FreeFile

sngTime = Timer
Open "c:\test\mytest.txt" For Input As #iFreeFile
Do While Not EOF(iFreeFile)
Line Input #iFreeFile, sTempInput
Loop
Close iFreeFile
sngTime = Timer - sngTime
txtTime.Text = "VB Elapsed:" & sngTime

End Sub


' PB Code **************************************************

FUNCTION LoadTextFile(BYVAL sFileName AS STRING) AS LONG


DIM iFreeFile AS INTEGER
DIM sTemp AS STRING
DIM sngTime AS SINGLE


iFreeFile = FREEFILE

sngTime = TIMER
OPEN sFileName FOR INPUT AS #iFreeFile
DO WHILE NOT EOF(iFreeFile)
LINE INPUT #iFreeFile, sTemp
LOOP
CLOSE iFreeFile
sngTime = TIMER - sngTime

END FUNCTION

'**************************************************************

I ran both in the debugger. The PB version I stop on the "End Function" to check the time.

The file size is 18,911 KB.


Rich




------------------

[Rich Brockway]

Patrice,

Thanks for the code!!

Rich

------------------

[Scott Turchin]

I tried this code in VB, it's not even CLOSE to comparing to PB, PB smokes VB on my P2/450 w/256 megs of ram on Windows 2000 server....
If I recall PB did the 37 meg file in less than 45 seconds, the RC4NcryptString uses pointers so has virtualy no overhead...

If it helps:

Function InitThread(ByVal x As Long) Export As Long
Local ErrType   As Integer
Local FilePos   As Long
Local FileSize  As Long
Local MaxBuff   As Long
Local InData    As String
Local Buff      As String
Local zText     As Asciiz * 255
g_lngInitThreadHandle = %TRUE
On Error GoTo ERRTRAP
'Set Mouse busy
MousePtr 11
If Exist(FileSpec) Then
    F = FreeFile
    Open FileSpec For Binary Access Read Lock Shared As #F
    G = FreeFile
    Open TmpFile For Binary Access Write Lock Shared As #G
Else
    MsgBox FileSpec + " does not exist!", %MB_ICONSTOP, Mine
    Exit Function
End If

FileSize = Lof(F)
BytesToRead& = FileSize
FilePos& = 1
MaxBuff& = 4096

Control Send hDlg, %PROGRESS, %PBM_SETRANGE, 0 ,MakLng(0,FileSize&\MaxBuff&)
Control Send hDlg, %PROGRESS, %PBM_SETSTEP, 1 , 0  'MakLng(0,MaxBuff&)


While IsTrue BytesToRead&
      BuffSize& = Min(BytesToRead&, MaxBuff&)
      BytesToRead& = BytesToRead& - BuffSize&
      Get$ F, BuffSize&, InData
      Buff = RC4NcryptString(ByVal InData, KeySt, DLL_KeyNum,DLL_CipherStrength)
'      Buff = rc4(InData)
      Control Send hDlg, %PROGRESS, %PBM_STEPIT , 0, 0
      Put$ G,Buff
      FilePos = FilePos + BuffSize&
      Seek F, FilePos         'Continue file operations
Wend
Close F
Close G  
nKill FileSpec
Name TmpFile As FileSpec
'Set Global Flag to False here
g_lngInitThreadHandle = %FALSE
'Reset mouse pointer
MousePtr 0

Dialog End hDlg, 1
Dialog End tDlg, 1

Exit Function
ERRTRAP:
ErrType = Err

Result = LoadString(hInst, LoWrd(ErrType), zText,SizeOf(zText))
MsgBox "Error " + Trim$(Str$(ErrType)) + zText, %MB_ICONSTOP, Mine
End Function

------------------
Scott
mailto:[email protected][email protected]</A>

[This message has been edited by Scott Turchin (edited April 19, 2000).]

[Semen Matusovski]

Originally posted by Rich Brockway:
I ran both in the debugger.

Rich --
It's incorrect test.
Compiled and IDE VB program executes (for this sample) approx. the same time.
But not PB.
On my PC: PB (compiled) - 320 ms. In debugger - 28 seconds !!!

------------------

[Chris Boss]

Rich;

Likely when you run the program from within the PB Debugger, PB is creating a "temporary" version of the program compiled with "Extra" debugger code after "every single command" in your code. This extra over head makes it possible to track things that normally could not be tracked in a normally compile program.

This "extra" code is what is making the program so slow.

VB is uniquely different than most compilers, so their is little difference between the compiled version and running it in the VB IDE. VB isn't really a true compiler, since it uses large runtimes to do most things and little native code is really created when compiled. PB on the other hand compiles to native code and has very little runtime code (embeded in your EXE). PB "smokes" compared to VB, but obviously there is a downside to using the debugger.


------------------

[Guest]

Rich, in Vb's IDE code the running code is threaded 'P' code.
It sometimes outperforms VB's so-called "Native Code" exe's.

The PowerBASIC debugger is not going to give you any measure of its performance whatsoever.

Compile the PowerBasic code and then perform your benchmark agains the Vb code compiled in
both output code models ('P' code and 'native' code).

Havagud1,
Ron

[Rich Brockway]

Final verdict from compiled exes with my file:

VB 2.3
PB 2.7

Rich

------------------

[Florent Heyworth]

Have a look at Patrice Terrier's fast line input replacement
BUFIN.bas in the "Source code Forum":

measured on my 400 MHz 256 MB computer:

tested on a 110 MB file

PB:
-------------------------------------------------
15.46 seconds

compiled VB with all optimisations and RecLen=32767
--------------------------------------------------
34.42 seconds

What more can be said?

Cheers

Florent

------------------