EOF detection causes prog hesitation

[Mike Trader]

I am looping through two files line by line.

On is a large file in memory (20MB) the other is a smaller file
being read line by line from disk using
INPUT# 100,

The code starts at the top of the large file and runs thru it
untill the date and time match the small file and then it steps
both files one line at a time untill the smaller file is exhausted.

then the next small file is processed and so on.

All is working, but every time the small file is exhausted the
program hits a brick wall for a second or two (as seen in the
progress bar). This is occuring when the EOF is detected

Why?

' open files etc etc
'
WHILE Done = 0 ' step thru the large file
    dLineStr = MID$(dFileStr, SPos+2, EPos-(SPos+2))
    dDateStr = PARSE$(dLineStr, 1)
    dDate    = DateToJulian(dDateStr)
    dTime    = TimeToMinutes(VAL(dTimeStr))
  
    WHILE (sDate < dDate OR (sDate = dDate AND sTime < dTime)) AND NOT EOF(100) AND Done = 0 ' advance the Small file to match
        IF sDate = dDate AND sTime = dTime THEN EXIT LOOP ' for first line only
        INPUT# 100, sDateStr, sTimeStr, OpenPrice, HighPrice, LowPrice, ClosePrice ' Input a Line
        sDate = DateToJulian(sDateStr)
        sTime = TimeToMinutes(VAL(sTimeStr))
    WEND
  
    IF sDate = dDate AND sTime = dTime THEN
        ' process the data
    END IF
  
    INCR LineNum : SPos = EPos : EPos = INSTR(SPos+2, dFileStr, CHR$(13)) : IF EPos = 0 THEN EXIT LOOP
    IF LineNum MOD 1000 = 0 THEN DIALOG SEND hDlgProgBar, %PROGBARID, ROUND(LineNum/LinesInFile*100 ,0), 0
WEND
' close small file
' loop for the rest of the small files

------------------
Kind Regards
Mike



[This message has been edited by Mike Trader (edited August 22, 2001).]

[Fred Buffington]

For what it's worth Mike, here's my thoughts.
Closing and reopening the next "small" file would cause a
delay though it shouldn't be 2 seconds or anywhere near that.
Another possibility though is an "input past end" on the
sequential file that is not being "reported".
Without more of the code it is difficult if not impossible to
give a definite answer to your slowdown question.

You might try putting the small files in random files and reading
them by record number or as binary files and see if there is
any difference.

also if you can open all the small files and read them by file
number on each loop rather than having to close and reopen
(which without the code is an assumption).

I don't see anything that sticks out as a cause of the slowdown.
Except it appears you exit the first while to close the small
file then reenter the big loop which seems inefficient but then
we are missing some code here (the processing stuff is probably
not needed --that is for determining an answer - but then ...)


------------------

[Lance Edmonds]

Mike, How have you verified that the EOF() function is the "problem"? I don't see any timing/benchmark code in the code snippet you posted.

A quick peruse of the code and your notes suggests quite a lot is going on in your app, along with items you have not posted code for, such as the GUI code for the progress bar, etc...

Basically, I'm not clear how you have verified the point where the delay is occurring.

Also, what O/S are you using, are the file(s) local or across a network, and what locking mode are you using in the OPEN statement? Also, are you specifying a buffer size in the OPEN statement (LEN=xxx)?



------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Mike Trader]

Lance.
You are right as allways. After MUCH testing the problem
turns out to have nothing to do with EOF as it originally appeared.

The problem is that I begin to do a:
STRDELETE$()
Followed by a
STRINSERT$()

in the middle of a 30MB file. The brick wall is windows shuffling
memory around.

I spent many hours on this and I was sure it was an EOF problem
so sorry

------------------
Kind Regards
Mike

[Lance Edmonds]

No problem at all Mike - that is what we are here for!



------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>