Sharing Data Files Over a Network

[Anthony Watson]

I would like to rewrite some of my old programs to work on a home network. Basically have a shared data file in one location, such as NAS, that the same program on two different computers could access. Unfortunately, network programming is relatively new to me, and I'm not sure the best way to keep everything synced and avoid corruption.

I could simply lock the file when open and prevent the other program from accessing it till I'm done. At least I guess that would still work if the data was stored on a NAS or something similar? But in some of my programs that wouldn't really work very well as the data needs to read and write in real time, and locking the file could cause issues. Though I could probably work around that if needed.

Also, if one computer adds data to the file, how do I know the file has been updated to reload that data on the other computer? I suppose I could check the file periodically for changes, but that seems sloppy somehow. Is there a way for my app to be notified if the file has changed on an external network drive?

I also wonder what happens if I edit the same record on my computer as another person is editing it on the other computer. Who takes priority, and what happens with the other edit?

Any tips would be appreciated.

Thanks!

[Dale Yarker]

Have only one program that accesses the file. All users have copies of a different program that requests reads and writes through the first program. The first program completely locks the file for itself and does record locking by user. If a user has a record locked too long to one user (with no activity) it switches locking the record to next waiting user.

Reading locked records could be allowed, but what if two users might make (different?) changes based on old contents?

(One opinion.)

Cheers,

[Michael Mattias]

"Fundamentals Of Multi-User Programming." Article published in December 1995 issue of "BASICally Speaking" magazine discussing the principles of writing multi-user programs; code samples in BASIC. Rich Text format; placed in the Public Domain June 2005
http://www.talsystems.com/tsihome_ht...rogramming.rtf


You're welcome.

[Frank Rogers]

When a program is just scrolling around and looking at the data file all data access is in the shared mode. It is only when you want to make changes to the data will things get ugly. There are two schools of thought.

One, lock the file, do the editing and then unlock the file.

Two, do the edit, lock the file, read and compare the data, write the changes and unlock.

The first is the easiest to program but if your users are not diligent about doing edits quickly, performance will come to a standstill.
The second is more work but it keeps the network running because you are only locked up while comparing and writing updates. The second requires you to never ever lock the file, ask the user a question and then unlock.

MCM's article is a good one, for some reason the link was not working. Maybe he will fix.

[Mike Doty]

This example uses file locking so all transactions are locked out to other users making things extremely simple. It is also thread safe.

As far as if somebody is in a record you have modified:
Every station gets there current record into a string. This station is in record 3
recnum = 3
sBefore = GetIt(recnum)
every so often
If sBefore <> GetIt(recnum) then notify user:update screen:sBefore = GetIt(recnum)


The below example from the help file doesn't open the file in shared mode nor does it do any error checking!!!
If the file is opened in exclusive mode like the below example, the file would be locked so locking of a record is not needed.
If you don't want somebody to be able to change a record once they have it then lock the record when you get it and unlock when done.

OPEN "PATIENTS.DAT" FOR RANDOM AS #1 LEN = 1024 ' determine the record number to retrieve LOCK #1, recnum GET #1, recnum ' process the record here PUT #1, recnum UNLOCK #1, recnum CLOSE #1

%killfile = 0           'kill $file at startup
%threads  = 3           'thread for each user
$file     = "junk.dat"  'file to open exclusive

TYPE RecType
 recnum AS STRING * 6
 User   AS STRING * 4
 Time   AS STRING * 10
 crlf   AS STRING * 2
END TYPE

FUNCTION GetIt(recnum AS LONG) THREADSAFE AS STRING
 LOCAL rec AS RecType
 LOCAL hFile,attempt,reclen AS LONG
 reclen = LEN(rec)
 hFile = FREEFILE
 FOR attempt = 1 TO 100
  ERRCLEAR
  OPEN $file FOR RANDOM AS #hFile LEN=reclen 'open exclusive
   IF ERR = 0 THEN EXIT FOR
  SLEEP 100
 NEXT
 IF ERR THEN ? "Could not open file":EXIT FUNCTION
 GET #hFile,recnum,rec
 CLOSE #hFile
 FUNCTION = rec
END FUNCTION

FUNCTION PutIt(usernum AS LONG) THREADSAFE AS LONG
 LOCAL rec AS RecType
 LOCAL hFile,attempt,reclen,recnum AS LONG
 reclen = LEN(rec)
 hFile = FREEFILE
 FOR attempt = 1 TO 100
  ERRCLEAR
  OPEN $file FOR RANDOM AS #hFile LEN=reclen 'open exclusive
   IF ERR = 0 THEN EXIT FOR
  SLEEP 100
 NEXT
 IF ERR THEN ? "Could not open file":EXIT FUNCTION
 recnum = LOF(hfile)\reclen + 1
 rec.recnum = FORMAT$(recnum)
 rec.user   = FORMAT$(usernum)
 rec.time   = TIME$
 rec.crlf   = $CRLF
 PUT #hFile,recnum,rec
 CLOSE #hFile
END FUNCTION

THREAD FUNCTION MyThread(BYVAL UserNum AS LONG) AS LONG
 PutIt UserNum
END FUNCTION

FUNCTION PBMAIN AS LONG
 IF %killfile THEN KILL $file:ERRCLEAR
 LOCAL threadnum,hthread,recnum AS LONG, s AS STRING
 FOR threadnum = 1 TO %Threads
  THREAD CREATE MyThread(threadnum) TO hThread
  THREAD CLOSE hThread TO hThread
 NEXT
 DO:SLEEP 100:LOOP UNTIL THREADCOUNT = 1
 SHELL "notepad.exe "+ $file
END FUNCTION

[Michael Mattias]

MCM's article is a good one, for some reason the link was not working. Maybe he will fix.

I contacted my web host because I could not open it either, although it shows up on a directory list. I can get everything else in that directory just fine.

I have it in other formats (kind of) but I converted to RTF specifically to put on the web site and it's been trouble free since approximately forever.

I'll drop a note to Mr. Drake about putting a ZIP file out on the PB site for everyone to use . It may include source code but the article is the really important part so I don't think it belongs in the Source Code Forum.


MCM

[Michael Mattias]

One, lock the file, do the editing and then unlock the file.

Two, do the edit, lock the file, read and compare the data, write the changes and unlock.

In these, you can lock the record; you don't need to lock the whole file (which may be impossible anyway). .

These use physical locks ; but you can also do a logical lock. This is covered in the article, too.

[Stuart McLachlan]

I contacted my web host because I could not open it either, although it shows up on a directory list. I can get everything else in that directory just fine.

I have it in other formats (kind of) but I converted to RTF specifically to put on the web site and it's been trouble free since approximately forever.

Firefox declined to download it because it was RTF,
But once I told it to download it anyway, I got it.

[Mike Doty]

Torture test writing to record number 1 by multiple threads leaving file open, but using flush after each unlock.

#INCLUDE "win32api.inc"
GLOBAL gRetries AS LONG 'not needed,  just curious number of retries
%Stations       = 2    'simulate number of work stations, 1-thread per station
%RecordsToWrite = 1000 'number of writes per station to record number 1

TYPE RecType
 counter AS STRING * 6
END TYPE

FUNCTION Lockit(hFile AS LONG,recnum AS LONG) AS LONG
 LOCAL attempt AS LONG
 FOR Attempt = 1 TO 1000
  ERRCLEAR
  LOCK #hfile,recnum
  IF ERR THEN gRetries = interlockedincrement(gRetries) 'threadsafe count number of retries
  IF ERR = 0 THEN EXIT FOR
  SLEEP 1
 NEXT
 FUNCTION = ERR
END FUNCTION

THREAD FUNCTION MyStation(BYVAL dummy AS LONG) AS LONG
 LOCAL rec AS RecType
 LOCAL x,hfile,recnum,reclen AS LONG

 reclen = LEN(rec)
 hfile = FREEFILE
 OPEN "junk.dat" FOR RANDOM SHARED AS #hFile LEN = reclen
 IF ERR THEN ? "open error" + ERROR$:EXIT FUNCTION

 FOR x = 1 TO %RecordsToWrite
  recnum = 1
  IF LockIt(hFile,recnum) THEN
   ? USING$("lock handle # record #",hfile,recnum)
   CLOSE #hfile
   EXIT FUNCTION
  END IF

  'make changes
  GET #hFile,recnum,rec
  rec.counter = FORMAT$(VAL(rec.counter)+1)

  PUT #hfile,recnum,rec

  UNLOCK #hfile,recnum
  FLUSH #hfile
 NEXT

END FUNCTION

FUNCTION PBMAIN AS LONG
 LOCAL rec AS RecType
 LOCAL Station,hThread AS LONG
 KILL "junk.dat" 'start fresh
 OPEN "junk.dat" FOR RANDOM SHARED AS #1 LEN = LEN(rec)
 FOR Station = 1 TO %Stations
  THREAD CREATE MyStation(station) TO hThread
  THREAD CLOSE hThread TO hThread
 NEXT
 DO:SLEEP 50:LOOP UNTIL THREADCOUNT = 1

 GET #1,1,rec
 ? FORMAT$(VAL(rec.counter),"#,"),,"Retrys"+STR$(gRetries)
 CLOSE #1
END FUNCTION

[Michael Mattias]

Firefox declined to download it because it was RTF,
But once I told it to download it anyway, I got it.

That's what happened to me. Instead of opening the file, it immediately asked me to save it instead. And... it saved it real nice. .

I will drop a note to my web provider, "Never Mind" or maybe I will thank them for their prompt action.

(Was worth the wait, huh?)

[Stuart McLachlan]

That's what happened to me. Instead of opening the file, it immediately asked me to save it instead. And... it saved it real nice. .

I will drop a note to my web provider, "Never Mind" or maybe I will thank them for their prompt action.

(Was worth the wait, huh?)

It may be worth converting any RTFs you want to share to PDF to avoid this issue.

(And there are six .exe files in that directory. It would be better to remove them after creating .zips for those that don't currently have them )

[Frank Rogers]

Looks like Edge is fine with .rtf files but chrome just ignores/refuses it. I guess a chrome update broke it.

[Stuart McLachlan]

Looks like Edge is fine with .rtf files but chrome just ignores/refuses it. I guess a chrome update broke it.

Maybe make that "a chrome update tightened security" ?

[Frank Rogers]

I'm still considering if Michael is "dangerous". Security Risk could be appropriate.

[Rod Macia]

Regarding the RTF from Michael is because it comes from a HTTP site (unencrypted) not a HTTPS site.
files downloaded from HTTP are potentially prone to an attach from "a man in the middle" of altering or changing the actual file by something else capable of comprising your system. So browsers are becoming more and more security centric.

Making them ZIP or PDF will not solve the problem.

MCM, Enable HTTPS hosting to solve.

others accept the risk, to download if offered at your own risk!

[Michael Mattias]

Mr. Drake suggested I just attach a file to a post in a "regular" forum. So I'll do that. I see this is an "https" site.

But not this AM as I am getting ready to leave for the Brewers vs. Dodgers game (Kershaw v Miley) with a group from the Port Washington Senior Center.