How to exchange data between processes?

[Gert Voland]

Hi,
I would like to exchange a small amount of data (about 16 variables) between two PB programs (.exe) which are running at the same time.
There should be a minimum of delay why a disk file is probably out of question.

Is the registry an appropriate means for doing that?

Who has experience with generating keys, writing and reading values and deleting them again? What has to be observed? And what are the pitfalls?

Maybe someone has some sample code, or are there better/simpler alternatives?

Thanks in advance,
Gert Voland.

[Gösta H. Lovgren-2]

Hi,
I would like to exchange a small amount of data (about 16 variables) between two PB programs (.exe) which are running at the same time.
There should be a minimum of delay why a disk file is probably out of question.

Gee, in today's world with disk caches, accessing a disk file is almost as fast as using in memory arrays. Probably as fast (and safer) than using the registry, especially if the 16 variables are in a UDT.

===============================================
Seriousness is the only refuge of the shallow.
Oscar Wilde
===============================================

[John Gleason]

The below two exe files show that writing to a file is probably not too slow and is a relatively simple option. It writes 16 EXT array variables to a file 512 times and measures the time taken per write. It is about 1,800,000 tix, well under a millisecond on most machines. Run simultaneously, each will access the same file when the other isn't.

'[B]shareDataBetweenExe1.exe[/B]
#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

    DIM ex(15) AS EXT
    LOCAL cnt, ii, ii2 AS LONG, t AS QUAD

   TIX t
   FOR ii2 = 1 TO 1550
    FOR ii = 0 TO 15
       ex(ii) = ii * 1234567890.12345 + ex(ii)
    NEXT

    INCR cnt

    IF (cnt AND 511) <> 0 THEN
       DO
          TRY
             OPEN "c:\variableExchangeFile.dat" FOR BINARY AS #1
             PUT #1,, ex()
             CLOSE #1
             EXIT DO
          CATCH
             SLEEP 0
          END TRY
       LOOP
    ELSE
       DO
          TIX END t
          TRY
             OPEN "c:\variableExchangeFile.dat" FOR BINARY AS #1
             GET #1,, ex()
             CLOSE #1
             ? STR$(t \ 512) & " tix per exchange of 16 EXT variables (exe #1)"
             TIX t
             EXIT DO
          CATCH
             SLEEP 0
          END TRY
       LOOP
   END IF
   NEXT

   ? "Done"

END FUNCTION

'[B]shareDataBetweenExe2.exe[/B]
#COMPILE EXE
#DIM ALL

FUNCTION PBMAIN () AS LONG

    DIM ex(15) AS EXT
    LOCAL cnt, ii, ii2 AS LONG, t AS QUAD

   TIX t
   FOR ii2 = 1 TO 1550
    FOR ii = 0 TO 15
       ex(ii) = ii * 2345678901.2345 + ex(ii)
    NEXT

    INCR cnt

    IF (cnt AND 511) <> 0 THEN
       DO
          TRY
             OPEN "c:\variableExchangeFile.dat" FOR BINARY AS #1
             PUT #1,, ex()
             CLOSE #1
             EXIT DO
          CATCH
             SLEEP 0
          END TRY
       LOOP
    ELSE
       DO
          TIX END t
          TRY
             OPEN "c:\variableExchangeFile.dat" FOR BINARY AS #1
             GET #1,, ex()
             CLOSE #1
             ? STR$(t \ 512) & " tix per exchange of 16 EXT variables (exe #2)"
             TIX t
             EXIT DO
          CATCH
             SLEEP 0
          END TRY
       LOOP
   END IF
   NEXT

   ? "Done"

END FUNCTION

[Eddy Van Esch]

Gert,

I would suggest to use the %WM_COPYDATA message to exchange data between 2 or more exe's (of which you can modify the source code).

See nice example code by Pierre Bellisle here:
WM_CopyData example

Here are some more options:
Sharing data between 2 exe's

Kind regards

[Edwin Knoppert]

(Which is actually based on a file as well )

[Michael Mattias]

Win32: Memory Mapped Files/Control User Count April 26, 2001

Complete with mutex-based memory locking.

[Gert Voland]

Thank you for the helpful tips so far.

- Exchange through files is probably the easiest in terms of using only PB/CC commands. My worry would be the cache being overwritten, and thus causing unwanted delays.
- The registry method seems to have a few difficult formatting cliffs.
- I would like to use PB/CC without WM_ messages .
- At the moment, the most promising way seems a memory mapped dataset (samples by Peter Lameijn or Michael Mattias).

In my case there are only two processes accessing the dataset (for a kind of full duplex communication). If I don't use the full mutex method, CreateFileMapping and MapViewFile could be enough to set up the dataset.

Does somebody know, if two SetEvent calls to the same (common) dataset could then possibly interfere with each other - or does one lock/block access for the time of writing?

Gert Voland.

[John Petty]

Mutexes are realy quite simple so worth the use. Memory mapped Files were basically included in Windows for exactly this purpose as there is no time delay or caching problems particularly if the amount of data to be transferred is not large as if correctly specified the data is never written to disk, it merely maps a section of physical memory into the virtual address space of both programs so allowing them both memory access. MCM's example is quite good so should be a good starting point.

[Mel Bishop]

The clipboard may be a possibility. Write to/Read from...

[Michael Mattias]

>The clipboard may be a possibility. Write to/Read from...

Not really. What if some other process changes the clipboard text between the time one process changes it and the other process gets around to checking it? Kind of selfish to think no other pair of processes is using the clipboard to exchange data, when that's exactly what you are doing, no?

FWIW, since your applications are cooperating, you could set up a named Windows event for the applications to signal each other, "Hey, something has changed, deal with it!" This would also eliminate any unnecessary checking (or worse, polling!) .... if you know it hasn't changed, just use the same values you had before!

MCM

[AFTERTHOUGHT]
Well, I guess the clipboard would be OK, as long as the cooperating applications use a private registered clipboard format.
[/AFTERTHOUGHT]

[Mike Doty]

Private clipboard.

http://www.powerbasic.com/support/pbforums/showthread.php?t=23335

[Michael Mattias]

Does somebody know, if two SetEvent calls to the same (common) dataset could then possibly interfere with each other - or does one lock/block access for the time of writing?

Huh?

You don't call SetEvent against a dataset... you call SetEvent against an event.

A call to SetEvent() puts the event into a signalled state until one WaitForxxxxx is satisified, at which time it either remains signalled or is reset depending on the options used when you created the event.

The mutex locking in the demo prevents more than one thread from obtaining ownership of the mutex object at one time ... and the way the code is written, if you do not obtain ownership of the mutex, you can't access the UDT... if the mutex is currently owned by another thread, the calling thread waits until it is available.

[Erich Schulman]

Hi,
I would like to exchange a small amount of data (about 16 variables) between two PB programs (.exe) which are running at the same time.
There should be a minimum of delay why a disk file is probably out of question.
Gert Voland.

Don't forget about the RAMdisk.

http://www.mydigitallife.info/2007/05/27/free-ramdisk-for-windows-vista-xp-2000-and-2003-server/

[Edwin Knoppert]

The clipboard may be a possibility. Write to/Read from...

It is actually a good idea, DDE is based on that.
You can also create a custom clipboardformat.

As simple notification could a message broadcast but you didn't want to use windows.
If you did the WM_COPYDATA would be the easiest.
So this scenario fails.

[Gert Voland]

You don't call SetEvent against a dataset... you call SetEvent against an event.

Yes, the question was not exact.

I will use a common data set (UDT) mapped in both programs.
Fortunately the data going from A --> B and from B --> A are not overlapping - which makes things quite easy.
In order to prevent interference of the events I will use two event objects, one created in A and checked in B and vice versa.
That way I get a full duplex connection at very little CPU load.

Gert.

[William Baugh]

I think Michael hit on this but I was going to suggest using shared memory:

https://www.powerbasic.com/support/pbforums/showthread.php?t=22584

[John Gleason]

Lotta good ideas here, and it looks like you're set Gert, but in case you're interested, here is a shorter and much faster improvement on my earlier post. As a bonus, it uses virtually just idle processor time. If you run one .exe, it will wait until you run a second instance, then will complete by shuttling the UDT back and forth 10,000 times, and finally show the total times taken. The shorter time of the two is the keeper.

'shareDataBetweenExe04.exe; PB 9.01
#COMPILE EXE
#DIM ALL

TYPE sixteenVars
   e0 AS EXT 'identifier we will reserve
   e1 AS EXT
   e2 AS EXT
   e3 AS EXT
   e4 AS EXT
   e5 AS EXT
   e6 AS EXT
   e7 AS EXT
   e8 AS EXT
   e9 AS EXT
   e10 AS EXT
   e11 AS EXT
   e12 AS EXT
   e13 AS EXT
   e14 AS EXT
   e15 AS EXT
   e16 AS EXT
END TYPE

FUNCTION PBMAIN () AS LONG

    LOCAL ii AS LONG, t AS QUAD, identifier AS EXT
    LOCAL sharType AS sixteenVars               'room for identifier and 16 EXT member elements

    TIX t                                       'get quad identifier guaranteed unique. (TIMER usually okay too unless exe's run < 18/th sec. apart)
    identifier = t                              'give this exe the unique identifier
    KILL "c:\shareDataFile0x1.dat"              'clear previous test runs

   TIX t                                        'speed tester
    sharType.e0 = identifier                    'put unique identifier in e0
    OPEN "c:\shareDataFile0x1.dat" FOR BINARY LOCK SHARED AS #1 'be sure to make SHARED
    PUT #1,, sharType                           'initialize. write opening string

   FOR ii = 1 TO 10000
  '--------Here is the actual sharing code...only 13 lines----------------------------------------------
  DO
       SEEK #1, 1                               'seek to beginning
       GET #1,, sharType                        'read in whole UDT
       IF sharType.e0 = identifier THEN         'if it's from this exe, it's not our turn yet, so iterate after short sleep
          SLEEP 0
          ITERATE DO
       ELSE
          sharType.e0 = identifier              'otherwise, our turn to write so set e0 member to our identifier
       END IF
       SEEK #1, 1                               'seek to beginning
       PUT #1,, sharType                        'and write it
       EXIT DO
  LOOP
  '---------End of the actual sharing code...only 13 lines----------------------------------------------
   NEXT

    CLOSE #1
    KILL "c:\shareDataFile0x1.dat"              'erase temp file
    TIX END t
   ? "Done" & STR$(t \ 10000)                   '112,000 tix per each share of UDT on my machine.
                                                'That's just 7000 tix per element shared.
END FUNCTION

[Michael Mattias]

but in case you're interested, here is a shorter and much faster improvement on my earlier post.

That code does not prevent simultaneous access... you never lock the disk file record.

Here is an article I wrote a long time ago about using files to share data across processes. The code is PB/DOS, but it's not really the code which is important, it's the concepts.

"Fundamentals Of Multi-User Programming." Article published in December 1995 issue of "BASICally Speaking" magazine discussing the principles of writing multi-user programs; code samples in BASIC. Rich Text format; placed in the Public Domain June 2005

http://www.talsystems.com/tsihome_html/downloads/multiuser_programming.rtf

MCM

[Knuth Konrad]

If possible, stay away from using files as a means to (real time) data exchange between running programs. Too much error conditions that can happen.

(I had to support an environment that used this technique for a while ... bad memories)

[Michael Mattias]

If possible, stay away from using files as a means to (real time) data exchange between running programs

Well, that makes two opinions.

IMO that used to be true, but today's disks are so fast and reliable that I think it no longer an issue.

I think probably you could make the same argument against using a Memory-Mapped File object: too many parity errors with those darned memory chips. Also passe.

FWIW, here's a demo (one process) which uses disk as intended: "secondary memory." While I concocted this in response to a specific challenge, it's the way I would do it in Real Life if I had to...(knowing what I know now, that is).....

Count unique keys and occurrences thereof in sequential file




MCM