Threadin Safely (again)

[jeroen brouwers]

Hi, I've posted some questions on threading and started working on it.
I've used only one global variable to see the result of the thread and respond to it.
Each thread (2 to be exact) use SQL statement, so thread 1 uses statementnumber 1, thread 2 uses statementnumber 2. Seems save. Also, I've used the SQL_THREAD %THREAD_START, < 1 for thread1, 2 for thread2> with the corresponding closes on all exits and of course the SQL_initialise and SQL_THREAD ThreadMax, 2

For the better part, it works just fine. Yet, after a couple of runs I get a error from the Kernel32.dll!
I call my PBDLL from a VB app so normally when there's a problem with my DLL I get error saying that the error comes from the DLL, not the Kernel32.

Any ideas what might be wrong ?

I'm looking at the Critical Section idea but got a little lost (maybe someone can help)


Sincerely

Jeroen



------------------

[Lance Edmonds]

Your description certainly sounds like a thread collision on the global variable is occurring.

If you can beg, borrow or buy a copy of Rector/Newcomer's "Win32 Programming", then they have an excellent section on thread synchronization, and explanations of why synchronization is essential when dealing with static and global variables from separate threads.


------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[Guest]

I use sql tools with sql_threads and OS threads, Jeroen.
One thing I see missing (from my experience) is a database
connections for each statement if they are both likely
running during the same time span. I tried multiple
"simultaneous" statements on the same connection and could not do so.
Try a database connection for each statement.
I do use Critical Sections when more than one thread "share" a
database connection.


Good luck,

Ron


------------------
Ron

[This message has been edited by Ron Pierce (edited December 01, 2000).]

[jeroen brouwers]

Thanks Ron,
I'll try multiple database connections.
But can you give me some 'light' (or code ...) on the critical section cos this is still new to me.

Thanks
Jeroen

------------------

[Lance Edmonds]

take a look at some of the previous discussions:

http://www.powerbasic.com/support/pb...ead.php?t=1559

a search for "critical section" reveals a lot of others that may also be pertinent...

i hope this helps!

------------------
lance
powerbasic support
mailto:[email protected][email protected]</a>

[jeroen brouwers]

Hi, just an additional question...

I've read Rector and NewComer on Synchronization, implemented the Critical Section, started each of the two threads using THREAD CREATE to call a wrapper function for each thread.
The wrapper then enters the CS and starts the SQL_THREAD and calls the corresponding function.
These functions are basically two ways to search a result in a database. I've only used one global variable which is read only for the functions. The results of the functions is either 1 (success) or 2 (failure) for tread1, and 3 and 4 for thread 2.
When one of the threads returns a value (either success or failure) I wait for the other one.

My question is this:
In case one of the threads returns a 'success' value, I don't need to wait for the second thread to return: I already have what I was looking for.

Yet, if I close both the threads by means of THREAD CLOSE and then delete the critical section, I get some problems.
First of all, the thread is still running. That's normal but - as the PB help file says - the handle should be released.
Yet, the closed thread that was still busy, provides its feedback of its result later on, which in turn (of course) interferes with what I'm doing with the success result from the first thread.

How can it be that a thread who's handle is released, still provides result back to it 'parent'? And how can I really disconnect it?

What I did as a resolution, is to provide a counter value. For instance, Hit1 and Hit2 are the global variables for the corresponding threads. So if Hit1 return with a value of 1 (success) I make Hit2 equal to -1 and in Thread2 I put in a lot of IF HIT2 = -1 THEN EXIT FUNCTION. In the 'parent' function that calls the 2 threads, I wait for both values to return. yet, this is taking up a lot of time that I would like to get rid of.
If I don't wait for both threads to end and just close the threads and delete the CS, it will eventually provide a critical error in the kernel32, especially in stress-testing with a large number of sequential inputs.

So, is there a better / faster way?

Hope you can help
Jeroen

------------------

[Bob Houle]

Jeroen,

Jeffrey Richter's 4th edition of: Programming Applications
for Windows discusses threads, etc.

Read more at: http://mspress.microsoft.com/prod/books/2345.htm

--Bob

------------------

[jeroen brouwers]

OK, the reviews of the book show that it's basically the same as the readings of Rector & Newcomer...
So here's some code example, which maybe helps to explain.

The function GetResult1 starts the threads and handles the result (if any).
The final functions are both called from a wrapper to make it's saver / not to forget anything.

GetResult1 send a kind of 'countervalue' to stop a function (using the value -1) when the result of that function is not needed anymore because the other function already provided a result. Then - after both functions have ended - , I close the threads.
Basically, GetResult2 is what I would LIKE to do. Yet, it provides the problems as mentioned earlier.

So, is there an alternative to sending the countervalues? Is it possible to really 'kill' the threads (and still keep it working under stress-conditions)?


Sincerely,

Jeroen

#COMPILE EXE "C:\MT_TEST.EXE"
$INCLUDE "C:\SQLTools\Win32API.inc"
$INCLUDE "C:\SQLTools\SQL_PRO.INC"

GLOBAL INPUTSTRING AS STRING
GLOBAL OUTPUTSTRING AS STRING
GLOBAL RecNr AS DOUBLE              ' final result record ID
GLOBAL Rec1 AS DOUBLE               ' Thread1 record ID
GLOBAL Rec2 AS DOUBLE               ' Thread2 record ID
GLOBAL SQLHIT AS DOUBLE             ' Thread1 signal    1 (success) - 2 (failure)
GLOBAL SQLHIT1 AS DOUBLE            ' Thread2 signal    3 (success) - 4 (failure)
GLOBAL g_CRITICALSECTION AS CRITICAL_SECTION

DECLARE FUNCTION SQLParse_Slow(BYVAL x AS LONG) AS LONG
DECLARE FUNCTION SQLParse_Slow_Wrap(BYVAL x AS LONG) AS LONG
DECLARE FUNCTION SQLParse_Fast(BYVAL x AS LONG) AS LONG
DECLARE FUNCTION SQLParse_Fast_Wrap(BYVAL x AS LONG) AS LONG

FUNCTION PBMAIN() AS LONG

    ' Code for opening 2 DATABASE CONNECTIONS goes here...

    LOCAL i AS INTEGER, res&
    
    FOR i = 1 TO 50
        InputString = "test" + TRIM$(STR$(i) + " thread " + TRIM$(STR$(I * 5))

        res& = GetResults1
        
    NEXT i
    
    
END IF

FUNCTION GetResults1
    
    LOCAL SlowResult AS LONG
    LOCAL FastResult AS LONG
    LOCAL hSlowThread AS LONG
    LOCAL hFastThread AS LONG


    SQLHit = 0
    SQLHIT1 = 0

    ' start Critical Section
    InitializeCriticalSection g_CriticalSection

    ' Start Threads
    SLEEP 1
    THREAD CREATE SQLParse_Slow_Wrap(50) TO hSlowThread
    SLEEP 1
    THREAD CREATE SQLParse_Fast_Wrap(50) TO hFastThread


    DO WHILE SQLHit = 0 AND SQLHIT1 = 0
        SLEEP 1
    LOOP

    IF SQLHit = 1 THEN
        ' _Fast provides resultaat: slow can be closed: provide countervalue

        SQLHIT1 = -1

    ELSEIF SQLHit = 2 THEN
        ' fast done, no result: wait for slow
        DO WHILE SQLHit1 <> 3 AND SQLHIT1 <> 4
            SLEEP 1
        LOOP

        IF SQLHit1 = 3 THEN
            ' slow provides result
        ELSEIF SQLHit1 = 4 THEN
            ' slow also no result
        END IF

    ELSEIF SQLHit1 = 3 THEN
        ' slow done first + result: fast can be closed: provide countervalue for Fast
        SQLHIT = -1

    ELSEIF SQLHit1 = 4 THEN
        ' slow done first, no result: end
        SQLHIT = -1

    END IF

    ' Wait for both to return
    DO WHILE (SQLHIT <> 1 AND SQLHIT <> 2) OR (SQLHIT1 <> 3 AND SQLHIT1 <> 4)
        SLEEP 1
    LOOP

    ' Close threads
    SLEEP 1
    THREAD CLOSE hSlowThread TO SlowResult
    SLEEP 1
    THREAD CLOSE hFastThread TO FastResult
    SLEEP 1

    ' close critical section
    DeleteCriticalSection g_CriticalSection

    IF SQLHit = 1 THEN
        RecNr = Rec1
    ELSEIF SQLHit1 = 3 THEN
        RecNr = Rec2
    ELSE
        RecNr = 0
    END IF

    LOCAL result&, SQLST AS STRING
    IF recnr = 0 THEN
        SQLST = "Select OutputString From Testtable Where ID = '" + TRIM$(STR$(RecNr)) + "'"
        result& = SQL_Statement(1, 2, %IMMEDIATE, SQLST)
        IF (result& <> %SUCCESS AND result& <> %SUCCESS_WITH_INFO) OR SQL_RESULTCOLUMNCOUNT(1, 2) = 0 THEN
            SQL_CLOSESTateMenT 1, 2
            OUTPUTSTRING = "NO DATA WITH ID " + STR$(RecNr)
            EXIT FUNCTION

        ELSE
            SQL_Fetchresult 1, 2, %FIRST_ROW

            OUTPUTSTRING = TRIM$(SQL_RESultCOLumntext(1, 2, 1))
            OUTPUTSTRING = EXTRACT$(OUTPUTSTRING, "[ CHR$(0) ]")
            OUTPUTSTRING = EXTRACT$(OUTPUTSTRING, "[CHR$(0)]")
            OUTPUTSTRING = EXTRACT$(OUTPUTSTRING, CHR$(0))
            OUTPUTSTRING = EXTRACT$(OUTPUTSTRING, "[NULL]")
            OUTPUTSTRING = EXTRACT$(OUTPUTSTRING, "<NULL>")

            IF OUTPUTSTRING = "" THEN
                OUTPUTSTRING = "NO DATA WITH ID " + STR$(RecNr)
            END IF

            SQL_CLOSESTateMenT 1, 2
        END IF
    ELSE
        OUTPUTSTRING = "NO RESULT"
    END IF
    
    EXIT FUNCTION
END FUNCTION

FUNCTION GetResults2
    
    LOCAL SlowResult AS LONG
    LOCAL FastResult AS LONG
    LOCAL hSlowThread AS LONG
    LOCAL hFastThread AS LONG

    SQLHit = 0
    SQLHIT1 = 0

    ' start Critical Section
    InitializeCriticalSection g_CriticalSection

    ' Start Threads
    SLEEP 1
    THREAD CREATE SQLParse_Slow_Wrap(50) TO hSlowThread
    SLEEP 1
    THREAD CREATE SQLParse_Fast_Wrap(50) TO hFastThread


    DO WHILE SQLHit = 0 AND SQLHIT1 = 0
        SLEEP 1
    LOOP

    IF SQLHit = 1 THEN
        ' _Fast provides resultaat: slow can be closed: provide countervalue

        SQLHIT1 = -1

    ELSEIF SQLHit = 2 THEN
        ' fast done, no result: wait for slow
        DO WHILE SQLHit1 <> 3 AND SQLHIT1 <> 4
            SLEEP 1
        LOOP

        IF SQLHit1 = 3 THEN
            ' slow provides result
        ELSEIF SQLHit1 = 4 THEN
            ' slow also no result
        END IF

    ELSEIF SQLHit1 = 3 THEN
        ' slow done first + result: fast can be closed: provide countervalue for Fast
        SQLHIT = -1

    ELSEIF SQLHit1 = 4 THEN
        ' slow done first, no result: end
        SQLHIT = -1

    END IF

    ' NO WAITING
    
    ' Close threads
    SLEEP 1
    THREAD CLOSE hSlowThread TO SlowResult
    SLEEP 1
    THREAD CLOSE hFastThread TO FastResult
    SLEEP 1

    ' close critical section
    DeleteCriticalSection g_CriticalSection
    
    '... REST IS THE SAME
    
    EXIT FUNCTION
END FUNCTION


FUNCTION SQLParse_Slow_Wrap(BYVAL x AS LONG) AS LONG

    LOCAL result&

    EnterCriticalSection g_CriticalSection

    SQL_THREAD %THREAD_START, 1

    result& = SQLParse_Slow(x)

    IF SQLHIT1 = -1 THEN
        SQLHIT1 = 4
    END IF
    SQL_THREAD %THREAD_STOP, 1

    LeaveCriticalSection g_CriticalSection

    EXIT FUNCTION
END FUNCTION


FUNCTION SQLParse_Fast_Wrap(BYVAL x AS LONG) AS LONG

    LOCAL result&

    EnterCriticalSection g_CriticalSection

    SQL_THREAD %THREAD_START, 2

    result& = SQLParse_Fast(x)

    IF SQLHIT = -1 THEN
        SQLHIT = 2
    END IF
    SQL_THREAD %THREAD_STOP, 2

    LeaveCriticalSection g_CriticalSection

    EXIT FUNCTION
END FUNCTION

FUNCTION SQLParse_Fast(BYVAL x AS LONG) AS LONG

    ' Lots of SQL Action, parsing of the results
    ' Method is 'shortcut' -> fast

    ' For instance:
    LOCAL A AS INTEGER, Part2$, SQLST1 AS STRING, SQLTemp1 AS STRING

    FOR A = 0 TO PARSECOUNT(InputString, " ")
        Part2$ = PARSE$(InputString, " ", A)

        IF LEN(TRIM$(Part2$)) <> 0 THEN
            IF SQLST1 = "" THEN
                SQLST1 = "Select * from Index1 Where Field1 = '" + TRIM$(Part2$) + "'"
            ELSE
                SQLST1 = SQLST1 + " Or Field1 = '" + TRIM$(Part2$) + "'"
            END IF
        END IF

        IF SQLHIT = -1 THEN
            EXIT FUNCTION
        END IF

    NEXT A

    Result& = SQL_STATEMENT(2, 2, %IMMEDIATE, SQLST1)
    IF (result& <> %SUCCESS AND result& <> %SUCCESS_WITH_INFO) OR  SQL_ResultColumnCount(2, 2) = 0 THEN
        SQLHit = 2
        SQL_CLOSESTateMenT 2, 2
        EXIT FUNCTION
    END IF

    SQLST1 = ""
    SQL_FetchRESULT 2, 2, %FIRST_ROW
    DO WHILE NOT SQL_EndOfData(2, 2)
        FOR A = 1 TO SQL_ResultColumnCount(2, 2)
            SQLTemp1 = TRIM$(SQL_RESULTCOLUMNTEXT(2, 2, A))
            SQLTemp1 = EXTRACT$(SQLTemp1, "[ CHR$(0) ]")
            SQLTemp1 = EXTRACT$(SQLTemp1, "[CHR$(0)]")
            SQLTemp1 = EXTRACT$(SQLTemp1, CHR$(0))
            SQLTemp1 = TRIM$(SQLTemp1)

            IF SQLTemp1 <> "" AND SQLTemp1 <> "[NULL]" AND SQLTemp1 <> "0" THEN
                IF SQLST1 = "" THEN
                    SQLST1 = "SELECT * FROM Table1 WHERE Field1 = '" + SQLTemp1 + "'"
                ELSE
                    SQLST1 = SQLST1 + " OR Field1 = '" + SQLTemp1 + "'"
                END IF
            END IF

            IF SQLHIT = -1 THEN
                SQL_CLOSESTateMenT 2, 2
                EXIT FUNCTION
            END IF

        NEXT A

        SQL_FetchRESULT 2, 2, %NEXT_ROW

        IF SQLHIT = -1 THEN
            SQL_CLOSESTateMenT 2, 2
            EXIT FUNCTION
        END IF

    LOOP

    IF SQLHIT = -1 THEN
        SQL_CLOSESTateMenT 2, 2
        EXIT FUNCTION
    END IF

    ' etc
    ' If result found -> rec1 filled, SQLHit = 1

END FUNCTION


FUNCTION SQLParse_Slow(BYVAL x AS LONG) AS LONG

    ' Lots of SQL Action, parsing of the results
    ' Method is 'shortcut' -> slow

    ' For instance:
    LOCAL I AS INTEGER, Part1$, SQLST AS STRING, result1&

    SQLST = "Select Field1 from Table2"
    result1& = SQL_STATEMENT(1, 1, %IMMEDIATE, SQLST)
    IF (result& <> %SUCCESS AND result& <> %SUCCESS_WITH_INFO) OR  SQL_ResultColumnCount(1, 1) = 0 THEN
        SQLHit1 = 4
        SQL_CLOSESTateMenT 1, 1
        EXIT FUNCTION
    END IF


    DIM ResultArr() AS STRING
    
    SQL_FetchRESULT 1, 1, %FIRST_ROW
    DO WHILE NOT SQL_EndOfData(1, 1)
        FOR I = 1 TO SQL_ResultColumnCount(1, 1)
            SQLST = TRIM$(SQL_RESULTCOLUMNTEXT(1, 1, I))
            SQLST = EXTRACT$(SQLST, "[ CHR$(0) ]")
            SQLST = EXTRACT$(SQLST, "[CHR$(0)]")
            SQLST = EXTRACT$(SQLST, CHR$(0))

            IF SQLST <> "" AND SQLST <> "[NULL]" AND SQLST <> "0" THEN
                result1& = 0
                ' parse result
                    '...
                IF result1& = 1 THEN
                    REDIM ResultArr(UBOUND(ResultArr) + 1)
                    ResultArr(UBOUND(ResultArr)) = SQLST
                END IF
            END IF

            IF SQLHIT1 = -1 THEN
                SQL_CLOSESTateMenT 1, 1
                EXIT FUNCTION
            END IF

        NEXT a

        IF SQLHIT1 = -1 THEN
            SQL_CLOSESTateMenT 1, 1
            EXIT FUNCTION
        END IF

        SQL_FetchRESULT 1, 1, %NEXT_ROW
    LOOP

    SQLST = ""
    FOR i = 0 TO UBOUND(ResultArr)
        IF i = 0 THEN
            SQLST = "Select * from Table1 Where Field1 = '" + TRIM$(ResultArr(i)) + "'"
        ELSE
            SQLST = SQLST + " OR Field1 = '" + TRIM$(ResultArr(i)) + "'"
        END IF
    NEXT i

    result1& = SQL_STATEMENT(1, 1, %IMMEDIATE, SQLST)
    IF (result& <> %SUCCESS AND result& <> %SUCCESS_WITH_INFO) OR  SQL_ResultColumnCount(1, 1) = 0 THEN
        SQLHit1 = 4
        SQL_CLOSESTateMenT 1, 1
        EXIT FUNCTION
    END IF

    SQLST = ""
    LOCAL SQLTemp AS STRING
    
    SQL_FetchRESULT 1, 1, %FIRST_ROW
    DO WHILE NOT SQL_EndOfData(1, 1)
        FOR i = 1 TO SQL_ResultColumnCount(1, 1)
            SQLTemp = TRIM$(SQL_RESULTCOLUMNTEXT(1, 1, i))
            SQLTemp = EXTRACT$(SQLTemp, "[ CHR$(0) ]")
            SQLTemp = EXTRACT$(SQLTemp, "[CHR$(0)]")
            SQLTemp = EXTRACT$(SQLTemp, CHR$(0))

            IF SQLTemp <> "" AND SQLTemp <> "[NULL]" AND SQLTemp <> "0" THEN
                IF SQLST = "" THEN
                    SQLST = "SELECT * FROM Table1 WHERE Field1 = '" + SQLTEMP + "'"
                    SQLST = "SELECT * FROM Table1 WHERE Field1 = '" + SQLTEMP + "'"
                ELSE
                    SQLST = SQLST + " OR Field1 = '" + SQLTEMP + "'"
                END IF
            END IF

            IF SQLHIT1 = -1 THEN
                SQL_CLOSESTateMenT 1, 1
                EXIT FUNCTION
            END IF

        NEXT A

        IF SQLHIT1 = -1 THEN
            SQL_CLOSESTateMenT 1, 1
            EXIT FUNCTION
        END IF

        SQL_FetchRESULT 1, 1, %NEXT_ROW
    LOOP


    IF SQLHIT1 = -1 THEN
        SQL_CLOSESTateMenT 1, 1
        EXIT FUNCTION
    END IF

    ' etc

    ' If result found -> rec2 filled, SQLHit1 = 3

END FUNCTION

------------------

[Lance Edmonds]

Although I have not tested your code, a casual glance indicates there are one or two problems with your code...

1. Although you are not calling GetResults1() and GetResults2() as threads, you are defining one (global) critical section structure, but initializing in two separate places! This will work as long as these two functions are never executed at the same time. If they do, then you will be interleaving two critical sections in an unpredictable manner, and this will guarantee a GPF. The "correct" way is to create a seperate critical section for each "group" of threads, so in your code above, GetResults1() and GetResults2() should probably have their "own" critical sections. Since a CS needs to be defined as a GLOBAL, you would need to define two global CS structures.

2. You are testing for the results of the threads by examining a global variable - in multi-threaded app's, you are likely to hit a point where the variable contains an "undefined" value due to a thread context switch (per Rector/Newcomer). This is exactly what a critical section is designed to help avoid, so ideally, you should be setting and testing these global variables within yet another CS.

Something like this:

GLOBAL g_CSResultVar AS CRITICAL_SECTION
...
InitializeCriticalSection g_SCResultVar
call GetResults1()
...
'in the primary thread where you query the variable's value
EnterCriticalSection g_CSResultVar
IF SQLHIT = <some value> OR SQLHIT1 = <some value> THEN  <do something>
LeaveCriticalSection gCSResultVar
...
'placed in the threads where we can set the variable's value:
EnterCriticalSection h_CSResultVar
SQLHIT = <some value>
LeaveCriticalSection h_CSresultVar

However, a much easier way is to use the thread return values instead... much easier!

Food for thought...

------------------
Lance
PowerBASIC Support
mailto:[email protected][email protected]</A>

[jeroen brouwers]

Thanks Lance, some food for through alright... I'll read up some more.

Yet:
It's true that I don't start GetResults1 as a thread: it's only the function to create the CS and start the threads; the wrapper functions are the threads that enter the CS and call the functions that really do the work.
It's also true that I would need 2 CS's but I don't start both GetResults1 and GetResults2.

GetResults2 is only to show what I would like: close both thread when I have a result from one of them. Yet, this provides errors because when thread1 (e.g. _Fast) provides a result, thread2 (e.g. _Slow) is still running. When I just close them both and delete the CS (like in GetResult2) I get a critical error in the end.
So my solution was to provide the countervalue for the thread that is still running in order to make it exit (therefore the code lines: if SQLHit = 1 Then exit function).
This works alright because before I close both threads and delete the CS, both threads are actually finished. But this means I have to wait for the running thread to end when I already have a result.

So my question still is: how can I do this faster? Is there a way to kill the running thread directly?

Hope you can help a little more.
Sincerely
Jeroen

------------------

[Eric Pearson]

Jeroen --

I have been reviewing some of what you have written about this (here and elsewhere) and it occurs to me that terminating the thread would not be a good idea.

If you find a way to terminate the second thread, the ODBC driver will not realize that anything is wrong, and it will continue working on the query. This will probably result in an "abandoned connection", because the driver will wait forever for your program to begin using SQL_Fetch, and that will never happen. If your program does that too many times, the driver will run out of available connections and you will have to reboot the computer (or possibly the server) before you can use the database again. This problem is covered in the SQL Tools Help File.

When you tell the ODBC driver to begin a query in a thread, you should either let it finish normally, or explicitly tell it to quit by using the SQL_StatementCancel function.

In your case, if Thread 1 finishes its job and wants Thread 2 to terminate, Thread 1 would use the SQL_StatementCancel function with the statement number that was used by Thread 2. (It is unusual for two different threads to operate on the same statement in this way, but that is exactly what SQL_StatementCancel was designed to do.)

Because of the SQL_StatementCancel command from Thread 1, Thread 2's SQL_Statement function would then exit with no results, and Thread 2 could be allowed to terminate normally.

Sorry this didn't occur to me before...

-- Eric


------------------
Perfect Sync: Perfect Sync Development Tools
Email: mailto:[email protected][email protected]</A>

[jeroen brouwers]

Thanks Eric,

I understand what you're saying, I'm going to implement it right now.
Could this be the reason that I got a critical error when just closing the threads and deleting the CS ?
Or is it still not a good idea to close a thread when it is still running (even when doing SQL_StatementCancel)?

Anyway, I tried it already and the average response has improved by 0.3 seconds without any crashes on 150 sequential inputs. GREAT!
So I guess it is save... No?

Regards
jeroen

------------------


[This message has been edited by jeroen brouwers (edited December 07, 2000).]

[Eric Pearson]

> Or is it still not a good idea to close
> a thread when it is still running

It's perfectly fine to CLOSE a thread -- that has no effect on the thread itself -- but it's not ok to terminate one.

> So I guess it is save... No?

That's up to you. It's working properly now, right?

-- Eric

------------------
Perfect Sync: Perfect Sync Development Tools
Email: mailto:[email protected][email protected]</A>

[jeroen brouwers]

Thanks for the tips. I'm learning... slowly. Anyway, it DOES work alright so I'll end with a thank you.

Sincerly
jeroen

------------------