I am facing a peculiar problem with QueueCollection.
PowerBasic Ver 10.04 running on i7/8GB RAM Windows 10
My application is a simulator that provides a TCP server to which around 1500+ TCP clients connect , send packets and receive responses and disconnect, at different times.
There are multiple threads handling the data.
One receiving thread corresponds to one activeTCP connection.
The receiving thread collects the incoming packet into a Datapacket variable and puts this data packet, along with Socket Handle , into a QUeue COllection
A separate processing thread takes out this data from the collection, generates a reply packet and sends it off to the corresponding socket
The program is very large, with lots of UI related code.
Packets arrive at a a very high rate - may be one packet in every 10 to 50 mS
Everything is working, but once in a while , say once in 100 to 200 packets ( randomly) , the data that is taken out of QueueCOllection is corrupted
Took lot of time to figure out that data corruption was happenning and after discarding corrupted packets , system started working fine and program response is also is good.
I have spent lots of time trying to figure out the reason for this data corruption, by selectively disabling parts of the code, but could not find the reason.
As a Final problem bypass, I replaced Queuecollection with StackCollection.
With no other code change whatsoever, Data corruption stopped !!!
If I put back Queuecollection in place of StackCollection, Corruption reappears.
So, for the time being, problem is solved and project is complete and I am not going to spend any more time on this,as the time spent has far exceeded the budget.
But , I am curious. Is there a bug in QueueCollection?
Wrote another program with two threads , one writing into Queuecollection and other reading from Queuecollection. No data corruption observed.
For information , Sequence of code shown below.
Only code snippets that are relevant to this problem given below:
PowerBasic Ver 10.04 running on i7/8GB RAM Windows 10
My application is a simulator that provides a TCP server to which around 1500+ TCP clients connect , send packets and receive responses and disconnect, at different times.
There are multiple threads handling the data.
One receiving thread corresponds to one activeTCP connection.
The receiving thread collects the incoming packet into a Datapacket variable and puts this data packet, along with Socket Handle , into a QUeue COllection
A separate processing thread takes out this data from the collection, generates a reply packet and sends it off to the corresponding socket
The program is very large, with lots of UI related code.
Packets arrive at a a very high rate - may be one packet in every 10 to 50 mS
Everything is working, but once in a while , say once in 100 to 200 packets ( randomly) , the data that is taken out of QueueCOllection is corrupted
Took lot of time to figure out that data corruption was happenning and after discarding corrupted packets , system started working fine and program response is also is good.
I have spent lots of time trying to figure out the reason for this data corruption, by selectively disabling parts of the code, but could not find the reason.
As a Final problem bypass, I replaced Queuecollection with StackCollection.
With no other code change whatsoever, Data corruption stopped !!!
If I put back Queuecollection in place of StackCollection, Corruption reappears.
So, for the time being, problem is solved and project is complete and I am not going to spend any more time on this,as the time spent has far exceeded the budget.
But , I am curious. Is there a bug in QueueCollection?
Wrote another program with two threads , one writing into Queuecollection and other reading from Queuecollection. No data corruption observed.
For information , Sequence of code shown below.
Only code snippets that are relevant to this problem given below:
Code:
. . . GLOBAL DataPacket1Queue AS IQUEUECOLLECTION GLOBAL g_CSDatapacket AS CRITICAL_SECTION GLOBAL DataPacket1Stack AS ISTACKCOLLECTION TYPE typInComingDataPacket lPacketSize AS LONG DataBytes(%DATAPACKET_DATABYTES_SIZE1+10) AS BYTE END TYPE . . [B]In Main Thread:[/B] LET DataPacket1Queue= CLASS "QueueCollection" DataPacket1Queue.Clear LET DataPacket1Stack= CLASS "StackCollection" DataPacket1Stack.Clear InitializeCriticalSection g_CSDatapacket ... . [B]In Receiving Thread - Data packet is put in the Data collection[/B] LOCAL IncomingPacket AS typIncomingdataPacket LOCAL s1 AS STRING . . . EnterCriticalSection g_CSDatapacket LOCAL vPacket AS VARIANT IF IncomingPacket.lPacketSize<>12 THEN IncomingPacket.lPacketSize=12 s1="Packet size reset:"+TRIM$(szPeerAddress) UserMessageMON(s1 ) 'send the message to MOnitor Process END IF LET vPacket= IncomingPacket AS STRING DataPacket1Queue.Enqueue(vPacket) 'DataPacket1Stack.Push(vPacket) ' This is for using Stackcollection in place of QueueCollection LeaveCriticalSection g_CSDatapacket . . . . [B]In Processing Thread, Data is taken out from the collection[/B] LOCAL v1 AS VARIANT LOCAL IncomingDataNew as typIncomingdataPacket LOCAL L1, lBytesRead AS LONG . . . EnterCriticalSection g_CSDatapacket v1=DataPacket1Queue.Dequeue 'v1=DataPacket1Stack.Pop ' This is for using Stackcollection in place of QueueCollection L1=OBJRESULT LeaveCriticalSection g_CSDatapacket IF L1=%S_FALSE OR VARIANTVT(v1)=%VT_EMPTY THEN ' error encountered while reading data from Que or there isno data in que lYield=1 ITERATE DO ' error. Ignore this and go back to beginning of the loop ELSE END IF IF VARIANTVT(v1)<>%VT_EMPTY THEN IncomingDataNew=VARIANT$(BYTE, v1) ' get teh bytes back lBytesRead=IncomingDataNew.lPacketSize END IF . . .
Comment