TCP - Getting extra FD_READ notifications

[garry star]

I am getting more FD_READ notifications than I should.

When my server sends me a message (approx 60k in one SEND), I receive an FD_READ and go into my RECV loop until I've received all the data using an 8k buffer. The message is received correctly.

However, I also sometimes get additional (1 to 3) FD_READ notifications with EOF true (no data) immediately after I've read my message. Where are these coming from? Am I doing something to cause them?

Thanks

Garry

[Michael Mattias]

Pure SDK syntax? PB-proprietary 'TCP' statements/functions? One from column A, one from column B?

[garry star]

I'm using all PB-proprietary functions
TCP OPEN
TCP NOTIFY
TCP RECV

[garry star]

Here's my sequence from the point of view of the client (who's receiving the extra FD_READs)

1. Connect to server
2. Send signon message (no response from server)
3. Send subscriptions request (no response from server)
4. some time later...
5. Receive message from server

Reading the message from the server is often followed by additional FD_READs with an empty buffer - If Eof(readFnum) returns true

Is it possible that the previous messages I sent to the server somehow get ACK'd with an empy buffer from the lower TCP layers (I am not doing any response in my server) ?

[Gösta H. Lovgren-2]

I am getting more FD_READ notifications than I should.
...
Garry

What is FD_READ? I can't find it either in PB Help or in WinApi?

Also, Garry, you'll get a lot more responsive/useful help here if you post code. Executable example preferred, or at least a longish snippet of the actual offending code.

================================
And when he is out of sight,
quickly also is he out of mind.
Thomas a Kempis
================================

[garry star]

Some code

After opening the port, I do a notify:

Tcp Notify gDsFnum, Recv Close To hMainWnd As %TCP_CONNECT

The dialogproc handling the NOTIFY messages looks like this

Function TcpDialogProc(ByVal hDlg As Long, _
ByVal Msg As Long, _
ByVal wParam As Long, _
ByVal lParam As Long) As Long

Local sBuffer As String
Function = 0

Select Case As Long Msg
Case %TCP_CONNECT
Select Case LoWrd(LPARAM)

Case %FD_READ
If wParam = %INVALID_SOCKET Then Exit Function
HandleRead(wParam, sBuffer)
Function = 1
End Select

Case Else
End Select


HandleRead has a loop to read the actual data
Do
If Eof(readFnum) Then
Debug "Ignore empty buffer"
Else
Try
Tcp Recv readFnum, 8192, sPacket
Catch
If Err = 24 Then Debug "Ignore timeout error"
End Try
End If
bytes = bytes + Len(sPacket)
sBuffer = sBuffer + sPacket
Loop Until Len(sPacket) = 0 Or IsTrue Eof(readFnum) Or IsTrue Err

Currently I'm checking for Eof first and just ignoring the additional FD_READ notifications when there is nothing to read

[Mike Doty]

http://www.powerbasic.com/support/pbforums/showthread.php?t=13533

[Knuth Konrad]

Gary,

try putting your code snippet into [ CODE ][ /CODE ] (without the spaces) brackets. This will preserve your source code formatting.

Example:

Tcp Notify gDsFnum, Recv Close To hMainWnd As %TCP_CONNECT

The dialogproc handling the NOTIFY messages looks like this

Function TcpDialogProc(ByVal hDlg As Long, _
   ByVal Msg As Long, _
   ByVal wParam As Long, _
   ByVal lParam As Long) As Long

[John Petty]

Mike
I have seen this posted before, but it makes no sense, TCP RECV is a blocking statement so is not returned until 1 of the 3 possible conditions is met ie full buffer, EOF or ERR so sleep is meaningless unless you are saying M$ has a problem. I have programs running PB syntax also used by a number of others that download hundreds of 30K to 60K web pages in a 6 hour period. Computer range in speed from 5YO and laptops to high speed multicore. Internet connections ranging from hopeless 3rd world to 28 MB ADSL2. Never seen the problem.

Gary
I have to assume the "server" is not a standard web server as the steps you list in 1 to 5 would be strange, ie the client cannot just "receive a file sometime later" unless it was also set up as a server. If it is a standard web server then it should be acknowledging ever transmission from the client and usually the client would not send the next information until that has been received. If you are not using a "keep alive" request then it would normally be sending a memory cookie so that at the next stage it knows who it is dealing with and that the credentials were accepted.
TCP is of course an acknowledge protocol unlike UDP in which there is no gaurentee of delivery or error. This is at a low layer level and I don't know what would be passed back to TCP RECV if there were no actual message returned.

[Mike Doty]

I spent weeks trying to figure out why packets over 1460 bytes were not reliable without using TCPSafeReceive.
I've noticed PB examples usually use 1024 bytes.
Note: counting the number of reads needed can not be done as winsock may change the size of each packet.

[John Petty]

Mike
I am not sure what you mean 1460 byte packets, I assume the amount of data you send in a single TCP SEND or TCP PRINT statement, the original ethernet standard allows 1500 bytes of data to be included in a maximum packet size of 1518 bytes, later standards allow maximum packet size of 1522 adding 4 extra control bytes and of course OS's and hardware can increase this on LAN's. Once you go over the internet then the IP protocol consumes 20 bytes of those 1500 so giving a normal MTU of 1480 ( though an IP packet can have a maximum size of 64K bytes) and I have seen a number of carriers halve this for various reasons. If the carrier uses ATM then the packets are broken down to 48 bytes of data (into which is all the IP headers) + 5 bytes of overhead for its own protocol.
The point of that long description? This is all handled at lower levels than the normal application can see and of course protocol and MTU changes across the internet are never seen, so should be totally ignored as the data should be fragmented and reassembled as needed.
For most downloads I use a 4K buffer and rarely use a callback notify scheme rather i put that whole portion of the internet access in a thread and allow my main program to carry on doing anything else it can while the thread is suspended with all the blocking statements. When the data is received (and in my case parsed) I pass it back to the main program (or an error) obviously using an appropriate synchronisation, normally one or more mutexs. I have never had to use this "safe" method or had problems.
John

[Dale Yarker]

Gosta,

In Help look in TCP NOTIFY and UDP NOTIFY. The value for %FD_READ in WSOCK32.INC

Cheers,

[Gösta H. Lovgren-2]

Gosta,

In Help look in TCP NOTIFY and UDP NOTIFY. The value for %FD_READ in WSOCK32.INC

Cheers,

Thanks Dale. I figured it was something like that (a specific Include) but as code was not shown ....

=====================================
"Ask her to wait a moment
I am almost done."
Carl Friedrich Gauss (1777-1855),
while working,
when informed that his wife is dying
=====================================

[Mike Doty]

In help, look for the code in:
An ECHO client and server using TCP

[John Petty]

Mike
I hope you don't mind me continuing in this thread but it has caused me a lot of thinking. In both the problems you have quoted and Gary they have been on the client side of a TCP application. I can't understand where TCP NOTIFY fits in as it is meant for the server side of an application.
John

[Mike Doty]

John,
This isn't my thread.
I'm not sure why NOTIFY was used for help on %FD_READ.
I pointed to the help file on using %FD_READ.

The complete Echo Server and Echo Client sample can be found in your PB\SAMPLES\INTERNET\TCP folder.
Finally, it should be noted that there is no direct correlation between the number of TCP SEND statements executed, compared to the number of %FD_READ messages received. This is because Winsock may concatenate multiple data packets and issue a lesser number of %FD_READ messages in response. Therefore, it is usually necessary to construct your code so that it continues to read data from the incoming data stream until either the returned string is empty, or an error is detected. For example:
DIM InBuffer AS STRING
...
CASE %FD_READ
InBuffer = ""
IF hEcho = %INVALID_SOCKET THEN EXIT SELECT

DO
TCP RECV hEcho, 1024, buffer
IF LEN(buffer) = 0 OR ISTRUE ERR THEN EXIT LOOP
InBuffer = InBuffer + buffer
TCP SEND hEcho, buffer
LogEvent $DQ + Buffer + $DQ
LOOP
...

Here is a threaded client that I used to find the problem I've mentioned:
There is no NOTIFY on the client side as mentioned.
 
#PBFORMS CREATED V1.51
#COMPILE EXE
#DIM ALL
'------------------------------------------------------------------------------
'   ** Includes **
'------------------------------------------------------------------------------
#PBFORMS BEGIN INCLUDES
#IF NOT %DEF(%WINAPI)
    #INCLUDE "WIN32API.INC"
#ENDIF
#PBFORMS END INCLUDES
'   ** Constants **
'------------------------------------------------------------------------------
#PBFORMS BEGIN CONSTANTS
%IDD_DIALOG1  = 1001
%IDC_LISTBOX1 =  201
#PBFORMS END CONSTANTS
'------------------------------------------------------------------------------
'   ** Declarations **
'------------------------------------------------------------------------------
DECLARE CALLBACK FUNCTION ShowDIALOG1Proc()
DECLARE FUNCTION ShowDIALOG1(BYVAL hParent AS DWORD) AS LONG
DECLARE FUNCTION AttachMENU1(BYVAL hDlg AS DWORD) AS DWORD
#PBFORMS DECLARATIONS
'------------------------------------------------------------------------------
%PacketLength       = 10000 'whatever you want
%TimesToSendReceive = 500
%Threads            = 10
$Server             = "127.0.0.1"
GLOBAL gCS AS CRITICAL_SECTION, ghDlg AS DWORD
'------------------------------------------------------------------------------
'   ** Main Application Entry Point **
'------------------------------------------------------------------------------
 FUNCTION PBMAIN AS LONG
  LOCAL result AS LONG
 
  REDIM gCounter(%Threads) AS LONG
 
  InitializeCriticalSection gCS
  ShowDIALOG1 %HWND_DESKTOP
 
  DO WHILE THREADCOUNT > 1
    DIALOG SET TEXT ghDlg, "Users"+ STR$(THREADCOUNT-1)
    SLEEP 1000
  LOOP
 
DeleteCriticalSection     gCS
END FUNCTION
SUB StartThread(ThreadNumber AS LONG)
  LOCAL result AS DWORD
  THREAD CREATE TEST(ThreadNumber) TO RESULT
  THREAD CLOSE result TO result
END SUB
'   ** CallBacks **
'------------------------------------------------------------------------------
CALLBACK FUNCTION ShowDIALOG1Proc()
    SELECT CASE AS LONG CBMSG
        CASE %WM_INITDIALOG
            ' Initialization handler
 
        CASE %WM_NCACTIVATE
            STATIC hWndSaveFocus AS DWORD
            IF ISFALSE CBWPARAM THEN
                ' Save control focus
                hWndSaveFocus = GetFocus()
            ELSEIF hWndSaveFocus THEN
                ' Restore control focus
                SetFocus(hWndSaveFocus)
                hWndSaveFocus = 0
            END IF
 
        CASE %WM_DESTROY
        CASE %WM_COMMAND
            ' Process control notifications
            SELECT CASE AS LONG CBCTL
               CASE 101 TO 109  'Button Numbers 1-9 start a thread
                 IF CBCTLMSG = %BN_CLICKED OR CBCTLMSG = 1 THEN
                   StartThread CBCTL-100
                 END IF
            END SELECT
        CASE %WM_SYSCOMMAND
            IF (CBWPARAM AND &HFFF0) = %SC_CLOSE THEN
              'If threads still running, don't end dialog
              Lb  "X clicked. Threads running" + STR$(THREADCOUNT)
              IF THREADCOUNT > 1 THEN
                 RemoveMenu(GetSystemMenu(CBHNDL, %True), %SC_CLOSE, %MF_BYCOMMAND)
                 drawMenuBar CBHNDL
                 FUNCTION = 1
              END IF
            END IF
    END SELECT
END FUNCTION
'------------------------------------------------------------------------------
'------------------------------------------------------------------------------
'   ** Dialogs **
'------------------------------------------------------------------------------
FUNCTION ShowDIALOG1(BYVAL hParent AS DWORD) AS LONG
    LOCAL lRslt AS LONG
#PBFORMS BEGIN DIALOG %IDD_DIALOG1->->
    LOCAL hDlg  AS DWORD
    DIALOG NEW hParent, "Dialog1", 355, 282, 270, 117, %WS_POPUP OR _
        %WS_DLGFRAME OR %WS_THICKFRAME OR %WS_CAPTION OR %WS_SYSMENU OR _
        %WS_CLIPCHILDREN OR %WS_VISIBLE OR %DS_MODALFRAME OR %DS_CENTER OR _
        %DS_3DLOOK OR %DS_NOFAILCREATE OR %DS_SETFONT, %WS_EX_CONTROLPARENT _
        OR %WS_EX_LEFT OR %WS_EX_LTRREADING OR %WS_EX_RIGHTSCROLLBAR, TO _
        hDlg
    CONTROL ADD LISTBOX, hDlg, %IDC_LISTBOX1, , 0, 15, 270, 87, %WS_CHILD OR _
        %WS_VISIBLE OR %WS_VSCROLL, %WS_EX_CLIENTEDGE OR %WS_EX_LEFT OR _
        %WS_EX_LTRREADING OR %WS_EX_RIGHTSCROLLBAR
#PBFORMS END DIALOG
    ghDlg = hdlg
    DIALOG SET TEXT hDlg, "(" + $Server + ")  Packet length" + STR$(%PacketLength)
    LOCAL ButtonNumber AS LONG
    FOR ButtonNumber = 1 TO 9
         CONTROL ADD BUTTON,  hDlg, ButtonNumber + 100, "User" + STR$(ButtonNumber), (ButtonNumber-1)*30, 0, 30, 15, _
        %WS_CHILD OR %WS_VISIBLE OR %BS_TEXT OR %BS_PUSHBUTTON OR %BS_CENTER _
        OR %BS_VCENTER, %WS_EX_LEFT OR %WS_EX_LTRREADING
    NEXT
    DIALOG SHOW MODAL hDlg, CALL ShowDIALOG1Proc TO lRslt
#PBFORMS BEGIN CLEANUP %IDD_DIALOG1
#PBFORMS END CLEANUP
    FUNCTION = lRslt
END FUNCTION
THREAD FUNCTION TEST(BYVAL ThreadNumber AS LONG) AS LONG
   LOCAL counter&,nSocket&, sBuffer$, sPacket$, Errors&, Result AS LONG
   LOCAL sDataSent AS STRING
   nSocket = FREEFILE   'Always send complete packets until the end!!!
   TCP OPEN PORT 999 AT $Server AS nSocket TIMEOUT 5000
   IF ERR THEN LB "Error opening":EXIT FUNCTION 'put into listbox
   DIALOG SET TEXT ghDlg, "Users"+ STR$(THREADCOUNT-1)
   RemoveMenu(GetSystemMenu(ghDlg, %false), %SC_CLOSE, %MF_BYCOMMAND)
   DrawMenuBar ghDlg 'don't allow ending
FOR Counter = 1 TO %TimesToSendReceive
    'send data here
    sDataSent = STRING$(%PacketLength,"X")
    TCP SEND nSocket, sDataSent
    IF ERR THEN
      LB "Send error" + STR$(ERRCLEAR) + " in Thread" + STR$(ThreadNumber) + " attempt" + STR$(Counter)
      EXIT FOR   'put into listbox
    END IF
    sPacket = "":    sBuffer = ""
    'receive data back here
    result = tcpSafeReceive(nSocket, %PacketLength,sPacket)   'Don Dickinson
    IF result THEN   'success
       'double check that data sent is the same as data that came back
       IF sPacket <> sDataSent THEN
         INCR Errors
          LB "Receive error, data sent does not match data received"
          EXIT FOR  'exit on first error
        'ELSE
        '  ? sPacket
        END IF
    ELSE
       LB "Receive error, user" + STR$(ThreadNumber)  'put into listbox
       EXIT FOR
    END IF
NEXT
  TCP CLOSE nSocket
  result = THREADCOUNT -1
  IF result =  1 THEN    'if only main thread then
    RemoveMenu(GetSystemMenu(ghDlg, %True), %SC_CLOSE, %MF_BYCOMMAND)  'enable
    DrawMenuBar ghDlg
    DIALOG SET TEXT ghDlg, "No users"
  ELSE
    DIALOG SET TEXT ghDlg, "Users" + STR$(result)
  END IF
END FUNCTION
'------------------------------------------------------------------------------
'tcpSaveReceive.Inc
 
FUNCTION tcpSafeReceive(BYVAL hSocket AS LONG, BYVAL iBufferLen AS LONG, _
                        recBuff AS STRING) AS LONG
   DIM iLeft AS LONG
   DIM sBuffer AS STRING
   recBuff = ""
   iLeft = iBufferLen
   DO
      sBuffer = SPACE$(iLeft)
      ON ERROR RESUME NEXT
      sBuffer = SPACE$(iBufferLen)
      TCP RECV hSocket, iLeft, sBuffer
      IF ERR THEN
         FUNCTION = %False
         EXIT FUNCTION
      END IF
      recBuff = recBuff + sBuffer
      IF LEN(recBuff) >= iBufferLen THEN
         EXIT DO
      END IF
      iLeft = iBufferLen - LEN(recBuff)
      SLEEP 5
   LOOP
   FUNCTION = %True
END FUNCTION
SUB EnterCritical
   EnterCriticalSection gCS
END SUB
SUB LeaveCritical
   LeaveCriticalSection gCS
END SUB
SUB LB(s AS STRING)
  LISTBOX ADD ghDlg, %IDC_LISTBOX1, s
END SUB

[garry star]

What I learned

FYI, based on what I read here and previous work with TCP in C++, I modified my message to have a constant sized header with the length within the first few bytes (20). I then do

recv of this header (size 20)
parse the length value
recv of the remainder of the message (may still need a recv loop)

The sender is another pwbasic program that sends the entire message (60k or more) in one send, after constructing the header in the first 20 bytes

This works reliably (and quickly) for me.

Garry

[garry star]

tcp send size limit?

Is there a limit to the size of message that I can call Tcp Send with? I'm sending up to 500k and it seems to work but I was just wondering.

Thanks...

[garry star]

FYI, based on what I read here and previous work with TCP in C++, I modified my message to have a constant sized header with the length within the first few bytes (20). I then do
...

This works reliably (and quickly) for me.

Garry

Well, it isn't quite as reliable as I thought.

I am doing at least 2 Tcp Recvs for each message (first to get the length, second to read the remainder of the message.) I then ignore any FD_READ if there is no data to be read. However, once in a while, I must be ignoring an FD_READ that I shouldn't since my server gets into a state where it doesn't receive an FD_READ when my client sends (and there are no errors on the send).

I think this means that I need to go back to doing a single Tcp Recv for each FD_READ and then continue reading/accumulating my message only after receiving another FD_READ.

Can anyone confirm that this is the correct approach?

Thanks!

Garry

[Mike Doty]

See help file under echo server.
See TcpSafeReceive in my posting above.

the complete echo server and echo client sample can be found in your pb\samples\internet\tcp folder.
Finally, it should be noted that there is no direct correlation between the number of tcp send statements executed, compared to the number of %fdçi6d messages received. This is because winsock may concatenate multiple data packets and issue a lesser number of %fd_read messages in response. Therefore, it is usually necessary to construct your code so that it continues to read data from the incoming data stream until either the returned string is empty, or an error is detected.