Problem communicating with System Service over TCP/IP

[Walter Schütz]

Hi

I try to get data from a system service that communicates over TCP/IP. The following code segments are incomplete but show the structure of my program (you can not test it anyway, as I can not post the service):

If I write my code like this it will work smooth:

tcpHandle = FREEFILE
FOR ix = 1 TO 10
  TCP OPEN PORT 6713 AT "127.0.0.1" AS tcpHandle TIMEOUT 5000     
  TCP SEND tcpHandle, dataIn
  DO
    TCP RECV tcpHandle, 4096, Buffer$
    dataOut = dataOut + Buffer$
  LOOP WHILE (LEN(Buffer$)<>0) AND (EOF(tcpHandle)= FALSE)
  TCP CLOSE tcpHandle
NEXT ix

However if I try to write it like this, the code will not work:

tcpHandle = FREEFILE
TCP OPEN PORT 6713 AT "127.0.0.1" AS tcpHandle TIMEOUT 5000
FOR ix = 1 TO 10
  TCP SEND tcpHandle, dataIn
  DO
    TCP RECV tcpHandle, 4096, Buffer$
    dataOut = dataOut + Buffer$
  LOOP WHILE (LEN(Buffer$)<>0) AND (EOF(tcpHandle)= FALSE)
NEXT ix
TCP CLOSE tcpHandle

The second sample returns correct data once, but in the second loop TCP RECV will raise an error 57: Device I/O error. Buffer$ seems empty, EOF is FALSE (before and after second TCP RECV)
My dataOut is a lot shorter than 4096 – so the code in the receive loop is executed only once.

Seems so simple - but I can not make it work. So, what is my mistake? Maybe impossible to do with TCP/IP?

Thanks!

[Dale Yarker]

Whatever service is at port 6713 apparently closes after it responds once. With TCP OPEN/CLOSE inside the the FOR/NEXT loop in upper code you're causing the service to set up for another single response each of the 10 times.

Cheers,

[Walter Schütz]

Hi Dale

So - in your oppinion - it is quite normal behaviour for such a service to close down after one single reply?

Thanks

[Thomas Tierney]

That all depends on the service, if its all at once the service will read all data until there is nothing more then post a response and close the socket. timing is the key, if your not fast enough to send, the service will close once it detects nothing on the port the next time it checks

[Dale Yarker]

Like Thomas said, depends on (undisclosed) service. An EWAG is that the service's main thread never closes. When you do a TCP OPEN the main thread creates a worker thread that receives the TCP SEND, sends the result, then the worker thread ends. In which case faster requests won't help.

Cheers,

[Mike Doty]

How does this work?

1. Error handling
2. Improved TCP RECV loop

The server may close if it doesn't get what it wants.

#COMPILE EXE
#DIM ALL
#INCLUDE "win32api.inc"
$Server = "127.0.0.1"
%Port   = 6713
FUNCTION PBMAIN () AS LONG
  LOCAL tcpHandle AS LONG
  LOCAL dataIn  AS STRING
  LOCAL sBuffer AS STRING
  LOCAL result  AS LONG
 
  tcpHandle = FREEFILE
  TCP OPEN PORT %PORT AT $Server AS tcpHandle TIMEOUT 5000
  IF ERR THEN ? "tcp open error"  + STR$(ERRCLEAR):SLEEP 1000:EXIT FUNCTION
 
  DataIn = "How now brown cow"
  TCP SEND tcpHandle, dataIn
  IF ERR = 0 THEN
    result = TcpSafeReceive(tcpHandle,4096,sBuffer)
    IF result = 0 THEN
      ? "Server sent back " + $DQ + sBuffer $DQ
    ELSE
      ? "TcpSafeReceive error" + STR$(result)
    END IF
  ELSE
    ? "TCP send error" + STR$(result)
  END IF
  TCP CLOSE #tcpHandle
  ? "tcp close"
  SLEEP 1000
 END FUNCTION
 
FUNCTION tcpSafeReceive(BYVAL hSocket AS LONG, BYVAL iBufferLen AS LONG, _
                        recBuff AS STRING) AS LONG
   'tcpSafeReceive by Don Dickinson
   DIM iLeft AS LONG
   DIM sBuffer AS STRING
   recBuff = ""
   iLeft = iBufferLen
   DO
      sBuffer = SPACE$(iLeft)
      ON ERROR RESUME NEXT
      sBuffer = SPACE$(iBufferLen)
      TCP RECV hSocket, iLeft, sBuffer
      IF ERR THEN
         FUNCTION = %False
         EXIT FUNCTION
      END IF
      recBuff = recBuff + sBuffer
      IF LEN(recBuff) >= iBufferLen THEN
         EXIT DO
      END IF
      iLeft = iBufferLen - LEN(recBuff)
      SLEEP 5
   LOOP
   FUNCTION = %True
END FUNCTION

[Walter Schütz]

Thanks for you help!

Mike please note:
Don Dickinson's code was written to read a fixed number of bytes from the server. So written as comment in some of the postings.

'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
'  tcpSafeReceive
'  Specify a length of data to wait for and this function
'  retrieves data until all data has filled the buffer or
'  an error occurs.
'  Returns: %True on success, %False on error
'~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In my case this will fail, as the response is variable length (what I did not write in my post).

If I correctly understand the threads that use Don Dickinson's code this solution was developed as a EOF()=TRUE might be received before all data is sent. So I think EOF() should be avoided i my case too. This lead me to the following code:

#COMPILE EXE
#DIM ALL
#INCLUDE "win32api.inc"

'%DebugView = 0 'No Output für DebugView
%DebugView = 1 'Output für DebugView

$Server = "127.0.0.1"
%Port   = 6713

MACRO TRUE  = -1
MACRO FALSE = 0

SUB MyTrace (strTxt AS STRING)
  #IF %DebugView = 1
    OutputDebugString BYCOPY("LMX-" & strTxt)
  #ENDIF
END SUB

FUNCTION TcpReceive    (BYVAL hSocket    AS LONG, _
                        BYVAL iBufferLen AS LONG, _
                              recBuff    AS STRING) AS LONG
DIM sBuffer AS STRING
  recBuff = ""
  DO
    ON ERROR RESUME NEXT
    sBuffer = SPACE$(iBufferLen)
    TCP RECV hSocket, iBufferLen, sBuffer
    MyTrace sBuffer
    recBuff = recBuff + sBuffer
    SLEEP 5
  LOOP UNTIL sBuffer="" OR ISTRUE ERR
  IF sBuffer="" THEN
    'ok, no more data
    MyTrace "ok, no more data  "
    FUNCTION = TRUE
  ELSE
    'Comm-Error
    MyTrace "Comm-Error" + STR$(ERRCLEAR)
    FUNCTION = FALSE
  END IF
END FUNCTION
 
FUNCTION PBMAIN () AS LONG
LOCAL tcpHandle AS LONG
LOCAL dataIn    AS STRING
LOCAL sBuffer   AS STRING
LOCAL result    AS LONG

  tcpHandle = FREEFILE
  TCP OPEN PORT %PORT AT $Server AS tcpHandle TIMEOUT 5000
  IF ERR THEN MyTrace "tcp open error"  + STR$(ERRCLEAR):SLEEP 1000:EXIT FUNCTION

  dataIn = "How now brown cow"
  TCP SEND tcpHandle, dataIn
  IF ERR = 0 THEN
    MyTrace "before tcpReceive"
    result = TcpReceive( tcpHandle, 4096, sBuffer)
    IF result = TRUE THEN
      MyTrace "Server sent back " + $DQ + sBuffer + $DQ
    ELSE
      MyTrace "TcpSafeReceive error" + STR$(result)
    END IF
  ELSE
    MyTrace "TCP send error" + STR$(result)
  END IF
  TCP CLOSE #tcpHandle
  MyTrace "tcp close"
  SLEEP 1000
END FUNCTION

[Mike Doty]

I'll test your modification, later today.
This should work. Added variable length buffer.
It should also be very fast since buffer can be very large.
Note:
My first posting showed an error if RESULT returned 1.
That should have been IF Result Then 'success

#COMPILE EXE
#DIM ALL
#INCLUDE "win32api.inc"
$Server = "127.0.0.1"
%Port   = 6713
FUNCTION PBMAIN () AS LONG
  LOCAL tcpHandle AS LONG
  LOCAL dataIn  AS STRING
  LOCAL sBuffer AS STRING
  LOCAL result  AS LONG
  LOCAL BufLen  AS LONG
  tcpHandle = FREEFILE
  TCP OPEN PORT %PORT AT $Server AS tcpHandle TIMEOUT 5000
  IF ERR THEN ? "tcp open error"  + STR$(ERRCLEAR):SLEEP 1000:EXIT FUNCTION
  DataIn = "How now brown cow"
  BufLen = LEN(DataIn)
  TCP SEND tcpHandle, dataIn
  IF ERR = 0 THEN
    result = TcpSafeReceive(tcpHandle,BufLen,sBuffer)
    IF result THEN  'success
      ? "Server sent back " + $DQ + sBuffer $DQ
    ELSE
      ? "TcpSafeReceive error" + STR$(result)
    END IF
  ELSE
    ? "TCP send error" + STR$(result)
  END IF
  TCP CLOSE #tcpHandle
  ? "tcp close"
  SLEEP 2000
 END FUNCTION
FUNCTION tcpSafeReceive(BYVAL hSocket AS LONG, BYVAL iBufferLen AS LONG, _
                        recBuff AS STRING) AS LONG
   DIM iLeft AS LONG
   DIM sBuffer AS STRING
   recBuff = ""
   iLeft = iBufferLen
   DO
      sBuffer = SPACE$(iLeft)
      ON ERROR RESUME NEXT
      sBuffer = SPACE$(iBufferLen)
      TCP RECV hSocket, iLeft, sBuffer
      IF ERR THEN
         FUNCTION = %False
         EXIT FUNCTION
      END IF
      recBuff = recBuff + sBuffer
      IF LEN(recBuff) >= iBufferLen THEN
         EXIT DO
      END IF
      iLeft = iBufferLen - LEN(recBuff)
      SLEEP 5
   LOOP
   FUNCTION = %True
END FUNCTION

[Mike Doty]

Walter,
Your code works with any buffer length, good job (always waits until timeout for received data.)
Not sure if this was of any use to you.

Note for any lurkers:
Don's code tests for the number of bytes by setting buffer size to what will be expected back.
Walter's code does not need to know the number of bytes that will be received.

[Walter Schütz]

Mike,

Thanks for having a look into my code.

Any readers please note: If you know the length of data to be received Don's code is more secure than mine. My SLEEP 5 might be too short if data is read e.g. from a disk.

If you have any other mean by recognizing data is complete this should be applied. In my case there is a LF at the end of the string so I'll check on that in my final code I will read until I receive that or until a timeout occurs.

Further: The LOOP should be coded as DO-LOOP with EXIT before the SLEEP 5 and not a DO LOOP UNTIL. This will save the waiting time in case data is already complete...

One more thing: Coding the TcpReceive this way returned never with my 5000ms timeout as set with TCP OPEN - well at least not when the server is correctly installed and correct data is sent to the server. I suppose that is due to the way the server is programmed. So I added my own timeout - just in case I do not get my terminating $LF...

FUNCTION TcpReceive    (BYVAL hSocket    AS LONG, _
                        BYVAL iBufferLen AS LONG, _
                              recBuff    AS STRING) AS LONG
DIM sBuffer   AS STRING
DIM startTime AS SINGLE
DIM maxTime   AS INTEGER
  maxTime = 5                 ' 5 seconds, NOTE: in my case TCP RECV can be called
                              ' forever (if set up correctly and correct inputdata)
                              ' without returning an ERR
  recBuff = ""
  FUNCTION = FALSE
  startTime = TIMER
  DO
    ON ERROR RESUME NEXT
    sBuffer = SPACE$(iBufferLen)
    TCP RECV hSocket, iBufferLen, sBuffer
    MyTrace sBuffer
    recBuff = recBuff + sBuffer
    IF (INSTR(-1, recBuff, $LF)>0) THEN
      'ok, no more data
      recBuff = RTRIM$( recBuff, $LF)
      MyTrace "ok, no more data  "
      FUNCTION = TRUE
      EXIT LOOP
    ELSEIF (ISTRUE ERR) THEN
      'Comm-Error
      MyTrace "Comm-Error" + STR$(ERRCLEAR)
      EXIT LOOP
    ELSEIF (TIMER-startTime>maxTime) THEN
      'Timeout
      MyTrace "Timeout" + STR$(ERRCLEAR)
      EXIT LOOP
    END IF
    SLEEP 5                 ' 5 milliseconds
  LOOP
  EXIT FUNCTION
END FUNCTION

[Brad D Byrne]

hmm, I might be a little confused with this tcp stuff, but as I recall, if the hand-shaking is setup right, (ie, acting upon/demanding a server response) then SLEEP is never needed,

am I wrong????

[Mike Doty]

My server-side code has no sleep statement.

I only use it because Don did on the client-side.
Without Don's TcpSafeReceive I would often lose data coming back
when sending very large packets to an echo server running multiple threads. I spent many hours trying to figure out what caused the
loss without any luck. TcpSafeReceive solved it. I wouldn't remove the SLEEP 5 without contacting DD.

Packets over 1,460 bytes caused a problem: http://www.powerbasic.com/support/pb...ad.php?t=39788

[Brad D Byrne]

ok, Mike, good to know! I'll look into Don's code, thanks!

[Don Dickinson]

i'm pretty sure i put the "sleep 5" in there to assist with multi-threading responsiveness and nothing more. i think i put it in there in response to my initial testing of the ttds data server. ttds was tested with literally hundreds of threads running on 4 or 5 machines steadily for several days. i don't recall the need for the sleep in any capacity other than to allow for smooth running from all the applications - meaning no one app or thread on an app would run without giving up time to other threads/apps/clients. there were no data receive problems that i remember. i don't see why you couldn't remove it.

best regards,
don