TCP RECV question

[Paul Johnson]

For the TCP RECV command, one provides a string to be used as a buffer, and the amount of data they want. My question is this:

In straight winsock, when doing a recv usually I would loop asking for a chunk at a time for very large transfers (such as 50mb or larger maybe). Perhaps chunks of 4k or 64k, etc. This was because recv wanted a buffer that was already allocated and I didn't want to be allocating 50 meg buffers and getting a few hundred K on each call.

So my question is how TCP RECV works. If I'm expecting a 250mb receive, should I be doing buffers of 4k (or some other value), or am I OK to loop, asking for the entire amount yet to be received each time. Does TCP RECV allocate the buffer of the size I specify all at once, or internally does it split it into chunks, etc?

How is that usually handled best to avoid wasting memory, but also to maximize transfer speed?

[Mike Doty]

http://www.powerbasic.com/support/pbforums/showthread.php?t=40734

I would use TCPSafeReceive no matter what buffer size is used.

[Paul Purvis]

In my experience, the internet is much slower than a intranet, so you need to allow for that.
I found it beneficial to have a parameter you can change that sets the buffer size. You're program may have a default size or even a size that is limiting, such as a small or large size.
If you're buffer is too large for a particular connection or even too small, it can run up your cpu usage, and bring your computer along with other computers connected nearby to a very slow pace.
Dependability is much more better than speed(trying to get every ounce of performance).
What i learned was to use smaller buffers then move up in size later after you have everything working. On windows 2000 systems, i have found out to get the performance up, you use a tool such as Drtcp to adjust your Rwin and MTU.

With a way in your program to alter(tweak) the buffer size, you can make adjustments later, without rewriting your program to suit whatever equipment is being used. You also never know when a dialup modem will have to be used, which means smaller buffers are usually better.

I inserted some SLEEP commands into some of my tcp programs too.
Most of all my programs are somewhere here on the forum.
If my memory servers me right, i believe as i approached a buffer size of 5000 bytes, i had lost connections over the internet, but the intranet was ok, but for large transfers, your network infastructure for 100mbps connections are probably going to suffer.

One one of my backup programs that transfers data over the internet to a remote site brings other computers to their knees if they too are accessing the internet at the same time, if i set the buffer to high.
I would say start with something like 1400 if you want to go over 1024.
I believe the buffer is mostly set on the client side. You can place the number of bytes in the first few characters of the string being sent too.
I like to have some kind of authentication string on the first packet sent/received if i am writing the client and server.
I am only a novice though so keep that in mind too.

[Thomas Tierney]

unfortuanately any implementation that requires a buffer to be full like the tcpsafereceive is prone to problems. If a user gets in all they have to do is send a single character at certain intervals to cause a ADOS attack. When the time out period has been determined (lets say 30 seconds) then a hacker can send a character every 29 seconds if the buffer is 65K in length then thats like 523 hours.. if the attack is distributed then the hacker can open 10,000 connections and drive legitimate traffic to the TCP Server into the ground. The only reasonable way to do this is to do character based timeouts. get one char at a time and timeout after fewer seconds..

[Mike Doty]

Security check after TCP RECV or TcpSafeReceive

unfortuanately any implementation that requires a buffer to be full like the tcpsafereceive is prone to problems. If a user gets in all they have to do is send a single character at certain intervals to cause a ADOS attack. When the time out period has been determined (lets say 30 seconds) then a hacker can send a character every 29 seconds if the buffer is 65K in length then thats like 523 hours.. if the attack is distributed then the hacker can open 10,000 connections and drive legitimate traffic to the TCP Server into the ground. The only reasonable way to do this is to do character based timeouts. get one char at a time and timeout after fewer seconds..

Why wouldn't other receive routines also be prone to problems?
Added a timeout counter.

TcpSafeReceive does not need to be changed in my opinion.
Add security after each receive.
This also assumed someone could get past any up front security.

Simulating reads in this functional example.
Please bring up more security concerns or start another thread.

%true = -1
FUNCTION PBMAIN () AS LONG
 
  LOCAL hSocket&,iBufferLen&,recBuff$,result&,counter&,timeoutcounter&
 
  timeoutcounter = 0
  iBufferLen = 1400
 
  FOR Counter = 1 TO 10 'have to end sometime
    result = tcpSafeReceiveSimulate(hSocket,iBufferLen,recBuff)
 
    [B]'Security routine 1[/B]: Timeout counter   (bring them on)
    IF LEN(recBuff) < iBufferLen THEN
      INCR timeoutcounter
      IF  timeoutcounter > 2 THEN  'give user a couple of chances
         REM ? "Timeouts" + STR$(timeoutcounter),%MB_ICONERROR, "Terminating user, too many timeouts"
         EXIT FOR
      END IF
    END IF
 
  NEXT
 
END FUNCTION
 
FUNCTION tcpSafeReceiveSimulate(BYVAL hSocket AS LONG, BYVAL iBufferLen AS LONG, recBuff AS STRING) AS LONG
 
  'Simulate reading packets, function logic is not changed.
  'TcpSafeReceive is by Don Dickinson and this is for simulation only
 
  STATIC counter AS LONG
  INCR  counter
  recbuff = SPACE$(ibufferlen)
  IF counter MOD 3 = 0 THEN recbuff = "x" 'cause timeout, not an error
  FUNCTION = %TRUE
 
END FUNCTION

[Thomas Tierney]

Why wouldn't other receive routines also be prone to problems?
Added a timeout counter.

Most receiver implementations are prone to this problem.

TcpSafeReceive does not need to be changed in my opinion.
Add security after each receive.

Sorry.. in the example code I dont see anything that makes me believe it is secure.

This also assumed someone could get past any up front security.

I deal with security all the time and the first rule is:
"Never assume the person that is there is the one that is supposed to be there"

if the server is availble on the net..then it is available. The biggest risk this year to security is the massive layoffs in the high tech sector.