python - Read from socket: Is it guaranteed to at least get x bytes?

Question

I have a rare bug that seems to occur reading a socket.

It seems, that during reading of data sometimes I get only 1-3 bytes of a data package that is bigger than this.

As I learned from pipe-programming, there I always get at least 512 bytes as long as the sender provides enough data.

Also my sender does at least transmit >= 4 Bytes anytime it does transmit anything -- so I was thinking that at least 4 bytes will be received at once in the beginning (!!) of the transmission.

In 99.9% of all cases, my assumption seems to hold ... but there are really rare cases, when less than 4 bytes are received. It seems to me ridiculous, why the networking system should do this?

Does anybody know more?

Here is the reading-code I use:

mySock, addr = masterSock.accept()
mySock.settimeout(10.0)
result = mySock.recv(BUFSIZE)
# 4 bytes are needed here ...
...
# read remainder of datagram
...

The sender sends the complete datagram with one call of send.

Edit: the whole thing is working on localhost -- so no complicated network applications (routers etc.) are involved. BUFSIZE is at least 512 and the sender sends at least 4 bytes.

Answer 1

I assume you're using TCP. TCP is a stream based protocol with no idea of packets or message boundaries.

This means when you do a read you may get less bytes than you request. If your data is 128k for example you may only get 24k on your first read requiring you to read again to get the rest of the data.

For an example in C:

int read_data(int sock, int size, unsigned char *buf) {
   int bytes_read = 0, len = 0;
   while (bytes_read < size && 
         ((len = recv(sock, buf + bytes_read,size-bytes_read, 0)) > 0)) {
       bytes_read += len;
   }
   if (len == 0 || len < 0) doerror();
   return bytes_read;
}