UNIX Socket FAQ

A forum for questions and answers about network programming on Linux and all other Unix-like systems

You are not logged in.

  • Index
  • » Networking
  • » socket recv waiting for some time even if data is available

#1 2013-05-09 12:52 PM

afxgs
Member
Registered: 2013-05-09
Posts: 6

socket recv waiting for some time even if data is available

I am sending a request to server and reading the response using recv() function in WinSock2. I have a requirement that the recv must be completed within 200 milliseconds. But there is a delay of more than 600ms for the recv() call.

I tried the same request in Java and it is getting the response within 10 or 12 milliseconds. So it is evident that this is not a server side delay. But I need to implement the tool for Windows in C++ and am struggling with this problem for a few weeks.

I also tried with non blocking sockets . But in this case, the select returns 0 for the first 400 milliseconds and only after that some data is available.

I tried setting TCP_NODELAY and SO_RCVTIMEO socket options but the delay still persists.

Am I doing anything wrong here? What could be causing the unnecessary delay in recv()? Is there any other Winsock function which is faster than recv()?

Last edited by afxgs (2013-05-09 01:11 PM)

Offline

#2 2013-05-09 07:02 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: socket recv waiting for some time even if data is available

I assume you're doing some send() of some request to the server prior to the recv()?  Are you sure the format of the data is correct, and the same as the data sent via your Java test?  I'm thinking it might be something like a missing CR/LF terminator or something like that, causing the server to delay a while in the expectation of more data arriving, before eventually timing out and handling the request as-is...  Just a guess, though...  If you can, it might be helpful to run something like wireshark to get a detailed view of the request when sent via Java vs. sent via your C++ program, in the hopes of spotting what is different and may be responsible for triggering the delay...

But, if it's not something like that, I'm not sure...  Setting TCP_NODELAY should've removed any possibility of delays on your end of things...  (Setting SO_RCVTIMEO is probably not helpful at all...  If it even works and does anything at all; on many systems, it doesn't...  And, where it does, its behavior is inconsistent and unportable, so I wouldn't recommend it in any case...  Best to do your own I/O timeouts via select() or similar...)

Offline

#3 2013-05-10 02:30 AM

asfxgs
Guest

Re: socket recv waiting for some time even if data is available

ya..i am using the same request as that in java..
I have already checked wireshark capture. It shows that the data has arrived within 30-40 milliseconds or so but the client is sending acknowledgement only after some time.
Another thing I noticed is that if I give the recieve buffer char [] as having the same size as content length, this delay doesnot happen. The response is available in 30-40 milliseconds.

#4 2013-05-10 12:13 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: socket recv waiting for some time even if data is available

Another thing I noticed is that if I give the recieve buffer char [] as having the same size as content length, this delay doesnot happen.

So, it's when you use a larger buffer in the recv() call that it does happen?  What flags are you passing to recv()?  Because, that sounds like MSG_WAITALL behavior...  Or, it could happen on a blocking socket normally, but generally doesn't on sane systems (but, since you're on Windows, we're not dealing with a sane system)...  But, you also mentioned putting the socket into non-blocking I/O mode, didn't you?  Certainly, a non-blocking socket should NEVER hang in recv() waiting for the full requested amount of data (unless maybe you tell it to by passing MSG_WAITALL)...

Offline

#5 2013-05-10 12:20 PM

afxgs
Member
Registered: 2013-05-09
Posts: 6

Re: socket recv waiting for some time even if data is available

my code is something like

char recvbuf[395];
int bufLen = 395;
int iResult = recv(SocketVar, recvbuf, bufLen, 0);

if iResult is exactly 395 bytes recv is not waiting. If it is not 395 (for even 1 byte difference, for both lesser and greater) recv is waiting for around 600ms.

Last edited by afxgs (2013-05-10 12:22 PM)

Offline

#6 2013-05-10 12:36 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: socket recv waiting for some time even if data is available

That just sounds utterly bizarre...  I have no idea WTF would cause that, then...

I assume this is with the socket in blocking I/O mode?  You mentioned that if you used non-blocking mode, even your select() would block for that long before registering data available?  How about, just as a test (it's not really something one should do for real-world use), you use a non-blocking socket, but don't bother with select(), and just spin in a tight loop retrying the non-blocking recv() over and over until it works?  Maybe with a slight delay between each attempt, to keep from hogging the CPU...  I'm just curious if that will read the data as soon as it arrives, or it'll only see it after the long delay too...

Also, out of curiosity, do you adjust your send or receive buffer sizes at all?  I wouldn't think either would be responsible for such strange behavior, but I'm just grasping at straws...  How about any other sockopts you set?

Offline

#7 2013-05-15 05:35 AM

afxgs
Member
Registered: 2013-05-09
Posts: 6

Re: socket recv waiting for some time even if data is available

RobSeace wrote:

don't bother with select(), and just spin in a tight loop retrying the non-blocking recv() over and over until it works?


I tried this also...still the delay is thr...

i didn''t try any other sockopts other than TCP_NODELAY and SO_RCVTIMEO

Offline

#8 2013-05-15 06:11 AM

Nope
Administrator
From: Germany
Registered: 2004-01-24
Posts: 385
Website

Re: socket recv waiting for some time even if data is available

If you use unmanaged C++ in WinVista or above (or the server equivalent) the communication between program and Windows is routed through a security layer, which does delay quite massively. So I/O heavy programs do actually run in, for example, C# a lot faster...

Offline

#9 2013-05-15 06:59 AM

afxgs
Member
Registered: 2013-05-09
Posts: 6

Re: socket recv waiting for some time even if data is available

I need to mark the DSCP bit...is this possible with C#?

Offline

#10 2013-05-15 10:56 PM

Nope
Administrator
From: Germany
Registered: 2004-01-24
Posts: 385
Website

Re: socket recv waiting for some time even if data is available

As far as I am aware, it is not possible to set the DSCP bits in managed code. Last time I checked, C# had no means to do anything as low level as that. Also as far as I am aware, WCF also offers no functionality to influence these bits in the header. There once was a method to use a windows service to accomplish something like that, but that was still TOS related.

But since I was barred by superiors command from using any low level stuff "no one but me might understand". I haven't checked on that since .NET 2.0, so there might have been some movement in this area, since QoS is a buzz word these days.

Sorry, but these things were one of the main reasons to stick with Linux for my private pleasure... There a single line would do the trick.

You might want to google in the direction of "traffic control API" or something like that. Or you might try your luck in the msdn forum.

Offline

#11 2013-05-16 04:50 AM

afxgs
Member
Registered: 2013-05-09
Posts: 6

Re: socket recv waiting for some time even if data is available

Nope wrote:

You might want to google in the direction of "traffic control API" or something like that. Or you might try your luck in the msdn forum.

It is from the MSDN site that I got a solution to set the DSCP bit...but it is in C++...and this code is causing the issues mentioned in my question. I cannot use any alternative language because of this one reason...

Offline

#12 2013-05-16 11:52 AM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: socket recv waiting for some time even if data is available

It's threads like these that make me so very, very thankful I don't ever have to write software to run on Windows...  It's like MS goes out of their way to make developers' lives hell just for pure, sadistic pleasure...

Offline

#13 2013-05-16 07:39 PM

Nope
Administrator
From: Germany
Registered: 2004-01-24
Posts: 385
Website

Re: socket recv waiting for some time even if data is available

afxgs wrote:

I need to mark the DSCP bit...is this possible with C#?

Wait a moment. You tell us the recv is slow, but the DSCP marking is needed only for sending data. Have you tried sending the data using C++ and receiving it with a C# program? If so, what delay could be seen?
Second, isn't the DSCP setting done on socket level? Why not define and create the socket in C++ and the using it in C#? I did comparable things when dealing with long file paths (longest was over 10000 chars with a filename being itself around 200 chars). I used a C++ method to find the file and open the stream but using C# to do the actual reading and processing.

Maybe mixing C++/C# could help you too.

Offline

#14 2013-05-17 10:03 AM

afxgs
Member
Registered: 2013-05-09
Posts: 6

Re: socket recv waiting for some time even if data is available

Nope wrote:

Have you tried sending the data using C++ and receiving it with a C# program? If so, what delay could be seen?
Second, isn't the DSCP setting done on socket level? Why not define and create the socket in C++ and the using it in C#?

I have never worked in C#..got to do some research..will try this...can u give me a small example on how to mix C++ and C#..

Last edited by afxgs (2013-05-17 10:04 AM)

Offline

#15 2013-05-18 10:48 AM

Nope
Administrator
From: Germany
Registered: 2004-01-24
Posts: 385
Website

Re: socket recv waiting for some time even if data is available

I think you should change over to stackoverflow.com or the like. I've not the time these days to guide you through C# and C++. Sorry.

Offline

#16 2013-05-19 10:33 AM

afxgss
Guest

Re: socket recv waiting for some time even if data is available

Ok...nyway thnx fr d idea...wil try it out..

#17 2014-11-04 06:33 PM

adwags
Member
Registered: 2014-11-04
Posts: 3

Re: socket recv waiting for some time even if data is available

afxgss...I'm experiencing a similar problem reading from a POSIX socket (RHEL6 x86_64 C++ icpc). Were you able to resolve your issue? If so, I'd love to hear how.

Offline

#18 2014-11-04 09:46 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: socket recv waiting for some time even if data is available

As far as any of us could tell, afxgss's problem was solely due to running on Windows...  So, if you're having some similar problem on Linux, it's very unlikely to be at all related...

Could you describe your problem in more detail?

Offline

#19 2014-11-04 10:40 PM

adwags
Member
Registered: 2014-11-04
Posts: 3

Re: socket recv waiting for some time even if data is available

Hi, sure!  I actually posted over on stack overflow first (based on a recommendation in this thread) and included a link to this thread as the closest thing I could find to the behavior I'm experiencing.  Here's a link...

http://stackoverflow.com/questions/2674 … -on-socket

I'd appreciate any insight you may have.

- Adam

Offline

#20 2014-11-05 01:04 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: socket recv waiting for some time even if data is available

Hmmm...  Not that it will probably change anything, but have you tried ppoll() instead of pselect()?  (Most ideal would be epoll_*(), but apparently there's no "p" version of epoll_wait(), and it seems like you're tied to the timespec representation of the timeout?)

When the app gets in this state, does anything else appear odd with it?  Does lsof (or just "ls /proc/<pid>/fd/") show an unusual amount of open FDs?  (This was just my pure gut first guess: it's leaking FDs somewhere, and you eventually exceded FD_SETSIZE, hence the above suggestion of moving from select to poll...)  Or, is its memory usage growing uncontrollably?  Does anything else it's doing seem to be suffering from the same sort of delays, or does it seem to be JUST the socket I/O?

You mention possible fork()/exec*() issues...  If the socket FD is getting shared between child and parent, and both are trying to do this same select test on it, then yeah that might be the cause...  With multiple waiters on a single socket, either only one will get woken up and notified, or both will but then will fight for CPU time to see who gets to read the data first...

Offline

#21 2014-11-05 07:57 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: socket recv waiting for some time even if data is available

So far there isn't any actual proof that it's blocking in the pselect or recv unnecessarily.
My guess is that it's a bug in either the calling code.

Run it with strace -f, perhaps -ff for different output files, and see how long the system calls actually take.

SOCK_CLOEXEC won't help, it only helps to close the sockets after an execve.

Offline

#22 2014-11-07 04:45 AM

adwags
Member
Registered: 2014-11-04
Posts: 3

Re: socket recv waiting for some time even if data is available

You're right, it wasn't blocking in the select. It was user error (facilitated by a flawed API design on my part) resulting in corruption of the deadline. We used strace and noticed tons of calls to clock_gettime.  Another (third) review of the calling code revealed a programming error resulting in the called code having a reference to corrupt stack data.  Sorry for the false alarm and thanks a lot for the replies.

Offline

#23 2014-11-08 09:58 AM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: socket recv waiting for some time even if data is available

Running your program through Valgrind to check for bad memory accesses is always good, I highly recommend it.

Offline

  • Index
  • » Networking
  • » socket recv waiting for some time even if data is available

Board footer

Powered by FluxBB