UNIX Socket FAQ

A forum for questions and answers about network programming on Linux and all other Unix-like systems

You are not logged in.

#1 2008-12-19 11:15 AM

Linuxhippy
Member
Registered: 2008-12-19
Posts: 3

Re: Unix Domain Sockets and writev questions

Hello,

I am currently using writev through the xcb library to communicate with the XServer directly.
I had quite a hard time finding informations about how writev works, would be cool if somebody could answer me the following questions:

1.) I somewhere read that there is a maximum per-writev size that may
submitted, otherwise glibc will allocate a temporary buffer. How large
is that size?
What are the conditions needed to make writev work with the least possible overhead?

2.) Does xcb_writev copy or block or even both? I have a dual-core
system but for now it seems when the x-server is processing commands
the client has to wait and vice-versa - so only one core is fully
loaded.
What would be needed to have producer and consumer processes to run simultaneously?

3.) Is it possible to use zero-copy with writev, and if so - for which sizes it starts to make sence?

Sorry for these newbie-style questions and thanks a lot in advance, Clemens

Offline

#2 2008-12-19 02:08 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,826
Website

Re: Unix Domain Sockets and writev questions

I somewhere read that there is a maximum per-writev size that may
submitted, otherwise glibc will allocate a temporary buffer. How large
is that size?

Actually, at least modern glibc will always try to hand the full iovec array off to the
kernel writev() syscall, and only if it fails will it fall back on doing the temp buffer
thing to emulate writev() on top of plain old write()...  And, I believe that Linux's true
internal IOV_MAX is 1024, so the syscall shouldn't fail unless you have more iovecs
than that...  However, there's also a bit of internal kernel optimization as well: if you
have 8 or less iovecs in your array, it will just use an on-stack array for them in the
kernel; if you have more than that, it needs to allocate some extra kernel memory to
hold them all...  So, your best bet is to stick with 8 or less, if you can...

As for question #2, I don't know anything about xcb, so I can't help you there...

And, for #3, I don't think there's any way of doing zero copy...  Unless maybe your
iovecs happened to point to something like an mmap()'d file or something, anyway...
Then, the kernel may be smart enough to pull the data straight out of its own file
cache instead of copying from user-space...  (Or may not, I'm not really sure...)

Offline

#3 2008-12-20 02:16 AM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Unix Domain Sockets and writev questions

As for 2 (keep in mind that I never used xcb):

XCB  is designed to be asynchronous, so X and your program should be able
to both run at the same time. That said, some synchronisation can happen,
perhaps for certain commands, or you ask for it explicitly somewhere.

Basically your program shouldn't wait for replies or answers from the X
server and then it should work as expected I think. So try finding out where
and why your program blocks, e.g. by examining strace output.

As for 3: zero-copy only makes sense for huge amounts of data, and even
then it's in general not worth the trouble. Writev doesn't do zero-copy, only
sendfile() does. Closest thing to zero-copy is using mmap() on a file +
write/writev/send/etc. But in general you want to have buffering, which
rules out zero-copy.

Offline

#4 2008-12-21 12:06 PM

Linuxhippy
Member
Registered: 2008-12-19
Posts: 3

Re: Unix Domain Sockets and writev questions

Thanks a lot for all the comments.

I always thought writev would copy all my data/payload into a large buffer if I have too many iovecs or so. Allocation of temoprary iovec's isn't a concern for me.

Hmm, one thing is not completly clear to me.
Lets forget xcb for the moment, just a simple producer (my client) and consumer (xserver) szenario.

When my client calls writev(), will it block immediatly until the server as consumed all the data I sent to it? Or will my data be buffered by the kernel, and writev() will only block once the buffer is full?
If so how large is that buffer?

Thanks again, Clemens

Offline

#5 2008-12-22 04:59 AM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,230

Re: Unix Domain Sockets and writev questions

It will be buffered and only block when the buffer is full. Or more likely,
return a short write and block if you call it again to send the rest. In the
case of Unix domain sockets, if the receiver is blocked and waiting for
more data, the kernel might copy the data directly to its destination, but
that's an optimisation and for all practical purposes everything you send
is being buffered.

As for the buffer size, it's not very important, except if you send huge
messages that must be read wholly before they can be processed. In that
case you can try to increase it by setting SO_SNDBUF with setsockopt().
You can get the current value with getsockopt. The default value is
/proc/sys/net/core/wmem_default, which should be plenty big as it is.

If you want to know if your program is blocking in writev() either look at
strace output or use a non-blocking socket and check for EAGAIN.

Offline

#6 2008-12-29 12:40 AM

Linuxhippy
Member
Registered: 2008-12-19
Posts: 3

Re: Unix Domain Sockets and writev questions

Thanks a lot :)

- Clemens

Offline

#7 2009-08-15 03:16 PM

0788629219
Guest

Re: Unix Domain Sockets and writev questions

Board footer

Powered by FluxBB