15 May
2003
15 May
'03
7:15 p.m.
On linux, which is faster, pipe, FIFO, or socket? What about shared
memory - is it faster, and if so is it faster enough to warrant the
extra programming overhead?
--
Hans Fugal | De gustibus non disputandum est.
http://hans.fugal.net/ | Debian, vim, mutt, ruby, text, gpg
http://gdmxml.fugal.net/ | WindowMaker, gaim, UTF-8, RISC, JS Bach
---------------------------------------------------------------------
GnuPG Fingerprint: 6940 87C5 6610 567F 1E95 CB5E FC98 E8CD E0AA D460
16 May
16 May
12:30 a.m.
On linux, which is faster, pipe, FIFO, or socket? What
about shared
memory - is it faster, and if so is it faster enough to warrant the
extra programming overhead?
depends what you want to do.
for communication that can wake another thread/process, shared memory
is not an option: there is no way to "wait" on shared memory. for
that, a pipe is notably faster than a tcp socket and a bit faster than
a unix socket. for actual data movement, shared memory beats
everything hands down since its zero copy.
linux pipes are about the fast IPC on any operating system so far,
although i have heard good things about futexes in kernel 2.5/2.6.
--p
7:48 a.m.
Hi there,
On Thu, May 15, 2003 at 10:23:56PM -0400, Paul Davis wrote:
linux pipes are about the fast IPC on any operating
system so far,
although i have heard good things about futexes in kernel 2.5/2.6.
I can second that. Pipes are a damn fast on Linux. And the copy
is good for most applications, because it decouples the processes
and thus reduces communication. It also simplifies programming a
lot.
If your programming model cannot make use of pipelines or you
need very deep ones (in terms of bytes), then the standard pipes
make no sense.
And futexes are available on RH9.0 also for 2.4.x.
But futexes need a new glibc with the new POSIX threading library
(NPTL) included. On closed systems there is another solution with
a special futex-library.
But this solution should only be used for embedded audio
applications or just for fun.
Futexes are hidden behind the standard POSIX threading API. So if
people program along POSIX, it will just be slower on non-futex-
enabled kernels, but fast on future kernels.
Regards
Ingo Oeser
8:51 a.m.
[...]
for communication that can wake another
thread/process, shared memory
is not an option: there is no way to "wait" on shared memory. for
that, a pipe is notably faster than a tcp socket and a bit faster than
a unix socket. for actual data movement, shared memory beats
everything hands down since its zero copy.
linux pipes are about the fast IPC on any operating system so far,
although i have heard good things about futexes in kernel 2.5/2.6.
Has anyone tried using realtime signals instead of a pipe for this?
--ms
9:53 a.m.
[...]
there seem to be mixed reports on the implementation of rtsigs in the
kernel. i've seen stories from some people saying they don't work,
and/or they don't work correctly.
anyway, they are inherently slower than a pipe, because the pipe model
doesn't require creation and use of a signal handling stack for the
task. the task is already blocked, and you just wake it up. with any
signal (including an rtsignal) you have to setup the signal call
stack first.
there is still, as i've mentioned before and as someone even posted a
kernel patch (here) for, the idea of pipes that are not protected by
fs locks, which would speed them up even more.
for communication that can wake another
thread/process, shared memory
is not an option: there is no way to "wait" on shared memory. for
that, a pipe is notably faster than a tcp socket and a bit faster than
a unix socket. for actual data movement, shared memory beats
everything hands down since its zero copy.
linux pipes are about the fast IPC on any operating system so far,
although i have heard good things about futexes in kernel 2.5/2.6.
Has anyone tried using realtime signals instead of a pipe for this?
10:25 a.m.
[...]
anyway, they are inherently slower than a pipe,
because the pipe model
doesn't require creation and use of a signal handling stack for the
task. the task is already blocked, and you just wake it up. with any
signal (including an rtsignal) you have to setup the signal call
stack first.
Also when using sigwait()?
--ms
10:39 a.m.
[...]
probably depends on the thread library implementation. sigwait in
linuxthreads is just a complete hack.
i never tried rtsignals, but the first implementation of JACK tried to
use signals for IPC, and i found it 3-5 times slower than using pipes.
anyway, they are inherently slower than a pipe,
because the pipe model
doesn't require creation and use of a signal handling stack for the
task. the task is already blocked, and you just wake it up. with any
signal (including an rtsignal) you have to setup the signal call
stack first.
Also when using sigwait()?
1:17 p.m.
Hi there,
On Fri, May 16, 2003 at 08:33:41AM -0400, Paul Davis wrote:
probably depends on the thread library implementation.
sigwait in
linuxthreads is just a complete hack.
Yes, that's the problem. Esp. for a threaded environment, as
signals are a dark chapter in POSIX threading.
i never tried rtsignals, but the first implementation
of JACK tried to
use signals for IPC, and i found it 3-5 times slower than using pipes.
Don't use signals together with threads, as the linux threading
libraries are not the only ones with bugs in this area.
So I can again confirm your experiences Paul.
Regards
Ingo Oeser
7:13 a.m.
Hi Hans,
On Thu, May 15, 2003 at 02:59:25PM -0600, Hans Fugal wrote:
On linux, which is faster, pipe, FIFO, or socket? What
about shared
memory - is it faster, and if so is it faster enough to warrant the
extra programming overhead?
This depends on how much data you usally transfer at a time, the
kernel used and the machine under it.
LMBench will show you the differences per system on any system
where it compiles. Just google for it (Author is Larry McVoy).
The decision you have to make is, whether a copy will take longer
than the synchronisation overhead. Don't forget the sleep/wakeup
overhead.
Regards
Ingo Oeser
1 Jun
1 Jun
6:14 a.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Thu, May 15, 2003 at 02:59:25PM -0600, Hans Fugal wrote:
On linux, which is faster, pipe, FIFO, or socket? What
about shared
memory - is it faster, and if so is it faster enough to warrant the
extra programming overhead?
FIFO pipes are a good choice.
i use them in a multithreaded environment, accessed by different threads
asynchronously, both in blocking/nonblocking modes, for read/write
operations; concurrent memory access is avoided using simple mutexes.
i did a small C++ class out of that, working on a codebase written by
Charles Samuels a while ago. My modifications on the original code
emphatize on speed and stability. Right now is very well debugged.
it is GNU GPL and you'll find it in the pipe.cpp and pipe.h files of
MuSE Streamer http://muse.dyne.org
ciao
- --
jaromil, dyne.org unix programmer, http://korova.dyne.org
< spring unveils cryptosexual anarchism in random forms >
dyne:bolic liveCD http://dynebolic.org | dyne.org productions
MuSE Streamer v0.8.1 http://muse.dyne.org | fugbachgasse 12 / 18
FreeJ v0.5 http://freej.org | 1020 Vienna, AUSTRIA
FARAH in Palestine http://farah.dyne.org | tel: +43 1 92 56 318
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
iD8DBQE+2bNIWLCC1ltubZcRAsEZAJ9HKaauMKJ5YlGcENFpCRuYkMDqFgCeJRIh
uMVxQIeM/5SIGFvF2ou++Wo=
=DxI4
-----END PGP SIGNATURE-----
7874
days inactive
7891
days old
linux-audio-dev@lists.linuxaudio.org
9 comments
5 participants
participants (5)
-
Hans Fugal -
Ingo Oeser -
jaromil -
Martijn Sipkema -
Paul Davis