4 Jun
2006
4 Jun
'06
9:04 p.m.
Hi!
I am trying to notify a high priority (nice -20 or nice -19) thread from
a realtime thread (from a jack callback to be precise). Of course I want
the realtime thread to not block, but I want the high priority thread to
react as soon as possible. For the first implementation, I used a
condition, and pthread_cond_signal. But as I can't aquire a mutex within
the realtime thread, at least not always (at most with trylock()), this
results in racy code.
Also, it doesn't integrate well with poll(), which the high priority
process should be able to use. So basically I want to switch to a pipe
write() instead of pthread_cond_signal.
Now the question is: am I producing priority inversion, then? I've seen
that the kernel code of pipe write has code like
mutex_lock(PIPE_MUTEX(*inode));
(in fs/pipe.c, linux-2.6.16.16), so can this mutex cause priority
inversion?
However, on the other hand, I examined the code of
glibc-2.3.6/ntpl/sysdeps/pthread, and found in pthread_cond_signal.c:
__pthread_cond_signal (cond)
pthread_cond_t *cond;
{
/* Make sure we are alone. */
lll_mutex_lock (cond->__data.__lock);
/* Are there any waiters to be woken? */
if (cond->__data.__total_seq > cond->__data.__wakeup_seq)
{
/* Yes. Mark one of them as woken. */
++cond->__data.__wakeup_seq;
++cond->__data.__futex;
/* Wake one. */
lll_futex_wake (&cond->__data.__futex, 1);
}
/* We are done. */
lll_mutex_unlock (cond->__data.__lock);
return 0;
}
So it looks like there is some kind of low level mutex locking involved
even when signalling a condition. Can't this also lead to priority
inversion?
So what is the best way to achieve that some lower priority thread will
definitely wakeup from a realtime priority thread, without causing
priority inversion, under recent linux kernels?
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
5 Jun
5 Jun
9:14 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Mon, 2006-06-05 at 01:21 +0200, Stefan Westerfeld wrote:
Hi!
I am trying to notify a high priority (nice -20 or nice -19) thread from
a realtime thread (from a jack callback to be precise). Of course I want
the realtime thread to not block, but I want the high priority thread to
react as soon as possible. For the first implementation, I used a
condition, and pthread_cond_signal. But as I can't aquire a mutex within
the realtime thread, at least not always (at most with trylock()), this
results in racy code.
int pipes[2];
RT-thread: char c;
write (pipes[0], &c, 1);
other-thread: poll (.... on pipes[1] ... and others)
writing to a pipe is not 100% RT safe, but if the pipe is created in a
shm filesystem, its as close to it as you will get without ...
futexes, which are used by current NPTL code for pthread signalling.
they can still block, but they have the benefit that you do not need new
code to use them - just use pthreads calls.
this is why JACK (which uses the pipe model) puts its pipes into a shm
filesystem by the way - not doing so leads to horrible xruns on most
systems because locking the pipe involves interactions with a real
filesystem on disk.
Now the question is: am I producing priority
inversion, then? I've seen
that the kernel code of pipe write has code like
mutex_lock(PIPE_MUTEX(*inode));
(in fs/pipe.c, linux-2.6.16.16), so can this mutex cause priority
inversion?
no, because the high priority thread cannot wake up take the lock - the
RT thread is already holding it.
priority inversion happens when an higher thread attempts to acquire a
lock currently held by a lower priority thread. in the design of a
system based around relatively long time intervals between lock
acquisition (10ths of msecs to many msecs), the high priority thread
will always be acquiring the lock again long after the lower priority
thread has finished with it.
if you want to get all theoretical, then yes, any scheme that involves
interactions between higher and lower priority thread can lead to
priority inversion if there is no system-level help to prevent it. but
in systems built around the data flow inherent in all current audio and
video interfaces, it does not need to be an issue.
if you cannot ensure that the low priority thread will release the lock
long before the high priority needs it again, i'd suggest a redesign of
your code. it will happen more quickly than the kernel acquiring
priority inversion support.
--p
7 Jun
7 Jun
6:13 a.m.
Paul Davis wrote:
writing to a pipe is not 100% RT safe, but if the pipe
is created in a
shm filesystem, its as close to it as you will get without ...
Nowadays, there's also available a very good interface from POSIX RT
extensions; posix message queues. See mq_send(3). You can use either
mq_receive(3) or mq_notify(3) for reception of the message.
I also measured posix message queues to be about twice as fast as for
example local (unix) sockets.
Now that Linux systems generally are pretty close to full POSIX
(IEEE-1003.1) compliance, I don't see any reason why not to make
advantage of it.
- Jussi
10:49 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-06-07 at 11:12 +0300, Jussi Laako wrote:
Paul Davis wrote:
nice to hear that they are faster. on the other hand, once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue.
writing to a pipe is not 100% RT safe, but if the
pipe is created in a
shm filesystem, its as close to it as you will get without ...
Nowadays, there's also available a very good interface from POSIX RT
extensions; posix message queues. See mq_send(3). You can use either
mq_receive(3) or mq_notify(3) for reception of the message.
I also measured posix message queues to be about twice as fast as for
example local (unix) sockets.
Now that Linux systems generally are pretty close to full POSIX
(IEEE-1003.1) compliance, I don't see any reason why not to make
advantage of it.
2:24 p.m.
Paul Davis wrote:
nice to hear that they are faster. on the other hand,
once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue.
POSIX message queues are not very new concept anyway, it dates back to
1993. However, I have to admit that I liked the old QNX concept of
send/receive/reply more.
That's true. On the other hand you can have multiple sending parties on
single queue, so you don't necessarily have to sleep on anything else.
Or you can be signal-like asynchronously woken up when something arrives
on message queue. Imo, generic concept on having to do something like
select/poll on multiple descriptors can be problematic. And you cannot
assign priorities for the descriptor like you can do for messages in
message queue. Most of the real realtime operating systems are
completely based on message queues and message passing.
Message queues need slightly different way of thinking, but on the other
hand, overcome some scheduling related problems when compared to
pipe-like implementations where you don't explicitly specify message
length, nor have priorities.
- Jussi
7:35 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, Jun 07, 2006 at 08:49:38AM -0400, Paul Davis wrote:
nice to hear that they are faster. on the other hand,
once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue.
You can build such a thing on top of condition variables - that
is what they exists for - to let a thread wait one any condition
you may want, no matter how complicated.
It's possible to do this in a very 'clean' way in C++.
First create 'service' classes for all the basic services you need:
message boxes, pipes, counting semaphores, whatever. These classes
should be handling the data only, not synchronisation. Derive them
all from an abstract base class that interfaces their state changes
to a second set of 'synchro' classes. Each of these uses just *one*
condition variable, and that is the thing a thread waits for, by
calling a 'synchro' object's wait().
Since _you_ design the condition in the 'synchro' objects it could
be anything you want, from simple readyness of any element in the
collection of 'service' objects you wait for (similar to poll/select)
up to things such as 'wake me up when all of mailboxes #1, #3 and #6
have some data and sema #5 has at least a count of ten'. You could
design a number of standard 'synchro' classes (as in libclthreads),
and/or create ad-hoc ones when you need them.
The problem then is all the system calls that use file descriptors,
as they don't provide the interface of the 'service' base class.
One solution is to delegate all use of such interfaces to 'helper
threads' that each wait on a single file, socket, or whatever.
You may want this delegation anyway for other reasons, e.g. disk
threads that read/write audio files in the background, or a thread
that receives and decodes OSC commands.
--
FA
Follie! Follie! Delirio vano e' questo!
7:42 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-06-07 at 20:32 +0200, Fons Adriaensen wrote:
On Wed, Jun 07, 2006 at 08:49:38AM -0400, Paul Davis
wrote:
But, from the original post it seems that pthread_cond_signal is not
realtime safe as it locks a mutex:
__pthread_cond_signal (cond)
pthread_cond_t *cond;
{
/* Make sure we are alone. */
lll_mutex_lock (cond->__data.__lock);
/* Are there any waiters to be woken? */
if (cond->__data.__total_seq > cond->__data.__wakeup_seq)
{
/* Yes. Mark one of them as woken. */
++cond->__data.__wakeup_seq;
++cond->__data.__futex;
/* Wake one. */
lll_futex_wake (&cond->__data.__futex, 1);
}
/* We are done. */
lll_mutex_unlock (cond->__data.__lock);
return 0;
}
How can glibc guarantee that we are not put to sleep if there is
contention?
Lee
nice to hear that they are faster. on the other
hand, once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue.
You can build such a thing on top of condition variables - that
is what they exists for - to let a thread wait one any condition
you may want, no matter how complicated.
8:09 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, Jun 07, 2006 at 05:42:26PM -0400, Lee Revell wrote:
But, from the original post it seems that
pthread_cond_signal is not
realtime safe as it locks a mutex:
...
How can glibc guarantee that we are not put to sleep if there is
contention?
The mutex associated with a CV is held only
- by the sender while modifying the condition
- by the receiver while checking the condition
So it is not held by the receiver while it is descheduled and waiting.
Suppose you use a CV from a JACK process callback to tell some other
thread in your app that a period of new samples is now available in a
circular buffer.
There is a _very small_ chance that the mutex you need to take is held
by the receiving thread - this will happen if JACK's thread pre-empted
the receiver at exactly the moment it was checking the condition, i.e.
in between its mutex_lock() and pthread_cond_wait ().
In that case, if you used mutex_lock(), the receiver will take over for
a very short time until it calls pthread_cond_wait(), and you will be
able to continue after that.
If that is not acceptable (i.e. if you have a *very* short period time),
use mutex_try_lock() instead. If it fails, don't do the pthread_cond_signal()
but remember you have to signal two periods next time.
--
FA
Follie! Follie! Delirio vano e' questo!
11:06 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-06-07 at 17:42 -0400, Lee Revell wrote:
On Wed, 2006-06-07 at 20:32 +0200, Fons Adriaensen
wrote:
The big question is: who cares? :) sem_post() is. A mutex/cond pair is
just a big ugly slow non-RT-safe semaphore anyway.
-DR-
On Wed, Jun 07, 2006 at 08:49:38AM -0400, Paul
Davis wrote:
But, from the original post it seems that pthread_cond_signal is not
realtime safe as it locks a mutex: nice to hear that they are faster. on the other
hand, once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue.
You can build such a thing on top of condition variables - that
is what they exists for - to let a thread wait one any condition
you may want, no matter how complicated.
9 Jun
9 Jun
8:17 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Hi!
On Wed, Jun 07, 2006 at 09:04:45PM -0400, Dave Robillard wrote:
On Wed, 2006-06-07 at 17:42 -0400, Lee Revell wrote:
As far as I understand from the linux-2.6.16.16 and glibc-2.3.6 code,
a semaphore has similar problems, as sem_post() is implemented in terms
of futexes on newer linuxes, and futexes have a list of waiters, which
is protected by a lock within the kernel.
But the unlikelyness of a priority inversion problem that has been
pointed out by others probably applies to any of the synchronization
primitives that came up in this thread, be it conditions, pipes or
semaphores.
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
On Wed, 2006-06-07 at 20:32 +0200, Fons
Adriaensen wrote:
The big question is: who cares? :) sem_post() is. A mutex/cond pair is
just a big ugly slow non-RT-safe semaphore anyway. On Wed, Jun 07, 2006 at 08:49:38AM -0400, Paul
Davis wrote:
But, from the original post it seems that pthread_cond_signal is not
realtime safe as it locks a mutex: nice to hear that they are faster. on the other
hand, once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue.
You can build such a thing on top of condition variables - that
is what they exists for - to let a thread wait one any condition
you may want, no matter how complicated.
10 Jun
10 Jun
12:03 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Sat, 2006-06-10 at 00:37 +0200, Stefan Westerfeld wrote:
Hi!
On Wed, Jun 07, 2006 at 09:04:45PM -0400, Dave Robillard wrote:
Well, sem_post is supposed to be async-signal safe (meaning it can't be
preempted, meaning it's realtime safe). If the Linux people have broken
that, they have done an evil stupid thing. POSIX semaphores are great,
especially since you can (or at least should, apparently) signal things
from signal callbacks and realtime threads with them (they work great
for event-driven systems).
Are you sure? Link?
-DR-
On Wed, 2006-06-07 at 17:42 -0400, Lee Revell
wrote:
As far as I understand from the linux-2.6.16.16 and glibc-2.3.6 code,
a semaphore has similar problems, as sem_post() is implemented in terms
of futexes on newer linuxes, and futexes have a list of waiters, which
is protected by a lock within the kernel. On Wed, 2006-06-07 at 20:32 +0200, Fons
Adriaensen wrote:
The big question is: who cares? :) sem_post() is. A mutex/cond pair is
just a big ugly slow non-RT-safe semaphore anyway. On Wed, Jun 07, 2006 at 08:49:38AM -0400, Paul
Davis wrote:
> nice to hear that they are faster. on the other hand, once again POSIX
> screws us all over by not integrating everything into a single blocking
> wait call. i've said it before, i'll say it again - this is one of the
> few things that the win32 API gets right - you can block in one call on
> almost *anything*. AFAICT, you cannot select/poll on a msg queue.
You can build such a thing on top of condition variables - that
is what they exists for - to let a thread wait one any condition
you may want, no matter how complicated.
But, from the original post it seems that pthread_cond_signal is not
realtime safe as it locks a mutex:
19 Jun
19 Jun
6:42 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Hi!
On Fri, Jun 09, 2006 at 10:03:14PM -0400, Dave Robillard wrote:
Well, sem_post is supposed to be async-signal safe (meaning it can't be
preempted, meaning it's realtime safe). If the Linux people have broken
that, they have done an evil stupid thing. POSIX semaphores are great,
especially since you can (or at least should, apparently) signal things
from signal callbacks and realtime threads with them (they work great
for event-driven systems).
Are you sure? Link?
The userspace code of sem_post() can be found in
glibc-2.3.6/nptl/sysdeps/unix/sysv/linux/sem_post.c:
int
__new_sem_post (sem_t *sem)
{
int *futex = (int *) sem;
int nr = atomic_increment_val (futex);
int err = lll_futex_wake (futex, nr);
if (__builtin_expect (err, 0) < 0)
{
__set_errno (-err);
return -1;
}
return 0;
}
There are assembler versions, but they also need the futex_wake syscall,
so it doesn't matter which version you read.
And here is the kernel code from linux-2.6.16.16/kernel/futex.c:
static int futex_wake(unsigned long uaddr, int nr_wake)
{
union futex_key key;
struct futex_hash_bucket *bh;
struct list_head *head;
struct futex_q *this, *next;
int ret;
down_read(¤t->mm->mmap_sem);
ret = get_futex_key(uaddr, &key);
if (unlikely(ret != 0))
goto out;
bh = hash_futex(&key);
spin_lock(&bh->lock);
head = &bh->chain;
list_for_each_entry_safe(this, next, head, list) {
if (match_futex (&this->key, &key)) {
wake_futex(this);
if (++ret >= nr_wake)
break;
}
}
spin_unlock(&bh->lock);
out:
up_read(¤t->mm->mmap_sem);
return ret;
}
The spin_lock(&bh->lock) is the one I was referring to in my earlier
mail.
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
The big question is: who cares? :) sem_post() is. A
mutex/cond pair is
just a big ugly slow non-RT-safe semaphore anyway.
As far as I understand from the linux-2.6.16.16 and glibc-2.3.6 code,
a semaphore has similar problems, as sem_post() is implemented in terms
of futexes on newer linuxes, and futexes have a list of waiters, which
is protected by a lock within the kernel.
6:48 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Mon, 2006-06-19 at 23:11 +0200, Stefan Westerfeld wrote:
The spin_lock(&bh->lock) is the one I was
referring to in my earlier
mail.
Taking a spinlock should be realtime safe - they are not supposed to be
held for long and cannot sleep. Worst case scenario should be that you
spin for a few hundred microseconds. It's a bug if a spinlock is held
for much longer or is heavily contended.
Lee
6:32 p.m.
On Thu, 2006-06-08 at 23:25 +0300, Jussi Laako wrote:
Lee Revell wrote:
The code in question was fron glibc, not the kernel.
Lee
But, from the original post it seems that
pthread_cond_signal is not
realtime safe as it locks a mutex:
Just about any syscall nowadays potentially acquires some sort of lock
inside kernel. 
3:56 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
* Fons Adriaensen:
You can build such a thing on top of condition
variables - that
is what they exists for - to let a thread wait one any condition
you may want, no matter how complicated.
But don't you need an additional context switch, from the event
listener thread to the actual processing thread? You could use a
leader/followers scheme to deal with this, I think, but it's quite
complicated. Doing it in the kernel would probably reduce overall
complexity.
3:10 p.m.
[Paul Davis]
On Wed, 2006-06-07 at 11:12 +0300, Jussi Laako wrote:
actually, according to my copy of
/usr/src/linux/ipc/mqueue.c
and
http://laurel.datsi.fi.upm.es/web/cgi-bin/man/man2html?7+mq_overview
linux takes us out of the posix stone age and implements the poll
syscall on message queues (which are simple file descriptors to a
linux application). thanks kernel guys!
portability questions aside, it remains the question whether mqd_t is
significantly faster than an ordinary pipe when used with poll().
cheers, tim
Paul Davis wrote:
nice to hear that they are faster. on the other hand, once again POSIX
screws us all over by not integrating everything into a single blocking
wait call. i've said it before, i'll say it again - this is one of the
few things that the win32 API gets right - you can block in one call on
almost *anything*. AFAICT, you cannot select/poll on a msg queue. writing to a pipe is not 100% RT safe, but if the
pipe is created in a
shm filesystem, its as close to it as you will get without ...
Nowadays, there's also available a very good interface from POSIX RT
extensions; posix message queues. See mq_send(3). You can use either
mq_receive(3) or mq_notify(3) for reception of the message.
I also measured posix message queues to be about twice as fast as for
example local (unix) sockets.
Now that Linux systems generally are pretty close to full POSIX
(IEEE-1003.1) compliance, I don't see any reason why not to make
advantage of it.
11 Jul
11 Jul
7:06 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Mon, 2006-06-05 at 01:21 +0200, Stefan Westerfeld wrote:
Hi!
I am trying to notify a high priority (nice -20 or nice -19) thread from
a realtime thread (from a jack callback to be precise). Of course I want
the realtime thread to not block, but I want the high priority thread to
react as soon as possible. For the first implementation, I used a
condition, and pthread_cond_signal. But as I can't aquire a mutex within
the realtime thread, at least not always (at most with trylock()), this
results in racy code.
Also, it doesn't integrate well with poll(), which the high priority
process should be able to use. So basically I want to switch to a pipe
write() instead of pthread_cond_signal.
I seem to be alone on this, but I still say semaphores are the best for
this - sem_post is async signal safe. It might not be pedantically
"realtime safe" (depending what the Linux guys have done with futexes I
guess) but it's certainly far better than a mutex/cond pair (and I would
guess better than writing to a pipe as well). A mutex obviously isn't
async signal safe, and I doubt writing to a pipe is either.
It's also handy that a semaphore is a counter, so you can easily
gracefully handle the case where the realtime thread is going a lot
faster than the consumer, or the consumer gets hung up for a little bit,
etc.
Semaphores seem about perfect for this to me.. am I missing something?
Why doesn't anyone ever recommend them?
-DR-
7:09 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Tue, 2006-07-11 at 17:06 -0400, Dave Robillard wrote:
Semaphores seem about perfect for this to me.. am I
missing something?
Why doesn't anyone ever recommend them?
i think mostly because in 2000-2001, they were very slow.
12 Jul
12 Jul
7:46 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Tue, Jul 11, 2006 at 05:10:27PM -0400, Paul Davis wrote:
On Tue, 2006-07-11 at 17:06 -0400, Dave Robillard
wrote:
IMHO they are still slow, especially when you port software to OSX then
pthreads and semaphores are *very* slow (well, it depends how much and
where you use them of course).
my solution so far is assuming that boolean is atomical.
all multi threaded handling i wrote is based on this assumption: i use
it in pipe and linklist classes, but semaphores could also be there.
i found no probems and good speed so far
... and life is boring without risks :))
ciao
- --
jaromil, dyne.org rasta coder, http://rastasoft.org
Semaphores seem about perfect for this to me.. am
I missing something?
Why doesn't anyone ever recommend them?
i think mostly because in 2000-2001, they were very slow. -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFEtW0Je2QxhLU0C14RAgoFAJ46wjWQ1TQcuisiWAGlNUe/r+4q0wCeKbhc
1LwvuKrz6u5FrljSDS6J4F0=
=RTwn
-----END PGP SIGNATURE-----
7:55 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-07-12 at 23:43 +0200, jaromil wrote:
my solution so far is assuming that boolean is
atomical.
all multi threaded handling i wrote is based on this assumption: i use
it in pipe and linklist classes, but semaphores could also be there.
i found no probems and good speed so far
... and life is boring without risks :))
How does that help with IPC? I thought this thread was about a realtime
safe IPC mechanism?
Lee
13 Jul
13 Jul
9:40 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Wed, Jul 12, 2006 at 05:56:00PM -0400, Lee Revell wrote:
On Wed, 2006-07-12 at 23:43 +0200, jaromil wrote:
the approach i'm using is multi-threading and shared memory: in critical
loops messages are passed as booleans (but i keep #defines to change
them into mutexes if necessary).
in fact i'm a bit off topic since i don't use multiple processes; for
that i can confirm that poll(2) is the most efficient and portable
method i found.
just my 2 pesetas :)
ciao
- --
jaromil, dyne.org rasta coder, http://rastasoft.org
my solution so far is assuming that boolean is
atomical.
all multi threaded handling i wrote is based on this assumption: i use
it in pipe and linklist classes, but semaphores could also be there.
i found no probems and good speed so far
... and life is boring without risks :))
How does that help with IPC? I thought this thread was about a realtime
safe IPC mechanism? -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFEtjBne2QxhLU0C14RAi9gAJ4mEDl8MtS8MPjqNtyjt/6cm96HlwCcCAtm
8pyuZA70O2BtDPW8J0nVcRY=
=mMJH
-----END PGP SIGNATURE-----
12 Jul
12 Jul
9:55 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-07-12 at 23:43 +0200, jaromil wrote:
OSX? Although I can see your argument (of convenience), it has no
beaeing to the implementation in linux. If you say that Darwin is
lacking, then fix it or use Linux, no?
i think mostly
because in 2000-2001, they were very slow.
IMHO they are still slow, especially when you port software to OSX then
pthreads and semaphores are *very* slow (well, it depends how much and
where you use them of course).
my solution so far is assuming that boolean is atomical.
all multi threaded handling i wrote is based on this assumption: i use
it in pipe and linklist classes, but semaphores could also be there.
i found no probems and good speed so far
... and life is boring without risks :))
... and sooner or later we will find an update for
multi-processors. :-D
ciao
cheers!
13 Jul
13 Jul
9:45 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Thu, Jul 13, 2006 at 01:55:25AM +0200, Jens M Andreasen wrote:
eheh, well, amazingly enough, i have no problems on SMP at all! :>
ciao
- --
jaromil, dyne.org rasta coder, http://rastasoft.org
On Wed, 2006-07-12 at 23:43 +0200, jaromil wrote:
OSX? Although I can see your argument (of convenience), it has no
beaeing to the implementation in linux. If you say that Darwin is
lacking, then fix it or use Linux, no?
i use Linux of course. i also dislike Darwin and would never spend time
fixing it for Apple; but many (l)users are using OSX and is somehow
interesting to port applications to it.
i think
mostly because in 2000-2001, they were very slow.
IMHO they are still slow, especially when you port software to OSX
then pthreads and semaphores are *very* slow (well, it depends how
much and where you use them of course). my solution so
far is assuming that boolean is atomical. all multi
threaded handling i wrote is based on this assumption: i use it in
pipe and linklist classes, but semaphores could also be there.
i found no probems and good speed so far
... and life is boring without risks :))
... and sooner or later we will find an update for multi-processors.
:-D -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFEtjGQe2QxhLU0C14RAkOIAKDvVgo5yuxC+LOm6eDA3YARCfm8VACguAw8
cUF+BBn9f3Ls5Ui5owXrtrE=
=pjM8
-----END PGP SIGNATURE-----
11 Jul
11 Jul
8:21 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Hi!
On Tue, Jul 11, 2006 at 05:06:03PM -0400, Dave Robillard wrote:
On Mon, 2006-06-05 at 01:21 +0200, Stefan Westerfeld
wrote:
Well after Lee's reply, and looking at the code again, I think I think
the Linux guys did everything right, and semaphores should be realtime
safe.
I am trying to notify a high priority (nice -20
or nice -19) thread from
a realtime thread (from a jack callback to be precise). Of course I want
the realtime thread to not block, but I want the high priority thread to
react as soon as possible. For the first implementation, I used a
condition, and pthread_cond_signal. But as I can't aquire a mutex within
the realtime thread, at least not always (at most with trylock()), this
results in racy code.
Also, it doesn't integrate well with poll(), which the high priority
process should be able to use. So basically I want to switch to a pipe
write() instead of pthread_cond_signal.
I seem to be alone on this, but I still say semaphores are the best for
this - sem_post is async signal safe. It might not be pedantically
"realtime safe" (depending what the Linux guys have done with futexes I
guess) but it's certainly far better than a mutex/cond
pair (and I would
guess better than writing to a pipe as well). A mutex obviously isn't
async signal safe, and I doubt writing to a pipe is either.
It's also handy that a semaphore is a counter, so you can easily
gracefully handle the case where the realtime thread is going a lot
faster than the consumer, or the consumer gets hung up for a little bit,
etc.
Semaphores seem about perfect for this to me.. am I missing something?
Since we've been comparing different methods here, I thought I might as
well write a benchmark, to look at the performance, too. I wrote a
little test which repeatedly switches between two threads, which wakeup
eachother using a pipe, cond or semaphore.
On an AMD64 3400+ (2200 MHz) running a 2.6.16.16 kernel with preemption
enabled, the timings for 1000000 iterations (each thread runs a brief
period of time 1000000 times) are
- about 5.7 seconds when using a wakeup pipe and poll
- about 5.7 seconds when using a condition with mutex
- about 2.0 seconds when using a semaphore
So: if what you're doing doesn't restrict you in any way, then
semaphores are probably the thing to use.
If you need to wait for multiple things simultaneously (like audio
device fd and another thread), then you can do it with pipes and poll,
but not with semaphores.
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
9:43 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-07-12 at 00:26 +0200, Stefan Westerfeld wrote:
Nice. I win! :) I'd noticed a set of the pants improvement (increased
event processing throughput) when I switch to semaphores, but never ran
a real benchmark.
Good point on waiting for multiple things though, poll is a lot more
flexible than semaphores (though I still say writing to a pipe in a
realtime thread is pretty sketchy...). Pipes let you communicate
between processes though - I havn't tried the fancier POSIX interprocess
stuff yet.
-DR-
Semaphores
seem about perfect for this to me.. am I missing something?
Since we've been comparing different methods here, I thought I might as
well write a benchmark, to look at the performance, too. I wrote a
little test which repeatedly switches between two threads, which wakeup
eachother using a pipe, cond or semaphore.
On an AMD64 3400+ (2200 MHz) running a 2.6.16.16 kernel with preemption
enabled, the timings for 1000000 iterations (each thread runs a brief
period of time 1000000 times) are
- about 5.7 seconds when using a wakeup pipe and poll
- about 5.7 seconds when using a condition with mutex
- about 2.0 seconds when using a semaphore
So: if what you're doing doesn't restrict you in any way, then
semaphores are probably the thing to use.
If you need to wait for multiple things simultaneously (like audio
device fd and another thread), then you can do it with pipes and poll,
but not with semaphores.
12 Jul
12 Jul
3:54 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Dave Robillard wrote:
realtime thread is pretty sketchy...). Pipes let you
communicate
between processes though - I havn't tried the fancier POSIX interprocess
stuff yet.
What do you mean by semaphore then, if not sem_*()?
sem_post(3)/sem_wait(3) are defined in POSIX realtime extensions (librt
in glibc). I would recommend _NOT_ to mix these semaphores (or shared
memory, or message queues) with anything from SysV which is usually
veeery far from being realtime safe, from implementation point of view.
IF the implementation happens to be the same in Linux, doesn't mean
that's the case in any other POSIX system.
Of course you can share semaphores between processes because the
semaphores reside within common namespace (see sem_open(3)).
- Jussi
P.S. There's also nice sem_overview(7) man page nowadays.
13 Jul
13 Jul
2:13 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
On Wed, 2006-07-12 at 20:54 +0300, Jussi Laako wrote:
Dave Robillard wrote:
Sorry, I /was/ referring to sem_post/sem_wait/etc. I mean havn't tried
any interprocess posix signalling stuff, like named semaphores or
message queues.
Benchmarking message queues against pipes would be interesting, maybe
Jack could benefit if they're faster?
-DR-
realtime thread is pretty sketchy...). Pipes let
you communicate
between processes though - I havn't tried the fancier POSIX interprocess
stuff yet.
What do you mean by semaphore then, if not sem_*()?
sem_post(3)/sem_wait(3) are defined in POSIX realtime extensions (librt
in glibc). I would recommend _NOT_ to mix these semaphores (or shared
memory, or message queues) with anything from SysV which is usually
veeery far from being realtime safe, from implementation point of view.
IF the implementation happens to be the same in Linux, doesn't mean
that's the case in any other POSIX system.
4:15 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Dave Robillard wrote:
Sorry, I /was/ referring to sem_post/sem_wait/etc. I
mean havn't tried
OK :)
Benchmarking message queues against pipes would be
interesting, maybe
Jack could benefit if they're faster?
I did some benchmarking between unix (local) sockets and message queues
and found speed (roundtrip) to be over twice as fast when using message
queues.
In test, process A sends message "Yoohoo" to process B and process B
replies "2U2" back. This is counted as a message exchange in the result.
On my P4/3000 I get this kind of number:
105193.5 messages / second
Nice feature of message queues is that you can get signal-style delivery
using mq_notify(3).
- Jussi
12 Jul
12 Jul
6:20 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Hi Stefan,
On Wed, 2006-07-12 at 00:26 +0200, Stefan Westerfeld wrote:
I wrote a
little test which repeatedly switches between two threads, which wakeup
eachother using a pipe, cond or semaphore.
Nice :) Can you upload it somewhere?
Cheers,
--
Andy Wingo
http://wingolog.org/
10:18 p.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Hi!
On Wed, Jul 12, 2006 at 10:20:40AM +0200, Andy Wingo wrote:
On Wed, 2006-07-12 at 00:26 +0200, Stefan Westerfeld
wrote:
Yes, you can get it here:
http://space.twc.de/~stefan/download/thread_switch_benchmark.tar.bz2
Originally I didn't write the code with the idea in mind that others
may want to use it, without having beasts birnet library available. But
I did some slightly hackish things to make it (hopefully) self-contained
now.
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
I wrote a
little test which repeatedly switches between two threads, which wakeup
eachother using a pipe, cond or semaphore.
Nice :) Can you upload it somewhere? 
13 Jul
13 Jul
7:18 a.m.
New subject: [linux-audio-dev] realtimeness: pthread_cond_signal vs. pipe write
Le 13 juil. 06 à 02:24, Stefan Westerfeld a écrit :
Hi!
On Wed, Jul 12, 2006 at 10:20:40AM +0200, Andy Wingo wrote:
There is code for real-time IPC in Jackdmp (http://www.grame.fr/
~letz/jackdmp.html)
- FIFO in JackFifo.cpp,h files
- POSIX semaphore in JackPosixSemaphore.cpp,h files
- Mach semaphore (OSX) in JackMachSemaphore.cpp,h files
- Event (Windows) in JackWinEvent.cpp,h files
Also code for timing bench in testSynchroClient.cpp,
testSynchroServer.cpp and testSynchroServerClient.cpp
Stephane
On Wed, 2006-07-12 at 00:26 +0200, Stefan
Westerfeld wrote:
Yes, you can get it here:
http://space.twc.de/~stefan/download/thread_switch_benchmark.tar.bz2
Originally I didn't write the code with the idea in mind that others
may want to use it, without having beasts birnet library available.
But
I did some slightly hackish things to make it (hopefully) self-
contained
now.
Cu... Stefan
-- I wrote a
little test which repeatedly switches between two threads, which
wakeup
eachother using a pipe, cond or semaphore.
Nice :) Can you upload it somewhere? 
12 Jul
12 Jul
10:12 a.m.
On 7/11/06, Stefan Westerfeld <stefan(a)space.twc.de> wrote:
Since we've been comparing different methods here,
I thought I might as
well write a benchmark, to look at the performance, too. I wrote a
little test which repeatedly switches between two threads, which wakeup
eachother using a pipe, cond or semaphore.
On an AMD64 3400+ (2200 MHz) running a 2.6.16.16 kernel with preemption
enabled, the timings for 1000000 iterations (each thread runs a brief
period of time 1000000 times) are
- about 5.7 seconds when using a wakeup pipe and poll
- about 5.7 seconds when using a condition with mutex
- about 2.0 seconds when using a semaphore
So: if what you're doing doesn't restrict you in any way, then
semaphores are probably the thing to use.
If you need to wait for multiple things simultaneously (like audio
device fd and another thread), then you can do it with pipes and poll,
but not with semaphores.
Cu... Stefan
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
What about POSIX real-time signals? I know mixing threads and signals
is taboo, but I thought I'd throw it out there anyway. Using pselect
then lets you wait atomically for events on file descriptors and/or
signals. Not sure how well it works with threads, though...or if it's
even implemented (though it is described in select(2)).
6490
days inactive
6529
days old
linux-audio-dev@lists.linuxaudio.org
32 comments
13 participants
participants (13)
-
Andy Wingo -
Dave Robillard -
Florian Weimer -
Fons Adriaensen -
jaromil -
Jens M Andreasen -
Jussi Laako -
Lee Revell -
Paul Coccoli -
Paul Davis -
Stefan Westerfeld -
Stéphane Letz -
Tim Goetze