17 Jun
2004
17 Jun
'04
8:59 a.m.
Hi.
I am currently trying to figure out how to restructure the code
of my little yatm tool to be flexible and extensible in the future.
Basically, I'd like to be able to become a JACK client at
some point in the future. Since JACK uses a poll model, I have
to majorly restructure my code. I think what I need is the following:
Three threads, one for the UI, one for the MPEG/Vorbis/Speex decoder,
and one for audio output (to either libao or JACK).
I'll also need a buffer between the decoder and the audio thread, and some
way to signal the decoder thread that more audio data is needed.
Additionally, I'd like to have the ability to make the buffer
in between remember the last n seconds of audio data and make
it possible to jump back in time <=n seconds during playback.
I've looked at the lock-free ringbuffer of JACK, but I am not
sure if this is really what I need here. I am very new to
linux audio programming, and especially to pthreads (wrote
my first thread using program two days ago).
I'd be very happy if some of you experts here could give me
a hand here and point me into the right direction.
Maybe there is already code out there which does roughly that
which I can look at?
Or maybe you can suggest which mechanisms of pthreads I should
be using for this kind of scenario.
--
CYa,
Mario | Debian Developer <URL:http://debian.org/>
| Get my public key via finger mlang(a)db.debian.org
| 1024D/7FC1A0854909BCCDBE6C102DDFFC022A6B113E44
17 Jun
17 Jun
2:39 p.m.
At Thu, 17 Jun 2004 12:50:06 +0200,
Mario Lang wrote:
I am currently trying to figure out how to restructure the code
of my little yatm tool to be flexible and extensible in the future.
Basically, I'd like to be able to become a JACK client at
some point in the future. Since JACK uses a poll model, I have
to majorly restructure my code. I think what I need is the following:
Three threads, one for the UI, one for the MPEG/Vorbis/Speex decoder,
and one for audio output (to either libao or JACK).
You actually probably only need two threads, one for the UI and one for
audio (more on this below).
Additionally, I'd like to have the ability to make
the
buffer in between remember the last n seconds of audio data and make
it possible to jump back in time <=n seconds during playback.
I think you could just change the value of the position indicator to
have it point that many frames back in time.
I've looked at the lock-free ringbuffer of JACK,
but I am not
sure if this is really what I need here.
The lock-free ringbuffer is necessary to communicate to the audio
thread in an RT-safe manner. RT doesn't really mean "goes fast" so
much as it means "has a guaranteed minimum speed." If you communicate
with the JACK thread just by mutex-protected global variables, that
guarantee becomes difficult to make. This is because the JACK thread
will have to lock each mutex to check the value, and there is a
possibility that it might end up waiting an indeterminate amount of
time to get that lock if your UI thread has it. If you use multiple
variables protected this way, things get worse.
So basically, if you need to tell your JACK thread something, you need
to do it with a ringbuffer.
What I'd recommend doing is creating a UI thread and having it
communicate to the audio thread via the ringbuffer. The audio thread
would have some setting indicating a chunk of memory or file
(whatever) to read sound data from, in addition to settings on how to
read it and where it currently is within the file. You would adjust
these settings by checking for messages from the UI in the ringbuffer
at the beginning of each callback invocation.
After checking for messages, just mixdown the number of frames
requested into the provided buffer, transforming it in accordance with
thread's current settings.
Hope that helps,
-Pete
4:17 p.m.
Pete Bessman <ninjadroid(a)gazuga.net> writes:
At Thu, 17 Jun 2004 12:50:06 +0200,
Mario Lang wrote:
I guess what I don't understand is where to do the reading/decoding.
I can't do it in the audio thread since read(2) might block,
and, especially since one of the libraries I use for decoding is
callback driven. I'm working with potentially very long sound files,
so I can't pre-decode them all into memory in advance either. I think for this
to work, I'd need some prebuffering in between, and the audio thread somehow
singaling to the decoder thread when the buffer is too empty to safely
continue sleeping so that the decoder can start to fill more data
into the buffer.
--
CYa,
Mario
I've looked at the lock-free ringbuffer of
JACK, but I am not
sure if this is really what I need here.
The lock-free ringbuffer is necessary to communicate to the audio
thread in an RT-safe manner. RT doesn't really mean "goes fast" so
much as it means "has a guaranteed minimum speed." If you communicate
with the JACK thread just by mutex-protected global variables, that
guarantee becomes difficult to make. This is because the JACK thread
will have to lock each mutex to check the value, and there is a
possibility that it might end up waiting an indeterminate amount of
time to get that lock if your UI thread has it. If you use multiple
variables protected this way, things get worse.
So basically, if you need to tell your JACK thread something, you need
to do it with a ringbuffer.
What I'd recommend doing is creating a UI thread and having it
communicate to the audio thread via the ringbuffer. The audio thread
would have some setting indicating a chunk of memory or file
(whatever) to read sound data from, in addition to settings on how to
read it and where it currently is within the file. You would adjust
these settings by checking for messages from the UI in the ringbuffer
at the beginning of each callback invocation. 
5:16 p.m.
On Thursday 17 Jun 2004 7:08 pm, Mario Lang wrote:
I think for this to work, I'd need some
prebuffering in between, and the audio thread somehow singaling to
the decoder thread when the buffer is too empty to safely continue
sleeping so that the decoder can start to fill more data into the
buffer.
One way to do this is to use pthread conditions. Have a ring buffer
between your decoder and RT threads, with the decoder sleeping for
short periods of time between reads using pthread_cond_timedwait or
similar. This causes the thread to sleep until a certain time has
elapsed or a condition has been signalled. You can then signal from
the RT thread using pthread_cond_signal to wake up the decoder thread
each time something is read from the ring buffer.
This is pretty much what we're doing in Rosegarden 0.9.8, but now that
I describe it, I realise I don't know for absolutely sure whether
pthread_cond_signal is itself RT-safe. More information or better
alternatives are welcome.
Chris
5:33 p.m.
Chris Cannam wrote on Thu, 17-Jun-2004:
On Thursday 17 Jun 2004 7:08 pm, Mario Lang wrote:
If you are using pthread_mutex_trylock() to lock the condition's
mutex in the RT thread, then you are ok. If you can't aqcuire
the lock (hence can't signal) that is usually OK because the other
thread is already doing work anyway.
jlc
I think for this to work, I'd need some
prebuffering in between, and the audio thread somehow singaling to
the decoder thread when the buffer is too empty to safely continue
sleeping so that the decoder can start to fill more data into the
buffer.
One way to do this is to use pthread conditions. Have a ring buffer
between your decoder and RT threads, with the decoder sleeping for
short periods of time between reads using pthread_cond_timedwait or
similar. This causes the thread to sleep until a certain time has
elapsed or a condition has been signalled. You can then signal from
the RT thread using pthread_cond_signal to wake up the decoder thread
each time something is read from the ring buffer.
This is pretty much what we're doing in Rosegarden 0.9.8, but now that
I describe it, I realise I don't know for absolutely sure whether
pthread_cond_signal is itself RT-safe. More information or better
alternatives are welcome. 
6:25 p.m.
Chris Cannam wrote on Thu, 17-Jun-2004:
If you are using pthread_mutex_trylock() to lock the condition's
mutex in the RT thread, then you are ok. If you can't aqcuire
the lock (hence can't signal) that is usually OK because the other
thread is already doing work anyway.
I have never understood the purpose of associating a mutex with a
condition variable. Here is the explanation given in the LinuxThreads man
page:
A condition variable must always be associated with a
mutex,
to avoid the race condition where a thread prepares to wait on
a condition variable and another thread signals the condition
just before the first thread actually waits on it.
Who cares if that happens? In this case, the signal should just be
ignored. Why is this worse than having another thread signal the
condition "well before" the first thread waits on it (instead of "just
before")? The only rationale I can think of is if the condition value
mechanism is not thread-safe and needs a mutex to prevent it from being
corrupted.
Josh
7:30 p.m.
On Thu, Jun 17, 2004 at 01:16:27PM -0700, Joshua Haberman wrote:
Unless I am misunderstanding, it's a classic race condition. Assume that
there is some period of time between when you start preparing to wait, and
when you are actually ON the waitqueue. Look:
Consumer Producer
- Check condition (=FALSE)
- Prepare to wait 1 - Set condition to TRUE, no waiters
- Prepare to wait 2
- Go to sleep
The consumer never sees that the condition was TRUE. It goes to sleep and
never wakes up. Now add the mutex:
Consumer Producer
- Take mutex
- Check condition (=FALSE)
- Prepare to wait 1 - block on mutex
- Prepare to wait 2
- Release mutex - Take mutex
- Go to sleep - Set condition to true, 1 waiter
- Wake, block on mutex - Release mutex
- Take mutex
- check condition (=TRUE)
See the difference. The problem is that you can always block on the
mutex.
A condition
variable must always be associated with a mutex,
to avoid the race condition where a thread prepares to wait on
a condition variable and another thread signals the condition
just before the first thread actually waits on it.
Who cares if that happens? In this case, the signal should just be
ignored. Why is this worse than having another thread signal the
7:50 p.m.
On Thursday 17 Jun 2004 10:21 pm, Tim Hockin wrote:
The consumer never sees that the condition was TRUE.
It goes to
sleep and never wakes up. Now add the mutex [...]
I think (correct me if I'm an idiot) that what Joshua's talking about
is that the thread that's waiting for the signal had to have been
doing something before it started waiting for the signal at all.
So, if it was doing a whole load of file read stuff, and then it
finished doing that and entered pthread_cond_wait, and it turned out
that someone else had signalled on the condition while it was doing
the file reads and before it ever entered the wait -- what then?
You'd miss the signal, obviously, and just as above, you'd never see
that the condition was ever true. What difference does it make, from
the point of view of the thread expecting the signal, whether the
signal arrived while it was busy or during that moment between busy
and actively waiting for the signal?
I think associating a mutex with a condition _is_ relevant -- if the
thread doing the signalling has to acquire a lock to do so, then that
can guarantee the other thread isn't busy at the time. In Rosegarden
we don't bother with a lock at all in the RT thread, we just signal()
and don't care if the signal is lost. Is there any advantage to
calling trylock() before doing that, or is signalling on a condition
you have no mutex interest in an OK thing to do?
Chris
9:32 p.m.
and don't care if the signal is lost. Is there any
advantage to
calling trylock() before doing that, or is signalling on a condition
you have no mutex interest in an OK thing to do?
the semantics of condition_wait/condition_signal require that
IF condition_wait() has been called AND condition_signal() is
called THEN the waiter will wake up
you cannot satisfy these semantics without the mutex, as tim
demonstrated. if you want some other semantics, maybe you can get by
without it. i know i've been bitten by race conditions caused by
trying to do this several times, and they are hard to track down and
identify. i also don't use conditions for RT stuff - they are not RT
safe. currently, i use FIFO's, and i plan to switch to futexes when
they become available. NPTL uses futexes to implement condition
variables, but linuxthreads uses signals.
--p
10:12 p.m.
[Paul Davis]
[...]
currently, i use FIFO's, and i plan to switch to
futexes when
they become available. NPTL uses futexes to implement condition
variables, but linuxthreads uses signals.
pipe()s here too. last time i benchmarked on an early 2.4 kernel, pipe
and socketpair gave about the same timing figures, quicker than
msgsnd/rcv. i don't remember the exact numbers but i remember being
positively surprised.
giving you an fd, both pipe and socketpair can be operated on within a
poll() loop, which is quite handy sometimes. don't know if futexes
will do that?
tim
10:52 p.m.
pipe()s here too. last time i benchmarked on an early
2.4 kernel, pipe
and socketpair gave about the same timing figures, quicker than
msgsnd/rcv. i don't remember the exact numbers but i remember being
positively surprised.
from the whitepapers that IBM did, the linux FIFO appears to be just
about the fastest IPC mechanism outside of a microkernel. i am not
sure if even futexes are actually significantly faster.
giving you an fd, both pipe and socketpair can be
operated on within a
poll() loop, which is quite handy sometimes. don't know if futexes
will do that?
you can. it was added as an nth-round iteration of their
design. otherwise, it would be just another example of the crappy *nix
design where "waiting" is a different system call depending on what
you're waiting for.
18 Jun
18 Jun
11:04 a.m.
[Paul Davis]
do you happen to have a pointer ready to those IBM whitepapers you
mention?
i've dug up the old benchmarking code today. it starts a RT thread
that poll()s n times on a pipe, then pthread_cond_wait()s n times and
finally msgrcv()s n times. the main thread writes the current
gettimeofday() to a global and triggers the wakeup of the respective
waiting mechanism, which compares the current gettimeofday() upon
waking up. here are average numbers for 20 wakeups each:
poll: avg 0.01705 ms
cond: avg 0.01576 ms
msg : avg 0.01045 ms
this is on 2.4.22-ll, an 1.7 GHz athlon XP. the last time (referred to
above in the OP) was on 2.4.5-ll (iirc) and a k6-III-450. so either
they sped up msgsnd/rcv (which i doubt), my memory failed me (highly
probable :) or there are significant differences in the respective
kernel code path and processor flavour pairing (too esoteric to
check).
the conclusion i draw is that if you pthread_cond_wait now, you are
probably better off switching to msgsnd/rcv if it fits into your
scheme and you really want the speed. my personal opinion is that the
ease of use of pipe and poll outweighs the microsecond-order gain
msgsnd/rcv gives. it's still blazingly fast. ymmv of course.
i can upload the benchmarking code if there's interest.
this sure is good news. though even if this weren't the case, i'm
quite comfortable with poll, and i'd rather work with 'crappy *nix
waiting' than ever MsgWaitForMultipleObjectsEx() another time in my
life :)
even if futexes will reach the speed msgsnd/rcv achieves, they'll have
to make them a real breeze to use to make me dump the working FIFO
code i am using now.
tim
pipe()s here
too. last time i benchmarked on an early 2.4 kernel, pipe
and socketpair gave about the same timing figures, quicker than
msgsnd/rcv. i don't remember the exact numbers but i remember being
positively surprised.
from the whitepapers that IBM did, the linux FIFO appears to be just
about the fastest IPC mechanism outside of a microkernel. i am not
sure if even futexes are actually significantly faster. giving you an
fd, both pipe and socketpair can be operated on within a
poll() loop, which is quite handy sometimes. don't know if futexes
will do that?
you can. it was added as an nth-round iteration of their
design. otherwise, it would be just another example of the crappy *nix
design where "waiting" is a different system call depending on what
you're waiting for.
6:22 a.m.
On Friday 18 Jun 2004 12:23 am, Paul Davis wrote:
i also don't use conditions for RT stuff - they
are
not RT safe. currently, i use FIFO's, and i plan to switch to
futexes when they become available. NPTL uses futexes to implement
condition variables, but linuxthreads uses signals.
Thanks, that's very informative.
Chris
7:39 a.m.
On Thu, Jun 17, 2004 at 07:23:00PM -0400, Paul Davis wrote:
i also don't use conditions for RT stuff - they
are not RT
safe.
Could you explain this ? AFAIK, if you use try_lock() on
the mutex (and thus accept that the operation may fail),
there is no problem. Occasional failure can easily be
worked around.
--
Fons
17 Jun
17 Jun
7:02 p.m.
On Thursday 17 June 2004 21:24, Jesse Chappell wrote:
Chris Cannam wrote on Thu, 17-Jun-2004:
In fact, you don't need to own the mutex to signal or broadcast the condition.
The one thing you lose by doing that is the guarantee that the
highest-priority waiter will run immediately, since it will need to reacquire
the mutex, and some lower-priority thread that was not waiting could be
holding it. In this situation here, where there is exactly one thread
waiting, and that thread is the only potential owner of the mutex, that's of
course not an issue.
Ciao,
Daniel.
[...]
This is pretty much what we're doing in Rosegarden 0.9.8, but now that
I describe it, I realise I don't know for absolutely sure whether
pthread_cond_signal is itself RT-safe. More information or better
alternatives are welcome.
If you are using pthread_mutex_trylock() to lock the condition's
mutex in the RT thread, then you are ok. If you can't aqcuire
the lock (hence can't signal) that is usually OK because the other
thread is already doing work anyway. 
7:53 p.m.
On Thu, Jun 17, 2004 at 08:26:51PM +0100, Chris Cannam wrote:
One way to do this is to use pthread conditions. Have
a ring buffer
between your decoder and RT threads, with the decoder sleeping for
short periods of time between reads using pthread_cond_timedwait or
similar. This causes the thread to sleep until a certain time has
elapsed or a condition has been signalled. You can then signal from
the RT thread using pthread_cond_signal to wake up the decoder thread
each time something is read from the ring buffer.
In the threads classes I'm using in Aeolus and Jaaa, each thread can
wait on any combination of up 16 counting semaphores and up to 16
message queues, and a timeout. All of this is controlled by just one
condition variable and its associated mutex. No other ITC primitives
are used.
There are two methods to signal the semaphores:
destination->put_event (sema_id, increment)
and
destination->put_event_try (sema_id, increment)
Both will increment one of the semas in <destination> by <increment>,
possibly waking up the thread if the semaphore is in the set of events
that <destination> is waiting for.
The first one uses mutex_lock(), and could block in the improbable
but possible case that the destination thread is pre-empted when
it is just between the mutex_lock() and cv_wait() of its get_event()
method, and holding the mutex.
The second one uses mutex_try_lock(). It will never block, but it may
fail. This one should be used by a RT audio thread.
In Jaaa for example, there is a circular buffer between the ALSA or
JACK thread, and the thread that is doing the FFT analysis. Each time
at least N (= 4096 IIRC) frames are ready in the circular buffer, the
RT audio thread will try to increment one of the semaphores in the
FFT thread by one. In the unlikely case this fails, this is reflected
in the return value of put_event_try(), and the RT thread will just
remember this for the next time, when it will try to increment the
sema by one more. So the worst that could happen is that the FFT
thread's idea of the buffer state lags by N frames, and if N is a
small fraction of the buffer size this just doesn't do any harm at all.
To get its commands from e.g. the GUI thread, the RT audio thread uses
a non-blocking version of get_event(), also using mutex_try_lock(), after
having done its normal processing. So the audio thread will never be
blocked.
--
FA
7259
days inactive
7260
days old
linux-audio-dev@lists.linuxaudio.org
15 comments
11 participants
participants (11)
-
Alfons Adriaensen -
Chris Cannam -
Daniel Schmitt -
Fons Adriaensen -
Jesse Chappell -
Joshua Haberman -
Mario Lang -
Paul Davis -
Pete Bessman -
Tim Goetze -
Tim Hockin