23 Aug
2008
23 Aug
'08
8:56 a.m.
Hi Dan,
why don't you answer on the mailing list?
Dan Mills wrote:
That worries me a bit. If my ringbuffer is really short, for responsive user
control, and the time stretching performed in a non-RT thread, isn't this going
to make things worse?
What about the following idea:
Say I make a Converter object, which performs time stretching. It doesn't know
anything about threading.
I also have a Reader object, with an internal ringbuffer, it reads raw audio
data, converts it using the Converter, and fills its ringbuffer with the
converted content.
The Reader objects exposes read(double time_ratio, int nframes) to the realtime
thread. When this function gets called, the Reader checks if it has data time
stretched according to this time ratio in its internal ringbuffer. If it does it
simply returns it. Otherwise, it calls the Converter on the fly, discarding the
irrelevant data in the ringbuffer, and causing a sudden small overhead.
Once the time ratio has stopped varying (the user has stopped playing with the
knob), the ringbuffer rapidly gets filled with data at the right time ratio, and
the overhead vanishes.
On Fri, 2008-08-22 at 20:26 +0200, Olivier Guilyardi
wrote:
I am unsure what you mean by "core".
In Jackbeat I perform sample rate conversion using libsamplerate in the realtime
thread. I have recently realized that might be a mistake in regard to
performance. The SRC is used on audio samples which sample rate might differ
from the external (Jack) one, and also as a simple pitch shifter.
<Snip>
In this regard, I thought about setting a (or
several) non-RT helper thread(s)
up, where the heavy stuff would happen, and feed the converted content into the
RT thread through a ringbuffer. I understand that this approach would require
implementing a sync callback for the helper threads to get ready (fill buffers,
etc...) whenever the transport position changes.
That is a fairly common approach, one thread per core is good. I have
code that reads audio, sample rate converts it, time-stretches it if
appropriate and sticks it in a ring buffer. The realtime thread does not
pay any attention to the transport at all, it just handles the realtime
controls (some mixing and routing in my case). One thing you do want to watch is that the disk IO
threads should
probably run at a realtime priority (just lower then the one for the
audio thread), otherwise a make -j10 on the kernel can result in
ringbuffer under run.
Disk IO is another problem to me. I would prefer to focus on computation load
here, that is: helper threads that performs audio conversion/transformation,
etc.... I assume that the thread priority you advise also concerns this though.
There are a few details to making everything work
right when you have
more playbacks then available cores, and the approach does not work well
if you need to vary things like timestretch in realtime.
Ah right, thanks, I had forgotten about this. Indeed, the good side of my
current approach (where the SRC is performed in the realtime thread) is that it
gives the user a good experience when varying the pitch: the effect is
immediate. So I'm not operating on live audio input, but I do handle live
"control" input, and in this regard the audio sample content is not exactly
known in advance anymore (since it is transformed according to realtime user
requests).
I'm asking
this because the SRC computation load currently has an impact on the
output quality (ticks, xruns) with many tracks or on slower systems, and I'm
thinking about adding real pitch and time shifting using Rubberband in the near
future, which is certainly going to be a lot heavier.
If you need to be able to vary the timestrtch in real time with a file
playing, any ring buffer after the timestretcher will need to be short,
otherwise it is fairly straightforward. Volatile is your friend as is pthread_mutex_trylock
(Particularly handy
if you have a vector of disk readers and a small handful of threads
servicing them).
Thanks, I knew little about volatile, I've read some more about it, and already
fixed a possible bug in my code. Yes, I recently read about
pthread_mutex_trylock being acceptable in the realtime thread on the jack-devel
mailing list.
Regards,
--
Olivier Guilyardi / Samalyse
25 Aug
25 Aug
6:21 p.m.
New subject: Realtime and helper threads
On Sat, 2008-08-23 at 14:56 +0200, Olivier Guilyardi wrote:
Processor core on the machine, having multiple threads will allow more
then one core to be utilised if required.
It would, in my app it is basically disk io that feeds the system but in
your case it seems to be something else, but the same principle applies.
I was advocating having this done in a RT thread at lower priority then
the audio thread.
How much control latency can you tolerate? That number will tell you
what is possible in terms of approach, quantify the problem.
Hi Dan,
why don't you answer on the mailing list?
I thought I had!
That is a
fairly common approach, one thread per core is good. I have
code that reads audio, sample rate converts it, time-stretches it if
appropriate and sticks it in a ring buffer. The realtime thread does not
pay any attention to the transport at all, it just handles the realtime
controls (some mixing and routing in my case).
I am unsure what you mean by "core". One thing you
do want to watch is that the disk IO threads should
probably run at a realtime priority (just lower then the one for the
audio thread), otherwise a make -j10 on the kernel can result in
ringbuffer under run.
Disk IO is another problem to me. I would prefer to focus on computation load
here, that is: helper threads that performs audio conversion/transformation,
etc.... I assume that the thread priority you advise also concerns this though.
If you need to be able to vary the timestrtch in real time with a file
playing, any ring buffer after the timestretcher will need to be short,
otherwise it is fairly straightforward.
That worries me a bit. If my ringbuffer is really short, for responsive user
control, and the time stretching performed in a non-RT thread, isn't this going
to make things worse? What about the following idea:
Say I make a Converter object, which performs time stretching. It doesn't know
anything about threading.
I also have a Reader object, with an internal ringbuffer, it reads raw audio
data, converts it using the Converter, and fills its ringbuffer with the
converted content.
The Reader objects exposes read(double time_ratio, int nframes) to the realtime
thread. When this function gets called, the Reader checks if it has data time
stretched according to this time ratio in its internal ringbuffer. If it does it
simply returns it. Otherwise, it calls the Converter on the fly, discarding the
irrelevant data in the ringbuffer, and causing a sudden small overhead.
Then you are no longer realtime as Linux audio understands the concept.
Once the time ratio has stopped varying (the user has
stopped playing with the
knob), the ringbuffer rapidly gets filled with data at the right time ratio, and
the overhead vanishes.
It still does not solve your problem as clicks and pops can happen while
the user is playing with the knob.
Regards, Dan.
27 Aug
27 Aug
7:27 a.m.
New subject: Realtime and helper threads
Dan Mills wrote:
It would, in my app it is basically disk io that feeds the system but in
your case it seems to be something else, but the same principle applies.
Okay, so just a side question about disk IO: can you do that in a realtime
thread? On http://rt.wiki.kernel.org/index.php/HOWTO:_Build_an_RT-application,
it says: "File handling is known to generate disastrous pagefaults. So,if there
is a need for file access from the context of the RT-application, then this can
be done best by splitting the application in an RT part and a file-handling part".
I was advocating having this done in a RT thread at lower priority then
the audio thread.
Right, sorry I misread this. Just a little practical question here: what could
be the event waking my RT helper thread? Sleeping for a fixed amount of time,
waking up, checking if there's some write space in the ringbuffer, if not then
sleeping again?
What do you mean?
No, I believe it could solve it: this is only a small overhead, on a single
track, for a short period of time. This is very different from, say, performing
time/pitch shifting in realtime on 16 tracks. In my idea, when the user vary the
pitch, you would only have one track that would temporarily perform some heavy
computation in realtime. The other 15 tracks would still rely on the background
job performed by the helper thread. This way I could have big ringbuffers (say
64k), to ensure smooth output.
Regards,
--
Olivier Guilyardi / Samalyse
On Sat, 2008-08-23 at 14:56 +0200, Olivier Guilyardi
wrote:
Processor core on the machine, having multiple threads will allow more
then one core to be utilised if required.
Right. Although this is one of the reason I thought about helper threads, I also
believed it could help on single cpu systems. For this to be true, is the only
solution to have a big ringbuffer?
That is a
fairly common approach, one thread per core is good. I have
code that reads audio, sample rate converts it, time-stretches it if
appropriate and sticks it in a ring buffer. The realtime thread does not
pay any attention to the transport at all, it just handles the realtime
controls (some mixing and routing in my case).
I am unsure what you mean by
"core". One thing you do want to watch is that the disk IO
threads should
probably run at a realtime priority (just lower then the one for the
audio thread), otherwise a make -j10 on the kernel can result in
ringbuffer under run.
Disk IO is another problem to me. I would prefer to focus
on computation load
here, that is: helper threads that performs audio conversion/transformation,
etc.... I assume that the thread priority you advise also concerns this though. If you need to be able to vary the timestrtch in real
time with a file
playing, any ring buffer after the timestretcher will need to be short,
otherwise it is fairly straightforward.
That worries me a bit. If my ringbuffer
is really short, for responsive user
control, and the time stretching performed in a non-RT thread, isn't this going
to make things worse? How much control latency can you tolerate? That number
will tell you
what is possible in terms of approach, quantify the problem.
It depends on the control type. I'm having problems thinking about this for time
stretching, I've never done it and it operates on time, which will interfere
with bpm, transport position, etc.. But in regard to pitch shifting, I would
like the latency to be as small as possible. Something less than 30ms. At
44100Hz, on a mono track, a buffer of 1024 bytes would do I suppose.
What about the
following idea:
Say I make a Converter object, which performs time stretching. It doesn't know
anything about threading.
I also have a Reader object, with an internal ringbuffer, it reads raw audio
data, converts it using the Converter, and fills its ringbuffer with the
converted content.
The Reader objects exposes read(double time_ratio, int nframes) to the realtime
thread. When this function gets called, the Reader checks if it has data time
stretched according to this time ratio in its internal ringbuffer. If it does it
simply returns it. Otherwise, it calls the Converter on the fly, discarding the
irrelevant data in the ringbuffer, and causing a sudden small overhead.
Then you are no longer realtime as Linux audio understands the concept. Once the time
ratio has stopped varying (the user has stopped playing with the
knob), the ringbuffer rapidly gets filled with data at the right time ratio, and
the overhead vanishes.
It still does not solve your problem as clicks and pops can happen while
the user is playing with the knob.
5972
days inactive
5976
days old
linux-audio-dev@lists.linuxaudio.org
3 comments
2 participants
participants (2)
-
Dan Mills -
Olivier Guilyardi