4 Nov
2011
4 Nov
'11
12:29 a.m.
Further to the conversation about Python to C++ ( with many helpful
responses, thanks everyone! ).
For my particular case, no drop outs is critical, and I really really want
to be able to run multiple UIs on lots of cheap machines talking to the
same engine over something (osc I expect). So I'm ok with the fact that
user input and requests for user interface updates may lag, as the queue is
likely to be really busy sometimes. I'm imagining:
Engine thread, which owns all the data actually getting played ( sequences,
wave tables, mixer/synth/effect controls, the works )
- gets called once per sample by audio subsystem ( rtaudio at the moment )
- does it's audio processing, sends out audio
- loop by Magical Mystery Number 1:
- get message off input queue describing change to a data point (
sequence or sample data )
- updates data point
- loop by mystery number 2:
- get message off 2nd UI queue requesting the state of a data point
- sends back a message with that data to the requester
done
GUI thread
- keeps it's own copy of whatever data is pertinent to that particular gui
at that point
- sends a bunch of requests if user changes the view
- sends messages data change requests according to user actions
Here's my question, how do I determine the magical mystery numbers? I need
to make sure engine callback is always done in time, no matter how many
messages are in the queue, which could be very high if someone is dropping
in a sample of audio. By making the data point messages very simple, I hope
that I'll have a pretty good idea of how long one takes to process. It's
just a lock-get-write to simple data structure. But how much audio
processing has happened before that point will be variable. Anyone have
suggestions on that? Is the system clock accurate enough to check the time
and see how much a sample period is left and make some safe calculation
with headroom left over there? It is totally ok for the queue and the
inputs to lag if the audio number crunching goes through a spike.
suggestions most welcome. (including 'that design sucks and here's why')
thanks
iain
4 Nov
4 Nov
1:42 a.m.
On Thu, 2011-11-03 at 16:29 -0700, Iain Duncan wrote:
Further to the conversation about Python to C++ ( with
many helpful
responses, thanks everyone! ).
For my particular case, no drop outs is critical, and I really really
want to be able to run multiple UIs on lots of cheap machines talking
to the same engine over something (osc I expect). So I'm ok with the
fact that user input and requests for user interface updates may lag,
as the queue is likely to be really busy sometimes. I'm imagining:
Engine thread, which owns all the data actually getting played
( sequences, wave tables, mixer/synth/effect controls, the works )
- gets called once per sample by audio subsystem ( rtaudio at the
moment )
- does it's audio processing, sends out audio
- loop by Magical Mystery Number 1:
- get message off input queue describing change to a data point
( sequence or sample data )
- updates data point
- loop by mystery number 2:
- get message off 2nd UI queue requesting the state of a data point
- sends back a message with that data to the requester
done
GUI thread
- keeps it's own copy of whatever data is pertinent to that particular
gui at that point
- sends a bunch of requests if user changes the view
- sends messages data change requests according to user actions
Here's my question, how do I determine the magical mystery numbers? I
need to make sure engine callback is always done in time, no matter
how many messages are in the queue, which could be very high if
someone is dropping in a sample of audio. By making the data point
messages very simple, I hope that I'll have a pretty good idea of how
long one takes to process. It's just a lock-get-write to simple data
structure. But how much audio processing has happened before that
point will be variable. Anyone have suggestions on that? Is the system
clock accurate enough to check the time and see how much a sample
period is left and make some safe calculation with headroom left over
there? It is totally ok for the queue and the inputs to lag if the
audio number crunching goes through a spike.
suggestions most welcome. (including 'that design sucks and here's
why')
Time stamp the events as they come in (e.g. with jack_frame_time()), and
aim to execute them at (time + block_size_in_frames). This avoids
jitter, and keeps the rate that you execute them bounded by the rate
they come in.
You'll also probably want some kind of hard upper limit to ensure
realtimeyness when things get crazy. That truly is a Magical Mystery
Number and will depend greatly on how expensive your events are. Make
one up.
-dr
2:33 a.m.
New subject: timing the processing of queues between engine and ui threads?
thanks Dave, that's what I was looking for! Have you used this technique
yourself? Do you have any suggestions on how that is done with non jack
systems? And any open source code that uses that technique?
thanks so much.
iain
On Thu, Nov 3, 2011 at 5:42 PM, David Robillard <d(a)drobilla.net> wrote:
On Thu, 2011-11-03 at 16:29 -0700, Iain Duncan wrote:
Further to the conversation about Python to C++ (
with many helpful
responses, thanks everyone! ).
For my particular case, no drop outs is critical, and I really really
want to be able to run multiple UIs on lots of cheap machines talking
to the same engine over something (osc I expect). So I'm ok with the
fact that user input and requests for user interface updates may lag,
as the queue is likely to be really busy sometimes. I'm imagining:
Engine thread, which owns all the data actually getting played
( sequences, wave tables, mixer/synth/effect controls, the works )
- gets called once per sample by audio subsystem ( rtaudio at the
moment )
- does it's audio processing, sends out audio
- loop by Magical Mystery Number 1:
- get message off input queue describing change to a data point
( sequence or sample data )
- updates data point
- loop by mystery number 2:
- get message off 2nd UI queue requesting the state of a data point
- sends back a message with that data to the requester
done
GUI thread
- keeps it's own copy of whatever data is pertinent to that particular
gui at that point
- sends a bunch of requests if user changes the view
- sends messages data change requests according to user actions
Here's my question, how do I determine the magical mystery numbers? I
need to make sure engine callback is always done in time, no matter
how many messages are in the queue, which could be very high if
someone is dropping in a sample of audio. By making the data point
messages very simple, I hope that I'll have a pretty good idea of how
long one takes to process. It's just a lock-get-write to simple data
structure. But how much audio processing has happened before that
point will be variable. Anyone have suggestions on that? Is the system
clock accurate enough to check the time and see how much a sample
period is left and make some safe calculation with headroom left over
there? It is totally ok for the queue and the inputs to lag if the
audio number crunching goes through a spike.
suggestions most welcome. (including 'that design sucks and here's
why')
Time stamp the events as they come in (e.g. with jack_frame_time()), and
aim to execute them at (time + block_size_in_frames). This avoids
jitter, and keeps the rate that you execute them bounded by the rate
they come in.
You'll also probably want some kind of hard upper limit to ensure
realtimeyness when things get crazy. That truly is a Magical Mystery
Number and will depend greatly on how expensive your events are. Make
one up.
-dr
4:05 a.m.
On Thu, 2011-11-03 at 18:32 -0700, Iain Duncan wrote:
thanks Dave, that's what I was looking for! Have
you used this
technique yourself? Do you have any suggestions on how that is done
with non jack systems? And any open source code that uses that
technique?
Yes, I use it in Ingen, where all control is via Events.
Event has three main methods:
pre_process() - ("prepare") Execute everything in a non-realtime thread
that has to happen before execution in the audio thread
execute() - ("apply") Execute/apply command in the audio thread
post_process() - ("finalize") Execute anything that needs to happen
after execution in a non-realtime thread, like clean up and notifying
the UI(s) about changes.
The only difference non-jack would make is you need some function to
tell you roughly what audio time it is you can call from another thread.
-dr
5 Nov
5 Nov
11:39 p.m.
New subject: timing the processing of queues between engine and ui threads?
On Thu, Nov 3, 2011 at 8:05 PM, David Robillard <d(a)drobilla.net> wrote:
On Thu, 2011-11-03 at 18:32 -0700, Iain Duncan wrote:
Does one use the system clock for that? Is it accurate enough? Any further
elaboration would be great. ( But I appreciate all the help so far either
way)
thanks
Iain
thanks Dave, that's what I was looking for!
Have you used this
technique yourself? Do you have any suggestions on how that is done
with non jack systems? And any open source code that uses that
technique?
Yes, I use it in Ingen, where all control is via Events.
Event has three main methods:
pre_process() - ("prepare") Execute everything in a non-realtime thread
that has to happen before execution in the audio thread
execute() - ("apply") Execute/apply command in the audio thread
post_process() - ("finalize") Execute anything that needs to happen
after execution in a non-realtime thread, like clean up and notifying
the UI(s) about changes.
The only difference non-jack would make is you need some function to
tell you roughly what audio time it is you can call from another thread.
-dr
6 Nov
6 Nov
8:20 a.m.
New subject: timing the processing of queues between engine and ui threads?
2011/11/5 Iain Duncan <iainduncanlists(a)gmail.com>om>:
On Thu, Nov 3, 2011 at 8:05 PM, David Robillard
<d(a)drobilla.net> wrote:
I think frame time (a frame of samples) is
meant here ? That time is
delivered in the jackd process callback.
The only difference non-jack would make is you need some function to
tell you roughly what audio time it is you can call from another thread.
Does one use the system clock for that? Is it accurate enough?
Depends on the system
clock used, I presume.
For best accuracy, you have to configure your kernel to support HPET
(high precision event) timers
and make ALSA use it as default.
--
E.R.
2:10 p.m.
New subject: timing the processing of queues between engine and ui threads?
On Sun, Nov 6, 2011 at 2:20 AM, Emanuel Rumpf <xbran(a)web.de> wrote:
2011/11/5 Iain Duncan
<iainduncanlists(a)gmail.com>om>:
the clock used for the system clock is less important than using a DLL
to "link" the audio clock and the system clock. this enables you to
answer the question "if its time T on clock1, what time is it on
clock2?"
fons wrote the canonical paper on this for a Linux Audio conference a
few years ago, and JACK contains a DLL for this purpose
(jack_get_microseconds() will return a prediction of the current time
according to the audio clock, based on the system clock and the DLL.
On Thu, Nov 3, 2011 at 8:05 PM, David Robillard
<d(a)drobilla.net> wrote:
I think frame time (a frame of samples) is
meant here ? That time is
delivered in the jackd process callback.
The only difference non-jack would make is you need some function to
tell you roughly what audio time it is you can call from another thread.
Does one use the system clock for that? Is it accurate enough?
Depends on the
system clock used, I presume.
For best accuracy, you have to configure your kernel to support HPET
(high precision event) timers
and make ALSA use it as default.
8:31 p.m.
New subject: timing the processing of queues between engine and ui threads?
The only difference non-jack would make is you need
some function to
tell you roughly what audio time it is you can call from another thread.
Does one use the system clock for that? Is it accurate enough?
Depends on the
system clock used, I presume.
For best accuracy, you have to configure your kernel to support HPET
(high precision event) timers
and make ALSA use it as default. to "link" the audio clock and the system
clock. this enables you to
answer the question "if its time T on clock1, what time is it on
clock2?"
fons wrote the canonical paper on this for a Linux Audio conference a
few years ago, and JACK contains a DLL for this purpose
(jack_get_microseconds() will return a prediction of the current time
according to the audio clock, based on the system clock and the DLL.
thanks, it's sounding increasing like I should be using jack for the time
being.
Iain
7 Nov
7 Nov
3:01 a.m.
New subject: timing the processing of queues between engine and ui threads?
On 11/06/2011 07:10 AM, Paul Davis wrote:
fons wrote the canonical paper on this for a Linux
Audio conference a
few years ago, and JACK contains a DLL for this purpose
(jack_get_microseconds() will return a prediction of the current time
according to the audio clock, based on the system clock and the DLL.
I wasn't aware of this paper... awesome!
/me finds a quiet place to read....
-gabriel
8 Nov
8 Nov
12:45 a.m.
New subject: timing the processing of queues between engine and ui threads?
2011/11/6 Paul Davis <paul(a)linuxaudiosystems.com>om>:
On Sun, Nov 6, 2011 at 2:20 AM, Emanuel Rumpf
<xbran(a)web.de> wrote:
Thanks for the hint, Paul, and to Fons of course.
An interresting paper, found it here:
http://kokkinizita.linuxaudio.org/papers/index.html
Still, I wonder:
Why not compute the time of any sample - relative - to a random start
time (using sample frequency) ?
( The start time would be the time, a recording or playback was started. )
For example:
At f = 48000 Hz, I would expect a soundcard to deliver exactly 48000 *
10 = 48E4 samples in 10 seconds.
Is this assumtion wrong ?
If this applied, one could easily calculate the time-position of
Sample S = 96000
based on a random start time (offset) T0 = 10 seconds:
(We know the answer : 10 sec + 2 sec = 12 sec )
Calculation:
1 sec / 48000 Hz * 96000 + 10 sec = 12 sec
(position of sample nr. 96000)
--
E.R.
2011/11/5 Iain Duncan
<iainduncanlists(a)gmail.com>om>:
the clock used for the system clock is less important than using a DLL
to "link" the audio clock and the system clock. this enables you to
answer the question "if its time T on clock1, what time is it on
clock2?"
fons wrote the canonical paper on this for a Linux Audio conference a
few years ago, and JACK contains a DLL for this purpose
(jack_get_microseconds() will return a prediction of the current time
according to the audio clock, based on the system clock and the DLL.
On Thu, Nov 3, 2011 at 8:05 PM, David Robillard
<d(a)drobilla.net> wrote:
I think frame time (a frame of samples) is
meant here ? That time is
delivered in the jackd process callback.
The only difference non-jack would make is you need some function to
tell you roughly what audio time it is you can call from another thread.
Does one use the system clock for that? Is it accurate enough?
Depends on the
system clock used, I presume.
For best accuracy, you have to configure your kernel to support HPET
(high precision event) timers
and make ALSA use it as default.
2:24 a.m.
New subject: timing the processing of queues between engine and ui threads?
On Mon, Nov 7, 2011 at 6:45 PM, Emanuel Rumpf <xbran(a)web.de> wrote:
For example:
At f = 48000 Hz, I would expect a soundcard to deliver exactly 48000 *
10 = 48E4 samples in 10 seconds.
Is this assumtion wrong ?
no, but its a bit like relativity: who defines "10 seconds"? the only
definition of 10 seconds that the soundcard cares about is the one
defined by *its* clock. its clock isn't the system clock and they do
not run at the same speed. 10 seconds on one clock might be 10.000002
or 9.998 seconds on the other.
4 Nov
4 Nov
2:39 a.m.
New subject: timing the processing of queues between engine and ui threads?
On Thu, Nov 3, 2011 at 7:29 PM, Iain Duncan <iainduncanlists(a)gmail.com> wrote:
Further to the conversation about Python to C++ ( with
many helpful
responses, thanks everyone! ).
For my particular case, no drop outs is critical, and I really really want
to be able to run multiple UIs on lots of cheap machines talking to the same
engine over something (osc I expect). So I'm ok with the fact that user
input and requests for user interface updates may lag, as the queue is
likely to be really busy sometimes. I'm imagining:
you're going to want at least 3 threads:
1) inside the engine, something to handle requests from a UI that
cannot be done under RT constraints
and route others that can into ...
2) the engine thread
3) some UI (not necessarily in the same process)
2:44 a.m.
New subject: timing the processing of queues between engine and ui threads?
For my particular case, no drop outs is critical, and
I really really want
to be able to run multiple UIs on lots of cheap machines talking to the
same
engine over something (osc I expect). So I'm ok
with the fact that user
input and requests for user interface updates may lag, as the queue is
likely to be really busy sometimes. I'm imagining:
you're going to want at least 3 threads:
1) inside the engine, something to handle requests from a UI that
cannot be done under RT constraints
and route others that can into ...
2) the engine thread
3) some UI (not necessarily in the same process)
Thanks, can you elaborate on the first two? ( I appreciate the time,
understand if not ). Is thread 1 spawned by thread 2? Is the idea that the
engine thread can then start stuff that it allows to be interrupted but
still owns all the data for? And how would that be handled if that thread
is being handled by the audio subsystem and I'm just writing a callback
function that runs once a sample?
thanks again
iain
3:12 a.m.
New subject: timing the processing of queues between engine and ui threads?
On Thu, Nov 3, 2011 at 9:44 PM, Iain Duncan <iainduncanlists(a)gmail.com> wrote:
in reverse order ... in such systems, the audio subsystem is providing
(2) for you, so you don't have to start it. however, consider that
many, many systems are now multiprocessor so that even though ardour
gets (2) from JACK, it also has its own pool of worker threads waiting
to function on behalf of "the engine thread". this is a complication
you probably don't want to deal with.
(1) ... it doesn't really matter who starts it. the idea is that the
engine thread (where it comes from) can run under strict RT
constraints. it does not example inter-process communication objects,
it does not perform significant data structure manipulation (what it
does is mostly limited to setting members of existing structures
and/or doing pointer swaps), and it does not have to do bursty cpu
consuming extra work from time to time. the IPC (or just communication
from the GUI) is handled by (1), which takes care of this and all the
other stuff that (2) should not be doing. but its *server* side, or
"backend" if you prefer, not part of any UI. in fact, it can be
handling requests from multiple UIs (as happens, for example, in
ardour where the GUI, OSC and MIDI could all be used to change the
state of things).
For my
particular case, no drop outs is critical, and I really really want
to be able to run multiple UIs on lots of cheap machines talking to the
same
engine over something (osc I expect). So I'm ok with the fact that user
input and requests for user interface updates may lag, as the queue is
likely to be really busy sometimes. I'm imagining:
you're going to want at least 3 threads:
1) inside the engine, something to handle requests from a UI that
cannot be done under RT constraints
and route others that can into ...
2) the engine thread
3) some UI (not necessarily in the same process)
Thanks, can you elaborate on the first two? ( I appreciate the time,
understand if not ). Is thread 1 spawned by thread 2? Is the idea that the
engine thread can then start stuff that it allows to be interrupted but
still owns all the data for? And how would that be handled if that thread is
being handled by the audio subsystem and I'm just writing a callback
function that runs once a sample?
4565
days inactive
4570
days old
linux-audio-dev@lists.linuxaudio.org
13 comments
5 participants
participants (5)
-
David Robillard -
Emanuel Rumpf -
Gabriel M. Beddingfield -
Iain Duncan -
Paul Davis