30 May
2003
30 May
'03
9:50 a.m.
Hi list,
Here is a propblem that is propably already solved in another context,
so i would like to know some opinions on it:
I am trying to implement a (hopefully:-) ) simple general-purpose event
delegation architecture.
The actual application will propably be something like a network of
modules that can do arbitrary filtering, generation and manipulation of
midi events in real time.
As propably different kinds of external event sources, like several midi
ports, maybe joystick device and of course a gui, are involved, how
would one efficiently organize the delegation of events passed between
the modules, so that the everything is still thread-safe?
The three main ideas i can currently think of are:
A: don't do it at all, that is, everything is implemented as simple
subject/observer patterns, so that the communication is a pure
module-to-module thing.
Mutexes et al, would have to be managed by each individual plugin.
B: use a global, mutexed event queue. This could be a priority queue for
time-stamped events, or a simple FIFO.
C: use local queues. As above, but for each individual module.
Each of the above aproaches seems to have its advantages and
disatvantages. E.g. if queues are used, this would as far as i an judge,
make it easy to have feedback cycles.
OTOH this would propably introduce some overhead which aproach A
wouldn't have. C would propably involve a single thread for each module,
or a global "clock" thread that periodicaly calls an "process_queue"
method on each module.
Is there a general "best" way to do this?
Kind regards,
Lukas
30 May
30 May
10:33 a.m.
On Friday 30 May 2003 13.39, Lukas Degener wrote:
Hi list,
Here is a propblem that is propably already solved in another
context, so i would like to know some opinions on it:
I am trying to implement a (hopefully:-) ) simple general-purpose
event delegation architecture.
The actual application will propably be something like a network of
modules that can do arbitrary filtering, generation and
manipulation of midi events in real time.
"Simple", "general purpose" and "real time" are terms that
don't mix
very well. :-)
As propably different kinds of external event sources,
like several
midi ports, maybe joystick device and of course a gui, are
involved, how would one efficiently organize the delegation of
events passed between the modules, so that the everything is still
thread-safe?
A rather common and simple approach is to just read all sources once
per buffer cycle, in non-blocking mode. Block only on reads or writes
from/to the "timing master" audio device, or what have you.
The disadvantage with this is that unless the APIs and devices you
read from provide timestamps, events will be quantized to buffer
cycle boundaries. This might be ok for some applications, but it
won't work well for synths and the like, unless you can run them with
less than some 1-3 milliseconds per buffer.
If you want to do your own timestamping, here's a (relatively) simple
approach:
1) Run one high priority thread for each event input
device. (Or one for all devices, if you can block
on all of them.) Priority should be *higher* than
the engine thread, or there's no point with this
arrangement.
2) Have the thread(s) timestamp incoming events and
pass them on to the engine thread via lock-free
FIFOs or similar thread safe non-blocking interfaces.
3) Have the engine thread check all FIFOs (or what
have you) once per buffer cycle, pretty much as if
they were direct interfaces to drivers, used in
non-blocking mode.
(The "engine thread" here would usually be the real time audio
thread.)
The three main ideas i can currently think of are:
A: don't do it at all, that is, everything is implemented as simple
subject/observer patterns, so that the communication is a pure
module-to-module thing.
Mutexes et al, would have to be managed by each individual
plugin.
B: use a global, mutexed event queue. This could be a priority
queue for time-stamped events, or a simple FIFO.
I'd be careful with those mutexes. When a lower priority thread locks
a resource, it prevents higher priority threads from getting at that
resource.
Simple version: If you have your GUI talk to the audio thread via a
lock protected object, you're asking for trouble.
C: use local queues. As above, but for each individual
module.
Same problems as B, basically, but here, it's easier to get around the
problem by using lock-free FIFOs or similar lock-free constructs.
Each of the above aproaches seems to have its
advantages and
disatvantages. E.g. if queues are used, this would as far as i an
judge, make it easy to have feedback cycles.
Yes, but feedback cycles mean you have either latency (buffer
size/cycle time dependent) or a chance of locking up your engine with
infinite loops. Just keep that in mind. :-)
OTOH this would propably introduce some overhead which
aproach A
wouldn't have.
C would propably involve a single thread for each
module, or a global "clock" thread that periodicaly calls an
"process_queue" method on each module.
JACK proves that it it's indeed possible. It's not trivial to get
right, and it's certainly not portable, unless we're talking about
real time operating systems. (Which often have non-standard APIs and
all sorts of peculiarities, so there are still portability issues.)
Anyway, why would you do this with *threads*? Multiple
threads/processes is definitely the hardest design to get to work
reliably in real time, so one would normally want to avoid it as far
as possible.
It's a different story when you want to integrate applications running
in separate processes. That's why we have JACK. No other way to do
that, without very deep kernel hacks.
Is there a general "best" way to do this?
Not really, as it depends on what you want to do, but pretty much
every (reliable) real time audio application does something like what
I described above, with a single audio thread and lock-free or at
least non-blocking communication.
You'll have to be more specific to get a more specific answer. :-)
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`-----------------------------------> http://audiality.org -'
--- http://olofson.net --- http://www.reologica.se ---
11:55 a.m.
(The "engine thread" here would usually be the real time audio
thread.)
(...)
You'll have to be more specific to get a more
specific answer. :-)
Ok, sorry, the scope of the whole proposal was somewhat ambigous, i'm
afraid. :-)
I don't intend to work with audio stream, at least not right now. (there
are dozens of apps that do this much more elegant than i would be ever
able to do it)
The main focus is on midi events. And i also would rather like a push
model for this. That is, the alsa sequencer client module listens on the
input ports and creates events which are pushed through the network. The
things happening within the individual modules will propably happen in
constant time, or O(n) respectively, if you send n events through them.
So i don't expect performance problems from this side. The only thing i
could imagine to have fatal impact on latency would be that a relevant
thread is blocked / not awoken in time by the system. So i thought if i
have a "master" thread runing with rt priority, which takes care of all
the event delegation (i.e. via a global queue), i should not run into
severe problems. For example: Any event source/subject, no matter which
thread it runs on, delivers events to the global queue (which should, as
you pointed out, be some kind of lock-free fifo). Every event is
associated with the observer/listener it is to be delivered to. Another
thread running with rt priority, reads the events from the queue and
delivers them.
As a result, all event _processing_ would run on a single thread, and
should happen in O(1).
I'd like to diregarded ui interaction for now. Anyway, events that
orginate in the gui could be managed as any other event. Events that
sould be send to the gui can be decoupled via another fifo.
As for the feedbacks: using a fifo in the way above should by itself
introduce a minimal (but required) delay, and therfore make feedbacks
controllable. (i.e. the loop is "flattned", no recursions.)
Ah, in the meantime, to other replies poped into my mailbox.
Kjetil:
yes i have tried pd some time ago, and was rather impressed by what it
could do. I guess by now it is possible to cook coffee and get the girl
next door laid with it. :-)
But i haven't ever looked at the code. And i imagine it to be rather
complex. :-)
Paul:
Interestingly, what you describe as the thing ardour is slowly into,
i.e. one RT thread to process moreless all events, is exactly what
becomes more and more my favourite alternative. Mostly because it seems
relatively easy to do, or rather i think i can imagine how this can be
done. Two issues remain, nevertheless:
A) What happens if any future plugin for some reason does something more
complex, let's say O(n^k) per event, would it still be possible to do
this on the rt thread? Propably not. But anyway, how could one possibly
guaranty rt processing of such a problem? Propably not at all?
B) How to implement a lock-free fifo? Or rather: is there some ready to
use implementation of it?
Thanks for the replies, everyone.
Lukas
12:10 p.m.
On Fri, 30 May 2003, Lukas Degener wrote:
Kjetil:
yes i have tried pd some time ago, and was rather impressed by what it
could do. I guess by now it is possible to cook coffee and get the girl
next door laid with it. :-)
But i haven't ever looked at the code. And i imagine it to be rather
complex. :-)
My thought was that you could make your program in pd.
--
3:42 p.m.
Kjetil S. Matheussen wrote:
On Fri, 30 May 2003, Lukas Degener wrote:
Yes, i did consider this, too. I am even convinced that this would by
far be the easiest solution.
I am just experimenting a bit with ideas that i gathered over time, to
see if i can fit them together into a working programm.
I sometimes try to connect this experiments to actual problems i have in
my everyday music making, but after all it's just for the fun of it.
Just want to see that it works ;-)
Regards,
Lukas
Kjetil:
yes i have tried pd some time ago, and was rather impressed by what it
could do. I guess by now it is possible to cook coffee and get the girl
next door laid with it. :-)
But i haven't ever looked at the code. And i imagine it to be rather
complex. :-)
My thought was that you could make your program in pd.
12:14 p.m.
Hi Lukas,
For the networking transportation you may want to have a look at IEEE
P1639, formerly known as DMIDI, and RTP-MIDI (aka MWPP).
Both provide a network transport framework for MIDI event transmission.
IEEE 1639 is layer 2, raw Ethernet, and RTP-MIDI is layer 3 based.
Regards
Phil
On Fri, 2003-05-30 at 14:44, Lukas Degener wrote:
(The "engine thread" here would usually be the real time audio
thread.)
(...)
You'll have to be more specific to get a more
specific answer. :-)
Ok, sorry, the scope of the whole proposal was somewhat ambigous, i'm
afraid. :-)
I don't intend to work with audio stream, at least not right now. (there
are dozens of apps that do this much more elegant than i would be ever
able to do it)
The main focus is on midi events. And i also would rather like a push
model for this. That is, the alsa sequencer client module listens on the
input ports and creates events which are pushed through the network. The
things happening within the individual modules will propably happen in
constant time, or O(n) respectively, if you send n events through them.
So i don't expect performance problems from this side. The only thing i
could imagine to have fatal impact on latency would be that a relevant
thread is blocked / not awoken in time by the system. So i thought if i
have a "master" thread runing with rt priority, which takes care of all
the event delegation (i.e. via a global queue), i should not run into
severe problems. For example: Any event source/subject, no matter which
thread it runs on, delivers events to the global queue (which should, as
you pointed out, be some kind of lock-free fifo). Every event is
associated with the observer/listener it is to be delivered to. Another
thread running with rt priority, reads the events from the queue and
delivers them.
As a result, all event _processing_ would run on a single thread, and
should happen in O(1).
I'd like to diregarded ui interaction for now. Anyway, events that
orginate in the gui could be managed as any other event. Events that
sould be send to the gui can be decoupled via another fifo.
As for the feedbacks: using a fifo in the way above should by itself
introduce a minimal (but required) delay, and therfore make feedbacks
controllable. (i.e. the loop is "flattned", no recursions.)
Ah, in the meantime, to other replies poped into my mailbox.
Kjetil:
yes i have tried pd some time ago, and was rather impressed by what it
could do. I guess by now it is possible to cook coffee and get the girl
next door laid with it. :-)
But i haven't ever looked at the code. And i imagine it to be rather
complex. :-)
Paul:
Interestingly, what you describe as the thing ardour is slowly into,
i.e. one RT thread to process moreless all events, is exactly what
becomes more and more my favourite alternative. Mostly because it seems
relatively easy to do, or rather i think i can imagine how this can be
done. Two issues remain, nevertheless:
A) What happens if any future plugin for some reason does something more
complex, let's say O(n^k) per event, would it still be possible to do
this on the rt thread? Propably not. But anyway, how could one possibly
guaranty rt processing of such a problem? Propably not at all?
B) How to implement a lock-free fifo? Or rather: is there some ready to
use implementation of it?
Thanks for the replies, everyone.
Lukas
3:46 p.m.
Phil Kerr wrote:
Hi Lukas,
For the networking transportation you may want to have a look at IEEE
P1639, formerly known as DMIDI, and RTP-MIDI (aka MWPP).
Oops, i think this is another misunderstanding. I shouldn't have used
words like 'network', 'port' and 'listen' .
Actualy i had nothing in mind that would be related to networking, i.e.
communication between two or more computers,
by 'network' i mean a virtual network of interconnected modules within
my programm.
Lukas
2 Jun
2 Jun
10:47 a.m.
Lukas Degener wrote:
Actualy i had nothing in mind that would be related to
networking, i.e.
communication between two or more computers,
by 'network' i mean a virtual network of interconnected modules within
my programm.
Since you concentrate on MIDI... That sounds a lot like the design of
MidiShare (http://www.grame.fr/MidiShare/), maybe you should take a look
at this?
Albert
--
Dr. Albert Gr"af
Email: Dr.Graef(a)t-online.de, ag(a)muwiinfa.geschichte.uni-mainz.de
WWW: http://www.musikwissenschaft.uni-mainz.de/~ag
30 May
30 May
12:26 p.m.
done. Two issues remain, nevertheless:
A) What happens if any future plugin for some reason does something more
complex, let's say O(n^k) per event, would it still be possible to do
this on the rt thread? Propably not. But anyway, how could one possibly
guaranty rt processing of such a problem? Propably not at all?
ardour has evolved a lot in this area in the last month. the transport
state is a classic non-RT mechanism (it often requires disk seeks, for
example). all changes to transport state with a non-RT element are
divided into two functions, the RT part and the non-RT part. the
non-RT part is queued with another thread (which not uncoincidentally
is the thread that would handle disk i/o anyway), and until its
finished its work, nothing much happens in the audio thread (we just
do silence or some equivalent). once its finished, we run the RT
component of the state change, and then the audio thread can run
"normally".
what is critical about this design is that it delegates "classes of
activity" to the correct thread, but (and this is the key part) the
audio thread is the initiator of all such delegation. as a result, the
audio thread is never "suprised" by anything happening behind its
back: if the disk thread is doing something, the audio thread already
knows about it.
B) How to implement a lock-free fifo? Or rather: is
there some ready to
use implementation of it?
i just use RingBuffer<Event*> (assuming you know C++ syntax). pseudo
code:
delivery:
Event* event;
event = new Event (arg1, arg2, ...);
pending_events.write (&event, 1);
receipt:
Event* event;
if (pending_events.read (&event, 1) == 1) {
process_event (event);
}
this does have one problem in that it limits the number of queued
events, but i set it to a large number (size is N * sizeof(Event*)),
and don't worry about it.
--p
1 Jun
1 Jun
11:43 a.m.
B) How to implement a lock-free fifo? Or rather:
is there some ready to
use implementation of it?
i just use RingBuffer<Event*> (assuming you know C++ syntax). pseudo
code:
delivery:
Event* event;
event = new Event (arg1, arg2, ...);
pending_events.write (&event, 1);
receipt:
Event* event;
if (pending_events.read (&event, 1) == 1) {
process_event (event);
}
30 May
30 May
10:56 a.m.
On Fri, 30 May 2003, Lukas Degener wrote:
Is there a general "best" way to do this?
Have you looked at pure data or jmax?
http://www.pure-data.org
http://www.ircam.fr/equipes/temps-reel/jmax/
--
11:07 a.m.
I am trying to implement a (hopefully:-) ) simple
general-purpose event
delegation architecture.
The actual application will propably be something like a network of
modules that can do arbitrary filtering, generation and manipulation of
midi events in real time.
As propably different kinds of external event sources, like several midi
ports, maybe joystick device and of course a gui, are involved, how
would one efficiently organize the delegation of events passed between
the modules, so that the everything is still thread-safe?
The three main ideas i can currently think of are:
interesting you should ask. in ardour, i have recently been shifting
from a rather complex model toward a very simple one. ardour has
several event sources (automation, GUI, MIDI, audio). there used to be
two different approaches to processing an event:
a) code to handle it is part of an object, and can be called
by any thread.
b) code to handle it is part of an object, but must be called
by a specific thread.
events in class (b) would be queued (lock-free) where they would wait
to be processed by the relevant thread. events in class (a) would get
handled immediately by whichever thread received/noticed the event.
slowly, but with all the certainty of a rising tide, everything is
shifting to a model in which more and more events are queued (lock
free) for the main audio thread, and all events are processed in FIFO
order by that thread.
the complications that i've discovered from attempting to have
different threads handle things without using locks are just to
signficant to be worth the effort (at least at this point).
in addition, lots of event processing in ardour consists of setting a
"desired" value for various things; the real-time code is responsible
for interpolating between the current value and the "desired"
value. you can imagine, i trust, that this can get quite complicated :)
--p
7873
days inactive
7876
days old
linux-audio-dev@lists.linuxaudio.org
11 comments
6 participants
participants (6)
-
David Olofson -
Dr.Graef＠t-online.de -
Kjetil S. Matheussen -
Lukas Degener -
Paul Davis -
Phil Kerr