27 Feb
2003
27 Feb
'03
7:58 p.m.
On Thursday 27 February 2003 20:25, Tim Jansen wrote:
Did the other platforms use a callback-driven approach
because it is a
superior API, or because it is the only way to have sound on
cooperative-multitasking OSes like MacOS <X and early Windows versions?
Callback-driven APIs are much harder to use, especially with many existing
codecs and frameworks that have been written with a push API in mind.
I try to list some pro / con for call back and push APIs:
push API:
Pro:
* simple to play a single file, basically a copy from source file to
destination.
Con:
* hard to write plugins that way. Take a look at arts plugins. They all have
a 'calculateBlock' = call back!
Why?
In a pure push model each processing step reads data from
one file/pipe/device, processes it, pushes it to a file/pipe/device
You get:
* lots of threads/processes that are not optimally synchronized.
Any thread is runnable when there are input available until the
output is full.
But that is not the important case, concider the case when the last
processing steps output is almost empty (if it gets empty you will hear
a click). How to prioritize it higher than all other threads? Should it
always be higher? Suppose it is a mixer that has several inputs...
Could be done by a super server that sets priorities depending
on position in the line? This is not easy...
* If plugins, with callback model, are used to build the application. Does
it not make sense to build the whole application (audio part) the same way?
There are some neat tricks that can be used, since the wrapper library
can know where the destination is.
* if the destination is in the same process, your output will end up somewere
in your process memory.
* On the other hand, suppose the destination is another application,
it can allocate shared memory and let the output of your pluggin end up
there.
* If the output is destined to go to an audio board
It could then give you a memory mapped hardware buffer instead of
ordinary memory to avoid the final copy. (you will get different buffers on
each process...)
* if your output type does not match the input type of the destination,
the library could automatically insert a converter/resampler either on
your side or on the destination side (pick the one that gives less
communication).
* Can the destination change during the run?
1. Your application starts alone, output format matches one supported
by hardware. => hardware buffers
2. Another application starts (suppose the device can have several
destinations open at once - like SB Live!) => no change for your pluggin
(but assume the format of this pluggin is not supported by hardware
=> in process buffer + automatic inserted convertion pluggin
+ hardware buffer)
3. Even more applications start... No more possible to output direct to
hardware for all... suppose the library checks for matching data types
- and the first application match perfectly!
=> the new application will get shared memory,
your application will be changed to ordinary memory, these buffers will be
mixed by an automatically inserted pluggin that outputs
to the hardware buffer...
4. The new application ends. => hardware buffers again
=> your application/pluggin does not need to care. [No framework that I know
of implements this today - especially not the automatic parts]
With the push model your application needs to know about possible
destinations, or treat everything as a file or shared memory.
But how to handle the dynamic changes in the push model?
Pipes, shared memory?
It could also use a library that directs the produced buffer in the right
direction (CSL?) - but it will be hard to eliminate extra copying.
Note: Arts and the KDE multimedia framework does a lot of things right today.
It even goes one step furter since it moves the pluggins, including input and
output into one process - artsd. But currently it does not work perfectly
together with applications using other frameworks.
/RogerL
--
Roger Larsson
Skellefteå
Sweden
28 Feb
28 Feb
9:57 a.m.
Roger Larsson writes:
I try to list some pro / con for call back and push
APIs:
push API:
Pro:
* simple to play a single file, basically a copy from source file to
destination.
Con:
* hard to write plugins that way. Take a look at arts plugins. They all have
a 'calculateBlock' = call back!
Why?
In a pure push model each processing step reads data from
one file/pipe/device, processes it, pushes it to a file/pipe/device
You get:
* lots of threads/processes that are not optimally synchronized.
Any thread is runnable when there are input available until the
output is full.
But that is not the important case, concider the case when the last
processing steps output is almost empty (if it gets empty you will hear
a click). How to prioritize it higher than all other threads? Should it
always be higher? Suppose it is a mixer that has several inputs...
Could be done by a super server that sets priorities depending
on position in the line? This is not easy...
There is no need to prioritize it. The possible bad synchronisation in
a 'push API' system arises from the fact that threads that do not need
any RT input (e.g. an oscillator) are essentially free running and are
allowed to 'work ahead', thereby stealing CPU time from the others.
This can easily be avoided by making these 'free-running' threads wait
on an event that has the right frequency, e.g. a counting semaphore
that is incremented by the output module each time a block is processed.
This way you can eleminate all buffering between the threads, and
you get a system that is as deterministic as e.g. JACK.
There is only on remaining problem: threads that need N inputs could be
scheduled up to N-1 times before they can proceed, which may generate
a number of unnecessary swaps. This is nicely solved by a system such as
JACK that will only wake op the processing thread (which then calls
your callback function) when all required resources are available.
The same effect can be had by using ITC mechanisms that allow you
to specify more complex conditions that a thread can wait for. The
result is a 'push-API' system that is completely deterministic and
as efficient as JACK. This is no imagination, I am using such a system
for telecom DSP applications. ITC is based on pthread condition variables.
--
Fons Adriaensen
ALCATEL TELECOM
10:49 a.m.
On Friday 28 February 2003 14:49, Fons Adriaensen wrote:
Roger Larsson writes:
But the point is that you get the push(!) from the wrong direction.
It is not important to work ahead it is important not to miss the deadline.
- - -
In a pure push model each processing step reads data from
one file/pipe/device, processes it, pushes it to a file/pipe/device
You get:
* lots of threads/processes that are not optimally synchronized.
Any thread is runnable when there are input available until the
output is full.
But that is not the important case, concider the case when the last
processing steps output is almost empty (if it gets empty you will hear
a click). How to prioritize it higher than all other threads? Should it
always be higher? Suppose it is a mixer that has several inputs...
Could be done by a super server that sets priorities depending
on position in the line? This is not easy...
There is no need to prioritize it. The possible bad synchronisation in
a 'push API' system arises from the fact that threads that do not need
any RT input (e.g. an oscillator) are essentially free running and are
allowed to 'work ahead', thereby stealing CPU time from the others.
This can easily be avoided by making these 'free-running' threads wait
on an event that has the right frequency, e.g. a counting semaphore
that is incremented by the output module each time a block is processed.
This way you can eleminate all buffering between the threads, and
you get a system that is as deterministic as e.g. JACK.
Easily avoided? By adding semaphores and more? The point is that you can
not keep it as simple as pipe in pipe out. By adding semaphores the push
model is already more complex that the call back. And all problems are not
solved yet...
Suppose you have four outputs 1, 2, 3, 4 where 1 and 2 is used by pluggin A
and 2, 3 and 4 is used by pluggin B. Then there have to be one counter per
pluggin and those pluggins that generates the outputs has to know of and
update both... or?
- - - [jack]
The same effect can be had by using ITC mechanisms that allow you
to specify more complex conditions that a thread can wait for. The
result is a 'push-API' system that is completely deterministic and
as efficient as JACK. This is no imagination, I am using such a system
for telecom DSP applications. ITC is based on pthread condition variables.
OK, but you did not mention simple - is it also as simple as JACK?
/RogerL
--
Roger Larsson
Skellefteå
Sweden
11:59 a.m.
Roger Larsson writes:
> This can easily be avoided by making these
'free-running' threads wait
> on an event that has the right frequency, e.g. a counting semaphore
> that is incremented by the output module each time a block is processed.
> This way you can eleminate all buffering between the threads, and
> you get a system that is as deterministic as e.g. JACK.
Easily avoided? By adding semaphores and more? The
point is that you can
not keep it as simple as pipe in pipe out. By adding semaphores the push
model is already more complex that the call back.
All the high level synchronisation functions such as poll(), select(),
pipes etc. are ultimately based on primitives such as semaphores and
condition variables. If you use these primitives directly (in the
implementation, not in the API - the user doesn't have to see them)
you can do things such as having a thread scheduled when all its
inputs are available, and then there is no need to analyse the
conection graph an plan ahead as JACK does. What you get is a
completely determinisic 'push-API' system.
OK, but you did not mention simple - is it also as
simple as JACK?
Implentation-wise, a lot simpler in fact. For the user (application
code writer) about the same. JACK looks simple because you don't see
what's happening behind the scenes, you only see the callback API.
By all this I don't want to leave the impression that I dislike JACK or
think it's badly written, on the contrary I think it's a high-quality,
and beautiful system, and it does a great service to the Linux audio
world.
The one thing I regret about JACK is that really tries to make things
*to* simple for the application developer, and thereby excludes some
possibilities that would come in handy.
The thread that calls the user's callback function is completely
hidden, and so it can wait only for the trigger form jackd and nothing
else. As a consequence you can't use it to communicate safely with your
callback which would be easy if the thread and the synchronisation
mechanism were visible.
--
FA
12:48 p.m.
On Fri, Feb 28, 2003 at 04:51:07 +0100, Fons Adriaensen wrote:
The thread that calls the user's callback function
is completely
hidden, and so it can wait only for the trigger form jackd and nothing
else. As a consequence you can't use it to communicate safely with your
callback which would be easy if the thread and the synchronisation
mechanism were visible.
Thats largly for portability reasons, the callbacks aren't allways in a
thread, eg. for IE clients.
Adding a thread which is syncronised with the calback thread is very easy
and more portable.
- Steve
7972
days inactive
7973
days old
linux-audio-dev@lists.linuxaudio.org
4 comments
3 participants
participants (3)
-
Fons Adriaensen -
Roger Larsson -
Steve Harris