27 Feb
2003
27 Feb
'03
7:58 p.m.
On Thursday 27 February 2003 20:25, Tim Jansen wrote:
Did the other platforms use a callback-driven approach
because it is a
superior API, or because it is the only way to have sound on
cooperative-multitasking OSes like MacOS <X and early Windows versions?
Callback-driven APIs are much harder to use, especially with many existing
codecs and frameworks that have been written with a push API in mind.
I try to list some pro / con for call back and push APIs:
push API:
Pro:
* simple to play a single file, basically a copy from source file to
destination.
Con:
* hard to write plugins that way. Take a look at arts plugins. They all have
a 'calculateBlock' = call back!
Why?
In a pure push model each processing step reads data from
one file/pipe/device, processes it, pushes it to a file/pipe/device
You get:
* lots of threads/processes that are not optimally synchronized.
Any thread is runnable when there are input available until the
output is full.
But that is not the important case, concider the case when the last
processing steps output is almost empty (if it gets empty you will hear
a click). How to prioritize it higher than all other threads? Should it
always be higher? Suppose it is a mixer that has several inputs...
Could be done by a super server that sets priorities depending
on position in the line? This is not easy...
* If plugins, with callback model, are used to build the application. Does
it not make sense to build the whole application (audio part) the same way?
There are some neat tricks that can be used, since the wrapper library
can know where the destination is.
* if the destination is in the same process, your output will end up somewere
in your process memory.
* On the other hand, suppose the destination is another application,
it can allocate shared memory and let the output of your pluggin end up
there.
* If the output is destined to go to an audio board
It could then give you a memory mapped hardware buffer instead of
ordinary memory to avoid the final copy. (you will get different buffers on
each process...)
* if your output type does not match the input type of the destination,
the library could automatically insert a converter/resampler either on
your side or on the destination side (pick the one that gives less
communication).
* Can the destination change during the run?
1. Your application starts alone, output format matches one supported
by hardware. => hardware buffers
2. Another application starts (suppose the device can have several
destinations open at once - like SB Live!) => no change for your pluggin
(but assume the format of this pluggin is not supported by hardware
=> in process buffer + automatic inserted convertion pluggin
+ hardware buffer)
3. Even more applications start... No more possible to output direct to
hardware for all... suppose the library checks for matching data types
- and the first application match perfectly!
=> the new application will get shared memory,
your application will be changed to ordinary memory, these buffers will be
mixed by an automatically inserted pluggin that outputs
to the hardware buffer...
4. The new application ends. => hardware buffers again
=> your application/pluggin does not need to care. [No framework that I know
of implements this today - especially not the automatic parts]
With the push model your application needs to know about possible
destinations, or treat everything as a file or shared memory.
But how to handle the dynamic changes in the push model?
Pipes, shared memory?
It could also use a library that directs the produced buffer in the right
direction (CSL?) - but it will be hard to eliminate extra copying.
Note: Arts and the KDE multimedia framework does a lot of things right today.
It even goes one step furter since it moves the pluggins, including input and
output into one process - artsd. But currently it does not work perfectly
together with applications using other frameworks.
/RogerL
--
Roger Larsson
Skellefteå
Sweden