[linux-audio-dev] XAP spec - early scribbles

David Olofson david at olofson.net
Thu Feb 6 13:17:01 UTC 2003


On Thursday 06 February 2003 06.11, Tim Hockin wrote:
> > The host struct is the plugin's interface to various host
> > provided resources. Unfortunately, making these resources
> > thread-safe would result in a major performance hit. (Consider
> > the event system, for example.)
> >
> > The easy way to deal with this is to just give plugins another
> > host struct when you want to mess with them in another thread.
> > The plugins won't be able to tell the difference, unless you want
> > them to.
>
> At what time do we need to change out the Host struct?  I'm not
> seeing where or why we need to do that.  Concrete example, please?

Changing any control that isn't RT safe. A delay time change that 
requires buffers to be reallocated, for example.

However, IMHO, this turns into just another motivation for worker 
callbacks, and even suggests that use of them should be encouraged. 
In fact, I think worker callbacks for non RT safe operations should 
be *required* for plugins that claim to be RT safe at all.

Problem is that the host cannot know when a plugin has received an 
event that will trigger a non-RT safe operation. Besides, you can't 
take a processing plugin out of the net anyway, as you'd disrupt 
anything that passes through it. Only the plugin can deal with this 
properly.

Taking the delay time change as an example, the plugin would do this:

	When the offending event is received:

		* Fire a worker callback to allocate and
		  initialize new buffers.

(Processing goes on as if nothing happened.)

	When the worker callback finishes:

		* Throw in the new buffers, change params etc.

		* Free the old buffers. (Worker call or
		  host provided RT safe free() replacement.)


I think this applies to most stuff that isn't RT safe, and it's the 
only way to have plugins with non RT safe operations work correctly 
in RT hosts. I also think implementations can be really rather simple 
most of the time.


> > > We can track silence per Port, or just per Plugin (simpler, I
> > > think).
> >
> >_
> > Simple, but insufficient for any but the simplest plugins. If you
> > have_
>
> Again, I think we are speaking of slightly different things.  I am
> talking about the time when (for example) the synth at the head of
> the chain has stopped playing notes.  A reverb with this as it's
> input would be told 'your input is now silent'.  It has a tail, of
> course.  It can be marked silent when it's tail is done.  Once it
> is silent, it does not need to be processed._

I see. I also see a big difference between synths and "effects" here. 
It's hard to tell when you need to start processing a synth again, 
after it's been silent. You can *guess* by looking at the input of an 
effect plugin, but you can't even be sure in that case with some 
effects. Such effects would just have to ignore this feature 
altogether and be processed at all times.


> I don't know what use it is to mark individual buffers/Ports in
> this case.

The point is that the effects that process the output from each 
channel of the synth can take great advantage of knowing whether 
input is silent or not.


> What is the point of having 'Silent' or NULL buffers?  I mean, I
> guess it can be a minor optimization.

It's a rather *major* optimization when it means you can return 
without doing any processing at all... That's what Audiality's old 
"reverb" does when it realizes the tail is out and there's no input.


>  Maybe silence is a
> per-channel thing?  I don't know that finer grain flags buy
> anything at all..

Why channels? If it's not per audio port, it might as well be 
anything... I think this depends entirely on plugin internals.


[...]
> > > Then the
> > > host can hot the bypass switch (is bypass optional?) or remove
> > > it from the net, or just stop processing it.
> >
> > No, that won't work. A plugin being silent doesn't mean it's
> > passive, or even that it won't start generating output again in a
> > moment, even if there's no new input. A long feed-back delay fed
> > with a short sound would intermittently generate non-silent
> > buffers for a good while, and plugins following it could still
> > have use for the "silence flag".
>
> In my view this is not silence.  A silent plugin is one that is
> totally passive.

A silent plugin is also one that you cannot know when to start 
processing again... Unless the host eavesdrop on all event queues of 
silent plugins. Plugins could provide a callback to check all Queues, 
but then what's the point? Plugins could just do this in run() 
instead. Then they could even chose to ignore some events, if 
appropriate.


> > Besides, you cannot safely stop calling a connected plugin. If it
> > has inputs directly connected to outputs of other plugins,
> > there's no way you can know when to start calling the plugin
> > again, since you won't even know about the events sent to it's
> > Queues. What you get is a dead plugin draining your event pool.
> > :-)
>
> Agagin, hrrm.  I always envision the host snooping events.  I'll
> think some more about this.

I think our direct plugin->plugin design is much more efficient than 
anything that requires events to be sent through the host. And as 
I've pointed out before, if that works, you automatically have an API 
that allows the host to really be just a *host*, with sequencers and 
whatnot running as plugins; real or host integrated.


> > >   * EventTarget:
> > >         A tuple of an EventQueue and a cookie.  Plugins have an
> > > EventTarget for each Control, as well as a master and
> > > per-Channel EventTarget.
> >
> > Not quite. Targets aren't really "input objects" in the API, but
> > rather two other things:
>
> I'm not following you - what did I say wrong?

Well, first of all, I've been thinking some, and I'm going to propose 
the complete elimination of the event "action code" as a required 
argument. I want to use EventTargets for *everything*, as it makes 
the API cleaner, faster and simpler.

There isn't really much point in having the physical ability to send 
events without a connection, especially not when it requires an extra 
level of decoding in all plugins, as well as an exception in the API 
and semantics.

That is, there has to be one EventTarget for every *action* on every 
object in a plugin. Some EventTargets may use an argument as an 
action code, but there's no reason to force action code decoding for 
events that are fully qualified through their cookies.

Anyway, what I initially reacted against was that it sounds like these 
EventTargets are visible objects that you can just get and connect 
to. However, this is not the case physically, since the objects would 
be generally useless to plugins internally.

When you ask for an EventTarget to make a connection, the plugin just 
figures out which EventQueue to return, and encodes the address 
fields in some suitable way. (So internal tables, switch()es and 
stuff can be used for quick decoding.) This is basically just an 
implementational issue, but I don't feel good about making it sounds 
as if EventTargets as physical objects exist inside plugins. (They 
*are* physical objects on the API level and usually also internally 
in the output handling of plugins.)

Now the interesting part for the API level: A plugin might want to 
know whether it's inputs are connected or not. If so, the plugin will 
have to rely on the host using the get/release (open/close) calls 
properly, so you can't just assume calling get_input_target() doesn't 
mean anything to the plugin.

In that light, stating that the EventTarget is a physical object, and 
calling the function to get it get_*() is really rather confusing. 
(To me at least. :-)


> > 	* The physical struct that you get filled in or
> > 	  returned when you ask a plugin for one of it's
> > 	  input controls.
> >
> > 	* An abstract object inside a plugin, representing
> > 	  a control output.
>
> right - did I say something different?

Well, I *interpteted* it differently. :-) See above.


[..]
> I don't mind the model.  The names I object to.  Targets aren't
> just control.  Controls are the biggest user, but there needs to be
> a master target and a per-channel target.  Targets are, in fact,
> event targets.

Yes, and this is in total agreement with the idea of eliminating 
action codes, as outlined above.


> {open,close}_event_input()
> {connect,disconnect}_event_output() or {open,close}_event_output().

Yes. I think I prefer {connect,disconnect}_event_output(), although 
{open,close} is shorter and suggests that it's related to the input 
counterpart... {connect,disconnect} is more accurate, though, as it 
actually *is* the output that makes the physical connection.


> > > Process all events up to and including 'now'.  For a 0 frame
> > > run(), all timestamps are 'now'.
> >
> > No, timestamps are still valid, so there's no way to guarantee
> > this. All events in the queue that match the current time will be
> > processed, but any future events will be left in the queue.
>
> Sorry, I wasn't clear.  If you want to change a control
> immediately, set the change event to the current time, and run() 0
> frames.  It will process events for 'now' which will get the
> change.  It will then see that there are 0 frames to process. 

Fine.


> Alternatively, 0-frame runs are kind of special. nframes=0,
> timestamps can be 0, too.

No, this has nasty implications. You'll either need a special event 
decoding loop in plugins, or you'd have to temporarily set the 
running time field in the host struct to 0, and *require* that event 
timestamps are 0.


//David Olofson - Programmer, Composer, Open Source Advocate

.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
   --- http://olofson.net --- http://www.reologica.se ---




More information about the Linux-audio-dev mailing list