[linux-audio-dev] XAP spec - early scribbles

[torbenh at gmx.de]

On Thu, Feb 06, 2003 at 07:09:38PM +0100, David Olofson wrote:
> On Thursday 06 February 2003 06.11, Tim Hockin wrote:
> > > The host struct is the plugin's interface to various host
> > > provided resources. Unfortunately, making these resources
> > > thread-safe would result in a major performance hit. (Consider
> > > the event system, for example.)
> > >
> > > The easy way to deal with this is to just give plugins another
> > > host struct when you want to mess with them in another thread.
> > > The plugins won't be able to tell the difference, unless you want
> > > them to.
> >
> > At what time do we need to change out the Host struct?  I'm not
> > seeing where or why we need to do that.  Concrete example, please?
> 
> Changing any control that isn't RT safe. A delay time change that 
> requires buffers to be reallocated, for example.
> 
> However, IMHO, this turns into just another motivation for worker 
> callbacks, and even suggests that use of them should be encouraged. 
> In fact, I think worker callbacks for non RT safe operations should 
> be *required* for plugins that claim to be RT safe at all.
> 
> Problem is that the host cannot know when a plugin has received an 
> event that will trigger a non-RT safe operation. Besides, you can't 
> take a processing plugin out of the net anyway, as you'd disrupt 
> anything that passes through it. Only the plugin can deal with this 
> properly.
> 
> Taking the delay time change as an example, the plugin would do this:
> 
> 	When the offending event is received:
> 
> 		* Fire a worker callback to allocate and
> 		  initialize new buffers.

It would be nice if the worker thread was fired when the event was queued.
consider a stepsequencer sequencing the delay time.

1. clock would generate an event scheduled in 1/Hz seconds.
2. event gets processed immediately by the sequencer and queued still with
   future timestamp onto the delay.
3. queueing the event on the delay could fire the worker thread.
4. when the event is due the worker thread could already be finished
   and the buffer can be exchanged on in Realtime.

This would require some flags on the sequencer plugin, which is a pure event
processor (maybe even not because a pure event processor does not have to be in sync
with audio)



> 
> (Processing goes on as if nothing happened.)
> 
> 	When the worker callback finishes:
> 
> 		* Throw in the new buffers, change params etc.
> 
> 		* Free the old buffers. (Worker call or
> 		  host provided RT safe free() replacement.)
> 
> 
> I think this applies to most stuff that isn't RT safe, and it's the 
> only way to have plugins with non RT safe operations work correctly 
> in RT hosts. I also think implementations can be really rather simple 
> most of the time.
> 
> 
> > > > We can track silence per Port, or just per Plugin (simpler, I
> > > > think).
> > >
> > >_
> > > Simple, but insufficient for any but the simplest plugins. If you
> > > have_
> >
> > Again, I think we are speaking of slightly different things.  I am
> > talking about the time when (for example) the synth at the head of
> > the chain has stopped playing notes.  A reverb with this as it's
> > input would be told 'your input is now silent'.  It has a tail, of
> > course.  It can be marked silent when it's tail is done.  Once it
> > is silent, it does not need to be processed._
> 
> I see. I also see a big difference between synths and "effects" here. 
> It's hard to tell when you need to start processing a synth again, 
> after it's been silent. You can *guess* by looking at the input of an 
> effect plugin, but you can't even be sure in that case with some 
> effects. Such effects would just have to ignore this feature 
> altogether and be processed at all times.

the synth could tell that it wants to be processed when the event
it just received becomes due.

> 
> 
> > I don't know what use it is to mark individual buffers/Ports in
> > this case.
> 
> The point is that the effects that process the output from each 
> channel of the synth can take great advantage of knowing whether 
> input is silent or not.

yes.

> > What is the point of having 'Silent' or NULL buffers?  I mean, I
> > guess it can be a minor optimization.
> 
> It's a rather *major* optimization when it means you can return 
> without doing any processing at all... That's what Audiality's old 
> "reverb" does when it realizes the tail is out and there's no input.
> 
> 
> >  Maybe silence is a
> > per-channel thing?  I don't know that finer grain flags buy
> > anything at all..
> 
> Why channels? If it's not per audio port, it might as well be 
> anything... I think this depends entirely on plugin internals.

i am for per port silence indication.

> 
> 
> [...]
> > > > Then the
> > > > host can hot the bypass switch (is bypass optional?) or remove
> > > > it from the net, or just stop processing it.
> > >
> > > No, that won't work. A plugin being silent doesn't mean it's
> > > passive, or even that it won't start generating output again in a
> > > moment, even if there's no new input. A long feed-back delay fed
> > > with a short sound would intermittently generate non-silent
> > > buffers for a good while, and plugins following it could still
> > > have use for the "silence flag".
> >
> > In my view this is not silence.  A silent plugin is one that is
> > totally passive.
> 
> A silent plugin is also one that you cannot know when to start 
> processing again... Unless the host eavesdrop on all event queues of 
> silent plugins. Plugins could provide a callback to check all Queues, 
> but then what's the point? Plugins could just do this in run() 
> instead. Then they could even chose to ignore some events, if 
> appropriate.

the whole thing would get simpler, if the queing of events was
done with a callback on the plugin.

i suggest you look at the galan code to see what i mean.
or i will post some excerpts if you are interested.

at the Moment it uses one global event queue which
makes the performance bad if you had 2 unrelated high frequency
clocks in the mesh. But the XAP model of individual event queues
would fix this with a not small coding effort involved.



> 
> 
> > > Besides, you cannot safely stop calling a connected plugin. If it
> > > has inputs directly connected to outputs of other plugins,
> > > there's no way you can know when to start calling the plugin
> > > again, since you won't even know about the events sent to it's
> > > Queues. What you get is a dead plugin draining your event pool.
> > > :-)
> >
> > Agagin, hrrm.  I always envision the host snooping events.  I'll
> > think some more about this.
> 
> I think our direct plugin->plugin design is much more efficient than 
> anything that requires events to be sent through the host. And as 
> I've pointed out before, if that works, you automatically have an API 
> that allows the host to really be just a *host*, with sequencers and 
> whatnot running as plugins; real or host integrated.

this management stuff could also be done in a library all plugins
link to. i dont see a reason for implementing graph traversal and
ordering in every host. This is a fairly complex thing and should
not be seen by an XAP user.

Also memory management for events has to be done without malloc
this should be also be taken care of by the XAP core.

> [stuff with too much detail for me, almost new to the discussion
>  deleted, i will have a glance at your website, where was it ?]

I hope it is not too late for me to push the API to the galan model :-)

regards..

-- 
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language