[linux-audio-dev] Plugin APIs (again)

[Tim Hockin]

> > So _every_ audio 'Channel' has one i/o port?  Assuming I buy this
> > whole model, it seems like multiple I/O (of one flavor - I or O) is
> > right.
> 
> You could say the Channel *is* the port (either I or O) for audio, 
> whereas for Controls, it's more like MIDI and CCs; one Channel has an 
> array of Controls.

But why does it have to be that each Channel (using your terms until we find
better ones) is a single I/O?  It seems like that is a lot of meta-data and
indirection to just get a single mono port.  You mention further down about
having a Bay for "Left Audio" and "Right Audio".  There is an implicit 1-1
mapping there that is not explicity defined anywhere and is pointless.  If
you want them bonded, make the Bay be a stereo bay.

This is how I am thinking of things.  I'll use an example of an old and
unrelated bit of equipment that I think exemplified my understanding.

US Robotics used to make modem chassis' called Total Control.  It was a 6U
monster with 16 slots in the front and 16 slots in the back joined by a
midplane.  In the front slots you could put any of the cards they made for
Total Control - single modems, 4-port modem cards, 16 port modem cards, ISDN
cards, and later even HDLC, I think.  In the back part of each of these
cards you connected it's I/O card.  Not every card had I/O mind you.  I
forget all the options, but anyway.  You could have 1 POTS, 4 POTS, 16 POTS,
1 T-Span, etc.  To make the Chassis useful you added in a Control card which
had software running on it to run the chassis.  It's I/O was
10/100/1000 ethernet or fibre or Token Ring or whatever.

As I see the 'Bay Model' working - it is like this.  Each plugin is a Total
Control Chassis.  First, you have to have a Master, and the master might
have some I/O.  Then you have a (limited or unlimited) number of slots into
which you put Control Channels.  Each control Channel could have (or not) an 
I/O Channel.  There are different I/O Channels that the plugin understands
(mono, stereo, etc) and different control Channels can be used with different
I/O Channels, within limits set by the plugin.  You want a Stereo mixer
slot, you add a stereo I/O and one of the compatible control channels
(simple, 3-band EQ, 7-band EQ, etc).

It is somewhat elegant.  The only ugly bit is that the Plugin writer needs
to define a compatibility matrix somehow.

Now, I'll accept your assertion that Channel Control and Voice Control are
different Bays.  I don't see ANY reason to have I/O be mono.

Now that all that is out, I'm still not convinced that splitting my Channel
struct { I/O, channel control, voice control } is needed.

> > > The reason is that the host would just see two normal Bays of
> > > Control Channels. One just happens to be hinted "these are Voice
> > > Controls for that other Bay", in case some higher levels of the
> > > host (or the user) cares.
> >
> > I think "of course they care!".  You've defined an implicit 1-1
> > mapping of voice-control channels to channel-control channels.
> 
> Well, how explicit does it have to be? Isn't the fact that the very 
> meaning of the link feature is to tell the host (and everyone else) 
> that there *is* a 1:1 mapping sufficient? Then, how is grouping them 
> in a struct used during initialization?

The 'link feature' is ugly, IMHO :)

> *heh* Yeah, Ports would be better. (And these Event Ports should be 
> renamed Event Queues or something.)

heh - I did that below :)

> Or we can use host callbacks, or build a queue of events to describe 
> the plugin. (The latter is what the tiny MAIA prototype did.) More 
> flexible, but may not look very clean... One advantage is that if you 
> forget to send something, the host will notice that it never came in, 
> so you won't have garbage or non-intentional NULLs without knowing 
> about it.

This is pretty much what GStreamer is doing.  I don't like a ton of
'register me' functions.  Ideally a simple descriptor for a simple plugin
can be static data, and all the plugin does is return a pointer to it, or
similar SIMPLE.

> It could work for custom audio formats, custom events and that kind 
> of stuff, but even then, you'd need host support for managing buffers 
> of custom sizes. (Interleaved multichannel for inter-plugin busses 
> and that kind of stuff.)
> 
> We'll see when we've dealt with the *basic* issues. :-)

yeah - I raised it as a joke, but it's not really :)

> > > Exactly. That's why I don't even want to *suggest* anything about
> > > 1:1 relationships on this level. :-)
> >
> > But this is full of them. A mixer's slot-controls bay is useless
> > without a corresponding audio-in.
> 
> But that's something decided by the plugin author - not by the API.

I think that almost every plugin will have a number of 1:1 releationships.
Every synth will have {1 Channel Control, 1 Voice Control} associations.

> Channels. That's what the link field means; "I must have the same 
> number of Channels as Bay X." (I didn't say that anywhere? Oops...)

An icky way of describing associations, IMHO.

> Yeah. There are too many objects in this design, or too many of them 
> have inappropriate names... *heh*

Well YOU are the one who wants to add Bays, which forces me to counter with
Templates :)

> Here's another confusion. (Probably caused by me not describing what 
> "linked bays" actually mean.) A Bay would not have controls, but each 
> one of it's Channels would. 

I mis-spoke, that's what I meant.  

> Channels are indeed a lot like MIDI channels in this respect.

Except for I/O Channels which are nothing more than an I/O port described by
two separate entities.

> > * Preset:
> > 	A stored or loaded set of values for all Controls in a Plugin.
> > 	//FIXME: can presets be per-Bay?  per Channel (group of bays)
> 
> A preset should always be for the entire Plugin. A Bay is just a 

Good - that's what I thought, too.  If we assume MIDI style channels,
though (multiple configurations of the same controls), then loading a preset
into a channel might be very cool.

> > * Event
> > 	A time-stamped notification of some change of a Control.
> > 	//FIXME: are events ONLY about controls?  What isn't?
> 
> That's a good questing, actually - and it demonstrates this need to 
> deal with events that are *not* about controls. There will most 
> probably be a few of those, but frankly, I can't tell for sure 
> whether they're *really* needed or not.

Right - even voice_on/off can be control(s)

> Anyway, no, events can be about other things as well.

Example?

> I would propose that the term "Audio Port" is used for audio, and 
> that the term "Control Port" replaces this silly "Control Channel" 

A Control Channel is a Channel (your terms :) with Controls as opposed to a
channel with Audio I/O.

> thing. That way, a Port is just something you use to make a 
> connection to a Channel - regardless of whether we're talking about 
> audio or controls.

Fine - Audio Port and Control Port.  And what lives in a Control Port is an
Event Queue.

> A Channel would then refer *only* to an abstract object inside the 
> plugin.

uggh.  For the time being let's take what you've been calling a Channel (an
instance of what a Bay describes) and call it a Foo.  Now, we can agree that
a Channel is a set of Foos, probably in different Bays.  Totally abstract -
a purely relational entity.

> But there's a problem with this as well: The Channels of a Plugin 
> don't *have* to be all of the same kind... For example, a Mixer would 

> So, what's the big deal? Well, nothing - just that it might be rather 
> confusing if there can be several "Channel 3" in the same Plugin, 
> that are in fact completely different things. One could be mixer 
> strip 3, while the other would be bus strip 3...

And this is why Channels should be addressed one-dimensionally :)  A host
really wants to instantiate Channels, usually.  By instantiating a Channel,
it gets all the Foos, each in their own Bay, all associated together some
how.  Channels are addressed linearly, and Channels of the same type are not
necessarily grouped together.

So why do we need Bays and Foos again, and not just ChannelDescriptors?
Simpler is better ...

> > * EventQueue:
> > 	A control input or output.  Plugins may internally have as many
> > 	EventQueues as they deem necessary.  For each Control, the Host
> > will ask the Plugin for the EventQueue on which to deliver Events.
> 
> Yes - although referring to an EventQueue as an output might be more 
> confusing that helpful. A Plugin will *never* use an EventQueue for 
> output, but rather just send each event to whatever EventQueue the 
> host told it to send that kind of events.

Hmm, I'd been debating whether it was simpler to have the Plugin write to a
real EventQueue and have the host detach the Event list and re-attach it
where it belongs.  It's a simple operation, and it removes the need for a
host to conditionally shadow EventQueues.  It means a HAIR more work in the
host, but less complexity.  However, then it needs to ALWAYS read ALL
Control-Out ports on a Plugin.  I guess a host that wants to can do that
itself and just shadow EVERY port.  Fine, you're right, probably.  Plugins
only write Control-Output iff the Control port is connected.  Does a Host
need to disable_port() on the Control out ports, like it does audio?

> So, instead, the stereo pairs could be expressed as two Bays, linked 
> together in Port Count, both with Audio Input Ports. One Bay would be 
> named "Left Inputs" and the other "Right Inputs".

Or there is a StereoOut bay which has two Audio Ports.  I prefer this.

Or Channels.  It's simple.  The terms are already understood and even map
onto a well-known paradigm reasonably well.  The complexity goes down.  It's
at least one less object in our model.  The ONLY things we lose:

* Ability to instantiate an arbitrary number of Audio ports - bad anyway,
since plugins will all have fixed or small ranges of allowable values.

* Ability to ignore Inputs if you have none - now you need to explicity say
'0 inputs'

* Ability to later add new Bay types besides the currently known ones.  The
only thing I see this as a loss on is Audio formats, and truthfully, I don;t
care.

Simple Channels can be expanded to:

struct XAP_ctrl_channel_desc {
	char *name;
	char *label;
	unsigned flags;
	int n_controls;
	XAP_control *controls;
};

struct XAP_io_channel_desc {
	char *name;
	char *label;
	unsigned flags;
	int n_ports;
	XAP_port *ports;
};

struct XAP_Descriptor {
	...
	XAP_ctrl_channel_desc *ctrl_channel_descs;
	XAP_io_channel_desc *io_channel_descs;
	int **channel_compat; /* if (compat[ctrl][io]) */
	...
	int (*xap_new_channel)(int ctrl_idx, int io_idx);
	...
};

Tim