[linux-audio-dev] Plugin APIs (again)

Tim Hockin thockin at hockin.org
Tue Dec 10 19:48:01 UTC 2002


> > I'm not conviced Bay has the correct connotation...
> 
> Well, the intention is that it should be thought of something like a 
> physical "panel" or area on a real device, where you have a number of 
> jacks, all of the same kind.

> Maybe there's a better word for it.

There has to be - see below for some discussion on names.

> So, with Audio, you have one Channel that handles exactly one mono 
> audio stream, and that's it.

So _every_ audio 'Channel' has one i/o port?  Assuming I buy this whole
model, it seems like multiple I/O (of one flavor - I or O) is right.

> A stereo in-stereo out effect would have two input Channels (one Bay) 
> and two Output Channels (another Bay), and at least one Control Input 
> Channel with a bunch of Controls (a third Bay), the way I see it.

Doesn't that strike you as a pretty complex way of doing it?  
Easier:
  1 Master Control
  1 Stereo Audio In Channel, 1 Stereo Audio Out Channel, 1 Control Channel
Easier:
  1 Master with 2-ins, 2-outs, and controls.

> There are not different controls per Channel, but rather, you may 
> have more than one Bay of Control Input Channels - and then, each Bay 
> may have it's own set of Controls, since the Bays are completely 
> independent objects on the API level.

so different controls per Bay, which is another way of saying Channel
Template.

> Yes. However, I don't see what is made simpler with the Template 
> approach. I'm probably missing some points, since I can't even tell 
> for sure whether your Templates do provide the same number of 
> dimensions of indexing or not.

The Template idea says "all channels have 0 or more controls and 0 or more
I/Os".  The Bay idea says "A single logical use of this plug has 0 or more
instances of n Channels".  You've moved the details of what is in a Channel
(think MIDI compatible terms) into metadata and hijacked the word Channel.

> The reason is that the host would just see two normal Bays of Control 
> Channels. One just happens to be hinted "these are Voice Controls for 
> that other Bay", in case some higher levels of the host (or the user) 
> cares.

I think "of course they care!".  You've defined an implicit 1-1 mapping of
voice-control channels to channel-control channels.  And suddenly terms
collide!

> And if someone comes up with a third kind of Controls (whatever that 
> would be), they won't need explicit API or host support either. If 
> you don't care about Voice Controls at all, you don't even have to 
> know they exist.

This is a mild win.  See below.

> Hmm... One might say that if Bays are horizontal, Templates are 
> vertical. Since you can have multiple arrays of different types of 
> channels inside a Template, one could say that a Template contains a 
> bunch of something that resembles Bays.
> 
> So, how about using the "Bays or Channels" system in the low level 
> API, and then provide (optional?) information that provides the 
> grouping information of the Templates?

We can certainly make both models work together.  I suspect that almost
everyone will use the higher-layer, though, making the extra abstraction
less important.

> 	Bay 0:
> 		Type:		INPUTS, CONTROL, CHANNEL
> 		Min Channels:	1
> 		Max Channels:	1
> 		Granularity:	1
> 		Linked w/ Bay:	-1
> 		Channel Names:	{"Control Input"}
> 		Info:		<list of control names + hints>
> 
> 	Bay 2:
> 		Type:		OUTPUTS, AUDIO, FLOAT32
> 		Min Channels:	2
> 		Max Channels:	2
> 		Granularity:	2
> 		Linked w/ Bay:	0
> 		Channel Names:	{"Left Output", "Right Output"}
> 		Info:		<>

So here you have this 'Info' field.  For CONTROL bays it is a pointer to an
array of controls.  For AUDIO bays it is something you have to set to NULL.

See, this suffers from one of the same problems as ChannelTemplates as I
described:  You have to populate fields about which you are not interested.
Now we can do some icky casting:

struct BasicBay {
	unsigned long type;
	int min_channels, max_channels;
	int granularity; /* what's this for? */
	int link_to_bay;
	char **names;
}

struct ControlBay {
	struct BasicBay super;
	ControlDesc *controls;
}

struct AudioBay {
	struct BasicBay super;
}

or we can use a macro to insert the base fields into each type.  Then
plugins expose an array of BasicBays.  This is similar to how I did controls
in my original proposal.  If we one day want to add a new Bay type, we make
a new struct.  Of course, plugins which implement a newer ver of the API and
use newer types won't work in older hosts.  Hosts will have to check that.

or we can have some sort of dynamically typed bay mechanism, where plugins
register BayTypes which have an array of the names of fields, and all are
implemented as hashes, where we look up the fieldname to get info.  Or not.
:)

So I could probably be convinced this is easier than growing the
ChannelTemplate struct.  At least it won't impact older plugins in newer
hosts.  But neither will a properly grown struct (bottom).  The host will
recognize that the plugin is old and not ask for newer fields.

> Besides, Bays + Channel Type enum means you can add Channel Types to 
> the API without breaking binary or source compatibility with older 
> plugins. You might even be able to use *new* plugins in old hosts; 
> you just won't be able to use Bays of unknown types.

Binary compatibility can be maintained either way.  Source-compat can't.  If
I write a plugin Foo, and we rev XAP and grow a struct and I want my plugin
to be XAP2 compatible, I _MUST_ be aware of changes and deal with them.
Bays mean I don't have to do anything for this.  Growing a struct means I
have to add a couple initializers.  A new API version will likely mean more
changes anyway, so this is not THAT compelling

> Exactly. That's why I don't even want to *suggest* anything about 1:1 
> relationships on this level. :-)

But this is full of them.  A mixer's slot-controls bay is useless without a
corresponding audio-in.

> What I don't like about your design is basically that the set of 
> supported connection types (audio and control) is hardcoded into the 
> Template struct, rather than available as an enumeration or similar.

Right.  And I can be swayed.  I just don't want it to become so gross that
people need a diagram to figure out the object model of our API.  And it's
getting there :)  We need to describe a potentially complex relationship
between bays.  A channel-control bay needs exactly 1 voice-control bay and 1
or 2 audio-in bays.  Whew.



ABOUT NAMES
---
We have a real problem.  This discussion has exposed it.  We've run out of
clever names.  Your proposal has hijacked the word channel to mean something
different than mine.   My use of Channel is pretty close to the MIDI meaning
of Channel.  Yours is closer to an instance of a bay.

What we NEED to do is define good words, even if they are made up or
acronyms.  Good names will make or break the way people perceive the API.
The names need to be simple but obvious enough that people can remember the
names and associate what they mean.  MIDI is not so good at this :)  I hate
having to stop and think "what does he mean by Port and Channel" here.

I'm going to take a stab.  I'll rehash an ealier email.
First the bits we agree on:

* Plugin:
	A chunk of code, loaded or not, that implements this API (e.g. a .so
	file or a running instance).
* Host
	The program responsible for loading and controlling Plugins.
* Instrument/Source: 
	An instance of a Plugin that supports the instrument API and is used
	to generate audio signals.  Many Instruments will implement audio
	output but not input, though they may support both and be used an an
	Effect, too.
* Effect:
	An instance of a Plugin that supports both audio input and output.
* Output/Sink:
	An instance of a Plugin that can act as a terminator for a chain of
	Plugins.  Many Outputs will will support audio input but not output,
	though they may support both and be used as an Effect, too.
* Voice:
	A playing sound within an Instrument.  Instruments may have multiple
	Voices, or only one Voice.
* Control:
	A knob, button, slider, or virtual thing that modifies behavior of
	the Plugin.  Controls can be master (e.g. master volume),
	per-Bay (e.g. channel pressure) or per-Voice (e.g. aftertouch).

Now some questions:

* Preset:
	A stored or loaded set of values for all Controls in a Plugin.
	//FIXME: can presets be per-Bay?  per Channel (group of bays)
* Event
	A time-stamped notification of some change of a Control.
	//FIXME: are events ONLY about controls?  What isn't?

Now a debate:

* Port: 
	An audio input or output. Ports are on AUDIO Bays.
* EventQueue:
	A control input or output.  Plugins may internally have as many 
	EventQueues as they deem necessary.  For each Control, the Host will
	ask the Plugin for the EventQueue on which to deliver Events.

And the clash:

* BayTemplate: A definition of a specific type of Bay within a Plugin.
	Plugins may define as many BayTemplates as they need.  BayTemplates
	can each be one of the following types:  AUDIO:IN:FLOAT,
	AUDIO:OUT:FLOAT, CONTROL:IN:CHANNEL, CONTROL:OUT:CHANNEL,
	CONTROL:IN:VOICE, CONTROL:OUT:VOICE.  
* Bay: 
	An instance of a BayTemplate.  Plugins may have 0 or more Bays per
	BayTemplate, depending on limits set within the BayTemplate.
* ChannelDescriptor: 
	A meta-entity which describes the relationship between Bays.

These last are my terms, but they're wrong.  You call them "Bay", "Channel"
and <unnamed>.

Let's make names that work.  Brainstorming, flow of consciousness follows:

A 'BayTemplate' (you say Bay) describes a group of controls or I/Os.  It is
a place where those things get put.  That is how we got to 'Bay'.  It's a
things we hook them up to.  A PCI slot. A panel. A header.  A FooGroup.  A
slot-in descriptor.  A blade.  A PanelDescriptor.  It describes the
different panels that can be plugged in to the Plugin.  A nook.  It really
is something that describes something else.  It's a class and 'Bay' is the
instance.

A 'Bay' (you say Channel) is a group of I/Os or controls or (can be
extended) of the same type.  Alone they may not be useful.  It's an instance
of a descriptor.  It's a pad.  A slot.  A card.  An IOGroup.

uggghhh.  



More information about the Linux-audio-dev mailing list