[linux-audio-dev] Plugin APIs (again)

David Olofson david at olofson.net
Tue Dec 10 21:48:00 UTC 2002


On Wednesday 11 December 2002 01.42, Tim Hockin wrote:
> > > I'm not conviced Bay has the correct connotation...
> >
> > Well, the intention is that it should be thought of something
> > like a physical "panel" or area on a real device, where you have
> > a number of jacks, all of the same kind.
> >
> > Maybe there's a better word for it.
>
> There has to be - see below for some discussion on names.
>
> > So, with Audio, you have one Channel that handles exactly one
> > mono audio stream, and that's it.
>
> So _every_ audio 'Channel' has one i/o port?  Assuming I buy this
> whole model, it seems like multiple I/O (of one flavor - I or O) is
> right.

You could say the Channel *is* the port (either I or O) for audio, 
whereas for Controls, it's more like MIDI and CCs; one Channel has an 
array of Controls.


> > A stereo in-stereo out effect would have two input Channels (one
> > Bay) and two Output Channels (another Bay), and at least one
> > Control Input Channel with a bunch of Controls (a third Bay), the
> > way I see it.
>
> Doesn't that strike you as a pretty complex way of doing it?

Not from the host side - but that's rather unimportant. (Stuff that 
goes into the host SDK anyway.)


> > There are not different controls per Channel, but rather, you may
> > have more than one Bay of Control Input Channels - and then, each
> > Bay may have it's own set of Controls, since the Bays are
> > completely independent objects on the API level.
>
> so different controls per Bay, which is another way of saying
> Channel Template.

Yes, that's about it. Bays can have only one array of Channels of a 
specified type, whereas Templates have one array of each supported 
Channel type, some of which may be disabled by allowing only 0 
Channels.


> > Yes. However, I don't see what is made simpler with the Template
> > approach. I'm probably missing some points, since I can't even
> > tell for sure whether your Templates do provide the same number
> > of dimensions of indexing or not.
>
> The Template idea says "all channels have 0 or more controls and 0
> or more I/Os". The Bay idea says "A single logical use of this
> plug has 0 or more instances of n Channels".  You've moved the
> details of what is in a Channel (think MIDI compatible terms) into
> metadata and hijacked the word Channel.

Yeah, that's about it. The word Channel isn't appropriate for this... 
Maybe Port or Jack would be better - but neither works very well with 
both audio and controls. Besides, those Control Channels/Ports/Jacks 
may take other events than Control events - we need to deal with 
those as well.


> > The reason is that the host would just see two normal Bays of
> > Control Channels. One just happens to be hinted "these are Voice
> > Controls for that other Bay", in case some higher levels of the
> > host (or the user) cares.
>
> I think "of course they care!".  You've defined an implicit 1-1
> mapping of voice-control channels to channel-control channels.

Well, how explicit does it have to be? Isn't the fact that the very 
meaning of the link feature is to tell the host (and everyone else) 
that there *is* a 1:1 mapping sufficient? Then, how is grouping them 
in a struct used during initialization?

Both are effectively just data as soon as the plugin is actually 
instantiated.


> And
> suddenly terms collide!

*heh* Yeah, Ports would be better. (And these Event Ports should be 
renamed Event Queues or something.)


> > And if someone comes up with a third kind of Controls (whatever
> > that would be), they won't need explicit API or host support
> > either. If you don't care about Voice Controls at all, you don't
> > even have to know they exist.
>
> This is a mild win.  See below.

Yeah, probably. We can't know in what way the API will be extended at 
this point, so preparing for it isn't worth all that much.


[...]
> or we can use a macro to insert the base fields into each type. 

Or we can use host callbacks, or build a queue of events to describe 
the plugin. (The latter is what the tiny MAIA prototype did.) More 
flexible, but may not look very clean... One advantage is that if you 
forget to send something, the host will notice that it never came in, 
so you won't have garbage or non-intentional NULLs without knowing 
about it.


> Then plugins expose an array of BasicBays.  This is similar to how
> I did controls in my original proposal.  If we one day want to add
> a new Bay type, we make a new struct.  Of course, plugins which
> implement a newer ver of the API and use newer types won't work in
> older hosts.  Hosts will have to check that.
>
> or we can have some sort of dynamically typed bay mechanism, where
> plugins register BayTypes which have an array of the names of
> fields, and all are implemented as hashes, where we look up the
> fieldname to get info.  Or not. :)

*hehe* Well, most useful types probably require some level of host 
support, so it's probably not very useful.

It could work for custom audio formats, custom events and that kind 
of stuff, but even then, you'd need host support for managing buffers 
of custom sizes. (Interleaved multichannel for inter-plugin busses 
and that kind of stuff.)

We'll see when we've dealt with the *basic* issues. :-)


[...]
> > Exactly. That's why I don't even want to *suggest* anything about
> > 1:1 relationships on this level. :-)
>
> But this is full of them. A mixer's slot-controls bay is useless
> without a corresponding audio-in.

But that's something decided by the plugin author - not by the API.

Either way, I think the issue I'm getting at here can be solved 
either way. Just have multiple Templates.


> > What I don't like about your design is basically that the set of
> > supported connection types (audio and control) is hardcoded into
> > the Template struct, rather than available as an enumeration or
> > similar.
>
> Right.  And I can be swayed.  I just don't want it to become so
> gross that people need a diagram to figure out the object model of
> our API.  And it's getting there :)

Yeah!


> We need to describe a
> potentially complex relationship between bays.  A channel-control
> bay needs exactly 1 voice-control bay and 1 or 2 audio-in bays. 
> Whew.

Actually, no. The relation is this: The channel control bay, the 
voice control bay and the audio in bay must have the same number of 
Channels. That's what the link field means; "I must have the same 
number of Channels as Bay X." (I didn't say that anywhere? Oops...)


> ABOUT NAMES
> ---
> We have a real problem.  This discussion has exposed it.  We've run
> out of clever names.

Yeah. There are too many objects in this design, or too many of them 
have inappropriate names... *heh*


> Your proposal has hijacked the word channel
> to mean something different than mine.   My use of Channel is
> pretty close to the MIDI meaning of Channel.  Yours is closer to an
> instance of a bay.
>
> What we NEED to do is define good words, even if they are made up
> or acronyms.  Good names will make or break the way people perceive
> the API.

Yes indeed.


> The names need to be simple but obvious enough that people
> can remember the names and associate what they mean.  MIDI is not
> so good at this :)  I hate having to stop and think "what does he
> mean by Port and Channel" here.
>
> I'm going to take a stab.  I'll rehash an ealier email.
> First the bits we agree on:
>
> * Plugin:
> 	A chunk of code, loaded or not, that implements this API (e.g. a
> .so file or a running instance).
> * Host
> 	The program responsible for loading and controlling Plugins.
> * Instrument/Source:
> 	An instance of a Plugin that supports the instrument API and is
> used to generate audio signals.  Many Instruments will implement
> audio output but not input, though they may support both and be
> used an an Effect, too.
> * Effect:
> 	An instance of a Plugin that supports both audio input and output.
> * Output/Sink:
> 	An instance of a Plugin that can act as a terminator for a chain
> of Plugins.  Many Outputs will will support audio input but not
> output, though they may support both and be used as an Effect, too.
> * Voice:
> 	A playing sound within an Instrument.  Instruments may have
> multiple Voices, or only one Voice.

Ok.


> * Control:
> 	A knob, button, slider, or virtual thing that modifies behavior of
> 	the Plugin.  Controls can be master (e.g. master volume),
> 	per-Bay (e.g. channel pressure) or per-Voice (e.g. aftertouch).

Here's another confusion. (Probably caused by me not describing what 
"linked bays" actually mean.) A Bay would not have controls, but each 
one of it's Channels would. Channels are indeed a lot like MIDI 
channels in this respect.


> Now some questions:
>
> * Preset:
> 	A stored or loaded set of values for all Controls in a Plugin.
> 	//FIXME: can presets be per-Bay?  per Channel (group of bays)

A preset should always be for the entire Plugin. A Bay is just a 
bunch of Channels, all of the same kind, so slicing that way would 
basically be like storing a preset for all the faders of a mixer, but 
not the other controls on each strip. (Although in the API, all strip 
controls would obiously be on the same Bay normally.)


> * Event
> 	A time-stamped notification of some change of a Control.
> 	//FIXME: are events ONLY about controls?  What isn't?

That's a good questing, actually - and it demonstrates this need to 
deal with events that are *not* about controls. There will most 
probably be a few of those, but frankly, I can't tell for sure 
whether they're *really* needed or not.

For example, you could express note on/off "events" as changes of a 
control "velocity" to positive and negative values, respectively. You 
would have to hint the control as "change trigged" rather than 
"continous", to make hosts and users aware of the fact that something 
that sends continous velocity will do strange things with this 
instrument.

Anyway, no, events can be about other things as well.


> Now a debate:
>
> * Port:
> 	An audio input or output. Ports are on AUDIO Bays.

I would propose that the term "Audio Port" is used for audio, and 
that the term "Control Port" replaces this silly "Control Channel" 
thing. That way, a Port is just something you use to make a 
connection to a Channel - regardless of whether we're talking about 
audio or controls.

A Channel would then refer *only* to an abstract object inside the 
plugin.

But there's a problem with this as well: The Channels of a Plugin 
don't *have* to be all of the same kind... For example, a Mixer would 
have Strip Channels as well as Bus Channels. The former would have a 
control port and some audio ports each, while the latter would have a 
control port and some audio ports as well, but a different set of 
controls and possibly different numbers of audio ports.

So, what's the big deal? Well, nothing - just that it might be rather 
confusing if there can be several "Channel 3" in the same Plugin, 
that are in fact completely different things. One could be mixer 
strip 3, while the other would be bus strip 3...

*Unless* all ports that belong to one "class" of Channels are kept in 
one "container" with a sensible name. Maybe that would be a Bay, 
provided that a Bay can have multiple types of Channels.


> * EventQueue:
> 	A control input or output.  Plugins may internally have as many
> 	EventQueues as they deem necessary.  For each Control, the Host
> will ask the Plugin for the EventQueue on which to deliver Events.

Yes - although referring to an EventQueue as an output might be more 
confusing that helpful. A Plugin will *never* use an EventQueue for 
output, but rather just send each event to whatever EventQueue the 
host told it to send that kind of events.

Often, the EventQueues you send to are the actual input EventQueues 
of other Plugins, but they may also be Shadow EventQueues that the 
host use to get a chance to efficiently sort/merge events when 
multiple senders are intended to send to the same EventQueue.


> And the clash:
>
> * BayTemplate: A definition of a specific type of Bay within a
> Plugin. Plugins may define as many BayTemplates as they need. 
> BayTemplates can each be one of the following types: 
> AUDIO:IN:FLOAT,
> 	AUDIO:OUT:FLOAT, CONTROL:IN:CHANNEL, CONTROL:OUT:CHANNEL,
> 	CONTROL:IN:VOICE, CONTROL:OUT:VOICE.

Yes. The BayTemplate also contains info on how many Ports (as in my 
"Audio Port" or "Control Port") the Bay may have.

BTW, granularity: That's about grouping Ports together. For example, 
stereo pairs would be expressed as a granularity of 2, which means 
that you must allocate even numbers of Ports on this Bay, and that 
all Ports in each granule belong to the same Channel.

This now appears to be a mess that is not needed if you can have 
multiple arrays of Ports of the same type, that are linked together 
by other means.

So, instead, the stereo pairs could be expressed as two Bays, linked 
together in Port Count, both with Audio Input Ports. One Bay would be 
named "Left Inputs" and the other "Right Inputs".


> * Bay:
> 	An instance of a BayTemplate.  Plugins may have 0 or more Bays per
> 	BayTemplate, depending on limits set within the BayTemplate.

My idea is to have exactly one instance of each BayTemplate.


> * ChannelDescriptor:
> 	A meta-entity which describes the relationship between Bays.
>
> These last are my terms, but they're wrong.  You call them "Bay",
> "Channel" and <unnamed>.
>
> Let's make names that work.  Brainstorming, flow of consciousness
> follows:
>
> A 'BayTemplate' (you say Bay) describes a group of controls or
> I/Os.  It is a place where those things get put.  That is how we
> got to 'Bay'.  It's a things we hook them up to.  A PCI slot. A
> panel. A header.  A FooGroup.  A slot-in descriptor.  A blade.  A
> PanelDescriptor.  It describes the different panels that can be
> plugged in to the Plugin.  A nook.  It really is something that
> describes something else.  It's a class and 'Bay' is the instance.
>
> A 'Bay' (you say Channel) is a group of I/Os or controls or (can be
> extended) of the same type.

No, that's a misunderstanding - my Bay instance is exactly what you 
describe here. (And the term Channel as I used it then would 
correspond to your Port.)


//David Olofson - Programmer, Composer, Open Source Advocate

.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
|    The Multimedia Application Integration Architecture    |
`----------------------------> http://www.linuxdj.com/maia -'
   --- http://olofson.net --- http://www.reologica.se ---



More information about the Linux-audio-dev mailing list