[linux-audio-dev] Plugin APIs (again)

David Olofson david at olofson.net
Tue Dec 3 09:03:01 UTC 2002


(Steve has made some good points that I agree with mostly, so I chose 
to comment on his reply.)

On Tuesday 03 December 2002 12.54, Steve Harris wrote:
> Fisrt off, I agree with Conrad, its better to make this fucused on,
> and really good at, one task (ie. instruments), that be OK at lots
> of things.

Yes. Though, I think using a virtual instruments capable API for 
effects can be great, especially when it comes to automation (both 
need to deal with accurate timing and control interpolation), but of 
course, there's always the ease-of-use/complexity trade-off; a 
virtual instruments API will always be more complex than LADSPA.


> On Mon, Dec 02, 2002 at 04:03:18 -0800, Tim Hockin wrote:
> > /* forward declarations of the fundamental OAPI types */
> > typedef struct OAPI_descriptor		OAPI_descriptor;
> > typedef struct OAPI_control		OAPI_control;
> > typedef struct OAPI_ctrl_int		OAPI_ctrl_int;
> > typedef struct OAPI_ctrl_float		OAPI_ctrl_float;
> > typedef struct OAPI_ctrl_enum		OAPI_ctrl_enum;
>
> Enum's good, I'm not so sure about int or string, a well defined
> enum type probably makes string unneccesary and int is probably not
> needed.

Well, I'm not sure it's worthwhile going beyond floats + "hints"... 
Strings could be interesting, as but how should the host deal with 
them? Are they file names, just names, scripts, or what? How to edit 
them? How to store them in "preset files" and the like?

I have a feeling that going down this route either results in complex 
hosts, or a pointless "official" API for stuff that only matters to 
plugins and their custom GUIs anyway - or both.

How about just supporting a "raw data block" control type, so that 
hosts can store the data in preset files or automation, but without 
understanding the data? Custom (G)UIs would be needed for editing the 
data - just as with VST, although the (G)UIs are not part of the 
plugin binaries here. (I think we have concluded long ago that GUI 
and DSP code should be entirely separated.)


> > /* Plugin types: all OAPI plugins are one or more of these. */
> > #define OAPI_TYPE_SOURCE		0x01
> > #define OAPI_TYPE_EFFECT		0x02
> > #define OAPI_TYPE_SINK			0x04
>
> I dont think this helps, it should be obvious by inspection whether
> an instrument is appropraite for the situation the host wants to
> fill.

Agreed. The VST API has something like this, and it seems to have 
been more harmful and confusing than helpful. Not sure if this was 
actually specified in the API originally, but people have come to 
assume that VST plugins and VSTi plugins have other implicit 
differences than the latter accepting timestamped events (like VSTis 
not being able to have audio inputs) - which results in trouble when 
people start exploring "unusual" configurations.

If it has outputs, it's a source. If it has inputs, it's a sink. If 
it has both, it's (probably) an effect. Either way, the relation 
between input and output can be of *any* kind (within the 
restrictions of the API), so there's not much point in trying to 
describe it to the host.

Likewise with timestamped "MIDI style" control events - either you 
accept them, or you don't. Whether you have audio inputs, outputs or 
both is another story entirely.


> > 	/*
> > 	 * These are display-friendly fields, which hosts can display to
> > 	 * users.
> > 	 */
> > 	const char *oapi_version;
> > 	const char *oapi_author;
> > 	const char *oapi_copyright;
> > 	const char *oapi_license;
> > 	const char *oapi_url;
> > 	const char *oapi_notes;
>
> If you're going to support metadata, support the Dublin Core, but
> generally extrnal metadata is better than internal. Just provide a
> unique ID and metadata can refer to that.

I'd definitely go for Unique IDs + external data here. Oh, and it's 
probably a good idea to structure the ID as "vendor" and "product" 
fields, so you don't have to aquire a global unique ID for every 
plugin you release... Just get a vendor ID, and then manage your 
plugin IDs yourself.


> > 	int oapi_inputs_min;
> > 	int oapi_inputs_max;
> > 	int oapi_outputs_min;
> > 	int oapi_outputs_max;
>
> Variable numbers of i/o is interesting, but I'm not sure if it
> overcomplicates things from the instruments point of view. Can you
> explain why you think this is a good idea?

Well, I don't know Tim's reasons, I could give it a try: "Horizontal" 
mixer modules, multipart synths and samplers and that kind of stuff. 
Sure, you could probably just say you have 128 ins and 128 outs (or 
whatever) - but then, would hosts assume that all are actually 
initialized and in use? I see a risk of complicating port management 
here, if ports can be "disabled" as well as "connected" or 
"disconnected". It might be better if they just don't exist if you 
don't need them.

Besides, I think fixed size arrays and the like are generally a bad 
idea, and should be avoided whenever practically possible. You alway 
hit the limit sooner or later...


> > 	int oapi_channels_min;
> > 	int oapi_channels_max;
>
> I'm pretty confident this is not the right way to go. If it matters
> to you what output belongs to what channel then well known labels
> would be better, I think.

Yeah, just as with input/output relations, there's no useful and 
reasonably simple way of describing channel/input/output relations. 
This just adds a dimension of indexing, for no gain.

Besides, you would at least have to have one pair of limits for each 
input and each output for this to be of any use. (Unless all inputs 
are required to be identical, and likewise for outputs.)

Consider a mixer, where you have insert sends and returns, as well as 
bus sends and returns. Obviously, you'd probably want to be able to 
use some 128 channels or so - but most probably not 128 busses!


> > 	/*
> > 	 * create: instantiate the plugin
>
> Maybe better to stiuck to LADSPA nomenclature? It might not be
> perfect, but it would avoid confusion.

Here's another idea (from Audiality):

typedef enum
{
	FX_STATE_CLOSED = 0,
	FX_STATE_OPEN,
	FX_STATE_READY,
	FX_STATE_PAUSED,
	FX_STATE_RUNNING,
	FX_STATE_SILENT,
	FX_STATE_RESTING
} a_fxstates_t;

typedef struct a_plugin_t
{
	...
	/* Plugin state management */
	int (*state)(struct a_plugin_t *p, a_fxstates_t new_state);
	...
} a_plugin_t;

The host is required to "climb" the state ladder, calling the 
plugin's state() callback, never skipping a state - but there's a 
handy wrapper in the API that deals with that. (So you can switch 
directly from CLOSED to RUNNING with a single call if you like.)

Dunno, but I've found this to be clean, simple and safe so far. It's 
hard to accidentally have the host confuse plugins by doing things in 
the wrong order - and it's also strictly specified what plugins may 
and may not do in each state, and when switching between any two 
adjacent states.


> > 	/*
> > 	 * set_rate: set the sample rate
>
> LADSPA definatly got this right, make it an argument to
> instantiate. Changing the sample rate of system is not an RT
> operation, so theres no need to make it ultra efficient, and
> instruments will have a very hard time rejigging everything.
> Simpler to just remove and instatiate a new one with the same
> settings.

I totally agree.


> > #if 1 //two options for note-control
> > 	/*
> > 	 * voice_on: trigger a note
> > 	 */
> > 	int (*oapi_voice_on)(void *plugin, int note);
> >
> > 	/*
> > 	 * manipulate a playing voice
> > 	 */
> > 	int (*oapi_note_off)(void *plugin, int voice);
>
> I prefer this one.
>
> After reading this I think the best solution is something very like
> LADSPA but with note_on and note_off calls. Other useful things
> like enum types /can/ be added via external metadata, eg RDF ;)

Here, I prefer timestamped events, for various reasons. Most 
importantly, timestamped events allow sample accurate control without 
the host splitting buffers, and it allows plugins to implement event 
handling in any way they like - including just executing all events 
before processing each audio buffer, for quick hacks. (You could even 
throw in a macro for that, for quick LADSPA-><new plugin API> 
porting.)

I've been using this for a while in Audiality (very similar to the 
event system designed for MAIA), and there's no turning back; 
timestamped events are simply superior, IMNSHO.

Might look a bit more complicated than direct function calls at 
first, but in my experience, it actually *simplifies* all but the 
very basic stuff. Filtering, proccesing and routing of events can be 
done by inserting plugins in between other plugins, and lots of it 
becomes trivial, as you can disregard the timestamps altogether in 
many cases.

I thought an event system would result in some overhead (linked 
lists, gloabal event pool and stuff), but in fact, the old "once per 
block" function call based implementation (*) was slower - and 
obviously, it was not sample accurate. (No buffer splitting - that 
would have slowed things down further.) It seems that staying within 
the inner loops (event processing and DSP) for one unit at a time 
more than makes up for the overhead that the event system brings.

(*) Note that that was mostly inlines messing with the internals of
    the respective unit - which obviously won't work with dynamically
    loaded plugins. There, every "action" has to be a real function
    call via function pointer. With an event system like the one in
    Audiality, there's no difference for dynamically loaded plugins,
    as you're still just passing little structs around.

Oh, BTW, events can easilly be tunnelled through whatever protocol 
you like (between threads, processe or systems), as the timing info 
is there already. Just make sure they arrive in time, and you're 
fine; order and accuracy is preserved, provided the "time base" is 
synchronized. (In a studio, you'd sync the audio interfaces to ensure 
that.)


//David Olofson - Programmer, Composer, Open Source Advocate

.- Coming soon from VaporWare Inc...------------------------.
| The Return of Audiality! Real, working software. Really!  |
| Real time and off-line synthesis, scripting, MIDI, LGPL...|
`-----------------------------------> (Public Release RSN) -'
.- M A I A -------------------------------------------------.
|    The Multimedia Application Integration Architecture    |
`----------------------------> http://www.linuxdj.com/maia -'
   --- http://olofson.net --- http://www.reologica.se ---



More information about the Linux-audio-dev mailing list