[linux-audio-dev] Plugin APIs (again)

David Olofson david at olofson.net
Tue Dec 3 21:10:01 UTC 2002


On Tuesday 03 December 2002 22.58, Tim Hockin wrote:
[...unique IDs...]
> Maybe I
> can be convinced - a few people chided in on this point - can
> anyone tell me WHY?

Well, a guaranteed unique ID is really rather handy when you want to 
load up a project on another system and be *sure* that you're using 
the right plugins... That's about the only strong motivation I can 
think of right now, but it's strong enough for me.

Now, you could claim that author, plugin name and version should do, 
but there are no guarantees... Unless you throw a unique author ID in 
there *as well*. How about this:

	int author_id;		/* Plugin author's unique ID */
	const char *name;	/* Plugin name */
	int version;		/* 8:8:16 major:minor:patchlevel */

(And then whatever optional data you may want, which is not 
considered by the host.)

Plugins are assumed to be compatible/equivalent if all of the 
following are true:

	* They have identical author_id fields.
	* They have identical names.
	* They have identical major version numbers.

When loading a net, the host must also ensure that the minor version 
numbers of loaded plugins are equal to, or greater than the minor 
versions in the net description. (Older minor verisons may lack 
features, but newer versions should only *add* capabilities, that 
shouldn't matter unless you use them. New major number effectively 
means that you have a new, different plugin.)


[...variable # of ins and outs...]
> It stems from experiences with virtual drum machines and synths. 
> It is very nice to provide a plugin that does things like slicing
> (thing .rex) or drum-pads and route each pad/slice to a different
> set of FX.  Further, I envision a mixer effect which has multiple
> inputs, and potentially multiple outputs, and wants to process each
> input discretely before mixing.  I didn't think it made sense to
> make a hard limit, much less a limit of 1.  The reality is that
> MOST instruments/effects will have 1 in and/or 1 out.  Those with
> more than 1 should be presented by the host as needing special
> attention, and can make assumptions as the author desires.

>  For
> example, a drum-machine with 16 outs which has only been connected
> on the first input can assume that the other 15 pads should be
> internally routed to the first output.

IMHO, plugins should not worry about whether or not their outputs are 
connected. (In fact, there are reasons why you'd want to always 
guarantee that all ports are connected before you let a plugin run. 
Open inputs would be connected to a dummy silent output, and open 
outputs would be connected to a /dev/null equivalent.)

Indeed, one could use the "connected"/"not connected" info of ports 
as some kind of interface to the plugin, but to me, it seems way too 
limited to be of any real use. You still need a serious interface 
that lets you configure the internals of the plugin the way you want, 
just like you program a studio sampler to output stuff to the outputs 
you want. This interface may be standardized or not - or there may be 
both variants - but either way, it has to be more sophisticated than 
one bit per output.


> Perhaps it is useful to design some way for the host to specify
> this: oapi_route_output(plug, 1..15, 0);.  Or perhaps it is just
> policy that if a plug has multiple outs but only the first out is
> connected, all outs re-route to out0.  Needs more thought.

I think there should be as little policy as possible in an API. As 
in; if a plugin can assume that all ins and outs will be connected, 
there are no special cases to worry about, and thus, no need for a 
policy.

Note that you can still tell the host that you can have anything from 
1 through 16 outputs. The plugin may do whatever it likes when the 
host says "i want only one out" - including being "smart" about the 
internal routing, disregarding any user presets. (Some hardware boxes 
seem to do this in some cases, but it seems to be meant more as a 
helper when you're in that "where the h*ll is my signal!?" situation 
in the studio. A good software studio would let you turn on various 
indicators and meters all over the place instead, and preferably also 
provide a "Free Listen" feature that lets you monitor anything you 
click on.)


> > > 	int oapi_channels_min;
> > > 	int oapi_channels_max;
> >
> > I'm pretty confident this is not the right way to go. If it
> > matters to you what output belongs to what channel then well
> > known labels would be better, I think.
>
> I wanted to make sure that this API handles mono plugs, stereo
> plugs, 5.1 plugs, and whatever turns up next without re-write.  I
> kind of assumed that 1 channel = MONO, 2 = LEFT,RIGHT, 6 =
> FL,FC,FR,RL,RR,LFE or some such mapping.  Again the idea being that
> a plugin may handle more than 1 channel in different ways, and that
> each in/out may not have the same # of channels. A multi-pad drum
> machine may have a stereo snare, but a mono kick.  Maybe this is
> not a good idea, I haven' decided.  That's why I sent it here for
> discussion :)

Strongest resason *not* to use multichannel ports: They don't mix 
well with how you work in a studio. If something gives you multiple 
channels as separate mono signals, you can connect each one wherever 
you want, independently, which gives you total control without the 
need for the rest of your plugin chain to understand any multichannel 
formats.

Strongest reason *for*: When implementing it as interleaved data, it 
may server as a performance hack in some situations. It is still 
unclear to me, though, whether or not this applies to "your average 
plugin" in a real virtual studio, and even if it does, whether the 
complexity/performance ration is anywhere near justifiable. (I 
suspect not.)


> Can you expound on the well-known labels idea?  I'm
> not sure I fully understand what you're thinking.

Like on a studio mixing desk; little notes saying things like 
"bdrum", "snare upper", "snare lower", "overhead left", "overhead 
right" etc.

In a plugin API, you could standardize the way of "naming" these in 
some way, such as always marking things MONO, LEFT, RIGHT, CENTER, 
BLEFT, BRIGHT, SUB etc etc. Use a string, or a struct with some 
enums, or whatever; just put it in the API, so hosts can make sense 
of it.

Point being that if the host understands the labels, it can figure 
out what belongs together and thus may bundle mono ports together 
into "multichannel cables" on the user interface level.


[...]
> > > 	/*
> > > 	 * set_rate: set the sample rate
> >
> > LADSPA definatly got this right, make it an argument to
> > instantiate. Changing the sample rate of system is not an RT
> > operation, so theres no need to make it ultra efficient, and
> > instruments will have a very hard time rejigging everything.
> > Simpler to just remove and instatiate a new one with the same
> > settings.
>
> ok, need to rework this then.  I had done it that way but decided
> this was simpler, but I suppose you are correct :)

Well, that's the hard part with APIs. Sometimes you forget to 
consider the implications that minor API changes may have upon 
implementations. :-)



> > > 	int (*oapi_voice_on)(void *plugin, int note);
> > > 	int (*oapi_note_off)(void *plugin, int voice);
> >
> > I prefer this one.
>
> I do too, but I don't want to have 10 APIs that are purely for
> instruments and not effects.  Add to those two the need for a
> voice_ison() api for proper voice accounting of things like
> triggered samples, and eventually a voice_param() to change dynamic
> parameters per-voice (something I have not introduced in this API
> yet).  I'd like to be able to say 'increase the volume of this
> existing voice'.

Well, I don't quite understand the voice_ison() call. I think voice 
allocation best handled internally by each synth, as it's highly 
implementation dependent.

If you want more direct ("tracker style") control over voices, just 
use monophonic patches, or use some kind of "note ID" system that 
lets you address individual notes.

In Audiality, I'm currently using actual voice numbers + "owner 
fingerprints" (channel numbers, IIRC) as note IDs, but that means you 
have to wait until the voice is actually allocated until you know 
your voice ID.

Not a major problem within the engine core (*), but it means you 
can't keep track of individual notes from, say, another machine over 
LAN. Starting a voice and getting it's ID (so you can control the 
voice) requires a client/engine roundtrip, and that's a bad idea in a 
distributed real time system...

(*) Actually, there is one minor problem: Voice allocation/stealing
    is restricted to buffer granularity, since when someone in the net
    allocates a voice, there is no guarantee that that voice doesn't
    belong to someone that runs *later* in the net. This hasn't been
    a real problem so far, but I dot't like it. I want *everything*
    sample accurate! ;-)


What I'll do in some future version is probably let each channel 
allocate a range of Virtual Voice IDs (plain integers; 0..N-1), and 
then have the engine keep track of these VVIDs on a per-channel 
basis. That way, a channel can always rely on it's allocated VVIDs to 
be valid - whether or not they have real voices connected to them. In 
fact, this potentially enables the engine to "unsteal" voices - not 
that I know if that would be worth the effort. (Normally, you should 
just get more power when you run out of voices... Voice stealing - 
however elegant - is just a brutal overload handling system.)


>  Fruityloops does it, and I've found it to be
> quite conventient for things like strings and pads.   FL does
> Velocity, Pan, Filter Cut, Filter Res, and Pitch Bend.  Not sure
> which of those I want to support, but I like the idea.

"None of those, but instead, anything" would be my suggestion. I 
think it's a bad idea to "hardcode" a small number of controls into 
the API. Some kind of lose "standard" such as the MIDI CC allocation, 
could be handy, but the point is that control ports should just be 
control ports; their function is supposed to be decided by the plugin 
author.


[...]
> > From: David Olofson <david at olofson.net>
> >
> > Yes. Though, I think using a virtual instruments capable API for
> > effects can be great, especially when it comes to automation
> > (both need to deal with accurate timing and control
> > interpolation), but of course, there's always the
> > ease-of-use/complexity trade-off; a virtual instruments API will
> > always be more complex than LADSPA.
>
> I don't think it needs to be TOO much more complex.  The areas I've
> added complexity:
> * multi-in/out

Yeah, that's required.


> * multi-channel

Should be handled on the UI level, IMHO. (See above.) Doing it down 
here only complicates the connection managment for no real gain.


> * type-specific controls

Yeah... But this is one subject where I think you'll have to search 
for a long time to find even two audio hackers that agree on the same 
set of data types. ;-)


> * voice_on/off

Just a note here: Most real instruments don't have an absolute start 
or end of each note. For example, a violin has it's pitch defined as 
soon as you put your finger on the string - but when is the note-on, 
and *what* is it? I would say "bow speed" would be much more 
appropriate than on/off events.


> > Well, I'm not sure it's worthwhile going beyond floats +
> > "hints"... Strings could be interesting, as but how should the
> > host deal with them? Are they file names, just names, scripts, or
> > what? How to edit them? How to store them in "preset files" and
> > the like?
>
> String is a primitive - the control flags can specify a filename or
> a directory name.  I'm sure that filename will be the most common
> case.  It seems like String controls probably do not want to be
> automated.  But I see no reason they CAN'T be.  All this complexity
> is to make things that plugins want to say (enumerated values,
> integral values) less complex and less hacked on the back of
> something else.  As simple as makes sense and no simpler.

Well, yes. There *has* to be a set of basic types that cover 
"anything we can think of". (Very small set; probably just float and 
raw data blocks.) I'm thinking that one might be able to have some 
"conveniency types" implemented on top of the others, rather than a 
larger number of actual types.

Dunno if this makes a lot of sense - I just have a feeling that 
keeping the number of different objects in a system to a functional 
minimum is generally a good idea. What the "functional minimum" is 
here remains to see...


> > I have a feeling that going down this route either results in
> > complex hosts, or a pointless "official" API for stuff that only
> > matters to plugins and their custom GUIs anyway - or both.
>
> Custom GUIs don't even need to identify filenames as controls -
> they can do it themselves.  The point of making it a control is to
> allow them to NOT have a custom GUI

Yeah, I know. It's just that I get nervous when something tries to do 
"everything", but leaves out the "custom format" fallback for cases 
that cannot be forseen. :-)


> > How about just supporting a "raw data block" control type, so
> > that hosts can store the data in preset files or automation, but
> > without
>
> I don't get what you're devolving it to.  The point of Strings was
> to say 'I want a filename' or 'Any string will do for this field
> named SPEECH'.

Right. I think we can conclude that "raw data block" is not a 
replacement for strings, but either a special kind of strings, or a 
different type entirely.


> Maybe a raw data block is useful, but I haven't
> seen an example. The types I specified are based on experience
> using FruityLoops and similar apps and dealing with VST instruments
> and other plugins and trying to proived the simplest subset of what
> they need without oversimplifying.

Well, you can put stuff in external files, but that seems a bit risky 
to me, in some situations. Hosts should provide per-project space for 
files that should always go with the project, and some rock solid way 
of ensuring that

	1) all required files can be found and stored in an
	   archive, for moving a project to another system,
	   or for backup, and that

	2) users don't accidentally edit global data used by
	   other projects while working with a project. (This
	   is where the per-project file space comes in.)


[...]
> > Likewise with timestamped "MIDI style" control events - either
> > you accept them, or you don't. Whether you have audio inputs,
> > outputs or both is another story entirely.
>
> In my worldview MIDI is ENTIRELY the host's problem.  The
> voice_on/voice_off/voice_change API is intended to provide the
> majority of MIDI controlability.  The host can translate from
> actual MIDI.

I totally agree. I'm not suggesting that the actual MIDI protocol 
should be used in any way in the API. (Although people on the VST 
list seem to believe MIDI is great as a control protocol for plugins, 
it's never going into another plugin API, if I can help it... What a 
mess - and how many plugins actually implement more than the most 
basic stuff? *heh*)


> > I'd definitely go for Unique IDs + external data here. Oh, and
> > it's
>
> but why?

See above. (And note that I'm not so sure about the "external data" 
part, actually. I just think the ID is required for user data 
portability reasons.)


> > probably a good idea to structure the ID as "vendor" and
> > "product"
>
> I though about doing a Vendor Code.  But why burden some person
> with distributing vendor IDs and maintaining that database, when
> storing a string is just as simple.

The string has to be unique. Just Google for some common "unique" 
brand names, and you'll see what I'm worried about. :-)


>  It's not like a few bytes of
> memory are going to matter.  Now, I'm NOT trying to be flippant
> about memory usage.  I just want to be realistic.

*hehe* Well, no, the size of strings won't matter - and unless I'm 
missing something, nor does the overhead of "parsing" them.


[...]
> > > I'm pretty confident this is not the right way to go. If it
> > > matters to you what output belongs to what channel then well
> > > known labels would be better, I think.
> >
> > Yeah, just as with input/output relations, there's no useful and
> > reasonably simple way of describing channel/input/output
> > relations. This just adds a dimension of indexing, for no gain.
>
> Perhaps.  Perhaps different channels-counts per in.out is a bad
> idea.  But I see a definate need for different channels-counts on
> input vs. output.  What of a mono-izer.  Or a mono->stereo.  Or
> Stereo->5.1.

Yes, definitely. A plugin must be able to have anything from 0 and up 
of each, independently. But I prefer to see it as M mono ins and N 
mono outs, labeled in a standardized and sensible way, so you and the 
host know what they are.


> > Besides, you would at least have to have one pair of limits for
> > each input and each output for this to be of any use. (Unless all
> > inputs are required to be identical, and likewise for outputs.)
>
> Limits are limits.  Current value is different.

Right - but I'm *am* talking about limits. :-)

If you have a heavily optimized mixer plugin, it's rather likely that 
it will only support say, 8 busses (no loop for bus sends!), but 
"any" number of strips. How do you describe that using the same limit 
for both strip insert loops and bus (or master) insert loops?


[...]
> > Consider a mixer, where you have insert sends and returns, as
> > well as bus sends and returns. Obviously, you'd probably want to
> > be able to use some 128 channels or so - but most probably not
> > 128 busses!
>
> I want to avoid pure hardware paradigms.

So do I. I'm just trying to base my examples on well known equipment 
and terminology.


> But you are correct in
> that it can get complex.  Maybe it needs to be curtailed, but it
> definitely needs to support multi-channel audio.  Ideas welcome.

Multichannel == multiple mono ins and/or outs... That's how you do it 
in analog, and it's how most plugin APIs (of the kind we're 
discussing here) do it. It's simple and very flexible.


> > Here's another idea (from Audiality):
> >
> > typedef enum
> > {
> > 	FX_STATE_CLOSED = 0,
> > 	FX_STATE_OPEN,
> > 	FX_STATE_READY,
> > 	FX_STATE_PAUSED,
> > 	FX_STATE_RUNNING,
> > 	FX_STATE_SILENT,
> > 	FX_STATE_RESTING
> > } a_fxstates_t;
>
> Similar to gstreamer.  Need to comtemplate this.  Can you elucidate
> on the different states?

>From the same header:

/*
 * Plugin States
 *	Hosts are allowed to inc/dec the FXC_STATE control
 *	by only 1 unit at a time. If you want to "leap", you
 *	must work through the the states in between, to allow
 *	the plugin to do the required actions for each
 *	transition. The toolkit function a_plugin_state()
 *	handles this automatically.
 *
 *	This arrangement makes it easier to implement plugins;
 *	just check whether the incoming FXC_STATE argument is
 *	higher or lower than the current value, to select one
 *	of two switch() statements; one with code for entering
 *	states, and another for leaving states.
 *
 * What To Do in the case of a transition from...
 *	CLOSED to OPEN:
 *		Create private instance user data, and set
 *		p->user to point at it.
 *
 *	OPEN to READY:
 *		Allocate larger buffers, and buffers that
 *		depend on system parameter settings.
 *		
 *	READY to PAUSED:
 *		Here you're *not* supposed to do anything
 *		that couldn't be done in "interrupt context"
 *		or similar, as this change may be made from
 *		within the actual real time engine core.
 *		Preferably, any time consuming buffer
 *		clearing and stuff should be done in the
 *		OPEN -> READY and PAUSED -> READY
 *		transitions instead of here.
 *
 *	PAUSED to RUNNING:
 *	RUNNING to PAUSED:
 *		In most cases, you should do absolutely
 *		nothing here. If your plugin cares about the
 *		world outside, these transitions can be used
 *		to manage the real world/engine time slip
 *		when pausing.
 *
 *	PAUSED to READY:
 *		Now you may kill reverb tails and that sort
 *		of things.
 *
 *	READY to OPEN:
 *		Free any system parameter dependent buffers.
 *		Basically, free everything except what you
 *		need to store any incoming system parameter
 *		values.
 *		
 *	OPEN to CLOSED:
 *		Delete any private instance data, and set
 *		p->user to NULL.
 *
 *	The SILENT and RESTING states:
 *		These are just special cases of the RUNNING
 *		state (see below). It's *not* possible to
 *		force a plugin to either of these states.
 *		Plugins may, but do not have to, switch
 *		between the RUNNING, SILENT and RESTING
 *		states by themselves.
 *
 * What To Expect, and How To Act when in the...
 *	CLOSED state:
 *		Well, basically, you don't exist yet! Expect
 *		someone to switch you into the OPEN state,
 *		passing a fresh a_plugin_t with a NULL
 *		'user' field for you to fill in.
 *
 *	OPEN state:
 *		You must accept calls to control() for setting
 *		system parameters, but you should only store
 *		the values (or whatever you calculate from
 *		them) for later use. Other parameter changes
 *		are not allowed.
 *
 *	READY state:
 *		If system parameters are changed while in the
 *		READY state, they are expected to take effect
 *		immediately. Preferably, work as quickly as
 *		possible, but assume that no one is stupid
 *		enough to tell you to reallocate you buffers
 *		from within a real time thread.
 *		
 *	PAUSED state:
 *		In this state, there will be no calls to
 *		process() or control(). (After a thread has
 *		put a plugin in this state, it's safe to
 *		operate the plugin from another thread, and
 *		then have that thread change the state back
 *		to RUNNING when done.)
 *
 *	RUNNING state:
 *		Same as for the PAUSED state, but of course,
 *		frequent calls to process() are to be expected.
 *
 *		RUNNING, SILENT and RESTING are the *only*
 *		states in which process() may be called.
 *
 *	    IMPORTANT:
 *			Note that process*() may get NULL
 *			input buffer pointers, which is a way
 *			for the host to say that you have no
 *			input.
 *
 *		The easiest way to handle that is just grabbing
 *		the ever present a_silent_buffer (provided
 *		by the host) and go on.
 *
 *		The in-place process() call will never get
 *		NULL input buffer pointers, as it has no
 *		separate output buffer pointer. This requires
 *		a special case: the process() callback must
 *		*always* do it's work, thus effectively
 *		ignoring the SILENT and RESTING states.
 *
 *		However, the idea is that you should consider
 *		this as a useful hint! Most DSP algorithms,
 *		when given silent input, will eventually
 *		produce silent output. When that point is
 *		reached, a plugin should spontaneously switch
 *		to the SILENT state, to tell everyone that
 *		there will be no output for a while, and thus,
 *		that there's no point in calling process().
 *
 *	SILENT state:
 *		Just handle control() calls. process() will
 *		be called as usual, but you're not required to
 *		do anything with the output buffers. (Or
 *		rather, you *shouldn't*, as it would just be
 *		a waste of CPU cycles.)
 *
 *		As soon as you feel like producing output
 *		again, switch back to the RUNNING state and
 *		do so.
 *
 *		If you know that you will not produce output
 *		again, until you get fresh input, switch to
 *		the RESTING state instead!
 *
 *		Host designers should note that a plugin in
 *		the SILENT state *is* actually still running,
 *		and also that it's most probably *not*
 *		producing valid output! It's also important
 *		that output buffer pointers passed to
 *		process() are valid at all times, as a SILENT
 *		plugin may switch to the RUNNING state at any
 *		time, for no obvious reason whatsoever. Input
 *		is *not* required.)
 *
 *		That is, it will need process() to be called
 *		as usual, *even* if only to pass NULL input
 *		buffers. This is to allow plugins to keep
 *		track of time, and of course, to detect any
 *		input that might - instantaneously or after
 *		some time - make the plugin switch back to
 *		RUNNING mode.
 *
 *	RESTING state:
 *		Do nothing (except the usual handling of
 *		control() calls) until you start getting
 *		input buffers again. You're *not* allowed to
 *		switch from this state unless you get
 *		non-NULL input buffer pointers.
 *
 *		This state exists to tell the host that a
 *		plugin is *really* done with any tails and
 *		stuff, and that there will be no more output
 *		without fresh input buffers. Hosts may use
 *		this to figure out when to stop recording
 *		and things like that.
 *
 *	    IMPORTANT:	You should *NEVER* return from
 *			process() in the SILENT or RESTING
 *			states unless the buffer you just
 *			generated (or should have generated)
 *			actually *is* garbage, or silent!
 *
 *			ANY OUTPUT WILL BE IGNORED.
 *
 * General Rules for Hosts:
 *	* System Parameter changes must *never* be done from
 *	  real time context, period.
 *
 *	* Plugins should *not* be expected to be thread safe.
 *	  That is, never use a plugin from more than one thread
 *	  at a time, without making sure only one callback is
 *	  running at any time.
 */

Note that the NULL input/SILENT/RESTING deal is a bit of a quick 
hack. It doesn't deal with individual outputs, since the state is for 
the whole plugin.

(But these sorry excuses for plugins I've written so far only have 1 
in/1 out interleaved int32 stereo anyway... This stuff was originally 
designed to scale to very low end gaming machines, so the FP stuff is 
still only in the off-line synth. I'll keep extending the scalability 
upwards in future versions, which includes optional float32 
processing and mixing.)


> It seems like the same as multiple
> callbacks, just in one callback.

Well, yes - although it seemed like a cool idea at the time. I've 
been happy with this design choice so far, but I haven't written all 
that many state() callbacks yet. :-)

I'm thinking about splitting state() into state_up() and state_down() 
or something, to simplify the plugin code further. (I don't like the 
depth of indentation that two switch()es in an if() results in. Tabs 
are 8 spaces, and there should be no more than 80 characters/line. :-)


> How about params?

Parameters are "normal" controls (called "system parameters" for now) 
that you set before the appropriate state changes.


> > the wrong order - and it's also strictly specified what plugins
> > may and may not do in each state, and when switching between any
> > two adjacent states.
>
> can you post docs on this?

See above. Here's what you *don't* have to do in host code, unless 
you really feel like it:

int a_plugin_state(a_plugin_t *p, a_fxstates_t new_state)
{
	int res, s = p->current_state;
	/* We don't care about SILENT and RESTING here. */
	if(s > FX_STATE_RUNNING)
		s = FX_STATE_RUNNING;

	/* Nor do we allow forced switches to SILENT or RESTING! */
	if(new_state > FX_STATE_RUNNING)
		return -1;

	while(new_state > s)
	{
		++s;
		res = p->state(p, s);
		if(res < 0)
			return res;

		p->current_state = s;
	}
	while(new_state < s)
	{
		--s;
		res = p->state(p, s);
		if(res < 0)
			return res;

		p->current_state = s;
		switch(new_state)
		{
		  case FX_STATE_CLOSED:
			free(p->ctl);
			p->ctl = NULL;
			break;
		  case FX_STATE_OPEN:
		  case FX_STATE_READY:
		  case FX_STATE_PAUSED:
		  case FX_STATE_RUNNING:	/* Can't happen. */
		  case FX_STATE_SILENT:		/* Can't happen. */
		  case FX_STATE_RESTING:	/* Can't happen. */
			break;
		}
	}
	return 0;
}


[...timestamped event systems...]
> Can you talk more about it - I don't know the paradigm.  I'm open
> to ideas
> :)

Well, it's basically about sending structured data around, with 
timestamps telling the receiver when to process the data. As an 
example, instead of calling

	voice_start(v, ev->arg1);

directly, at exactly the right time (which would mean you have to 
split buffers for sample accurate timing), I do this:

	aev_send1(&v->port, 0, VE_START, wave);

where aev_send1() is an inline function that grabs an event struct 
from the pool, fills it in and sends it to the voice's event port. 
The sender does nothing more about it for now; it just keeps 
processing it's entire buffer and then returns. Just as if it had 
been processing only audio buffers. In fact, the timestamped events 
are very similar to audio data in that they contain both actual data 
and timing information - it's just that the timing info is explicit 
in events.
 
Then, eventually, the host decides it's time for the voice mixer to 
run (some synth specific safety checks, debug code and crap stripped):

void voice_process_mix(a_voice_t *v, int *busses[], unsigned frames)
{
	unsigned s, frag_s;

	if((VS_STOPPED == v->state) && (aev_next(&v->port, 0) > frames))
		return;	/* Stopped, and no events for this buffer --> */

	/* Loop until buffer is full, or the voice is "dead". */
	s = 0;
	while(frames)
	{
		unsigned frag_frames;
		while( !(frag_frames = aev_next(&v->port, s)) )
		{
			aev_event_t *ev = aev_read(&v->port);
			switch(ev->type)
			{
			  case VE_START:
				voice_start(v, ev->arg1);
				break;
			  case VE_STOP:
				voice_kill(v);
				aev_free(ev);
				return;	/* Back in the voice pool! --> */
			  case VE_SET:
				v->c[ev->index] = ev->arg1;
				if(VC_PITCH == ev->index)
					v->step = __calc_step(v);
				break;
			  case VE_ISET:
				v->ic[ev->index].v = ev->arg1 << RAMP_BITS;
				v->ic[ev->index].dv = 0;
				break;
			  case VE_IRAMP:
				v->ic[ev->index].dv = ev->arg1 << RAMP_BITS;
				v->ic[ev->index].dv -= v->ic[ev->index].v;
				v->ic[ev->index].dv /= ev->arg2;
				break;
			}
			aev_free(ev);
		}

		if(frag_frames > frames)
			frag_frames = frames;

		/* Handle fragmentation, end-of-waveform and looping */
		frag_s = (VS_PLAYING == v->state) ? 0 : frag_frames;
		while(frag_s < frag_frames)
		{
			unsigned offs = (s + frag_s) << 1;
			unsigned do_frames = __endframes(v, frag_frames - frag_s);
			if(do_frames)
			{
				bustab[v->fx1].in_use = 1;
				if(v->use_double)
				{
					bustab[v->fx2].in_use = 1;
					 __fragment_double(v, busses[v->fx1] + offs,
							busses[v->fx2] + offs,
							do_frames);
				}
				else
					__fragment_single(v, busses[v->fx1] + offs,
							do_frames);

				frag_s += do_frames;

				// This is just for that damn oversampling...
				if(v->position >= v->section_end)
					do_frames = 0;
			}
			if(!do_frames && !__handle_looping(v))
			{
				voice_kill(v);
				return;
			}
		}
		s += frag_frames;
		frames -= frag_frames;
	}
}

In short, this function will process the buffer it's given, stopping 
and handling any pending events at the correct time.

Indeed, there is buffer splitting done in here (or you cannot have 
sample accurate timing without going to audio rate control streams 
for all controls - deal with it), but it can be done entirely inside 
the loop, and it can be optimized in whatever way the plugin designer 
wants, including "shortcuts" such as reducing the timing accuracy so 
4 or 8 sample granularity, or whatever.


> I REALLY appreciate all the feedback.  I will incorporate the bits
> I definitely agree with and hopefully I can get more discussion
> about the bits I am unsure about.  I'm willing to be convinced if
> you are :)

Well, it's nice to see some action in this area again. (Right in time 
for the Audiality alpha release - so I might get some bonus publicity 
there. ;-)

Seriously, it's probably time to move on to the VSTi/DXi level now. 
LADSPA and JACK rule, but the integration is still "only" on the 
audio processing/routing level. We can't build a complete, seriously 
useful virtual studio, until the execution and control of synths is 
as rock solid as the audio.

I mean, being that I'm tired of sqeezing my ideas into hardware 
synths, and don't work with any "real" instruments (well, apart from 
my voice, if that counts ;-), what am I going to *record* with 
Ardour...? Audiality, among other things, I hope, and I might use 
MusE or something else to drive it - but either way, it must all be 
rock solid and ultra low latency.


//David Olofson - Programmer, Composer, Open Source Advocate

.- Coming soon from VaporWare Inc...------------------------.
| The Return of Audiality! Real, working software. Really!  |
| Real time and off-line synthesis, scripting, MIDI, LGPL...|
`-----------------------------------> (Public Release RSN) -'
.- M A I A -------------------------------------------------.
|    The Multimedia Application Integration Architecture    |
`----------------------------> http://www.linuxdj.com/maia -'
   --- http://olofson.net --- http://www.reologica.se ---



More information about the Linux-audio-dev mailing list