[linux-audio-dev] Synth APIs, GluttonAudio

David Olofson david at olofson.net
Fri Dec 13 15:24:01 UTC 2002


On Friday 13 December 2002 20.07, Sami P Perttu wrote:
> On Fri, 13 Dec 2002, David Olofson wrote:
> > > What is that? If a plugin got an audio port event it would just
> > > update a pointer. Past samples should be kept in temporary
> > > variables and future samples should not be looked at.
> >
> > Correct. But consider this example:
> >
> > 				out[s] = left[s] + right[s];
> > 				++s;
> >
> > Now, if you get a new buffer for left in the middle of the
> > buffer, what happens...? This is what I'd call "non-obvious
> > implications".
>
> Oops, I didn't consider that. Solution: require that pointers point
> to start of block even though there may be no data there :).

Well, you'd have to adjust the pointer yourself, since the host can't 
know how your plugin is implemented.

And BTW, the host that does this kind of stuff may well bo doing it 
as a way of avoiding data copying to merge fragments into complete 
buffers. Then, there is no obvious logic in this at all. :-)


> > IMHO, this is complicating the API for no good reason.
>
> Then we are back to square one: multiple versions of plugins that
> differ only in control and audio port details.

Why? Sample accurate buffer splits won't help, since it only applies 
to Audio Ports. You need control<->audio converters either way. There 
is no way we can merge audio and control into one kind of port 
without either doing it all as audio rate streams, or actually doing 
conversions somewhere, or forcing plugins to have "dual" inputs.


> If the plugins will,
> as you say, indeed be higher level than CSound then it won't be a
> problem.

It will be much *less* of a problem, but I'm 100% conviced that 
people will still want to do "weird" stuff with the plugins. Even VST 
users dream about this! ;-)

(They might even be doing it to some extent; dunno... MIDI 
CCs<->audio? Hahaaa! :-)


> > > I don't see anything physical about audio ports in XAP,
> > > although it might be better if they were actual objects.
> >
> > 	typedef	XAP_sample *XAP_buffer;
> >
> > Do you need an object for that? :-)
>
> No, sorry, I should make myself clearer. I was thinking about the
> pull model again.

Ah, I see. Well, see my post on asynchronous streaming; that's 
basically what you get if you want to do it in a real time context. 
(And that's not even a complete solution, as it doesn't include QoS 
control.)


> > BTW, I am indeed going to look into some sort of "process only
> > this range" feature for the rendering engine. Some sounds take a
> > while to render, and there's no point in waiting for the whole
> > sound to render when you're just tweaking the attack.
>
> ...or let the rendering run in a background process.

Doesn't work, because the current system is still totally imperative, 
and not block based. One statement at a time operates on the whole 
buffer, so no part of the waveform is ready for playback until the 
script has returned.

Speaking of which, switching to implicit full script block based 
processing might be a rather performance hack. Should reduce cache 
thrashing quite a bit!


> > How? It's not the host that sends these events in general; it's
> > other plugins. The host never touches the events in the normal
> > case.
>
> Okay. I'm lost here because I don't know what a XAP app would look
> like. I only know trackers :).

Well, it's not *that* different. Just think of the different parts of 
a tracker (MIDI input, song, pattern, "classic" effects, synth, DSP 
effects, audio output) as separate plugins, running under a host that 
does pretty much nothing but loading, connecting and running plugins. 
The sequencer becomes a plugin, rather than a part of the host, and 
you can do the same with I/O and pretty much everything.


> > > Logically, controls are like ports: they have a definite value
> > > at each point in time.
> >
> > You *could* say that - but don't forget that many plugins are
> > oversampling internally...
> >
> > internally. As a result, the filters and oscillators see zipper
> > noise
> >
> > Now, if control was seen as continous rather than sampled, it
> > would [...]
> > That said, the problem is that you cannot get away without
> > actually implementing this. If there are cubic interpolation
> > events, all [...]
> > Well, you could just use the extra argument as "slope", to
> > suggest the desired slope when "target" is reached. So, for
> > chained sections, you get <value, slope> for start and end of
> > each section. Either just ignore the slopes and do linear
> > interpolation, or do something more sophisticated, to take the
> > slopes in account as well. (Basic splines; "smoothest path".)
>
> What happens between sample values. Now we are too far in the
> hair-splitting territory :).

*hehe*

Does matter with some oversampled algos, though. They may not be very 
happy at all with the zipper-noise, since it lands well inside the 
frequency range the algo is supposed to work with. This may result in 
distortion that affects the final downsampled result.

Think of it as running all audio at 48 kHz, but updating controls 
only every 3rd sample. (The "feeping" should be pretty annoying at 
that rate.) Now, downsample to 16 kHz. Can you be *sure* that all 
distortion goes away...?


> If we are dealing with 3rd order polys then both point-slope and
> coefficient formats contain the same information. I guess we should
> pick a representation that allowed lower-order approximations to be
> made efficiently. If plugins can't handle anything but constants
> they would use the average value; linearly ramping plugins would
> use the initial value and the slope as you describe. 2nd order
> approximation is probably useless. Plugin picks the order and
> invokes a macro to get the coefficients. Conversion between 3rd
> order poly coeffs and point-slope is pretty trivial. Take your
> pick. Perhaps some spline expert could comment on this?

Yeah, suggestions are welcome. Something that easily translates to 
"coefficients" for this:

	for(...)
	{
		...
		value += dvalue;
		dvalue += ddvalue;
		ddvalue += dddvalue;
	}

or this:

	for(...)
	{
		...
		value += dvalue;
	}

where both should start and end at the same respective values.

If you have:

	start_value, start_dvalue	(from current state)
	end_value, end_dvalue		(from spline event)
	segment_length			(from spline event)

(*_dvalue are the slopes at the respective *_value), it's trivial to 
do linear:

	d_value = (end_value - start_value) / segment_lenght;
	for(...)
	{
		...
		value += dvalue;
	}

To do the a "reasonably smooth" (*) spline, you'll need to calculate 
dvalue, ddvalue and dddalue in a way that solves this:

	value[0] = start_value;
	dvalue[0] = start_dvalue;
	value[segment_length] = end_value;
	dvalue[segment_length] = end_dvalue;

value and dvalue are given, obviously. We need ddvalue and dddvalue. 
(I was looking into that before, but got interrupted, and lost track. 
Shouldn't be too complicated, though. Just a basic 1D polynomial 
spline. :-)


(*) We have to assume something reasonable as a "vague reference"
    here, so plugin authors have some useful common guidelines.
    Anything stricter than that will just make optimizations harder
    and more risky, and may have authors of control generators
    assume a bit too much about the shape of splines.


//David Olofson - Programmer, Composer, Open Source Advocate

.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
|    The Multimedia Application Integration Architecture    |
`----------------------------> http://www.linuxdj.com/maia -'
   --- http://olofson.net --- http://www.reologica.se ---



More information about the Linux-audio-dev mailing list