11 Dec
2002
11 Dec
'02
8:19 a.m.
First, I
don't understand why you want to design a "synth API". If
you want to play a note, why not instantiate a DSP network that
does the job, connect it to the main network (where system audio
outs reside), run it for a while and then destroy it? That is what
events are in my system - timed modifications to the DSP network.
99% of the synths people use these days are hardcoded, highly
optimized monoliths that are easy to use and relatively easy to host.
We'd like to support that kind of stuff on Linux as well, preferably
with an API that works equally well for effects, mixers and even
basic modular synthesis.
Besides, real time instantiation is something that most of us want to
avoid at nearly any cost. It is a *very* complex thing to get right
(ie RT safe) in any but the simplest designs. However, if
you want, you can define functions like C x =
exp((x - 9/12) * log(2)) * middleA, where middleA is another
function that takes no parameters. Then you can give pitch as "C 4"
(i.e. C in octave 4), for instance. The expression is evaluated and
when the event (= modification to DSP network) is instantiated it
becomes an input to it, constant if it is constant, linearly
interpolated at a specified rate otherwise. I should explain more
about MONKEY for this to make much sense but maybe later.
This sounds interesting and very flexible - but what's the cost? How
many voices of "real" sounds can you play at once on your average PC?
(Say, a 2 GHz P4 or someting.) Is it possible to start a sound with
sample accurate timing? How many voices would this average PC cope
with starting at the exact same time? You could think of our API as...
It seems to be a solid design so far. I will definitely comment on it when
you have a first draft for a proposal.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
11 Dec
11 Dec
12:27 p.m.
On Wednesday 11 December 2002 13.14, Sami P Perttu wrote:
[...]
That sounds a lot like a specialized event system, actually. You have
structured data - and that is essentially what events are about.
Well, the naming scheme isn't that solid... ;-)
But I think we have solved most of the hard technical problems.
(Event routing, timestamp wrapping, addressing of synth voices, pitch
control vs scales,...)
It's probably time to start working on a prototype, as a sanity check
of the design. Some things are hard to see until you actually try to
implement something.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
| The Multimedia Application Integration Architecture |
`----------------------------> http://www.linuxdj.com/maia -'
--- http://olofson.net --- http://www.reologica.se ---
This sounds
interesting and very flexible - but what's the cost?
How many voices of "real" sounds can you play at once on your
average PC? (Say, a 2 GHz P4 or someting.) Is it possible to
start a sound with sample accurate timing? How many voices would
this average PC cope with starting at the exact same time?
Well, in MONKEY I have done away with separate audio and control
signals - there is only one type of signal. However, each block of
a signal may consist of an arbitrary number of consecutive
subblocks. There are three types of subblocks: constant, linear and
data. A (say) LADSPA control signal block is equivalent to a MONKEY
signal block that has one subblock which is constant and covers the
whole block. Then there's the linear subblock type, which specifies
a value at the beginning and a per-sample delta value. The data
subblock type is just audio rate data. The native API then provides for conversion between
different types
of blocks for units that want, say, flat audio data. This is
actually less expensive and complex than it sounds.
Well, it doesn't sound tremendously expensive to me - and the point
is that you can still accept the structured data if you can do a
better job with that.
About the cost: an expression for pitch would be
evaluated, say,
100 times a second, and values in between would be linearly
interpolated, so that overhead is negligible.
I see. This is what I intend to do in Audiality later on, although it
will be more event centered and not "just" expressions. As an
alternative to the current mono, poly and sequencer "patch plugins",
there will be one that lets you code patch plugins in a byte compiled
scripting language. Timing is sample accurate, but since we're
dealing with "structured control", there's no need to evaluate once
per sample, or even once per buffer. You just do what you want when
you want.
It probably does not
matter that e.g. pitch glides are not exactly logarithmic, a
piece-wise approximation should suffice in most cases.
Yes, but there is a problem with fixed control rate, even if you can
pick one for each expression: If you set it low, you can't handle
fast transients (percussion attacks and the like), and if you set it
high, you get constantly high CPU utilization.
That's one of the main reason why I prefer timestamped events: One
less descision to make. You always have sample accurate timing when
you need it, but no cost when you don't.
I'm not sure about the overhead of the whole
system but I believe
the instantiation overhead to be small, even if you play 100 notes
a second.
Yes, the "note frequency" shouldn't be a major issue in itself; no
need to go to extremes optimizing the handling of those events.
However, even relatively simple FIR filters and the like may have
rather expensive initialization that you cannot do much about,
without instantiating "something" resident when you load the plugin.
However, I haven't measured instantiation times,
and
there certainly is some overhead. We are still talking about
standard block-based processing, though. Yes, sample accurate
timing is implemented: when a plugin is run it is given start and
end sample offsets.
As in "start processing HERE in your first buffer", and similarly for
the last buffer? Couldn't that be handled by the host, though "buffer
splitting", to avoid explicitly supporting that in every plugin?
Hmm, that might have sounded confusing, but I intend
to write a
full account of MONKEY's architecture in the near future.
Ok, that sounds like an interesting read. :-)
You could
think of our API as...
It seems to be a solid design so far. I will definitely comment on
it when you have a first draft for a proposal.
3:30 p.m.
On Wed, 11 Dec 2002, David Olofson wrote:
Hmm, that's one way of looking at it. I had thought of the subblock aspect
as something that is "peeled away" to get at the continuous signal
underneath.
Sounds cool. So these would be scripts that read and write events..? I
also have something similar in mind but writing the compiler is an effort
in itself. Especially because it has to be as fast as possible: in MONKEY
real-time control is applied by redefining functions. So when you turn a
knob an arbitrary number of expressions may have to be re-evaluated or
even reparsed. The benefit is that since basically all values are given as
expressions, the system is very flexible.
No, as in "process this block from offset x to offset y". The complexity
is hidden inside an iterator - plugins can mostly ignore it. The clever
plugin writer can also parameterize her processing for different subblock
types via C++ templates, etc.
Well, in
MONKEY I have done away with separate audio and control
signals - there is only one type of signal. However, each block of
a signal may consist of an arbitrary number of consecutive
subblocks. There are three types of subblocks: constant, linear and
data. A (say) LADSPA control signal block is equivalent to a MONKEY
signal block that has one subblock which is constant and covers the
whole block. Then there's the linear subblock type, which specifies
a value at the beginning and a per-sample delta value. The data
subblock type is just audio rate data.
That sounds a lot like a specialized event system, actually. You have
structured data - and that is essentially what events are about. About the
cost: an expression for pitch would be evaluated, say,
100 times a second, and values in between would be linearly
interpolated, so that overhead is negligible.
I see. This is what I intend to do in Audiality later on, although it
will be more event centered and not "just" expressions. As an
alternative to the current mono, poly and sequencer "patch plugins",
there will be one that lets you code patch plugins in a byte compiled
scripting language. Timing is sample accurate, but since we're
dealing with "structured control", there's no need to evaluate once
per sample, or even once per buffer. You just do what you want when
you want. Yes, but there is a problem with fixed control rate,
even if you can
pick one for each expression: If you set it low, you can't handle
fast transients (percussion attacks and the like), and if you set it
high, you get constantly high CPU utilization.
That's one of the main reason why I prefer timestamped events: One
less descision to make. You always have sample accurate timing when
you need it, but no cost when you don't.
Isn't that one more decision to make? :) What do you do in between events?
Do you have a set of prescribed envelope shapes that you can choose from,
or something else?
However, even relatively simple FIR filters and the
like may have
rather expensive initialization that you cannot do much about,
without instantiating "something" resident when you load the plugin.
True; I don't have that problem yet because I only have a class interface,
and classes can have static data.
standard
block-based processing, though. Yes, sample accurate
timing is implemented: when a plugin is run it is given start and
end sample offsets.
As in "start processing HERE in your first buffer", and similarly for
the last buffer? Couldn't that be handled by the host, though "buffer
splitting", to avoid explicitly supporting that in every plugin? It's probably time to start working on a
prototype, as a sanity check
of the design. Some things are hard to see until you actually try to
implement something.
Especially when it comes to the user interface. Ever since I started to
design the GUI I have found myself evaluating features based more on their
value to the user and less on their technical merits.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
5:05 p.m.
On Wednesday 11 December 2002 20.25, Sami P Perttu wrote:
[...]
A sort of combined "rendering language" and compressed format.
An event system with "set" and "ramp" events can do the same thing -
although it does get pretty inefficient when you want to transfer
actuall audio rate data! ;-)
Sounds cool. So these would be scripts that read and write
events..?
Yes. The same language (although interpretted) is alreay used for
rendering waveforms off-line. (Optimized for quality and flexibility,
rather than speed.) It will eventually be able to construct (or
rather, describe) simple networks that the real time part of the
scripts can control. Currently, the real time synth is little more
than a sampleplayer with an envelope generater, so there isn't much
use for "net definition code" yet. :-)
This is a good point. So far, only "set" and "linear ramp" has been
discussed, really, and that's what some of the proprietary plugin
APIs use. It seems to work well enough for most things, and in the
cases where linear is insufficient for quality reasons, plugins are
*much* better off with linear ramp input than just points with no
implied relation to the actual signal.
Why? Well, if you consider what a plugin would have to do to
interpolate at all, it becomes obvious that it either needs two
points, or a time constant. Both result in a delay - and a delay that
is not known to the host or other plugins, at that! Nor can it be
specified in the API in any useful way (some algos are more sensitive
than others), and agreeing on a way for plugins to tell the host
about their control input latency may not be easy either.
Linear ramping doesn't eliminate this problem entirely, but at least,
it lets you tell the plugin *explicitly* what kind of steepness you
have in mind.
A ramp event effectiely spans the whole time of the change, whereas
"set" events always arrive exactly when you want the target value
reached - and that's rather late if you want to avoid clicks! :-)
I see. Then you actually *have* a form of load time initialization
for plugins. The major difference for us would be that we have that
per-instance, rather than per-class.
No, as in "process this block from offset x to offset y".
Ah! Ok.
Well, of course - that's what actually matters in the end. If it does
the job; great! If not, clean designs and sophisticated solutions are
entirely worthless.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
| The Multimedia Application Integration Architecture |
`----------------------------> http://www.linuxdj.com/maia -'
--- http://olofson.net --- http://www.reologica.se ---
That sounds a
lot like a specialized event system, actually. You
have structured data - and that is essentially what events are
about.
Hmm, that's one way of looking at it. I had thought of the subblock
aspect as something that is "peeled away" to get at the continuous
signal underneath. About the cost: an expression for pitch would be
evaluated,
say, 100 times a second, and values in between would be
linearly interpolated, so that overhead is negligible.
I see. This is what I intend to do in Audiality later on,
although it will be more event centered and not "just"
expressions. As an alternative to the current mono, poly and
sequencer "patch plugins", there will be one that lets you code
patch plugins in a byte compiled scripting language. Timing is
sample accurate, but since we're dealing with "structured
control", there's no need to evaluate once per sample, or even
once per buffer. You just do what you want when you want. I also have something similar in mind but writing the
compiler is an effort in itself.
No kidding...!
Especially because it has to be as
fast as possible: in MONKEY real-time control is applied by
redefining functions. So when you turn a knob an arbitrary number
of expressions may have to be re-evaluated or even reparsed.
I prefer to think in source->compiler->code terms, to avoid getting
into these kinds of situations. (I guess the 8 and 16 bit ages still
have some effect on me. ;-)
The
benefit is that since basically all values are given as
expressions, the system is very flexible.
Yeah, that's a great idea. I'm not quite sure I see how that can
result in expressions being reparsed, though. When does this happen?
I would have thought you could just compile all expressions used in
your net, and then plugin the compiled "code". You can create, load
or modify a net, and then you compile and run.
Yes, but there
is a problem with fixed control rate, even if you
can pick one for each expression: If you set it low, you can't
handle fast transients (percussion attacks and the like), and if
you set it high, you get constantly high CPU utilization.
That's one of the main reason why I prefer timestamped events:
One less descision to make. You always have sample accurate
timing when you need it, but no cost when you don't.
Isn't that one more decision to make? :) What do you do in between
events? Do you have a set of prescribed envelope shapes that you
can choose from, or something else? However, even
relatively simple FIR filters and the like may have
rather expensive initialization that you cannot do much about,
without instantiating "something" resident when you load the
plugin.
True; I don't have that problem yet because I only have a class
interface, and classes can have static data. standard block-based processing, though. Yes, sample
accurate
timing is implemented: when a plugin is run it is given start
and end sample offsets.
As in "start processing HERE in your first buffer", and similarly
for the last buffer? Couldn't that be handled by the host, though
"buffer splitting", to avoid explicitly supporting that in every
plugin? The
complexity is hidden inside an iterator - plugins can mostly ignore
it.
Slightly harder to do that in C - but no language can avoid the
overhead. (Ooooh, and that's *some* overhead! ;-D)
The clever plugin writer can also parameterize her
processing
for different subblock types via C++ templates, etc.
That's when it *really* starts to work like an event based system. :-)
It's
probably time to start working on a prototype, as a sanity
check of the design. Some things are hard to see until you
actually try to implement something.
Especially when it comes to the user interface. Ever since I
started to design the GUI I have found myself evaluating features
based more on their value to the user and less on their technical
merits.
12 Dec
12 Dec
9:17 a.m.
On Wed, 11 Dec 2002, David Olofson wrote:
Well, real-time control is exerted like this: you tie (for instance) some
MIDI control to a function (with no parameters - it is a constant
function) - the value of the function at any time is the value of the MIDI
control. An arbitrary number of expressions for all kinds of things can
refer to the function, and thus their value can depend on it. Now if you
turn a knob on your MIDI keyboard and the control changes, the function is
effectively redefined, and any expression that depends on it has to be at
least re-evaluated. I have all kinds of glue that take care of the details
- it is not simple but not impossible either.
An event system with "set" and
"ramp" events can do the same thing -
although it does get pretty inefficient when you want to transfer
actuall audio rate data! ;-)
Yes. It seems that audio architecture design is dominated by the audio
versus control rate issue. The complexity can be shifted to various parts
in the system but it will be there as long as we want to avoid everything
being audio rate.
In XAP, then, things like filter cutoff and Q would be controls and
respond to "set" and "ramp" events. Conversions or possibly plugins
that
convert between audio rate signals and events will still be needed, I
think, but they are no problem to implement. What should be avoided is the
proliferation of plugins that perform the same task and only differ in
audio versus control rate details - CSound is a notorious example of this.
Hmm, events and subblocks are so close but not quite the same. I was
almost dreaming of abstracting everything to a single signal type.
But already the interleaving of different controls into a single event
queue is a major point in favor of events. So how about integrating audio
data into event queues as well? These events would contain a pointer to
a buffer. There - a single structured signal type that caters for
everything. Goodbye audio-only ports. Now all we need are some functions
in the API converting between set, ramp and data events, so that plugins
that can deal only in one type get what they want. And each plugin
instance would need just one input port. Is this totally crazy?
Yes. The same language (although interpretted) is
alreay used for
rendering waveforms off-line. (Optimized for quality and flexibility,
rather than speed.)
What kind of semantics does your language have? Imperative, functional,
something in between? So far, mine is just a "mathematical" expression
language with simple function definitions.
The
benefit is that since basically all values are given as
expressions, the system is very flexible.
Yeah, that's a great idea. I'm not quite sure I see how that can
result in expressions being reparsed, though. When does this happen? This is a good point. So far, only "set" and
"linear ramp" has been
discussed, really, and that's what some of the proprietary plugin
APIs use. It seems to work well enough for most things, and in the
cases where linear is insufficient for quality reasons, plugins are
*much* better off with linear ramp input than just points with no
implied relation to the actual signal.
True, true. Piece-wise linear is just fine by me. Some folks might want to
see log-linear implemented as well, I don't know. It is difficult to
optimize your DSP calculations with respect to more complex functions than
linear but I suppose the real advantage lies in not having to read stuff
in from memory all the time.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
5:18 p.m.
On Thursday 12 December 2002 14.12, Sami P Perttu wrote:
[...]
Being that it started out as little more than a list of "commands"
and arguments (as an alternative to a bunch of hardcoded C calls), it
is currently *very* imperative.
However, due to the nature of the underlying audio rendering API, it
would not be far off to interpret the scripts as net descriptions
instead of step by step instructions. Just add plugins instead of
running commands, and then run the result in real time.
I have seriously considered this approach, but I'm not sure I'll do
it that way - at least not as an implicit, only way of using the
language. (The language itself is indeed imperative; this is more a
matter of what the extension commands do.)
Well, real-time control is exerted like this: you tie (for
instance) some MIDI control to a function (with no parameters - it
is a constant function) - the value of the function at any time is
the value of the MIDI control. An arbitrary number of expressions
for all kinds of things can refer to the function, and thus their
value can depend on it. Now if you turn a knob on your MIDI
keyboard and the control changes, the function is effectively
redefined, and any expression that depends on it has to be at least
re-evaluated. I have all kinds of glue that take care of the
details - it is not simple but not impossible either.
Ok, I see.
I've considered something spline-like, with one control point or
something, but you really need 5 arguments for something useful;
<duration, start, end, start_k, end_k>. (With just one control point,
you would still have transients in between curve sections.)
That said, if curves are *chained*, you need only three arguments
<duraction, end, end_k>. <start, start_k> of the next segment will be
taken directly from the end parameters of the previous segment. (When
you use "set" events, those will be set to the target value and 0,
respectively, so "set" followed by a chain of curve segments will
work right.)
Since the actual *value* is still in there, plugins may chose to pass
curve segment events to the same case as the "ramp" events, which
would effectively just mean that the end_k argument is ignored. Or
you can approximate the curve with N linear sections internally, or
whatever.
In XAP, then, things like filter cutoff and Q would be
controls and
respond to "set" and "ramp" events. Conversions or possibly plugins
that convert between audio rate signals and events will still be
needed, I think, but they are no problem to implement.
Right, you can have Control -> Audio and vice versa - but the way
instruments are usually implemented in this kind of APIs, such
conersions are rarely needed.
What should
be avoided is the proliferation of plugins that perform the same
task and only differ in audio versus control rate details - CSound
is a notorious example of this.
Yes, but CSound opcodes are *very* low level in relation to XAP.
Still, that does not prevent a gread deal of overlap, although XAP
gets more and more expensive the more you "abuse" it - just like full
audio rate or blockless systems tend to be less efficient when used
more like "VSTi style" synths.
Hmm, events and subblocks are so close but not quite
the same. I
was almost dreaming of abstracting everything to a single signal
type. But already the interleaving of different controls into a
single event queue is a major point in favor of events.
Especially since XAP plugins may chose to get events for any Control
in any Queue they want. Have one Queue per internal loop, and there's
only one Queue to check at a time, regardless of how many Controls
there are. (This is what motivated me to go for one Queue per Plugin
in MAIA - but the XAP way is much better.)
So how
about integrating audio data into event queues as well? These
events would contain a pointer to a buffer. There - a single
structured signal type that caters for everything. Goodbye
audio-only ports.
However, Audio Ports would still exist, just like before. They have
just become abstract object, rather than physical objects, so the
Host must access them through events to the Plugin.
I mentioned in another mail that I did consider this for MAIA. I also
mentioned that it brings implementational issues for Plugins...
(Asynchronous buffer splitting is nasty to deal with.)
My conclusion has to be that for both complexity and performance
reasons, you're better off with physical Audio Ports, that
effectively are nothing but buffer pointers, maintained by the Host.
(LADSPA style.) This is simple and effective, and avoids some
complexity in Plugins.
Now all we need are some functions in the API
converting between set, ramp and data events, so that plugins that
can deal only in one type get what they want. And each plugin
instance would need just one input port. Is this totally crazy?
No, but (as explained above) there are reasons why passing buffers
via the event system, and enforcing that there be only one event
queue per plugin, are not as great ideas as they may seem at first.
Yes. The same
language (although interpretted) is alreay used for
rendering waveforms off-line. (Optimized for quality and
flexibility, rather than speed.)
What kind of semantics does your language have? Imperative,
functional, something in between? So far, mine is just a
"mathematical" expression language with simple function
definitions. The
benefit is that since basically all values are given as
expressions, the system is very flexible.
Yeah, that's a great idea. I'm not quite sure I see how that can
result in expressions being reparsed, though. When does this
happen? This is a good
point. So far, only "set" and "linear ramp" has
been discussed, really, and that's what some of the proprietary
plugin APIs use. It seems to work well enough for most things,
and in the cases where linear is insufficient for quality
reasons, plugins are *much* better off with linear ramp input
than just points with no implied relation to the actual signal.
True, true. Piece-wise linear is just fine by me. Some folks might
want to see log-linear implemented as well, I don't know. It is
difficult to optimize your DSP calculations with respect to more
complex functions than linear
Depends. Polynomial is just another addition per sample...
value += dvalue;
becomes
value += dvalue;
dvalue += ddvalue;
Calculating dvalue and ddvalue becomes more expensive, though, but I
have a feeling that might be compensated for by the lower number of
segments needed for a good approximation.
Not sure, though, at least not for the buffer sizes we're dealing
with in real time engines. I've explained in another post why you
must not assume things about the future (you don't know what will
happen in the next block), and this applies to ramp/curve events as
well!
but I suppose the real advantage lies
in not having to read stuff in from memory all the time.
Well, at least for reasonably simple calculations on fast CPUs, where
memory bandwidth is a problem.
The *major* advantage though, is when you have filters and other
algorithms for which the input -> coefficient transform is expensive.
With audio rate control input, you have to do this for every sample.
With events, you can do it once per event or so, and then internally
ramp the coefficients instead. If linear ramping is not sufficient,
extend the transform (or do it for a center point as well) so you can
use polynomial "ramping" instead.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
| The Multimedia Application Integration Architecture |
`----------------------------> http://www.linuxdj.com/maia -'
--- http://olofson.net --- http://www.reologica.se ---
13 Dec
13 Dec
9:11 a.m.
New subject: [linux-audio-dev] Synth APIs, GluttonAudio
On Thu, 12 Dec 2002, David Olofson wrote:
What is that? If a plugin got an audio port event it would just update a
pointer. Past samples should be kept in temporary variables and future
samples should not be looked at.
This is what I meant with optimizations. If you have a biquad with a
linearly changing cutoff frequency, for instance, you can optimize away
calls to trigonometric functions inside the inner loop without resorting
to approximation; just use the old "spinning point on the unit circle"
trick. Hmm, I realize now that the same can be done with a polynomial of
any degree but the computational benefit gets smaller and smaller. For
this reason it might make sense to just stick to constant values and
linear ramps. You could then have plugins that generated piece-wise linear
approximations of splines and the like with any accuracy desired.
So: I vote for constant and linear controls.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
Still, that does not prevent a gread deal of overlap,
although XAP
gets more and more expensive the more you "abuse" it - just like full
audio rate or blockless systems tend to be less efficient when used
more like "VSTi style" synths.
I thought XAP was intended to replace LADSPA (in addition to extending
it).
about
integrating audio data into event queues as well? These
events would contain a pointer to a buffer. There - a single
[...]
I mentioned in another mail that I did consider this for MAIA. I also
mentioned that it brings implementational issues for Plugins...
(Asynchronous buffer splitting is nasty to deal with.) My conclusion has to be that for both complexity and
performance
reasons, you're better off with physical Audio Ports, that
effectively are nothing but buffer pointers, maintained by the Host.
(LADSPA style.) This is simple and effective, and avoids some
complexity in Plugins.
Point conceded, although some of your arguments apply to controls as well.
I don't see anything physical about audio ports in XAP, although it might
be better if they were actual objects.
Being that it started out as little more than a list
of "commands"
and arguments (as an alternative to a bunch of hardcoded C calls), it
is currently *very* imperative.
However, due to the nature of the underlying audio rendering API, it
would not be far off to interpret the scripts as net descriptions
instead of step by step instructions. Just add plugins instead of
running commands, and then run the result in real time.
I have seriously considered this approach, but I'm not sure I'll do
it that way - at least not as an implicit, only way of using the
language. (The language itself is indeed imperative; this is more a
matter of what the extension commands do.)
Okay, I can see how that would be useful. I have a graphical network
editor in GluttonAudio (I've just decided to name it that) but without any
scripting it's not flexible enough. When you start adding script plugins,
however...
I've considered something spline-like, with one
control point or
something, but you really need 5 arguments for something useful;
<duration, start, end, start_k, end_k>. (With just one control point,
you would still have transients in between curve sections.)
That said, if curves are *chained*, you need only three arguments
<duraction, end, end_k>. <start, start_k> of the next segment will be
taken directly from the end parameters of the previous segment. (When
you use "set" events, those will be set to the target value and 0,
respectively, so "set" followed by a chain of curve segments will
work right.)
Since the actual *value* is still in there, plugins may chose to pass
curve segment events to the same case as the "ramp" events, which
would effectively just mean that the end_k argument is ignored. Or
you can approximate the curve with N linear sections internally, or
whatever.
Logically, controls are like ports: they have a definite value at each
point in time. So it would make sense to have a XAP_control_variable type
or something whose value could be set (via a macro) from an event and
whose value could be updated (also via a macro) after each sample is
processed.
Whatever the sections end up being, spline calculations should be done by
the host. What the plugin would see are just polynomial coefficients. With
third-order sections you get continuity of first derivatives, etc.
Example (well this is just copy-pasting from your post):
struct XAP_control_variable {
double value, dvalue, ddvalue, dddvalue;
};
typedef struct XAP_control_variable XAP_control_variable;
#define XAP_SET_CTRL_VAR (var, event) (var).value = (event) -> value; \
(var).dvalue = (event) -> dvalue; \
(var).ddvalue = (event) -> ddvalue; \
(var).dddvalue = (event) -> dddvalue;
#define XAP_UPDATE_CTRL_VAR (var) (var).value += (var).dvalue; \
(var).dvalue += (var).ddvalue; \
(var).ddvalue += (var).dddvalue;
It is
difficult to optimize your DSP calculations with respect to more
complex functions than linear
The *major* advantage though, is when you have filters and other
algorithms for which the input -> coefficient transform is expensive.
With audio rate control input, you have to do this for every sample.
With events, you can do it once per event or so, and then internally
ramp the coefficients instead. If linear ramping is not sufficient,
extend the transform (or do it for a center point as well) so you can
use polynomial "ramping" instead.
11:35 a.m.
New subject: [linux-audio-dev] Synth APIs, GluttonAudio
On Friday 13 December 2002 14.06, Sami P Perttu wrote:
What is that? If a plugin got an audio port event it would just
update a pointer. Past samples should be kept in temporary
variables and future samples should not be looked at.
Correct. But consider this example:
process(..., int frames)
{
int s = 0;
while(frames)
{
...process events...
for(samples until next event)
{
out[s] = left[s] + right[s];
++s;
}
}
}
Now, if you get a new buffer for left in the middle of the buffer,
what happens...? This is what I'd call "non-obvious implications".
Sure, you can deal with this by having one index for each buffer, or
simply incrementing the pointers, but that impacts performance in the
inner loops. (Yes, incrementing pointers is a bad idea on most
reasonably modern CPUs! They're designed for indexing.)
I'm pretty sure most plugins would crash the first time a host
actually makes use of this feature, unless possibly if you keep
yelling at all plugin developers all the time.
IMHO, this is complicating the API for no good reason.
Yes, but controls are *intended* to be sample accurate, and for good
reasons. Deal with it, or use LADSPA.
There are (IMNSHO) no good reasons whatsoever to support sample
accurate buffer splitting on a per-Port basis. (And I'm the one who
introduced this very concept when MuCoS/MAIA was discussed...)
Yeah... I started in the other end, basically. What I have in mind
doing next is a sort of "smart" text editor, that understands the
commands of the rendering backend, and lets you view and edit them
graphically, as little panels that replace them in the source. That
way, you can edit envelopes and stuff, even inside functions and
control flow statements.
[...chained ramp events...]
On Thu, 12 Dec 2002, David Olofson wrote:
Well, it might - unless people consider XAP too complex for their
plugins.
Obiously, we'll try to make XAP as simple, efficient and easy to use
as realistically possible - but given the requirements, it's not
realistic to make it as simple as LADSPA.
Still, that does not prevent a gread deal of
overlap, although
XAP gets more and more expensive the more you "abuse" it - just
like full audio rate or blockless systems tend to be less
efficient when used more like "VSTi style" synths.
I thought XAP was intended to replace LADSPA (in addition to
extending it). about integrating audio data into event queues as
well? These
events would contain a pointer to a buffer. There - a single
[...]
I mentioned in another mail that I did consider this for MAIA. I
also mentioned that it brings implementational issues for
Plugins... (Asynchronous buffer splitting is nasty to deal with.) My conclusion
has to be that for both complexity and performance
reasons, you're better off with physical Audio Ports, that
effectively are nothing but buffer pointers, maintained by the
Host. (LADSPA style.) This is simple and effective, and avoids
some complexity in Plugins.
Point conceded, although some of your arguments apply to controls
as well. I don't see anything physical about audio ports in
XAP,
although it might be better if they were actual objects.
Well...
typedef XAP_sample *XAP_buffer;
Do you need an object for that? :-)
Point is, plugins will probably need their own per-Port data
internally, and since we cannot know what they need, there is no
point in suggesting anything in the API. Let Audio Ports be abstract
objects, and just pass a pointer to an array of buffer pointers when
you connect an array of Audio Ports.
> Being that it started out as little more than a
list of
> "commands" and arguments (as an alternative to a bunch of
> hardcoded C calls), it is currently *very* imperative.
>
> However, due to the nature of the underlying audio rendering API,
> it would not be far off to interpret the scripts as net
> descriptions instead of step by step instructions. Just add
> plugins instead of running commands, and then run the result in
> real time.
BTW, I am indeed going to look into some sort of "process only this
range" feature for the rendering engine. Some sounds take a while to
render, and there's no point in waiting for the whole sound to render
when you're just tweaking the attack.
I have
seriously considered this approach, but I'm not sure I'll
do it that way - at least not as an implicit, only way of using
the language. (The language itself is indeed imperative; this is
more a matter of what the extension commands do.)
Okay, I can see how that would be useful. I have a graphical
network editor in GluttonAudio (I've just decided to name it that)
but without any scripting it's not flexible enough. When you start
adding script plugins, however... Logically, controls are like ports: they have a
definite value at
each point in time.
You *could* say that - but don't forget that many plugins are
oversampling internally...
Even Audiality's off-line engine is affected. Envelopes generate
audio rate output, while some oscillators and filters are oversampled
internally. As a result, the filters and oscillators see zipper noise
on their inputs, unless they filter or ramp interally in some way,
just like they should do with the audio input.
Now, if control was seen as continous rather than sampled, it would
just have been a matter of applying the ramp events at the internal
sample rate. More accurate, easier, and cheaper.
So it would make sense to have a
XAP_control_variable type or something whose value could be set
(via a macro) from an event and whose value could be updated (also
via a macro) after each sample is processed.
Yeah, that's an idea I considered for MAIA; wrapping most of the
standard event handling in the plugin SDK. It could work for
oversampled inner loops as well, if done right. (Just a matter of
being able to scale the "duration" parameter before the "set up ramp"
macro gets at it.)
Whatever the sections end up being, spline
calculations should be
done by the host.
How? It's not the host that sends these events in general; it's other
plugins. The host never touches the events in the normal case.
What the plugin would see are just polynomial
coefficients. With third-order sections you get continuity of first
derivatives, etc.
Events should probably contain something less specific that
polynomial coefficients. Many plugins can't use them directly anyway,
since they have to ramp/interpolate internally for performance
reasons.
OTOH, calculating a few points from given polynomial coefficients
isn't all that hard... Just multiply with x, x*x and x*x*x, where x
is the offset from the start of the section to the point you want, in
sample frames.
That said, the problem is that you cannot get away without actually
implementing this. If there are cubic interpolation events, all
plugins must support them, or there will be lots of little converter
plugins everywhere...! :-)
[...]
This is what I meant with optimizations. If you have a
biquad with
a linearly changing cutoff frequency, for instance, you can
optimize away calls to trigonometric functions inside the inner
loop without resorting to approximation; just use the old "spinning
point on the unit circle" trick. Hmm, I realize now that the same
can be done with a polynomial of any degree but the computational
benefit gets smaller and smaller. For this reason it might make
sense to just stick to constant values and linear ramps. You could
then have plugins that generated piece-wise linear approximations
of splines and the like with any accuracy desired.
So: I vote for constant and linear controls.
Well, you could just use the extra argument as "slope", to suggest
the desired slope when "target" is reached. So, for chained sections,
you get <value, slope> for start and end of each section. Either just
ignore the slopes and do linear interpolation, or do something more
sophisticated, to take the slopes in account as well. (Basic splines;
"smoothest path".)
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
| The Multimedia Application Integration Architecture |
`----------------------------> http://www.linuxdj.com/maia -'
--- http://olofson.net --- http://www.reologica.se ---
3:12 p.m.
New subject: [linux-audio-dev] Synth APIs, GluttonAudio
On Fri, 13 Dec 2002, David Olofson wrote:
Oops, I didn't consider that. Solution: require that pointers point to
start of block even though there may be no data there :).
No, sorry, I should make myself clearer. I was thinking about the pull
model again.
What is that?
If a plugin got an audio port event it would just
update a pointer. Past samples should be kept in temporary
variables and future samples should not be looked at.
Correct. But consider this example:
out[s] = left[s] + right[s];
++s;
Now, if you get a new buffer for left in the middle of the buffer,
what happens...? This is what I'd call "non-obvious implications". IMHO, this is complicating the API for no good reason.
Then we are back to square one: multiple versions of plugins that differ
only in control and audio port details. If the plugins will, as you say,
indeed be higher level than CSound then it won't be a problem.
I don't
see anything physical about audio ports in XAP,
although it might be better if they were actual objects.
typedef XAP_sample *XAP_buffer;
Do you need an object for that? :-) BTW, I am indeed going to look into some sort of
"process only this
range" feature for the rendering engine. Some sounds take a while to
render, and there's no point in waiting for the whole sound to render
when you're just tweaking the attack.
...or let the rendering run in a background process.
How? It's not the host that sends these events in
general; it's other
plugins. The host never touches the events in the normal case.
Okay. I'm lost here because I don't know what a XAP app would look like. I
only know trackers :).
Logically,
controls are like ports: they have a definite value at
each point in time.
You *could* say that - but don't forget that many plugins are
oversampling internally... internally. As a result, the filters and oscillators
see zipper noise
Now, if control was seen as continous rather than sampled, it would
[...]
That said, the problem is that you cannot get away without actually
implementing this. If there are cubic interpolation events, all
[...]
Well, you could just use the extra argument as "slope", to suggest
the desired slope when "target" is reached. So, for chained sections,
you get <value, slope> for start and end of each section. Either just
ignore the slopes and do linear interpolation, or do something more
sophisticated, to take the slopes in account as well. (Basic splines;
"smoothest path".)
What happens between sample values. Now we are too far in the
hair-splitting territory :).
If we are dealing with 3rd order polys then both point-slope and
coefficient formats contain the same information. I guess we should pick a
representation that allowed lower-order approximations to be made
efficiently. If plugins can't handle anything but constants they would use
the average value; linearly ramping plugins would use the initial value
and the slope as you describe. 2nd order approximation is probably
useless. Plugin picks the order and invokes a macro to get the
coefficients. Conversion between 3rd order poly coeffs and point-slope is
pretty trivial. Take your pick. Perhaps some spline expert could comment
on this?
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
4:22 p.m.
New subject: [linux-audio-dev] Synth APIs, GluttonAudio
On Friday 13 December 2002 20.07, Sami P Perttu wrote:
Why? Sample accurate buffer splits won't help, since it only applies
to Audio Ports. You need control<->audio converters either way. There
is no way we can merge audio and control into one kind of port
without either doing it all as audio rate streams, or actually doing
conversions somewhere, or forcing plugins to have "dual" inputs.
No, sorry, I should make myself clearer. I was thinking about the
pull model again.
Ah, I see. Well, see my post on asynchronous streaming; that's
basically what you get if you want to do it in a real time context.
(And that's not even a complete solution, as it doesn't include QoS
control.)
Doesn't work, because the current system is still totally imperative,
and not block based. One statement at a time operates on the whole
buffer, so no part of the waveform is ready for playback until the
script has returned.
Speaking of which, switching to implicit full script block based
processing might be a rather performance hack. Should reduce cache
thrashing quite a bit!
Well, it's not *that* different. Just think of the different parts of
a tracker (MIDI input, song, pattern, "classic" effects, synth, DSP
effects, audio output) as separate plugins, running under a host that
does pretty much nothing but loading, connecting and running plugins.
The sequencer becomes a plugin, rather than a part of the host, and
you can do the same with I/O and pretty much everything.
What happens between sample values. Now we are too far in the
hair-splitting territory :).
*hehe*
Does matter with some oversampled algos, though. They may not be very
happy at all with the zipper-noise, since it lands well inside the
frequency range the algo is supposed to work with. This may result in
distortion that affects the final downsampled result.
Think of it as running all audio at 48 kHz, but updating controls
only every 3rd sample. (The "feeping" should be pretty annoying at
that rate.) Now, downsample to 16 kHz. Can you be *sure* that all
distortion goes away...?
On Fri, 13 Dec 2002, David Olofson wrote:
Oops, I didn't consider that. Solution: require that pointers point
to start of block even though there may be no data there :).
Well, you'd have to adjust the pointer yourself, since the host can't
know how your plugin is implemented.
And BTW, the host that does this kind of stuff may well bo doing it
as a way of avoiding data copying to merge fragments into complete
buffers. Then, there is no obvious logic in this at all. :-)
What is
that? If a plugin got an audio port event it would just
update a pointer. Past samples should be kept in temporary
variables and future samples should not be looked at.
Correct. But consider this example:
out[s] = left[s] + right[s];
++s;
Now, if you get a new buffer for left in the middle of the
buffer, what happens...? This is what I'd call "non-obvious
implications". IMHO, this is
complicating the API for no good reason.
Then we are back to square one: multiple versions of plugins that
differ only in control and audio port details. If the plugins will,
as you say, indeed be higher level than CSound then it won't be a
problem.
It will be much *less* of a problem, but I'm 100% conviced that
people will still want to do "weird" stuff with the plugins. Even VST
users dream about this! ;-)
(They might even be doing it to some extent; dunno... MIDI
CCs<->audio? Hahaaa! :-)
I don't see anything physical about audio ports in
XAP,
although it might be better if they were actual objects.
typedef XAP_sample *XAP_buffer;
Do you need an object for that? :-) BTW, I am
indeed going to look into some sort of "process only
this range" feature for the rendering engine. Some sounds take a
while to render, and there's no point in waiting for the whole
sound to render when you're just tweaking the attack.
...or let the rendering run in a background process. How? It's
not the host that sends these events in general; it's
other plugins. The host never touches the events in the normal
case.
Okay. I'm lost here because I don't know what a XAP app would look
like. I only know trackers :). Logically, controls are like ports: they have a
definite value
at each point in time.
You *could* say that - but don't forget that many plugins are
oversampling internally...
internally. As a result, the filters and oscillators see zipper
noise
Now, if control was seen as continous rather than sampled, it
would [...]
That said, the problem is that you cannot get away without
actually implementing this. If there are cubic interpolation
events, all [...]
Well, you could just use the extra argument as "slope", to
suggest the desired slope when "target" is reached. So, for
chained sections, you get <value, slope> for start and end of
each section. Either just ignore the slopes and do linear
interpolation, or do something more sophisticated, to take the
slopes in account as well. (Basic splines; "smoothest path".) If we are dealing with 3rd order polys then both
point-slope and
coefficient formats contain the same information. I guess we should
pick a representation that allowed lower-order approximations to be
made efficiently. If plugins can't handle anything but constants
they would use the average value; linearly ramping plugins would
use the initial value and the slope as you describe. 2nd order
approximation is probably useless. Plugin picks the order and
invokes a macro to get the coefficients. Conversion between 3rd
order poly coeffs and point-slope is pretty trivial. Take your
pick. Perhaps some spline expert could comment on this?
Yeah, suggestions are welcome. Something that easily translates to
"coefficients" for this:
for(...)
{
...
value += dvalue;
dvalue += ddvalue;
ddvalue += dddvalue;
}
or this:
for(...)
{
...
value += dvalue;
}
where both should start and end at the same respective values.
If you have:
start_value, start_dvalue (from current state)
end_value, end_dvalue (from spline event)
segment_length (from spline event)
(*_dvalue are the slopes at the respective *_value), it's trivial to
do linear:
d_value = (end_value - start_value) / segment_lenght;
for(...)
{
...
value += dvalue;
}
To do the a "reasonably smooth" (*) spline, you'll need to calculate
dvalue, ddvalue and dddalue in a way that solves this:
value[0] = start_value;
dvalue[0] = start_dvalue;
value[segment_length] = end_value;
dvalue[segment_length] = end_dvalue;
value and dvalue are given, obviously. We need ddvalue and dddvalue.
(I was looking into that before, but got interrupted, and lost track.
Shouldn't be too complicated, though. Just a basic 1D polynomial
spline. :-)
(*) We have to assume something reasonable as a "vague reference"
here, so plugin authors have some useful common guidelines.
Anything stricter than that will just make optimizations harder
and more risky, and may have authors of control generators
assume a bit too much about the shape of splines.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
| The Multimedia Application Integration Architecture |
`----------------------------> http://www.linuxdj.com/maia -'
--- http://olofson.net --- http://www.reologica.se ---
16 Dec
16 Dec
10 a.m.
New subject: [linux-audio-dev] XAP: What is it for
On Fri, 13 Dec 2002, David Olofson wrote:
[...valid points...]
Well, it's not *that* different. Just think of the different parts of
a tracker (MIDI input, song, pattern, "classic" effects, synth, DSP
effects, audio output) as separate plugins, running under a host that
does pretty much nothing but loading, connecting and running plugins.
The sequencer becomes a plugin, rather than a part of the host, and
you can do the same with I/O and pretty much everything.
My first reaction to reading the above paragraph was "you are insane".
That is a very ambitious plan! Would it really be possible to accomplish
all that with one API...? Is it possible to design it without a divine
intervention...? In the end I fear you may find the host doing a whole lot
more than "pretty much nothing". For instance, the metadata needed to
describe all this could easily get out of hand. No metadata = no UI =
useless API.
If I were me I would keep XAP a low-level block processing API. The idea
of audio ports on the one hand and sample-accurate control events
interleaved in a single queue on the other is ingenious. Add to that voice
allocation and well defined metadata capabilities and we have a backbone
for such a softstudio as you outline above. Then design more APIs on top
of it as needed. Don't let the event aspect fool you into thinking it is
meaningful to make XAP encompass everything.
Summa theologica: dump musical time, note pitch, asynchronous I/O. Use Hz
for pitch.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
How?
It's not the host that sends these events in general; it's
other plugins. The host never touches the events in the normal
case.
Okay. I'm lost here because I don't know what a XAP app would look
like. I only know trackers :).
8:38 p.m.
New subject: [linux-audio-dev] XAP: What is it for
On Monday 16 December 2002 21.24, Tim Goetze wrote:
Sami P Perttu wrote:
Right; that would basically give you LADSPA + buffer splitting for
sample accurate timing, only more efficient.
We must aim higher than that, or there's no point in a new API.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
Summa theologica: dump musical time, note pitch,
asynchronous I/O.
Use Hz for pitch.
and never even think of using the system to actually
produce music.
17 Dec
17 Dec
8:04 a.m.
New subject: [linux-audio-dev] XAP: What is it for
On Mon, 16 Dec 2002, Tim Goetze wrote:
That's true, it would require something built on top of it to be musically
useful. I'm just concerned that you/we will run out of power with XAP - it
has some inherent limitations, namely a) it is block-based and b) it is
causal. But I understand that you/we can get far enough by including the
above items that it could be worth trying. I've voiced my opinion now so
I'll concentrate my efforts on helping in what little way I can.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
Summa
theologica: dump musical time, note pitch, asynchronous I/O. Use Hz
for pitch.
and never even think of using the system to actually
produce music.
tim 
11:21 a.m.
New subject: [linux-audio-dev] XAP: What is it for
Sami P Perttu wrote:
> and never even think of using the system to
actually
> produce music.
That's true, it would require something built on
top of it to be musically
useful.
david is right in saying that we already have ladspa for audio
only i think. if we basically just add sample-accurate events,
we'll need yet another layer/API to deal with the implications
of musical time. i think it is sensible to assume that one API
can be made to handle both; at least i'm running a system that
does to my, arguably subjective, content.
I'm just concerned that you/we will run out of
power with XAP - it
has some inherent limitations, namely a) it is block-based and b) it is
causal.
both are very much required by the technical foundation it is
to be designed to work on i fear.
But I understand that you/we can get far enough by
including the
above items that it could be worth trying. I've voiced my opinion now so
I'll concentrate my efforts on helping in what little way I can.
i like your stance.
tim
16 Dec
16 Dec
8:30 p.m.
New subject: [linux-audio-dev] XAP: What is it for
On Monday 16 December 2002 14.55, Sami P Perttu wrote:
On Fri, 13 Dec 2002, David Olofson wrote:
Well, that's probably true, but I'm not sure it's relevant. ;-)
[...valid points...]
Well, it's not *that* different. Just think of the different
parts of a tracker (MIDI input, song, pattern, "classic" effects,
synth, DSP effects, audio output) as separate plugins, running
under a host that does pretty much nothing but loading,
connecting and running plugins. The sequencer becomes a plugin,
rather than a part of the host, and you can do the same with I/O
and pretty much everything.
My first reaction to reading the above paragraph was "you are
insane". How?
It's not the host that sends these events in general;
it's other plugins. The host never touches the events in the
normal case.
Okay. I'm lost here because I don't know what a XAP app would
look like. I only know trackers :). That is a very ambitious plan!
Yes. Guess why MuCoS/MAIA never got off the ground. Audiality is
(sort of :-) flying, though, and it's designed around the same ideas,
basically.
Would it really be possible
to accomplish all that with one API...?
Yes, I think so. Timestamped structured data is an incredibly
powerful and accurate tool. While being an alternative to audio rate
control data, it is also capable of acting much like the message
passing APIs used in many real time operating systems. The latter is
what makes this basic form of interface so powerful, and yet easy to
use.
Is it possible to design it
without a divine intervention...?
That remains to see. :-)
In the end I fear you may find
the host doing a whole lot more than "pretty much nothing". For
instance, the metadata needed to describe all this could easily get
out of hand. No metadata = no UI = useless API.
The "metadata" doesn't have to be all that much more complicated than
for LADSPA. We have Control Ports and Audio Ports as well. Our
control ports are slightly more complex, since we want to take
advantage of some of the possibilites that timestamped events add.
Our audio ports are just as simples as those in LADSPA, although we
want to be able to arrange them in more structured ways than a single
1D array.
It is going to be more complex than LADSPA, but I'm quite sure it
well be simpler, cleaner, easier to use and more powerful than VST,
at least in the areas we have the knowledge and resources to deal
with now.
For example, we'll probably not have an entire side of the API
dedicated to off-line processing. Not in 1.0. Maybe we'll never think
of that as an integral part of an API of this kind.
In short, if you can host LADSPA plugins, you'll know how to host XAP
plugins. There isn't all that much more you need to understand. The
event system is 90% inline macros, and the rest comes in a Host
Support Library. Just as plugin registering, loading, connection
managment and most other stuff.
I suspect that you'll be able to write a basic host in *less* time
than you can write a LADSPA host. You should get away rather easy
with basic plugins as well. There is a lot you *can* do, but very
little you *have to* do.
If I were me I would keep XAP a low-level block
processing API.
Well, that's what it *is*. We're not really stretching beyond what
VST does, and people seem to do pretty well with that API, despite
the lack of documentation and a host SDK, and despite it mixing MIDI
and all sorts of stuff into it's event system, and despite it having
a redundant but non-optional interface for parameters and presets.
(Note that this on VST is my personal opinion; not that of the XAP
"team".)
The
idea of audio ports on the one hand and sample-accurate control
events interleaved in a single queue on the other is ingenious.
There's a reason why everyone and his dog is using it. ;-)
Add
to that voice allocation and well defined metadata capabilities and
we have a backbone for such a softstudio as you outline above.
Yep, but you're forgetting a few details that part simple modular
synth APIs from APIs that allow you to what people have been doing on
Windoze and Mac for a good while. We intend to cover that, and
possibly some more, in a clean and easy-to-use API.
Then
design more APIs on top of it as needed.
Well, you can *extend* the API, or possibly implement some protocols
on top of it - but there's no big difference. Most useful things seem
to require host services and/or that all or many plugins understand
the "new stuff" anyway. So, it's the same thing as releasing a now,
incompatible API anyway.
Consider LADSPA:
Did it ever get timestamped events?
What would be the implications of adding them?
Then look at VST 1.0 vs 2.0:
VST 1.0 did not have events.
VST 2.0 added them - which resulted in massive redundancy,
and the fact that their event system is essentially crippled
in many ways.
It's not like we were introduced to timestamped events, or any other
of these concepts last week. Many of us have spent *years* thinking
about this, hacking, testing and learning what works and what doesn't.
This is not our first shot at an audio plugin API. We've all learned
a lot since I came here with Audiality/RT-Linux, and IMHO, we should
do our best to make XAP reflect our current knowledge and experience.
I think we can do better than a slightly enhanced LADSPA.
Don't let the event aspect
fool you into thinking it is meaningful to make XAP encompass
everything.
Summa theologica: dump musical time,
This won't work. Beat sync effects are really rather important in any
serious synth setup these days.
note pitch,
That, IMHO, only gives users another way to mix things up, without a
good reason.
Either way, it's just a hint, and nothing that anyone really has to
care about. It's only purpose is to allow nice hosts to suggest to
lost users that they might want to place note based effects *before*
the scale converter - in case they actually use one. (With 12tET,
there's no need, because NOTEPITCH == PITCH in that case.)
asynchronous I/O.
Yes, I think I agree that that should be left out, at least until we
really know what we're doing. Until it allows us to split
Linuxsampler into a streaming plugin and a sample player plugin,
there isn't much point in having this in the official API.
Use Hz for pitch.
Why Hz? Linear pitch is so much easier to work with, whether you
think in musical or pure pitch terms.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
17 Dec
17 Dec
11:16 a.m.
New subject: [linux-audio-dev] XAP: Control events
On Tue, 17 Dec 2002, David Olofson wrote:
This is not our first shot at an audio plugin API.
We've all learned
a lot since I came here with Audiality/RT-Linux, and IMHO, we should
do our best to make XAP reflect our current knowledge and experience.
I think we can do better than a slightly enhanced LADSPA.
Thanks for the explanations... there is still one thing I don't
understand. Do you intend to have both frequency (Hz) and pitch
(logarithmic Hz) controls? It seems redundant.
While you're discussing loftier matters I might as well try to do
some low-level grunt work on controls. This is an amalgamation of several
posts:
1) A control is a signal whose value against time is determined by control
events.
2) The control events divide the signal into sections. The signal is
identically zero before the first control event arrives.
3) Each section is a third-order polynomial. This gives continuity of
first derivatives where it is needed and is a compromise between
efficiency and flexibility. Note that some or all of the coefficients may
be zero.
4) What happens between sample values? This is up to each plugin, mainly
because the continuous signal is, in general, not bandlimited. However, to
most practical purposes, it can be made "smooth enough" . Most plugins
that oversample will use the macros below to get an interpolated version.
Some might do additional low-pass filtering before applying the control
signal internally.
5) The event type SET_CONTROL (name?) begins a new section of a control
signal. It has these attributes:
float start_value; /* value now */
float start_slope; /* slope now */
float duration; /* in samples */
float end_value; /* value at now + duration */
float end_slope; /* slope at now + duration */
All attributes must be real-valued. Duration must be greater than zero.
Slopes are given as "(instantaneous) change in value per sample".
As stated above, the attributes should be taken as constraints specifying
a third-order polynomial. Denoting the polynomial as f(x), the constraints
are:
f(now) = start_value
f'(now) = start_slope
f(now + duration) = end_value
f'(now + duration) = end_slope
Note: the section is specified completely in each event to make the
handling of controls as stateless as possible.
The section lasts until the next SET_CONTROL event. The timing of the next
event need not agree with the duration given in the current event - the
next event may arrive at any time.
6) The plugin may approximate the control internally as piece-wise
constant or piece-wise linear. In the first case the value of
the section is the average of start and end values. The following macros
extract these representations:
#define GET_CONSTANT_VALUE(event) (((event).start_value \
+ (event).end_value) * 0.5)
#define GET_LINEAR_VALUE(event) ((event).start_value)
#define GET_LINEAR_SLOPE(event) (((event).end_value \
- (event).start_value) / (event).duration)
#define GET_OVERSAMPLED_LINEAR_SLOPE(event, ratio) (((event).end_value \
- (event).start_value) / ((event).duration * (ratio))
Note: GET_LINEAR_VALUE returns the initial value.
The last macro provides for plugins that oversample. The second argument
is the oversampling ratio.
Note: it is assumed that plugins choose their approximation irregardless
of the actual polynomial, which can be constant or linear. I think this is
a good assumption. Exact linearity cannot be detected anyway due to the
inaccuracy of floating-point arithmetic.
7) The following struct and macros provide a "control iterator". They are
an efficient way of implementing the full third-order form.
/* TODO: should some/all of these be floats? */
struct control_variable {
double value;
double dvalue;
double ddvalue;
double dddvalue;
};
/* Initializes a control variable. */
#define INITIALIZE_CONTROL(var) \
(var).value = (var).dvalue = (var).ddvalue = (var).dddvalue = 0;
/* Initializes a control variable from a SET_CONTROL event. */
#define SET_CONTROL_FROM_EVENT(var, event) \
{ \
double tmp = 1 / (event).duration; \
(var).value = (event).start_value; \
(var).dvalue = (event).start_slope; \
(var).dddvalue = ((event).start_slope + (event).end_slope \
- 2 * ((event).end_value - (event).start_value) \
* tmp) * tmp * tmp; \
(var).ddvalue = (((event).end_value - (event).start_value) * tmp \
- (event).start_slope) * tmp \
- (var).dddvalue / tmp; \
(var).dvalue += (var).dddvalue + (var).ddvalue; \
(var).dddvalue *= 6; \
(var).ddvalue += (var).dddvalue + (var).ddvalue; \
}
/* Initializes an oversampled control variable from a SET_CONTROL event.
*/
#define SET_OVERSAMPLED_CONTROL_FROM_EVENT(var, event, ratio) \
{ \
double tmp = 1 / ((event).duration * (ratio)); \
(var).value = (event).start_value; \
(var).dvalue = (event).start_slope; \
(var).dddvalue = ((event).start_slope + (event).end_slope \
- 2 * ((event).end_value - (event).start_value) \
* tmp) * (tmp * tmp); \
(var).ddvalue = (((event).end_value - (event).start_value) * tmp \
- (event).start_slope) * tmp \
- (var).dddvalue / tmp; \
(var).dvalue += (var).dddvalue + (var).ddvalue; \
(var).dddvalue *= 6; \
(var).ddvalue += (var).dddvalue + (var).ddvalue; \
}
/* Increments a control variable. Should be called after each sample is
processed. */
#define INCREMENT_CONTROL(var) \
(var).value += (var).dvalue; \
(var).dvalue += (var).ddvalue; \
(var).ddvalue += (var).dddvalue;
I have checked the above for correctness.
8) The following macros generate control events.
/* TODO: set event type, too? */
#define SET_CONTROL_CONSTANT(event, value) \
(event).start_value = (event).end_value = (value); \
(event).start_slope = (event).end_slope = 0; \
(event).duration = 1;
/* TODO: set event type, too? */
#define SET_CONTROL_RAMP(event, start_value, slope) \
(event).start_value = (start_value); \
(event).start_slope = (event).end_slope = (slope); \
(event).duration = 1; \
(event).end_value = (event).start_value + (event).slope;
/* TODO: set event type, too? */
#define SET_CONTROL_POLYNOMIAL(event, start_value, start_slope, \
duration, end_value, end_slope) \
(event).start_value = (start_value); \
(event).start_slope = (start_slope); \
(event).duration = (duration); \
(event).end_value = (end_value); \
(event).end_slope = (end_slope);
Comments?
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
3:12 p.m.
New subject: [linux-audio-dev] XAP: Control events
On Tuesday 17 December 2002 16.11, Sami P Perttu wrote:
[...]
Thanks for the explanations... there is still one
thing I don't
understand. Do you intend to have both frequency (Hz) and pitch
(logarithmic Hz) controls? It seems redundant.
There will not be Hz at all, if I can help it; just linear PITCH, and
hopefully "linear" pitch hinted as "I really mean *linear*" - ie
NOTEPITCH. :-)
[...]
1) A control is a signal whose value against time is
determined by
control events.
Yes.
2) The control events divide the signal into sections.
The signal
is identically zero before the first control event arrives.
You should probably not assume that. (0 might be an illegal value.)
The metadata will contain default values for all controls, it seems
to make sense that the host should load the defaults as part of the
instantiation of a plugin. That would make it a matter of "do it once
and for all in the host SDK."
3) Each section is a third-order polynomial. This
gives continuity
of first derivatives where it is needed and is a compromise between
efficiency and flexibility. Note that some or all of the
coefficients may be zero.
Yes, this looks nice.
4) What happens between sample values? This is up to
each plugin,
mainly because the continuous signal is, in general, not
bandlimited. However, to most practical purposes, it can be made
"smooth enough" . Most plugins that oversample will use the macros
below to get an interpolated version. Some might do additional
low-pass filtering before applying the control signal internally.
Yes. The best we can do is polynomial splines - but that doesn't
automatically eliminate the problem. It's just a bit better than
without ramps, or with just linear, but it's not perfect. Some
plugins might still need to do some conditioning.
5) The event type SET_CONTROL (name?) begins a new
section of a
I call it XAP_A_SET, IIRC. (XAP "Action" SET.)
control signal. It has these attributes:
float start_value; /* value now */
float start_slope; /* slope now */
float duration; /* in samples */
float end_value; /* value at now + duration */
float end_slope; /* slope at now + duration */
Well, we'd have to switch to 64 byte events for that. However,
there's no good reason to double the start/end data, for reasons
explained below.
All attributes must be real-valued.
Yes, probably. Not sure about the duration, though, as you'll need it
in both integer and float if you must implement both ramping and
"ramp stops here" handling efficiently.
With the "ramp stops when I tell you" design (target + duration or
just slope), this is not an issue, since the control data generator
will tell you what to do in time.
Duration must be greater than
zero.
Not sure. Short ramps easilly round to 0 length. Where you you want
the special case; in senders or receivers? Well, probably in senders.
(See below.)
Slopes are given as "(instantaneous) change in
value per
sample".
Yes.
As stated above, the attributes should be taken as
constraints
specifying a third-order polynomial. Denoting the polynomial as
f(x), the constraints are:
f(now) = start_value
f'(now) = start_slope
f(now + duration) = end_value
f'(now + duration) = end_slope
Note: the section is specified completely in each event to make the
handling of controls as stateless as possible.
That's basically a good idea, but in my experience, it's pointless in
real life applications. These events are not asynchronous messages,
but rather a continous stream of structured data. If you can't keep
track of what you're telling other plugins, you can't generate useful
control data anyway.
Besides, all you need to do to have a complete section is send
XAP_A_SET followed by XAP_A_RAMP, with the same timestamp value.
(Assuming that XAP_A_SET comes with value + slope, and XAP_A_RAMP
comes with target value + target slope. A normal SET would always
have 0 for slope.)
The section lasts until the next SET_CONTROL event.
How? Either it ends when the ramp is finished, or it doesn't.
BTW, Audiality's internal voice control events don't even keep track
of that, so you *explicitly* have to tell a voice when to stop
ramping a control, by sending a "SET" event with the exact target
value. (*) This isn't nearly as hard to manage as it may sound, since
you *have* to keep track of what you're doing anyway, as a sender.
That said, giving ramp events a specific duration, after which the
ramping should stop *might* be nice. I'm still suspecting that this
is an illusion that won't help real code at all, though. (It didn't
in Audiality, at least, and I can't think of a real situation where
it would be useful.)
Want to have safe disconnections? Just do what you should do anyway:
Load the default value with a slope of 0. That kills any ramping in
progress, and reverts to the default value.
Click free? Just have the host take over the port for a while, before
really disconnecting it. (The plugin that owns the input will never
notice this.) Send a ramp event of suitable duration, that takes the
control back to the default value. Then disconnect.
Obiously, this *requires* the chained ramp approach I'm using in
Audiality, since you can't read back the value of a XAP control
input. You have to eavesdrop to do that, if you really want to.
(Quite possible to do, of course.)
(*) This is to avoid ramping rounding errors affecting DC levels
between ramps. I'm basically just doing it to avoid a special
"STOP_RAMP" event. It's not required to avoid drift, since
ramping is always chasing a target value, rather than building
on the current state. You can actually ramp to any value
without even knowing the current value of the control!
The timing of
the next event need not agree with the duration given in the
current event - the next event may arrive at any time.
What's the point? As a control data generator, you can't just jump in
and out as you like. Either you're connected, or you're not. When
you're connected, you do your job, and deliver a stream of structured
data that represents a continous signal.
[...]
Note: it is assumed that plugins choose their
approximation
irregardless of the actual polynomial, which can be constant or
linear. I think this is a good assumption. Exact linearity cannot
be detected anyway due to the inaccuracy of floating-point
arithmetic.
Yes, that's an important point. Plugins should be concistent rather
than "smart" in this case. It would be rather annoying if some
plugins toggled "randomly" between two different sets of artifacts...
*heh* (That's a great way to make it glaringly obvious that there
*are* artifactsl even if they're very subtle.)
7) The following struct and macros provide a
"control iterator".
They are an efficient way of implementing the full third-order
form.
/* TODO: should some/all of these be floats? */
Well, given that control ramping is not reliable enough to be used
for long sections and big changes, I don't think there's much point
in using doubles at all. Control ramping is just an *approximation*,
and since you can't know how well the receiver actually implements
it, you can't rely too much on the accuracy anyway.
In short, if we can deal with ramps over a few hundred samples at
most, that should be just fine. If you do longer ramps than that,
you're in for unpleasant surprises with many of the plugins that
interpolate internal coefficients. Besides, you can't really make a
longer ramp than one block anyway, without breaking this "future"
rule. Process one block at a time...
As to "hitting the target", well, I think we must analyze this more
carefully, to see how big a difference float vs double makes here,
and when. How far off can you land?
struct control_variable {
double value;
double dvalue;
double ddvalue;
double dddvalue;
};
/* Initializes a control variable. */
#define INITIALIZE_CONTROL(var) \
(var).value = (var).dvalue = (var).ddvalue = (var).dddvalue = 0;
Or just use XAP_A_SET, and wait for the host to give you your default
values that way. I think that's the best way, actually, since it
eliminates lots of boring and potentially incorrect initialization
code in plugins.
/* Initializes a control variable from a SET_CONTROL
event. */
[...]
/* Initializes an oversampled control variable from a
SET_CONTROL
event. */
[...]
Yes. Thanks for doing the maths! :-)
/* Increments a control variable. Should be called
after each
sample is processed. */
#define INCREMENT_CONTROL(var) \
(var).value += (var).dvalue; \
(var).dvalue += (var).ddvalue; \
(var).ddvalue += (var).dddvalue;
I have checked the above for correctness.
Great. :-)
8) The following macros generate control events.
/* TODO: set event type, too? */
Well, in Audiality, I just have inline functions that allocate, fill
in and send events of the common kinds. That's the cleanest way by
far, from the plugin code POV.
As to inline funcs; how likely is it that we'll find a compiler these
days that won't do it, or will generaty "slow" code?
(Not that I care all that much, really. gcc seems to do well enough,
and that's what most real OSes come with anyway. ;-)
[...fill-in-event macros...]
Comments?
Nice work!
We'll have to settle on the conventions for control generators,
though. IMNSHO, *they* should be the "smart side" of the connection,
while the receiver just does what it's told. I'm basically trying to
achieve this:
1. Plugins that only receive control data become very
simple and easy to get right.
2. Control generators contain all logic, and as a result,
have more alternatives when it comes to optimizations.
3. Keeping the "active" logic in one place will most
probably make debugging easier, since you can
basically run the output into a "control tester"
plugin to find out if the output data is correct.
If you have a problem that's caused by *two* plugins
being broken, you're in for some great fun...
4. Keeping the most complex part right behind a standard
output interface is less likely to result in
weirdness that shows up in the middle of your plugin,
where you can't easily view the data in useful way.
5. Optimized control ramp implementations will probably
be hairy enough without conditionals and stuff
required by the API.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
18 Dec
18 Dec
7:44 a.m.
New subject: [linux-audio-dev] XAP: Control events
On Tue, 17 Dec 2002, David Olofson wrote:
What about things like filter center frequency, isn't it "unnatural" to
give these as logarithmic frequencies? Not to mention that most (?)
plugins need to convert to Hz/samplerate anyway, involving an expensive
exp(). Also, you can't specify negative frequencies in logarithmic Hz, and
0 Hz is the awkward minus infinity. But maybe they are not needed.
Well, I think it is a good semantics for duration: nobody knows what
will happen in the next block, and the plugin should not need to check if
the section has ended.
Thanks for the
explanations... there is still one thing I don't
understand. Do you intend to have both frequency (Hz) and pitch
(logarithmic Hz) controls? It seems redundant.
There will not be Hz at all, if I can help it; just linear PITCH, and
hopefully "linear" pitch hinted as "I really mean *linear*" - ie
NOTEPITCH. :-) Well, we'd have to switch to 64 byte events for
that. However,
there's no good reason to double the start/end data, for reasons
explained below.
Okay, I realize it makes sense, especially the bit about switching
connections.
That said, giving ramp events a specific duration,
after which the
ramping should stop *might* be nice. I'm still suspecting that this
is an illusion that won't help real code at all, though. (It didn't
in Audiality, at least, and I can't think of a real situation where
it would be useful.)
I agree. Plus it adds a conditional per sample, which is not nice.
Click free? Just have the host take over the port for
a while, before
really disconnecting it. (The plugin that owns the input will never
notice this.) Send a ramp event of suitable duration, that takes the
control back to the default value. Then disconnect.
This is cool. With a target slope of 0 the control settles nicely to the
new value.
The timing of
the next event need not agree with the duration given in the
current event - the next event may arrive at any time.
What's the point? As a control data generator, you can't just jump in
and out as you like. Either you're connected, or you're not. When
you're connected, you do your job, and deliver a stream of structured
data that represents a continous signal. [...]
True. Plugins would need to generate different code for the different
approximations, anyway. Not to mention that the linear approximation is
very different from the full form... if the sign of the target slope
differs from the sign of the difference between target and current value,
the polynomial will overshoot somewhere in the middle. Etc.
Note: it is assumed that plugins choose their
approximation
irregardless of the actual polynomial, which can be constant or
linear. I think this is a good assumption. Exact linearity cannot
be detected anyway due to the inaccuracy of floating-point
arithmetic.
Yes, that's an important point. Plugins should be concistent rather
than "smart" in this case. It would be rather annoying if some
plugins toggled "randomly" between two different sets of artifacts...
*heh* (That's a great way to make it glaringly obvious that there
*are* artifactsl even if they're very subtle.) As to "hitting the target", well, I think we
must analyze this more
carefully, to see how big a difference float vs double makes here,
and when. How far off can you land?
I'll investigate.
Or just use XAP_A_SET, and wait for the host to give
you your default
values that way. I think that's the best way, actually, since it
eliminates lots of boring and potentially incorrect initialization
code in plugins.
Okay.
Well, in Audiality, I just have inline functions that
allocate, fill
in and send events of the common kinds. That's the cleanest way by
far, from the plugin code POV.
Okay.
We'll have to settle on the conventions for
control generators,
though. IMNSHO, *they* should be the "smart side" of the connection,
while the receiver just does what it's told. I'm basically trying to
achieve this: [...]
Agreed. We are on the right track.
There was an error in SET_OVERSAMPLED_..., I'll fix it. Then I have to
check how to recover slope from a control variable. Then float vs. double.
I'll send a revised version shortly.
--
Sami Perttu "Flower chase the sunshine"
Sami.Perttu(a)hiit.fi http://www.cs.helsinki.fi/u/perttu
9:07 a.m.
New subject: [linux-audio-dev] XAP: Control events
On Wednesday 18 December 2002 12.38, Sami P Perttu wrote:
Yeah. (If we have splines, that is. Otherwise, just RAMP to the
default value, and then stop ramping with a SET before
disconnecting.)
Well, I think it is a good semantics for duration: nobody knows
what will happen in the next block, and the plugin should not need
to check if the section has ended.
Right. That's why you basically don't let ramps run across block
boundaries.
That said, it's perfectly legal to aim for longer distances, since
you can just send a new ramp or set event at any time. The current
state when the new event is handled is the starting point of the new
ramp. It doesn't matter where the original ramp was *supposed* to
end, since that's basically just an aim point. (If you sent slope
instead of target + duration, you would have rounding errors build up
over time.)
[...]
True. Plugins would need to generate different code for the
different approximations, anyway. Not to mention that the linear
approximation is very different from the full form... if the sign
of the target slope differs from the sign of the difference between
target and current value, the polynomial will overshoot somewhere
in the middle. Etc.
Yeah - and there's another reason not to do splines: overshoot.
Doesn't mix with controls with hard ranges.
[...]
Ok. Although it seems like we're not having splines after all, due to
various mathematical and implementational issues mentioned earlier.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
On Tue, 17 Dec 2002, David Olofson wrote:
What about things like filter center frequency, isn't it
"unnatural" to give these as logarithmic frequencies?
Why? At least, I'd find Hz *really* inconvenient when programming
filters on synths. I want them in the same unit as pitch, to make it
easy to have filters track pitch and that kind of stuff.
Thanks
for the explanations... there is still one thing I don't
understand. Do you intend to have both frequency (Hz) and pitch
(logarithmic Hz) controls? It seems redundant.
There will not be Hz at all, if I can help it; just linear PITCH,
and hopefully "linear" pitch hinted as "I really mean *linear*" -
ie NOTEPITCH. :-) Not to
mention that most (?) plugins need to convert to Hz/samplerate
anyway, involving an expensive exp().
Someone, somewhere has to do that anyway. And calculations differ
depending on what you're doing, especially when you're processing
events rather than just generating audio from them.
Also, you can't specify
negative frequencies in logarithmic Hz, and 0 Hz is the awkward
minus infinity. But maybe they are not needed.
Well, I can't think of any serious use for exactly 0 Hz, or even
anything very close to it.
[...]
Click free?
Just have the host take over the port for a while,
before really disconnecting it. (The plugin that owns the input
will never notice this.) Send a ramp event of suitable duration,
that takes the control back to the default value. Then
disconnect.
This is cool. With a target slope of 0 the control settles nicely
to the new value. The timing of
the next event need not agree with the duration given in the
current event - the next event may arrive at any time.
What's the point? As a control data generator, you can't just
jump in and out as you like. Either you're connected, or you're
not. When you're connected, you do your job, and deliver a stream
of structured data that represents a continous signal. Note: it is assumed that plugins choose their
approximation
irregardless of the actual polynomial, which can be constant or
linear. I think this is a good assumption. Exact linearity
cannot be detected anyway due to the inaccuracy of
floating-point arithmetic.
Yes, that's an important point. Plugins should be concistent
rather than "smart" in this case. It would be rather annoying if
some plugins toggled "randomly" between two different sets of
artifacts... *heh* (That's a great way to make it glaringly
obvious that there *are* artifactsl even if they're very subtle.) We'll have
to settle on the conventions for control generators,
though. IMNSHO, *they* should be the "smart side" of the
connection, while the receiver just does what it's told. I'm
basically trying to achieve this: [...]
Agreed. We are on the right track.
There was an error in SET_OVERSAMPLED_..., I'll fix it. Then I have
to check how to recover slope from a control variable. Then float
vs. double. I'll send a revised version shortly. 
2:37 p.m.
New subject: [linux-audio-dev] XAP: Control events
Yeah. (If we have splines, that is. Otherwise, just
RAMP to the
default value, and then stop ramping with a SET before
disconnecting.)
Why do you need to stop ramping? Set the value to 0, and ramp for 1 block
or 1/2 block or whatever
Right. That's why you basically don't let
ramps run across block
boundaries.
That said, it's perfectly legal to aim for longer distances, since
given that we know nothing about the future, I say no. We've set a rule
that "things happen now and in this block only". Let's stick to it.
6:07 p.m.
New subject: [linux-audio-dev] XAP: Control events
On Wednesday 18 December 2002 19.31, Tim Hockin wrote:
That only works if you're about to kill the plugin as well.
Otherwise, it will basically ignore that the port was disconnected,
and keep ramping forever.
The alternative would be to require that disconnect() resets the
control automatically, but that falls in the same domain as loading
control default values, which IMO is something that the host should
do. (Explained earlier.)
Sounds logical, but one could say that the ramping events break the
rule anyway, since the ramping isn't stopped automatically at the aim
point. The aim point is just a different way of expressing slope, and
it's used only to keep rounding errors from building up. What a ramp
event *really* says is this:
"Start ramping from the current value so that
<target> is hit in <length> sample frames."
It doesn't say that the aim point is where the ramp stops. It's more
similar to just changing the ramp value directly.
Either way, it's just a performance hack, and I'm not sure it's of
much use. I've already said myself that you shouldn't do long ramps,
because they may not be as linear as you think, due to plugin
optimizations.
So, yes better stick to the rule.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
Yeah. (If we
have splines, that is. Otherwise, just RAMP to the
default value, and then stop ramping with a SET before
disconnecting.)
Why do you need to stop ramping? Set the value to 0, and ramp for
1 block or 1/2 block or whatever Right.
That's why you basically don't let ramps run across block
boundaries.
That said, it's perfectly legal to aim for longer distances,
since
given that we know nothing about the future, I say no. We've set a
rule that "things happen now and in this block only". Let's stick
to it.
11:28 p.m.
New subject: [linux-audio-dev] XAP: Control events
Why do you
need to stop ramping? Set the value to 0, and ramp for
1 block or 1/2 block or whatever
That only works if you're about to kill the plugin as well.
Otherwise, it will basically ignore that the port was disconnected,
and keep ramping forever. given that we
know nothing about the future, I say no. We've set a
rule that "things happen now and in this block only". Let's stick
to it.
Sounds logical, but one could say that the ramping events break the
rule anyway, since the ramping isn't stopped automatically at the aim
19 Dec
19 Dec
9:38 a.m.
New subject: [linux-audio-dev] XAP: Control events
On Thursday 19 December 2002 04.23, Tim Hockin wrote:
nonono. I always envisioned a ramp as being finite. That's the
whole point of the duration field. "Ramp to this value over N
frames" where you know (timestamp + duration) <= (buffer_start +
nsamples)
The duration + target level interface is the "only" way to do this
right - that's the primary reason to use it. The alternative would be
to use only slope, but that would mean you have to know the current
value, and - what is worse - you become dependent on the accuracy of
that value, as any rounding errors will build up through chains of
ramps.
A second reason not to view ramps as finite is that it requires
receivers to do one of two things:
1. Test for the target value for each sample. (Inner
loop conditional!)
2. Prequeue an event to itself to stop the ramping at
the right point. (Means you have to scan the event
queue for the right place to insert this event.)
Meanwhile, the sender must keep track of what it's doing anyway, so
it really doesn't matter whether ramps stop automatically or not -
there will always be a new event at the end of the ramp anyway.
Consider writing an envelope generator that outputs linear ramp
events. When would you ever have a use for stopping ramping without
an explicit event?
See above - they don't have to.
No, but I see no motivation to force implementational issues upon
plugins to implement a feature that will practically never be of any
use.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
Why do you need to stop ramping? Set the value to 0,
and ramp
for 1 block or 1/2 block or whatever
That only works if you're about to kill the plugin as well.
Otherwise, it will basically ignore that the port was
disconnected, and keep ramping forever. given that we know nothing about the future, I say no.
We've
set a rule that "things happen now and in this block only".
Let's stick to it.
Sounds logical, but one could say that the ramping events break
the rule anyway, since the ramping isn't stopped automatically at
the aim
5:57 p.m.
New subject: [linux-audio-dev] XAP: Control events
A second reason not to view ramps as finite is that it
requires
receivers to do one of two things:
1. Test for the target value for each sample. (Inner
loop conditional!)
2. Prequeue an event to itself to stop the ramping at
the right point. (Means you have to scan the event
queue for the right place to insert this event.)
Meanwhile, the sender must keep track of what it's doing anyway, so
it really doesn't matter whether ramps stop automatically or not -
there will always be a new event at the end of the ramp anyway.
OK, I'll buy that. Send a SET event to stop a ramp. It just means that a
ramp is (almost always) two events. Not too big a problem, I think.
6:07 p.m.
New subject: [linux-audio-dev] XAP: Control events
On Thursday 19 December 2002 22.51, Tim Hockin wrote:
No, no major bandwidth abuse or anything. And in fact, it's not even
as bad as two events per ramp. You don't have to send any SET
"stoppers" if you chain ramps - which is what you'll be doing most of
the time anyway. Especially if you stick strictly to the rule of not
letting ramps run across block boundaries. I don't in Audiality, but
it doesn't really make any difference there: I *know* that the send
level ramping is truly linear. ;-)
That's an example of the "aim ahead and change your ming" approach,
BTW. The EG runs each stage of the envelope as a single linear ramp,
but may still switch to the release stage at any point.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
A second
reason not to view ramps as finite is that it requires
receivers to do one of two things:
1. Test for the target value for each sample. (Inner
loop conditional!)
2. Prequeue an event to itself to stop the ramping at
the right point. (Means you have to scan the event
queue for the right place to insert this event.)
Meanwhile, the sender must keep track of what it's doing anyway,
so it really doesn't matter whether ramps stop automatically or
not - there will always be a new event at the end of the ramp
anyway.
OK, I'll buy that. Send a SET event to stop a ramp. It just means
that a ramp is (almost always) two events. Not too big a problem,
I think. 
18 Dec
18 Dec
10:20 a.m.
New subject: [linux-audio-dev] XAP: Control events
On Wed, Dec 18, 2002 at 01:38:53 +0200, Sami P Perttu wrote:
What about things like filter center frequency,
isn't it "unnatural" to
give these as logarithmic frequencies? Not to mention that most (?)
No, most synth processes use filters with pitch control, eg. modulation
with the envelope and pitch tracking.
A few modular synth filters have FM inputs as well, but its texception,
rather than the rule.
plugins need to convert to Hz/samplerate anyway,
involving an expensive
exp(). Also, you can't specify negative frequencies in logarithmic Hz, and
0 Hz is the awkward minus infinity. But maybe they are not needed.
Yes, but if you do this inside the instrument then it only has to be done
once, if you send frequency data around then you have to keep converting
it back and forwards.
In any case there are well known optimisations for going from float pitch
to float Hz.
- Steve
8000
days inactive
8008
days old
linux-audio-dev@lists.linuxaudio.org
26 comments
5 participants
participants (5)
-
David Olofson -
Sami P Perttu -
Steve Harris -
Tim Goetze -
Tim Hockin