20 Mar
2003
20 Mar
'03
8:48 p.m.
I fear we've all been very busy. I know I have. I have not had time to
devote to XAP, though I have certainly not lost interest.
So today I start looking at my evolving spec doc again. The biggest chunk
of undecidedness lies in the area of Channels. We never really resolved the
Bays/Channels etc debate. We just put it off. Here it is.
I'd like to propose the following - it is simple and flexible. Not too
confusing.
Plugins define 'Modules'. Every plugin has a master Module, and 0-n extra
Modules. The host can instantiate Modules as needed. A Module is analogous
to a MIDI Channel in a synth (maybe Module is the wrong word). It is
analogous to a slot in a mixer.
struct XAP_descriptor {
...
/* master module descriptor */
XAP_module master;
/* other channel descriptors */
int n_modules;
XAP_module *modules;
/*
* Instantiate a module, return a plugin-unique
* id/index for it - it's 'Channel'.
*/
int (*add_module)(void *plugin, int module);
/*
* Remove a module.
*/
int (*del_module)(void *plugin, int index);
};
struct XAP_module {
char *label;
char *name;
uint32_t flags;
/* how many instances of this module are allowed? */
int count_max;
/* per-module controls and I/O */
int n_controls;
XAP_control **controls;
int n_ports;
XAP_port **ports;
};
21 Mar
21 Mar
6:24 p.m.
On Thu, Mar 20, 2003 at 04:35:51PM -0800, Tim Hockin wrote:
struct XAP_module {
char *label;
char *name;
uint32_t flags;
/* how many instances of this module are allowed? */
int count_max;
/* per-module controls and I/O */
int n_controls;
XAP_control **controls;
int n_ports;
XAP_port **ports;
};
why support the difference between ports and controls ?
This is done in the C galan and tony somehow managed to remove this
difference in the C++ galan.
I have not read much of this code but tony said he managed to remove
this.
--
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language
7:02 p.m.
why support the difference between ports and controls
?
This is done in the C galan and tony somehow managed to remove this
difference in the C++ galan.
Because throughout all the XAP discussions we have made it clear that
controls are event-driven and ports are audio-rate data (specifically audio
data). Controls are more flexible in their data typing.
We have argued many hours about th ebenefits/drawbacks of using normalized
float data for everything and the decision we came to was that it just isn't
the best model. I point you at th earchives for the discussions :)
I have not read much of this code but tony said he
managed to remove
this.
LADSPA is much the same way - connect anything to anything. But several
people in the XAP discussion feel that normalized data (0 to 1.0 or
whatever) is bad. I am still of the position that I could be convinced to
support two basic control types: numerical (normalized) and other (strings,
data block, etc). This, however, is still not the same as audio-rate
controls, which is what you get when you plug an oscillator into a knob.
The simplest concept is that they are different things. Audio and Control
data. Reconsidering this notion would takes us WAY back to early XAP
discussions. Maybe that is OK - anyone want to make a case for a new
fundamental model?
Tim
7:29 p.m.
On Fri, Mar 21, 2003 at 02:49:53PM -0800, Tim Hockin wrote:
i know that. In galan it is the same.
But why should this difference be so clearly stated in the API ?
If there was only one list of pads which would either be events or
audiorate push, or pull or whatever type it would be cleaner.
If in the face of gmpi it still makes sense to specify XAP we should
make it supersede gmpi. But as acceptance is the biggest API feature i
doubt this makes sense.
no... galan is almost an XAP implementation. read the XAP early
scribbles thread i respawned.
--
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language
why support
the difference between ports and controls ?
This is done in the C galan and tony somehow managed to remove this
difference in the C++ galan.
Because throughout all the XAP discussions we have made it clear that
controls are event-driven and ports are audio-rate data (specifically audio
data). Controls are more flexible in their data typing.
We have argued many hours about th ebenefits/drawbacks of using normalized
float data for everything and the decision we came to was that it just isn't
the best model. I point you at th earchives for the discussions :) I have not read much of this code but tony said
he managed to remove
this.
LADSPA is much the same way - connect anything to anything. But several
people in the XAP discussion feel that normalized data (0 to 1.0 or
whatever) is bad. I am still of the position that I could be convinced to
support two basic control types: numerical (normalized) and other (strings,
data block, etc). This, however, is still not the same as audio-rate
controls, which is what you get when you plug an oscillator into a knob.
The simplest concept is that they are different things. Audio and Control
data. Reconsidering this notion would takes us WAY back to early XAP
discussions. Maybe that is OK - anyone want to make a case for a new
fundamental model?
7:52 p.m.
i know that. In galan it is the same.
But why should this difference be so clearly stated in the API ?
Can you show me a nice API that makes the two feel similar?
Controls: multiple datatypes, receive events in time-order queues
Ports: are assigned buffers (possibly datatyped) of arrays of data
We could, I suppose, send an event for each block which contains the buffer
pointer for each port, thereby making a Port a type of control. Is there a
benefit to that? It seems to just hide what a Port is behind a control.
If in the face of gmpi it still makes sense to specify
XAP we should
make it supersede gmpi. But as acceptance is the biggest API feature i
doubt this makes sense.
Even if GMPI is a smash success, having XAP to the alpha or beta stage will
lend us a lot of credibility when we say to GMPI "This is how XAP does it,
here is the code, and here is the analysis". And GMPI won't be done for at
LEASt a year.
no... galan is almost an XAP implementation. read the
XAP early
scribbles thread i respawned.
Nothing can be a XAP implementation yet, because we have fundamental model
issues we have not solved. I know David uses Audiality to model and prove
some of the XAP ideas, but it is hardly a XAP implementation, yet :)
Tim
22 Mar
22 Mar
3:07 p.m.
<quote who="Tim Hockin">
I've been partially involved with a project called Open Sound World for
some time (see http://osw.sourceforge.net/). Yes, this is Yet Another
Modular Synth. This is a project was part of the Phd thesis for a student
working at UC Berkeley CNMAT. There is naturally a somewhat academic
mindset to the design used in this project, so bear with me...
In OSW, there is no distiction between 'control' data and 'audio' data.
There is a very heavy emphasis on C++ generic programming and a rigorous
type model. Blocks of audio data are classes of type Samples (actually
they are just typedef vectors of floats), control data is of class
'Float', 'Int', 'Boolean', 'String', 'List', etc.
The advantage of not specifying a difference is there is a vastly
increased power for runtime dynamicism.
There is a type called 'Any' which can be converted into any other type.
Some objects have inlets and outlets specified as type 'Any', and the type
is changed upon connection with another object of specific type. This is
useful for making a 'general purpose' object. For example, named
Send/Recv pairs, List pack/unpacking, and control flow (FanOut, FanIn,
Switch/Gate). So, there is nothing preventing us from packing up a list
of audio blocks and doing whatever we want from there. In addition, there
are Open Sound Control objects which accept and transmit lists in the
patch; those lists might contain audio data, which means we can easily
send audio data over the network.
Additionally, it is trivial to implement alternative stream types
including audio streams running at different sample rates, different bit
types, etc.
Another point is that it -might- be possible to effectively utilize
SMP/parallelizm under this increased flexibility. That is a current
research topic wrt. this architecture.
Systems such as the PD family and Jack have fixed audio types; this
enables you to approach audio processing a different way; i.e. the audio
graph ordering is precomputed by the engine. You sacrifice runtime
dynamic routing in this case... but basically I think the idea is that you
don't really want it anyways. :)
So, there are downsides to having a fully dynamic routed graph:
1) you don't know how long the paths are in the audio graph, its
impossible to compute 'total latency' ahead of time.
2) you can't seperate the control and audio calculations; i.e. you cannot
decide to give one priority over the other. There are arguements for and
against this point. Any max user can sing a song about overdrive and
various insane scheduling hacks necessary to get control data to flow in a
timely manner. If control data is subject to latency jitter, then what is
the point of having it at all?
3) performance; generalization *always* costs you something.
specifically, I think modular synths suffer from having to do a of mixing
which could be avoided easily if you were coding straight C/C++. Some of
these overheads are probably somewhat easier to mitigate using the
traditional audio-graph model (e.g. jack is able to mitigate cost in many
ways, by providing zero-copy buffers when possible, etc).
--aws
mailto:andy*a2hd,com
http://www.a2hd.com/
i know that.
In galan it is the same.
But why should this difference be so clearly stated in the API ?
Can you show me a nice API that makes the two feel similar?
Controls: multiple datatypes, receive events in time-order queues
Ports: are assigned buffers (possibly datatyped) of arrays of data 
3:41 a.m.
Tim Hockin wrote:
LADSPA is much the same way - connect anything to
anything. But several
people in the XAP discussion feel that normalized data (0 to 1.0 or
whatever) is bad. I am still of the position that I could be convinced to
support two basic control types: numerical (normalized) and other (strings,
data block, etc). This, however, is still not the same as audio-rate
controls, which is what you get when you plug an oscillator into a knob.
The simplest concept is that they are different things. Audio and Control
data. Reconsidering this notion would takes us WAY back to early XAP
discussions. Maybe that is OK - anyone want to make a case for a new
fundamental model?
Ok. Mostly for fun, though...
+++A Case for a New Fundamental Model+++
The current model isn't particularly fundamental. It compounds aspects
of the problem which are actually quite separate:
1. What kind of data is entering/leaving a port?
2. When and how often is it entering/leaving?
3. What do various control data types mean?
4. How is the meaning encoded?
5. How/when is the encoding compressed?
6. What can be sensibly connected to what?
It proposes one fixed set of answers for "audio" data and another,
different fixed set of answers for "control" data.
Is this a clean, natural division? I don't think so. Why does the
model treat the output of an envelope generator as being (in many
respects) more like a string than it is like an oscillator output?
Why isn't it clear what an LFO's output type should be? Why does
something as simple as a ramp spawn so much complexity?
Even where the audio/event distinction looks clear cut, its not. The
user toggles a switch. Thats an event, surely? Wellll... maybe it is,
but then, what does the plugin do with the event? If the switch is
going anywhere near the audio path then it probably makes itself a
little ramp or curve and uses that instead. Why? "To avoid clicks".
The ramp/curve is a crude low-pass filter! The plugin is doing audio-rate,
audio resolution signal processing on something the model tells us is
fundamentally different from and incompatible with an audio signal.
Something has to make this ramp/curve(/whatever), because at its
source (eg UI widget or incoming midi message) the toggling of a
switch *is* an event. But the rise and fall characteristics should
belong to the owner of the source, not to the destination. An app
that displays a button on its GUI should be able to determine the
characteristics of that button. Is it hard edged? Soft edged?
Asymmetrical? Different circumstances call for different behaviour.
Some undeveloped ideas towards an alternative model:
~ Rather than making the distinction between "audio" and "control"
data, make it between "signal-path" and "non-signal-path" data.
~ Signal-path data is at full audio rate/resolution and doesn't
contain unpleasant things such as instantaneous edges. This doesn't
necessarily mean that it's always represented by a buffer full of
floats in memory (though that would be one way to implement it) but
whatever turns up at the input must *encode* a buffer's worth of
data, sample-synchronised with all the other signal-path data in
the system.
~ Non-signal path data is event-like. Something happens at a
particular instant. The encoding is some reasonable representation
of what happenned. The instant is a particular sample period, ie
non-signal-path data is still sample-synchronised with signal-path
data, it just doesn't occur on every sample.
~ Anything which can start life as an event but end up interacting
with the signal path has two representations: The event representation
(a bit like MIDI) and the audio-like signal-path representation (a bit
like CV). Crucially, plugins don't directly combine signal-path and
non-signal-path inputs. A plugin which switches audio, for example,
has a signal-path switch input not an event switch input. A plugin
which switches events, OTOH, has an event switch input.
~ Finally, define some default conversion plugins which can "cast"
between various events and signals, allowing most things to be
connected to most others.
Simon Jenkins
(Bristol, UK)
4:47 a.m.
On Sat, Mar 22, 2003 at 08:30:22 +0000, Simon Jenkins wrote:
Something has to make this ramp/curve(/whatever),
because at its
source (eg UI widget or incoming midi message) the toggling of a
switch *is* an event. But the rise and fall characteristics should
belong to the owner of the source, not to the destination. An app
This is the fundamental problem here, the characteristics should belong to
the receiver.
FWIW I see little point pressing ahead with XAP, even if we were to come
up with a novel representation system, I doubt the GMPI people would
regard it as having enough pracical experience in the time between a XAP
implementation and GMPI design stage, and rightly so. once GMPI exists
there is little or no justification for anyone supporting XAP, even if it
technologically superiour. Please bear this in mind.
- Steve
28 Mar
28 Mar
5:14 a.m.
On Sat, Mar 22, 2003 at 08:38:39AM +0000, Steve Harris wrote:
On Sat, Mar 22, 2003 at 08:30:22 +0000, Simon Jenkins
wrote:
we have one significant advantage over gmpi...
they (the commercial developers) can not distribute their source.
if we find a way to compile an XAP plugins into 1 frame processing code,
and galan-plugin, and ams plugin, and LADSPA etc...
something similar to your xml-ladspa but for all formats and machines...
there should be methods to get a gtk, qt or fltk widget from a plugin.
and this nice totally abstract port concept.
just some thoughts....
--
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language
Something has to make this ramp/curve(/whatever),
because at its
source (eg UI widget or incoming midi message) the toggling of a
switch *is* an event. But the rise and fall characteristics should
belong to the owner of the source, not to the destination. An app
This is the fundamental problem here, the characteristics should belong to
the receiver.
FWIW I see little point pressing ahead with XAP, even if we were to come
up with a novel representation system, I doubt the GMPI people would
regard it as having enough pracical experience in the time between a XAP
implementation and GMPI design stage, and rightly so. once GMPI exists
there is little or no justification for anyone supporting XAP, even if it
technologically superiour. Please bear this in mind.
24 Mar
24 Mar
7:22 p.m.
Interesting stuff worth talking about.
Is this a clean, natural division? I don't think
so. Why does the
model treat the output of an envelope generator as being (in many
respects) more like a string than it is like an oscillator output?
Why isn't it clear what an LFO's output type should be? Why does
something as simple as a ramp spawn so much complexity?
Some good questions. In fact, maybe an LFO _should_ be audio data. Or at
least Audio-rate. But one can argue that an LFO is LF enough that you can
represent it reasonably well with a series of short linear ramps, and have
less overhead.
Maybe XAP should consider some method of identifying Audio-rate control
ports. If your control expects audio-rate input, you should be able to ask
for it? Why not use actual audio ports for that, then, and just hint that
they are control signals? An envelope generator or an LFO should be audio
outputs that go into audio inputs. Maybe.
Even where the audio/event distinction looks clear
cut, its not. The
user toggles a switch. Thats an event, surely? Wellll... maybe it is,
but then, what does the plugin do with the event? If the switch is
switch *is* an event. But the rise and fall
characteristics should
belong to the owner of the source, not to the destination. An app
Disagree, here. The flip of a switch is a binary event. The receiver needs
to handle it. Non-binary things like knob turns will already either be
interpolated by the sequencer, by a custom UI, or by the plugin. Or all
three.
28 Mar
28 Mar
5:14 a.m.
On Mon, Mar 24, 2003 at 03:08:12PM -0800, Tim Hockin wrote:
yeah you are right.
the switch can not know what type its output will be converted too, so you
can not provide all typecast params....
i think we need parameterized typecasts.
a cast from Event/Binary to Signal/Norm needs a parameter.
in galan the signal to event converter needs to be fed by a clock also.
i am not sure how to model this correctly but an implicit typecast must
be configured somehow.
--
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language
Interesting stuff worth talking about.
yes... but it still represents an audiorate signal, whether it is a
stream of events or not.
i like the idea of having a stream, and interrupting code.
audio is the stream and then there is timestamped control data
interrupting the stream. but the user changing the graph is interrupting
the whole graph.
hmmm... dont know how to explain this better. hope you get it.
Is this a clean, natural division? I don't
think so. Why does the
model treat the output of an envelope generator as being (in many
respects) more like a string than it is like an oscillator output?
Why isn't it clear what an LFO's output type should be? Why does
something as simple as a ramp spawn so much complexity?
Some good questions. In fact, maybe an LFO _should_ be audio data. Or at
least Audio-rate. But one can argue that an LFO is LF enough that you can
represent it reasonably well with a series of short linear ramps, and have
less overhead.
Maybe XAP should consider some method of identifying Audio-rate control
ports. If your control expects audio-rate input, you should be able to ask
for it? Why not use actual audio ports for that, then, and just hint that
they are control signals? An envelope generator or an LFO should be audio
outputs that go into audio inputs. Maybe.
I would rather try to build a class hierarchy for ports...
in galan a port type is only an interface.
only push and pull semantics should be handled nicely but i think simon
knows the answer.
Even where the audio/event distinction looks
clear cut, its not. The
user toggles a switch. Thats an event, surely? Wellll... maybe it is,
but then, what does the plugin do with the event? If the switch is
switch *is* an event. But the rise and fall
characteristics should
belong to the owner of the source, not to the destination. An app
Disagree, here. The flip of a switch is a binary event. The receiver needs
to handle it. Non-binary things like knob turns will already either be
interpolated by the sequencer, by a custom UI, or by the plugin. Or all
three.
7951
days inactive
7959
days old
linux-audio-dev@lists.linuxaudio.org
10 comments
5 participants
participants (5)
-
Andy Schmeder -
Simon Jenkins -
Steve Harris -
Tim Hockin -
torbenh＠gmx.de