17 Dec
2002
17 Dec
'02
7:40 p.m.
We concluded a while ago that string events are really rather handy
for referencing external files. (For that to be truly useful, hosts
need to be able to understand what these strings are; ie directory
names, file names, temporary storage, output, input. This, so they
can keep track of what belongs in a project.)
Anyway, what I'm seeing here is that strings and raw data blocks
overlap quite significantly on the protocol level. Both are just
control ports that take or send "strings" - either NULL terminated,
or "pascal style". The latter *could* be replace former... Just hint
them differently. Same transfer protocol.
Problem with NULL terminated strings is that they cannot take binary
data (without encoding), and that you have to scan them for the
terminator to find out their length.
OTOH, sending a C string as raw data means you have to do this, even
if the receiver will just ignore the "length" field, and treat it as
the C string it actually is...
I'm leaning towards the "strings in raw data blocks" approach,
despite the little strlen() inconveniency for senders. Mostly because
it's one control data type less to care about, without loss of real
functionality.
Ideas?
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
17 Dec
17 Dec
7:55 p.m.
On Wednesday 18 December 2002 00.35, David Olofson wrote:
[...]
I'm leaning towards the "strings in raw data
blocks" approach,
despite the little strlen() inconveniency for senders. Mostly
because it's one control data type less to care about, without loss
of real functionality.
On second thoughts, there might be something more to this: Who *owns*
the memory used for data to and from string/raw data Controls? How is
it allocated in the first place?
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
18 Dec
18 Dec
5:51 a.m.
On Wed, Dec 18, 2002 at 12:50:00 +0100, David Olofson wrote:
That is a good point.
I think the host needs to own it because only the host knows when it can
be free'd again.
That does require the host to provide malloc functions though, but IIRC
they are in there allready.
Data blocks that will be need in an RT context (eg. bulk parameter pasing)
can be allocated ahead of time.
If an instrument knows that its not going to need to block again it cabn
tell the host to free it, if its going to be sent out then the host can
track it and free it if it consumes it itsself.
This is all quite a lot of work though. The alternative would be some sort
of garbage collection (even just ref. counting would do) I guess.
- Steve
I'm
leaning towards the "strings in raw data blocks" approach,
despite the little strlen() inconveniency for senders. Mostly
because it's one control data type less to care about, without loss
of real functionality.
On second thoughts, there might be something more to this: Who *owns*
the memory used for data to and from string/raw data Controls? How is
it allocated in the first place?
7:50 a.m.
On Wednesday 18 December 2002 10.45, Steve Harris wrote:
On Wed, Dec 18, 2002 at 12:50:00 +0100, David Olofson
wrote:
That is a good point.
I think the host needs to own it because only the host knows when
it can be free'd again.
No, that would be senders, not hosts, I think. (Remember that
connected plugins send *directly* to each other's queues. The host
doesn't know what's going on.)
That said, the memory management definitely has to go through a host
service of some kind.
I'm
leaning towards the "strings in raw data blocks" approach,
despite the little strlen() inconveniency for senders. Mostly
because it's one control data type less to care about, without
loss of real functionality.
On second thoughts, there might be something more to this: Who
*owns* the memory used for data to and from string/raw data
Controls? How is it allocated in the first place? That does require the host to provide malloc functions
though, but
IIRC they are in there allready.
Yes, but those are generic, and don't have to be implemented RT safe,
even in RT hosts. I think it might be nice if plugins can send
strings of reasonable sizes without RT trouble in "simple" hosts.
(We're talking about control *values*, really; not internal buffers
or anything.)
Data blocks that will be need in an RT context (eg.
bulk parameter
pasing) can be allocated ahead of time.
How? A plugin can't know what it will send until it's in process() -
and then it's too late...
If an instrument knows that its not going to need to
block again it
cabn tell the host to free it,
Yes...
if its going to be sent out then the
host can track it
Track it, how? Eavesdrop?
and free it if it consumes it itsself.
That's not a special case. The plugin would have to work like any
plugin when publishing controls in the net.
This is all quite a lot of work though. The
alternative would be
some sort of garbage collection (even just ref. counting would do)
I guess.
Yeah... Still need RT allocation for sending, though. *heh*
OTOH, it might be useful to assume that strings and raw data blocks
you get as control input are *read only*. Then you could send string
events referencing statically or previously allocated data buffers.
Rather than allocate memory from the host, you allocate a Memory
Block Handle, which is passed to the receiving control. MBHes would
have refcounting, so you can keep track of blocks sent.
If we state that the old MBH on a control becomes invalid when you
receive a now one, we can reuse them without double buffering. That
is, if you send one MBH to only one receiver, you're allowed to
change the data and resend the same MBH, as long as you send this or
another MBH to the original receiver.
Sounds a bit hairy, though.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
9:46 a.m.
On Wed, Dec 18, 2002 at 12:44:46 +0100, David Olofson wrote:
OK, I thought that the host was responsible for mashalling the queues. The
plugin must eb responsible for allocating the space, yes (unless it
originates form the host, eg. host sequenced data). The 'thing' that
consumes the data needs to free it.
Well if its always sending 256 floats as a parameter it know its going to
be 256*sizeof(float) vytes long, so it can preallocate a buffer that bit.
I guess the problem is that it doesn't know when to reallocate a new one.
This is definatly a difficult problem. In the simple example case of a UI
sending a block of floats to the DSP code, its not a problem, because the
UI part is running non-RT anyway.
- Steve
I think the
host needs to own it because only the host knows when
it can be free'd again.
No, that would be senders, not hosts, I think. (Remember that
connected plugins send *directly* to each other's queues. The host
doesn't know what's going on.)
That said, the memory management definitely has to go through a host
service of some kind. Data blocks
that will be need in an RT context (eg. bulk parameter
pasing) can be allocated ahead of time.
How? A plugin can't know what it will send until it's in process() -
and then it's too late... 
10:09 a.m.
Steve Harris wrote:
Well if its always sending 256 floats as a parameter it know its going to
be 256*sizeof(float) vytes long, so it can preallocate a buffer that bit.
I guess the problem is that it doesn't know when to reallocate a new one.
the solution i use is to allocate n events with every output port
instantiated (n being fixed thereafter). all event info can be
changed as the event travels the network; all except the field
that says where it originated from. from the node that eventually
consumes the event it is returned it to this origin, ie. the output.
tim
Data blocks that will be need in an RT context (eg.
bulk parameter
pasing) can be allocated ahead of time.
How? A plugin can't know what it will send until it's in process() -
and then it's too late...
10:16 a.m.
On Wednesday 18 December 2002 14.40, Steve Harris wrote:
Well if its always sending 256 floats as a parameter it know its
going to be 256*sizeof(float) vytes long, so it can preallocate a
buffer that bit.
Yes, but how many such data blocks are needed per engine cycle? You
can't change a string or data control more than once per block,
unless there's an allocation system similar to that for events.
On Wed, Dec 18, 2002 at 12:44:46 +0100, David Olofson
wrote:
OK, I thought that the host was responsible for mashalling the
queues.
Only when you send events from different contexts to the same Queue.
There's normally no need for it otherwise.
I think
the host needs to own it because only the host knows
when it can be free'd again.
No, that would be senders, not hosts, I think. (Remember that
connected plugins send *directly* to each other's queues. The
host doesn't know what's going on.)
That said, the memory management definitely has to go through a
host service of some kind. The plugin must eb responsible for allocating the
space,
yes (unless it originates form the host, eg. host sequenced data).
The 'thing' that consumes the data needs to free it.
Yes. That's how the event structs are handled, so it would be sort of
nice if we could stick with that throughout the API.
Data blocks that will be need in an RT context (eg.
bulk
parameter pasing) can be allocated ahead of time.
How? A plugin can't know what it will send until it's in
process() - and then it's too late... I guess the problem is that it doesn't know when
to reallocate a
new one.
That too.
This is definatly a difficult problem.
Well, that's RT memory management... *heh*
In the simple example case
of a UI sending a block of floats to the DSP code, its not a
problem, because the UI part is running non-RT anyway.
Yes, and that's one of the few cases where the host is marshalling
events. (It has to, because the event system in itself isn't thread
safe.) The host can either use an RT memory manager, or just restrict
things to one string/data block per engine cycle.
There is a feature for allocating audio buffers from the host.
Problem is that they're not of a hardcoded size, and they're
generally rather small; at most a few hundred bytes for low latency
hosts.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
11:26 a.m.
On Wed, Dec 18, 2002 at 03:10:42PM +0100, David Olofson wrote:
Yes, and that's one of the few cases where the
host is marshalling
events. (It has to, because the event system in itself isn't thread
safe.) The host can either use an RT memory manager, or just restrict
things to one string/data block per engine cycle.
That sounds reasonable.
There is a feature for allocating audio buffers from
the host.
Problem is that they're not of a hardcoded size, and they're
generally rather small; at most a few hundred bytes for low latency
hosts.
I can think of situations where you might want a few kB, eg. FFT plugins.
- Steve
2:17 p.m.
On Wednesday 18 December 2002 16.21, Steve Harris wrote:
Yes...
Let's assume a few things:
* Raw data blocks belong to the sender.
* Received raw data blocks are only valid until the
the plugin returns from process().
* Senders can figure out at init time how many
and how big data blocks they need.
That would make things quite trivial, as you can just assume that the
blocks you send are yours to use again for the next block. However,
I'm not sure how useful such a scheme is. It's sort of weird that you
end up with undefined "value" on some controls, unless they get input
first thing every block. *heh*
Alternatively, mark the block with an "owner ID", so they can be sent
back to the right place upon deallocation, as suggested by Tim
Goetze. I rather like that idea, since it allows receivers to just
keep the last block until a new one arrives.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
There is a
feature for allocating audio buffers from the host.
Problem is that they're not of a hardcoded size, and they're
generally rather small; at most a few hundred bytes for low
latency hosts.
I can think of situations where you might want a few kB, eg. FFT
plugins.
19 Dec
19 Dec
4:14 p.m.
On Wed, Dec 18, 2002 at 07:12:29 +0100, David Olofson wrote:
* Raw data blocks belong to the sender.
* Received raw data blocks are only valid until the
the plugin returns from process().
* Senders can figure out at init time how many
and how big data blocks they need.
That would make things quite trivial, as you can just assume that the
blocks you send are yours to use again for the next block. However,
I'm not sure how useful such a scheme is. It's sort of weird that you
end up with undefined "value" on some controls, unless they get input
first thing every block. *heh*
Well, they would presumably keep the values from the previous data block.
This sounds quite reasonable, and efficient.
- Steve
5:17 p.m.
On Thursday 19 December 2002 21.09, Steve Harris wrote:
On Wed, Dec 18, 2002 at 07:12:29 +0100, David Olofson
wrote:
> * Raw data blocks belong to the sender.
>
> * Received raw data blocks are only valid until the
> the plugin returns from process().
>
> * Senders can figure out at init time how many
> and how big data blocks they need.
[...]
Well, they would presumably keep the values from the
previous data
block.
Yes, but *where*?
A float control has the whole value in the event, and it's small and
fixed size, so you can just copy it into your internal Control data
array. Then you just throw the event struct away.
With raw data block events, OTOH, you get a block that could be of
any size. *If* you could copy it somewhere, you'd just throw the
event away, and the sender would officially own the block from then
on.
Problem is that if you can't copy the data, you'll need the block
until you get a new one.
We could either have an explicit "free_block()" action of some kind,
or we could just do it implicitly like this:
Cycle Actions (S == sender, R == receiver)
-----------------------------------------------------------
1 S: Sends an event with block X to R.
R: Gets the event and uses block X.
(R owns block X.)
2 S: Does nothing.
R: Does whatever.
(R owns block X.)
3 S: Sends a new block Y to R.
R: Gets block Y, and forgets about block X.
(S now owns block X again.)
(R owns block Y.)
4 (Sender may now reuse block X.)
Note that this *does* work even if R decides to forward the block to
a third plugin. Since blocks are read only, you may even "keep" the
block and still pass it on. The only requirement is that when S sends
a new block to replace it, the old block implicitly becomes property
of S again, before the next cycle begins.
That is, you can't delay raw data blocks with this scheme, without
copying the blocks. No major issue, though, as you're normally
supposed to have your own blocks anyway, if you have raw data
outputs. (You don't *need* to if you're not changing the data in the
block - but we can just consider delaying a form of change.)
This might look a bit hairy, but the rules are quite simple:
* When you get a new block, you must forget about
the old one before returning from process().
* When you send a block of your own, it is property
of the receiver until the cycle after you've sent
a new block to the same control.
This will cause trouble in hosts that have sub-nets with smaller
block size, if you're sending from within such a sub net. You'll
expect a block to be back in the cycle after the one you in which you
send the new block - but if the receiver has a larger block size, it
won't have a chance to run before you want your block back!
Ok, such a host *could* deal with this, but there's another bunch of
RT memory management issues...
No, I think this calls for either ever prevent RT memory management,
or that blocks are handled by the host and marked with their "home
address". That is, any plugin may set up one or more global memory
block pools. The host's "free_block()" call will know where each
block belongs.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
18 Dec
18 Dec
2:45 p.m.
OK, I thought
that the host was responsible for mashalling the
queues.
Only when you send events from different contexts to the same Queue.
There's normally no need for it otherwise.
6:27 p.m.
On Wednesday 18 December 2002 19.39, Tim Hockin wrote:
If the host expects to draw a UI that corresponds to an actual knob
moving, it needs to snoop events.
Well, you're bringing up an interesting set of problems.
First, note that a Control is either an input or an output - so you
can't drive a knob that way from a single Control. You need an output
control to move the knob, and an input control to get user input from
the knob. For an input widget, the plugin will normally have only an
input Control, so the output must come from somewhere else: The
sequencer.
So, the actual connections don't have much to do with snooping at
all, but rather look something like this:
|-----------| |-------|
| |--+----------->| Synth |
| | | |-----| |-------|
| Sequencer | +->| |
| | | GUI |
| |<----| |
|-----------| |-----|
That is, the sequencer sends data to the synth *and* the synth's GUI.
This is what makes the GUI track the automation.
When automation is on "record", the sequencer takes input from the
GUI as well, and basically records it and forwards it to the Synth at
the same time.
Display widgets in GUIs would be the same, although without control
output. Obviously, all widgets must have inputs.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
OK, I thought that the host was responsible for
mashalling the
queues.
Only when you send events from different contexts to the same
Queue. There's normally no need for it otherwise.
11:31 p.m.
So, the actual connections don't have much to do
with snooping at
all, but rather look something like this:
<snip nice ascii diagram>
That is, the sequencer sends data to the synth *and*
the synth's GUI.
This is what makes the GUI track the automation.
When automation is on "record", the sequencer takes input from the
GUI as well, and basically records it and forwards it to the Synth at
the same time.
I'd call that snooping. And if a plugin controls another plugin, the host
has to snoop those events, too, or the UI is out of sync.
19 Dec
19 Dec
9:41 a.m.
On Thursday 19 December 2002 04.25, Tim Hockin wrote:
My point is that it isn't really snooping from the API POV. You're
basically inserting Y splits in the net, sending the same events to
two places. The host should never mess with, or try to understand
these events; just make sure they get to both inputs.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
So, the actual
connections don't have much to do with snooping at
all, but rather look something like this:
<snip nice ascii diagram>
That is, the sequencer sends data to the synth
*and* the synth's
GUI. This is what makes the GUI track the automation.
When automation is on "record", the sequencer takes input from
the GUI as well, and basically records it and forwards it to the
Synth at the same time.
I'd call that snooping. And if a plugin controls another plugin,
the host has to snoop those events, too, or the UI is out of sync.
18 Dec
18 Dec
5:46 a.m.
On Wed, Dec 18, 2002 at 12:35:15 +0100, David Olofson wrote:
I'm leaning towards the "strings in raw data
blocks" approach,
despite the little strlen() inconveniency for senders. Mostly because
it's one control data type less to care about, without loss of real
functionality.
That sounds reasonable, we could require that strings (like filenames) are
NULL terminated as well, for compatability with libc.
- Steve
6:59 a.m.
Steve Harris wrote:
On Wed, Dec 18, 2002 at 12:35:15 +0100, David Olofson
wrote:
agree. steve, you were also right when you were pointing out
that inheritance would come in handy handling events i think.
tim
I'm leaning towards the "strings in raw
data blocks" approach,
despite the little strlen() inconveniency for senders. Mostly because
it's one control data type less to care about, without loss of real
functionality.
That sounds reasonable, we could require that strings (like filenames) are
NULL terminated as well, for compatability with libc.
7:33 a.m.
On Wednesday 18 December 2002 10.41, Steve Harris wrote:
On Wed, Dec 18, 2002 at 12:35:15 +0100, David Olofson
wrote:
Yes, definitely. Strings must be NULL terminated, but data blocks for
controls not hinted as strings don't have to be.
Actually, strings and raw data blocks probably should be different
types in the metadata, as they're not compatible anyway. (Sending raw
data to a string control can result in a segfault.)
The point is to have them use the same infrastructure. Same events,
same memory management system, same semantics. String controls just
make some asumptions about the format of the data, whereas raw data
controls don't.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---
I'm leaning towards the "strings in raw
data blocks" approach,
despite the little strlen() inconveniency for senders. Mostly
because it's one control data type less to care about, without
loss of real functionality.
That sounds reasonable, we could require that strings (like
filenames) are NULL terminated as well, for compatability with
libc.
8000
days inactive
8002
days old
linux-audio-dev@lists.linuxaudio.org
17 comments
4 participants
participants (4)
-
David Olofson -
Steve Harris -
Tim Goetze -
Tim Hockin