8 Nov
2007
8 Nov
'07
8:07 a.m.
On Thursday 08 November 2007, Nedko Arnaudov wrote:
[...]
The point is that some plugins need *realtime safe*
memory
allocation.
Well, yes; that part is quite obvious.
What I meant was, if all the host provides is a pool of uniformly
sized chunks that are allocated when the plugin is initialized, there
doesn't seem to be much point in implementing it on the host side.
The naïve host side implementatio would add exactly nothing, compared
to plugins just allocating their own pools during initialization.
A proper real time manager, with arbitrary chunk sizes, would be more
motivated, as it adds functionality that plugins can't implement
internally; namely a shared memory pool.
I
need this functionality for lv2zynadd plugin (like arbitrary voices
count allocation)
The "standard" solution is to pre-allocate and pre-initialize voice
structures for a fixed maximum polyphony.
Obviously, this becomes troublesome with modular synths and the like,
where the "voice structure" doesn't have a fixed size.
One solution is to allocate the voice pool as a response to "program
change" events. Of course, this makes the "programe change" operation
non real time safe, but it usually is anyway, due to samples being
loaded from disk and stuff. Many systems, both hardware and software,
are based entirely on the idea that patch loading is part of
setup/initialization, as that is often the only practical solution.
It's really only entirely ROM based synths (or "small fixed sample
set", in the case of software), virtual analog synths and the like
that *can* implement real time safe "program change" at all.
and for lv2dynparam plugin library internal
operation too.
It needs to "regroup" internally as a response to certain parameter
changes?
There were rumors in #lad that such functionality may
be useful
without lv2dynparam extension.
Well, yes; real time safe dynamic memory management can make life a
lot easier for some types of plugins, and/or reduce memory
requirements by having a shared pool. However, I think it needs to be
more generic than just a pool of fixed size chunks for the "shared
pool" part to be viable.
[...]
That reminds me that LV2 may need a way to specify
optional features
that interdepend. I.e. features (extensions) A and B are both
optional but if A is provided B is required. Of course plugin can
check this explicitly on instantiate and refuse to use feature, but
I'm not sure how vital is such approach.
Haven't really thought about this... Isn't it just a matter of plugins
and hosts listing *all* extensions? I mean, if you provide or ask for
this feature A, but not feature B, you have a bug. An automatic
validation tool would trap this right away - but of course,
someone/something has to tell the *tool* about these extension
interdependencies...
//David Olofson - Programmer, Composer, Open Source Advocate
.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
'-- http://www.reologica.se - Rheology instrumentation --'
8 Nov
8 Nov
9:53 a.m.
New subject: LV2 realtime safe memory pool extension
David Olofson <david(a)olofson.net> writes:
in zynadd, voices are not poliphony voices (they are called notes there)
but voice modules. zynadd is semi modular synth.
no idea what you mean by regroup, for each parameter, group or message
there are internal fixed size structures.
Point is whether arbitrary and fixed chunk allocators be in one
extension. I tend to think that they should be separate because
algorythms behind them are quite different and host may choose to
implement only one of them. Also most lock free algorithms are patented
so this can be point of host supporting only one feature set.
On Thursday 08 November 2007, Nedko Arnaudov wrote:
[...]
There is point when you have several dynparam plugins that all user
dynparam helper library requesting same sized chunks.
The point is that some plugins need *realtime
safe* memory
allocation.
Well, yes; that part is quite obvious.
What I meant was, if all the host provides is a pool of uniformly
sized chunks that are allocated when the plugin is initialized, there
doesn't seem to be much point in implementing it on the host side.
The naïve host side implementatio would add exactly nothing, compared
to plugins just allocating their own pools during initialization. A proper real time manager, with arbitrary chunk
sizes, would be more
motivated, as it adds functionality that plugins can't implement
internally; namely a shared memory pool.
I'm not against it but I dont see use of it for lv2 zyn plugins /
lv2dynparam / zynjacku. Except for enum value strings that I plan to
make fixed (runtime) max size.
I
need this functionality for lv2zynadd plugin (like arbitrary voices
count allocation)
The "standard" solution is to pre-allocate and pre-initialize voice
structures for a fixed maximum polyphony.
Obviously, this becomes troublesome with modular synths and the like,
where the "voice structure" doesn't have a fixed size. One solution is to allocate the voice pool as a
response to "program
change" events. Of course, this makes the "programe change" operation
non real time safe, but it usually is anyway, due to samples being
loaded from disk and stuff. Many systems, both hardware and software,
are based entirely on the idea that patch loading is part of
setup/initialization, as that is often the only practical solution.
It's really only entirely ROM based synths (or "small fixed sample
set", in the case of software), virtual analog synths and the like
that *can* implement real time safe "program change" at all.
One of ultimate goals of zynjacku is to make this happen with zyn
plugins. Patch loading involves lv2dynparam communication that is
realtime safe. It make take some time during which synth "parameters"
will be in intermediate state, but this will depend on cpu power and
preallocation size. Things that user has control of.
and for
lv2dynparam plugin library internal
operation too.
It needs to "regroup" internally as a response to certain parameter
changes? There were
rumors in #lad that such functionality may be useful
without lv2dynparam extension.
Well, yes; real time safe dynamic memory management can make life a
lot easier for some types of plugins, and/or reduce memory
requirements by having a shared pool. However, I think it needs to be
more generic than just a pool of fixed size chunks for the "shared
pool" part to be viable. [...]
The point is how should you define that feature A depends on feature
B. LV2 in its current state, to my knowledge, does not treat this as
bug.
--
Nedko Arnaudov <GnuPG KeyID: DE1716B0>
That reminds me that LV2 may need a way to
specify optional features
that interdepend. I.e. features (extensions) A and B are both
optional but if A is provided B is required. Of course plugin can
check this explicitly on instantiate and refuse to use feature, but
I'm not sure how vital is such approach.
Haven't really thought about this... Isn't it just a matter of plugins
and hosts listing *all* extensions? I mean, if you provide or ask for
this feature A, but not feature B, you have a bug. An automatic
validation tool would trap this right away - but of course,
someone/something has to tell the *tool* about these extension
interdependencies...
12:05 p.m.
New subject: LV2 realtime safe memory pool extension
Seems like the post I'm replying to below isn't getting through to the
list. I got only the CC - which, BTW, I would normally not need at
all, as I'm on the list. :-)
On Thursday 08 November 2007, Nedko Arnaudov wrote:
[...]
Yes... Point taken. Also, there is likely some probability of various
other plugins using "common" chunk sizes, or sizes somehow related to
the audio buffer size.
Then again, something that uses real time scripting would probably not
fit into such patterns at all - or maybe it would? Depends on the
implementation of the language, and how it's being used...
And of course, if there's no suitably licensed implementation that
works out of the box, I can se why you wouldn't want to go there
right now...
[...]
Well, yes; if it's only a matter of doing some calculations and
initializations "fast enough", it makes sense to go all the way and
make it properly real time safe.
no idea what you mean by regroup,
Sorry; I was thinking modifications of processing graphs
(adding/removing units), reallocation of delay buffers and that sort
of stuff.
What I meant
was, if all the host provides is a pool of uniformly
sized chunks that are allocated when the plugin is initialized,
there doesn't seem to be much point in implementing it on the host
side.
The naïve host side implementatio would add exactly nothing,
compared to plugins just allocating their own pools during
initialization.
There is point when you have several dynparam plugins that all user
dynparam helper library requesting same sized chunks. A proper real
time manager, with arbitrary chunk sizes, would be
more motivated, as it adds functionality that plugins can't
implement internally; namely a shared memory pool.
I'm not against it but I dont see use of it for lv2 zyn plugins /
lv2dynparam / zynjacku. Except for enum value strings that I plan to
make fixed (runtime) max size. It's
really only entirely ROM based synths (or "small fixed sample
set", in the case of software), virtual analog synths and the like
that *can* implement real time safe "program change" at all.
One of ultimate goals of zynjacku is to make this happen with zyn
plugins. Patch loading involves lv2dynparam communication that is
realtime safe. It make take some time during which
synth "parameters"
will be in intermediate state, but this will depend on cpu power and
preallocation size. Things that user has control of. and for lv2dynparam plugin library internal
operation too.
It needs to "regroup" internally as a response to certain
parameter changes? for each parameter, group or
message there are internal fixed size structures.
Ok. We're talking "pool of event structs", basically.
[...]
Point is whether arbitrary and fixed chunk allocators
be in one
extension. I tend to think that they should be separate because
algorythms behind them are quite different and host may choose to
implement only one of them.
Well, yes; pools of fixed size chunks are trivial to implement, and
very fast and cache effective - whereas a "real" memory manager is
not. Even a host that *does* implement the latter might take
advantage of knowing when a plugin really just wants a pool of fixed
size chunks, even if the systems are somehow connected behind the
scenes.
Also most lock free algorithms are patented so this
can be point of
host supporting only one feature set.
You shouldn't need any lock-free constructs, unless I'm missing
something... Is it a requirement that real time safe allocations can
be done in other threads than the real time audio thread?
I would think that it's sufficient if other threads can make non RT
safe allocations. Then you can just have the corresponding call use
malloc() and mark the blocks so that the host's RT safe free() will
know how to handle these blocks. (They'd probably have to be passed
to a background thread for destruction. I simple - AFAIK, non
patented - single-writer, single-reader lock-free FIFO could be used
for that.) RT reallocations of such "non RT" blocks would just not
work, or you'd have to grab a block from the RT allocator and copy
the data, even when the original block could theoretically be
extended.
[...]
The point is how should you define that feature A
depends on feature
B. LV2 in its current state, to my knowledge, does not treat this as
bug.
Right... It does have to be defined somewhere, no matter what.
Probably not a good idea to just leave it to documentation and (the
as of now, non-existent) validation tools.
//David Olofson - Programmer, Composer, Open Source Advocate
.------- http://olofson.net - Games, SDL examples -------.
| http://zeespace.net - 2.5D rendering engine |
| http://audiality.org - Music/audio engine |
| http://eel.olofson.net - Real time scripting |
'-- http://www.reologica.se - Rheology instrumentation --'
9 Nov
9 Nov
5:13 a.m.
New subject: LV2 realtime safe memory pool extension
David Olofson <david(a)olofson.net> writes:
You dont missin anything. I think you are right about lock-free
algorithms in rt-safe memory manager. I said a non-sense.
--
Nedko Arnaudov
Also most lock
free algorithms are patented so this can be point of
host supporting only one feature set.
You shouldn't need any lock-free constructs, unless I'm missing
something... Is it a requirement that real time safe allocations can
be done in other threads than the real time audio thread? 
8 Nov
8 Nov
8:58 p.m.
New subject: LV2 realtime safe memory pool extension
2007/11/8, Nedko Arnaudov <nedko(a)arnaudov.name>me>:
David Olofson <david(a)olofson.net> writes:
I do totally agree.
Stefano D'Angelo
On Thursday 08 November 2007, Nedko Arnaudov
wrote:
[...]
Point is whether arbitrary and fixed chunk allocators be in one
extension. I tend to think that they should be separate because
algorythms behind them are quite different and host may choose to
implement only one of them. Also most lock free algorithms are patented
so this can be point of host supporting only one feature set. There were rumors in #lad that such functionality
may be useful
without lv2dynparam extension.
Well, yes; real time safe dynamic memory management can make life a
lot easier for some types of plugins, and/or reduce memory
requirements by having a shared pool. However, I think it needs to be
more generic than just a pool of fixed size chunks for the "shared
pool" part to be viable.
6219
days inactive
6220
days old
linux-audio-dev@lists.linuxaudio.org
4 comments
3 participants
participants (3)
-
David Olofson -
Nedko Arnaudov -
Stefano D'Angelo