[LAD] LV2 realtime safe memory pool extension

[David Olofson]

Seems like the post I'm replying to below isn't getting through to the 
list. I got only the CC - which, BTW, I would normally not need at 
all, as I'm on the list. :-)


On Thursday 08 November 2007, Nedko Arnaudov wrote:
[...]
> > What I meant was, if all the host provides is a pool of uniformly 
> > sized chunks that are allocated when the plugin is initialized,
> > there doesn't seem to be much point in implementing it on the host
> > side.  
> > The naïve host side implementatio would add exactly nothing,
> > compared to plugins just allocating their own pools during
> > initialization.
> 
> There is point when you have several dynparam plugins that all user
> dynparam helper library requesting same sized chunks.

Yes... Point taken. Also, there is likely some probability of various 
other plugins using "common" chunk sizes, or sizes somehow related to 
the audio buffer size.

Then again, something that uses real time scripting would probably not 
fit into such patterns at all - or maybe it would? Depends on the 
implementation of the language, and how it's being used...


> > A proper real time manager, with arbitrary chunk sizes, would be
> > more motivated, as it adds functionality that plugins can't
> > implement internally; namely a shared memory pool.
> 
> I'm not against it but I dont see use of it for lv2 zyn plugins /
> lv2dynparam / zynjacku. Except for enum value strings that I plan to
> make fixed (runtime) max size.

And of course, if there's no suitably licensed implementation that 
works out of the box, I can se why you wouldn't want to go there 
right now...


[...]
> > It's really only entirely ROM based synths (or "small fixed sample 
> > set", in the case of software), virtual analog synths and the like 
> > that *can* implement real time safe "program change" at all.
> 
> One of ultimate goals of zynjacku is to make this happen with zyn
> plugins. Patch loading involves lv2dynparam communication that is
> realtime safe. It make take some time during which
> synth "parameters" 
> will be in intermediate state, but this will depend on cpu power and
> preallocation size. Things that user has control of.

Well, yes; if it's only a matter of doing some calculations and 
initializations "fast enough", it makes sense to go all the way and 
make it properly real time safe.


> >> and for lv2dynparam plugin library internal 
> >> operation too.
> >
> > It needs to "regroup" internally as a response to certain
> > parameter changes?
> 
> no idea what you mean by regroup,

Sorry; I was thinking modifications of processing graphs 
(adding/removing units), reallocation of delay buffers and that sort 
of stuff.


> for each parameter, group or 
> message there are internal fixed size structures.

Ok. We're talking "pool of event structs", basically.


[...]
> Point is whether arbitrary and fixed chunk allocators be in one
> extension. I tend to think that they should be separate because
> algorythms behind them are quite different and host may choose to
> implement only one of them.

Well, yes; pools of fixed size chunks are trivial to implement, and 
very fast and cache effective - whereas a "real" memory manager is 
not. Even a host that *does* implement the latter might take 
advantage of knowing when a plugin really just wants a pool of fixed 
size chunks, even if the systems are somehow connected behind the 
scenes.


> Also most lock free algorithms are patented so this can be point of
> host supporting only one feature set.

You shouldn't need any lock-free constructs, unless I'm missing 
something... Is it a requirement that real time safe allocations can 
be done in other threads than the real time audio thread?

I would think that it's sufficient if other threads can make non RT 
safe allocations. Then you can just have the corresponding call use 
malloc() and mark the blocks so that the host's RT safe free() will 
know how to handle these blocks. (They'd probably have to be passed 
to a background thread for destruction. I simple - AFAIK, non 
patented - single-writer, single-reader lock-free FIFO could be used 
for that.) RT reallocations of such "non RT" blocks would just not 
work, or you'd have to grab a block from the RT allocator and copy 
the data, even when the original block could theoretically be 
extended.


[...]
> The point is how should you define that feature A depends on feature
> B. LV2 in its current state, to my knowledge, does not treat this as
> bug.

Right... It does have to be defined somewhere, no matter what. 
Probably not a good idea to just leave it to documentation and (the 
as of now, non-existent) validation tools.


//David Olofson - Programmer, Composer, Open Source Advocate

.-------  http://olofson.net - Games, SDL examples  -------.
|        http://zeespace.net - 2.5D rendering engine       |
|       http://audiality.org - Music/audio engine          |
|     http://eel.olofson.net - Real time scripting         |
'--  http://www.reologica.se - Rheology instrumentation  --'