[LAD] Plugin buffer size restrictions

[David Robillard]

On Sat, 2012-05-26 at 10:02 +0000, Fons Adriaensen wrote:
> On Fri, May 25, 2012 at 10:43:36PM -0400, David Robillard wrote:
> 
> > I am making an LV2 extension for accessing and/or restricting the buffer
> > size.  This is straightforward, but I need to know just what
> > restrictions are actually needed by various sorts of DSP.
> > 
> > The sort of thing we're looking for here is "buffer size is always at
> > least 123 frames" or "buffer size is always a power of 2" or "buffer
> > size is always a multiple of 123".
> > 
> > I know "multiple of a power of two" is needed for convolution.  Not sure
> > what else...
> 
> It's not really *needed*, an FFT-based process can be designed to
> work with arbitrary and variable buffer sizes provided 
> 
> * this is known at init time,
> * and the user accepts the additional latency.

Yes, I meant w/o buffering.  It would be known at init time.

> 'Power of 2' in practice means 'a power of two that is not
> too small to make things inefficient'. So the plugin should
> have the option to require a minimum size as well, e.g. 64.
> 
> But I doubt very much if your extension is going to have much
> impact. I expect most hosts to simply ignore it, as implementing
> it could be quite invasive. Imagine a host having a chain a plugins
> in e.g. a channel strip. If one of them requires this extension
> and the host relies on variable buffer size to provide 'sample
> accurate' control of the others, then the host is required to
> implement extra buffering for that plugin. Probably no-go, some
> hosts even refuse plugins that don't work 'in place', even if
> the solution in that case is much simpler.

This is not a problem.  If a plugin exists that requires this
functionality, it won't work in hosts that can't provide that
functionality.  Compared to the situation of that plugin not existing at
all, something has been gained, not lost.  There is literally no
downside to this situation.  Feature, not a bug.

If, as you go on to argue, a fixed size solution is so superior, then
plugins will require it, and in turn hosts will support it.

> The following is part of a report on LV2 I was asked to 
> write some months ago:
> 
> ======
> 
> Audio processing (on PCs) is usually done in blocks of frames,
> so all components are designed to work that way, There are at
> least 4 possible ways the block size of a module or plugin
> could be defined:
> 
>   1. Fixed at compile time.    
>   2. Fixed at instantiation, power of 2 [*]
>   3. Fixed at instantiation, any (reasonable) value.
>   4. Variable - can be different in each process() call.
> 
> [*]  Or a similar restriction, e.g. either 2^N or 3*2^N.
> For example some FFT based processes could be designed to
> allow a block size of 3*2^N, or even 5*2^N, while making
> it accept any size would be out of the question in many
> cases. 
> 
> Clearly (1) is undesirable, and it will not be considered
> further. Some algorithms require (2) in order to meet their
> specs. They could be made to work with any block size, but
> this usually means extra latency, or some other form of loss
> of performance.
> 
> Almost all more sophisticated audio processing algorithms can 
> be implemented more easily and usually more efficiently if (3) 
> is guaranteed, even if the block size itself plays no role in
> their definition and does not affect their output in any way.
> This is in particular true for algorithms that partially run
> using some internal block size (determined by e.g. the sample
> rate and some combination of parameters), or those that have
> to manage a long history or complex state as part of their
> operation.
> 
> It is almost always possible to allow (4), the exceptions are
> the cases mentioned above that require (2). But an algorithm
> that should perform well will have to be designed so that the
> output it produces does not in any way depend on the actual 
> block sizes used at run time.
> 
> This includes ignoring any attempt to e.g. control the rate of 
> change of its internal parameters by a caller playing with the
> block size for each process() call. The idea that one could
> achieve 'sample accurate' control of an audio process in this
> way is a either a fallacy or irrelevant, depending on the case
> and how you look at it.
> 
> All this means that there is *no reason* to ever require (4),
> of a plugin or module, and no serious plugin standard or
> modular audio application should ever do this.
> 
> Yet that is precisely what LV2 does. While it is designed
> to be flexible and extensible, the one and only part of its
> specs that is fixed but shouldn't have been gets it wrong.

It is, for well-known historical reasons, adopted from LADSPA.  It's not
"fixed" (You say that because the following argument depends on it, but
it is simply not true).

Starting from LADSPA remains the reason there is a thriving LV2
ecosystem, working, in actual reality, right now, which only continually
gets better.  I said years ago 'growing' was the only way to do it, and
here we are: LV2 is doing just fine, and the magical monolithic API that
does everything everybody wants, initially, and correctly, still doesn't
exist.  Interesting.

> And allowing an extension to override this is not going to
> work, because such an extension can be very invasive to a
> host. If a host, e.g. Ardour, is from the start designed to
> exploit the freedom of (4), be it for the wrong reasons, it
> is never going to adopt an extension that would cripple major
> parts of its functionality, e.g. automation, or force them
> to be redesigned. Even less so if the plugin system does not
> provide an alternative way, guaranteed to work with any plugin,
> to obtain the same result.

You criticize LV2 for having control ports, but then say an alternative
is impossible because it would be invasive?  This does not make sense.
Ardour supports LADSPA in any case, and several other APIs as well.

This is basically equivalent to saying Ardour (or pretty much any
existing program) could not implement the right solution for this.
Nothing whatsoever to do with LV2 (i.e. would be true of an API that
*only* had the right solution)... but you blame it on LV2 anyway?

> This is the really sad aspect of the situation: LV2 requires
> so little of a plugin that in order to make things like e.g.
> automation possible it must allow hacks such as playing with
> the block size of each call. So hosts will do this, and that
> wart will never go away.
> 
> Things should have been the other way round: allowing hosts
> to use variable block sizes should have been the extenstion.
> Of course nobody would ever use it, because in a well designed
> system or application it is not necessary (not even for the
> things e.g. Ardour uses it for).

Frankly, this whole thing is more like FUD than a reasonable technical
report, and I don't really consider that an appropriate thing to do when
consulted as an expert to report on a technology.  You have misled
whoever you wrote this for.

I could write a long rebuttal here, but that would just get tedious.  I
will simply summarize:

100% of the things you blame on LV2 here are either:

1) Inherent to the problem/solution
2) Nothing to do with LV2 at all

and 100% of the things you say can't happen - can.

Rather than argue about it, which history has shown to be pointless, I
will simply prove it, by actually backing up my talk with action.

Please describe your good solution (if you indeed even have one), and
(if it is actually good) I will implement it.

I'll make a spec for it.

I'll implement a plugin that uses it.

I'll implement it in Ardour, among others.

I'll make it all work just fine together, personally.  All you have to
do is describe it.

FUD, or solution.  Your choice.  I'll be over here solving problems if
you need me.

-dr