26 May
2012
26 May
'12
4:43 a.m.
Hello laddies,
I am making an LV2 extension for accessing and/or restricting the buffer
size. This is straightforward, but I need to know just what
restrictions are actually needed by various sorts of DSP.
The sort of thing we're looking for here is "buffer size is always at
least 123 frames" or "buffer size is always a power of 2" or "buffer
size is always a multiple of 123".
I know "multiple of a power of two" is needed for convolution. Not sure
what else...
-dr
26 May
26 May
12:02 p.m.
On Fri, May 25, 2012 at 10:43:36PM -0400, David Robillard wrote:
I am making an LV2 extension for accessing and/or
restricting the buffer
size. This is straightforward, but I need to know just what
restrictions are actually needed by various sorts of DSP.
The sort of thing we're looking for here is "buffer size is always at
least 123 frames" or "buffer size is always a power of 2" or "buffer
size is always a multiple of 123".
I know "multiple of a power of two" is needed for convolution. Not sure
what else...
It's not really *needed*, an FFT-based process can be designed to
work with arbitrary and variable buffer sizes provided
* this is known at init time,
* and the user accepts the additional latency.
'Power of 2' in practice means 'a power of two that is not
too small to make things inefficient'. So the plugin should
have the option to require a minimum size as well, e.g. 64.
But I doubt very much if your extension is going to have much
impact. I expect most hosts to simply ignore it, as implementing
it could be quite invasive. Imagine a host having a chain a plugins
in e.g. a channel strip. If one of them requires this extension
and the host relies on variable buffer size to provide 'sample
accurate' control of the others, then the host is required to
implement extra buffering for that plugin. Probably no-go, some
hosts even refuse plugins that don't work 'in place', even if
the solution in that case is much simpler.
The following is part of a report on LV2 I was asked to
write some months ago:
======
Audio processing (on PCs) is usually done in blocks of frames,
so all components are designed to work that way, There are at
least 4 possible ways the block size of a module or plugin
could be defined:
1. Fixed at compile time.
2. Fixed at instantiation, power of 2 [*]
3. Fixed at instantiation, any (reasonable) value.
4. Variable - can be different in each process() call.
[*] Or a similar restriction, e.g. either 2^N or 3*2^N.
For example some FFT based processes could be designed to
allow a block size of 3*2^N, or even 5*2^N, while making
it accept any size would be out of the question in many
cases.
Clearly (1) is undesirable, and it will not be considered
further. Some algorithms require (2) in order to meet their
specs. They could be made to work with any block size, but
this usually means extra latency, or some other form of loss
of performance.
Almost all more sophisticated audio processing algorithms can
be implemented more easily and usually more efficiently if (3)
is guaranteed, even if the block size itself plays no role in
their definition and does not affect their output in any way.
This is in particular true for algorithms that partially run
using some internal block size (determined by e.g. the sample
rate and some combination of parameters), or those that have
to manage a long history or complex state as part of their
operation.
It is almost always possible to allow (4), the exceptions are
the cases mentioned above that require (2). But an algorithm
that should perform well will have to be designed so that the
output it produces does not in any way depend on the actual
block sizes used at run time.
This includes ignoring any attempt to e.g. control the rate of
change of its internal parameters by a caller playing with the
block size for each process() call. The idea that one could
achieve 'sample accurate' control of an audio process in this
way is a either a fallacy or irrelevant, depending on the case
and how you look at it.
All this means that there is *no reason* to ever require (4),
of a plugin or module, and no serious plugin standard or
modular audio application should ever do this.
Yet that is precisely what LV2 does. While it is designed
to be flexible and extensible, the one and only part of its
specs that is fixed but shouldn't have been gets it wrong.
And allowing an extension to override this is not going to
work, because such an extension can be very invasive to a
host. If a host, e.g. Ardour, is from the start designed to
exploit the freedom of (4), be it for the wrong reasons, it
is never going to adopt an extension that would cripple major
parts of its functionality, e.g. automation, or force them
to be redesigned. Even less so if the plugin system does not
provide an alternative way, guaranteed to work with any plugin,
to obtain the same result.
This is the really sad aspect of the situation: LV2 requires
so little of a plugin that in order to make things like e.g.
automation possible it must allow hacks such as playing with
the block size of each call. So hosts will do this, and that
wart will never go away.
Things should have been the other way round: allowing hosts
to use variable block sizes should have been the extenstion.
Of course nobody would ever use it, because in a well designed
system or application it is not necessary (not even for the
things e.g. Ardour uses it for).
======
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
8:23 p.m.
On Sat, 2012-05-26 at 10:02 +0000, Fons Adriaensen wrote:
On Fri, May 25, 2012 at 10:43:36PM -0400, David
Robillard wrote:
Yes, I meant w/o buffering. It would be known at init time.
I am making an LV2 extension for accessing and/or
restricting the buffer
size. This is straightforward, but I need to know just what
restrictions are actually needed by various sorts of DSP.
The sort of thing we're looking for here is "buffer size is always at
least 123 frames" or "buffer size is always a power of 2" or "buffer
size is always a multiple of 123".
I know "multiple of a power of two" is needed for convolution. Not sure
what else...
It's not really *needed*, an FFT-based process can be designed to
work with arbitrary and variable buffer sizes provided
* this is known at init time,
* and the user accepts the additional latency. 'Power of 2' in practice means 'a power of
two that is not
too small to make things inefficient'. So the plugin should
have the option to require a minimum size as well, e.g. 64.
But I doubt very much if your extension is going to have much
impact. I expect most hosts to simply ignore it, as implementing
it could be quite invasive. Imagine a host having a chain a plugins
in e.g. a channel strip. If one of them requires this extension
and the host relies on variable buffer size to provide 'sample
accurate' control of the others, then the host is required to
implement extra buffering for that plugin. Probably no-go, some
hosts even refuse plugins that don't work 'in place', even if
the solution in that case is much simpler.
This is not a problem. If a plugin exists that requires this
functionality, it won't work in hosts that can't provide that
functionality. Compared to the situation of that plugin not existing at
all, something has been gained, not lost. There is literally no
downside to this situation. Feature, not a bug.
If, as you go on to argue, a fixed size solution is so superior, then
plugins will require it, and in turn hosts will support it.
The following is part of a report on LV2 I was asked
to
write some months ago:
======
Audio processing (on PCs) is usually done in blocks of frames,
so all components are designed to work that way, There are at
least 4 possible ways the block size of a module or plugin
could be defined:
1. Fixed at compile time.
2. Fixed at instantiation, power of 2 [*]
3. Fixed at instantiation, any (reasonable) value.
4. Variable - can be different in each process() call.
[*] Or a similar restriction, e.g. either 2^N or 3*2^N.
For example some FFT based processes could be designed to
allow a block size of 3*2^N, or even 5*2^N, while making
it accept any size would be out of the question in many
cases.
Clearly (1) is undesirable, and it will not be considered
further. Some algorithms require (2) in order to meet their
specs. They could be made to work with any block size, but
this usually means extra latency, or some other form of loss
of performance.
Almost all more sophisticated audio processing algorithms can
be implemented more easily and usually more efficiently if (3)
is guaranteed, even if the block size itself plays no role in
their definition and does not affect their output in any way.
This is in particular true for algorithms that partially run
using some internal block size (determined by e.g. the sample
rate and some combination of parameters), or those that have
to manage a long history or complex state as part of their
operation.
It is almost always possible to allow (4), the exceptions are
the cases mentioned above that require (2). But an algorithm
that should perform well will have to be designed so that the
output it produces does not in any way depend on the actual
block sizes used at run time.
This includes ignoring any attempt to e.g. control the rate of
change of its internal parameters by a caller playing with the
block size for each process() call. The idea that one could
achieve 'sample accurate' control of an audio process in this
way is a either a fallacy or irrelevant, depending on the case
and how you look at it.
All this means that there is *no reason* to ever require (4),
of a plugin or module, and no serious plugin standard or
modular audio application should ever do this.
Yet that is precisely what LV2 does. While it is designed
to be flexible and extensible, the one and only part of its
specs that is fixed but shouldn't have been gets it wrong.
It is, for well-known historical reasons, adopted from LADSPA. It's not
"fixed" (You say that because the following argument depends on it, but
it is simply not true).
Starting from LADSPA remains the reason there is a thriving LV2
ecosystem, working, in actual reality, right now, which only continually
gets better. I said years ago 'growing' was the only way to do it, and
here we are: LV2 is doing just fine, and the magical monolithic API that
does everything everybody wants, initially, and correctly, still doesn't
exist. Interesting.
And allowing an extension to override this is not
going to
work, because such an extension can be very invasive to a
host. If a host, e.g. Ardour, is from the start designed to
exploit the freedom of (4), be it for the wrong reasons, it
is never going to adopt an extension that would cripple major
parts of its functionality, e.g. automation, or force them
to be redesigned. Even less so if the plugin system does not
provide an alternative way, guaranteed to work with any plugin,
to obtain the same result.
You criticize LV2 for having control ports, but then say an alternative
is impossible because it would be invasive? This does not make sense.
Ardour supports LADSPA in any case, and several other APIs as well.
This is basically equivalent to saying Ardour (or pretty much any
existing program) could not implement the right solution for this.
Nothing whatsoever to do with LV2 (i.e. would be true of an API that
*only* had the right solution)... but you blame it on LV2 anyway?
This is the really sad aspect of the situation: LV2
requires
so little of a plugin that in order to make things like e.g.
automation possible it must allow hacks such as playing with
the block size of each call. So hosts will do this, and that
wart will never go away.
Things should have been the other way round: allowing hosts
to use variable block sizes should have been the extenstion.
Of course nobody would ever use it, because in a well designed
system or application it is not necessary (not even for the
things e.g. Ardour uses it for).
Frankly, this whole thing is more like FUD than a reasonable technical
report, and I don't really consider that an appropriate thing to do when
consulted as an expert to report on a technology. You have misled
whoever you wrote this for.
I could write a long rebuttal here, but that would just get tedious. I
will simply summarize:
100% of the things you blame on LV2 here are either:
1) Inherent to the problem/solution
2) Nothing to do with LV2 at all
and 100% of the things you say can't happen - can.
Rather than argue about it, which history has shown to be pointless, I
will simply prove it, by actually backing up my talk with action.
Please describe your good solution (if you indeed even have one), and
(if it is actually good) I will implement it.
I'll make a spec for it.
I'll implement a plugin that uses it.
I'll implement it in Ardour, among others.
I'll make it all work just fine together, personally. All you have to
do is describe it.
FUD, or solution. Your choice. I'll be over here solving problems if
you need me.
-dr
8:42 p.m.
On Sat, May 26, 2012 at 02:23:52PM -0400, David Robillard wrote:
[Something about buffer sizes]
I didn't follow the discussion, but maybe you want to address something
related, too: make the buffers 32-byte aligned, so that plugins can use
AVX.
I don't know who's responsible for providing the buffers (the plugin or
the host) in LV2, so my comment may not apply at all.
Just my €0.01
--
mail: adi(a)thur.de http://adi.thur.de PGP/GPG: key via keyserver
10:05 p.m.
On Sat, May 26, 2012 at 02:23:52PM -0400, David Robillard wrote:
This is not a problem. If a plugin exists that
requires this
functionality, it won't work in hosts that can't provide that
functionality. Compared to the situation of that plugin not existing at
all, something has been gained, not lost. There is literally no
downside to this situation. Feature, not a bug.
Compared to the actual situation (without that extension) something
is gained indeed. But it's like paying off $5 if you start with a
debt of $10000.
If, as you go on to argue, a fixed size solution is so
superior,
then plugins will require it, and in turn hosts will support it.
I doubt it. That would happen if host and plugin authors
* strive to make high-quality plugins only,
* take decision on technical grounds rather than
opportunistic ones.
The first condition is contradicted by history - I won't
hesitate to say that 50% of all LADSPA plugins are crap
(somewhat better for LV2), and that 90% have serious
flaws that are more often than not related to the way
that control parameters are handled.
Your own argumentation of why certain things in LV2 are what
they are (because LADSPA did it that way) is an example of
the contrary of the second.
It is, for well-known historical reasons, adopted from
LADSPA.
I know.
It's not "fixed" (You say that because
the following argument
depends on it, but it is simply not true).
It was part of the 'core spec' last time I looked. All plugins
are supposed to accept a process call for any number of frames.
Which leads to an interesting observation: you extension
actually negates part of the core specs. So that is allowed ?
You criticize LV2 for having control ports, but then
say an alternative
is impossible because it would be invasive? This does not make sense.
Not for having control ports. For encouraging (almost mandating -
there is no alternative in the core spec) a way to provide control
values at arbitrary points by manipulating the nframes of a process()
call. There is a world of difference between
1. telling a plugin that at N frames from the current position the
parameter P should have value V, and
2. doing the same, while also requiring that the plugin outputs
N frames at that time.
My argumentation is that doing (2) is a bad idea, and even more so if
you consider that (1) is a silly thing to in the first place, at least
for the usual audio processing tasks such as EQ, dynamics, and most
effects. The exception is synthesis modules of course, but those should
have a dedicated mechanism for it anyway, or accept audio rate controls.
The full report included a detailed anaysis of why (1) is a bad
idea in most cases (with the exception of synthesis modules). It
is because it makes it almost impossible for the plugin code to
compute good parameter trajectories. A well-designed EQ, effect,
compressor, etc. should actually *ignore* attempts to control its
internals in such a way. So there was never any need to allow
arbitrary nframes values. The correct thing to do would be to
remove that from the core spec, and provide an extension for
finer-grained 'sample accurate' control.
FUD, or solution. Your choice. I'll be over here
solving problems if
you need me.
Been there before, I get used to it.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
10:23 p.m.
as once again another discussion that could be a useful technical
discussion turns into a stupid spitting match. sigh.
look fons, variable size frame counts were one approach to a genuine
problem: how to deliver automation data to plugins. but they were not
added solely for that reason.
you write as if when we designed LADSPA to "support" this, that no
other plugin API formed a part of the considerations that went into
that design process. every other plugin API that had a visible spec
specifically provided for variable frame counts. even in the AudioUnit
spec, which additionally provides much more sophisticated, scheduled,
non-scalar mechanisms to deliver automation data, it is *still*
mandatory for plugins to be able to accept any frame count from zero
up to some size specified during the last reset.
this is also a characteristic of ASIO and EASI and Rewire and many
other audio-related APIs.
now, we can sit here and listen to you and dave mudslinging about the
sanity of this or that. you can, if you want, insist that everyone who
designed AU and RTAS and VST also don't understand audio programming.
maybe you're even right about that (though it seems less likely). but
that doesn't actually solve anything or move anything forward.
ardour has no requirement to subdivide blocks for automation data.
precisely because i wasn't interested in implementing support for AU's
slightly complex automation API, its entirely possible to have do
block-accurate automation only. and if someone wanted to a bit of work
to use an API like AU where you can pre-schedule parameter changes, it
could revert to sample accurate when necessary. plugins are free to do
whatever they want with the host's attempt to do this.
On 5/26/12, Fons Adriaensen <fons(a)linuxaudio.org> wrote:
On Sat, May 26, 2012 at 02:23:52PM -0400, David
Robillard wrote:
This is not a problem. If a plugin exists that
requires this
functionality, it won't work in hosts that can't provide that
functionality. Compared to the situation of that plugin not existing at
all, something has been gained, not lost. There is literally no
downside to this situation. Feature, not a bug.
Compared to the actual situation (without that extension) something
is gained indeed. But it's like paying off $5 if you start with a
debt of $10000.
If, as you go on to argue, a fixed size solution
is so superior,
then plugins will require it, and in turn hosts will support it.
I doubt it. That would happen if host and plugin authors
* strive to make high-quality plugins only,
* take decision on technical grounds rather than
opportunistic ones.
The first condition is contradicted by history - I won't
hesitate to say that 50% of all LADSPA plugins are crap
(somewhat better for LV2), and that 90% have serious
flaws that are more often than not related to the way
that control parameters are handled.
Your own argumentation of why certain things in LV2 are what
they are (because LADSPA did it that way) is an example of
the contrary of the second.
It is, for well-known historical reasons, adopted
from LADSPA.
I know.
It's not "fixed" (You say that
because the following argument
depends on it, but it is simply not true).
It was part of the 'core spec' last time I looked. All plugins
are supposed to accept a process call for any number of frames.
Which leads to an interesting observation: you extension
actually negates part of the core specs. So that is allowed ?
You criticize LV2 for having control ports, but
then say an alternative
is impossible because it would be invasive? This does not make sense.
Not for having control ports. For encouraging (almost mandating -
there is no alternative in the core spec) a way to provide control
values at arbitrary points by manipulating the nframes of a process()
call. There is a world of difference between
1. telling a plugin that at N frames from the current position the
parameter P should have value V, and
2. doing the same, while also requiring that the plugin outputs
N frames at that time.
My argumentation is that doing (2) is a bad idea, and even more so if
you consider that (1) is a silly thing to in the first place, at least
for the usual audio processing tasks such as EQ, dynamics, and most
effects. The exception is synthesis modules of course, but those should
have a dedicated mechanism for it anyway, or accept audio rate controls.
The full report included a detailed anaysis of why (1) is a bad
idea in most cases (with the exception of synthesis modules). It
is because it makes it almost impossible for the plugin code to
compute good parameter trajectories. A well-designed EQ, effect,
compressor, etc. should actually *ignore* attempts to control its
internals in such a way. So there was never any need to allow
arbitrary nframes values. The correct thing to do would be to
remove that from the core spec, and provide an extension for
finer-grained 'sample accurate' control.
FUD, or solution. Your choice. I'll be over
here solving problems if
you need me.
Been there before, I get used to it.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
10:27 p.m.
On Sat, 2012-05-26 at 20:42 +0200, Adrian Knoth wrote:
On Sat, May 26, 2012 at 02:23:52PM -0400, David
Robillard wrote:
[Something about buffer sizes]
I didn't follow the discussion, but maybe you want to address something
related, too: make the buffers 32-byte aligned, so that plugins can use
AVX.
I don't know who's responsible for providing the buffers (the plugin or
the host) in LV2, so my comment may not apply at all.
Ah, right, that's another thing. The hosts do. In reality most
probably are already aligned, but the weird cycle size thing some do
circumvents that, of course.
As it happens, none of this buffer size stuff is actually guaranteed by
Jack, which would need to happen to make implementing buffer size
guarantees actually possible. IIRC in Jack1 it is always true, but not
always for Jack2. If so, there needs to be Jack API for the host to
know what it can actually guarantee.
-dr
10:59 p.m.
On Sat, May 26, 2012 at 04:22:58PM -0400, Paul Davis wrote:
as once again another discussion that could be a
useful technical
discussion turns into a stupid spitting match. sigh.
So far I didn't spit and I have no intention to do such a thing.
look fons, variable size frame counts were one
approach to a genuine
problem: how to deliver automation data to plugins. but they were not
added solely for that reason.
That is probably true. But that doesn't make using that mechanism
to deliver automation data a good idea. A well-designed plugin
should actually impose its own limits on the rate of change or
the bandwidth of its controls, which means it should ignore some
of the things the host tries to make it do. Which in turn means
it's futile for the host to even try.
you write as if when we designed LADSPA to
"support" this, that no
other plugin API formed a part of the considerations that went into
that design process. every other plugin API that had a visible spec
specifically provided for variable frame counts. even in the AudioUnit
spec, which additionally provides much more sophisticated, scheduled,
non-scalar mechanisms to deliver automation data, it is *still*
mandatory for plugins to be able to accept any frame count from zero
up to some size specified during the last reset.
I didn't comment *at all* on the desing process of the LADSPA plugin
standard. Please re-read. I did imply that copying some aspects of it
- the requirement for a plugin to accept any value of nframes as an
easy way to allow 'sample accurate' control - into the 'next generation'
was probably a bad idea. Some others parts of the LADSPA specs were not
retained - for good reasons.
ardour has no requirement to subdivide blocks for
automation data.
precisely because i wasn't interested in implementing support for AU's
slightly complex automation API, its entirely possible to have do
block-accurate automation only. and if someone wanted to a bit of work
to use an API like AU where you can pre-schedule parameter changes, it
could revert to sample accurate when necessary. plugins are free to do
whatever they want with the host's attempt to do this.
Question: do A2 and/or A3 actually subdivide a period in order to
deliver automation data or not ?
If yes, do they also try to deliver 'sample accurate' control values
in case these are real-time (from GUI widgets, MIDI, OSC) ?
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
11:08 p.m.
On Sat, 2012-05-26 at 20:05 +0000, Fons Adriaensen wrote:
This is a bit difficult to decipher. You are saying you think
block-accurate controls at a given fixed minimum size are good enough?
That's just LADSPA with a simple restriction on sample_count parameter.
However, making "synthesis modules" an exception is kind of pointless
over-classification. Either a sample accurate control mechanism is
required, or it isn't. There is no hard distinction between "synthesis
modules" and other things anyway, it is not a useful (false) dichotomy.
On Sat, May 26, 2012 at 02:23:52PM -0400, David
Robillard wrote:
[...]
Which leads to an interesting observation: you
extension
actually negates part of the core specs. So that is allowed ?
Sure it is. In this case, it's not even weird, if we do this now it
will look EXACTLY the same as if we did it from day one. It's a
textbook case of what the feature mechanism was designed to do, and not
weird or unnatural at all.
In LV2, stuff like this is done via "features". A feature is either
optionally supported, or required, by a plugin. In code, the host
passes a struct to instantiate for each supported feature which may
contain whatever (in this case, probably a get_buffer_size() function
and so on).
The host may not instantiate a plugin if it *requires* a feature it can
not provide. So, in this case, if the plugin requires e.g. a power of 2
buffer size, and the host can't provide that, well.. it can't provide
that and the plugin can't be used. This isn't crazy decentralized
metadata pipe-dream stuff, it's the way the API would be made from day
one anyway; extremely straightforward basic C stuff.
Because of what "required" means, you can do pretty much literally
anything with a required feature. You could define a feature where the
host must fetch the fifteenth byte from http://linuxaudio.org and pass
that to instantiate if you wanted to. It's crazy, and *requiring* that
feature surely means your plugin isn't going to work almost anywhere,
but that's fine.
In this case, however, it's not crazy at all. It's the most natural
thing in the world.
Here, notice a plugin not working in the host is *inherent*. This is
why LV2 folks generally don't consider this a "problem" - it isn't one.
If the host literally *can't* provide a given feature (e.g. fixed buffer
size) and the plugin *requires* it to function at all, well, clearly the
plugin can't work in that host. LV2 merely provides a mechanism to
express such things. The decision of whether to do them is inherent.
It is not my job, or your job, to tell plugin and host authors what they
can and can not do. Ad hoc extensions are where innovation happens,
where solutions get sorted out *before* they become part of the main
spec. You can't sort them out without actually implementing them.
Software simply does not work that way. Ever.
Different host authors have different opinions about conservatism and
compatibility and such. This is fine and good and expected and natural.
It will all work out eventually. You've been around here long enough to
know everything ain't happening tomorrow regardless :)
tl;dr: If buffer size restrictions are the way to go, then let's go that
way. It's just that simple.
You criticize
LV2 for having control ports, but then say an alternative
is impossible because it would be invasive? This does not make sense.
Not for having control ports. For encouraging (almost mandating -
there is no alternative in the core spec) a way to provide control
values at arbitrary points by manipulating the nframes of a process()
call. There is a world of difference between
1. telling a plugin that at N frames from the current position the
parameter P should have value V, and
2. doing the same, while also requiring that the plugin outputs
N frames at that time.
My argumentation is that doing (2) is a bad idea, and even more so if
you consider that (1) is a silly thing to in the first place, at least
for the usual audio processing tasks such as EQ, dynamics, and most
effects. The exception is synthesis modules of course, but those should
have a dedicated mechanism for it anyway, or accept audio rate controls. The full report included a detailed anaysis of why (1)
is a bad
idea in most cases (with the exception of synthesis modules). It
is because it makes it almost impossible for the plugin code to
compute good parameter trajectories. A well-designed EQ, effect,
compressor, etc. should actually *ignore* attempts to control its
internals in such a way. So there was never any need to allow
arbitrary nframes values. The correct thing to do would be to
remove that from the core spec, and provide an extension for
finer-grained 'sample accurate' control.
It's odd to speak of this like there has been some active decision to
"allow" arbitrary nframes values. It's more accurate to say that
basically nothing has been said about this thus far. It is simply an
absence of well-defined restriction because nobody has defined them yet.
If you're content with LADSPA style controls with just a restriction on
block size, that's trivial. I don't really see why you've made such a
big deal about it. There's plenty of questions raised by a sample
accurate control port (mainly whether to MUX the controls to a single
event buffer or not), but no questions about that. From a spec POV,
it's easy.
It'd have been solved and implemented long long ago if it was brought up
in useful channels and not weird cloak and dagger reports...
-dr
27 May
27 May
12:02 a.m.
On Sat, May 26, 2012 at 05:08:47PM -0400, David Robillard wrote:
Here, notice a plugin not working in the host is
*inherent*. This is
why LV2 folks generally don't consider this a "problem" - it isn't
one.
If the host literally *can't* provide a given feature (e.g. fixed buffer
size) and the plugin *requires* it to function at all, well, clearly the
plugin can't work in that host. LV2 merely provides a mechanism to
express such things. The decision of whether to do them is inherent.
Understood, and not for the first time. But it leads to chicken/egg
situations. I'm not going to write an LV2 requiring N features if
there isn't a host providing them. And probably vice versa for host
authors. And that is why I'd prefer a more demanding core spec.
Dave, if there is anything I find contradictory in your argumentation
(and that is often why I react to it) it is that you seem to sweep
this point under the carpet, while in other cases you can be very
pragmatic.
It is not my job, or your job, to tell plugin and host
authors what they
can and can not do. Ad hoc extensions are where innovation happens,
where solutions get sorted out *before* they become part of the main
spec. You can't sort them out without actually implementing them.
Software simply does not work that way. Ever.
Mmm, I'd like to disagree. Most open source software may be developed
in such conditions - but that is both a strength and a curse. Having
worked some ten years developing software for space and aviation I
can tell you things are different there. And that has some advantages.
It will all work out eventually. You've been
around here long enough to
know everything ain't happening tomorrow regardless :)
Yes. But that's no reason to not learn from the past. Which is what
LV2 started out to do...
This is a bit difficult to decipher. You are saying
you think
block-accurate controls at a given fixed minimum size are good enough?
For anything except synthesis, yes. Not only good enough, but actually
much easier to implement correctly.
However, making "synthesis modules" an
exception is kind of pointless
over-classification. Either a sample accurate control mechanism is
required, or it isn't. There is no hard distinction between "synthesis
modules" and other things anyway, it is not a useful (false) dichotomy.
If 'sample accurate' control would be the only difference between
'general purpose' and 'synthesis' modules I'd agree with you.
But it isn't. 'Polyphonic' is not the same as 'multichannel' for one.
And synthesis modules may require real-time instantiation - no need
for that in case of an effect in a DAW. My conclusion is that the
two are fundamentally different. Which doesn't mean that a plugin
standard couldn't provide both. But probably not by ignoring the
differences.
It's odd to speak of this like there has been some
active decision to
"allow" arbitrary nframes values. It's more accurate to say that
basically nothing has been said about this thus far. It is simply an
absence of well-defined restriction because nobody has defined them yet.
AFAIK, the 'core spec' requires plugins to accept a non-constant nframes:
"Thus the "control rate" is determined by the block size, which is
controlled by the host (and not necessarily constant)."
It'd have been solved and implemented long long
ago if it was brought up
in useful channels and not weird cloak and dagger reports...
Nothing cloak and dagger about it. If someone is prepared to pay me for
writing a report on something I do have strong opinions about (and very
probably because of that), should I refuse that ? It's a consultancy
job just like any other. And at least one other well-known developer
got the same.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
12:34 a.m.
On Sat, May 26, 2012 at 4:59 PM, Fons Adriaensen <fons(a)linuxaudio.org>wrote;wrote:
On Sat, May 26, 2012 at 04:22:58PM -0400, Paul Davis
wrote:
I think you know precisely what I mean.
as once again another discussion that could be a
useful technical
discussion turns into a stupid spitting match. sigh.
So far I didn't spit and I have no intention to do such a thing.
look fons, variable size frame counts were one
approach to a genuine
problem: how to deliver automation data to plugins. but they were not
added solely for that reason.
That is probably true. But that doesn't make using that mechanism
to deliver automation data a good idea.
in the absence of a part of the API designed specifically to deliver
automation data, its one of the few reasonably straightforward choices.
A well-designed plugin
should actually impose its own limits on the rate of change or
plugins are free to do this with or without a requirement that they handle
0..N frames. certainly, doing so can be a bit more complex with the 0..N
requirement, but its not impossible - there is an entire company's suite of
plugins that all do this across more than 6 plugin APIs all of which have
the 0..N requirement.
I didn't comment *at all* on the desing process of the LADSPA plugin
standard. Please re-read. I did imply that copying
some aspects of it
LADSPA and LV2 are not independent here. the requirement that you're
talking about:
- the requirement for a plugin to accept any value of
nframes
is shared across just about every plugin API out there. on the other hand,
its use as an
easy way to allow 'sample accurate' control -
into the 'next generation'
is host-specific, and doesn't have much to do with the API.
Question: do A2 and/or A3 actually subdivide a period
in order to
deliver automation data or not ?
depends on the plugin API in use. in some cases yes, in others no.
If yes, do they also try to deliver 'sample
accurate' control values
in case these are real-time (from GUI widgets, MIDI, OSC) ?
data from control surfaces of any kind is quantized to the blocksize, and
when delivered during non-automation playback, delayed by up to one block.
automation data is recorded by sampling plugin control port values. the
only thing that can generate sub-block size resolution is GUI-based (or
similar) editing of the data.
1:08 a.m.
On Sat, 2012-05-26 at 16:22 -0400, Paul Davis wrote:
[...]
now, we can sit here and listen to you and dave
mudslinging about the
sanity of this or that. you can, if you want, insist that everyone who
designed AU and RTAS and VST also don't understand audio programming.
maybe you're even right about that (though it seems less likely). but
that doesn't actually solve anything or move anything forward.
I mud-sling only about *behaviour* which is actively harmful to the
adoption of open audio technology. I am indeed publicly critical of
such behaviour, and proudly so. Unhelpful fallacious FUDdey nonsense
will be exposed for what it is.
The actual technical points, however, I largely agree with, and would be
more than happy to incorporate said functionality into this malleable
spec that was specifically designed to be able to improve this way (not
that I or anyone actually has to in order for it to happen). This has
been known and idly discussed and experimented with for years, nothing
new, it just has never proved that important (if nobody needs it, now,
it is not important, period).
This buffer size stuff is *not* an error. It is simply work that is not
done yet. That is all.
I, as it happens, do personally need it now. Specifically, I just need
a maximum buffer size (to allocate auxiliary buffers). Knowing there is
more involved, however, I asked this list for input on what other
restrictions are necessary.
Input I am still actively looking for. So I can do it. Like, really
actually do it. Myself. Right now.
For example, is there any reasonable case where a plugin needs to
specify an (arbitrary? minimum?) block size, or can the host's value
provided at instantiation time (possibly required to be a power of 2)
always be accommodated? If an arbitrary/minimum size must be specified,
statically (highly preferable) or dynamically?
Thanks,
-dr
1:47 a.m.
On May 26, 2012 10:02:32 AM Fons Adriaensen wrote:
On Fri, May 25, 2012 at 10:43:36PM -0400, David
Robillard wrote:
I added variable run-length processing to MusE.
It can break a period into single frame runs if necessary.
If you have a project with a lot of plugins, or power-hungry ones,
where you are forced to use large Jack buffer sizes to make
everything work reasonably well, vari-runs work well.
One reason is because only those plugins whose controls changed will
consume time during your cycle, versus attempting to run Jack at
very low periods.
Thus we can run Jack at 2048 period size and operate a dssi filter or
oscillator control for example, and the result is so smooooth sounding,
responsive, and near-instantaneous that you'd swear you were running 32
period size, because the the runs are broken up into a reasonable near-frame
resolution.
What about encoder and decoder plugins? Yer switching matrices etc.
Their precise control rates and timestamps must be preserved right?
I guess if LV2 allows hi-speed control rates, it would cure the issue with
some ladspa audio ports being used for hi-speed control, with no way to
know that fact.
So with MusE then, it's /possible/ to have hi-speed control port rates.
Although, I don't know of any ladspa plugin that uses it for say,
encoder/decoder switching matrices, they all use audio ports for that.
Probably more efficient to use audio ports, I suppose.
Caveat: Dssi-vst no likey!
I admit, we needed a special hack to turn off vari-runs for dssi-vst plugins.
But other than the entire dssi-vst, I have not found a ladspa/dssi plugin
or synth yet that doesn't like it. Can you think of any that I might want
to reconsider and test?
Also, we're not out of our woods yet, it was a fun experiment, but we still
need to provide that kind of resolution on our playback, which I admit may
ultimately require fixed run lengths (but still multiple per period), for
interpolated floating point controls (not integer), because if even one
single control is currently on a slope between two points we don't have
much choice but to do the whole plugin period in adjustable fixed
run-length segments.
(We do provide a 'Minimum Control Period' user setting as a lower limit
for such and other uses.)
Whaddya you guys think?
Or did I actually just provide arguments /against/ vari-runs? Ha ha...
Cheers. Tim.
I am making an LV2 extension for accessing and/or
restricting the buffer
size. This is straightforward, but I need to know just what
restrictions are actually needed by various sorts of DSP.
The sort of thing we're looking for here is "buffer size is always at
least 123 frames" or "buffer size is always a power of 2" or "buffer
size is always a multiple of 123".
I know "multiple of a power of two" is needed for convolution. Not sure
what else...
It's not really *needed*, an FFT-based process can be designed to
work with arbitrary and variable buffer sizes provided
* this is known at init time,
* and the user accepts the additional latency.
'Power of 2' in practice means 'a power of two that is not
too small to make things inefficient'. So the plugin should
have the option to require a minimum size as well, e.g. 64.
But I doubt very much if your extension is going to have much
impact. I expect most hosts to simply ignore it, as implementing
it could be quite invasive. Imagine a host having a chain a plugins
in e.g. a channel strip. If one of them requires this extension
and the host relies on variable buffer size to provide 'sample
accurate' control of the others, then the host is required to
implement extra buffering for that plugin. Probably no-go, some
hosts even refuse plugins that don't work 'in place', even if
the solution in that case is much simpler.
The following is part of a report on LV2 I was asked to
write some months ago:
======
Audio processing (on PCs) is usually done in blocks of frames,
so all components are designed to work that way, There are at
least 4 possible ways the block size of a module or plugin
could be defined:
1. Fixed at compile time.
2. Fixed at instantiation, power of 2 [*]
3. Fixed at instantiation, any (reasonable) value.
4. Variable - can be different in each process() call.
[*] Or a similar restriction, e.g. either 2^N or 3*2^N.
For example some FFT based processes could be designed to
allow a block size of 3*2^N, or even 5*2^N, while making
it accept any size would be out of the question in many
cases.
Clearly (1) is undesirable, and it will not be considered
further. Some algorithms require (2) in order to meet their
specs. They could be made to work with any block size, but
this usually means extra latency, or some other form of loss
of performance.
Almost all more sophisticated audio processing algorithms can
be implemented more easily and usually more efficiently if (3)
is guaranteed, even if the block size itself plays no role in
their definition and does not affect their output in any way.
This is in particular true for algorithms that partially run
using some internal block size (determined by e.g. the sample
rate and some combination of parameters), or those that have
to manage a long history or complex state as part of their
operation.
It is almost always possible to allow (4), the exceptions are
the cases mentioned above that require (2). But an algorithm
that should perform well will have to be designed so that the
output it produces does not in any way depend on the actual
block sizes used at run time.
This includes ignoring any attempt to e.g. control the rate of
change of its internal parameters by a caller playing with the
block size for each process() call. The idea that one could
achieve 'sample accurate' control of an audio process in this
way is a either a fallacy or irrelevant, depending on the case
and how you look at it.
All this means that there is *no reason* to ever require (4),
of a plugin or module, and no serious plugin standard or
modular audio application should ever do this.
Yet that is precisely what LV2 does. While it is designed
to be flexible and extensible, the one and only part of its
specs that is fixed but shouldn't have been gets it wrong.
And allowing an extension to override this is not going to
work, because such an extension can be very invasive to a
host. If a host, e.g. Ardour, is from the start designed to
exploit the freedom of (4), be it for the wrong reasons, it
is never going to adopt an extension that would cripple major
parts of its functionality, e.g. automation, or force them
to be redesigned. Even less so if the plugin system does not
provide an alternative way, guaranteed to work with any plugin,
to obtain the same result.
This is the really sad aspect of the situation: LV2 requires
so little of a plugin that in order to make things like e.g.
automation possible it must allow hacks such as playing with
the block size of each call. So hosts will do this, and that
wart will never go away.
Things should have been the other way round: allowing hosts
to use variable block sizes should have been the extenstion.
Of course nobody would ever use it, because in a well designed
system or application it is not necessary (not even for the
things e.g. Ardour uses it for).
2:48 a.m.
On Sat, 2012-05-26 at 22:02 +0000, Fons Adriaensen wrote:
They most certainly are, but here is simply not there. They are not in
any way similar, the culture, organization, requirements, and
methodologies are entirely different.
That does have some advantages. So does this. What you do there could
never work here, and vice-versa.
(Much like in politics) it's helpful to imagine yourself *not* in the
position of power. In LV2, for example, the fantasy monolithic scenario
is not *you* dictating everything, it's *me* dictating everything, i.e.
not giving a damn what you have to say because *I* drive this bus and to
hell with you. Doesn't sound so great now, does it? :)
I, in fact, *do* drive this bus, and (having learned a thing or two) am
slowly moving the project to *appear* as more of a monolithic effort
because it helps with adoption. It's just an illusion, though. I am
toning down the decentralized talk and moving more to a role of
benevolent dictator on purpose, but I'm only the benevolent dictator
because I did all the damned work. The API itself always has been, and
always will be, extensible by anyone.
Failing to learn from the past would be trying to have this mailing list
come to a universal agreement on everything, and actually expecting a
project to work out that can't be released until that happens.
This increasingly ironic thread is a perfect example of why.
*That* lesson is the core thing LV2 was designed to work around, and it
worked beautifully.
LV2 was designed to be feature compatible with LADSPA with the
additional requirement that improvements be able to be added on as they
get figured out, and people actually interested in solving them (i.e.
not just bitching on l-a-d) have the ability to do so. Many have. This
particular issue is no different.
That it can and will be solved now, years after all sorts of fancy new
plugins have been in production, is all the proof you need that the
requisite lesson has been learned, and applied.
The problem here is conflating all kinds of orthogonal issues. Being
unimportant to the problem at hand, and categorization ("synthesis" vs
not) not being useful anyway, I will ignore them.
No, you clearly wouldn't refuse it, but you could at least attempt to do
a competent job. Since you didn't, I in turn clearly am not going to
pull any punches:
Did they pay for FUD? Because it sounds to me like what is a relatively
straightforward problem they could have easily overcome if they needed
to was hyped up as a fundamental flaw, and quite possibly someone didn't
invest in open technology because of it. Because of some stupid trivial
bit of missing API that could be added in a day.
Anyone with an even basic grasp of how LV2 works would have included how
trivial of an addition this is, which would be actually useful.
Whatever they paid you, they could have paid for a solution. Hell, if
it was mentioned on lv2-dev they could have had that for free.
You got your money, so hey, you win. Free Software didn't, however, and
that's the part that saddens me.
Can I really be expected to not personally be thoroughly pissed off
here? I - and several others - spend so much time working on this
effort to provide a powerful free audio API, and not only do you have
nothing but endless public ignorant criticism, you get privately *paid*
to do so?
You make more money in a day in your fancy aerospace job than I do in an
entire month, and on the side you're taking contracts to write essays on
how shit the entire effort is? Not any actual solution, not even a peep
on any of the available channels to mention this issue which would have
surely been addressed, nothing helpful whatsoever, just project libel
for profit. Not only being literally paid to hurt, but the arguments
used don't stand up to even the a hint of public scrutiny.
Gee, thanks, Fons. Thanks a lot.
Enjoy your money.
-dr
On Sat, May 26, 2012 at 05:08:47PM -0400, David
Robillard wrote:
Yes, it does. But what you sweep under the carpet is that this would be
true of the hypothetical monolithic spec you allude to. That would just
give you a mountain of chicken & egg all at the same time.
The only difference between LV2 and that is that LV2 actually exists.
This is not at all a coincidence.
I am using a producer mentality. You are using a consumer mentality.
You don't like that a monolithic specification that does everything you
want has not been handed to you. I sympathize, I really do. I would
much rather just be writing and using the plugins and apps I want,
believe me.
The big thing you're missing - and this is important - is that this
process, right here, right now, IS the process of making that
specification you want.
Read that again. Several times, ideally.
*Nothing* emerges instantaneously, they get worked on. This one is
being worked on in the open, and as we work on it, we're making sure the
bits actually work along the way, and making sure things are sensible in
the one and only way it is possible to do so: implementing them.
If you like, and are not interested in influencing the specification,
you can simply pretend it has not been "released" yet. Of course, this
means you voluntarily forfeit your right to have any input on its
design. It's when you criticize this *design in process* without
actually trying to help that things turn sour, and I'd say rightly so.
In fact, I actively advocate NOT adopting LV2 some times, for
people/projects/companies that need an extremely conservative very long
term stable API. For them, it is not done yet. We're working on it.
For others, for whom either everything they need is done yet, or are
agile enough to keep up, it is done yet.
It's simply an open development process. The only difference is, you
know about it, and - lucky you! - you even get to participate and
influence it if you wish.
Here, notice a plugin not working in the host is
*inherent*. This is
why LV2 folks generally don't consider this a "problem" - it isn't
one.
If the host literally *can't* provide a given feature (e.g. fixed buffer
size) and the plugin *requires* it to function at all, well, clearly the
plugin can't work in that host. LV2 merely provides a mechanism to
express such things. The decision of whether to do them is inherent.
Understood, and not for the first time. But it leads to chicken/egg
situations. I'm not going to write an LV2 requiring N features if
there isn't a host providing them. And probably vice versa for host
authors. And that is why I'd prefer a more demanding core spec.
Dave, if there is anything I find contradictory in your argumentation
(and that is often why I react to it) it is that you seem to sweep
this point under the carpet, while in other cases you can be very
pragmatic. It is not my
job, or your job, to tell plugin and host authors what they
can and can not do. Ad hoc extensions are where innovation happens,
where solutions get sorted out *before* they become part of the main
spec. You can't sort them out without actually implementing them.
Software simply does not work that way. Ever.
Mmm, I'd like to disagree. Most open source software may be developed
in such conditions - but that is both a strength and a curse. Having
worked some ten years developing software for space and aviation I
can tell you things are different there. And that has some advantages. It will all
work out eventually. You've been around here long enough to
know everything ain't happening tomorrow regardless :)
Yes. But that's no reason to not learn from the past. Which is what
LV2 started out to do... This is a bit
difficult to decipher. You are saying you think
block-accurate controls at a given fixed minimum size are good enough?
For anything except synthesis, yes. Not only good enough, but actually
much easier to implement correctly.
However, making "synthesis modules" an
exception is kind of pointless
over-classification. Either a sample accurate control mechanism is
required, or it isn't. There is no hard distinction between "synthesis
modules" and other things anyway, it is not a useful (false) dichotomy.
If 'sample accurate' control would be the only difference between
'general purpose' and 'synthesis' modules I'd agree with you.
But it isn't. 'Polyphonic' is not the same as 'multichannel' for
one.
And synthesis modules may require real-time instantiation - no need
for that in case of an effect in a DAW. My conclusion is that the
two are fundamentally different. Which doesn't mean that a plugin
standard couldn't provide both. But probably not by ignoring the
differences. It's odd
to speak of this like there has been some active decision to
"allow" arbitrary nframes values. It's more accurate to say that
basically nothing has been said about this thus far. It is simply an
absence of well-defined restriction because nobody has defined them yet.
AFAIK, the 'core spec' requires plugins to accept a non-constant nframes:
"Thus the "control rate" is determined by the block size, which is
controlled by the host (and not necessarily constant)."
It'd have been solved and implemented long
long ago if it was brought up
in useful channels and not weird cloak and dagger reports...
Nothing cloak and dagger about it. If someone is prepared to pay me for
writing a report on something I do have strong opinions about (and very
probably because of that), should I refuse that ? It's a consultancy
job just like any other. And at least one other well-known developer
got the same.
2:58 a.m.
On Sat, 2012-05-26 at 19:47 -0400, Tim E. Real wrote:
I added variable run-length processing to MusE.
It can break a period into single frame runs if necessary.
[...]
What about encoder and decoder plugins? Yer switching
matrices etc.
Their precise control rates and timestamps must be preserved right?
I guess if LV2 allows hi-speed control rates, it would cure the issue with
some ladspa audio ports being used for hi-speed control, with no way to
know that fact.
LV2 does have CV ports, which are identical to audio ports but
semantically the values are control values.
For discrete controls, actually you can easily do this with a normal
event port, just as we do MIDI. There's just a lack of convention, or
standard event type to convey control values.
Once you go that far, though, you start questioning why you're using
different port buffers when it's easy to multiplex controls to a single
buffer, which is:
* More space efficient
* Easier to iterate over and process as time progresses (but harder for
some other things)
* Allows for dynamic controls (e.g. n-band EQs and such), a highly
desirable feature
* Faster than multiple run() calls due to decreased
function-call-via-a-pointer overhead
Whaddya you guys think?
Or did I actually just provide arguments /against/ vari-runs? Ha ha...
Basically you've provided arguments that the control rate not always
being the Jack buffer size is good. Few would argue that.
It's splitting the cycle on odd numbers that causes problems for certain
kinds of plugins. However it should be mentioned (despite suggestions
to the contrary) that if a solution to this was implemented, it is
trivial to make old plugins adapt to restricted buffer sizes. It's the
other way around that doesn't work.
Cheers,
-dr
1:16 p.m.
On Sat, May 26, 2012 at 08:48:07PM -0400, David Robillard wrote:
The full report is over 100 pages, and with what you've seen of
it you are not in any position to judge its merits or faults.
And it was certainly not all 'negative', on the contrary.
Nothing cloak
and dagger about it. If someone is prepared to pay me for
writing a report on something I do have strong opinions about (and very
probably because of that), should I refuse that ? It's a consultancy
job just like any other. And at least one other well-known developer
got the same.
No, you clearly wouldn't refuse it, but you could at least attempt to do
a competent job. Since you didn't, I in turn clearly am not going to
pull any punches: You got your money, so hey, you win. Free Software
didn't, however,
and that's the part that saddens me.
First, I wouldn't be so sure of that.
Second, you wrote:
I mud-sling only about *behaviour* which is actively
harmful to the
adoption of open audio technology.
So it's my *behaviour* that bothers you. So be it. I'm not going
to change my opinions in function of the advancement of Free
Software or whatever 'higher cause' or marketing campaign, ever.
I've done my part of advocacy for Linux Audio, and I'm still doing
that, by example and pointing out both its great features *and* its
shortcomings, not by keeping my mouth shut about the latter. If some
potential user asks me what is the state of audio plugins in Linux
I'll tell him/her that there are some great ones, but also that some
things are missing, and that 50% of what is available is pure crap.
Since I'm usually known or introduced to sich persons as 'the Linux
Audio nerd', that may come as a surprise to them, But they generally
appreciate the honesty.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
4:01 p.m.
On Sun, May 27, 2012 at 7:16 AM, Fons Adriaensen <fons(a)linuxaudio.org>wrote;wrote:
So it's my *behaviour* that bothers you. So be it. I'm not going
to change my opinions in function of the advancement of Free
Software or whatever 'higher cause' or marketing campaign, ever.
I've done my part of advocacy for Linux Audio, and I'm still doing
that, by example and pointing out both its great features *and* its
shortcomings, not by keeping my mouth shut about the latter. If some
potential user asks me what is the state of audio plugins in Linux
I'll tell him/her that there are some great ones, but also that some
things are missing, and that 50% of what is available is pure crap.
Since I'm usually known or introduced to sich persons as 'the Linux
Audio nerd', that may come as a surprise to them, But they generally
appreciate the honesty.
i don't see any problem with this, and i'm guessing that david doesn't
either (though i shouldn't try to speak for him). i spend a lot of my time
trying to explain to people all the shortcomings in my own software out of
a similar need for honesty.
the problem is that this isn't "some potential user asking me what is the
state of audio plugin in linux". its the linux audio develop(ers|ment)
mailing list.
in this context, i think that criticizing a feature of a plugin API as
though it is fundamentally misconceived, and as if the people who came up
with the feature basically don't know what they are doing, when the same
feature is found in every other even moderately used plugin API just seems
... well, i don't know a good adjective for this. i concede that there is
room for a much more generalized discussion about whether plugin APIs or
DSP in general should avoid variable block sizes. that discussion should
draw on the variety of reasons why variable block sizes exist at all and
this needs to cover the large variety of different contexts in which plugin
APIs are used. it could be that the outcome of a focused discussion on this
would reach the clear conclusion that variable block size is a detrimental
feature in such an overwhelming way that it should never be part of such an
API. personally i find this highly unlikely, but i'm willing to concede
that its a possibility.
but instead, as i correctly though preemptively anticipated (can there be
such a thing as preemptive anticipation?) this discussion once again turned
into a very personally-driven spitting match in which david's personal
involvement and belief in LV2 and his committed work on it slams up against
a distaste for many aspects of LV2 that you've made clear for several
years. this type of clash isn't useful to anyone, and this is why i think
david gets so upset with them - that rather than there being discussion
that targets "how can this be done (right)?" there ends up being a tone of
"well, this is wrong, and that is wrong, and also this other thing is
wrong, and by the way the whole basic idea, conception and architecture is
wrong".
and maybe that is actually what you think of LV2, and if so you're not
alone in that assessment and its clearly OK. its just that having public
battles that superficially claim to be about a detail of the API but are
really just surrogates for "this whole thing is just misconceived" is
really pointless.
if thats not what you think of LV2 overall, then wouldn't it be more
helpful, when bringing up an issue like this, to put it in the context of
all plugin APIs? I don't trust the people in of the particular individual
tiny sub-niches of audio software (digidesign, windows, coreaudio, ALSA,
pulseaudio, linux, audiounits, vst, h/w DSP, consoles-on-linux,
audio-over-IP etc, etc,) to get things right. but i do think that if you
scan across the appropriate sub-niches (in this case, a half dozen or more
plugin APIs) and you find a consistent pattern, then it becomes much more
important to make the claim "this is the wrong thing to do" be backed up
with explanations of why the other instances of the same kind of thing also
got it wrong. perhaps the argument is "they all copied each other", and
perhaps that is the only possible argument. if so, its a weak one though i
will grant that it could be true.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
6:05 p.m.
On Sun, 2012-05-27 at 11:16 +0000, Fons Adriaensen wrote:
[...]
So it's my *behaviour* that bothers you. So be it.
I'm not going
to change my opinions in function of the advancement of Free
Software or whatever 'higher cause' or marketing campaign, ever.
I've done my part of advocacy for Linux Audio, and I'm still doing
that, by example and pointing out both its great features *and* its
shortcomings, not by keeping my mouth shut about the latter.
You could have done your part (i.e. pointed out those things in a useful
way/place), had the relevant issues addressed, and improved the Linux
Audio plugin situation.
You didn't, they weren't, and it wasn't.
That is all there is to it, and all I care to say about the matter.
-dr
7:19 p.m.
On Sun, 2012-05-27 at 10:01 -0400, Paul Davis wrote:
[...]
this type of clash isn't useful to anyone, and
this is why i think
david gets so upset with them - that rather than there being
discussion that targets "how can this be done (right)?" there ends up
being a tone of "well, this is wrong, and that is wrong, and also this
other thing is wrong, and by the way the whole basic idea, conception
and architecture is wrong".
and maybe that is actually what you think of LV2, and if so you're not
alone in that assessment and its clearly OK. its just that having
public battles that superficially claim to be about a detail of the
API but are really just surrogates for "this whole thing is just
misconceived" is really pointless.
Bingo.
Of course, pulling that crap on a mailing list isn't news. It kind of
goes with the territory.
However, doing it for pay is professionally dishonest. When you are
paid to report on something as an expert, you are supposed to set a
higher bar for yourself than mailing list trolls. While the buffer size
analysis is fine (as expected, since this is something the author
*actually* knows something about), the conclusions drawn about the API
itself are simply wrong and serve only to illustrate a fundamental lack
of understanding about the most basic principles of the thing, and even
contradicts itself several times in the few paragraphs I have seen (to
say nothing of how divorced from reality it is, as Paul has mentioned).
It also manages to blame aspects of almost every single piece of audio
software in existence on LV2, despite all of these things predating it
by decades (a pretty good indicator of how rational and objective this
work is). It is, in short, nonsense; a lunatic preaching his ego
stroking religion of anti-everything, not what a decent person would
expect of paid work from an "expert".
I am really just amazed that anyone would actually do something so
shameful and dishonest (and admittedly somewhat bitter that somebody
actually paid them to do so). I may be just as colourful a character on
this list, but never in a million years would I consider attaching my
name to paid "professional" work like this delusional drivel.
I'd write a rebuttal, but there's no point, and this has wasted enough
of my and everyone else's time.
That said, I don't like how one grumpy old fool gets to exploit the
balance fallacy here, so I will mention a few people who have actually
done their part and helped:
Steve Harris: Original LV2 idea, first real plugin set, etc.
Lars Luthman: Early pioneer, many experiments, lv2-c++-tools, UI and
event extensions, etc.
Nedko Arnaudov: Early pioneer, external UI extension, Zyn*, early work
on dynamic plugin control, etc
Krzysztof Foltman: Calf plugins (first fancy GUIs?), state extension,
several ad-hoc extensions (some making their way to core, more to
follow), lots of involvement in the early days
Stefano D'Angelo: dyn-manifest extension, NASPRO (bridging LADSPA, VST,
etc. to LV2), current experimental work on web UIs, etc.
Gabriel M. Beddingfield: URID extension, lots of useful feedback on the
mailing list
Leonard Ritter: State extension
... and probably others I have forgotten (apologies). This is only
people who have directly contributed to the spec; there are of course,
many more who have written plugins and hosts, packaged, contributed
usefully to the list and/or wiki, and so on. Equally important work,
apologies for not mentioning all your names, but they are too numerous
to list here.
Everyone one of them awesome, having directly contributed tangible
improvements, whose valuable efforts would have been missed.
Thankfully, they are the norm, not the exception.
The ever improving functionality, quality, and hundreds (thousands?) of
plugins didn't come from nowhere, it came from an active community of
people working together to get the job done.
Keep on keepin' on, laddies. Haters gonna hate.
-dr
8:59 p.m.
2012/5/26 Fons Adriaensen <fons(a)linuxaudio.org>rg>:
1. telling a plugin that at N frames from the current
position the
 parameter P should have value V, and
2. doing the same, while also requiring that the plugin outputs
 N frames at that time.
My argumentation is that doing (2) is a bad idea, and even more so if
you consider that (1) is a silly thing to in the first place, at least
for the usual audio processing tasks such as EQ, dynamics, and most
effects. The exception is synthesis modules of course, but those should
have a dedicated mechanism for it anyway, or accept audio rate controls.
The full report included a detailed anaysis of why (1) is a bad
idea in most cases (with the exception of synthesis modules). It
is because it makes it almost impossible for the plugin code to
compute good parameter trajectories. A well-designed EQ, effect,
compressor, etc. should actually *ignore* attempts to control its
internals in such a way. So there was never any need to allow
arbitrary nframes values. The correct thing to do would be to
remove that from the core spec, and provide an extension for
finer-grained 'sample accurate' control.
If I understand correctly an implication would be that you get uniform
sampling of parameter signals with control rate = sample rate /
nframes. I assume that computing parameter trajectories basically
means interpolating, and that inevitably introduces some latency if
you want the audio and control streams to be in sync (e.g., nframes
for linear interpolation, 2 * nframes for splines, etc.).
In practice, this would mean that we might want to have two flavours
of each and every algorithm: one for offline use that keeps audio and
controls in sync (and adds latency) and another for live use that
doesn't keep the sync (since in practice a couple of blocks of unsync
between audio and control is unnoticeable when controlling through
GUIs, MIDI, OSC or similar).
The way I usually coped with parameter trajectories (not really) was
to add things like leaky integrators and use filtered parameters at
the audio rate (mostly for smoothing to avoid clicks and noises), but
more orthodox resampling equivalents (whether uniform or not) have at
least performance advantages, or are at least alternatives worth
considering.
In practical terms, especially w.r.t. LV2, there may be a third way:
let the host pass a limited number of future parameter samples at each
run() (could be negotiated at instantiation time), so that the plugin
doesn't have to add latency to the audio streams in any case. Would be
only supported by "offline hosts". If the block sizes are variable,
future block sizes should be passed as well (argh?). But I don't know
if this really makes sense or has downsides... ideas, folks?
Stefano
9:56 p.m.
On Sun, May 27, 2012 at 01:19:36PM -0400, David Robillard wrote:
However, doing it for pay is professionally dishonest.
When you are
paid to report on something as an expert, you are supposed to set a
higher bar for yourself than mailing list trolls. While the buffer size
analysis is fine (as expected, since this is something the author
*actually* knows something about), the conclusions drawn about the API
itself are simply wrong and serve only to illustrate a fundamental lack
of understanding about the most basic principles of the thing,
The buffer size issue is the only one mentioned in the excerpt
from my report that I quoted. Everything else, good or bad, about
LV2 that I may have written or not written is just the product of
your imagination. Which I can assure yout gets its mostly wrong -
my overall conclusion was not negative at all.
Judging from the (quite limited) feedback I got on my report,
what you present as an inevitable quality of the whole LV2
project - things are 'designed' iteratively and as the result
of a lot of social interaction - is what scared off my customer.
This is not what you need if you want to launch a product in a
determined time and commit yourself to support it. An idealist
may do that, a company wanting to make a profit and survive for
some time won't.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
10:01 p.m.
On Sun, 2012-05-27 at 21:59 +0300, Stefano D'Angelo wrote:
[...]
In practical terms, especially w.r.t. LV2, there may
be a third way:
let the host pass a limited number of future parameter samples at each
run() (could be negotiated at instantiation time), so that the plugin
doesn't have to add latency to the audio streams in any case. Would be
only supported by "offline hosts". If the block sizes are variable,
future block sizes should be passed as well (argh?). But I don't know
if this really makes sense or has downsides... ideas, folks?
Or just send time-stamped ramped controls (value now is 1.2, value at
some point in the future past this block is 3.9, and so on). This could
be done even online in the case of automated parameters, though of
course not things being actively twiddled by the user live.
I suppose the host could just indicate whether it can send future
control values, so plugins that interpolate can avoid adding control
latency when they don't need to. This maps nicely to a feature, though
it'd be a questionable decision to require it and make a plugin that
doesn't work at all without receiving parameters from the future, since
it'd be impossible to use live. One could argue the host always has to
do the latency part if necessary, I guess.
In terms of API, these can thankfully be tacked completely independently
(even though they impact each other in terms of what plugins actually do
with them). Block size restriction stuff is one thing, a way of getting
more expressive control values into a plugin is another. The former is
easier and more pressing.
-dr
10:17 p.m.
2012/5/27 David Robillard <d(a)drobilla.net>et>:
On Sun, 2012-05-27 at 21:59 +0300, Stefano
D'Angelo wrote:
[...]
Sure, it's all sound and nice. I was only wondering whether there is a
smarter way to do things.
Stefano
In practical terms, especially w.r.t. LV2, there
may be a third way:
let the host pass a limited number of future parameter samples at each
run() (could be negotiated at instantiation time), so that the plugin
doesn't have to add latency to the audio streams in any case. Would be
only supported by "offline hosts". If the block sizes are variable,
future block sizes should be passed as well (argh?). But I don't know
if this really makes sense or has downsides... ideas, folks?
Or just send time-stamped ramped controls (value now is 1.2, value at
some point in the future past this block is 3.9, and so on). Â This could
be done even online in the case of automated parameters, though of
course not things being actively twiddled by the user live.
I suppose the host could just indicate whether it can send future
control values, so plugins that interpolate can avoid adding control
latency when they don't need to. Â This maps nicely to a feature, though
it'd be a questionable decision to require it and make a plugin that
doesn't work at all without receiving parameters from the future, since
it'd be impossible to use live. Â One could argue the host always has to
do the latency part if necessary, I guess.
In terms of API, these can thankfully be tacked completely independently
(even though they impact each other in terms of what plugins actually do
with them). Â Block size restriction stuff is one thing, a way of getting
more expressive control values into a plugin is another. Â The former is
easier and more pressing.
10:26 p.m.
On Sun, May 27, 2012 at 09:59:29PM +0300, Stefano D'Angelo wrote:
If I understand correctly an implication would be that
you get uniform
sampling of parameter signals with control rate = sample rate /
nframes. I assume that computing parameter trajectories basically
means interpolating, and that inevitably introduces some latency if
you want the audio and control streams to be in sync (e.g., nframes
for linear interpolation, 2 * nframes for splines, etc.).
Indeed, interpolation introcudes delay. If control values are
delivered at a constant rate it's quite easy for a host to
compensate for this. What is needed in that case is that the
delay is *defined*. But there is anothor issue you touch on.
If automation data is the result of 'real time' input, then
that delay doesn't matter - the user, while creating the data
using some GUI widget or a HW controller, will compensate for
it because he/she simply has no other choice. Then, if the host
delivers the data at the same time (w.r.t. the audio) as when
it was recorded, then the plugin will do the same things and
the user will hear the result he/she wanted.
If OTOH the automation data is created by e.g. drawing some
curves in a graphical interface (with probaby the audio wave-
form as a reference) then you don't want that delay, and the
host shoudl compensate for this.
In most cases (general audio effects in a DAW) it just doesn't
matter much. The only thing (apart from synthesis/instruments)
that need accurate timing is the 'mute' function when used to
remove unwanted noises etc. But that is usually handled by the
host itself, and anyway automation is not the best way to deal
with such things - editing is.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
11 p.m.
2012/5/27 Fons Adriaensen <fons(a)linuxaudio.org>rg>:
On Sun, May 27, 2012 at 09:59:29PM +0300, Stefano
D'Angelo wrote:
So far so good, but in practical terms how could a plugin API allow
all use cases without requiring the plugin writer to do twice the
work? The only solution I can think of is "getting future values" in
case a host can provide them, but maybe there is a better way?
Stefano
If I understand correctly an implication would be
that you get uniform
sampling of parameter signals with control rate = sample rate /
nframes. I assume that computing parameter trajectories basically
means interpolating, and that inevitably introduces some latency if
you want the audio and control streams to be in sync (e.g., nframes
for linear interpolation, 2 * nframes for splines, etc.).
Indeed, interpolation introcudes delay. If control values are
delivered at a constant rate it's quite easy for a host to
compensate for this. What is needed in that case is that the
delay is *defined*. But there is anothor issue you touch on.
If automation data is the result of 'real time' input, then
that delay doesn't matter - the user, while creating the data
using some GUI widget or a HW controller, will compensate for
it because he/she simply has no other choice. Then, if the host
delivers the data at the same time (w.r.t. the audio) as when
it was recorded, then the plugin will do the same things and
the user will hear the result he/she wanted.
If OTOH the automation data is created by e.g. drawing some
curves in a graphical interface (with probaby the audio wave-
form as a reference) then you don't want that delay, and the
host shoudl compensate for this.
In most cases (general audio effects in a DAW) it just doesn't
matter much. The only thing (apart from synthesis/instruments)
that need accurate timing is the 'mute' function when used to
remove unwanted noises etc. But that is usually handled by the
host itself, and anyway automation is not the best way to deal
with such things - editing is.
11:33 p.m.
On Mon, May 28, 2012 at 12:00:36AM +0300, Stefano D'Angelo wrote:
So far so good, but in practical terms how could a
plugin API allow
all use cases without requiring the plugin writer to do twice the
work? The only solution I can think of is "getting future values" in
case a host can provide them, but maybe there is a better way?
If you really want 'sample accurate' control for bandlimited
controls then the host must provide future values. Which is not
more difficult than providing latency compensation for audio
signals, at least not if the control rate is fixed. It's not
at all an issue for the plugin code and you certainly don't
need two versions of it.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
28 May
28 May
12:33 a.m.
On Sun, 2012-05-27 at 19:56 +0000, Fons Adriaensen wrote:
On Sun, May 27, 2012 at 01:19:36PM -0400, David
Robillard wrote:
Forgive me if I can only extrapolate from what I have seen: a constant
stream of FUD and endless criticism.
I am at least glad - if surprised - to hear there was anything else in
there.
However, doing it for pay is professionally
dishonest. When you are
paid to report on something as an expert, you are supposed to set a
higher bar for yourself than mailing list trolls. While the buffer size
analysis is fine (as expected, since this is something the author
*actually* knows something about), the conclusions drawn about the API
itself are simply wrong and serve only to illustrate a fundamental lack
of understanding about the most basic principles of the thing,
The buffer size issue is the only one mentioned in the excerpt
from my report that I quoted. Everything else, good or bad, about
LV2 that I may have written or not written is just the product of
your imagination. Which I can assure yout gets its mostly wrong -
my overall conclusion was not negative at all. Judging from the (quite limited) feedback I got on my
report,
what you present as an inevitable quality of the whole LV2
project - things are 'designed' iteratively and as the result
of a lot of social interaction - is what scared off my customer.
That's what FUD is meant to do. It worked.
This is not what you need if you want to launch a
product in a
determined time and commit yourself to support it. An idealist
may do that, a company wanting to make a profit and survive for
some time won't.
What you need is an API that actually does what you need. Period.
You argue as if it would be better if it was *guaranteed* that it can
*not* do what the product needs.
You're comparing *real* technology with hypothetical perfect *fantasies*
that do not exist - could not ever exist - and claim that the *real*
thing is the victim of blind idealism.
Well, what can I say? Out here in actual reality it's pretty clear
what's pragmatic and what's blind idealism.
/me pushes a key on his keyboard
I love the sound of code in the morning. Sounds like victory.
-dr
1:58 a.m.
2012/5/28 Fons Adriaensen <fons(a)linuxaudio.org>rg>:
On Mon, May 28, 2012 at 12:00:36AM +0300, Stefano
D'Angelo wrote:
(Maybe just for the record) I meant one version for sample accurate
control (whether by accessing future values or by the host providing
latency compensation) and another for live usage (virtually
latency-free, but less accurate w.r.t. parameters).
Stefano
So far so good, but in practical terms how could
a plugin API allow
all use cases without requiring the plugin writer to do twice the
work? The only solution I can think of is "getting future values" in
case a host can provide them, but maybe there is a better way?
If you really want 'sample accurate' control for bandlimited
controls then the host must provide future values. Which is not
more difficult than providing latency compensation for audio
signals, at least not if the control rate is fixed. It's not
at all an issue for the plugin code and you certainly don't
need two versions of it.
12:52 p.m.
On Sun, May 27, 2012 at 06:33:25PM -0400, David Robillard wrote:
They are not that stupid. And they didn't need me to find
out the things they did not like: extensions being created
and then deprecated, extensions that 'exist' on some web
page but are implemented nowhere, tutorials that start with
saying that everything has changed and they should be ignored,
etc.
At one time, pointing out that the Catholic church maybe had
made some mistakes was 'blasphemy' and you could end up on
the stake. You seem to have a similar definition of 'fud' -
any statement that does not glorify the excellence of LV2
is immediately classified as such. Nothing new really.
Which they already have. AFAIK it does all they need quite well.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
Judging from
the (quite limited) feedback I got on my report,
what you present as an inevitable quality of the whole LV2
project - things are 'designed' iteratively and as the result
of a lot of social interaction - is what scared off my customer.
That's what FUD is meant to do. It worked. This is not
what you need if you want to launch a product in a
determined time and commit yourself to support it. An idealist
may do that, a company wanting to make a profit and survive for
some time won't.
What you need is an API that actually does what you need. Period 
1:54 p.m.
On 05/28/2012 12:52 PM, Fons Adriaensen wrote:
They are not that stupid. And they didn't need me
to find
out the things they did not like: extensions being created
and then deprecated, extensions that 'exist' on some web
page but are implemented nowhere, tutorials that start with
saying that everything has changed and they should be ignored,
etc.
LV2 may suffer from those issues, all hard to avoid for a free software
project that moved slow at times, due to a lack of total qualified
developer time.
If nobody takes control, it's yucky anarchism. If someone does step up,
he becomes a "dictator" and there will be people who call him a fool for
taking choices different from those they would have taken, but didn't
and likely won't.
It's like you are asking again and again: "David and Co, why didn't you
just publish a complete specification at once, one all important plugin
and host authors would agree with, magically. As you did not do this,
why don't you throw away what you have to do it now?"
"Oh, and David, why don't you at least mention all the negatives,
every-time you mention the project that your investing so many hours
(weeks, months ...) in?"
As I understand it, what you are insisting on would require a change to
the core of LV2 and invalidate everything out there. Which would lead to
deprecation and outdated documentation on a much larger scale. All that
to gain something the creators of several other standards apparently
didn't consider important or at all.
I think it is the impression that you keep measuring LV2 against an
hypothetic solution that irritates not just David. Especially if it
sounds like that hypothetic solution has to somehow appear at once,
without the friction of experiments and visible social interaction. "LV2
Creators, why aren't you a company and invest a few man-years , working
in secret, to then publish a *complete* spec and reference
implementation at once?"
It would be nice, if you could limit your further input in LV2 matters,
if any, to things that LV2 developers can act on, realistically.
--
Thorsten Wilms
thorwil's design for free software:
http://thorwil.wordpress.com/
2:17 p.m.
On Mon, May 28, 2012 at 02:58:55AM +0300, Stefano D'Angelo wrote:
(Maybe just for the record) I meant one version for
sample accurate
control (whether by accessing future values or by the host providing
latency compensation) and another for live usage (virtually
latency-free, but less accurate w.r.t. parameters).
How much filtering or interpolation a plugin needs to do in
order to avoid clicks, zipper noise or worse (e.g. instability)
depends on what the plugin does. And on how it does it: for most
functions you can have different algorithms that are absolutely
equivalent as long as the parameters are fixed but that behave
very differently when they change.
If there is any difference between the two scenarios you mention
it will be in the other sense: if the control data is played back
from storage it can be 'smoothed' before.
If the control input is 'live', that is from GUI events, MIDI,
OSC or HID it will always need some filtering. The question then
is if this is responsability of the host or of the plugin, or
both.
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
5:05 p.m.
2012/5/28 Fons Adriaensen <fons(a)linuxaudio.org>rg>:
On Mon, May 28, 2012 at 02:58:55AM +0300, Stefano
D'Angelo wrote:
IMO it's easily said: if control rate < audio rate it's plugin's
responsibility, otherwise the host feeds upsampled/filtered control
signals at audio rate to the plugin and all problems evaporate... but
then the plugin should "ask the host" for guarantees on how the
smoothing is performed to both avoid useless performance penalties and
to avoid noises/instabilities.
Stefano
(Maybe just for the record) I meant one version
for sample accurate
control (whether by accessing future values or by the host providing
latency compensation) and another for live usage (virtually
latency-free, but less accurate w.r.t. parameters).
How much filtering or interpolation a plugin needs to do in
order to avoid clicks, zipper noise or worse (e.g. instability)
depends on what the plugin does. And on how it does it: for most
functions you can have different algorithms that are absolutely
equivalent as long as the parameters are fixed but that behave
very differently when they change.
If there is any difference between the two scenarios you mention
it will be in the other sense: if the control data is played back
from storage it can be 'smoothed' before.
If the control input is 'live', that is from GUI events, MIDI,
OSC or HID it will always need some filtering. The question then
is if this is responsability of the host or of the plugin, or
both.
7:01 p.m.
On Mon, May 28, 2012 at 06:05:20PM +0300, Stefano D'Angelo wrote:
IMO it's easily said: if control rate < audio
rate it's plugin's
responsibility, otherwise the host feeds upsampled/filtered control
signals at audio rate to the plugin and all problems evaporate...
The don't evaporate, they explode.
Take a filter plugin. Calculating the actual filter coefficients
from the 'user' parameters (frequency, gain, etc..) can easily be
10..1000 times more complex than actually using those coefficients
to process one sample. So you really don't want to do that at the
audio sample rate. A normal EQ as used in a DAW could do this once
every 10 ms or so and use simple linear interpolation in between.
And even that doesn't mean the filter should accept arbitrary
'user' parameters on each such update - these have to be rate
limited first as well.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
7:30 p.m.
On Mon, 2012-05-28 at 17:01 +0000, Fons Adriaensen wrote:
On Mon, May 28, 2012 at 06:05:20PM +0300, Stefano
D'Angelo wrote:
This is also an argument for not using signals for controls at all.
It's better in many cases to be explicitly told when the control value
changes.
That said, the comparison used to fake it isn't that expensive, but it's
more expensive than nothing (and for other port types, it is
significantly expensive)
All signs lately have been pointing to control signals sucking (except
when you actually want audio rate CV). Events are better. The details
of how to actually do that right are still unknown.
-dr
IMO it's easily said: if control rate <
audio rate it's plugin's
responsibility, otherwise the host feeds upsampled/filtered control
signals at audio rate to the plugin and all problems evaporate...
The don't evaporate, they explode.
Take a filter plugin. Calculating the actual filter coefficients
from the 'user' parameters (frequency, gain, etc..) can easily be
10..1000 times more complex than actually using those coefficients
to process one sample. So you really don't want to do that at the
audio sample rate. A normal EQ as used in a DAW could do this once
every 10 ms or so and use simple linear interpolation in between.
And even that doesn't mean the filter should accept arbitrary
'user' parameters on each such update - these have to be rate
limited first as well.
9:48 p.m.
On Mon, May 28, 2012 at 01:30:37PM -0400, David Robillard wrote:
Assuming that by 'signals' you mean audio rate controls, yes.
Again, the exception is 'synthesis' modules. You'd expect a
VCA in a modular synth to do what the ADSR tells it to do,
even if that results in artefacts. But you wouldn't want a
fader in a DAW (essentially the same thing if implemented as
a plugin) to do the same.
Take a filter
plugin. Calculating the actual filter coefficients
from the 'user' parameters (frequency, gain, etc..) can easily be
10..1000 times more complex than actually using those coefficients
to process one sample. So you really don't want to do that at the
audio sample rate. A normal EQ as used in a DAW could do this once
every 10 ms or so and use simple linear interpolation in between.
And even that doesn't mean the filter should accept arbitrary
'user' parameters on each such update - these have to be rate
limited first as well.
This is also an argument for not using signals for controls at all. It's better in many cases to be explicitly told
when the control value
changes.
Yes, but if that 'when' is not a regular period, and assuming you
want to actually reach the value, there's little you can do excecpt
linearly interpolate from the previous value. This makes the control
value continuous, but not its derivative(s). And that can be more
than enought to generate clicks, zipper noise, etc. Not only you
don't know what the next value will be, you don't even know when
it will arrive. It could be one millisecond later or one minute.
So the only thing you can do to have click and zipper-free output
is to apply enough filtering to guarantee this, whatever the input.
Which means ignoring most of the 'detail' in the input. Which in
turn means it's futile to provide such input in the first place.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
11:08 p.m.
On Mon, 2012-05-28 at 19:48 +0000, Fons Adriaensen wrote:
Well, in the event case we are *not* regularly sampling a single value
(which is a signal), we are describing a parameter change over time. If
it didn't change this time, it's the same as last time, but of course:
On Mon, May 28, 2012 at 01:30:37PM -0400, David
Robillard wrote:
Assuming that by 'signals' you mean audio rate controls, yes.
Again, the exception is 'synthesis' modules. You'd expect a
VCA in a modular synth to do what the ADSR tells it to do,
even if that results in artefacts. But you wouldn't want a
fader in a DAW (essentially the same thing if implemented as
a plugin) to do the same.
Sorry, that term isn't clear. By 'signals' I meant basically a single
numeric value (float), whatever the rate.
As opposed to an 'event' (also an ambiguous term) which actually says
"hey, this control changed to 4.0 now" when it changes - and only when
it changes. An 'event' could also have ramp information for
interpolation and whatever else.
Take a
filter plugin. Calculating the actual filter coefficients
from the 'user' parameters (frequency, gain, etc..) can easily be
10..1000 times more complex than actually using those coefficients
to process one sample. So you really don't want to do that at the
audio sample rate. A normal EQ as used in a DAW could do this once
every 10 ms or so and use simple linear interpolation in between.
And even that doesn't mean the filter should accept arbitrary
'user' parameters on each such update - these have to be rate
limited first as well.
This is also an argument for not using signals for controls at all. It's
better in many cases to be explicitly told when the control value
changes.
Yes, but if that 'when' is not a regular period , and assuming you
want to actually reach the value, there's little you can do excecpt
linearly interpolate from the previous value. This makes the control
value continuous, but not its derivative(s). And that can be more
than enought to generate clicks, zipper noise, etc. Not only you
don't know what the next value will be, you don't even know when
it will arrive. It could be one millisecond later or one minute.
So the only thing you can do to have click and zipper-free output
is to apply enough filtering to guarantee this, whatever the input.
Which means ignoring most of the 'detail' in the input. Which in
turn means it's futile to provide such input in the first place.
(The following is my perspective on the issue with respect to the plugin
interface, not a rebuttal: I agree completely)
Right, you need not only 'now' information in the event, but 'future'
information in that event to make it work. I see these issues as more
evidence that a single value just isn't good enough, since getting the
requisite information when the control itself is just a single float
involves mandated latency (i.e. the value this cycle is actually for the
future, and the value now is actually one that came earlier). With the
right rules, that can works, but it's screwy: the control value at frame
32 doesn't actually correspond to frame 32, the 'synchronous' feel of
run() has been ruined, and more importantly, sample accurate offline
rendering just breaks outright (unless you start tacking on a bunch of
additional crap to work around that with special cases).
I think the same requirements can be better met by moving past a single
float. Or: potentially flexible buffer size is not the error in LV2;
the use of a single float for controls is. The buffer size feature is
mostly orthogonal and simple to add; ControlPort is the thing that
probably shouldn't have been inherited from LADSPA at all.
To reframe slightly what you've said: in order for interpolation to work
correctly the value must be known for 'now', and at 'now plus 1
block' (see below for more on 'block'). This is basically the eventey
equivalent to saying you need a "regular period", but slightly different
to a sampled signal for a control. Most notably, the value described
now indeed represents *now*.
Of course, in some cases (like the ones you are thinking of, I believe)
you may not be able to actually affect the sample at t=32 with the value
at t=32; the interpolation or other factors might add some additional
hidden control latency by necessity. This is fine. The important
distinction is that what the host is describing is a single synchronous
stream of control and audio, where t=32 is t=32 for both control and
audio.
The big tangible difference is that the latency is *optional*. With
this scheme, you could e.g. write a trivial amplifier plugin that simply
multiplies by the gain control, and there would be no additional latency
whatsoever. The host can describe a step-wise function and have it
actually applied to the audio (in whatever way) without latency.
This is crucial if you want to build stuff out of lower-level plugins in
modular synths and such (but is not limited to synthesis), a case where
a "float value with guaranteed control latency" scheme completely fails.
(There is the question of what to do in live situations where future
values do not exist, but there are only two possible things to do, and
the solution is obvious, so I will ignore that and stay on topic)
Relating back to buffer size, there is one time parameter here: how far
into the future the plugin must know controls in advance (L).
This raises a big question: ignoring convolution and such, is that value
actually related to the audio block size at all? If you have a
description of a continuous parameter where you know values at least L
frames in advance (for sufficiently large L), you can interpolate that
parameter smoothly, in whatever way is appropriate for the plugin.
It seems the desire for a restricted block size (in cases where it is
not otherwise necessary) is actually just a consequence of the control
description (i.e. a single float) sucking.
I think the actual problem is inadequate controls, and fixing that seems
like a dramatically better solution than restricting the block size and
mandating latency to work around it. The latter is more of a kludge
than an ideal solution and has many downsides; not the least of which
being that it's dramatically less feasible. The only thing every person
in this thread has agreed on is that restricting the buffer size (to
make controls work properly) is a massive difficulty for hosts. As far
as I can tell, we don't have to.
-dr
29 May
29 May
12:40 a.m.
On Mon, May 28, 2012 at 05:08:02PM -0400, David Robillard wrote:
Dave, thanks for this reply. Reading your complete post, there's
a lot to discuss - more than I'm prepared to write at this time
(past midnight here) - but I'll do so ASAP.
It's not a simple issue. For example you mention that control
values should somehow look ahead (how is another matter, and
it can get hairy). That is true, but otoh in many cases the
delay you get when not doing this doesn't matter, so there's
good reason to avoid the complexity in those cases, etc. etc.
So it ends up as an exercise in weighting pros and cons, you
can't reduce it to some simple 'binary' statement. We wouldn't
have this debate if things were so simple.
More later,
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
Assuming that
by 'signals' you mean audio rate controls, yes.
Again, the exception is 'synthesis' modules. You'd expect a
VCA in a modular synth to do what the ADSR tells it to do,
even if that results in artefacts. But you wouldn't want a
fader in a DAW (essentially the same thing if implemented as
a plugin) to do the same.
Sorry, that term isn't clear. By 'signals' I meant basically a single
numeric value (float), whatever the rate.
As opposed to an 'event' (also an ambiguous term) which actually says
"hey, this control changed to 4.0 now" when it changes - and only when
it changes. An 'event' could also have ramp information for
interpolation and whatever else.
5:55 a.m.
On Mon, 2012-05-28 at 22:40 +0000, Fons Adriaensen wrote:
On Mon, May 28, 2012 at 05:08:02PM -0400, David
Robillard wrote:
Dave, thanks for this reply. Reading your complete post, there's
a lot to discuss - more than I'm prepared to write at this time
(past midnight here) - but I'll do so ASAP.
No rush. I am particularly interested in your thoughts on my question
about how L is related to the run buffer size, and what restrictions
there are there (e.g. L of course at least has some minimum value
dependent on the sample rate to make bandlimited interpolation possible
at all, but surely there is more to it in various cases)
Assuming
that by 'signals' you mean audio rate controls, yes.
Again, the exception is 'synthesis' modules. You'd expect a
VCA in a modular synth to do what the ADSR tells it to do,
even if that results in artefacts. But you wouldn't want a
fader in a DAW (essentially the same thing if implemented as
a plugin) to do the same.
Sorry, that term isn't clear. By 'signals' I meant basically a single
numeric value (float), whatever the rate.
As opposed to an 'event' (also an ambiguous term) which actually says
"hey, this control changed to 4.0 now" when it changes - and only when
it changes. An 'event' could also have ramp information for
interpolation and whatever else. It's not a simple issue. For example you mention
that control
values should somehow look ahead (how is another matter, and
it can get hairy). That is true, but otoh in many cases the
delay you get when not doing this doesn't matter, so there's
good reason to avoid the complexity in those cases, etc. etc.
Actually I think it's globally significantly *less* complicated to
provide future values.
So it ends up as an exercise in weighting pros and
cons, you
can't reduce it to some simple 'binary' statement. We wouldn't
have this debate if things were so simple.
Absolutely true. All of these issues, various scenarios and the
different ways they must be handled, are inherently there no matter
what.
However, everything gets quite ugly and complicated when the values the
plugin has during run() have different time bases, because all those
different scenarios then cross-cut the plugin API, and it's *everyone's*
problem to carefully deal with *every* case that exists. This is
probably literally impossible.
Even if it isn't, as you mentioned - lots of code sucks. If plugins
have to have fancy configurable control latency settings it takes a
mailing list of intelligent people weeks to even figure out - they're
simply going to do it wrong.
However, it seems possible to move all of that stuff out of the plugin
API so those different scenarios become only the host's problem.
Everything then becomes clean and feasible.
In my opinion, having several different interpretations for controls
depending on scenario in the plugin is an impossible solution. It would
never happen. Avoiding this, then, is an overriding requirement. We
need *one* way of describing controls in run() that works in all cases.
I think providing synchronous control events, with 'future' values (at
least some distance L in the future) is the way to get that. Let's
pretend that the Ultimate Plugin Interface (UPI) 1.0 exists, works this
way, is stable and unmalleable, and all you have to work with to deliver
your product (a plugin).
From the plugin author perspective: is there anything
that is
*impossible* to do correctly?
-dr
2 Jun
2 Jun
12:09 a.m.
On Mon, 2012-05-28 at 17:08 -0400, David Robillard wrote:
[...]
The buffer size feature is
mostly orthogonal and simple to add; ControlPort is the thing that
probably shouldn't have been inherited from LADSPA at all.
Back to the buffer size issue... (which is actually tangent to what this
discussion ended up being about but is sometimes useful anyway)
I am experimenting with adding the following callback. As far as
describing the block length goes, I can't imagine this not being able to
represent any reasonable situation, but just in case, I present:
/**
Get the limits on block length.
The values here are restrictions on the sample_count parameter of
LV2_Descriptor::run().
@param min Set to the minimum block length.
@param max Set to the maximum block length, or 0 for unlimited.
@param multiple_of Set to a number the block length will always be
a multiple of, possibly 1 for arbitrary block lengths.
@param power_of Set to a number the block length will always be
a power of, or 0 if no such restriction applies.
*/
LV2_Buf_Size_Status
get_block_length(LV2_Buf_Size_Buf_Feature_Handle handle,
uint32_t* min,
uint32_t* max,
uint32_t* multiple_of,
uint32_t* power_of);
This callback just describes reality to the plugin at instantiation
time. As far as plugins specifying requirements goes, it seems complete
to allow the plugin to specify, for each field:
1) A specific value
2) That some value must be given
The goal here is simply to be able to convey any reasonable restriction
on the block length. Whether or not people will/can actually implement
a given restriction is irrelevant.
That said, in practice, I suspect:
* max will be usually available (e.g. the Jack buffer size) and is
widely useful
* convolution plugins may require power_of 2 and perhaps min
* requirements for multiple_of, or power_of any value but 2, are
probably useless
If anyone can think of a block length restriction that can not be
described by this scheme, do tell.
Cheers,
-dr
12:22 a.m.
On Fri, Jun 01, 2012 at 06:09:06PM -0400, David Robillard wrote:
LV2_Buf_Size_Status
get_block_length(LV2_Buf_Size_Buf_Feature_Handle handle,
uint32_t* min,
uint32_t* max,
uint32_t* multiple_of,
uint32_t* power_of);
This callback just describes reality to the plugin at
instantiation
time.
That 'to the plugin' confuses me - isn't this supposed to be
the plugin telling the host what it (the plugin) needs ?
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
12:42 a.m.
On Fri, 2012-06-01 at 22:22 +0000, Fons Adriaensen wrote:
On Fri, Jun 01, 2012 at 06:09:06PM -0400, David
Robillard wrote:
This is a host-provided callback which the plugin can use at
instantiation time to figure out the block length characteristics,
whatever they may be.
-dr
LV2_Buf_Size_Status
get_block_length(LV2_Buf_Size_Buf_Feature_Handle handle,
uint32_t* min,
uint32_t* max,
uint32_t* multiple_of,
uint32_t* power_of);
This callback just describes reality to the
plugin at instantiation
time.
That 'to the plugin' confuses me - isn't this supposed to be
the plugin telling the host what it (the plugin) needs ?
12:56 a.m.
On Fri, Jun 01, 2012 at 06:42:01PM -0400, David Robillard wrote:
On Fri, 2012-06-01 at 22:22 +0000, Fons Adriaensen
wrote:
(Maybe it's just too late for me to grok this...)
So it's the plugin asking the host what it can expect ?
Isn't the LV2 way to do this via features (i.e. the plugin
checking which features the host provides) ?
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
On Fri, Jun 01, 2012 at 06:09:06PM -0400, David
Robillard wrote:
This is a host-provided callback which the plugin can use at
instantiation time to figure out the block length characteristics,
whatever they may be. LV2_Buf_Size_Status
get_block_length(LV2_Buf_Size_Buf_Feature_Handle handle,
uint32_t* min,
uint32_t* max,
uint32_t* multiple_of,
uint32_t* power_of);
This callback just describes reality to the
plugin at instantiation
time.
That 'to the plugin' confuses me - isn't this supposed to be
the plugin telling the host what it (the plugin) needs ?
2:22 a.m.
On Fri, 2012-06-01 at 22:56 +0000, Fons Adriaensen wrote:
On Fri, Jun 01, 2012 at 06:42:01PM -0400, David
Robillard wrote:
Yes.
On Fri, 2012-06-01 at 22:22 +0000, Fons
Adriaensen wrote:
(Maybe it's just too late for me to grok this...)
So it's the plugin asking the host what it can expect ? On Fri, Jun 01, 2012 at 06:09:06PM -0400, David
Robillard wrote:
This is a host-provided callback which the plugin can use at
instantiation time to figure out the block length characteristics,
whatever they may be. LV2_Buf_Size_Status
get_block_length(LV2_Buf_Size_Buf_Feature_Handle handle,
uint32_t* min,
uint32_t* max,
uint32_t* multiple_of,
uint32_t* power_of);
This callback just describes reality to the
plugin at instantiation
time.
That 'to the plugin' confuses me - isn't this supposed to be
the plugin telling the host what it (the plugin) needs ? Isn't the LV2 way to do this via features (i.e.
the plugin
checking which features the host provides) ?
Yes, this callback is in a feature struct. Essentially "feature" just
means "thing provided by the host"[1].
Restrictions would also be features, but nevermind that. Regardless of
restrictions, plugins need to know the actual situation (for example,
right now I need to know max, and have no restrictions). It decomposes
nicely.
Not mentioning any of that was no mistake[2]; I am attempting to farm
precisely one piece of information: whether or not the above callback
can adequately express any reasonable block length situation; i.e.
whether or not any plugin could know "yes, I can run" or "no, my
requirements to run are not met".
There is no point confusing the thread with restrictions since that part
follows directly from the above.
Also, I said something stupid about constant size earlier. Of course,
constant can be expressed with min == max.
-dr
[1] Poor choice in terminology here, it turns out
[2] I know this stupid list and how to extract useful information from
it and ignore the rest ;)
4:40 a.m.
On Fri, 2012-06-01 at 20:22 -0400, David Robillard wrote:
[...]
I am attempting to farm
precisely one piece of information: whether or not the above callback
can adequately express any reasonable block length situation; i.e.
whether or not any plugin could know "yes, I can run" or "no, my
requirements to run are not met".
Also, I suppose, if restrictions need to be dynamic, or if static will
suffice.
Specifying exact minimums and/or multiples seems pretty esoteric and of
dubious benefit to me. Doing so dynamically even more so...
-dr
11:32 a.m.
[David Robillard]
On Fri, 2012-06-01 at 20:22 -0400, David Robillard
wrote:
[...]
If you will allow me to go back and question the rationale for a bit:
Consider a simplistic convolution plugin designed around a fixed
FFT-dictated block size. Obviously somebody has to decouple this
fixed size from whatever is factually mandated by the audio hardware
or used by a freewheeling host's main audio loop. As long as the host
isn't *required* to implement this decoupling, our plugin *must* do it
itself and work with whatever the host asks it to process, be it 1,
23, or 16384 frames at a time.
As a plugin author, I consider all assumptions or guarantees about
the block size useless unless the plugin can tell the host exactly
what it wants, and get it everytime (and I'm not even sure it would be
worth it).
Cheers, Tim
I am attempting to farm
precisely one piece of information: whether or not the above callback
can adequately express any reasonable block length situation; i.e.
whether or not any plugin could know "yes, I can run" or "no, my
requirements to run are not met".
Also, I suppose, if restrictions need to be dynamic, or if static will
suffice.
Specifying exact minimums and/or multiples seems pretty esoteric and of
dubious benefit to me. Doing so dynamically even more so...
6:16 p.m.
On Sat, 2012-06-02 at 11:37 +0200, Tim Goetze wrote:
[David Robillard]
LV2's feature mechanism is specifically designed so plugins can
*require* features if they need to. A plugin that requires a feature
will not be instantiated if it is not available.
This lets us hammer out functionality without having to fight about what
everyone *has* to implement. Essentially everything is as you say,
except *you* get to decide what you require, there is no central
authority (me) mandating what everyone must require.
There are some situations where a given feature (like power of 2 block
length) are sensible/possible, and some where they are not. In this
case, for example, a restricted block length is required for latency
free convolution plugins, period.
So, the API should support this. That does not imply it will/can/should
be available in every scenario. If you don't want to require it -
don't. If you want to do convolution in hosts that can't provide it,
you have to add latency.
LV2 is not in the business of crippling *everything* because *something*
can't do it. The decisions about whether to make use of or implement
something like this, as you described, are inherent to the problem, and
depend on the situation. The API should make it possible to do what you
want - not get in the way.
Cheers,
-dr
On Fri, 2012-06-01 at 20:22 -0400, David Robillard
wrote:
[...]
If you will allow me to go back and question the rationale for a bit:
Consider a simplistic convolution plugin designed around a fixed
FFT-dictated block size. Obviously somebody has to decouple this
fixed size from whatever is factually mandated by the audio hardware
or used by a freewheeling host's main audio loop. As long as the host
isn't *required* to implement this decoupling, our plugin *must* do it
itself and work with whatever the host asks it to process, be it 1,
23, or 16384 frames at a time.
As a plugin author, I consider all assumptions or guarantees about
the block size useless unless the plugin can tell the host exactly
what it wants, and get it everytime (and I'm not even sure it would be
worth it). I am attempting to farm
precisely one piece of information: whether or not the above callback
can adequately express any reasonable block length situation; i.e.
whether or not any plugin could know "yes, I can run" or "no, my
requirements to run are not met".
Also, I suppose, if restrictions need to be dynamic, or if static will
suffice.
Specifying exact minimums and/or multiples seems pretty esoteric and of
dubious benefit to me. Doing so dynamically even more so...
4 Jun
4 Jun
10:08 a.m.
[David Robillard]
On Sat, 2012-06-02 at 11:37 +0200, Tim Goetze wrote:
Hmmm, thanks for clearing that up; I suspected as much.
Cheers, Tim
As a plugin author, I consider all assumptions or
guarantees about
the block size useless unless the plugin can tell the host exactly
what it wants, and get it everytime (and I'm not even sure it would be
worth it).
LV2's feature mechanism is specifically designed so plugins can
*require* features if they need to. A plugin that requires a feature
will not be instantiated if it is not available.
4368
days inactive
4377
days old
linux-audio-dev@lists.linuxaudio.org
46 comments
8 participants
participants (8)
-
Adrian Knoth -
David Robillard -
Fons Adriaensen -
Paul Davis -
Stefano D'Angelo -
Thorsten Wilms -
Tim E. Real -
Tim Goetze