Specification issues in open systems - Linux-audio-dev

27 Sep 2008

The following is a cross-post of an exchange that took
place on the rosegarden-devel mailing list. I'm posting
it here because I think it hints at something fairly
serious with the current state of open audio specifications
and issues with their implementation.

It is not my intent to start a flame war. I come at this
with over twelve years of involvement (albeit chiefly at a
hobbyist level) with audio development and production on
windows system (predominantly VSTis). I've used linux
since the late nineties, but these issues have become more
important to me since fully ditching Windows about 18
months ago.

The open source philosophy has always been that,
if something were not to your tastes, you were completely
free to build it, or alter something else. This is true,
and one of the things greatly appreciated about the system.
The key issue, though, is the ability to use that facility;
sometimes it is not as available as it appears. There are a
great many people who would offer much more to open audio,
I feel, but issues like the following need to be addressed.

all the best,
Chris Williams

Chris Williams <yikyak> wrote:
...
  Chris Cannam wrote:
  Chris Williams <yikyak> wrote:
  Hello,

 I've been developing a toy DSSI plugin, partly to become
 more familiar with the spec and partly as a platform for
 writing base code for use with less toy instruments. I've
 been using Rosegarden and jack-dssi-host as test hosts.

 All was going well until I increased the number of DSSI /
 LADSPA output audio ports in the plugin. For some reason,
 I expected that rosegarden would create extra synth
 channels in the audio mixer for these outputs, but this
 didn't happen. Instead, one 'synth' audio channel was
 maintained in the mixer for the synth, out of which all
 outputs could seemingly be heard and controlled globally.

 Does Rosegarden mix-down all the audio outputs of a given
 synth before feeding the signal to its own internal signal
 path, or am I misunderstanding something / being an idiot?
 The number of outputs shows up correctly in the synths
 'controls' dialog along with its ID, but this seems to be
 the only place at which they're discernible outside of the
 synth.  (I'm using v.1.7.0 from Arch's repository).

 No, you aren't misunderstanding anything.  Rosegarden is
 very simplistic in this respect -- it mixes the number of
 audio outputs up or down to match the number of channels
 on the track, which is always either 1 or 2 depending on
 whether the stereo button in the instrument parameters is
 pushed in or not.

 If the aim is to accept multi-MIDI-channel input and send
 multi-channel output in order to support multiple effective
 instances on separate tracks, the "DSSI way" to do that is
 to have several actual instances (perhaps sharing data
 behind the scenes) and then call run_multiple_synths once
 for all of them instead of run_synth on each one.
 Rosegarden will do this if you have the same plugin set for
 more than one track (and it supports run_multiple_synths).

 Unfortunately, that mechanism is rather different from any
 other plugin format.

 Thanks Chris, that's just the information I was looking for.

 I was thinking more of the situation where where you have
 *single*-midi-channel input but, as with some synths and
 samplers, want to run the output to different banks of
 effects (e.g.  LADSPA plugins) depending on the specific
 midi note or the range in which that note lies (output
 groups). This seems "theoretically" possible under DSSI
 using only run_synth() (given an idiosyncratic parsing of
 the DSSI spec) but not if the host routes all output ports
 through the same audio channel. At the same time, I can see
 the problem from the host developers' perspective: the DSSI
 spec uses LADSPA's port system and there's no good reason
 for an effect's output ever to be routed to multiple 'host
 audio channels'.

 The two other ways of doing it would seem to be:

 1) Incorporate a 'channel' system and ladspa hosting system
 *internal* to the instrument which would then only ever
 need a stereo output to the host (the massive downside of
 this being that it unnecessarily replicates complex
 functionality already provided by the host); or

 2) Use run_multiple_synths() in a hackish manner for which
 it probably wasn't fully intended, as you'd be writing to
 separate LADSPA handles but essentially using only one midi
 event queue (this would also have the nasty side effect of
 requiring multiple redundant instances simply to use their
 output ports). You could probably do something equally
 nasty by playing with the descriptor returning and
 instantiate functions.

 Anyway, thanks again. It's not a Rosegarden-specific issue,
 I know; more strangeness in the DSSI spec coupled with my
 own ignorance. 
I've been thinking about this some more, and can't let it
lie. It seems to me that there's definitely a bug or broken
implementation involved somewhere.

Under jack-dssi-host, if I request n outputs, I get n
outputs, any one of which is routable to wherever I wish
to send it using jack_connect or qjackctl. Under
rosegarden, if I request n outputs, I always get either one
or two, depending on the stereo settings of the host-
ascribed audio channel.

This is a problem, because any summing rosegarden does
*could* have been done internally in the plugin. What's the
point of requesting, configuring and filling >2 output
ports if the client is never *effectively* going to get
them? If the client wanted them bumped down to one (or two)
buffers, it could -- would -- have done this itself and
needn't have requested and configured >2 outputs in the
first place.

Now, one way of approaching this is to say, "Oh, well; it's
an implementation bug in rosegarden and there's nothing
that can be done". That's OK as far as it goes, but it
doesn't actually solve the problem / ambiguity and it's
questionable as to whether it's true.

Another approach is to say that rosegarden implements the
DSSI spec correctly and the bug is actually in the spec
itself. This is *far more serious* -- and it's far more
serious for two reasons.

Firstly, rosegarden is currently the only fully-featured
standalone DSSI host. How are the other hosts currently
implementing it (MuSE etc.) going to behave? Are they going
to implement rosegarden's behaviour, or are they going to
implement jack-dssi-host's behaviour? In either case, you
have a situation where plugin developers simply don't know
what the host is going to do, which kind of defeats the
point of a specification.

At this point a cynic might say that the uptake of DSSI is
such that it doesn't matter too much and that the likes of
LV2 will solve this problem anyway, but this brings us to
the second serious problem (which the more well-versed
among you will already have realised):

If this ambiguity isn't specific to rosegarden but is
inherent in the spec, then *LV2 doesn't solve it either*.
Part of the problem with DSSI, if this is where the
problem lies, is that it encourages host developers to
treat synth / sampler plugins as extensions of effects
plugins for the sake of simplicity. This works to an
extent, but there's a subset of functionality that they do
not have in common (the ability to leverage other effects
plugins and host capabilities for two).

There is no output-negotiation scheme in either LV2 *or*
DSSI beyond the output port request and instantiation
stage, and neither spec makes clear what a plugin can
expect having acquired those outputs, or, from the other
side, what a host is fully expected to provide in response
to a given request. The assumption goes unstated, which is
why there is this problem to begin with.

Perhaps an extra negotiation is required, so that both the
host *and* the plugin know unambiguously the contexts in
which they are operating. It's not reasonable to expect a
host to grant any and all resources that a synth plugin
might request (2^32 outputs, for example), but at the same
time the *plugin* needs to *know* the context in which it's
operating, and not have to guess it based on external
knowledge of specific host implementations.

Anyway, I appreciate that the rosegarden developers are not
fully responsible (if at all) for this situation, but it is
a major issue, I feel, when it comes to standard adoption
and plugin development with open systems. I may cross-post
this to the LAD list as it seems just as relevant to them
as to you.

Thanks for your time,

Chris Williams