6 Feb
2003
6 Feb
'03
6:12 p.m.
On Thursday 06 February 2003 12.58, Laurent de Soras [Ohm Force]
wrote:
That's one point, although I think you could have the plugin factory
of a wrapper do the instantiation "secretly". A wrapper *is* a plugin
host anyway, so this shouldn't be a major issue.
Either way, for various other reasons, I'm beginning to think that
there are other reasons to use lightweight plugin instances for
metadata requests. We have already mentioned plugins that can
"mutate", and generic wrappers (ie wrappers that have a control for
selecting the alien plugin to wrap) belong in that category.
Yeah, but it doesn't really solve the problem, does it? If you get
endless recusion (or rather ping-pong, if we're talking about events;
not much better though), it's because there's something wrong with
the host, the plugin or both.
There has to be a way to guarantee that dependencies don't become
circular... Can we have some actual examples to break down? I've
never written a host that changes it's parameters based on plugin
properties, so I've never seen the problem in real life.
What state transitions?
Yeah... Though I think all this should be left to the host,
regardless. Just tell the host where your GUI is. If it's a "local
half" plugin (that in turn refers to an external GUI), it could be in
the same binary, as it's still just a plugin with a minimum of
external dependencies. (Whether or not it wants to run in the audio
thread is another matter. It can still use the same API - and the
same rules WRT GUI toolkits still apply; don't use them.)
Not really. The difference is that my approach doesn't require GUIs
and hosts to use a special API just for talking to controls.
Also, tokens don't mix with the direct connection system we're using.
(XAP plugins generally don't send events through the host, but rather
directly to the input queues of other plugins.)
Good idea, of course. And besides, this even further reduces the need
for "advanced" methods of maintaining supported and unsupported
calls.
Yeah, but you can't really get around that. You can still implement a
host that does small block (not the Chevy V8! :-) sub nets for
feedback loops, but it's going to be rather expensive. OTOH, that's
*always* expensive...
My point though; expensive is something you could live with. It might
even matter to you. *Impossible* is a different story entirely.
[...]
Or just make it a setting in "advanced options"...?
(If you're not very interested in users upgrading your software, that
is! ;-)
They don't really have much of a choice, do they?
Very good point! If it *crashes*, who cares if it's slightly faster?
Yes.
As a matter of fact, I have *exactly* this problem with the FX plugins
in Audiality. Initializing the parameters of the xdelay or reverb
effects means they have to recalculate all taps almost once per
parameter. *heh* They're about the last controls to use function
calls, and it's a real problem.
I could handle it by implementing the changes in the process() call,
though, just like with events. It's just that the changes are not RT
safe, so they have to send the work off to another thread. Dealing
with that *and* the function call API, well... I'll just leave it
until I've fixed the plugin API.
There's a *big* hidden issue with thinking about different interfaces
yor different states. States are related to what context a plugin
runs in, and using different APIs in different states doesn't really
help. You can't perform non RT operations on a running plugin just
because it has a separate interface for it! Either the plugin must be
thread safe, or the host has to take the plugin out of the RT thread
first.
You're totally right. I wasn't thinking straight.
[...]
Yeah, I realize that, but there's still a difference between explicit
1.0/note and 1.0/octave with the scale assumed to be 12tET. If the
scale *isn't* 12tET, rounding Base Pitch won't work as expected.
Yes, I remember now that you say it. However, it has very limited
range (cents; sint8_t), and the MIDI pitch field still doubles as the
"note ID", so this is in no way a substitute for continous pitch, or
even sufficient for handling scales that are substantially different
from 12tET.
[...]
What's wrong with using a generic scale converter plugin *before* any
synth you want to do this with? That's one of the major benefits of
using continous, linear pitch, IMO. I don't think synths should have
internal scale converters, unless they have *very* good reasons. (And
I can't think of any.)
Yeah, but what is a "default" velocity in real life...?
It's something that was invented for the MIDI protocol, to support
controllers without velocity sensitivity. As even the cheapest "toy"
MIDI controllers are velocity sensitive these days, it is essentially
irrelevant, even to MIDI devices.
[...]
Somewhat like in VST; I ask for a time info struct.
I'm not denying that.
Sure, but multiplication of two integers always gives you an exact
result. (Yes, even if it doesn't fit! It's just that high mid and
level languages can't make use of the upper half of the result -
which I find really rather annoying... *heh*)
There will be conversion work to do *somewhere* no matter what.
However, I don't like the idea of making all controls have hard
ranges. It restricts the users in ways not necessarily intended by
plugin authors, since there's no way to set controls to values
outside the range the plugin author decided to use, even if there's
no technical reason why it couldn't be done. You're basically
throwing away interesting possibilities.
Yes, I'm perfectly aware of that... *heh*
Some of it can be made optional, of course. In fact, even range and
default values could be made optional. Just define the default range
for an unmarked control as [0, 1], and the default value as 0.
Whether the range should be soft or hard, I'm not sure, but soft seems
more useful.
Good point. I think we can count on standard C calling conventions to
work and be supported by all languages on every platform. No big deal
if it isn't as fast as the default conventions - the heavy traffic
goes through the event system anyway, and there are no per-event
function calls.
Of course, a language that cannot interface with the event struct
would be a problem. Requirements: Pointers, 32 bit integers, 32 bit
floats and 64 bit floats. (We *could* make all integers 64 bit on 64
bit CPUs, but it seems insane and pointless to me. 64 bit machines
don't have infinite memory bandwidth...)
[...]
Late reply, I was quite busy and english writing takes
me
time. I commented various people quotes here. However I
haven't read recent posts yet, I'm a bit overhelmed by
the reply avalanche. :)
Well, we're crazy about plugin APIs around here! ;-)
[...]
I'm
questioning whether
having a simpler query based system may be easier. I don't like
the idea that you have to instantiate to query, though.
Many plug-in standards require instanciation before fetching
properties. PTAF must be able to make good wrappers for existing
plug-ins, industry requires it. Yeah, I've
thought about that too. Rather hairy stuff. The only
easy way to avoid it is to strictly define what the host should
tell the plugin before asking anything, and then just leave it at
that.
We can also add somewhere a watchdog limiting the recursion. [...the
"sequencer" thread...]
As suggested we can call it "host", or "main" thread. In my opinion
generic forms of hosts have several "virtual" threads :
- Audio: handles audio processing and related sequencing
- Main/Background: handles generic operations, state transitions, as well as background disk streaming, data preparing
for audio
thread, etc.
- GUI thread (at least on Win and Mac OSs), can theoretically
be merged with the main thread, but it's not recommended (Mac
programmers have the bad habit to poll the mouse within a
blocking loop when user is holds a click).
Well, they didn't have much choise in the old days... Win16 had the
same problem. Provided we're talking about Mac OS X, it certainly is
nothing but a bad habit, though.
Anyway, I know what you mean by these threads - but I still don't see
it as relevant to a plugin API. A GUI-less audio server may well have
only one thread, and either way, plugins should not be required to be
thread safe, unless they explicitly ask for it (trouble, that is ;-)
themselves.
[...]
Well, you
might still be able to comment on an idea: Splitting up
the GUI in two parts; one that runs in the same process as the
DSP plugin, and one that runs in another process, possibly on
another machine.
Yeah, I had this in mind when i was talking about lauching the
GUI from the plug-in. You would need
a feature that allows control outputs to be marked
as "active" or "passive". This allows hosts to control how things
are handled.
If I see a knob that is turning via some old automation, I
should be able to grab it in the UI and take over. It is not
clear to me, however, whether the host should remember that and
leave control with the highest prio controller, or just make it
temporary. This needs more thought. It may not be the domain of
the API at all, but the host.
But the token system is just about the same concept, isn't it ? For a given parameter, there is at most one active
client at a
time, others are passive and receive only change notifications.
Arbitration is done by host, who is allowed to deprive a client
from his token to give it to a client requesting it.
This doesn't work if some clients are out-of process. Control outputs
with active/passive modes do, since you never have to inform client
whether or not their data is used.
For example it makes sense to give less priority to
the
automation system than to user via GUI, so the latter can
steal its token, just by requesting it. When user has released
the knob, it give back the token to the host making it
available for any use (can transmits it implicitly to the
automation system).
Sure, but it just seems like a more complicated way of doing it,
especially when threading and out-of-process issues are taken in
account.
I actually
think having to try to *call* a function to find out
whether it's supported or not is rather nasty...
Good point.
Anyway I would like to minimize the number of functions. Functions are handy when the calling conventions
of both parts are compatible. If the language you use
doesn't support this convention natively, all functions
have to be wrapped, which is a real pain.
Really?
Either way, we also have to consider the importance of wrapping the
API. How many write VST hosts and plugins in other languages than
C++? (I know that some do. FruityLoops is written in Delphi, IIRC.)
We also have to consider the balance between the work required to
implement a wrapper/language binding and the impact simplifying that
has on the native API, in terms of performance and ease of use.
More, there are still issues with existing plug-ins,
reporting
wrongly if they support some opcode or not (talk to this issue
to Plogue people :).
Bugs are always bugs. A simple testing host in the SDK would eliminate
all valid excuses for this particular bug. (And it could of course be
included in the host SDK as well, to warn every user about broken
plugins.)
Either way, I don't see a real problem here. Just don't call a NULL
pointer, ok? :-)
I think this
separation is a mistake. Of course, we can't have an
API for complex monolith synths scale perfectly to modular synth
units, but I don't see why one should explicitly prevent some
scaling into that range. Not a major design goal, though; just
something to keep in mind.
The separation between coarse/fine grained plug-ins was just
a design goal for the API. Fine grained plug-ins would have
more issues, like the ability to do fast sample-per-sample
processing (for feedback loops). How about
defining buffer alignment as "what works good for
whatever extensions that are available on this hardware"...?
Good, but it requires the host to be updated when new hardware
is released. We can add a host property indicating the current
alignment so the plug-in can check it before continuing.
Very good idea. There *are* platforms where a plugin could crash the
host if they ignore this.
you'll get
a lot of arguments about that on this list - linux
people tend to have a 486 or pentium stashed somewhere. :)
Professional musicians and studios have recent hardware, at
least it's what survey results show us. Recent audio software
produced by the industry also tend to be hungry, requiring
fast hardware - I don't say it's good, it's just a fact, and
industry is the main target for PTAF.
It's a fact, and it's not as much a result of inefficient APIs as it
is a result of people wanting "cooler" DSP effects. I don't have a
problem with this, but I *do* have, and always will have, a problem
with wasting resources for no good reason. Thus, I tend to look for
optimal solutions at all times. If it's something I'll have to live
with for years, I look harder - just in case it turns out that it
actually matters.
If you plan to make a standard, it's better to
build it for
having use over years. After all, MIDI is still here and will
probably last 20 years again. OK things are a bit different
with pure software protocols, but arguing little performance
boost against obvious programming safety is pointless for me
(sadly programmer's brain doesn't follows the Moore law).
Very good points, and of course, I'm not proposing that we should make
life hard for plugin authors just to save a few cycles.
It's just that the wasted cycles is not the *only* problem I have with
the dispatcher approach, so if there's a better solution (that also
happens to be a bit faster), why not use that?
Agreed - but
they also attach value to efficient software.
Yes, but given my experience with customer support, unsafty
displeasure is one or two orders of magnitude above efficency
pleasure. That's one
way... I thought a lot about that for MAIA, but I'm
not sure I like it. Using events becomes somewhat pointless when
you change the rules and use another callback. Function calls are
cleaner and easier for everyone for this kind of stuff.
The advantage of events is that you can have several of them at
once, giving the plug-in the possibility to optimise operations. We've just
spoken of calling the process() function with a
duration of 0 samples. Events are then processed immediately,
and no second API is needed.
Dunno if this factorisation is really good. Different goals,
different states for call, it should be a different opcode/func. Anyway,
I'm nervous about the (supposedly real time) connection
event, as it's not obvious that any plugin can easilly handly
connections in a real time safe manner.
Don't mix up audio configuration (which is not real-time)
and connections. In most of cases I see, (de)connections
is reflected in the plug-in by changing a flag or a variable,
making the processing a bit different. For example a plug-in
has a 2-in/2-out configuration. When only one input or output
channel is connected, it gives the ability to make the
processing mono, or to (de)activate vu-meters, etc. This is where
it gets hairy. We prefer to think of scale tuning
as something that is generally done by specialized plugins, or by
whatever sends the events (host/sequencer) *before* the synths.
That way, every synth doesn't have to implement scale support to
be usable.
That's why Base Pitch is float. But because MIDI is still there,
existing plug-ins will be happy to round (after scaling if 1/oct)
Base Pitch to get the note number. As it is with
VST, most plugins are essentially useless to people
that don't use 12tET, unless multiple channels + pitch bend is a
sufficient work-around.
VST has also a finetuning delivered with each Note On. The only part
I don't like is assuming that Base Pitch isn't
linear pitch, but effectively note pith, mapped in unknown ways.
However many (hard/soft) synth can be tuned this way. It is a
convenient way to play exotic scales using traditional keyboard,
piano roll or other editors / performance devices. Right, but
MIDI is integers, and the range defines the
resolution. With floats, why have 2.0 if you can have 1.0...?
To make 1.0 the middle, default position. Anyway, what
I'm suggesting is basically that this event should
use the same format as the running musical time "counter" that
any tempo or beat sync plugin would maintain. It seems easier to
just count ticks, and then convert to beats, bars or whatever
when you need those units.
But how do you know on which bar the plug-in is ? Tempo and
time signature can change in the course of the song and it
is not obvious to deduce the song position from just an
absolute time reference or a number of ticks or beats.
That's why tempo and beat sync plugins in VST have to ask for time
info for every block.
The only difference with XAP is that if you have input controls for
musical time, you'll get timestamped events whenever interesting
changes occur, so if you just listen, you can have a sample accurate
view of the musical timeline at all times.
(XAP beats VST in this regard, since VST can only give you time info
for the first frame in each block. XAP plugins can maintain sample
accurate beat sync even if there are several *meter* changes within a
single block. All VST can tell you is the tick time for points in
audio time, so you could miss meter changes.)
Yes. And bars
and beats is just *one* valid unit for musical
time. There's also SMPTE, hours/minutes/seconds, HDR audio time
and other stuff. Why bars and beats of all these, rather than
just some linear, single value representation?
Could be seconds or ticks, but the latter make more sense as it's
locked to the song even if the tempo is changed.
For reasons mentioned above, both time in second an musical
representation are needed. SMPTE etc can be deduce from the
absolute time in seconds or samples.
Yep, but that's true for musical time as well - although musical time
*is* indeed of much more interest to many more plugins, so
special-casing it in the API is probably motivated. (Which is why XAP
has a set of controls specifically for musical time.)
Accuracy is
*not* infinite. In fact, you'll have fractions with
more decimals that float or double can handle even in lots of
hard quantized music, and the tick subdivisions used by most
sequencers won't produce "clean" float values for much more than
a few values.
But is that so important ? Float or int, we manipulate numbers
with a finite precision. And if someone is happy with tick
scales multiple of 3, someone else will complain about not
supporting multiples of 19, etc.
Yeah, that's why sequencers use these "weird" numbers... They don't
cover *everything* of course, but they're high enough that rounding
times to the nearest tick is ok - and then you can keep calculations
in the sequencer exact.
For the drifting question, it is neglictible with
double, and
can be easily resync'd if required. Ohm Force's plug-ins use
double as positions and steps for the LFO phases and it works
fine without resync during hours, on any fraction of beat.
If you need exact subdivisions, you can still convert the
floating point value into the nearest integer tick.
Well, I can't disagree, as I used to argue for your position, and have
yet to see actual proof of the rounding errors/drift being a problem.
Accurate musical time calculations belong in the sequencer domain, as
we can't support them anyway, unless possibly if we switch to musical
time timestamps.
The host would
of course know the range of each control, so the
only *real* issue is that natural control values mean more work
for hosts.
No it just would store and manipulate normalized values.
Natural values would be needed only for parameter connection
between plug-ins, if user chooses to connect natural values. What's
wrong with text and/or raw data controls?
They often require a variable amount of memory, making
them difficult to manage in the real-time thread. However there is a need for this kind of stuff for
sure.
I thought more about encapsulating them into specific
events.
Yeah, that's the approach I have in mind. Chained fixed size blocks or
something... It's not too much fun working with, but we *are* dealing
with real time here; not word processing. People wanting to write
real time software will simply have to accept that it's a different
world, since there's no way we can changes the laws of nature.
1) none - host
can autogenerate from hints
2) layout - plugin provides XML or something suggesting it's UI,
host draws it
3) graphics+layout - plugin provides XML or something as well as
graphics - host is responsible for animating according to plugin
spec 4) total - plugin provides binary UI code (possibly bytecode
or lib calls)
I agree with 1) and 4). For my experience, when a plug-in
has 4), it has also 1), but it's not *always* the case
because developers don't feel necessary to implement the
hints correctly (dev time matters). Adding more UI
description is likely not to be supported by either host
or plug-in, making them useless. Good solution would
be using portable UI libs, as someone suggested here,
but everyone knows it's the Holy Grail.
Yeah... *heh* Even the nicest toolkit/scripting solution will only
work for *most* plugins; not the ones that want new, cool visual
effects.
Though SDL is as portable as it gets, and what you can't do with that
can't be done, basically. I can assure you that an API like that
*will* work for any plugin GUI, one way or another.
How? All it provides is basic blitting and pixel level access to the
screen and other surfaces. This is stuff that any platform with
graphics can support, and most GUI toolkits also have some form of
low level canvas or other component that can do it. I've even
implemented the SDL 2D API on top of OpenGL (glSDL), for lightning
fast rendering where h/w acceleration is available. (Pretty much the
only way to have fully accelerated SDL on X at this point.)
If we want to spend time on anything along the lines of a portable GUI
toolkit, SDL's graphics functionality is the *first* thing we should
implement, or it's just a waste of time. Then we can start thinking
about building something like VSTGUI and/or other stuff on top of
that.
The spec
should not dictate ABI, either. The ABI is an articat
of the platform. If my platform uses 32 registers to pass C
function arguments, it is already binary imcompatible with your
PC :)
The only way to guarantee real portability through platforms
and programming languages is to describe the API at the byte
level. This include calling convention. IMHO We can assume
that every target system has a stack, and can pass parameters
on it. If you want to generate only ASCII in your program,
just write
only ASCII, it will be supported by UTF-8. However if you have
to read strings, it's better to be aware that it's UTF-8,
because incoming strings may contain multi-byte characters.
...and what that means to your average plugin is basically "Don't
start splitting strings arbitrarily!"
Would be interesting to know which ASCII values are valid inside
multibyte charatcers, BTW. Is there a risk you'll see false slashes,
colons and things like that in paths, if you don't parse the UTF-8
properly? (There isn't IIRC, but I'll have to read up on this.)
[...]
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
--- http://olofson.net --- http://www.reologica.se ---