Re: [linux-audio-dev] Plugin APIs (again)

On Sunday 08 December 2002 06.00, Tim Hockin wrote: [...] I totally agree with this. I'd just like to add that, depending on the API design, Ports might come in four kinds total: * Audio Input Ports * Audio Output Ports * Event Input Ports * Event Output Ports Of course. MASTER, PART and VOICE controls are completely separate things when it comes to numbers, valid ranges, internal handling and all that. You could say that's the "relationship" between them. As to "control struct", I'm not sure what you mean. The way Audiality is designed, there is only one struct - the event struct - which is used for all communication through the event system. Inside that "protocol", where only the linked list and timestamp stuff is strictly defined, various parts of the engine define their own, higher level protocols, such as; * Low level voices (ie "oscillators") with these events: VE_START Start playing waveform <arg1>. VE_STOP Stop and free voice. VE_SET Set control <index> to <arg1>. VE_ISET Set interpolated control <index> to <arg1>. VE_IRAMP Ramp interpolated control <index> to <arg1> over <arg2> frames. VE_SET has one set of controls... VC_WAVE Waveform index (latched on loop) VC_LOOP looping on/off VC_PITCH linear frequency VC_RETRIG, retrig after N samples VC_RANDTRIG random mod. to RETRIG VC_PRIM_BUS target bus for ACC_VOLUME VC_SEND_BUS target bus for ACC_SEND VC_RESAMPLE Resampling mode while VE_ISET and VE_IRAMP have their own set: VIC_LVOL Left volume VIC_RVOL Right volume VIC_LSEND Left send VIC_RSEND Right send There's not conflict and no extra conditionals; just two sets of controls, each with their own event(s). (BTW, note that VC_WAVE and VIC_LVOL share the same actual enum value - but that's irrelevant, as they're never used as arguments to the same event type.) Now, this is hardwired stuff, but it's no different with a "real" API. Just have one set of events for each class of controls. Obviously, that applies to the "negotiation" API as well; you need separate calls, or an extra argument to tell the control classes apart. Something like that. I'd like to use an enum for "control kind" (corresponding to those "standard" MIDI CCs), and some hints and stuff to tell the host about range, linearity, absolute limits, sensible default/"no effect" value and that sort of stuff. Or we could use strings, but I *definitely* prefer enums, since those can be checked at compile time. You can still have two-way binary compatibility. Just have a default case "I don't know what this is" in the host. No, no MIDIisms on that level, please. It's easy enough to throw a suitable factor into the int->float conversion. Just like with LADSPA, the plugin will have to tell the host whether or not it has absolute upper and/or lower limits. In VST, the range is [0,1], period, but I think that's too restrictive. Scaling that range to whatever you need is fine, but that's not the point: Some controls just don't *have* sensible absolute limits - so why force plugin designers to "invent" some? Yes, that's what I've been thinking every now and then all the time. However, it seems that the term "velocity" is generally thought of as "instantaneous velocity at the time of impact". MIDI brainwashing again; that's why I tend to think of "velocity" as a parameter rather than anything that could be a continous control... Another reason would be that if you accelerate a drum stick up to a certain velocity, it'll bounce at impact - and that doesn't map all that well to one-way communication and "don't rely on short roundtrip latency!", does it? Force feedback MIDI controllers? :-) Normally, plugins just do what you tell them. They don't change their own input controls. (Well, they *could*, but that would be pretty hairy, I think... Or maybe not. Anyone dare to try it? ;-) Continuous control...? ;-) So have I (when designing MAIA) - until I actually implemented Audiality. There *should* be a single per-instance event port - the Master Event Port - but you may optionally have one event port for each channel as well, if you have any use for per-channel events. You may well just use the Master Event Port for everything in a multichannel plugin, but then you're on your own when it comes to addressing of channels and stuff. (That would require more dimensions of addressing in each event, which means a bigger event struct, more decoding overhead etc.) It wouldn't have to. If there's only one Channel, Master and Channel events are effectively equivalent - so you won't need to tell them apart, right? Consequently, you can just have a single Master Event Port, and register your full range of events on that. You *could* have a Channel Event Port for your single channel (if you even have something like that internally), but there's not much point, unless you really have two loops in your plugin, and want events for them delivered separately. Yes, definitely. As it is in Audiality (which made me realize this) you could view each Channel as a plugin instance - and they do indeed have one event port (which implies one queue) per instance. Each Channel has a typical "MAIA style" loop: while(frames_left_to_process) { while(next_event->timestamp == now) { process_and_remove(next_event); } for(samples_left_to_next_event) { process_audio(); } } It just sits there for one buffer at a time, worrying only about it's *own* events and it's own audio, and that's it. Now, if you would merge multiple Channels into one plugin instance, it would not make much sense to squeeze all events through the same port. You would only have to split them up internally in the plugin, or you would get a very inefficient loop structure, that effectively splits buffers for the whole synth for every single event. run() (or process(), as I prefer to call it) should be called only once per buffer, and there should be only one per instance. That's the one major point with supporting multichannel at all; the *plugin* gets to decide how and when things are done for the whole set of Channels. If processing all channels in parallel is better in some places; fine, do so. Besides, multiple run()/process() calls would mean more overhead than a single call + internal looping. You can probably disregard the actual function call overhead in most cases, but plugins may have a whole lot of work to do in each call before they can actually enter the loop. Another thing to keep in mind is that a multichannel plugin may have a highly optimized master mixing/routing section hooked up to the Master Event Port. Where would you run that if you have one run()/process() call for each Channel? You would effectively end up with at the very least N+1 plugins for N Channels. If the Master section is also involved before, or in the middle of the signal frow of Channels, you'll soon have countless little plugins, communicating through Audio and Event Ports, instead of one, potentially highly optimized "monolith" plugin. No, not one per *voice*. Voices have way too short lifetime, so it just wouldn't make sense to try and track them with actual objects on the API level. They should be considered Abstract Objects, that are only addessed through a field in Voice events that are sent to the Channel Event Port. (Note that I actually do have one event port for each voice in Audiality - but that's just because it happens to be the simplest and most efficient way of controlling them. Implementation stuff.) [...] No. Let's not repeat Steinberg's mistake here as well. Yes. It could also listen to an audio track, extracting pitch information from it. You get a *range* of valid VVIDs from the host (which has to ask the plugin at some point), by asking for it, saying how many VVIDs you need. Plugins that want to reallocate internal tables to deal with the VVIDs will generally have to do this outside the RT thread, but plugins that use the tag/search approach (like Audiality as of now) don't even have to be asked; the host could maintain the VVIDs all by itself. It seems like this would be a host global matter... What would be the point in that? All you need is a way to address Voices within *one Channel* - you don't have to worry about other Plugins, or even other Channels on the same Plugin. (For table based implementations, plugins have one table per Channel, and for tag/search implementations, you just tag Voices with VVIDs *and* Channel indices.) You do this when the host tells you to connect to an event port, and what you do is actually: first_id = host->get_vvids(target_port, number_of_VVIDs); if(first_id < 0) give_up; Note that the first_id thing is actually *only* needed when the host connects multiple output ports to the same input port. Normally, it will be 0. Definitely not. That would mean at least two function calls for every VOICE_ON event... *heh* (Meanwhile, Audiality event allocation, sending and handling macros will only ever make a function call when the Event Pool is empty - and that's normally a critical error. Each common event operation is only a matter of a few CPU instructions.) The host will at some point ask the plugin, or rather *tell* the plugin how many VVIDs are needed for each Event Input port it wants to connect. Plugins may say "I don't have Voices!" (No need for VVIDs...) or "Anything that fits makes a fine VVID for me." (Full 32 bit range, no actual allocation needed. or "I need internal tables. How many VVIDs do you need?" (Host says N, and plugin allocates tables for N VVIDs, addressed as 0..N-1.) The last case would be the most interesting one, since it requires that plugins allocate memory whenever the host needs more VVIDs. That is, if plugins do this themselves using malloc() or something, you simply cannot connect to Event Ports without taking the plugin out of the RT engine - *unless* you decide on a number of VVIDs to allocate for each Channel of every plugin, right when they're instantiated. A third solution would be to have the *host* allocate memory for the tables needed, and just pass it to the plugin for initialization. Or even better (so you don't have to run the table init code in RT context), just have a Channel flag that tells the host that calling plugin->allocate_vvids() is not RT safe, but *is* thread safe. The host can safely call plugin->allocate_vvids() for a Channel that is not in use, while the plugin is running in the RT thread. [...] Yes. If the event pool is part of the host struct (which I don't even have one in Audiality - yet! *heh*), you're safe. All you have to do is put proxy ports in between when connecting plugins accross the engine threads, so you never touch ports that might be read by someone else at the same time. Getting that to work without events arriving late is less trivial, though! :-) (When you have more than one thread, plugin execution order is no longer guaranteed, obviously - unless you enforce it by having blocking or spinning synchronization at critical points in the net or something.) Well, the host's event queue is the smallest problem. Getting plugins to run properly in more than one thread is the big deal - but if that works, reply events to the host is trivial. Just put the repyl event port in the host struct as well, and have one host struct for each engine thread. //David Olofson - Programmer, Composer, Open Source Advocate .- The Return of Audiality! --------------------------------. | Free/Open Source Audio Engine for use in Games or Studio. | | RT and off-line synth. Scripting. Sample accurate timing. | `---------------------------> http://olofson.net/audiality -' .- M A I A -------------------------------------------------. | The Multimedia Application Integration Architecture | `----------------------------> http://www.linuxdj.com/maia -' --- http://olofson.net --- http://www.reologica.se ---