[LAD] LV2 implementer poll (event buffer)

[David Robillard]

On Mon, 2012-02-06 at 18:25 +0100, Albert Graef wrote:
> On 02/06/2012 12:43 AM, David Robillard wrote:
> > If anyone has any other gripes/suggestions for the event extension,
> > now's the time to make them heard.  Speak now or forever hold your
> > peace, etc.
> 
> AFAICT I don't use any of the items that you mentioned.
> 
> > P.S. I have avoided the "why does event need to be replaced" part to
> > keep it to the point, but if anyone is curious, I can explain
> 
> I'm curious. :)

(I'll do the "why", and the "what's new" separately.  If you're only
interested in what's coming, skip to the latter)

= Why Event Needs Replacing =

This stems from the uri-map extension being an oops.

Many things in LV2 depend on mapping URIs to integers (called URIDs),
which gives us decentralized extensibility, human/machine readable
documentation, etc., but with the performance of simple integers.  It's
a pretty fundamental idea to LV2 moving beyond LADSPA, used everywhere.

A host-provided URI mapping function is used for this, e.g.

uint32_t map_uri(const char* uri);

One use of this is the type field in event structs.  Events are generic
blobs with an int type and size, the type can be anything (e.g.
midi:Event, which simply means the payload is normalized MIDI bytes).

Because events are small, we wanted to shrink this header, and chose to
use uint16_t for type and size (surely nobody is going to invent 65535
event types!).  However, for URIs in general, that limit is too small,
even within the scope of just LV2.  So, a context argument was added to
uri-map to allow specifying different contexts for URIs, one of which
was "event" which had the restriction that the range is uint16_t, e.g.:

uint32_t map_uri(const char* context_uri, const char* uri);

Problem solved.  Except... oops, the big problem, mapping URIs to
integers, was unsolved.  This is not a mechanism to map a URI to an
integer, it's a mechanism to map a URI to *two* integers at best (a
context and the URI).  In other words, if I give you just an integer,
you can't figure out which URI it is.  That's a problem when we start
communicating the things in wider contexts.

That slippery slope results in every different place URIDs are used
having a different context, and plugins and hosts having to carefully
translate between them, the actual map implementation is bloated and
vastly more annoying to write than a simple map, extensions to convert
quickly need to be invented, etc. etc.  A big mess, and it's caused by
the simple oops: we didn't *actually* have a mechanism to map a URI
to/from an integer.  We broke it trying to save a few bytes.  Duh.

So, live and learn (this problem has been known/looming for a while),
Gabriel M. Beddingfield submitted the "urid" extension[1] which is a
simplified uri-map with no context parameter, and unmapping built in.
This went stable last November.

Now, event needs to go because it's using this incompatible type field.

= LV2 Atoms =

The replacement is a more generic thing I call an "atom":

typedef struct {
    uint32_t type;    /* Type URID. */
    uint32_t size;    /* Size of body in bytes. */
    uint8_t  data[];  /* Body. */
} LV2_Atom;

This is intended to be used wherever LV2 stuff needs to talk data,
including but beyond MIDI-like uses.  The atom extension defines
primitive types like Int32, Int64, Float, Double, String, etc. and POD
collections like Tuple, Vector, and Object (i.e. a dictionary with URID
keys).  These are all POD.

Defining new binary atom types is discouraged unless necessary;
everything uses these basic types to describe whatever they need.  This
keeps everything transparent.  Thus, implementations need only to
implement these types (to e.g. communicate between processes/machines,
or serialise) and, voila, full power.  atom:Object is elegantly
described in Turtle (among other things), and is intended to give us the
higher-level "messages" we so badly need.  For example, a message to set
the filename and mode of a plugin could be (in Turtle, how you would
think of it):

[
    a            foo:SetMessage ;
    bar:filename "/whatever.wav" ;
    bar:mode     bar:backwards ;
]

At runtime this would be a single POD chunk with 5 ints, the string, and
some atom headers.

Since the logistics of working with atoms (copying, serialising,
whatever) is simple and generic, plugin/host authors are free to
describe whatever they want (e.g. defining new message types), without
being held back by other implementations having to keep up.

Object types (e.g. foo:SetMessage above) can be defined/documented the
same way everything else is (i.e. in Turtle), our doc tools can generate
the same fancy resolvable documentation for them, it's straightforward
to describe what you support in a plugin's data file, etc.

IMO, this decoupling of a plugin author's ability to innovate from host
or other plugin authors ability to keep up is crucial.  We definitely
want to avoid every different little thing being a different C struct
which you only understand if you actively support whatever defined it,
that would really cripple progress.  I envision working with this stuff
being sort of "Pythoney", for lack of a better term.  If you receive a
<whatever> from <wherever>, you can pretty-print it out to see what it
is, save it for later, fire it to/from the UI via ports, etc.  More of a
high-level feel than a maze of different little C structs, all alike.

I am not sure if any of this will make sense to anyone else, but there's
that.  I think it will be pretty awesome, anyway.  :)

-dr

[1] http://lv2plug.in/ns/ext/urid

P.S. Of course, synths and such will probably just stick to using MIDI
atoms for existing MIDI uses, which is expected and fine.  The door is
wide open for playing with something nicer, though.  Individual note
control, perhaps?  Go wild.