Re: [LAD] LV2 implementer poll (event buffer)

On Mon, 2012-02-06 at 18:25 +0100, Albert Graef wrote: (I'll do the "why", and the "what's new" separately. If you're only interested in what's coming, skip to the latter) = Why Event Needs Replacing = This stems from the uri-map extension being an oops. Many things in LV2 depend on mapping URIs to integers (called URIDs), which gives us decentralized extensibility, human/machine readable documentation, etc., but with the performance of simple integers. It's a pretty fundamental idea to LV2 moving beyond LADSPA, used everywhere. A host-provided URI mapping function is used for this, e.g. uint32_t map_uri(const char* uri); One use of this is the type field in event structs. Events are generic blobs with an int type and size, the type can be anything (e.g. midi:Event, which simply means the payload is normalized MIDI bytes). Because events are small, we wanted to shrink this header, and chose to use uint16_t for type and size (surely nobody is going to invent 65535 event types!). However, for URIs in general, that limit is too small, even within the scope of just LV2. So, a context argument was added to uri-map to allow specifying different contexts for URIs, one of which was "event" which had the restriction that the range is uint16_t, e.g.: uint32_t map_uri(const char* context_uri, const char* uri); Problem solved. Except... oops, the big problem, mapping URIs to integers, was unsolved. This is not a mechanism to map a URI to an integer, it's a mechanism to map a URI to *two* integers at best (a context and the URI). In other words, if I give you just an integer, you can't figure out which URI it is. That's a problem when we start communicating the things in wider contexts. That slippery slope results in every different place URIDs are used having a different context, and plugins and hosts having to carefully translate between them, the actual map implementation is bloated and vastly more annoying to write than a simple map, extensions to convert quickly need to be invented, etc. etc. A big mess, and it's caused by the simple oops: we didn't *actually* have a mechanism to map a URI to/from an integer. We broke it trying to save a few bytes. Duh. So, live and learn (this problem has been known/looming for a while), Gabriel M. Beddingfield submitted the "urid" extension[1] which is a simplified uri-map with no context parameter, and unmapping built in. This went stable last November. Now, event needs to go because it's using this incompatible type field. = LV2 Atoms = The replacement is a more generic thing I call an "atom": typedef struct { uint32_t type; /* Type URID. */ uint32_t size; /* Size of body in bytes. */ uint8_t data[]; /* Body. */ } LV2_Atom; This is intended to be used wherever LV2 stuff needs to talk data, including but beyond MIDI-like uses. The atom extension defines primitive types like Int32, Int64, Float, Double, String, etc. and POD collections like Tuple, Vector, and Object (i.e. a dictionary with URID keys). These are all POD. Defining new binary atom types is discouraged unless necessary; everything uses these basic types to describe whatever they need. This keeps everything transparent. Thus, implementations need only to implement these types (to e.g. communicate between processes/machines, or serialise) and, voila, full power. atom:Object is elegantly described in Turtle (among other things), and is intended to give us the higher-level "messages" we so badly need. For example, a message to set the filename and mode of a plugin could be (in Turtle, how you would think of it): [ a foo:SetMessage ; bar:filename "/whatever.wav" ; bar:mode bar:backwards ; ] At runtime this would be a single POD chunk with 5 ints, the string, and some atom headers. Since the logistics of working with atoms (copying, serialising, whatever) is simple and generic, plugin/host authors are free to describe whatever they want (e.g. defining new message types), without being held back by other implementations having to keep up. Object types (e.g. foo:SetMessage above) can be defined/documented the same way everything else is (i.e. in Turtle), our doc tools can generate the same fancy resolvable documentation for them, it's straightforward to describe what you support in a plugin's data file, etc. IMO, this decoupling of a plugin author's ability to innovate from host or other plugin authors ability to keep up is crucial. We definitely want to avoid every different little thing being a different C struct which you only understand if you actively support whatever defined it, that would really cripple progress. I envision working with this stuff being sort of "Pythoney", for lack of a better term. If you receive a <whatever> from <wherever>, you can pretty-print it out to see what it is, save it for later, fire it to/from the UI via ports, etc. More of a high-level feel than a maze of different little C structs, all alike. I am not sure if any of this will make sense to anyone else, but there's that. I think it will be pretty awesome, anyway. :) -dr [1] http://lv2plug.in/ns/ext/urid P.S. Of course, synths and such will probably just stick to using MIDI atoms for existing MIDI uses, which is expected and fine. The door is wide open for playing with something nicer, though. Individual note control, perhaps? Go wild.