Re: [LAD] "enhanced event port" LV2 extension proposal

Dave Robillard wrote: It *is* extensible. Note that commands 0x00-0x6F and 0x73-0x7F are unused, so further extensions are free to define them (perhaps we need a scheme for binding extension URIs to command numbers, to make it more LV2-ey). And 0x72 command can be used for pretty much any data larger than 8+2 octets. While it's "efficient first" and not "generic first", so to speak, it should be fine for the intended uses. I think it's not just "not a bad one". The other possibility (multiple event ports) is less efficient, and speed is crucial here. It's also more complex, looking from plugin author's perspective. So I had little choice. Please don't jump to conclusions, and take more time to read and analyse the proposal. Of course, it is possible to add new event types with arbitrary length data, and the limitation of 8 octets per extended block is not that bad, because you can always fit an interface pointer (32-bit or 64-bit) there. Just look how binary data extension is implemented. Notice that I just took the approach of optimizing for most common case (MIDI events), and tried to maximize functionality while keeping block size small and constant (to avoid pointer arithmetic that was complicating Lars' proposal a bit). Well, sometimes you need to find the right tradeoff between being efficient (memory- and speed-wise) and generic. I think I've found an acceptable tradeoff (definitely favouring speed, but not losing generality and not very memory-hungry). However, I had to make some assumptions about how it will be used (mostly implemented by inexperienced people, mostly used for MIDI and float parameters, seldom used for advanced stuff). Oh well, I'm repeating myself here :) I think those are correct assumptions, but you seem to have a different angle for looking at those things. Well, it took me years (and failed/inadequate designs) to grow out of the "everything should be as generic as possible" approach, so I understand why you're doing that, but I still prefer the priority-based optimization approach that I've used. I still think my proposal could be improved, and I don't like some decisions that I made (basically, I made them because the alternatives looked even more nasty), but stripping off optimizations is not the way to go, IMO. You're suggesting a "classic" textbook chunked data approach, which works, no doubt. However, it has some problems with it, which might not be considered very major, but seem to make my approach slightly more favourable: - too much data to be accessed in the most common use case (in 32-bit environment, 16 bytes of header plus event data possibly in distant memory); we don't need to save every byte of RAM, but when you need to read and write twice as much RAM as you could, then maybe it's worth rethinking it - separation of event header and event data in the most common case; it would be better not to cause cache thrashing too much - it encourages memory fragmentation (experienced people will allocate event data for all events in the same buffer, wonder about inexperienced ones, one malloc per event data? :) ) - it doesn't deal with large data properly (because the plugin cannot start "owning" the raw event data instead of copying it from the buffer provided); imagine copying a video buffer in the process() function of a plugin! I'm not saying that approach is Really Bad - just that it's kind of a pre-optimization version of my proposal (I made MIDI data very efficient, float parameter data slightly less efficient, float parameter data with deltas even less efficient, and binary data are pretty inefficient :) ). The fact that event has to be handled is annoying enough on its own :) - I have to end the inner loop, store state information somewhere etc. - I don't want some additional, unnecessary memory accesses which may throw sample data and buffers out of the cache. In network context, yes. However, _optimizing_ for uncommon case is not a preferable approach to me. The arbitrary binary data command (0x72) mentioned in my proposal can give you practically everything you need, and can be used in a network-transparent way, as long as data in the "binary data" chunks are self-contained (don't refer to other buffers). However, my proposal lacks any mechanism to be used for serializing arguments of future commands defined by extensions. Still, that problem was solved many times in history, by deriving extra interfaces for the new commands from an interface that provides reference counting and marshalling. IPersistStream type stuff, for the victims of Microsoft APIs. It is a bit complex (or at least not as trivial as plain MIDI), but it would be only used in hosts and complex plugins that use extension events, so I guess it'd be fine. On the other hand, "100% generic" means "almost 100% unoptimized". By throwing away extra information, you often throw away the chances for optimization, so to speak. Instead of thinking in terms of MIDI vs non-MIDI, try thinking of "my" event types as "short" (8 octets), "medium" (16 octets) and "large" (arbitrary-sized blobs). The fact that all short events are MIDI events is, I think, less important. It's also not set in stone. On the other hand, it deals with a trivial problem, and solves it in a complex way. That's not an engineer's dream :) Regards, Krzysztof