[LAD] "enhanced event port" LV2 extension proposal

[Dave Robillard]

On Wed, 2007-11-28 at 10:47 +0000, Krzysztof Foltman wrote:
> Dave Robillard wrote:
> 
> > "extensible" like SYSEX.  It isn't 1980 anymore...
> 
> Which doesn't mean we need to waste CPU cycles on comparing URIs in
> event processing loop :)

Again, your optimisation claims are baseless.  Please stop making them
10 times per mail ;)

> I already gave you one example how it can be handled beyond what SysEx
> offers (by using interfaces). And another (by binding command numbers to
> URIs, preferably on plugin load). Read my previous mail more carefully.
> 
> By the way, how old is IP? It is old, it is crufty, still, it won with
> competing protocols for some reasons :)

IP doesn't dictate packet contents whatsoever.  It does what it needs to
do quite well, I don't think it's crufy at all.  It would be crufty if
it tried to define, say HTTP error codes...

> > A stamped chunk of "some sort of event" is simple to understand,
> > guaranteed 100% extensible and generic without any weird sysexey
> > 1980'sness,
> 
> Are you sure you are solving right problems here? Look at what purpose
> is it meant to serve.

Yep.

> > Your efficiency claims make no sense, frankly.  Put everything in a flat
> > buffer and voila, it's exactly how MIDI events work right now.  Your
> > proposal has no inherent performance advantage over a more obvious and
> > cleaner one.
> 
> I've already said why it is more efficient. Please give statements that
> refer to details, not just vague "make no sense". For example, tell me
> why reading more data from DRAM to cache has no impact on performance in
> your opinion.

Because, for the nth time, it's a flat buffer.

Please look at the definition of the LV2_MIDI struct in lv2-midiport.h.
Note the char* parameter, and read it's comment.  The buffer is flat,
there is no cache thrashing or any such issues here whatsoever.

Take that struct definition, remove the stuff about what sort of event
goes in the data part (which is literally in comments alone), and voila
- generic events.  If you have a good reason to mess up the header part
with event specific things, speak up, but efficiency is not one.

The only problem that needs to be handled is how to get the type in
there.  I would like to find a good solution to this problem that's as
extensible as URIs but doesn't actually stick a URI in the event struct
(there are a few other future extensions that have the same problem.
strcmp of URIs in the audio thread is, as you say, completely out of the
question, but so is handing out a flat numeric space.  This is /the/
problem that needs solving here, and I'm desperately trying to guide the
conversation in a direction that will get it solved nicely ;)

> I think we make a serious mistake of not trying to sketch out some code
> for reading a buffer, writing a buffer, handling binary data etc.
> 
> > Exactly.  You are making assumptions about what people want to put in
> > there.  This is a Bad Thing(TM).  If it can be avoided, it should be
> > (and it can definitely be avoided).
> 
> I should be avoided, but not at cost of efficiency.

Agreed.  There is no efficiency hit.  QED.

> > too much is on the table to be figured out at one time (how far would
> > the Internet have come if every high level protocol was defined in the
> > IP specification?)
> 
> That's a good argument... almost. But there are counterarguments:
> 
> - whenever you add more layers, you decrease efficiency

Not true in this case, since it's not really a 'layer'.  There is ZERO
performance penalty.  See LV2_MIDI struct.

> > Your optimisation comments (the bulk of your email) are based on a
> > misunderstanding.  I don't mean the data pointer to point somewhere else
> > in memory, it's a flat buffer (ie almost identical to how MIDI works
> > right now).
> 
> You didn't make it possible for the MIDI data to follow the event
> directly (well, you could say that if data pointer is equal to the first
> byte after event header, then size+padding should be added to header
> pointer, but that would decrease the elegance of your proposal).

Ding ding ding!  You figured it out; that is indeed the case.  Lars
didn't write that big helpful comment for nothing. ;)

Padding is an issue, as is contant time event indexing.  I have dealt
with these things a bit in my LV2 OSC experiments (I have working OSC
patching in Ingen BTW).

Now that we're on the same page about the flat buffer thing, I think we
should ignore padding and indexing and such and focus on the important
bit.  We have this:

/* (LV2_MIDI with the comments stripped out) */
typedef struct {
  uint32_t  event_count;
  uint32_t  capacity;
  uint32_t  size;
  unsigned char* data;
} LV2_EVENTS_BUFFERS;

and data is filled (flatly!) with something like this:

typedef struct {
  timestamp_t time; /* again ignoring type/size nitpickeys */
  uint32_t    size; /* size of data (__which directly follows__) */
  char*       data; /* data */
} LV2_EVENT;

We need to stick a type in the LV2_EVENT bit.  How?

* URI is no good.  Infinitely extensible, but too slow.
* Number is no good.  Not extensible, requires central authority.

So, we need some kind of (DYNAMIC!) URI<->number 'negotiation' scheme or
something.  Almost certainly will have something to do with the LV2
"feature" mechanism.  Thinking caps on.......

Cheers,

-Dave