[LAD] "enhanced event port" LV2 extension proposal

[Krzysztof Foltman]

Dave Robillard wrote:
> What is the point of having two separate event definitions when one will
> do for both, though?
>   
Just efficiency. 8 bytes instead of 16 for empty messages. 16 bytes 
instead of 32 for MIDI messages. Sounds less stupid than just "saving 
two bytes" as you picture it.

And speaking about translation - of all the timing-related work required 
to translate between different event protocols (beat:tick, frame-based, 
picosecond-based), different bit layout is the least problem you'll be 
dealing with! :) Plus, for us C++ folks, it would probably be nicely 
solved with some sort of class template ;)

Perhaps, if we go 64-bit (although I still think it's overkill!) it 
might make sense to make a timestamp a sort of an union, so some event 
types will use one int64_t member (say, 200 picosecond units) instead of 
integer+fractional? Just cosmetics.

So we'd have something like:

struct LV2_EVENT_HEADER {
  union {
    int64_t timestamp64; // for event streams that use one large number 
as timestamp, like OSC???
    struct {
      int32_t timestamp;
      int32_t timestamp_fract;
    };
  };
  uint32_t type; // don't see any other choice but int, opaque pointer 
won't fit on x86-64, plus, index is all we need, given a good URI 
mapping mechanism
  uint32_t size;
};

With 16-byte granularity for payload (ie. size is rounded up to nearest 
multiple of 16 to determine next header address). Or perhaps 8-byte, 
winning some memory at cost of messier code.

Assuming the compiler and language used accepts anonymous structs and 
unions. Otherwise, too many dots ;)

And that probably is what we need, it looks OKish for most intended 
uses. Not optimal for *all* uses, but good enough for my purposes, and 
the generic-efficient-easy tradeoff is OK to me.
> Why?  The cons are obvious, what are the pros?  A few bits?
>   
Using a small structure when a small structure is just fine. When you 
have lots of events, this may be very important. In other situations, no 
clear advantages of 8-byte struct over 16-byte.
> A byte here and a byte there in the header makes no difference to any of
> this.  We should try to keep it small as possible, yes, but it doesn't
> affect what the using code looks like at all.
>   
Well, let me try to rephrase it, because I sense a huge miscommunication 
here.

When payload is always aligned to header size (8 bytes in my case), the 
loop can look just like this:

for (size_t i = 0; i < count; i += (events[i].size+7) >> 3) {
   // use events[i] here, cast to event type struct if needed
}

or (events[i].size+15)>>4 with 16 byte header

Short, simple, efficient.

How would the same code look like if header was 12 bytes long? (assuming 
we want to guarantee 8 byte alignment for 64-bit pointers or int64_t 
values - I think it's important because, again, on some platforms, 
misaligned access = SIGBUS)
I can only think of some ugly multiple pointer casting crap, can you 
give a clean example? Other than sweeping the ugliness under the carpet 
by closing the pointer arithmetic in some sort of standard macros :)

We can always use 16 bytes, but then payload is 16 bytes too. Which is 
15 bytes and not just 2 bytes wasted. Per (short) event. Some of those 
15 bytes (if not all) will probably end up in cache anyway, because 
cache is not byte-addressable (I don't remember the addressing 
granularity, 32, 64 bytes?), and will be wasted. Again, not a huge deal, 
except people who do granular synthesis will probably curse me at this 
point.
> Well... good thing you don't have to use those stamps for the 'others'.
> Again, I'm not proposing anyone use OSC style stamps in place of frame
> stamps...
>   
So we end up with two-three event stream types, each using different 
timing scheme. Not as bad as it sounds, certainly, because each of these 
types will be used for different things.

What I proposed was to use different event structure layout for those 
event streams. As I said above, the differences between those streams 
are so huge, that translation between 8-byte and 16-byte headers will be 
the least of the problems :)
> I think being directly compatible with Jack, with higher resolution,
> while also capable of being an absolute or tempo based time stamp is a
> pretty clear winner when the only downside is 4 bytes.  16:16
> fixed /decreases/ the stamp range compared to Jack's by a factor of 2^16
> remember.  That's a big decrease.
>   
1. 16 bytes, not 4. Unless you want char * casting ugliness and 
misalignment problems.
2. When exactly does JACK use buffers of 65536 samples and more? Not 
very often, I guess. It's half a second or more of latency. Multiplied 
by number of buffers.
> somewhere is not quite a convincing enough argument for me to be happy
> dealing with 5 different event structs (and all the translation) instead
> of 1 ;)
>   
2 structs, not 5. And the translation will have to be there anyway, 
unless you expect sample-based plugins to read OSC-style timestamps.
> Bit... odd.  Sure, saves 2 bytes, but at the cost of throwing out that
> OSC stamp compatibility (which I guarantee will be actually useful).
> Plus... well, 2 bytes.  Recentish chips can keep a few million of them
> in cache. :)
>   
By saving 2 bytes, it saves 8-16 bytes. Magic, isn't it? :)
> You don't always need to memcpy it.  Arbitrary restrictions are neither
> wise or necessary in the spec.  uint32_t is a good limit for size (it's
> used in both Jack MIDI and OSC, so everything matches)
>   
Typically you need to memcpy it at least once, in host - the data must 
get into the flat buffer somehow, right? (data of this size aren't 
usually produced ad-hoc in-place?)

The "JACK does it" is not a winning argument to me - the fact somebody 
arbitrarily picked int32_t (perhaps just because there's usually no 
point in passing parameters to functions which are less than 32 bits 
wide, because they get passed as 32 bits anyway, and using fields <32 
bits in structures leads to alignment troubles) doesn't mean that you 
have to follow that choice.

If you want to store OSC messages transparently, though, 32 bits would 
be nice.

Krzysztof