[linux-audio-dev] Softsynth latency control [Jack?]

[Florian Schmidt]

Hi,

i was thinking about latencies of softsynths and how cubase vst handles
this when playing back recorded midi tracks as opposed to playing the
softsynth directly. I'm talking about the vst-softsynths here. not the
ones that get controlled via usual midi ports.

It seems that during playback of a prerecorded midi track cubase vst
knows the latency of the softsynth and and arranges for that so that the
softsynth is really tight [no offset to other recorded audio material]. 
This is, of course, only possible because the midi signal if routed
vst-internally is handled different than midi that gets sent out. 

I suppose that todays available audio/midi-sequencer support midi in a
form that it is synchronized to the latency of the audio signal that is
sent out. So, if the audio signal takes 2*128/44100s [2 buffers a 128
frames] to reach the ear of the user, then the prerecorded midi signal
is delayed this exact amount of time, because midi signals is supposed
to be of zero latency. And for external hardware synthesizers this is
pretty much true. This way prerecorded audio and midi tracks are nicly
synchronised during playback.

Now, for the case of a sofstsynth this scenario doesn't fit, because the
synth is not zero latency. This means that a "tight" midi track is
audible with the softsynths output latency.

Now, Jack is really an audio server but it could also be used to
communicate the latency of the softsynth to the Audio/Midi Sequencer. I
know there's calls to get the output latency of physical output ports.
But it would be nice to have calls to ask for the physical output
latency of clients [this value can be different for different clients -
maybe with different output devices]. 

This way, the audio/midi-sequencer could ask jack for a list of clients
and offer this choice to the user who then can select which midi tracks
correspond to which softsynth. The Sequencer could then send the midi
events a tad bit earlier [the output latency of the softsynth].

What do you think? I have no insight into the implementation details of
jack, so i don't know how possible this scenario is and if it fits with
design decisions.

Regards
Florian Schmidt