30 Oct
2003
30 Oct
'03
8:58 p.m.
Hi,
i was thinking about latencies of softsynths and how cubase vst handles
this when playing back recorded midi tracks as opposed to playing the
softsynth directly. I'm talking about the vst-softsynths here. not the
ones that get controlled via usual midi ports.
It seems that during playback of a prerecorded midi track cubase vst
knows the latency of the softsynth and and arranges for that so that the
softsynth is really tight [no offset to other recorded audio material].
This is, of course, only possible because the midi signal if routed
vst-internally is handled different than midi that gets sent out.
I suppose that todays available audio/midi-sequencer support midi in a
form that it is synchronized to the latency of the audio signal that is
sent out. So, if the audio signal takes 2*128/44100s [2 buffers a 128
frames] to reach the ear of the user, then the prerecorded midi signal
is delayed this exact amount of time, because midi signals is supposed
to be of zero latency. And for external hardware synthesizers this is
pretty much true. This way prerecorded audio and midi tracks are nicly
synchronised during playback.
Now, for the case of a sofstsynth this scenario doesn't fit, because the
synth is not zero latency. This means that a "tight" midi track is
audible with the softsynths output latency.
Now, Jack is really an audio server but it could also be used to
communicate the latency of the softsynth to the Audio/Midi Sequencer. I
know there's calls to get the output latency of physical output ports.
But it would be nice to have calls to ask for the physical output
latency of clients [this value can be different for different clients -
maybe with different output devices].
This way, the audio/midi-sequencer could ask jack for a list of clients
and offer this choice to the user who then can select which midi tracks
correspond to which softsynth. The Sequencer could then send the midi
events a tad bit earlier [the output latency of the softsynth].
What do you think? I have no insight into the implementation details of
jack, so i don't know how possible this scenario is and if it fits with
design decisions.
Regards
Florian Schmidt
30 Oct
30 Oct
10:10 p.m.
Now, for the case of a sofstsynth this scenario
doesn't fit, because the
synth is not zero latency. This means that a "tight" midi track is
audible with the softsynths output latency.
most softsynths are zero latency. only those that work in the
frequency domain tend to have non-zero latency. by this i mean that
the synth will generate output for a given time period at the same
time as processing any input it receives (which may be nothing at all)
for the same time period.
Now, Jack is really an audio server but it could also
be used to
communicate the latency of the softsynth to the Audio/Midi Sequencer. I
know there's calls to get the output latency of physical output ports.
But it would be nice to have calls to ask for the physical output
latency of clients [this value can be different for different clients -
maybe with different output devices].
the jack API supports this for every port of every client. its nothing
to do with particular classes of ports. it *is* up to the client to
set the port latency correctly.
This way, the audio/midi-sequencer could ask jack for a
list of clients
and offer this choice to the user who then can select which midi tracks
correspond to which softsynth. The Sequencer could then send the midi
events a tad bit earlier [the output latency of the softsynth].
ardour does exactly this. it delays all tracks by various amounts so
that they sync up with the most latent/delayed one. this includes
internal delays caused by plugins, and external delays due to signal
routing.
What do you think? I have no insight into the
implementation details of
jack, so i don't know how possible this scenario is and if it fits with
design decisions.
its already done :)
--p
10:55 p.m.
On Thu, 30 Oct 2003 21:09:46 -0500
Paul Davis <paul(a)linuxaudiosystems.com> wrote:
most softsynths are zero latency. only those that work
in the
frequency domain tend to have non-zero latency. by this i mean that
the synth will generate output for a given time period at the same
time as processing any input it receives (which may be nothing at all)
for the same time period.
Yes, but this output has to pass through a buffer and that introduces
delay. With filters in the frequency domain you mean stuff like fourier
and wavelet analysis where you have to have a certain windowsize to get
low frequencies.. In these cases you would have two latencies which add
up. the filter/softsynth latency and the output latency.
In the case that the softsynth is connected to a track in ardour,
there's also a possible delay introduced for this [softsynth
output]->[ardour track input] - connection (i don't know the details)? I
suppose, this is also compensated for? In this case handling the output
latency would be trivial because after entering the track in ardour the
signal has the same latency as the other tracks from ardour (well,
exccept for those that go through other plugins/jackclients inside and
outside of ardour, but this is handled, too, i suppose).
ardour does exactly this. it delays all tracks by
various amounts so
that they sync up with the most latent/delayed one. this includes
internal delays caused by plugins, and external delays due to signal
routing.
This is so cool :) I have to try that and find a midi sequencer that can
act as jack transport slave..
its already done :)
11:07 p.m.
most
softsynths are zero latency. only those that work in the
frequency domain tend to have non-zero latency. by this i mean that
the synth will generate output for a given time period at the same
time as processing any input it receives (which may be nothing at all)
for the same time period.
Yes, but this output has to pass through a buffer and that introduces
delay. and wavelet analysis where you have to have a certain
windowsize to get
low frequencies.. In these cases you would have two latencies which add
up. the filter/softsynth latency and the output latency.
see above. the client sets its own port
latency. jack_port_get_gotal_latency() will sum the latencies
between the port and physical port.
In the case that the softsynth is connected to a track
in ardour,
there's also a possible delay introduced for this [softsynth
output]->[ardour track input] - connection (i don't know the details)? I
suppose, this is also compensated for? In this case handling the output
there are no delays. any data generated by the softsynth on cycle N is
processed by ardour on cycle N. the results are output on cycle N.
ardour does
exactly this. it delays all tracks by various amounts so
that they sync up with the most latent/delayed one. this includes
internal delays caused by plugins, and external delays due to signal
routing.
This is so cool :) I have to try that and find a midi sequencer that can
act as jack transport slave..
7687
days inactive
7687
days old
linux-audio-dev@lists.linuxaudio.org
3 comments
2 participants
participants (2)
-
Florian Schmidt -
Paul Davis