22 Apr
2005
22 Apr
'05
1:13 p.m.
Hi,
In Jackbeat, I want to include support for realtime pitch shifting. One
of the option is to use a quality resampler as libsamplerate's
SRC_SINC_BEST_QUALITY converter. In the future, I may add support for
libsoundtouch1 (http://sky.prohosting.com/oparviai/soundtouch/index.html).
Can I do that in the jack process callback ? Or should I span a new
thread and do the heavy job in there, ringbuffer'ing the result to the
RT thread ?
--
og
http://xung.org/jackbeat
22 Apr
22 Apr
1:48 p.m.
On Fri, Apr 22, 2005 at 05:13:47PM +0200, Olivier Guilyardi wrote:
Hi,
In Jackbeat, I want to include support for realtime pitch shifting. One
of the option is to use a quality resampler as libsamplerate's
SRC_SINC_BEST_QUALITY converter. In the future, I may add support for
libsoundtouch1 (http://sky.prohosting.com/oparviai/soundtouch/index.html).
Can I do that in the jack process callback ? Or should I span a new
thread and do the heavy job in there, ringbuffer'ing the result to the
RT thread ?
Doing heavy jobs in the process() callback is fine, as long as the CPU
load is deterministic. I this case I think it should be.
- Steve
2:23 p.m.
Steve Harris wrote:
On Fri, Apr 22, 2005 at 05:13:47PM +0200, Olivier
Guilyardi wrote:
By deterministic do you mean that the CPU time consumed by a given
process() should remain (quite) the same for each call ? Do you mean
that Jack mesures it and adapt consequently ?
Say the user starts a raw playback, and then choose to add
pitch-shifting, while playing. The CPU load would suddenly increase. Is
this bad ?
--
og
In Jackbeat, I want to include support for
realtime pitch shifting.
Can I do that in the jack process callback ? Or should I span a new
thread and do the heavy job in there, ringbuffer'ing the result to the
RT thread ?
Doing heavy jobs in the process() callback is fine, as long as the CPU
load is deterministic. I this case I think it should be.
6:33 p.m.
On Fri, Apr 22, 2005 at 06:23:51 +0200, Olivier Guilyardi wrote:
Steve Harris wrote:
I mean that the CPU load should be the same from call to call, assuming
hte user hasn't changed any settings or whatever.
On Fri, Apr 22, 2005 at 05:13:47PM +0200, Olivier
Guilyardi wrote:
By deterministic do you mean that the CPU time consumed by a given
process() should remain (quite) the same for each call ? Do you mean
that Jack mesures it and adapt consequently ? In Jackbeat, I want to include support for
realtime pitch shifting.
Can I do that in the jack process callback ? Or should I span a new
thread and do the heavy job in there, ringbuffer'ing the result to the
RT thread ?
Doing heavy jobs in the process() callback is fine, as long as the CPU
load is deterministic. I this case I think it should be.
Say the user starts a raw playback, and then choose to
add
pitch-shifting, while playing. The CPU load would suddenly increase. Is
this bad ?
No, as long as it stays at that level. Problems come when for eg. the CPU
load is very high for a few cycles, then goes low for a bit, then high
again, without any configuration changes being made.
- Steve
23 Apr
23 Apr
8:07 a.m.
Steve Harris wrote:
Ok I'm gonna do the job in process(). By the way, is there any good
reading around to understand what's involved in this ?
Thanks
--
og
Say the user
starts a raw playback, and then choose to add
pitch-shifting, while playing. The CPU load would suddenly increase. Is
this bad ?
No, as long as it stays at that level. Problems come when for eg. the CPU
load is very high for a few cycles, then goes low for a bit, then high
again, without any configuration changes being made.
22 Apr
22 Apr
3:14 p.m.
On Fri, 22 Apr 2005 16:48:32 +0100
Steve Harris <S.W.Harris(a)ecs.soton.ac.uk> wrote:
Doing heavy jobs in the process() callback is fine, as
long as the CPU
load is deterministic. I this case I think it should be.
Yep, the CPU load of the sample rate conversion process of libsamplerate
is pretty close to linearly related to the size of the blocks you are
converting.
The only mallocing etc that is done by libsamplerate is done in
src_new(), the obly freeing if memory is done in src_delete().
Erik
--
+-----------------------------------------------------------+
Erik de Castro Lopo nospam(a)mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
The National Multiple Sclerosis Society of America recently started an
advertising campaign with the slogan "MS: It's not a software company".
Seasoned IT professionals will have no trouble telling the two MS's
apart. One is a debilitating and surprisingly widespread affliction
that renders the sufferer barely able to perform the simplest task.
The other is a disease.
4:10 p.m.
Erik de Castro Lopo wrote:
On Fri, 22 Apr 2005 16:48:32 +0100
Steve Harris <S.W.Harris(a)ecs.soton.ac.uk> wrote:
I want to vary the pitch in realtime, and even to totally enable/disable
the pitch-shifting routines when needed, always in realtime, while rolling.
I'm not sure if all that's still linear. The CPU load of my process()
callback may remain constant when the user vary the pitch, but it will
suddenly increase/decrease if the user enables/disables pitch-shifting.
Is this still deterministic/linear ? If not, is spanning a new thread
the only alternative ?
--
og
Doing heavy jobs in the process() callback is
fine, as long as the CPU
load is deterministic. I this case I think it should be.
Yep, the CPU load of the sample rate conversion process of libsamplerate
is pretty close to linearly related to the size of the blocks you are
converting.
4:36 p.m.
On Fri, Apr 22, 2005 at 08:10:45PM +0200, Olivier Guilyardi wrote:
Erik de Castro Lopo wrote:
If you want to do this in realtime, this doesn't help at all. If your jack process
callback takes too long, you get kicked out. If your spawned thread doesn't keep up,
you get cracks in your sound. Neither is really an option.
The determinic running time is only a nescessary criterium, not a sufficient one. The code
still has to be fast enough to keep up.
cheers,
Christian
On Fri, 22 Apr 2005 16:48:32 +0100
Steve Harris <S.W.Harris(a)ecs.soton.ac.uk> wrote:
I want to vary the pitch in realtime, and even to totally enable/disable
the pitch-shifting routines when needed, always in realtime, while rolling.
I'm not sure if all that's still linear. The CPU load of my process()
callback may remain constant when the user vary the pitch, but it will
suddenly increase/decrease if the user enables/disables pitch-shifting.
Is this still deterministic/linear ? If not, is spanning a new thread
the only alternative ?
Doing heavy jobs in the process() callback is
fine, as long as the CPU
load is deterministic. I this case I think it should be.
Yep, the CPU load of the sample rate conversion process of libsamplerate
is pretty close to linearly related to the size of the blocks you are
converting.
23 Apr
23 Apr
3:15 a.m.
On Fri, 2005-22-04 at 20:10 +0200, Olivier Guilyardi wrote:
Erik de Castro Lopo wrote:
Yes, this is fine to do.
Having another thread doing this, and having the RT audio thread read
from it through a ringbuffer won't help - if the slave thread is too
slow, everything will go to hell anyway. There's a million reasons why
doing your audio work in another thread is an awful idea (the most
obvious being you need to /block/ the Jack thread waiting for it, which
is a no-no).
Ever used one of those programs that had Jack support grafted on after
the fact, that zombie every five minutes? That's precisely because they
do this, instead of processing audio in the callback thread in a
"pull-based" manner.
Blocking IO is satan, avoid at all costs.
-DR-
On Fri, 22 Apr 2005 16:48:32 +0100
Steve Harris <S.W.Harris(a)ecs.soton.ac.uk> wrote:
I want to vary the pitch in realtime, and even to totally enable/disable
the pitch-shifting routines when needed, always in realtime, while rolling.
I'm not sure if all that's still linear. The CPU load of my process()
callback may remain constant when the user vary the pitch, but it will
suddenly increase/decrease if the user enables/disables pitch-shifting.
Is this still deterministic/linear ? If not, is spanning a new thread
the only alternative ? Doing heavy jobs in the process() callback is
fine, as long as the CPU
load is deterministic. I this case I think it should be.
Yep, the CPU load of the sample rate conversion process of libsamplerate
is pretty close to linearly related to the size of the blocks you are
converting.
8:18 a.m.
Dave Robillard wrote:
I would never block in process(). I guess I'd try to detect if there's a
xrun, and play silence if that's the case.
I want to vary
the pitch in realtime, and even to totally enable/disable
the pitch-shifting routines when needed, always in realtime, while rolling.
I'm not sure if all that's still linear. The CPU load of my process()
callback may remain constant when the user vary the pitch, but it will
suddenly increase/decrease if the user enables/disables pitch-shifting.
Is this still deterministic/linear ? If not, is spanning a new thread
the only alternative ?
Yes, this is fine to do.
Having another thread doing this, and having the RT audio thread read
from it through a ringbuffer won't help - if the slave thread is too
slow, everything will go to hell anyway. There's a million reasons why
doing your audio work in another thread is an awful idea (the most
obvious being you need to /block/ the Jack thread waiting for it, which
is a no-no).
Ever used one of those programs that had Jack support
grafted on after
the fact, that zombie every five minutes? That's precisely because they
do this, instead of processing audio in the callback thread in a
"pull-based" manner.
Ok, so doing the real job in process() is usually the right way, as long
as you don't perform syscalls. I was confused about this, and wondering
if the process() callback wasn't just a place where to memcpy(), with
worker threads doing the real work aside. I guess this is still true, if
you need to malloc(), free(), etc..
Anyway, doing the DSP in process() makes it much easier for me as well.
I'll go that way.
Thanks
--
og
22 Apr
22 Apr
3:10 p.m.
On Fri, 22 Apr 2005 17:13:47 +0200
Olivier Guilyardi <ml(a)xung.org> wrote:
Hi,
In Jackbeat, I want to include support for realtime pitch shifting. One
of the option is to use a quality resampler as libsamplerate's
SRC_SINC_BEST_QUALITY converter.
Err, the only think you can do with libsamplerate alone that even comes
close to pitch shifting is the vary speed effect (change the playback
speed and the pitch changes).
Secondly, for any real time operation, SRC_SINC_BEST_QUALITY is complete
overkill. I would honestly suggest SRC_SINC_FASTEST.
In the future, I may add support for
libsoundtouch1 (http://sky.prohosting.com/oparviai/soundtouch/index.html).
Libsoundtouch can do constant pitch time stretching. If you combine
that with libsamplerate's vary speed capabilities then you get
pitch shifting.
Erik
--
+-----------------------------------------------------------+
Erik de Castro Lopo nospam(a)mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
"Always code as if the person who ends up maintaining your
code will be a violent psychopath who knows where you live."
-- Martin Golding
3:48 p.m.
Erik de Castro Lopo wrote:
I know that changing the playback rate is not exactly pitch-shifting,
but it's what it's called in many drummachines and trackers : increasing
the "pitch" means playing the sample faster. It's a lightweight and
oldschool solution.
If I use libsoundtouch, why should I need libsamplerate to perform
pitch-shifting ?
From the libsoundtouch's homepage :
*- Tempo* *(time-stretch)*: Changes the sound to play at faster or
slower speed than original, without affecting the sound pitch.
*- Pitch (key)* : Changes the sound pitch or key, without affecting the
sound tempo or speed.*
- Playback Rate* : Changes both the sound tempo and pitch, as if an LP
disc was played at wrong RPM rate.
The second feature seems to be able to perform what's properly called
"pitch-shifting".
About vary speed : I could use libsoundtouch instead of libsamplerate,
but I'm not sure the former can compare to the later's quality.
--
og
In Jackbeat, I
want to include support for realtime pitch shifting. One
of the option is to use a quality resampler as libsamplerate's
SRC_SINC_BEST_QUALITY converter.
Err, the only think you can do with libsamplerate alone that even comes
close to pitch shifting is the vary speed effect (change the playback
speed and the pitch changes).
Secondly, for any real time operation,
SRC_SINC_BEST_QUALITY is complete
overkill. I would honestly suggest SRC_SINC_FASTEST.
Thanks for this tip. I've been wondering about this for weeks. However I
think the best is to include a user option for this.
In the future,
I may add support for
libsoundtouch1 (http://sky.prohosting.com/oparviai/soundtouch/index.html).
Libsoundtouch can do constant pitch time stretching. If you combine
that with libsamplerate's vary speed capabilities then you get
pitch shifting.
6:45 p.m.
On Fri, 22 Apr 2005 19:48:47 +0200
Olivier Guilyardi <ml(a)xung.org> wrote:
About vary speed : I could use libsoundtouch instead
of libsamplerate,
but I'm not sure the former can compare to the later's quality.
Last time I looked, libsoundtouch used linear interpolation for
doing vary speed. Linear interpolation is evil. Here is a table
from a talk I presented a the audio minconf attached the LCA.
The table is for doing linear interpolation for sample rate
converting from 44100 to 48000. The signal is a single sine wave
at the frequencies in the left column. The right column is the
measured signal to noise ratio at the output:
333 Hz & 146.0 dB \\
666 Hz & 115.8 dB \\
1332 Hz & 103.8 dB \\
2664 Hz & 49.8 dB \\
5328 Hz & 38.7 dB \\
10656 Hz & 28.4 dB \\
21312 Hz & 19.5 dB \\
Linear interpolation is simply no damn good unless you are *positive*
the maximum frequency in your source material is less that 0.06 of
the sample rate. For any real world signal, this is simply not the
case.
Secondly, I think the pitch shifter in libsoundtouch works by doing
time stretching first and then vary speed on top of that (someone
please correct me if I am wrong). In this is the case, it might be
possible to replace the second vary speed stage with the vary
speed from libsamplerate.
Erik
--
+-----------------------------------------------------------+
Erik de Castro Lopo nospam(a)mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
"Windows was created to keep stupid people away from UNIX."
-- Tom Christiansen
23 Apr
23 Apr
8:23 a.m.
Erik de Castro Lopo wrote:
Last time I looked, libsoundtouch used linear
interpolation for
doing vary speed. Linear interpolation is evil. Here is a table
from a talk I presented a the audio minconf attached the LCA.
Linear interpolation is simply no damn good unless you are *positive*
the maximum frequency in your source material is less that 0.06 of
the sample rate. For any real world signal, this is simply not the
case.
My guess was good, then, about libsamplerate providing better quality.
Secondly, I think the pitch shifter in libsoundtouch
works by doing
time stretching first and then vary speed on top of that (someone
please correct me if I am wrong). In this is the case, it might be
possible to replace the second vary speed stage with the vary
speed from libsamplerate.
Together with a Jackbeat user, we agreed that varying the speed is
enough. We don't need (lo-fi) pitch-shifting. I'll stick with libsamplerate.
By the way, thanks for your work !
--
og
24 Apr
24 Apr
7:27 a.m.
Erik de Castro Lopo wrote:
On Fri, 22 Apr 2005 19:48:47 +0200
Olivier Guilyardi <ml(a)xung.org> wrote:
Well, I agree with you that linear interpolation is not great, but
realisticaly, compare to the artefacts created by a standart
time-stretching algorithm, it is not that significant. It all depends on
the time-stretch algorithm, and soundtouch is using a wsola algorithm.
Having implemented and tested a WSOLA algorithm myself, I would say the
sample rate convertor has almost no impact compared to the artefacts of
WSOLA.
Cheers,
David
About vary speed : I could use libsoundtouch
instead of libsamplerate,
but I'm not sure the former can compare to the later's quality.
Last time I looked, libsoundtouch used linear interpolation for
doing vary speed. Linear interpolation is evil. Here is a table
from a talk I presented a the audio minconf attached the LCA.
The table is for doing linear interpolation for sample rate
converting from 44100 to 48000. The signal is a single sine wave
at the frequencies in the left column. The right column is the
measured signal to noise ratio at the output:
333 Hz & 146.0 dB \\
666 Hz & 115.8 dB \\
1332 Hz & 103.8 dB \\
2664 Hz & 49.8 dB \\
5328 Hz & 38.7 dB \\
10656 Hz & 28.4 dB \\
21312 Hz & 19.5 dB \\
Linear interpolation is simply no damn good unless you are *positive*
the maximum frequency in your source material is less that 0.06 of
the sample rate. For any real world signal, this is simply not the
case.
S
7096
days inactive
7098
days old
linux-audio-dev@lists.linuxaudio.org
14 comments
6 participants
participants (6)
-
Christian Henz -
Cournapeau David -
Dave Robillard -
Erik de Castro Lopo -
Olivier Guilyardi -
Steve Harris