[LAU] sample preview program?
nielsmayer at gmail.com
Tue Aug 17 22:12:12 UTC 2010
On Wed, Aug 4, 2010 at 11:21 PM, Niels Mayer <nielsmayer at gmail.com> wrote:
> It's works very nice and all, and it's good to know I can build a jack
> app this easily... so how would you go about adding a
> spectrogram/waveform display for the samples you display, e.g.
> http://www.mega-nerd.com/libsndfile/tools/#spectrogram . And then
> using that as a visual reference, be able to zoom and drag out regions
> on-top of the waveform or spectrogram.
FYI, the Vala-based spectrogram display, and better decoding options
for sound files is here:
This one talks to gstreamer and ffmpeg and from my quick tests, seems
blazing fast and more functional compared to similar programs using
python ( http://soundviewer.svn.sourceforge.net/viewvc/soundviewer/svt.py?view=markup&pathrev=1
) and http://www.ar.media.kyoto-u.ac.jp/members/david/softwares/audiolab/
(and I can code in a decent/modern language instead of Python).
http://code.google.com/p/spek/issues/detail?id=40 for spek-0.6
configuration/compilation issues and solution for Fedora12)
.... use FFmpeg libraries to decode audio files. There are several
reasons for the switch from GStreamer:
GStreamer is a fantastic framework for building complex multimedia
pipelines, however what Spek really needs is a simple decoder and
FFmpeg’s libavformat and libavcodec do just that.
To handle some audio formats (e.g. APE and DTS), GStreamer relies on
FFmpeg anyway, so the switch will result in lesser dependencies. It
doesn’t matter too much on GNU/Linux, but this will reduce the size of
the Windows and Mac OS X installers.
Spek used GStreamer’s spectrum plugin to perform the actual spectral
analysis, with FFmpeg I had to implement it myself. The code I ended
up with is very compact and gives room for a lot of experimentation,
from using different window functions (it’s still Hamming) and working
on performance optimisations to switching to a faster FFT library.
The last bit is actually done, Spek now uses FFTW which in my tests is
1.5x to 2x faster than Kiss FFT used by GStreamer. Apart from that,
FFTW can scale to multiple threads with near linear performance
increase, future versions of Spek will take advantage of this.
UPDATE: As one of commenters pointed out, FFTs on small number of
samples are not very parallelisable and my benchmarks confirm this.
Also, I switched from FFTW to avfft which is built into FFmpeg. It’s a
little bit faster than FFTW for my particular use case. Lastly, 1.5x
to 2x speed up was actually caused by a faster decoder, not by a
faster FFT library.
Another thing that would be hard with GStreamer is static
cross-compilation using mingw-cross-env to produce a single Windows
executable. Because FFmpeg doesn’t employ a plugin architecture,
static linking is not an issue.
Last, but not least, the whole experience was very educational. I now
remember why I loved C back in the days, the trick is to not even
attempt to write any GObject code with it, that’s what Vala is for.
Fast Fourier Transform and the maths behind it is much fun, and NR was
very helpful here.
For the curious, the code is merged to git master and pushed to
Gitorious. The next version of Spek will be released sometime in July.
More information about the Linux-audio-user