[LAD] Replaygain for video?

[Robin Gareus]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hoi Philipp,

On 12/09/2011 09:20 PM, Philipp wrote:
> Hi there,
> I could use some advise.
> 
> You may or may not heard of replaygain. It's reasonably widely used in
> consumer audio, but sometimes I wish it was available for video as well.
> By this I mean I wish it was available for the audio part of the video.
> 
> Well, I need a programming project for a university course and this is
> just one of my ideas that I want to propose to my teacher and
> prospective teammates. In order to do this I'd like to narrow it down a
> bit further and especially want to find out whether I have the right
> idea of how it can be achieved.
> 
> Scanning/tagging
> Since replaygain works on whole audio files I think I need to extract
> the whole audio track from the container. How easily this can be
> achieved I don't know. 

#ffmpeg -i YOUR_VIDEO_FILE -vn -acodec pcm_f32le -f wav /tmp/file.wav
#sndfile-info /tmp/file.wav # prints max gain.

Actually, you just need to decode the audio-track, not extract it; Alas
sndfile-info only calculates the peak-value for seekable audio files not
pipes.

https://github.com/avuserow/ffmpeg-replaygain seems to do what you want
but requires ffmpeg av* libs <=0.5 and/or a bit of patching.

> After that, the scanning process should work as
> with any audio file. Afterwards the calculated replaygain values have to
> be added to the metadata of the file. I have no idea how hard it is to
> add new metadata fields to video formats.

Theoretically easy. Practically it depends on the video-codec,
container-format and library support to re-write the meta-data in-place.
Worst case: you'll have to create a new file, copy the codecs, but
rewrite the container.

Note that you may need to calculate and store the gain for _each_
audio-track. Many films include two or more tracks: e.g. stereo and 5.1.
# ffmpeg-options -ac <channel-count> -map <audio-track>


I don't know any command-line apps to modify meta-data on a per track
basis in-place (maybe ffmpeg can do that?), but the new liav* meta-data
API is pretty nifty.


> Playback
> Video players need to be aware of those tags, read the metadata and scale
> the playback volume accordingly. This is probably not hard per se, but
> there are many players out there. 

Spot on. Have you checked #replaygain-video on irc.freenode.net ?
Might be dead by now.. http://forum.xbmc.org/showthread.php?t=39320

> However, I plan to start with a single
> player, even with a single file format, and go from there.
> 
> Question 1: Is there anything better than replaygain that should be used
> instead?

Yes. the big knob on your PA labeled "Volume" :)


I don't think it'll be particularly useful to have replay gain for movies.

A compressor plugin (for video-players that don't yet have one) would be
much more useful in the real world because
 - purists won't care for neither replaygain nor compression
 - consumers will complain that replay-gain is not sufficient for
   long[er] films and won't care about sound-degradation by compression


> Question 2: Which player would be easiest to hack to add such
> functionality? Could it be a gstreamer plugin? mplayer?

gstreamer has all the building blocks (audio-gain, meta-data parser);
for a quick/dirty prototype.

But adding a gain-factor is trivial in every player [that has already
mete-data parsing]. The most difficult will be to find an entry-point
and the place where to add the gain scaling code. esp. in larger
projects like mplayer, VLC or totem.

Unless you want this to remain a school project, check for
"easiest player to get it included upstream"; that'll indirectly make it
easy to hack :)

> Question 3: How much work would it be?
> The project should be done in C++ if possible, otherwise C. Group size
> 2-4 Students, all rather new at C/C++ and rather inexperienced in
> general.

rather inexperienced -> /build one to throw [it] away/.
Re-build it with then experienced students :)

On a scale from 1(trivial) to 10 (very hard). I'd place it on 6.
One one side there's a lot of code that can be re-used (libreplaygain,
ffmpeg,..) yet their documentation is very compact for experts. On the
other side it requires diligence and some maths knowledge. Integration
in desktop-enviroments also needs some digging (e.g. hooks on mime-types
for file-manager integration, trigger re-scan on demand, in the
background,..)

It's a great educational project with many problems that one will
encounter in other /real-world/ projects.

For an experienced coder the analyzer, meta-data re-write, mplayer gain
filter code would be less than a day of work.
Documentation, [social] interaction with upstream, maintenance, etc can
easily take over a year.

> Other ideas I have are in short:
> - A CLI (using readline) connection manager for jack audio/midi and alsa
> midi that can handle large numbers of ports. 

Did you try http://www.akjmusic.com/software/jackctl20110317.py ??!

> More detailed ideas exist
> thanks to Julien Claasen.
> 
> - A simple but hopefully sane mplayer GUI

RFTL.

http://www.mplayerosx.ch/ :(

> - A new GUI for ecasound
> 
> Another problem I might have is that most students in the course are
> Windows users, not sure whether I can go solo.

You can x-compile for windows and test with wine. ffmpeg is x-platform,
as is mplayer and gstreamer. ..just be prepared to throw in a day or two
to set up a x-platform build-environment.

> Thanks for any advice,
> regards,
> Philipp
> 
> _______________________________________________
> Linux-audio-dev mailing list
> Linux-audio-dev at lists.linuxaudio.org
> http://lists.linuxaudio.org/listinfo/linux-audio-dev
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk7jbAMACgkQeVUk8U+VK0InZwCgwgy6m+9gd9XQ1nKmYP34XFyd
jVYAnje6txWjyoe1hICfCzlHyeFOrSFy
=Y0id
-----END PGP SIGNATURE-----