9 Dec
2011
9 Dec
'11
9:20 p.m.
Hi there,
I could use some advise.
You may or may not heard of replaygain. It's reasonably widely used in
consumer audio, but sometimes I wish it was available for video as well.
By this I mean I wish it was available for the audio part of the video.
Well, I need a programming project for a university course and this is
just one of my ideas that I want to propose to my teacher and
prospective teammates. In order to do this I'd like to narrow it down a
bit further and especially want to find out whether I have the right
idea of how it can be achieved.
Scanning/tagging
Since replaygain works on whole audio files I think I need to extract
the whole audio track from the container. How easily this can be
achieved I don't know. After that, the scanning process should work as
with any audio file. Afterwards the calculated replaygain values have to
be added to the metadata of the file. I have no idea how hard it is to
add new metadata fields to video formats.
Playback
Video players need to be aware of those tags, read the metadata and scale
the playback volume accordingly. This is probably not hard per se, but
there are many players out there. However, I plan to start with a single
player, even with a single file format, and go from there.
Question 1: Is there anything better than replaygain that should be used
instead?
Question 2: Which player would be easiest to hack to add such
functionality? Could it be a gstreamer plugin? mplayer?
Question 3: How much work would it be?
The project should be done in C++ if possible, otherwise C. Group size
2-4 Students, all rather new at C/C++ and rather inexperienced in
general.
Other ideas I have are in short:
- A CLI (using readline) connection manager for jack audio/midi and alsa
midi that can handle large numbers of ports. More detailed ideas exist
thanks to Julien Claasen.
- A simple but hopefully sane mplayer GUI
- A new GUI for ecasound
Another problem I might have is that most students in the course are
Windows users, not sure whether I can go solo.
Thanks for any advice,
regards,
Philipp
10 Dec
10 Dec
11:38 a.m.
On 12/09/11 21:20, Philipp wrote:
Hi!
Well, I need a programming project for a university
course and this is
just one of my ideas that I want to propose to my teacher and
prospective teammates. In order to do this I'd like to narrow it down a
bit further and especially want to find out whether I have the right
idea of how it can be achieved.
Let me start with discouraging you. ;) Don't do it.
Maybe you need to motivate your wish for "replaygain for audio tracks in
video files" a bit more, but basically, I see two points against it:
1. Real movies always have the right audio level, there's a universal
standard in the industry that's obeyed by mastering engineers. In
other words, replaygain is not needed here.
I'm only quoting Bob Katz on this, I've never mastered a Hollywood
production. ;)
2. Players like VLC have a normalize function. I don't know if it's
one-pass (on the fly, more like automated gain control) or two-pass,
but it basically solves your problem of audio levels at playback
time.
Given that (2) is universal wrt input formats, I see little to no
additional value in your proposed project.
Scanning/tagging Since replaygain works on whole audio
files I think I
need to extract the whole audio track from the container.
Demuxing, yes.
How easily this can be achieved I don't know.
There are special tools. If nothing helps, vlc and mplayer can do this.
mplayer with -dumpvideo and -dumpaudio, vlc with the transcode commands.
After that, the scanning process should work as with
any audio file.
Afterwards the calculated replaygain values have to be added to the
metadata of the file.
Provided that the audio track supports meta data. Depending on the
container, you can embed almost everything from pcm to ogg, mp3, mp4/aac
and so on.
Depending on the container, it might be possible to add the meta data to
the muxed file.
Playback Video players need to be aware of those tags,
read the
metadata and scale the playback volume accordingly. This is probably
See, this is where your problem starts. Too many formats, too many
players. This alone should be reason enough to move on and do something
useful instead.
That is why normalizing in the player's output chain (see (2) above) or
starting with decent levels (1) is the right approach.
Other ideas I have are in short: - A CLI (using
readline) connection
manager for jack audio/midi and alsa midi that can handle large
numbers of ports. More detailed ideas exist thanks to Julien Claasen.
Hmm, IIRC, we have at least one oomidi user (sorry, forgot the name, a
guy from Russia) who's making heavy use of this.
- A simple but hopefully sane mplayer GUI
smplayer? I don't know if mplayer already supports DVD menus. That's
something you could work on.
In any case:
http://www.mplayerhq.hu/design7/projects.html
Scroll down to "frontends".
Another problem I might have is that most students in
the course are
Windows users, not sure whether I can go solo.
Create them an image to be used with virtualbox. ;)
Cheers
1:10 p.m.
Excerpts from Adrian Knoth's message of 2011-12-10 11:38:03 +0100:
Ah, I didn't think of adding metadata to the audio itself, another
possibility. However, adding it to the container would probably be more
universal.
Yes, this is the major issue, an uphill battle. Replaygain fights it
since ten years. I realize that the result of the project would not be a
practical solution but rather a proof of concept or a first step.
I mostly use smplayer, but I'm not exactly happy with it although it's
pretty much the official frontend at the moment.
Here's an audio related bug report I filed more than a year ago that got
no response:
http://sourceforge.net/tracker/?func=detail&aid=3052419&group_id=18…
Even more annoying is that the volume suddenly increases after a while,
apparently once a certain threshold is reached.
I want to write something that doesn't have bugs like those and that has
sane file and playlist management.
That's a rather good idea, thanks.
If you have a good idea, please tell me. I'll have to find a team on
monday and it might help to have some good proposals.
Regards,
Philipp
On 12/09/11 21:20, Philipp wrote:
Hi!
Anyway, thanks for helping :)
Well, I need a programming project for a
university course and this is
just one of my ideas that I want to propose to my teacher and
prospective teammates. In order to do this I'd like to narrow it down a
bit further and especially want to find out whether I have the right
idea of how it can be achieved.
Let me start with discouraging you. ;) Don't do it. Maybe you need to motivate your wish for
"replaygain for audio tracks in
video files" a bit more, but basically, I see two points against it:
1. Real movies always have the right audio level, there's a universal
standard in the industry that's obeyed by mastering engineers. In
other words, replaygain is not needed here.
Counter argument:
most videos these days aren't real movies but amateur productions
off youtube or whatnot. Admittedly most people watch those
in-browser using flash. Either way, I would argue that
professionally produced movies are a minority these days.
I'm only quoting Bob Katz on this, I've
never mastered a Hollywood
production. ;)
2. Players like VLC have a normalize function. I don't know if it's
one-pass (on the fly, more like automated gain control) or two-pass,
but it basically solves your problem of audio levels at playback
time.
Counter argument:
If normalization would be sufficient, there wouldn't have been a
need for replaygain in the first place. Explanation on
http://www.replaygain.org/.
Given that (2) is universal wrt input formats, I see little to no
additional value in your proposed project.
Right, this should do. Is there an API for that? Either way, I think the
calculation could be done on the dumped audio and the dumped audio file
could be deleted immediately afterwards, provided it's possibly to add
tags without re-encoding or re-muxing.
Scanning/tagging Since replaygain works on whole
audio files I think I
need to extract the whole audio track from the container.
Demuxing, yes.
How easily this can be achieved I don't know.
There are special tools. If nothing helps, vlc and mplayer can do this.
mplayer with -dumpvideo and -dumpaudio, vlc with the transcode commands. After that,
the scanning process should work as with any audio file.
Afterwards the calculated replaygain values have to be added to the
metadata of the file.
Provided that the audio track supports meta data. Depending on the
container, you can embed almost everything from pcm to ogg, mp3, mp4/aac
and so on.
Depending on the container, it might be possible to add the meta data to
the muxed file. Playback Video
players need to be aware of those tags, read the
metadata and scale the playback volume accordingly. This is probably
See, this is where your problem starts. Too many formats, too many
players. This alone should be reason enough to move on and do something
useful instead. That is why normalizing in the player's output
chain (see (2) above) or
starting with decent levels (1) is the right approach.
Alex Stone, I guess :)
I discussed connection management with him a long while ago, the idea to
write something that can handle hundreds of ports well stems from there.
Other ideas I have are in short: - A CLI (using
readline) connection
manager for jack audio/midi and alsa midi that can handle large
numbers of ports. More detailed ideas exist thanks to Julien Claasen.
Hmm, IIRC, we have at least one oomidi user (sorry, forgot the name, a
guy from Russia) who's making heavy use of this. - A simple but
hopefully sane mplayer GUI
smplayer? I don't know if mplayer already supports DVD menus. That's
something you could work on. In any case:
http://www.mplayerhq.hu/design7/projects.html
Scroll down to "frontends".
Many of those are dead links and even the reasonably popular frontends
are rather bad (gnome-mplayer, which doesn't depend on gnome and has
lots of really stupid interface decisions that make it unusable).
But yes, chances are I'd just add another dead link.
Another
problem I might have is that most students in the course are
Windows users, not sure whether I can go solo.
Create them an image to be used with virtualbox. ;)
3:01 p.m.
On Sat, Dec 10, 2011 at 01:10:38PM +0100, Philipp Überbacher wrote:
Huh? I cannot find anything on this website that says "normalization is
not sufficient":
"Although music is encoded to a digital format with a clearly defined
maximum peak amplitude, and although most recordings are normalized
to utilize this peak amplitude, not all recordings sound equally loud.
This is because once this peak amplitude is reached, perceived loudness
can be further increased through signal-processing techniques such as
dynamic range compression and equalization"
I have to admit I'm not too familiar with the details of replaygain, but
I'm not aware that it actually does compression or equalization. In
contrast, they explicitly state:
"The player reads the corresponding gain metadata value from the file
and scales the audio data as appropriate. Scaling the audio data simply
means multiplying each sample value by a constant value."
That's gain, nothing more, nothing less.
According to the wiki, they store four values:
1. Peak track amplitude
2. Peak album amplitude
3. Track replay gain
4. Album replay gain
We can directly forget about the album values, since we're talking
videos here.
Let's assume we're brave and normalize to 0dBFS, then this is obviously
the resulting peak track amplitude.
Asking for additional replay gain would simply cause distortion.
Long story short: only the peak track amplitude could be a useful
information if you don't want to apply automatic gain control or
read the entire file every time you play it just to determine the
correct gain for normalization.
Sure, libavformat from ffmpeg or libav, whatever you prefer.
Ah, I didn't think of adding metadata to the audio itself,
another
possibility. However, adding it to the container would probably be more
universal.
It's probably not that simple. If the container doesn't provide the
possibility to add sane metadata, you'd be lost. Likewise, there might
be multiple audio streams (stereo, multichannel, different languages).
You'd need a way to relate to those substreams from within your global
metadata.
It could be easier to work on the audio streams directly, but this would
require re-muxing the file, causing subtle problems like A-V sync or the
necessity to write arbitrary containers (MPEG, MPEG-TS, MP4, ogv...).
ffmpeg/libav might help.
Either way, it's complex. ;)
[jackd connection handling]
2. Players
like VLC have a normalize function. I don't know if it's
one-pass (on the fly, more like automated gain control) or two-pass,
but it basically solves your problem of audio levels at playback
time.
Counter argument:
If normalization would be sufficient, there wouldn't have been a
need for replaygain in the first place. Explanation on
http://www.replaygain.org/. There are
special tools. If nothing helps, vlc and mplayer can do this.
mplayer with -dumpvideo and -dumpaudio, vlc with the transcode commands.
Right,
this should do. Is there an API for that? After that, the scanning process should work as with
any audio file.
Afterwards the calculated replaygain values have to be added to the
metadata of the file.
Provided that the audio track supports meta data. Depending on the
container, you can embed almost everything from pcm to ogg, mp3, mp4/aac
and so on.
Depending on the container, it might be possible to add the meta data to
the muxed file. Alex Stone, I guess :)
Exactly.
If you have a good idea, please tell me. I'll have
to find a team on
monday and it might help to have some good proposals.
I'm still looking for somebody to rewrite hdspmixer. ;) While we're at
it, how about an HTML based approach: you fire up your browser, either
have a matrix mixer for everything or can select individual output
buses, and then have the input/playback faders for this destination
only. (I have a couple of details, if you want to go this route)
Other idea: a P1722 streamer, no idea if Christoph Kuhr is still working
on that.
Or ask Paul if he needs some help with jack3. ;)
Cheers
--
mail: adi(a)thur.de http://adi.thur.de PGP/GPG: key via keyserver
3:22 p.m.
On Sat, Dec 10, 2011 at 03:01:54PM +0100, Adrian Knoth wrote:
Huh? I cannot find anything on this website that says
"normalization is
not sufficient":
Depends on what is normalised. Peak level has no relation to
loudness, so peak normalisation doesn't help. RMS comes as lot
closer, RMS after some EQ (as in EBU R-128) is even better.
Ciao,
--
FA
Vor uns liegt ein weites Tal, die Sonne scheint - ein Glitzerstrahl.
9:42 p.m.
Excerpts from Adrian Knoth's message of 2011-12-10 15:01:54 +0100:
Sure, libavformat from ffmpeg or libav, whatever you prefer.
Ok, thanks.
Ah, I didn't think of adding metadata to the audio itself,
another
possibility. However, adding it to the container would probably be more
universal.
It's probably not that simple. If the container doesn't provide the
possibility to add sane metadata, you'd be lost. Likewise, there might
be multiple audio streams (stereo, multichannel, different languages).
You'd need a way to relate to those substreams from within your global
metadata.
This should be possible, shouldn't it? The player needs to be able to
identify those anyway.
On Sat, Dec 10, 2011 at 01:10:38PM +0100, Philipp
Überbacher wrote:
Huh? I cannot find anything on this website that says "normalization is
not sufficient":
"Although music is encoded to a digital format with a clearly defined
maximum peak amplitude, and although most recordings are normalized
to utilize this peak amplitude, not all recordings sound equally loud.
This is because once this peak amplitude is reached, perceived loudness
can be further increased through signal-processing techniques such as
dynamic range compression and equalization"
This page tells you that peak normalization is not sufficient, but
that's nothing new, it's logical that the highest peak has basically no
relation to the perceived loudness of a piece of audio.
2.
Players like VLC have a normalize function. I don't know if it's
one-pass (on the fly, more like automated gain control) or two-pass,
but it basically solves your problem of audio levels at playback
time.
Counter argument:
If normalization would be sufficient, there wouldn't have been a
need for replaygain in the first place. Explanation on
http://www.replaygain.org/. I have to admit I'm not too familiar with the
details of replaygain, but
I'm not aware that it actually does compression or equalization. In
contrast, they explicitly state:
"The player reads the corresponding gain metadata value from the file
and scales the audio data as appropriate. Scaling the audio data simply
means multiplying each sample value by a constant value."
That's gain, nothing more, nothing less.
Exactly, no equalisation, no compression, that's why purists like it.
According to the wiki, they store four values:
1. Peak track amplitude
2. Peak album amplitude
3. Track replay gain
4. Album replay gain
We can directly forget about the album values, since we're talking
videos here.
I agree, album mode makes little sense unless we have music video albums
or similar.
Let's assume we're brave and normalize to
0dBFS, then this is obviously
the resulting peak track amplitude.
Asking for additional replay gain would simply cause distortion.
Long story short: only the peak track amplitude could be a useful
information if you don't want to apply automatic gain control or
read the entire file every time you play it just to determine the
correct gain for normalization.
Wrong. This is in short how (track) replaygain works:
1) Calculate the tracks loudness. Loudness implies psychoacoustics. It
uses some algorithm that takes the human perception into account.
2) Compare the measurement value to a reference (pink noise at -14 dB)
and store the difference as metadata.
3) The audio player reads the metadata and adjusts its output level
according to the metadata.
The result: levels of 'loud' songs get lowered to the reference level,
levels of 'soft' songs get raised to the reference level. No
compression, now eq, no need to twist the volume knob all the time.
Again, the reference level is below 0 dBFS, loud songs get attenuated.
Peak normalization in contrast just tries to make everything as loud as
possible and doesn't take human perception into account at all.
There are special tools. If nothing helps, vlc and
mplayer can do this.
mplayer with -dumpvideo and -dumpaudio, vlc with the transcode commands.
Right,
this should do. Is there an API for that? After that,
the scanning process should work as with any audio file.
Afterwards the calculated replaygain values have to be added to the
metadata of the file.
Provided that the audio track supports meta data. Depending on the
container, you can embed almost everything from pcm to ogg, mp3, mp4/aac
and so on.
Depending on the container, it might be possible to add the meta data to
the muxed file. It could be easier to work on the audio streams
directly, but this would
require re-muxing the file, causing subtle problems like A-V sync or the
necessity to write arbitrary containers (MPEG, MPEG-TS, MP4, ogv...).
ffmpeg/libav might help.
Either way, it's complex. ;)
Yes, especially with all those containers and formats out there.
However, I'd start with one where it's reasonably easy to do :)
[jackd connection handling]
I don't have such a mixer. The html approach is probably outside the
scope of this class, it's a beginner C/C++ class.
Alex Stone, I guess :)
Exactly.
If you have a good idea, please tell me. I'll
have to find a team on
monday and it might help to have some good proposals.
I'm still looking for somebody to rewrite hdspmixer. ;) While we're at
it, how about an HTML based approach: you fire up your browser, either
have a matrix mixer for everything or can select individual output
buses, and then have the input/playback faders for this destination
only. (I have a couple of details, if you want to go this route) Other idea: a P1722 streamer, no idea if Christoph
Kuhr is still working
on that.
I guess my teacher could like it, but I know basically nothing about
network stuff at this point.
Or ask Paul if he needs some help with jack3. ;)
Cheers
Haven't heard of that one yet and I doubt we need another implementation
:)
Thanks for your help,
regards,
Philipp
1:04 p.m.
On 12/09/2011 09:20 PM, Philipp wrote:
Question 1: Is there anything better than replaygain
that should be used
instead?
not better as such, but carrying more weight, for sure: the EBU R128
recommendation. fons has presented an implementation at lac 2011:
http://lac.linuxaudio.org/2011/papers/26.pdf
this is what all european radio stations will implement over the next
few years, and similar plans are being discussed among tv broadcasters.
there is a german forum on loudness at
http://www.tonmeister.de/forum/viewforum.php?f=34&sid=41ae6829b326641ee…
, which includes some useful links. you'll have to register to post, though.
best,
jörn
3:26 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hoi Philipp,
On 12/09/2011 09:20 PM, Philipp wrote:
Hi there,
I could use some advise.
You may or may not heard of replaygain. It's reasonably widely used in
consumer audio, but sometimes I wish it was available for video as well.
By this I mean I wish it was available for the audio part of the video.
Well, I need a programming project for a university course and this is
just one of my ideas that I want to propose to my teacher and
prospective teammates. In order to do this I'd like to narrow it down a
bit further and especially want to find out whether I have the right
idea of how it can be achieved.
Scanning/tagging
Since replaygain works on whole audio files I think I need to extract
the whole audio track from the container. How easily this can be
achieved I don't know.
#ffmpeg -i YOUR_VIDEO_FILE -vn -acodec pcm_f32le -f wav /tmp/file.wav
#sndfile-info /tmp/file.wav # prints max gain.
Actually, you just need to decode the audio-track, not extract it; Alas
sndfile-info only calculates the peak-value for seekable audio files not
pipes.
https://github.com/avuserow/ffmpeg-replaygain seems to do what you want
but requires ffmpeg av* libs <=0.5 and/or a bit of patching.
After that, the scanning process should work as
with any audio file. Afterwards the calculated replaygain values have to
be added to the metadata of the file. I have no idea how hard it is to
add new metadata fields to video formats.
Theoretically easy. Practically it depends on the video-codec,
container-format and library support to re-write the meta-data in-place.
Worst case: you'll have to create a new file, copy the codecs, but
rewrite the container.
Note that you may need to calculate and store the gain for _each_
audio-track. Many films include two or more tracks: e.g. stereo and 5.1.
# ffmpeg-options -ac <channel-count> -map <audio-track>
I don't know any command-line apps to modify meta-data on a per track
basis in-place (maybe ffmpeg can do that?), but the new liav* meta-data
API is pretty nifty.
Playback
Video players need to be aware of those tags, read the metadata and scale
the playback volume accordingly. This is probably not hard per se, but
there are many players out there.
Spot on. Have you checked #replaygain-video on irc.freenode.net ?
Might be dead by now.. http://forum.xbmc.org/showthread.php?t=39320
However, I plan to start with a single
player, even with a single file format, and go from there.
Question 1: Is there anything better than replaygain that should be used
instead?
Yes. the big knob on your PA labeled "Volume" :)
I don't think it'll be particularly useful to have replay gain for movies.
A compressor plugin (for video-players that don't yet have one) would be
much more useful in the real world because
- purists won't care for neither replaygain nor compression
- consumers will complain that replay-gain is not sufficient for
long[er] films and won't care about sound-degradation by compression
Question 2: Which player would be easiest to hack to
add such
functionality? Could it be a gstreamer plugin? mplayer?
gstreamer has all the building blocks (audio-gain, meta-data parser);
for a quick/dirty prototype.
But adding a gain-factor is trivial in every player [that has already
mete-data parsing]. The most difficult will be to find an entry-point
and the place where to add the gain scaling code. esp. in larger
projects like mplayer, VLC or totem.
Unless you want this to remain a school project, check for
"easiest player to get it included upstream"; that'll indirectly make it
easy to hack :)
Question 3: How much work would it be?
The project should be done in C++ if possible, otherwise C. Group size
2-4 Students, all rather new at C/C++ and rather inexperienced in
general.
rather inexperienced -> /build one to throw [it] away/.
Re-build it with then experienced students :)
On a scale from 1(trivial) to 10 (very hard). I'd place it on 6.
One one side there's a lot of code that can be re-used (libreplaygain,
ffmpeg,..) yet their documentation is very compact for experts. On the
other side it requires diligence and some maths knowledge. Integration
in desktop-enviroments also needs some digging (e.g. hooks on mime-types
for file-manager integration, trigger re-scan on demand, in the
background,..)
It's a great educational project with many problems that one will
encounter in other /real-world/ projects.
For an experienced coder the analyzer, meta-data re-write, mplayer gain
filter code would be less than a day of work.
Documentation, [social] interaction with upstream, maintenance, etc can
easily take over a year.
Other ideas I have are in short:
- A CLI (using readline) connection manager for jack audio/midi and alsa
midi that can handle large numbers of ports.
Did you try http://www.akjmusic.com/software/jackctl20110317.py ??!
More detailed ideas exist
thanks to Julien Claasen.
- A simple but hopefully sane mplayer GUI
RFTL.
http://www.mplayerosx.ch/ :(
- A new GUI for ecasound
Another problem I might have is that most students in the course are
Windows users, not sure whether I can go solo.
You can x-compile for windows and test with wine. ffmpeg is x-platform,
as is mplayer and gstreamer. ..just be prepared to throw in a day or two
to set up a x-platform build-environment.
Thanks for any advice,
regards,
Philipp
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
-----BEGIN
PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iEYEARECAAYFAk7jbAMACgkQeVUk8U+VK0InZwCgwgy6m+9gd9XQ1nKmYP34XFyd
jVYAnje6txWjyoe1hICfCzlHyeFOrSFy
=Y0id
-----END PGP SIGNATURE-----
11:04 p.m.
Excerpts from Robin Gareus's message of 2011-12-10 15:26:11 +0100:
Do you refer to http://www.libav.org/ ?
Yes, the channel doesn't exist.
Agreed, if purists
are people who only watch professionally produced
movies or similar. In such a case there'd be no need for it.
Yes, those are the big questions. Maybe gstreamer is the way to go as I
won't have much time.
Thank you very much for that, I fear that it might be too hard. Most
students have very little coding experience. Most courses require little
coding and that is mostly done in java. The course in question is a
beginner C/C++ course and for many students this was the first experience
with those languages. I think it would be the first real world project
too. Thanks a lot for your input, it gets increasingly clear to me that
it most likely is too hard for this project.
I sadly don't what that means :)
It looks reasonably nice, similar to smplayer, but the devil is in the
details. And this one seems to be for OSX only.
That's another possibility, thanks. I guess I'll need to do something
like this no matter what the project will be.
Thanks a lot Robin,
I'm pretty much back at square one now, but that's ok. Maybe I can
develop the ecasound GUI idea, maybe I should find a team first and try
to come up with an idea together with the others.
Regards,
Philipp
Hash: SHA1
Hoi Philipp,
On 12/09/2011 09:20 PM, Philipp wrote:
Thanks, guess I'd need the whole track anyway before making
calculations.
Hi there,
I could use some advise.
You may or may not heard of replaygain. It's reasonably widely used in
consumer audio, but sometimes I wish it was available for video as well.
By this I mean I wish it was available for the audio part of the video.
Well, I need a programming project for a university course and this is
just one of my ideas that I want to propose to my teacher and
prospective teammates. In order to do this I'd like to narrow it down a
bit further and especially want to find out whether I have the right
idea of how it can be achieved.
Scanning/tagging
Since replaygain works on whole audio files I think I need to extract
the whole audio track from the container. How easily this can be
achieved I don't know.
#ffmpeg -i YOUR_VIDEO_FILE -vn -acodec pcm_f32le -f wav /tmp/file.wav
#sndfile-info /tmp/file.wav # prints max gain.
Actually, you just need to decode the audio-track, not extract it; Alas
sndfile-info only calculates the peak-value for seekable audio files not
pipes. https://github.com/avuserow/ffmpeg-replaygain seems to
do what you want
but requires ffmpeg av* libs <=0.5 and/or a bit of patching.
My ffmpeg is rather recent (20111123) but the program doesn't build, so
I guess it needs patching. Seems like my ffmpeg is too new. The
implementation looks insanely complex to me.
After that,
the scanning process should work as
with any audio file. Afterwards the calculated replaygain values have to
be added to the metadata of the file. I have no idea how hard it is to
add new metadata fields to video formats.
Theoretically easy. Practically it depends on the video-codec,
container-format and library support to re-write the meta-data in-place.
Worst case: you'll have to create a new file, copy the codecs, but
rewrite the container.
Note that you may need to calculate and store the gain for _each_
audio-track. Many films include two or more tracks: e.g. stereo and 5.1.
# ffmpeg-options -ac <channel-count> -map <audio-track>
I don't know any command-line apps to modify meta-data on a per track
basis in-place (maybe ffmpeg can do that?), but the new liav* meta-data
API is pretty nifty. Playback
Video players need to be aware of those tags, read the metadata and scale
the playback volume accordingly. This is probably not hard per se, but
there are many players out there.
Spot on. Have you checked #replaygain-video on irc.freenode.net ?
Might be dead by now.. http://forum.xbmc.org/showthread.php?t=39320 However, I
plan to start with a single
player, even with a single file format, and go from there.
Question 1: Is there anything better than replaygain that should be used
instead?
Yes. the big knob on your PA labeled "Volume" :)
I don't think it'll be particularly useful to have replay gain for movies.
A compressor plugin (for video-players that don't yet have one) would be
much more useful in the real world because
- purists won't care for neither replaygain nor compression - consumers will complain that replay-gain is not
sufficient for
long[er] films and won't care about sound-degradation by compression
Why would it not be sufficient for longer films? It can't do anything to
improve bad mixing, that's true, but there's not much one can do about
that anyway (except maybe apply some eq to make dialogs understandable).
Question 2:
Which player would be easiest to hack to add such
functionality? Could it be a gstreamer plugin? mplayer?
gstreamer has all the building blocks (audio-gain, meta-data parser);
for a quick/dirty prototype.
But adding a gain-factor is trivial in every player [that has already
mete-data parsing]. The most difficult will be to find an entry-point
and the place where to add the gain scaling code. esp. in larger
projects like mplayer, VLC or totem.
Unless you want this to remain a school project, check for
"easiest player to get it included upstream"; that'll indirectly make it
easy to hack :) Question 3:
How much work would it be?
The project should be done in C++ if possible, otherwise C. Group size
2-4 Students, all rather new at C/C++ and rather inexperienced in
general.
rather inexperienced -> /build one to throw [it] away/.
Re-build it with then experienced students :)
On a scale from 1(trivial) to 10 (very hard). I'd place it on 6.
One one side there's a lot of code that can be re-used (libreplaygain,
ffmpeg,..) yet their documentation is very compact for experts. On the
other side it requires diligence and some maths knowledge. Integration
in desktop-enviroments also needs some digging (e.g. hooks on mime-types
for file-manager integration, trigger re-scan on demand, in the
background,..)
It's a great educational project with many problems that one will
encounter in other /real-world/ projects. For an experienced coder the analyzer, meta-data
re-write, mplayer gain
filter code would be less than a day of work.
Documentation, [social] interaction with upstream, maintenance, etc can
easily take over a year.
I thought I knew them all...
Thanks, this is great and pretty close to what I/we imagined. I don't
fully understand the alsa midi mode, I think it doesn't use a pager for
many ports and I guess a way to connect a range of ports to another
would be useful as well, but it's really close to what I had in mind.
Julien will be happy to hear about this.
That pretty much takes care of this idea :)
Other ideas I have are in short:
- A CLI (using readline) connection manager for jack audio/midi and alsa
midi that can handle large numbers of ports.
Did you try http://www.akjmusic.com/software/jackctl20110317.py ??! More detailed
ideas exist
thanks to Julien Claasen.
- A simple but hopefully sane mplayer GUI
RFTL. - A new GUI
for ecasound
Another problem I might have is that most students in the course are
Windows users, not sure whether I can go solo.
You can x-compile for windows and test with wine. ffmpeg is x-platform,
as is mplayer and gstreamer. ..just be prepared to throw in a day or two
to set up a x-platform build-environment.
11 Dec
11 Dec
12:49 p.m.
On 12/10/2011 11:04 PM, Philipp Überbacher wrote:
[..]
edit ffs.c
replace "CODEC_TYPE_AUDIO" with "AVMEDIA_TYPE_AUDIO" and you are good
to
go with ffmpeg >0.5.
For ffmpeg <0.5, replace the 'avcodec_decode_audio3() call with:
int len = avcodec_decode_audio2(ffs->cc, (int16_t *) (buf + rdec),
&size, tmppkt.data, tmppkt.size)
[..]
Sorry, a typo slipped in. Yes, libavformat, libavcodec, libavutil, etc..
I prefer libav* from ffmpeg.org not the branch libav.org.
The single loud explosion (show down) or a slammed door will dwarf all
the all whispered (sexy) dialog.
I just ran `fbff` (ffmpeg replay gain) on a The Great Dictator (1940),
Clockwork Orange (1971) and Avatar(2009). as well as
http://www.youtube.com/watch?v=f2bcPIXl8kc
They _all_ came out on zero (Max PCM value found: 1.000000, Title gain:
64.820000). Looks like they have been normalized already. (maybe youtube
and DVD master[er]s do that?!) But the dynamics are completely different.
For comparison, ebur128 reported:
Loundness-range 19.1 LU; Peak 6.7 LU for the Chaplin movie.
and
Loundness-range 11.3 LU; Peak 15.3 LU for the youtube video,
(Quite good actually, RHCP's 'Around the world' only features a
Loundness-range 4.0 LU; Peak. 19.2.)
[..]
If you want to learn hands-on CS, don't study Informatics.
Two years ago I taught the "advanced programming" class at the
Kunst-universität Linz. 3rd semester *Art* school: > 1/3 of the
students were fluent with openframeworks & openCV C++. The rest knew
either JAVA/processing.org or Pure-data. I was pleasantly surprised.
[..]
I think Aaron is on the LAD list. IIRC he announced it on LAU and LAA
and may be willing to add a pager or other features. - just change the
subject-line :)
I sadly don't what that means :) AWW snap. ROFL.
meaning: Mission impossible.
"Die Idee ist gut, die Welt noch nicht bereit." (Tocotronic)
guess I'd need the whole track anyway before
making
calculations.
Yes, but not all of it at the same time. You can calculate it sequentially.
Have a look at https://www.ohloh.net/p/Samplecat it includes a ffmpeg
audio decoder, max. gain calculation, Fons' ebur128 analysis and more..
Comment out the mime-type filter in main.c:788 to read the first audio
track from a video file.
https://github.com/avuserow/ffmpeg-replaygain seems to do what you want
but requires ffmpeg av* libs <=0.5 and/or a bit of patching.
My ffmpeg is rather recent (20111123) but the program doesn't build, so
I guess it needs patching. Seems like my ffmpeg is too new. The
implementation looks insanely complex to me. I don't
know any command-line apps to modify meta-data on a per track
basis in-place (maybe ffmpeg can do that?), but the new liav* meta-data
API is pretty nifty.
Do you refer to http://www.libav.org/ ? >> Playback
[..]
A compressor
plugin (for video-players that don't yet have one) would be
much more useful in the real world because
- purists won't care for neither replaygain nor compression
Agreed, if
purists are people who only watch professionally produced
movies or similar. In such a case there'd be no need for it.
- consumers will complain that replay-gain is
not sufficient for
long[er] films and won't care about sound-degradation by compression
Why would it not be sufficient for longer films? It can't do anything to
improve bad mixing, that's true, but there's not much one can do about
that anyway (except maybe apply some eq to make dialogs understandable). It's a
great educational project with many problems that one will
encounter in other /real-world/ projects.
Thank you very much for that, I fear that it might be too hard. Most
students have very little coding experience. Most courses require little
coding and that is mostly done in java.
The course in question is a
beginner C/C++ course and for many students this was the first experience
with those languages. Did you try
http://www.akjmusic.com/software/jackctl20110317.py ??!
I thought I knew them all...
Thanks, this is great and pretty close to what I/we imagined. I don't
fully understand the alsa midi mode, I think it doesn't use a pager for
many ports and I guess a way to connect a range of ports to another
would be useful as well, but it's really close to what I had in mind.
Julien will be happy to hear about this.
That pretty much takes care of this idea :) More detailed ideas exist
thanks to Julien Claasen.
- A simple but hopefully sane mplayer GUI
RFTL.
It looks reasonably nice, similar to smplayer, but the devil is in the
details. And this one seems to be for OSX only.
That's why I'm sad.
>> - A new GUI for ecasound
>>
>> Another problem I might have is that most students in the course are
>> Windows users, not sure whether I can go solo.
>
> You can x-compile for windows and test with wine. ffmpeg is x-platform,
> as is mplayer and gstreamer. ..just be prepared to throw in a day or two
> to set up a x-platform build-environment.
install mingw32 and wine. As long as a project uses autoconf:
#export WINEBASEDIR=$HOME/.wine/drive_c/x-prefix
#export PKG_CONFIG_PATH=$WINEBASEDIR/lib/pkgconfig/
#./configure --host=i586-mingw32msvc --build=i386-linux \
--prefix=$HOME/.wine/drive_c/x-prefix/
#make install
Problem is: for ffmpeg you'll need a lot of time consuming dependencies.
http://nsis.sf.net is the packager/installer tool of choice.
but I'm getting more off-topic.
That's another possibility, thanks.
I guess I'll need to do something
like this no matter what the project will be.
qemu or virtualbox? is nice for running or testing apps; but IMO
developing in a VM - esp windows sucks much more than setting up a
x-compile environment.
Thanks a lot Robin,
I'm pretty much back at square one now, but that's ok. Maybe I can
develop the ecasound GUI idea, maybe I should find a team first and try
to come up with an idea together with the others.
I know that a /real wold use-case/ greatly excites students. Just you
trade that for /real world experience/. The project is superb and IMHO
not too complex:
The /lazy/ student could just write a batch script:
call `ffmpeg.exe` - parse output.., call `replaygain.exe` - parse output
call `ffmpeg.exe -acodec copy -vcodec copy -metadata replaygain=X ..`
and finish while you're still struggling with a gst-launch pipeline...
Those students who are into C/C++ will at least manage to glue existing
code and may learn about the basic mathematical complexities involved:
bit-depth, logarithms..
The rich student will outsource it, and the creative student just
delivers a remote-control for manual Volume control..
How many movies are you going to watch in your life vs. how long does it
take to adjust the volume?
robin
1:19 p.m.
On Sun, Dec 11, 2011 at 12:49:10PM +0100, Robin Gareus wrote:
For comparison, ebur128 reported:
Loundness-range 19.1 LU; Peak 6.7 LU for the Chaplin movie.
and
Loundness-range 11.3 LU; Peak 15.3 LU for the youtube video,
?? ebur128 does not report anything called 'Peak' in LU.
The relevant values are the first two: 'Integrated loudness'
and 'Loudness range'. The first should determine any gain
correction, the second can be used to decide if some com-
pression would help.
The last two, 'Momentary max' and 'Short term max' (I guess
one of these is what you call 'Peak') are for information
only and should not be used for the purpose discussed in
this thread.
Updates of ebur128 will show only the first two, unless
you explicitly ask for the others as well.
Ciao,
--
FA
Vor uns liegt ein weites Tal, die Sonne scheint - ein Glitzerstrahl.
2:27 p.m.
On 12/11/2011 01:19 PM, Fons Adriaensen wrote:
On Sun, Dec 11, 2011 at 12:49:10PM +0100, Robin Gareus
wrote:
No it does not. "Momentary max" did not fit in the line w/o line break
in my mailer :)
For comparison, ebur128 reported:
Loundness-range 19.1 LU; Peak 6.7 LU for the Chaplin movie.
and
Loundness-range 11.3 LU; Peak 15.3 LU for the youtube video,
?? ebur128 does not report anything called 'Peak' in LU. The relevant values are the first two: 'Integrated
loudness'
and 'Loudness range'. The first should determine any gain
correction, the second can be used to decide if some com-
pression would help.
The last two, 'Momentary max' and 'Short term max' (I guess
one of these is what you call 'Peak') are for information
only and should not be used for the purpose discussed in
this thread.
Thanks for the clarification. So let me correct that:
The Great dictator DVD has an
Integrated Loudness of 4.1 LU, Loudness Range 19.1 LU.
and 'Music for one appart. and 6 drummers' from you-tube:
Integrated Loudness: 12.9 LU, Loudness Range 11.3 LU
Both are already normalized to 0 dbFS.
Checking some more, revealed that you-tube does not actually normalize
audio volume automatically: e.g.
http://www.youtube.com/watch?v=hJMEvDu0K2E peaks at -16.5 dbFS
Integrated Loudness: -1.7 LU, Loudness Range 9.9 LU
Updates of ebur128 will show only the first two,
unless
you explicitly ask for the others as well.
Ciao,
Cheers!
robin
24 Dec
24 Dec
1:11 p.m.
Quoting Robin Gareus (2011-12-11 14:27:35)
On 12/11/2011 01:19 PM, Fons Adriaensen wrote:
Thanks everyone,
sorry for the late answer, I had a combination of email client trouble
with Christmas stress among other things.
It would be very nice if at some point youtube would use such a system.
It's rather common today that people use youtube playlists instead of
local music. This could at least solve the problem of different volume.
Anyway, I won't be implementing replaygain at this time, but I still
think it's worth doing. Maybe I'll pick up the idea some other time or
someone else does.
For the project I decided to go with the ecasound GUI. The plan is to
write the GUI using JUCE. The minimal set of goals is, well, minimal,
but I have a few ideas, I'll see where to got from there. No idea how
usable it will become. Ecasound sure could use a GUI other than the Tk
thing nama provides. Imho ecasound has also become more interesting
with the recent addition of LV2 support.
Thanks everyone for sharing your thoughts,
I wish you happy holidays.
Regards,
Philipp
On Sun, Dec 11, 2011 at 12:49:10PM +0100, Robin
Gareus wrote:
No it does not. "Momentary max" did not fit in the line w/o line break
in my mailer :)
For comparison, ebur128 reported:
Loundness-range 19.1 LU; Peak 6.7 LU for the Chaplin movie.
and
Loundness-range 11.3 LU; Peak 15.3 LU for the youtube video,
?? ebur128 does not report anything called 'Peak' in LU. The relevant values are the first two:
'Integrated loudness'
and 'Loudness range'. The first should determine any gain
correction, the second can be used to decide if some com-
pression would help.
The last two, 'Momentary max' and 'Short term max' (I guess
one of these is what you call 'Peak') are for information
only and should not be used for the purpose discussed in
this thread.
Thanks for the clarification. So let me correct that:
The Great dictator DVD has an
Integrated Loudness of 4.1 LU, Loudness Range 19.1 LU.
and 'Music for one appart. and 6 drummers' from you-tube:
Integrated Loudness: 12.9 LU, Loudness Range 11.3 LU
Both are already normalized to 0 dbFS.
Checking some more, revealed that you-tube does not actually normalize
audio volume automatically: e.g.
http://www.youtube.com/watch?v=hJMEvDu0K2E peaks at -16.5 dbFS
Integrated Loudness: -1.7 LU, Loudness Range 9.9 LU
Updates of ebur128 will show only the first two,
unless
you explicitly ask for the others as well.
Ciao,
Cheers!
robin
4506
days inactive
4521
days old
linux-audio-dev@lists.linuxaudio.org
12 comments
6 participants
participants (6)
-
Adrian Knoth -
Fons Adriaensen -
Jörn Nettingsmeier -
Philipp -
Philipp Überbacher
-
Robin Gareus