[LAD] Replaygain for video?

[Philipp Überbacher]

Excerpts from Adrian Knoth's message of 2011-12-10 11:38:03 +0100:
> On 12/09/11 21:20, Philipp wrote:
> 
> Hi!
> 
> > Well, I need a programming project for a university course and this is
> > just one of my ideas that I want to propose to my teacher and
> > prospective teammates. In order to do this I'd like to narrow it down a
> > bit further and especially want to find out whether I have the right
> > idea of how it can be achieved.
> 
> Let me start with discouraging you. ;) Don't do it.

Anyway, thanks for helping :)

> Maybe you need to motivate your wish for "replaygain for audio tracks in
> video files" a bit more, but basically, I see two points against it:
> 
>    1. Real movies always have the right audio level, there's a universal
>       standard in the industry that's obeyed by mastering engineers. In
>       other words, replaygain is not needed here.

Counter argument:
    most videos these days aren't real movies but amateur productions
    off youtube or whatnot. Admittedly most people watch those
    in-browser using flash. Either way, I would argue that
    professionally produced movies are a minority these days.

>       I'm only quoting Bob Katz on this, I've never mastered a Hollywood
>       production. ;)
> 
>    2. Players like VLC have a normalize function. I don't know if it's
>       one-pass (on the fly, more like automated gain control) or two-pass,
>       but it basically solves your problem of audio levels at playback
>       time.

Counter argument:
    If normalization would be sufficient, there wouldn't have been a
    need for replaygain in the first place. Explanation on
    http://www.replaygain.org/.

> 
> Given that (2) is universal wrt input formats, I see little to no
> additional value in your proposed project.
> 
> > Scanning/tagging Since replaygain works on whole audio files I think I
> > need to extract the whole audio track from the container.
> 
> Demuxing, yes.
> 
> > How easily this can be achieved I don't know.
> 
> There are special tools. If nothing helps, vlc and mplayer can do this.
> mplayer with -dumpvideo and -dumpaudio, vlc with the transcode commands.

Right, this should do. Is there an API for that? Either way, I think the
calculation could be done on the dumped audio and the dumped audio file
could be deleted immediately afterwards, provided it's possibly to add
tags without re-encoding or re-muxing.

> > After that, the scanning process should work as with any audio file.
> > Afterwards the calculated replaygain values have to be added to the
> > metadata of the file.
> 
> Provided that the audio track supports meta data. Depending on the
> container, you can embed almost everything from pcm to ogg, mp3, mp4/aac
> and so on.
>
> Depending on the container, it might be possible to add the meta data to
> the muxed file.

Ah, I didn't think of adding metadata to the audio itself, another
possibility. However, adding it to the container would probably be more
universal.

> > Playback Video players need to be aware of those tags, read the
> > metadata and scale the playback volume accordingly. This is probably
> 
> See, this is where your problem starts. Too many formats, too many
> players. This alone should be reason enough to move on and do something
> useful instead.

Yes, this is the major issue, an uphill battle. Replaygain fights it
since ten years. I realize that the result of the project would not be a
practical solution but rather a proof of concept or a first step.

> That is why normalizing in the player's output chain (see (2) above) or
> starting with decent levels (1) is the right approach.
> 
> > Other ideas I have are in short: - A CLI (using readline) connection
> > manager for jack audio/midi and alsa midi that can handle large
> > numbers of ports. More detailed ideas exist thanks to Julien Claasen.
> 
> Hmm, IIRC, we have at least one oomidi user (sorry, forgot the name, a
> guy from Russia) who's making heavy use of this.

Alex Stone, I guess :)
I discussed connection management with him a long while ago, the idea to
write something that can handle hundreds of ports well stems from there.

> > - A simple but hopefully sane mplayer GUI
> 
> smplayer? I don't know if mplayer already supports DVD menus. That's
> something you could work on.

I mostly use smplayer, but I'm not exactly happy with it although it's
pretty much the official frontend at the moment.
Here's an audio related bug report I filed more than a year ago that got
no response:
http://sourceforge.net/tracker/?func=detail&aid=3052419&group_id=185512&atid=913573
Even more annoying is that the volume suddenly increases after a while,
apparently once a certain threshold is reached.
I want to write something that doesn't have bugs like those and that has
sane file and playlist management.

> In any case:
> 
>    http://www.mplayerhq.hu/design7/projects.html
> 
> Scroll down to "frontends".

Many of those are dead links and even the reasonably popular frontends
are rather bad (gnome-mplayer, which doesn't depend on gnome and has
lots of really stupid interface decisions that make it unusable).
But yes, chances are I'd just add another dead link.

> > Another problem I might have is that most students in the course are
> > Windows users, not sure whether I can go solo.
> 
> Create them an image to be used with virtualbox. ;)

That's a rather good idea, thanks.
If you have a good idea, please tell me. I'll have to find a team on
monday and it might help to have some good proposals.

Regards,
Philipp