[LAU] state of multimedia

[Robin Gareus]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Patrick Shirkey wrote:
> Hi,
> 
> I have just spent the past few days working on a project to sync a badly 
> recorded a/v track.
> 
> In the process I have had to do a crash course in Linux Multimedia apps.
> 
> It's like going back 5 years compared to Linux Audio. I haven't had to 
> do this kind of compiling for quite some time with Linux Audio apps due 
> to the Fedora yum repository being upto date and the stability issues 
> for basic operations have been sorted. Most of the audio apps I use just 
> work on 64 bit for basic editing tasks.
> 
> I have used several programs in my quest for a/v sync.
> 
> Avidemux
> openmovieeditor
> kdenlive
> LiVES
> cinelerra-cv (community version)
> kino
> 
> I have also used
> 
> ffmpeg
> mencoder
> mplayer
> gmerlin
> 
> I also would have liked to try out jahshaka /cinefx and cinelerra-4  but 
> I couldn't get either installed. I ended up using avidemux and 
> openmovieeditor.
> 
> I had to install several apps from source due to various bugs that 
> happen on my dual core amd , fedora 10 x64 packaged versions. Many of 
> the apps have got Fedora 10 rpm builds but it took several hours of 
> searching google to find the links and details for installing them. Then 
> I had the headache of figuring out why libs that were compiling and 
> installing were not being found as deps. Why does Fedora not setup the 
> qt4 and pkgconfig paths by default? Has that been fixed in Fedora 11? 
> For a development install this seems like a prerequisite.
> 
> In the end after 30 hours of trying to use the various apps above to 
> realign a single audio track the best I could do was to extract the 
> audio track from the original with mplayer -dumpaudio, cut the original 
> mpg with avidemux, render (export) the new track to disk, split off the 
> audio track with mplayer, import the original audio track to audacity, 
> cut it to the right start points, export it to wav, import the new 
> tracks to openmovieeditor, align them, export them to .mov.

yay: Unix spirit: one tool per task ;)

I'm surprised that openmovieeditor failed. It works pretty stable here
for doing simple editing. kino even more so.

As for aligning and editing audio; I'd recommend ardour over audacity.
You can see the video in sync with xjadeo. Ardour can read timecode from
the broadcast-info in the audio-file's header and automatically align
chunks/regions in a session.

Your final step of multiplexing and transcoding can be done on the
commandline. eg. for a PAL-DVD with two audio-tracks:

ffmpeg -target pal-dvd -b 9000k -bt 1000k -i film.avi -acodec mp2 -ab
192k -alang en -i bounce_stereo.wav -acodec ac3 -ab 384k -ac 6 -alang en
- -newaudio -i bounce_cinema.ac3 -map 0:0 -map 1:0 -map 2:0
/tmp/master.mpg  -author xxx -title "Title" -year 2009

The devede(1) GUI can build this command for you.

> I would have preferred to use one app but none of them could do all of 
> the above or if they could they were buggy and kept crashing or couldn't 
> read the mp3 or couldn't export the file in a readable format, etc.... 
> Kdenlive looked like it would be almost perfect but it kept crashing, 
> openmovieeditor is also good but editing is limited and buggy, of them 
> all avidemux was definitely the most rock solid performer and the 
> interface was a pleasure to work with. LiVES has a nice clean interface 
> too but is not really an editor. The other apps left a lot to be desired 
> in the visual appeals dept. Why do multimedia devs insist on using the 
> gui libraries like sdl and tkinter?
> 
> BTW, why the different jargon for audio (export) and video (render) 
> processing?

I guess that's because historically they were coined and used by
different departments. BTW. Audio-export is still often called "bounce"
(analog tape-tracks).

Rendered or bounced tracks are always processed-data. However an
"Export" can also just be raw-data & edit-info. Modern Workflow has
somewhat blurred the lines here.

> What I am sorely missing is a video time stretch function as the final 
> edit is still badly out of sync. 

While some software-players support it, many video playback devices are
limited to 24, 25, 29.97 or 30 fps. - Time-stetching film by non-integer
multipliers is tricky. google for eg. telecine, pulldown.

Usually you're better of editing, re-sampling or time-streching the
audio. *hint* ardour's stretch/shrink region edit-mode.

> At least now it starts in sync but the 
> drift sets in after about 5 seconds. 

What is the cause of this? Is the source material already out of sync
(eg. indepenent audio and video recordings)? Does it happen when cutting
the clips (in your case: after avidemux)?  or is it a problem of the
final rendering/mastering?

A possible mistake could be that you simply specified a wrong framerate
or samplerate for the target.. (eg. you use 24fps source instead of
25fps?! or supplied an 44100 SPS audio to a PAL-DVD which should be in
48kSPS).

> Maybe I could do it with liVES or 
> ffmpeg but I was surprised that none of the apps I tried offered 
> "Stretch" as a core feature/tool.

mencoder is your friend. It includes pullup and filmdint (inverse
telecine) filters; and it can do crude slow/fast motion via -fps , -ofps

That being said: you better fix the audio than tempering with video
framerates.

> Through all of this process I have been consistently amazed at the state 
> of Linux Multimedia apps. Maybe it's just my Fedora 10 64 bit system but 
> it seems to me that Linux Audio Apps are a good 5 years ahead of Linux 
> Multimedia.

yes, if not 10-15 years. While most core features have been present for
quite some time, there's a huge gap  when it comes to interoperability,
usability and proper front-ends.

> However I was glad to see that most of the apps had JACK 
> support which made editing and listening a lot easier. 
> 
> I still had to close all the apps occasionally because the fedora 
> mplayer doesn't have jack support and pulseaudio jack support isn't 
> obvious to setup.

have you tried  `mplayer -ao jack <file>`
  "Could not open/initialize audio device" ? -> recompile mplayer ;)

> If the multimedia devs can get jack support right why 
> can't the pulse audio team?
>
> I have a new level of respect for the amazing progress that Ardour 
> represents for Linux Audio and *cough*soon*cough* to be multimedia 
> (xjadeo). Ardour is without a doubt the most advanced and stable editing 
> system we have to work with.
> 
> I can see now that Ardour could well become the defacto "multimedia" 
> editing suite for Linux if work continues and we continue to support the 
> developers.
> 
> I would like to give cineFX (jahshaka) a shot too once I get the 
> openlibraries to install.
> 

lumiera.org is on the horizon.

> Cheers.
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkoApSMACgkQeVUk8U+VK0K+xgCgoeI0uS7fGT9rWRAW0juQjy5t
FKYAnjhXZMbE/gSKX13tjt6wFaGWiiCl
=978M
-----END PGP SIGNATURE-----