2 Dec
2009
2 Dec
'09
12:46 a.m.
This seems to be due to the massive amounts of IO needed for realtime video
When I was working with YUV4Linux (basically a DVD stream without encryption
or compression) for a 2 GB file I was getting 2 minutes and 18 seconds at
PAL DV resolution. Raw rgb should be almost double that and rgba more than
double. HD resolutions will double and quadruple the amount of
memory/storage needed.
To get data between applications I was using Fifos. Which work, but are
difficult to set up and tempremental at the best of times.
I don't think it would be that realistic to work with compression between
applications in jack type situation. though colour space compression should
be possible
to give some idea of the memory/disk IO requirements needed.
I did some quick calculations
comparing a pal dv sized stream 720 x 576 @ 25 frames a second
vs 48 000 samples per second floating point audio stream.
rgba 32bit float = 864 48k audio streams.
rgba 8bit int = 216 48k audio streams.
rgb 8 bit int = 162 48k audio streams
YCrCb 422 = 108 48k audio streams
YCrCb 420 = 81 48k audio streams
for 720p
rgb 8bit = 360 48k audio streams
YCrCb 420 = 180 " " "
for 1080p
rgb 8bit = 810 48k audio streams
Are there any other color spaces I should be worried about?
Most compositors use RGBA internally either as 8bit int or floats.
FreiOr is a set of video plugins used by a lot of different video editing
programs it prefers rgba 8888, (bgra8888 is the other option) so that format
must at least be acceptable to a lot of video editors. Effectv is the other
shared video plugin set I am aware of and it uses rgb888.
Cinelerra supports RGBA float, RGBA 8888 & YUVA 8888 and non Alpha varients
of the above.
Perhaps the most sensible Idea would be to concider purhaps RGB(A) 888(8)
and YUV(A) 888(8) as different types of port.
On Wed, Dec 2, 2009 at 11:33 AM, Paul Davis <paul(a)linuxaudiosystems.com>wrote;wrote:
On Tue, Dec 1, 2009 at 8:02 PM, Danni Coy
<danni.coy(a)gmail.com> wrote:
You would be suprised. RGBA32 is used quite
commonly in the compositor
world, usually stored as a sequence of images, with even higher
precisions
available. Raw RGB24 is reasonably common with
the video professionals I
know, when I questioned them about it they said that they could acheive
more
film like colour grading than they could in any
of the YUV colour spaces.
It's probably not the best format for realtime work given the amount of
data
that needs to be transmitted (or maybe it is), I
have previously worked
with
piping raw YUV (yuv4linux) between applications
and having something like
jack to manage the connections would be a godsend.
my understanding is that one of the many problems that video faces is
that there truly is no equivalent to "32 bit floating point for
audio". whereas this format for audio samples is pretty much
acceptable for just about all purposes, and the ones not satisfied by
it are corner cases, in video there are many common cases that prefer
quite different data representations. for an audio analogy, imagine a
JACK world where some clients wanted FFT-bin data while others wanted
floating point encoding of PCM.
until or unless someone can step up and authoritatively say "the
interchange format for video is XXXXXX", its hard to imagine a "jack
for video" system really working very usefully for a significant
number of people. those who work entirely in the RGBA32 or YUV spaces
would be happy with just that format; anyone mixing processing that is
better suited to different formats is going to take a hell of a
performance hit.
--p