[LAD] Ardour session archiver (Was: new lossless/lossy audio compressor)

[Phil Frost]

On Wed, Aug 01, 2007 at 02:12:41PM -0400, Paul Winkler wrote:
> On Wed, Aug 01, 2007 at 10:45:59AM -0700, Gregory Alan Hildstrom wrote:
> > Are there any wav/audio programs out there that use the full float range of 3.4e+/-38? It seems to
> > me that this larger range would use more of the 32 bits.
> 
> That's not how float works.  *Any* floating point number uses the full
> precision of all 32 bits (or 64, or whatever). This is because the
> radix point (like decimal point, but binary) can be anywhere.
> 
> What you're saying would be true of fixed-point numbers.
> 
> http://en.wikipedia.org/wiki/Floating_point

I reckon there is more to it than that. I'd say any AC signal centered
on 0 uses at least all the bits available in the mantissa. Clearly one
can not be using all 32 bits if one's signal does not have 2**32
possible values. I can't tell you off the top of my head how many
distinct values can represented within -1 to +1 with a 32 bit float, but
since it's less than the full range I can tell you it's more than 2**24
and less than 2**32.

True, a bigger range may not result in a higher quality signal, but in
the context of compression, it may be important. No value between -1 and
+1 has a biased exponent greater than 0x7f. Consequently, there are at
best 2**31 values an audio signal could use, and one bit that will be
always zero.

Of those 2**31 possible values, the vast majority of them lie around
zero. Cutting the range in half (to -.5 to +.5) excludes only 1/127th of
the possible values. After cutting the range in half 64 times to about
+/- .00000000000000000005421, roughly half of the 2**31 previously
possible values still lie within this range. Would you say half of a
typical audio signal lies within this range? I'm not about to go write a
program to find out, but I hope you will accept my guess of "no". The
point is this: floating point numbers have a lot of potential values
around zero that are unlikely to be used.

But wait, it gets worse! When was the last time you saw an audio
interface with a 31 bit analog to digtal converter? Maybe they exist,
but I consider myself lucky to get 24 bits. No matter how you cram 24
bits in to a 32 bit float, you still have 8 bits that mean nothing, even
if they have been shuffled around a bit.

I would suspect that many ardour sessions contain WAVs with floats which
came from a 24 or 16 bit source. I haven't looked at the jack source,
but it seems to me that the conversion between 32 bit floats in +/- 1
and 24 bit integers can be lossless. A trivial archiver could do this
conversion and make the session roughly 75% of the original size.

Of course, if this isn't the case for some sessions, this trivial
archiver can't be useful and lossless anymore, which raises another
point worth considering: if testing a lossless compression for float
signals, consider testing with something that didn't come straight from
a digital integer source. I'd suspect that IIR filters and reverbs in
the right circumstances will fill in a good many of the values near zero
that may typically be unused.