10 Nov
2008
10 Nov
'08
1:27 p.m.
As a double blind test when seeing what changes were made to Wav files
when translating to and from Flac, I also used Audacity to make a
direct copy using 'Export'.
To my surprise, the copy had a different md5sum signature.
Thinking that the export operation might be creating the differences I
exported another copy, but this turned out to have a different
signature to both the previous ones. A third copy was again different
to all the others.
Listening to the files I could detect no difference, nor could i see
any difference in Audacity, even when stretching the display enough to
see the actual waveforms.
It occurred to me that the differences might be due to some sort of
timestamp embedded into the file header (I don't really know what
meta-data Wav files contain) so I looked at the files in a Hex editor,
cecking at several identical locations through the files. They all
showed up as different, and it wasn't just the same pattern but shifted
a few bytes.
Can anyone suggest what might be happening here?
--
Will J Godfrey
http://www.musically.me.uk
10 Nov
10 Nov
3:37 p.m.
Folderol wrote:
As a double blind test when seeing what changes were
made to Wav files
when translating to and from Flac, I also used Audacity to make a
direct copy using 'Export'.
To my surprise, the copy had a different md5sum signature.
Thinking that the export operation might be creating the differences I
exported another copy, but this turned out to have a different
signature to both the previous ones. A third copy was again different
to all the others.
WAV files have a header as well as the audio data. Its highly
likely that the audio data is the same each time and that the
header is different.
Listening to the files I could detect no difference,
nor could i see
any difference in Audacity, even when stretching the display enough to
see the actual waveforms.
It occurred to me that the differences might be due to some sort of
timestamp embedded into the file header (I don't really know what
meta-data Wav files contain)
They can indeed contain time stamps.
so I looked at the files in a Hex editor,
cecking at several identical locations through the files. They all
showed up as different, and it wasn't just the same pattern but shifted
a few bytes.
Can anyone suggest what might be happening here?
Can you put two example files that are supposed to be the same up on
a web server somewhere?
Also, I you grab the latest pre-release of libsndfile at:
http://www.mega-nerd.com/tmp/
and compile it, you will find an example program called sndfile-cmp
in the programs/ directory which compares just the audio data.
Cheers,
Erik
--
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"Using Java as a general purpose application development language
is like going big game hunting armed with Nerf weapons."
-- Author Unknown
4:32 p.m.
On Mon, Nov 10, 2008 at 05:27:32PM +0000, Folderol wrote:
As a double blind test when seeing what changes were
made to Wav files
when translating to and from Flac, I also used Audacity to make a
direct copy using 'Export'.
Maybe 'Export' adds dithering ...
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Lascia la spina, cogli la rosa.
5:30 p.m.
On Mon, 10 Nov 2008 17:27:32 +0000
Folderol <folderol(a)ukfsn.org> wrote:
As a double blind test when seeing what changes were
made to Wav files
when translating to and from Flac, I also used Audacity to make a
direct copy using 'Export'.
To my surprise, the copy had a different md5sum signature.
Thinking that the export operation might be creating the differences I
exported another copy, but this turned out to have a different
signature to both the previous ones. A third copy was again different
to all the others.
Listening to the files I could detect no difference, nor could i see
any difference in Audacity, even when stretching the display enough to
see the actual waveforms.
It occurred to me that the differences might be due to some sort of
timestamp embedded into the file header (I don't really know what
meta-data Wav files contain) so I looked at the files in a Hex editor,
cecking at several identical locations through the files. They all
showed up as different, and it wasn't just the same pattern but
shifted a few bytes.
Can anyone suggest what might be happening here?
I think we experienced something similar once but I don't know problem
nor solution. wav-files have a header but I don't know of any kind of
meta-data, so this is unlikely.
My guess was and is a bug in audacity, one should contact the authors,
I haven't done so when we discovered this oddness.
Philipp
6:25 p.m.
hollunder(a)gmx.at wrote:
I think we experienced something similar once but I
don't know problem
nor solution. wav-files have a header but I don't know of any kind of
meta-data, so this is unlikely.
WAV files can potentially have an abundance of meta data. Don't
discount this too quickly as a possibility.
My guess was and is a bug in audacity, one should
contact the authors,
I haven't done so when we discovered this oddness.
Well I'm the author of libsndfile which Audacity uses for its WAV
file export. Lets investigate the issue here and then if audacity is
actually the problem we report complete findings to them :-).
Erik
--
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
C offers you enough rope to hang yourself. C++ offers a fully equipped
firing squad, a last cigarette and a blindfold.
6:58 p.m.
On Tue, 11 Nov 2008 09:25:05 +1100
Erik de Castro Lopo <mle+la(a)mega-nerd.com> wrote:
hollunder(a)gmx.at wrote:
OK, there are four files:
http://www.musically.me.uk/music/Test1.wav
http://www.musically.me.uk/music/Test2.wav
http://www.musically.me.uk/music/Test3.wav
http://www.musically.me.uk/music/Test4.wav
Test1 is a short excerpt from the original recording that I
investigated. It is well clear of any clipping margins etc.
Test2-4 are copies made with Audacity's export function, and were made
one after the other, within seconds of each other, and without touching
anything else in Audacity (or any other application for that matter).
All have exactly the same byte count, and look identical to me in
Audacity.
Below are the md5sums I get for these.
will@folderol:~$ md5sum /share/Music/Test1.wav
184c0fb2af68f5f79438c4ba4a9f5352 /share/Music/Test1.wav
will@folderol:~$ md5sum /share/Music/Test2.wav
8e40bd5e2931fc1a9c53117ffbe22bf8 /share/Music/Test2.wav
will@folderol:~$ md5sum /share/Music/Test3.wav
351f5295df6b771a012f0bc1a959719f /share/Music/Test3.wav
will@folderol:~$ md5sum /share/Music/Test4.wav
f07427e5b0d87e3158094d88f830f63f /share/Music/Test4.wav
will@folderol:~$
--
Will J Godfrey
http://www.musically.me.uk
I think we experienced something similar once but
I don't know problem
nor solution. wav-files have a header but I don't know of any kind of
meta-data, so this is unlikely.
WAV files can potentially have an abundance of meta data. Don't
discount this too quickly as a possibility.
My guess was and is a bug in audacity, one should
contact the authors,
I haven't done so when we discovered this oddness.
Well I'm the author of libsndfile which Audacity uses for its WAV
file export. Lets investigate the issue here and then if audacity is
actually the problem we report complete findings to them :-).
Erik
7:19 p.m.
Folderol wrote:
OK, there are four files:
http://www.musically.me.uk/music/Test1.wav
http://www.musically.me.uk/music/Test2.wav
http://www.musically.me.uk/music/Test3.wav
http://www.musically.me.uk/music/Test4.wav
Test1 is a short excerpt from the original recording that I
investigated. It is well clear of any clipping margins etc.
Test2-4 are copies made with Audacity's export function, and were made
one after the other, within seconds of each other, and without touching
anything else in Audacity (or any other application for that matter).
All have exactly the same byte count, and look identical to me in
Audacity.
Visual inspection is not really up the the task :-).
I've got a little curses hexdump utility that I use for this kind of
comparison:
http://www.mega-nerd.com/erikd/Blog/Tools/hv.html
which shows something like this for "hv Test[234].wav":
00000000: 52 49 46 46 A8 02 49 00 57 41 56 45 66 6D 74 20 RIFF..I. WAVEfmt
52 49 46 46 A8 02 49 00 57 41 56 45 66 6D 74 20 RIFF..I. WAVEfmt
52 49 46 46 A8 02 49 00 57 41 56 45 66 6D 74 20 RIFF..I. WAVEfmt
00000010: 10 00 00 00 01 00 02 00 44 AC 00 00 10 B1 02 00 ........ D.......
10 00 00 00 01 00 02 00 44 AC 00 00 10 B1 02 00 ........ D.......
10 00 00 00 01 00 02 00 44 AC 00 00 10 B1 02 00 ........ D.......
00000020: 04 00 10 00 64 61 74 61 84 02 49 00 00 00 02 00 ....data ..I.....
04 00 10 00 64 61 74 61 84 02 49 00 FE FF 02 00 ....data ..I.....
04 00 10 00 64 61 74 61 84 02 49 00 01 00 FF FF ....data ..I.....
00000030: FC FF 05 00 FB FF 02 00 01 00 FE FF 02 00 FE FF ........ ........
FD FF 03 00 FD FF 01 00 02 00 FC FF 05 00 FA FF ........ ........
00 00 00 00 00 00 FF FF 02 00 FE FF 02 00 FD FF ........ ........
The differences are small, about 2 least significant bits, which suggests
that the audio has been dithered.
My guess is that Audacity has a default internal format of normalized
floats (ie range [-1.0, 1.0]) and that by default it dithers when saving
internal floating point data to a 16 bit WAV files.
In general, when converting from float to 16 bit int, dither should be
applied. However, one case where it makes sense not to dither is when
you load 16 bit integer audio data into a float internal format and then
save it back to a 16 bit integer format. In other words, the int16_t ->
float -> int16_t chain should be lossless.
If you are interested in lossless conversion from integer format to
integer format, you should try sndfile-convert which (obviously)
handles all the formats supported by libsndfile. The sndfile-convert
program by default does all conversions in 32 bit integer format
(hence int16_t -> int16_t is lossless) and only uses floating
point if either the source or destination format is float.
HTH,
Erik
--
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"UNIX was not designed to stop you from doing stupid things, because
that would also stop you from doing clever things." -- Doug Gwyn
11 Nov
11 Nov
3:44 p.m.
On Tue, 11 Nov 2008 10:19:14 +1100
Erik de Castro Lopo <mle+la(a)mega-nerd.com> wrote:
Folderol wrote:
Thank you very much for going into this in such detail and so quickly.
I can confirm that using sndfile-convert directly gives completely
identical files.
This is true when going wav -> wav and also wav -> flac -> wav
--
Will J Godfrey
http://www.musically.me.uk
OK, there are four files:
http://www.musically.me.uk/music/Test1.wav
http://www.musically.me.uk/music/Test2.wav
http://www.musically.me.uk/music/Test3.wav
http://www.musically.me.uk/music/Test4.wav
Test1 is a short excerpt from the original recording that I
investigated. It is well clear of any clipping margins etc.
Test2-4 are copies made with Audacity's export function, and were made
one after the other, within seconds of each other, and without touching
anything else in Audacity (or any other application for that matter).
All have exactly the same byte count, and look identical to me in
Audacity.
Visual inspection is not really up the the task :-).
I've got a little curses hexdump utility that I use for this kind of
comparison:
http://www.mega-nerd.com/erikd/Blog/Tools/hv.html
which shows something like this for "hv Test[234].wav":
00000000: 52 49 46 46 A8 02 49 00 57 41 56 45 66 6D 74 20 RIFF..I. WAVEfmt
52 49 46 46 A8 02 49 00 57 41 56 45 66 6D 74 20 RIFF..I. WAVEfmt
52 49 46 46 A8 02 49 00 57 41 56 45 66 6D 74 20 RIFF..I. WAVEfmt
00000010: 10 00 00 00 01 00 02 00 44 AC 00 00 10 B1 02 00 ........ D.......
10 00 00 00 01 00 02 00 44 AC 00 00 10 B1 02 00 ........ D.......
10 00 00 00 01 00 02 00 44 AC 00 00 10 B1 02 00 ........ D.......
00000020: 04 00 10 00 64 61 74 61 84 02 49 00 00 00 02 00 ....data ..I.....
04 00 10 00 64 61 74 61 84 02 49 00 FE FF 02 00 ....data ..I.....
04 00 10 00 64 61 74 61 84 02 49 00 01 00 FF FF ....data ..I.....
00000030: FC FF 05 00 FB FF 02 00 01 00 FE FF 02 00 FE FF ........ ........
FD FF 03 00 FD FF 01 00 02 00 FC FF 05 00 FA FF ........ ........
00 00 00 00 00 00 FF FF 02 00 FE FF 02 00 FD FF ........ ........
The differences are small, about 2 least significant bits, which suggests
that the audio has been dithered.
My guess is that Audacity has a default internal format of normalized
floats (ie range [-1.0, 1.0]) and that by default it dithers when saving
internal floating point data to a 16 bit WAV files.
In general, when converting from float to 16 bit int, dither should be
applied. However, one case where it makes sense not to dither is when
you load 16 bit integer audio data into a float internal format and then
save it back to a 16 bit integer format. In other words, the int16_t ->
float -> int16_t chain should be lossless.
If you are interested in lossless conversion from integer format to
integer format, you should try sndfile-convert which (obviously)
handles all the formats supported by libsndfile. The sndfile-convert
program by default does all conversions in 32 bit integer format
(hence int16_t -> int16_t is lossless) and only uses floating
point if either the source or destination format is float.
HTH,
Erik
10 Nov
10 Nov
7:58 p.m.
On Mon, Nov 10, 2008 at 10:58:26PM +0000, Folderol wrote:
The difference is just noise. Either Audacity is dithering
the exported signal (but using too much noise to do it), or
it is mixing in some other noise.
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Lascia la spina, cogli la rosa.
11 Nov
11 Nov
2:44 p.m.
On Mon, Nov 10, 2008 at 10:58:26PM +0000, Folderol wrote:
I had a closer look. As posted before the difference is
noise, numerically generated (not analog) and strangely
lowpass filtered: -3dB at around 8 Khz going down to
-20dB at Fs/2.
The noise power is -81dB FS, which is at least 14 dB
higher than would be required for any form of dithering
of 16-bit PCM. At least 2.5 bits are lost. No idea what
Audacity is trying to do, but it looks quite wrong.
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Lascia la spina, cogli la rosa.
3:47 p.m.
On Tue, 11 Nov 2008 19:35:30 +0100
Fons Adriaensen <fons(a)kokkinizita.net> wrote:
On Mon, Nov 10, 2008 at 10:58:26PM +0000, Folderol
wrote:
I had a closer look. As posted before the difference is
noise, numerically generated (not analog) and strangely
lowpass filtered: -3dB at around 8 Khz going down to
-20dB at Fs/2.
The noise power is -81dB FS, which is at least 14 dB
higher than would be required for any form of dithering
of 16-bit PCM. At least 2.5 bits are lost. No idea what
Audacity is trying to do, but it looks quite wrong.
Ciao,
Thanks for going further into this. It's quite puzzling.
The thing is, where do we go from here? As I said in my previous post I
can confirm the ability to make identical copies directly, and via flac,
but I don't know how relevant the Audacity devs will think this issue
is.
--
Will J Godfrey
http://www.musically.me.uk
3:58 p.m.
Folderol wrote:
Thanks for going further into this. It's quite
puzzling.
The thing is, where do we go from here?
I think *now* is a good time to bring this up with the Audacity
developers. Probably worth sending them links to my response
containing the hexdump of the differences as well as Fons'
analysis that it seems to be low-pass filtered noise.
Cheers,
Erik
--
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"These are the finest moments in (post)modern life, when satire is completly
indistinguishable from reality... I usually have to rely on the presidential
elections for such dada." -- frenomulax on Jesux a christian Linux distro.
4:39 p.m.
On Wed, 12 Nov 2008 06:58:09 +1100
Erik de Castro Lopo <mle+la(a)mega-nerd.com> wrote:
Folderol wrote:
Further investigation reveals just what is going on (I think).
After a look at the Audacity help archive, I somewhat shamefaced looked
at the preferences tab.
The default sample format is indeed 32 bit float, but is selectable as
16 or 24bit integer as well.
Also dither can be 'None' 'Rectangle' 'Triangle' and most
interestingly
'Shaped'. Guess what, out-of-the box default is for 'Shaped'.
I'll do a few more tests myself, but I'm rather of the opinion that
I've just stirred up a non-issue.
I shall get my coat and close the door quietly behind me :/
--
Will J Godfrey
http://www.musically.me.uk
Thanks for going further into this. It's
quite puzzling.
The thing is, where do we go from here?
I think *now* is a good time to bring this up with the Audacity
developers. Probably worth sending them links to my response
containing the hexdump of the differences as well as Fons'
analysis that it seems to be low-pass filtered noise.
Cheers,
Erik
5:26 p.m.
On Tue, Nov 11, 2008 at 08:39:52PM +0000, Folderol wrote:
The default sample format is indeed 32 bit float, but
is selectable as
16 or 24bit integer as well.
Also dither can be 'None' 'Rectangle' 'Triangle' and most
interestingly
'Shaped'. Guess what, out-of-the box default is for 'Shaped'.
The code is based on Jack's but implemented quite differently.
I'll do a few more tests myself, but I'm
rather of the opinion that
I've just stirred up a non-issue.
I don't think so...
I just ripped Audacity's dither code and put it in a simple
jack app to test it. Things seem OK... The 'shaped' spectrum
is not entirely what I expected, but it's quite different and
*way* lower than what I measused on you files.
There must be something else going on...
Caio,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Lascia la spina, cogli la rosa.
15 Nov
15 Nov
2:13 p.m.
On Tue, 11 Nov 2008 22:16:56 +0100
Fons Adriaensen <fons(a)kokkinizita.net> wrote:
On Tue, Nov 11, 2008 at 08:39:52PM +0000, Folderol
wrote:
I haven't had time yet, but I will put a bug report together for
audacity with all these points included.
--
Will J Godfrey
http://www.musically.me.uk
The default sample format is indeed 32 bit float,
but is selectable as
16 or 24bit integer as well.
Also dither can be 'None' 'Rectangle' 'Triangle' and most
interestingly
'Shaped'. Guess what, out-of-the box default is for 'Shaped'.
The code is based on Jack's but implemented quite differently.
I'll do a few more tests myself, but I'm
rather of the opinion that
I've just stirred up a non-issue.
I don't think so...
I just ripped Audacity's dither code and put it in a simple
jack app to test it. Things seem OK... The 'shaped' spectrum
is not entirely what I expected, but it's quite different and
*way* lower than what I measused on you files.
There must be something else going on...
Caio,
11 Nov
11 Nov
4:02 p.m.
On Tue, Nov 11, 2008 at 07:47:01PM +0000, Folderol wrote:
The thing is, where do we go from here? As I said in
my previous post I
can confirm the ability to make identical copies directly, and via flac,
but I don't know how relevant the Audacity devs will think this issue
is.
If they are happy with exporting 16-bit files that have
a S/N ratio corresponding to 13 bits, they can ignore it :-)
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Lascia la spina, cogli la rosa.
5855
days inactive
5860
days old
linux-audio-user@lists.linuxaudio.org
15 comments
4 participants
participants (4)
-
Erik de Castro Lopo -
Folderol -
Fons Adriaensen -
hollunderï¼ gmx.at