There's a couple things you're hinting at that can be explained with math and one that can be explained psychologically:

Signals with compact support in the time domain have infinitely long support in the frequency domain.  When you color a sound with non-flat frequency response, you lose the compactness of the original signal.  Muddiness comes about when the frequencies have different group delays or when some frequencies are cut.

The best mix preserves the distinctness of each part.  One of my favorite articles is "Tone and Voice" by David Huron.  The article makes it clear that musical practice derives itself from balance and independence of its parts (specifically as applied to voice leading but when applied more generally it has some insights for engineering more modern music)

Chuck

On Apr 5, 2013 9:32 PM, "Ralf Mardorf" <ralf.mardorf@alice-dsl.net> wrote:
>
> I try to find better words in English now.
>
> I care less about the bandwidth of the frequency. If the bandwith
> becomes smaller or does colour some frequencies, it's ok for my taste,
> assumed it is not too much. If the range endet at 18KHz for the original
> and a copy or transformed by a codec has loss, by ending at 15KHz, I'm
> able to notice this, but won't name it "I hear loss of highs", as long
> as the transparency and loudness dynamic, IOW the atmosphere of the
> recording is the same.
>
> It's important that the loudness ratio between the instruments is kept,
> the room is kept.
>
> Good recordings made with a 4-Track cassette recorder do sound less good
> than a recording with professional gear, if you take care about those
> things, when listening on professional gear, but it's impossible to hear
> it on an averaged consumer hifi.
>
> Taking care about the bandwith of the frequencies is not completely
> unimportant, but even something important as signal separation, is not
> very important, as long as you take care about this limitations, when
> doing the mastering.
>
> The music will sound completely different biased on different consumer
> hifi gear, it doesn't matter if it was recorded in a professional or in
> a home studio.
>
> A good audio engineer takes care that the sound is most transparent,
> even if the frequencies are cut etc..
>
> Sure, if you play a good recording on a very bad hifi, it does sound
> ugly, but very often it does sound better, than at least the old MP3s I
> know, because the ratio between the instruments etc. seems to be more
> biased by MP3s than by bad gear.
>
> Optimal is 48 KHz, but I also mentioned that for some recordings even
> 32 KHz is ok. For me it's bad when "transparent" sound becomes "muddy".
> "Muddy" doesn't mean less highs only (it often comes with less highs),
> "muddy" is for a loss of transparency. The original sound might separate
> guitar and trompete very good, even when they are in the same position
> of the room, but they have accentuations by different frequency bands.
> If a codec does remove frequencies, those instruments will be less good
> separated. This is the kind of loss I can't tolerate. The loss of highs
> and a little bit noise are less important for me.
>
> Many people nowadays aren't aware how to make sound transparent even
> with cheap analog home recording gear, but it's possible. We even don't
> need a compressor, many things a compressor does, can be done by EQs.
>
> Quality doesn't mean 2 Hz to 22 KHz and 200 dB S/N ratio. When using an
> analog tape 40 Hz to 14 KHz and 67 dB S/N ratio (EIAJ) and I take care
> about frequency issues and less good channel separation by the mix, I
> can do an amazing transparent mixes.
>
> _______________________________________________
> Linux-audio-user mailing list
> Linux-audio-user@lists.linuxaudio.org
> http://lists.linuxaudio.org/listinfo/linux-audio-user