Doesn't anybody test audio quality in depth any more?

over here
http://forums.stevehoffman.tv/threa...-res-digital-while-chewing-out-gizmodo.409061
folks are debating analog versus digital.

Actually there is another dog in the hunt so to speak, and that is the QUALITY of the bits and perhaps the sampling. Just because you have 24 bits samples 96,000 times per second does not mean that the data is really valid. The A/D conversion used to be and perhaps still is incapable of really generating valid data over such a dynamic range. In other words, you weren't really getting 24 bits of real data (or even 16 back in the day) but maybe 18-20 bits plus a bunch of noise and crud. I'm less sure about the time accuracy of the A/D conversion back then and today [anyone?]. Different schemes and engineering can produce different results, but just as very very very very few D/A converters can deliver anywhere near 24 bits of actual signal-to-noise, I bet very very very very few A/D converters can do so either. Certainly anything being sucked off ancient analog tape for the umpteenth time isn't going to have anywhere near that dynamic range.

There are many tests that can be applied to audio systems (analog or digital testing):
- Intermodulation, for instance 19 kHZ + 20 kHz tones together and see what sideband products pop out
- A high amplitude, very low frequency tone with a tiny high frequency tone riding on top. This was on a Denon test disc, for examining the linearity of a digital system throughout the bit range. I suppose you could do this in the analog domain as well, especially for A/D conversion.
- Reproduction of tiny signals. For example, a favorite of mine is the 22.05 kHz @ 1 LSB test which I think was from the NAB test disc. Since there was data, DACs couldn't mute to show amazing but fake low noise floors. I would then look at an FFT of the rest of the spectrum. The 1 kHz -90 (??) dB yes signal is another one which lets you look at the noise floor and then there is a test which let you look at noise floor "pumping" as the music signal changed.

BUT. I rarely see these tests anymore! I'm unaware if any pro gear EVER gets tested in this detail [??]. Actually, the ONLY place I see tests like this is Stereophile so kudos to John Atkinson (with a bit of a raspberry for not testing receivers, which are by far the most popular amplification and would be so interesting to see digital quality results on).

Actually even in the analog domain, tests seem to be few, far between, and very cursory and uncritical. I see for example receiver tests routinely showing massive power droops as all channels are driven, FAILURES TO MEET SPECIFICATIONS, and ignored or dismissed with stupidity such as "well all the channels never get driven at once"-OK, so why bother testing at all? If it was some $5000 multichannel power amp you'd be criticizing it! Grrrrrr.

I see no detailed tests of noise floor characteristics or…much of anything. Barely better than the old "oh it sounds so great" days of what the heck was that magazine?

Am I missing information sources? Has the publishing economics just squeezed this out?

We certainly don't know how to measure everything that is important to sound, and still don't understand much about hearing perception. And we could definitely debate the importance of the things that DO get measured. But it is better to have some numeric information than none, and I feel we have gone backwards in this area.

???

 

When it comes to portable and walk-around music, that segment of the population who are religious about that playback format really do not care much about the sound quality ...

 

Don't even get me started on the near total lack of serious testing and measuring of phono cartridges, tonearms, and turntables. The test records to run a lot of the more telling and interesting tests don't even exist anymore.

 

Those of us who care about audio quality represent a rapidly shrinking population.

 

It's a great point, and it's true not only with audio. Go to any car enthusiast forum, and you will see people agonizing about what to do to modify their cars to solve this problem or that shortcoming - and they haven't even taken delivery yet! We are a society of specs and digital opinions. How could high end audio be immune?

Regarding quality of "bits," I just think about digital cameras on phones. I had a 12 MP camera on an Android phone that just plain sucked. The 8MP camera on my iPhone, however, is rich and detailed, and takes a much better quality shot than the Android. Why? It's the lens and the light sensor, most likely. But the specs, by themselves, don't tell the story.

As for music, I just trust my ears. I can listen to hours upon hours of vinyl, and my only inclination to stop is usually when it has become so late that I need to get to bed. With digital, I just cannot stay focused on the music. That said, I do enjoy hi-res done correctly...it's very close.

 

Right. I can imagine owners of Koetsu cartridges, SAT tonearms and TechDas turntables losing sleep over lack of tests which would prove that their acquisitions are not hoax.

 

Customers of AV receivers likely wouldn't be worried about how they would perform on the Stereophile test bench, as AV receivers likely all perform decently in their intended role of playing compressed movie soundtracks loudly.

Customers of phono cartridges, especially the more esoteric ones, likely aren't worried about test measurements because its hard to measure what it is that people enjoy about them. I'm reminded of commenters on the Stereophile review of the DeVore 0/96s attacking the speaker they'd never heard because it "failed" a test. Oh well!

 

I'm not sure which definition of compressed you meant, but it doesn't apply in either case. Movie sountracks have enormous dynamic range, much much more than music, and they're lossless (presuming you're watching BDs). So, neither lossy data compression nor dynamic range compression.

 

Sure thing.

 

Right, exactly. I was definitely referring primarily to equipment at those price points and above. I want people with big budgets to sleep better. Why should anyone care about finding the tracking ability limits or channel separation characteristics of a sub-$500 phono cartridge under various conditions?

 

This is true. In my experience, most folks complain that movie soundtracks are not adequately compressed in the dynamic range sense.

 

I'm unaware of any DAC that can resolve 24bits of information, some of the best are aroung 20. Not that it is of any practical concern, 20 bits already allows a dynamic range and noise floor that is beyond human hearing capacity. 24 bits, fully recorded and resolved would have a noise floor low enough to reproduce the sound of electrons crash in the resistors within the audio chain, which of course no person would ever be able to hear it. In any event, with the mastering of many modern music compressed within 20db, even 16 bits is largely wasted.

But back to the main point of you post, I think that the main reason these measures are rarely published or discussed these days is that most DACs are competent enough to not be the weakest link the audio chain. Same is true for ADCs, particularly as used in recording studios. Back in the 80s and possibly the 90s it had more relevance. Even the Ipod and Samsung DACs perform very well. Most of the improvements today are evolutionary and by large, within the analogue circuitry.

 

For movies, many people do like compression simply to allow better clarity of speech and hence make it easier to follow the dialogue. For music DVD's though, it may be a different matter.

 

Totally thought the same thing when I read that.

 

Especially when it comes to watching Netflix on a laptop!

 

A/V receivers have DSPs modes that can deal with this somewhat, plus center channel speakers help a ton with speech clarity and a lot of people just set the center channel +2 decibels above the rest of the soundtrack. But yeah, movie soundtracks are incredibly dynamic and probably the most dynamic material most people own.

 

Yep, they all seemed to quietly disapear after the advent of CDs. Perhaps it was a bit deflating with CD's beating the metrics of T/T and various carts on every relevant measure. I can remember a time when audiophiles used to brag about any slight improvement in THD, wow and flutter, dynamic range and stereo separation measures and often used these to bang against the heads of cassette deck enthusiasts.

 

I don't think that is a good analogy - the pixel density of a CMOS sensor is not the same thing as bit depth in audio. However, like audio, there is a range of metrics for video, or cameras, which provide real world comparisons.

As for listening, I am sort of inclined to agree with you, though your mileage will vary. I enjoy listening to records, I find it is quite a relaxing sound but somewhat booring if listening to it after a session with good mastered digital. I liken it to the "night" function some TVs have which soften the brightness and resolution of the picture to create a more relaxed viewing experience when unwinding late at night.

 

Magazine testing is flawed, because magazine reviews are flawed and not really worth taking notice of.

I welcome that people are trusting their own ears more - that's how Chord did the Hugo, and that turned out
pretty well.

 

It is nice to have testing of components alongside a narrative. If nothing else it illustrates how measured performance doesn't always tie-in vs perception.

With or without measurements though, it’s even more useful is when the findings are compared/contextualised against some equivalent components.

What I find increasingly blah are the 'I tested X expensive component and, lo, it was good' '…and her voice was more real [than what?] and one knew all too well she'd had two buttered croissants for breakfast' / 'This budget component performed well above its price point' kind of reviews without any of the above. Fine appetite-whetting coffee table fare, but not all that helpful when you get down to it.

 

I know this is repeated often on audio forums however my own experience is that bit depth has a lot more to do with the quality of reproduction than the theoretical and practical noise floors and dynamic range might suggest. For my own part, at least when it comes to classical music, 24 bits not an option per see - it's a basic requirement in order to achieve reasonably accurate music reproduction. It is possible to hear how reducing the number of bits effects the timbre of acoustic instruments, especially when it comes to violins (though I might say that because having played the instrument to the cusp of a professional level, I know exactly how they are supposed to sound). The less bits, the more aggressive and metallic the sound - to the point where by the time the music is 16 bit (either native or dithered from a higher bit depth), it is no longer possible to accurately reproduce the sound, even though there is more than enough dynamic range and the noise floor remains ostensibly inaudible at normal listening levels.

It is even possible to set up test scenarios to see how reduced bit depth negatively effects the sound. Take a classical piece of music recorded natively at 24 bits, then apply dither at 24 bits, 20 bits and 16 bits whilst retaining the file in a 24 bit container. Then compare the sound of the 4 files. The lower the bit depth, the less accurate the sound compared to the original 24 bit file, even though the actual noise added and dynamic range of all 4 files are still more than accommodating of the limitations of human hearing (and yes, I agree the original 24 bit file would still be encumbered with noise and dynamic range to suggest that only 21 or 22 bits were "used"). Even when you add dither at 24 bits (where the added noise floor to the original file is insanely low), it is still possible to hear the negative effects on instrumental timbre even at normal listening levels.

The bottom line then, at least in all of my experience is that theoretical noise floors and theoretical dynamic range don't tell the whole story, since if they did, I shouldn't be able to hear any difference between a classical piece of music at 24 bits or 20 bits or 16 bits. But as I say, the differences are quite noticeable even though when the bit depth is decreased, the total amount of noise added is still far below my or anyone else's hearing thresholds.

Even when I am editing original 24 bit files on a workstation, I can hear the difference in the final output if I am working at 64 bit precision versus 32 bit precision, even though the actual differences between the two result in added noise even far below a 24 bit level. So however it all works (and I don't pretend to have a scientific or even plausible explanation), humans with a lot of critical listening experience can hear the differences between two otherwise identical recordings, with the differences being nothing more than added noise far below the thresholds of audibility. And the less bits, the worse it sounds.

 

There's a sucker born every minute.

I'm not saying all those brands are BS or any of them for that matter but high quality and rigerous testing keeps people and manufacturers honest, drives competition and makes for a better product. Do you really think that everyone trying to sell you something is looking out for your best interests or that just because a few hundred or thousand people believe something makes it true?

 

I can understand, and have many times experienced, the difference that 24bit depth and higher has on recording and mixing, but for playback?

I'm not having a dig at you, but many people claim that they can hear the difference but, assuming all other variables are controlled, this has never been proven. I know I can't tell the difference in a double blind test. Then there is the peer review paper below I posted in the other thread. In the 10 years since it has been out there, no credible test has been published which refutes its findings as far as I am aware.
http://drewdaniels.com/audible.pdf

Just out of interest, the OP in the link below articulates the 24bit vs 16bit debate quite well.
http://www.head-fi.org/t/415361/24bit-vs-16bit-the-myth-exploded

 

Figures lie, and liars figure... I'm elated that numbers and graphs don't explain the human condition of enjoyment!

So much of it is subjective anywho, isn't it?

Happy Listening!