Doesn't anybody test audio quality in depth any more?

Magazine testing is flawed because the mags are funded by there advertisers, ( the equipment manufactures and distributers ). Bad reviews mean less advertising dollars. Sad but true.

 

I'm not really surprised it hasn't been easy to demonstrate because not only are the differences subtle in the scheme of things, but the listener also needs to not only be incredibly tuned in to the strengths and weaknesses of the test system, intimately familiar with the source material, but also highly experienced in terms of what real instruments sound like - preferably through having played those instruments to a professional level or at least to an extremely competent amateur level (such as achieving an A. Mus A diploma in violin, for example).

I have actually done double blind tests and passed them but not so much to prove a point but in order to determine which dithering and word length reductions were the most sonically transparent. But it is a difficult test to do - my audio workstation for example (the only way I can do a double blind test since I needed access to Foobar) simply does not have the resolving power in terms of playback amplification and playback transducers to make subtle differences obvious. I can for example, use different dithers and different resample parameters and it is almost impossible to reliably hear the differences (with subtle changes) simply because the purpose of the setup is to perform editing and "flat" recording - it's not meant for high calibre playback or remastering. But my other system is extremely revealing so I had to temporarily set everything up so that system was working out of the workstation outputs.

The other problem is that when it comes to 16 bit (assuming one is working at 24 bits during the mastering) is that all these 16 bit algorithms are - in my experience - optimised for a 44.1 kHz sample rate. That makes sense because about the only commercial resolution that uses 16 bit is 16/44.1 (CD and standard res downloads) and the much-rarer 16/48, with the latter being extremely rare on the consumer side of things (though Decca used to record at 18/48 and later 20/48 and of course 16/48 was the DAT standard).

The bottom line is that it's my experience when converting high sample rate material from 24 bit to 16 bit (which I only did for self-educational purposes), I really can't hear a difference with any reliability (96 kHz and upwards). At 48 kHz the difference is actually the most noticeable and I put this down to the algorithms not being designed to work well at 48 kHz because it is such an orphaned resolution. At 44.1 KHz, the inevitable compromises of the low sample rate (proximity of actual recorded content to the point where the filtering slope takes effect) already causes enough sonic issues as it is, without adding the dithering effects to it. But suffice to say I find the low 44.1 kHz sample rate about 65% responsible for the reduced sound quality from native high res and the dithering responsible for the other 35%. I obviously don't mean the quality drops by 100% - I'm just saying that the low sample rate does about twice the damage to the sound compared to top notch dithering (which I have found after years of experimentation to be PSP X-Dither - the most benign of all products I have ever tested, including iZotope).

Of course it's the internet so anyone can say what they like, so me saying I've done blind testing and passed can't carry much weight and I don't blame people for being sceptical. And I will also reiterate that give me a system with which I am not already intimately familiar and source material which I am not intimately familiar and I'd completely fail any test. Again, I suspect this is the reason all the official tests never prove the point. I'd be completely lost as well in such circumstances.

I need to point out too that I am not some sort of golden-eared guru - my hearing is only normal for my age and sex. But I have an advantage in that I studied violin to diploma level and grew up being both a musician and audiophile. So you could say that I had trained my "ears" (well, really my brain I suppose) to be extremely well tuned and receptive to what the real sound should be like and to detect even the most subtle changes. It's like someone who has trained themselves to appreciate fine art - I myself just think a French Impressionist painting looks incredibly cool and aesthetically appealing. But there are other people who are trained to understand subtleties that would completely elude me - probably no matter how hard I tried to learn.

So I focus on acoustic stringed instruments because that's my thing. I would not hear the differences (I suspect) on rock or pop and I probably wouldn't hear them on brass instruments, harps, oboes or bassoons and definitely not double basses where the frequencies are well below the range where I honed my musician's skills over the years. Keep in mind that some violinists can hear the differences between a tiny change in sound post position, sound post tension, a miniscule change in string gauge and a miniscule change in bridge position - things that 99.9% of people wouldn't have a hope of hearing.

If we really had to prove a point that 16 versus 24 bit is officially audible to a human (as opposed to humans in general), we'd be better off getting world class classical musicians together who also happen to be hardcore audiophiles and getting them to listen to material on their own systems. I suspect that then the story would be different than is the commonly held perception. Sure, maybe 99% of people (I'm pulling that off the top of my head) still can't hear the difference, but it would only take one human to hear a difference and then we would have to say that yes - the difference is actually audible, though you need to be incredibly gifted as a musician and audiophile to hear it.

 

There are all kinds of issues which could theoretically make a 16bit version potentially sound worse than a 24bit version but none of it would be due with 16bit in itself. These include appropriate whether appropriate differing was applied (as you said), and whether the 24bit source was itself dithered to start with. Properly implemented any difference would be below the noise floor of a listening environment. The point you make about different the sample rate conversion is interesting and has some theoretical basis to it. In fact, from a pure theoretical perspective 88.2, rather than 96 is the optimum higher sampling rate required given the maths involved to get back to 44.1. However properly done, the difference between 16bit and 24bit is an extra 48db of dynamic range, and nothing else. Given 16bit is already below the noise floor of most listening environments, and no recording exists currently that uses all of 16bits 96db range, where is the difference that can be heard, or at very least be material?

That AES study I posted has already covered this. Some of the subjects were audiophiles and musicians using their own hi res material on their own equipment, with the same 50/50 result. Of course that does not prove that there may not be any one individual within the 50% correct group that actually did hear a difference and was not guessing, or thinking they heard a difference, though the science would suggest that is not the case. But at least you have subjected yourself to a double blind test when you say you can hear a difference. If it was me personally, I would then be looking at why I can hear this difference by assessing the equipment, software and conversion. If the difference is an actual difference, then you must admit from the gist of your post, that it is so subtle as to be immaterial. Hardly the stuff supporting the hi res fanboy crowd you come across in some other threads.

 

I'm not sure why you think I wouldn't have quadruple checked everything. I'm a pretty thorough person. And it easy to check the software. You just null the two files out. All you are left with is noise - precisely the same noise you get if you applied the dither to a silent section of 24 bit audio and dithered that. And I've double blind tested with and without noise shaping - so the dither without noise shaping simply adds a consistent level of noise across all reproduced frequencies so effectively is the most "pure" redution to 16 bit as it reduces the signal to noise ratio consistently across all frequencies. As I say, the noise itself is inaudible - I agree with that. But what this inaudible noise does is effect what we do actually hear. It's a similar mechanism to jitter - which, afterall is for practical purposes nothing more than noise as well - which again is inaudible in itself except in extreme cases, yet it affects the tonality of what we hear.

As for the hi res fanboy club, no I don't agree with your interpretation of my post since high res by definition is high sample rate 24 bit audio (I disagree that 44.1/24 bit is "high res" even though technically resolution is defined by bit depth). The sample rate difference between 44.1 and 96 kHz is easily audible - as I say it effects the sound about twice as much as dithering does. Yes, dither is more difficult to hear, but once a person is intimately acquainted with their system and the source material, subtle differences become larger even to the point where they can make or break the sound in a highly tuned system.

I can't speak on behalf of anyone else but I find all these differences including dither pretty easy to hear on a very highly resolving system. It's just extremely difficult to hear on an average to good system. If as you opine dither is for practical purposes inconsequential, I wouldn't have literally spent nearly 1000 listening hours over the last three years experimenting with many dithering products and endless settings in the (impossible) attempt of making it's application audibly transparent. If I did not hear the ill effects I could have just spent 5 minutes on it all - just buy iZotope and hit the OK button.

 

There is some rather rare material that features 100+dB of musical dynamic range, BTW. The 16/44 downsampled/dithered version of the 24/88 master track can be found here. This track generated few heated exchanges in now closed monumental thread (check my posts about 16-bit dithered / non-dithered versions of the master). The differences can be clearly heard due to 0-crossing distortions in 16-bit non-dithered track...

 

I always have a bit of problems when someone talks about a 'highly resolving system', for some reason I sort of loose interest.

 

Apologies if it came across that I thought dither is inconsequential. It isn't. I have heard several examples where both the source and downsample were dithered, which can cause all sorts of issues. I would have thought that noise shaping the dither signal, so that the white noise is moved further up the frequency band to where human hearing is less sensitive, would result in an improvement and a perceptual increase in dynamic range? When 24 to 16 conversion is professionally and competently implemented it is not possible to hear any noise, unless you pick a silent passage and turn the volume up to insane levels. If conversion of 96kfps to 44.1 is creating noticeable effects then I would be suspecting something is not quite right. There is no physiological or practical reason supporting that frequencies above 20khz can be heard or perceived by humans, and no credible study supporting such a view. Indeed, there is nearly a hundred years of hearing tests being conducted every day around the world in hearing clinics. I doubt there are any audiologists that would concur otherwise.

The other, somewhat related issue, is that 16bits can exceed the capability of analogue circuits in the chain. Even in the digital world, there are no DACs or speakers that are able to resolve 24bits. Perhaps someday we may get there but there is no urgency because at 16bits we already are at the limit of what our hearing and brains are able to resolve.

And I'm not making a judgement whether or not you can hear a difference, just questioning why, but I am curious that with the peer review papers not supporting people can hear this difference in properly controlled tests (including audiophiles and musicians using their own equipment, listening environment and reference sound material), and no studies of peer review standard that support people can hear them, why don't you document your methodology and foobar results and post them in hydrogenaudio? If valid, and replicable it could lead to a solid rebuttal of those peer review papers and a well deserved reputation on your part.

 

Interesting, thanks for the references.

 

Amen!

 

You mean something like this?..

Code:

foo_abx 2.0 report
foobar2000 v1.3.7
2015-01-30 21:53:45

File A: 13 - Bozza- Children’s Overture (from La Voie Triomphale) (2488).flac
SHA1: 19315719e5ca526122ab992abacfa9e33d2d8da1
File B: 13 - Bozza- Children’s Overture (from La Voie Triomphale) [16-bit, S-TPDF] (2488).flac
SHA1: 41e04c7c50aec3cf434ffdc321376d477ad2ab43

Output:
ASIO : ASIO4ALL v2
Crossfading: NO

21:53:45 : Test started.
21:55:42 : 01/01
21:55:58 : 02/02
21:56:10 : 03/03
21:56:23 : 04/04
21:56:35 : 05/05
21:56:42 : 06/06
21:56:52 : 07/07
21:57:05 : 08/08
21:57:17 : 09/09
21:57:26 : 10/10
21:57:35 : 11/11
21:57:42 : 12/12
21:57:49 : 13/13
21:57:59 : 14/14
21:58:09 : 15/15
21:58:17 : 16/16
21:58:26 : 17/17
21:58:34 : 18/18
21:58:44 : 19/19
21:58:52 : 20/20
21:58:52 : Test finished.

----------
Total: 20/20
Probability that you were guessing: 0.0%

-- signature --
8830edcc4001d30845f1fdf2ae7707e1c1b00adb

For those interested, here are the files I used in my 24- vs 16-bit blind comparisons:

A: 24-bit original (24/88 FLAC);
B: 16-bit S-TPDF dithered version (padded 24/88 FLAC).

Give 'em a try & see if you can successfully pass the ABX test yourself. To show your geniune audiophile skills try to avoid the fadeout (unlike me)...

 

I will give it a try a bit later. Do you know what dither was applied to the 16bit file? Was the 24bit original dithered at all?

I'd recommend you also post the results on hydrogenaudio and see what discussion results.

On another matter in relation to the 100db dynamic range, that just exceeds 16bits of 98db (putting aside any perceptual gains through dithering). I'm curious as to what happens if the file is used as a source to cut a vinyl record. The maximum dynamic range of vinyl is at best 70db. So what happens to the extra information? Is it cut-off at the extremes or is it compressed within that range?

 

Sure, the end product is subjective and that is a good thing otherwise we'd all have the same taste in music, listening on identical stereos. Doesn't mean someone can't enjoy the music and be interested in the science and technicalities behind it.

 

The dither I used for a 16/44 file referred to in my 1st post (Krakow) was SoX' moderate noise-shaped one (Modified E-Weighted). 24/88 original is apparently an untouched studio master recording...

 

Pretty much the only thing I'm really interested in when it comes to reveiws is the measurements (and explanations of the circuits or drivers etc.), whether that's measurements Stereophile does or Soundstage has done or whatever. After 20 or 30 years of reading subjective audio reviews it just becomes a mind-numbing blur of undifferentiated adjectives -- warm, organic, natural, accurate, hard, bright, wide, narrow; or barely meaningful comparisions -- slight differences between two componenets I've never heard noticed by a reviewer listening in who-knows-what evironment. At least with the other stuff I'm getting useful information.

 

But I also think that 16 bit dynamic range doesn't tell the whole story in any case. Take a violin concerto for example. You have maximum peaks when the full orchestra is playing flat out and that might be at -1.0 dBFS in the final recording. Then you have the solo violin part. That's one treble stringed instrument that has to be clearly heard on its own as well as an orchestra that might have 60 or more players in it.

Then you have the violin playing solo passages and some of those may be piano (as in dynamics, not the instrument). Some of these passages will be over 50 dB down from the maximum peak on the recording. And that is just the volume of the fundamental note - not the harmonics that give the instrument it's characteristic timbre. The harmonics produced by the instrument, as opposed to the fundamental tone are complex and obviously far lower in volume than the fundamental. As a result, a sufficient number of these harmonics are completely obliterated by a 16 bit noise floor yet remain intact at 24 bits. This is the reason why 16 bits simply cannot reproduce violin sound correctly - period. And it also explains why 16 bit sonata recordings sound more creditable in 16 bits than a large scale concerto recording - because the average recorded level of the violin in a sonata recording is so much higher to begin with so that less of the harmonics are killed off in the noise floor (practically speaking you probably preserve around an extra 30 dB more of the harmonics with 24 bit versus 16 bit).

So sure, if music was just a collection of pure sine waves over a 96 dB range in an anechoic chamber then yes, 16 bits is sufficient and it won't sound any different whatsoever to a 24 bit recording. But acoustic music and orchestral music in particular relies on the accurate reproduction of all those harmonic structures and 16 bits is only good enough to cover all the fundamental tones but fails because it cannot preserve as many of the harmonics as 24 bit recording can.

And it's not just timbre - it's ambient decay. 16 bit cannot handle the subtle reverberations in the concert hall either. This in turn effects the imaging and the precise placement of instruments in the performing space and their relative positions to each other as well as defining the shape of the hall they are performing in. Mind you, 24 bit recordings can't do this perfectly either. They might if they really were 24 bits, but at least they do far better than 16 bits.

 

Well, you can listen to 2 lingering trumpet notes at the end of the track in question (24-bit master dithered to 16/44) & hear how they sound below -90dB...

 

So you believe you can hear the harmonics and subtleties that may exist below the noise floor of 16bits? An extremely quiet listening room would be an essential just to hear the noise floor and even so, any music content would mask any subtleties that would exist at that level. Not saying it is impossible, but rather, improbable and that there is no credible study supporting the contention. Saying that 24bits do far better than 16bits within the bounds of human hearing is an overstatement.

 

Look, you are obviously a 16 bit, 44.1 kHz is perfect and an audio objectivist. I'm different to that. I use my ears as a trained musician and audiophile. Do you do audio engineering for a living and do you have a performing diploma on the violin (9 years minimum study for most people)? I do in both cases so I think that gives me a pretty creditable standing in my assertions. I don't subscribe to theories that are blown apart in actual practice, as is the case for 16 bit, 44.1 kHz audio being perfect. There are many reasons why it isn't. You obviously are not absorbing anything I have been trying to say over these last few days in this thread, because if you had, you would have realised I've said umpteen times that there is a clear difference between what we can / cannot hear at extremely low levels as opposed to how what is missing / included at those levels DOES influence what we actually hear. I've also said it umpteen times that a 16 bit recording of a violin does not sound like the real thing. A 24 bit recording is much closer but even that doesn't sound like the real thing. But you just keep on an on about 16 bit being perfectly fine and then you tell me to go to Hydrogen Audio so they can repeat my results. The mere fact that over their they almost without exception think CD (and many compression algorithms) sound "perfect" by my definition means that the hope of any of them hearing the difference between 16 bit and 24 bit is non-existent even when I spoon-feed them and tell them precisely what to listen for.

Quite frankly, I'm not interested in Hydrogen Audio and I'm not interested in your white papers because I've already proven to myself 100 times over that 16 bits is not good enough. I think that you and others here need to show a little bit more respect to people who both do all this stuff for a living and also have spent 10,000 hours learning a difficult acoustic instrument to a professional performance level. You can direct your back handed insults elsewhere.

 

Easy mate, where have I insulted? Where have I said 16/44 is perfect? The discussion was the context of the bounds of human hearing.

Ok, I get it. You dismiss the peer review evidence which supports people do not hear a difference and the real day to day evidence from audiologists around the world. 16bits is not good enough for you, fine. I'm sorry I don't know which "white papers" you are referring to, nothing to do with me, but if you mean the real controlled studies from the peak audio science bodies which have not been refuted, I'm sure they should throw it all away in favour of your golden ears.

Perhaps there is something mystical about all this. Many people also say that they know more or can feel effects of things which science and practical real world evidence does not in the fields of medicine such as homeopaths, astrologists and the like. They too have special superhuman abilities and are dismissive of science, peer review papers etc. As in now, any discussion outside the true believers end up in ad-hominen attacks. And it doesn't surprise me a bit that you aren't interested in hydrogenaudio as that forum is evidence based and grounded in basic science and more is required than simple assertions about one's superior hearing abilities and mystical speculation about things we cannot hear somehow impacting on what we can.

I'm done here.

 

Something that I think so many people miss - if a person can only detect a difference in something because they've devoted their life to being able to hear the difference, and acknowledges in at least a de-facto way that a person who hasn't devoted their life to being able to hear the difference probably can't, then its such an incredibly minor difference its not worth arguing about. Continuing down the path makes you an audio narcissist, not somebody who is doing something better the the rest of the pack.

 

We all hear about the same at a given age, I don't believe that some can hear things that others can't. But do you pickup on certain q's, are you a more astute listener and more importantly do you care .

 

Well, now we get down to it. It's always mystified me as to why every objectivist cowboy seems hellbent on marching into every audio saloon on the web itching for a digital gunfight.

Art Dudley said it far better than I ever could:

Though it appears that the most contentious skeptics are unlikely to take heed, I urge them: Please disregard any product, or any genre of product, if its manufacturer is unable to produce a reason for buying with which you are satisfied. Buy what you want. Don't buy what you don't want. Enjoy recorded music in whatever manner suits you best. Believe what you believe—but for God's sake, keep an open mind. And if you can't manage that, then at least stop worrying so damn much about the way the hobby is approached by people who aren't you.

Read more at http://www.stereophile.com/content/listening-143-page-2#fBi7R9hdI1XDyyoA.99

 

I don't disagree, but it does often times seem as if in the same conversation or even same post, we'll have both "a trained ear is necessary to make out the subtle effects on the transients" and "the difference is quite obvious" without acknowledgement these things are mutually exclusive. It's the latter of the two statements that becomes a point of contention vs. the former IMO.

 

It's because some people want to just take numeric "evidence" into account without considering that each person's hearing varies, and tests may not show everything about hearing.

On the other hand, others feel/believe they hear some difference, and are psychologically invested in that. To be "disproved" is in some sense to be told "you have been fooled, or fooling yourself" => "you are a fool"

That's why we see all this intensity. And why we see people arguing over bits and sampling rate, and ignoring the fact that those raw things do not matter very much, it is the practices around them (for example, higher sampling rates allowing gentler filtering, and greater bit depth enabling greater manipulation in studio mixing and effects without losing dynamic range) which really enable better sound.

We also see people forget that to hear subtle effects you need progressively more resolving (=expensive) systems. You're not going to hear subtleties in a 96 kbps MP3 over some $50 Bluetooth speaker.

It's also kind of like Ford vs. Chevy (and Dodge maybe too). Is there REALLY that much difference? Nah. Let's face it, not really. A Camaro and a Mustang are not alike, but their overall performance over the years has been roughly comparable. But fanatics of each are, frankly, just following a religion. Same with objectivists and subjectivists-they just follow religions, with precious little absolute proof either way. (Against the objectivists, because trying to prove NOBODY can hear certain differences is like trying to prove there is no God-it's basically impossible to prove a negative. All those tests are subject to criticism of the equipment, the room, the music, the listeners, etc etc ad nauseam. The most you can conclude is that "most listeners" can't hear a difference).

 

Yeah, I get bugged by people who say "the difference is quite obvious"-it is a dumb statement if not qualified. Some differences may indeed be obvious…to SOME people on SOME systems once they know what to listen for. Others are blithely happy disregarding, say, tape hiss (which drives me nuts) or clicks/pops/ticks (which for ME disqualify vinyl as a totally high fidelity medium, except perhaps in the mega-stratosphere of perfect vinyl perfectly cleaned or something)