INTERNET BLIND TEST: MQA Core Decoding vs. Standard Hi-Res (24/88 or 24/96)

Hey Arch

It's great someone has put the time an effort into an objective test of MQA (well objective as it can be and far better than sighted opinions) given its purveyors seem very reluctant to do so, thanks for that.

I'd loved to have participated but only stumbled into this thread today. Looking forward to the results.

Btw, excellent posts from @testikoff too.

 

Time for some results folks:
Archimago's Musings: MQA Core vs. Hi-Res Blind Test Part I: Procedure
Archimago's Musings: MQA Core vs. Hi-Res Blind Test Part II: Core Results

More to come over the weeks...

 

Brilliant work. Thanks for doing this. Bears out what many of us thought about this and valuable to the wider audio community too I think. I hope somebody points Meridian to this study.

 

IMHO 83 music lovers sitting in the comfort of their home and listening with their favorite equipment sure beat a bunch of old geezers corralled into a room who later proceed to explain why we are not worthy of having our own opinions.

Now unless I missed something the results are failing in a spectacular way to show a clear difference in the perception of sound between the samples. This is eerily reminding me of the various blind tests between CDs and MP3 that were popular 10 years ago.

 

A/M,
will it ever be possible to recall which ones we voted for and file type? Somehow I lost my reference.

 

Thanks for putting in the effort. The results are unsurprising as is the reason why Meridian do not back their claims with controlled tests.

 

I finished reading the article and the methodology looks sound. The population sample is small but HiFi is a niche market so it might still be representative.

What interest me is how the choices differed between the various tiers. Did the expensive setup fared better at correctly identifying MQA or hearing noticeable differences? What about age? We are getting old and we don't hear as well as we did in our 20's. Is there a correlation between younger age and noticing differences?

 

Fantastic work! I would be interested to know if there is anything that links the participants that consistently chose the more 'popular' sample.

 

Interesting - I had the misconception that MQA was the size of a MP3.

Ugh - sometimes marketing can influence the popular choice - even though one method is more efficient/clearly better than the other.

Sometimes even with solid analysis and evidence, one cannot go against the tidal wave. (no pun intended)

A very good example is the Betamax and VCR war...

I was very curious about how good the MQA format is and this has satisfied my intrigue.

Thanks everyone for participating and Archimago for setting up, analysing and writing up the report.

 

Archimago - will you be using non-parametric statistics on the data?

 

Hi Ralf,
Happy to have a look if you have ideas...

In the Part III: Subgroup Analysis post, I did try looking specifically if I could find "outliers" and tried some hypothesis driven exploration of the data to determine if I could find significance. Categorical correlations like musicians vs. engineers vs. audio reviewers, "golden ear" responses, any age correlation, and equipment costs...

 

A way forward with a non-parametric test could be a 2x2 contingency table and a chi-square test (or Fisher's exact if a cell count is lower than 5) for differences between different groups. Or maybe a logistic regression with not too many explanatory variables in the model.

Hope this give some ideas.

R-T

 

Thank you kindly for this test, @Archimago. I've been curious about the reality of how MQA differs from PCM.

I spent a couple of minutes on each track pair and was very confident when writing down my results. I had an advantage in that I listen to all three tracks frequently (particularly Arnesen and Mozart), so any differences from the original were easy to spot. I'm not sure how successful I would be with unfamiliar tracks.

I was able to correctly choose which of the tracks was MQA without issue. I'd like to claim that they sounded overtly artificial compared to the real tracks, but really all that I was sure of was that they were not the original tracks (being familiar with the original tracks).

My listening tests were performed with the following setup:

JRiver -> Lynx AES16e -> Schiit Gungnir Multibit -> Cavalli Liquid Crimson -> Sennheiser HD650

This test has handily convinced me that MQA is basically the devil. It changed the sound of tracks and made them feel comparatively artificial.

 

Successful ABX test report(s)?

 

Great work! Thank you.

 

Yes, all this is only relevant to audiophiles, and if there had been a clear difference to them, the population would have been plenty big. In psychoacoustic experiments with animals, there are rarely more than 10-15 individuals tested. Another thing is the methodology of AB'ing with quick switches. I can't perceive small differences that way, and I would have to listen to MQA-only music for a couple of weeks and then shift to PCM-only (or vice versa) in order to get a clear sense of whether I can spot a difference. Anyway, my DAC only goes to 96 kHz, so for that reason alone I couldn't participate in the test.

 

Absolutely no opinion on merits of and I'm unable to play Hires files but this test is statistically meaningless, too many uncontrolled variables.

 

Why do you say that?

 

Well most statistical scientific studies haven't been reliable, even when rigorous, as reported by New Scientist, can't remember which issue but this,Most scientific papers are probably wrong , talks to same problem.

Plus we don't even know if people listened before voting.

 

Obviously the sample is non-random. The panel is unbalanced in sex, age, headphone versus speaker users. Respondents does not use the same hardware and we know nothing about respondents hearing, intoxication etc. But for fun, why not play with some statistical methods and inference something about the group? You can always slag someone else's (bad) design. Because it is cheap and cost you nothing at all.

 

Well, it is true that a lot of scientists may apply the wrong statistics, be selective in their interpretation of data, and display a lot of other imperfections. I heard a scientist (sorry, forgotten his name, but he's a wellknown and respected biologist) give a talk where he addressed just that. He found that the most statistical significant result he has ever had was that the closer to the 0.05 confidence limit the a paper's results were, the less likely it was that the authors would respond to his request to see the original data (that they are obliged to reveal). Authors of papers with higher stronger statistical significance were much more likely to share the original data.

But in this case, Archimago has been completely open about his methods and the data, and I, for one, think that the various segments that he divided the respondents into made perfectly sense. This transparence allows anybody with sufficient knowledge of statistics may pitch in and find faults.

Possible, but unlikely. But the respondents in this test were highly motivated individuals, so it is very unlikely.

 

I wouldn't want to criticise Archimango's great work, just pointing out this could potentially just help to enforce held beliefs when it shouldn't, especially as it may be confused with science.

 

as a scientist, I say this is science

if people did not listen first, that would bias the test - but how do you think it would be biased?

 

I thought that these were a taboo subject on this forum...

 

You thought wrong.