Objective ways to measure gear performance vs cost?

It happens but usually not. You spent your $3K unwisely unfortunately.

 

Good question and we all as audio consumers certainly wish there were.
Unfortunately measurements of any audio component get you in the ballpark and are useful- watts @ ohm, %THD, frequency response, cartridge and phono preamp specifications / measurements, sensitivity, efficiency, impedance, etc. - but they fall short of adequately representing how said component will sound- especially within a specific system or room. Kind of like words trying to describe a picture. When you cross reference our God given 5-senses, things just do not translate well enough to do it justice- e.g. reading words and numbers to convey sound and hearing.
For me to get a sense of determining (sound vs. cost= value) required exposure, lots of exposure to lots of systems over the years. There are trends and giant killers but in general to get a certain sound quality requires a certain investment. The "exposure" part helps you spend it wisely.

 

Well, I haven´t said we are. What I think is that in reality we need both, both objectivity and subjectivity. Nothing is really designed by either alone, some parameters are better detected being measured, some might be better appreciated by listening.

 

Well, sure, there are all the traditional bench tests -- harmonic distortion, intermodulation distortion, output before clipping, SNR, slew rate, square wave response at various frequencies, frequency response, channel balance, etc.

With amps it's tough because a bench test with an amp driving a 4 or 8 ohm resistor won't tell you that much about how a amp is going to perform under a complex load like a speaker playing music. So that's one way the bench tests aren't going to tell you much about how an amp will sound, because how an amp sounds is how th amp sounds driving a real world load. But even there, you can look at things like output into various impedances, and amp output impedance/damping factor, to have some sense of how an amp might perform driving a specific speaker (There's no point in evaluating an amp separate from asking the question about how well suited it is to driving the speaker you want it to drive).

The problem is the bench tests aren't going to tell you anything about the sonic character of the amp -- the things we audiophiles listen to: do the highs sound hard or easy -- or anything about imagining, which we audiophiles care so much about.

And the other problem is we don't have outside bench tests of all this gear, taken under identical conditions, by disinterested third parties. Sometimes we have that as some publishers do bench testing, but mostly what we have are manufacturer's specs which are as much sales documents as test results, so they're always taken and presented in ways that make the piece their selling look it's best. It's like a horsepower or mileage number in a car spec -- the HP number might be at an RPM you almost never get the engine too, and try getting the same gas mileage in average driving conditions that the manufacturers can get under test conditions for a sticker! And then the specs provided my manufacturers are taken all kinds of different ways, at different frequencies, etc. making them hard to compare.

 

Excellent post. I can't and wouldn't disagree with any of it.
-Bill

 

Actually there have been just this done by Consumer Reports. Unfortunately, their "top picks", "best performers", and "best values" have been laughable. Objective measurements used as a sole guiding indicator of results in audio is simply a red herring. It is a point that many engineers whom I have conversed with use as a crutch for being too cheap, or too disinterested in art, or too self absorbed to actually listen to different audio products in an attempt to better the sound. I can't am am not saying this is universal though as I also have many customers who are engineers, physicists, etc who do listen as their base measurement of performance, and who are interested in experiencing better sound. The proof is in the pudding, not the bottle.
-Bill

 

Unfortunately, that is usually a recipe for missing out on many of the best sounding products available. It works to a lesser degree only in cases where there is a particular need for a particular parameter of performance. Often however, even then, it is made a rather ineffective tactic by the opposing tactic of manufacturers playing that hand. I gave up on trying to sort out products by spec in the '80s as I learned the hard way a few times that it results in simply a chance outcome.
-Bill