Let's test the output of our DACs with RightMark Audio Analyzer.

More then theoretically. But the problem is you'd get identical results no matter which cable you used, ergo the test would be declared invalid.

 

I'm not getting what is so strange? That a premium 96/24 sound card has a frequency response +/- 0.5dB 12Hz-40,000Hz? (Note my graph goes an octave lower than the Rightmark software.) That there is any rolloff at all at the ends of the frequency response?

The ADC must perform filtering to avoid aliasing, so the largest frequency response contribution is likely from the recording component of the loop. The DAC output also filters, integrating over 1/2 sample rate, otherwise you would have a bunch of (inaudible) distortion from a series of square waves and their infinite harmonics. Most chipset-style converters these days are high-order multibit delta-sigma, so these filters are merely design parameters, not limitations.

Analysis of my test tone alone without loopback is +/- 0.0005 dB, so there is no problem with the methodology. See my thread http://forums.stevehoffman.tv/threads/testing-a-preamps-riaa-frequency-response.290022/ I could isolate just the output's response by sending tones to my Tektronix oscilloscope with RMS voltmeter.

To analyze if RMAA software is making calculation errors or to test its tolerances, it should be possible to loopback digitally through a rewire audio driver like VB Audio virtual cable. You must use a bit-accurate driver such as ASIO or WASAPI to bypass the Windows mixer and resampling. That should give analysis revealing the quantization distortion limits.

The sound card I'm using was not cheap either, more expensive than OP's DAC:

 

They should mean the maximum deviation from zero, plus or minus, in decibels. But I guess as you've already noticed, they don't tally with the chart - or certainly not in the case of the data + chart I posted above. I would find it a lot easier to accept the figures than the chart!

Sorry everyone - harby has correctly identified my error. Because his chart was just below another that looked much the same, I assumed it had the same scale, but in fact the scale was very different (both axes) and I now see that his DAC is close to flat between 20 Hz and 20 kHz.

In fact, speaking of scale, I think I was a bit intemperate in my previous post. The vertical scale on the RMAA frequency response charts is marked in half decibels, not whole decibels, and does tend to emphasise small deviations from zero in a way that other charts might not.

In any event, this has motivated me to do a test that I hope to set up this weekend in which I'll use two separate PCs to check the frequency response of my DACs, then compare the results with the RMAA loopback tests.

 

Great!

 

I was Googling to see if I could find some way of using RightMark on one PC to test a signal coming from another PC, and stumbled upon this very long and detailed critique of the software:

NwAvGuy on the RightMark Audio Analyser

There's a lot of boring stuff about "15 things RMAA does not measure" but if you skip all that and move on to "All the nasty details", it gets quite interesting.

The article is very long but the basic message seems to be, you can have a bit of fun with RMAA on your own PC if you understand its limitations but you can't use it to produce any absolute measurements and you can't use it to compare the results obtained on one PC with the results obtained on another PC. In other words, what most of us were probably suspecting already!

I'm still planning to do those PC-to-PC tests, though.

 

NwAvGuy is right to point out RMAA's weaknesses, note also the age of that post however. I think the fact that, in my case, I was able to get measurements close to the manufacturer's spec is useful

 

Yes, I think it does have its uses - for example, if you get excellent results or results in line with the manufacturers' spec, that's a good indication that everything's OK on your system. It's probably less useful when problems start to show up because people who don't know about its drawbacks could leap to the wrong conclusions and/or blame the wrong components. Not always, but it's a risk.

I wasn't able to do those tests at the weekend BTW but I'm still hoping to do them in the next few days.

 

Agreed!

I have a new soundcard from E-Mu coming hopefully this week, so I'm looking forward to measuring it against the Asus Xonar U7!

 

Oh, spoke too soon. E-Mu out of stock!

 

I've done a few tests, mainly on frequency response. I think the system only allows you three charts per post so I'll have to do a lot of posts.

I've been comparing:

1. The built-in DAC /amp on my desktop PC's motherboard.

2. My iFi Nano DSD portable DAC/headphone amp.

3. My Yuolong U200 desktop DAC/headphone amp.

To begin, I did an RMAA test of all three devices, using the loopback technique. Here are the results:



The noise figures look terrible to me - not surprisingly, because the noise is so bad when I do a loopback test that it's easily audible over the headphones.

For me, then, the noise is virtually untestable using this method. But I was more interested in frequency response because I wanted to see whether the little iFi was capable of delivering the same sort of frequency range as a desktop headphone amp.

Continued in next post.

 

In this post, I'll look at the frequency response of the built-in Dac on the motherboard.

I should have said that in all of these tests, including the ones referred to in the previous post, I conducted the tests with a pair of Audeze LCD-2 headphones connected to the DAC/amp output via a Y-splitter. These are moderately hard headphones to drive.

This is what the RMAA test told me, using the loop-back method through the internal A/DC:



That looks pretty shocking at first sight, but keep in mind the scale - in reality it's only 2 dB down at 20 Hz and only 1 dB down at 30 Hz. Not bad for a little chip on the motherboard driving an enormous pair of LCD-2 headphones!

But let's see what another test shows.

To check the veracity of the RMAA technique, I disconnected the loopback and used a second desktop PC. I connected the headphone output on the DAC/amp being tested to the line-in input of a high-end Lynx Hilo Reference A/D D/A Converter System, then connected the Lynx Hilo to the second PC via USB. I recorded the input to the second PC in Adobe Audition and I then used Voxengo Curve EQ to compare the recorded output to the source file.

In this first test, I played a one-minute pink noise test file. Here is the result:



Please note the different scale. In this case the general picture is fairly similar and there's little doubt the built-in DAC is rolling off, but the roll-off is only a little over 1 dB at 20 Hz and only 0.75 dB at 30 Hz.

In the following test, the setup is exactly the same as in the previous test but this time, instead of pink noise, I play the whole of track "Everybody Plays The Fool" from the CD Warm Your Heart by Aaaron Neville, which has lots of deep bass among other things.

Sorry, can't seem to post it. I'll have to leave off while I try to fix it.

 

I'd imagine the low dynamic range numbers are due to the load introduced by the headphones?

 

Good question but I've no idea what the answer is. The iFi spec says DR is >104 dB and the Yulong spec says >115 dB so we're looking at a huge shortfall. I tell you what, after I've finished with frequency response, I'll do another quick loopback test with no load.

Now, returning to the frequency response charts, I've now managed to get reconnected to Dropbox so here's the final chart missing from the previous post.

This is the in-built DAC but in a real-life situation, playing real music - the track I named above.



It's still rolling off the bass but only 1 dB at 20 Hz, and that's under load.

So my quick takeaway from the in-built DAC comparisons is that the RMAA test exaggerates the bass roll-off. RMAA said it was 2 dB down at 20 Hz but my own tests which I think are more reliable say 1 dB.

 

Next, the iFi. Here's what RMAA says:






Hmm, rolling off both ends, apparently.

Now, here's my own test using pink noise and Voxengo CurveEQ, PC-to-PC:




And now, a real-life test using real music - same track as named previously:



I'm a bit disappointed to find that all tests show the iFi rolling off the bass. However, even RMAA says it's only 0.5 dB down at 20 Hz while my own real-life test shows it only about 0.2 dB down at 20 Hz, which I'm certain would be completely inaudible. The top end roll-off doesn't bother me at all - it's quite small and mostly affects frequencies beyond my audible range.

So, broad agreement between the tests there although RMAA does seem to exaggerate the bass roll-off very slightly.

 

Finally, the same three tests for the Yulong U200.

First, RMAA loop-back:



Again, rolled off at both ends. Notice the channel imbalance - the left channel is 0.5 dB louder than the right channel. That's true! I've known about that from the start.

Now, my own two tests - first, pink noise, then real music:





Look at that last chart! As close to perfection as it gets, I reckon. Remember, these aren't just testing the DACs, they're testing the amps as well, and the amp in the Yulong is Class A.

It's interesting to see how the RMAA loopback test shows a bass roll-off while neither of my tests shows any sign of this. Another thing I've noticed is that the lines in the RMAA charts are remarkably straight - they must use an awful lot of smoothing.

 

I don't think it's possible to conclude much from these tests. It does seem to me that, on my particular PC, using the loopback method, RMAA rolled off the bass a little and I'm wondering if that's something that happens on the return path as the signal goes back into the PC through the A/DC on the motherboard. Based on these tests, I wouldn't personally trust RMAA to do an accurate frequency response test on a DAC or DAC/amp using a loopback test on my particular PC but having that said, RMAA wasn't far off and did remarkably well considering the poor quality of the loop. No doubt it would do a lot better if I could figure out a way of running an RMAA test from one PC to another, using the Lynx Hilo, instead of using a poor quality loop.

 

Robert, I ran the RMAA tests again without the headphone load and the DR range for the Yulong U200 and the iFi DSD Nano were the same - 59.5 and 51.5 respectively. I don't understand it at all but I'm just going to assume it's another quirk of the measurement system - it seems unlikely that both amps would have a DR that low.

 

Tell me, is there any way you can use your Lynx Hilo for (ordered but) simultaneous processing of D to A and A to D? That is, loop USB or other digital output from a single computer through a Lynx D to A section, then using external (or Lynx internal) interconnects run through the Lynx once more, in the A to D direction, then back into the same single computer via USB (or other digital)? I don't think I've seen that in your interesting set of tests.

I'm not on top of the full Lynx Hilo capability set, but I know there's a lot it can do. It seems to me that such a loop could completely eliminate any of your computer internal D to A or A to D processing. It should give you the cleanest possible loop through...and serve as an RMAA processing cross check within your set of system capabilities.

I've been thinking about trying out my Sound Devices USBPre 2 in exactly such a fashion. I've long had my own copy of RMAA Pro and this seems like an entertaining thing to take a look at. In fact the USBPre 2 can internally connect one channel of a stereo pair for loop-through test and measurement of D to A to D. I'd post my results.

 

Are you using your computer's realtek card to do the recording? Is it set to 24 bit?

 

Yes, the Lynx Hilo will indeed do D/A and A/D at the same time - it will even do them through the same USB connection. But the problem is, that would only solve one of the two problems I have when doing a loopback test through my desktop PC. It would eliminate the problem of the poor quality components on the PC, but it wouldn't eliminate the problem of the noise, which I think may be caused by some kind of ground loop or something. I know this because I already tried it, and that's why I went down the road of using two separate PCs.

But maybe you'll have better luck with your PC! (Hope so.)

 

Yes on both counts - if you flip back to the table in my post #35 above, that was done using a loopback test via the DAC and A/DC on the motherboard, and as you can see in the table, all the tests were done at 24 bits. To re-test without the headphone load, I repeated exactly the same tests except with the headphones disconnected.

BTW, there was one more thing I meant to say. Looking at the charts I did, it was interesting to me that real music produced EQ charts that were different from the charts done from test tones or pink noise. It seems that the nature of the test signal can make a difference - but if that's really the case, it's a bit worrying for the reliability of tests generally.

 

All right, I've now been able to run some loop-through tests of my Sound Devices USBPre 2. For those that don't know, it's one of these:


With the site page here.


The USBPre 2 is easily thought of as an external soundcard, but that's a slightly limited way to think of it. In fact it also has portable standalone purposes while not actually connected to a computer. All tests in this post were run with RMAA Pro and the USBPre 2. The USBPre 2 was looped through in a D/A/D configuration--with the use of a throw-away stereo RCA interconnect cable running from the USBPre's (D/A), line level, Aux Outs to the USBPre's (A/D), line level, Aux Ins.


The loop-through test results are a bit further down, but first a small amount of meta analysis of RMAA...

Posts throughout this thread, including one of my own, have pointed out that RMAA needs to be used with extreme interpretive caution--and the linked NwAvGuy post decimated RMAA rather nicely. That was all completely valid. RMAA has it's qualities, but it's less a precision instrument than it might look and, at best, RMAA really only has use within a single system...when the complete system RMAA is running within is clearly understood.

This thread's posts form a valid "devil's advocacy" rather against RMAA. Let me, however, surface a slight bit of RMAA "angel's advocacy". The chief designer/developer of RMAA is Alexey Lukin. At this point Alexey Lukin has been Principal DSP Engineer at iZotope Inc. for a good number of years and has been teaching Introductory Signal Processing at Moscow State University for about the same number of years. RMAA is a bit of freeware (or "Pro" version cheapware) that must be used cautiously, but there's probably a fair amount of correctness in how RMAA goes about things.


Okay, my test results. Oh, I've mentioned the USBPre 2 box, but the machine it was connected to is an HP Professional Division "EliteBook Mobile Workstation" (laptop) running Windows 7 Pro 64. The in/out connection to the USBPre is a single USB cable...

I ran four loop-through tests: 44.1k/16, 44.1k/24, 96k/24, and 192k/24.

Here's the summary table:



And here is a subset of the USBPre 2 specifications:



It may be worth pointing out that tested noise levels and dynamic ranges are in agreement and, naturally, a function of bit depth. If the RMAA measured dynamic ranges falls short of the spec's, perhaps that's because the RMAA testing is a two-way round trip. But the RMAA tested noise levels and dynamic ranges aren't so very bad.

A couple of notes on frequency response, using a picture of the 192k/24 "swept sine" graph...



When I put the cursor at 65 kHz on the graph I got a reading of -3.52 dB, that's certainly in close agreement with the spec's "-3 dB at 65 kHz"...and the RMAA reading is two-way. But the left side of the plot is interesting, with a strange trough at 45 Hz. I believe that's an artifact of the FFT analysis. Due to the way FFT bins frequency ranges (each bin gets a precisely equal, linear, range of frequencies), low frequencies can get a very poor FFT analysis as the overall analyzed measurement range expands. In my RMAA lower sampling rate tests, the lower frequencies plot and measure much better.

And, finally, what is happening with the 192k RMAA THD swept frequencies test??? Beats me, and the corresponding plot looks pretty messed up (and I tried this a few times). The "%" THD line looks consistent, and good, for all tests. I'm going to call this an "outlier testing artifact", and am going to completely eliminate this strange result from consideration. . . .By the way, I opened the identical file several times and looked at the RMAA table, each time the file was opened a different (but still very bad) number showed in the table. Alexey! What's going on here!!

 

Thanks for posting that I will post a new set of results with my Xonar U7 at the same bit/sample rates as you. I too am miffed by the swept frequencies read-outs. I've disabled them in my copy of RMAA

 

Asus Xonar U7.

First off, let's take a look at the manufacturer's specifications:

Here are my RMAA results at various rates and resolutions:



The figures look good with the noise level and dynamic range results practically matching Asus' specs. A 24-bit dynamic range of 112.2 dBA is the equivalent of 18.7 bits.



The frequency response is interesting to me as it clearly shows a rising line from ~7 kHz, peaking at 20 kHz, before rolling down and out by ~60 kHz. I use this sound-card when DJ'ing and I've found that I have to notch the high frequencies down a bit at the mixer to prevent the sound over the PA from being too bright. Looks like I've found the reason!

Zoomed in frequency response...



The noise level graph below shows a 50 Hz peak at -112 dB, which seems to repeat at 100 Hz intervals. I wonder if this is power related? Perhaps I can smooth it down with my Audioquest Jitterbug



Experimentations with power coming soon

 

Not much difference with the Jitterbug installed. Slightly worse?