Let's test the output of our DACs with RightMark Audio Analyzer.

Discussion in 'Audio Hardware' started by Robert C, May 12, 2016.

  1. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    RightMark Audio Analyzer is intended for testing the quality of analog and digital sound sections of any audio equipment, be it a sound card, portable MP3 player, consumer CD/DVD player, or a speaker system. The measurements are conducted by playing and recording test signals passed through the examined sound section, using frequency analysis algorithms.

    I thought it would be fun and interesting for us to test our DACs and see how they measure up to the published specifications.

    I should mention that RMAA isn't the sharpest tool on the test bench. It's free and has its quirks, but it seems perfectly capable of testing a DAC as fed by a computer.

    My soundcard is the Asus Xonar U7 and my DAC is the Rega DAC-R. My soundcard has a published input signal-to-noise ratio of 110 dBA, so this limits its ability to test anything with a higher SNR. My Rega DAC-R has a published signal-to-noise ratio of 105 dB, so the soundcard should be able to measure it accurately.

    Below are the test results for my Asus U7 soundcard (first column), followed by the Rega DAC-R tested three times at 16/44, 24/96, and 24/192. You can see that the U7's dynamic range is recorded as being 112 dBA which is close enough to the published 110 dBA (both 110 and 112 dBA = 18 bits).

    Rega states the DAC has a SNR of 105 dB and, under 24/96 and 24/192, RMAA gives it 103.8, which is essentially the same as 105 dB (both ~17.5 bits). However, it looks like there is some discrepancy with the 16/44 results. I would expect noise level and dynamic range to both be ~96 dBA but the dynamic range recorded by RMAA is 91.8, or 15 bits of resolution. I will have to double check that!

    [​IMG]

    The Rega DAC-R has 3 filters, described in the Stereophile review as: "'standard', 'extended', and 'gradual'... The extended filter extends to the maximum response allowed by the sample rate. The gradual filter rolls off sooner than the standard filter". I used RMAA to check the frequency response of the 3 filters, fed by a 24/96 RMAA test signal.

    [​IMG]

    Here we can see:
    1. Standard
    2. Extended
    3. Gradual
    Subjectively, I prefer the sound of Filter 3. It's interesting to see that Filter 3 removes the most ultra-sonic information. The filters behave similarly at 44 kHz, with Filter 3 rolling off from 19 kHz.

    I hope some readers will find this interesting and I would love to see your own results with your DACs!

    You need the following:
    • RightMark Audio Analyser: Latest News. Audio Rightmark ยป
    • A USB soundcard with line-in
    • An external USB DAC.
    • An analogue cable to connect your DAC to your soundcard's line-in.
    First perform a loop-around test with your soundcard to set a reference. For this you need to connect the analog inputs of your soundcard to the analog outputs, and this way it measures the DA-AD path of your soundcard. It sends a test signal through the chain and analyses that. It can only analyse it's own test signals. Try to run this test first to get yourself familiarised with RMAA. It's performed by pressing the button with the loopback cable in the RMAA software.

    Then, connect your DAC to your computer so that audio is being transferred to the DAC (whilst recording remains with your soundcard). Connect the DAC's analogue out to the line-in on your soundcard, and perform the test again. This time, you are sending the test signal from your computer to your DAC, which is outputting it in the analogue domain to your soundcard to be recorded.

    You will now have two results: The reference loop-back test for your sound card, and the results of your DAC as fed by your computer. Make sure you do both of these tests at the same resolution and sampling rate. 24/96 should reveal the full extent of your soundcard and DAC's dynamic range. You can also repeat the test at different resolution and sampling rates to see if the data changes. If the 16/44 data for my DAC is correct, then it looks like it performs better at higher resolutions.

    Looking forward to seeing your results! :)
     
    Last edited: May 12, 2016
    acdc7369, Krzych, Plan9 and 2 others like this.
  2. Jimi Floyd

    Jimi Floyd Forum Resident

    Location:
    Italy
    Windows only, unfortunately.
     
  3. Shawn

    Shawn Forum Resident

    Are you out of luck if you don't have a dedicated sound card?
     
  4. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    You could use your computer sound card if it has a line-in, but you will need to do the loop-back test first so we can at least see the limitations (if any) of your computer's sound card :)
     
    Shawn likes this.
  5. c-eling

    c-eling Forum Resident

    No way in hell I'm using the line in on my PC, absolute garbage :laugh:
    Neat test though Rob :cheers:
     
  6. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    Test it and see :)
     
  7. russk

    russk Forum Resident

    Location:
    Syracuse NY
    Wow that is cool. Sadly I use a tablet PC. Then again I have a DAC-r to so thanks for the information.
     
    Ntotrar likes this.
  8. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    Tomorrow I am going to re-test the Rega at 16/44, but setting my ADC at 24 bit to see how much of my soundcard's 24 bit container the Rega DAC can fill playing a 16 bit test signal :)

    (this is also a bump) ;)
     
  9. Archimago

    Archimago Well-Known Member

    Good job!

    Welcome to the world of objective measurements with the ability to test not just your DAC but also other things like the difference between streamers/laptops/computers/cables/etc...

    Yes, do try the ADC set at 24/44 to see just what the Rega with a 16/44 test signal can do. I bet you'll see the noise level and dynamic range end up being similar... Around 96-97dB as expected for a clean 16-bit signal. With both the Rega and ASUS locked at 16/44 you're probably seeing the cumulative limitation of the 16-bit noise floor both on the DAC + ADC ends.

    Also, try the different filter settings and see what happens with the frequency response at 16/44. At 24/192, it really doesn't matter what filter you're using since the only difference is >20kHz. How the roll-off looks at 44kHz is what will distinguish the "sound" of each of the DAC's settings.

    Have fun!

    While you're at it... Go start a blog :)
     
  10. back2vinyl

    back2vinyl Forum Resident

    Location:
    London, UK
    Very interesting. I would like to try this but it may take me a day or two. In the past I've found the weakness of doing comparative tests using this kind of loop-back is the line-in on your PC's sound card which distorts the results and can be very noisy. From a quick read, it seems this test calibrates for that effect so maybe that's a remedy, although ideally I think you'd have two PC's and record the output from one into a high quality input on the other.
     
  11. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    I will certainly test the different filters at 16/44 also! It was your blog that inspired me to test my DAC (and led to me running into an annoying issue with my Raspberry Pi!) Looking forward to your next article, whatever that may be.
     
  12. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    Using an external soundcard should remedy that :)
     
  13. back2vinyl

    back2vinyl Forum Resident

    Location:
    London, UK
    But I don't see how! Please follow my plodding logic.

    Let's say I'm starting out by setting up a reference using my noisy, inaccurate internal soundcard.

    The signal path is:

    1. RMAA creates digital test signal in software.
    2. Digital signal converted to analogue by audio codec chip (DAC) on motherboard.
    3. Analogue signal sent through internal PC wiring to headphone jack on PC casing.
    4. I plug a cable with a stereo mini jack into the headphone jack.
    5. I plug the other end of the cable, also fitted with a stereo mini jack, into the line-in microphone jack on the PC casing.
    6. The analogue signal follows this looped-back cable into the line-in microphone jack.
    7. Analogue signal now follows internal wiring to the motherboard.
    8. Analogue signal is now converted back to digital again by same audio codec chip (this time working as an A/DC) on motherboard.
    9. Digital signal now read by RMAA software and compared with original.

    Evidently, there are many opportunities for noise and distortion to enter the signal chain. The big problem, as I see it, is that you don't know how much noise and distortion is coming from any particular part of the chain. This is a big problem because, without knowing that, you can't calibrate for the PART of the chain you will be using in your tests.

    For example, let's say I'm now going to test an external DAC.

    The signal path now is:

    1. RMAA creates digital test signal in software,
    2. Digital signal sent (possibly via internal wiring) to USB output on PC casing.
    3. Digital signal carried by USB cable to external DAC.
    4. Digital signal converted to analogue by external DAC.
    5. Analogue signal sent by analogue cable to line-in microphone jack on PC casing.
    6. Analogue signal now follows internal wiring to the motherboard.
    7. Analogue signal is now converted back to digital again by audio codec chip (this time working as an A/DC) on motherboard.
    8. Digital signal now read by RMAA software and compared with original.

    So, in the real test, how do I know to what extent I'm testing my DAC and to what extent I'm testing the analogue line-in components of my PC? I can't use the reference test to calibrate because the reference test included the line-out section of my PC as well as the line-in. The reference test is pretty much useless unless I can find a way to measure only the line-in section of my PC.

    I don't mean to sound negative - I'm really interested in this idea.
     
  14. darkmass

    darkmass Well-Known Member

    I'm also having trouble understanding how the testing procedure is capable of testing only the DAC operation.

    It seems to me that to test only the DAC, a person needs to be able to make a loop that can function without the DAC, for baseline testing, then have precisely the same loop except that the DAC is now inserted for a proper comparison against the baseline. Does that make sense?

    1) baseline loop

    2) baseline loop + DAC (no more and no less--the DAC can, however, be inserted into the loop at any point)

    But a DAC by its defined operating characteristics takes a digital in and converts that to an analog out. That seems to indicate that the baseline (DACless) loop must have the digital signal, that was the DAC's input, connected directly to the analog (DACless) "line out". Don't work that way, or DACs wouldn't even be needed.

    And if you cannot do a two stage test exactly given by "1" and "2" above, a different system is being tested than "DAC output". While such a test may have its own validity, it really should be properly understood for the system that the test results apply to.
     
    Kyhl likes this.
  15. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    This is why you use the soundcard to set the reference. Clearly, the test you perform is limited by the capabilities of your soundcard and it's uncertain whether the results are limited by the soundcard's DAC or ADC. However, you can check your soundcard results against the manufacturer's published specs and see how they compare. Mine do compare with Asus' specs, which indicates to me that an okay reference level has been achieved. It just so happens that Rega's published specs are "worse" than my soundcard's. Thus, it follows that the soundcard has enough "room" to accommodate the Rega DAC's test signal output.

    Give it a try :)
     
    Shawn likes this.
  16. back2vinyl

    back2vinyl Forum Resident

    Location:
    London, UK
    I'm still not convinced! I can see that your results suggest your Rega DAC probably has an SNR broadly in line with its spec, but I'm not sure how I'd personally use the test to discover anything useful.

    For example, I ran the test, first using the built-in audio codec on the motherboard of my desktop PC, and then using a newly-acquired iFi DSD Nano DAC + headphone amp. I was particularly interested to discover whether the iFi rolled off the bass in any way.

    Here's the result. The chart shows the frequency response of my system, with the built-in codec in white and the iFi in green.

    [​IMG]

    But what use is that? I can see from the chart that the bass is being rolled off somewhere in the system but I have no idea whether the DACs are doing it, the A/DC on the motherboard is doing it or something else in the system is doing it.

    BTW here are the data for the two DACs:

    [​IMG]

    I'm not familiar with a lot of these numbers but it looks to me as if I'm being told that my iFi is far worse in almost every way than the chip on the motherboard, and although I don't rule out that possibility, it does seem a bit unlikely.

    Maybe it's something to do with the testing method. I find that loop-back testing creates terrible noise on my desktop PC, possibly because of some kind of ground loop effect. The noise is different every time but here's how it sounded today, amplified by 15 dB:

    Background noise when using loop-back

    The noise stops as soon as you break the loop.

    I think there are probably some good uses for this test and I'm very interested in it but at first sight I don't really feel it's very well suited for measuring the performance of DACs and I would be very wary of any results obtained through a loop-back test.
     
    Krzych and Shawn like this.
  17. Rolltide

    Rolltide Forum Resident

    Location:
    Vallejo, CA
    "useful" is kind of relative here. It seems to me the most benefit in this is to test variables surrounding the same DAC, like power supplies, cables, etc. Otherwise it doesn't seem like we're doing much more then proving our DACs aren't broken.
     
    F1nut and back2vinyl like this.
  18. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    Is your computer's line-in capable of 24 bit recording? Try the test with 24 bits if possible :)
     
  19. Shawn

    Shawn Forum Resident

    Very interesting thread! I'm wondering that once the baseline results are determined, wouldn't anything different than those baseline results be due to whatever new device (in your case the iFI) you're introducing? I honestly do not know, just throwing this out there to get your thoughts.
     
  20. harby

    harby Forum Resident

    Location:
    Portland, OR, USA
    Not using that software, but just analysis with my own generated tones and software. Here's what loopback on my Creative Soundblaster Audigy 2 ZS looks like:
    [​IMG]
    I have <0.003 THD after piping 1kHz through my receiver and back, at levels right below clipping.

    You can get one at your local computer recycler for under $20 on a lucky day. An "audiophile" DAC should be untestable by a computer sound card, but you can see from your own tests that they hold their own.
     
  21. krlpuretone

    krlpuretone Forum Resident

    Location:
    Grantham, NH
    Theoretically, you could use this to test USB cables as well.
     
  22. back2vinyl

    back2vinyl Forum Resident

    Location:
    London, UK
    Yes, that ought to be true but I don't think it is in this case. I'm rapidly coming to the conclusion that you can't believe anything you see coming out of a loop-back test done on a PC. Any $2 DAC chip will give you an almost perfectly flat frequency response from 20 Hz to 20 kHz and yet look at harby's chart just above this post. How can that possibly be? No DAC on the market today would give you a result like that. And the funny thing is, it bears an extraordinarily close resemblance to the chart I posted for my iFi, a little further up - both charts having been obtained through loop-back tests.

    Or is it the RightMark Audio Analyser that's the problem? Coincidentally I was reading the following review of the iFi last week which provoked an argument about frequency response charts that were obtained using the RightMark Audio Analyser:

    Review with disputed charts

    Again, the disputed frequency response charts showed a rolled-off top and bottom which don't accord with other tests using other methodologies. I'm not here to defend the iFi or any other DAC but at a time when you can make a flat DAC for next to nothing and anyone can test it with free software, it's almost inconceivable that a manufacturer would sell one that performed so poorly in terms of frequency response.

    All very strange.
     
    Shawn likes this.
  23. back2vinyl

    back2vinyl Forum Resident

    Location:
    London, UK
    OK, Robert, I'll see what I can do. :thumbsup:
     
    Robert C likes this.
  24. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    Here are the corrected results for the Rega DAC-R in 16/44 mode:

    [​IMG]

    And the filter frequency responses. You can see a very slight roll-off beginning at 7 kHz. The DAC is described as having a "firm grip on bass", which might be the result of the slightly softening of treble we can see here.

    [​IMG]

    I've zoomed in from 15-20 kHz to show the filter roll-offs.

    [​IMG]

    And the 16/44, 24/96, 24/192 measurements for ease of comparison:

    [​IMG]
     
    Last edited: May 13, 2016
  25. Robert C

    Robert C Forum Resident Thread Starter

    Location:
    London, UK
    Could someone (@Archimago ?) explain what is meant by the positive and negative figures in the frequency response columns?
     

Share This Page