Do measurements tell you how good a DAC sounds, and is Rob an objectivist or a subjectivist?

Measurements are necessary but not sufficient

In August 2014, on the Hugo thread, Watts wrote that "Measurements are very important, as they tell you what is wrong objectively", while also saying that "you will never measure problems with interconnect cables, but you can hear the effect of changing interconnect cables" Head-Fi, Aug 2014. In July 2015 he wrote that "if you can measure an error you can probably hear it, but I have had countless times where I hear a clear and easily discernible difference but with absolutely no measurable difference" Head-Fi, Jul 2015. Giving DAVE's distortion and noise figures that month, he said "But these numbers, although very important, won't tell you how good it sounds." Head-Fi, Jul 2015.

The rule he states most often is one directional: lower measured distortion, with nothing else changed, sounds better. He said so for DAVE in December 2015, for Hugo 2 in March and April 2017, at RMAF in October 2017, to Music Photo Life in March 2018 and to What Hi-Fi? in March 2022 Head-Fi, Dec 2015Head-Fi, Mar 2017Head-Fi, Apr 2017miceblue425 50:53article, Mar 2018article, Mar 2022. In his 2016 blog post on listening tests his rule of thumb was "if I can measure it, and it is signal dependent error, then its audible" Head-Fi, Mar 2016. At CanJam NYC in late February 2023 (video published March 2023) he repeated the rule but said of the "everything else unchanged" condition: "And that's a big assumption, by the way." The Headphone Show 40:57.

The converse does not hold for him. In March 2023 he said conventional measurements "can give you an inkling as to how bad something will sound, they certainly won't tell you how good it will sound" Head-Fi, Mar 2023, and in an interview the same month: "They tell you how bad something is. They'll never tell you how good something is." Eric's Hi-Fi 28:42. His example there was valve power amplifiers, which he said generally measure worse than solid state but are likely to have better sound staging and depth and sound more musical Eric's Hi-Fi 28:42Eric's Hi-Fi 29:25. His next sentence reads "That's not to say that if you then take that valve power amplifier and reduce the distortion by a hundred times you will get better sound quality so long as you don't screw up anything else"; the context suggests he meant that reducing the distortion would still help, but the wording should be checked against the recording Eric's Hi-Fi 29:25. In August 2022 he wrote that most buyers want to know how a device will improve their musical enjoyment, "And you can't get that from measurements." Head-Fi, Aug 2022. In March 2021 he wrote "I could easily put together a DAC that had poor technical performance - as defined conventionally - but sounded fantastic." Head-Fi, Mar 2021. In an interview published May 2026 (recording date not stated) he said "some things that you can measure have no impact on sound quality", and that listening is how he decides whether an error matters Audio Force 3:30Audio Force 5:12.

In March 2017, publishing his Hugo 2 slides, he wrote "I can tell from a suite of measurements pretty much how something will sound", along with the designer's intent and ability Head-Fi, Mar 2017. In February 2023, talking about linear power supplies, he said "Measurements don't matter. What's important is the sound quality.", in the sense that measurements are used to reduce problems already identified Soundnews 35:05. That line is about linear power supplies, in a Q&A.

Audible errors below what can be measured

His reason measurements cannot settle sound quality is that he reports hearing errors smaller than analog test gear can resolve. He measures these with digital simulation (Verilog) of each module, which he says shows errors at -300 dB and below, and then listens to them Head-Fi, Oct 2017Head-Fi, Apr 2018Head-Fi, Apr 2022article, Mar 2022. Examples he gives include noise shapers developed for DAVE improving in depth from 200 dB to 350 dB performance (June 2017, November 2017, April 2022), a sample rate converter for his ADC project audible at -310 dB versus -350 dB aliasing (May 2021), and noise floor modulation below measurable limits (June 2015 and March 2024) Head-Fi, Jun 2017Head-Fi, Nov 2017Head-Fi, Apr 2022Head-Fi, May 2021Head-Fi, Jun 2015MFG Audio Reviews 15:45. He ties this to cables measuring identically but sounding different, in 2018 (a November 2018 open-day talk), 2021, 2024 and late 2025 (a podcast published in March 2026) Pursuit Perfect System 29:21Head-Fi, Nov 2021MFG Audio Reviews 15:45ecoustics 47:23. In January 2018 he described cutting DAVE's distortion harmonics from -130 dB to below -150 dB and hearing it, saying "-130 dB harmonics are well below the threshold of audibility, and reducing it should not be audible; but it is nonetheless" Head-Fi, Jan 2018. A day later he hedged: "I for sure recognise the strong possibility that the change in SQ had nothing to do with the measured change but with something else that had changed", citing later tests as further evidence Head-Fi, Jan 2018. Figures from transcripts should be checked against the recordings.

Which measurements he thinks matter

He dismisses the headline numbers. He called DAC dynamic range "a pretty meaningless number" for DAVE in February 2016, wrote in January 2020 that "SINAD and distortion are very simple tests and do not tell you how it sounds like at all", and in July 2022 that "SINAD is worthless as a useful measurement parameter", arguing that a fixed inaudible noise level matters less than distortion and noise floor modulation Head-Fi, Feb 2016Head-Fi, Jan 2020Head-Fi, Jul 2022. He still publishes SINAD-type figures: 125 dB for Mojo 2 and 127 dB for Hugo TT2, A-weighted and referenced to full output, in February 2022 Head-Fi, Feb 2022.

What he values instead: noise floor modulation, which in April 2018 he called "one of the top 3 performance indicators that correlate with subjective sound quality with DAC's" Head-Fi, Apr 2018; small-signal accuracy, tested by passing a -301 dB signal through each digital module with amplitude and phase held within tight limits Head-Fi, Apr 2022Head-Fi, Jan 2022; and, for chip DACs in 2021, brick wall response on a noise test, small signal accuracy and noise floor modulation Head-Fi, Mar 2021. He says standard tests miss transients: "All our measurements that we do None of them have transients because they're just simple steady state signals." (Passion for Sound interview, Part 3, published December 2019; recording date not stated), and in January 2025 that the industry's steady state measurements are "ignoring the very thing that makes music music" Passion for Sound 20:54Head-Fi, Jan 2025. In January 2018 he said he was developing a test to put a number on transient error, and worried the measured difference between DAVE's taps and the M Scaler's 1M taps might be small Head-Fi, Jan 2018. In January 2025 he said he had built such a test, which showed the M Scaler 22 bit accurate, but did not pursue it because "a raw number does not tell you what happens perceptually" Head-Fi, Jan 2025.

Test gear, setup and published reviews

Watts treats measurement as fallible. In March 2017 he wrote "A measurement is not objective reality, it is just data subject to error done at a particular time with particular test equipment.", and that dropping measurements is "throwing out the baby with the bath water" Head-Fi, Mar 2017. In his RMAF 2017 DAC design masterclass (given in October 2017, video published December 2017) he said "Measurements is a kind of objective reality. It's not the reality" RMAF 47:15. In July 2019: "It is not an objective reality as people make out." and "So measurements, like listening tests, must be done very carefully, and conclusions reached must be tentative and subject to constant reappraisal." Head-Fi, Jul 2019. In September 2022 he named two problems: the correlation with listening, "which is very weak and subject to interpretation", and the tester's intent, adding that his own published measurements show the device in its best light but are typical units Head-Fi, Sep 2022.

He says his DACs need an Audio Precision APx555 Head-Fi, Mar 2017Head-Fi, May 2018. In April and May 2018 he put Stereophile's poor DAVE results down to their SYS2722, which he called inadequate to measure DAVE Head-Fi, Apr 2018Head-Fi, May 2018. In August 2021 he wrote that the review unit, once returned to him, "measured perfectly" on his APx555, and that he "could find no technical reason to explain Stereophile's findings" with the SYS2722, while still calling it inadequate; in July 2022 he said he "never got to the bottom of why" Head-Fi, Aug 2021Head-Fi, Jul 2022. In March 2023, asked about criticism of DAVE and M Scaler measurements (the interviewer and Watts refer to ASR reviews), he said criticizing the M Scaler on jitter "doesn't make any kind of sense because it's not relevant", and that "the APx555 is incredibly sensitive to ground", describing a manufacturer whose bad TT2 and M Scaler results came from ground loops Eric's Hi-Fi 24:01Eric's Hi-Fi 25:05Eric's Hi-Fi 25:47Eric's Hi-Fi 26:36.

Objectivist or subjectivist

In his first blog post (March 2016) he framed the debate as between "the extreme objectivists (you don't need to listen it's all nonsense)" and "the extreme subjectivists (you don't need to measure it's all nonsense)", and argued for structured listening while accepting the objectivists' points about placebo and bias Head-Fi, Mar 2016. In May 2019 he described himself as "a measurement/objective listening based designer, and am driven entirely upon measurements and carefully controlled listening tests." Head-Fi, May 2019. In Part 4 of the Passion for Sound interview cited above, published in January 2020 (the series ran from December 2019; recording date not stated): "Throwing listening tests away is a bad, bad thing. You have to use both." Passion for Sound 13:50. In May 2021: "Objectivity is absolutely essential", meaning a heard change must have a science based reason that explains it Head-Fi, May 2021.

Answering a poster's "Objectivist or subjectivist?" on the DAVE thread in March 2023, he said he spends most of his time on objective measurement and simulation but "So subjectivism trumps objectivism.", with the condition that "you have to be dreadfully careful about subjective assessment - it needs to be done objectively" through repeated or blind tests Head-Fi, Mar 2023. His other statements on priority run the same way: "Of course, listening takes priority." (What Hi-Fi?, March 2022) article, Mar 2022; "Sound quality listening tests is a reality, and it's the most important reality." (March 2023) Eric's Hi-Fi 30:09; "I'm technically based, but I do a lot of careful listening." (September 2023, CanJam SoCal) The Source AV 23:59; and "So measurements are important, but listening is by far much more important" (interview published May 2026; recording date not stated) Audio Force 5:12. He keeps the condition attached: in May 2024 he wrote "we can certainly be objective in characterising subjective impressions" if it is "rigorously done by a sensitive listener who actually knows what real acoustic instruments sound like" Head-Fi, May 2024, and, in an interview published October 2026 (recording date not stated, though the transcript places it before May 2026), he said "that subjective listening test needs to be an objective measurement" HiFi Buys 7:20. In an interview published August 2022 (recording date not stated) and again in the October 2026 interview he described two camps of designers, those who only measure and those who only listen, and placed himself in neither; in the 2026 interview he said "I'm strong on both those factors" AVstore 43:59AVstore 44:33HiFi Buys 8:50. In October 2014, discussing how listening tests can mislead, he wrote that "musicality is always the most important arbiter of performance", and in the same post "So I never do serious listening until it has passed lots of measurement tests." Head-Fi, Oct 2014.

How he listens

He measures first, then listens; he says he does no listening tests until a design measures properly Head-Fi, Oct 2014RMAF 48:25article, Mar 2022. His routine tests are sighted AB comparisons of one variable at a time on short excerpts; when a result seems implausibly small he runs single blind tests, usually with his son Daniel, who is trained to describe sound in the same terms Head-Fi, Oct 2014Head-Fi, Mar 2016Head-Fi, Jan 2023The Source AV 35:31. He is wary of double blind testing: in March 2016 he wrote that "double blind testing, where stress is put on listeners can degrade sensitivity by two orders of magnitude" Head-Fi, Mar 2016; at RMAF in October 2017, answering an audience member who said listeners fail double blind tests, he began "I have big problems with with with double blind testing." (the transcript breaks off there) miceblue425 1:00:28; and in December 2019, answering a buyer who asked about blind testing the M Scaler, he wrote "Blind tests are not very sensitive, as people worry about what they are trying to listen too." Head-Fi, Dec 2019. He cites his own single blind checks as support: in March 2024 he said of the 330 dB versus 350 dB noise shaper change that "if I do this as a blind listening test, which I have done, you can still perceive these changes", and in an October 2025 podcast (published March 2026) that a blind listener came back with the same result he heard MFG Audio Reviews 45:45ecoustics 39:09. In June 2017 he conceded that confirmation bias in his non double blind noise shaper tests could not be ruled out, while saying repeated tests convinced him the results were real Head-Fi, Jun 2017. In April 2022 he wrote that his observations "should be treated as just my opinion", that "It is entirely rational to be sceptical", and that his listening tests "would be considered anecdotal and not scientific evidence; but my job is to close the gap from live to recorded sound, and the tests I do are extremely carefully done and objective…" Head-Fi, Apr 2022Head-Fi, Apr 2022. He advises buyers to listen at home for themselves rather than trust a review or him Head-Fi, Sep 2022Head-Fi, Jan 2020.

What he has not said: he names no single measurement that predicts how good a DAC will sound, and he has not published controlled listening test data for his sub-measurement claims; he says he has not written them up for the AES because of time Head-Fi, Apr 2022Head-Fi, Jan 2025.

Quotes

“Measurements are very important, as they tell you what is wrong objectively.”2014-08-24 Head-Fi, Aug 2014

“I have had countless times where I hear a clear and easily discernible difference but with absolutely no measurable difference.”2015-07-19 Head-Fi, Jul 2015

“They tell you how bad something is. They'll never tell you how good something is.”2023-03-25 Eric's Hi-Fi 28:42

“So subjectivism trumps objectivism. But I must add that you have to be dreadfully careful about subjective assessment - it needs to be done objectively”2023-03-03 Head-Fi, Mar 2023

Dated positions

Caveats and open points

• The February 2023 line 'Measurements don't matter' is from a transcribed Q&A about linear power supplies and conflicts with his many statements that measurements are vital; it is reported in context, not as his general position.
• The March 2017 claim that he can tell from measurements pretty much how something will sound sits against his 2015 and 2023 statements that measurements cannot show how good something sounds; he does not reconcile them.
• His sub-measurement audibility claims rest on his own listening tests, mostly sighted with single blind checks; he calls them tentative and his opinion.
• The interviewer, not Watts, introduces the objective versus subjective framing in the 2023 Eric's Hi-Fi and 2026 Audio Force interviews, and a forum poster raised it in the March 2023 DAVE thread post; only Watts's replies are cited for his position.
• The German Hifistatement DAVE text found in search is the publisher's translation and is not quoted.
• The March 2023 Eric's Hi-Fi transcript line on reducing valve amplifier distortion reads as a negation; the answer reports its likely sense but flags it.
• Stereophile DAVE measurements: his explanation shifted from blaming the SYS2722 (2018) to saying he could find no technical reason (2021) and never got to the bottom of it (2022).

Topics: Measurement vs listening, Measurements, test equipment and simulation