Another test finds HomePod frequency response flat, but results potentially meaningless

Posted:
in General Discussion edited February 15
An acoustical analysis of Apple's HomePod published Wednesday found the speaker boasts a relatively flat frequency response, characteristics often associated with accurate sound reproduction, but those results might be misleading.


HomePod frequency response. Source: NTi via Fast Company


In the third and final installment of its HomePod review, Fast Company tested frequency response using specialized hardware and software from NTi Audio, an acoustic evaluation specialist based in Liechtenstein.

With HomePod set up in a typical living room, the publication saw the device deliver what is characterized as a flat response, meaning the magnitude and phase of output was for the most part in line with input across the frequency spectrum.

Specifically, HomePod output lies within 4 decibels of flat between 70Hz and 6KHz. Further, Total Harmonic Distortion, which in basic terms can be defined as a calculation measuring the difference between a source signal and observed speaker output, was notably slim.

"We found distortion of less than 10% from the 40 Hz to 10,000 Hz range, which is very good, and less than 2.5% from 150 Hz to 10,000 Hz, which is excellent," according to Brian MacMillan, associate general manager at NTi.

The results are similar to measurements seen by Reddit user "WinterCharm," who ran HomePod through a gamut of tests over the weekend and also found the speaker to reproduce "near-perfectly flat" sound.

Both tests found steep decreases in output at the high end of the spectrum, just shy of its 20KHz range limit. The Fast Company evaluation saw its test unit begin to taper on the low end at around 60Hz, results not shown in the Reddit test. And therein lies the problem.

As noted by Reddit user "edechamps" in response to WinterCharm's rundown, acoustic testing was performed in a reverberant room without any apparent steps taken to reduce reflections such as impulse response windowing. Further, the graph in the Reddit post was presented at a scale that does not provide enough granularity to make accurate comparisons.

Whether Fast Company implemented windowing in its procedure is unknown, but the publication did perform multiple measurements from different points of the evaluation room. Still, as the test was not conducted in a controlled environment free of reverberations, its results are questionable.

As edechamps puts it, any conclusion gleaned from a frequency response evaluation that is not performed in an anechoic chamber, or does not attempt to nullify aberrations caused by real-world factors, should be taken with a hefty grain of salt.

Still, Apple was able to create a speaker that performs more than adequately in real-world scenarios, as evidenced by numerous positive reviews lauding the device's ability to reproduce premium sound in a variety of environments. According to Apple, HomePod's adaptive acoustic technology, powered by an advanced DSP, is largely to thank for that level of quality.
«134567

Comments

  • Reply 1 of 123
    Apple advertised “distortion free music” & HomePod seems to do the job. The only other company that advertised it to my knowledge is Devialet. Sonos & Google didn’t say about such a thing.
    edited February 14 watto_cobrajony0
  • Reply 2 of 123
    foggyhillfoggyhill Posts: 4,692member
    An acoustical analysis of Apple's HomePod published Wednesday found the speaker boasts a relatively flat frequency response, characteristics often associated with accurate sound reproduction, but those results might be misleading.


    HomePod frequency response. Source: NTi via Fast Company


    In the third and final installment of its HomePod review, Fast Company tested frequency response using specialized hardware and software from NTi Audio, an acoustic evaluation specialist based in Liechtenstein.

    With HomePod set up in a typical living room, the publication saw the device deliver what is characterized as a flat response, meaning the magnitude and phase of output was for the most part in line with input across the frequency spectrum.

    Specifically, HomePod output lies within 4 decibels of flat between 70Hz and 6KHz. Further, Total Harmonic Distortion, which in basic terms can be defined as a calculation measuring the difference between a source signal and observed speaker output, was notably slim.

    "We found distortion of less than 10% from the 40 Hz to 10,000 Hz range, which is very good, and less than 2.5% from 150 Hz to 10,000 Hz, which is excellent," according to Brian MacMillan, associate general manager at NTi.

    The results are similar to measurements seen by Reddit user "WinterCharm," who ran HomePod through a gamut of tests over the weekend and also found the speaker to reproduce "near-perfectly flat" sound.

    Both tests found steep decreases in output at the high end of the spectrum, just shy of its 20KHz range limit. The Fast Company evaluation saw its test unit begin to taper on the low end at around 60Hz, results not shown in the Reddit test. And therein lies the problem.

    As noted by Reddit user "edechamps" in response to WinterCharm's rundown, acoustic testing was performed in a reverberant room without any apparent steps taken to reduce reflections such as impulse response windowing. Further, the graph in the Reddit post was presented at a scale that does not provide enough granularity to make accurate comparisons.

    Whether Fast Company implemented windowing in its procedure is unknown, but the publication did perform multiple measurements from different points of the evaluation room. Still, as the test was not conducted in a controlled environment free of reverberations, its results are questionable.

    As edechamps puts it, any conclusion gleaned from a frequency response evaluation that is not performed in an anechoic chamber, or does not attempt to nullify real-world , should be taken with a hefty grain of salt.
    I truly don't get the last fracking paragraph... Man, it's like they don't know wth they're testing.

    Yeah, all other speakers are shit at doing something because they couldn't do it, so hey, lets test them in a way were we don't know how on earth they'll sound in the real world and say that actually means something....

    And you wonder why people have stopped buying good stereo amps and speaker pairs, well that kind of talk says it all.


    netmagejony0
  • Reply 3 of 123
    Edechamps dorsnt know anything so why are you repeating what he said when you don’t understand it anyway. 
    ‘Nobody will is this in a treated room and omnidirectional wouldn’t work in a treated room anyway. 
  • Reply 4 of 123
    slurpyslurpy Posts: 5,008member
    I think we've seen that the consensus is that the HP sounds really, really fucking good. And now that we know that just the raw component cost is around $220, the price is pretty damn good as well.
    lolliverwatto_cobrazroger73williamlondonjahbladeStrangeDaysjony0randominternetperson
  • Reply 5 of 123
    Is this an editorial?
    brakken
  • Reply 6 of 123
    snookie said:
    Edechamps dorsnt know anything so why are you repeating what he said when you don’t understand it anyway. 
    ‘Nobody will is this in a treated room and omnidirectional wouldn’t work in a treated room anyway. 
    I think what people are missing in this reddit argument is even though that reddit guy doesn't have an anechoic chamber to do the test but Apple has one, and PS had boasted about "distortion free music" on stage of WWDC. It's hard to think he would do that lightly without measurements to back it up.
    edited February 15 lolliver2old4funjony0
  • Reply 7 of 123

    Whether Fast Company implemented windowing in its procedure is unknown, but the publication did perform multiple measurements from different points of the evaluation room. Still, as the test was not conducted in a controlled environment free of reverberations, its results are questionable.

    As edechamps puts it, any conclusion gleaned from a frequency response evaluation that is not performed in an anechoic chamber, or does not attempt to nullify real-world, should be taken with a hefty grain of salt.


    They don't list all the equipment, just the XL2 to measure. They obviously need a signal source, and NTi's tool for this (I can only assume they used only their tools and not something from another company) can output signals for impulse response. Whether or not they bothered to do so is unknown.

    Second paragraph is interesting. "anechoic chamber, or does not attempt to nullify real-world".

    So...you CAN perform these tests if you take steps to nullify the room? Like doing an impulse response? Funny, I was told an anechoic chamber was necessary. /s
    edited February 15 lolliver
  • Reply 8 of 123
    Listing measurements of distortion and bass response are pretty meaningless without specifying the SPL.
    Also, the graph shows a range of 8dB in response between 60Hz and 7KHz, or ± 4dB.
  • Reply 9 of 123
    The reason for an anechoic chamber is that room reflections cause variations in frequency response measurements that have nothing to do with the speaker.

    HOWEVER, Apple says the HomePod performs automatic room analysis and acoustic correction. That means the variations caused by reflections in the room should be nullified by the HomePod's automatic correction, eliminating the need for an anechoic chamber because the variable the anechoic chamber is intended to eliminate SHOULD be eliminated by the HomePod itself.

    If variations in frequency response exist as a result of the HomePod being tested in a reflective environment (i.e. not an anechoic chamber) it means the HomePod's automatic room correction is not effective (or not effective enough).
    edited February 15 baconstangcjaerlollivermuthuk_vanalingamlarryawilliamlondonnetmagejony0macgui
  • Reply 10 of 123
    When selecting speakers for production work, one of the things the buyer has to be careful about is not being seduced by speakers that sound "pleasing" even though they're not very accurate. An accurate speaker should reveal flaws in the source that a pleasing speaker may mask.

    The flip side of the equation is that, with certain source material, an accurate speaker may be less enjoyable to listen to than a pleasing one. So for home entertainment the question becomes one of priorities: would you rather have a speaker that is as accurate as possible, revealing every detail in the source, including the warts, or do you prefer something less likely to ever offend the ears?

    I was initially alarmed by the high-end roll-off of the HomePod. Then I remembered the guidelines for balancing monitoring in the studio call for some high-end roll-off: 2db per octave starting at about 2,000 Hz. That would put the output of the monitors down about 6dB-ish at around 15,000 Hz. The HomePod is in that general ballpark, so the response may actually seem quite natural and pleasing, rather than "lacking highs" as a quick glance at the graph might lead one to expect.

    The bottom line is that even if the HomePod is not as accurate as a production monitor chain, it might be quite enjoyable and pleasing to the ear. If it sounds good in your setting (and the test results indicate that setting DOES matter, despite Apple's claims to the contrary), and if deadly accuracy is not important to you as long as the experience is pleasant and enjoyable, then none of the pontificating by experts and debate over testing methods matters.
    d_2lolliverbaconstang2old4funroundaboutnowjony0randominternetpersonmacgui
  • Reply 11 of 123
    zoetmbzoetmb Posts: 2,292member
    That's not a flat frequency response.  Back in the hi-fi days, the measurement (with pink noise input) had to be from 20 to 20,000 Hz +/- 3db with no more than X% distortion (preferably 1% or under).

    But even if it was flat, a flat frequency response won't sound very good to most people.  That's why hi-fi equipment had tone controls and loudness compensation (which increased the bass and treble at low volume levels according to the Fletcher-Munson curve and which gradually had less effect as the volume was raised).

    I happened to be at an Apple store yesterday to listen to the HP.   IMO, it has an over-emphasized bass.   On new, digitally recorded material, it sounded pretty good.  But on tracks I tried from the 1960's-1970's (which I was pleasantly surprised were in the Apple Music library), it sounded kind of dull.   And while the Apple person was a bit confused and had to look things up, a single unit is NOT stereo.   At first, he claimed it was because "it contains lots of speakers", but when I told him that the number of speakers was not relevant and that it sounded mono to me, he did some more research and admitted it would take two for stereo and that stereo is not currently supported even with two units, but it would be with some future update.   

    Most of the reviews said the sound was great but that Siri sucked.  But in my tests and even in the noisy store environment, Siri did really well - it never played the wrong track or artist that I called out and it properly heard me say "louder" and "softer" and "pause".  It was far more accurate than the Apple TV achieved in the same environment. 
    baconstang2old4funlarrya
  • Reply 12 of 123
    foggyhillfoggyhill Posts: 4,692member
    zoetmb said:
    That's not a flat frequency response.  Back in the hi-fi days, the measurement (with pink noise input) had to be from 20 to 20,000 Hz +/- 3db with no more than X% distortion (preferably 1% or under).

    But even if it was flat, a flat frequency response won't sound very good to most people.  That's why hi-fi equipment had tone controls and loudness compensation (which increased the bass and treble at low volume levels according to the Fletcher-Munson curve and which gradually had less effect as the volume was raised).

    I happened to be at an Apple store yesterday to listen to the HP.   IMO, it has an over-emphasized bass.   On new, digitally recorded material, it sounded pretty good.  But on tracks I tried from the 1960's-1970's (which I was pleasantly surprised were in the Apple Music library), it sounded kind of dull.   And while the Apple person was a bit confused and had to look things up, a single unit is NOT stereo.   At first, he claimed it was because "it contains lots of speakers", but when I told him that the number of speakers was not relevant and that it sounded mono to me, he did some more research and admitted it would take two for stereo and that stereo is not currently supported even with two units, but it would be with some future update.   

    Most of the reviews said the sound was great but that Siri sucked.  But in my tests and even in the noisy store environment, Siri did really well - it never played the wrong track or artist that I called out and it properly heard me say "louder" and "softer" and "pause".  It was far more accurate than the Apple TV achieved in the same environment. 
    It's not mono either so not sure wth you're saying and I've got very good hearing too, or so says the audiologist who tested me.

    If you read about it, you'd know that it actually uses Fletcher-Munson loudness compensation,
    So, the curve is EXACTLY where they want it to be.

    Many of the1960s to 1970s recording are truly horribly mastered and sound like crap.
    (and yeah, I've collected music since the late 1970s and got things that go back to the 1920s)

    Why not use some live modern recordings, some have minimal post processing to test instead.

    Compressed audio of mainstream pop is probably the worst music to test with any speaker.


    baconstangbb-15
  • Reply 13 of 123
    foggyhill said: 
    It's not mono either
    This is one of several claims Apple makes about the HomePod that make me scratch my head a little...

    Apple says the HomePod beams "direct" sound to the middle of the room and "ambient" sound to the left and right. But then Apple ALSO claims that the HomePod sounds the same no matter where you are in the room. Those are conflicting statements.

    Since people are saying the sound of the HomePod does remain very consistent as one moves around the room, the logical conclusion is that there can't be much in the way of directional cues coming out of it. If there were, listeners would perceive a change in the sound as they moved around.

    If that assessment is correct, it means that despite what Apple's marketing claims, it's really essentially a mono device.
    muthuk_vanalingamlarryawilliamlondon
  • Reply 14 of 123
    irnchrizirnchriz Posts: 1,553member
    So, unless you are sitting in an anechoic chamber with it all of the time, it sounds ok?
    edited February 15 netmagepotatoleeksoup
  • Reply 15 of 123
    jidojido Posts: 102member
    foggyhill said:
    Still, as the test was not conducted in a controlled environment free of reverberations, its results are questionable. 

    As edechamps puts it, any conclusion gleaned from a frequency response evaluation that is not performed in an anechoic chamber, or does not attempt to nullify real-world , should be taken with a hefty grain of salt.
    I truly don't get the last fracking paragraph... Man, it's like they don't know wth they're testing.

    Yeah, all other speakers are shit at doing something because they couldn't do it, so hey, lets test them in a way were we don't know how on earth they'll sound in the real world and say that actually means something....

    And you wonder why people have stopped buying good stereo amps and speaker pairs, well that kind of talk says it all.


    You don't get it. Unless you measure this in a controlled environment (anechoic), the flat response is meaningless - it could very well sound like crap to human ears and have a pretty nice response curve according to the instruments.

    That is not the case of the HomePod, which sounds great according to users. But you can't make conclusions about sound from the graphs.
  • Reply 16 of 123
    I think that for an object that does Fletcher-Munson loudness compensation,and that does beam forming analyzing the environment where it is (and this do not necessarly include only frequency response compensation, it may include phase, delay compensation and what not), measuring frequency response just make no sense, either in a real room or in a anechoic chamber. Result depends, by definition, on the environment you are measuring them.

    An objective, scientific approach would require a lot of work, if only to define the reference framework you are defining.

    And anyway, this kind of approach, while heralded by hifi equipment producer, doesn't really correspond to the final user experience; it is useful only for places like recording studio, where you design a complete system (room, electronics, speakers, etc) by an high end professional.

    For example, the artifact of compression will have a stronger effect on the sound than the speaker frequency response.

    I strongly suggest every body relax and go back to good old subjective strategy: if it sound ok for you, it is ok. If anything in this price range sound better for you, buy elsewhere.

    Maurizio
    2old4fun
  • Reply 17 of 123
    Can we all just give this a rest and enjoy the HomePod? Or don’t. You do you. But these reviews are getting quite ridiculous. It’s like people are tripping over each other trying to get the farthest decimal place. 

    when I’m listening to music, graphs and numbers are honestly the last things on my mind. 
    gatorguy2old4funlarryawilliamlondonbb-15StrangeDaysjcs2305potatoleeksoup
  • Reply 18 of 123
    sully54 said:
    Can we all just give this a rest and enjoy the HomePod? Or don’t. You do you. But these reviews are getting quite ridiculous. It’s like people are tripping over each other trying to get the farthest decimal place. 

    when I’m listening to music, graphs and numbers are honestly the last things on my mind. 
    We are in rationalization mode now to explain the less than unanimous reviews. For many, admiration of Apple and its products is now a religion.  If you like it, buy and enjoy it. But learn to accept that others, who are just as intelligent, may not come to the same conclusions. 
  • Reply 19 of 123
    lkrupplkrupp Posts: 6,091member
    I have just one question. Did the Google and Amazon products go through all this analysis by audiophiles, all this editorializing, all this discussion (pro and con) when they were released? To be honest I had never heard of the Google Home Max until it started being compared to the HomePod.
    williamlondonnetmagetmayStrangeDaysrandominternetperson
  • Reply 20 of 123
    wizard69wizard69 Posts: 12,468member
    A flat response may indicate electronics quality but it is meaningless because almost no body leaves their tone controls alone.  
Sign In or Register to comment.