A deep dive into HomePod's adaptive audio, beamforming and why it needs an A8 processor

Posted:
in General Discussion edited January 2018
The HomePod stands to be an audiophile's ideal speaker when it ships on Feb. 9, in no small part due to the considerable effort Apple has put into audio processing for the device. Audio expert Matt Hines of iZotope explains some of the technical hurdles Apple and other smart speaker producers have to overcome to provide a high-quality audio experience in a wide variety of environments.




The first problem Apple and similar firms face when creating an intelligent speaker, like the HomePod, is to ensure there is an optimal audio experience for the user, regardless of where they are in the room. Hines, product manager of Spire by iZotope and guest on this week's AppleInsider Podcast, says the exact position of the user can dramatically affect the sound from a normal speaker, with the problem compounded by local acoustics.

Moving a speaker around a room alters the sound reflections in a dramatic way, Hines suggests, such as a loss of bass frequencies when moving towards the corner of a room. While bouncing around the room, the reflections can multiply the sound in different ways, potentially causing some resonant frequencies to become inconsistently louder, an effect that is particularly noticeable for longer wavelengths.

"Depending on where the ear is, you'll hear sudden, violent variations in frequency levels, which is why bass might disappear in the corner of a room," advises Hines. "It also helps explain why you'll argue with your family about whether the TV is too quiet or loud enough -- and you could all be right -- just sat in juxtaposed locations within a room."

Items located within the local environment can also change sound quality, with soft furnishings absorbing some noise while harder surfaces have a tendency to reflect it. This is why places relating to audio production or audio consumption, like a recording studio or a high-end home cinema room, carefully select the materials they place on their walls, floors, and ceilings to shape the sound to the user's needs.

In the case of Apple, it's using array of six microphones and a digital signal processor (DSP) to understand the environment based on its acoustics, and adapt the device's output to better suit its physical location and the room's audio profile. Considering the multiple speakers and the microphone array, it is possible for such a system to customize the output of each speaker to allow for a similar sound to be heard through as much of the environment's space as possible.




If HomePod is running a realtime DSP that can alter the sound emitted from each of the device's seven tweeter speakers, it can constantly change the profile even if the environment itself changes, such as the mass of the listener moving to a different location in the same area. The experience is consistent regardless of position, with Hines adding the consistency will make listening to music "more seamless and more immersive."

The DSP and this adaptive acoustic processing system will also help when the HomePod gains its stereo audio feature in a future update, one that will offer stereo audio split across two HomePods.

In a typical stereo audio setup, "adjusting the listening position even an inch will have a very material impact on the arrival time of the audio to your ear," Hines stresses. "The crosstalk between speakers is now different and the room reflections are too."



This raises an issue for the HomePod if it plays straight stereo without any adaptations, as the audio will almost certainly be affected by the position of the user. "A true stereo setup for HomePod would be a bad user experience, which Apple aren't in the business of offering."

By using this audio adaptation, which Apple has attempted to patent, it means that the HomePods are able to take each other into account, so the stereo experience will still be present and enjoyable for those near enough to them to hear it, while still keeping the audio listenable in other areas of the room.

The microphone array, DSP, and speaker collection also helps the HomePod in other ways, aside from playing audio. As the HomePod has voice-enabled features, such as those used by Siri, the system also helps with its reception of speech from users.

The beamforming technology allows the HomePod to determine the position of the user relative to the device, and in turn adjust its microphone settings to better acquire audio from that direction. In effect, the audio feeds from all of the microphones in the array are analyzed for speech, with any extra noises picked up by the array used to remove extraneous sounds from the sound of the voice, making it clearer.

"A beam-form mic array means that while there are multiple mics pointed in every direction, they can weight the signal received by each mic differently depending on a sort of correlation matrix," Hines explains, continuing that the processing "helps reject environmental surrounding ambience, and highlight only the more transient, desired speech material."




Hines illustrates the difference in audio quality via an example of typical speakerphone systems, which use a single microphone that degrades in audio quality if the user moves further away or turns to speak in a different direction. While using an array of microphones pointed in all directions would provide more than enough coverage of a room to pick up the user's voice wherever they are, it would also pick up unwanted sounds from around the room that would need processing to remove.

"This weighting is a constant computational process, so as you walk around a HomePod, those microphones are smart enough to know about it, and to ensure that you sound just as clear."

This system also enables acoustic echo cancellation, significantly reducing the potential for feedback, an issue commonly endured when a speaker is placed too close to a connected microphone, with the resulting audio loop usually causing a high-pitched whine. Hines notes HomePod is intelligent enough to recognize occasions where the sound its making is being fed back to itself, such as a reflection from a wall or other nearby flat surface, allowing it to cancel that sound from the audio signal.

The echo cancellation and beamforming technology will help with more than the ability to hand off calls from an iPhone to the HomePod. Providing as clean an audio feed as possible will help improve Siri's voice recognition accuracy, making it less likely to misinterpret a command or a query.

"Simply put, there is a ton of stuff happening automatically in order to offer you a smart speaker that sounds good and works well regardless of where you are in your home," Hines declares. "Elegant simplicity is the hardest thing to build by far."

Hines notes all of this audio processing, as well as other features like AirPlay 2, Siri, and HomeKit, is performed using the A8 processor, as used in the iPhone 6. Apple's need to use such a powerful chip to perform all of these functions and more, which Hines suggests could be seen by users as seemingly doing "less" than the usual workload of an iPhone is a "testament to how much is actually going on inside" the HomePod.
mmatzTomEanantksundaramlostkiwijahbladelolliver
«1345

Comments

  • Reply 1 of 89
    dewmedewme Posts: 2,098member
    Very nicely done article! Thank you. 
    lostkiwisergiozScot1RobPalmer9jony0lolliverwatto_cobra
  • Reply 2 of 89
    LucioguidoLucioguido Posts: 9unconfirmed, member
    I am extremely curious how the HomePod will respond to other speakers playing music. I’m planning on using an airplay receiver to pair (eventually) a couple HomePods to a top tier subwoofer and other high end speakers. My guess is that the audio analysis and beam forming capabilities of the HomePod will pair up perfectly to meet offer some truly astounding audio quality and sound stages. 
    lostkiwiwatto_cobra
  • Reply 3 of 89
    robjnrobjn Posts: 210member
    The HomePod is in some respects better than Stereo. The DSP respects the intent of the original stereo mix. For example, if it detects backing vocals mixed toward the left channel it will send these more toward the left tweeters. At the same time lead vocals will be beamed to the center of the room. This creates a sound stage intelligently from the stereo mix and it allows each of the 7 tweeters to be dedicated to reproducing a smaller number of sounds.

    There is just one woofer in a HomePod but this fact does not completely destroy the stereo effect because the human ear and brain are not as good at detecting the direction of very low frequency sounds. Nether-the-less when two HomePods are able to work together they will do an even better job, especially if a track has a bass guitarist mixed to one side.

    I can’t wait to hear it. I just listened to The Accidental Podcast which contained an awesome deep dive into the technical implementation of AirPlay 2. However, the guys on there seemed to base their expectations of the HomePod audio output on it’s size - this is kind of like thinking the camera in iPhone won’t be as good as a physically larger 1990’s era digital camera. Computational audio will enable HomePod to punch way above it’s size and it is a quantum leap forward compared to anything else in this price range.
    redgeminipaStrangeDayscbrookerjahbladepatchythepiratelolliver
  • Reply 4 of 89
    xmhillxxmhillx Posts: 109member
    Tell all that to Brian Tong at CNET. Man that dude is an Apple hater disguised as a self proclaimed fan. His shtick is saying he loved the Apple of the past and everything today is dog shit, while praising and loving all things google and Alexa and amazon. Hahah he’s entertaining but not very well versed in tech. Not like the guys on the Apple Insider podcast, for example. 
    redgeminipabb-15JWSClostkiwicbrookerpatchythepiratejony0lolliverwatto_cobra
  • Reply 5 of 89
    mike54mike54 Posts: 339member
    I would like to see waste reduced. The speakers in this seems very good and it will probably last many years as speakers usually do. The electronics in this will get dated and will likely not be supported by Apple in 3 - 4 years or even sooner. I would like see the electronics in a module that can be user replaced. Apple has talented engineers and funds that could design such methods. These techniques can also be done for the iMac.
    williamlondonGeorgeBMacavon b7dws-2caladanianScot1SpamSandwich[Deleted User]muthuk_vanalingam
  • Reply 6 of 89
    Hurry up Apple and get that stereo firmware update to us so I have an excuse to purchase a second HomePod.
    2old4funjahbladepatchythepirateScot1lolliverwatto_cobra
  • Reply 7 of 89
    mjtomlinmjtomlin Posts: 1,876member
    mike54 said:
    I would like to see waste reduced. The speakers in this seems very good and it will probably last many years as speakers usually do. The electronics in this will get dated and will likely not be supported by Apple in 3 - 4 years or even sooner. I would like see the electronics in a module that can be user replaced. Apple has talented engineers and funds that could design such methods. These techniques can also be done for the iMac.

    Electronics? Not sure what you mean by that? Are you referring to the A8? 

    My AirportExpress (AirPlay) router has outlasted a couple sets of speakers - it’s 13 years old!
    williamlondonredgeminipachabigStrangeDaysJWSCjahbladenetmagelolliverwatto_cobra
  • Reply 8 of 89
    "The HomePod stands to be an audiophile's ideal speaker"

    It will be the GREATEST!
    It will be the BIGGEST SOUND!
    You will get tired of all the GREAT sound!
    larryawlymSolibaconstangJaiOh81lolliverwatto_cobra
  • Reply 9 of 89
    mike54mike54 Posts: 339member
    mjtomlin said:
    mike54 said:
    I would like to see waste reduced. The speakers in this seems very good and it will probably last many years as speakers usually do. The electronics in this will get dated and will likely not be supported by Apple in 3 - 4 years or even sooner. I would like see the electronics in a module that can be user replaced. Apple has talented engineers and funds that could design such methods. These techniques can also be done for the iMac.

    Electronics? Not sure what you mean by that? Are you referring to the A8? 

    My AirportExpress (AirPlay) router has outlasted a couple sets of speakers - it’s 13 years old!
    My speakers are 30 years and still sound great, router lasted 11 years now replaced (out of necessity). And yes, I am referring the A8 and other components that make up the 'smart' features.
    GeorgeBMacfeudalist
  • Reply 10 of 89
    mike54 said:
    The electronics in this will get dated and will likely not be supported by Apple in 3 - 4 years or even sooner. 
    What are you basing the "3-4 years or even sooner" on? It doesn't have to drive a retina display and runs a customized version of iOS that is focused on the audio and voice command aspect only for the most part. The CPU probably isn't the part of it that would become outdated. These days, it's the wireless standards that are the more likely culprit for making things obsolete. 
    wlymbb-15StrangeDaysJWSClostkiwiradarthekatnetmagejony0lolliverwatto_cobra
  • Reply 11 of 89
    xmhillx said:
    Tell all that to Brian Tong at CNET. Man that dude is an Apple hater disguised as a self proclaimed fan. His shtick is saying he loved the Apple of the past and everything today is dog shit, while praising and loving all things google and Alexa and amazon. Hahah he’s entertaining but not very well versed in tech. Not like the guys on the Apple Insider podcast, for example. 
    Agreed. I stopped visiting C|Net several years ago, since they LOVE to feed the trolls, and Brian Tong is exactly as you describe. C|Net is the Fox News of the tech world. Sadly, MacWorld is heading in that direction, too.
    StrangeDayszroger73baconstanglostkiwicbrookerlolliverwatto_cobra
  • Reply 12 of 89
    k2kwk2kw Posts: 1,756member
    I've read that the HomePod sounds as good as a Sonos 1 or Sonos 3 .    I wonder how it matches up with the Sonos Play 5.   The HomePod like Sonos 1 doesn't have an auxiliary line in.   You can get that with a Sonos 5 (which is $500).   I wonder how you do the same with HomePod.   It's hard to tell if the HomePod is the first of multiple products or will be Apple's only home audio product.    I would be more interested in a HomePodPlus with line in and HomePodTheater sound bar.   

    Also can HomePod play Hi-Res audio files?
    Scot1
  • Reply 13 of 89
    mjtomlinmjtomlin Posts: 1,876member
    k2kw said:
    I've read that the HomePod sounds as good as a Sonos 1 or Sonos 3 .    I wonder how it matches up with the Sonos Play 5.   The HomePod like Sonos 1 doesn't have an auxiliary line in.   You can get that with a Sonos 5 (which is $500).   I wonder how you do the same with HomePod.   It's hard to tell if the HomePod is the first of multiple products or will be Apple's only home audio product.    I would be more interested in a HomePodPlus with line in and HomePodTheater sound bar.   

    Also can HomePod play Hi-Res audio files?

    After reading Serenity Caldwell's "write up", she found that sound reproduction and separation is better on the HomePod. Which would push it more towards the higher end of home audio. I also read the HomePod also has the ability to "see" where you are in the room and can optimize the direction of the sound in real-time. But Sonos:5 will definitely have a "bigger" (louder) sound because of its size and number of speakers.

    It's a pure WiFi speaker and does not have a line-in. And it does play ALAC, AIFF, WAV, and FLAC all of which are capable of supporting Hi-Res Audio.

    I would be inclined to guess that this will be Apple's only home audio product. 
    edited January 2018 StrangeDayslolliver
  • Reply 14 of 89
    macxpressmacxpress Posts: 4,896member
    mike54 said:
    I would like to see waste reduced. The speakers in this seems very good and it will probably last many years as speakers usually do. The electronics in this will get dated and will likely not be supported by Apple in 3 - 4 years or even sooner. I would like see the electronics in a module that can be user replaced. Apple has talented engineers and funds that could design such methods. These techniques can also be done for the iMac.
    Why are you push the same already tired BS response to HomePod in every thread? You have yet to explain why Apple needs better than the A8 chip in it to begin with? Its a speaker, not a computer, phone, AppleTV, etc. Did you expect Apple to use the A11 Bionic chip? If you think its not worth it because of some stupid chip, then don't buy it. 

    This is an accessory (product), that I doubt Apple will keep around for years and years so its not like every year or even every other year you're going to see HomePod 2, HomePod 3, etc, etc. 
    edited January 2018 StrangeDaysJWSCnetmagelolliverwatto_cobra
  • Reply 15 of 89
    larryalarrya Posts: 548member
    mjtomlin said:
    k2kw said:
    I've read that the HomePod sounds as good as a Sonos 1 or Sonos 3 .    I wonder how it matches up with the Sonos Play 5.   The HomePod like Sonos 1 doesn't have an auxiliary line in.   You can get that with a Sonos 5 (which is $500).   I wonder how you do the same with HomePod.   It's hard to tell if the HomePod is the first of multiple products or will be Apple's only home audio product.    I would be more interested in a HomePodPlus with line in and HomePodTheater sound bar.   

    Also can HomePod play Hi-Res audio files?

    After reading Serenity Caldwell's "write up", she found that sound reproduction and separation is better on the HomePod. Which would push it more towards the higher end of home audio. I also read the HomePod also has the ability to "see" where you are in the room and can optimize the direction of the sound in real-time. But Sonos:5 will definitely have a "bigger" (louder) sound because of its size and number of speakers.

    It's a pure WiFi speaker and does not have a line-in. And it does play ALAC, AIFF, WAV, and FLAC all of which are capable of supporting Hi-Res Audio.

    I would be inclined to guess that this will be Apple's only home audio product. 
    Her listening comparisons still only included a play:1, but she mentions that the HP fills a room like a Play:5.  I would still take the Play:5 for the line input and ability to stream more sources (incidentally, for those who don’t know, a Sonos input can routed to any other Sonos speaker). 
  • Reply 16 of 89
    What a load of rubbish! Sure, Apple does a great job of engineering audio devices that work well in difficult circumstances, and the HomePod devotes impressive technology to the task of providing a voice-controlled interface to internet-connected devices. Good quality music reproduction is a welcome by-product, but it's not automatically "an audiophile's ideal speaker".

     Matt Hines is correct to say that speaker location can affect performance. But "causing some resonant frequencies to become inconsistently louder", "sudden, violent variations in frequency levels" and "bass might disappear"? Really, Mr. Hines? I challenge you to prove in a blind listening test that "adjusting the listening position even an inch will have a very material impact on the arrival time of the audio to your ear"?

     If your ear can hear that difference, I will gladly eat it.
    edited January 2018 avon b7GeorgeBMacbaconstang
  • Reply 17 of 89
    macxpressmacxpress Posts: 4,896member
    tardis said:
    What a load of rubbish! Sure, Apple does a great job of engineering audio devices that work well in difficult circumstances, and the HomePod devotes impressive technology to the task of providing a voice-controlled interface to internet-connected devices. Good quality music reproduction is a welcome by-product, but it's not automatically "an audiophile's ideal speaker".

     Matt Hines is correct to say that speaker location can affect performance. But "causing some resonant frequencies to become inconsistently louder", "sudden, violent variations in frequency levels" and "bass might disappear"? Really, Mr. Hines? I challenge you to prove in a blind listening test that "adjusting the listening position even an inch will have a very material impact on the arrival time of the audio to your ear"?

     If your ear can hear that difference, I will gladly eat it.
    What?
    StrangeDaysJWSCnetmagemike1lolliverwatto_cobra
  • Reply 18 of 89
    mike54 said:
    I would like to see waste reduced. The speakers in this seems very good and it will probably last many years as speakers usually do. The electronics in this will get dated and will likely not be supported by Apple in 3 - 4 years or even sooner. I would like see the electronics in a module that can be user replaced. Apple has talented engineers and funds that could design such methods. These techniques can also be done for the iMac.
    The HP won’t be support in 3 years how? I have an old 2011 iMac that I was able to get a replacement GPU for a year ago. So what are you talking about?
    edited January 2018 netmagelolliverwatto_cobra
  • Reply 19 of 89

    larrya said:
    mjtomlin said:
    k2kw said:
    I've read that the HomePod sounds as good as a Sonos 1 or Sonos 3 .    I wonder how it matches up with the Sonos Play 5.   The HomePod like Sonos 1 doesn't have an auxiliary line in.   You can get that with a Sonos 5 (which is $500).   I wonder how you do the same with HomePod.   It's hard to tell if the HomePod is the first of multiple products or will be Apple's only home audio product.    I would be more interested in a HomePodPlus with line in and HomePodTheater sound bar.   

    Also can HomePod play Hi-Res audio files?

    After reading Serenity Caldwell's "write up", she found that sound reproduction and separation is better on the HomePod. Which would push it more towards the higher end of home audio. I also read the HomePod also has the ability to "see" where you are in the room and can optimize the direction of the sound in real-time. But Sonos:5 will definitely have a "bigger" (louder) sound because of its size and number of speakers.

    It's a pure WiFi speaker and does not have a line-in. And it does play ALAC, AIFF, WAV, and FLAC all of which are capable of supporting Hi-Res Audio.

    I would be inclined to guess that this will be Apple's only home audio product. 
    Her listening comparisons still only included a play:1, but she mentions that the HP fills a room like a Play:5.  I would still take the Play:5 for the line input and ability to stream more sources (incidentally, for those who don’t know, a Sonos input can routed to any other Sonos speaker). 
    HP is an AirPlay endpoint you can stream any source to it that has an app. 
    edited January 2018 lolliver
  • Reply 20 of 89

    tardis said:
    What a load of rubbish! Sure, Apple does a great job of engineering audio devices that work well in difficult circumstances, and the HomePod devotes impressive technology to the task of providing a voice-controlled interface to internet-connected devices. Good quality music reproduction is a welcome by-product, but it's not automatically "an audiophile's ideal speaker".

     Matt Hines is correct to say that speaker location can affect performance. But "causing some resonant frequencies to become inconsistently louder", "sudden, violent variations in frequency levels" and "bass might disappear"? Really, Mr. Hines? I challenge you to prove in a blind listening test that "adjusting the listening position even an inch will have a very material impact on the arrival time of the audio to your ear"?

     If your ear can hear that difference, I will gladly eat it.
    I’ve perosnally sat in a small sound recording studio with calibrated monitors. There is a small sweet spot in the room, and if you move out of its space you can definitely hear it. 
    netmagemike1lolliverwatto_cobra
Sign In or Register to comment.