Apple details Siri's machine learning upgrades for better listening on HomePod

Posted:
in General Discussion
Apple on Monday updated its Machine Learning Journal with a post by the company's Siri speech and audio software engineering teams, explaining how the company uses machine learning to help the HomePod hear people under tougher circumstances than iPhones and iPads.

Apple HomePod internals


Siri on the HomePod had to be upgraded to cope with loud music, ambient noise, and distant talkers, the journal entry notes. Accordingly the HomePod employs not just far-field microphones, but "mask-based multichannel filtering using deep learning" to strip out echo and background noise, and "unsupervised" learning to split up multiple sound sources and use only the one including "Hey Siri" as a trigger phrase.

The entry goes into considerable technical detail, being oriented mainly towards professionals in the machine learning field. It does however mention that multichannel signal processing happens "continuously" on the HomePod's A8 processor, even in low-power states, and is able to adjust to both changing environments and moving talkers.

Apple suggests that while "other state-of-the-art systems" use multi-microphone processing, they typically only focus on echo and noise reduction.

The HomePod is a relative late comer to the smartspeaker market, having launched just this February -- Amazon's Echo debuted in 2014, and the Google Home shipped in 2016. Apple though has taken a different tack than many vendors, concentrating on sound quality with technologies like beamforming. A HomePod will automatically tune itself to match its position in a room.

Siri though has been criticized as limited next to Amazon and Google's voice assistants, for instance only natively supporting Apple Music in streaming services. The HomePod hardware is also expensive at $349 -- Apple has been rumored as working on a cheaper model to be more competitive.

The journal update coincides with Apple's appearance at the 32nd Conference on Neural Information Processing Systems in Montreal, Canada. The company has tried to open itself more to the academic community, presumably to appeal to potential recruits and to appease the researchers it already has, who previously complained about Apple's paper publishing restrictions.

The Machine Learning Journal is another one of those compromises. It began in July 2017 with a paper discussing neural net training with collated images, and has since gone on to cover a variety of topics, such as face detection and "differential" privacy.

A year ago the company's s director of AI research, Ruslan Salakhutdinov, spoke about the company's self-driving car project, the ultimate purpose of which is still shrouded in mystery. The company may or may not be working on a self-designed car -- it at least briefly abandoned the idea in favor of pure platform development.

Comments

  • Reply 1 of 18
    zompzomp Posts: 62member
    I'll venture to say that Apple is advancing all their products for the purpose of delivering an amazing car. They aren't interested in selling product through HomePod, but I do believe their HomePod listening advances and Portrait mode camera's (facebook is converting to 3d photos) will be used to advance safety in vehicles. I'm not a tech guy by no means and look forward to reading the opinion of others whom feel the same.
    caladanian
  • Reply 2 of 18
    fallenjtfallenjt Posts: 4,053member
    To be honest, I’m Apple freak and HomePod was the only Apple product I bought and returned. Beside listening to music, you don’t have much use for it. I use Amazon Echo more at home because of its capabilities and compatibility. I can get 2 Echoes (2nd generation) and an Echo Sub for $200 and they blow HomePod away!
    williamlondon
  • Reply 3 of 18
    We've had both Google Home and HomePod in our house for the past 9 months. We don't use the Google much at all - turns out that the "smart" part isn't really very compelling the vast majority of the time, and I'll probably put it back in its box along with its ghastly sub-par audio. The HomePods (we have 2 now) are used constantly as music sources, they sound absolutely great. I still have nearly no use for asking Siri for anything but 1) music and 2) where is iPhone? I'd consider an Echo for fun, but can't think of anything I want to ask it to do and they sound mediocre. Not yet sold on the voice-command lifestyle except for a few specific tasks.
    rtmyers5
  • Reply 4 of 18
    russwrussw Posts: 21member
    Once you have more HomeKit automation, the HomePod becomes more useful to turn on and off devices or enable scenes. Hey Siri, set scene dining (or just say hey Siri, dinning) to setup a set of lights in our home. Sure, you could do that with your iPhone but when you're scrambling around in the kitchen it's a lot easier to just talk. Not saying it's worth the expense but it's a nice extravagance. I'd like a smaller HomePod that cost 1/2 as much so I could add to a few more areas in our home. It's nice they started high end but they are expensive.
    caladanianwilliamlondon
  • Reply 5 of 18
    auxioauxio Posts: 2,717member
    We've had both Google Home and HomePod in our house for the past 9 months. We don't use the Google much at all - turns out that the "smart" part isn't really very compelling the vast majority of the time, and I'll probably put it back in its box along with its ghastly sub-par audio. The HomePods (we have 2 now) are used constantly as music sources, they sound absolutely great. I still have nearly no use for asking Siri for anything but 1) music and 2) where is iPhone? I'd consider an Echo for fun, but can't think of anything I want to ask it to do and they sound mediocre. Not yet sold on the voice-command lifestyle except for a few specific tasks.
    I'd agree.  Music is my primary use since I don't have any HomeKit accessories, and HomePod excels at that.  The only things I tend to ask Siri for other than to play/stop music are what the weather forecast is and to set timers.
  • Reply 6 of 18
    maestro64maestro64 Posts: 5,043member
    All I have to say is Siri on homepod still does weird things. When there is conversations going on in the room Siri will blurt out random responses and no one said hey Siri. The other issue is TV shows or commercials will say Hey Siri and the dame thing will response to the TV. It happens all the time on NFL games, they will say Hey Siri. 
  • Reply 7 of 18
    Is this a joke? Siri on HomePod is one of the most useless and frustrating iterations on Siri. 

    For instance, I just told Siri to “play chill house music” and the reply was “I have a hunch you’ll like this tropical house”. Wrong and incorrect. 

    So, not only do I have to deal with siri’s obnoxious and repetitive “personality,” Siri as well as apple
    music continues to be complete shit. 

    I crige every time I have to use HomePod for anything other than rote HomeKit commmads.

    Even as an airplay speaker HomePod sucks. Sometimes it doesn’t respond at all. And when I use it as an airplay speaker it drops the connection to my iPhone all the time, so I usually have to resort to a separate Bluetooth speaker. I keep all my iOS products new and updated, including the latest OS, the latest iPhone, and the latest Apple TV, yet there’s always problems. 

    Continued, huge gaps in the Apple ecosystem. An ongoing slap in the face to Apple users. Over and over. 

    No wonder investors are skittish about AAPL. The only thing that is consistently reliable is the core hardware. Services continue to be almost fucking useless (except for App Store revenue, which of course exists only bc the iOS hardware is so fantastic). 

    Edit: forgot to mention that my HomePod likes to activate itself randomly when I’m watching tv (using the crappy built in tv sv speakers). 
    edited December 2018
  • Reply 8 of 18
    Rayz2016Rayz2016 Posts: 6,957member
    auxio said:
    We've had both Google Home and HomePod in our house for the past 9 months. We don't use the Google much at all - turns out that the "smart" part isn't really very compelling the vast majority of the time, and I'll probably put it back in its box along with its ghastly sub-par audio. The HomePods (we have 2 now) are used constantly as music sources, they sound absolutely great. I still have nearly no use for asking Siri for anything but 1) music and 2) where is iPhone? I'd consider an Echo for fun, but can't think of anything I want to ask it to do and they sound mediocre. Not yet sold on the voice-command lifestyle except for a few specific tasks.
    I'd agree.  Music is my primary use since I don't have any HomeKit accessories, and HomePod excels at that.  The only things I tend to ask Siri for other than to play/stop music are what the weather forecast is and to set timers.
    Yup, same here. I only ask for music, the time, the weather, the location of my phone, set reminders, alarms, traffic, the lighting, and shopping lists. 

    And I occasionally ask it to flip a coin. 
    caladanian
  • Reply 9 of 18
    auxioauxio Posts: 2,717member
    For instance, I just told Siri to “play chill house music” and the reply was “I have a hunch you’ll like this tropical house”. Wrong and incorrect. 
    Music genres are a joke.  I remember when I used to tag genres in my MP3s and the list of options iTunes would pop up (based on other files in my collection) were completely random and useless.  Maybe it's better with Apple Music now since songs are more professionally curated, but everyone has their own take on what constitutes any genre (and there are always songs which cross genres).  Completely subjective and arbitrary in my experience, so it's not surprising Siri (or any algorithm) gets it wrong when trying to pick music you like based on genre alone.  It's like trying to categorize every colour based on the 64 Crayola box colours.
    Even as an airplay speaker HomePod sucks. Sometimes it doesn’t respond at all. And when I use it as an airplay speaker it drops the connection to my iPhone all the time, so I usually have to resort to a separate Bluetooth speaker. I keep all my iOS products new and updated, including the latest OS, the latest iPhone, and the latest Apple TV, yet there’s always problems. 
    Ever consider that your WiFi router is the problem?  The only time Airplay has failed is when my WiFi network has bogged to a crawl for whatever reason (I miss my Airport Express).
    No wonder investors are skittish about AAPL. The only thing that is consistently reliable is the core hardware. Services continue to be almost fucking useless (except for App Store revenue, which of course exists only bc the iOS hardware is so fantastic). 
    Seriously?  Do you even understand what Apple's services are?

    iCloud is a godsend.  My phone was stolen the other day and it took almost no time to get a new one set up with all of the same information again.  I remember the days when I'd have to pray that I'd backed it up recently, and then spend a number of hours syncing everything over to it again (and having to keep it attached to my computer).  Now it just happens automagically once I log in to my iCloud account.

    Then there's iTunes Match, which works beautifully for getting my music collection on every device.
    Edit: forgot to mention that my HomePod likes to activate itself randomly when I’m watching tv (using the crappy built in tv sv speakers). 
    This is my only gripe with HomePod.  But the random activations seem to be happening far less often these days, so I'm assuming Siri is getting better behind the scenes.

  • Reply 10 of 18
    auxioauxio Posts: 2,717member

    Rayz2016 said:
    auxio said:
    We've had both Google Home and HomePod in our house for the past 9 months. We don't use the Google much at all - turns out that the "smart" part isn't really very compelling the vast majority of the time, and I'll probably put it back in its box along with its ghastly sub-par audio. The HomePods (we have 2 now) are used constantly as music sources, they sound absolutely great. I still have nearly no use for asking Siri for anything but 1) music and 2) where is iPhone? I'd consider an Echo for fun, but can't think of anything I want to ask it to do and they sound mediocre. Not yet sold on the voice-command lifestyle except for a few specific tasks.
    I'd agree.  Music is my primary use since I don't have any HomeKit accessories, and HomePod excels at that.  The only things I tend to ask Siri for other than to play/stop music are what the weather forecast is and to set timers.
    Yup, same here. I only ask for music, the time, the weather, the location of my phone, set reminders, alarms, traffic, the lighting, and shopping lists. 

    And I occasionally ask it to flip a coin. 
    Using it for a shopping list is a great idea!  I'll have to try that.
    Rayz2016
  • Reply 11 of 18
    macguimacgui Posts: 2,350member
    I've tried integrating Alexa with HomeKit, but it fails after a short period. I've tired if trying to enable new skills only to have them fail. I have to raise my voice noticeably as the distance increases, or if Alexa is playing music or talking. However she excels at providing answers. And I often set multiple timers/

    I can't find a comprehensive how-to source, so she's no longer any help with home automation. Millions of people don't seem to have that/those problems, but that's no help for me.

    Siri works more reliably for me, and seems to have been upgraded, if slightly, over the last part of the week and/or weekend. Siri isn't nearly as bright as Alex but is more obedient. If BB has another $100 off deal, I'll give the HP a shot. I wish B&H would do the same.

    For the time being, they both compliment each other as neither does everything I want. Both have flaws, such as activating when hearing something on TV that may or may not sound like 'Hey, Siri...' or 'Alexa...' to me. My living room Dot activates occasionally for no reason I can see, with no audio in the room. I just assume it's the NSA checking in.
  • Reply 12 of 18
    MacProMacPro Posts: 19,718member
    Well, maybe I am alone but I actually bought my HomePods to listen to music not ask them trivia. Put them up against the Alexa devices for audio quality and decide.  It's a no brainer.
    williamlondon
  • Reply 13 of 18
    welshdogwelshdog Posts: 1,897member
    My HomePod regularly activates Siri from something it hears on the TV. The machine learning improvements are not doing that much for us I guess.

    Other than that, it's been a great addition. The sound is wonderful and combined with Apple Music, it suits our needs well.
  • Reply 14 of 18
    Rayz2016Rayz2016 Posts: 6,957member
    MacPro said:
    Well, maybe I am alone but I actually bought my HomePods to listen to music not ask them trivia. Put them up against the Alexa devices for audio quality and decide.  It's a no brainer.
    I’ve never heard an Alexa device so I can’t judge, but the HomePod in my office is eerily good. And the way it picks out my voice when it’s playing loud? Also spooky. 

    Something else I use it for: spelling out words while I’m writing. 
    williamlondon
  • Reply 15 of 18
    evilutionevilution Posts: 1,399member
    Apple were just too slow to market. By the time I got a HomePod, I had an Amazon Dot in every room and it could do more than the HomePod that cost 10 times the price. Yes it sounds impressive but I don’t have Apple Music. Again, too slow. I have Amazon Prime so can get buy with the free music nailed onto that package.

    The HomePod can’t even stream radio stations. A smart speaker it isn’t.

    My HomePod was bought on day 1 of release. By week 3 it had made a ring on my wooden table, been disconnected and put away in the hope that Apple would eventually add abilities that made it useful to non Apple Music users.
    williamlondon
  • Reply 16 of 18

    Would most of the complaints about Apple Music go away if it was cheaper?

    I find it to be of tremendous value, but then again, I just pay the equivalent of ~$2 per month for a family subscription.

  • Reply 17 of 18
    MacProMacPro Posts: 19,718member
    evilution said:
    Apple were just too slow to market. By the time I got a HomePod, I had an Amazon Dot in every room and it could do more than the HomePod that cost 10 times the price. Yes it sounds impressive but I don’t have Apple Music. Again, too slow. I have Amazon Prime so can get buy with the free music nailed onto that package.

    The HomePod can’t even stream radio stations. A smart speaker it isn’t.

    My HomePod was bought on day 1 of release. By week 3 it had made a ring on my wooden table, been disconnected and put away in the hope that Apple would eventually add abilities that made it useful to non Apple Music users.
    I stream radio stations on my HomePods no problem.  Perhaps you need to try again.  For example (it may vary in different localities) is "Hey Siri play radio station NPR" or "Hey Siri, play the news on CNN".  The streams are actually podcasts but they are updated so news is current, it is not simply streaming FM radio as such.  Neither of my HomePods has left a mark on the wood where they sit.  
    edited December 2018
  • Reply 18 of 18
    auxio said:

    I think the simplest way to respond will be to follow up your comments in bold:
    For instance, I just told Siri to “play chill house music” and the reply was “I have a hunch you’ll like this tropical house”. Wrong and incorrect. 
    Music genres are a joke.  I remember when I used to tag genres in my MP3s and the list of options iTunes would pop up (based on other files in my collection) were completely random and useless.  Maybe it's better with Apple Music now since songs are more professionally curated, but everyone has their own take on what constitutes any genre (and there are always songs which cross genres).  Completely subjective and arbitrary in my experience, so it's not surprising Siri (or any algorithm) gets it wrong when trying to pick music you like based on genre alone.  It's like trying to categorize every colour based on the 64 Crayola box colours.

    I think you seem to be agreeing with me here. It's pretty ridiculous that Apple has dominated online-accessed music for over a decade and they still cant get basic sub-genres right. Unbelievably pathetic apple. I wish eddie cue would have the decency and integrity to get this right, especially with music being such an integral (and important for future growth) part of the Apple ecosystem. It's even more bizarre that the creation of Beats music in the first place was entirely based on being able to say "play relaxing jazz music" or "play upbeat party music" etc. Now it's worse than it was before Apple bought it. If I ask Apple music to play "chill ____", it should't offer me a different sub-genera just play the fucking music I'm paying Apple to play ffs.
    Even as an airplay speaker HomePod sucks. Sometimes it doesn’t respond at all. And when I use it as an airplay speaker it drops the connection to my iPhone all the time, so I usually have to resort to a separate Bluetooth speaker. I keep all my iOS products new and updated, including the latest OS, the latest iPhone, and the latest Apple TV, yet there’s always problems. 
    Ever consider that your WiFi router is the problem?  The only time Airplay has failed is when my WiFi network has bogged to a crawl for whatever reason (I miss my Airport Express).

    I use an airport extreme with multiple airport expresses, as well as the latest iphone and latest OS. Yet it drops the connection all the time, and even more inexplicably isn't even able to pick it back up again half the time.
    No wonder investors are skittish about AAPL. The only thing that is consistently reliable is the core hardware. Services continue to be almost fucking useless (except for App Store revenue, which of course exists only bc the iOS hardware is so fantastic). 
    Seriously?  Do you even understand what Apple's services are?

    Yes. Apple's services are almost entirely the App store (revenue-wise), and the gap is probably even wider profit-wise.

    iCloud is a godsend.  My phone was stolen the other day and it took almost no time to get a new one set up with all of the same information again.  I remember the days when I'd have to pray that I'd backed it up recently, and then spend a number of hours syncing everything over to it again (and having to keep it attached to my computer).  Now it just happens automagically once I log in to my iCloud account.

    Agreed. iCloud is great. An essential part of the Apple ecosystem. But not a profit driver.

    Then there's iTunes Match, which works beautifully for getting my music collection on every device.

    Again, this is a basic, rote function that should work easily, despite the rollout being bumbled from what I heard (I don't use it personally).
    Edit: forgot to mention that my HomePod likes to activate itself randomly when I’m watching tv (using the crappy built in tv sv speakers). 
    This is my only gripe with HomePod.  But the random activations seem to be happening far less often these days, so I'm assuming Siri is getting better behind the scenes.

    I havn't noticed any change in frequency, but of course that's just my experience. But doesn't it seem like, with 6 microphones, that the homepod should be a bit more accurate in distinguishing a real activation from a false trigger (not even mentioning "siri") from a crappy tv speaker all the way across the room?

    Just about the only functional products/services in the services division seem to be rote tasks that should be relatively simple from a technological standpoint (eg press purchase button, card is charged, app is downloaded; eg download something on server, other devices get access to it). Why is it that everything else in the services division is either crap, or was horribly bumbled (siri, music, maps, apple pay, mail, search.. on and on)? I'm not one who enjoys bashing on others, but enough is enough is enough.... how long are they going to keep letting eddie cue mess up Apple? It's very frustrating as both an Apple consumer and an Apple fan.

    edited December 2018
Sign In or Register to comment.