Test finds HomePod's Siri 'at the bottom of the totem pole' in smartspeaker AI
While the Apple HomePod is the "best sounding" smartspeaker and has a "measurably better" user experience in many areas, its underlying AI assistant -- Siri -- failed dramatically in a query test versus Amazon Alexa, Google Assistant, and Microsoft Cortana, according to Loup Ventures.
Three HomePods were subjected to 782 queries by the firm, said analyst Gene Munster. While Siri understood 99.4 percent of them, it was only able to answer 52.3 percent of them correctly. The latter figure compares with rates of 81 percent for Google, 64 percent for Alexa, and 57 percent for Cortana.
Siri did beat out Alexa and Cortana in test categories like "Local" (e.g. "Where can I find a good cup of coffee around here?") and "Commerce" ("Help me buy some new shoes"), but still failed in part because a number of functions -- such as email, calling, calendars, and navigation -- are simply inaccessible on a HomePod. Likewise, where Siri would display a list of Google search results on an iPhone, it outright fails to provide anything on a HomePod.
Removing navigation, calling, email, and calendar-related categories improves Siri's correct answer rate to 67 percent, and Munster argues that Apple will probably improve the HomePod's options over time, making it "vastly more useful and integrated with your other Apple devices."
The Loup Ventures analysis lines up with early HomePod reviews, which have typically pegged Siri as the weakest link. Apple has directed its marketing focus to sound quality, emphasizing technologies like beamforming and the use of seven separate tweeters.
Loup Ventures is forecasting sales of 7 million HomePods in 2018, boosting Apple's revenue and earnings by 1 percent. Sales are expected to hit 10.9 million in 2019, and grow 40 to 45 percent annually during the next three years.
Three HomePods were subjected to 782 queries by the firm, said analyst Gene Munster. While Siri understood 99.4 percent of them, it was only able to answer 52.3 percent of them correctly. The latter figure compares with rates of 81 percent for Google, 64 percent for Alexa, and 57 percent for Cortana.
Siri did beat out Alexa and Cortana in test categories like "Local" (e.g. "Where can I find a good cup of coffee around here?") and "Commerce" ("Help me buy some new shoes"), but still failed in part because a number of functions -- such as email, calling, calendars, and navigation -- are simply inaccessible on a HomePod. Likewise, where Siri would display a list of Google search results on an iPhone, it outright fails to provide anything on a HomePod.
Removing navigation, calling, email, and calendar-related categories improves Siri's correct answer rate to 67 percent, and Munster argues that Apple will probably improve the HomePod's options over time, making it "vastly more useful and integrated with your other Apple devices."
The Loup Ventures analysis lines up with early HomePod reviews, which have typically pegged Siri as the weakest link. Apple has directed its marketing focus to sound quality, emphasizing technologies like beamforming and the use of seven separate tweeters.
Loup Ventures is forecasting sales of 7 million HomePods in 2018, boosting Apple's revenue and earnings by 1 percent. Sales are expected to hit 10.9 million in 2019, and grow 40 to 45 percent annually during the next three years.
Comments
I would personally much rather be taking things slow with all of this new artificial intelligence than blindly following the race to the first without ever thinking about the potential consequences.
That aside... whatever we we may say about who is first and who is best. Majority of people still don’t even talk to their assistants.
There’re so many fundamental things SIRI could be improved. If Apple won’t do it I hope they get called out like this until they understand it will hurt their brand.
(I’m talking about SIRI in general, like on iPhone, not specifically on HomePod.)
The Echo is a poor speaker with a quality voice assistant added on. It’s sold as a voice assistant.
However, software updates won’t improve the sound quality of the Echo but a software update can improve Siri.
Hopefully all of the reviews will encourage Apple to improve Siri.
Then hopefully that improvement will force Amazon to improve Alexa. Ideally to understand more than 3 languages.
That being said, who really ever uses a voice assistant? I set alarms for cooking and that’s it.
As a Speaker version of an iPod, it's outstanding - I can stream music from the BBC over airplay, or BBC talk, and it's so clear its uncanny. the frequency response seems incredibly neutral - some pieces of music sound as good as when using pretty decent headphones.
HomePod also doubles as a really very good speakerphone - I called my son using FaceTime audio on his iPhone X, and the call quality was very very good, it was as though he was in the same room as myself.
Siri on HomePod will get better I assume, but I don't really understand why its so poor out of the gate - no multiple timers, inability to answer simple questions that Siri on my iPhone knows, I could go on - Per a comment above, I for one often as Siri (or Alexa - I got a dot as a Christmas gift) for asking a simple question.
Apple is so focused with gimmick tech for the tweenies that there stuff no longer "just works".
http://www.androidpolice.com/2018/02/10/ultimate-guide-google-home-tips-tricks-understanding-making-assistant-speaker/
Get rid of this embarrassment, Apple; Man up, call it quits, buy IBM’s Watson, and build on it. Siri-ously.
Wow. I am sure your use case = The World’s.
Thanks in advance.