Siri's reliance on Google cut in half with iOS 6
Tests with Apple's voice-driven personal assistant, Siri, have found that the service turns to Google for an answer in iOS 6 half as much as it did in iOS 5.
When investment firm Piper Jaffray initially tested Siri in June, its list of common queries found that Google was responsible for about 60 percent of the data returned. Now, with iOS 6, Google is responsible for only 30 percent of the data.
"The biggest change in query resolution to Google was the removal of Google Maps in iOS 6, as 23% of queries resolved to Apple Maps," analyst Gene Munster wrote on Thursday. "Yelp and Yahoo! also benefited slightly. Net-net we view this as a slight negative for consumers on Apple products as we view Apple Maps as an inferior product to Google Maps."
The latest tests also found evidence of behind-the-scenes improvements to Siri, though the changes have been slight. In iOS 5, Siri understood 89 percent of queries spoke in a quiet environment, while that improved to 91 percent in iOS 6, while 76 percent of queries were answered correctly in iOS 5 and that improved to 77 percent with iOS 6.
Since Piper Jaffray's first Siri test, Google has launched its own voice-driven assistant service, dubbed Google Now. For the latest test, Google Now was included to offer a head-to-head comparison with Siri, and found that Apple's service has a very slight advantage.
"It appears the two voice assistants are comparable to one another in terms of understanding the spoken query and returning the correct result," Munster wrote. "In our test, Siri correctly understood our queries 91% of the time in a quiet environment compared to Google Now at 88%. In terms of accuracy, we determined that Siri accurately answered understood queries 77% of the time compared to 75% for Google now."
Munster said the tests showed that Siri's greatest strengths are in local search and operating-system-level commands. Google Now, meanwhile, has its strengths in navigation and fetching information.
"For example, Siri enables full control of the music application via Siri, but Google Now does not understand all song change/pause commands," Munster said. "We believe Google will eventually strengthen its ability to control the OS in future launches."
One area where both Google Now and Siri are lacking, he said, is in commerce. Munster believes that Apple will integrate with local deal companies such as Groupon and Living Social to enhance commerce assistance with Siri.
When investment firm Piper Jaffray initially tested Siri in June, its list of common queries found that Google was responsible for about 60 percent of the data returned. Now, with iOS 6, Google is responsible for only 30 percent of the data.
"The biggest change in query resolution to Google was the removal of Google Maps in iOS 6, as 23% of queries resolved to Apple Maps," analyst Gene Munster wrote on Thursday. "Yelp and Yahoo! also benefited slightly. Net-net we view this as a slight negative for consumers on Apple products as we view Apple Maps as an inferior product to Google Maps."
The latest tests also found evidence of behind-the-scenes improvements to Siri, though the changes have been slight. In iOS 5, Siri understood 89 percent of queries spoke in a quiet environment, while that improved to 91 percent in iOS 6, while 76 percent of queries were answered correctly in iOS 5 and that improved to 77 percent with iOS 6.
Since Piper Jaffray's first Siri test, Google has launched its own voice-driven assistant service, dubbed Google Now. For the latest test, Google Now was included to offer a head-to-head comparison with Siri, and found that Apple's service has a very slight advantage.
"It appears the two voice assistants are comparable to one another in terms of understanding the spoken query and returning the correct result," Munster wrote. "In our test, Siri correctly understood our queries 91% of the time in a quiet environment compared to Google Now at 88%. In terms of accuracy, we determined that Siri accurately answered understood queries 77% of the time compared to 75% for Google now."
Munster said the tests showed that Siri's greatest strengths are in local search and operating-system-level commands. Google Now, meanwhile, has its strengths in navigation and fetching information.
"For example, Siri enables full control of the music application via Siri, but Google Now does not understand all song change/pause commands," Munster said. "We believe Google will eventually strengthen its ability to control the OS in future launches."
One area where both Google Now and Siri are lacking, he said, is in commerce. Munster believes that Apple will integrate with local deal companies such as Groupon and Living Social to enhance commerce assistance with Siri.
Comments
How about something more than your opinion? So far, no one has provided evidence that Apple Maps returns incorrect information more frequently than Google Maps. This is just one of those things that people start believing when it's repeated often enough.
Interestingly, they did think it was important to provide data on the Siri v Google Now issue. So why not provide data on Apple Maps v Google Maps?
Quote:
Originally Posted by jragosta
How about something more than your opinion? So far, no one has provided evidence that Apple Maps returns incorrect information more frequently than Google Maps. This is just one of those things that people start believing when it's repeated often enough.
Interestingly, they did think it was important to provide data on the Siri v Google Now issue. So why not provide data on Apple Maps v Google Maps?
Besides Munster, Pogue and Siegler, all generally pro-Apple bloggers, Walt Mossberg chimed in to agree Google Maps is a great app.
"Google’s rich, reliable Maps app is back on the iPhone, and that means iPhone users can stop relying on the flawed, fledgling Apple maps app that replaced it as a built-in feature in September."
"I’ve been testing the new Google Maps on iPhone for a week or so in the San Francisco and Washington metro areas, and I really like it. It isn’t perfect, but I prefer it to any other iPhone Maps app I’ve used..."
It's very unusual, perhaps unheard of till now, to see this particular group of Apple-friendly tech columnists all agree that something from Google outperforms an Apple offering., wouldn't you agree Jragosta? Can't imagine what their motivation would be unless they truly believe what they've written. None of them could be accused of being Google shills.
Quote:
Originally Posted by jragosta
How about something more than your opinion? So far, no one has provided evidence that Apple Maps returns incorrect information more frequently than Google Maps.
So you don't believe that Google has more POI data than Apple at this point in time?
What part of "facts" do you not understand? An opinion that is repeated 10,000 times is still an opinion.
Apple Maps has x errors.
Google Maps has y errors.
Let me know when you have evidence that x is significantly greater than y. So far, no one has provided any.
I don't know. I am simply pointing out that no one has yet proven the oft-repeated claim that Apple Maps is inferior.
Stock analyst = failed business person.
Quote:
Originally Posted by jragosta
What part of "facts" do you not understand? An opinion that is repeated 10,000 times is still an opinion.
Apple Maps has x errors.
Google Maps has y errors.
Let me know when you have evidence that x is significantly greater than y. So far, no one has provided any.
I don't know. I am simply pointing out that no one has yet proven the oft-repeated claim that Apple Maps is inferior.
Yup. You and everyone else, including tech columnists, are posting their opinions. I completely agree. FWIW note I've posted none myself as I've not used both of them yet. Have you?
Quote:
Originally Posted by jragosta
What part of "facts" do you not understand? An opinion that is repeated 10,000 times is still an opinion.
Apple Maps has x errors.
Google Maps has y errors.
Let me know when you have evidence that x is significantly greater than y. So far, no one has provided any.
I don't know. I am simply pointing out that no one has yet proven the oft-repeated claim that Apple Maps is inferior.
I searched the other day for Chinese restaurants in my area. Apple Maps brought up Chinese AND teriyaki places.
Hello?
And if they rely on Foursquare to augment their data, God help Apple. I tried out Foursquare and it is sadly incomplete and a bit of a joke.
Google Maps (Apple app with Google data) has been my go-to app for years. It's very reliable. And transit information? Traffic information? It's great.
(Apple user 32 years)
Quote:
Originally Posted by TBell
That is his point, he doesn't want to rely on somebody's beliefs. He wants comparative data.
I would venture to say that's impossible. I've watched arguments over Navteq vs. Teleatlas map quality for years. When Google maps rolled out I saw more discussion about whether Google had better maps than either TeleAtlas (TomTom) or Navteq. the arguments still take place. Add in disagreements on who has the better validation and data gathering methods, which throws companies like Waze and OpenStreetMaps into the mix.
Much more qualified commenters than the ones in this thread (afaik) have been unable to prove beyond any doubt which mapping effort has the better data. I doubt a couple of guys at AI can change that. All that's left is opinion.
mostly due to maps...
Quote:
Originally Posted by Ireland
Now we need on-device speech recognition, to help speed up him/her.
Google's speech recognition is built in offline to devices running 4.0 and up. It takes up 23 MB.
Apple will catch up eventually. Probably iOS 7 next summer. This is the only reason Google Now is so much faster than Siri.
And that's exactly the point. THERE IS NO EVIDENCE that Apple Maps is inferior. None. Nada. Zip.
There are some anecdotes (like the person complaining above about Apple listing some teriyaki restaurants), but there are also errors with Google Maps. So all the whining about how bad Apple Maps is has no basis in fact. It's just more of the common Internet phenomenon - the loud people get heard. Just like all the other 'gates' that people have been throwing out about Apple. Most of them were either completely fabricated or blown so far out of proportion as to lose any sense of reality.
In the end, both Apple Maps and Google Maps do the job most of the time.
Quote:
Originally Posted by jragosta
In the end, both Apple Maps and Google Maps do the job most of the time.
So you've actually tried out both, given each a fair shake, or are you going by what others have said?
Quote:
Originally Posted by jragosta
And that's exactly the point. THERE IS NO EVIDENCE that Apple Maps is inferior. None. Nada. Zip.
Apple maps may be almost as good as Google or even better in some areas which has been demonstrated with countless examples, however, there is usually a USA bias in those supporting the opinion that Apple maps is just as accurate and detailed as Google maps. By in large Apple Maps is quite lacking in other countries. People say that Apple Maps will get better as more people use it and correct POI through the feedback system, but that only works if the maps are semi useful to start with. If the data is so erroneous or missing that you can't even use it, where do you start in trying to fix it?
Below are two comparisons I just captured as an example of the complete uselessness of Apple Maps in Central America.
Quote:
Originally Posted by jragosta
How about something more than your opinion? So far, no one has provided evidence that Apple Maps returns incorrect information more frequently than Google Maps. This is just one of those things that people start believing when it's repeated often enough.
Interestingly, they did think it was important to provide data on the Siri v Google Now issue. So why not provide data on Apple Maps v Google Maps?
No, but Apple Maps does have far inferior points of interest data, especially outside of the USA.
Quote:
Originally Posted by Gatorguy
I would venture to say that's impossible. I've watched arguments over Navteq vs. Teleatlas map quality for years. When Google maps rolled out I saw more discussion about whether Google had better maps than either TeleAtlas (TomTom) or Navteq. the arguments still take place. Add in disagreements on who has the better validation and data gathering methods, which throws companies like Waze and OpenStreetMaps into the mix.
Much more qualified commenters than the ones in this thread (afaik) have been unable to prove beyond any doubt which mapping effort has the better data. I doubt a couple of guys at AI can change that. All that's left is opinion.
I agree with your sentiment. Yet, most reviewers aren't pointing to any testing to say why Google's maps are better. Moreover, has anybody done any tests to see if Apple's Maps App has improved? In my area, it has always been pretty accurate.
I performed a very limited test of both the Google and Apple Apps in Ann Arbor Michigan on two routes of about ten miles each. The first trip I wanted to find the Mall, the other Moe's Southwestern Grill. With Apple's App I just told Siri to plot the routes. Easy. With Google's App I performed a search, and found the destinations with no problem. So, both Apps had little trouble with the Point of Interest data. However, with the Google App it wasn't clear how to turn my search into directions using my current location and turn by turn. I had to fiddle with the App for a while and the method wasn't super intuitive. None of these glowing reviews seem to address that issue.
Once I had the routes programed in, I ignored the directions and took my own route. Apple's App auto corrected with no problem. Google's struggled. At times, I wasn't sure what Google's App was doing at all. The most annoying thing was Google's App simply failed to tell me to turn at least twice, once on 1-94.
Combine this experience, with the fact that most reviewers who downloaded the Google App gave it 5 stars within a couple minutes of downloading it (e.g. no testing done), I am skeptical Google's App is actually being compared side by side with Apple's Map App.
Besides not calling it's App a Beta, I think one of Apple's biggest mistakes was not making turn by turn work on older model handsets. Perhaps, this is somehow tied to Siri. However, I guess there is a whole bunch of people who will use Google Maps on older model phones and become accustomed to it.