First Look: Using iPhone 4S with Siri voice assistant
Most of the new features in iOS 5 simply run faster on the iPhone 4S, but one major new feature is exclusive to the new model: Siri voice assistant. Here's a look at how it works in action, responding to a variety of requests.
Apart from iPhone 4S, other iOS devices upgraded to run the free new iOS 5 continue to supply Voice Control, which responds to a variety of voice commands such as "what time is it?" or "play songs by MGMT," or "call Tim Cook."
The new Siri dramatically expands upon Voice Control, offering a much wider, more powerful vocabulary for creating Notes, composing Emails and Messages, navigating and changing Calendar events, setting new Reminders, and taking dictation within apps virtually anywhere you could type.
Siri also plugs into external services, integrating with smart artificial intelligence servers hosted by Apple to decipher not just basic commands but natural language questions. It then provides answers through Google or Bing, Wolfram Alpha, Wikipedia, Yelp, and Yahoo weather and stock market information.
Voice recognition
Siri offloads voice recognition to Apple's servers, which apparently run software licensed from Nuance. The text to speech component of Siri is very good, far easier to begin using and more accurate than other systems I've used.
More importantly, however, Siri isn't just voice recognition and dictation. The heart of Siri is a new AI system that works to understand what you mean, not just transcribe the words you say. So while it's nice to be able to talk rather than type, the real functionality of Siri is the ability to respond to what you want to do.
This isn't always perfect, but it's very usable. Generally, when Siri fails to understand what you ask, it's because of a lack of context. You can start over and the same words will be correctly recognized.
As a test of Siri's core voice recognition, I read back the previous lines into the iPhone 4S, and quickly got the following text back as a result. I wasn't speaking with any special diction and was using a normal speaking speed. The result isn't perfect, but is very good, especially considering that you can touch to select problems and correct them. Siri didn't drop in some of the punctuation (I was barely stopping for commas), and got a few words wrong (marked in bold below):
Siri offloads voice recognition to Apple servers which apparently run software license from Nuance. The text-to-speech component of Siri is very good far easier to begin using a more accurate than other systems I've used. More importantly however serious and just voice recognition and dictation. The heart of Siri is a new AI system that works to understand what you mean not just transcribe the words you say. so what's nice to be old to talk rather than type the real functionality of Siri's ability to respond to what you want to do. This isn't always perfect but very usable generally when Siri fails to understand what you ask is because a lot of context you can start over in the same words of me correctly recognized.
Idea recognition
It's apparent when using Siri that you don't have to say everything perfectly, or mouth just the right words in the just the right order, or even have what you say be correctly recognized, because Siri looks at the context of your conversation with it to provide a useful answer more often than not.
If the response you get isn't what you were looking for, simply repeating or correcting your request generally delivers what you want. You don't have to say things according to a specific set of commands.
This use of context to interpret your commands means that Siri is most likely to get confused if you suddenly change what you're talking about without signaling that you want to start something new. You can always request to "start over."
Voice navigation
Some of the most useful things Siri does is perform jobs that would otherwise require typing in a bunch of text. Asking for a map of an area, or directions to a destination, is a key example. Or asking for weather, stock prices, definitions of terms and so on. You can also type the microphone icon in the virtual keyboard to enter text via your voice in most apps.
To create new Notes or to address and compose emails, you don't even need to hit a dictation button. You just tell Siri what you want to do. In the video below, I ask for directions, pull up an answer to a conversion question, ask Siri to compose a new Note, and ask to "call my mom," which prompts Siri to ask what my mom's name is, cross reference her Contact, and present three phone numbers I have associated with her. It will remember who I have identified as my mom, sister, brother or father next time I ask.
Also shown in the video is how Siri handles interruptions from multitasking notifications. The alarm that goes off while Siri is transcribing a Note for me has no impact on the user interface or the accuracy of the Note.
On page 2 of 3: Conversational context
Conversational context
In a second video, I ask about a weekday, a question that isn't correctly recognized; asking it again works however. An answer about stock prices and weather work as expected.
A general knowledge question and a request for a city's population get quick answers from Wolfram Alpha. Siri's use of context in answering questions is indicated when I ask about breakfast in San Francisco's Cow Hollow neighborhood. At first, while correctly recognizing the text of my question, Siri doesn't provide a useful answer. But when I ask again, despite failing to recognize the words I used, the context of the previous request resulted in Siri offering useful information.
I immediately changed the topic to getting gas in the Mission District, and Siri responds with breakfast options there instead. But when I repeat
that I'm looking for gas, Siri provides the right information.
Siri also helps you pull up information saved in your Notes (including those you've just dictated), offering a powerful way to delegate things you're supposed to remember to your smartphone. This feels addictive and easy.
Siri also helps you schedule and change your calendar, although in my example, it failed to immediately do what I asked. However, with a little persistence the system not only got it right, but also presented helpful information about potential calendar conflicts. After I thought I was finished with the calendar, I asked an unrelated question, which Siri stumbled on as it continued to talk about my calendar.
After appearing to double book the appointment I had been working on, I simply asked Siri to review what it had done to tomorrow's schedule. Everything was fine, no duplications and no missing events due to the change not being finished.
I tried asking about changing Pesos into US dollars, but Siri seemed stuck in my Calendar and unable to figure out what I was trying to do, despite catching the words correctly. However, after asking to "start over," Siri seemed able to get back on track and provide a useful answer.
On page 3 of 3: Siri outside of iPhone 4S
Siri outside of iPhone 4S
It's not clear how much of Siri's exclusive availability on the iPhone 4S is due to its far more powerful processor, and how much is simply Apple reserving the feature as an incentive to upgrade.
In previous iPhone hardware releases, Apple similarly reserved video capture as an exclusive of the iPhone 3GS, and made FaceTime exclusive to iPhone 4. While those features have been hacked to work on earlier models, they don't run nearly as well on the less powerful hardware.
Were Apple to make Siri available on the iPhone 4, it would likely work half as fast and half as well, likely tainting the experience of the majority of those able to immediately try it out. On iPhone 4S, the new feature works brilliantly, although not always flawlessly.
Apart from being far more sophisticated in what it can do, Siri also has a significant limitation over the Voice Control it replaces on the 4S: it only works if you have a network connection. In Airplane Mode, Siri won't work at all. The old Voice Control will, but its commands (and accuracy) are so limited that it isn't likely to get used that much.
Siri promises to change how often people use voice command by making it far more likely to work as expected, and to do functions that are valuable enough to remember how to use it. At some point, Apple will likely bring Siri to the Mac along with iOS 5's new iMessage chat, just as it did over the last year with FaceTime in iOS 4.
Siri is also likely to make its way to iPad 2, which is certainly powerful enough to run the software. It's less certain that Siri will be adapted to last year's iPhone 4 or the current crop of iPod touch models, none of which use the same speedy A5 chip that power the iPad 2 and iPhone 4S.
Apart from iPhone 4S, other iOS devices upgraded to run the free new iOS 5 continue to supply Voice Control, which responds to a variety of voice commands such as "what time is it?" or "play songs by MGMT," or "call Tim Cook."
The new Siri dramatically expands upon Voice Control, offering a much wider, more powerful vocabulary for creating Notes, composing Emails and Messages, navigating and changing Calendar events, setting new Reminders, and taking dictation within apps virtually anywhere you could type.
Siri also plugs into external services, integrating with smart artificial intelligence servers hosted by Apple to decipher not just basic commands but natural language questions. It then provides answers through Google or Bing, Wolfram Alpha, Wikipedia, Yelp, and Yahoo weather and stock market information.
Voice recognition
Siri offloads voice recognition to Apple's servers, which apparently run software licensed from Nuance. The text to speech component of Siri is very good, far easier to begin using and more accurate than other systems I've used.
More importantly, however, Siri isn't just voice recognition and dictation. The heart of Siri is a new AI system that works to understand what you mean, not just transcribe the words you say. So while it's nice to be able to talk rather than type, the real functionality of Siri is the ability to respond to what you want to do.
This isn't always perfect, but it's very usable. Generally, when Siri fails to understand what you ask, it's because of a lack of context. You can start over and the same words will be correctly recognized.
As a test of Siri's core voice recognition, I read back the previous lines into the iPhone 4S, and quickly got the following text back as a result. I wasn't speaking with any special diction and was using a normal speaking speed. The result isn't perfect, but is very good, especially considering that you can touch to select problems and correct them. Siri didn't drop in some of the punctuation (I was barely stopping for commas), and got a few words wrong (marked in bold below):
Siri offloads voice recognition to Apple servers which apparently run software license from Nuance. The text-to-speech component of Siri is very good far easier to begin using a more accurate than other systems I've used. More importantly however serious and just voice recognition and dictation. The heart of Siri is a new AI system that works to understand what you mean not just transcribe the words you say. so what's nice to be old to talk rather than type the real functionality of Siri's ability to respond to what you want to do. This isn't always perfect but very usable generally when Siri fails to understand what you ask is because a lot of context you can start over in the same words of me correctly recognized.
Idea recognition
It's apparent when using Siri that you don't have to say everything perfectly, or mouth just the right words in the just the right order, or even have what you say be correctly recognized, because Siri looks at the context of your conversation with it to provide a useful answer more often than not.
If the response you get isn't what you were looking for, simply repeating or correcting your request generally delivers what you want. You don't have to say things according to a specific set of commands.
This use of context to interpret your commands means that Siri is most likely to get confused if you suddenly change what you're talking about without signaling that you want to start something new. You can always request to "start over."
Voice navigation
Some of the most useful things Siri does is perform jobs that would otherwise require typing in a bunch of text. Asking for a map of an area, or directions to a destination, is a key example. Or asking for weather, stock prices, definitions of terms and so on. You can also type the microphone icon in the virtual keyboard to enter text via your voice in most apps.
To create new Notes or to address and compose emails, you don't even need to hit a dictation button. You just tell Siri what you want to do. In the video below, I ask for directions, pull up an answer to a conversion question, ask Siri to compose a new Note, and ask to "call my mom," which prompts Siri to ask what my mom's name is, cross reference her Contact, and present three phone numbers I have associated with her. It will remember who I have identified as my mom, sister, brother or father next time I ask.
Also shown in the video is how Siri handles interruptions from multitasking notifications. The alarm that goes off while Siri is transcribing a Note for me has no impact on the user interface or the accuracy of the Note.
On page 2 of 3: Conversational context
Conversational context
In a second video, I ask about a weekday, a question that isn't correctly recognized; asking it again works however. An answer about stock prices and weather work as expected.
A general knowledge question and a request for a city's population get quick answers from Wolfram Alpha. Siri's use of context in answering questions is indicated when I ask about breakfast in San Francisco's Cow Hollow neighborhood. At first, while correctly recognizing the text of my question, Siri doesn't provide a useful answer. But when I ask again, despite failing to recognize the words I used, the context of the previous request resulted in Siri offering useful information.
I immediately changed the topic to getting gas in the Mission District, and Siri responds with breakfast options there instead. But when I repeat
that I'm looking for gas, Siri provides the right information.
Siri also helps you pull up information saved in your Notes (including those you've just dictated), offering a powerful way to delegate things you're supposed to remember to your smartphone. This feels addictive and easy.
Siri also helps you schedule and change your calendar, although in my example, it failed to immediately do what I asked. However, with a little persistence the system not only got it right, but also presented helpful information about potential calendar conflicts. After I thought I was finished with the calendar, I asked an unrelated question, which Siri stumbled on as it continued to talk about my calendar.
After appearing to double book the appointment I had been working on, I simply asked Siri to review what it had done to tomorrow's schedule. Everything was fine, no duplications and no missing events due to the change not being finished.
I tried asking about changing Pesos into US dollars, but Siri seemed stuck in my Calendar and unable to figure out what I was trying to do, despite catching the words correctly. However, after asking to "start over," Siri seemed able to get back on track and provide a useful answer.
On page 3 of 3: Siri outside of iPhone 4S
Siri outside of iPhone 4S
It's not clear how much of Siri's exclusive availability on the iPhone 4S is due to its far more powerful processor, and how much is simply Apple reserving the feature as an incentive to upgrade.
In previous iPhone hardware releases, Apple similarly reserved video capture as an exclusive of the iPhone 3GS, and made FaceTime exclusive to iPhone 4. While those features have been hacked to work on earlier models, they don't run nearly as well on the less powerful hardware.
Were Apple to make Siri available on the iPhone 4, it would likely work half as fast and half as well, likely tainting the experience of the majority of those able to immediately try it out. On iPhone 4S, the new feature works brilliantly, although not always flawlessly.
Apart from being far more sophisticated in what it can do, Siri also has a significant limitation over the Voice Control it replaces on the 4S: it only works if you have a network connection. In Airplane Mode, Siri won't work at all. The old Voice Control will, but its commands (and accuracy) are so limited that it isn't likely to get used that much.
Siri promises to change how often people use voice command by making it far more likely to work as expected, and to do functions that are valuable enough to remember how to use it. At some point, Apple will likely bring Siri to the Mac along with iOS 5's new iMessage chat, just as it did over the last year with FaceTime in iOS 4.
Siri is also likely to make its way to iPad 2, which is certainly powerful enough to run the software. It's less certain that Siri will be adapted to last year's iPhone 4 or the current crop of iPod touch models, none of which use the same speedy A5 chip that power the iPad 2 and iPhone 4S.
Comments
Suggestion to AppleInsider: make sure you CLEARLY communicate to your readership the parameters of the article (if they aren't clear). You have a lot of ground to make up.
In no point in this article is the fact that Siri is BETA software mentioned.
Suggestion to AppleInsider: make sure you CLEARLY communicate to your readership the parameters of the article (if they aren't clear). You have a lot of ground to make up.
When it works, it appears to be pretty good for beta software.
It's up to you to make sure how your experience translates into page views for this site's operator.
Good luck.
Your appreciation is duly noted.
It's up to you to make sure how your experience translates into page views for this site's operator.
Good luck.
I will absolutely do that. Just as soon as I figure out what on earth you are talking about...
I will absolutely do that. Just as soon as I figure out what on earth you are talking about...
Siri is absolutely awesome!
can't wait for the Siri API and move improvements and integration with the more apps!
"...you can start over in the same words of me correctly recognized."
Whaaaaaat?
I assume Siri will reach '1.0' status alongside iPad 3 in the spring of 2012. Then we'll see updates with each iOS increment. Give it a year and it will be extraordinary.
I would think it's initially a smart move to make siri live to a central network to learn and adapt- but it's had dot say right now because it's kind of back firing for now. no doubt it will clear up and become awesome- but apple needs to make sure the end game is to make it a client on each device- that syncs between devices for each user. connect it with Apis to apps- and we have a revolution of tech on ours hands. no doubt.
and of course- a wide variety of voices!
This is not a *question*; why are you surprised that there is no answer?
No wonder Siri was confused...
"...you can start over in the same words of me correctly recognized."
Whaaaaaat?
"Change 500 mexican pesos into US dollars."
Evidently, not only is proofreading nonexistent, half the time Siri didn't understand his requests was because DED speaks like he has a mouth-full of marbles....
Are we all in such a hurry that we can't take a minute to re-read our comments prior to posting?
I really hope Siri does continue to improve over time. I'm pretty blown away with what Siri can do today, at launch. Just imagine where it will be in a year with what it will learn based on interaction from users! It's my understanding that it does "learn" based on user interaction.
Wish I were in Ireland again. It'd be funny to see how Siri would deal with their version of spoken English... I love the Irish folks, but understanding what they're saying can be tough, as an American that is. Hope they've got an Irish Siri worked out...
We tested this today against a $49 Samsung Focus running Windows Phone 7 Mango. Siri Effed up with understanding us just as much or as less really as the Windows Phone.
Siri was just as slow as the WP7 as well. Yes siri can come up with some funny canned answers, but it is not light years ahead of everything else.
The 4S was not really any faster than the Windows Phone, in fact in lagged behind ever so slightly in a few cases. I think that Metro is just so watered down it moves faster.
The 4 inch Super Amoled screen on the Focus was brighter and more vibrant. Angry Birds looked more eye poping on the Focus.
The build quality of the 4S is simply much better...but its a phone....that gets put in a case to protect the glass.
The camera on the 4S is its best feature. It can take good photos as fast as you can tap the button....very amazing.
The flow of iOS is starting to get old. There are some nice features in the Metro UI that really make it easier to navigate, plus stuff like email just looks better compared to iOS.
DED you spin it well my brother. The iPhone 4S is a nice phone and will sell well, but its the same shite different day and Apple better step it up next year.
The closest siri can get it to this is " 's one " or "This one" - neither of which will get people to come down the pub. Even spelling it out produces weird results like "SW a N" and there doesn't seem to be any way to correct mistakes or train the recognition.
If I can't use it to send the simplest of text messages I send every week then I certainly won't be using it for anything more complicated.