First Look: Using iPhone 4S with Siri voice assistant

Posted:
in iPhone edited January 2014
Most of the new features in iOS 5 simply run faster on the iPhone 4S, but one major new feature is exclusive to the new model: Siri voice assistant. Here's a look at how it works in action, responding to a variety of requests.



Apart from iPhone 4S, other iOS devices upgraded to run the free new iOS 5 continue to supply Voice Control, which responds to a variety of voice commands such as "what time is it?" or "play songs by MGMT," or "call Tim Cook."



The new Siri dramatically expands upon Voice Control, offering a much wider, more powerful vocabulary for creating Notes, composing Emails and Messages, navigating and changing Calendar events, setting new Reminders, and taking dictation within apps virtually anywhere you could type.



Siri also plugs into external services, integrating with smart artificial intelligence servers hosted by Apple to decipher not just basic commands but natural language questions. It then provides answers through Google or Bing, Wolfram Alpha, Wikipedia, Yelp, and Yahoo weather and stock market information.







Voice recognition



Siri offloads voice recognition to Apple's servers, which apparently run software licensed from Nuance. The text to speech component of Siri is very good, far easier to begin using and more accurate than other systems I've used.



More importantly, however, Siri isn't just voice recognition and dictation. The heart of Siri is a new AI system that works to understand what you mean, not just transcribe the words you say. So while it's nice to be able to talk rather than type, the real functionality of Siri is the ability to respond to what you want to do.



This isn't always perfect, but it's very usable. Generally, when Siri fails to understand what you ask, it's because of a lack of context. You can start over and the same words will be correctly recognized.



As a test of Siri's core voice recognition, I read back the previous lines into the iPhone 4S, and quickly got the following text back as a result. I wasn't speaking with any special diction and was using a normal speaking speed. The result isn't perfect, but is very good, especially considering that you can touch to select problems and correct them. Siri didn't drop in some of the punctuation (I was barely stopping for commas), and got a few words wrong (marked in bold below):



Siri offloads voice recognition to Apple servers which apparently run software license from Nuance. The text-to-speech component of Siri is very good far easier to begin using a more accurate than other systems I've used. More importantly however serious and just voice recognition and dictation. The heart of Siri is a new AI system that works to understand what you mean not just transcribe the words you say. so what's nice to be old to talk rather than type the real functionality of Siri's ability to respond to what you want to do. This isn't always perfect but very usable generally when Siri fails to understand what you ask is because a lot of context you can start over in the same words of me correctly recognized.







Idea recognition



It's apparent when using Siri that you don't have to say everything perfectly, or mouth just the right words in the just the right order, or even have what you say be correctly recognized, because Siri looks at the context of your conversation with it to provide a useful answer more often than not.



If the response you get isn't what you were looking for, simply repeating or correcting your request generally delivers what you want. You don't have to say things according to a specific set of commands.



This use of context to interpret your commands means that Siri is most likely to get confused if you suddenly change what you're talking about without signaling that you want to start something new. You can always request to "start over."



Voice navigation



Some of the most useful things Siri does is perform jobs that would otherwise require typing in a bunch of text. Asking for a map of an area, or directions to a destination, is a key example. Or asking for weather, stock prices, definitions of terms and so on. You can also type the microphone icon in the virtual keyboard to enter text via your voice in most apps.



To create new Notes or to address and compose emails, you don't even need to hit a dictation button. You just tell Siri what you want to do. In the video below, I ask for directions, pull up an answer to a conversion question, ask Siri to compose a new Note, and ask to "call my mom," which prompts Siri to ask what my mom's name is, cross reference her Contact, and present three phone numbers I have associated with her. It will remember who I have identified as my mom, sister, brother or father next time I ask.



Also shown in the video is how Siri handles interruptions from multitasking notifications. The alarm that goes off while Siri is transcribing a Note for me has no impact on the user interface or the accuracy of the Note.







On page 2 of 3: Conversational context



Conversational context



In a second video, I ask about a weekday, a question that isn't correctly recognized; asking it again works however. An answer about stock prices and weather work as expected.



A general knowledge question and a request for a city's population get quick answers from Wolfram Alpha. Siri's use of context in answering questions is indicated when I ask about breakfast in San Francisco's Cow Hollow neighborhood. At first, while correctly recognizing the text of my question, Siri doesn't provide a useful answer. But when I ask again, despite failing to recognize the words I used, the context of the previous request resulted in Siri offering useful information.



I immediately changed the topic to getting gas in the Mission District, and Siri responds with breakfast options there instead. But when I repeat

that I'm looking for gas, Siri provides the right information.



Siri also helps you pull up information saved in your Notes (including those you've just dictated), offering a powerful way to delegate things you're supposed to remember to your smartphone. This feels addictive and easy.



Siri also helps you schedule and change your calendar, although in my example, it failed to immediately do what I asked. However, with a little persistence the system not only got it right, but also presented helpful information about potential calendar conflicts. After I thought I was finished with the calendar, I asked an unrelated question, which Siri stumbled on as it continued to talk about my calendar.



After appearing to double book the appointment I had been working on, I simply asked Siri to review what it had done to tomorrow's schedule. Everything was fine, no duplications and no missing events due to the change not being finished.



I tried asking about changing Pesos into US dollars, but Siri seemed stuck in my Calendar and unable to figure out what I was trying to do, despite catching the words correctly. However, after asking to "start over," Siri seemed able to get back on track and provide a useful answer.







On page 3 of 3: Siri outside of iPhone 4S



Siri outside of iPhone 4S



It's not clear how much of Siri's exclusive availability on the iPhone 4S is due to its far more powerful processor, and how much is simply Apple reserving the feature as an incentive to upgrade.



In previous iPhone hardware releases, Apple similarly reserved video capture as an exclusive of the iPhone 3GS, and made FaceTime exclusive to iPhone 4. While those features have been hacked to work on earlier models, they don't run nearly as well on the less powerful hardware.



Were Apple to make Siri available on the iPhone 4, it would likely work half as fast and half as well, likely tainting the experience of the majority of those able to immediately try it out. On iPhone 4S, the new feature works brilliantly, although not always flawlessly.



Apart from being far more sophisticated in what it can do, Siri also has a significant limitation over the Voice Control it replaces on the 4S: it only works if you have a network connection. In Airplane Mode, Siri won't work at all. The old Voice Control will, but its commands (and accuracy) are so limited that it isn't likely to get used that much.



Siri promises to change how often people use voice command by making it far more likely to work as expected, and to do functions that are valuable enough to remember how to use it. At some point, Apple will likely bring Siri to the Mac along with iOS 5's new iMessage chat, just as it did over the last year with FaceTime in iOS 4.



Siri is also likely to make its way to iPad 2, which is certainly powerful enough to run the software. It's less certain that Siri will be adapted to last year's iPhone 4 or the current crop of iPod touch models, none of which use the same speedy A5 chip that power the iPad 2 and iPhone 4S.





«13456711

Comments

  • Reply 1 of 219
    muppetrymuppetry Posts: 3,324member
    Trying to test Siri on a friend's 4S this evening was completely unsuccessful. It understood the words just fine - reproducing them on the screen - but then, after 10 or 15 seconds, responded that something "seemed to have gone wrong" and to "try again". I'm wondering if it is another case of the servers being swamped.
  • Reply 2 of 219
    In no point in this article is the fact that Siri is BETA software mentioned.



    Suggestion to AppleInsider: make sure you CLEARLY communicate to your readership the parameters of the article (if they aren't clear). You have a lot of ground to make up.
  • Reply 3 of 219
    muppetrymuppetry Posts: 3,324member
    Quote:
    Originally Posted by cvaldes1831 View Post


    In no point in this article is the fact that Siri is BETA software mentioned.



    Suggestion to AppleInsider: make sure you CLEARLY communicate to your readership the parameters of the article (if they aren't clear). You have a lot of ground to make up.



    When it works, it appears to be pretty good for beta software.
  • Reply 4 of 219
    Your appreciation is duly noted.



    It's up to you to make sure how your experience translates into page views for this site's operator.



    Good luck.



  • Reply 5 of 219
    muppetrymuppetry Posts: 3,324member
    Quote:
    Originally Posted by cvaldes1831 View Post


    Your appreciation is duly noted.



    It's up to you to make sure how your experience translates into page views for this site's operator.



    Good luck.







    I will absolutely do that. Just as soon as I figure out what on earth you are talking about...
  • Reply 6 of 219
    [EDIT: no, not worth the time nor effort.]
  • Reply 7 of 219
    Pretty damn good for a beta software, how much more when Apple gets it out of beta, adds more services and integration. Not to mention when they open it to 3rd party developers.
  • Reply 8 of 219
    paxmanpaxman Posts: 4,517member
    Quote:
    Originally Posted by muppetry View Post


    I will absolutely do that. Just as soon as I figure out what on earth you are talking about...



  • Reply 9 of 219
    I've been playing with Siri for a few hours AND I LOVE IT.





    Siri is absolutely awesome!



    can't wait for the Siri API and move improvements and integration with the more apps!
  • Reply 10 of 219
    No wonder Siri was confused...



    "...you can start over in the same words of me correctly recognized."



    Whaaaaaat?
  • Reply 11 of 219
    Siri looks great, and has great potential. Like a human being, you learn what it responds to and how it will feel about what you're saying and you know exactly what to say. I'm sure each person who uses it will see improvements and realise when to say things such as 'start over'. My iPhone 4S is still awaiting delivery but I'll be using Siri for hours when it does!



    I assume Siri will reach '1.0' status alongside iPad 3 in the spring of 2012. Then we'll see updates with each iOS increment. Give it a year and it will be extraordinary.
  • Reply 12 of 219
    If Apple was looking for this to make me forget that I got the SAME phone for the 2nd year in a row they have failed. The ONLY reason I upgraded was because I wanted a white iPhone and since I have gotten my phone things have going wrong with it! My partner got one at the same time and when I opened his to get it ready for him I noticed he had bars immediately! My phone said no service out of the package !! Then his phone was ready immediately out of the box and mine didnt work for 2-3 hrs. Now my Siri keeps telling me its having a hard time connecting to the network and hardly ever gets what I am saying!
  • Reply 13 of 219
    macrrmacrr Posts: 488member
    IT seems on the west coast- as all of us get our phones- siri can't connect to the network.. yet a friend of mine in Chicago says her phone is working great with Siri.



    I would think it's initially a smart move to make siri live to a central network to learn and adapt- but it's had dot say right now because it's kind of back firing for now. no doubt it will clear up and become awesome- but apple needs to make sure the end game is to make it a client on each device- that syncs between devices for each user. connect it with Apis to apps- and we have a revolution of tech on ours hands. no doubt.
  • Reply 14 of 219
    macrrmacrr Posts: 488member
    ...it seems that another end game is that it should always be aware- and literally button free.. just announce Siri - and it activates.. but then again- it'd be nice to nam fit shithead too. or fartsmoker. or peen or. you know.. even uncle buck. or mr belvedere. or hank.





    and of course- a wide variety of voices!
  • Reply 15 of 219
    "Change 500 mexican pesos into US dollars."



    This is not a *question*; why are you surprised that there is no answer?
  • Reply 16 of 219
    Quote:
    Originally Posted by SpamSandwich View Post


    No wonder Siri was confused...



    "...you can start over in the same words of me correctly recognized."



    Whaaaaaat?



    Quote:
    Originally Posted by The-Steve View Post




    "Change 500 mexican pesos into US dollars."





    Evidently, not only is proofreading nonexistent, half the time Siri didn't understand his requests was because DED speaks like he has a mouth-full of marbles....
  • Reply 17 of 219
    I hope the majority of people can put a 'spoken' sentence together better than the above posters have structured their comments. If not, Siri's going to have a tough time figuring out what the hell most of you are talking about... Maybe you guys are not native English speakers or maybe some of you haven't completed grade school yet. Don't mean to be harsh, but take two seconds to re-read your comments prior to posting... I'm struggling to get the full meaning from what some of you have written.



    Are we all in such a hurry that we can't take a minute to re-read our comments prior to posting?



  • Reply 18 of 219
    Thanks for another detailed and well written article Daniel. I've been waiting to read about Siri's real-world performance. Guess if Siri can understand you, with your "mouth full of marbles" then it might have a chance... Not sure I agree with the marbles comment, but it was kind of funny to read.



    I really hope Siri does continue to improve over time. I'm pretty blown away with what Siri can do today, at launch. Just imagine where it will be in a year with what it will learn based on interaction from users! It's my understanding that it does "learn" based on user interaction.



    Wish I were in Ireland again. It'd be funny to see how Siri would deal with their version of spoken English... I love the Irish folks, but understanding what they're saying can be tough, as an American that is. Hope they've got an Irish Siri worked out...
  • Reply 19 of 219
    DED who is your doctor because brother you are getting some GOOD DRUGS.



    We tested this today against a $49 Samsung Focus running Windows Phone 7 Mango. Siri Effed up with understanding us just as much or as less really as the Windows Phone.



    Siri was just as slow as the WP7 as well. Yes siri can come up with some funny canned answers, but it is not light years ahead of everything else.



    The 4S was not really any faster than the Windows Phone, in fact in lagged behind ever so slightly in a few cases. I think that Metro is just so watered down it moves faster.



    The 4 inch Super Amoled screen on the Focus was brighter and more vibrant. Angry Birds looked more eye poping on the Focus.



    The build quality of the 4S is simply much better...but its a phone....that gets put in a case to protect the glass.



    The camera on the 4S is its best feature. It can take good photos as fast as you can tap the button....very amazing.



    The flow of iOS is starting to get old. There are some nice features in the Metro UI that really make it easier to navigate, plus stuff like email just looks better compared to iOS.



    DED you spin it well my brother. The iPhone 4S is a nice phone and will sell well, but its the same shite different day and Apple better step it up next year.
  • Reply 20 of 219
    I'm totally underwhelmed with siri so far. The name of my local pub is "The Swan" and it appears frequently in my texts...



    The closest siri can get it to this is " 's one " or "This one" - neither of which will get people to come down the pub. Even spelling it out produces weird results like "SW a N" and there doesn't seem to be any way to correct mistakes or train the recognition.



    If I can't use it to send the simplest of text messages I send every week then I certainly won't be using it for anything more complicated.
Sign In or Register to comment.