Apple refining Siri to cut down on mistaken activations, and to draw less power

Posted:
in General Discussion edited November 2020
Apple is researching ways where Siri can be better at assessing whether you really meant to call it, and also when you're done speaking to it.

Siri could become better at only responding when you want it to
Siri could become better at only responding when you want it to


You've done this. You've had to hurriedly slap your palm over your Apple Watch because Siri has begun talking. And you've had that perplexing moment of trying to fathom out what you could possibly have just said that sounded anything like "Hey, Siri." Apple wants to change that.

There is also the separate issue of when Siri doesn't respond, or when you turn your Watch and the iPhone in your pocket replies instead, but that's a problem for another day. For now, Apple is focusing on how Siri can decide whether to respond based on how interested you seem.

Apple has previously applied for a patent where the iPhone's Face ID camera interprets your emotions when you ask Siri to do something. But a new patent application starts with whether you're even looking at your device when you speak.

"Attention Aware Virtual Assistant Dismissal," is a newly revealed patent application which both lists criteria for assessing whether you want Siri, and then what the device should do next.

"[The] process includes determining, based on data obtained using one or more sensors of the electronic device, whether one or more criteria representing expressed user disinterest are satisfied," says the application.

There are a lot of factors involved in determining this "disinterest," and effectively in when Siri should take you seriously or not. For instance, a key way of determining whether you're interested is by having the device's cameras look to detect your gaze.

If you're looking directly at the device when you've activated Siri, the iPhone can be pretty certain that you meant to call for it and that you're going to say a command next. However, you might well say "Hey, Siri," and then be momentarily distracted by someone or something around you.

So the patent application discusses the length of time that might or might not be reasonable in working out your attention. Apple refers to this as the "characteristic intensity" of a contact, or in other words, of your saying "Hey, Siri."

"[This] characteristic intensity is, optionally, based on a predefined number of intensity samples," says Apple, "or a set of intensity samples collected during a predetermined time period... relative to a predefined event (e.g., after detecting the contact)."

The patent application avoids specifying actual durations for this delay, preferring instead to say that it varies depending on the situation. The application spends less time on how long a device should wait, and much more on how it can assess intention or disinterest.

As well as whether you're looking at the device, for instance, the criteria for concluding that you're disinterested includes using many different sensors. The device's accelerometer can tell whether you're lowering the device, or picking it up, for instance.

Detail from the patent application showing the start of determining whether a user wanted to activate Siri or not
Detail from the patent application showing the start of determining whether a user wanted to activate Siri or not


The system can also determine whether you've placed the device face-down on a surface. Similarly, if you have the display covered by your hand, perhaps as you carry it, then the front-facing light sensor can tell that.

That same sensor can also contribute to the device recognizing whether it is in an enclosed space. If you have your iPhone in a pocket or a bag, for instance, you're less likely to be calling for Siri.

This isn't just about cutting down on the number of times a phrase similar to "Hey, Siri," is followed by your saying something a little more fruity. Apple is also looking to how it can maximize the performance of its devices.

"Operating a digital assistant requires electric power, which is a limited resource on handheld or portable devices that rely on batteries and on which digital assistants often run," it says. "Accordingly, it can be desirable to operate a digital assistant in an energy efficient manner."

So if your iPhone can tell very quickly that you weren't calling for Siri, it doesn't have to activate it pointlessly.

This patent application is credited to three inventors, including Rohit Dasari whose previous work includes related patents such as one about how apps can integrate with a digital assistant.

Keep up with all the Apple news with your iPhone, iPad, or Mac. Say, "Hey, Siri, play AppleInsider Daily," -- or bookmark this link -- and you'll get a fast update direct from the AppleInsider team.

Comments

  • Reply 1 of 17
    MacProMacPro Posts: 19,471member
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    razorpitThinkmanDuvSpamSandwichBeatsJWSC
  • Reply 2 of 17
    Afganblues Afganblues Posts: 10unconfirmed, member
    Several times a day, Siri askes me to repeat myself when I haven’t even said “hey Siri “. And it’s extremely frustrating to call out Siri on my phone and have my HomePod respond 45 feet away in the other room. I’ve had my HomePod Respond when I’m in my bedroom and one of my home Pod in the kitchen, and there is my 22 foot wide living room in between my bedroom and the kitchen. It’s ridiculous. I don’t know why Apple simply won’t allow us to change the activation for HomePod to something like “hey HomePod “instead of hey Siri. Steve Jobs would’ve been furious about the way Siri works much like the way he was furious about mobile me.
    edited September 2020 razorpitmuthuk_vanalingamJWSC
  • Reply 3 of 17
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    I haven't called Apple regarding the same issue you're having yet as I'd like some time between support calls that had become all too frequent. To be fair, I have a lot of Apple gear, and 99% of the time they work as expected. Also, Apple support typically is excellent in resolving my issues, though often I need to speak with a Sr. Advisor to get there. The issue with the HomePods is one that I've semi-resolved by using Handoff from my iPhone. After doing this, I often find my HomePods playing what I request. This is a very weird workaround, but given the number of times I use the HomePods, it's acceptable for now. 
    watto_cobra
  • Reply 4 of 17
    mike1mike1 Posts: 2,948member
    Several times a day, Siri askes me to repeat myself when I haven’t even said “hey Siri “. And it’s extremely frustrating to call out Siri on my phone and have my HomePod respond 45 feet away in the other room. I’ve had my HomePod Respond when I’m in my bedroom and one of my home Pod in the kitchen, and there is my 22 foot wide living room in between my bedroom and the kitchen. It’s ridiculous. I don’t know why Apple simply won’t allow us to change the activation for HomePod to something like “hey HomePod “instead of hey Siri. Steve Jobs would’ve been furious about the way Siri works much like the way he was furious about mobile me.

    My setup works flawlessly. I turned off Siri voice activation on all devices except the HomePod. If I am within ear shot of my HomePod, the HomePod responds, which is exactly what I want. Otherwise, I manually activate Siri on my Watch or Phone. My wife, on the other hand, still has hers active and the the HomePod still always responds rather than her Phone or iPad.
    cornchipbonobobFileMakerFellerJWSCwatto_cobra
  • Reply 5 of 17
    MacProMacPro Posts: 19,471member
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    I haven't called Apple regarding the same issue you're having yet as I'd like some time between support calls that had become all too frequent. To be fair, I have a lot of Apple gear, and 99% of the time they work as expected. Also, Apple support typically is excellent in resolving my issues, though often I need to speak with a Sr. Advisor to get there. The issue with the HomePods is one that I've semi-resolved by using Handoff from my iPhone. After doing this, I often find my HomePods playing what I request. This is a very weird workaround, but given the number of times I use the HomePods, it's acceptable for now. 
    Yes, I have so much Apple gear it looks like an Apple Store here and it all works perfectly.  My only niggle is Siri on HomePods and frankly, I'd use them without Siri, the audio quality is so good.  

    Good idea on handoff for HomePod but it is a few extra steps and it used to work so well with Siri.  I've read countless posts with 'cures', none work for me.  Weirdly she can play some of the music e.g. my Eagles albums but not my Glen Frey albums.  This all started when Apple Music was born which I don't subscribe to. Also after an update or a HomePod restart, everything works like it used to for about five minutes and will play anything... then a short while later... 'Sorry I can't find Glen Frey in your Music Library' again.

    I half wonder if it could be related to where I bought the CDs which were ripped into iTunes in the first place.  Many were from the UK before I moved here 30 years ago.  Perhaps the new algorithms only work now on CDs purchased in the country where you live? I will try to investigate this.
    edited September 2020 muthuk_vanalingamwatto_cobra
  • Reply 6 of 17
    cornchipcornchip Posts: 1,897member
    For months my wife was hating on her new iPad because it wasn’t receiving non-Apple texts. I just let her flounder for a few weeks until the other day she brought it up again. I was like “well... is that setting turned on”? Of course it wasn’t. Something else like that also happened recently but I can’t remember what it was.

    There’s probably a setting for that. 
    watto_cobra
  • Reply 7 of 17
    DAalsethDAalseth Posts: 1,988member
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    Heck I'd settle for it being able to answer a question, or take a command at all. It's an amusing gadget but I've found it almost never work as I want it to. 
    Simple questions, "What time do the Twins play?" works, though inexplicably half the time it wants Minnesota Twins and the rest it doesn't care. But more complex questions, "Is Home Depot open?" almost never give me an answer other than "Here is what I found on the web for..." It should be smart enough to know that I mean my closest Home Depot, etc. the rest of the time it just doesn't understand.
    As far as commands, fugettaboutit. "Remind me to take out hamburger when I get home" or "Remind me to turn on the Game at 4:00."  is pointless. That sort of command just confuses Siri.  I've tried different phrasing, different patterns of the question "Set a reminder for noon to water the plants." Nope, it's always been a hard fail for me.
  • Reply 8 of 17
    Meh. An annoying & useless feature I disable.

    How about “refining” Siri so that it’s actually useful for more than “five minute timer” and has some capability to respond to CONTEXT??

    Can’t do it. It’s still just a pipe dream. Until someone figures out actual intelligence, all of this so-called (and inappropriately named) “AI” is going to keep going nowhere. It’s algorithms, not intelligence. The more the industry obsesses over algorithms for everything [that they don’t want real humans employed to do], the worse things get (looking at you, Google & Facebook).

    These “assistants” can’t do much of anything (and only speed up a couple of the few tasks they can accomplish, like “five minute timer”). The more they try to mimic the appearance of human intelligence, the more obviously stupid it looks. “Artificial Stupidity” is the result. It’s the uncanny valley for intelligence simulation.
  • Reply 9 of 17
    RIP Siri. It (?) won’t be missed.
  • Reply 10 of 17
    Rayz2016Rayz2016 Posts: 6,957member
    DAalseth said:
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    Heck I'd settle for it being able to answer a question, or take a command at all. It's an amusing gadget but I've found it almost never work as I want it to. 
    Simple questions, "What time do the Twins play?" works, though inexplicably half the time it wants Minnesota Twins and the rest it doesn't care. But more complex questions, "Is Home Depot open?" almost never give me an answer other than "Here is what I found on the web for..." It should be smart enough to know that I mean my closest Home Depot, etc. the rest of the time it just doesn't understand.
    As far as commands, fugettaboutit. "Remind me to take out hamburger when I get home" or "Remind me to turn on the Game at 4:00."  is pointless. That sort of command just confuses Siri.  I've tried different phrasing, different patterns of the question "Set a reminder for noon to water the plants." Nope, it's always been a hard fail for me.
    Weird. 

    If I ask Siri if Tesco is open, it gives me the opening time of the closest one, then it tells me the address of the next closest one and asks if I want the opening times for that. 
    watto_cobra
  • Reply 11 of 17
    I find Siri to work really well & accurately in most cases. I do have issues with accidental activations on both my Apple Watch & iPhone. Lessening those would be fantastic.

    My biggest Siri complaints are it’s non-usefulness when I have bad/no cell reception (do I really need internet connection to have Siri set a timer) and Siri’s varying abilities per device. So frustrating to ask something on my Watch just to be sent to my phone. 


    watto_cobra
  • Reply 12 of 17
    Better be good algorithm, because I use Siri exclusively when my phone is in my pocket or away from me! 
    watto_cobra
  • Reply 13 of 17
    Rayz2016 said:
    DAalseth said:
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    Heck I'd settle for it being able to answer a question, or take a command at all. It's an amusing gadget but I've found it almost never work as I want it to. 
    Simple questions, "What time do the Twins play?" works, though inexplicably half the time it wants Minnesota Twins and the rest it doesn't care. But more complex questions, "Is Home Depot open?" almost never give me an answer other than "Here is what I found on the web for..." It should be smart enough to know that I mean my closest Home Depot, etc. the rest of the time it just doesn't understand.
    As far as commands, fugettaboutit. "Remind me to take out hamburger when I get home" or "Remind me to turn on the Game at 4:00."  is pointless. That sort of command just confuses Siri.  I've tried different phrasing, different patterns of the question "Set a reminder for noon to water the plants." Nope, it's always been a hard fail for me.
    Weird. 

    If I ask Siri if Tesco is open, it gives me the opening time of the closest one, then it tells me the address of the next closest one and asks if I want the opening times for that. 
    I’m with you. I just asked HomePod Siri, “Is Home Depot open?” Siri replied, “Here are 15 options, the Home Depot on (whatever) street is open until 6pm. Would you like to hear the next one?” The one Siri gave me the hours for is the closest to me. I also tried the reminders that @DAalseth mentioned and they both worked for me, on HomePod still.

    Sometimes I have an issue where I’m intending HomePod to hear me but something else will respond. For instance, I’ll have HomePod Siri to play a song and my iPad, which is sitting on a table nearby will start playing OR my Watch will respond and my phone will start playing even though my wrist is by my side and my phone is in my pocket.

    The thing I find most baffling is how different people get different results when asking the same question when phrased the exact same way. I think it was @SpamSandwich who had an issue a couple of years ago. When asking Siri who starred in a particular movie the response was “here’s what I found on the web.” When I asked the exact same way Siri (on my iPhone) replied with who starred in the movie, along with the movie poster, run time and other information about the movie. It’s bizarre.
    watto_cobra
  • Reply 14 of 17
    BeatsBeats Posts: 2,902member
    Apple has been working on Siri for years with little sign of improvement.

    When will Siri get out of beta??????
    razorpit
  • Reply 15 of 17
    razorpitrazorpit Posts: 1,796member
    Rayz2016 said:
    DAalseth said:
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    Heck I'd settle for it being able to answer a question, or take a command at all. It's an amusing gadget but I've found it almost never work as I want it to. 
    Simple questions, "What time do the Twins play?" works, though inexplicably half the time it wants Minnesota Twins and the rest it doesn't care. But more complex questions, "Is Home Depot open?" almost never give me an answer other than "Here is what I found on the web for..." It should be smart enough to know that I mean my closest Home Depot, etc. the rest of the time it just doesn't understand.
    As far as commands, fugettaboutit. "Remind me to take out hamburger when I get home" or "Remind me to turn on the Game at 4:00."  is pointless. That sort of command just confuses Siri.  I've tried different phrasing, different patterns of the question "Set a reminder for noon to water the plants." Nope, it's always been a hard fail for me.
    Weird. 

    If I ask Siri if Tesco is open, it gives me the opening time of the closest one, then it tells me the address of the next closest one and asks if I want the opening times for that. 
    Whatever you do don't tell them you got them at Tesco's.  :D

    Sorry, for the last 30 years or so that line comes to mind every time I hear the word Tesco.
    watto_cobra
  • Reply 16 of 17
    razorpit said:
    Rayz2016 said:
    DAalseth said:
    MacPro said:
    "Siri could become better at only responding when you want it to" I'd settle for her being able to answer a question on HomePod without apologizing for not being able to,  not to mention play my iTunes Match music on HomePods again like she used to be able to.
    Heck I'd settle for it being able to answer a question, or take a command at all. It's an amusing gadget but I've found it almost never work as I want it to. 
    Simple questions, "What time do the Twins play?" works, though inexplicably half the time it wants Minnesota Twins and the rest it doesn't care. But more complex questions, "Is Home Depot open?" almost never give me an answer other than "Here is what I found on the web for..." It should be smart enough to know that I mean my closest Home Depot, etc. the rest of the time it just doesn't understand.
    As far as commands, fugettaboutit. "Remind me to take out hamburger when I get home" or "Remind me to turn on the Game at 4:00."  is pointless. That sort of command just confuses Siri.  I've tried different phrasing, different patterns of the question "Set a reminder for noon to water the plants." Nope, it's always been a hard fail for me.
    Weird. 

    If I ask Siri if Tesco is open, it gives me the opening time of the closest one, then it tells me the address of the next closest one and asks if I want the opening times for that. 
    Whatever you do don't tell them you got them at Tesco's.  :D

    Sorry, for the last 30 years or so that line comes to mind every time I hear the word Tesco.
    Oh, so you're a Waitrose shopper? :P
    razorpit
  • Reply 17 of 17
    I like the idea of Siri shutting off if I frown at my phone. Would work for everyone except Grumpy Cat, lol.
    watto_cobra
Sign In or Register to comment.