Google Assistant response speed getting improved by on-device processing

Posted:
in General Discussion edited May 7
The Google Assistant will be quicker to respond to user queries faster by performing processing on the device, the search company revealed at Google I/O, with the next generation of the virtual assistant able to quickly switch between apps and help with multitasking in Android than for previous versions.




The promise of on-device processing of digital assistants offer the possibility of faster response times for queries based on locally-held data, without necessarily requiring an internet connection. It also has the prospect of fending off any potential privacy issues caused through dispatching requests to the cloud, where the data may be analyzed and used for other purposes.

At Google I/O, the search company suggested it was getting closer to the idea of on-device virtual assistants, with the next version capable of processing requests up to ten times faster.

An on-stage demonstration showed someone rattling through multiple queries in quick succession, with a small corner transcript of what was being heard revealing a lot of the processing was being carried out at a very high speed. Much faster than what would normally be expected by offloading processing to a separate server.

According to Google, it has reduced the size of the artificial intelligence models used for speech recognition and language understanding down from hundreds of gigabytes in size to less than half a gigabyte, making it more manageable to store on a smartphone. This enables the next generation Assistant to perform near real-time transcription without an Internet connection.





The Assistant also includes query features like Continued Conversation, where multiple queries could be made in a row without requiring the use of "Hey Google" every time. The examples also showed how a user could write a message out verbally and insert an image from a separate app, all without requiring any finger presses or vocal prompts to change task.

Google anticipates the next-generation Assistant will be arriving initially on its new Pixel smartphones later this year. It is unclear when it will roll out to other Android device, or even to the iOS app.

Google is not the only company grappling with the idea of on-device processing for virtual assistants.

Siri, Apple's digital assistant that is on a par with Google Assistant in terms of worldwide usage, could have an offline mode if a November patent application comes to fruition. Titled "Offline personal assistant," the Apple filing suggests the use of speech-to-text processing and validation to occur on the device, and if a connection was available, could perform both local and remote processing for two possible interpretations of a query, which could be used to further confirm a transcription as correct.
«1

Comments

  • Reply 1 of 21
    gatorguygatorguy Posts: 20,698member
    Everyone will appreciate a feature available starting today:
    "Users in English-speaking countries will be able to stop ringing alarms and timers on Google Home and Smart Display devices simply by saying "stop," no hotword required."
    dws-2forgot usernamemuthuk_vanalingamCarnage
  • Reply 2 of 21
    Duh...

    Microsoft figured this out in 2005 with Voice Command, which existed on the Windows Phones.

    Streaming requests that can be handled locally (i.e. "Call Julie on Mobile", or "what appointments do I have today?") over the Internet is just dumb, inefficient, and exposes privacy concerns.  Items such as "Who put the bop in the bop-she-bop-she-bop" or "What movies are playing on El Cajon Blvd theaters tonight?" are those that parsed on the device, then a web query can be used to search the Internet.

    A central place for all of my, and everyone else's queries are the ingredients for 1984.
    watto_cobra
  • Reply 3 of 21
    avon b7avon b7 Posts: 3,901member
    We all knew this capacity was coming but it's good to see it 'go live' so to speak. Things like music playback control by voice should be doable via assistants without requiring an internet connection.

    Two years ago we were told onboard AI would be a major advance for low latency, ultra fast response situations and where privacy is paramount. It's becoming reality.
    edited May 7
  • Reply 4 of 21
    genovellegenovelle Posts: 967member
    Duh...

    Microsoft figured this out in 2005 with Voice Command, which existed on the Windows Phones.

    Streaming requests that can be handled locally (i.e. "Call Julie on Mobile", or "what appointments do I have today?") over the Internet is just dumb, inefficient, and exposes privacy concerns.  Items such as "Who put the bop in the bop-she-bop-she-bop" or "What movies are playing on El Cajon Blvd theaters tonight?" are those that parsed on the device, then a web query can be used to search the Internet.

    A central place for all of my, and everyone else's queries are the ingredients for 1984.
    Interesting. Device and media control has been handled on device since 2001 on macs. The original iPhone and even some iPods before it could as well via voice commands. Apple announced for iOS 11 that most Siri request were being processed on device. Only database or internet queries would require a connection. So, Google is announcing they are going to do something Apple implemented 2 years ago, and the article pretends Apple is working on it, when in fact they did it in 2017. Odd.  Here’s an article about it from Sept 2017. https://www.fastcompany.com/40443055/apple-explains-how-its-making-siri-smart-without-endangering-user-privacy
    racerhomie3gutengeljbdragonchaickafastasleepn2itivguywatto_cobra
  • Reply 5 of 21
    gatorguygatorguy Posts: 20,698member
    genovelle said:
    Duh...

    Microsoft figured this out in 2005 with Voice Command, which existed on the Windows Phones.

    Streaming requests that can be handled locally (i.e. "Call Julie on Mobile", or "what appointments do I have today?") over the Internet is just dumb, inefficient, and exposes privacy concerns.  Items such as "Who put the bop in the bop-she-bop-she-bop" or "What movies are playing on El Cajon Blvd theaters tonight?" are those that parsed on the device, then a web query can be used to search the Internet.

    A central place for all of my, and everyone else's queries are the ingredients for 1984.
    Interesting. Device and media control has been handled on device since 2001 on macs. The original iPhone and even some iPods before it could as well via voice commands. Apple announced for iOS 11 that most Siri request were being processed on device. Only database or internet queries would require a connection. So, Google is announcing they are going to do something Apple implemented 2 years ago, and the article pretends Apple is working on it, when in fact they did it in 2017. Odd.  Here’s an article about it from Sept 2017. https://www.fastcompany.com/40443055/apple-explains-how-its-making-siri-smart-without-endangering-user-privacy
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 

    https://www.youtube.com/watch?time_continue=7&v=FDdorhY7fb8
    edited May 7 muthuk_vanalingam
  • Reply 6 of 21
    chasmchasm Posts: 1,585member
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.
    edited May 7 StrangeDayslolliver
  • Reply 7 of 21
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
  • Reply 8 of 21
    entropysentropys Posts: 1,745member
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    watto_cobra
  • Reply 9 of 21
    FolioFolio Posts: 566member
    And maybe a few for the subcategory: "Vonbrick, likely susceptibilities"
    lolliverwatto_cobra
  • Reply 10 of 21
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]
  • Reply 11 of 21
    FolioFolio Posts: 566member
    If in real world, it works as well in demos then pretty impressive. The AR walking directions in Maps looks interesting too.
  • Reply 12 of 21
    canukstormcanukstorm Posts: 1,949member
    vonbrick said:
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]
    The biggest beef is with Google's business model which is predicated on selling all the information it's collected on people to advertisers and in return inundating you with ads.  At its heart it's still predominantly an advertising company.
    n2itivguylolliverwatto_cobra
  • Reply 13 of 21
    vonbrick said:
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]
    DED had a good example the other day. (https://appleinsider.com/articles/19/05/04/editorial-the-new-services---how-will-apple-arcades-exclusivity-privacy-affect-android-google-play)

    Modern ad tracking goes well beyond just profiling users into various demographics to show them "relevant ads." Ad networks exist to find cross-app and cross-site web browsing correlations they can market to ad buyers. 

    So when their user surveillance notices, for example, that a large number of users who install a specific workout app and also use a food delivery service are also statistically likely to pay for a subscription to Grindr, they can offer strategic ad placement to the vendor of that gay hookup app within every app that particular population of users will see. 

    ...

    The specific example above of an attempt at "relevant advertising" is potentially valuable to an advertiser seeking to target their ad budget at gay men willing to pay for a cruising tool, but it may make faulty assumptions about the user of a device. That could end up being embarrassing at best and plausibly even cause a person to lose their job in any number of states or countries where there is no legal protection from discrimination launched in the mere suspicion of a person's private life details. 

    That targeting could also destroy a relationship. And if you're traveling in some countries, it could potentially threaten you with detainment and even persecution at the border. All because your private behaviors were interpreted by an algorithm to imply a potential interest that others might notice. Targeted advertising can appear to reveal private things about you that may not even be accurate, yet with such confidence that it makes you look like you're hiding something.

    Perhaps this doesn’t reflect you or your habits, but it’s an example of what is going on behind the scenes that many people are unaware of.
    n2itivguylolliverflyingdpwatto_cobra
  • Reply 14 of 21
    gatorguygatorguy Posts: 20,698member
    vonbrick said:
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]
    DED had a good example the other day. 

    Modern ad tracking goes well beyond just profiling users into various demographics to show them "relevant ads." Ad networks exist to find cross-app and cross-site web browsing correlations they can market to ad buyers. 

    So when their user surveillance notices, for example, that a large number of users who install a specific workout app and also use a food delivery service are also statistically likely to pay for a subscription to Grindr, they can offer strategic ad placement to the vendor of that gay hookup app within every app that particular population of users will see. 

    ...

    The specific example above of an attempt at "relevant advertising" is potentially valuable to an advertiser seeking to target their ad budget at gay men willing to pay for a cruising tool, but it may make faulty assumptions about the user of a device. That could end up being embarrassing at best and plausibly even cause a person to lose their job in any number of states or countries where there is no legal protection from discrimination launched in the mere suspicion of a person's private life details. 

    That targeting could also destroy a relationship. And if you're traveling in some countries, it could potentially threaten you with detainment and even persecution at the border. All because your private behaviors were interpreted by an algorithm to imply a potential interest that others might notice. Targeted advertising can appear to reveal private things about you that may not even be accurate, yet with such confidence that it makes you look like you're hiding something.

    Perhaps this doesn’t reflect you or your habits, but it’s an example of what is going on behind the scenes that many people are unaware of.
    Interesting example too that might apply to some ad networks. It would not apply to Google's since they don't permit ad placement or assist marketing efforts based on sexual persuasion, religion, political leanings, medical conditions or a plethora of other categories that some other companies might (and probably do) offer access to. Google actually goes further than that and does not allow companies using their advertising services to collect those personal details either. 

    canukstorm
    said:
    vonbrick said:
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]
    The biggest beef is with Google's business model which is predicated on selling all the information it's collected on people to advertisers and in return inundating you with ads.  At its heart it's still predominantly an advertising company.
    Google does not sell personal information. Period.

    Google sells ad placement services that are made more valuable by being able to lump people with similar interests and/or demographics together into big de-personalized baskets of potential customers for a company's product/service. 

    You are far from the only person here who has misunderstood or been mislead about Google's ad services and how personal information that may be entrusted with them is protected and not sold. 
    edited May 7 forgot usernamepratikindiamuthuk_vanalingamCarnagejony0
  • Reply 15 of 21
    ivanhivanh Posts: 360member
    Google Home Hub (with Google Assistant) is multi-user and instantly recognise the user voice and switch to that user profile. E.g. a family of six can ask “hey google who am I?” And the Google Home Hub can instantly response “You name is whoeverasking”. 

    It can play your music library, YouTube Recommanded, Spotify, Google Photos of the user asking, instantly, at the volume you commanded, and control the lighting, turning on/off fans and aircond  as setup from Google Home app, all by voice.

    Google Home also sync setting changes from Apple HomeKit for wherever changes, but not vice versa. That’s Apple HomeKit’s problem.  
    forgot usernamegregoriusmpratikindia
  • Reply 16 of 21
    mulasienmulasien Posts: 3member
    vonbrick said:
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]
    The biggest beef is with Google's business model which is predicated on selling all the information it's collected on people to advertisers and in return inundating you with ads.  At its heart it's still predominantly an advertising company.
    You sure about that? https://safety.google/privacy/ads-and-data/

    "We do not sell your personal information to anyone"


    Been listening too much to the other tin foil types
    pratikindiamuthuk_vanalingamCarnagebigtdsgatorguy
  • Reply 17 of 21
    michelb76michelb76 Posts: 93member
    Google presented some pretty cool stuff yesterday. Realtime transcription looked useful, the camera improvements and te smaller ML models are really cool, but of-course building on the massive lead Google has in that area. I hope Apple has similar things in the pipeline.
    bigtdswatto_cobra
  • Reply 18 of 21
    kruegdudekruegdude Posts: 340member
    Folio said:
    If in real world, it works as well in demos then pretty impressive. The AR walking directions in Maps looks interesting too.
    What? And deprive you of the opportunity to use google to search for that answer? 
    watto_cobra
  • Reply 19 of 21
    dws-2dws-2 Posts: 238member
    vonbrick said:
    entropys said:
    vonbrick said:
    chasm said:
    gatorguy said:
    As far as I know Siri voice requests are still processed "in the cloud" rather than on-device. What Google is planning is something improved from that aren't they? 
    My understanding is that that's partially incorrect. Where it is possible, Siri processes requests or questions on-device, but also does do some (most? No idea) queries via internet relay to Apple servers, albeit encrypted and anonymized. I believe there's a white paper about this, but I'm not able to grab a link just at the moment.

    As for Google, they're just trying to speed things up a bit -- no change in their privacy invasion.  
    =====================================

    As a user of Google search (and Bing, too, actually)...I'm always interested to hear of how Google "invaded my privacy" today...?  What exactly did Google do to me after several web searches today to make my private life less private?
    It merely added today’s activities to the vast and extensive database records it already has collectively called “Vonbrick the innocent”
    Cool snark, bro.  And yet, I await a substantive answer as to what it is that Google does that invades my personal privacy.  [TICK TOCK TICK TOCK]

    I think the general idea is that Google is collecting all available information about you when you use its services. This information is then used to target ads to you. Some people think that invades their privacy; others do not.
  • Reply 20 of 21
    _honez_honez Posts: 5member
    mulasien said:
    You sure about that? https://safety.google/privacy/ads-and-data/

    "We do not sell your personal information to anyone"


    Been listening too much to the other tin foil types
    That may be so, but they sure as hell slurp as much highly personal information as they can for their own use, analyse it, infer from it and then sell *that* information.
    Google do sell information about you, inferred from your personal information, to anyone.
    The amount of passively gathered, inferred and not-specifically-disclosed information they extricate from PCs, phones, TVs, tablets, cars, smart-speakers, their software, their services, 3rd-party websites, etc, etc, is tantamount to a massive invasion of privacy by anyone's reckoning. If this passive and inferred data collection gathering was no longer passive and had to be granted by users, almost no-one would grant it or be comfortable with it.
    If this was a nation-state people would be apoplectic.
Sign In or Register to comment.