Google Assistant iOS app update coming this year, teases Duplex AI phone calls

AppleInsider

Google has previewed some of the changes it will be making to the Google Assistant app for iOS at the Google I/O keynote address, including food delivery and smart home-related requests, and an experimental feature where the digital assistant makes calls to businesses on the user's behalf.

The Google Assistant will become a more visual tool than in its current state, with responses to queries providing bigger pictures and more imagery to the user. An on-stage demonstration querying about a singer provided the usual basic information, along with a large photograph of Camila Cabello, the subject of the query.

More control will also be provided when using the assistant with smart home appliances. When asked to cool down the living room, Google Assistant displayed a dial, allowing the user to more directly control the home's thermostat, with similar controls presumably offered for other types of device.

Google is also making it possible for users to order food and drinks for pickup and delivery through the assistant. An in-app menu of items served at specific restaurants can be requested, with items able to be ordered without needing to go into a specific app for that store.

The Food pickup and delivery function will initially work with DoorDash, Domino's, 7-Eleven, Panera Bread, Starbucks, Applebees, Just Eat, and Dunkin' Donuts.

Google is rolling out the changes to Google Assistant to Android users first, later this summer, followed by iOS later this year.

Another Assistant feature teased on stage is Google Duplex, which allows Assistant to make calls on behalf of the user to businesses. A request to Assistant to make an appointment at a store, such as a restaurant or a hair salon, will prompt a call to the business over the phone, with no further interaction required from the user.

Google CEO Sundar Pichai told attendees approximately sixty percent of small businesses do not offer an online booking system, but do take appointments over the phone.

In the demonstration calls, Google Duplex used different voices, asking to place an appointment and offering the business relevant details when required. To make the virtual assistant seem more realistic, it included a number of filler sounds, such as "umm" and "ah," while speaking or before responding, with the cadence of voice also varying when appropriate.

The technology will also help update Google's business listings for holiday hours and to update Google maps automatically, potentially helping others searching to see if the same business is open.

While not a full feature, Google bills Duplex as an experiment, and will be rolled out in the coming weeks, though it is unclear if this means to all Google Assistant users or just those on Android.

KITA

AppleInsider said:

In the demonstration calls, Google Duplex used different voices, asking to place an appointment and offering the business relevant details when required. To make the virtual assistant seem more realistic, it included a number of filler sounds, such as "umm" and "ah," while speaking or before responding, with the cadence of voice also varying when appropriate.

That's really impressive.

randominternetperson

KITA said:

AppleInsider said:

In the demonstration calls, Google Duplex used different voices, asking to place an appointment and offering the business relevant details when required. To make the virtual assistant seem more realistic, it included a number of filler sounds, such as "umm" and "ah," while speaking or before responding, with the cadence of voice also varying when appropriate.

That's really impressive.

If that actually works that would be a "killer app" for me, making these devices start to be real "assistants."  If I could say "Hey Siri make a dentist appointment for some time in the next two or three weeks" and the phone has a conversation with a real person and together they find a reasonable time on my calendar, that would be great.  It's hard to believe that we're getting close to that, but perhaps we are.

(Actually, try saying that to your iPhone now.  The results aren't terrible.)

franklinjackcon

Oh this is what I've been dreaming of but didn't expect it would happen so soon. I get massive anxiety making phone calls because half the time they work out fine and half the time I hang up not knowing if I've made the right appointment or not (I live in a country where my grasp of the local language isn't perfect and have turned up to appoints on wrong days before or struggled with technical requests). A pity this'll probably take years before it rolls out into multiple languages. Or maybe not, better I keep trying. I recall Depp Mind was experimenting with creating voices from scratch and making them more realistic by adding in umms and ahhs. I guess this is one of the reasons for it. Nobody wants to talk to the standard Google Assistant but if there are dozens of voices, it'll go more smoothly.

eightzero

randominternetperson said:

KITA said:

AppleInsider said:

In the demonstration calls, Google Duplex used different voices, asking to place an appointment and offering the business relevant details when required. To make the virtual assistant seem more realistic, it included a number of filler sounds, such as "umm" and "ah," while speaking or before responding, with the cadence of voice also varying when appropriate.

That's really impressive.

If that actually works that would be a "killer app" for me, making these devices start to be real "assistants."  If I could say "Hey Siri make a dentist appointment for some time in the next two or three weeks" and the phone has a conversation with a real person and together they find a reasonable time on my calendar, that would be great.  It's hard to believe that we're getting close to that, but perhaps we are.

(Actually, try saying that to your iPhone now.  The results aren't terrible.)

You said it. That's an assistant. It is about time someone fought back against these phone-hell systems. "Press 1 for this" etc. I always just lean on zero and # to see if it kicks me out to a live person. An app that robo-calls on your behalf would be killer. 

I've called dentists that then wanted to send me emails and texts with documents they want me to fill out. I refuse. That's their job. I'll provide information, but I am not their typist. Have an app that says, "I've called you with information you will need. Press 1 to hear my contact information. Press 2 to be connected directly to me." When they press 2, I get a call back.

Nifty.

avon b7

franklinjackcon said:

Oh this is what I've been dreaming of but didn't expect it would happen so soon. I get massive anxiety making phone calls because half the time they work out fine and half the time I hang up not knowing if I've made the right appointment or not (I live in a country where my grasp of the local language isn't perfect and have turned up to appoints on wrong days before or struggled with technical requests). A pity this'll probably take years before it rolls out into multiple languages. Or maybe not, better I keep trying. I recall Depp Mind was experimenting with creating voices from scratch and making them more realistic by adding in umms and ahhs. I guess this is one of the reasons for it. Nobody wants to talk to the standard Google Assistant but if there are dozens of voices, it'll go more smoothly.

Just a quick note of support and understanding. There are few things that seem as unimportant but are actually totally nerve-racking as making a telephone call in a language you are not fluent in.

I can laugh at some of my failed attempts now (many of them worthy of a place in the script of Fawlty Towers) but they were definitely scary moments. I know how it feels.

Keep at it and try not to let mistakes get the better of you. Things in language never get worse and if you are living in the country of your non-native language, you will eventually be able to look back and laugh at your mistakes.

I spent more than three months trying to find my contact lens liquids in Barcelona but without 'condoms' in them!

Hopefully, if this Google idea works out you'll use it as a convenience and not a necessity.

franklinjackcon

avon b7 said:

franklinjackcon said:

Oh this is what I've been dreaming of but didn't expect it would happen so soon. I get massive anxiety making phone calls because half the time they work out fine and half the time I hang up not knowing if I've made the right appointment or not (I live in a country where my grasp of the local language isn't perfect and have turned up to appoints on wrong days before or struggled with technical requests). A pity this'll probably take years before it rolls out into multiple languages. Or maybe not, better I keep trying. I recall Depp Mind was experimenting with creating voices from scratch and making them more realistic by adding in umms and ahhs. I guess this is one of the reasons for it. Nobody wants to talk to the standard Google Assistant but if there are dozens of voices, it'll go more smoothly.

Just a quick note of support and understanding. There are few things that seem as unimportant but are actually totally nerve-racking as making a telephone call in a language you are not fluent in.

I can laugh at some of my failed attempts now (many of them worthy of a place in the script of Fawlty Towers) but they were definitely scary moments. I know how it feels.

Keep at it and try not to let mistakes get the better of you. Things in language never get worse and if you are living in the country of your non-native language, you will eventually be able to look back and laugh at your mistakes.

I spent more than three months trying to find my contact lens liquids in Barcelona but without 'condoms' in them!

Hopefully, if this Google idea works out you'll use it as a convenience and not a necessity.

Even if Google Assistant makes my appointments for me, I'll still need to contend with the daily hilarity that ensues every time I talk to my neighbour. The guy must think I'm insane based on some of the conversations we've had although fortunately he's more patient than Basil Fawlty.

slurpy

Technically impressive, but fucking creepy (and wrong, in my opinion) on so many levels. It's actively tricking the person on the other end into thinking they're talking to a real human being, and forcing them to follow niceties which are completely useless to a machine. Just feels dishonest and ethically messed up. At the least it should state "This is Google Assistant" at the start.

gatorguy

slurpy said:

Technically impressive, but fucking creepy (and wrong, in my opinion) on so many levels. It's actively tricking the person on the other end into thinking they're talking to a real human being, and forcing them to follow niceties which are completely useless to a machine. Just feels dishonest and ethically messed up. At the least it should state "This is Google Assistant" at the start.

Why? It seems like that would only serve to give the person on the other end an excuse to be less cooperative and helpful. The whole point of it is to be helpful to the smartphone owner. Anything that makes that task harder or less likely to succeed isn't beneficial.

mazda 3s

AI should have embedded the video!! The text description doesn't do it justice. It's flipping INCREDIBLE. Video below is primed right before the demo begins (1:55 Mark): 

SpamSandwich

randominternetperson said:

KITA said:

AppleInsider said:

In the demonstration calls, Google Duplex used different voices, asking to place an appointment and offering the business relevant details when required. To make the virtual assistant seem more realistic, it included a number of filler sounds, such as "umm" and "ah," while speaking or before responding, with the cadence of voice also varying when appropriate.

That's really impressive.

If that actually works that would be a "killer app" for me, making these devices start to be real "assistants."  If I could say "Hey Siri make a dentist appointment for some time in the next two or three weeks" and the phone has a conversation with a real person and together they find a reasonable time on my calendar, that would be great.  It's hard to believe that we're getting close to that, but perhaps we are.

(Actually, try saying that to your iPhone now.  The results aren't terrible.)

That Duplex demo is very, very impressive. Hats off to Google for making first what Apple should have. Absolutely amazing.

https://www.theverge.com/2018/5/8/17332070/google-assistant-makes-phone-call-demo-duplex-io-2018

randominternetperson

mazda 3s said:

AI should have embedded the video!! The text description doesn't do it justice. It's flipping INCREDIBLE. Video below is primed right before the demo begins (1:55 Mark): 

Thanks for the link. Very impressive indeed. As the dude says, this is something Google has been working on "for many years."

chasm

Let's see how it responds when the called entity opens the conversation with "For English, press 1. Our menu options have changed. For Luis, press 7. For Andrea, press 8, for Camelita, press 9. Otherwise, leave a message."

SpamSandwich

chasm said:

Let's see how it responds when the called entity opens the conversation with "For English, press 1. Our menu options have changed. For Luis, press 7. For Andrea, press 8, for Camelita, press 9. Otherwise, leave a message."

Funny, but you raise an interesting point. Companies could use Google’s AI to replace their own systems, so both the customer and the business would hand off the drudgery of negotiating details to these systems and then the AI would simply inform both parties after there is agreement or schedules sync. Of course, the real test comes when there is no agreeable conclusion to the negotiation. Does the AI go back to the user and inform them of the failure or does it explain what happened? What happens when a child answers the phone instead of an adult? There are a near infinite number of things that could conceivably go wrong.

evilution

I’m not a fan of Google at all but that tech demo was incredible. For someone like myself who has autism and a phobia of using phones, it would be a life changing addition to me.

ericthehalfbee

I don't find it impressive or useful, for two reasons.

First, I, like I’m betting most people, don’t micro-manage my schedule. I have appointments in my calendar, but only for major things booked in advance (like dentist/doctor or holidays/trips). My regular schedule is pretty much a routine and there’s no reason to put that into a calendar. If my buddies are going to watch the game Friday night I sure as hell don’t need to create a calendar event for it. Likewise if my kids weekly Karate class is cancelled I don’t need to record it.

So how is Duplex going to book an appointment for me without detailed knowledge of what I’m doing and when I’m doing it? Is it going to tell the person on the line “hold on while I ask my master if Thursday is OK”? I don’t see how this will work without intimate knowledge of my schedule

Second, how will Duplex know my tastes/preferences? If I’m booking a restaurant, does it know if I want a table in the dining room or one in the bar/lounge? If I tell it beforehand to book in the dining room, and nothing’s available, does it know if I’ll accept the lounge or if I’ll adjust my schedule to make sure I get the dining room?

Theres too many variables that I think will confuse Duplex if you’re booking anything but the most basic type of appointment.

gatorguy

ericthehalfbee said:

I don't find it impressive or useful, for two reasons.

First, I, like I’m betting most people, don’t micro-manage my schedule. I have appointments in my calendar, but only for major things booked in advance (like dentist/doctor or holidays/trips). My regular schedule is pretty much a routine and there’s no reason to put that into a calendar. If my buddies are going to watch the game Friday night I sure as hell don’t need to create a calendar event for it. Likewise if my kids weekly Karate class is cancelled I don’t need to record it.

So how is Duplex going to book an appointment for me without detailed knowledge of what I’m doing and when I’m doing it? Is it going to tell the person on the line “hold on while I ask my master if Thursday is OK”? I don’t see how this will work without intimate knowledge of my schedule

Second, how will Duplex know my tastes/preferences? If I’m booking a restaurant, does it know if I want a table in the dining room or one in the bar/lounge? If I tell it beforehand to book in the dining room, and nothing’s available, does it know if I’ll accept the lounge or if I’ll adjust my schedule to make sure I get the dining room?

Theres too many variables that I think will confuse Duplex if you’re booking anything but the most basic type of appointment.

You're only looking at it from the consumer side which is causing you to miss some of the potential. Expect this to be used more in enterprise in the beginning IMHO. 

ericthehalfbee

gatorguy said:

ericthehalfbee said:

I don't find it impressive or useful, for two reasons.

First, I, like I’m betting most people, don’t micro-manage my schedule. I have appointments in my calendar, but only for major things booked in advance (like dentist/doctor or holidays/trips). My regular schedule is pretty much a routine and there’s no reason to put that into a calendar. If my buddies are going to watch the game Friday night I sure as hell don’t need to create a calendar event for it. Likewise if my kids weekly Karate class is cancelled I don’t need to record it.

So how is Duplex going to book an appointment for me without detailed knowledge of what I’m doing and when I’m doing it? Is it going to tell the person on the line “hold on while I ask my master if Thursday is OK”? I don’t see how this will work without intimate knowledge of my schedule

Second, how will Duplex know my tastes/preferences? If I’m booking a restaurant, does it know if I want a table in the dining room or one in the bar/lounge? If I tell it beforehand to book in the dining room, and nothing’s available, does it know if I’ll accept the lounge or if I’ll adjust my schedule to make sure I get the dining room?

Theres too many variables that I think will confuse Duplex if you’re booking anything but the most basic type of appointment.

You're only looking at it from the consumer side which is causing you to miss some of the potential. Expect this to be used more in enterprise in the beginning IMHO. 

Then they should have demoed a business use case, not a consumer one. I think the reason they didn’t is because it’s nowhere near good enough yet.

gatorguy

ericthehalfbee said:

gatorguy said:

ericthehalfbee said:

I don't find it impressive or useful, for two reasons.

First, I, like I’m betting most people, don’t micro-manage my schedule. I have appointments in my calendar, but only for major things booked in advance (like dentist/doctor or holidays/trips). My regular schedule is pretty much a routine and there’s no reason to put that into a calendar. If my buddies are going to watch the game Friday night I sure as hell don’t need to create a calendar event for it. Likewise if my kids weekly Karate class is cancelled I don’t need to record it.

So how is Duplex going to book an appointment for me without detailed knowledge of what I’m doing and when I’m doing it? Is it going to tell the person on the line “hold on while I ask my master if Thursday is OK”? I don’t see how this will work without intimate knowledge of my schedule

Second, how will Duplex know my tastes/preferences? If I’m booking a restaurant, does it know if I want a table in the dining room or one in the bar/lounge? If I tell it beforehand to book in the dining room, and nothing’s available, does it know if I’ll accept the lounge or if I’ll adjust my schedule to make sure I get the dining room?

Theres too many variables that I think will confuse Duplex if you’re booking anything but the most basic type of appointment.

You're only looking at it from the consumer side which is causing you to miss some of the potential. Expect this to be used more in enterprise in the beginning IMHO. 

Then they should have demoed a business use case, not a consumer one. I think the reason they didn’t is because it’s nowhere near good enough yet.

Ummm...
because it was impressive nonetheless and gets free press?
..and in truth it is.... Very much so, (I don't believe you when you say you weren't) but even Google was upfront about certain scenarios being far less, shall we say, "successful". It's not ready yet for primetime.

eightzero

I'm now seeing some of the demos. Pretty interesting. Given these are in fact just demos, it is worth considering the potential. For perhaps the first time, I'm actually excited and interested in a google product. 

A few things pop to mind. As a basic premise, there needs to be acknowledgement that this is indeed a robotic system under the control of, and for interaction with humans. Like self driving cars, humans will intentionally screw with them. Some humans, being called by a robot, will either simply hang up (I do that when I reach someone else's robot) or will be tempted to screw with them. Give a fake name, don't book the appointment, then blame the callers robot. Think "we don't serve their kind" in the cantina scenein Star Wars ("IV"). Pretty sure the consumer will need a receipt directly from the other party, not their robotic confirmation. And oh yes, my robot will first inform the other party they are on a recorded line for "quality assurance purposes." 

These systems will also need to navigate a call to someone else's robot, aka the phone-hell system systems of press 1 to hear how technology doesn't improve anyone's lives etc. Oddly enough, robot to robot stuff should be the easiest part. Take humans out of the loop, and this get dramatically more predictable. Betcha google knows this in spades.

Many call centers need info to get started. Will your robot be authorized to speak your SSN or other ID (credit card number?) into a call? Because..well...you know...

This is the first time I've seen something on this AI stuff that gets my intention. Kinda interesting, and I am now (edit) curious what Apple will say about it at WWDC.

eightzero

ericthehalfbee said:

I don't find it impressive or useful, for two reasons.

First, I, like I’m betting most people, don’t micro-manage my schedule. I have appointments in my calendar, but only for major things booked in advance (like dentist/doctor or holidays/trips). My regular schedule is pretty much a routine and there’s no reason to put that into a calendar. If my buddies are going to watch the game Friday night I sure as hell don’t need to create a calendar event for it. Likewise if my kids weekly Karate class is cancelled I don’t need to record it.

So how is Duplex going to book an appointment for me without detailed knowledge of what I’m doing and when I’m doing it? Is it going to tell the person on the line “hold on while I ask my master if Thursday is OK”? I don’t see how this will work without intimate knowledge of my schedule

Second, how will Duplex know my tastes/preferences? If I’m booking a restaurant, does it know if I want a table in the dining room or one in the bar/lounge? If I tell it beforehand to book in the dining room, and nothing’s available, does it know if I’ll accept the lounge or if I’ll adjust my schedule to make sure I get the dining room?

Theres too many variables that I think will confuse Duplex if you’re booking anything but the most basic type of appointment.

You comments are well taken. Like any tech, the selection and use is quite personal. The short answer to your question is...your robot will need to learn from you. And that can be time consuming in and of itself, and deeply frustrating. "Predictive Siri" is particularly annoying. It starts suggesting things to me, and thus is like the annoying child that demands to be taught right now, interfering with my current task. Unlike a child, to whom you've presumably made a commitment, Siri is specifically and exclusively my servant, and my servants don't interfere. 

Don't need or want this? OK. I don't have any children either.