WWDC 2018: Apple, Siri and the future of mobile voice automation in iOS 12
Apple's Worldwide Developer Conference is just over three months away, but we already have some idea of what the company will likely be showing off in iOS 12 to drive the next generation of our mobile devices. Here's a look at Apple's Siri voice assistant and automation.
A recent piece by The Information described the entire history of Siri at Apple as being non-stop dramatic infighting over a series of foundational mistakes and fragile code all falling apart while new acquisitions were thrown on top.
The profile sounded a lot like the same site's earlier depictions of infighting between Android frenemies Google and Samsung related to everything from Tizen smartwatches to enterprise goals, or the bumbling changes in strategic direction within Google and its infighting hardware fiefdoms of Android, Chrome and acquisitions including Motorola Mobility and Nest.
Like its previous juicy tell-all depictions of drama in Silicon Valley, The Information largely cited its sources among disgruntled former employees whose stories were contradicted by others. But the main difference between the three portrayals is in final outcomes.
Google and Samsung actually failed to sell smartwatches, whether Android Wear or Tizen, and fumbled their enterprise pitch, while in parallel Google's Android, Chrome, Motorola and Next hardware have also all been huge expensive catastrophes without sales of any real commercial significance. The last several years of Apple's Siri look like a genius home run next to all of that.
Apple doesn't sell Siri as a product at all, so the only way to brand Siri as a "failure" would be to establish that rivals were somehow undercutting Apple's business or future potential with their own voice services. Yet even The Information admitted that Apple remains by far the largest supplier of voice requests, and the only one to support more than 20 languages (Alexa supports three). Siri is at worst assisting in profitable sales of Apple hardware.
If Android gets a leadership success trophy just for voluntarily participating in the volume sales of low-end phones that don't make money despite its variety of serious architectural problems, performance issues and security flaws, surely we can throw Siri a bone for riding along as a supporting actor in Apple's massive, global cash machine, even if it can't tell you what's in a drink on demand faster than you can just look it up on your iPhone.
Hey Siri, where's Alexa?
Nobody is passing over Apple Watch (the most popular smartwatch by a vast margin) to get Alexa-connected smartwatches. Buyers are not shrugging off CarPlay--which is so popular it has forced every major carmaker to offer support, even those who once actively resisted it such as Toyota--to instead get car-mounted Alexa hardware. HomePod actually outsold comparably-priced "smart speakers" and much cheaper Amazon Echo devices in its debut, and even Apple TV has no competitors asking a similar price for hardware just because they're powered by Alexa.
For being such a celebrated business, Alexa doesn't seem to exercise much commercial muscle.
The Information didn't dwell on the value of these skills. Yet if you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff. For example, TomsGuide listed a series of 50 Skills that included lots of things Siri already does, such as music playback; dictating messages; asking about stocks prices; asking for news reports from NPR, CNN or the BBC; controlling home lighting automation; listening to podcasts; asking how to spell a word; or requesting a car from Uber or Lyft.
Other "Skills" were odd novelties, such as playing cat noises or thunderstorm sound effects, asking for a word definition of the day or an inspiring quote. Others Skills offered voice assistance for things Siri can look up on your phone, such as how to make a drink or get first aid advice. Other Skills were audio-only versions of apps. The most frequently cited among "top Skills" is a banking app for Capital One that accesses your bank balance and can make a payment. To use it, you have to speak out a security PIN code. That just seems like a bad idea. Alexa integration sometimes uses the same type of "security" to unlock your doors.
Other Skills involved devices, such as telling your (high-end model) Roomba to begin vacuuming or to adjust the temperature on a digital Sous Vide. You can also start your car remotely (if you have one of a few new vehicles) or control future home appliances from Whirlpool. It's almost as hard to see the valuable utility here as it is to discern any proprietary advantage Amazon Skills can wield in the long term.
Hey Siri: Workflow makes complex tasks into a triggerable event
Workflow is the graphical (rather than voice) version of what a lot of people seem to imagine as the ideal goal of Siri: it lets you define a set of actions and then trigger them with a touch, from the Home page, from within apps (on the Share Sheet), from the Today widgets page or even from your Apple Watch. Given how turns a trigger into complex action, a Workflow Intent for Siri could enable third parties to craft any sort of skill for users to launch with their voice, or graphically from any iOS device.
The insurmountable Skills advantage of Alexa largely goes away with a way for iOS app developers to add Siri automation via Workflow. The pizza order that Alexa lets you repeat would now be something that any app developer could expose as a voice trigger, much the same way that HomeKit allows devices to respond to a set of controls via Siri, or from direct control in apps or widgets.
Note that Alexa doesn't have to go away for Siri to work well or improve. Users might choose to have both, using an Alexa mic to order products while using HomePod to fill their living room with great sound and provide access to their personal information. However, the narrative that Apple is headed toward oblivion because Amazon has sold tens of millions of Alexa mics over the past few years is really not true by any stretch.
There was no mention of the fact that those Viv employees (Dag Kittlaus, Adam Cheyer and Chris Brigham) not only launched a project to compete against Siri (which they publicly demonstrated in early 2016) but also sold themselves to Samsung toward the end of that year to work on what was intended to become Bixby.
Viv Labs was in the news as the Siri-killer until it failed to kill Siri | Source: The Washington Post
The Viv team promoted their product as enabling more conversational, complex voice interactions than Apple's Siri could. But they couldn't ultimately get it to work. According to the Wall Street Journal, Samsung was forced to revert to the existing code of its languishing internal S Voice project.
Last year Samsung had to delay its Bixby voice service from the very flagship that it was supposed to differentiate. And across the next year reviews continued to describe it as "unforgivable," or "the kindest way I can describe it right now is utterly unimpressive."
Rather than shipping Viv's "better than Siri" on an imagined 400 million devices, Bixby ended up as an anti-feature dogging Samsung's flagship Galaxy S8 and doing no favors for the latest S9 this year. "Bixby is, by far, the most frustrating part of the S9," wrote Brian Heater for TechCrunch last week.
Isn't the massive failure of Viv to deliver a "better Siri" across the last three years--despite all the market power afforded by the world's largest shipper of smartphones--more important to understanding the voice assistant market than an anecdote about Apple not wanting Vivi to poach their employees on its own campus?
Selective reporting that portrays Alexa's largely frivolous Skills as mountains of pure gold while brushing off the fact that selling voice appliance devices to the world would require support for more than three languages and at least some presence in both of the world's largest two markets isn't really good information.
"My own home is littered with two Echo 'First Gen' tubes, two Echo Dots, a Google Home and a HomePod--and they're used almost entirely for weather (all roughly equal), timers (advantage HomePod, because it's audible a room away), spelling (advantage Echo dot, because it's cheap and can be placed beside a computer) and music (massive advantage to HomePod, which gives the only acceptable music stream.
"We have an iPhone 7, iPhone X and Samsung s8+, upon which the digital assistants are used to make phone calls, deliver weather forecasts, set timers, set alarms and give driving directions. I'd rank Siri and Google Assistant about equal at these tasks - and Siri is actually a lot more useful to me because she's on Apple Watch, which I use far more often than any of the speaking tubes. Google Maps are probably more reliable in tests than Apple Maps, but I find the latter good enough in Car Play, which I find myself using far more often than Android Auto.
"You'll note I haven't once mentioned IoT. We do have a home theater and thermostats that are Alexa-enabled. I could enable the skills and memorize the commands, buteh. The Android and iOS Apps are simpler, and a Control4 remote appeared spontaneously on my Apple Watch. Both functions are simpler to control WITHOUT voice."
HomePod is "audible a room away"
On the other hand, Apple has been enhancing how different Siri devices work together on the same network, an issue that neither Amazon nor Google really have. This is still a work in progress, but HomePod already does a good job of silencing your iPhone when both are listening for "Hey Siri."
What could be improved is Continuity handoff between devices, allowing HomePod to delegate questions to your iPhone or Apple Watch when it can't answer them, potentially even working together to deliver a UI on your phone or watch while listening to you from HomePod.
Similar HomePod integration with Apple TV, enabling you to ask it to play a specific show, also seems likely, enabling third-party tvOS developers to build new types of apps and games that can listen for feedback and commands using HomePod's fancy array of microphones.
HomePod will also likely expand its ability to support new types of requests (Hey Siri, where's my iPhone?), and expand what apps and data it can access under a Family Sharing account. You might also expect Apple to expand upon Type to Siri as a way to discreetly invoke Siri-type searches without using your voice at all.
What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity? Add your comments below.
An inflated tale of constant political battles among incompetents
According to an emerging media narrative, Apple fumbled its lead in voice assistants with Siri, and has been relegated into a distant third behind the new leaders in talking to cloud-connected microphones: Amazon's Alexa-powered WiFi mics and its ecosystem of connected devices and a platform of third-party Alexa Skills; and Google's me-too efforts tied to its Assistant running on Android, iOS and some Google-branded appliances.A recent piece by The Information described the entire history of Siri at Apple as being non-stop dramatic infighting over a series of foundational mistakes and fragile code all falling apart while new acquisitions were thrown on top.
The profile sounded a lot like the same site's earlier depictions of infighting between Android frenemies Google and Samsung related to everything from Tizen smartwatches to enterprise goals, or the bumbling changes in strategic direction within Google and its infighting hardware fiefdoms of Android, Chrome and acquisitions including Motorola Mobility and Nest.
Like its previous juicy tell-all depictions of drama in Silicon Valley, The Information largely cited its sources among disgruntled former employees whose stories were contradicted by others. But the main difference between the three portrayals is in final outcomes.
Google and Samsung actually failed to sell smartwatches, whether Android Wear or Tizen, and fumbled their enterprise pitch, while in parallel Google's Android, Chrome, Motorola and Next hardware have also all been huge expensive catastrophes without sales of any real commercial significance. The last several years of Apple's Siri look like a genius home run next to all of that.
Siri is failing all the way to the bank
People love to hate on Siri, but the worst thing Siri ever did was sell lots of iPhone 4s models, draw attention to other Apple introductions (from Apple Watch to CarPlay to Apple TV 4 to HomePod) and, of course, fail to understand a lot of questions from frustrated users. Ask your HomePod how to make a Negroni and it doesn't help at all. But Apple doesn't sell Siri as a voice first platform.Apple doesn't sell Siri as a voice first platform
Apple doesn't sell Siri as a product at all, so the only way to brand Siri as a "failure" would be to establish that rivals were somehow undercutting Apple's business or future potential with their own voice services. Yet even The Information admitted that Apple remains by far the largest supplier of voice requests, and the only one to support more than 20 languages (Alexa supports three). Siri is at worst assisting in profitable sales of Apple hardware.
If Android gets a leadership success trophy just for voluntarily participating in the volume sales of low-end phones that don't make money despite its variety of serious architectural problems, performance issues and security flaws, surely we can throw Siri a bone for riding along as a supporting actor in Apple's massive, global cash machine, even if it can't tell you what's in a drink on demand faster than you can just look it up on your iPhone.
Hey Siri, where's Alexa?
Nobody is passing over Apple Watch (the most popular smartwatch by a vast margin) to get Alexa-connected smartwatches. Buyers are not shrugging off CarPlay--which is so popular it has forced every major carmaker to offer support, even those who once actively resisted it such as Toyota--to instead get car-mounted Alexa hardware. HomePod actually outsold comparably-priced "smart speakers" and much cheaper Amazon Echo devices in its debut, and even Apple TV has no competitors asking a similar price for hardware just because they're powered by Alexa.
For being such a celebrated business, Alexa doesn't seem to exercise much commercial muscle.
Alexa and the 25,000 Skills
The real power of Amazon's Alexa is supposedly the third-party support for its voice action extensions, called "Skills." These allow app developers to expand its voice capabilities to support controlling devices and tasks such as ordering a favorite pizza from Dominos.If you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff
The Information didn't dwell on the value of these skills. Yet if you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff. For example, TomsGuide listed a series of 50 Skills that included lots of things Siri already does, such as music playback; dictating messages; asking about stocks prices; asking for news reports from NPR, CNN or the BBC; controlling home lighting automation; listening to podcasts; asking how to spell a word; or requesting a car from Uber or Lyft.
Other "Skills" were odd novelties, such as playing cat noises or thunderstorm sound effects, asking for a word definition of the day or an inspiring quote. Others Skills offered voice assistance for things Siri can look up on your phone, such as how to make a drink or get first aid advice. Other Skills were audio-only versions of apps. The most frequently cited among "top Skills" is a banking app for Capital One that accesses your bank balance and can make a payment. To use it, you have to speak out a security PIN code. That just seems like a bad idea. Alexa integration sometimes uses the same type of "security" to unlock your doors.
Other Skills involved devices, such as telling your (high-end model) Roomba to begin vacuuming or to adjust the temperature on a digital Sous Vide. You can also start your car remotely (if you have one of a few new vehicles) or control future home appliances from Whirlpool. It's almost as hard to see the valuable utility here as it is to discern any proprietary advantage Amazon Skills can wield in the long term.
Apple, Apps, Siri and Automation at WWDC 18
Virtually all of these Skills are tasks you can do from a smartphone app, just triggered by your voice via a speaker instead. How will Apple ever catch up? The answer may lie with Workflow, an app the company acquired nearly a year ago.Hey Siri: Workflow makes complex tasks into a triggerable event
Workflow is the graphical (rather than voice) version of what a lot of people seem to imagine as the ideal goal of Siri: it lets you define a set of actions and then trigger them with a touch, from the Home page, from within apps (on the Share Sheet), from the Today widgets page or even from your Apple Watch. Given how turns a trigger into complex action, a Workflow Intent for Siri could enable third parties to craft any sort of skill for users to launch with their voice, or graphically from any iOS device.
The insurmountable Skills advantage of Alexa largely goes away with a way for iOS app developers to add Siri automation via Workflow. The pizza order that Alexa lets you repeat would now be something that any app developer could expose as a voice trigger, much the same way that HomeKit allows devices to respond to a set of controls via Siri, or from direct control in apps or widgets.
Note that Alexa doesn't have to go away for Siri to work well or improve. Users might choose to have both, using an Alexa mic to order products while using HomePod to fill their living room with great sound and provide access to their personal information. However, the narrative that Apple is headed toward oblivion because Amazon has sold tens of millions of Alexa mics over the past few years is really not true by any stretch.
A surprising deficit of the Information
Reinforcing the huge gap in logic that accompanies much of the recent reporting related to Siri, The Information appeared to leave out material facts in order to contrive a tale of "everyone else in the voice universe" muscling into or otherwise disrupting Siri. The piece noted that Apple banned its former Siri employees (who left to launch Viv Lab) from playing basketball on its campus among concerns they'd recruit away the Siri talent Apple supposedly doesn't have.There was no mention of the fact that those Viv employees (Dag Kittlaus, Adam Cheyer and Chris Brigham) not only launched a project to compete against Siri (which they publicly demonstrated in early 2016) but also sold themselves to Samsung toward the end of that year to work on what was intended to become Bixby.
Viv Labs was in the news as the Siri-killer until it failed to kill Siri | Source: The Washington Post
The Viv team promoted their product as enabling more conversational, complex voice interactions than Apple's Siri could. But they couldn't ultimately get it to work. According to the Wall Street Journal, Samsung was forced to revert to the existing code of its languishing internal S Voice project.
Last year Samsung had to delay its Bixby voice service from the very flagship that it was supposed to differentiate. And across the next year reviews continued to describe it as "unforgivable," or "the kindest way I can describe it right now is utterly unimpressive."
Rather than shipping Viv's "better than Siri" on an imagined 400 million devices, Bixby ended up as an anti-feature dogging Samsung's flagship Galaxy S8 and doing no favors for the latest S9 this year. "Bixby is, by far, the most frustrating part of the S9," wrote Brian Heater for TechCrunch last week.
Isn't the massive failure of Viv to deliver a "better Siri" across the last three years--despite all the market power afforded by the world's largest shipper of smartphones--more important to understanding the voice assistant market than an anecdote about Apple not wanting Vivi to poach their employees on its own campus?
Selective reporting that portrays Alexa's largely frivolous Skills as mountains of pure gold while brushing off the fact that selling voice appliance devices to the world would require support for more than three languages and at least some presence in both of the world's largest two markets isn't really good information.
Real people on the utility of voice
Responding to dramatic portrayal of Siri woes at Apple, one reader of The Information wrote, "I wish Aaron and Kevin had also looked outside the Bay Bubble to see what motivates buyers of smartphones and 'Smart Speakers' - and find out what they really want out of their Digital Assistants."My own home is littered with two Echo 'First Gen' tubes, two Echo Dots, a Google Home and a HomePod--and they're used almost entirely for weather (all roughly equal), timers (advantage HomePod, because it's audible a room away), spelling (advantage Echo dot, because it's cheap and can be placed beside a computer) and music (massive advantage to HomePod, which gives the only acceptable music stream.
"We have an iPhone 7, iPhone X and Samsung s8+, upon which the digital assistants are used to make phone calls, deliver weather forecasts, set timers, set alarms and give driving directions. I'd rank Siri and Google Assistant about equal at these tasks - and Siri is actually a lot more useful to me because she's on Apple Watch, which I use far more often than any of the speaking tubes. Google Maps are probably more reliable in tests than Apple Maps, but I find the latter good enough in Car Play, which I find myself using far more often than Android Auto.
"You'll note I haven't once mentioned IoT. We do have a home theater and thermostats that are Alexa-enabled. I could enable the skills and memorize the commands, buteh. The Android and iOS Apps are simpler, and a Control4 remote appeared spontaneously on my Apple Watch. Both functions are simpler to control WITHOUT voice."
HomePod is "audible a room away"
Siri expectations for WWDC 18
So while you can expect Apple to expand Siri with some new Intents and to potentially add third-party task automation support in Workflow, don't expect the company to devote major competitive efforts toward replicating the extremely low-value Amazon Skills ecosystem that offers users little more than a more clumsy way to use apps without looking at them (the way Google has blindly, desperately tried to copy Alexa).On the other hand, Apple has been enhancing how different Siri devices work together on the same network, an issue that neither Amazon nor Google really have. This is still a work in progress, but HomePod already does a good job of silencing your iPhone when both are listening for "Hey Siri."
What could be improved is Continuity handoff between devices, allowing HomePod to delegate questions to your iPhone or Apple Watch when it can't answer them, potentially even working together to deliver a UI on your phone or watch while listening to you from HomePod.
Similar HomePod integration with Apple TV, enabling you to ask it to play a specific show, also seems likely, enabling third-party tvOS developers to build new types of apps and games that can listen for feedback and commands using HomePod's fancy array of microphones.
HomePod will also likely expand its ability to support new types of requests (Hey Siri, where's my iPhone?), and expand what apps and data it can access under a Family Sharing account. You might also expect Apple to expand upon Type to Siri as a way to discreetly invoke Siri-type searches without using your voice at all.
What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity? Add your comments below.
Comments
https://techpinions.com/apple-siri-and-dealing-with-failure/52484
"What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity?" => Given that the rumors point to iOS 12 / macOS 14 being a "Snow Leopard" type of release, I'm not expecting much this year.
Given that Siri languished for many years under Eddy Cue's leadership (Federighi now being in charge of Siri) & how important Siri is to Apple's future, I am surprised he still around at Apple.
No matter how much BS the media spews, the reality is little to no people care about voice .
It was well into the Post Jobs era that iOS 7 and the A7 shipped, that iCloud development was fixed and dramatically expanded, that Maps was made usable in most countries (enough to relegate Google Maps into a minor position on iOS) and where iPad crushed all tablets and no competition is left outside of Android placeholders shipping PC-like generic boxes with razor-thin margins.
Also, Samsung's leadership is fresh out of jail and Google has experienced a wave of new managers. So the whole Jobs thing really needs to be put to bed.
Apple views siri as complementary to the iPhone like like the watch, they have very little to gaine by dedicating the gargantuan amounts of resources it would take to make sirri get to that Samantha level, I think I read somewhere that Bezos' got over 100K people working on Alexa don't know how true that is but unlike Apple Alexa could realistically makes AMZ serious $$$ because Alexa is complementary to AMZ's business model not the products they sell, so maybe you could work on something to show us how Apple getting "Sirious" about Siri will meaning fully contribute to the bottom line since you love to bring up in almost every artticle, & why do you do that by the way, how has talking about how much money Apple's making become the thing Apple fans care about now?
Only AAPL fans care about that and those guys car about Apple only as far as the next financial statement's good for them no?
Siri works aross all of those Apple devices fine for me. I also have a CarPlay audio system so I can also use Siri in the car to send and read messages, play music, ask for directons etc.
Siri could absolutely be better, but I don’t see Alexa or Assistant doing the things that Siri struggles with all that well either.
One comment from the article that rings true for me is that Siri just works across all my devices. I can use it from my watch if I’m in the backyard and need to turn the outside lights on and so I’m fascinated to see where Apple goes with Siri, HomeKit, CarPlay and Workflow and how it all comes together across their products.
As the article mentions, Apple definitely leads the way in terms of voice assistant integration across the range of its ecosystem, something Google and Amazon haven’t been able to do.
Alexa is claimed to be some sort of champion, but as this article details thoroughly I’m not seeing the value add compared to apps. The basics are well served by either platform — home automation, reminders, alarms, texts, calendar, etc.
When I do use Siri it’s either on my Watch or spoken to whatever idevice is sitting closest to me.
I'm sure what I'm asking for is difficult - but that's the Apple Engineer's job.
Expanding Siri to do more useful things should be the job of app developers - they just need a flexible API to do it with.
The fact I can't add something to my shopping list whilst handsfree/driving is very annoying.
It's a trivial example - but there are 100 minor things I do with my phone every day, and 98 of them still require me to touch it to do them:
- shopping list
- check bank balance
- transfer funds
- find where my bus is (using the TripView app)
- find the next connecting bus (TripView)
- is there an accident ahead (LiveTraffic app)
- read the latest Facebook posts from close friends
- read a book (Kindle) or newspaper article (Kindle)
- read an InMail (linked in)
- open a camera (Nest)
- make a doctors appointment (HotDoc)
- note the time I put the baby to bed (Baby Tracker)
- note the time the baby woke up (Baby Tracker)
- note the quantity and time of medication (Baby Tracker)
etc. etc.
What can I do?
- set a timer
- text/SMS my wife
No way can Apple add all those features - but they can work on better API's so that developers can create the apps with those features.
I think the best example on how inflexible the Siri API is - is the HotDoc app and doctors appointments. Siri can make a calendar appointment, but it's not flexible enough to handle a doctors appointment, because it needs some ability to select the doctor and restrict the appointments available based on available appointment slots and appointment durations (long or short).
It was a good 'show us you are at least thinking about it' API - but it needs a follow up to 'show us you are serious about it' API.