WWDC 2018: Apple, Siri and the future of mobile voice automation in iOS 12

Posted:
in iOS edited March 2018
Apple's Worldwide Developer Conference is just over three months away, but we already have some idea of what the company will likely be showing off in iOS 12 to drive the next generation of our mobile devices. Here's a look at Apple's Siri voice assistant and automation.

An inflated tale of constant political battles among incompetents

According to an emerging media narrative, Apple fumbled its lead in voice assistants with Siri, and has been relegated into a distant third behind the new leaders in talking to cloud-connected microphones: Amazon's Alexa-powered WiFi mics and its ecosystem of connected devices and a platform of third-party Alexa Skills; and Google's me-too efforts tied to its Assistant running on Android, iOS and some Google-branded appliances.

A recent piece by The Information described the entire history of Siri at Apple as being non-stop dramatic infighting over a series of foundational mistakes and fragile code all falling apart while new acquisitions were thrown on top.

The profile sounded a lot like the same site's earlier depictions of infighting between Android frenemies Google and Samsung related to everything from Tizen smartwatches to enterprise goals, or the bumbling changes in strategic direction within Google and its infighting hardware fiefdoms of Android, Chrome and acquisitions including Motorola Mobility and Nest.

Like its previous juicy tell-all depictions of drama in Silicon Valley, The Information largely cited its sources among disgruntled former employees whose stories were contradicted by others. But the main difference between the three portrayals is in final outcomes.

Google and Samsung actually failed to sell smartwatches, whether Android Wear or Tizen, and fumbled their enterprise pitch, while in parallel Google's Android, Chrome, Motorola and Next hardware have also all been huge expensive catastrophes without sales of any real commercial significance. The last several years of Apple's Siri look like a genius home run next to all of that.

Siri is failing all the way to the bank

People love to hate on Siri, but the worst thing Siri ever did was sell lots of iPhone 4s models, draw attention to other Apple introductions (from Apple Watch to CarPlay to Apple TV 4 to HomePod) and, of course, fail to understand a lot of questions from frustrated users. Ask your HomePod how to make a Negroni and it doesn't help at all. But Apple doesn't sell Siri as a voice first platform.Apple doesn't sell Siri as a voice first platform

Apple doesn't sell Siri as a product at all, so the only way to brand Siri as a "failure" would be to establish that rivals were somehow undercutting Apple's business or future potential with their own voice services. Yet even The Information admitted that Apple remains by far the largest supplier of voice requests, and the only one to support more than 20 languages (Alexa supports three). Siri is at worst assisting in profitable sales of Apple hardware.

If Android gets a leadership success trophy just for voluntarily participating in the volume sales of low-end phones that don't make money despite its variety of serious architectural problems, performance issues and security flaws, surely we can throw Siri a bone for riding along as a supporting actor in Apple's massive, global cash machine, even if it can't tell you what's in a drink on demand faster than you can just look it up on your iPhone.


Hey Siri, where's Alexa?


Nobody is passing over Apple Watch (the most popular smartwatch by a vast margin) to get Alexa-connected smartwatches. Buyers are not shrugging off CarPlay--which is so popular it has forced every major carmaker to offer support, even those who once actively resisted it such as Toyota--to instead get car-mounted Alexa hardware. HomePod actually outsold comparably-priced "smart speakers" and much cheaper Amazon Echo devices in its debut, and even Apple TV has no competitors asking a similar price for hardware just because they're powered by Alexa.

For being such a celebrated business, Alexa doesn't seem to exercise much commercial muscle.

Alexa and the 25,000 Skills

The real power of Amazon's Alexa is supposedly the third-party support for its voice action extensions, called "Skills." These allow app developers to expand its voice capabilities to support controlling devices and tasks such as ordering a favorite pizza from Dominos. If you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff

The Information didn't dwell on the value of these skills. Yet if you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff. For example, TomsGuide listed a series of 50 Skills that included lots of things Siri already does, such as music playback; dictating messages; asking about stocks prices; asking for news reports from NPR, CNN or the BBC; controlling home lighting automation; listening to podcasts; asking how to spell a word; or requesting a car from Uber or Lyft.

Other "Skills" were odd novelties, such as playing cat noises or thunderstorm sound effects, asking for a word definition of the day or an inspiring quote. Others Skills offered voice assistance for things Siri can look up on your phone, such as how to make a drink or get first aid advice. Other Skills were audio-only versions of apps. The most frequently cited among "top Skills" is a banking app for Capital One that accesses your bank balance and can make a payment. To use it, you have to speak out a security PIN code. That just seems like a bad idea. Alexa integration sometimes uses the same type of "security" to unlock your doors.

Other Skills involved devices, such as telling your (high-end model) Roomba to begin vacuuming or to adjust the temperature on a digital Sous Vide. You can also start your car remotely (if you have one of a few new vehicles) or control future home appliances from Whirlpool. It's almost as hard to see the valuable utility here as it is to discern any proprietary advantage Amazon Skills can wield in the long term.

Apple, Apps, Siri and Automation at WWDC 18

Virtually all of these Skills are tasks you can do from a smartphone app, just triggered by your voice via a speaker instead. How will Apple ever catch up? The answer may lie with Workflow, an app the company acquired nearly a year ago.


Hey Siri: Workflow makes complex tasks into a triggerable event


Workflow is the graphical (rather than voice) version of what a lot of people seem to imagine as the ideal goal of Siri: it lets you define a set of actions and then trigger them with a touch, from the Home page, from within apps (on the Share Sheet), from the Today widgets page or even from your Apple Watch. Given how turns a trigger into complex action, a Workflow Intent for Siri could enable third parties to craft any sort of skill for users to launch with their voice, or graphically from any iOS device.

The insurmountable Skills advantage of Alexa largely goes away with a way for iOS app developers to add Siri automation via Workflow. The pizza order that Alexa lets you repeat would now be something that any app developer could expose as a voice trigger, much the same way that HomeKit allows devices to respond to a set of controls via Siri, or from direct control in apps or widgets.

Note that Alexa doesn't have to go away for Siri to work well or improve. Users might choose to have both, using an Alexa mic to order products while using HomePod to fill their living room with great sound and provide access to their personal information. However, the narrative that Apple is headed toward oblivion because Amazon has sold tens of millions of Alexa mics over the past few years is really not true by any stretch.

A surprising deficit of the Information

Reinforcing the huge gap in logic that accompanies much of the recent reporting related to Siri, The Information appeared to leave out material facts in order to contrive a tale of "everyone else in the voice universe" muscling into or otherwise disrupting Siri. The piece noted that Apple banned its former Siri employees (who left to launch Viv Lab) from playing basketball on its campus among concerns they'd recruit away the Siri talent Apple supposedly doesn't have.

There was no mention of the fact that those Viv employees (Dag Kittlaus, Adam Cheyer and Chris Brigham) not only launched a project to compete against Siri (which they publicly demonstrated in early 2016) but also sold themselves to Samsung toward the end of that year to work on what was intended to become Bixby.


Viv Labs was in the news as the Siri-killer until it failed to kill Siri | Source: The Washington Post


The Viv team promoted their product as enabling more conversational, complex voice interactions than Apple's Siri could. But they couldn't ultimately get it to work. According to the Wall Street Journal, Samsung was forced to revert to the existing code of its languishing internal S Voice project.

Last year Samsung had to delay its Bixby voice service from the very flagship that it was supposed to differentiate. And across the next year reviews continued to describe it as "unforgivable," or "the kindest way I can describe it right now is utterly unimpressive."

Rather than shipping Viv's "better than Siri" on an imagined 400 million devices, Bixby ended up as an anti-feature dogging Samsung's flagship Galaxy S8 and doing no favors for the latest S9 this year. "Bixby is, by far, the most frustrating part of the S9," wrote Brian Heater for TechCrunch last week.

Isn't the massive failure of Viv to deliver a "better Siri" across the last three years--despite all the market power afforded by the world's largest shipper of smartphones--more important to understanding the voice assistant market than an anecdote about Apple not wanting Vivi to poach their employees on its own campus?

Selective reporting that portrays Alexa's largely frivolous Skills as mountains of pure gold while brushing off the fact that selling voice appliance devices to the world would require support for more than three languages and at least some presence in both of the world's largest two markets isn't really good information.

Real people on the utility of voice

Responding to dramatic portrayal of Siri woes at Apple, one reader of The Information wrote, "I wish Aaron and Kevin had also looked outside the Bay Bubble to see what motivates buyers of smartphones and 'Smart Speakers' - and find out what they really want out of their Digital Assistants.

"My own home is littered with two Echo 'First Gen' tubes, two Echo Dots, a Google Home and a HomePod--and they're used almost entirely for weather (all roughly equal), timers (advantage HomePod, because it's audible a room away), spelling (advantage Echo dot, because it's cheap and can be placed beside a computer) and music (massive advantage to HomePod, which gives the only acceptable music stream.

"We have an iPhone 7, iPhone X and Samsung s8+, upon which the digital assistants are used to make phone calls, deliver weather forecasts, set timers, set alarms and give driving directions. I'd rank Siri and Google Assistant about equal at these tasks - and Siri is actually a lot more useful to me because she's on Apple Watch, which I use far more often than any of the speaking tubes. Google Maps are probably more reliable in tests than Apple Maps, but I find the latter good enough in Car Play, which I find myself using far more often than Android Auto.

"You'll note I haven't once mentioned IoT. We do have a home theater and thermostats that are Alexa-enabled. I could enable the skills and memorize the commands, buteh. The Android and iOS Apps are simpler, and a Control4 remote appeared spontaneously on my Apple Watch. Both functions are simpler to control WITHOUT voice."


HomePod is "audible a room away"

Siri expectations for WWDC 18

So while you can expect Apple to expand Siri with some new Intents and to potentially add third-party task automation support in Workflow, don't expect the company to devote major competitive efforts toward replicating the extremely low-value Amazon Skills ecosystem that offers users little more than a more clumsy way to use apps without looking at them (the way Google has blindly, desperately tried to copy Alexa).



On the other hand, Apple has been enhancing how different Siri devices work together on the same network, an issue that neither Amazon nor Google really have. This is still a work in progress, but HomePod already does a good job of silencing your iPhone when both are listening for "Hey Siri."

What could be improved is Continuity handoff between devices, allowing HomePod to delegate questions to your iPhone or Apple Watch when it can't answer them, potentially even working together to deliver a UI on your phone or watch while listening to you from HomePod.

Similar HomePod integration with Apple TV, enabling you to ask it to play a specific show, also seems likely, enabling third-party tvOS developers to build new types of apps and games that can listen for feedback and commands using HomePod's fancy array of microphones.

HomePod will also likely expand its ability to support new types of requests (Hey Siri, where's my iPhone?), and expand what apps and data it can access under a Family Sharing account. You might also expect Apple to expand upon Type to Siri as a way to discreetly invoke Siri-type searches without using your voice at all.

What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity? Add your comments below.
watto_cobraFolio
«1

Comments

  • Reply 1 of 26
    aylkaylk Posts: 54member
    What the hell did I just read?
  • Reply 2 of 26
    FranculesFrancules Posts: 122member
    Awesome sauce!!! I love it!! 
    StrangeDayswatto_cobrajony0
  • Reply 3 of 26
    paxmanpaxman Posts: 4,729member
    I have never used any other voice service than Siri, and I have all but given it up. Bottom line - for Siri to be useful it needs to be MUCH better. Quicker, more reliable and more user friendly. Sending a text is an exercise in futility. And when the half recorded, cut out too quickly, mis-interpreted text is ready to send there is no easy way to fix it. Occasionally it works great. I do use if for dictation a fair amount, but it is hit and miss from one day to another. I don't care how much better or worse Siri is compared to other services, and I can live without Siri altogether, but as it stands Siri gets two thumbs down  from me as a digital assistant.

    edited March 2018 patchythepiratewlymmuthuk_vanalingam
  • Reply 4 of 26
    canukstormcanukstorm Posts: 2,729member
    "Siri’s development is one area where we need to watch closely from Apple in the post-Jobs era of Apple. There was a major concern that Apple would not be same, and their run on innovation would end. That certainly has not been the case, however, what I’m not sure we have seen is how Apple in the post Jobs era deals with failure or potentially impactful struggles to the companies future."

    https://techpinions.com/apple-siri-and-dealing-with-failure/52484

    "What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity?" => Given that the rumors point to iOS 12 / macOS 14 being a "Snow Leopard" type of release, I'm not expecting much this year.  

    Given that Siri languished for many years under Eddy Cue's leadership (
    Federighi now being in charge of Siri) & how important Siri is to Apple's future, I am surprised he still around at Apple.
    patchythepiratedoozydozenwlym
  • Reply 5 of 26
    mavemufcmavemufc Posts: 326member
    Really can't wait to see what iOS 12 looks like and what new features they add.
    watto_cobra
  • Reply 6 of 26
    simply258simply258 Posts: 133member
    How is iOS 12 going to fix Siri, which is server-side and has nothing to do with the OS? Siri can be improved without any device software updates.
    doozydozenaylk
  • Reply 7 of 26
    FolioFolio Posts: 698member
    Great insights in this feature, like the Workflow potential to boost Siri usefulness. But on first read looks like it was hastily edited by none other than "Bixby." Hope some human editor can polish it, as it deserves wide audience.
    doozydozenwatto_cobramuthuk_vanalingamjony0
  • Reply 8 of 26
    macapfelmacapfel Posts: 575member
    I think, Siri on HomePod should at least be smart enough to understand the command ‘Play XXX next’. But something like ‘play next’ or ‘add to queue’ seems not to be in Siri’s HomePod vocabulary. Or has anyone been successful with this?
    edited March 2018 doozydozenwatto_cobra
  • Reply 9 of 26
    racerhomie3racerhomie3 Posts: 1,264member
    Siri just works fine.
    No matter how much BS the media spews, the reality is little to no people care about voice .
    mavemufcStrangeDays
  • Reply 10 of 26
    frantisekfrantisek Posts: 760member
    For me, us, would be enough if it would support or language, that is Czech. We have dictations, reading, but no Quickty prediction, neither Siri. And I would say our language is easier to understand even is more complicated to learn. Only command I use is Turn low power mode on as I am still on ios 10.
  • Reply 11 of 26
    That was quick: Siri brought up recipes for a Negroni when I just asked...
    watto_cobra
  • Reply 12 of 26
    That was quick: Siri brought up recipes for a Negroni when I just asked...
    On an iOS device it will search google, but it doesn't offer to do anything on HomePod. 
    watto_cobra
  • Reply 13 of 26
    macapfel said:
    I think, Siri on HomePod should at least be smart enough to understand the command ‘Play XXX next’. But something like ‘play next’ or ‘add to queue’ seems not to be in Siri’s HomePod vocabulary. Or has anyone been successful with this?
    Yeah it should work like "Play Later" in the UI, but it appears it doesn't yet work from Siri yet.
    edited March 2018 watto_cobra
  • Reply 14 of 26
    "Siri’s development is one area where we need to watch closely from Apple in the post-Jobs era of Apple. There was a major concern that Apple would not be same, and their run on innovation would end. That certainly has not been the case, however, what I’m not sure we have seen is how Apple in the post Jobs era deals with failure or potentially impactful struggles to the companies future."

    https://techpinions.com/apple-siri-and-dealing-with-failure/52484

    "What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity?" => Given that the rumors point to iOS 12 / macOS 14 being a "Snow Leopard" type of release, I'm not expecting much this year.  

    Given that Siri languished for many years under Eddy Cue's leadership (Federighi now being in charge of Siri) & how important Siri is to Apple's future, I am surprised he still around at Apple.
      The talk about Post Jobs seems to forget that when he passed, Apple was shipping iOS 5, iCloud was a hot mess, Maps was a year away from shipping with serious problems, Android looked credible in tablets and Windows Mobile was still around. 

    It was well into the Post Jobs era that iOS 7 and the A7 shipped, that iCloud development was fixed and dramatically expanded, that Maps was made usable in most countries (enough to relegate Google Maps into a minor position on iOS) and where iPad crushed all tablets and no competition is left outside of Android placeholders shipping PC-like generic boxes with razor-thin margins. 

    Also, Samsung's leadership is fresh out of jail and Google has experienced a wave of new managers. So the whole Jobs thing really needs to be put to bed.  
    Rayz2016StrangeDaysgilly017watto_cobramuthuk_vanalingamFolio
  • Reply 15 of 26
    What was the purpose of this article?
    muthuk_vanalingam
  • Reply 16 of 26
    Not all of the Alexa skills are frivolous.  I use the Sonos skill, as well as the Logitech harmony Skill.  When I get home, I say "Alexa turn on Sonos", she complies, then I say "Alexa play Jethro Tull in the Living Room" and she dutifully complies by streaming Tull via Spotify.  (The biggest shortcoming here BTW is Spotify - I'm relatively new to it, but compared to Apple Music, Spotify is a real dud.  It doesn't understand King Crimson, as well as many other "older" bands.  It also has a tendency to play the same handful of songs over and again if you just specify to listen to a particular artist.  But I digress).  Another cool Alexa skill is Anylist.  This is how we make our shopping list, and its remarkably good at understanding ingredients you wouldn't expect it to understand.  And with the new Alexa feature where you don't need to say the wake-word if you are already conversing with Alexa, then its even more natural to make a shopping list just by talking.
  • Reply 17 of 26
    holyoneholyone Posts: 398member
    Hey @DanielEran if Apple makes no money from Siri how is she "failing all the way to the bank?" If siri was 100x better tomorrow and sounded just like Scarlet Johanson in "Her" how would that get people who already have iPhones to buy new ones? You're wrightup doesn't seem to follow its own logic.

    Apple views siri as complementary to the iPhone like like the watch, they have very little to gaine by dedicating the gargantuan amounts of resources it would take to make sirri get to that Samantha level, I think I read somewhere that Bezos' got over 100K people working on Alexa don't know how true that is but unlike Apple Alexa could realistically makes AMZ serious $$$ because Alexa is complementary to AMZ's business model not the products they sell, so maybe you could work on something to show us how Apple getting "Sirious" about Siri will meaning fully contribute to the bottom line since you love to bring up in almost every artticle, & why do you do that by the way, how has talking about how much money Apple's making become the thing Apple fans care about now?

    Only AAPL fans care about that and those guys car about Apple only as far as the next financial statement's good for them no?  
    muthuk_vanalingam
  • Reply 18 of 26
    jasejase Posts: 8member
    I am now heavily invested in the Apple ecosystem -  iPhone, watch, ATVs, iPad, HomePod, Mac Mini. 

    Siri works aross all of those Apple devices fine for me. I also have a CarPlay audio system so I can also use Siri in the car to send and read messages, play music, ask for directons etc. 

    Siri could absolutely be better, but I don’t see Alexa or Assistant doing the things that Siri struggles with all that well either. 

    One comment from the article that rings true for me is that Siri just works across all my devices. I can use it from my watch if I’m in the backyard and need to turn the outside lights on and so I’m fascinated to see where Apple goes with Siri, HomeKit, CarPlay and Workflow and how it all comes together across their products.

    As the article mentions, Apple definitely leads the way in terms of voice assistant integration across the range of its ecosystem, something Google and Amazon haven’t been able to do. 
    StrangeDayswatto_cobra
  • Reply 19 of 26
    StrangeDaysStrangeDays Posts: 12,955member
    What was the purpose of this article?
    I thought it was quite informative. It’s yanking the rug on the claim that “Siri is the worst product in Apple’s history” — a claim made by Walt Mosserg on twitter and bolstered by Gruber and Marco Arment's Techie Echo Chamber of Doom, Worry, and Concern (tm). My point back was identical to DED’s — this is bunk, as it’s not a stand alone product and the Apple products which offer it as a feature all kill it in sales. 

    Alexa is claimed to be some sort of champion, but as this article details thoroughly I’m not seeing the value add compared to apps. The basics are well served by either platform — home automation, reminders, alarms, texts, calendar, etc. 

    When I do use Siri it’s either on my Watch or spoken to whatever idevice is sitting closest to me. 
    edited March 2018 gilly017watto_cobra
  • Reply 20 of 26
    arthurbaarthurba Posts: 155member
    It needs some full-function API's.  Restricting the released API's to calls (VoIP) and maps makes their job easier - but it doesn't let app developers truly expand the scope of what Siri can do.

    I'm sure what I'm asking for is difficult - but that's the Apple Engineer's job.

    Expanding Siri to do more useful things should be the job of app developers - they just need a flexible API to do it with.

    The fact I can't add something to my shopping list whilst handsfree/driving is very annoying.  

    It's a trivial example - but there are 100 minor things I do with my phone every day, and 98 of them still require me to touch it to do them:
    - shopping list
    - check bank balance
    - transfer funds
    - find where my bus is (using the TripView app)
    - find the next connecting bus (TripView)
    - is there an accident ahead (LiveTraffic app)
    - read the latest Facebook posts from close friends
    - read a book (Kindle) or newspaper article (Kindle)
    - read an InMail (linked in)
    - open a camera (Nest)
    - make a doctors appointment (HotDoc)
    - note the time I put the baby to bed (Baby Tracker)
    - note the time the baby woke up (Baby Tracker)
    - note the quantity and time of medication (Baby Tracker)
    etc. etc.

    What can I do?
    - set a timer
    - text/SMS my wife

    No way can Apple add all those features - but they can work on better API's so that developers can create the apps with those features.

    I think the best example on how inflexible the Siri API is - is the HotDoc app and doctors appointments.  Siri can make a calendar appointment, but it's not flexible enough to handle a doctors appointment, because it needs some ability to select the doctor and restrict the appointments available based on available appointment slots and appointment durations (long or short).

    It was a good 'show us you are at least thinking about it' API - but it needs a follow up to 'show us you are serious about it' API.

    Folio
Sign In or Register to comment.