Apple exposes how it teaches new languages to Siri, staying ahead of competition

Posted:
in iPhone
A report on Thursday offered a behind-the-scenes look at how Apple introduces new languages for Siri, suggesting that broader language support is one of the few things keeping the voice assistant competitive in the face of rivals from Amazon and Google.




To bring in a new language, Apple first starts by having people read passages in a variety of accents and dialects, Apple speech team head Alex Acero explained to Reuters. These passages are manually transcribed so computers know exactly what they're supposed to be learning. After being supplemented by sounds captured in a range of voices, Apple builds a language model that tries to predict word sequences.

Before actually upgrading Siri, Apple rolls out dictation. The company is said to collect and anonymize a small percentage of recordings from customers, and then have humans transcribe these, something Acero said can cut recognition errors in half.

After gathering enough data, the company finally prepares to launch Siri in the new language, including hiring one or more actors. The first release may still only be able to answer the most frequent questions -- Apple, though, will update its codebase every two weeks with adjustments.

The result is that Siri is now in 21 languages and 36 countries. Microsoft's Cortana supports 8 languages in 13 countries, while Google Assistant -- a relatively new AI, replacing Google Now -- speaks 4 languages. Amazon's Alexa works only in English and German.

One of Apple's next languages will be Shanghainese, a Chinese dialect spoken only around that city. Dictation support is coming in impending software updates such as macOS 10.12.4.

Apple's approach could be problematic, Reuters hinted, because of how difficult it is to scale, demanding more and more writers. Other assistants, like the Samsung-owned Viv, may be better designed for this task.

Siri has increasingly been accused of lagging behind its rivals in terms of capabilities. Amazon's Alexa regularly gains new skills, for instance, while Google Assistant is able to understand context, enabling simpler, more conversational responses.

Apple is rumored to be planning Siri enhancements which could debut on new iPhones shipping this fall. What those might be aren't clear, though the company has been investing large sums into AI and machine learning.
«1

Comments

  • Reply 1 of 30
    Jacobh101Jacobh101 Posts: 1unconfirmed, member
    its difficult to scale? they are blowing away the competition!
    patchythepiratelollivercornchip
  • Reply 2 of 30
    bloggerblogbloggerblog Posts: 2,003member
    "[Siri] staying ahead of competition" sounds a little far fetched. I would rather have Siri be effective with fewer languages than suck at all of them. I cringe at how much worse Siri must be with some of the lesser used languages.
    Roger_Fingasbrucemconeof52boxcatcher
  • Reply 3 of 30
    joe28753joe28753 Posts: 82member
    I don't think transcribing the words she hears is the problem. Being able to do what I want is the problem. I still feel like I need to memorize a list of commands she knows. Too often I get a "would you like me to search the web?" or "I'm sorry I can't do that." I won't be likely to use Siri much until I can speak to her like she's actually a person, or can have a conversation and figure out what I mean more reliably.
    caliSpamSandwichoneof52jdunysdamn_its_hot[Deleted User]cornchip
  • Reply 4 of 30
    "Airplay This is England." "Everything you need to know about Apple products is at Apple's website." "Airplay This is England to Apple TV." "OK, I found this on the web for 'Airplay this is England to Apple TV." Thanks, Siri, I'll do it myself like some kind of animal.
    [Deleted User]cornchip
  • Reply 5 of 30
    SoliSoli Posts: 10,025member
    Siri can say “I don’t know what you mean" in the most languages.
    patchythepirateoneof52bloggerblogboxcatchercornchip
  • Reply 6 of 30
    avon b7avon b7 Posts: 5,676member
    The iOS 10 Spanish Spanish Siri sounds half South American. Not natural at all. I hope they bring back the old pronunciation.
  • Reply 7 of 30
    zroger73zroger73 Posts: 775member
    I rarely use Siri. On the rare occasion I do, it seems to be an exercise in frustration. It seems like an update somewhere along the way changed the way Siri activated using the home button and I've never got the hang of it.

    Alexa, on the other hand, is very useful. I talk to her several times each day on my three Echo devices to control lighting, read news and weather, set timers, make conversions and calculations, add items to my shopping list, read audio books, and play music.

    I'm a huge Apple "fanboy" and I'd much prefer to be tied to their ecosystem instead of Amazon's, but Alexa has Siri beat in terms of ease of use and the ability to actually accomplish things - at least for me. I've not used Google Home and have no desire to at this time.
    cornchip
  • Reply 8 of 30
    SoliSoli Posts: 10,025member
    zroger73 said:
    I rarely use Siri. On the rare occasion I do, it seems to be an exercise in frustration. It seems like an update somewhere along the way changed the way Siri activated using the home button and I've never got the hang of it.

    Alexa, on the other hand, is very useful. I talk to her several times each day on my three Echo devices to control lighting, read news and weather, set timers, make conversions and calculations, add items to my shopping list, read audio books, and play music.

    I'm a huge Apple "fanboy" and I'd much prefer to be tied to their ecosystem instead of Amazon's, but Alexa has Siri beat in terms of ease of use and the ability to actually accomplish things - at least for me. I've not used Google Home and have no desire to at this time.
    As frustrating as Siri is, I'd wager that Siri has a better back end than Alexa. I think the primary benefit for Alexa is that the Echo has much better HW in being able to understand your comments. I use an Echo Dot as my bedroom clock, alarm clock, and with a Skill for meditation to help me fall asleep at night (which I prefer over a standard alarm clock due to the lack of always-on lights or having to look at it), and it's not as good at understanding commands as its larger, more expensive brother.
  • Reply 9 of 30
    frantisekfrantisek Posts: 730member
    "[Siri] staying ahead of competition" sounds a little far fetched. I would rather have Siri be effective with fewer languages than suck at all of them. I cringe at how much worse Siri must be with some of the lesser used languages.
    There is no guaranty that it would help. But Tim was talking how machine learning makes big difference in Siri abilities. Just not sure they were already incorporated or we are still waiting.

    I would prefer my language included even with limited feature set to be able to call/message people by their names.

    Next level for Siri is at least bilingual operation. But seeing that we got just bilingual spellcheck for few languages it will take long time. AI asistants have long way to go.
    edited March 2017
  • Reply 10 of 30
    SoliSoli Posts: 10,025member
    frantisek said:
    "[Siri] staying ahead of competition" sounds a little far fetched. I would rather have Siri be effective with fewer languages than suck at all of them. I cringe at how much worse Siri must be with some of the lesser used languages.
    There is no guaranty that it would help. But Tim was talking how machine learning makes big difference in Siri abilities. Just not sure they were already incorporated or we are still waiting.

    I would prefer my language included even with limited feature set to be able to call/message people by their names.

    Next level for Siri is at least bilingual operation. But seeing that we got just bilingual spellcheck for few languages it will take long time. AI asistants have long way to go.
    I believe machine learning wasn't only included, but also included locally into IOS 10 on your device.


    edited March 2017
  • Reply 11 of 30
    calicali Posts: 3,494member
    Not sure why people were excited that Siri engineers can now expose secrets. Why do we have to know this? The competition is gonna be quick to copy alright. 
  • Reply 12 of 30
    SpamSandwichSpamSandwich Posts: 33,408member
    joe28753 said:
    I don't think transcribing the words she hears is the problem. Being able to do what I want is the problem. I still feel like I need to memorize a list of commands she knows. Too often I get a "would you like me to search the web?" or "I'm sorry I can't do that." I won't be likely to use Siri much until I can speak to her like she's actually a person, or can have a conversation and figure out what I mean more reliably.
    Yes, that is a huge failing with Siri. Instead of being given the wrong results more than half the time, I'd prefer if Siri would instead ask follow-up questions for clarity. I cannot remember what specific trigger words Siri requires or what particular sentence wiould optimize my requests to the point where I've given up on any kind of reliance on the service.
    cornchip
  • Reply 13 of 30
    I wish Apple would make a device that is a hybrid of the Echo dot and the Eero WiFi System.

    For $300 you get 3 "Siri hubs"
    Replaces your current WiFi routers with a mesh network that works well.
    Typical placement: 1) Kitchen 2) Livingroom 3) Master bedroom
    Each Siri hub has a microphone array that is always listening for questions or commands.

    Eero already works with Amazon Alexa and can do the following...


    This is the kind of skills Siri/HomeKit need!
    While the Eero solution is nice you would need to purchase 3 Eeros($375) and 3 Echo Dots($150)...$525!
    Soliboxcatcher
  • Reply 14 of 30
    sog35 said:
    "[Siri] staying ahead of competition" sounds a little far fetched. I would rather have Siri be effective with fewer languages than suck at all of them. I cringe at how much worse Siri must be with some of the lesser used languages.
    Apple is playing the long game

    Apple is playing the long game with regard to HomeKit as well and I think that is why we haven't seen a competitor to the Amazon dot and echo.
    Apple doesn't want to spook their HomeKit partners by competing with them in the HomeKit hardware space.
    lolliver
  • Reply 15 of 30
    It's genuinely impressive and good news that Siri is capable in so many languages.

    However, I am still continually frustrated with just the basic functionality, and I know I'm not alone. The most recent frustration: since siri, somehow, cant understand "call ____ on ____ street," I entered my local pharmacy in my contacts as "Midtown Walgreens." Then, inexplicably, when I say "call midtown walgreens" siri's reply is "I couldn't find any wallgreens locations in midtown, here are some walgreens locations that are near you" (this is nearly verbatim IIRC). And of course, multiple, logical work-arounds didn't work; they never do with siri. For example, "call midtown walgreens from my contacts," siri's response was, and I'm not joking, "I couldn't find the contact 'contact midtown walgreens' in your contacts."

    Siri is endlessly frustrating.

    I have a strong feeling that Apple's issue with Siri is something along the lines of 'the perfect is the enemy of the good,' in that Apple is focusing on advanced functionality while neglecting simple, basic functionality.
    StrangeDaysSpamSandwichlolliver
  • Reply 16 of 30
    StrangeDaysStrangeDays Posts: 11,153member
    It's genuinely impressive and good news that Siri is capable in so many languages.

    However, I am still continually frustrated with just the basic functionality, and I know I'm not alone. The most recent frustration: since siri, somehow, cant understand "call ____ on ____ street," I entered my local pharmacy in my contacts as "Midtown Walgreens." Then, inexplicably, when I say "call midtown walgreens" siri's reply is "I couldn't find any wallgreens locations in midtown, here are some walgreens locations that are near you" (this is nearly verbatim IIRC). And of course, multiple, logical work-arounds didn't work; they never do with siri. For example, "call midtown walgreens from my contacts," siri's response was, and I'm not joking, "I couldn't find the contact 'contact midtown walgreens' in your contacts."

    Yes, this is annoying. I have multiple chain grocery stores near me and if I ask for one on XXXX street, it fails to identify that one and instead just tells me there are a bunch and which do i want. So annoying. Have they not also experienced this in dev land??
    patchythepiratelolliver
  • Reply 17 of 30
    avon b7avon b7 Posts: 5,676member
    frantisek said:
    "[Siri] staying ahead of competition" sounds a little far fetched. I would rather have Siri be effective with fewer languages than suck at all of them. I cringe at how much worse Siri must be with some of the lesser used languages.
    There is no guaranty that it would help. But Tim was talking how machine learning makes big difference in Siri abilities. Just not sure they were already incorporated or we are still waiting.

    I would prefer my language included even with limited feature set to be able to call/message people by their names.

    Next level for Siri is at least bilingual operation. But seeing that we got just bilingual spellcheck for few languages it will take long time. AI asistants have long way to go.
    Basic bilingual support would be nice. Just catching Siri's attention in one language and following on with that same language would be good. Having Siri able to detect languages within a language or speakers of different languages would be even nicer.
  • Reply 18 of 30
    Apple really needs to get better at dictation. Most of the time Siri doesn't understand what I say while Google assignment or Cortana is very accurate. That's why I stopped using the Quicktype keyboard because dictation is extremely bad compared to Gboard.  
  • Reply 19 of 30
    emoelleremoeller Posts: 521member
    Since using AirPods my dictation has improved dramatically with Siri.  I think it is the microphones in each earpiece that makes the difference.  Once I started doing more dictation I learned that some of the problems I had been having with Sire not understanding me were related to syntax issues.   Here is a nice listing of the Siri syntax:

    http://www.siriuserguide.com/siri-dictation-guide/

    For example the person who had problems "And of course, multiple, logical work-arounds didn't work; they never do with siri. For example, "call midtown walgreens from my contacts," siri's response was, and I'm not joking, "I couldn't find the contact 'contact midtown walgreens' in your contacts."  Works much better when one says Call "XXX" as it adds context for Siri.  

    Spending some time with Siri and playing with the various syntax options, and using AirPods has made me a regular Siri user now even in the car, which previously was unusable.
  • Reply 20 of 30
    bloggerblogbloggerblog Posts: 2,003member
    sog35 said:
    "[Siri] staying ahead of competition" sounds a little far fetched. I would rather have Siri be effective with fewer languages than suck at all of them. I cringe at how much worse Siri must be with some of the lesser used languages.
    Apple is playing the long game
    Playing the long game would be focusing on getting a few languages to work well before delving into a plethora of languages. Apple is definitely not playing the long game here, they seem to be sucking up to as many markets as possible.
Sign In or Register to comment.