Apple working on how to read back iMessages in the sender's voice

Posted:
in iOS
A new Apple patent application describes converting an iMessage into a voice note played back in a voice that the user has created with samples of the sender's voice.

Messages
Messages


Apple users can already send audio recordings in iMessage or have Siri read text messages back to them, but the patent describes a way to have the device read the text message in the sender's voice instead of Siri, using a voice file.

"The voice model is provided to a second electronic device," according to the patent. "In some examples, a message is received from a respective user of a second electronic device."

Meaning, when someone sends an iMessage, they can choose to attach a voice file, which would be stored on the device. If this happens, the receiver will be prompted to decide if they want to receive both the message and the voice recording.




"In response to receiving the message, a voice model of the respective user is received," the patent reads. "Based on the voice model, an audio output corresponding to the received message is provided."

According to the patent, the iPhone in question would then build a Siri-like profile for the sender's voice, and then simulated it when reading that message and any future messages they receive from that sender. The voice simulation model could also be sent on its own so a person's contacts can download it before messages.

It would offer more personalization when friends and family text each other, instead of hearing Siri's voice when it reads messages. Couples could also hear messages in a more personal way, such as hearing "I love you" in their partner's voice.

The patent's inventors are Qiong Hi, Jiangchuan Li, and David A. Winarsky. Winarsky is Apple's director of text-to-speech technology, while Li is a senior Siri software engineer for machine learning at Apple, and Hu formerly worked on Siri at the company.

As usual, with patents, it won't necessarily become a reality, but it's possible given Apple's recent work with artificial intelligence and voices. For example, with iOS 11, Apple switched Siri's voice from relying on recordings from voice actors to a text-to-speech model using machine learning.

In 2020, Apple acquired a company called Voysis that worked on improving natural language processing in virtual assistants. They used WaveNet technology, which was introduced by Google's DeepMind program in 2016.

WaveNets are "deep generative models of raw audio waveforms" that can be used to generate speech that mimics any human voice.

Apple has also started using artificial intelligence to narrate specific audiobook genres instead of using humans. So the patent is entirely in the realm of possibility of having an Apple device eventually learn to read messages in a person's voice.

Read on AppleInsider

Comments

  • Reply 1 of 18
    DAalsethDAalseth Posts: 2,836member
    A bit torn. Sounds like it might be cute. But also sounds like it could be creepy beyond all reason. You know,, depending on the circumstances. 
    muthuk_vanalingamlolliver
  • Reply 2 of 18
    maltzmaltz Posts: 474member
    Patenting does not necessarily mean "working on".  I can't imagine anyone would want or care about this.
    williamlondon
  • Reply 3 of 18
    This is so possible today. A friend of mine asked ChatGPT a question, he took the answer and uploaded a few samples of his voice to a different AI and then had it read the answer from ChatGPT with his own voice, and let me tell you... it was dead on, with all the quirks and nuances in his voice. Crazy stuff... So this patent is 100% plausible
    lolliver
  • Reply 4 of 18
    DAalseth said:
    A bit torn. Sounds like it might be cute. But also sounds like it could be creepy beyond all reason. You know,, depending on the circumstances. 
    I vote for "creepy".
    Japheyboboliciousmaltz
  • Reply 5 of 18
    mike1mike1 Posts: 3,316member
    Not sure. Might be OK for close contacts like my wife or kids. In reality, those would likely be the only people I would have a voice sample from anyway.
  • Reply 6 of 18
    mike1 said:
    Not sure. Might be OK for close contacts like my wife or kids. In reality, those would likely be the only people I would have a voice sample from anyway.
    If I’m reading the article correctly the iPhone of the person who sent you the message would also send a voice profile of the sender’s voice. Your iPhone would then read back the message in the sender’s voice using the profile that was sent along. You wouldn’t have to provide the voice sample to hear a message in your wife’s voice, her phone would do it automatically. Same goes for that new plumber you just hired, you’d never need to have met but if they sent a text it could be read in their voice.
    lolliver
  • Reply 7 of 18
    This sounds like a phone call with extra steps
    maltzbaconstangwilliamlondon
  • Reply 8 of 18
    maltz said:
    I can't imagine anyone would want or care about this.
    This sounds like a you problem.
    williamlondonSpitbathlolliverStrangeDays
  • Reply 9 of 18
    dewmedewme Posts: 5,492member
    Interesting. If the sender gets to submit the voice sample, couldn’t they submit a sample based on a voice recording obtained from any number of openly available audio sources? I suppose this concern (for things like catfishing) could be mitigated if the sample capture involves reading a unique/traceable script. 

    I suppose this means we’ll no longer feel obliged to respond to Siri with “I love you too Siri” when Siri transcribes texts into voice. 


    lolliverbaconstang
  • Reply 10 of 18
    mike1mike1 Posts: 3,316member
    dewme said:

    I suppose this means we’ll no longer feel obliged to respond to Siri with “I love you too Siri” when Siri transcribes texts into voice. 

    :D
    lolliver
  • Reply 11 of 18
    DAalseth said:
    A bit torn. Sounds like it might be cute. But also sounds like it could be creepy beyond all reason. You know,, depending on the circumstances. 
    I vote for "creepy".
    ... and in the wrong hands could this be hugely dangerous ...?
    Is this the brave new world of data scraping and soon AI / AR that has become Apple ...?
    williamlondonDAalseth
  • Reply 12 of 18
    mknelsonmknelson Posts: 1,128member
    ItsDeCia said:
    This sounds like a phone call with extra steps
    Or just messaging an audio clip.

    I suppose in the long run this could use less data.

    Seems like another patent for IPs sake, rather than intended for a product. 
  • Reply 13 of 18
    mknelson said:
    ItsDeCia said:
    This sounds like a phone call with extra steps
    Or just messaging an audio clip.

    I suppose in the long run this could use less data.

    Seems like another patent for IPs sake, rather than intended for a product. 
    Years ago I read (on AI, I think) that sending voice messages was very popular in some countries (Brazil?). My first thought was that a system like this might be preferable in a situation where sending a message via CarPlay only sends text. If that text sent via CarPlay could be converted to the sender’s voice on the receiver’s end that may be very similar to sending a voice message, something that can’t be currently done with CarPlay (for example). 
    lolliver
  • Reply 14 of 18
    Just great- deep fake messages that sound like you.
    Someone gets mad- makes a message saying someone is going to do something bad- and leaves a real sounding message on someone elses voicemail...
    no, just no...
    Apple, I am starting to want to break up with you....
    williamlondon
  • Reply 15 of 18
    Oh, I think this would be cool.
  • Reply 16 of 18
    Oh yeah!  

    I'm gonna have Queen Elizabeth read all my txts to me!
  • Reply 17 of 18
    Just great- deep fake messages that sound like you.
    Someone gets mad- makes a message saying someone is going to do something bad- and leaves a real sounding message on someone elses voicemail...
    no, just no...
    Apple, I am starting to want to break up with you....
    Did you read something I didn’t? This article is about text messaging, not voicemail. It’s also about using the sender’s voice to read a text message, not some random person’s voice. I’ll make an assumption here that if I send a message to someone, even if I can manage to do it in someone else’s voice, the message will still come from me, so I’m not sure who would be fooled in that made up scenario of yours.
    edited February 2023 williamlondon
  • Reply 18 of 18
    mattinozmattinoz Posts: 2,371member
    Memojis with voice profile that you can share and indeed retract with other users would be great. 

    I do think the owner of the voice should be in control of creation and use.
Sign In or Register to comment.