Future Memoji could be created automatically from user's photos

Posted:
in General Discussion
An iPhone may automatically generate Memoji for a user in the future, by analyzing a photograph of the user's face to determine the closest combination of facial features to produce a digital avatar with a similar likeness, without the user needing to make any major choices.




Apple's Animoji and Memoji features have proven popular, having been used for advertising purposes as well as for social media trends like Animoji Karaoke. In iOS 13, Apple extended Memoji to allow users to create collections of stickers using the user-based character, which can be added to Messages.

Even so, the feature does rely on users going through a process to create their Memoji, which can take a few minutes to accomplish, though to get a perfect likeness it can take quite a while longer. Some users, such as the visually impaired, simply cannot use the feature as intended, and either have to rely on another person to capture their likeness accurately, or resign themselves to using an Animoji.

In a patent granted by the US Patent and Trademark Office on Tuesday, Apple 's "Technique for creating avatar from image data" suggests this need not be the case, as one could be produced from a photograph.

In short, the patent uses an image of the user to determine the facial features of the subject, then selects digital representations and compiles them into an avatar.

The processing method measures the spatial facial attributes of the subject, classifying the expression of the person at the same time, and determining which from a selection of avatar templates is the closest match. The template avatars are deformed based on the expression classification to aid matching.

An illustration of how images of a subject can be broken down into facial features and location measurements
An illustration of how images of a subject can be broken down into facial features and location measurements


Other attributes of the subject are then identified and the selected avatar template is deformed further, before the process is completed an a user's avatar is available to use.

The classification can include the identifying of spatial locations of hair on the subject, then to determine multiple different templates of hair styles which can be attached to the template, as well as for identifying eyeglasses, facial hair, and other elements. Eye, hair, and skin color can also be used as part of the final deformation process.

The data for the compiled avatar could also be transferred into other templates, such as to provide users with variations on an appearance. This can include costume changes or those of expressions, or even the aforementioned Memoji stickers.

Apple files numerous patent filings on a weekly basis, but while the existence of a patent suggests areas of interest for Apple's research and development efforts, they don't guarantee the described concepts will appear in a future product or service.

In this case, as it is an image processing patent that is entirely based in software, and given Apple's existing work with Memoji, it seems plausible some form of automated Memoji creation feature could be made available in the future.

While the patent is more about a flat 2D image rather than using three-dimensional data, the existence of Face ID and the TrueDepth camera array may help Apple further the concept. By using the available 3D mapping capability, Apple has the potential to take detailed measurements of a user's face for avatar creation, with the relative ease potentially allowing users to generate new Memoji whenever they want.

This is not the only way Apple believes it can enhance Memoji and Animoji for its users. One patent application that surfaced in November 2018 suggested the recording of audio with Animoji, but rather than leave the audio plain, Apple proposed switching out spoken key words for sound effects and custom character animations, such as a dog's bark when the user says "woof" to the iPhone.

Comments

  • Reply 1 of 18
    lkrupplkrupp Posts: 9,121member
    I ran across a hilarious thread on the Apple Discussions Forums where some users were going apoplectic over the unicorn emojis. They demanded to know how to remove them permanently OR ELSE, Apple. 
    DAalseth
  • Reply 2 of 18
    hmurchisonhmurchison Posts: 12,341member
    The fixation and brain power entrusted to Emoji is approaching Idiocracy levels.  Should I put my can of Brawndo next to my Mac at all times?
    mwhitemuthuk_vanalingamSpamSandwich
  • Reply 3 of 18
    Good! I hope that they accurately portray wrinkles and crows feet. I get a chuckle at all the Memoji (and Bitmoji) that show the person who it's supposed to represent appearing to be 20 or 30 years younger than they actually are.
    mwhite
  • Reply 4 of 18
    sflagelsflagel Posts: 617member
    There has got to be an on-line service where I can send a pic of myself and who would then recommend to me the most accurate selection... does anyone know?
  • Reply 5 of 18
    dewmedewme Posts: 3,612member
    It seems like it would be a pretty easy series of steps to go from apps that transform photos into cartoons, e.g., ToonCamera, to ones that create a Memoji from a photo. 

    What's the purpose of all of this? My take is that smartphones and social media have allowed us to move away from personal, intimate, and reality based forms of human interaction that exist in meatspace to virtual interactions created behind a personally curated facade that represents how we want to be perceived by other virtual souls that interact in cyberspace. Social media allows us to create the "Virtual Me" and our smartphones and Memoji allow us to carry our Virtual Me around with us so we are never have to be exposed in our actual human form. Just like clothing allows us to cover up our naked human form, the Virtual Me allows us to cover up our outward physical appearance, our true personality, social flaws, and our inherent communicative capabilities and weaknesses into something more palatable for the rest of society to see. In other words, we're simply adding another layer of "clothing" to cover up a broader range of things we're looking to hide. At some point I expect there will be professional design studios that will sell you a much more fashionable Virtual Me version of yourself than what you can come up with on your own. 

      
    randominternetpersonhmurchison
  • Reply 6 of 18
    lkrupp said:
    I ran across a hilarious thread on the Apple Discussions Forums where some users were going apoplectic over the unicorn emojis. They demanded to know how to remove them permanently OR ELSE, Apple. 
    They are mildly annoying, seeming to appear at intervals when I hit the emoji button, hiding the other ones.  It would nice to be able to permanently put them off the screen until I ask for them.

    Not sure who thought it was a good idea to put them in the spot once reserved for oft used emojis, but that person was wrong.

    Not sussed enough about it to rant and rave though.  :smiley: 

  • Reply 7 of 18
    Should make the process of making something close to reality easier.


    DAalseth
  • Reply 8 of 18
    mac_dogmac_dog Posts: 888member
    Are you fucking kidding me???
  • Reply 9 of 18
    Why is everyone so negative? Apple is one of the only companies that truly understands the value of delight, a UX principle that has been the core of Apple's products since the return of Jobs. It is why Apple dumped billions into the iPhone while Ballmer laughed at it. Even till this day very few companies appreciate it, they talk about its importance but when it's time to pull out that checkbook, they're like oh no that's a "nice to have feature" but not priority. The lack of it is plastered all over Google and Samsung products, they bring out gimmicky features because no one really believed in delight to begin with.
    This feature is going to be key to iPhone success.
    fastasleep
  • Reply 10 of 18
    mac_dog said:
    Are you fucking kidding me???
    This feature makes sense to me, even though it isn't something I would use other than to just mess with it. Here's why it makes sense to me: after upgrading to iOS 13 I was suddenly getting a bunch of "so-and-so wants to share a new photo with you" in the Messages app. People are clearly using MeMoji so why shouldn't Apple do something to engage them even more?
    fastasleep
  • Reply 11 of 18
    hmurchisonhmurchison Posts: 12,341member
    Why is everyone so negative? Apple is one of the only companies that truly understands the value of delight, a UX principle that has been the core of Apple's products since the return of Jobs. It is why Apple dumped billions into the iPhone while Ballmer laughed at it. Even till this day very few companies appreciate it, they talk about its importance but when it's time to pull out that checkbook, they're like oh no that's a "nice to have feature" but not priority. The lack of it is plastered all over Google and Samsung products, they bring out gimmicky features because no one really believed in delight to begin with.
    This feature is going to be key to iPhone success.

    I truly don’t mean to be negative.   I view computing as being a tool for problem solving.   That delight for me comes from great software and hardware in concert with each other with clever and extensible features that really make a difference in my life or at least turn that light bulb on.  I’m not adverse to Memoji it has a place in our lives as will Apple TV+  but these are essentially diversions.  

    There are many frustrations that Memoji simply won’t fix.    The Internet has become a cess pool.   It’s now so bad even Tim Berners Lee says it’s broken.   We needed  Apple Screen Time in its current form like 3 years ago so that today we had a robust solution.   Parent with young children would pay for this as a service.   I appreciate Apple’s commitment to privacy and wish they’d take more leadership in areas of moving computing forward.  
    muthuk_vanalingam
  • Reply 12 of 18
    dewme said:
    It seems like it would be a pretty easy series of steps to go from apps that transform photos into cartoons, e.g., ToonCamera, to ones that create a Memoji from a photo. 

    What's the purpose of all of this? My take is that smartphones and social media have allowed us to move away from personal, intimate, and reality based forms of human interaction that exist in meatspace to virtual interactions created behind a personally curated facade that represents how we want to be perceived by other virtual souls that interact in cyberspace. Social media allows us to create the "Virtual Me" and our smartphones and Memoji allow us to carry our Virtual Me around with us so we are never have to be exposed in our actual human form. Just like clothing allows us to cover up our naked human form, the Virtual Me allows us to cover up our outward physical appearance, our true personality, social flaws, and our inherent communicative capabilities and weaknesses into something more palatable for the rest of society to see. In other words, we're simply adding another layer of "clothing" to cover up a broader range of things we're looking to hide. At some point I expect there will be professional design studios that will sell you a much more fashionable Virtual Me version of yourself than what you can come up with on your own. 
      
    I'm bewildered by many of the comments here. Has nobody heard of avatars before? This isn't a new concept by any stretch of the imagination. Memoji is just building on where Bitmoji was going with this (now owned by Snap), which is incredibly popular. Obviously using avatars will become more ubiquitous across AR/VR, gaming, and other applications beyond messaging and traditional social media. This has been around for decades, it's not going anywhere.

    ToonCamera is just a photo filter app, like a "find edges" Photoshop plugin from the 90's. This is building an actual interactive 3D model that can be used in many different ways. There's literally nothing in common between the two, and the tech required to analyze a photo using ML to build this model is far, far more complex.


  • Reply 13 of 18
    DAalsethDAalseth Posts: 1,556member
    Interesting article, but it made me realize that I wasn't sure what a Memoji was. I remember hearing about them. But it was part of Apple's presentation on Emoji and Animoji, so I tuned it out. Curious, I found a page on the Apple site explaining what they were, how to make them, and how to use them. Uh yeaaaa...pass. Not interested. I'vr got better things to do with my time, and my phone.
  • Reply 14 of 18
    sflagel said:
    There has got to be an on-line service where I can send a pic of myself and who would then recommend to me the most accurate selection... does anyone know?

    I discovered my emoji posting to Seeking Arrangements.  Should I be worried?
  • Reply 15 of 18
    fastasleepfastasleep Posts: 5,396member
    DAalseth said:
    Interesting article, but it made me realize that I wasn't sure what a Memoji was. I remember hearing about them. But it was part of Apple's presentation on Emoji and Animoji, so I tuned it out. Curious, I found a page on the Apple site explaining what they were, how to make them, and how to use them. Uh yeaaaa...pass. Not interested. I'vr got better things to do with my time, and my phone.
    You could’ve made one in the time it took to post this comment. 
  • Reply 16 of 18
    dewmedewme Posts: 3,612member
    dewme said:
    It seems like it would be a pretty easy series of steps to go from apps that transform photos into cartoons, e.g., ToonCamera, to ones that create a Memoji from a photo. 

    What's the purpose of all of this? My take is that smartphones and social media have allowed us to move away from personal, intimate, and reality based forms of human interaction that exist in meatspace to virtual interactions created behind a personally curated facade that represents how we want to be perceived by other virtual souls that interact in cyberspace. Social media allows us to create the "Virtual Me" and our smartphones and Memoji allow us to carry our Virtual Me around with us so we are never have to be exposed in our actual human form. Just like clothing allows us to cover up our naked human form, the Virtual Me allows us to cover up our outward physical appearance, our true personality, social flaws, and our inherent communicative capabilities and weaknesses into something more palatable for the rest of society to see. In other words, we're simply adding another layer of "clothing" to cover up a broader range of things we're looking to hide. At some point I expect there will be professional design studios that will sell you a much more fashionable Virtual Me version of yourself than what you can come up with on your own. 
      
    I'm bewildered by many of the comments here. Has nobody heard of avatars before? This isn't a new concept by any stretch of the imagination. Memoji is just building on where Bitmoji was going with this (now owned by Snap), which is incredibly popular. Obviously using avatars will become more ubiquitous across AR/VR, gaming, and other applications beyond messaging and traditional social media. This has been around for decades, it's not going anywhere.

    ToonCamera is just a photo filter app, like a "find edges" Photoshop plugin from the 90's. This is building an actual interactive 3D model that can be used in many different ways. There's literally nothing in common between the two, and the tech required to analyze a photo using ML to build this model is far, far more complex.


    The patent described in this article is exactly about 2D imaging.The 3D references are all speculative discussion by the author of the article and not part of the patent. I've gone through the memoji creation process on the iPhone and it seemed silly that I could only use the built-in images to build the model rather than starting with a cartoon rendering of my actual image. While the "how" part of Memoji/avatar creation is a fascinating technological topic the "why" part is still something that I can't fully get my head wrapped around.
  • Reply 17 of 18
    dewme said:
    dewme said:
    It seems like it would be a pretty easy series of steps to go from apps that transform photos into cartoons, e.g., ToonCamera, to ones that create a Memoji from a photo. 

    What's the purpose of all of this? My take is that smartphones and social media have allowed us to move away from personal, intimate, and reality based forms of human interaction that exist in meatspace to virtual interactions created behind a personally curated facade that represents how we want to be perceived by other virtual souls that interact in cyberspace. Social media allows us to create the "Virtual Me" and our smartphones and Memoji allow us to carry our Virtual Me around with us so we are never have to be exposed in our actual human form. Just like clothing allows us to cover up our naked human form, the Virtual Me allows us to cover up our outward physical appearance, our true personality, social flaws, and our inherent communicative capabilities and weaknesses into something more palatable for the rest of society to see. In other words, we're simply adding another layer of "clothing" to cover up a broader range of things we're looking to hide. At some point I expect there will be professional design studios that will sell you a much more fashionable Virtual Me version of yourself than what you can come up with on your own. 
      
    I'm bewildered by many of the comments here. Has nobody heard of avatars before? This isn't a new concept by any stretch of the imagination. Memoji is just building on where Bitmoji was going with this (now owned by Snap), which is incredibly popular. Obviously using avatars will become more ubiquitous across AR/VR, gaming, and other applications beyond messaging and traditional social media. This has been around for decades, it's not going anywhere.

    ToonCamera is just a photo filter app, like a "find edges" Photoshop plugin from the 90's. This is building an actual interactive 3D model that can be used in many different ways. There's literally nothing in common between the two, and the tech required to analyze a photo using ML to build this model is far, far more complex.


    While the "how" part of Memoji/avatar creation is a fascinating technological topic the "why" part is still something that I can't fully get my head wrapped around.
    The "why" is the easy part, at least sometimes. I worked with a guy who changed up his hair every day, sometimes he wore it down, sometimes he had it in a bun, sometimes he had pony tails, etc. He was super excited when Memoji was introduced because he wanted to be able to send an up-to-date avatar of what he looked like to people he may not see that day. There's also my MIL, who is never, for as long as I've known her, been happy with the way she looks in photos. She is much happier to have a not-quite-accurate avatar of herself being shared with people than she is with having her own photo shared. Others just think it's fun. You must have gotten people sending you Bitmoji in text messages before, I know I have hundreds of times. The people sending them just think they're fun or cute or whatever. I'm sure I'm only scratching the surface but there are a few "whys" for you.
  • Reply 18 of 18
    fastasleepfastasleep Posts: 5,396member
    dewme said:
    dewme said:
    It seems like it would be a pretty easy series of steps to go from apps that transform photos into cartoons, e.g., ToonCamera, to ones that create a Memoji from a photo. 

    What's the purpose of all of this? My take is that smartphones and social media have allowed us to move away from personal, intimate, and reality based forms of human interaction that exist in meatspace to virtual interactions created behind a personally curated facade that represents how we want to be perceived by other virtual souls that interact in cyberspace. Social media allows us to create the "Virtual Me" and our smartphones and Memoji allow us to carry our Virtual Me around with us so we are never have to be exposed in our actual human form. Just like clothing allows us to cover up our naked human form, the Virtual Me allows us to cover up our outward physical appearance, our true personality, social flaws, and our inherent communicative capabilities and weaknesses into something more palatable for the rest of society to see. In other words, we're simply adding another layer of "clothing" to cover up a broader range of things we're looking to hide. At some point I expect there will be professional design studios that will sell you a much more fashionable Virtual Me version of yourself than what you can come up with on your own. 
      
    I'm bewildered by many of the comments here. Has nobody heard of avatars before? This isn't a new concept by any stretch of the imagination. Memoji is just building on where Bitmoji was going with this (now owned by Snap), which is incredibly popular. Obviously using avatars will become more ubiquitous across AR/VR, gaming, and other applications beyond messaging and traditional social media. This has been around for decades, it's not going anywhere.

    ToonCamera is just a photo filter app, like a "find edges" Photoshop plugin from the 90's. This is building an actual interactive 3D model that can be used in many different ways. There's literally nothing in common between the two, and the tech required to analyze a photo using ML to build this model is far, far more complex.


    The patent described in this article is exactly about 2D imaging.The 3D references are all speculative discussion by the author of the article and not part of the patent. I've gone through the memoji creation process on the iPhone and it seemed silly that I could only use the built-in images to build the model rather than starting with a cartoon rendering of my actual image. While the "how" part of Memoji/avatar creation is a fascinating technological topic the "why" part is still something that I can't fully get my head wrapped around.
    You’re wrong. Memoji are 3D. It’s talking about taking 2D data from a photo to determine which of those 3D feature get applied. The article says it doesn’t discuss using 3D data from the TrueDepth camera to apply that data to the model, but it doesn’t mean it’s not already using the 2D data to build a 3D model as we know Memoji to function already. 
Sign In or Register to comment.