Future Memoji could be created automatically from user's photos
An iPhone may automatically generate Memoji for a user in the future, by analyzing a photograph of the user's face to determine the closest combination of facial features to produce a digital avatar with a similar likeness, without the user needing to make any major choices.
Apple's Animoji and Memoji features have proven popular, having been used for advertising purposes as well as for social media trends like Animoji Karaoke. In iOS 13, Apple extended Memoji to allow users to create collections of stickers using the user-based character, which can be added to Messages.
Even so, the feature does rely on users going through a process to create their Memoji, which can take a few minutes to accomplish, though to get a perfect likeness it can take quite a while longer. Some users, such as the visually impaired, simply cannot use the feature as intended, and either have to rely on another person to capture their likeness accurately, or resign themselves to using an Animoji.
In a patent granted by the US Patent and Trademark Office on Tuesday, Apple 's "Technique for creating avatar from image data" suggests this need not be the case, as one could be produced from a photograph.
In short, the patent uses an image of the user to determine the facial features of the subject, then selects digital representations and compiles them into an avatar.
The processing method measures the spatial facial attributes of the subject, classifying the expression of the person at the same time, and determining which from a selection of avatar templates is the closest match. The template avatars are deformed based on the expression classification to aid matching.
An illustration of how images of a subject can be broken down into facial features and location measurements
Other attributes of the subject are then identified and the selected avatar template is deformed further, before the process is completed an a user's avatar is available to use.
The classification can include the identifying of spatial locations of hair on the subject, then to determine multiple different templates of hair styles which can be attached to the template, as well as for identifying eyeglasses, facial hair, and other elements. Eye, hair, and skin color can also be used as part of the final deformation process.
The data for the compiled avatar could also be transferred into other templates, such as to provide users with variations on an appearance. This can include costume changes or those of expressions, or even the aforementioned Memoji stickers.
Apple files numerous patent filings on a weekly basis, but while the existence of a patent suggests areas of interest for Apple's research and development efforts, they don't guarantee the described concepts will appear in a future product or service.
In this case, as it is an image processing patent that is entirely based in software, and given Apple's existing work with Memoji, it seems plausible some form of automated Memoji creation feature could be made available in the future.
While the patent is more about a flat 2D image rather than using three-dimensional data, the existence of Face ID and the TrueDepth camera array may help Apple further the concept. By using the available 3D mapping capability, Apple has the potential to take detailed measurements of a user's face for avatar creation, with the relative ease potentially allowing users to generate new Memoji whenever they want.
This is not the only way Apple believes it can enhance Memoji and Animoji for its users. One patent application that surfaced in November 2018 suggested the recording of audio with Animoji, but rather than leave the audio plain, Apple proposed switching out spoken key words for sound effects and custom character animations, such as a dog's bark when the user says "woof" to the iPhone.
Apple's Animoji and Memoji features have proven popular, having been used for advertising purposes as well as for social media trends like Animoji Karaoke. In iOS 13, Apple extended Memoji to allow users to create collections of stickers using the user-based character, which can be added to Messages.
Even so, the feature does rely on users going through a process to create their Memoji, which can take a few minutes to accomplish, though to get a perfect likeness it can take quite a while longer. Some users, such as the visually impaired, simply cannot use the feature as intended, and either have to rely on another person to capture their likeness accurately, or resign themselves to using an Animoji.
In a patent granted by the US Patent and Trademark Office on Tuesday, Apple 's "Technique for creating avatar from image data" suggests this need not be the case, as one could be produced from a photograph.
In short, the patent uses an image of the user to determine the facial features of the subject, then selects digital representations and compiles them into an avatar.
The processing method measures the spatial facial attributes of the subject, classifying the expression of the person at the same time, and determining which from a selection of avatar templates is the closest match. The template avatars are deformed based on the expression classification to aid matching.
An illustration of how images of a subject can be broken down into facial features and location measurements
Other attributes of the subject are then identified and the selected avatar template is deformed further, before the process is completed an a user's avatar is available to use.
The classification can include the identifying of spatial locations of hair on the subject, then to determine multiple different templates of hair styles which can be attached to the template, as well as for identifying eyeglasses, facial hair, and other elements. Eye, hair, and skin color can also be used as part of the final deformation process.
The data for the compiled avatar could also be transferred into other templates, such as to provide users with variations on an appearance. This can include costume changes or those of expressions, or even the aforementioned Memoji stickers.
Apple files numerous patent filings on a weekly basis, but while the existence of a patent suggests areas of interest for Apple's research and development efforts, they don't guarantee the described concepts will appear in a future product or service.
In this case, as it is an image processing patent that is entirely based in software, and given Apple's existing work with Memoji, it seems plausible some form of automated Memoji creation feature could be made available in the future.
While the patent is more about a flat 2D image rather than using three-dimensional data, the existence of Face ID and the TrueDepth camera array may help Apple further the concept. By using the available 3D mapping capability, Apple has the potential to take detailed measurements of a user's face for avatar creation, with the relative ease potentially allowing users to generate new Memoji whenever they want.
This is not the only way Apple believes it can enhance Memoji and Animoji for its users. One patent application that surfaced in November 2018 suggested the recording of audio with Animoji, but rather than leave the audio plain, Apple proposed switching out spoken key words for sound effects and custom character animations, such as a dog's bark when the user says "woof" to the iPhone.
Comments
What's the purpose of all of this? My take is that smartphones and social media have allowed us to move away from personal, intimate, and reality based forms of human interaction that exist in meatspace to virtual interactions created behind a personally curated facade that represents how we want to be perceived by other virtual souls that interact in cyberspace. Social media allows us to create the "Virtual Me" and our smartphones and Memoji allow us to carry our Virtual Me around with us so we are never have to be exposed in our actual human form. Just like clothing allows us to cover up our naked human form, the Virtual Me allows us to cover up our outward physical appearance, our true personality, social flaws, and our inherent communicative capabilities and weaknesses into something more palatable for the rest of society to see. In other words, we're simply adding another layer of "clothing" to cover up a broader range of things we're looking to hide. At some point I expect there will be professional design studios that will sell you a much more fashionable Virtual Me version of yourself than what you can come up with on your own.
Not sure who thought it was a good idea to put them in the spot once reserved for oft used emojis, but that person was wrong.
Not sussed enough about it to rant and rave though.
This feature is going to be key to iPhone success.
I truly don’t mean to be negative. I view computing as being a tool for problem solving. That delight for me comes from great software and hardware in concert with each other with clever and extensible features that really make a difference in my life or at least turn that light bulb on. I’m not adverse to Memoji it has a place in our lives as will Apple TV+ but these are essentially diversions.
ToonCamera is just a photo filter app, like a "find edges" Photoshop plugin from the 90's. This is building an actual interactive 3D model that can be used in many different ways. There's literally nothing in common between the two, and the tech required to analyze a photo using ML to build this model is far, far more complex.
I discovered my emoji posting to Seeking Arrangements. Should I be worried?