Apple looks to the future of video conferencing with Memoji avatars

Posted:
in General Discussion edited May 2020
Instead of every meeting attendee staring at a flat Zoom screen, Apple is looking to the future of video conferencing with Memoji-style avatars arranged in augmented reality around each meeting attendee.

We're all used to Zoom-style meetings, but Apple wants to replace them with  avatars -- but also arranged in AR.
We're all used to Zoom-style meetings, but Apple wants to replace them with avatars -- but also arranged in AR.


Forget video conferencing through Zoom, Apple has been looking at a way of using virtual and augmented reality to stage meetings in 3D space. Each attendee can be at their own table and "see" the others arranged around them. It's not quite like having a "Star Wars" type of hologram of the person, but through screens or glasses, you'd be able to see avatars that move and interact.

The application, "Communication system and method for providing a bionic virtual meeting room," couches the idea in typical highly detailed legalese, but the concepts behind it are fascinating.

"The invention relates to a method for operating a communication system for providing a virtual meeting of at least one first user and at least one second user," starts the patent. "The invention relates to... displaying a virtual meeting space on a display device [and] detecting a biometric characteristic of a first user.

"[It involves] determining a level of abstraction associated with the first user from among a plurality of different settable levels of abstraction," it continues, "and displaying, in the virtual meeting space, an avatar representing the first user having the biometric characteristic of the first user."

"This way, objects, especially virtual objects, like an avatar representing the first user, can be presented to the second user over any distance, which can advantageously be used to provide virtual meetings between different users, especially any arbitrary number of users, in a common virtual meeting room," says the patent.

Think of the way that Animoji and Memoji present an animated face that shows the same mouth and head movements that the user is doing. The patent stresses that "providing a biometric characteristic" of each user is not just for fun, it is for directly improving communication.

"The main area of application of such virtual realities lies in the field of entertainment and games," admits the patent. "But it would be desirable to make use of such virtual realities also in other fields, especially in the field of human interaction."

Detail from a patent drawing describing the devices that could be used to show these meeting avatars
Detail from a patent drawing describing the devices that could be used to show these meeting avatars


"Such adaptions by biometric characteristics of the first user are especially important for human interaction," it says. "For example the addition of gaze directions, eye blinking or mimics as biometric characteristics to virtual characters in the virtual meeting space can greatly improve empathy and perception of the person."

"In many situations, especially in international, professional or other intercultural contact it has been determined as useful to modulate and moderate individual behavior and appearance to either tone it down, bias or amplify it to support the purpose of the interaction and communication," continues the patent.

If this feels like just one step below having holograms of people in the same meeting room, it does still require a method of displaying these avatars to each attendee. That means the patent goes to lengths to describe how such avatars could be shown on devices.

These could be "computers, mobile virtual reality or augmented reality glasses, tablets or smart phones" and the specifics of how each avatar is displayed varies between the devices. Consistently, however, the patent is concerned with the fidelity of presenting a human's characteristics through these avatars.

The invention is credited to Eberhard Schmidt, who has previously been listed on patents regarding display apparatus and presenting images on display devices.

This patent does not concern itself with the details of how an avatar may actually be created or chosen by the user, just with how it can react and display their characteristics. A previous Apple patent, however, has described how a future iPhone may be able to generate a user's Memoji automatically from a photograph.

Comments

  • Reply 1 of 14
    MplsPMplsP Posts: 3,356member
    Before we get 'personalized augmented reality Memoji' meetings, how about we get meetings with audio that actually works? Between delays in the audio and the conference call systems automatically muting people when someone else is talking, conference calls are frequently an exercise in frustration. Video is nice, but Audio is essential. Augmented Reality? whatever - I chalk this up to another solution looking for a problem.
    SpamSandwich
  • Reply 2 of 14
    KidAKidBKidAKidB Posts: 19member
    What a joke. This is fine for children, but which professional setting would ever accept such a thing? This is not what anyone is asking for, but I guess Apple is showing once again that they have no idea what a professional would want.
  • Reply 3 of 14
    nceencee Posts: 856member
    And simply letting folks post to a bulletin /web site, what they'd like to see, see fixed, see changed, wouldn't be a bad thing. YES theist would be long, but if there were enough serious suggestions, at least then Apple would know it would be worth checking out.

    I'm serene of the many issues and reasons why this would never happen, is the large number of trolls wo=ho would than insist of a piece of the profits from their ideas and suggestions.
  • Reply 4 of 14
    KidAKidB said:
    What a joke. This is fine for children, but which professional setting would ever accept such a thing? This is not what anyone is asking for, but I guess Apple is showing once again that they have no idea what a professional would want.
    Many of my “professional” business partners from global enterprises (some with a “C” in front of their title) use Zoom backdrops with all kinds of funny pictures. So, I “seriously” doubt your assumption is right.
    bonobobmknelsonlolliverchasm
  • Reply 5 of 14
    nceencee Posts: 856member
    Damn spell check "I'm sure" some of the many reasons …
  • Reply 6 of 14
    rob53rob53 Posts: 2,679member
    MplsP said:
    Before we get 'personalized augmented reality Memoji' meetings, how about we get meetings with audio that actually works? Between delays in the audio and the conference call systems automatically muting people when someone else is talking, conference calls are frequently an exercise in frustration. Video is nice, but Audio is essential. Augmented Reality? whatever - I chalk this up to another solution looking for a problem.
    Agree. I'd much rather see Apple work on a multi-platform version of FaceTime so I don't have to keep telling my family I'm never going to use Zoom (we have mainly iPhone users but also a few Android users). I got one of them to use WebEx but it was choppy. Group FaceTime can handle 32+1 users, which is plenty for the majority of users so just figure out a way to get a full or semi-restricted version of FaceTime to Windows and Android. I know Apple thinks this will hurt iPhone sales but it would make my life a whole lot easier. I still have problems sending an image along with text in the same message to Android users. Come on Apple, think about something useful for everyone, not just a gimmick. 
  • Reply 7 of 14
    bonobobbonobob Posts: 309member
    Quote:
    Forget video conferencing through Zoom
    Forget video conferencing through FaceTime.  Apple can add all the cute and/or useful features to FaceTime that they can possibly ever think of, and I'll still be using Zoom because FaceTime doesn't work on Android or Windows, and not everyone in my social circles is in the Apple ecosystem.
  • Reply 8 of 14
    bonobobbonobob Posts: 309member
    ncee said:
    Damn spell check "I'm sure" some of the many reasons …
    Did you know that you could have edited your post to fix the error for up to four hours after posting it (which means you still can do so as of this writing)?
  • Reply 9 of 14
    gutengelgutengel Posts: 363member
    Well, I cant forget about zoom because apparently you cant do a group call if you are on a Facetime voice call. I cant forget about zoom because when I want to start a facetime call sometimes it calls an old phone number on the contact and you end up talking to a random person. I cant forget about zoom because sometimes the FT call goes just to your iPhone instead of your other Apple devices. I cant forget about zoom because when someone on the group doesn't answer or hang up the screen stays awkwardly divided. I cant forget about zoom because a lot of times the call doesn't ring on the other end and it doesn't notify the other person that I called them. Apple really needs to improve and simplify their iMessage, Facetime and phone integration, because its simply broken. I like that everything is encrypted & secure, but as of now, I only use Facetime audio to do 1:1 calls because it's what mostly works.
    minicoffee
  • Reply 10 of 14
    KidAKidBKidAKidB Posts: 19member
    KidAKidB said:
    What a joke. This is fine for children, but which professional setting would ever accept such a thing? This is not what anyone is asking for, but I guess Apple is showing once again that they have no idea what a professional would want.
    Many of my “professional” business partners from global enterprises (some with a “C” in front of their title) use Zoom backdrops with all kinds of funny pictures. So, I “seriously” doubt your assumption is right.
    There's a difference between a backdrop and a memoji. It's not even remotely the same thing and I can guarantee you that your C level partners won't be using them.
  • Reply 11 of 14
    tuckerjjtuckerjj Posts: 30member
    KidAKidB said:
    What a joke. This is fine for children, but which professional setting would ever accept such a thing? This is not what anyone is asking for, but I guess Apple is showing once again that they have no idea what a professional would want.
    Once AR glasses become a thing this might be necessary for FaceTime calls because short of a tiny camera drone flying in front of your face it won’t be possible to get a picture of you.
  • Reply 12 of 14
    DAalsethDAalseth Posts: 1,767member
    If someone invites me to a conference call and shows up with one on these Animoji faces, I’m just ending the call right then. If you don’t want to show me your real face when I’m talking to you, we’ve got nothing to talk about.
  • Reply 13 of 14
    dyonoctisdyonoctis Posts: 48member
    I don't get it. i understand that it's impressive on a technical stand point, but that seems more in line with the like of Facebook or snapchat. I don't see why Apple is spending so much energy on this. There must be a forest that I can't see behind that tree.
  • Reply 14 of 14
    beowulfschmidtbeowulfschmidt Posts: 1,453member
    The Robots of Dawn (and numerous others).

    Prior art. 😜
Sign In or Register to comment.