Unreal Engine releases an app that uses Face ID to capture 3D facial animations

AppleInsider

Unreal Engine's new Live Link Face app brings big-budget movie-style effects to the iPhone, with Face ID able to animate 3D characters in real time or capture face tracking data for later use.

Live Link Face app by Unreal Engine

The company announced the new Live Link Face app on Thursday morning and boasted the usefulness of Face ID's TrueDepth Camera and ARKit for 3D animation capture. The app can be used to capture live performances or be used to animate digital avatars for live streams.

#UE4 has their Live Link Face iOS app available now for free!! You can capture realtime facial data straight into sequencer or save the takes for later. Do what you will with this knowledge #gamedev pic.twitter.com/7CXPh1oCIq

-- Chase Shields (@chaseanimation)

July 8, 2020

The app can intelligently adjust based on how it is being used due to utilizing the motion coprocessors in the iPhone. If the iPhone is attached to the user's head, it will only record facial movements into the data, where if it on a desk it will capture head and neck movements as well.

It is built for professional collaborative environments, which will allow the app to sync up to the Multi-User Editor used in Unreal productions. Other syncing features will ensure timecodes are perfectly in sync with the other recordings by connecting to the stage master clock using bluetooth.

Live Link Face app being used for live motion capture

Addy Ghani, Director of Animation Technology at Verizon Media praised the app's usefulness.

"The Live Link Face app harnesses the amazing facial capture quality of iPhone ARKit and turns it into a streamlined production tool. At RYOT we believe in the democratization of capture technology and real time content and this solution is perfect for a creator at home or a professional studio team like ours."

Unreal Engine is known for its cross platform gaming engine used by developers to make games on iOS, macOS, and game consoles. The technology has also been used to create digital movie sets for media like Disney's "The Mandalorian."

The Live Link Face app is available on the App Store for free with no in-app purchases. The app needs to be connected to professional software used in Unreal Engine production to utilize the data.

mjtomlin

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

SpamSandwich

Very cool tool.

Rayz2016

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

As soon as I read the headline I knew AI had got it wrong again. 🙄

JinTech

Rayz2016 said:

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

As soon as I read the headline I knew AI had got it wrong again. 🙄

Love AI but they really need to stop with the click bate. At this point, they already have a pretty large audience and we all know better.

svanstrom

JinTech said:

Rayz2016 said:

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

As soon as I read the headline I knew AI had got it wrong again. 🙄

Love AI but they really need to stop with the click bate. At this point, they already have a pretty large audience and we all know better.

Problem might just be that the ones writing the articles don't always know better.

JinTech

svanstrom said:

JinTech said:

Rayz2016 said:

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

As soon as I read the headline I knew AI had got it wrong again. ߙ䦬t;/div>

Love AI but they really need to stop with the click bate. At this point, they already have a pretty large audience and we all know better.

Problem might just be that the ones writing the articles don't always know better.

This is true though AI as a whole probably have rough guidelines on how to write articles, title them, etc. They are indeed owned by Quiller Media, Inc.

Wesley Hilliard

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

Thank you for pointing this out. We are aware that Face ID and TrueDepth are different things, but the title needs to be informative and approachable to everyone. Not everyone knows what TrueDepth means or if their phone has it, but they will know what Face ID is. We edited the text to disambiguate the two for those who care.

If a title is too technical, people are less likely to care. It’s like referring to a boat as a ship, you’re technically wrong, but not everyone knows that water displacement determines what you call a boat, and pointing the difference out might be more annoying than informative to the offender.

The person wondering if people write these articles without knowledge of the items they are writing is concerning though. Our business wouldn’t last if the writers and editors didn’t have prior knowledge of the topics they discuss... food for thought.

svanstrom

Wesley Hilliard said:

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

Thank you for pointing this out. We are aware that Face ID and TrueDepth are different things, but the title needs to be informative and approachable to everyone. Not everyone knows what TrueDepth means or if their phone has it, but they will know what Face ID is. We edited the text to disambiguate the two for those who care.

If a title is too technical, people are less likely to care. It’s like referring to a boat as a ship, you’re technically wrong, but not everyone knows that water displacement determines what you call a boat, and pointing the difference out might be more annoying than informative to the offender.

The person wondering if people write these articles without knowledge of the items they are writing is concerning though. Our business wouldn’t last if the writers and editors didn’t have prior knowledge of the topics they discuss... food for thought.

I've long ago just assumed that what's published at AI is based on facts, but always at the risk of having been written by someone actually completely clueless about the subject (writers just rewriting press releases, "articles" only to include sponsored links, rewriting articles from other sources, and so on); so I always expect to have to "adjust" what's written here to get the truth. But, that we're still talking about minor details rather than something taken to the point where it's sort of too far from the actual facts.

Saying that your articles become more approachable by replacing "TrueDepth" with "Face ID" is a bit worrying for me, though, because it makes me seriously wonder about the judgment of the people writing the articles; to the point of wondering if texts might be rewritten to the point where they start distorting the facts way too much.

Look at these:

1. Unreal Engine releases an app that uses Face ID to capture 3D facial animations
2. Unreal Engine releases an app that uses TrueDepth to capture 3D facial animations
   

Is #1 really such a revolutionary leap in approachability that it's worth mislabeling the technology, which perpetuates having to dumb down all future articles to fit your established baseline of simplicity?!

If #1 really is required to attract people to read the article then they have zero interest of the actual subject (which is "3D facial animations"), and were just triggered by the use of "Face ID"; and that will not only get them reading an article not quite for them, but they will also make erroneous assumptions about what the technology actually does… like whether or not the 3D animations represent the person behind an avatar having a verified identity, or if this means that Face ID is available to third party developers to without notification track and identify all people infront of the cameras. And so on.

So instead of long-term educating your audience clicks have become your ruling guideline for how to write articles; and if that's the measured performance of what you publish, then how you rework information/truths is way too malleable for my liking.

Then, of course… maybe I'm just the odd one that don't fit in here (or anywhere   ). 

dk49

Downloaded the app. But not sure how to get the animation out of it? Recording shows the normal video, not the animation.

beowulfschmidt

The people writing the articles might not be the same people authoring the titles, or might not have the final say in the title.

fastasleep

dk49 said:

Downloaded the app. But not sure how to get the animation out of it? Recording shows the normal video, not the animation.

Uhh, you connect it to Unreal Editor. If you don't use that, this app is not for you. The playback is limited to the video of each take; it doesn't display the mesh.

https://docs.unrealengine.com/en-US/Engine/Animation/FacialRecordingiPhone/index.html

mr lizard

Wesley Hilliard said:

mjtomlin said:

Technically speaking, they are not actually using Face ID, they're using the True Depth camera system with ARKit to capture depth and facial landmark location data. Face ID is the iPhone's security authorization API, which also uses the True Depth camera for facial recognition.

Thank you for pointing this out. We are aware that Face ID and TrueDepth are different things, but the title needs to be informative and approachable to everyone. Not everyone knows what TrueDepth means or if their phone has it, but they will know what Face ID is. We edited the text to disambiguate the two for those who care.

If a title is too technical, people are less likely to care. It’s like referring to a boat as a ship, you’re technically wrong, but not everyone knows that water displacement determines what you call a boat, and pointing the difference out might be more annoying than informative to the offender.

The person wondering if people write these articles without knowledge of the items they are writing is concerning though. Our business wouldn’t last if the writers and editors didn’t have prior knowledge of the topics they discuss... food for thought.

Wesley, the problem is your post title might make people distrust Face ID as it implies that developers can tap into their face data, which Apple goes to great lengths to stress is kept absolutely secure and not shared with apps. You could have just said “front facing camera”.

applesincider

I, too, found the obvious misuse of the term “Face ID” very distracting. 

As mentioned in the first comment, Face ID is an authentication mechanism, and the app and article simply have nothing to do with authentication. 

Unreal Engine releases an app that uses Face ID to capture 3D facial animations

No — that’s factually incorrect, not just technically speaking, but in product, feature, and branding terms as well. 

If you’re that worried about it, a good editor would say it’s unnecessary, anyway. 

Unreal Engine releases app that captures facial animations

Done. That would be good. 

Then later:

… with Face ID able to animate 3D characters in real time…

Again, just factually incorrect. Face ID is not involved here. Face ID and ARKit both use the TrueDepth camera, that’s the point of intersection. 

If you’re that worried about saying “TrueDepth”, then “front-facing camera” or simply “camera” would be less specific but at least still 100% accurate.

And again later:

The company[…] boasted the usefulness of Face ID's TrueDepth Camera…

Again, no. You haven’t disambiguated anything, you’ve added more confusion to the mix. It isn’t “Face ID’s TrueDepth Camera”. 

It’s the iPhone’s/iPad’s TrueDepth Camera, which Face ID employs alongside other technologies for one use case of the camera’s technology. There are other use cases, like ARKit and the face tracking of this very app. It would be a great opportunity to educate your readership by explaining that. 

Instead, it’s like you were determined to get it exactly wrong in each instance. 

Now we know you know better, so now we know you can do better. Please?

svanstrom

beowulfschmidt said:

The people writing the articles might not be the same people authoring the titles, or might not have the final say in the title.

Sure, but someone somewhere is in charge, and no matter if it's all done by one person or a team, the problems still persist; and lots of people over on their side should be able to spot the problems, or else they're in deep trouble.

To screw things up even more they've now added some sort of click-tracker among the comments so that links can't be cmd-opened in new tabs.

Rn I'm still at AI simply because I've been too lazy to check at what level their competition is at today.