Apple's Craig Federighi details Face ID in new interview [u]

Posted:
in iPhone edited September 2017
In a brief interview on Friday, Apple SVP of Software Engineering Craig Federighi answered a few questions about the Face ID facial recognition feature that will ship out to customers in November, dispelling some of the more contentious rumors, providing background on its development and touting the system's benefits.


Craig Federighi demonstrates Face ID on iPhone X.


Federighi spoke with TechCrunch to clear up some questions and concerns about Apple's biometric authentication solution that have cropped up in the days following Face ID's debut and first public demonstration on Tuesday.

Starting with Face ID's data backbone, Federighi said Apple collected a "quite exhaustive" library of face scans from consenting subjects that was subsequently used to train the system. To maximize accuracy, Apple captured high quality depth maps of each person's face from various angles.

"Phil mentioned that we'd gathered a billion images and that we'd done data gathering around the globe to make sure that we had broad geographic and ethnic data sets. Both for testing and validation for great recognition rates," Federighi said. "That wasn't just something you could go pull of [sic] the internet."

He goes on to say that Apple is holding on to the high fidelity depth map data for further algorithm training, noting the dataset is needed for system optimization. As can be expected of a valuable, and potentially sensitive, asset, Apple's facial scan library is protected.

The same can be said about user data. Media outlets, pundits and potential buyers have raised concerns over what Apple does with data generated from a Face ID scan. Some have suggested data is sent to the cloud for processing, while others pondered whether Apple keeps a repository of scans to train its neural network.

"We do not gather customer data when you enroll in Face ID, it stays on your device, we do not send it to the cloud for training data," Federighi said.

Just like Touch ID, Apple does not retain Face ID data, that information lives in the iPhone X Secure Enclave. Also like Touch ID, raw Face ID image data is processed and stored as a mathematical model that cannot be reverse engineered back into "model of a face."

Developers are also not privy to facial scan data through ARKit. Instead, apps leveraging the API are granted access to a depth map, not raw sensor data, that can be applied to photographic effects and other software features, Federighi said.

Further, TrueDepth's dot projector and infrared receiver are designed to work only at short distances, a hardware limitation noted by industry analysts in the weeks leading up to iPhone X's unveiling. Extrapolating this information out into the real world, it can be assumed that Apple's phone is not capable of scanning faces of passersby.

The executive notes iPhone X boasts a redesigned Secure Enclave capable of performing the re-training procedures that power Face ID's adaptive features. Mentioned during Tuesday's keynote, Face ID is able to adapt to changes in a user's face. For example, the system will work when a user grows a beard, wears a hat or puts on glasses. As Federighi noted in an email to a curious customer, Face ID even works with most sunglasses.

There are limitations to the system's capabilities. Face ID will be unable to recognize a user whose face is obscured by a mask or niqab, the report says. In such cases, users can opt to enter a passcode instead. A user passcode is also required after five failed attempts or if a user has not unlocked their phone with Face ID in 48 hours. There are also idle time limits, meaning passcode unlock is required if an iPhone X has not been unlocked in 6.5 days and Face ID was not used to unlock the device within 4 hours.

Federighi earlier this week noted users can manually disable Face ID by gripping iPhone X's side buttons, a feature that could come in handy when a user is forced to hand over the device unwillingly. The executive confirmed to TechCrunch that the feature works by depressing either volume button and the sleep/wake button. The squeeze to passcode feature is also available on iPhone 8.

"So, if you were in a case where the thief was asking to hand over your phone -- you can just reach into your pocket, squeeze it, and it will disable Face ID," he said. "It will do the same thing on iPhone 8 to disable Touch ID."

From Federighi's description, Face ID appears to be extremely flexible insomuch as users need not worry about holding iPhone X at exact distance on a specific angle.

"It's quite similar to the ranges you'd be at if you put your phone in front facing camera mode [to take a picture]," he said. "Once your space from eyes to mouth come into view that would be the matching range - it can work at fairly extreme angles -- if it's down low because your phone is in your lap it can unlock it as long as it can see those features. Basically, If you're using your phone across a natural series of angles it can unlock it."

Users can also turn off attention detection, one of the safeguards built into Face ID to protect against errant unlocking. When activated, attention detection requires users to look at iPhone X in order to trigger the authentication procedure. This might not be possible for people with certain conditions, for example blind users.

Alternatively, Face ID can be disabled altogether, leaving users to unlock their device with a passcode.

As for usability, Federighi says Face ID was developed using face data collected from a broad spectrum of geographic locations, ages and ethnic backgrounds. In other words, the system should function normally, no matter your cultural background.

A day after Face ID was unveiled, U.S. Senator Al Franken penned a letter to CEO Tim Cook requesting additional information on the system, including questions touching on data acquisition and retention. Aside from answers already available in Apple's PR and website, Federighi's answers today address many of Franken's concerns.

Finally, Federighi said Apple intends to release a white paper on Face ID for security researchers and other interested in learning more about the cutting-edge facial recognition technology.

Update: Apple has provided AppleInsider with additional commentary on Face ID:
Our teams have been developing the technologies behind Face ID for several years, and our users' privacy has been a priority since the very beginning.

Face ID provides intuitive and secure authentication enabled by the TrueDepth camera system and the A11 Bionic chip, which uses advanced technologies to accurately map and match the geometry of a user's face. Face ID data never leaves the device, is encrypted and protected by the Secure Enclave.

We've tested Face ID on people from many countries, cultures, races and ethnicities, using over one billion images to train our neural networks and defend against spoofing.

We're confident that our customers will love using the feature and find it an easy and natural way to unlock their iPhone X. We will offer more details on Face ID as we near the product's availability.
«13

Comments

  • Reply 1 of 44
    There are always concerns raised about any new security technology. Remember, "what if thieves cut off my fingers" or "could I use this/that body part" or numerous such silly questions with TouchID. Yet, it was a remarkable innovation. 

    I have little doubt that Apple knows what it's going. And I trust Apple to protect my security and privacy more than any other organization on earth, whether in business, government, or the non-profit sector. End of story.
    edited September 2017 macxpresseightzeroStrangeDaysnetroxradarthekatiqatedoslprescottcalilostkiwilolliver
  • Reply 2 of 44
    Using a picture of Craig using Snapchat and saying it's Face ID is confusing.  In the pic, Snapchat is really using an API built atop FaceID.
    edited September 2017 lolliverjony0
  • Reply 3 of 44
    There are always concerns raised about any new security technology. Remember, "what if thieves cut off my fingers" or "could I use this/that body part" or numerous such silly questions with TouchID. Yet, it was a remarkable innovation. 

    I have little doubt that Apple knows what it's going. And I trust Apple to protect my security and privacy more than any other organization on earth, whether in business, government, or the non-profit sector. End of story.
    Right on.

    This info is interesting. So cool.
    calilostkiwilollivercornchip
  • Reply 4 of 44
    Infrared can be washed out by bright ambient light. Does FaceID have any limitations with respect to bright sunlight or bright room light?
    cornchip
  • Reply 5 of 44
    Hopefully this ease some of those suffering great panic. Yet I see on MR they are still having fits over it and this article.
    watto_cobra
  • Reply 6 of 44
    Hopefully this ease some of those suffering great panic. Yet I see on MR they are still having fits over it and this article.

    It's MacRumors what do you expect? Its amazing how a community that represents something AI also represents (Apple users) can be so different. 
    calilolliverwatto_cobracornchipjony0
  • Reply 7 of 44
    Every new tech that works takes sometimes for non tech savy to get used to them....even my wife still used pass codes, she ain't got nothing to do with touch ID or 3D touch, she said....hmmmm
    watto_cobra
  • Reply 8 of 44
    GBannis said:
    Infrared can be washed out by bright ambient light. Does FaceID have any limitations with respect to bright sunlight or bright room light?
    depends on the part of the spectrum.
    cornchip
  • Reply 9 of 44
    I'd like a personal apology from Craig for letting the entire world catch him in an onstage gaff. He should have realized any action that might be construed as some feature failure would be multiplied by a gazillion times on the internet and news media. People are so eager to know that Apple has made some mistake, and now Apple must suffer huge amounts of humiliation. When you sell a $1000 smartphone, it must be absolutely perfect no matter what. Heck, no one would care if it was a $200 Android smartphone because that's expected to get screwed up. But a failure on a $1000 smartphone is just too much to bear. I demand an apology or he must immediately commit hara-kiri using an iPhone sim-tool. /s
    flyingdpMintzjony0anton zuykov
  • Reply 10 of 44
    jd_in_sbjd_in_sb Posts: 1,600member
    Al Franken penned a letter with questions that were already answered in the keynote. 
    radarthekatbloggerblogcgWerkscalibb-15SpamSandwichwatto_cobracornchipjony0
  • Reply 11 of 44
    MplsPMplsP Posts: 3,911member
    Hopefully this ease some of those suffering great panic. Yet I see on MR they are still having fits over it and this article.

    Well, it is MacRUMORS....

     From the article: "passcode unlock is required if an iPhone X has not been unlocked in 6.5 days and Face ID was not used to unlock the device within 4 hours." If it requires a passcode if face ID hasn't been used in 4 hours, what's the point of the 6.5 day limit? (and why 6.5 days?)
    edited September 2017 cali
  • Reply 12 of 44
    JinTechJinTech Posts: 1,020member
    I'd like a personal apology from Craig for letting the entire world catch him in an onstage gaff. He should have realized any action that might be construed as some feature failure would be multiplied by a gazillion times on the internet and news media. People are so eager to know that Apple has made some mistake, and now Apple must suffer huge amounts of humiliation. When you sell a $1000 smartphone, it must be absolutely perfect no matter what. Heck, no one would care if it was a $200 Android smartphone because that's expected to get screwed up. But a failure on a $1000 smartphone is just too much to bear. I demand an apology or he must immediately commit hara-kiri using an iPhone sim-tool. /s
    It's people that keep posting fud like this that keeps the memory alive. Apple has already come out and publicly stated what the problem was and that it was not a failure of the product and that it actually worked as advertised. Time to let it go. 

    Edit: Just realized there was a sarcasm tag but the fact that this post was long enough made me think otherwise and I just stopped reading lol
    edited September 2017 randominternetpersonwatto_cobra
  • Reply 13 of 44
    GBannis said:
    Infrared can be washed out by bright ambient light. Does FaceID have any limitations with respect to bright sunlight or bright room light?
    850nm works well indoor in any type of lighting conditions. 850nm was used by Kinect 1 and worked well in bright lit rooms with good coverage at a large room. 

    940nm works well outside in bright sunshine. At 940nm, there is a notch caused by the Earth's atmosphere that atsorbs this particular wavelength. Thus, 940nm works well under bright sunshine. It also works well indoors since lights do not typically produce much signal at these wavelengths. 
  • Reply 14 of 44
    cgWerkscgWerks Posts: 2,952member
    AppleInsider said:
    ... There are limitations to the system's capabilities. Face ID will be unable to recognize a user whose face is obscured by a mask or niqab, the report says. In such cases, users can opt to enter a passcode instead. ...
    I wonder if there will be any SJW groups taking to the streets over Apple's obvious Islamophobia. ;) (And, what the heck are bank-robbers to do?)
    edited September 2017
  • Reply 15 of 44
    haarhaar Posts: 563member
    cgWerks said:
    AppleInsider said:
    ... There are limitations to the system's capabilities. Face ID will be unable to recognize a user whose face is obscured by a mask or niqab, the report says. In such cases, users can opt to enter a passcode instead. ...
    I wonder if there will be any SJW groups taking to the streets over Apple's obvious Islamophobia. ;) (And, what the heck are bank-robbers to do?)

    Correct me... a niqab (or a neck covering head scarf) EXPOSES the face ... so faceid requires a neck? ... REALLY DOUBT THAT...  otherwise sounds like niqab really means burka...

  • Reply 16 of 44
    jd_in_sb said:
    Al Franken penned a letter with questions that were already answered in the keynote. 
    I think a lot of people didn't actually watch the keynote...they just read headlines and made assumptions. Yes, they look like idiots. Its a shame some have to work in government. I guess its too hard to sometimes for these people to engage their brain, before their mouths. 
  • Reply 17 of 44
    haar said:
    cgWerks said:
    AppleInsider said:
    ... There are limitations to the system's capabilities. Face ID will be unable to recognize a user whose face is obscured by a mask or niqab, the report says. In such cases, users can opt to enter a passcode instead. ...
    I wonder if there will be any SJW groups taking to the streets over Apple's obvious Islamophobia. ;) (And, what the heck are bank-robbers to do?)

    Correct me... a niqab (or a neck covering head scarf) EXPOSES the face ... so faceid requires a neck? ... REALLY DOUBT THAT...  otherwise sounds like niqab really means burka...

    Just eyes, nose and mouth must be open, the rest may be covered (for the Face ID, I mean ;-)). On the other hand, people with niqabs and burkas don't wear these all the time, these are only for outside, in their personal living space those people can wear anything. So their dressing code won't have any effect on their use of Face ID thus on their purchase of X.
    edited September 2017
  • Reply 18 of 44
    cgWerkscgWerks Posts: 2,952member
    haar said:
    Correct me... a niqab (or a neck covering head scarf) EXPOSES the face ... so faceid requires a neck? ... REALLY DOUBT THAT...  otherwise sounds like niqab really means burka...
    While I know a reasonable amount about Islam, I'm not great on all the terms. But, no, I think they actually got that one right.
    I was just joking though, about how anyone with a cause seems to have the ability to misuse terms to make a fuss. :) (i.e.: if you don't agree with and support my cause, you're something-phobic)
  • Reply 19 of 44
    I still want to know the answer to Franken's question: Where did Apple get a billion faces from exactly? 'Federighi said. "That wasn't just something you could go pull off the internet."' Ya, no kidding.
    edited September 2017
  • Reply 20 of 44
    jgfitzjgfitz Posts: 1unconfirmed, member
    I wonder if sometime in the future a third party could develop a more powerful scanner that could operate at a greater range that could mimic the apple scan and get an equivalent dataset for a face.     If so, what would be the implications?
Sign In or Register to comment.