Future iPhones may unlock, hide messages based on a user's face

Posted:
in General Discussion edited January 2014
Apple on Tuesday was granted a patent that uses facial recognition technology to control a computing device, like an iPhone, iPad or Mac, allowing for a more secure and productive operating environment.

Face
Source: USPTO


The U.S. Patent and Trademark Office awarded Apple U.S. Patent No. 8,600,120 for "Personal computing device control using face detection and recognition," a property that looks to beef up device security as well as make the computing experience more convenient for users.

As noted in the document, face detection and recognition are two different processes. Detection involves locating faces within an image, while recognition goes deeper by pairing those faces with a particular person or user. Typically, facial recognition follows detection.

Apple's patent comprises three separate systems: a face detection decision application; a face recognition application; and an input/output control application. These systems work together to detect whether a user is authorized to operate a given device, and if so, activate certain functions within the computing environment.

Face


In practice, the detection application can scan a defined area where a user is expected to appear, using knowledge-based, feature-based or template matching techniques to identify faces. A number of features are taken into consideration with the matching techniques, including relationships of facial features, identification of facial structures like skin tone, shape, and skin texture, and encoded learning models.

As for facial recognition, at least one embodiment compares face feature vector data, gathered from an image output by an on-board camera, with stored vector data. These vectors can range from face shapes to distance between facial features, such as eyes and nose.

Using the vector data, the system determines whether a user is authorized to operate the device and controls data input/output based on this information.

For example, during an incoming phone call, an iPhone may be able to "sense" that someone is looking at the device's screen. If the person is not an authorized user, the iPhone's screen remains off and only a ringtone or vibration alert is provided. If the person is an authorized user, the usual incoming call UI is displayed.

In another example, an incoming email can be blocked from view unless the authorized user's face is detected by the system. This implementation would be helpful for users who choose to display part of the message in the lock screen.

Face


The document describes another use scenario in which a user can be monitored while positioned in front of a computer. As long as the user's face is recognized over a set period of time, the system can activate or inhibit various device functions like screen savers, video settings, audio settings and communications settings, among others.

The patent goes on to describe in detail the facial recognition application, including a unique algorithm that uses face pattern recognition to "learn" a user's face.

It is unknown if or when Apple plans to implement the technology into its computers or mobile devices, but facial recognition is quickly gaining momentum in the consumer electronics industry. The new Xbox One and PlayStation 4 gaming consoles tout the feature for user logins, while rival smartphone makers have toyed with the tech to varying degrees of success.

Face


Most recently, Apple purchased PrimeSense, an Israeli motion-sensing hardware and software firm responsible for the technology behind Microsoft's first Kinect sensor. While the system described in Tuesday's patent is primitive compared to PrimeSense's applications, the property could be a small part of a larger initiative focusing on novel forms of user input.

Apple's face recognition patent was first filed for in 2008 and credits Jeff Gonion and Duncan Robert Kerr as its inventors.

Comments

  • Reply 1 of 16
    Ok, let's hear the trolls.
    ------------------------------
    On a serious note. Here is where Apple triumphs: implementation. I'm sure it'll be awesome once Apple gets it together. I'm sure it'll be better than the half-cooked version of face recognition I have tried in other devices. It is so lame that you can unlock a device just by using another phone to show the device a picture of the user to unlock it. I'm really curious to see if or how Apple would address that huge issue. Because believe me, if all someone needs to bypass this feature is a picture of you, well...
  • Reply 2 of 16
    mjtomlinmjtomlin Posts: 2,490member
    eckergus wrote: »
    Ok, let's hear the trolls.
    On a serious note. Here is where Apple triumphs: implementation. I'm sure it'll be awesome once Apple gets it together. I'm sure it'll be better than the half-cooked version of face recognition I have tried in other devices. It is so lame that you can unlock a device just by using another phone to show the device a picture of the user to unlock it. I'm really curious to see if or how Apple would address that huge issue. Because believe me, if all someone needs to bypass this feature is a picture of you, well...

    This is where PrimeSense's spacial mapping comes into play... A photo of a face would appear as a flat surface and wouldn't count as a real face.
  • Reply 3 of 16
    rogifanrogifan Posts: 10,669member
    Hmm,,,,this patent was first filed in 2008. Why did it take so long to get approved?
  • Reply 4 of 16
    Quote:

    Originally Posted by eckergus View Post



    Ok, let's hear the trolls.

    On a serious note. Here is where Apple triumphs: implementation. I'm sure it'll be awesome once Apple gets it together. I'm sure it'll be better than the half-cooked version of face recognition I have tried in other devices. It is so lame that you can unlock a device just by using another phone to show the device a picture of the user to unlock it. I'm really curious to see if or how Apple would address that huge issue. Because believe me, if all someone needs to bypass this feature is a picture of you, well...

     

    They will likely address that issue the same way it was already (quickly) addressed in Android.  Face unlock requires the user to blink.

  • Reply 5 of 16
    MacProMacPro Posts: 19,426member
    eckergus wrote: »
    Ok, let's hear the trolls.
    On a serious note. Here is where Apple triumphs: implementation. I'm sure it'll be awesome once Apple gets it together. I'm sure it'll be better than the half-cooked version of face recognition I have tried in other devices. It is so lame that you can unlock a device just by using another phone to show the device a picture of the user to unlock it. I'm really curious to see if or how Apple would address that huge issue. Because believe me, if all someone needs to bypass this feature is a picture of you, well...

    Detecting 2D from 3D is probably not too hard, identical evil twin on the other hand .....
  • Reply 6 of 16
    solipsismxsolipsismx Posts: 19,566member
    nexusphan wrote: »
    They will likely address that issue the same way it was already (quickly) addressed in Android.  Face unlock requires the user to blink.

    Don't blink. Blink and you're dead. They are fast. Faster than you can believe. Don't turn your back. Don't look away. And don't blink.
  • Reply 7 of 16
    takeotakeo Posts: 428member
    They could combine this with the fingerprint ID too.
  • Reply 8 of 16
    paxmanpaxman Posts: 4,678member
    Without checking the image that comes to mind is of dynasaurs. (?)
  • Reply 9 of 16
    jupiteronejupiterone Posts: 1,564member
    Detecting 2D from 3D is probably not too hard, identical evil twin on the other hand .....

    Nah, the brunette hair (Jeannie-Jeannie/Samantha-Serena) or the goatee (Spock) will give it away.
  • Reply 10 of 16
    tbelltbell Posts: 3,146member
    Quote:
    Originally Posted by mjtomlin View Post





    This is where PrimeSense's spacial mapping comes into play... A photo of a face would appear as a flat surface and wouldn't count as a real face.

     

    Agreed. The software will make a 3D map of a users face. I suspect this is years away first. I also doubt Apple would use both fingerprint recognition and facial ID. 

  • Reply 11 of 16
    Quote:

    Originally Posted by digitalclips View Post





    Detecting 2D from 3D is probably not too hard, identical evil twin on the other hand .....

    Again,   the main(probabilistically) threat vectors are someone you know anyway (Wives, husbands, mothers

    Fathers, sisters and brothers), with apologies to Alabama 3*).  Then people you let near your phone (waiters, co-workers, etc.)  

     

    If they can get the 3D portion of it down (or verify analog movement), this makes for a nice 'notification shield'.  e.g. I'd love to show my messages on my phone when the lock screen is on, but corporate policy [and client sensitivity] precludes it.   Now, you can 'ping' me [message arrived] and based on my face when I look the phone can decide whether or not to show the message, or require a password [or both].  But like TouchID, if I can record my wifes' face so she can log into my account, or even allow her to access her own account on my iOS device, then the appropriate level of security is there, and it's removes 2 or 3 major steps of 'identification'

     

    And this would be good.... because I see iPads being home shared devices, as well as iPods, and to a lesser extent iPhones.  having multiple profiles,that use a 'face' and a 'fingerprint' to switch user contexts  is a good thing.   This to me is more important than 'running 2 apps at once' (which it does in the background)...

     

    *the cut out rap section of 'Woke Up this Morning' the Sopranos Theme

  • Reply 12 of 16
    Quote:

    Originally Posted by TBell View Post

     

     

    Agreed. The software will make a 3D map of a users face. I suspect this is years away first. I also doubt Apple would use both fingerprint recognition and facial ID. 


    Why years away... if they can do it about things on the ground in mapping software, then I see them doing it with a face.  pretty quickly.

     

    as for both face and fingerprint, look at the 'shared device' use case.

     

    User A picks up device... device 'sees' who it is...  That maps to a data set of authenticators, including finger prints

    Displays missed notifications to you based on your identity.

    User A logs in does stuff, locks phone

    User B (Mrs A) picks up device... devices 'sees' who it is. her authenticators are queued up.   Her notifications show up... she logs in with fingerprint.

     

    Eliminates typing in an appleID to log in... and typing int '[email protected]' will be a pain.

  • Reply 13 of 16
    jfc1138jfc1138 Posts: 3,090member

    Given all the facial recognition already built into iPhoto and Aperture this is an easy to see progression.

  • Reply 14 of 16
    nexusphan wrote: »
    They will likely address that issue the same way it was already (quickly) addressed in Android.  Face unlock requires the user to blink.
    Well a video could be shown then.

    Detecting 2D from 3D is probably not too hard, identical evil twin on the other hand .....
    Obviously twins will not use the feature.
    tbell wrote: »
    Agreed. The software will make a 3D map of a users face. I suspect this is years away first. I also doubt Apple would use both fingerprint recognition and facial ID. 
    Well they are both sorta used, face detection is in the photos app, and the fingerprint sensor could be more of a mini trackpad in the future.

    jfc1138 wrote: »
    Given all the facial recognition already built into iPhoto and Aperture this is an easy to see progression.
  • Reply 15 of 16

    Now this is a patent worthy of granting.  This is a brilliant way to use facial recognition, and nothing like it has been done before.

  • Reply 16 of 16
    solipsismx wrote: »
    Don't blink. Blink and you're dead. They are fast. Faster than you can believe. Don't turn your back. Don't look away. And don't blink.

    Possibly the greatest adversaries created for the good Doctor!
Sign In or Register to comment.