Apple's future iPhones & iPads could automatically personalize via face recognition

Posted:
in Future Apple Hardware edited January 2014
Using a forward-facing camera to recognize an individual user, future iPhones and iPads from Apple could automatically customize applications, settings and features to a user's personal preferences once they pick up the device.



Apple's concept was revealed in a new patent application published this week by the U.S. Patent and Trademark Office and discovered by AppleInsider. Entitled "Low Threshold Face Recognition," it describes a low-computation solution for quickly and accurately recognizing a user.



The filing provides a simple way for multiple users to share a single device, like an iPad. Each user could customize their personal profile with unique wallpaper, applications and settings, and that profile would be immediately accessed once the iPad recognizes a user's face.



Apple's application notes that robust facial recognition systems that work in various lighting conditions and orientations can be taxing on an electronic device, requiring resources and draining battery life.



Its solution would reduce the impact of lighting conditions and biometric distortions on an image. The filing describes a "low-computation solution for reasonably effective (low threshold) face recognition that can be implemented on camera-equipped consumer portable appliances."



Rather than aggressively analyzing a user's entire face and using up time and resources, Apple's concept would rely on a "high information portion" of a human face. Potential high information portions could include eyes, a mouth, or the tip of a user's nose.



By recognizing the individual features on a user's face, the system could scale the distance between someone's eyes and their mouth. That distance could then be measured against the reference image originally captured by the user in order to confirm it is in fact the same person.







Apple's application notes that its facial recognition capabilities could be constantly active due to lower power consumption. This means a user could simply point an iPhone or iPad at their face, without pressing a button, and have the screen automatically turn on and unlock the device.



This could be accomplished through an "orange-distance filter," which would capture the "likely presence" of a human face in front of a camera. This filter would also be used to detect a person's skin tone, and measure the distance of their face from the camera.



Once a user has been recognized, the facial recognition technology could not only grant them access to the device, but also customize its settings based on a unique user profile. Each user would be presented with a personalized configuration, as an iPhone or iPad would be able to "modify screen saver slide shows or other appliance non-security preferences," the application reads.







The proposed invention, made public this week, was first filed by Apple in June of 2009. It is credited to Robert Mikio Free.



Apple was rumored in September of 2010 to have acquired a Swedish facial recognition company, Polar Rose, for $29 million. One of the company's products, Recognizr, could take a photo of a user and recognize that same person when shown on video.



Reports have indicated that Apple has long shown interest in having its devices identify users with facial recognition technology and customize a device based on their personal preferences. In early 2010, before the iPad was announced, The Wall Street Journal revealed that an early prototype of Apple's touchscreen tablet would use a forward-facing camera to recognize users' faces, allowing it to be one device easily shared by the entire family.



Apple reportedly experimented with the ability to customize the device, and have it automatically switch to a user's personal settings once they picked it up. One early feature included virtual "sticky notes" that one user could leave for another, and would be read the next time they picked up the iPad.
«13

Comments

  • Reply 1 of 58
    Quote:

    By recognizing the individual features on a user's face, the system could scale the distance between someone's eyes and their mouth. That distance could then be measured against the reference image originally captured by the user in order to confirm it is in fact the same person.



    Okay, before we get buried under an avalanche of Ice Cream Sandwiches, can someone tell me how (or IF) this is different from that gimmick in Android and how this implementation wouldn't also be a pointless gimmick?
  • Reply 2 of 58
    too bad for Apple competitors : another possible unlock mechanism patent protected ...
  • Reply 3 of 58
    And how is this useful since you can only have one "user" on an iPhone?
  • Reply 4 of 58
    MacProMacPro Posts: 18,216member
    I can imagine Siri saying .. "Either that isn't you Mike or you look terrible this morning!"
  • Reply 5 of 58
    this is so weird, I was literally just doing a search for multiuser iPad !! Its the one missing feature that is stopping me get one
  • Reply 6 of 58
    juandljuandl Posts: 228member
    Is it because I only read AppleInsider and similar sites. Or do other phone manufacturers actually care about 'Patent' actual good ideas to put on their phones?

    I know that in Samsung and HTC 's case they just copy.

    But is there anybody else innovating?
  • Reply 7 of 58
    poochpooch Posts: 768member
    i think "the cloud" needs to be a bit more accessible such that when i start storing all my personally identifiable information in it, biometrics included, the information is accessible by any corporate entity. just wanting to do my part.
  • Reply 8 of 58
    drdoppiodrdoppio Posts: 1,132member
    Why not voice recognition (building on existing technologies)? Using face recognition is flawed from a security point of view, just hold a picture in front of the camera.
  • Reply 9 of 58
    nairbnairb Posts: 253member
    Quote:
    Originally Posted by Kevinneal View Post


    this is so weird, I was literally just doing a search for multiuser iPad !! Its the one missing feature that is stopping me get one



    Multiple user is already available on Microsoft Tablets - maybe that is an option for you.



    Oh, so apple have been beaten to multiple users, beaten to facial recognition of user, but they think they are entitled to a patent of facial recognition for multiple users. The innovation is amazing. No-one else would have ever thought about this.
  • Reply 10 of 58
    What about using a thumb print as you pick it up ? since there is a camera how about a retina scan, voice reco....Lots of different options here. Let the user pick how they want it unlocked. Self destruct mode would be awesome when you are ready to ask the wife for a new one cuz that one is broken
  • Reply 11 of 58
    wingswings Posts: 261member
    Quote:
    Originally Posted by Nairb View Post


    Multiple user is already available on Microsoft Tablets - maybe that is an option for you.



    Oh, so apple have been beaten to multiple users, beaten to facial recognition of user, but they think they are entitled to a patent of facial recognition for multiple users. The innovation is amazing. No-one else would have ever thought about this.



    It's not so much WHAT they're doing but the WAY they're doing it. That's what patents are all about.
  • Reply 12 of 58
    Quote:
    Originally Posted by Wings View Post


    It's not so much WHAT they're doing but the WAY they're doing it. That's what patents are all about.



    Yeah you patent mechanisms and techniques not results
  • Reply 13 of 58
    Quote:
    Originally Posted by juandl View Post


    Is it because I only read AppleInsider and similar sites. Or do other phone manufacturers actually care about 'Patent' actual good ideas to put on their phones?

    I know that in Samsung and HTC 's case they just copy.

    But is there anybody else innovating?



    They copy what now? Maybe you do need to get out of AI once in a while. Samsung Electronics has consistently been ranked second after IBM for the most patents granted. Is that not innovation?
  • Reply 14 of 58
    Quote:
    Originally Posted by AppleInsider View Post




    Using a forward-facing camera to recognize an individual user, future iPhones and iPads from Apple could automatically customize applications, settings and features to a user's personal preferences once they pick up the device.






    I'm not certain that when I'm looking at an iPad that I want the iPad looking back at me.



    Too much like Big Brother. I hope that this "feature" will have a secure kill mode.
  • Reply 15 of 58
    Quote:
    Originally Posted by Tallest Skil View Post


    Okay, before we get buried under an avalanche of Ice Cream Sandwiches, can someone tell me how (or IF) this is different from that gimmick in Android and how this implementation wouldn't also be a pointless gimmick?



    first thing I noticed was that this said "non security" unlike Ice Cream which seems to be going full on for all preferences including security



    As for the gimmick thing, of course it is. Heck Apple might not ever implement it. Why cut into sales by making their devices multiuser, right? But the way that Ice Cream is doing it is apparently not so great. Apple figures out a better way, patents it and then Android would be stuck having to license it from Apple if they want to use that improved way of doing the whole trick. tad devious but it's a stunt Apple has pulled before as have other companies.
  • Reply 16 of 58
    It's about time.



    I don't care if it's implemented by facial recognition, voice or login screen, I just want multiple users on the iPad ASAP.



    Especially on a device like the iPad that is often shared. Pick it up and get YOUR Mail, YOUR iCal, YOUR music, YOUR bookmarks in Safari, etc.



    Even if I had an iPad for every family member I wouldn't want everyone to have to carry their iPad around the house. Pick up the iPad in the family room or kitchen and get YOUR info.



    I know memory is one of the holdups, but I'd buy the larger capacity for the ability to have multiple users on an iPad like I have on my 27" iMac.



    Wouldn't be as useful on an iPhone or iPod Touch, but definitely on the iPad.
  • Reply 17 of 58
    bwikbwik Posts: 562member
    If I were a DHS or DOD warrantless surveillance jerk, I would certainly be intrigued by this Apple technology. Which they would share with feds, presumably.
  • Reply 18 of 58
    Quote:
    Originally Posted by charlituna View Post


    Why cut into sales by making their devices multiuser, right?



    I really don't think it will hurt sales. If anything customers will buy the more profitable 64GB (hopefully 128GB with iPad 3) versions.



    For me it's all about convenience and privacy. When you have young kids, you want to monitor what they can access (parental controls), protect your data from being accidentally deleted, etc.



    Desktops, laptops and iPads are often shared. Being able to login and get the device how you left it is paramount!



    It's the one missing feature I asked for when the first iPad launched and the one I'm still eagerly waiting for. Much better than workarounds like using webmail.
  • Reply 19 of 58
    Quote:
    Originally Posted by Tallest Skil View Post


    Okay, before we get buried under an avalanche of Ice Cream Sandwiches, can someone tell me how (or IF) this is different from that gimmick in Android and how this implementation wouldn't also be a pointless gimmick?



    Wasn't the Android gimmick fooled by a picture?



    Not great security if It can't tell the difference between a real person and a picture|
  • Reply 20 of 58
    There are a finite set of elements that are used to determine facial recognition. The more powerful the processing system, the more variables it can use to determine a facial match as a part of a "security" system. It's not really a security feature* - a point I will touch on further down. Facial recognition depends on certain "geographies" and "geometries" of the face, as well as the resolution and depth of field of the camera used to measure those elements. Among the geographic and geometric elements looked at are is the object in the camera actually a "face", the distance between the eyes, midpoint between eyes, realtional measurements between eyes, nose, cheekbones, mouth, or in greater detail, skin textures, etc. Geometric elements include the solid object Euler angles, and overall head shape. Reference photos are used to composite a reference frame for comparison purposes. Deeper analysis for recognition involves processor demanding multi-variate analysis.



    Stationary security device applications currently are trying out a 3D approach which would defeat the use of 2D picture representations that can be used to fool traditional recognition systems. However the more data points required for comparison the more processor intensive and slower the results.



    For example, Google's Picasa and Apple's iPhoto both use a fairly robust set of facial recognition algorithms. ICS used a more basic approach which relies on a small composite set coupled with simple algorithms measuring the eye distance, midpoint and Euler angles to resolve recognition. The downside to the less intensive approach was demonstrated when it was introduced - where low-lighting caused the recognition to fail, and subsequent demonstrations of photos being used to fool the system.



    For facial recognition to "work" reliably on a mobile device, there is a necessary balance between detail processing and accurate confirmation. This means that a better camera should be used to capture detail, multiple cameras to capture 3D. Deeper eigen-style analysis maybe still too much processing for mobile devices, which means that facial recognition is not a recommended security method for locking/unlocking your device.



    Google and Apple both have several different approaches they are using, and will patent anything they see as singularly important to use for various projects.



    * As far as security is concerned for a mobile device, facial recognition is actually more of a "convenience" feature rather than a security feature - at least for now.
Sign In or Register to comment.