Apple's secure facial recognition system would rely on 3D rendering
Apple has shown interest in building advanced, secure facial recognition technology that would take a two-dimensional image of a person's face and render it in 3D for greater accuracy.
The concept was detailed in a new patent application discovered this week by AppleInsider entitled "3D Object Recognition." Assigned to Apple, the new application actually stems from an earlier patent filing in Sweden which the company now owns, thanks to its acquisition of Polar Rose for $29 million in 2010.
Because the application was filed by Polar Rose before the company was purchased by Apple, parts of it are less consumer-oriented than Apple would likely utilize. For example, the filing goes into detail on potential government security uses, such as verifying that a person matches their passport or other identification card, and automatically identifying a person's face from a live video without their knowledge.
Last year, with the launch of the iPhone 4S, Apple introduced face detection when shooting still images. Apple's technology allows the iPhone 4S to detect up to 10 faces in a single photo.
The company has also offered facial recognition in iPhoto for years, letting users match a name to a face in pictures, and the system will automatically recognize them in other photos.
But the application published by the U.S. Patent and Trademark Office this week suggests Apple could go beyond simply detecting a face or recognizing it in still photos. By rendering a two-dimensional image of a person from a photo or video in three dimensions, Apple could implement new security features in Macs or iOS devices, allowing it to more accurately identify an individual person.
Google introduced a "Face Unlock" feature with its Galaxy Nexus smartphone and Android 4.0 released last October. But some tests have found that the system can be tricked by holding the picture of a person in front of the device rather than having them physically present.
But with the technology from Polar Rose, Apple's filing describes a more accurate method of identifying a person's face by utilizing a 3D rendering. When setting up a facial recognition system, users could provide a more accurate render of their face to a device like an iPhone by using pictures or video from multiple angles.
This original 3D image would then be the source that the system would refer to when attempting to identify an individual user. The system could then take a two-dimensional image of the person, render it in 3D, and compare it to the original 3D model to verify a person's identity.
Apple has shown interest in secure face detection in the past. For example, last month AppleInsider detailed a separate patent application that described using a forward facing camera on devices like the iPhone to unlock the device and customize its layout based on detecting the identity of the person using it. And a similar filing from last year described iPhones and iPads that would automatically customize applications, settings and features to a user's personal preference once they pick up the device.
The latest application goes well beyond face detection, and describes methods for identifying a range of features such as an entire human body, inner organs, blood vessels, or even tumors. It also mentions inanimate items like vehicles, aircrafts, ships, and military objects.
The application, made public this week, was first filed by Apple with the USPTO in November of 2011. The proposed invention is credited to Jan Eric Solem and Fredrik Kahl of Polar Rose.
The concept was detailed in a new patent application discovered this week by AppleInsider entitled "3D Object Recognition." Assigned to Apple, the new application actually stems from an earlier patent filing in Sweden which the company now owns, thanks to its acquisition of Polar Rose for $29 million in 2010.
Because the application was filed by Polar Rose before the company was purchased by Apple, parts of it are less consumer-oriented than Apple would likely utilize. For example, the filing goes into detail on potential government security uses, such as verifying that a person matches their passport or other identification card, and automatically identifying a person's face from a live video without their knowledge.
Last year, with the launch of the iPhone 4S, Apple introduced face detection when shooting still images. Apple's technology allows the iPhone 4S to detect up to 10 faces in a single photo.
The company has also offered facial recognition in iPhoto for years, letting users match a name to a face in pictures, and the system will automatically recognize them in other photos.
But the application published by the U.S. Patent and Trademark Office this week suggests Apple could go beyond simply detecting a face or recognizing it in still photos. By rendering a two-dimensional image of a person from a photo or video in three dimensions, Apple could implement new security features in Macs or iOS devices, allowing it to more accurately identify an individual person.
Google introduced a "Face Unlock" feature with its Galaxy Nexus smartphone and Android 4.0 released last October. But some tests have found that the system can be tricked by holding the picture of a person in front of the device rather than having them physically present.
But with the technology from Polar Rose, Apple's filing describes a more accurate method of identifying a person's face by utilizing a 3D rendering. When setting up a facial recognition system, users could provide a more accurate render of their face to a device like an iPhone by using pictures or video from multiple angles.
This original 3D image would then be the source that the system would refer to when attempting to identify an individual user. The system could then take a two-dimensional image of the person, render it in 3D, and compare it to the original 3D model to verify a person's identity.
Apple has shown interest in secure face detection in the past. For example, last month AppleInsider detailed a separate patent application that described using a forward facing camera on devices like the iPhone to unlock the device and customize its layout based on detecting the identity of the person using it. And a similar filing from last year described iPhones and iPads that would automatically customize applications, settings and features to a user's personal preference once they pick up the device.
The latest application goes well beyond face detection, and describes methods for identifying a range of features such as an entire human body, inner organs, blood vessels, or even tumors. It also mentions inanimate items like vehicles, aircrafts, ships, and military objects.
The application, made public this week, was first filed by Apple with the USPTO in November of 2011. The proposed invention is credited to Jan Eric Solem and Fredrik Kahl of Polar Rose.
Comments
It would be interesting to have a facial recognition system that actually works and this seems like a good approach, but I will believe it when I see it.
We've been seeing hype over facial recognition for over 30 years and not a single one of them has ever been useable, let alone reliable. So far all a computer can do is recognise that a face exists in a picture about 80% of the time and tell one face from another about 45% of the time.
In Mac OS 9 (or was it 8.6?) you could login with your voice. It worked like crap, but a combination of voice and face would be really nice. Although perfect face recognition would be best.
So far all biometrics have been nothing more than an expensive joke. Using a passcode that is contained only within your head will continue to the safest method.
That said, I can see this tech being useful in being able to track the proper direction and zoom required for FaceTime calls from a TV, which has been one of the major issues I've pondered since the idea of FaceTime in the HEC was considered.
Can you have perfect facial recognition? For it to be perfect it would have to be able to discern between identical twins who are trying to look as similar as possible and then working flawlessly 100% of the time.
Quote:
Originally Posted by SolipsismX
That patent looks absolutely offensive.
So far all biometrics have been nothing more than an expensive joke. Using a passcode that is contained only within your head will continue to the safest method.
That said, I can see this tech being useful in being able to track the proper direction and zoom required for FaceTime calls from a TV, which has been one of the major issues I've pondered since the idea of FaceTime in the HEC was considered.
Can you have perfect facial recognition? For it to be perfect it would have to be able to discern between identical twins who are trying to look as similar as possible and then working flawlessly 100% of the time.
There is non-commodity facial recognition software that provides ~95% accuracy and multiple, different solutions can reach near 100% accuracy for identification though these numbers are published by the companies and I haven't seen independent studies researching the specificity and sensitivity of said solutions. Identification provides different challenges than verification though. The real issue is that most systems even today require full frontal scanning and "daylight" lighting. The government will have proposals for solutions this year that effectively recognize:
One-to-many identification in a set of mugshot images
One-to-one verification of visa images
Multi-sample still facial imagery
Recognition of persons in video sequences
Twins
90 degree profile-view
Gender estimation
Age estimation
Pose estimation
Expression neutrality
The best security solution use three factor authentication but most consumers don't desire to be troubled with the additional two seconds to use even two factor authentication:
"Something you are" (facial recognition, fingerprint recognition, gate recognition, occular recognition, voice recognition)
"Something you know" (typically challenge and password)
"Something you have" (typically a "SmartCard" or SecurID aka USB dongle or soft token)
Quote:
Originally Posted by MacBook Pro
There is non-commodity facial recognition software that provides ~95% accuracy and multiple, different solutions can reach near 100% accuracy though these numbers are published by the companies and I haven't seen independent studies researching the specificity and sensitivity of said solutions. The real issue is that most systems even today require full frontal scanning and "daylight" lighting. The government will have proposals for solutions this year that effectively recognize:
One-to-many identification in a set of mugshot images
One-to-one verification of visa images
Multi-sample still facial imagery
Recognition of persons in video sequences
Twins
90 degree profile-view
Gender estimation
Age estimation
Pose estimation
Expression neutrality
The best security solution use three factor authentication but most consumers don't desire to be troubled with the additional two seconds to use even two factor authentication:
"Something you are" (facial recognition, fingerprint recognition, gate recognition, occular recognition, voice recognition)
"Something you know" (typically challenge and password)
"Something you have" (typically a "SmartCard" or SecurID aka USB dongle or soft token)
The multi-factor identification idea is really the only solution for any kind of secure identification. Even if you are using biometrics instead of questions and answers or possession of physical keys, it's really necessary to use more than one technique. Facial identification, plus voiceprint analysis, plus gait analysis, plus fingerprints and you might have a secure system. Any one of these in isolation is bound to fail sooner or later.
One of the things about face recognition that people always ignore is that faces are not that unique. They are less unique than fingerprints (at least on the level of detail typically analysed), and fingerprints are not that unique at all. What a lot of the CSI type detective shows on TV rarely spell out is that in a given large metropolitan area there can be a half dozen people with the the same (or at least indistinguishable from each other) fingerprints. Even if a facial recognition device got the ID of the person right 100% of the time, there is no guarantee that because the face passes the test, that it's actually the same person. It's the other factors (as you point out) that make the identification sure.
Quote:
Originally Posted by Gazoobee
The multi-factor identification idea is really the only solution for any kind of secure identification. Even if you are using biometrics instead of questions and answers or possession of physical keys, it's really necessary to use more than one technique. Facial identification, plus voiceprint analysis, plus gait analysis, plus fingerprints and you might have a secure system. Any one of these in isolation is bound to fail sooner or later.
One of the things about face recognition that people always ignore is that faces are not that unique. They are less unique than fingerprints (at least on the level of detail typically analysed), and fingerprints are not that unique at all. What a lot of the CSI type detective shows on TV rarely spell out is that in a given large metropolitan area there can be a half dozen people with the the same (or at least indistinguishable from each other) fingerprints. Even if a facial recognition device got the ID of the person right 100% of the time, there is no guarantee that because the face passes the test, that it's actually the same person. It's the other factors (as you point out) that make the identification sure.
This is why I immediately edited my statement to indicate the juxtaposition between identification and verification. The patent in question regards verification (authentication) rather than identification although the AppleInsider article uses the term "identification," this is used incorrectly.
A notable aspect of identification is that often positive identification is not necessary. Often, negative identification is sufficient. If, for example, a transportation terminal has a terrorist watch list they simply need to identify those who (apparently) pose no threat. Anyone identified as a potential threat via facial recognition, gate recognition or behavorial abnormalities can be segregated and more thoroughly investigated.
"Gate" should be "gait" just realized this was typed (and autocorrected) incorrectly.
Authentication aside, it would be neat to have a quick and easy way to get a 3D model of someone's face. There are solutions now, but they are kind of a pain.
Quote:
Originally Posted by afrodri
Authentication aside, it would be neat to have a quick and easy way to get a 3D model of someone's face. There are solutions now, but they are kind of a pain.
As of yesterday, the free 123D Catch iPad app from Autodesk is a pretty cool solution
Like anything else, Apple will introduce face recognition for unlocking, etc when it can be done RIGHT, not like the gimmick it currently is. Yes, I have a Nexus. And yes, it's an absolute gimmick. Works maybe 20% of the time, and seems completely random. I know others with the phone and not a single person that actually uses it.
Quote:
Originally Posted by Slurpy
Like anything else, Apple will introduce face recognition for unlocking, etc when it can be done RIGHT, not like the gimmick it currently is. Yes, I have a Nexus. And yes, it's an absolute gimmick. Works maybe 20% of the time, and seems completely random. I know others with the phone and not a single person that actually uses it.
And this is the argument that most Fandroids make, that Android (or someone else) already has facial recognition...way before Apple. But the reason why Apple is late to the game and is later credited with implementing the technology is because Apple waits until it is (almost) perfected technology and not a novelty.
I don't yet have an iPad so I might be missing something, but as far as I am aware there isn't a 'multi-user' equivalent.
Maybe the font facing camera could utilize the facial recognition to provide the equivalent of this. In this scenario the emphasis is slightly less about security and more on usability. I guess in most families you'll want to keep your own email private but it's not such of an issue that a system that's not 100% secure wouldn't be acceptable.
Taking advantage of the 3D recognition, Siri will now ask you to: "Turn your head and cough"
Quote:
Originally Posted by joelbruner
Taking advantage of the 3D recognition, Siri will now ask you to: "Turn your head and cough"
``Ever been to prison, Doc?''
Quote:
Originally Posted by SolipsismX
That patent looks absolutely offensive.
So far all biometrics have been nothing more than an expensive joke. Using a passcode that is contained only within your head will continue to the safest method.
That said, I can see this tech being useful in being able to track the proper direction and zoom required for FaceTime calls from a TV, which has been one of the major issues I've pondered since the idea of FaceTime in the HEC was considered.
Can you have perfect facial recognition? For it to be perfect it would have to be able to discern between identical twins who are trying to look as similar as possible and then working flawlessly 100% of the time.
Someone will get this right one day and I would like Apple to be the one. Identical twins have been known to successfully fool boyfriends and teachers etc and so, should a computer be expected to perform better. Perhaps one day they will. Without doubt, access security will be a multi-tiered process involving two or more metrics and even then, the user will set preferences. This field has been researched for as long as computers have been available to run algorithms and even by purely optical means (via non-linear optics employing substances such as lithium niobate etc). Further, it is quite likely that only a non-static image will be acceptable for security purposes.
All the best.
If being used as the primary authentication method for accessing a computer then yes.
This has much bigger uses if it works. I am a 3D artist and this kind of technology would turn an iPhone or iPad into a very neat 3D scanner.
A lot of the methods to get 3D from a single camera have scale and distortion problems. Typically the dedicated 3D scanners can get a damn good 3D model (mm accuracy or better) very quickly. But the single view camera techniques may not have over all scale correct and may have afine distortions that would make 3D recognition less accurate.