Apple wins auto-focus speed, exposure metering camera patents
The U.S. Patent and Trademark Office on Tuesday granted Apple two patents related to the operation of a digital camera, one which helps speed up auto-focus time based on object recognition and another for dynamic exposure metering.
Apple's U.S. Patent No. 8,233,789 for "Dynamic exposure metering based on face detection" describes a system and method of automatically changing a digital camera's exposure settings based on a subject's face while U.S. Patent No. 8,233,078 for "Auto focus speed enhancement using object recognition and resolution" uses similar object-detection software to hasten focus speed.
It should be noted that both of Tuesday's patents rely on face or object-recognition assets which are already found in iOS 5.
The '789 patent, first applied for in April 2010, offers a method to automatically adjust the exposure of a digital image by metering a face detected in the picture's frame.
Apple notes the invention could prove useful in situations where a background's lighting is in high contrast relative to the subject. In such cases some automatic metering systems fail because they take an average or weighted average of the complete frame meaning the usually larger background could throw off the exposure. To overcome the limitations of existing metering technologies, the invention employs facial recognition software which the camera uses to apply the appropriate exposure. Continuous metering is also covered by the patent, allowing users to simply open an app, point and shoot.
Face detection-based metering. | Source: USPTO
From the abstract:
In the '078 patent, an interesting autofocusing method is described where a processor detects a face in a captured image and selects the correct lens position for optimal focus based on calculations of the face's size.
The invention selects the appropriate setting for a moveable lens system by comparing a subject's face with "stored face sizes" which are associated with corresponding "in-focus" lens positions. Field of view is also taken into consideration in the patent's claims.
Illustration of the '078 patent's field of view claims. | Source: USPTO
From the patent claims:
Apple's U.S. Patent No. 8,233,789 for "Dynamic exposure metering based on face detection" describes a system and method of automatically changing a digital camera's exposure settings based on a subject's face while U.S. Patent No. 8,233,078 for "Auto focus speed enhancement using object recognition and resolution" uses similar object-detection software to hasten focus speed.
It should be noted that both of Tuesday's patents rely on face or object-recognition assets which are already found in iOS 5.
The '789 patent, first applied for in April 2010, offers a method to automatically adjust the exposure of a digital image by metering a face detected in the picture's frame.
Apple notes the invention could prove useful in situations where a background's lighting is in high contrast relative to the subject. In such cases some automatic metering systems fail because they take an average or weighted average of the complete frame meaning the usually larger background could throw off the exposure. To overcome the limitations of existing metering technologies, the invention employs facial recognition software which the camera uses to apply the appropriate exposure. Continuous metering is also covered by the patent, allowing users to simply open an app, point and shoot.
Face detection-based metering. | Source: USPTO
From the abstract:
The patent helps to make digital cameras found on many "personal electronics" like smartphones more usable as the units are smaller and less feature-rich than dedicated photography devices.In one embodiment disclosed herein, a face detection algorithm is run on the current picture or video frame, and the exposure metering region is inset over the detected face. Exposure time, gain, or other exposure parameters may be set based on the pixels within the exposure metering region. In another embodiment, the exposure metering region tracks a moving face according to lag parameters so that the exposure metering region remains substantially over the face. In yet another embodiment, a plurality of faces may be tracked, with the exposure parameters set based on a weighted average of the pixels within the plurality of face-containing exposure metering regions.
In the '078 patent, an interesting autofocusing method is described where a processor detects a face in a captured image and selects the correct lens position for optimal focus based on calculations of the face's size.
The invention selects the appropriate setting for a moveable lens system by comparing a subject's face with "stored face sizes" which are associated with corresponding "in-focus" lens positions. Field of view is also taken into consideration in the patent's claims.
Illustration of the '078 patent's field of view claims. | Source: USPTO
From the patent claims:
Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems. Instead of moving through a number of positions to gather focus values, the '078 patent allows a lens assembly to "jump directly to a lens position that is expected to result in a sharp picture of the scene, without having to go through a full sweep of lens positions."8. [?] determining angle of view used when capturing the image; and selecting a group of previously stored face sizes and associated lens focus positions, from amongst a plurality of such groups, that are associated with the determined angle of view.
9. [?} calculating field of view used when capturing the image, and wherein calculating the size of the detected face comprises measuring the area of the detected face in relation to the calculated field of view.
10. The method of claim 9 wherein calculating the size of the detected face comprises calculating a ratio of the measured area of the detected face to the calculated field of view.
11. The method of claim 8, wherein calculating the size of the detected face comprises measuring dimensions of the smallest rectangle or square in which the detected face fits.
Comments
OT, but it would be great if the next iPhones' display has the same AR as the CCD. Don't know what I mean? Open the Photo app and pinch to zoom - out, not in. You'll see that the picture is actually larger than what is being displayed. That's because the camera shoots in 4:3 while the screen is 3:2 I really hope they'll make the screen and CCD have the same AR.
I'm sorry, but I lurk here regularly and never post.This has driven me to comment.
What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?
Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.
Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.
Quote:
Originally Posted by AppleInsider
OMG, Apple has patented the rectangle!!!
/s
Quote:
Originally Posted by andrewb123
I'm sorry, but I lurk here regularly and never post.This has driven me to comment.
What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?
Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.
Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.
The precise mechanism is probably different than that which you have used before. And those which you have used before have probably patented their own take on the problem.
Just because something appears to do the same thing does not mean it does the same thing. The 789 patent is fairly detailed. It describes a specific approach. It is conceivable that the Panasonic method is different. Apple did not patent exposure control based on face recognition. They patented a method of doing that.
Don't let all the hyperbole from the software patent haters confuse you.
Quote:
Originally Posted by andrewb123
I'm sorry, but I lurk here regularly and never post.This has driven me to comment.
What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?
Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.
Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.
The question is does your Panasonic use the method Apple's patent uses or does it use the method as was outlined above i.e.:-
"Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems."
Exactly.
And if Apple IS doing exactly the same thing that Panasonic did before Apple applied for the patent - and someone can prove it - the patent will likely be invalidated.
Wow.
Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits. For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):
http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003
If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)
So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.
Quote:
Originally Posted by Jim Cockfield
Wow.
Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits. For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):
http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003
If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)
So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.
See replies to a similar post earlier.
Quote:
Originally Posted by Jim Cockfield
Wow.
Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits. For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):
http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003
If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)
So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.
Does it really bother you *that* much?
Everyone's got an opportunity to patent ideas and technologies, same as Apple. Everyone's got an opportunity to defend them, challenge them, you name it. That's what the courts are there for.
It's all fair.
Are they using the same specific progress as outlined in the patent? Because those specifics matter.
How can Apple patent taking pictures? This nonsense has got to stop. /S
Off topic, but I see wh already have two brand-new accounts with single posts, and coincidentally they're both complaining about Apple patenting something that supposedly already exists.
Quote:
Originally Posted by EricTheHalfBee
Off topic, but I see wh already have two brand-new accounts with single posts, and coincidentally they're both complaining about Apple patenting something that supposedly already exists.
Yup noticed that
.
#next_pages_container { width: 5px; hight: 5px; position: absolute; top: -100px; left: -100px; z-index: 2147483647 !important; }
I noticed the second similar post too from a newly registered user, but it isn't me. Admins can check if they like, I work from a fixed IP address and have nothing to hide.
The patent system is like the UK House of Lords. Clearly needs major reform, but never changes. The problem is that while it may have adequately served its purpose in the whimsical world of yesteryear, it just isn't capable of keeping pace with today's rate of innovation and litigation. It's been re-purposed by evil big companies to attack each other and troll, like the investment banking sector was reshaped to exploit pensioners and private investors.
The article reports the filing in two parts, one of which describes using facial recognition to control focus and exposure. Okay, so the detail of how this is achieved may be slightly different to the multitude of pre-existing methods, but it's the same basic idea. Perhaps Apple feel the need to register their own slightly different method simply to protect themselves from litigation? That sounds like a smart move to me. Otherwise, somebody somewhere might try to take a pop when Apple eventually catch up and add it to their own imaging products. Whatever, the goon that chimed in with how Apple continue to innovate with this patent was way off beam, it's old news, old technology. The ancient Panasonic IA system allows you to register family and friends with the camera so it optimizes focus and exposure for them in preference to 'strangers' in a group photo. Pretty neat, eh?
Quote:
Originally Posted by andrewb123
The patent system is like the UK House of Lords. Clearly needs major reform, but never changes.
In your opinion. I personally think it's fine.
Quote:
Originally Posted by andrewb123
The article reports the filing in two parts, one of which describes using facial recognition to control focus and exposure. Okay, so the detail of how this is achieved may be slightly different to the multitude of pre-existing methods, but it's the same basic idea. Perhaps Apple feel the need to register their own slightly different method simply to protect themselves from litigation? That sounds like a smart move to me. Otherwise, somebody somewhere might try to take a pop when Apple eventually catch up and add it to their own imaging products. Whatever, the goon that chimed in with how Apple continue to innovate with this patent was way off beam, it's old news, old technology. The ancient Panasonic IA system allows you to register family and friends with the camera so it optimizes focus and exposure for them in preference to 'strangers' in a group photo. Pretty neat, eh?
Just because it's "the same basic idea" does not mean innovation and improvements should stop! Without reward, innovation halts. There are always improvements to be made to any basic "idea".
It's a socialist dream to think otherwise. I speak from much experience, and no longer innovate in the industry I once did due to plagiarism. Innovation is hard work and progresses humanity, blatant copying has the adverse affect.
Quote:
Originally Posted by Quadra 610
Does it really bother you *that* much?
Everyone's got an opportunity to patent ideas and technologies, same as Apple. Everyone's got an opportunity to defend them, challenge them, you name it. That's what the courts are there for.
It's all fair.
Actually, yes it does bother me that much. And no, I do not believe it's "all fair".
Bottom line, the Patent System is broken, and only companies with substantial resources can defend themselves against frivolous patent litigation, even when the patents disputed are based on very obvious evolutions in technology and prior art, and should not have been granted patent production to begin with.
The patents being discussed here were predated by many patents filed over 10 years before these, that were awarded to Fuji, Kodak, Sony and even the U.S. Government, involving the use of facial recognition, exposure and autofocus; and a number of camera manufacturers were producing products using this technology years before the Apple Patent Applications were filed.
The Patent System as it stands now serves to stifle innovation because of smaller companies' concerns of lawsuits, and that's bad for consumers.
Quote:
Originally Posted by Jim Cockfield
Wow.
Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits. For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):
http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003
If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)
So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.
"Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems."
Quote:
Originally Posted by EricTheHalfBee
How can Apple patent taking pictures? This nonsense has got to stop. /S
Off topic, but I see wh already have two brand-new accounts with single posts, and coincidentally they're both complaining about Apple patenting something that supposedly already exists.
I joined this morning after reading the announcement here about the patents awarded to Apple. I work for a digital camera review site, and the patent awards got my attention. So, I decided to comment on them. I also wrote an article about them that we'll probably publish later today (with a "via appleinsider.com" type link for publishing the info about the patents being awarded). It's probably a bit "opinionated", but I did "tone it down" some from my first draft.