or Connect
AppleInsider › Forums › General › General Discussion › Apple wins auto-focus speed, exposure metering camera patents
New Posts  All Forums:Forum Nav:

Apple wins auto-focus speed, exposure metering camera patents

post #1 of 29
Thread Starter 
The U.S. Patent and Trademark Office on Tuesday granted Apple two patents related to the operation of a digital camera, one which helps speed up auto-focus time based on object recognition and another for dynamic exposure metering.

Apple's U.S. Patent No. 8,233,789 for "Dynamic exposure metering based on face detection" describes a system and method of automatically changing a digital camera's exposure settings based on a subject's face while U.S. Patent No. 8,233,078 for "Auto focus speed enhancement using object recognition and resolution" uses similar object-detection software to hasten focus speed.

It should be noted that both of Tuesday's patents rely on face or object-recognition assets which are already found in iOS 5.

The '789 patent, first applied for in April 2010, offers a method to automatically adjust the exposure of a digital image by metering a face detected in the picture's frame.

Apple notes the invention could prove useful in situations where a background's lighting is in high contrast relative to the subject. In such cases some automatic metering systems fail because they take an average or weighted average of the complete frame meaning the usually larger background could throw off the exposure. To overcome the limitations of existing metering technologies, the invention employs facial recognition software which the camera uses to apply the appropriate exposure. Continuous metering is also covered by the patent, allowing users to simply open an app, point and shoot.

Exposure
Face detection-based metering. | Source: USPTO


From the abstract:

In one embodiment disclosed herein, a face detection algorithm is run on the current picture or video frame, and the exposure metering region is inset over the detected face. Exposure time, gain, or other exposure parameters may be set based on the pixels within the exposure metering region. In another embodiment, the exposure metering region tracks a moving face according to lag parameters so that the exposure metering region remains substantially over the face. In yet another embodiment, a plurality of faces may be tracked, with the exposure parameters set based on a weighted average of the pixels within the plurality of face-containing exposure metering regions.


The patent helps to make digital cameras found on many "personal electronics" like smartphones more usable as the units are smaller and less feature-rich than dedicated photography devices.

In the '078 patent, an interesting autofocusing method is described where a processor detects a face in a captured image and selects the correct lens position for optimal focus based on calculations of the face's size.

The invention selects the appropriate setting for a moveable lens system by comparing a subject's face with "stored face sizes" which are associated with corresponding "in-focus" lens positions. Field of view is also taken into consideration in the patent's claims.

width=
Illustration of the '078 patent's field of view claims. | Source: USPTO


From the patent claims:

8. [?] determining angle of view used when capturing the image; and selecting a group of previously stored face sizes and associated lens focus positions, from amongst a plurality of such groups, that are associated with the determined angle of view.

9. [?} calculating field of view used when capturing the image, and wherein calculating the size of the detected face comprises measuring the area of the detected face in relation to the calculated field of view.

10. The method of claim 9 wherein calculating the size of the detected face comprises calculating a ratio of the measured area of the detected face to the calculated field of view.

11. The method of claim 8, wherein calculating the size of the detected face comprises measuring dimensions of the smallest rectangle or square in which the detected face fits.


Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems. Instead of moving through a number of positions to gather focus values, the '078 patent allows a lens assembly to "jump directly to a lens position that is expected to result in a sharp picture of the scene, without having to go through a full sweep of lens positions."
post #2 of 29
Keep em coming Apple, really great to see you continue to innovate!

OT, but it would be great if the next iPhones' display has the same AR as the CCD. Don't know what I mean? Open the Photo app and pinch to zoom - out, not in. You'll see that the picture is actually larger than what is being displayed. That's because the camera shoots in 4:3 while the screen is 3:2 I really hope they'll make the screen and CCD have the same AR.
post #3 of 29

I'm sorry, but I lurk here regularly and never post.This has driven me to comment.

 

What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?

 

Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.

 

Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.

post #4 of 29
Quote:
Originally Posted by AppleInsider View Post

11. The method of claim 8, wherein calculating the size of the detected face comprises measuring dimensions of the smallest

rectangle or square in which the detected face fits.

 

OMG, Apple has patented the rectangle!!!

 

/s

Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
post #5 of 29
Quote:
Originally Posted by andrewb123 View Post

I'm sorry, but I lurk here regularly and never post.This has driven me to comment.

 

What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?

 

Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.

 

Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.

 

The precise mechanism is probably different than that which you have used before. And those which you have used before have probably patented their own take on the problem. 


Edited by monstrosity - 7/31/12 at 6:19am
post #6 of 29
Quote:
Originally Posted by andrewb123 View Post

I'm sorry, but I lurk here regularly and never post.This has driven me to comment.

What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?

Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.

Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.

Just because something appears to do the same thing does not mean it does the same thing. The 789 patent is fairly detailed. It describes a specific approach. It is conceivable that the Panasonic method is different. Apple did not patent exposure control based on face recognition. They patented a method of doing that.

Don't let all the hyperbole from the software patent haters confuse you.
post #7 of 29
Quote:
Originally Posted by andrewb123 View Post

I'm sorry, but I lurk here regularly and never post.This has driven me to comment.

 

What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years. My three year old Panasonic TZ10 has 'Intelligent Auto Mode' which does exactly the same as this 'new innovation'. Panasonic's Intelligent Auto algorithms clearly pre-date the patent application. Don't the USPO know how to use Google?

 

Is this new because it's on a phone and not a camera? Come on Apple, where's the innovation in this? You make yourselves look like patent trolls. I can't help feeling that Apple are trying to register anything and everything, no matter how obvious or pre-existing, knowing that the US patent system is a complete joke.

 

Panasonic (and a dozen other camera makers) had better keep an eye out for the mailman because it looks like he'll be delivering a letter from Apple's lawyers any day now.

 

The question is does your Panasonic use the method Apple's patent uses or does it use the method as was outlined above i.e.:-

 

"Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems."

Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
post #8 of 29
Quote:
Originally Posted by Wovel View Post

Just because something appears to do the same thing does not mean it does the same thing. The 789 patent is fairly detailed. It describes a specific approach. It is conceivable that the Panasonic method is different. Apple did not patent exposure control based on face recognition. They patented a method of doing that.
Don't let all the hyperbole from the software patent haters confuse you.

Exactly.

And if Apple IS doing exactly the same thing that Panasonic did before Apple applied for the patent - and someone can prove it - the patent will likely be invalidated.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #9 of 29

Wow.

 

Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits.     For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):

 

http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003

 

If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)

 

So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.

post #10 of 29
Quote:
Originally Posted by Jim Cockfield View Post

Wow.

 

Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits.     For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):

 

http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003

 

If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)

 

So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.

See replies to a similar post earlier. 

post #11 of 29
Quote:
Originally Posted by Jim Cockfield View Post

Wow.

 

Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits.     For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):

 

http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003

 

If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)

 

So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.

 

Does it really bother you *that* much?

 

Everyone's got an opportunity to patent ideas and technologies, same as Apple. Everyone's got an opportunity to defend them, challenge them, you name it. That's what the courts are there for. 

 

It's all fair. 

post #12 of 29
Quote:
Originally Posted by andrewb123 View Post

I'm sorry, but I lurk here regularly and never post.This has driven me to comment.

What the hell is the USPO up to here? Digital cameras have been using face detection to optimize exposure and focus to the human subject for years and years.

Are they using the same specific progress as outlined in the patent? Because those specifics matter.

A non tech's thoughts on Apple stuff 

(She's family so I'm a little biased)

Reply

A non tech's thoughts on Apple stuff 

(She's family so I'm a little biased)

Reply
post #13 of 29

How can Apple patent taking pictures? This nonsense has got to stop. /S

 

 

Off topic, but I see wh already have two brand-new accounts with single posts, and coincidentally they're both complaining about Apple patenting something that supposedly already exists.

Author of The Fuel Injection Bible

Reply

Author of The Fuel Injection Bible

Reply
post #14 of 29
Quote:
Originally Posted by EricTheHalfBee View Post

 

Off topic, but I see wh already have two brand-new accounts with single posts, and coincidentally they're both complaining about Apple patenting something that supposedly already exists.

 

Yup noticed that ;) 

post #15 of 29

.

 
 
post #16 of 29

I noticed the second similar post too from a newly registered user, but it isn't me. Admins can check if they like, I work from a fixed IP address and have nothing to hide.

 

The patent system is like the UK House of Lords. Clearly needs major reform, but never changes. The problem is that while it may have adequately served its purpose in the whimsical world of yesteryear, it just isn't capable of keeping pace with today's rate of innovation and litigation. It's been re-purposed by evil big companies to attack each other and troll, like the investment banking sector was reshaped to exploit pensioners and private investors.

 

The article reports the filing in two parts, one of which describes using facial recognition to control focus and exposure. Okay, so the detail of how this is achieved may be slightly different to the multitude of pre-existing methods, but it's the same basic idea. Perhaps Apple feel the need to register their own slightly different method simply to protect themselves from litigation? That sounds like a smart move to me. Otherwise, somebody somewhere might try to take a pop when Apple eventually catch up and add it to their own imaging products. Whatever, the goon that chimed in with how Apple continue to innovate with this patent was way off beam, it's old news, old technology. The ancient Panasonic IA system allows you to register family and friends with the camera so it optimizes focus and exposure for them in preference to 'strangers' in a group photo. Pretty neat, eh?

post #17 of 29
Quote:
Originally Posted by andrewb123 View Post

The patent system is like the UK House of Lords. Clearly needs major reform, but never changes. 

 

In your opinion. I personally think it's fine.

post #18 of 29
Quote:
Originally Posted by andrewb123 View Post

The article reports the filing in two parts, one of which describes using facial recognition to control focus and exposure. Okay, so the detail of how this is achieved may be slightly different to the multitude of pre-existing methods, but it's the same basic idea. Perhaps Apple feel the need to register their own slightly different method simply to protect themselves from litigation? That sounds like a smart move to me. Otherwise, somebody somewhere might try to take a pop when Apple eventually catch up and add it to their own imaging products. Whatever, the goon that chimed in with how Apple continue to innovate with this patent was way off beam, it's old news, old technology. The ancient Panasonic IA system allows you to register family and friends with the camera so it optimizes focus and exposure for them in preference to 'strangers' in a group photo. Pretty neat, eh?

 

 

Just because it's "the same basic idea" does not mean innovation and improvements should stop! Without reward, innovation halts. There are always improvements to be made to any basic "idea".

 

 

It's a socialist dream to think otherwise. I speak from much experience, and no longer innovate in the industry I once did due to plagiarism. Innovation is hard work and progresses humanity, blatant copying has the adverse affect.


Edited by monstrosity - 7/31/12 at 7:14pm
post #19 of 29
Quote:
Originally Posted by Quadra 610 View Post

 

Does it really bother you *that* much?

 

Everyone's got an opportunity to patent ideas and technologies, same as Apple. Everyone's got an opportunity to defend them, challenge them, you name it. That's what the courts are there for. 

 

It's all fair. 

 

Actually, yes it does bother me that much.  And no, I do not believe it's "all fair".

 

Bottom line, the Patent System is broken, and only companies with substantial resources can defend themselves against frivolous patent litigation, even when the patents disputed are based on very obvious evolutions in technology and prior art, and should not have been granted patent production to begin with.

 

The patents being discussed here were predated by many patents filed over 10 years before these, that were awarded to Fuji, Kodak, Sony and even the U.S. Government, involving the use of facial recognition, exposure and autofocus; and a number of camera manufacturers were producing products using this technology years before the Apple Patent Applications were filed.

 

The Patent System as it stands now serves to stifle innovation because of smaller companies' concerns of lawsuits, and that's bad for consumers.

post #20 of 29
Quote:
Originally Posted by Jim Cockfield View Post

Wow.

 

Virtually all of the major camera manufacturers were shipping products with face detection that adjusted both focus and exposure to get the best portraits.     For example, read this page at bestbuy I found with a quick google search with detailed information about how it worked in the PowerShot G9 and SX100 (introduced in 2007, years before Apple filed for that patent):

 

http://www.bestbuy.com/site/Canon/Canon-Advanced-Face-Detection/pcmcat149500050003.c?id=pcmcat149500050003

 

If you read through feature descriptions and press releases for other camera models with face detection,you'll find that they all are adjusting both focus and exposure for faces in scenes (basically, the same thing Apple describes in that patent application)

 

So, that Apple was granted a patent for the same thing that Canon, Fujifilm, Pentax, Panasonic, Sony and others were doing before that application was ever filed is just one more example of a patent system that's out of control.

 

"Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems."

Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
Better than my Bose, better than my Skullcandy's, listening to Mozart through my LeBron James limited edition PowerBeats by Dre is almost as good as my Sennheisers.
Reply
post #21 of 29
Quote:
Originally Posted by EricTheHalfBee View Post

How can Apple patent taking pictures? This nonsense has got to stop. /S

 

 

Off topic, but I see wh already have two brand-new accounts with single posts, and coincidentally they're both complaining about Apple patenting something that supposedly already exists.

 

I joined this morning after reading the announcement here about the patents awarded to Apple.  I work for a digital camera review site, and the patent awards got my attention.   So, I decided to comment on them.  I also wrote an article about them that we'll probably publish later today (with a "via appleinsider.com" type link for publishing the info about the patents being awarded).  It's probably a bit "opinionated", but I did "tone it down" some from my first draft.

post #22 of 29
Quote:
Originally Posted by hill60 View Post

 

"Apple notes the invention saves time by eschewing the sweep of positions a moveable lens assembly goes through to find optimal focus in some existing autofocus systems."

 

Do you not think that other camera manufacturers are doing the same type of thing?  The technology has been around for a long time (years before Apple filed their patent applications).  Of course other manufacturers are looking at details like the percentage of the frame the faces occupy to help with AF speed, also taking things like Depth of Field for a given focal length and aperture into consideration to accelerate AF performance.  The Apple patent assumes the use of very old AF technology in it's filings.

 

If companies like Minolta (with their camera related technology now owned by Sony) released their algorithms for technology being used to improve AF speed based on subject detection and percentage of the frame occupied, and (even before digital cameras were popular), I think you'll find that the Apple patent is based on prior art.    AF technology used by today's digital cameras is very sophisticated, as it has been for years before Apple filed these patent applications.

 

This is just another example of a broken patent system.

post #23 of 29
Quote:
Originally Posted by Jim Cockfield View Post

 

and only companies with substantial resources can defend themselves against frivolous patent litigation

 

 

Why do you assume it's frivolous?

 

Is it because you don't run Apple and aren't responsible for protecting its greatest asset (intellectual property) and aren't responsible to shareholders?

 

Probably. 

 

WAKE UP.  There are billions of dollars at stake here. If you were running Apple, you would do exactly the same thing, if not more. 

 

As far as "innovation" goes, if the rest of the industry feels it can't innovate around Apple's implementation of tech, then it's a pretty sad state of affairs for the industry outside Cupertino: one that is beset by galactic laziness and a lack of imagination. This is what happens when you sit on your hands and let a leaner, meaner, faster competitor run away with the market. And a smarter one, too, that used the same patent system that is open to everyone else. 

 

If "smaller" companies feel they can't innovate, it's their fault.  They will come and go and live and die until someone comes along who can innovate. 

 

The price of poker suddenly goes up after June 2007, and afterward in January 2010 and everyone else cries foul. It's because they can't be bothered to step up their game. Which is exactly why Apple does so well in the market. Apple is terrified of stagnation. Everyone else is terrified of work.

 

The real problem here isn't the patent system. It's everyone else's LAZINESS. Too much work to do it better. Oh well. I'm sure they'll appreciate your sympathy as they circle the drain. Because they'll get none whatsoever from me.  

post #24 of 29
Quote:
Originally Posted by Quadra 610 View Post

 

 

Why do you assume it's frivolous?

 

 

read the entire post from which you responded; its latter half answers your question.

post #25 of 29

Innovation means creating a new method, look it up. This isn't a new method, this is a slightly different method based on existing tech. They are expanding something, not innovating, which they shouldn't be able to patent.

post #26 of 29
Quote:
Originally Posted by emacs72 View Post

 

read the entire post from which you responded; its latter half answers your question.

 

 

We'll let the courts decide that, if and when the time comes. 

post #27 of 29

The typical way of handling face detection AF is using the focus value from face region. This is not necessary same thing that "acclerating the AF based on face detection". Are they reltaed? sure...  Will those are considered same in patent world? I don't think so. 

 

The only purpose of face detection AF is to make sure face gets good focus from camera. I'm sure this feature is done by many company. It seems like what apple claims is that they can reduce the AF search range by using the face detecton data.

They are not claiming the right for face detecton AF or not even new way of doing face detectiion AF. I think what they claiming is basically "improving AF speed using face detection data".  I am not very sure if anyone already done this or not before Apple (I think that is the work for patent office) but it is not certainly very common idea like just plain "face detection AF".

post #28 of 29
Quote:
Originally Posted by Wovel View Post


Just because something appears to do the same thing does not mean it does the same thing. The 789 patent is fairly detailed. It describes a specific approach. It is conceivable that the Panasonic method is different. Apple did not patent exposure control based on face recognition. They patented a method of doing that.
Don't let all the hyperbole from the software patent haters confuse you.

 

So you're saying it's ok even if it looks and functions the same, as long as the underlying mechanism is different?

I think you need to rethink your statement, because you just completely invalidated Apple's suit against Samsung.

post #29 of 29
Quote:
Originally Posted by Quadra 610 View Post

 

 

Why do you assume it's frivolous?

 

Is it because you don't run Apple and aren't responsible for protecting its greatest asset (intellectual property) and aren't responsible to shareholders?

 

Probably. 

 

WAKE UP.  There are billions of dollars at stake here. If you were running Apple, you would do exactly the same thing, if not more. 

 

As far as "innovation" goes, if the rest of the industry feels it can't innovate around Apple's implementation of tech, then it's a pretty sad state of affairs for the industry outside Cupertino: one that is beset by galactic laziness and a lack of imagination. This is what happens when you sit on your hands and let a leaner, meaner, faster competitor run away with the market. And a smarter one, too, that used the same patent system that is open to everyone else. 

 

If "smaller" companies feel they can't innovate, it's their fault.  They will come and go and live and die until someone comes along who can innovate. 

 

The price of poker suddenly goes up after June 2007, and afterward in January 2010 and everyone else cries foul. It's because they can't be bothered to step up their game. Which is exactly why Apple does so well in the market. Apple is terrified of stagnation. Everyone else is terrified of work.

 

The real problem here isn't the patent system. It's everyone else's LAZINESS. Too much work to do it better. Oh well. I'm sure they'll appreciate your sympathy as they circle the drain. Because they'll get none whatsoever from me.  

 

Apple is patenting the wedge, and tech that other companies have already had on the market for years, but they're the INNOVATIVE ones?

 

Yup, chug that Koolaid, because you clearly swallowed any objectivity you had a long time ago.

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: General Discussion
AppleInsider › Forums › General › General Discussion › Apple wins auto-focus speed, exposure metering camera patents