or Connect
AppleInsider › Forums › General › General Discussion › Apple looks to end blurry iPhone photos with new invention
New Posts  All Forums:Forum Nav:

Apple looks to end blurry iPhone photos with new invention

post #1 of 45
Thread Starter 
In a patent filing discovered on Thursday, Apple describes a digital camera implementation that continuously captures and stores images in a buffer until the user releases the shutter, at which time the system automatically selects the best picture based on a number of predetermined variables.

Continuous Imaging
Source: USPTO


Filed with the U.S. Patent and Trademark Office in October of 2012, Apple's "Image capturing device having continuous image capture" offers owners of small, portable devices more leeway when trying to get the perfect shot.

While smartphones like the iPhone have relatively high-quality camera systems, the products are not purpose-built for picture taking and come with a multitude of compromises. For example, a smartphone's optics and imaging sensor are minuscule compared to modern equivalents seen in full-size DSLRs and pocketable point-and-shoots. The lack of a powerful image processor and other vital components just add to the challenge of getting high quality photographs from a handset's camera.

From the patent filing's background:

These image capturing devices typically use a preview resolution for capturing a preview image of a scene. Subsequently, a user provides an input to take a photograph. The device switches from preview resolution to full resolution prior to capturing an image. Switching from preview to full resolution causes a time lag, which may lead to user frustration. Also, camera shake during the time when a user presses a button or touches a touchscreen can degrade image quality.


The iPhone 5, for example, offers a preview image not quite at full resolution. This allows for fast screen refresh times that give a better overall user experience by simulating a "live" environment. Preview quality is most noticeable when zooming in on a subject, when the image becomes pixelated and sometimes blurry.

Apple's system starts up when a user launches a photo app like Camera, continuously capturing and storing sequential full-resolution images to a buffer. When a request is given (shutter press or screen touch), the system pulls from the pool and chooses one image based on when it was captured, its quality, or a combination of the two.

Depending on the quality of the image, the processing logic can select the photo from either the buffer or concurrent to when the shutter is pressed. The system uses a "focus score" to based on contrast, image resolution, dynamic range and color rendering properties. By weighting the scores of tagged images, along with factoring in exposure time, the logic can choose which photo to use. Memory is conserved by purging the buffer at a predetermined time, or when capacity reaches a certain threshold.

Camera Flowchart
Example flowchart of processing logic.


In one embodiment, the selected picture can be displayed on screen in full resolution immediately after a request as confirmation for the user.

It is not clear if this exact technology is being implemented in iOS and devices like the iPhone, iPad and iPod touch, but some aspects of the invention can be seen in Apple's latest products.

The patent application was first filed for in October of 2012 as a division to another co-pending filing from 2009, and lists Ralph Brunner, Nikhil Bhogal and James David Batson as its inventors.
post #2 of 45

Why not use that fancy blurry picture clearing technology?  Too much for iPhone?  I doubt it.

post #3 of 45
Quote:
Originally Posted by Eriamjh View Post

Why not use that fancy blurry picture clearing technology?  Too much for iPhone?  I doubt it.

Can such algorithms make an image as good as one that was not blurry in the first place? Too much for the imagination? I doubt it.

post #4 of 45
I have seen something like this. Win 8? Nokia?
post #5 of 45
Coming to iPhone 6? Hope so!

Tallest Skil:


"Eventually Google will have their Afghanistan with Oracle and collapse"

"The future is Apple, Google, and a third company that hasn't yet been created."


 


 

Reply

Tallest Skil:


"Eventually Google will have their Afghanistan with Oracle and collapse"

"The future is Apple, Google, and a third company that hasn't yet been created."


 


 

Reply
post #6 of 45

I'd be surprised if this patent is granted. A company called Scalado has been touting an identical technology for years.

post #7 of 45
post #8 of 45
Quote:
Originally Posted by Eriamjh View Post

Why not use that fancy blurry picture clearing technology?  Too much for iPhone?  I doubt it.


I imagine Adobe patented it.

post #9 of 45
This is a well known technique to get non-blurry shots, so I wonder what is novel about their algorithm. Cue Mark Cuban and his gang of SW patent nay-sayers...
post #10 of 45
does not do much for my unsteady hands...
post #11 of 45
Well, I can't even begin to think of the drain it will have on battery! *if you know what I mean*
post #12 of 45
lol why not just have a faster shutter speed? If you are going to spend all that die space for cache and processing a bad image, why not just increase the speed of your shutter and the ability of your sensor to gather light?

Blurry images are due to motion. Motion is offset by shutter speed. Fast shutter speed can be achieved when enough light is gathered (unless you want black images).

Lots of good reading on Canon's website. (I'm sure Nikon also has some great articles)
post #13 of 45
Quote:
Originally Posted by haar View Post

does not do much for my unsteady hands...

 

That is why you need a shutter speed of 120 or above. At 120 I can capture crisp floral shots in the wind. 

post #14 of 45
Quote:
Originally Posted by NotScott View Post

Amazing demo - http://www.youtube.com/watch?feature=player_embedded&v=xxjiQoTp864

 

I love this demo also. But I stress that it is always best to capture a great exposure to start, then to fix it in software afterwards. 

post #15 of 45

Keep in mind this is a filing, not an issue...  

 

At the same time, I am pretty sure they should go and look at the trove of patents from Kodak...  Am betting they'll find one just like this somewhere in there...

 

Just sayin'

post #16 of 45

Most issues can be fixed.  Being out of focus is not one of those issues.

post #17 of 45
Quote:
Originally Posted by Richard Getz View Post

That is why you need a shutter speed of 120 or above. At 120 I can capture crisp floral shots in the wind. 

It's a tiny lens. Very short 'shutter speeds' (in quotes because it doesn't actually use a shutter) may not allow for enough light capture.
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
"I'm way over my head when it comes to technical issues like this"
Gatorguy 5/31/13
Reply
post #18 of 45

Just fix the zoom already. 

post #19 of 45
Quote:
Originally Posted by jpadhiyar View Post

Well, I can't even begin to think of the drain it will have on battery! *if you know what I mean*

 

Quote:
Originally Posted by Richard Getz View Post

lol why not just have a faster shutter speed? If you are going to spend all that die space for cache and processing a bad image, why not just increase the speed of your shutter and the ability of your sensor to gather light?

 

Apple could simply make use of the ISP already embedded on all A5 and above SoCs - would mean insignificant power consumption while processing the images.

 

Cameras in cell phones are tiny compared to what's in dedicated cameras. They cannot possibly capture the same amount of light and as such "shutter" speed will (most likely) always be much slower. Hence the reason for trying to develop other ways to capture higher quality images.

 

As far as cache space, we're probably talking about micro seconds of photo capturing before the person presses the capture button.

Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
Disclaimer: The things I say are merely my own personal opinion and may or may not be based on facts. At certain points in any discussion, sarcasm may ensue.
Reply
post #20 of 45
Originally Posted by NotScott View Post
Amazing demo - http://www.youtube.com/watch?feature=player_embedded&v=xxjiQoTp864

 

Not at all the same technology.


Still neat, though.

Originally Posted by Marvin

The only thing more insecure than Android’s OS is its userbase.
Reply

Originally Posted by Marvin

The only thing more insecure than Android’s OS is its userbase.
Reply
post #21 of 45
Quote:
Originally Posted by paxman View Post

I have seen something like this. Win 8? Nokia?

 

Galaxy S3 is one example.   Burst mode, with "Best Photo" turned on.

 

Takes eight images (one every 0.3 secs).

 

Then it determines the "best photo" based on lighting, subject smiles, and least amount of blur... and puts a "thumbs up" on a mini of that image.  You can decide otherwise and choose a different shot to save, if you were after a different effect.  (Go to about 0:55 below if you want to skip the initial settings tutorial.)

 


Edited by KDarling - 2/14/13 at 9:26am
post #22 of 45

This is a brilliant simple idea.  This was not possible with the 35mm camera.  And it was difficult to impossible for digital camera to do because of lack of processing power.  So Apple has cleverly utilized the resources of a mobile device to do things that was previously impossible.  This is what I like of this company. 

post #23 of 45
Quote:
Originally Posted by KDarling View Post

 

Galaxy S3 is one example.   Burst mode, with "Best Photo" turned on.

 

Takes eight images (one every 0.3 secs).

 

Then it determines the "best photo" based on lighting, subject smiles, and least amount of blur... and puts a "thumbs up" on a mini of that image.  You can decide otherwise and choose a different shot to save, if you were after a different effect.  (Go to about 0:55 below if you want to skip the initial settings tutorial.)

 

This is different from Apple invention.  The eight photos are taken after user press the camera button.  I smell that Samsung will try to steal Apple invention and claim that it innovated it first.  

post #24 of 45
Quote:
Originally Posted by tzeshan View Post

This is different from Apple invention.  The eight photos are taken after user press the camera button.  I smell that Samsung will try to steal Apple invention and claim that it innovated it first.  

Actually, burst mode will stop at 8 shots. Lets you pick best shot. Saves it. Disposes the other 7 shots after saving.

 

Quote:
Originally Posted by tzeshan View Post

This is a brilliant simple idea.  This was not possible with the 35mm camera.  And it was difficult to impossible for digital camera to do because of lack of processing power.  So Apple has cleverly utilized the resources of a mobile device to do things that was previously impossible.  This is what I like of this company. 

 

http://www.samsung.com/in/promotions/galaxycamera/#shoot camera with processing power.

post #25 of 45
Originally Posted by bleh1234 View Post

http://www.samsung.com/in/promotions/galaxycamera/#shoot camera with processing power.

 

And this disproves his point how?

Originally Posted by Marvin

The only thing more insecure than Android’s OS is its userbase.
Reply

Originally Posted by Marvin

The only thing more insecure than Android’s OS is its userbase.
Reply
post #26 of 45
Whatever works! Because of weakness in my arms, I am more likely to get a blurry picture from my iPhone 5 than a sharp one. When it's a bright scene, obviously there is less of a problem. I'm wondering why they haven't been able to institute some form of Image Stabilization technology to make it easier for everyone to get sharper images, even in less than ideal lighting situations!

As a sidebar, the Olympus OMD 4:3rds camera is without doubt THE best at image stabilization. I will be likely selling all my Nikon gear and switching to this for serious photography, particularly because of my condition. I love my Nikon D7000 setup, but I have to go to high ISOs for everyday shooting if I want to be sure my images are sharp. This introduces "graininess" that normal ISOs wouldn't. Not a good trade off.
post #27 of 45
Quote:
Originally Posted by bleh1234 View Post

Actually, burst mode will stop at 8 shots. Lets you pick best shot. Saves it. Disposes the other 7 shots after saving.

 

I know that.  I have read that article carefully.  Apple's invention is revolutionary.  It take the shots BEFORE you press the camera button.  

post #28 of 45
Quote:
Originally Posted by stelligent View Post

Can such algorithms make an image as good as one that was not blurry in the first place? Too much for the imagination? I doubt it.

There's also this one. Not as good picture quality yet as even a top-notch camera phone, but it's expected to get better fast with a bit more processing power behind it.

 

http://www.businessweek.com/videos/2013-02-12/how-the-lytro-camera-puts-blurry-photos-into-focus


Edited by Gatorguy - 2/14/13 at 10:56am
melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #29 of 45
Quote:
Originally Posted by tzeshan View Post

This is different from Apple invention.  The eight photos are taken after user press the camera button.  

 

I see what you're saying.  One captures photos before the shutter is clicked, and the other captures after.  

 

Perhaps someone should get a patent on a method that does both, which would take care of the user clicking in anticipation of an action.

 

I smell that Samsung will try to steal Apple invention and claim that it innovated it first.  

 

Oh for goodness' sake.  This is not exactly a revolutionary invention, considering the number of apps that allow choosing still photos from a video.  This variation simply requires the willingness to chew up resources during a time when the user normally doesn't think anything is happening.  Personally, I'd rather let it know when I'm close to being ready.

 

Heck, it's basically doing the same thing that a DVR does when you turn it on... it starts recording the channel that you're flipped to, in case you later decide to pause, or go back a bit.  I'd even guess that could've been the inspiration.


Edited by KDarling - 2/14/13 at 11:04am
post #30 of 45
Quote:
Originally Posted by bleh1234 View Post

Actually, burst mode will stop at 8 shots. Lets you pick best shot. Saves it. Disposes the other 7 shots after saving.

 

 

http://www.samsung.com/in/promotions/galaxycamera/#shoot camera with processing power.

There's a much better description of it here:

http://samsunggeeks.com/2012/08/08/galaxy-s3-best-photo-what-is-it-and-how-to-use-it/#.UR0z5h3UkSE

 

A bit fanboyish but still explains it pretty well. 

melior diabolus quem scies
Reply
melior diabolus quem scies
Reply
post #31 of 45
Quote:
Originally Posted by mjtomlin View Post

 

 

Apple could simply make use of the ISP already embedded on all A5 and above SoCs - would mean insignificant power consumption while processing the images.

 

Cameras in cell phones are tiny compared to what's in dedicated cameras. They cannot possibly capture the same amount of light and as such "shutter" speed will (most likely) always be much slower. Hence the reason for trying to develop other ways to capture higher quality images.

 

As far as cache space, we're probably talking about micro seconds of photo capturing before the person presses the capture button.

 

Quote:
Originally Posted by jragosta View Post


It's a tiny lens. Very short 'shutter speeds' (in quotes because it doesn't actually use a shutter) may not allow for enough light capture.

 

 

So instead of increasing the lens size, or improving light gathering, let's try to enhance a poor exposure? I can see software assist as the filers assist when high ISO is used, however, the idea is always to first capture the best exposure.  

post #32 of 45
Quote:
Originally Posted by tzeshan View Post

I know that.  I have read that article carefully.  Apple's invention is revolutionary.  It take the shots BEFORE you press the camera button.

 

 

Quote:
In a patent filing discovered on Thursday, Apple describes a digital camera implementation that continuously captures and stores images in a buffer until the user releases the shutter, at which time the system automatically selects the best picture based on a number of predetermined variables.

 

The button is pressed.

post #33 of 45
Quote:
Originally Posted by bleh1234 View Post

 

 

 

The button is pressed.

I think you caught a mistake in the writing.  Because later in the article it said

"Depending on the quality of the image, the processing logic can select the photo from either the buffer or concurrent to when the shutter is pressed."

post #34 of 45
Quote:
Originally Posted by Richard Getz View Post

lol why not just have a faster shutter speed? If you are going to spend all that die space for cache and processing a bad image, why not just increase the speed of your shutter and the ability of your sensor to gather light?

Blurry images are due to motion. Motion is offset by shutter speed. Fast shutter speed can be achieved when enough light is gathered (unless you want black images).

Lots of good reading on Canon's website. (I'm sure Nikon also has some great articles)

 

Obviously because they are ALREADY taking the most advantage of the image sensor they have. What this does is improve what the user captures.
I'm wondering if they might also be doing a bit of "HDR" -- changing focus and exposure to get more value on a combined image. I know what we see so far is just "find the most in focus of the samples." It's a bit like a video capture and you use the buffer to make them all photo quality for a brief moment.
It's an improvement -- so what's to complain about?
post #35 of 45
It would be nice to combine this with the technology that determines when people's eyes are open, so if two pictures are comparably sharp but one has a subject with open eyes, it chooses that one.
post #36 of 45

Always check the claims to see what the patent is really about:

 

"1. A method comprising: continuously, after entering a mode, capturing a sequence of images with an image capturing device; continuously, after entering the mode, storing a predetermined number of the sequence of images in a buffer; receiving a user request to capture an image; and in response to the user request, automatically selecting one of the buffered images based on an image contrast parameter that compares pixel values within each buffered image, wherein the sequence of images were captured prior to receiving the user request."

 

So, it captures pictures after entering a certain mode, and before the user touches the screen. 

 

The main purpose noted later on, is to avoid shutter shake caused by tapping the screen shutter button.

 

I think it might end being kind of peculiar to use at first, because most of us expect the pictures to be taken AT or AFTER we click, not during the time beforehand.

post #37 of 45
I'm shocked they've even applied for this patent. I use to work for Apple and now work for *cough*. I'm somewhat of a Apple fan but as a camera engineer I have to say this method is extremely obvious. It pops up every single time deblurring comes up in discussions with customers. 3 times this year alone.

This is a natural consequence of implementations of NSL and burst in fact. You're already capturing repeated stills... picking the best one is not a patent IMHO. The method of determining which one is the best could be patentable but I think almost everyone that works in the field could figure that one out too...

FYI "blurry picture clearing technology" is a figment of Hollywood's imagination. If it was easy I could sell you a 1MP camera with no lens. It'll take a crappy blurry picture but you could just "blurry picture clear" it into a 10MP sharp image.

The best thing to do is avoid it in the first place which is what this patent is all about.
post #38 of 45
Originally Posted by Serendip View Post
I'm shocked they've even applied for this patent. I use to work for Apple and now work for *cough*.

 

The draw to Apple forums was so great you switched companies. 'Course now you can't post on *cough*'s forums… lol.gif


FYI "blurry picture clearing technology" is a figment of Hollywood's imagination.

 

Really (as posted earlier)?

Originally Posted by Marvin

The only thing more insecure than Android’s OS is its userbase.
Reply

Originally Posted by Marvin

The only thing more insecure than Android’s OS is its userbase.
Reply
post #39 of 45
Quote:
Originally Posted by KDarling View Post

 

I think it might end being kind of peculiar to use at first, because most of us expect the pictures to be taken AT or AFTER we click, not during the time beforehand.

I think the opposite.  You press the camera button thinking you want the picture shown on the screen.  Because of the time taken to press the button and the shutter delay all cameras always take the picture AFTER the click.  And often it is not what is wanted.  

post #40 of 45
Quote:
Originally Posted by Serendip View Post

I'm shocked they've even applied for this patent. I use to work for Apple and now work for *cough*. I'm somewhat of a Apple fan but as a camera engineer I have to say this method is extremely obvious. It pops up every single time deblurring comes up in discussions with customers. 3 times this year alone.

This is a natural consequence of implementations of NSL and burst in fact. You're already capturing repeated stills... picking the best one is not a patent IMHO. The method of determining which one is the best could be patentable but I think almost everyone that works in the field could figure that one out too...

FYI "blurry picture clearing technology" is a figment of Hollywood's imagination. If it was easy I could sell you a 1MP camera with no lens. It'll take a crappy blurry picture but you could just "blurry picture clear" it into a 10MP sharp image.

The best thing to do is avoid it in the first place which is what this patent is all about.

I think whether it is patentable is whether it is prior art.  If no company has implemented it then it is not a prior art.  Samsung's Burst Mode is clearly not a prior art.  

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: General Discussion
AppleInsider › Forums › General › General Discussion › Apple looks to end blurry iPhone photos with new invention