Apple patents Lytro-like refocusable camera for iPhone
The U.S. Patent and Trademark Office on Tuesday awarded Apple a patent for a camera system that uses a microlens adaptor to enable refocusing of an image after initial shot is taken, much like the Lytro Light Field camera.
Source: USPTO
Apple's U.S. Patent No. 8,593,564 for a "Digital camera including refocusable imaging mode adaptor," describes a method in which a user can take a picture at a certain resolution and refocus the resulting image after the fact. Also noted in the patent is the inclusion of such a system in portable devices, like an iPhone.
The patent describes a plenoptic camera, otherwise known as a "light-field camera," that uses a microlens array in the form of an adaptor rather than a fixed and integrated component.
Arguably the most famous light field camera is the Lytro. Founded by Stanford University computer science graduate student Ren Ng, Lytro Inc. has drawn funding from heavy-hitting investors despite its limited hardware lineup. Interestingly, late Apple cofounder Steve Jobs reportedly invited Ng to his Palo Alto, Calif., home for a live demonstration of the camera before it went public.
In fact, Apple's patent cites Ng's invention as prior art, but notes improvements can be made in regard to picture quality and resolution.
Light field technology uses so-called microlenses disposed in front of a CMOS or CCD sensor, which together form a "ray sensor" capable of capturing intensity of light as a function of position and angle. The microlens array directs light in a specific and predictable manner based on the lens aperture. In the case of Apple's patent, each microlens corresponds to a fixed set of imaging pixels. After the light information has been recorded, software can be used to focus and refocus a picture.
Unlike Lytro, Apple's design employs a movable adaptor between the lens element and the imaging sensor. Situated on the adaptor is the microlens array, meaning the camera can operate in two separate modes: high-resolution non-refocusable and low-resolution refocuasable. Lytro's microlens component can also be moved closer and farther from the sensor, but cannot be completely removed from the light path.
Apple's patent describes the adaptor as a switchable optical module mounted to a bracket along with a glass plate.
When in low-resolution refocusing mode, as seen above, the optical module is positioned between the lens and the sensor, thus creating a ray sensor. The patent notes the sensor records light data from the microlens (positioning) and angle of incidence (direction). Spatial resolution is limited by that of the microlens array.
In high-resolution non-refocusing mode, the switchable optical module moves to a different position in which the glass plate is situated between the lens and imaging sensor. Light is redirected by the glass pane onto the sensor to create an image of the scene as would a normal camera. The glass has a thickness and refraction index that compensates for the focal plane adjustment made to accommodate the microlens array. This mode allows for the sensor's full resolution to be used.
In some embodiments, the hardware includes smartphone-like features, like the ability to place phone calls and connect to a wireless network. Other forms of the patent include the system can be used as an add-on accessory for any existing camera.
It is unknown if Apple will employ the technology in a later iPhone model, though the solution would solve numerous problems suers face with small-format camera platforms.
Apple's plenoptic adaptor patent was first filed for in 2011 and credits John Norvold Border and Richard D. Young.
Source: USPTO
Apple's U.S. Patent No. 8,593,564 for a "Digital camera including refocusable imaging mode adaptor," describes a method in which a user can take a picture at a certain resolution and refocus the resulting image after the fact. Also noted in the patent is the inclusion of such a system in portable devices, like an iPhone.
The patent describes a plenoptic camera, otherwise known as a "light-field camera," that uses a microlens array in the form of an adaptor rather than a fixed and integrated component.
Arguably the most famous light field camera is the Lytro. Founded by Stanford University computer science graduate student Ren Ng, Lytro Inc. has drawn funding from heavy-hitting investors despite its limited hardware lineup. Interestingly, late Apple cofounder Steve Jobs reportedly invited Ng to his Palo Alto, Calif., home for a live demonstration of the camera before it went public.
In fact, Apple's patent cites Ng's invention as prior art, but notes improvements can be made in regard to picture quality and resolution.
Light field technology uses so-called microlenses disposed in front of a CMOS or CCD sensor, which together form a "ray sensor" capable of capturing intensity of light as a function of position and angle. The microlens array directs light in a specific and predictable manner based on the lens aperture. In the case of Apple's patent, each microlens corresponds to a fixed set of imaging pixels. After the light information has been recorded, software can be used to focus and refocus a picture.
Unlike Lytro, Apple's design employs a movable adaptor between the lens element and the imaging sensor. Situated on the adaptor is the microlens array, meaning the camera can operate in two separate modes: high-resolution non-refocusable and low-resolution refocuasable. Lytro's microlens component can also be moved closer and farther from the sensor, but cannot be completely removed from the light path.
Apple's patent describes the adaptor as a switchable optical module mounted to a bracket along with a glass plate.
When in low-resolution refocusing mode, as seen above, the optical module is positioned between the lens and the sensor, thus creating a ray sensor. The patent notes the sensor records light data from the microlens (positioning) and angle of incidence (direction). Spatial resolution is limited by that of the microlens array.
In high-resolution non-refocusing mode, the switchable optical module moves to a different position in which the glass plate is situated between the lens and imaging sensor. Light is redirected by the glass pane onto the sensor to create an image of the scene as would a normal camera. The glass has a thickness and refraction index that compensates for the focal plane adjustment made to accommodate the microlens array. This mode allows for the sensor's full resolution to be used.
In some embodiments, the hardware includes smartphone-like features, like the ability to place phone calls and connect to a wireless network. Other forms of the patent include the system can be used as an add-on accessory for any existing camera.
It is unknown if Apple will employ the technology in a later iPhone model, though the solution would solve numerous problems suers face with small-format camera platforms.
Apple's plenoptic adaptor patent was first filed for in 2011 and credits John Norvold Border and Richard D. Young.