Apple's dual-sensor imaging patent promises higher quality iPhone pictures

Posted:
in General Discussion edited January 2014
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent for an imaging system that processes data from multiple sensors to detect and correct for various photographic anomalies, such as defective pixels, lens aberrations, and ineffective demosaicing, among others.

Dual Sensor
Source: USPTO


Apple's U.S. Patent No. 8,493,482 for a "Dual image sensor image processing system and method," describes a unique technology that can interleave imaging data from two or more imaging sensors, resulting in higher quality photos than a setup with a single sensor.

The lengthy patent notes that said device can be either mobile or non-portable, but does mention a number of Apple products by name, including the iPhone, iPod, and a few Mac models. Disregarding device form factor, the invention can be used on any suitable camera system, which can be described as a sensor, color filters, lens elements, processing circuitry, and storage. Other components specified include various device hardware not directly associated with the camera module.

According to the patent abstract, there are two main modes in which the imaging system operates. One uses a single sensor, as is the case with many digital consumer cameras, while another uses image signal processing to process data from two distinct sensors.

While the idea of having two or more sensors disposed in a device is a novel, the backend processing and control logic comprising the image processing circuitry is most impressive.

Going into highly technical detail, the document describes the various functions of the system, starting with the detection of image flaws pertaining to filters, dead pixels, luminance, chroma, and others. Responsible for this step of processing is the front-end logic, which compares raw image data against a litany of statistical predefined metrics and statistics stored in system memory.

Dual Sensor
Illustration of raw image data and applied Bayer filtering techniques.


In one embodiment, the system moves data through the ISP pipe processing logic before outputting to a display device. The image undergoes further processing in this step, which in some cases may involve the replacement of image data from one sensor with data from a second sensor, depending on the level of corruption. For example, if a sensor's image data presents a threshold that indicates a dead, "hot," "stuck," or otherwise defective pixel, the system will compensate with data from the second sensor.

The patent notes that the method can be used for both still pictures and video by using the same image processing circuitry. Key to the invention is not the per-pixel replacement of data, but more importantly the logic that can interleave the information from two separate frames into one before outputting to a display or memory.

Dual Sensor
Per-pixel motion history lookup for video.


While current rumors don't have Apple implementing a dual-lens or dual-camera system in its iOS device lineup in the near future, the technology could one day be implemented to bring a new level of image quality to compatible products.

Apple's dual image sensor patent was first filed for in 2010 and credits Guy Cote, Jeffrey E. Frederiksen, Joseph P. Bratt, Shun Wai Go, and Timothy J. Millet as its inventors.

Comments

  • Reply 1 of 10
    ppietrappietra Posts: 288member
    I don%u2019t believe this patent covers a two cameras device! As you can see in the first image there is only one lens for possible multiple sensors - only one camera.
    In fact it would actually be quite difficult to use another camera to correct a dead pixel on one camera because of different viewing angles and the world being 3 dimensional, meaning each camera would see a different picture.
  • Reply 2 of 10
    SpamSandwichSpamSandwich Posts: 33,407member
    Reminds me of the Lytro.
  • Reply 3 of 10
    shogunshogun Posts: 362member
    I agree with ppietra, above. Surely it would use one lens but with two sensors. Very cool!
  • Reply 4 of 10
    froodfrood Posts: 771member


    Analysts now predict Apple will soon release a new helmet-like wearable iDevice as evidenced by a lleaked photo in one of their patent filings.  No word yet on a release date for the iHelmet

  • Reply 5 of 10
    MacProMacPro Posts: 19,718member
    Flies are claiming prior art! ;)
  • Reply 6 of 10
    rivertriprivertrip Posts: 142member

    Quote:

    Originally Posted by SpamSandwich View Post



    Reminds me of the Lytro.


    Why?

  • Reply 7 of 10
    jessijessi Posts: 302member
    Even with one lens, I would think registration issues would be a problem, unless one sensor is transparent and the other sensor is behind it.
  • Reply 8 of 10
    mjtomlinmjtomlin Posts: 2,673member

    Quote:

    Originally Posted by Ppietra View Post



    I don%u2019t believe this patent covers a two cameras device! As you can see in the first image there is only one lens for possible multiple sensors - only one camera.

    In fact it would actually be quite difficult to use another camera to correct a dead pixel on one camera because of different viewing angles and the world being 3 dimensional, meaning each camera would see a different picture.


     


    Not two cameras... two image sensors. The light coming through a single lens can be split to two sensors. Image sensors aren't all equal. Some may have flaws. Having a redundant sensor will provide the system another set of data to correct for any issues found in the image.

  • Reply 9 of 10
    richard getzrichard getz Posts: 1,142member

    Quote:

    Originally Posted by Ppietra View Post



    I don%u2019t believe this patent covers a two cameras device! As you can see in the first image there is only one lens for possible multiple sensors - only one camera.

    In fact it would actually be quite difficult to use another camera to correct a dead pixel on one camera because of different viewing angles and the world being 3 dimensional, meaning each camera would see a different picture.


     


    I agree with your comment, but let me take it further. If one lens and two sensors, how will they get the light from one lens onto two sensors? Split the light? But would that not reduce the quality of the light on one or both sensors? 

  • Reply 10 of 10
    richard getzrichard getz Posts: 1,142member

    Quote:

    Originally Posted by mjtomlin View Post


     


    Not two cameras... two image sensors. The light coming through a single lens can be split to two sensors. Image sensors aren't all equal. Some may have flaws. Having a redundant sensor will provide the system another set of data to correct for any issues found in the image.



     


    I should have read all the way down first : )


     


    If you split the light, would you not effectively cut it in half? What would this do for low light captures? 

Sign In or Register to comment.