Future iPhones may shift camera sensor to take perfect pictures with clip-on lenses

Posted:
in iPhone
Camera accessories that attach to an iPhone or iPad could work better in the future, as Apple is considering ways to adjust a mobile device's camera to make shots taken through the extra external lens look great.




Accessories like those from Olloclip and other third-party producers provide customers with ways to make the camera included in iPhones and iPads better, usually in the form of putting additional lenses in front of the camera to change the photograph. While this can help create effects not normally possible on an iPhone, such as extending the zoom or widening the field of view, errors in physically attaching it to the device, such as failing to center it on the lens, could cause some unwanted results.

Granted on Tuesday by the U.S. Patent and Trademark Office, the patent for "Adapting camera systems to accessory lenses" was originally filed in August 2016, and attempts to fix such issues using a combination of methods.

Apple's suggestion includes a lens system, a photosensor, camera control, image analysis, and image processing software, which can be used to analyze the picture quality and detect problems. The analysis element would look at the focus, sharpness, relative illumination, optical distortion, chromatic aberration, magnification, vignetting, and other metrics to see how the image has changed from existing results for the stock camera lens.




A part of the same system, actuators could change the physical location of the camera itself on multiple axes based on the results of the analysis, in order to properly line itself up with the center of the accessory's lens. In theory, this could include tilting the camera if the accessory is fitted at an angle, and in some cases could be moved closer or further away from the accessory's lens.

While this can be performed by the camera detecting an accessory, Apple suggests other ways can be used to determine if an accessory is attached, including electronics, magnets, and radio sensing. In the case where the device knows what specific accessory is attached, this could be used to enable extra options in the camera interface, giving users more ways to interact with the add-on hardware.

Apple regularly files patent applications with the USPTO, and the publication of an application or granting of a patent is not necessarily a sign that Apple is actively working on introducing the technology in its consumer products.

In this case, such a system seems fairly plausible to add to an iPhone or iPad, as some of the elements are already in place. Aside from the software-based image analysis that could be processed on a mobile device, Apple does use mechanical optical image stabilization systems that help produce sharp images with minimal motion blur.

While it would be a stretch to adjust the tilt of the lens using an existing OIS system, its ability to make minor adjustments to the camera sensor's position could feasibly be useful for the final lining-up of the camera's lens with the center of the accessory, if it is almost perfectly placed from the start.
Alex1N

Comments

  • Reply 1 of 11
    Sounds like a useful patent if one were mounting a camera behind a lens... even a standard optical eye lens. Perhaps this may have threads into AR devices.
    repressthisAlex1Nwatto_cobra
  • Reply 2 of 11
    MacProMacPro Posts: 17,574member
    With the amazing Sony Alpha cameras and iPhone quality Canon and Nikon must be scratching their heads. All my Canon gear is for sale. Loving the a7 iii.
    watto_cobra
  • Reply 3 of 11
    melgrossmelgross Posts: 31,036member
    I’ve been suggesting a mini bayonet around the lens for several years. With two lenses, that wouldn’t work, unless the lenses were further apart. But there’s got to be a simple way to do this. This seems overly complex. Possibly some indentations around the lenses that manufacturers could use to line their lenses to, and to provide a spacing.

    giving the optical specs. Out to manufacturers would also help in their design work. There are some very good lenses for the iPhone out there, but they require cases, or clip-on mounts that either shift, or are clumsy. I’ve got a number of these things.
    radarthekatAlex1N
  • Reply 4 of 11
    melgrossmelgross Posts: 31,036member
    MacPro said:
    With the amazing Sony Alpha cameras and iPhone quality Canon and Nikon must be scratching their heads. All my Canon gear is for sale. Loving the a7 iii.
    Meh. Not so thrilled about Sony’s work. Neither is the rest of the camera buying public. While Canon has just over 50% of the camera market, and Nikon about 35%, Sony is still stuck at about 5%.
    edited July 24 Alex1N
  • Reply 5 of 11
    To get out in front on photographic quality, Apple could conceivably sell cases with a rotating disk containing paired lenses—one for each of two lenses with rotation for different pairings— for alternate focal lengths and macro ability. Another add-on to up-sell.  
    I’m also curious if they are working on a 3 lense system.  
    Another exciting area is computational photography as is done with the two lenses used together.

    I read earlier this year that Apple has around a thousand people that work on computational photography (Jean-Louis Gassée’s Monday Note blog—former Apple exec, BE OS founder and very smart guy). That’s much more than Canon, Nikon or Sony can afford to invest. 

    Apple is very serious about photography and AR. Since their pipelines for products can be as much as 5 to 7 years, believe Tim Cook when he says AR is going to be a big deal. 

    Alex1N
  • Reply 6 of 11
    icoco3icoco3 Posts: 1,453member
    melgross said:
    I’ve been suggesting a mini bayonet around the lens for several years. With two lenses, that wouldn’t work, unless the lenses were further apart. But there’s got to be a simple way to do this. This seems overly complex. Possibly some indentations around the lenses that manufacturers could use to line their lenses to, and to provide a spacing.

    giving the optical specs. Out to manufacturers would also help in their design work. There are some very good lenses for the iPhone out there, but they require cases, or clip-on mounts that either shift, or are clumsy. I’ve got a number of these things.
    There is that lens bump on the back that the clip on lens manufacturers could use to center their lenses.
    edited July 24
  • Reply 7 of 11
    melgrossmelgross Posts: 31,036member
    icoco3 said:
    melgross said:
    I’ve been suggesting a mini bayonet around the lens for several years. With two lenses, that wouldn’t work, unless the lenses were further apart. But there’s got to be a simple way to do this. This seems overly complex. Possibly some indentations around the lenses that manufacturers could use to line their lenses to, and to provide a spacing.

    giving the optical specs. Out to manufacturers would also help in their design work. There are some very good lenses for the iPhone out there, but they require cases, or clip-on mounts that either shift, or are clumsy. I’ve got a number of these things.
    There is that lens bump on the back that the clip on lens manufacturers could use to center their lenses.
    Yes, but it doesn’t seem to be enough, so they don’t use it.
  • Reply 8 of 11
    i guess (hope) this means apple will be creating their own series of olloclips as well? maybe? i like the X but find the camera excessive for my needs. extensions can serve those who need to take photos and exempt those of us who are just casual photographers.
  • Reply 9 of 11
    MacProMacPro Posts: 17,574member
    melgross said:
    MacPro said:
    With the amazing Sony Alpha cameras and iPhone quality Canon and Nikon must be scratching their heads. All my Canon gear is for sale. Loving the a7 iii.
    Meh. Not so thrilled about Sony’s work. Neither is the rest of the camera buying public. While Canon has just over 50% of the camera market, and Nikon about 35%, Sony is still stuck at about 5%.
    Sorry this is OT but have to respond to that.  I should qualify that by saying I am talking high end gear and I think those numbers are changing fast in that arena.  Perhaps you should test an Alpha before deciding. The new sensors are sci-fi level.  Plus it is a full frame mirrorless beast made of all metal.  I've been shooting Canon for decades and have a lot of L lenses.  Nice thing is the Canon lenses all mount on the Sony (EF and EF-S) and have 100% functionality including continuous AF in 4K video and up to 120 fps HD.  Not to mention eye tracking let alone face tracking.  This all thanks to the Metabones V adapter.  You should check out all the reviwers who have been Canon aficionados comments on testing the new Sony a7 iii.  Here is just one https://kenrockwell.com/sony/a7-iii.htm. Just look at those images!  This is only Sony's entry level full frame too!  Want to buy a well loved but now obsolete 5D iii?  ;)
    Alex1N
  • Reply 10 of 11
    melgrossmelgross Posts: 31,036member
    MacPro said:
    melgross said:
    MacPro said:
    With the amazing Sony Alpha cameras and iPhone quality Canon and Nikon must be scratching their heads. All my Canon gear is for sale. Loving the a7 iii.
    Meh. Not so thrilled about Sony’s work. Neither is the rest of the camera buying public. While Canon has just over 50% of the camera market, and Nikon about 35%, Sony is still stuck at about 5%.
    Sorry this is OT but have to respond to that.  I should qualify that by saying I am talking high end gear and I think those numbers are changing fast in that arena.  Perhaps you should test an Alpha before deciding. The new sensors are sci-fi level.  Plus it is a full frame mirrorless beast made of all metal.  I've been shooting Canon for decades and have a lot of L lenses.  Nice thing is the Canon lenses all mount on the Sony (EF and EF-S) and have 100% functionality including continuous AF in 4K video and up to 120 fps HD.  Not to mention eye tracking let alone face tracking.  This all thanks to the Metabones V adapter.  You should check out all the reviwers who have been Canon aficionados comments on testing the new Sony a7 iii.  Here is just one https://kenrockwell.com/sony/a7-iii.htm. Just look at those images!  This is only Sony's entry level full frame too!  Want to buy a well loved but now obsolete 5D iii?  ;)
    Their sensors are at about the same level as Canon’s And Nikon’s. A small difference here and there.

    i love to pixel peep as much as the next guy. But, when you get to real photos in print, the differences just melt away. Ten years ago, we could see differences in print, from high quality commercial ink jet. But even then, when going four color, it was almost impossible to tell. The same thing is true now, though it’s even more difficult.

    all cameras from about three years ago far exceed the limits of four color printing, and even the best inkjet printing. Even websites, using sRGB, which is the standard for the internet, have problems showing the differences between the newest models. I’ll take my 5Dmk IV any day. 
    Alex1N
  • Reply 11 of 11
    Alex1NAlex1N Posts: 37member
    MacPro said:
    With the amazing Sony Alpha cameras and iPhone quality Canon and Nikon must be scratching their heads. All my Canon gear is for sale. Loving the a7 iii.
    I'm not chucking my Canon Powershot SuperZoom - old though it is - or my relatively recent Olympus 4/3 OM D EM5 Mk II system just yet. Especially not just for an iPhone (although I realise that you aren't saying that).

    If I remember right, Sony uses Zeiss lenses, doesn't it? What does Apple use? Their own? Someone else's (e.g., Huawei and Leica)?
    edited July 27
Sign In or Register to comment.