Apple exploring multi-camera systems for 3D picture taking
Apple has toyed with the idea of adding two or more rear-facing cameras and sensors to a device like an iPhone, allowing pictures to be taken and displayed in 3D.
The U.S. Patent and Trademark Office this week revealed a patent application from Apple related to 3D picture taking, entitled "Systems and Methods for an Imaging System Using Multiple Image Sensors." First discovered by AppleInsider, the filing describes a system that would capture, process and render 3D images with the accompanying dual-camera hardware.
The application refers to Apple's approach as a "paradigm shift from the known software-based approaches." It notes that current software must "guess" at how to perform stereo disparity compensation for 3D images.
Those guesses, Apple said, can result in images with artifacts in a composed 3D image that would result in a poor quality photo. Apple's method would instead employ a hardware-based approach with a "deterministic calculation for stereo disparity compensation."
Apple would create hardware with multiple imaging systems, employing separate luma, chroma and depth/distance sensors to capture images that can be turned into a single, three-dimensional picture.
In addition to still images, Apple's system would also be able to record video in 3D. Setup and calibration of such a system would be a simple one-time event, the application claims.
3D picture taking capabilities for the average consumer just arrived in the U.S. this week, with the launch of the new Nintendo 3DS handheld gaming console. Nintendo's portable device features two rear cameras for capturing still pictures and playing augmented reality games in 3D.
On Nintendo's gaming system, the 3D content is shown on a glasses-free 3D display. Of course, if Apple were to offer 3D picture taking capabilities on a future device, it too would need to feature 3D display capabilities.
The patent application revealed this week is not the first time Apple has shown interest in 3D imaging. In 2009, the company explored 3D display capabilities with head tracking technology, while in 2008 it was revealed to be researching auto-stereoscopic 3D display hardware. The Cupertino, Calif., company has also shown interest in a 3D interface for Mac OS X.
The patent application revealed this week by the USPTO is credited to Brett Bilbrey and Guy Cote. It was first filed in September of 2009.
The U.S. Patent and Trademark Office this week revealed a patent application from Apple related to 3D picture taking, entitled "Systems and Methods for an Imaging System Using Multiple Image Sensors." First discovered by AppleInsider, the filing describes a system that would capture, process and render 3D images with the accompanying dual-camera hardware.
The application refers to Apple's approach as a "paradigm shift from the known software-based approaches." It notes that current software must "guess" at how to perform stereo disparity compensation for 3D images.
Those guesses, Apple said, can result in images with artifacts in a composed 3D image that would result in a poor quality photo. Apple's method would instead employ a hardware-based approach with a "deterministic calculation for stereo disparity compensation."
Apple would create hardware with multiple imaging systems, employing separate luma, chroma and depth/distance sensors to capture images that can be turned into a single, three-dimensional picture.
In addition to still images, Apple's system would also be able to record video in 3D. Setup and calibration of such a system would be a simple one-time event, the application claims.
3D picture taking capabilities for the average consumer just arrived in the U.S. this week, with the launch of the new Nintendo 3DS handheld gaming console. Nintendo's portable device features two rear cameras for capturing still pictures and playing augmented reality games in 3D.
On Nintendo's gaming system, the 3D content is shown on a glasses-free 3D display. Of course, if Apple were to offer 3D picture taking capabilities on a future device, it too would need to feature 3D display capabilities.
The patent application revealed this week is not the first time Apple has shown interest in 3D imaging. In 2009, the company explored 3D display capabilities with head tracking technology, while in 2008 it was revealed to be researching auto-stereoscopic 3D display hardware. The Cupertino, Calif., company has also shown interest in a 3D interface for Mac OS X.
The patent application revealed this week by the USPTO is credited to Brett Bilbrey and Guy Cote. It was first filed in September of 2009.
Comments
Sounds like they wanted to patent an idea to have a foot in the door, than push through an iPod Touch with a 3D camera?
How would this fit today, or with Lion?
Of course, if Apple were to offer 3D picture taking capabilities on a future device, it too would need to feature 3D display capabilities.
That's not true. Twin cameras make AR apps much more powerful, but you could happily interact with those apps using a regular 2D display. That's not to say people might not clamor for a 3D display, just because they wanted one. But twin cameras have utility without such a display. (Edit: I was a little too fast to reply, and I think it looks like I'm commenting about something just to the side of the point you're making. What I had in mind was a video I saw of a guy who used a 2-camera device to turn a flat real-life game board into an AR zombies game, where zombies are walking down the street, and the device screen acts as a camera on the scene. He moves around gets a different view of the streets below. It was very cool. The device was essentially taking a 3D picture over and over and used that to generate scene information for the AR)
The only iDevice with a good camera is iP4. Then they followed up with disappointing sensors in the iPod Touch and iPad 2. Whether the rationale was costs or design (to keep the devices thin), it was still disappointing.
Hopefully this signals a growing interest to conquer the hearts and imagination of more photo-buffs.
Let's see if all the iFanboys will suddenly change their mind and start liking 3D on their phones. The same iFans who blasted this feature on Android phones.
Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.
From theaters, to TVs, to playstation, to Nintendo handhelds...I've yet to see a single implementation of 3D that does not absolutely suck.
Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.
And why is OpenGL rendering bad?
Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.
HTC EVO 3D
4.3" (960x540) glasses-free 3D display
1.2GHz dual-core Qualcomm processor,
Dual 5 megapixel cameras
Front-facing 1.3 megapixel camera
Support for 1080p video playback via the HDMI out (or 720p for 3D content)
Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.
As usual, iFans are in denial. Unless Apple comes up with a feature, it is totally crap and worthless!
What I had in mind was a video I saw of a guy who used a 2-camera device to turn a flat real-life game board into an AR zombies game, where zombies are walking down the street, and the device screen acts as a camera on the scene. He moves around gets a different view of the streets below. It was very cool.
I just picked up a Nintendo 3DS and it includes an AR game that works the same way. It uses a card to have a known position in the world and you walk around it to get different views. You can play the game with the 3D display on or off.
http://www.youtube.com/watch?v=uOa3DHgoHs0
I've also been having fun testing out the 3D camera
http://www.atariage.com/forums/topic/179570-3ds-photos/
3D is the [very near] future. It would be nice to see Apple developing something in this area as well. As to the current patent application, it doesn't seem particularly original; it may not pass due to existing prior art.
Maybe you know someone else who has taken this approach? If so, could you provide a link?
It looks like this describes a method of using specific aspects of the sensor data (chroma, luminance) to calculate the disparity between left and right views with NEW levels of precision. Like using metadata at chip level to refine the parallax geometry.
I think this is new, and if anybody has knowlege of 'prior art' can we hear about it?
One imaginable application: calculate in real time and control by software the spacing between the two camera lenses (or sensor fields perhaps) according to whether foreground or background subjects are of interest, determined by autofocus routines in the viewfinder.
I'm just guessing here. I'll be looking for this patent document, should have a link soon.
Current and near future 3D remains a novelty and it is also not recommended for kids to be exposed to for extended periods. Just like in the 50s, then again in the 70s, it'll fade away or remain a niche.
Last time there was a rumor about 3D, someone posted a link to an article which explained that the way current electronic devices display 3D is different from the way the human eye is built to experience 3D. That basic incompatibility is responsible for the eye strain and vertigo many people get from viewing current 3D. Until there are some kind of holographic displays, I agree that 3D will remain a novelty. I do acknowledge that the presence of 3D cameras on an iPhone would be one way of differentiating between multiple models of iPhone, should Apple ever decide to do that.
Last time there was a rumor about 3D, someone posted a link to an article which explained that the way current electronic devices display 3D is different from the way the human eye is built to experience 3D. That basic incompatibility is responsible for the eye strain and vertigo many people get from viewing current 3D. Until there are some kind of holographic displays, I agree that 3D will remain a novelty. I do acknowledge that the presence of 3D cameras on an iPhone would be one way of differentiating between multiple models of iPhone, should Apple ever decide to do that.
That would have been the Walter Murch letter to Roger Ebert. There's a rejoinder to Murch's 'misguided' conclusions here:
http://www.crunchgear.com/2011/01/24...ignment-of-3d/
Murch's point about the loss of brightness in current projection systems and TV viewing systems is valid, I think. The movie has to be worth it to transcend that loss. So far some are, many aren't.
Edit: Headaches and eyestrain are caused by bad 3D, not 3D itself, OR, by one's own binocular misalignment, not detected in ordinary eye exams. (See Sue Barry's website on 'Fixing My Gaze' if you get headaches when others don't.) There's also a learning curve for those with marginal stereo vision, and a fatigue curve, depending on the viewing system. I felt tired around the eyes after Avatar, and I've been messing with stereo for many years. It was maybe too long, a dreadfully dark and noisome movie. Ambitious, though, and quite an accomplishment.
To create 3D photographs one needs special processing. Already the photographic film industry is almost dead. Three dimensional photography has been in existence with quality cameras for decades and it hasn't caught on.
I don't like 3D photography. It is a novelty that is OK when looking at one image. It becomes annoying with many images. Movies in 3D just take away from the story. Good stories aren't made better with 3D. I watched Avatar on my computer without the 3D effects. I really liked the movie. Having 3D effects always takes attention off the story and puts it on the effects, thus lessening the impact of the story.
The thing that makes 3D bad is that the parallax of a camera isn't the same as the one your eyes produce. All 3D therefore seems unnatural to each viewer.
When holograms that resemble actual things can be produced, our own eyes will then observe the 3D without strain. Of course people on the left side of a theater will not see the movie the people on the right side of the screen see. It would be more like viewing a play than a movie.
I just hope that 3D goes away and stays away.
That would have been the Walter Murch letter to Roger Ebert. There's a rejoinder to Murch's 'misguided' conclusions here:
http://www.crunchgear.com/2011/01/24...ignment-of-3d/
Murch's point about the loss of brightness in current projection systems and TV viewing systems is valid, I think. The movie has to be worth it to transcend that loss. So far some are, many aren't.
Edit: Headaches and eyestrain are caused by bad 3D, not 3D itself, OR, by one's own binocular misalignment, not detected in ordinary eye exams. (See Sue Barry's website on 'Fixing My Gaze' if you get headaches when others don't.) There's also a learning curve for those with marginal stereo vision, and a fatigue curve, depending on the viewing system. I felt tired around the eyes after Avatar, and I've been messing with stereo for many years. It was maybe too long, a dreadfully dark and noisome movie. Ambitious, though, and quite an accomplishment.
From the article (emphasis added):
"Mr. Murch is perfectly right to point out the convergence/focus problem as a limitation of the medium, but for him and Ebert (whom I have rebutted before on the topic) to consider it fatal is, in my opinion, a lack of imagination and faith in the ingenuity of filmmakers."
I don't interpret that as saying that Murch is misguided, rather, it seems to agree he has a point.
No doubt filmmakers can film things differently to make the effect less painful on people, but the incompatibility with human physiology remains. If a person can view their real world surroundings without pain and only when viewing 3D electronically do the symptoms of binocular misalignment or marginal stereo vision occur, I place the blame on the medium, not the eyes.