or Connect
AppleInsider › Forums › Mac Hardware › Future Apple Hardware › Apple exploring multi-camera systems for 3D picture taking
New Posts  All Forums:Forum Nav:

Apple exploring multi-camera systems for 3D picture taking

post #1 of 26
Thread Starter 
Apple has toyed with the idea of adding two or more rear-facing cameras and sensors to a device like an iPhone, allowing pictures to be taken and displayed in 3D.

The U.S. Patent and Trademark Office this week revealed a patent application from Apple related to 3D picture taking, entitled "Systems and Methods for an Imaging System Using Multiple Image Sensors." First discovered by AppleInsider, the filing describes a system that would capture, process and render 3D images with the accompanying dual-camera hardware.

The application refers to Apple's approach as a "paradigm shift from the known software-based approaches." It notes that current software must "guess" at how to perform stereo disparity compensation for 3D images.

Those guesses, Apple said, can result in images with artifacts in a composed 3D image that would result in a poor quality photo. Apple's method would instead employ a hardware-based approach with a "deterministic calculation for stereo disparity compensation."

Apple would create hardware with multiple imaging systems, employing separate luma, chroma and depth/distance sensors to capture images that can be turned into a single, three-dimensional picture.

In addition to still images, Apple's system would also be able to record video in 3D. Setup and calibration of such a system would be a simple one-time event, the application claims.



3D picture taking capabilities for the average consumer just arrived in the U.S. this week, with the launch of the new Nintendo 3DS handheld gaming console. Nintendo's portable device features two rear cameras for capturing still pictures and playing augmented reality games in 3D.

On Nintendo's gaming system, the 3D content is shown on a glasses-free 3D display. Of course, if Apple were to offer 3D picture taking capabilities on a future device, it too would need to feature 3D display capabilities.



The patent application revealed this week is not the first time Apple has shown interest in 3D imaging. In 2009, the company explored 3D display capabilities with head tracking technology, while in 2008 it was revealed to be researching auto-stereoscopic 3D display hardware. The Cupertino, Calif., company has also shown interest in a 3D interface for Mac OS X.

The patent application revealed this week by the USPTO is credited to Brett Bilbrey and Guy Cote. It was first filed in September of 2009.
post #2 of 26
Would be interesting for a MacBook or iMac. I think it could also link to their previous patents on being able to move the desktop display through tracking potentially? Also depth perception, possible uses could link up to it. We haven't seen much from Apple in regard Kinect like body or hand gestures (or being able to identify hand position and gestures. Guess we'll see if they want to go in that direction.
Sounds like they wanted to patent an idea to have a foot in the door, than push through an iPod Touch with a 3D camera?
How would this fit today, or with Lion?
post #3 of 26
Quote:
Originally Posted by AppleInsider View Post

Of course, if Apple were to offer 3D picture taking capabilities on a future device, it too would need to feature 3D display capabilities.

That's not true. Twin cameras make AR apps much more powerful, but you could happily interact with those apps using a regular 2D display. That's not to say people might not clamor for a 3D display, just because they wanted one. But twin cameras have utility without such a display. (Edit: I was a little too fast to reply, and I think it looks like I'm commenting about something just to the side of the point you're making. What I had in mind was a video I saw of a guy who used a 2-camera device to turn a flat real-life game board into an AR zombies game, where zombies are walking down the street, and the device screen acts as a camera on the scene. He moves around gets a different view of the streets below. It was very cool. The device was essentially taking a 3D picture over and over and used that to generate scene information for the AR)
post #4 of 26
Let's see if all the iFanboys will suddenly change their mind and start liking 3D on their phones. The same iFans who blasted this feature on Android phones.
post #5 of 26
I'm interested, would be real cool if the Live Chat sites use it (until you click on a good-look'n thumbnail and the live 3D baaadddddd). Meaning maybe 3D isn't isn't the best for everything.
post #6 of 26
Photography in general has not been an area that Apple has shown consistent passion for.

The only iDevice with a good camera is iP4. Then they followed up with disappointing sensors in the iPod Touch and iPad 2. Whether the rationale was costs or design (to keep the devices thin), it was still disappointing.

Hopefully this signals a growing interest to conquer the hearts and imagination of more photo-buffs.
post #7 of 26
Quote:
Originally Posted by srathi View Post

Let's see if all the iFanboys will suddenly change their mind and start liking 3D on their phones. The same iFans who blasted this feature on Android phones.

Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.
you only have freedom in choice when you know you have no choice
Reply
you only have freedom in choice when you know you have no choice
Reply
post #8 of 26
Even if Apple could do it, and make it not suck, I still think it would remain the simple Fad that it is.

From theaters, to TVs, to playstation, to Nintendo handhelds...I've yet to see a single implementation of 3D that does not absolutely suck.
post #9 of 26
Quote:
Originally Posted by cy_starkman View Post

Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.

And why is OpenGL rendering bad?
post #10 of 26
3D is the [very near] future. It would be nice to see Apple developing something in this area as well. As to the current patent application, it doesn't seem particularly original; it may not pass due to existing prior art.
post #11 of 26
Quote:
Originally Posted by cy_starkman View Post

Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.

HTC EVO 3D



4.3" (960x540) glasses-free 3D display
1.2GHz dual-core Qualcomm processor,
Dual 5 megapixel cameras
Front-facing 1.3 megapixel camera
Support for 1080p video playback via the HDMI out (or 720p for 3D content)
post #12 of 26
Quote:
Originally Posted by cy_starkman View Post

Dude, I haven't read one article about a 3D video taking / stereoscopic display android device. I have read about android phones with silly interfaces wasting mass resources to present a OpenGL type rendered look. Notionally a 3G navigation system could be good, still waiting to see it done successfully yet.

As usual, iFans are in denial. Unless Apple comes up with a feature, it is totally crap and worthless!
post #13 of 26
Current and near future 3D remains a novelty and it is also not recommended for kids to be exposed to for extended periods. Just like in the 50s, then again in the 70s, it'll fade away or remain a niche.

Proud AAPL stock owner.

 

GOA

 

Get the lowdown on the coming collapse:  http://www.cbo.gov/publication/45010

Reply

Proud AAPL stock owner.

 

GOA

 

Get the lowdown on the coming collapse:  http://www.cbo.gov/publication/45010

Reply
post #14 of 26
Quote:
Originally Posted by Ronbo View Post

What I had in mind was a video I saw of a guy who used a 2-camera device to turn a flat real-life game board into an AR zombies game, where zombies are walking down the street, and the device screen acts as a camera on the scene. He moves around gets a different view of the streets below. It was very cool.

I just picked up a Nintendo 3DS and it includes an AR game that works the same way. It uses a card to have a known position in the world and you walk around it to get different views. You can play the game with the 3D display on or off.
http://www.youtube.com/watch?v=uOa3DHgoHs0

I've also been having fun testing out the 3D camera
http://www.atariage.com/forums/topic/179570-3ds-photos/
Darrell Spice, Jr.
SpiceWare- We don't do Windows, it's too much of a chore
Artistic Overpass - nobody said roadways had to be drab
Medieval Mayhem - This ain't your father's Warlords
Reply
Darrell Spice, Jr.
SpiceWare- We don't do Windows, it's too much of a chore
Artistic Overpass - nobody said roadways had to be drab
Medieval Mayhem - This ain't your father's Warlords
Reply
post #15 of 26
Quote:
Originally Posted by DrDoppio View Post

3D is the [very near] future. It would be nice to see Apple developing something in this area as well. As to the current patent application, it doesn't seem particularly original; it may not pass due to existing prior art.

Maybe you know someone else who has taken this approach? If so, could you provide a link?

It looks like this describes a method of using specific aspects of the sensor data (chroma, luminance) to calculate the disparity between left and right views with NEW levels of precision. Like using metadata at chip level to refine the parallax geometry.

I think this is new, and if anybody has knowlege of 'prior art' can we hear about it?

One imaginable application: calculate in real time and control by software the spacing between the two camera lenses (or sensor fields perhaps) according to whether foreground or background subjects are of interest, determined by autofocus routines in the viewfinder.

I'm just guessing here. I'll be looking for this patent document, should have a link soon.
post #16 of 26
Quote:
Originally Posted by SpamSandwich View Post

Current and near future 3D remains a novelty and it is also not recommended for kids to be exposed to for extended periods. Just like in the 50s, then again in the 70s, it'll fade away or remain a niche.

Last time there was a rumor about 3D, someone posted a link to an article which explained that the way current electronic devices display 3D is different from the way the human eye is built to experience 3D. That basic incompatibility is responsible for the eye strain and vertigo many people get from viewing current 3D. Until there are some kind of holographic displays, I agree that 3D will remain a novelty. I do acknowledge that the presence of 3D cameras on an iPhone would be one way of differentiating between multiple models of iPhone, should Apple ever decide to do that.
post #17 of 26
I’ve been taking digital 3D photos for years. I bolted identical digital cameras together at the tripod mount making the cameras look like a pair of strange binoculars. I try and snap the pictures at the same time, then combine the left and right images side by side in Photoshop making sure the horizontal details line up on the same line. I then print the combined image and look at it with a the aid of a stereoscope or a lorgnette. This form of 3D image making goes back to the 1860’s and is still exciting just for the sheer sense of seeing in 3D. It has to be seen to be really appreciated, even the most ordinary of shots has a fascination because of the perceived depth. Last summer a Yankee game was broadcast with a similar kind of twin lens visual experience. It would really be something to have an iPhone, iPod, or iPad that could shoot such images.
post #18 of 26
Quote:
Originally Posted by quinney View Post

Last time there was a rumor about 3D, someone posted a link to an article which explained that the way current electronic devices display 3D is different from the way the human eye is built to experience 3D. That basic incompatibility is responsible for the eye strain and vertigo many people get from viewing current 3D. Until there are some kind of holographic displays, I agree that 3D will remain a novelty. I do acknowledge that the presence of 3D cameras on an iPhone would be one way of differentiating between multiple models of iPhone, should Apple ever decide to do that.

That would have been the Walter Murch letter to Roger Ebert. There's a rejoinder to Murch's 'misguided' conclusions here:

http://www.crunchgear.com/2011/01/24...ignment-of-3d/

Murch's point about the loss of brightness in current projection systems and TV viewing systems is valid, I think. The movie has to be worth it to transcend that loss. So far some are, many aren't.

Edit: Headaches and eyestrain are caused by bad 3D, not 3D itself, OR, by one's own binocular misalignment, not detected in ordinary eye exams. (See Sue Barry's website on 'Fixing My Gaze' if you get headaches when others don't.) There's also a learning curve for those with marginal stereo vision, and a fatigue curve, depending on the viewing system. I felt tired around the eyes after Avatar, and I've been messing with stereo for many years. It was maybe too long, a dreadfully dark and noisome movie. Ambitious, though, and quite an accomplishment.
post #19 of 26
Just give us ONE camera and Hipstamatic as standard camera, now THAT would make me really happy!
post #20 of 26
Stereo cameras on Apple devices must be for something else besides making images. They must be for measuring distance for a game or some other motion intensive use.

To create 3D photographs one needs special processing. Already the photographic film industry is almost dead. Three dimensional photography has been in existence with quality cameras for decades and it hasn't caught on.

I don't like 3D photography. It is a novelty that is OK when looking at one image. It becomes annoying with many images. Movies in 3D just take away from the story. Good stories aren't made better with 3D. I watched Avatar on my computer without the 3D effects. I really liked the movie. Having 3D effects always takes attention off the story and puts it on the effects, thus lessening the impact of the story.

The thing that makes 3D bad is that the parallax of a camera isn't the same as the one your eyes produce. All 3D therefore seems unnatural to each viewer.

When holograms that resemble actual things can be produced, our own eyes will then observe the 3D without strain. Of course people on the left side of a theater will not see the movie the people on the right side of the screen see. It would be more like viewing a play than a movie.

I just hope that 3D goes away and stays away.
post #21 of 26
Quote:
Originally Posted by Flaneur View Post

That would have been the Walter Murch letter to Roger Ebert. There's a rejoinder to Murch's 'misguided' conclusions here:

http://www.crunchgear.com/2011/01/24...ignment-of-3d/

Murch's point about the loss of brightness in current projection systems and TV viewing systems is valid, I think. The movie has to be worth it to transcend that loss. So far some are, many aren't.

Edit: Headaches and eyestrain are caused by bad 3D, not 3D itself, OR, by one's own binocular misalignment, not detected in ordinary eye exams. (See Sue Barry's website on 'Fixing My Gaze' if you get headaches when others don't.) There's also a learning curve for those with marginal stereo vision, and a fatigue curve, depending on the viewing system. I felt tired around the eyes after Avatar, and I've been messing with stereo for many years. It was maybe too long, a dreadfully dark and noisome movie. Ambitious, though, and quite an accomplishment.

From the article (emphasis added):
"Mr. Murch is perfectly right to point out the convergence/focus problem as a limitation of the medium, but for him and Ebert (whom I have rebutted before on the topic) to consider it fatal is, in my opinion, a lack of imagination and faith in the ingenuity of filmmakers."

I don't interpret that as saying that Murch is misguided, rather, it seems to agree he has a point.
No doubt filmmakers can film things differently to make the effect less painful on people, but the incompatibility with human physiology remains. If a person can view their real world surroundings without pain and only when viewing 3D electronically do the symptoms of binocular misalignment or marginal stereo vision occur, I place the blame on the medium, not the eyes.
post #22 of 26
My grandfather took thousands of 3D slides in the 1950's. Had I never seen these I would dismiss 3D as a gimmick. But those slides captured the moment in a way no regular photo could match. 3D really did add value to those pictures. The only thing odd about 3D pictures is that they can be a little creepy. You see everyone in the photo as if they are alive right in front of you, but they are frozen in motion like a Twilight Zone episode.

Please update the AppleInsider app to function in landscape mode.

Reply

Please update the AppleInsider app to function in landscape mode.

Reply
post #23 of 26
Quote:
Originally Posted by Flaneur View Post

...
It looks like this describes a method of using specific aspects of the sensor data (chroma, luminance) to calculate the disparity between left and right views with NEW levels of precision. Like using metadata at chip level to refine the parallax geometry.
...

You're merely restating the flowchart. As I said previously, there's nothing novel about it. Everyone doing 3D is using aspects of the same approach -- there's simply no other data coming from the sensor to use. As to prior art, look at the numerous examples of palpable products already provided in the thread.

It may seem that it's enough to put together a few technical terms into a flowchart in order to make up a patent-worthy idea, but in reality some actual ingenuity goes a long way...
post #24 of 26
Quote:
Originally Posted by DrDoppio View Post

You're merely restating the flowchart. As I said previously, there's nothing novel about it. Everyone doing 3D is using aspects of the same approach -- there's simply no other data coming from the sensor to use. As to prior art, look at the numerous examples of palpable products already provided in the thread.

It may seem that it's enough to put together a few technical terms into a flowchart in order to make up a patent-worthy idea, but in reality some actual ingenuity goes a long way...

You are right, I was working off the 'flowchart,' but now that I've seen the application:

http://appft.uspto.gov/netacgi/nph-P...age+Sensors%22


It looks totally novel to me -- three sensors, one being only for 3D disparity data -- but what do I know about stereo imaging at this level?

"Palpable products already provided in the thread"?? What products, what thread? Perhaps you can mention something specific that is prior art.
post #25 of 26
Quote:
Originally Posted by quinney View Post

From the article (emphasis added):
"Mr. Murch is perfectly right to point out the convergence/focus problem as a limitation of the medium, but for him and Ebert (whom I have rebutted before on the topic) to consider it fatal is, in my opinion, a lack of imagination and faith in the ingenuity of filmmakers."

I don't interpret that as saying that Murch is misguided, rather, it seems to agree he has a point.
No doubt filmmakers can film things differently to make the effect less painful on people, but the incompatibility with human physiology remains. If a person can view their real world surroundings without pain and only when viewing 3D electronically do the symptoms of binocular misalignment or marginal stereo vision occur, I place the blame on the medium, not the eyes.

Where Murch and Ebert are misguided is in condemning the medium by judging how it is misused.

I like the challenge in your last two sentences. The capture and display of stereo images isolate the two views in a way not encountered in real-world vision. This means that the 3D photographer has to be very mindful of the viewer's tolerance of disparity between the two views according to the background/foreground interest in any given scene. Many if not most current movies ignore these considerations. But it can be done, to great advantage, particularly in documentaries like the Space Station IMAX films. Werner Herzog's coming doc on the cave paintings should be definitive; I haven't seen it yet.

On your other point, there are millions of people walking around with no or dodgy stereo vision, and they're doing just fine, but not when they go to a 3D movie. (Again, Sue Barry is a must-read on this subject, if you care.) This is because of the isolation of the two views I mentioned earlier. Still no fault of the medium, because people with good stereo vision can see a kind of deep photography they've never seen before.

The problem really is that 3D is being exploited and oversold right now. It's not for everyone and every subject.
post #26 of 26
Quote:
Originally Posted by srathi View Post

Let's see if all the iFanboys will suddenly change their mind and start liking 3D on their phones. The same iFans who blasted this feature on Android phones.

It'll be one hilarious sight to see.

HP Omni 100-5100z, 500GB HDD, 4GB RAM; ASUS Transformer, 16GB, Android 4.0 ICS
Although I no longer own Apple products like I did before, I'll continue to post my opinions.

Reply

HP Omni 100-5100z, 500GB HDD, 4GB RAM; ASUS Transformer, 16GB, Android 4.0 ICS
Although I no longer own Apple products like I did before, I'll continue to post my opinions.

Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Future Apple Hardware
AppleInsider › Forums › Mac Hardware › Future Apple Hardware › Apple exploring multi-camera systems for 3D picture taking