Apple researching autostereoscopic 3-D display hardware

[AppleInsider]

Apple has been conducting research on a new breed of display hardware that would employ autostereoscopy to produce three-dimensional images that can be viewed by multiple users without the need for special headgear or glasses, AppleInsider has discovered.

The technology would consist of a projection screen having a predetermined angularly-responsive reflective surface function which would produce depth perception to the viewer even though the image is produced by a flat device, according to a filing with United States Patent and Trademark Office.

"Modern three-dimensional (3D) display technologies are increasingly popular and practical not only in computer graphics, but in other diverse environments and technologies as well," Apple said in the 25-page filing. "Growing examples include medical diagnostics, flight simulation, air traffic control, battlefield simulation, weather diagnostics, entertainment, advertising, education, animation, virtual reality, robotics, biomechanical studies, scientific visualization, and so forth."

While common forms of such displays require shuttered or passively polarized eyewear, those approaches have not met with widespread acceptance because observers generally do not like to wear equipment over their eyes, the company said. Such approaches are also said to be impractical, and essentially unworkable, for projecting a 3D image to one or more casual passersby, to a group of collaborators, or to an entire audience such as when individuated projections are desired.

As a result, Apple proposes a three-dimensional display system having a projection screen with a predetermined angularly-responsive reflective surface function. Three-dimensional images would be respectively modulated in coordination with the predetermined angularly-responsive reflective surface function to define a programmable mirror with a programmable deflection angle.

This form of technology would cater to the continuing need for such practical autostereoscopic 3D displays that can also accommodate multiple viewers independently and simultaneously, the company said. Unlike 3D glasses or googles, it would provide simultaneous viewing in which each viewer could be presented with a uniquely customized autostereoscopic 3D image that could be entirely different from that being viewed simultaneously by any of the other viewers present, all within the same viewing environment, and all with complete freedom of movement.



According to the filing, this form of display could include a 3D/stereoscopic rendering engine that renders 3D images and may be implemented in firmware, software, or hardware. The 3D/stereoscopic rendering engine could also be part of a graphics card, code running on a graphics chip's graphics processor unit, a dedicated application specific integrated circuit, specific code running on the host CPU, and so forth.

"The 3D images that are rendered by the 3D/stereoscopic rendering engine are sent to a 3D/stereoscopic display through a suitable interconnect, such as an interconnect based upon the digital video interface (DVI) standard," Apple said. "The interconnect may be either wireless (e.g., using an 802.11x Wi-Fi standard, ultra wideband (UWB), or other suitable protocol), or wired (e.g., transmitted either in analog form, or digitally such as by transition minimized differential signaling (TMDS) or low voltage differential signaling (LVDS))."



A display interface and image splitter inside the 3D/stereoscopic display would divide the 3D images from the 3D/stereoscopic rendering engine into two 3D sub-images, namely a left sub-image and a right sub-image. The left and right sub-images would be modulated (including being turned on and off) in respective image modulators to enable and control optical projection by a projector of the left and right sub-images respectively into the observer's left and right eyes

"The observer's brain then combines the two projected optical sub-images into a 3D image to provide a 3D viewing experience for the observer," the filing explains. "The deflection into the observer's respective left and right eyes is accomplished using a projection screen. The projection screen, in combination with image data properly modulated [...] forms a mirror device that is a programmable mirror with a programmable deflection angle."



Broadly speaking, Apple said, this combination constitutes the projection screen as a programmable mirror that is a spatial filter, because the combination operates to cause light to reflect from the projection screen to the observer's particular left and right eyes as a function of the spatial locations of those respective eyes, and otherwise does not reflect light -- as if the light were filtered out.

A digital signal processor (DSP) in combination with a 3D imager would also determine the correct location of an observer with respect to the projection screen. Characteristics about the observer, such as the observer's head position, head tilt, and eye separation distance with respect to the projection screen would also be determined by the DSP and the imager.

"The 3D imager may be any suitable scanner or other known device for locating and determining the positions and characteristics of each observer," the company went on to say. "Such characteristics may include, for example, the heights of the observers, head orientations (rotation and tilt), arm and hand positions, and so forth."



In some embodiments, the 3D imager may be configured as an integral part of the projector, which could be configured to directly illuminate the observer as well as the projection screen. An appropriately located light sensor would then be positioned to pick up the illumination light that is reflected from the observer, determining his or her position relative to the display.

Apple added that the 3D imager and the light sensor could also provide a means for observer input: "For example, the volume in front of the projection screen in which the observer is positioned may be constituted by the 3D display system as a virtual display volume that is echoed as a 3D display on the projection screen. The virtual display volume can then be used for observer input. In one embodiment, the observer can then actuate, for example, a 3D representation of a button to activate certain features on a virtual active desktop. Such an active desktop would be represented virtually in the virtual display volume and, by virtue of the 3D projection on the projection screen, would appear to the observer as a 3D image in the virtual display volume in the immediate presence and proximity of the observer. Other human interface behaviors are similarly possible, as will be understood by persons of ordinary skill in the art in view of the present disclosure."

In concluding its filing, originally submitted back in September of 2006, the Cupertino-based company asserts that such display technology is "straight-forward, cost-effective, uncomplicated, highly versatile and effective, can be surprisingly and unobviously implemented by adapting known technologies, and are thus fully compatible with conventional manufacturing processes and technologies."

[ View this article at AppleInsider.com ]

[Virgil-TB2]

Originally Posted by AppleInsider

... Apple added that the 3D imager and the light sensor could also provide a means for observer input: "For example, the volume in front of the projection screen in which the observer is positioned may be constituted by the 3D display system as a virtual display volume that is echoed as a 3D display on the projection screen. ...

This part kind of implies a system wherein it would appear to be an ordinary mirror when you look into it, displaying a 3-D image of yourself looking back, but also to allow for editing/alteration of that image. In other words the thing in many sci-fi movies where you can try out different "looks" at a store in what looks like some kind of magic mirror. Very cool.

[sausage&Onion]

Imagine trying to capture user position with an iSight camera!
In regards to the display not reflecting light, that technology would be incredibly useful to every designer on the planet, regardless of everything else. Very interesting indeed.

[Hobbes]

Whoa.

[Slewis]

Null.

[Crtaylor]

I could definitely see this type of technology implemented in so many ways. They could even put this in that Shake reincarnation that was rumored in 2006! After all, this patent and the "end of Shake support" announcement were in the same year.

It would also be cool if they integrated this technology into their Core Imaging and/or Animation architectures. They could even make an image-editing software around this thing!

Either way, I hope Apple is still working on all these patents that have been surfacing because all of them would kick some major PC a$$!

[blastfamy]

~"can be surprisingly and unobviously implemented by adapting known technologies, and are thus fully compatible with conventional manufacturing processes and technologies."

can you say mac OS 11?

[jbh0001]

So, now we know why refreshes to the Cinema displays are taking so long. (Just kidding.)

[Buck]

Hmm... isn't it what Philips has already done with their 3D TV sets?
http://www.research.philips.com/tech...ov_3ddisp.html
http://www.wowvx.com/
I've seen one in action in a store.

[wbrasington]

Originally Posted by AppleInsider

Unlike 3D glasses or googles, it would provide simultaneous viewing......

A little slip there..... wonder if the yahoos are laughing at the googles?

[HyteProsector]

Heres an example of what I'm envisioning it to look like. Minus the part where you wear something. Mostly just the way the screen looks. Not sure though. Check this out any how its pretty impressive, and theres not doubt that this is definitely the future for gaming. Definitely.

http://www.youtube.com/watch?v=Jd3-eiid-Uw

[Olternaut]

All I can say is....oh my freakin gawd!! For me this is totally out of the blue.
What the hell is Apple up to I wonder?

[desarc]

they're just trying to copy the tremendous success of the virtual boy.

[yeah i read the article, they said no goggles, i know i know i know]

[fARGINaPPLES]

Originally Posted by Crtaylor

I could definitely see this type of technology implemented in so many ways. They could even put this in that Shake reincarnation that was rumored in 2006! After all, this patent and the "end of Shake support" announcement were in the same year.

It would also be cool if they integrated this technology into their Core Imaging and/or Animation architectures. They could even make an image-editing software around this thing!

Either way, I hope Apple is still working on all these patents that have been surfacing because all of them would kick some major PC a$$!

Ok, ok, calm down! What you're imagining and what is realistic are two completely separate things. Let me start by saying that just because Apple is working on it, doesn't mean it's going to be any good. I worked in a virtual reality lab as a research assistant, and we exhausted the term "Presence". In a nutshell, presence is the spectator's rating of how close to real something is. For example, iChat is a really good product because it allows me to video chat with my neice who lives 3,000 miles away from me. Yet, there is no doubt in my mind that she is still 3,000 miles away.

Holographics and synthesized 3-dimensional images are still very early in their respective development, that a spectator would hardly feel the presence required to make their experience a "genuinely realistic" one. Let's put aside the novelty of the idea that this technology might occur. Instead, think about how practical this technology would be from a normal user position. How many of you would greatly benefit from having a holographic representation of what you see on your computer screen? You can't touch it, you can't move it around without some type of plugged in device. You can't really do anything that you wouldn't already be able to do with the "magical computer screen" in front of you!

On another note, system resources would become an issue immediately, bogging your system memory down to the point that you could only run one program at a time, and not very efficiently.

So, I don't mean to be the guy who brings the room down, but just because Apple is doing it doesn't mean it's gonna be perfect. Furthermore, just because Apple is doing it, it doesn't mean that you have to immediately go out and buy it as soon as it hits the shelves. I honestly feel that this is going to take a lot of development, and tons of R & D, and still will flop. Oh, and did i mention that it's going to be expensive as h3ll?

Thanks for your time!

[Mind_Fast]

Considering how groundbreaking and innovative this company has been over the years what one of the ultimate challenges would be (and I SERIOUSLY think that's what they're after) is bringing touch to a 3D interface.

Apple brought touch to the masses, now combine it with this patent! It's what Tom Cruise did in Minority Report...even without the Gloves (although I think that was the coolest gadget in the movie).

Everything you do on your mac in 3D, imagine pressing a button in the air, an interfare that reacts actively to your movement not simply following your head position.

Wouldn't that be AMAZING...?

[jragosta]

Originally Posted by fARGINaPPLES

On another note, system resources would become an issue immediately, bogging your system memory down to the point that you could only run one program at a time, and not very efficiently.

While I agree with the rest of your comments, I don't know about this one. As I read the patent, it's simply a matter of storing and manipulating two images rather than one. Any reasonably competent graphics card can handle that.

I believe that Dell already has a computer which does essentially this - they have two different images that map to the screen and have a funky screen where half the pixels display to the right and half display to the left. You only get 3D if you're exactly in front of it. Apple's solution allows the viewer(s) to sit at an angle, but it's not obvious (to me, at least) that it's going to require orders of magnitude more computer power or memory.

[fARGINaPPLES]

Originally Posted by jragosta

While I agree with the rest of your comments, I don't know about this one. As I read the patent, it's simply a matter of storing and manipulating two images rather than one. Any reasonably competent graphics card can handle that.

I believe that Dell already has a computer which does essentially this - they have two different images that map to the screen and have a funky screen where half the pixels display to the right and half display to the left. You only get 3D if you're exactly in front of it. Apple's solution allows the viewer(s) to sit at an angle, but it's not obvious (to me, at least) that it's going to require orders of magnitude more computer power or memory.

This brings to light another issue that would make me hesitate to follow the sheep. If we're basically watching a movie, then how is this innovative? So, it sounds like two movies of the same thing from slightly different views playing simultaneously, and over-laid. First things first, don't try to trick my eye with smoke and mirrors. If Apple is going to create the "illusion" of 3-D, then i don't want it. I want the real 3-D experience!

Then of course buying the "special equipment" to view the cheap imagery will be another issue. I say we have Adobe develop all of this using Flash! j/k! but seriously!

[melgross]

In another thread, some of us were talking about how the iMac could evolve from the form that it has now. I was saying that the way it is now is about as far as it can go, even though we could see some advances that wouldn't change the computer in a screen concept.

My reasoning was that this is about the prefect end to a computer design. The only other advance would be holographic displays, and that they would not be good for everything.

This comes at a good time for the discussion.

This couldn't replace the thin one piece computer that the iMac represents.

[echosonic]

3D Peep Shows.

That's all I'm gonna say.

[fARGINaPPLES]

Originally Posted by echosonic

3D Peep Shows.

That's all I'm gonna say.

word!

[ajmas]

A few things worth mentioning:
- Johnny Chung Lee did something like this, but with a Wiimote. See last video here: http://www.cs.cmu.edu/~johnny/projects/wii/
- OpenGL is able to support stereoscopic imaging
- How ever good this is, it is only useful for the main viewer.

[Olternaut]

Originally Posted by fARGINaPPLES

Ok, ok, calm down! What you're imagining and what is realistic are two completely separate things. Let me start by saying that just because Apple is working on it, doesn't mean it's going to be any good. I worked in a virtual reality lab as a research assistant, and we exhausted the term "Presence". In a nutshell, presence is the spectator's rating of how close to real something is. For example, iChat is a really good product because it allows me to video chat with my neice who lives 3,000 miles away from me. Yet, there is no doubt in my mind that she is still 3,000 miles away.

Holographics and synthesized 3-dimensional images are still very early in their respective development, that a spectator would hardly feel the presence required to make their experience a "genuinely realistic" one. Let's put aside the novelty of the idea that this technology might occur. Instead, think about how practical this technology would be from a normal user position. How many of you would greatly benefit from having a holographic representation of what you see on your computer screen? You can't touch it, you can't move it around without some type of plugged in device. You can't really do anything that you wouldn't already be able to do with the "magical computer screen" in front of you!

On another note, system resources would become an issue immediately, bogging your system memory down to the point that you could only run one program at a time, and not very efficiently.

So, I don't mean to be the guy who brings the room down, but just because Apple is doing it doesn't mean it's gonna be perfect. Furthermore, just because Apple is doing it, it doesn't mean that you have to immediately go out and buy it as soon as it hits the shelves. I honestly feel that this is going to take a lot of development, and tons of R & D, and still will flop. Oh, and did i mention that it's going to be expensive as h3ll?

Thanks for your time!

Yeah but umm......this isn't the company you worked for......this is Apple. Your company couldn't make it work and/or couldn't make a viable product out of the tech. But when Apple approaches a problem they work on it.....til it works.
We might not see Apple debuting something using this tech in a product tommorow. But I have a feeling it might be sooner than you think.

[SpamSandwich]

I could see this becoming an Apple 3D movie system... bring on the 3D RED cameras!

[fARGINaPPLES]

Originally Posted by Olternaut

Yeah but umm......this isn't the company you worked for......this is Apple. Your company couldn't make it work and/or couldn't make a viable product out of the tech.

wow! haha! calm down. I never said that my research team could or couldn't make it work. Olternaut, I think you're missing the point here. I'm not suggesting that it's impossible or that it will be a long time coming. To put it simply, I'm saying that the tech that Apple is working on is something that will cost a lot of money for lay-people end-users like you and me, and that it won't be as much of an innovative break-through as what you might think.

Furthermore, Olternaut, you made these claims with no support. All you said was that Apple can do it. Am i going to have to pull out my freshman level philosophy book and explain rhetoric and fallacy to you? What i want to see is some research that shows that this product will be feasible for consumers, and that it won't cause me to take out a second mortgage to own it.

I appreciate your enthusiasm for Apple, as I share the same sentiment, but we are all entitled to our opinions in this forum, and I think you shouldn't flame someone until you know the whole story about them. At least give me an opportunity to be an idiot before you call me one.

[Frank777]

Wonderful. Everyone's waiting for new displays, but Apple wants to build a holodeck.

Don't those clowns realize how much trouble that thing can cause? Moriarty will kill us all.

[webmail]

This isn't anything new. Many companies have been offering these displays even in laptops at CES for several years now.

It's just that most of them don't follow through by offering it "built into the OS". If anyone can make this technology stick it's Apple.

[8CoreWhore]

I'd be happy with a new 2D display.

[melgross]

I've done work with holography, so a few years ago, Mercedes Benz, a client of mine, asked me to build a display that would be one of their cars, complete. They wanted it to be an outside view, but when you walked through the side, you would see the insides.

I told them to come back to me in 2020.

Hmmm! That may have been early.

[Olternaut]

Originally Posted by fARGINaPPLES

wow! haha! calm down. I never said that my research team could or couldn't make it work. Olternaut, I think you're missing the point here. I'm not suggesting that it's impossible or that it will be a long time coming. To put it simply, I'm saying that the tech that Apple is working on is something that will cost a lot of money for lay-people end-users like you and me, and that it won't be as much of an innovative break-through as what you might think.

Furthermore, Olternaut, you made these claims with no support. All you said was that Apple can do it. Am i going to have to pull out my freshman level philosophy book and explain rhetoric and fallacy to you? What i want to see is some research that shows that this product will be feasible for consumers, and that it won't cause me to take out a second mortgage to own it.

I appreciate your enthusiasm for Apple, as I share the same sentiment, but we are all entitled to our opinions in this forum, and I think you shouldn't flame someone until you know the whole story about them. At least give me an opportunity to be an idiot before you call me one.

Who said you were being flamed?
Can't a guy have a strong opinion without being accused of being a flamer? Its not like I insulted you or something.
And I just re-read your post. You can't just casually say stuff like I called you an idiot when I clearly didn't say so. And thats even if you said that in jest.

[Olternaut]

Originally Posted by melgross

I've done work with holography, so a few years ago, Mercedes Benz, a client of mine, asked me to build a display that would be one of their cars, complete. They wanted it to be an outside view, but when you walked through the side, you would see the insides.

I told them to come back to me in 2020.

Hmmm! That may have been early.

The holodeck will still make whatever people are working on....even Apple's project look like crap. But one thing I've learned about apple is that they should not be underestimated....ever.
So lets see what this thing of theirs turns out to be. Maybe we will get a peak at it at MacWorld 2009.

[melgross]

Originally Posted by Olternaut

The holodeck will still make whatever people are working on....even Apple's project look like crap. But one thing I've learned about apple is that they should not be underestimated....ever.
So lets see what this thing of theirs turns out to be. Maybe we will get a peak at it at MacWorld 2009.

I'm sorry to have to say it, but the Holodeck is one of those things that's not likely to ever work. I wish it could be done, but it can't.

[palomine]

Each new patent filing just excites me with the greatest joy. I love to pore over these stories and imagine what could be. Probably not a common reaction, (We have inventors in the family) but I get a real kick out of watching Apple.

In this haunting, dire time we live in!

I am grateful for a company that sends up new fireworks for us all to ooh and ahhh over. Some are duds, some are those 5-in-1 splendors.

Who else is doing anything like this nowadays?
Every critique has an effect on Apple. Do they study everything?

There was a guy in here who wanted to track his podcasts automatically so he could get the next lesson for his foreign language study. 2 days later Apple had some code ready and filed a patent.

3D image screens. Even with that inventor DNA in me, I am puzzling over how this one would work. Wow.

[palomine]

Let me start by saying that just because Apple is working on it, doesn't mean it's going to be any good. I worked in a virtual reality lab as a research assistant, and we exhausted the term "Presence".

So, I don't mean to be the guy who brings the room down, but just because Apple is doing it doesn't mean it's gonna be perfect. Furthermore, just because Apple is doing it, it doesn't mean that you have to immediately go out and buy it as soon as it hits the shelves. I honestly feel that this is going to take a lot of development, and tons of R & D, and still will flop.

Duhh, yeah, it's well known that all VR (Virtual reality) attempts have ended in way over budget disaster. Same can be said for the quest for AI. That's obvious. It's been difficult to even get funding for either due to the legendary failures in both.

If Apple is taking it on it will be in a sensible stepwise and coherent way, No?

[Ronbo]

Originally Posted by fARGINaPPLES

So, I don't mean to be the guy who brings the room down, but just because Apple is doing it doesn't mean it's gonna be perfect.

You're right, of course.

Originally Posted by fARGINaPPLES

Furthermore, just because Apple is doing it, it doesn't mean that you have to immediately go out and buy it as soon as it hits the shelves.

You're wrong, of course.

[palegolas]

Ok I only read like 70% of the article, so perhaps I missed some vital detail. But I still don't understand how this will work on multiple persons at once. It sounds like it's stereoscopic*100 or something to cover all angles. Hm...

[jragosta]

Originally Posted by fARGINaPPLES

This brings to light another issue that would make me hesitate to follow the sheep. If we're basically watching a movie, then how is this innovative? So, it sounds like two movies of the same thing from slightly different views playing simultaneously, and over-laid. First things first, don't try to trick my eye with smoke and mirrors. If Apple is going to create the "illusion" of 3-D, then i don't want it. I want the real 3-D experience!

What's innovative is that it apparently allows the 3D experience with a single projector and screen as seen by multiple viewers.

If you want the 'REAL 3D experience' instead of illusions, perhaps you should turn your TV off and go out into the real world. Even if we reach the technology level of the Holodeck, it's still illusion. Only the real world is real.

[Olternaut]

Originally Posted by palomine

Each new patent filing just excites me with the greatest joy. I love to pore over these stories and imagine what could be. Probably not a common reaction, (We have inventors in the family) but I get a real kick out of watching Apple.

In this haunting, dire time we live in!

I am grateful for a company that sends up new fireworks for us all to ooh and ahhh over. Some are duds, some are those 5-in-1 splendors.

Who else is doing anything like this nowadays?
Every critique has an effect on Apple. Do they study everything?

There was a guy in here who wanted to track his podcasts automatically so he could get the next lesson for his foreign language study. 2 days later Apple had some code ready and filed a patent.

3D image screens. Even with that inventor DNA in me, I am puzzling over how this one would work. Wow.

I wonder...does anyone think that anyone important over at Apple actually studies these forums? Do they even care?
Do some of our sometimes crazy opinions and theories really effect Apple in any way?

[jragosta]

Originally Posted by Olternaut

I wonder...does anyone think that anyone important over at Apple actually studies these forums? Do they even care?
Do some of our sometimes crazy opinions and theories really effect Apple in any way?

I could see them reading Crazy Apple Rumors every day, but now that they're gone, what's the point?

[Aqua OS X]

Apple's attempts to address crappy stereoscopic viewing angles are interesting, but, holographic displays already do that, and stereo graphics will never be able to address ocular accommodation. In other words, you can't shift your focus between the foreground and background of an image. If you had to look at a stereo graphic display all day long you would likely have a splitting headache.

Volumetric displays are the only real solution to this problem...
http://www.actuality-medical.com/indexAS.html
http://www.eventdesignmag.com/viewmedia.asp?prmMID=257
...but they require space. :shrug:

[vinea]

Originally Posted by melgross

I've done work with holography, so a few years ago, Mercedes Benz, a client of mine, asked me to build a display that would be one of their cars, complete. They wanted it to be an outside view, but when you walked through the side, you would see the insides.

I told them to come back to me in 2020.

Hmmm! That may have been early.

You could do it with wide FOV HMD...of course it's somewhat bulky and not all that elegant.

It is actually already used by car companies though for design reviews.

An alternative is a hand held tablet (say as light as an air) that allows you to view an invisible car through it in 3D space based on "head tracking" tied to the tablet.

Many ways to skin the cat...but for holodeck...yes, 2020 is a bit soon.