Apple researching autostereoscopic 3-D display hardware
Apple has been conducting research on a new breed of display hardware that would employ autostereoscopy to produce three-dimensional images that can be viewed by multiple users without the need for special headgear or glasses, AppleInsider has discovered.
The technology would consist of a projection screen having a predetermined angularly-responsive reflective surface function which would produce depth perception to the viewer even though the image is produced by a flat device, according to a filing with United States Patent and Trademark Office.
"Modern three-dimensional (3D) display technologies are increasingly popular and practical not only in computer graphics, but in other diverse environments and technologies as well," Apple said in the 25-page filing. "Growing examples include medical diagnostics, flight simulation, air traffic control, battlefield simulation, weather diagnostics, entertainment, advertising, education, animation, virtual reality, robotics, biomechanical studies, scientific visualization, and so forth."
While common forms of such displays require shuttered or passively polarized eyewear, those approaches have not met with widespread acceptance because observers generally do not like to wear equipment over their eyes, the company said. Such approaches are also said to be impractical, and essentially unworkable, for projecting a 3D image to one or more casual passersby, to a group of collaborators, or to an entire audience such as when individuated projections are desired.
As a result, Apple proposes a three-dimensional display system having a projection screen with a predetermined angularly-responsive reflective surface function. Three-dimensional images would be respectively modulated in coordination with the predetermined angularly-responsive reflective surface function to define a programmable mirror with a programmable deflection angle.
This form of technology would cater to the continuing need for such practical autostereoscopic 3D displays that can also accommodate multiple viewers independently and simultaneously, the company said. Unlike 3D glasses or googles, it would provide simultaneous viewing in which each viewer could be presented with a uniquely customized autostereoscopic 3D image that could be entirely different from that being viewed simultaneously by any of the other viewers present, all within the same viewing environment, and all with complete freedom of movement.
According to the filing, this form of display could include a 3D/stereoscopic rendering engine that renders 3D images and may be implemented in firmware, software, or hardware. The 3D/stereoscopic rendering engine could also be part of a graphics card, code running on a graphics chip's graphics processor unit, a dedicated application specific integrated circuit, specific code running on the host CPU, and so forth.
"The 3D images that are rendered by the 3D/stereoscopic rendering engine are sent to a 3D/stereoscopic display through a suitable interconnect, such as an interconnect based upon the digital video interface (DVI) standard," Apple said. "The interconnect may be either wireless (e.g., using an 802.11x Wi-Fi standard, ultra wideband (UWB), or other suitable protocol), or wired (e.g., transmitted either in analog form, or digitally such as by transition minimized differential signaling (TMDS) or low voltage differential signaling (LVDS))."
A display interface and image splitter inside the 3D/stereoscopic display would divide the 3D images from the 3D/stereoscopic rendering engine into two 3D sub-images, namely a left sub-image and a right sub-image. The left and right sub-images would be modulated (including being turned on and off) in respective image modulators to enable and control optical projection by a projector of the left and right sub-images respectively into the observer's left and right eyes
"The observer's brain then combines the two projected optical sub-images into a 3D image to provide a 3D viewing experience for the observer," the filing explains. "The deflection into the observer's respective left and right eyes is accomplished using a projection screen. The projection screen, in combination with image data properly modulated [...] forms a mirror device that is a programmable mirror with a programmable deflection angle."
Broadly speaking, Apple said, this combination constitutes the projection screen as a programmable mirror that is a spatial filter, because the combination operates to cause light to reflect from the projection screen to the observer's particular left and right eyes as a function of the spatial locations of those respective eyes, and otherwise does not reflect light -- as if the light were filtered out.
A digital signal processor (DSP) in combination with a 3D imager would also determine the correct location of an observer with respect to the projection screen. Characteristics about the observer, such as the observer's head position, head tilt, and eye separation distance with respect to the projection screen would also be determined by the DSP and the imager.
"The 3D imager may be any suitable scanner or other known device for locating and determining the positions and characteristics of each observer," the company went on to say. "Such characteristics may include, for example, the heights of the observers, head orientations (rotation and tilt), arm and hand positions, and so forth."
In some embodiments, the 3D imager may be configured as an integral part of the projector, which could be configured to directly illuminate the observer as well as the projection screen. An appropriately located light sensor would then be positioned to pick up the illumination light that is reflected from the observer, determining his or her position relative to the display.
Apple added that the 3D imager and the light sensor could also provide a means for observer input: "For example, the volume in front of the projection screen in which the observer is positioned may be constituted by the 3D display system as a virtual display volume that is echoed as a 3D display on the projection screen. The virtual display volume can then be used for observer input. In one embodiment, the observer can then actuate, for example, a 3D representation of a button to activate certain features on a virtual active desktop. Such an active desktop would be represented virtually in the virtual display volume and, by virtue of the 3D projection on the projection screen, would appear to the observer as a 3D image in the virtual display volume in the immediate presence and proximity of the observer. Other human interface behaviors are similarly possible, as will be understood by persons of ordinary skill in the art in view of the present disclosure."
In concluding its filing, originally submitted back in September of 2006, the Cupertino-based company asserts that such display technology is "straight-forward, cost-effective, uncomplicated, highly versatile and effective, can be surprisingly and unobviously implemented by adapting known technologies, and are thus fully compatible with conventional manufacturing processes and technologies."
The technology would consist of a projection screen having a predetermined angularly-responsive reflective surface function which would produce depth perception to the viewer even though the image is produced by a flat device, according to a filing with United States Patent and Trademark Office.
"Modern three-dimensional (3D) display technologies are increasingly popular and practical not only in computer graphics, but in other diverse environments and technologies as well," Apple said in the 25-page filing. "Growing examples include medical diagnostics, flight simulation, air traffic control, battlefield simulation, weather diagnostics, entertainment, advertising, education, animation, virtual reality, robotics, biomechanical studies, scientific visualization, and so forth."
While common forms of such displays require shuttered or passively polarized eyewear, those approaches have not met with widespread acceptance because observers generally do not like to wear equipment over their eyes, the company said. Such approaches are also said to be impractical, and essentially unworkable, for projecting a 3D image to one or more casual passersby, to a group of collaborators, or to an entire audience such as when individuated projections are desired.
As a result, Apple proposes a three-dimensional display system having a projection screen with a predetermined angularly-responsive reflective surface function. Three-dimensional images would be respectively modulated in coordination with the predetermined angularly-responsive reflective surface function to define a programmable mirror with a programmable deflection angle.
This form of technology would cater to the continuing need for such practical autostereoscopic 3D displays that can also accommodate multiple viewers independently and simultaneously, the company said. Unlike 3D glasses or googles, it would provide simultaneous viewing in which each viewer could be presented with a uniquely customized autostereoscopic 3D image that could be entirely different from that being viewed simultaneously by any of the other viewers present, all within the same viewing environment, and all with complete freedom of movement.
According to the filing, this form of display could include a 3D/stereoscopic rendering engine that renders 3D images and may be implemented in firmware, software, or hardware. The 3D/stereoscopic rendering engine could also be part of a graphics card, code running on a graphics chip's graphics processor unit, a dedicated application specific integrated circuit, specific code running on the host CPU, and so forth.
"The 3D images that are rendered by the 3D/stereoscopic rendering engine are sent to a 3D/stereoscopic display through a suitable interconnect, such as an interconnect based upon the digital video interface (DVI) standard," Apple said. "The interconnect may be either wireless (e.g., using an 802.11x Wi-Fi standard, ultra wideband (UWB), or other suitable protocol), or wired (e.g., transmitted either in analog form, or digitally such as by transition minimized differential signaling (TMDS) or low voltage differential signaling (LVDS))."
A display interface and image splitter inside the 3D/stereoscopic display would divide the 3D images from the 3D/stereoscopic rendering engine into two 3D sub-images, namely a left sub-image and a right sub-image. The left and right sub-images would be modulated (including being turned on and off) in respective image modulators to enable and control optical projection by a projector of the left and right sub-images respectively into the observer's left and right eyes
"The observer's brain then combines the two projected optical sub-images into a 3D image to provide a 3D viewing experience for the observer," the filing explains. "The deflection into the observer's respective left and right eyes is accomplished using a projection screen. The projection screen, in combination with image data properly modulated [...] forms a mirror device that is a programmable mirror with a programmable deflection angle."
Broadly speaking, Apple said, this combination constitutes the projection screen as a programmable mirror that is a spatial filter, because the combination operates to cause light to reflect from the projection screen to the observer's particular left and right eyes as a function of the spatial locations of those respective eyes, and otherwise does not reflect light -- as if the light were filtered out.
A digital signal processor (DSP) in combination with a 3D imager would also determine the correct location of an observer with respect to the projection screen. Characteristics about the observer, such as the observer's head position, head tilt, and eye separation distance with respect to the projection screen would also be determined by the DSP and the imager.
"The 3D imager may be any suitable scanner or other known device for locating and determining the positions and characteristics of each observer," the company went on to say. "Such characteristics may include, for example, the heights of the observers, head orientations (rotation and tilt), arm and hand positions, and so forth."
In some embodiments, the 3D imager may be configured as an integral part of the projector, which could be configured to directly illuminate the observer as well as the projection screen. An appropriately located light sensor would then be positioned to pick up the illumination light that is reflected from the observer, determining his or her position relative to the display.
Apple added that the 3D imager and the light sensor could also provide a means for observer input: "For example, the volume in front of the projection screen in which the observer is positioned may be constituted by the 3D display system as a virtual display volume that is echoed as a 3D display on the projection screen. The virtual display volume can then be used for observer input. In one embodiment, the observer can then actuate, for example, a 3D representation of a button to activate certain features on a virtual active desktop. Such an active desktop would be represented virtually in the virtual display volume and, by virtue of the 3D projection on the projection screen, would appear to the observer as a 3D image in the virtual display volume in the immediate presence and proximity of the observer. Other human interface behaviors are similarly possible, as will be understood by persons of ordinary skill in the art in view of the present disclosure."
In concluding its filing, originally submitted back in September of 2006, the Cupertino-based company asserts that such display technology is "straight-forward, cost-effective, uncomplicated, highly versatile and effective, can be surprisingly and unobviously implemented by adapting known technologies, and are thus fully compatible with conventional manufacturing processes and technologies."
Comments
... Apple added that the 3D imager and the light sensor could also provide a means for observer input: "For example, the volume in front of the projection screen in which the observer is positioned may be constituted by the 3D display system as a virtual display volume that is echoed as a 3D display on the projection screen. ...
This part kind of implies a system wherein it would appear to be an ordinary mirror when you look into it, displaying a 3-D image of yourself looking back, but also to allow for editing/alteration of that image. In other words the thing in many sci-fi movies where you can try out different "looks" at a store in what looks like some kind of magic mirror. Very cool.
In regards to the display not reflecting light, that technology would be incredibly useful to every designer on the planet, regardless of everything else. Very interesting indeed.
It would also be cool if they integrated this technology into their Core Imaging and/or Animation architectures. They could even make an image-editing software around this thing!
Either way, I hope Apple is still working on all these patents that have been surfacing because all of them would kick some major PC a$$!
can you say mac OS 11?
http://www.research.philips.com/tech...ov_3ddisp.html
http://www.wowvx.com/
I've seen one in action in a store.
Unlike 3D glasses or googles, it would provide simultaneous viewing......
A little slip there..... wonder if the yahoos are laughing at the googles?
http://www.youtube.com/watch?v=Jd3-eiid-Uw
What the hell is Apple up to I wonder?
[yeah i read the article, they said no goggles, i know i know i know]
I could definitely see this type of technology implemented in so many ways. They could even put this in that Shake reincarnation that was rumored in 2006! After all, this patent and the "end of Shake support" announcement were in the same year.
It would also be cool if they integrated this technology into their Core Imaging and/or Animation architectures. They could even make an image-editing software around this thing!
Either way, I hope Apple is still working on all these patents that have been surfacing because all of them would kick some major PC a$$!
Ok, ok, calm down! What you're imagining and what is realistic are two completely separate things. Let me start by saying that just because Apple is working on it, doesn't mean it's going to be any good. I worked in a virtual reality lab as a research assistant, and we exhausted the term "Presence". In a nutshell, presence is the spectator's rating of how close to real something is. For example, iChat is a really good product because it allows me to video chat with my neice who lives 3,000 miles away from me. Yet, there is no doubt in my mind that she is still 3,000 miles away.
Holographics and synthesized 3-dimensional images are still very early in their respective development, that a spectator would hardly feel the presence required to make their experience a "genuinely realistic" one. Let's put aside the novelty of the idea that this technology might occur. Instead, think about how practical this technology would be from a normal user position. How many of you would greatly benefit from having a holographic representation of what you see on your computer screen? You can't touch it, you can't move it around without some type of plugged in device. You can't really do anything that you wouldn't already be able to do with the "magical computer screen" in front of you!
On another note, system resources would become an issue immediately, bogging your system memory down to the point that you could only run one program at a time, and not very efficiently.
So, I don't mean to be the guy who brings the room down, but just because Apple is doing it doesn't mean it's gonna be perfect. Furthermore, just because Apple is doing it, it doesn't mean that you have to immediately go out and buy it as soon as it hits the shelves. I honestly feel that this is going to take a lot of development, and tons of R & D, and still will flop. Oh, and did i mention that it's going to be expensive as h3ll?
Thanks for your time!
Apple brought touch to the masses, now combine it with this patent! It's what Tom Cruise did in Minority Report...even without the Gloves (although I think that was the coolest gadget in the movie).
Everything you do on your mac in 3D, imagine pressing a button in the air, an interfare that reacts actively to your movement not simply following your head position.
Wouldn't that be AMAZING...?
On another note, system resources would become an issue immediately, bogging your system memory down to the point that you could only run one program at a time, and not very efficiently.
While I agree with the rest of your comments, I don't know about this one. As I read the patent, it's simply a matter of storing and manipulating two images rather than one. Any reasonably competent graphics card can handle that.
I believe that Dell already has a computer which does essentially this - they have two different images that map to the screen and have a funky screen where half the pixels display to the right and half display to the left. You only get 3D if you're exactly in front of it. Apple's solution allows the viewer(s) to sit at an angle, but it's not obvious (to me, at least) that it's going to require orders of magnitude more computer power or memory.
While I agree with the rest of your comments, I don't know about this one. As I read the patent, it's simply a matter of storing and manipulating two images rather than one. Any reasonably competent graphics card can handle that.
I believe that Dell already has a computer which does essentially this - they have two different images that map to the screen and have a funky screen where half the pixels display to the right and half display to the left. You only get 3D if you're exactly in front of it. Apple's solution allows the viewer(s) to sit at an angle, but it's not obvious (to me, at least) that it's going to require orders of magnitude more computer power or memory.
This brings to light another issue that would make me hesitate to follow the sheep. If we're basically watching a movie, then how is this innovative? So, it sounds like two movies of the same thing from slightly different views playing simultaneously, and over-laid. First things first, don't try to trick my eye with smoke and mirrors. If Apple is going to create the "illusion" of 3-D, then i don't want it. I want the real 3-D experience!
Then of course buying the "special equipment" to view the cheap imagery will be another issue. I say we have Adobe develop all of this using Flash! j/k! but seriously!
My reasoning was that this is about the prefect end to a computer design. The only other advance would be holographic displays, and that they would not be good for everything.
This comes at a good time for the discussion.
This couldn't replace the thin one piece computer that the iMac represents.
That's all I'm gonna say.
3D Peep Shows.
That's all I'm gonna say.
word!
- Johnny Chung Lee did something like this, but with a Wiimote. See last video here: http://www.cs.cmu.edu/~johnny/projects/wii/
- OpenGL is able to support stereoscopic imaging
- How ever good this is, it is only useful for the main viewer.