Apple's display tech lets users interact with 3D objects in mid-air
A document discovered on Thursday describes an interactive three-dimensional display system that allows users to "touch" objects in mid-air, presenting the illusion of an advanced hologram.
Giant Mirage 3D hologram device uses parabolic mirrors to create illusion of floating three-dimensional objects. | Source: Optigone
The U.S. Patent and Trademark Office published an Apple patent application for an "Interactive three-dimensional display system," which details a method of presenting users with what appears to be a 3D image that can be manipulated with touches, swipes and other gestures.
In practice, Apple's invention uses a variety of known display techniques, along with a bevy of sensors, to present the pseudo-hologram. The basis of the technology can be found in the popular UFO-shaped mirror boxes that "project" a 360-degree 3D image of an object placed inside. To the user, a strawberry, stone or other solid entity seemingly floats just above an opening in the middle of the box.
While the generic mirror box relies solely on optical illusions to function, Apple's solution is a bit more complex.
The technique is comprised of three parts: a display system that creates a primary 3D image, an optical system that translates the first image into a secondary 3D image in mid-air and a sensor assembly to log user input. Support structures include a processing unit and control circuitry to facilitate feedback.
Source: USPTO
First, the image being projected is digital and not a reflection of a physical object. Apple describes a system in which infrared lasers, or other light emitting devices, project an image into a medium such as a non-linear crystal.
The medium itself may serve as an optical frequency up-converter for light passing through. When configured correctly, the medium can mix and up-convert infrared laser light to the visible spectrum, thus creating a primary 3D image.
According to the document, the non-linear medium is located between two parabolic mirrors, like the aforementioned mirror box. The primary image is reflected off the upper mirror to the lower mirror and ultimately out of a hole in the top mirror. As a consequence of this internal reflection, a secondary hologram-like image appears just above the upper mirror.
Next, a 3D input detection system collects and translates user motion data. To detect movement in 3D space, an infrared, ultraviolet, x-ray or other laser is coupled to a beam expander positioned in the bottom mirror assembly. The laser emits a beam that exits the top mirror and strikes a user's hand, finger or other control object. Reflected light is collected by image sensors and translated into positioning data by the onboard processor.
In addition to narrowing down a control surface's location, other sensors can be disposed to track gestures such as "pinch-to-zoom," swipes and presses. This data is fed into the feedback mechanism, which adjusts the primary 3D image accordingly.
Apple's 3D image presentation and tracking system is similar to a 360-degree viewable display debuted by Microsoft Research in 2011. As seen above, the system, dubbed "Vermeer," used many of the same concepts as Apple's invention to achieve a fairly high-resolution interactive image.
Apple's interactive 3D image display patent application was first filed for in 2012 and credits Christoph H. Krah and Marduke Yousefpor as its inventors.
Giant Mirage 3D hologram device uses parabolic mirrors to create illusion of floating three-dimensional objects. | Source: Optigone
The U.S. Patent and Trademark Office published an Apple patent application for an "Interactive three-dimensional display system," which details a method of presenting users with what appears to be a 3D image that can be manipulated with touches, swipes and other gestures.
In practice, Apple's invention uses a variety of known display techniques, along with a bevy of sensors, to present the pseudo-hologram. The basis of the technology can be found in the popular UFO-shaped mirror boxes that "project" a 360-degree 3D image of an object placed inside. To the user, a strawberry, stone or other solid entity seemingly floats just above an opening in the middle of the box.
While the generic mirror box relies solely on optical illusions to function, Apple's solution is a bit more complex.
The technique is comprised of three parts: a display system that creates a primary 3D image, an optical system that translates the first image into a secondary 3D image in mid-air and a sensor assembly to log user input. Support structures include a processing unit and control circuitry to facilitate feedback.
Source: USPTO
First, the image being projected is digital and not a reflection of a physical object. Apple describes a system in which infrared lasers, or other light emitting devices, project an image into a medium such as a non-linear crystal.
The medium itself may serve as an optical frequency up-converter for light passing through. When configured correctly, the medium can mix and up-convert infrared laser light to the visible spectrum, thus creating a primary 3D image.
According to the document, the non-linear medium is located between two parabolic mirrors, like the aforementioned mirror box. The primary image is reflected off the upper mirror to the lower mirror and ultimately out of a hole in the top mirror. As a consequence of this internal reflection, a secondary hologram-like image appears just above the upper mirror.
Next, a 3D input detection system collects and translates user motion data. To detect movement in 3D space, an infrared, ultraviolet, x-ray or other laser is coupled to a beam expander positioned in the bottom mirror assembly. The laser emits a beam that exits the top mirror and strikes a user's hand, finger or other control object. Reflected light is collected by image sensors and translated into positioning data by the onboard processor.
In addition to narrowing down a control surface's location, other sensors can be disposed to track gestures such as "pinch-to-zoom," swipes and presses. This data is fed into the feedback mechanism, which adjusts the primary 3D image accordingly.
Apple's 3D image presentation and tracking system is similar to a 360-degree viewable display debuted by Microsoft Research in 2011. As seen above, the system, dubbed "Vermeer," used many of the same concepts as Apple's invention to achieve a fairly high-resolution interactive image.
Apple's interactive 3D image display patent application was first filed for in 2012 and credits Christoph H. Krah and Marduke Yousefpor as its inventors.
Comments
[IMG]http://forums.appleinsider.com/content/type/61/id/42243/width/300/height/600[/IMG]
http://appleinsider.com/articles/13/08/20/apple-patents-3d-gesture-ui-for-ios-based-on-proximity-sensor-input
Both patent applications were filed with the USPTO in 2012.
Samsung will try to enter this as evidence of prior art at their patent trial, someday.
Perhaps a point of sale gimmick for advertisers? "See the new BMW in an exciting new demo and interact with the features in 3-D!"...
I've having trouble thinking of a single consumer use case scenario for this type of technology.
Pron :>)
Wasn't there a arcade (video) game based on this type of tech? It was from the 1990s and it was called Time Traveler.
I remember seeing this and I thought it looked just like the hologram from Star Wars.
JARVIS, fetch me some coffee!
Same thing was said about fax & copy machines and personal computers in the 70's. People clamor for innovation but can't see the down-the-road benefits of, literaly in this case, thinking outside the box.
Note that I didn't say it wasn't useful only that 'I' can't see a usage case.
My comment wasn't meant to pooh-pooh the patent but rather a solicitation to get others to enlighten me.
Signed,
Fandroids and Samsung
Top of my head I'm thinking interactive games. If I knew for sure I'd be jumping on a use for it right now. I can think of many uses for this tech in the medical and engineering industries. Imagine a doctor using 3d x-rays to explore a persons inner body before cutting them open. Imagine an auto company (or other construction industries) working out a complex series of parts in a virtual world before having to put up the money to tool dummy parts to see if they work. This is the kind of stuff that bleeds into the commercial sector after it's been in use for awhile. All someone needs to do is figure out a way to make boatloads of money from this tech, then it will be in the marketplace. I never thought there would be a printer that would make a 3 dimensional object, but can see myself owning one in the next decade or less.
I can think of a many things I want the mythical iWatch to do for me but I come up blank with 3D for the consumer. I'm not a gamer so that isn't something I consider. Engineering, science and medical fields, sure, but Apple focused on the consumer market so that is where my query went.
PS: Prior art¡
I thought it was God that invented the tech that lets users interact with 3D objects.
I loved that game as a kid! If I recall correctly it was quite expensive for a play.
I can think of a many things I want the mythical iWatch to do for me but I come up blank with 3D for the consumer. I'm not a gamer so that isn't something I consider. Engineering, science and medical fields, sure, but Apple focused on the consumer market so that is where my query went.
PS: Prior art¡
Like other posters have mentioned, there are a few use cases in commercial environments or for games, but I wonder whether there would be a big enough/profitable enough market for Apple to pursue.
PS: More prior art: