Apple investigates 3D head-tracking, battery life improvements
Future Apple hardware could track the location of a user and adjust a 3D display according to their position, creating the illusion that an on-screen object is physically present.
Known as head-tracking, the technology has been known and implemented for sometime. But a new patent application filed by Apple this week suggests the Mac maker could employ the advanced method to allow users greater interactivity with their computer.
Through the use of a camera, the system could detect a user's position, and adjust the image on screen accordingly. In addition, the camera could use the image of the user and overlay it onto an on-screen object, giving the impression of a "reflection" and creating a more immerse experience.
Apple noted that the conventional methods of interacting with 3D objects on screen, by using a mouse and keyboard, are not intuitive for most users. In addition, on-screen objects often lack realism because they do not interact with the living environment.
Apple's newly described technology would address that through the use of a camera or appropriate "sensing mechanism."
"Using the detected position of the user, the electronic device may use any suitable approach to transform the perspective of three-dimensional objects displayed on the display," the application reads. "For example, the electronic device may use a parallax transform by which three-dimensional objects displayed on the screen may be modified to give the user the impression of viewing the object from a different perspective."
The technology would be capable of defining the visual properties of different surfaces, and determining how well it would reflect light. Using this, images of the user and their environment could be recreated on the screen with effects added.
"Using this approach, surfaces with low reflectivity (e.g., plastic surfaces) may not reflect the environment, but may reflect light, while surfaces with high reflectivity (e.g. polished metal or chrome) may reflect both the environment (e.g. the user's face as detected by the camera) and light," the application states. "To further enhance the user's experience, the detected environment may be reflected differently along curved surfaces of a displayed object (e.g. as if the user were actually moving around the displayed object and seeing his reflection based on his position and the portion of the object reflecting the image."
It's a new level of virtual reality, meshing the real world with a nonexistent one, but giving the visual perception that an object is real. Apple is not alone in exploring this technology: Microsoft's "Project Natal" is an unreleased video game accessory that aims to use full-body tracking through the use of cameras to allow a user's physical actions to be translated into Xbox 360 games.
Programmer Johnny Chung Lee also gained attention 2007 when he created a homemade head tracking device using the infrared remote controller for the Nintendo Wii. The Carnegie Mellon University student created a simple interactive display that adjusted its picture based on the relative position of the user. His technology has been viewed more than 7.6 million times on YouTube.
This week's patent application isn't the first time Apple has investigated different kinds of 3D technology. In 2008, the company applied for a patent on new display hardware that would employ autostereoscopy to produce 3D, viewable without head gear or glasses. Apple also, in 2008, revealed details on a possible 3D user interface for Mac OS X. And the company has also shown interest in a 3D gaming controller for the Apple TV.
A different kind of virtual reality, known as augmented reality, has proven popular in various applications for the iPhone. Made possible with the release of iPhone OS 3.1, the display of markers and content over live pictures from the iPhone camera is made possible through the use of the autofocusing camera of the iPhone 3GS and its built-in compass. With them, the phone is capable of recognizing the direction the phone is facing and getting a detailed look at a subject to tag it with information.
Forecasting media player battery life
In another patent application, Apple has described a system that would calculate the amount of power needed to play content like a video, and determines whether there is enough left in the battery to complete the task. The system could then adjust its playback settings, either automatically or through user input, to reduce the amount of power needed to play the video in its entirety.
Apple said with current technology, a user might begin playing a video without the knowledge that there is not enough power left in the battery to complete it. But those issues can usually be addressed from the beginning by adjusting picture resolution, video bit-rate, backlight strength, sound volume and other factors.
The "intelligent power management method" would take all of those into consideration when trying to maximize battery life and ensure that a video will be played in its entirety. The software would employ a "priority ranking" for video settings, which can be customized by the user.
Known as head-tracking, the technology has been known and implemented for sometime. But a new patent application filed by Apple this week suggests the Mac maker could employ the advanced method to allow users greater interactivity with their computer.
Through the use of a camera, the system could detect a user's position, and adjust the image on screen accordingly. In addition, the camera could use the image of the user and overlay it onto an on-screen object, giving the impression of a "reflection" and creating a more immerse experience.
Apple noted that the conventional methods of interacting with 3D objects on screen, by using a mouse and keyboard, are not intuitive for most users. In addition, on-screen objects often lack realism because they do not interact with the living environment.
Apple's newly described technology would address that through the use of a camera or appropriate "sensing mechanism."
"Using the detected position of the user, the electronic device may use any suitable approach to transform the perspective of three-dimensional objects displayed on the display," the application reads. "For example, the electronic device may use a parallax transform by which three-dimensional objects displayed on the screen may be modified to give the user the impression of viewing the object from a different perspective."
The technology would be capable of defining the visual properties of different surfaces, and determining how well it would reflect light. Using this, images of the user and their environment could be recreated on the screen with effects added.
"Using this approach, surfaces with low reflectivity (e.g., plastic surfaces) may not reflect the environment, but may reflect light, while surfaces with high reflectivity (e.g. polished metal or chrome) may reflect both the environment (e.g. the user's face as detected by the camera) and light," the application states. "To further enhance the user's experience, the detected environment may be reflected differently along curved surfaces of a displayed object (e.g. as if the user were actually moving around the displayed object and seeing his reflection based on his position and the portion of the object reflecting the image."
It's a new level of virtual reality, meshing the real world with a nonexistent one, but giving the visual perception that an object is real. Apple is not alone in exploring this technology: Microsoft's "Project Natal" is an unreleased video game accessory that aims to use full-body tracking through the use of cameras to allow a user's physical actions to be translated into Xbox 360 games.
Programmer Johnny Chung Lee also gained attention 2007 when he created a homemade head tracking device using the infrared remote controller for the Nintendo Wii. The Carnegie Mellon University student created a simple interactive display that adjusted its picture based on the relative position of the user. His technology has been viewed more than 7.6 million times on YouTube.
This week's patent application isn't the first time Apple has investigated different kinds of 3D technology. In 2008, the company applied for a patent on new display hardware that would employ autostereoscopy to produce 3D, viewable without head gear or glasses. Apple also, in 2008, revealed details on a possible 3D user interface for Mac OS X. And the company has also shown interest in a 3D gaming controller for the Apple TV.
A different kind of virtual reality, known as augmented reality, has proven popular in various applications for the iPhone. Made possible with the release of iPhone OS 3.1, the display of markers and content over live pictures from the iPhone camera is made possible through the use of the autofocusing camera of the iPhone 3GS and its built-in compass. With them, the phone is capable of recognizing the direction the phone is facing and getting a detailed look at a subject to tag it with information.
Forecasting media player battery life
In another patent application, Apple has described a system that would calculate the amount of power needed to play content like a video, and determines whether there is enough left in the battery to complete the task. The system could then adjust its playback settings, either automatically or through user input, to reduce the amount of power needed to play the video in its entirety.
Apple said with current technology, a user might begin playing a video without the knowledge that there is not enough power left in the battery to complete it. But those issues can usually be addressed from the beginning by adjusting picture resolution, video bit-rate, backlight strength, sound volume and other factors.
The "intelligent power management method" would take all of those into consideration when trying to maximize battery life and ensure that a video will be played in its entirety. The software would employ a "priority ranking" for video settings, which can be customized by the user.
Comments
http://johnnylee.net/projects/wii/
With a WiiMote, glasses with IR-LEDs and the Software you can try out Head-Tracking
See a preview by Johnny Chung Lee:
http://johnnylee.net/projects/wii/
With a WiiMote, glasses with IR-LEDs and the Software you can try out Head-Tracking
Thanks for the link... very cool stuff there! I'm going to have to play with that when I get home.
See a preview by Johnny Chung Lee:
http://johnnylee.net/projects/wii/
With a WiiMote, glasses with IR-LEDs and the Software you can try out Head-Tracking
If only Apple had bought zcam? Guess they might not have done Project Natal like Microsoft is doing. Good to see potential of pseudo coming through. guess there is an option to bring this in, as an alternative/prior to 3D monitors becoming mainstream (if at all).
Time for them to patent having accelerometers in the Apple remote/sync up with Touch/iPhones?
Is Apple late on this - Seeing as Johny joined Microsoft a few months ago (his http://procrastineering.blogspot.com/ blog has him mentioning it Project Natal back in June - does Microsoft have patents prior to Apple on this?
http://bmson.is/bin/ht/
If they can properly combine face tracking technology from a camera (or your PC's built in cam), and somehow gauge distance as well, you've got most of what you need already built into a basic Mac.
You need a corporate team like Apple with a dynamic, demanding leader like Steve to be productive and profitable enough in these times to have the wherewithal and capacity to have this kind of forward vision and prediction.
yes, and with that, come all the parasites (ie psystar, microsoft...)
PS: That guy lives a very utilitarian life or he is getting raped in a divorce.
I?m glad I saw that, I jut hope Apple can use sensors at the edge of monitor bezels because I?m not wearing headgear to achieve the experience.
PS: That guy lives a very utilitarian life or he is getting raped in a divorce.
I think he is just a student.
The "intelligent power management method" would take all of those into consideration when trying to maximize battery life and ensure that a video will be played in its entirety. The software would employ a "priority ranking" for video settings, which can be customized by the user.
[ View this article at AppleInsider.com ]
The first thing was interesting, but patenting the above? Wow, I could patent everything I think about at work every day, except I'd be too busy writing patent-speak BS to actually *do* anything....
I thought everone knew about this two years ago. Strange that no-one has really capitalized on it. I want to play a first person shooter where you dodge real bullets.
It's interesting that AAA games haven't capitalized on this. It seems to me be a natural progression of the Nvidia 3D vision stuff that's being pushed these days.
Instead of wearing a head mounted pair of LEDs - wouldn't there be some way to use the built in camera in most Macs these days to detect your eyes and use those for tracking?
other examples of using the built in camera to detect the user include this one http://www.freeverse.com/mac/product/?id=45
-> Edges filter
-> Noise reduction
-> Here you go, the perfectly detectable moving pattern.
Sure, filtering should be hardware one.