or Connect
AppleInsider › Forums › Mac Hardware › Future Apple Hardware › Apple investigates 3D head-tracking, battery life improvements
New Posts  All Forums:Forum Nav:

Apple investigates 3D head-tracking, battery life improvements

post #1 of 23
Thread Starter 
Future Apple hardware could track the location of a user and adjust a 3D display according to their position, creating the illusion that an on-screen object is physically present.

Known as head-tracking, the technology has been known and implemented for sometime. But a new patent application filed by Apple this week suggests the Mac maker could employ the advanced method to allow users greater interactivity with their computer.

Through the use of a camera, the system could detect a user's position, and adjust the image on screen accordingly. In addition, the camera could use the image of the user and overlay it onto an on-screen object, giving the impression of a "reflection" and creating a more immerse experience.

Apple noted that the conventional methods of interacting with 3D objects on screen, by using a mouse and keyboard, are not intuitive for most users. In addition, on-screen objects often lack realism because they do not interact with the living environment.

Apple's newly described technology would address that through the use of a camera or appropriate "sensing mechanism."

"Using the detected position of the user, the electronic device may use any suitable approach to transform the perspective of three-dimensional objects displayed on the display," the application reads. "For example, the electronic device may use a parallax transform by which three-dimensional objects displayed on the screen may be modified to give the user the impression of viewing the object from a different perspective."

The technology would be capable of defining the visual properties of different surfaces, and determining how well it would reflect light. Using this, images of the user and their environment could be recreated on the screen with effects added.



"Using this approach, surfaces with low reflectivity (e.g., plastic surfaces) may not reflect the environment, but may reflect light, while surfaces with high reflectivity (e.g. polished metal or chrome) may reflect both the environment (e.g. the user's face as detected by the camera) and light," the application states. "To further enhance the user's experience, the detected environment may be reflected differently along curved surfaces of a displayed object (e.g. as if the user were actually moving around the displayed object and seeing his reflection based on his position and the portion of the object reflecting the image."



It's a new level of virtual reality, meshing the real world with a nonexistent one, but giving the visual perception that an object is real. Apple is not alone in exploring this technology: Microsoft's "Project Natal" is an unreleased video game accessory that aims to use full-body tracking through the use of cameras to allow a user's physical actions to be translated into Xbox 360 games.



Programmer Johnny Chung Lee also gained attention 2007 when he created a homemade head tracking device using the infrared remote controller for the Nintendo Wii. The Carnegie Mellon University student created a simple interactive display that adjusted its picture based on the relative position of the user. His technology has been viewed more than 7.6 million times on YouTube.



This week's patent application isn't the first time Apple has investigated different kinds of 3D technology. In 2008, the company applied for a patent on new display hardware that would employ autostereoscopy to produce 3D, viewable without head gear or glasses. Apple also, in 2008, revealed details on a possible 3D user interface for Mac OS X. And the company has also shown interest in a 3D gaming controller for the Apple TV.

A different kind of virtual reality, known as augmented reality, has proven popular in various applications for the iPhone. Made possible with the release of iPhone OS 3.1, the display of markers and content over live pictures from the iPhone camera is made possible through the use of the autofocusing camera of the iPhone 3GS and its built-in compass. With them, the phone is capable of recognizing the direction the phone is facing and getting a detailed look at a subject to tag it with information.

Forecasting media player battery life

In another patent application, Apple has described a system that would calculate the amount of power needed to play content like a video, and determines whether there is enough left in the battery to complete the task. The system could then adjust its playback settings, either automatically or through user input, to reduce the amount of power needed to play the video in its entirety.

Apple said with current technology, a user might begin playing a video without the knowledge that there is not enough power left in the battery to complete it. But those issues can usually be addressed from the beginning by adjusting picture resolution, video bit-rate, backlight strength, sound volume and other factors.

The "intelligent power management method" would take all of those into consideration when trying to maximize battery life and ensure that a video will be played in its entirety. The software would employ a "priority ranking" for video settings, which can be customized by the user.

post #2 of 23
So this is a pseudo 3D achieved by perspective and lighting but still shown in 2D. I am still hoping we get genuine 3D one day. I can imagine moving back into the finder to locate folders on shelves I know this is not required with spotlight but it would be neat all the same.
Enjoying the new Mac Pro ... it's smokin'
Been using Apple since Apple ][ - Long on AAPL so biased
nMac Pro 6 Core, MacBookPro i7, MacBookPro i5, iPhones 5 and 5s, iPad Air, 2013 Mac mini.
Reply
Enjoying the new Mac Pro ... it's smokin'
Been using Apple since Apple ][ - Long on AAPL so biased
nMac Pro 6 Core, MacBookPro i7, MacBookPro i5, iPhones 5 and 5s, iPad Air, 2013 Mac mini.
Reply
post #3 of 23
You need a corporate team like Apple with a dynamic, demanding leader like Steve to be productive and profitable enough in these times to have the wherewithal and capacity to have this kind of forward vision and prediction.

Daniel Swanson

Reply

Daniel Swanson

Reply
post #4 of 23
I wish Apple had something like BumpTop. If you haven't seen it yet go check it out! It is a desktop environment with 3D objects and realistic physics. Currently it is windows only.
post #5 of 23
See a preview by Johnny Chung Lee:
http://johnnylee.net/projects/wii/
With a WiiMote, glasses with IR-LEDs and the Software you can try out Head-Tracking
post #6 of 23
That's awesome.
80 million iPhones by 2012. That's only 15% of the market.

http://www.iphonethailand.net
Reply
80 million iPhones by 2012. That's only 15% of the market.

http://www.iphonethailand.net
Reply
post #7 of 23
Quote:
Originally Posted by tonicstuff View Post

See a preview by Johnny Chung Lee:
http://johnnylee.net/projects/wii/
With a WiiMote, glasses with IR-LEDs and the Software you can try out Head-Tracking

Thanks for the link... very cool stuff there! I'm going to have to play with that when I get home.
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
post #8 of 23
Quote:
Originally Posted by tonicstuff View Post

See a preview by Johnny Chung Lee:
http://johnnylee.net/projects/wii/
With a WiiMote, glasses with IR-LEDs and the Software you can try out Head-Tracking

If only Apple had bought zcam? Guess they might not have done Project Natal like Microsoft is doing. Good to see potential of pseudo coming through. guess there is an option to bring this in, as an alternative/prior to 3D monitors becoming mainstream (if at all).

Time for them to patent having accelerometers in the Apple remote/sync up with Touch/iPhones?


Is Apple late on this - Seeing as Johny joined Microsoft a few months ago (his http://procrastineering.blogspot.com/ blog has him mentioning it Project Natal back in June - does Microsoft have patents prior to Apple on this?
Apple!

Think Different
Reply
Apple!

Think Different
Reply
post #9 of 23
Here is a head tracking demo in Flash.

http://bmson.is/bin/ht/
post #10 of 23
The youtube clip was pretty amazing. Especially given the fact that even for someone watching the video, it had a true 3-D appearance. I could literally see the targets floating in front of the screen. Given a properly narrow viewing angle, it would seem to have 3-D properties, even if you aren't seeing the exact perspective of the targeted user.

If they can properly combine face tracking technology from a camera (or your PC's built in cam), and somehow gauge distance as well, you've got most of what you need already built into a basic Mac.
iMac 27" 2.8 Quad i7 / 24" Dual Core 3.06 / 17" Macbook Pro Unibody / Mac Mini HTPC / iPhone 4
Reply
iMac 27" 2.8 Quad i7 / 24" Dual Core 3.06 / 17" Macbook Pro Unibody / Mac Mini HTPC / iPhone 4
Reply
post #11 of 23
Quote:
Originally Posted by DanielSW View Post

You need a corporate team like Apple with a dynamic, demanding leader like Steve to be productive and profitable enough in these times to have the wherewithal and capacity to have this kind of forward vision and prediction.

yes, and with that, come all the parasites (ie psystar, microsoft...)
post #12 of 23
I can barely read the faded/pixelated credits and I can only slightly hear the sound, but I'm glad I saw the whole movie!
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
Citing unnamed sources with limited but direct knowledge of the rumoured device - Comedy Insider (Feb 2014)
Reply
post #13 of 23
awesome
post #14 of 23
I’m glad I saw that, I jut hope Apple can use sensors at the edge of monitor bezels because I’m not wearing headgear to achieve the experience.

PS: That guy lives a very utilitarian life or he is getting raped in a divorce.
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
Dick Applebaum on whether the iPad is a personal computer: "BTW, I am posting this from my iPad pc while sitting on the throne... personal enough for you?"
Reply
post #15 of 23
Quote:
Originally Posted by solipsism View Post

Im glad I saw that, I jut hope Apple can use sensors at the edge of monitor bezels because Im not wearing headgear to achieve the experience.

PS: That guy lives a very utilitarian life or he is getting raped in a divorce.

I think he is just a student.
post #16 of 23
Quote:
Originally Posted by AppleInsider View Post


The "intelligent power management method" would take all of those into consideration when trying to maximize battery life and ensure that a video will be played in its entirety. The software would employ a "priority ranking" for video settings, which can be customized by the user.


The first thing was interesting, but patenting the above? Wow, I could patent everything I think about at work every day, except I'd be too busy writing patent-speak BS to actually *do* anything....
post #17 of 23
I thought everone knew about this two years ago. Strange that no-one has really capitalized on it. I want to play a first person shooter where you dodge real bullets.
post #18 of 23
Quote:
Originally Posted by Dunks View Post

I thought everone knew about this two years ago. Strange that no-one has really capitalized on it. I want to play a first person shooter where you dodge real bullets.

It's interesting that AAA games haven't capitalized on this. It seems to me be a natural progression of the Nvidia 3D vision stuff that's being pushed these days.
PC Gamer, Musician, Mac Geek. | Jerion.us
Reply
PC Gamer, Musician, Mac Geek. | Jerion.us
Reply
post #19 of 23
I didn't read the article, but 3D head-tracking has been around for a long time...

post #20 of 23
that video clip was pretty cool - though for everyday tasks it could be disorienting.

Instead of wearing a head mounted pair of LEDs - wouldn't there be some way to use the built in camera in most Macs these days to detect your eyes and use those for tracking?

other examples of using the built in camera to detect the user include this one http://www.freeverse.com/mac/product/?id=45
post #21 of 23
IR image
-> Edges filter



-> Noise reduction
-> Here you go, the perfectly detectable moving pattern.

Sure, filtering should be hardware one.

We mean Apple no harm.

People are lovers, basically. -- Engadget livebloggers at the iPad mini event.

Reply

We mean Apple no harm.

People are lovers, basically. -- Engadget livebloggers at the iPad mini event.

Reply
post #22 of 23
This has no purpose I imagine in increasing productivity, but it could definitely add something to gaming, multi-person interactive apps (whiteboards?) and videoconferencing.

Proud AAPL stock owner.

 

GOA

Reply

Proud AAPL stock owner.

 

GOA

Reply
post #23 of 23
Quote:
Originally Posted by SpamSandwich View Post

This has no purpose I imagine in increasing productivity, but it could definitely add something to gaming, multi-person interactive apps (whiteboards?) and videoconferencing.

What about a structural engineer analyzing a new bridge design. Or a geologist studying the readings from an earthquake.

Or add in a few more sensors and have ability to "touch" and "move" the objects without using a keyboard or mouse?

At this point I believe it is an interesting idea that deserves further development - and while initial applications will very likely be games - I could easily see it becoming the standard interface for some applications - and with sufficient advances in hardware, software, sensors, etc - could one day be mainstream - hard to imagine a time when there will be no need to hard input devices at least in some applications - since no matter how good things like speech recognition become there may always be conditions under which those technologies cannot be used.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Future Apple Hardware
AppleInsider › Forums › Mac Hardware › Future Apple Hardware › Apple investigates 3D head-tracking, battery life improvements