Apple exploring Kinect-like 3D input for controlling Macs
Apple has shown interest in an entirely new way for users to interact with and control their Mac, with a new system allowing users to perform gestures with their hands in a three-dimensional space.
The concept was disclosed this week in a U.S. patent application discovered by AppleInsider entitled "Three-Dimensional Imaging and Display System." It describes a system akin to Microsoft's Xbox Kinect platform, which allows for controller-free interaction with devices.
Apple's solution would optically detect a user's hands and fingertips and measure their movements. In this manner, users could use their hands to control a Mac without the need for a mouse or keyboard.
In its application, Apple notes that current computer input devices like the mouse allow users to control a system in two dimensions, along the 'X' axis and 'Y' axis. But when manipulating objects in three dimensions, input methods like a joystick or mouse can be cumbersome.
"A need thus remains for uncomplicated, economical, yet highly effective 3D input devices for computers," the filing reads. "Such devices need to be able to detect, analyze and measure objects located in a 3D volume, and to observe and track any motions thereof."
Apple's filing notes that for any system to be successful, it needs to be "economical, small, portable and versatile." It must also provide a user with "integral, immediate feedback," and it needs to be adapted well for use with a number of devices, ranging from full-fledged Macs to portable electronics like the iPhone.
The hardware described by Apple in the application would be capable of tracking a user's head as well as their hands. Potential components include an infrared sensor or visible laser, high-speed photo detector, digital signal processor, dual-axis scanning device, and subsystems for analog and video.
Apple envisions a user interface that would allow a person to more naturally use their hands and fingertips to control a device. An accompanying display could show knobs, sliders and buttons that a user could virtually manipulate in a three-dimensional space.
In one image accompanying the application, a user is shown using both of their hands to select inputs on an iMac. With their left hand, the user is twisting a virtual knob, while on the right hand, their index finger is being used to press a button.
The system would aid the user with on-screen visual cues, allowing them to more easily manipulate objects on the screen. For example, Apple's system includes a virtual representation of the user's hand displayed on the screen of the iMac.
In the application, Apple provides a number of potential uses for its described system. For example, head tracking could be used to tilt an image or zoom in on the screen based on the location of a user's head. Accordingly, moving person could be followed and tracked with a motorized camera system.
With hand gesturing, the system could track individual elements including the fingers, thumb and palm. Using gestures, a user could accomplish complex tasks like 3D rotate, zoom, scroll and volume control.
Another feature of the system described by Apple is user presence detection. With this, the system could detect whether a user is sitting in front of a display, and could even identify unique users. The system could also automatically shut down and save power when the user walks away.
Other potential uses listed by Apple are surveillance, bar code reading, object measuring, image substitution or replacement, or a virtual keyboard. In the keyboard example, Apple notes that individual fingers could be tracked for typing, and the image of a keyboard could be projected onto a desktop to give users a visual aid while typing.
The proposed invention, made public this week by the U.S. Patent and Trademark Office, was first filed by Apple in August of this year. It is credited to Christoph H. Krah.
The concept was disclosed this week in a U.S. patent application discovered by AppleInsider entitled "Three-Dimensional Imaging and Display System." It describes a system akin to Microsoft's Xbox Kinect platform, which allows for controller-free interaction with devices.
Apple's solution would optically detect a user's hands and fingertips and measure their movements. In this manner, users could use their hands to control a Mac without the need for a mouse or keyboard.
In its application, Apple notes that current computer input devices like the mouse allow users to control a system in two dimensions, along the 'X' axis and 'Y' axis. But when manipulating objects in three dimensions, input methods like a joystick or mouse can be cumbersome.
"A need thus remains for uncomplicated, economical, yet highly effective 3D input devices for computers," the filing reads. "Such devices need to be able to detect, analyze and measure objects located in a 3D volume, and to observe and track any motions thereof."
Apple's filing notes that for any system to be successful, it needs to be "economical, small, portable and versatile." It must also provide a user with "integral, immediate feedback," and it needs to be adapted well for use with a number of devices, ranging from full-fledged Macs to portable electronics like the iPhone.
The hardware described by Apple in the application would be capable of tracking a user's head as well as their hands. Potential components include an infrared sensor or visible laser, high-speed photo detector, digital signal processor, dual-axis scanning device, and subsystems for analog and video.
Apple envisions a user interface that would allow a person to more naturally use their hands and fingertips to control a device. An accompanying display could show knobs, sliders and buttons that a user could virtually manipulate in a three-dimensional space.
In one image accompanying the application, a user is shown using both of their hands to select inputs on an iMac. With their left hand, the user is twisting a virtual knob, while on the right hand, their index finger is being used to press a button.
The system would aid the user with on-screen visual cues, allowing them to more easily manipulate objects on the screen. For example, Apple's system includes a virtual representation of the user's hand displayed on the screen of the iMac.
In the application, Apple provides a number of potential uses for its described system. For example, head tracking could be used to tilt an image or zoom in on the screen based on the location of a user's head. Accordingly, moving person could be followed and tracked with a motorized camera system.
With hand gesturing, the system could track individual elements including the fingers, thumb and palm. Using gestures, a user could accomplish complex tasks like 3D rotate, zoom, scroll and volume control.
Another feature of the system described by Apple is user presence detection. With this, the system could detect whether a user is sitting in front of a display, and could even identify unique users. The system could also automatically shut down and save power when the user walks away.
Other potential uses listed by Apple are surveillance, bar code reading, object measuring, image substitution or replacement, or a virtual keyboard. In the keyboard example, Apple notes that individual fingers could be tracked for typing, and the image of a keyboard could be projected onto a desktop to give users a visual aid while typing.
The proposed invention, made public this week by the U.S. Patent and Trademark Office, was first filed by Apple in August of this year. It is credited to Christoph H. Krah.
Comments
Notice how old this is? That iMac is from the early 2000's.
Is this the predecessor comment to saying Microsoft must have copied Apple again?
Is this the predecessor comment to saying Microsoft must have copied Apple again?
It is obvious that Apple thought of it first. Dont you know that apple have a patent on thinking of ideas before patenting them?
Nice and all, but how's this 'entirely new' if MS' Kinect already allows "users to perform gestures with their hands in a three-dimensional space" as a substitute for device input/controllers?
It just is!!! Anyone that know anything about Apple knows that they invented this in August and were the first to invent it.
All the similar things like:
- People making this sort of thing a few years ago using Nintend Wii controllers
- Microsofts well known plans for this going back a few years, releasing Kinect last year and the SDK for Windows at the start of this year
- My Sony Vaio that I bought earlier this year accepting hand gestures using the web cam
are all copy's of Apple's patent filed this year after all of them existing.
If you can see how they copies Apple your just an idiot!!!
http://www.pranavmistry.com/projects/sixthsense
Looks like another copy to me
Terrible, terrible idea. No matter who implements it. At least touch screens have some kind of tactile feedback.
Too bad Apple didn't purchase the technology in Kinect since it was offered to them first.
Microsoft did not invent the motion controls in Kinect, they licensed the technology from Prime Sense. Prime Sense is not owned by or controlled by Microsoft and licenses its technology to any company willing to pay.
Nice and all, but how's this 'entirely new' if MS' Kinect already allows "users to perform gestures with their hands in a three-dimensional space" as a substitute for device input/controllers?
Who said it was "entirely new"? Are you making up quotes or just building a straw man?
Who said it was "entirely new"? Are you making up quotes or just building a straw man?
DaHarder is just being himself. You'll learn to ignore him like pretty much everyone else does
DaHarder is just being himself. You'll learn to ignore him like pretty much everyone else does
I should have learned long ago I'm sure. Back when he said he was buying 3 tablets for each member of his family.
Nice and all, but how's this 'entirely new' if MS' Kinect already allows "users to perform gestures with their hands in a three-dimensional space" as a substitute for device input/controllers?
Because ideas like this are not supposed to be patentable, only the specific implementation of the idea. And while I will admit that the patent office violates that concept all the time (like permitting Amazon to patent "one-click", which Apple then had to license from Amazon), it's how patents are supposed to work. Otherwise, "information is presented on a screen" would be patentable.
Well, at least when the late Steve Jobs was around.
Hopefully it won't need any extra cameras such as those employed by M$ and previously Sony with their PS2 EyeToy. I'd seen Kinect working but it isn't like this - this one will enable us to 'feel' the objects rather than just waving our hands to interact slides.
As for Project Sixth Sense, it uses a wearable finger band and a camera - the Apple one looks like it only requires a board. Project Sixth Sense would rather make more sense if Sony would invest in them to advance their Move gadget (really, it's design is embarrassing - it currently looks like a microphone or worse, a dildo).
Apple's track record in touches, both in performance and commercial results, is great. Surely, everyone remembers the multi-gesture trackpad on the Macbook Pro? I think that was before touch-screens was popular. In fact, touch screens became popular AFTER Apple innovate them - there were touch screens invented before but it never really took off! Apple had just put a gas propeller on that and then some!
Now imagine if Apple pulled this air gesture gadget off:
- 3D models can be sculpted, modelled and move around freely
- Drawing on canvas will be easier - no more worrying about smudges or stylus pen nibs exhausted
- Games will be more interactive; we saw these on the iPad and iPhone, now imagine doing so WITHOUT touching the screen or anything!
- Web browsing will also be enhanced
- HECK, a lot of possibilities I can't even think of!
And because this is Apple, quality is assured. It does not only works but it works smoothly. As always, every new piece of tech has it's flaws but, again, because this is Apple, people are going to be more confident in buying and then using them.
I know I am.
* based on that we are just waving our hands and moving our body without touching anything
Where gestures would really come in to their own is controlling the TV. But MS is way ahead on this one.
Because ideas like this are not supposed to be patentable, only the specific implementation of the idea. And while I will admit that the patent office violates that concept all the time (like permitting Amazon to patent "one-click", which Apple then had to license from Amazon), it's how patents are supposed to work. Otherwise, "information is presented on a screen" would be patentable.
Patents, including the "one click purchase system" by Amazon which is actually called "METHOD AND SYSTEM FOR PLACING A PURCHASE ORDER VIA A COMMUNICATIONS NETWORK", require a very detailed explanation to accomplishing your task. Amazon had to explain a method and system to accomplish that task, so if another person comes out with a different process or system to accomplish the same task, Amazon can't sue. And there certainly are many other ways to accomplish a one click purchase, it's just that Apple decided to go with Amazon's solution.
Amazon had to submit a patent that begins with explaining that there are many clients connected to the internet which are browsing Amazon's site, etc.. . So you can imagine the details required to explain the exact process in accomplishing the one click purchase.
You can see the 18 page document here.
Notice how old this is? That iMac is from the early 2000's.
I know, strange this would just be filed now with the diagram showing an early G4 iMac (notice the screen size/ratio). They obviously were working on this back in 2001/2002 (I think this iMac was introduced at Macworld SF 2002). But why submit the patent in August of 2011, nearly 10 years later? strange indeed.
Is this the predecessor comment to saying Microsoft must have copied Apple again?
Well, no. MS has something on the market now, so Microsoft wins this round in the to production race. Having just gotten and Xbox 360/Kinect on Black Friday, I have to say, it is cool. BUT... It is slow. it isn't as simple as "touching" the air in the space you want it to be. You have to hover your hand in the area of the (rather large by necessity) button and hold it there for a few seconds while the sensor recognizes the hand, and to prevent any stray interaction, while a circle fills up around the "hand" to confirm. This works for gaming menus, but would be too slow to be useful on a computer. The dashboard update that came out Tuesday allows more voice interaction, which, in my limited use of it over the 40-48 hours it has been live, is pretty good, especially as a first "live" implementation. It isn't quite to the point of me being able to say "Play Fruit Ninja" from anywhere but more menu navigation in steps. "Xbox...Games...Play"
Because ideas like this are not supposed to be patentable, only the specific implementation of the idea. And while I will admit that the patent office violates that concept all the time (like permitting Amazon to patent "one-click", which Apple then had to license from Amazon), it's how patents are supposed to work. Otherwise, "information is presented on a screen" would be patentable.
BINGO!! Not sure why this is so hard to grasp. ugh.
Who said it was "entirely new"? Are you making up quotes or just building a straw man?
It's the very first line of the AI article.
"Apple has shown interest in an entirely new way for users to interact. . ."