Apple investigating thinner touch panels that can sense 3D 'hover' gestures
Apple is looking beyond its new Force Touch pressure-sensitive input method, and may in the future allow users to perform gestures by pointing or moving their hands in front of a device, without even touching it, new patents reveal.
The company has filed for a pair of new touch-related inventions with the U.S. Patent and Trademark Office that were published this week, as discovered by AppleInsider. The first, entitled "Touch and Hover Sensor Compensation," describes how a device might use multiple adjacent sensors to sense movement and gestures away from a screen.
Known as "hover" gestures, these three-dimensional movements can open up entirely new ways for users to interact with a device. Apple's filing notes that there are motion sensing devices on the market, but providing reliable and accurate responses to hover gestures is incredibly difficult.
Apple's proposed invention is related to compensation for sensors. The company aims to reduce errors by using adjacent sensors together, and applying a gain factor to measurements they capture, reducing sensitivity variation at different sensor locations on the device.
"Sensor compensation can advantageously provide improved touch and hover sensing," the filing reads.
In addition, the USPTO disclosed a second filing on Thursday for a patent application entitled "On-Cell Touch Architecture," describing ways to make touch panels on devices like the iPhone and iPad even thinner.
But the second filing also makes note of 3D hover gestures. In the proposed patent, Apple notes that some capacitive touch sensing systems use fringing electrical fields to sense not only touches on the display, but also sensing fingertips beyond the surface of the display.
"Objects approaching near the surface may be detected near the surface without actually touching the display," the application reads.
Both filings are particularly noteworthy as Apple acquired gesture input company PrimeSense in a $360 million deal in 2013. PrimeSense is best known for creating the technology that powered Microsoft's first-generation Kinect sensor, used for motion controls and gameplay on its Xbox 360 game console.
Why Apple acquired PrimeSense remains a mystery, but the existing use of its company's technology in the living room has led to speculation that a next-generation Apple TV could offer some sort of advanced motion input. Such capabilities would allow users to control their Apple TV without the need to grab a remote control, iPhone, or other accessory.
As for Apple's new Force Touch input method, it was first introduced on the Apple Watch as a pressure-sensitive way to allow for new ways of interacting on a smaller display, but the company quickly adopted the technology for trackpads on its latest MacBook Pros and the new 12-inch MacBook with Retina display. It's also expected that Force Touch will be a key addition to this fall's anticipated "iPhone 6s," "iPad Air 3" and "iPad mini 4" upgrades.
The company has filed for a pair of new touch-related inventions with the U.S. Patent and Trademark Office that were published this week, as discovered by AppleInsider. The first, entitled "Touch and Hover Sensor Compensation," describes how a device might use multiple adjacent sensors to sense movement and gestures away from a screen.
Known as "hover" gestures, these three-dimensional movements can open up entirely new ways for users to interact with a device. Apple's filing notes that there are motion sensing devices on the market, but providing reliable and accurate responses to hover gestures is incredibly difficult.
Apple's proposed invention is related to compensation for sensors. The company aims to reduce errors by using adjacent sensors together, and applying a gain factor to measurements they capture, reducing sensitivity variation at different sensor locations on the device.
"Sensor compensation can advantageously provide improved touch and hover sensing," the filing reads.
In addition, the USPTO disclosed a second filing on Thursday for a patent application entitled "On-Cell Touch Architecture," describing ways to make touch panels on devices like the iPhone and iPad even thinner.
But the second filing also makes note of 3D hover gestures. In the proposed patent, Apple notes that some capacitive touch sensing systems use fringing electrical fields to sense not only touches on the display, but also sensing fingertips beyond the surface of the display.
"Objects approaching near the surface may be detected near the surface without actually touching the display," the application reads.
Both filings are particularly noteworthy as Apple acquired gesture input company PrimeSense in a $360 million deal in 2013. PrimeSense is best known for creating the technology that powered Microsoft's first-generation Kinect sensor, used for motion controls and gameplay on its Xbox 360 game console.
Why Apple acquired PrimeSense remains a mystery, but the existing use of its company's technology in the living room has led to speculation that a next-generation Apple TV could offer some sort of advanced motion input. Such capabilities would allow users to control their Apple TV without the need to grab a remote control, iPhone, or other accessory.
As for Apple's new Force Touch input method, it was first introduced on the Apple Watch as a pressure-sensitive way to allow for new ways of interacting on a smaller display, but the company quickly adopted the technology for trackpads on its latest MacBook Pros and the new 12-inch MacBook with Retina display. It's also expected that Force Touch will be a key addition to this fall's anticipated "iPhone 6s," "iPad Air 3" and "iPad mini 4" upgrades.
Comments
Yup.. It was done through the cameras.. Different and inferior approach
"Apple is looking beyond its new Force Touch pressure-sensitive input method, and may in the future allow users to perform gestures by pointing or moving their hands in front of a device, without even touching it, new patents reveal." -- AppleInsider
Uh, no. Patents like these existed BEFORE Force Touch. Force Touch just happened to be introduced to the world first perhaps due to technology being advanced enough to support Force Touch.
I believe Apple has been working on multiple paths that will eventually become one path of awesomeness. From your own archives...
http://appleinsider.com/articles/14/04/24/apples-display-tech-lets-users-interact-with-3d-objects-in-mid-air
http://appleinsider.com/articles/13/08/20/apple-patents-3d-gesture-ui-for-ios-based-on-proximity-sensor-input
Here is to hoping the iPad Pro and/or the next iPhone has these features. Better yet, these patents could be the prelude to a brand new hardware product that opens a brand new platform for the Apple ecosystem that exists outside the iPad, iPhone and Apple TV platforms yet can coexist with those platforms.
Apple can investigate whatever they want, but I'll be very surprised if hover gestures ever take the place of tactility.
Wouldn't the hover gestures and 3D technology shown in the Iron Man movies be awesome to have Apple bring to the masses? This is where I see the technology heading.
It's more useful as a supplementary input rather than primary. Your arms/hands would get tired holding them up for long periods:
[VIDEO]
[VIDEO]
The LEAP demo shows how it can be used for some games. It would be able to detect objects like when you are holding a pen to be able to do accurate palm rejection. It can support gestures for multi-tasking, split view, zoom/rotate/pan. It can precache website links that you are about to touch, it can type ahead keys you are about to press. It can do facial recognition by holding your face in the 3D scanner as well as fast avatar creation in games to put your face in the game. It can detect hand size for age controls and block inappropriate content or expensive purchases.
Where it can replace tactile input for prolonged use would be if you could sit the iPad vertically on a table and be able to simply move your hands horizontally on the desk in front of it and it would detect passive objects like a mock keyboard that has no electronics inside, it can be a rubber sheet. It could tell which keys you are pressing based on the edges of the rubber sheet. The keyboard can alternatively be on-screen and your hand position on the table would determine which keys you are pressing.
Wouldn't the hover gestures and 3D technology shown in the Iron Man movies be awesome to have Apple bring to the masses? This is where I see the technology heading.
No. It'd be super annoying and strenuous. Good demo, crap practice.
If there was some way they could build a tactile feedback into a holographic interface with micro interaction then that might be something.
Apple also owns PrimeSense.
Your idea is definitely Star Trek Next Generation! That would be great!!
I can't wait to crack my iPhone screen if I can just hover over it after! no more getting glass in your finger! lol
my ipad and iphone do this all the time. it's annoying when you move your hand over top of the display surface and the system responds as if it has been touched.
they all have fairly definite xyz planar surfaces, this thing kind of cuts through the xz and makes a sort of rhombus in cross section type touch surface. maybe that has some advantage in the method
Didn't Samsung try something like this? I swear I remember seeing a hokey commercial with some guy whose hands were covered in BBQ sauce but he was still able to manipulate his phone just by hovering his finger over the screen. Seemed very gimmicky.
Yes and Samsung still does.
The Feature is called Air View and exists at least since Galaxy S4 and it just works good. Actually it behaves like hovering with a mouse pointer. For instance:
- In phone app, you hover over the numbers and it shows you the linked contacts
- In gallery you hover over the small preview pictures and it opens them in a much bigger preview
- In Video player you hover over the time scale and like youtube it shows you the time and the scene in preview mode
- In email client in the list view you just hover over the emails and it shows you the first view lines without opening the emails
- In Internet Browser it works like a magnifier
Actually it is a much better feature than Force Touch on a touch Screen.
There's quite a few patents related to holographic touch interfaces, some pre-dating even TNG. At least one large tech has several in that space so it seems to be a focus of theirs. I doubt they're the only ones but I didn't spend more than a few minutes looking.
If you want to see what's being done in that area use the specialty Google search product Advanced Patent Search (simply type Advanced Patent Search in the search bar to get to the page) and enter whatever search terms you're wanting to find. It's pretty straightforward. I don't know of any other general search provider with a similar free service but there might be if you prefer not to use Google.