Apple investigating thinner touch panels that can sense 3D 'hover' gestures

Posted:
in Future Apple Hardware edited June 2015
Apple is looking beyond its new Force Touch pressure-sensitive input method, and may in the future allow users to perform gestures by pointing or moving their hands in front of a device, without even touching it, new patents reveal.




The company has filed for a pair of new touch-related inventions with the U.S. Patent and Trademark Office that were published this week, as discovered by AppleInsider. The first, entitled "Touch and Hover Sensor Compensation," describes how a device might use multiple adjacent sensors to sense movement and gestures away from a screen.

Known as "hover" gestures, these three-dimensional movements can open up entirely new ways for users to interact with a device. Apple's filing notes that there are motion sensing devices on the market, but providing reliable and accurate responses to hover gestures is incredibly difficult.




Apple's proposed invention is related to compensation for sensors. The company aims to reduce errors by using adjacent sensors together, and applying a gain factor to measurements they capture, reducing sensitivity variation at different sensor locations on the device.

"Sensor compensation can advantageously provide improved touch and hover sensing," the filing reads.




In addition, the USPTO disclosed a second filing on Thursday for a patent application entitled "On-Cell Touch Architecture," describing ways to make touch panels on devices like the iPhone and iPad even thinner.

But the second filing also makes note of 3D hover gestures. In the proposed patent, Apple notes that some capacitive touch sensing systems use fringing electrical fields to sense not only touches on the display, but also sensing fingertips beyond the surface of the display.

"Objects approaching near the surface may be detected near the surface without actually touching the display," the application reads.




Both filings are particularly noteworthy as Apple acquired gesture input company PrimeSense in a $360 million deal in 2013. PrimeSense is best known for creating the technology that powered Microsoft's first-generation Kinect sensor, used for motion controls and gameplay on its Xbox 360 game console.

Why Apple acquired PrimeSense remains a mystery, but the existing use of its company's technology in the living room has led to speculation that a next-generation Apple TV could offer some sort of advanced motion input. Such capabilities would allow users to control their Apple TV without the need to grab a remote control, iPhone, or other accessory.

As for Apple's new Force Touch input method, it was first introduced on the Apple Watch as a pressure-sensitive way to allow for new ways of interacting on a smaller display, but the company quickly adopted the technology for trackpads on its latest MacBook Pros and the new 12-inch MacBook with Retina display. It's also expected that Force Touch will be a key addition to this fall's anticipated "iPhone 6s," "iPad Air 3" and "iPad mini 4" upgrades.

Comments

  • Reply 1 of 20
    rogifanrogifan Posts: 10,669member
    Didn't Samsung try something like this? I swear I remember seeing a hokey commercial with some guy whose hands were covered in BBQ sauce but he was still able to manipulate his phone just by hovering his finger over the screen. Seemed very gimmicky.
  • Reply 2 of 20
    yojimbo007yojimbo007 Posts: 1,143member
    rogifan wrote: »
    Didn't Samsung try something like this? I swear I remember seeing a hokey commercial with some guy whose hands were covered in BBQ sauce but he was still able to manipulate his phone just by hovering his finger over the screen. Seemed very gimmicky.

    Yup.. It was done through the cameras.. Different and inferior approach
  • Reply 3 of 20

    "Apple is looking beyond its new Force Touch pressure-sensitive input method, and may in the future allow users to perform gestures by pointing or moving their hands in front of a device, without even touching it, new patents reveal." -- AppleInsider

     

    Uh, no. Patents like these existed BEFORE Force Touch. Force Touch just happened to be introduced to the world first perhaps due to technology being advanced enough to support Force Touch.

     

    I believe Apple has been working on multiple paths that will eventually become one path of awesomeness. From your own archives...

     

    http://appleinsider.com/articles/14/04/24/apples-display-tech-lets-users-interact-with-3d-objects-in-mid-air

    http://appleinsider.com/articles/13/08/20/apple-patents-3d-gesture-ui-for-ios-based-on-proximity-sensor-input

     

    Here is to hoping the iPad Pro and/or the next iPhone has these features. Better yet, these patents could be the prelude to a brand new hardware product that opens a brand new platform for the Apple ecosystem that exists outside the iPad, iPhone and Apple TV platforms yet can coexist with those platforms.

  • Reply 4 of 20
    fotoformatfotoformat Posts: 300member
    Maybe this idea is for the day when the proposed Apple iCar arrives, and it won't have a steering wheel... just point your pinkie and the vehicle will turn exactly where you want to go!
  • Reply 5 of 20
    fastasleepfastasleep Posts: 5,689member
    Here comes the hover-to-reveal mystery meat navigation a la iTunes 12 to iOS.
  • Reply 6 of 20
    Hover, touch, force touch.
  • Reply 7 of 20
    crowleycrowley Posts: 8,717member
    Apple can investigate whatever they want, but I'll be very surprised if hover gestures ever take the place of tactility.
  • Reply 8 of 20
    Quote:

    Originally Posted by Crowley View Post



    Apple can investigate whatever they want, but I'll be very surprised if hover gestures ever take the place of tactility.



    Wouldn't the hover gestures and 3D technology shown in the Iron Man movies be awesome to have Apple bring to the masses? This is where I see the technology heading.

  • Reply 9 of 20
    MarvinMarvin Posts: 14,541moderator
    crowley wrote: »
    Apple can investigate whatever they want, but I'll be very surprised if hover gestures ever take the place of tactility.

    Wouldn't the hover gestures and 3D technology shown in the Iron Man movies be awesome to have Apple bring to the masses? This is where I see the technology heading.

    It's more useful as a supplementary input rather than primary. Your arms/hands would get tired holding them up for long periods:


    [VIDEO]


    [VIDEO]


    The LEAP demo shows how it can be used for some games. It would be able to detect objects like when you are holding a pen to be able to do accurate palm rejection. It can support gestures for multi-tasking, split view, zoom/rotate/pan. It can precache website links that you are about to touch, it can type ahead keys you are about to press. It can do facial recognition by holding your face in the 3D scanner as well as fast avatar creation in games to put your face in the game. It can detect hand size for age controls and block inappropriate content or expensive purchases.

    Where it can replace tactile input for prolonged use would be if you could sit the iPad vertically on a table and be able to simply move your hands horizontally on the desk in front of it and it would detect passive objects like a mock keyboard that has no electronics inside, it can be a rubber sheet. It could tell which keys you are pressing based on the edges of the rubber sheet. The keyboard can alternatively be on-screen and your hand position on the table would determine which keys you are pressing.
  • Reply 10 of 20
    crowleycrowley Posts: 8,717member
    Quote:

    Originally Posted by leavingthebigG View Post

     



    Wouldn't the hover gestures and 3D technology shown in the Iron Man movies be awesome to have Apple bring to the masses? This is where I see the technology heading.


    No.  It'd be super annoying and strenuous.  Good demo, crap practice.

     

    If there was some way they could build a tactile feedback into a holographic interface with micro interaction then that might be something.

  • Reply 11 of 20

    Wouldn't the hover gestures and 3D technology shown in the Iron Man movies be awesome to have Apple bring to the masses? This is where I see the technology heading.

    Apple also owns PrimeSense.
  • Reply 12 of 20
    Yes, I know this. I think Apple is doing the "Apple" think different thing with the Prime Sense technology.
  • Reply 13 of 20
    crowley wrote: »
    No.  It'd be super annoying and strenuous.  Good demo, crap practice.

    If there was some way they could build a tactile feedback into a holographic interface with micro interaction then that might be something.

    Your idea is definitely Star Trek Next Generation! That would be great!!
  • Reply 14 of 20
    gregqgregq Posts: 62member
    Personally couldn't care less about this, I'm sure others find more merit to this. I'm waiting for apple to deploy eye-tracking auto-stereoscopic 3d displays on iPhones/iPads.
  • Reply 15 of 20

    I can't wait to crack my iPhone screen if I can just hover over it after! no more getting glass in your finger! lol

  • Reply 16 of 20
    Microsoft had this exact tech with their cancelled phone mclaren that was supposed to come out this past December. Supposedly they couldn't come up with a good me enough use for it. They were going to use it as extra options when you hovered over a tile
  • Reply 17 of 20
    xixoxixo Posts: 430member

    my ipad and iphone do this all the time. it's annoying when you move your hand over top of the display surface and the system responds as if it has been touched.

  • Reply 18 of 20
    not sure which axis it measures on, from those drawings it certainly isn't designed for an iPhone or any other existing device.

    they all have fairly definite xyz planar surfaces, this thing kind of cuts through the xz and makes a sort of rhombus in cross section type touch surface. maybe that has some advantage in the method
  • Reply 19 of 20
    Quote:
    Originally Posted by Rogifan View Post



    Didn't Samsung try something like this? I swear I remember seeing a hokey commercial with some guy whose hands were covered in BBQ sauce but he was still able to manipulate his phone just by hovering his finger over the screen. Seemed very gimmicky.

    Yes and Samsung still does.

    The Feature is called Air View and exists at least since Galaxy S4 and it just works good. Actually it behaves like hovering with a mouse pointer. For instance:

    - In phone app, you hover over the numbers and it shows you the linked contacts

    - In gallery you hover over the small preview pictures and it opens them in a much bigger preview

    - In Video player you hover over the time scale and like youtube it shows you the time and the scene in preview mode

    - In email client in the list view you just hover over the emails and it shows you the first view lines without opening the emails

    - In Internet Browser it works like a magnifier

    Actually it is a much better feature than Force Touch on a touch Screen.

  • Reply 20 of 20
    gatorguygatorguy Posts: 23,165member
    crowley wrote: »
    No.  It'd be super annoying and strenuous.  Good demo, crap practice.

    If there was some way they could build a tactile feedback into a holographic interface with micro interaction then that might be something.

    Your idea is definitely Star Trek Next Generation! That would be great!!
    There's quite a few patents related to holographic touch interfaces, some pre-dating even TNG. At least one large tech has several in that space so it seems to be a focus of theirs. I doubt they're the only ones but I didn't spend more than a few minutes looking.

    If you want to see what's being done in that area use the specialty Google search product Advanced Patent Search (simply type Advanced Patent Search in the search bar to get to the page) and enter whatever search terms you're wanting to find. It's pretty straightforward. I don't know of any other general search provider with a similar free service but there might be if you prefer not to use Google.
Sign In or Register to comment.