Apple patents accurate touch and hover panel, embedded heart rate monitor

Posted:
in General Discussion edited January 2014
Apple on Tuesday was granted two iOS device-targeted patents, one for a "touch and hover" display panel that is made more accurate by compensating for signal drift, and another for heart rate monitor that can be seamlessly integrated into a handset.

Touch and hover

The newer of Tuesday's patents, Apple's extensive U.S. Patent No. 8,614,693 for "Touch and hover signal drift compensation" describes a system in which a touchscreen display can accurately determine both hover and touch events.

Hover
Source: USPTO


A number of modern computing devices, like Apple's iPhone and iPad, incorporate touch-sensitive panels that enhance and define the user experience by affording unprecedented GUI manipulation and control. Some systems also incorporate what is known as "hover" controls, which allow users to interface with a device without actually touching it.

Using specialized internal components, these touch sensitive devices can recognize an object hovering above a display panel, like a user's finger or stylus. Once a hover event has been detected, the device may process it as a touch event, handling subsequent actions according to the general rules of traditional touch input. For example, if a hover event occurs over a specific app, that app may be opened.

Hover


A general overview of hover sensing technology is provided, including a driver control system in which can generate electrical fields that extend out from the touch panel. When a finger or other object passes through these fields it changes the capacitance of trace lines in the sensor array. By processing the capacitance fluctuations, the system is able to detect a hover event.

As noted in Apple's patent, incorporating touch and hover capabilities into one device is a difficult proposition when considering reliability and accuracy. A user may, for example, be holding a phone with their left hand while inputting hover events with the right. If said user's left thumb accidentally touches the screen, the device may register it as a touch event.

In one embodiment, touch event compensation can be applied to ignore errant touches. Much like iOS palm rejection technology, touch signal compensation can intelligently determine a user's intentions by inferring activity based on signal strength.

Hover


For example, as seen above, touch panel may be segmented into quadrants, each of which measures capacitance signals of both touch and hover events. Depending on the location and saturation of a signal, as well as duration and other metrics, the system can classify an action as a legitimate touch or hover event. Managing the system is a set of rules based on predefined thresholds.

In other embodiments, object shape and profiling may be implemented to parse out hover events. For example, a user may point at a UI asset such as an app icon, which results in a certain shape profile as "seen" by the touch screen's sensors. Size and distance from a sensor panel may also be used to determine whether a user intended to invoke a hover event. Also provided for are solutions for concurrent touch/hover events and multi-hover events as measured and processed by capacitance and threshold rules.

Hover


Apple's patent particularly focuses on compensation for signal drift in a touch sensor panel. Touch sensing devices measure changes from a baseline capacitance -- that is when no object is touching or hovering over the device -- generated by touching or hovering over the array. Environmental factors like ambient temperature, humidity and pressure play a large role in how well the system operates. Operating changes, such as component shifts, expansion or contraction, could also impact baseline.

When these changes to the baseline capacitance substantially affect measurements indicative of a touch or hover event, it is known as signal drift.

Hover


To combat signal drift, Apple proposes a compensation method that resets the baseline capacitance of a touch sensor at periodic intervals, thereby accounting for environmental, operating, mechanical and other changes. When a new baseline is defined, the system can apply the change to capacitance measurement thresholds to correct for signal drift.

In one embodiment, a grounded plate can cover the touch panel when not in use, such as a protective device cover. With the plate temporarily covering the panel, and no touch or hover events being logged, a baseline reading can be taken and a subsequent signal drift compensation applied.

Hover


When the plate is not covering the screen, recalibration is suspended because it is more likely that the user will be touching or interacting with the device during this time. Alternatively, the sensor panel can recalibrate when the device is attached to a grounded dock.

In another embodiment, the device can take readings dynamically in between touch and hover events. By using on-board sensors, the recalibration system can intelligently select the most opportune time in which to reset the baseline. For example, recalibration might be suspended if the device is thought to be in motion, or if touch and hover events are being logged. When the unit is stationary and no touch or hover event is recognized, the system may set for baseline signal drift.

The patent continues with an in-depth look at other touch/hover panel implementations, including touch and however switching, signal resistance calibration and sensitivity variation compensation, among others. Also covered are various input techniques, including cameras and on-board motion sensors, and their integration into a display.

Hover


It is unknown if Apple will one day choose to incorporate hover controls into its products, but some companies are already working on their own solutions. Rival Samsung, for example, recently rolled out hover-based Air View and Air Gestures features with its Galaxy S4 handset.

Apple's touch and hover sensor panel patent was first filed for in 2010 and credits Brian Michael King, Omar Leung, Paul G. Puskarich, Jeffrey Traer Bernstein, Andrea Mucignat, Avi E. Cieplinski, Muhammad U. Choudry, Praveen R. Subramani, Marc J. Piche, David T. Amm and Duncan Robert Kerr as its inventors.

Heart rate monitor

As for the heart rate monitor, Apple's U.S. Patent No. 8,615,290 for a "Seamlessly embedded heart rate monitor" remains largely unchanged from AppleInsider's initial report on the property's application in 2010. The patent employs sensors to read a user's EKG data, which can then be used to authenticate said user or indicate their mood.

Patent 1


Basically, Apple's invention measures cardiac signals read by sensors embedded into metallic conductive portions of a device housing. The sensor leads can be successfully hidden or embedded, into the chassis housing, possibly in the screen bezel. These leads pick up minute electrical signals unique to a specific user's heart, which can be measured to a degree of accuracy high enough as to provide an adequate means of identification.

The latest Apple patent is evidence that the company is actively investigating deployable biometric security solutions that rely on users' bodies rather than stored or memorized codes, one such system being the Touch ID fingerprint reader.

Apple credits Gloria Lin, Taido Nakajima, Pareet Rahul, and Andrew Hodge as the patent's inventors. The property was originally filed for in 2009.

Comments

  • Reply 1 of 17
    I'm currently looking for a watch that can give me all the data I need for my 70.3 Ironman (half a triathlon) training. Looks like my preference goes to the Suunto Ambit2. Maybe I'm too focused on this, but this patent looks like it would be great for a watch. The calibration description, the hovering, errant touches...Then there is the drawing of the square display, or rectangle in landscape orientation. I only don't know what to make of the cover, could it be for protection during a mountain bike trip? Cycling through the bushes, branches and what not might be what the cover is for?

    Thoughts? Christopher126?
  • Reply 2 of 17
    solipsismxsolipsismx Posts: 19,566member
    I'd think this is for a wearable computer, like one worn on a wrist, and something that will hopefully come next year. The touch sensor would be an impressive bit of tech that could easily be used while jogging or doing other activities to take your pulse by simply touching the display. That sounds easy and natural to me. The hover gesture could be used in a watch as well and certainly more practical than on a smartphone.

    I'd also like a touch sensitive remote control for the Apple TV (Apple HDTV) that can tell which user is operating the remote and then alter the UI, lineup, saved and favorites, etc. accordingly.
  • Reply 3 of 17
    My iPhone 5 already does the "hover touch".

    Many times it will record a touch when my finger hasn't hit the surface of the screen yet.

    Alien technology? Next: ouija board tech?
  • Reply 4 of 17
    Quote:
    Originally Posted by PhilBoogie View Post



    I'm currently looking for a watch that can give me all the data I need for my 70.3 Ironman (half a triathlon) training. Looks like my preference goes to the Suunto Ambit2. Maybe I'm too focused on this, but this patent looks like it would be great for a watch. The calibration description, the hovering, errant touches...Then there is the drawing of the square display, or rectangle in landscape orientation. I only don't know what to make of the cover, could it be for protection during a mountain bike trip? Cycling through the bushes, branches and what not might be what the cover is for?



    Thoughts? Christopher126?

    I don't think you'll go wrong with the Ambit2, Phil. Here's another well-done video review. Goes right thru the features for training and racing. I like the heart rate monitor a lot and the GPS function. Display looks really good, with large numbers.

     

    I wouldn't wait for the Apple version.... Idea: You could get the Ambit2, use it, get fit. (Keep all the packaging.)

     

    When the Apple watch comes out, and if it's superior to the Ambit, sell the AMbit.

     

    Basically you "rented" the Ambit for, say $10-$15/mo. or so.

     

    That's how I rationalize all my Apple purchases. Sell my 3 year old (at the time) MacBook for $300 to get the new iPad and basically "rented" the MacBook for $25/mo ($900/36mo.s=$25/mo).

     

    Caveat: This may be a "stupid" rationalization! :)

     

    Best. 

     

  • Reply 5 of 17
    Quote:
    Originally Posted by SolipsismX View Post



    I'd think this is for a wearable computer, like one worn on a wrist, and something that will hopefully come next year. The touch sensor would be an impressive bit of tech that could easily be used while jogging or doing other activities to take your pulse by simply touching the display. That sounds easy and natural to me. The hover gesture could be used in a watch as well and certainly more practical than on a smartphone.



    I'd also like a touch sensitive remote control for the Apple TV (Apple HDTV) that can tell which user is operating the remote and then alter the UI, lineup, saved and favorites, etc. accordingly.

    'The Remote' will end up being your iDevice.   If you go with Apple's basic premise that every man, woman, child, and dog will have their TouchID identified iP*, then the moment you click on the 'Remote' App, you'll be identified to the AppleTV (or whatever home system you are trying to remote into... (e.g. thermostat, garage door opener, Kuerig [oh Solipsism, here's your half caf, no fat, 2 sugar latte], gun case, whatever)*).  This is consistent with 'identifying' you as an AppleID to the world through Apples' secret sauce.

     

    Same with your car... if you plug in your phone [BT, AirPlay, cable, and touchID authenticate], you'll set up to your 'specs'  (your playlist, FM channel choice, Environmentals, mirror positions, shifting preferences)*

     

    I'm more curious about the HR monitor.   This is given for health management, which is a major cash cow (if Apple can get 30% of all 'remote monitoring fees' under Obamacare, that would be printing money;-), but the 'mood sensor'?   I'd like it to be more a reverfication of 'who I am' (oh, your Twave is this... your relative amplitude is that... your slight murmur is whatsit ... you're 90% likely to be ToG... and let's see if the fingerprint confirms....)

     

    *pure speculation and humorous list... YMMV.

  • Reply 6 of 17
    solipsismxsolipsismx Posts: 19,566member
    'The Remote' will end up being your iDevice.   If you go with Apple's basic premise that every man, woman, child, and dog will have their TouchID identified iP*, then the moment you click on the 'Remote' App, you'll be identified to the AppleTV (or whatever home system you are trying to remote into... (e.g. thermostat, garage door opener, Kuerig [oh Solipsism, here's your half caf, no fat, 2 sugar latte], gun case, whatever)*).  This is consistent with 'identifying' you as an AppleID to the world through Apples' secret sauce.

    Same with your car... if you plug in your phone [BT, AirPlay, cable, and touchID authenticate], you'll set up to your 'specs'  (your playlist, FM channel choice, Environmentals, mirror positions, shifting preferences)*

    I'm more curious about the HR monitor.   This is given for health management, which is a major cash cow (if Apple can get 30% of all 'remote monitoring fees' under Obamacare, that would be printing money;-), but the 'mood sensor'?   I'd like it to be more a reverfication of 'who I am' (oh, your Twave is this... your relative amplitude is that... your slight murmur is whatsit ... you're 90% likely to be ToG... and let's see if the fingerprint confirms....)

    *pure speculation and humorous list... YMMV.

    1) I like your speculation on the remote monitoring fees business.

    2) I think using an iDevice for the remote should exist but I personally never use mine as such simply because it seems so clunky compared to the included remote since the buttons are all there and I don't need to look away from the TV to use it.
  • Reply 7 of 17
    ^ post

    Thanks for that video. Yep, this watch seems to do it all, seems to be the most robust, and the fact that you can get/write apps for it is just amazing: don't like the pre-defined info it's displaying? Modify it yourself, or download an app.

    And your rationalisation is considered deductive reasoning!
  • Reply 8 of 17
    Quote:

    Originally Posted by PhilBoogie View Post





    Thanks for that video. Yep, this watch seems to do it all, seems to be the most robust, and the fact that you can get/write apps for it is just amazing: don't like the pre-defined info it's displaying? Modify it yourself, or download an app.



    And your rationalisation is considered deductive reasoning!

    Thanks for that, Phil.

     

    Merry Xmas

  • Reply 9 of 17
    philboogie wrote: »
    Yep, this watch seems to do it all...

    Everything but the 70.3 km you're training for. :p I don't know how people do that. That is well beyond my ability to commitment to such a grueling mental and physical effort. Good luck.
  • Reply 10 of 17
    solipsismx wrote: »
    philboogie wrote: »
    Yep, this watch seems to do it all...

    Everything but the 70.3 km you're training for. :p I don't know how people that do that. That is well beyond my ability to commitment to such a grueling mental and physical effort. Good luck.

    Ha! 'everything but' Excellent response.

    Yes, quite the challenge. This winter I will focus on the swimming part. I've been cycling for 5 years now, I really got into running just this summer, earlier attempt got me 'off the wrong foot'. But I'm getting to the point where it isn't a gruelling experience anymore. Once you get into the 'mood' of the sweating, there aren't any muscle aches anymore and your times get better it's rely a 'fun' thing to do.

    Just don't ask me on progress or lap times; I'm merely trying to train for it. If I am fast enough to enlist (there's usually a cut-off time, I believe 8 hours for the official 70.3) I just want to finish. Maybe I can 'set some sort of 'record' in my age group, but that's not my goal.

    I'm sure your surfing isn't a 'light event' either. I did it once, as a kid, but I was way too impatient to wait for the water to drop from the sail so I tried to 'just pull' the sail out of the water, instead of using my weight. Fail, and haven't tried since. Occasionally I watch it though, they all seem to be so free, surfing at sea.

    Merry Christmas sir.
  • Reply 11 of 17

    Originally Posted by AppleInsider View Post

    ... sensors to read a user's EKG data, which can then be used to authenticate said user ...

     

    iPhone - Touch ID.

    "iWatch" - Heart ID.

     

    And if it's capable of generating an EKG, maybe the "iWatch" could send an emergency signal if you're about to have a heart attack (or if you flatline).  And maybe "iWatch" will be a must-have in the modern hospital.  Just a thought.

  • Reply 12 of 17
    jakebjakeb Posts: 562member

    Wait a minute, you can identify someone by the electrical signals of their heart? Fascinating.

  • Reply 13 of 17
    Originally Posted by jakeb View Post

    Wait a minute, you can identify someone by the electrical signals of their heart? Fascinating.

     

    Not only that, you can identify different users by the capacitance of their fingers!

     

    NOTE: The following is for illustrative purposes only and does not represent a case of me condoning multi-user portable touchscreen devices.

     

    Imagine, just by touching the screen, without even a fingerprint to remember, the device knows who is using it and changes settings accordingly. So think a multitouch terminal in an office. Just start using it and your files come up on it.

  • Reply 14 of 17
    I have ZERO interest in biometric authentication. No thanks Apple -- how about something actually useful, like Push-To-Talk and a better speakerphone?!
  • Reply 15 of 17
    Quote:

    Originally Posted by libertyforall View Post



    how about something actually useful, like Push-To-Talk and a better speakerphone?!

     

    agree re phone speakerphone quality, but why is PTT considered better than plain old just talk?

     

    you like hearing the annoying PTT 'chirp' everytime?

     

    i think someone still makes CB radios...

  • Reply 16 of 17
    jakebjakeb Posts: 562member
    PTT was considered so cool in the early 2000s. I think it just made people feel important or that they were using a newer ( and therefore fancier) kind of phone.
  • Reply 17 of 17
    I have ZERO interest in biometric authentication. No thanks Apple -- how about something actually useful, like Push-To-Talk and a better speakerphone?!

    It's ironic that you, with your ridiculously extreme desire for privacy, would want walkie-talkie style communication.
Sign In or Register to comment.