Apple invention uses vibrations and temperature to simulate different materials on touchscreens, tra

Posted:
in General Discussion edited April 2015
An Apple patent application published by the U.S. Patent and Trademark Office on Thursday details an advanced haptic feedback system that simulates the feeling of different textures on a device display, such as wood and metal, using localized vibrations and temperature changes.

iPhone


Aptly titled "Touch Surface for Simulating Materials," Apple's invention takes haptic feedback a step further than current systems seen on Apple Watch or even the MacBook line's mind-bending Force Touch multitouch trackpad.

Based on the basic principals of haptic feedback, Apple's proposed system applies an actuator capable of moving horizontally, vertically, or a combination of both to create a vibratory sensation on a user's finger. Varying the amplitude of such vibrations over time and space can simulate rough or smooth surfaces. Rough surfaces and textures may be presented as stronger vibrations compared to smooth surfaces, for example.

Touch
Source: USPTO


The system also harnesses data from capacitive sensors, positioning sensors, force sensors and other sensing components to detect the location and pressure of a user's finger on the screen. Tying it all together with computer display technology, Apple asserts that its haptic system can reproduce the feeling of different textures, surfaces and materials on a glass or plastic substrate.

In one embodiment, a user would be able to "feel" wood grain when they draw their finger over an image of a plank of wood. Vibrations can be adjusted based on finger pressure, reflective of increased force on a simulated material. Although left unmentioned in the patent application, such a system could adapted for iOS or OS X to distinguish borders of app icons, lines of text and more.

Perhaps most interesting is Apple's mention of Peltier devices, otherwise known as thermoelectric coolers or heat pumps. Operation of a TEC is based on the Peltier effect, which describes heating or cooling resulting from an electric current flowing across the meeting point of two conductive materials. Arranging conductors in alternating series expands the effect across a plane, or in most cases a plate, drawing heat to one side while cooling the other.

Peltier devices can be found in certain consumer gadgets and scientific instruments, though computer buffs may recognize TECs as a technology used in overclocking.

TouchA touch surface is broken into regions for localized vibratory/temperature feedback.


As applied to Apple's patent, embedded Peltier devices can enhance simulated touch by adding temperature fluctuations to the mix. Continuing with the example above, a screen's temperature may be made warmer when displaying a piece of wood, or cooler when displaying a sheet of steel. Taking the idea even further, Apple points out that surface temperatures can change dynamically, warming simulated metal or plastic as it is "heated" by a user's finger.

These TECs need not be distinct components, Apple says, and can instead be piezoelectric actuators that also produce heat or a combination of components stacked on top of each other.

iPhone


Apple is showing greater interest in haptic feedback systems, as seen in Apple Watch and trackpads incorporated into the latest MacBook models. Called Force Touch, the MacBook version is a completely self-contained system employing force sensors, a vibration motor and controller (Taptic Engine) to respond -- physically and onscreen -- to user clicks, presses and other gestures. Unlike the hinge-based multitouch trackpad it replaces, Apple's Force Touch unit doesn't move, but the effect of simulated clicks is so convincing that many people can't tell the difference.

A number of industry players are already developing haptic feedback systems for generating simulated textures on flat displays, including a unique friction-based solution from Disney Research and various designs by Microsoft. The field is still in its infancy, however, leaving Apple in a good position to break new ground with potential consumer-facing products.

Apple's surface simulation patent application was first filed for in October 2013 and credits Paul G. Puskarich as its inventor. Interestingly, Puskarich is also named as a principle inventor on another Apple patent detailing a haptic feedback concept nearly identical to MacBook's Force Touch trackpad. That particular patent was uncovered by AppleInsider three years ago.

Comments

  • Reply 1 of 14
    xixoxixo Posts: 449member
    [VIDEO]

    NSFW, but iPhone compatible. Talk about haptic feedback...
  • Reply 3 of 14
    Holy. Shit.
  • Reply 4 of 14
    So where are the updates to the external user devices that are in need of a refresh?
    - updating the Apple Trackpad and even better the top surface of the Apple Mouse?
    - updating the Apple Wireless Keyboard with the new butterfly switches?
  • Reply 5 of 14
    mike1mike1 Posts: 3,299member

    Wow. Implementation of that into real products is true innovation. I can think of so many uses and I'm sure I'm not even scratching the surface. E-brochures that mimic the feel of real high-quality paper and materials, more immersive games, realistic-feeling photos.

  • Reply 6 of 14
    So where are the updates to the external user devices that are in need of a refresh?
    - updating the Apple Trackpad and even better the top surface of the Apple Mouse?
    - updating the Apple Wireless Keyboard with the new butterfly switches?

    Let's consider: AA's in the MagicTrackpad and Magic Mouse would take a beating with haptic feedback. And is there anything wrong with the current keyboard?
  • Reply 7 of 14
    chadmaticchadmatic Posts: 285member



    A nod to String Theory

  • Reply 8 of 14
    This might be part of the UI if Apple ever goes back to skeumorphic UIs. Otherwise, when would it be used apart from games?
  • Reply 9 of 14
    xixoxixo Posts: 449member
    Let's consider: AA's in the MagicTrackpad and Magic Mouse would take a beating with haptic feedback. And is there anything wrong with the current keyboard?

    Agreed. Not everyone is enamored with the new trackpad and keyboard in the MacBook.

    Since this affects the UI, I'm sure much more will be said at WWDC '15.
  • Reply 10 of 14
    MacProMacPro Posts: 19,750member
    This might be part of the UI if Apple ever goes back to skeumorphic UIs. Otherwise, when would it be used apart from games?

    I couldn't help imaging a possible conversation about that very point ... somewhere deep inside Apple R&D ... "Wow feel this, it is amazing ... close your eyes and it feels like you are touching leather. It would be amazing for the ...oh damn ... I just remembered." :D
  • Reply 11 of 14
    This was probably Scott Forstall's idea.
    Imagine the feeling of rich Corinthian leather in Address Book and the feel of the felt table in Game Center!
  • Reply 12 of 14
    smiffy31smiffy31 Posts: 202member
    Quote:
    Originally Posted by xixo View Post



    Agreed. Not everyone is enamored with the new trackpad and keyboard in the MacBook.


    I agree with the keyboard but show me someone who was not totally won over by the new trackpad.
  • Reply 13 of 14
    Let's consider: AA's in the MagicTrackpad and Magic Mouse would take a beating with haptic feedback. And is there anything wrong with the current keyboard?
    There isn't just that, they could put in lithium ion batteries, add backlit(even to the mouse?), update the keyboard to basically same thing as the MacBook has wouldn't hurt.
    This was probably Scott Forstall's idea.
    Imagine the feeling of rich Corinthian leather in Address Book and the feel of the felt table in Game Center!
    Oh my, that would have been lovely, the current ui would be all glass, maybe some metal, oh wait that's already there.

    This sounds like Apple is reinnovating the touchscreen. Something that's essentially the #1 priority for smartphones and tablets.
  • Reply 14 of 14
    19831983 Posts: 1,225member
    This is one of Apple's most interesting patients to date. I hope we don't have to wait too long for products incorporating this tech. Having it introduced on the iPhone 7 would be nice but unlikely, probably will have to wait for something like this a good few years down the line, on the iPhone 8 or 9 maybe.
Sign In or Register to comment.