Future iPhone screen could simulate surface textures and temperatures for AR
Touching a future iPhone or iPad display could provide a sensation of texture, with Apple coming up with a way to alter how a touch-enabled surface can be controlled to simulate the feel of different types of materials.
iPhone 11 Pro and iPhone 11 Pro Max
Touchscreens are a common interface for the vast majority of mobile devices, with the visual element and haptic feedback providing users with a considerable amount of stimulus in relation to the content they are interacting with. However, no matter how it appears on the screen, users are still touching a glass or plastic surface, and won't get a full physical feedback experience.
Regardless of whether it looks like you're touching sandpaper or a kitten's fur on an iPad display, at the end of the day you're just going to feel glass.
In a patent granted by the U.S. Patent and Trademark Office on Tuesday titled "Touch surface for simulating materials," Apple suggests ways that a surface could be altered to give a variety of sensations that somewhat mimic that of different textures.
The patent suggests the use of actuators, temperature control devices, and a main control unit that manages the use of both types of feedback generators over a specific portion of a touch surface. In the case of the actuators, the control unit will use it to vibrate the display to give a sensation of texture.
The user of a Peltier device or another temperature control component can be used to simulate the thermal conductivity of a material. For example, a wooden table may "feel" warm, while a metal surface could feel cold to the touch or warm up over time.
Depending on the implementation, the system could be set up to function in only one region of a touch surface, multiple areas, or across the entire surface. In instances where multiple regions are in play, each region could be made to simulate different materials at the same time, so that one hand could feel something soft and warm while the other could sense cold and roughness.
Selecting specific areas of the display to apply the technology to could be due to the difficulty of applying multiple sets of components all across a display. By limiting it to designated regions, such as opposite edges of a screen that are usually touched for gaming in landscape orientation, this would make such a system be more practical from a manufacturing standpoint.
Apple's patent illustration showing the use of actuators, temperature controls, and a diamond material to simulate materials on a touch surface
Apple suggests the touch surface could use a layer of a diamond material, possibly using a chemical vapor deposit method, which will provide "extremely high thermal conductivity" and "extreme mechanical hardness" to the display while maintaining optical transparency.
While the patent has more immediate applications for touchscreen-enabled devices, Apple does highlight its usage on the touchpad area of a MacBook in one image. This may make sense as it would be simpler to implement on a touch-enabled surface that doesn't include a display element.
Apple files numerous patent applications on a weekly basis, but while the existence of a patent suggests areas of interest for the company's research efforts, it doesn't guarantee the concept will make an appearance in a future product or service.
In the case of texture, it is arguable that Apple could implement part of the patent in devices right now. Apple's Taptic Engine in its iPhones provides a variety of different buzzing patterns and levels, which could easily be used to buzz in different ways to simulate a surface material.
Of course, temperature is not something the iPhone is capable of simulating right now. Given the need to keep devices as safe as possible for consumers, it is probable that Apple would want to ensure any thermal-altering system doesn't have the opportunity to cause injury, such as burning a user's finger by simulating a hot frying pan.
Apple has previously looked into the possibility of providing a more tactile experience to its users, such as in the March 2017 patent for "User Interface having changeable topography," where the display changes shape to have raised sections. For example, this could include raised keys on a keyboard or on a calculator.
An earlier patent filing from July 2015 involved a stylus with onboard haptic feedback components to vibrate the implement, based on information collected about the material the stylus is "touching" on the display.
iPhone 11 Pro and iPhone 11 Pro Max
Touchscreens are a common interface for the vast majority of mobile devices, with the visual element and haptic feedback providing users with a considerable amount of stimulus in relation to the content they are interacting with. However, no matter how it appears on the screen, users are still touching a glass or plastic surface, and won't get a full physical feedback experience.
Regardless of whether it looks like you're touching sandpaper or a kitten's fur on an iPad display, at the end of the day you're just going to feel glass.
In a patent granted by the U.S. Patent and Trademark Office on Tuesday titled "Touch surface for simulating materials," Apple suggests ways that a surface could be altered to give a variety of sensations that somewhat mimic that of different textures.
The patent suggests the use of actuators, temperature control devices, and a main control unit that manages the use of both types of feedback generators over a specific portion of a touch surface. In the case of the actuators, the control unit will use it to vibrate the display to give a sensation of texture.
The user of a Peltier device or another temperature control component can be used to simulate the thermal conductivity of a material. For example, a wooden table may "feel" warm, while a metal surface could feel cold to the touch or warm up over time.
Depending on the implementation, the system could be set up to function in only one region of a touch surface, multiple areas, or across the entire surface. In instances where multiple regions are in play, each region could be made to simulate different materials at the same time, so that one hand could feel something soft and warm while the other could sense cold and roughness.
Selecting specific areas of the display to apply the technology to could be due to the difficulty of applying multiple sets of components all across a display. By limiting it to designated regions, such as opposite edges of a screen that are usually touched for gaming in landscape orientation, this would make such a system be more practical from a manufacturing standpoint.
Apple's patent illustration showing the use of actuators, temperature controls, and a diamond material to simulate materials on a touch surface
Apple suggests the touch surface could use a layer of a diamond material, possibly using a chemical vapor deposit method, which will provide "extremely high thermal conductivity" and "extreme mechanical hardness" to the display while maintaining optical transparency.
While the patent has more immediate applications for touchscreen-enabled devices, Apple does highlight its usage on the touchpad area of a MacBook in one image. This may make sense as it would be simpler to implement on a touch-enabled surface that doesn't include a display element.
Apple files numerous patent applications on a weekly basis, but while the existence of a patent suggests areas of interest for the company's research efforts, it doesn't guarantee the concept will make an appearance in a future product or service.
In the case of texture, it is arguable that Apple could implement part of the patent in devices right now. Apple's Taptic Engine in its iPhones provides a variety of different buzzing patterns and levels, which could easily be used to buzz in different ways to simulate a surface material.
Of course, temperature is not something the iPhone is capable of simulating right now. Given the need to keep devices as safe as possible for consumers, it is probable that Apple would want to ensure any thermal-altering system doesn't have the opportunity to cause injury, such as burning a user's finger by simulating a hot frying pan.
Apple has previously looked into the possibility of providing a more tactile experience to its users, such as in the March 2017 patent for "User Interface having changeable topography," where the display changes shape to have raised sections. For example, this could include raised keys on a keyboard or on a calculator.
An earlier patent filing from July 2015 involved a stylus with onboard haptic feedback components to vibrate the implement, based on information collected about the material the stylus is "touching" on the display.
Comments
Some degree of drag applied to the Pencil would provide better control of the strokes, as already implemented on the iPad Pros. Or an accurate implementation of the Touch ID under the display. Besides those I don’t think anyone needs to feel any kind of wetness on the display.
Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
If this is a matter of "research" then the "research" may be targeted more on current products than imaginative products.
Wow never heard this quote. It defines me to the core. When I read patents I can think of tens or hundreds of users while others simply say "this won't work".
Ummm...WHAT? This can go beyond "feeling" app icons. This can help with everything from typing to gaming to medical.
Awesome idea! May even be implemented on Watch/iPhone if it detects you're driving/riding a bike.
While not nearly as interesting as a tactile display, one thing I’d like to see Apple do sooner rather than later is to use its face and eye tracking along with the onboard gyros to automatically adjust the hit zones on the on-screen keyboard keys to help prevent or reduce on-screen keyboard entry errors caused by parallax. I find myself hitting the wrong keys quite often when I’m not holding my phone exactly parallel and centered on my face. This isn’t a trivial problem to solve but it seems like a resolvable one that lends itself to the sensors and processing that Apple already has in its latest iPhones. It’s somewhat similar to the changes being proposed for FaceTime to auto compensate the appearance of your eyes to make it appear that you’re looking straight at the person you’re FaceTiming with.
Apple wouldn't implement it till battery usage was acceptable. They are not in the habit of rushing out features at the cost of battery life.