Future iPhone screen could simulate surface textures and temperatures for AR

Posted:
in iPhone
Touching a future iPhone or iPad display could provide a sensation of texture, with Apple coming up with a way to alter how a touch-enabled surface can be controlled to simulate the feel of different types of materials.

iPhone 11 Pro and iPhone 11 Pro Max
iPhone 11 Pro and iPhone 11 Pro Max


Touchscreens are a common interface for the vast majority of mobile devices, with the visual element and haptic feedback providing users with a considerable amount of stimulus in relation to the content they are interacting with. However, no matter how it appears on the screen, users are still touching a glass or plastic surface, and won't get a full physical feedback experience.

Regardless of whether it looks like you're touching sandpaper or a kitten's fur on an iPad display, at the end of the day you're just going to feel glass.

In a patent granted by the U.S. Patent and Trademark Office on Tuesday titled "Touch surface for simulating materials," Apple suggests ways that a surface could be altered to give a variety of sensations that somewhat mimic that of different textures.

The patent suggests the use of actuators, temperature control devices, and a main control unit that manages the use of both types of feedback generators over a specific portion of a touch surface. In the case of the actuators, the control unit will use it to vibrate the display to give a sensation of texture.

The user of a Peltier device or another temperature control component can be used to simulate the thermal conductivity of a material. For example, a wooden table may "feel" warm, while a metal surface could feel cold to the touch or warm up over time.

Depending on the implementation, the system could be set up to function in only one region of a touch surface, multiple areas, or across the entire surface. In instances where multiple regions are in play, each region could be made to simulate different materials at the same time, so that one hand could feel something soft and warm while the other could sense cold and roughness.

Selecting specific areas of the display to apply the technology to could be due to the difficulty of applying multiple sets of components all across a display. By limiting it to designated regions, such as opposite edges of a screen that are usually touched for gaming in landscape orientation, this would make such a system be more practical from a manufacturing standpoint.

Apple's patent illustration showing the use of actuators, temperature controls, and a diamond material to simulate materials on a touch surface
Apple's patent illustration showing the use of actuators, temperature controls, and a diamond material to simulate materials on a touch surface


Apple suggests the touch surface could use a layer of a diamond material, possibly using a chemical vapor deposit method, which will provide "extremely high thermal conductivity" and "extreme mechanical hardness" to the display while maintaining optical transparency.

While the patent has more immediate applications for touchscreen-enabled devices, Apple does highlight its usage on the touchpad area of a MacBook in one image. This may make sense as it would be simpler to implement on a touch-enabled surface that doesn't include a display element.

Apple files numerous patent applications on a weekly basis, but while the existence of a patent suggests areas of interest for the company's research efforts, it doesn't guarantee the concept will make an appearance in a future product or service.

In the case of texture, it is arguable that Apple could implement part of the patent in devices right now. Apple's Taptic Engine in its iPhones provides a variety of different buzzing patterns and levels, which could easily be used to buzz in different ways to simulate a surface material.

Of course, temperature is not something the iPhone is capable of simulating right now. Given the need to keep devices as safe as possible for consumers, it is probable that Apple would want to ensure any thermal-altering system doesn't have the opportunity to cause injury, such as burning a user's finger by simulating a hot frying pan.

Apple has previously looked into the possibility of providing a more tactile experience to its users, such as in the March 2017 patent for "User Interface having changeable topography," where the display changes shape to have raised sections. For example, this could include raised keys on a keyboard or on a calculator.

An earlier patent filing from July 2015 involved a stylus with onboard haptic feedback components to vibrate the implement, based on information collected about the material the stylus is "touching" on the display.
macplusplus

Comments

  • Reply 1 of 16
    That would burn through your battery like nobody’s business, and shorten its life.  Sounds interesting, but not on a phone.
  • Reply 2 of 16
    Skeuomorphism to the extreme ! Then why did you remove leather backgrounds?

    Some degree of drag applied to the Pencil would provide better control of the strokes, as already implemented on the iPad Pros. Or an accurate implementation of the Touch ID under the display. Besides those I don’t think anyone needs to feel any kind of wetness on the display.

    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    edited October 2019
  • Reply 3 of 16
    StrangeDaysStrangeDays Posts: 12,963member
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    philboogietmayhodarmike1fastasleep
  • Reply 4 of 16
    tjwolftjwolf Posts: 424member
    Did you just throw the letters AR into the title as click bait?  No idea how the actual subject of the article -imitating different materials using tech - has anything to do with AR.
  • Reply 5 of 16
    philboogiephilboogie Posts: 7,675member
    tjwolf said:
    Did you just throw the letters AR into the title as click bait?  No idea how the actual subject of the article -imitating different materials using tech - has anything to do with AR.
    "Imagination is more important than knowledge"
  • Reply 6 of 16
    This would be awesome with a CarPlay screen.
  • Reply 7 of 16
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    I am under the impression that a few core teams are overloaded while others are busy with filing their epiphanies.
    edited October 2019
  • Reply 8 of 16
    Rayz2016Rayz2016 Posts: 6,957member
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    I am under the impression that a few core teams are overloaded while others are busy with filing their epiphanies.
    Right, so you think that every team at Apple has exactly the same skill set. Got it. 
    tmaymike1fastasleep
  • Reply 9 of 16
    Rayz2016 said:
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    I am under the impression that a few core teams are overloaded while others are busy with filing their epiphanies.
    Right, so you think that every team at Apple has exactly the same skill set. Got it. 
    This is not a matter of skill set or training or whatever. This is a matter of management, adequate distribution of the workforce etc.

    If this is a matter of "research" then the "research" may be targeted more on current products than imaginative products.
    edited October 2019
  • Reply 10 of 16
    mike1mike1 Posts: 3,333member
    Rayz2016 said:
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    I am under the impression that a few core teams are overloaded while others are busy with filing their epiphanies.
    Right, so you think that every team at Apple has exactly the same skill set. Got it. 
    This is not a matter of skill set or training or whatever. This is a matter of management, adequate distribution of the workforce etc.

    If this is a matter of "research" then the "research" may be targeted more on current products than imaginative products.
      Didn't realize that managing a trillion dollar company and tens of thousands of employees is so simple.
    fastasleep
  • Reply 11 of 16
    This patent reminds me of the idea (Sci-fi) of a person running on a treadmill like device, where you can feel the terrain (rocks, sand, snow etc.) under your feet.  The user wears an AR headset so it’s simulating running in the Australian outback or Alps (for example).
  • Reply 12 of 16
    AppleExposedAppleExposed Posts: 1,805unconfirmed, member
    tjwolf said:
    Did you just throw the letters AR into the title as click bait?  No idea how the actual subject of the article -imitating different materials using tech - has anything to do with AR.
    "Imagination is more important than knowledge"

    Wow never heard this quote. It defines me to the core. When I read patents I can think of tens or hundreds of users while others simply say "this won't work".


    Skeuomorphism to the extreme ! Then why did you remove leather backgrounds?

    Some degree of drag applied to the Pencil would provide better control of the strokes, as already implemented on the iPad Pros. Or an accurate implementation of the Touch ID under the display. Besides those I don’t think anyone needs to feel any kind of wetness on the display.

    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...

    Ummm...WHAT? This can go beyond "feeling" app icons. This can help with everything from typing to gaming to medical.


    dkhaley said:
    This would be awesome with a CarPlay screen.

    Awesome idea! May even be implemented on Watch/iPhone if it detects you're driving/riding a bike.
  • Reply 13 of 16
    dewmedewme Posts: 5,551member
    The first thing that came to mind of when I saw “tactile display” was using MEMS technology. Sure enough, someone already wrote a white paper about this very topic in 2015.  See https://www.sciencedirect.com/science/article/pii/S0141938214000870

    While not nearly as interesting as a tactile display, one thing I’d like to see Apple do sooner rather than later  is to use its face and eye tracking along with the onboard gyros to automatically adjust the hit zones on the on-screen keyboard keys to help prevent or reduce on-screen keyboard entry errors caused by parallax. I find myself hitting the wrong keys quite often when I’m not holding my phone exactly parallel and centered on my face. This isn’t a trivial problem to solve but it seems like a resolvable one that lends itself to the sensors and processing that Apple already has in its latest iPhones. It’s somewhat similar to the changes being proposed for FaceTime to auto compensate the appearance of your eyes to make it appear that you’re looking straight at the person you’re FaceTiming with. 
  • Reply 14 of 16
    slurpyslurpy Posts: 5,385member
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    I am under the impression that a few core teams are overloaded while others are busy with filing their epiphanies.
    Maybe you should apply to consult about their team management? Cause obviously Apple doesn't know what the fuck it's doing, just a massive fucking mess with failure after failure. Thank God for people like you, who obviously know better, and have an intimate understanding of Apple's internal team structures and how they should be improved. 
    fastasleepbadmonk
  • Reply 15 of 16
    slurpy said:
    Apparently some “creative” engineers have no way to be heard by the management other than filing a patent... And HomePods get bricked while they are so busy with their epiphanies...
    You're under the impression the researchers patenting new technology within Apple are the same software engineers responsible for the HomePod OS bug? That's absurd. 
    I am under the impression that a few core teams are overloaded while others are busy with filing their epiphanies.
    Maybe you should apply to consult about their team management? Cause obviously Apple doesn't know what the fuck it's doing, just a massive fucking mess with failure after failure. Thank God for people like you, who obviously know better, and have an intimate understanding of Apple's internal team structures and how they should be improved. 
    Calm down. Fortunately Apple doesn’t need supporters like you to compete and succeed. Neither it will get doomed after criticisms like mine.
  • Reply 16 of 16
    That would burn through your battery like nobody’s business, and shorten its life.  Sounds interesting, but not on a phone.

    Apple wouldn't implement it till battery usage was acceptable. They are not in the habit of rushing out features at the cost of battery life.
Sign In or Register to comment.