Apple patents shape changing device display for tactile interactions
The U.S. Patent and Trademark Office on Tuesday granted Apple a patent for a portable device with a display capable of dynamically changing its shape, potentially opening the door to truly tactile user interaction.
Apple's U.S. Patent No. 9,600,070 for a "User interface having changeable topography" covers an innovative hardware solution in which a consumer device dynamically changes its shape to better serve user needs.
In particular, the invention notes a user interface, like the display on an iPhone, can provide a variable tactile feel by incorporating sub-display electromechanical elements that push or pull against a deformable surface. With iPhone, for example, this surface would be a flexible screen capable of adjusting digital assets to correspond with the raised and lowered topography.
In some embodiments, the operating state of a device is dependent on a particular user interface. For example, screen sections might raise or lower to outline keys in a calculator app. A mapping app might raise those portions of a display to highlight potential routes -- or indeed an accurate tactile representation of topographical data -- while a music app could manipulate a screen to form arrow symbols denoting playback and shuttle controls.
The proposed technology gives virtual assets form, thereby enhancing function. Beyond the added appeal of a tactile experience, shape changing displays would be a boon for users with visual impairments. Depending on the implementation and level of topographic definition, such systems might also be deployed for controlling a device without looking at a screen, perhaps while it's in a user's pocket.
Achieving a shape changing display would require the installation of one or more configurable nodes within a screen's surface. Arranged in a matrix, these nodes act upon a stretchable substrate, or display, to define shapes corresponding with a user interface, generate patterns and textures, highlight UI elements, provide tactile feedback and more.
The document details a variety of shape changing actuators, from electromechanical pistons to specialized materials like nitinol or piezocrystals. While piston-type actuators are likely easier to manufacture, shape changing material systems that respond to heat, electric, magnetic or other stimuli are smaller and allow for fine installations.
Thanks to their unique properties, shape changeable actuators are able to deform when stimulated. Depending on the material, orientation and stimulus, these components can stretch, raise, elongate, extend or otherwise deform from a natural "at rest" state to push against a flexible display surface. Alternatively, a separate stimulus will cause the same actuator to shorten, retract, lower, retreat, thus pulling away from the display.
In yet another embodiment, actuators are crafted to tilt, rotate or shift position when stimulated, thereby manipulating a screen's topography.
Independent of the underlying technology at play, each embodiment calls for actuators to work together to define specific shapes above or below a device surface. Acceptable deformations include changes in height, width, length, orientation, configuration, layout, texture, pattern, three-dimensional form and other adjustments.
Alternate embodiments allow for the insertion of shape change devices in other hardware solutions like a MacBook's touchpad or (dating the patent) an iPod's touch interface. In addition to virtual buttons and UI-related tasks, the actuator system is also described as being useful for creating a 3D effect for displayed images.
It should be noted that Apple's invention as detailed in today's grant appears to deal with larger surface areas suitable for UI buttons and app icons. It is possible that more advanced systems based on the above described tech might one day achieve pixel-level resolutions for fine circuitous lines. Variable height actuators might also be used, allowing Apple to create dynamically undulating surfaces similar to a shallow 3D printed object.
Considering the patent's age -- it was first submitted in 2008 -- it is highly unlikely that Apple will bring its shape changing technology to market anytime soon. The concept, however, offers insight into the company's ongoing research into alternative user interfaces, a journey that started with the first Mac and continues to this day.
Apple's shape changing display patent was first filed for in March 2009 and credits Shuvo Chatterjee and Quin C. Hoellwarth as its inventors.
Apple's U.S. Patent No. 9,600,070 for a "User interface having changeable topography" covers an innovative hardware solution in which a consumer device dynamically changes its shape to better serve user needs.
In particular, the invention notes a user interface, like the display on an iPhone, can provide a variable tactile feel by incorporating sub-display electromechanical elements that push or pull against a deformable surface. With iPhone, for example, this surface would be a flexible screen capable of adjusting digital assets to correspond with the raised and lowered topography.
In some embodiments, the operating state of a device is dependent on a particular user interface. For example, screen sections might raise or lower to outline keys in a calculator app. A mapping app might raise those portions of a display to highlight potential routes -- or indeed an accurate tactile representation of topographical data -- while a music app could manipulate a screen to form arrow symbols denoting playback and shuttle controls.
The proposed technology gives virtual assets form, thereby enhancing function. Beyond the added appeal of a tactile experience, shape changing displays would be a boon for users with visual impairments. Depending on the implementation and level of topographic definition, such systems might also be deployed for controlling a device without looking at a screen, perhaps while it's in a user's pocket.
Achieving a shape changing display would require the installation of one or more configurable nodes within a screen's surface. Arranged in a matrix, these nodes act upon a stretchable substrate, or display, to define shapes corresponding with a user interface, generate patterns and textures, highlight UI elements, provide tactile feedback and more.
The document details a variety of shape changing actuators, from electromechanical pistons to specialized materials like nitinol or piezocrystals. While piston-type actuators are likely easier to manufacture, shape changing material systems that respond to heat, electric, magnetic or other stimuli are smaller and allow for fine installations.
Thanks to their unique properties, shape changeable actuators are able to deform when stimulated. Depending on the material, orientation and stimulus, these components can stretch, raise, elongate, extend or otherwise deform from a natural "at rest" state to push against a flexible display surface. Alternatively, a separate stimulus will cause the same actuator to shorten, retract, lower, retreat, thus pulling away from the display.
In yet another embodiment, actuators are crafted to tilt, rotate or shift position when stimulated, thereby manipulating a screen's topography.
Independent of the underlying technology at play, each embodiment calls for actuators to work together to define specific shapes above or below a device surface. Acceptable deformations include changes in height, width, length, orientation, configuration, layout, texture, pattern, three-dimensional form and other adjustments.
Alternate embodiments allow for the insertion of shape change devices in other hardware solutions like a MacBook's touchpad or (dating the patent) an iPod's touch interface. In addition to virtual buttons and UI-related tasks, the actuator system is also described as being useful for creating a 3D effect for displayed images.
It should be noted that Apple's invention as detailed in today's grant appears to deal with larger surface areas suitable for UI buttons and app icons. It is possible that more advanced systems based on the above described tech might one day achieve pixel-level resolutions for fine circuitous lines. Variable height actuators might also be used, allowing Apple to create dynamically undulating surfaces similar to a shallow 3D printed object.
Considering the patent's age -- it was first submitted in 2008 -- it is highly unlikely that Apple will bring its shape changing technology to market anytime soon. The concept, however, offers insight into the company's ongoing research into alternative user interfaces, a journey that started with the first Mac and continues to this day.
Apple's shape changing display patent was first filed for in March 2009 and credits Shuvo Chatterjee and Quin C. Hoellwarth as its inventors.
Comments
Why did it take 8 or 9 years to grant this patent, I wonder.
I expect this will be about as relevant as Liquid Metal in Apple's products, but if they figure out how to do this durably and cost-effectively it could be amazing.
I've been saying for ages that Apple are moving towards "virtual" control surfaces that uses haptic feedback to make it feel like there are physical controls (like, say a keyboard) on a flat surface (like a phone screen). This is a different approach that would allow them to do the same thing, possibly more reliably. This will finally allow for fully customised controls for apps, so if you're typing, there's a keyboard with seemingly physical keys, if you're mixing music or video, there are sliders that feel like physical controls.
If they (or any other company) were to implement this, it could be the biggest revolution in Human Interface Design since the mouse. (Or it could be a complete failure that winds up on that list of failed products everyone likes to trot out every year or so.)