Apple's patented GUI compensates for iPhone motion, minimizes errant touches

Posted:
in General Discussion edited January 2014
Apple on Tuesday was awarded an interesting patent describing a graphical user interface that offers a solution to inadvertent input errors users may see when interacting with a touchscreen while moving.

Motion


The U.S. Patent and Trademark Office granted Apple U.S. Patent No. 8,631,358 for a "Variable device graphical user interface," which outlines a system where user movement is compensated for via software adjustments.

While not a particularly in-depth patent, the invention looks to solve a common problem many iPhone users face when using the device while moving. Due to its relatively small screen and touch-only interface, Apple's smartphone could be difficult to use when a user is in motion.

For example, the bobbing motion caused by walking could lead to errant screen touches. Running or other activities may exaggerate the problem to a point where interacting with an iPhone is next to impossible.

To compensate, one or more sensors can be used to detect device motion or patterns of motion, which will trigger dynamic UI changes like enlarging a virtual button's touch area or shifting visual assets around the screen. Apple mentions accelerometers, gyroscopes and more as adequate on-board sensors, most of which are already installed on current iPhone models.

Changes in metrics such as device acceleration and orientation can be used to interpret the motion of a device. More importantly, patterns can be detected and matched to a database of predetermined criteria. For example, if the device senses bobbing, it can be determined that the user is walking. An oscillating motion may denote running, while minute bounces could signal that a user is riding in a car.

Motion


In each of the above scenarios, the detected pattern of motion is mapped to one or more GUI adjustments. In some embodiments, user interface changes are based on type and magnitude of motion. Depending on what criteria the motion meets, a corresponding response will be applied to the GUI.

According to some implementations, the system will resize UI elements and touch areas to compensate for a certain motion. For example, the rows of a contact list may be enlarged by increasing the height of each entry. The touch area is also resized, affording the user a bigger target to hit while moving.

Another embodiment calls for graphical assets to be dynamically mapped to different positions on a screen in relation to device motion. In this case, a row of contacts may be shifted vertically or horizontally in a direction opposite to the detected motion, thereby simulating a stabilized display. Touch sensitivity can also be customized to help with accuracy.

In yet another embodiment, the UI can be skewed to compensate for angle of hold. Further, a "fisheye" effect can be mapped to certain elements, directing focus to more important assets while minimizing the size of others.

Finally, the system is able to "learn" particular characteristics of device motion and how a user's touch accuracy relates to said movement. This data can be stored and later retrieved to predict where the user will touch during a particular pattern of motion. The GUI can be remapped based on these predictions.

Motion


As with most Apple patents, this property may never be deployed in a consumer device. It is interesting to note, however, that the company is looking to expand the iPhone's potential as an activity monitor, a fact evidenced by the iPhone 5s' M7 motion coprocessor.

Apple's variable device GUI patent was first filed for in 2007 and credits John O. Louch as its inventor.

Comments

  • Reply 1 of 8
    Sounds like stuff one would want on an "iBand" or "iWatch" to me.
  • Reply 2 of 8
    Of course actual real-world use case scenarios would suggest otherwise:
    https://24.media.tumblr.com/tumblr_mbs77cDMT81qzdn5qo1_500.gif
  • Reply 3 of 8
    Originally Posted by nicknormal View Post

    Of course actual real-world use case scenarios would suggest otherwise:

    https://24.media.tumblr.com/tumblr_mbs77cDMT81qzdn5qo1_500.gif

     

    Yeah, I’m sure that image is proof of anything whatsoever.

  • Reply 4 of 8
    retrogustoretrogusto Posts: 1,109member
    Oh man, I had this idea years ago, but I thought about it mostly to stabilize text when you're reading in a bouncing vehicle. I hope they do it though--it would be fun to see.
  • Reply 5 of 8
    retrogusto wrote: »
    Oh man, I had this idea years ago, but I thought about it mostly to stabilize text when you're reading in a bouncing vehicle. I hope they do it though--it would be fun to see.

    That's not a bad idea. Develop it and make a reader app.
  • Reply 6 of 8
    gatorguygatorguy Posts: 24,176member
    retrogusto wrote: »
    Oh man, I had this idea years ago, but I thought about it mostly to stabilize text when you're reading in a bouncing vehicle. I hope they do it though--it would be fun to see.

    Qualcomm must have been listening in on your thoughts.
    https://www.google.com/patents/WO2012135837A1?cl=en&dq=stabilize+text+on+a+mobile+device+when+moving&hl=en&sa=X&ei=KmnVUsyJKa7hsAT7-4DACg&ved=0CDcQ6AEwAA

    Abstract:
    "Apparatus and methods for stabilizing an image displayed on a mobile device are presented. In some circumstances, the display on a mobile device moves with respect to the ground, which may result from vibrations in a moving vehicle. Embodiments use a sequence of images to determine a displacement of a stationary feature relative to the mobile device. This determined displacement may be a predicted displacement that estimates a displacement that probably will occur in the immediate future. Next, the displacement may be projected to a flat plane of the display. Finally, information presented on the display is moved in an opposite direction of the determined or projected displacement, thus compensating for the displacement and having the effect of stabilizing the image on the mobile device, such that the displayed image appears to be still in space even though the mobile device is vibrating or shaking relative to the Earth."
  • Reply 7 of 8
    dacloodacloo Posts: 890member
    Great for Whatsapping your wife while having sex.
  • Reply 8 of 8
    Quote:

    Originally Posted by Retrogusto View Post



    Oh man, I had this idea years ago, but I thought about it mostly to stabilize text when you're reading in a bouncing vehicle. I hope they do it though--it would be fun to see.

    I'm pretty sure that an iOS web browser was released a while ago that had this feature, though it didn't work very well.

Sign In or Register to comment.