Apple's patented physics-based iPad GUI translates file size into mass, supports intuitive gestures

Posted:
in General Discussion edited September 2014
In its quest to humanize computers and digital user interfaces, Apple has invented a physics-based GUI that assigns mass to files and folders based on their data size, allowing for novel, yet intuitive, user interactions.


Source: USPTO


The U.S. Patent and Trademark Office on Monday granted Apple U.S. Patent No. 8,839,150 for "Graphical objects that respond to touch or motion input," which describes a hybrid mobile operating system and corresponding graphical user interface that responds to touches based on implied physical forces like mass, gravity, friction or drag.

As described by Apple, the technology can be applied to a tablet's UI, though the system described is more full-featured than iOS in that it handles files and folders. For example, instead of arranging apps in rigid grids, icons can be placed and selected in free space, a method familiar to OS X users. Unlike Apple's current desktop operating system, however, the patent relies on touch gestures as a main form of input.

At the heart of Apple's patent is organic user interaction facilitated through graphical animations driven by sensors, touch gestures and intelligent design. The document offers a first example that involves selecting a batch of folders.

In a usual desktop operating environment, selecting one or more UI elements implies using a mouse to create a highlighted box around one or more icons representative of folders, files or apps. With Apple's invention, a user draws a circle around the same two-dimensional assets, but when they lift their finger, the elements are transformed into graphical objects.




While the representation can be any number of animations or other graphics, the document describes a floating bubble or sphere that appears detached from the background UI. In this way, a user can quickly distinguish what folders are active, or selected.

In addition, by leveraging onboard sensors like accelerometers and gyroscopes, the system can add physical properties like mass to the floating ball, which correspond to device motion. In one embodiment, the ball can bounce off screen borders or move in accordance to simulated drag and friction.

Further, the mass of a graphical asset can correspond to its data size. For example, a large file or folder will create a graphical object of with proportional mass. The larger the file, the larger the mass, which in turn dictates how the graphical asset responds to user interaction. In some embodiments, a large file will move more slowly as its mass is larger than other files in the UI.

Another interesting concept tied to physical file attributes is how these graphically represented files respond to actions like transferring, compressing and sorting, among others.

In the illustration at the top of this article, a user makes a "pouring" gesture with one device onto another device connected via Bluetooth, Wi-Fi or similar wireless technology. Again, based on file size, the assets obey the rules of gravity and "fall" into an opening on one device, then show up on the recipient machine's display as if being poured.

As an example for compression and file archiving, users can make a pinching gesture around a selected asset, which shrinks the graphical representation once the operation is completed. Alternative embodiments include tags, text or file replacement confirming the compression action and displaying the new file size.

The document again points to a graphical representation of data as mass, this time applying it to file sorting. For example, users can shake their device to create a "snow globe" effect that results in large data files sinking to the bottom of a UI, while smaller files remain at the top. Similar sized files and folders are arranged in bands.




While the invention would be an interesting addition to an Apple product, the property appears to be aimed at a GUI that lies somewhere between a touch-capable version of OS X and a more feature-laden iOS. Since Apple seems reticent to join the two operating systems, the patented technology will likely remain a curiosity for the foreseeable future.

Apple's graphical objects patent was first filed for in February 2010 and credits Nicholas V. King, Brett Bilbrey and Todd Benjamin as its inventors.
«13

Comments

  • Reply 1 of 44
    And the hits just keep on coming...
    Go Apple!
  • Reply 2 of 44
    UI for the iPad Pro :D
  • Reply 3 of 44
    UI for the HoloPad.
  • Reply 4 of 44
    clemynxclemynx Posts: 1,552member
    It's the first time I see an OSX bar on a tablet patent.
  • Reply 5 of 44
    MacProMacPro Posts: 19,843member
    Fascinating!
  • Reply 6 of 44
    I hate to say it but this patent was merely a knee-jerk response to Google's 2010 acquisition of BumpTop.
    http://mashable.com/2010/05/02/google-bumptop/
  • Reply 7 of 44
    I hate to say it but this patent was merely a knee-jerk response to Google's 2010 acquisition of BumpTop.
    http://mashable.com/2010/05/02/google-bumptop/

    Calling a patent a "knee-jerk response" is ridiculous.
  • Reply 8 of 44
    ecatsecats Posts: 274member
    @Johnny: not really as apple already has a patent very similar to bump top. (Involving the use of eye tracking technology to present a depth-based UI environment.) That same line of patents also describes stacks.

    Unlike bump top this patent covers interactions that give file metadata real-world attributes. The simple example above is that larger files can be heavier, which makes a useful sorting device.
  • Reply 9 of 44
    asciiascii Posts: 5,936member
    That's the kind of GUI design I like, actually thinking about how things can be made more intuitive. Instead of just making everything transparent and then going on stage an WWDC to tell the crowd how incredible it is.
  • Reply 10 of 44
    Quote:
    Originally Posted by Johnny Mozzarella View Post



    I hate to say it but this patent was merely a knee-jerk response to Google's 2010 acquisition of BumpTop.

    http://mashable.com/2010/05/02/google-bumptop/



    The patent filing date is three months before Google acquired BumpTop.

  • Reply 11 of 44
    MacProMacPro Posts: 19,843member
    shadow700 wrote: »

    The patent filing date is three months before Google acquired BumpTop.

    That was a good find. Love it when facts mess with those Google / Android support posts.
  • Reply 12 of 44

    BumpTop's public demonstration at TED was 2007.

    http://www.ted.com/talks/anand_agarawala_demos_his_bumptop_desktop

  • Reply 13 of 44
    calicali Posts: 3,494member
    So when will this come to android??

    (seeing how U.S. patents have become useless)
  • Reply 14 of 44
    gatorguygatorguy Posts: 24,628member
    cali wrote: »
    So when will this come to android??

    (seeing how U.S. patents have become useless)
    Hard to say. Google has been filing several patents that relate to it.
    https://www.google.com/search?tbo=p&tbm=pts&hl=en&q=ininventor:Anand+ininventor:Agarawala&num=10&qscrl=1&gws_rd=ssl
  • Reply 15 of 44
    jd_in_sbjd_in_sb Posts: 1,600member
    Samsung will immediately copy this and then Judge Koh will invalidate the patent.
  • Reply 16 of 44
    I wonder if this GUI will be used in future, new devices that Apple does not currently market but has in development.

    It would seem that a multi-core device using the A-8 or similar chip, larger touch screen, providing a more-desktop feel while maintaining portability could be a possibility with such a GUI.
  • Reply 17 of 44
    shadow700 wrote: »

    The patent filing date is three months before Google acquired BumpTop.
    Ah come on! Those guys threw this patent together in an afternoon after reading an article on Ars, Wired, or wherever.

    This is an example of something that is far "too obvious" and will not stand the 101 ruling, "Doing It On A Computer" test. I do this with my hands every day with nuts and screws on my messy workbench.

    /s
  • Reply 18 of 44
    If anyone hasn't taken a look at the BumpTop pres... they should. Quite funny, interesting, and does "appear' to look like a basis for this patent.

    Also... I bet Scott Forstall had an orgasm when he first saw it...it's a skeuomorphic fetishist's wet dream.... :D

    [VIDEO]
  • Reply 19 of 44
    Quote:

    Originally Posted by AppleInsider View Post

    Apple's graphical objects patent was first filed for in February 2010 and credits Nicholas V. King, Brett Bilbrey and Todd Benjamin as its inventors.

     

    Quote:

    Originally Posted by shadow700 View Post

    The patent filing date is three months before Google acquired BumpTop.


    BumpTop released a beta for Mac OS X in January of 2010 and was actively trying to be acquired.

    I'm sure the folks at BumpTop were trying to play Apple off Google to get the biggest payday.

    Ultimately, Google acquired them and Apple opted to try to patent some similar concepts.

     

    Quote:

    Originally Posted by digitalclips View Post

    That was a good find. Love it when facts mess with those Google / Android support posts.

    The fact is Android is a giant turd but that doesn't mean every post critical of Apple is from an Android apologist.

  • Reply 20 of 44
    Quote:

    Originally Posted by macaholic_1948 View Post



    I wonder if this GUI will be used in future, new devices that Apple does not currently market but has in development.

    No.  Apple under Jony Ive is clearly going in the opposite direction, away from skeuomorphic 3D UIs.

Sign In or Register to comment.