Apple working on 3D Mac OS X user interface (images)

Posted:
in macOS edited January 2014
A series of Apple patent filings published this week reveal the Mac maker has spent a considerable amount of time outlining a new multi-dimensional interface for Mac OS X that would make better use of screen real estate by increasing the number of virtual surfaces capable of housing application and interface elements.



The most extensive of those filings is labeled "Multi-Dimensional Desktop" and was submitted to the United States Patent and Trademark Office back in June of last year, around the same time the company took the wraps off the feature set for its upcoming release of Mac OS X 10.6 Snow Leopard.



None of the latest interface enhancements described in the June patent are present in current pre-release builds of Snow Leopard, however, suggesting they'll be candidates for inclusion in versions of Mac OS X that would surface in years to come.



Generally speaking, the filings depict a 3D interface by which side walls, a top, and a floor all protrude from a back surface that resembles today's two-dimensional Mac OS X desktop. A few examples also suggest a radical departure from traditional interface design by which the Mac OS X menubar would be removed from the top of the screen and thrown into a stack or floating element.



An illustrated description of each of Apple's proposed design changes can be found below:







In the example above, Apple portrays an exemplary multidimensional desktop environment that includes a back surface like today's traditional two-dimensional desktop environment, which includes the Mac OS X menu bar and desktop surface. Protruding from this surface would be one or more side surfaces capable of housing object receptacles, including a "floor" that would contain the Mac OS X Dock and icon stacks positioned in the background. Other side surfaces would intersect with the floor, forming a three-dimensional box, though Apple says a "top" surface may not always be necessary.







Most of the diagrams in the filing show the multi-dimentional Finder with side surfaces, or "walls," as shown above. As is the case with the floor surface, object receptacles like the Dock can be placed along the walls at certain perceived depths. The example above shows the Dock displaced behind the 3D icon stacks sitting on the floor, but could also be arranged closer to the viewing surface so that it displays in front of the icon stacks.







The 3D icon stacks in the multi-dimensional Finder would fan outwards as they do in today's version of the Mac OS X but would not necessarily be tied to a linear path. They could instead fan in a three-dimensional arch towards a central region of the display, producing the perception of depth.







Alternatively, a short stack of icons could instead expand into a two dimensional matrix that would also display behind other elements such as the Dock. Apple adds that the stack items could corresponding to the categories of online buddies and music, which could be controlled contextually. For example, a stack item can automatically appear when the Finder receives a notification that an event associated with another user that is designated as an "online buddy" has occurred, such as an "online buddy" logging onto a network.







Apple in another example goes on to explain that the floor can be vast and sprawling, with only a portion of the surface displayed at any given time. "For example, upon a selection of the surface, [...] the surface can be scrolled in the directions indicated by the arrows," the company said. "The floor surface can include a scroll ingress and a scroll egress in which a scroll direction transitions from the scroll ingress to the scroll egress."



More specifically, the intersections of the two side walls may define a scroll ingress and a scroll egress for a left-to-right scroll direction, or the left edge and the right edge of the reflection region may define a scroll ingress and a scroll egress for a left-to-right scroll direction.



During a scroll, the stack items can be displaced from the floor surface by fading from view or by a failing effect. Meanwhile, application windows can be dragged to one of the side walls, where they'd appear in thumbnail form, or as stacks of windows. The windows could be restored to their traditional form by dragging them back to the back surface (Desktop) or through a simple double-click.



The side walls could also be colored to emphasize the back surface and the reflection regions of the interface. For example, the side surfaces could be black in color, or respective patterns, colors, or images can be rendered on each side surface. Other differentiation schemes including color schemes and image schemes can also be applied, Apple said.



A lighting aspect could also be employed to generate an illumination effect from window thumbnails or widgets placed on the side walls. For example, the illumination effect could comprise a simulated sunbeam emanating from one of the window thumbnails or widgets on the left wall. The illumination effect could change according to local environmental states. For example, it could track across the surfaces according to a local time, or its intensity could be modulated by local weather conditions that are received over the network -- high intensity for sunny days, low intensity for overcast days and during the early evening. Each element on the side walls could also cast its own shadow on subsequent interface elements to emphasize a depth aspect and an overall user interface relationship.







The above example shows how stack items on the floor can move in response to the addition of a new stack item. As the new stack item drops onto the surface, the stack items already on the surface can move in response to the displacement induced by the new stack item, shifting those stack items forwards, backwards, to the right or left.



"In one implementation, a maximum number of stack items can be displayed on the surface. If the addition of a new stack item causes the number of displayed stack items to be exceeded, then a stack item nearest a surface intersection can be displaced from the surface," Apple explained. "For example, if the maximum number of stack items to be displayed is four, then [one of the stacks] can continue to move to the edge of the surface, where [it would be] displaced, e.g., fades from view, atomizes, etc."







Another example shows a three-dimensional Finder that lacks a traditional menubar on the back surface. In this case, a stack item could be utilized to access menu items traditionally found at the top of the Mac OS X dektop. Alternatively, selection of the menu stack item could throw a traditional menubar back into place at the top of the screen, or display menu items on the back surface, or one of the walls.



The same diagram shown above also depicts a running application by which one or more application control elements -- such as palettes -- can be displayed on one or more side walls. For example, a tool bar or layers palette can be displayed on the side walls when Photoshop is running.







The above diagram expands on the concept of having application control elements reside on the walls of a multi-dimensional Finder by portraying those elements as 3D icons, which would expand their functionality. For example, each three-dimensional function icon could include a separate function command on each surface, and each three-dimensional function icon could be be rotated, positioned, and manipulated through the use of an input device, such as a mouse.



Additional 3D icons could be added to the surface by use of a menu, while removing one of the 3D icons can cause the remaining icons to "fall" in a downward direction on the surface wall. Apple said 3D icons can also be used to transition between work spaces, or to transition between a 2D Finder and a 3D Finder.







Windows in a 3D Mac OS X interface can also be dragged or displaced across one or more surfaces. For example, windows can transition out of view by sliding across the back surface and side walls. Side surfaces could also be used to implement a deletion characteristic.



"For example, the last access time for icons and other system object representations can be monitored. If the last access time for an icon or other system object representation exceeds a first threshold, the icon or other system object representation can be automatically transitioned to the surface implementing the deletion characteristic," Apple said. "Additionally, if the last access time for the icon or other system object representation located on the surface exceeds a second threshold, the icon or other system object representation can be automatically deleted from view."







One final multidimensional desktop environment described in the filing includes an arcuate back surface with side surfaces defined by arcuate regions having curvature intersections.



"Other multidimensional desktop environment geometries can also be used. For example, in one implementation, the multidimensional desktop environment can conform to a tetrahedron-shaped environment in which a front surface of the tetrahedron defines a viewing surface, and the remaining three surfaces define a left surface, a bottom surface, and a side surface," Apple said. "In another implementation, the multidimensional desktop environment can conform to a triangular environment, in which one axis of the triangle defines the viewing surface and the remaining two sides of the triangle define a left surface and a right surface."



The 54-page filing is credited to Apple employees Imran Chaudhri, John Louch, Christopher Hynes, Timothy Bumgarner, and Eric Peyton.
«13456

Comments

  • Reply 1 of 104
    successsuccess Posts: 1,039member
    obviously reversed engineered about 20 years ago from alien technology.
  • Reply 2 of 104
    Too bad the implementation seen in the patent diagrams would be utter garbage.



    Stick with a 2D UI for 2D displays, Apple. When the time is right, a 3D interface for 3D displays will make sense. And hopefully, it won't be that pseudo-3D (read 2.5D) crap seen in the diagrams.
  • Reply 3 of 104
    NO! PLEASE NO! This is awful!
  • Reply 4 of 104
    irelandireland Posts: 17,616member
    Rest assured whatever they do will be more subtle that this patent suggests.
  • Reply 5 of 104
    wow. Talk about cluttering up a desktop.
  • Reply 6 of 104
    sandausandau Posts: 1,230member
    Time Machine turned out pretty good for a 3d-ish environment. I'd be interested in more apps like it including the OS.
  • Reply 7 of 104
    Hey, ever since 3d graphics came out all those years ago, people were saying the desktop of the future would be 3d based. This is just Apple exploring a possible way of using 3D in a productive way. I doubt we'll see it anytime soon.



    The Time machine view seems very similar to this patent filing as well. Maybe some of this research was used for that.
  • Reply 8 of 104
    That circle diagram reminded me of the Cardassian user interface on Deep Space Nine.
  • Reply 9 of 104
    Personally I do not like the look of it. It looks cluttered and fake.



    I like a 2D environment as afterall the monitor is 2D. Why try to fake a 3D environment on it.



    I already have the dock on the right of my screen (in 2d) as I do not like how it takes up screen space (in fake 3D) at the bottom even though it is pretty. With more "3D" I just see less and less screen space as the rest is taken up by walls that I do not want.



    Hopefully this is not the future of OS 11?
  • Reply 10 of 104
    Quote:
    Originally Posted by sandau View Post


    Time Machine turned out pretty good for a 3d-ish environment. I'd be interested in more apps like it including the OS.



    well said
  • Reply 11 of 104
    >_>>_> Posts: 336member
    It could work well if it had the ability to detect the position of your head via your iSight camera, and adjust the 3d space accordingly.



    Could be like looking into a little room within your computer.
  • Reply 12 of 104
    crees!crees! Posts: 501member
    Quote:
    Originally Posted by kim kap sol View Post


    Too bad the implementation seen in the patent diagrams would be utter garbage.



    Stick with a 2D UI for 2D displays, Apple. When the time is right, a 3D interface for 3D displays will make sense. And hopefully, it won't be that pseudo-3D (read 2.5D) crap seen in the diagrams.



    Do you remember the patents we saw a few months ago? Take the pieces and build the puzzle. If not for use in the same product, at least there's a common theme.



    http://www.appleinsider.com/articles..._displays.html

    http://www.appleinsider.com/articles..._hardware.html
  • Reply 13 of 104
    "It's a UNIX system. I know this!"
  • Reply 14 of 104
    jeffdmjeffdm Posts: 12,946member
    Obviously there are details that need to be worked out for a 3D interface, but I don't think 2D displays are a problem. Humans can deal with 3D representations on a 2D surface, take a look at movies and games. Windowing GUI interfaces are already 3d-ish in a way, with layers. Dashboard comes down on top of the screen.



    That said, I can deal with not having 3D, I can get all my work done in 2D-land just fine, and I don't even need fancy GUI animations, as far as I'm concerned, when I'm working, those just take time away from actual work and are often superflous, especially the "bounce" in iPhone dialog boxes and pans, the zoom in and out for application start and end.
  • Reply 15 of 104
    Quote:
    Originally Posted by kerryn View Post


    Personally I do not like the look of it. It looks cluttered and fake.



    I agree.



    However, I will reserve judgement because I think it is obvious that they are not showing their hand on the pattent application which they know will be public. They put in only what they have to to protect their ideas, but they DO NOT want to give MS or anyone any ideas of what they need to catch up to.



    I don't know if if will be possible to come up with a clean 3-D interface, but I do have some hope that Apple will not implement it if they cannot...
  • Reply 16 of 104
    This wouldn't just be useful for a 2D environment. It could define a way to navigate 3d environments as well. It's not hard to imagine a time when the walls and ceiling would be covered by a display material with you and your Mac left to determine it's use.
  • Reply 17 of 104
    mactelmactel Posts: 1,275member
    Quote:
    Originally Posted by qnp9999 View Post


    NO! PLEASE NO! This is awful!



    Yeah it looks awful from the patent application but you see elements of it in the current OS.



    It makes sense now about Apple's push for OpenCL. The OS will need the GPU power to render the OS landscape. Snow Leopard is laying the foundations but this 3D OS in full form will probably be the version afterwards and dropping all legacy ties.
  • Reply 18 of 104
    hobbeshobbes Posts: 1,252member
    Interesting stuff.



    It's particularly interesting to see Apple trying and playing with new ideas - even when they don't look 100% successful - and you can see how they took elements from this direction that were utilized (in a more conservative, modest way) both in Leopard's Stacks and specialized Time Machine UI.



    I don't think we're going to see a 3D desktop as described by the above patents anytime soon.
  • Reply 19 of 104
    Quote:
    Originally Posted by kerryn View Post


    Personally I do not like the look of it. It looks cluttered and fake.



    I like a 2D environment as afterall the monitor is 2D. Why try to fake a 3D environment on it.



    I already have the dock on the right of my screen (in 2d) as I do not like how it takes up screen space (in fake 3D) at the bottom even though it is pretty. With more "3D" I just see less and less screen space as the rest is taken up by walls that I do not want.



    Hopefully this is not the future of OS 11?



    Same here, it looks so cluttered and untidy



    ____________

  • Reply 20 of 104
    I haven't yet seen anyone mention multi-touch. A 3-D interface would be exactly what is needed to fully realize multi-touch, bridging the gap between the computer and the real world for the user. Adding Stacks into Leopard was just a baby step in this direction, and was probably part of this on-going research. You can also see elements of a real-world interface on iPhone, where windows are "springy" and pop in and out, as they respond to your touch.



    People who say it looks cluttered aren't really seeing the big picture. This is just conceptual research in motion.
Sign In or Register to comment.