Apple exploring motion-adaptive Mac OS X interface elements

[AppleInsider]

As rumors swirl over Apple's plans instate changes to the Mac OS X user interface with this year's release of Snow Leopard, a newly discovered filing shows the company has been exploring the use of user interface elements that change their appearance based on movement.

The 10-page document, published for the first time this week, was filed with the United States Patent and Trademark Office less than a month prior to the release of Mac OS X 10.5 Leopard back in October of 2007. Its short list of credits includes some of the Cupertino-based company's more prominent interface designers holding rich patent portfolios, including Bas Ording, Imran Chaudhri, and Elizabeth Furches.

As for the filing, it describes techniques for generating common Mac OS X user interface elements through the use of multiple graphical layers -- or a composite of a base layer, one or more pattern layers, and a shaping layer.

The base layer would provide the base color of the interface element, while one or more pattern layers would define one or more graphical patterns of the interface element. These pattern layers could, for example, include respective patterns of spaced-apart wave shapes that could be translated relative to each other to produce visual effects. A final layer, called the shaping layer, would sit atop these layers and dictate the overall shape of the particular interface element.

Although the concepts in the filing can be applied broadly, Apple focuses largely on describing its techniques in relation to Mac OS X scroll bars that would be formed from layer composites.

"When the scroll bar is moved rightward, the thin waves pattern (i.e., pattern layer 1) is translated rightward relative to the thick waves pattern (i.e., pattern layer 2)," the filing explains. "If the scroll bar moves leftward, the patterns are reversed and the thin waves pattern (i.e., pattern layer 1) is translated leftward relative to the thick waves pattern (i.e., pattern layer 2)."



While this implementation describes translating pattern layers in response to movement of the user interface element, Apple goes on to say that other graphical changes to the interface element can be performed in response to movement of the interface element.



"For example, a user interface element can include one or more layers that define lighting, shading, or color characteristics for the user interface element," the company says. "When the user moves the user interface element, the layer or layers change to change the lighting, shading, or color characteristics of the user interface element. For example, a vertical scrollbar can have a brighter shading as it is moved closer to the top, and a darker shading as it is moved closer to the bottom."

Speaking more generally, Apple adds that various graphical characteristics of one or more elements in a graphical user interface can be changed in response to various conditions or inputs at the computer, or in response to satisfaction of particular conditions or criteria, not just user input from a user input device.



"For example, the color or brightness of a menu bar can be changed based on the time of day, amount of computer use, activity or idleness of the computer, and so on," the filing says. "Further, in some implementations, the entire graphical user interface, not just particular user interface elements, can be changed in response to satisfaction of some condition or criterion."

Apple is rumored to be tailoring Mac OS X interface changes that will be adopted by Mac OS X 10.6 Snow Leopard prior to its release later this year. Specific details, however, are few and far between.

[ View this article at AppleInsider.com ]

[Outsider]

Hmm, interesting. But is this just eye candy or does it have some applicable interface advantages?

[Kasper]

Originally Posted by Outsider

Hmm, interesting. But is this just eye candy or does it have some applicable interface advantages?

From what I read, it seems to be just eye-candy. But the filing is more about the methods of composition for the interface elements, not their practical use.

K

[hiimamac]

Originally Posted by Kasper

From what I read, it seems to be just eye-candy. But the filing is more about the methods of composition for the interface elements, not their practical use.

K

The thing that amaze me, is that Apple is the only OS dveloper that can make eye candy that doesn't big done the OS. Additionally, their eye candy always seems to make sense. It's never useless. It will probably tie in to Pro apps too.

Could also further the use for special effects, background, blue screen from motion to iChat video. Let's hope so. Wouldn't be surprised if Apple delivers the first affordable blue screen that is at a Pro level, especially if the iPhone has video backgrounds.

[aToMac]

Can somebody explain what the difference is with the current aqua scroll bars? Maybe the way the composition takes place, but I already see such an effect on current scoll bars.

Edit: These patent describes some sorft of Doppler Effect in the scroll bars. The current ones don't have that.

[DyingSun]

Originally Posted by aToMac

Can somebody explain what the difference is with the current aqua scroll bars? Maybe the way the composition takes place, but I already see such an effect on current scoll bars.

Edit: These patent describes some sorft of Doppler Effect in the scroll bars. The current ones don't have that.

I still don't understand the relevance of what's being discussed here, if it's nothing more that a simple visual effect with no apparent usefulness.

I am definitely ready to be blown away by some cool, inovative and above all, useful interface changes at WWDC and I do hope it's more than just a cosmetic makeover.

[Virgil-TB2]

Originally Posted by DyingSun

I still don't understand the relevance of what's being discussed here, if it's nothing more that a simple visual effect with no apparent usefulness. ...

that's pretty much it. It's an interesting technique that has a lot of promise for making interesting UI elements.

It's just that the actual examples given here of what it could be used for are either dull, trivial, or downright annoying ("waves" on a scroll-bar thumb? Yikes!)

In general, it would be a way of making something like a scroll bar, that has to be really easy to see when you need it, and fade into the background when you don't, be more flexible in appearance. Personally, I'd go for 3D interface elements that were 2D until they were highlighted or rolled-over. Much classier than "waves."

[foljs]

Originally Posted by DyingSun

I still don't understand the relevance of what's being discussed here, if it's nothing more that a simple visual effect with no apparent usefulness.

It's called VISUAL FEEDBACK, and yes, it *is* useful. For one, it speedups the user-interaction with the UI, by removing uncertainty on certain actions.

It can also be used for so many things. Haven't you read the end of TFA?

"For example, the color or brightness of a menu bar can be changed based on the time of day, amount of computer use, activity or idleness of the computer, and so on," the filing says. "Further, in some implementations, the entire graphical user interface, not just particular user interface elements, can be changed in response to satisfaction of some condition or criterion."

As an example, think of the URL bar on Safari 3.0 going blue showing the progress of loading a page. (sadly, they have changed this behavior in Safari 4.0)

[elroth]

It looks more annoying than useful to me.

[VL-Tone]

Wow, interesting... Before Leopard was released and it was confirmed that it would include a vectorized GUI framework, and based on some other Apple patents hinting such things, I had a "vision" of what could be a new GUI for Leopard:

Here's what I posted on digg at the time:

Mark my words, Leopard will feature dynamic live re-coloring of windows, menus and widgets according to parameters like time of the day and/or user settings.

For example, during the day you could have your windows and menus with a bluish tint and black text, and they would gradually fade to a white text on black theme as the evening comes.

It's not like Apple has a fundamental philosophy that windows should be gray. The reason why they didn't implement such re-colorization feature before is that the old OS X UI framework was an archaic mess of bitmaps that was hard to maintain. The two current themes, "blue" and "graphite" each had their own sets of bitmaps, adding new color themes would've required creating new sets of bitmaps in that obsolete and messy Extra.rsrc file.

Leopard includes an all new XML based UI framework, and all widgets and window parts are being vectorized. Implementing dynamic re-colorization of the main theme will be trivial in Leopard.

And while Windows had the ability to have different color themes for years, Leopard will push the envelope further, with a dynamic aspect to it.

Don't you think Apple is aware that moving to a unified theme makes every window look the same, making Exposé harder to use? Colorization could be implemented on an application basis, or even a window basis, as a user choice from a color palette. That would fix the "recognition" problem while keeping a unified theme.

Other possible dynamic features could include a window that smoothly pulse red when it needs your attention, or changing color according to its activity, the outside temperature or the passage of time.

You guys seriously think that Apple will present a "dull", gray-themed interface in Leopard to compete with Vista, when the vectorization of the interface at last enables easy color customization in OS X?

Now I was wrong when predicting it would make it into 10.5, and while the vectorized UI framework is present, it's disabled when the resolution independence ratio is at 1.0x, and buggy when it's on. (I was the one that found a way to enable vectorized UI elements at 1.0x, but it slows down the system because of a constant stream of errors in the console, and there are a few cosmetic bugs).

So maybe the vector UI is fixed in Snow Leopard and Apple intends on implementing some of these "crazy" ideas?

[anantksundaram]

Originally Posted by elroth

It looks more annoying than useful to me.

I am sure it does, at this point, given our ability to forecast in the context of what we have have as information and know as experiences.

I am also equally sure that some people said the same thing about Apple's GUI or touch-screen or mouse or......

So, who knows.

[benny-boy]

i-sight could change the colors based on ambient light in the room, possibly sense where light comes from in the room and cast shadows on objects based on this?

[rasnet]

Back in the day, I seem to recall a fair number of people (probably including myself) thinking rippling scrollbars were gimmicky eye candy. If they could have composited them to ripple directionally back then, it might have sold us on Aqua faster

[rrabu]

Originally Posted by benny-boy

i-sight could change the colors based on ambient light in the room, possibly sense where light comes from in the room and cast shadows on objects based on this?

Or use the ambient light sensor that is used for the backlit keyboard brightness in notebooks.

Dimming certain bright elements even seems much smarter that just dimming the entire GUI...

[Shunnabunich]

I don't see how such things would be useless, if they're used in the right contexts. Essentially, this is no different at all from having animation in a UI. Gratuitous animation is pointless and irritating indeed, but what about seeing a minimized window shrink into its place on the Dock? Seeing an icon rubber-band back to its original location if it's released on top of something that can't use it? Seeing tabs or bookmarks move out of the way as you drag one around to rearrange it? That kind of animation helps users understand what's happening (and what they are doing) more clearly, and motion/time/conditionally-adaptive UI elements, when used according to the same principles, would be helpful in the same way.

[MyopiaRocks]

This sounds like exactly the sort of feature that makes older computers seem slower, post-upgrade...

...I love it.

[Amorph]

OK, first, we have to remember something about how Apple files patents: They never, ever, disclose what they're actually thinking of doing. Apple is secretive; patents are public documents.

Remember the diagram of the notebook computer sliding into a monitor to dock it? Yeah, they ended up releasing that as the 24" Cinema Display. No sliding anything into anything else. But they needed to describe the functionality, so they made something up.

So of course their examples are going to be banal, or glosses on behaviors they've already implemented some other way, e.g. the rippling scrollbars in Aqua.

In fact there's nothing new here in terms of functionality, just the implementation of using layers over and under user interface elements. Think of dragging a file in Mac OS 9: The file icon would turn translucent-ish when "picked up."

Some of this use could be frivolous, but some of it is related to touch technology. The genius of the iPhone's interface lies in its behavior. It acts physical and mechanical. You swipe, push, drag, pinch, or whatever and the behavior underneath directly correlates to the movement of your fingers. It's this kind of as-if physicality that appears to be Apple's direction, and the more convincing they can make the illusion the more intuitive the interface will be. So think of the way you drag icons around on the iPhone: The icons currently wiggle to let you know that they're movable. What if instead the screen appeared to thaw into liquid with the icons bobbing in it, and they sloshed around when moved. Touching the menu button would freeze the interface again. It's a bit gratuitous, yes, but it's also intuitive insofar as it appeals to a real, physical metaphor.

[JeffDM]

I tried reading the article a couple times, I still don't get it. But frankly, I don't need most motion effects, for example, I'd rather minimize and un-minimize just simply happen and pop into place rather than take two seconds with graphical transitions that I don't need. I know they're not useless, it's just not useful *to me*. I feel the same way about OS X dialog boxes transitions where they take a second or two to contract or expand.

[ascii]

This patent is for CoreAnimation. Most Mac programmers know CA is called "LayerKit" behind the scenes and works just as described...

[Messiah]

And there was me hoping that those Aqua scroll bars were going to be a thing of the past!