Apple developing active desktop feature for Mac OS X

[AppleInsider]

An enhancement to the Mac OS X operating system under development by Apple looks to pave the way for active desktop pictures, or desktop backgrounds that can include motion graphics and alter themselves based on user actions or the time of the day.

In a patent filing detailing the development, Apple notes that desktop pictures have traditionally been drawn using an image stored in a static picture file that consumes considerable resources in RAM and VRAM (video RAM). In addition, the Mac maker also notes that because desktop pictures are static, transitions between the login screen and the desktop picture are generally abrupt and nonorganic.

Instead of loading a file that contains the desktop image, Apple's design looks to provide for a system and method for opening and retaining a procedural "recipe" and a small set of instructions that can be executed to compute a desktop picture over time. The technique is said to reduce requirements for both VRAM and RAM, hence taking less memory away from the system.

"Because the desktop picture is computed using a procedural recipe, the storage for the desktop picture can be eliminated," Apple wrote. "This includes both main memory (e.g., RAM) and video memory (e.g., VRAM) copies of the picture. Advantageously, using a procedural recipe to compute a desktop picture (or a portion thereof) allows the unused VRAM and RAM to be used for other operations."

The Cupertino-based systems builder goes on to say in the filing that seamless integration between login, the desktop picture, and log out also provides a visual hook that can further distinguish products.

"Since the desktop picture can be computed very quickly using a GPU, it may be made to move on demand," the company explained. "This includes movement, for example, when logging in, logging out, and transitioning to and from a screen saver, providing a seamless experience. It can also include slow movement, such as seen when a soft tree shadow is cast, with the gentle rustling of leaves in the breeze, or slow movement over time, or concerted movement to mark the passing of time (e.g., a noticeable change to pattern or color every hour)."

Apple also hinted at an editing tool that could allow desktop picture designers to edit and specify the user experience. "In certain embodiments of the present invention, a single frame may be computed," the company said. "In other embodiments of the present invention, multiple frames may be computed. Furthermore, transitions between frames effectively provide movement of desktop pictures on demand."

A seamless transition from one desktop picture to another may be used to simulate motion or animation, according to the filing, where several gradations may be computed over time. "For example, the color gradation may be computed based on the time of day to mimic the changes in the colors of the sky," Apple wrote.

Aside from the time of the day, a variety of other types of events may be used to effect a change to the desktop picture. "For example," Apple continued. "the desktop picture may change upon a user event, such as launching a predetermined application. For instance, a particular desktop picture may be computed when the application iTunes.RTM. is launched; a different desktop picture may be computed when the application QuickTime.RTM. Player is launched."

Additionally, desktop pictures may change upon exiting a predetermined application, where upon exiting a particular application, the desktop picture may revert to the pervious desktop picture that was displayed just prior to the launching of that application.

As yet another example, Apple said the desktop picture may change after the computer is idle for a particular period of time or when the computer comes out of an idle state. It could similarly change when transitioning to or from a screen saver.

"As will be appreciated by those of skill in the art, the foregoing examples are provided as just a few examples of the many types of events that may be used to trigger a change in the desktop picture, and are not intended to be an exhaustive list," Apple wrote. "It will be appreciated that various other types of events may be used to trigger a change in the desktop picture and are within the scope of the present invention."

The October 14, 2005 filing with the United States Patent and Trademark Office is titled "System and method for computing a desktop picture." It was published for the first time on Thursday with credits to Apple employees Ralph Brunner, Imran Chaudhri, and Mark Zimmer.

[ View this article at AppleInsider.com ]

[ Digg this story ]

[gloss]

This is something I've always noticed with any OS (re: the 'inorganic' transition from login to desktop). It'll be great if they can do something about it.

[Booga]

This is a good idea. Essentially, use pixel shaders to compute an abstract background image instead of a static huge picture. For a gigantic display or a very high DPI display, an image is going to use up huge resources.

Of course, this is the kind of good idea that once everyone hears about it, retroactively assumes was "obvious". I'm sure SlashDot is going to have a field day crying foul about this patent.

[JupiterOne]

In other news, Microsoft patents an idea for tiling icons. Imagine!

[solipsism]

A static desktop image uses a lot of resources? I have trouble believing this; can someone point me to some info?

Originally Posted by Booga

This is a good idea. Essentially, use pixel shaders to compute an abstract background image instead of a static huge picture. For a gigantic display or a very high DPI display, an image is going to use up huge resources.

Of course, this is the kind of good idea that once everyone hears about it, retroactively assumes was "obvious". I'm sure SlashDot is going to have a field day crying foul about this patent.

And Diggers will point out that MS has had "Active Desktop" for years.

[mmmdoughnuts]

Most pics I see as desktop backgrounds are only a few MBs. With system memories being 1GB+ now this is a silly argument.

What it really sounds like is that they are off loading much of the windowing environment off of the CPU and turning it in to calculations that are efficient for the GPU and directly to the display.

[syklee26]

Isn't this feature already in Vista or something? Maybe I am misunderstanding the whole concept...

[wirc]

Well I'm glad I didn't vote for Atmosphere. But could that program interfere with the patent?

[Louzer]

How exciting! I guess leave it to me to be one who has to ask the obvious. Do people actually see their desktop that much to care? To me, any desktop I see is space that could be used for some actual purpose (you know, sort of like how having 'free' memory in a computer is a complete waste, since it just sits there, while even if it just stores programs that have been quit, it can at least help speed opening it again).

[pgb0517]

Waste of Apple's resources! How does this help me administer hundreds of Macs in a corporate environment? Let Apple focus on making the Active Directory plug-in and other business-related tools work flawlessly before they waste time on eye candy. And hey, how about that Leopard release date?

[Clive At Five]

"The Cupertino-based systems builder."

Getting creative, are we?

-Clive

[Clive At Five]

Originally Posted by syklee26

Isn't this feature already in Vista or something? Maybe I am misunderstanding the whole concept...

Vista may have this , but the patent was filed in 2005... long before Vista was released. I think we were still calling it "Longhorn" then...

As for the whole concept, I think this is probably part of a much broader environment beyond desktop backgrounds. The particular feature is unremarkable, but if combined with other UI elements, this could be the workings of a new OS-platform altogether. I mean it'll still be Mac... but perhaps something for OS "XI" - a reinvention of the GUI.

Then again, it could be another one of those patents for Apple to cover their ass.

-Clive

[gloss]

Originally Posted by JupiterOne

In other news, Microsoft patents an idea for tiling icons. Imagine!

That's pretty shameless.

[SMQT]

The way I read the patent, it is broader than just generating and rotating Desktop pics. Seems like they plan to use the Desktop as part of the interface of applications.
I'm imagining something along the lines of the Time Machine interface, where the whole environment changes, instead of an application just being a window.
That would be cool IMO.

The not so cool part is that this opens up space for advertisers, right on your Desktop: If you open an application all of a sudden your desktop displays ads.

[Clive At Five]

Originally Posted by pgb0517

Waste of Apple's resources! How does this help me administer hundreds of Macs in a corporate environment? Let Apple focus on making the Active Directory plug-in and other business-related tools work flawlessly before they waste time on eye candy. And hey, how about that Leopard release date?

Seeing as how this was a 2005 patent filing, I would highly doubt that it has been instrumental in delaying Leopard. It seems like a rather trivial feature on its own, but (like I said above) it could be paired with other UI elements for a future Apple reinvention of the GUI.

But speaking of resources, this seems to me like this feature would use more resources than it would save - especially GPU resources...

-Clive

[Gee4orce]

You can do this now, with QuartzComposer and one of the several programs that let you run a screensaver (or Quartz composition) as your desktop picture.

[eAi]

Originally Posted by Clive At Five

"The Cupertino-based systems builder."

Getting creative, are we?

-Clive

We had "the Calfornia system builder" yesterday too! http://www.appleinsider.com/articles...ite_vista.html

[Johnny Mozzarella]

Originally Posted by pgb0517

Waste of Apple's resources! How does this help me administer hundreds of Macs in a corporate environment? Let Apple focus on making the Active Directory plug-in and other business-related tools work flawlessly before they waste time on eye candy. And hey, how about that Leopard release date?

I've got a great idea! Let's do all the things you want to do!

[desarc]

how is a picture more resource intensive than an active, moving image that reacts to multiple triggers?

i can see this being used not for a "background", but to use the entire desktop as the "window" for the current application - no more windows one on top of another as we know them now. i hardly ever just click on a background application - almost always use apple-tab.

i could care less about a smooth transition from log-in to desktop, or saving .001% of my ram/vram

[nagromme]

Originally Posted by solipsism

A static desktop image uses a lot of resources? I have trouble believing this; can someone point me to some info?

And Diggers will point out that MS has had "Active Desktop" for years.

A desktop image uses a chunk of VRAM, which some systems have to spare. Some don't. (Assuming it's rendered in OpenGL like everything else, I imagine it might be even worse if GPUs must round up to the nearest power of 2, and if they store the texture uncompressed. A 1600x1050 20" wallpaper would then have to be an uncompressed 24-bit 2048x2048 texture.)

As for Active Desktop (which Microsoft removed from Vista), that's something entirely different--HTML pages as wallpapers.

Originally Posted by Clive At Five

But speaking of resources, this seems to me like this feature would use more resources than it would save - especially GPU resources...

It describes saving VRAM, not GPU processing time. Except when gaming, GPUs often have processing time to spare... and during hardcore 3D gaming the desktop isn't visible anyway.

But saving VRAM is just a side benefit--you could use a solid black if that's all you wanted. The point of this is that it animates and responds to things.

A neat if minor feature. But it sounds to me like yet another cool patent that Apple probably won't ever do anything with--a patent "just in case" they ever need it.

(AI's headline that Apple is "developing" this seems to be an overstatement.)

[Johnny Mozzarella]

Originally Posted by Clive At Five

Vista may have this , but the patent was filed in 2005... long before Vista was released. I think we were still calling it "Longhorn" then...

As for the whole concept, I think this is probably part of a much broader environment beyond desktop backgrounds. The particular feature is unremarkable, but if combined with other UI elements, this could be the workings of a new OS-platform altogether. I mean it'll still be Mac... but perhaps something for OS "XI" - a reinvention of the GUI.

Then again, it could be another one of those patents for Apple to cover their ass.

-Clive

procedural desktops + resolution independent UI + Spaces

This should make the whole UI a lot more "snappy".

[donebylee]

Hmmm, I wonder if this could be combined with the different desktops in Spaces to give a different look to each virtual desktop? That would offer some advantages.

And from a designer/illustrator/photoshop user's point of view, the ability to dim or change the desktop to a non-obtrusive theme when working could be useful.

[nascarnate326]

I think this is a great idea, and I would love to have an ocean scene where the water comes and goes, the sun does what its supposed to, and the wind blows. Maybe some people swimming or boating. The ideas I have!!

Only thing...how much power will this take, I hope my C2D macBook isnt too old to do this kind of stuff.

[pragmatic67]

I read this and thought, Oh Well there goes any chance of people paying for Atmosphere. it sounds just the same. is It? - http://mydreamapp.com/

[JupiterOne]

Originally Posted by nascarnate326

I think this is a great idea, and I would love to have an ocean scene where the water comes and goes, the sun does what its supposed to, and the wind blows. Maybe some people swimming or boating. The ideas I have!!

Only thing...how much power will this take, I hope my C2D macBook isnt too old to do this kind of stuff.

Yeah, but that's not very interactive is it? What if, as the sun crosses the sky, it makes your icons cast shadows over the desktop. Or when you "Move to Trash", a big wave comes in and drags the icons out to sea.

Although, I'm not sure how productive I'd be by watching all of this...

[marshall]

I imagine a nature scene with the sun traveling across the sky and creating shadows as it should be outside. While it may not directly influence my work, for those of us who spend the workday in cubes with no natural light, and walk outside to gray skies (in the Norhtwest), it might provide some mood elevation and maybe help internal clocks to stay in sync.

Sounds good to me.

[design1]

So... this would be Quartz Destop, with like Coverflow?

[Booga]

Most posters are completely missing the point. Let's say you have a 200dpi screen that's 21" diagonal, in a 4:3 configuration, running at 32-bit pixel depth. That's 33MB of VRAM used up for a background image and taken away from any games, other window buffers, Core Video, etc. If you instead define an abstract background pattern procedurally and run it on the GPU, you're using up some GPU processing power, but no VRAM. As monitors get more resolution, this advantage will increase.

This isn't about just putting some active content on the background, it's about how to implement the background image on the GPU.

[JeffDM]

Originally Posted by eAi

We had "the Calfornia system builder" yesterday too! http://www.appleinsider.com/articles...ite_vista.html

Next, maybe they'll call Apple an "earthling system builder".

[auxio]

Originally Posted by Booga

Most posters are completely missing the point. Let's say you have a 200dpi screen that's 21" diagonal, in a 4:3 configuration, running at 32-bit pixel depth. That's 33MB of VRAM used up for a background image and taken away from any games, other window buffers, Core Video, etc. If you instead define an abstract background pattern procedurally and run it on the GPU, you're using up some GPU processing power, but no VRAM. As monitors get more resolution, this advantage will increase.

I don't quite buy this argument. Regardless of what is being displayed on the screen (a dynamically created image, or a static image), you still need to put it into a framebuffer so that it can be displayed on the screen. Whether you take a JPEG image, convert it to the required image format for the framebuffer, and load it there, or generate the image data on the fly and load it, you still need that framebuffer memory to be allocated in VRAM. So unless video card manufacturers come up with a new way to display screen contents, you can't really get around this.

The only difference I can see here is the actual loading of an image file into main memory or not. Which is larger than simply the size of the image file itself (ie. most images are compressed in some way), but still not all that massive given that most people have around 1GB of RAM these days.

One other benefit I can think of is reduced usage of the system bus. If the image data doesn't need to be loaded from main memory to the framebuffer a number of times per second, and instead is simply loaded there directly from the GPU on the video card, then that frees up a bit of bandwidth on the system bus. Although perhaps the desktop image is somehow cached on the video card itself or maintained in the framebuffer once it's loaded there (I'm not exactly sure of the technical details of video card framebuffer compositing).

[Hiro]

I think the point most folks are missing is that this is old tech written up and finally filed. The OS X desktop is just an OpenGL display layer, and has been since 10.2. This just codifies that without explicitly mentioning OpenGL. Everything mentioned in the patent is trivially implementable using OpenGL calls.

I think this is more a case of building the portfolio for defense than creating something radically new. An unfortunate side effect of the whole software patent process.

[Marvin]

Originally Posted by Appleinsider

An enhancement to the Mac OS X operating system under development by Apple looks to pave the way for active desktop pictures, or desktop backgrounds that can include motion graphics and alter themselves based on user actions or the time of the day.

Originally Posted by Marvin

They could even have color sets to allow users to easily switch for seasons or even the time of day. You could have the theme light platinum at midday and then gradually fade to black at night.

http://forums.appleinsider.com/showp...43&postcount=5

Wow, it's like Apple are reading my mind...

or my posts.

This will be the kind of thing that blows people away. When you see Vista looking all shiny but static and a Mac with some leaves gently blowing in the wind and an animated interface, there will be no contest. Not that a nice GUI is what should define an operating system but it will to some consumers.

[Macvault]

Originally Posted by pgb0517

Waste of Apple's resources! How does this help me administer hundreds of Macs in a corporate environment? Let Apple focus on making the Active Directory plug-in and other business-related tools work flawlessly before they waste time on eye candy. And hey, how about that Leopard release date?

Ditto! Ditto! Ditto! Ditto!
My thoughts EXACTLY

[SpamSandwich]

Originally Posted by nascarnate326

I think this is a great idea, and I would love to have an ocean scene where the water comes and goes, the sun does what its supposed to, and the wind blows. Maybe some people swimming or boating. The ideas I have!!

Only thing...how much power will this take, I hope my C2D macBook isnt too old to do this kind of stuff.

I've had a dream for years of a desktop that would change over time to reflect the outside temperature or weather patterns or the seasons. It would reduce the stir-crazy effect people get in offices when they are not near a window or able to see outside.

There a millions of variations on this theme that could now be possible.

[Ireland]

In other news; where are all the Dell Printer drivers for OS X?

[Kolchak]

I sure hope this isn't one of the "top secret features" that's been holding Leopard up.

[MarcUK]

exactly! ^

Its bollocks like this I dont want to read when the OS has just been delayed 6 months.

[Ireland]

Originally Posted by MarcUK

exactly! ^

Its bollocks like this I dont want to read when the OS has just been delayed 6 months.

It's 4 months.

[i386]

Originally Posted by SpamSandwich

I've had a dream for years of a desktop that would change over time to reflect the outside temperature or weather patterns or the seasons. It would reduce the stir-crazy effect people get in offices when they are not near a window or able to see outside.

There a millions of variations on this theme that could now be possible.

mmh haven't you tried OSXPlanet ?

[fanboy_uk]

Active Desktop - hmm. Does this mean we could get the e-mail 'send mail video' that we have always dreamed of. You know the one. It's been in countless movies..
You write your message, click send and then a graphic neatly folds the message, puts into an envelope and then posts in it in a mail box and does the opposite when you receive mail.

I like the idea of the change time, seasons or temperatures.

As for business/networks I wonder how 'user serviceable' this feature will be.
Could you for instance use it to show a network diagram showing managed computers on your network, when they become active, who is logged on etc.

Other thoughts I had are...
Link to it iCal and have upcoming meetings shown with a count down.
Link it to iChat, Skype, GMail etc to show when people in your address book are logged on.
There are alot of possibilities but I guess it depends on whether you can write script or automator flows to make it more personal.

Finally, with my conspiracy theory hat on, maybe Apple are doing some canny market research.
File the patent, see what people think. If they like it commit to continuing development if they don't ditch it.
But the suggestion that this could be a precursor to a new interface for the operating system could suggest that OS XI is on the horizon.