Filing details Apple's plans for Retina-ready resolution-independent OS X
Apple's behind-the-scenes work to upgrade its OS X operating system for future high-resolution Retina display Macs has been detailed in a new patent filing.
The application, published this week by the U.S. Patent and Trademark Office, is entitled "User-Interface Design." Apple re-filed the continuation with the USPTO this January, along with previous related filings that date back to 2004.
The proposed invention relates to specifying user interface objects in "a procedural and largely display resolution independent manner," the filing states. Graphical user interface elements are typically designed for use in a specified resolution, but as a user changes the resolution of their display, the originally designed object can become distorted or, in the case of a very high resolution display, unusably small.
Operating systems have previously used standard techniques to address this problem, such as taking a low-resolution object, like an icon, and up-sampling it to create a larger image. The problem with this method is it creates blurry edges, much like with iOS icons designed for the iPad 2 when they are displayed on the Retina display of the new third-generation iPad.
Another way of accommodating multiple resolutions is to take an object designed for a high-resolution screen and then downsampling it to an unknown target resolution. But Apple's filing states the problem with this method is it's impossible to know what width to give an object's edge to ensure it is crisp when down-sampled for a lower resolution.
"Line width is a critical factor in GUI design as the width of lines define the edge of graphical objects," Apple explains in the filing. "If edges appear blurry or ill-defined, the entire GUI design may be compromised."
Apple's solution as described in the filing is to create a plurality of attributes for a graphical user interface object, such as an icon or cursor, and store those multiple attributes and their associated values in a file. These attributes can be associated with distinct display resolutions, ensuring that the operating system will appear appropriately on any screen.
Apple's system could even accommodate resolutions for which graphical elements have not been specifically created. For example, if a user's machine is operating at a screen resolution between two of the resolutions specified by the designer, the rendering engine could interpolate between the two values and provide a better display than up-sampling or down-sampling of GUI elements would provide.
Operating system files that contain multiple attributes for various resolutions are referred to by Apple in the filing as "recipe" files. These types of files can already be found in the OS X operating system.
Last week, AppleInsider was first to reveal that Apple has continued to make behind-the-scenes improvements to OS X that will allow Macs with high-resolution Retina displays to show higher quality images. Evidence of this came in a new, larger TextEdit icon file, which contains new Retina display-caliber 1,204-by-1,024-pixel icons.
In a manner similar to what is described in Apple's patent application published by the USPTO this week, the icon files contain a number of images designed to be displayed by the operating system at various screen resolutions. The new higher-resolution icons could be utilized by a Retina display Mac with an ultra-high resolution screen.
Rumors have indicated that Apple plans to upgrade its Mac lineup to new high-resolution Retina displays this year. Double-resolution Retina displays were previously introduced with Apple's iOS products starting with the iPhone 4 and iPod touch, and brought this year to the third-generation iPad.
Apple is expected to unveil new, thinner MacBook Pro models with Retina displays at its annual Worldwide Developers Conference, which begins on June 11 in San Francisco, Calif. It has also been suggested that Apple's next-generation iMacs will also receive a Retina upgrade at WWDC.
The application, published this week by the U.S. Patent and Trademark Office, is entitled "User-Interface Design." Apple re-filed the continuation with the USPTO this January, along with previous related filings that date back to 2004.
The proposed invention relates to specifying user interface objects in "a procedural and largely display resolution independent manner," the filing states. Graphical user interface elements are typically designed for use in a specified resolution, but as a user changes the resolution of their display, the originally designed object can become distorted or, in the case of a very high resolution display, unusably small.
Operating systems have previously used standard techniques to address this problem, such as taking a low-resolution object, like an icon, and up-sampling it to create a larger image. The problem with this method is it creates blurry edges, much like with iOS icons designed for the iPad 2 when they are displayed on the Retina display of the new third-generation iPad.
Another way of accommodating multiple resolutions is to take an object designed for a high-resolution screen and then downsampling it to an unknown target resolution. But Apple's filing states the problem with this method is it's impossible to know what width to give an object's edge to ensure it is crisp when down-sampled for a lower resolution.
"Line width is a critical factor in GUI design as the width of lines define the edge of graphical objects," Apple explains in the filing. "If edges appear blurry or ill-defined, the entire GUI design may be compromised."
Apple's solution as described in the filing is to create a plurality of attributes for a graphical user interface object, such as an icon or cursor, and store those multiple attributes and their associated values in a file. These attributes can be associated with distinct display resolutions, ensuring that the operating system will appear appropriately on any screen.
Apple's system could even accommodate resolutions for which graphical elements have not been specifically created. For example, if a user's machine is operating at a screen resolution between two of the resolutions specified by the designer, the rendering engine could interpolate between the two values and provide a better display than up-sampling or down-sampling of GUI elements would provide.
Operating system files that contain multiple attributes for various resolutions are referred to by Apple in the filing as "recipe" files. These types of files can already be found in the OS X operating system.
Last week, AppleInsider was first to reveal that Apple has continued to make behind-the-scenes improvements to OS X that will allow Macs with high-resolution Retina displays to show higher quality images. Evidence of this came in a new, larger TextEdit icon file, which contains new Retina display-caliber 1,204-by-1,024-pixel icons.
In a manner similar to what is described in Apple's patent application published by the USPTO this week, the icon files contain a number of images designed to be displayed by the operating system at various screen resolutions. The new higher-resolution icons could be utilized by a Retina display Mac with an ultra-high resolution screen.
Rumors have indicated that Apple plans to upgrade its Mac lineup to new high-resolution Retina displays this year. Double-resolution Retina displays were previously introduced with Apple's iOS products starting with the iPhone 4 and iPod touch, and brought this year to the third-generation iPad.
Apple is expected to unveil new, thinner MacBook Pro models with Retina displays at its annual Worldwide Developers Conference, which begins on June 11 in San Francisco, Calif. It has also been suggested that Apple's next-generation iMacs will also receive a Retina upgrade at WWDC.
Comments
This is the sort of thing that makes being an Apple developer a pleasure.
Every platform has its up and down sides, but from a practical perspective, Apple continues to provide a great toolset.
They make mistakes, including the overly restrictive iOS environment, but unlike on other platforms, I feel like I have some basic confidence that code will work for more than just the people that have exactly the same machine as I do.
When my Windows user friends ask me why I prefer OSX, I have to tell them its because of its core technology and tools, not for any "its easier to use" reason. I like the toolset, I like the API, I like my familiar BSD environment.
I do think the emphasis Apple places on core platform development like this makes the whole of the experience much more stable than on other platforms.
This is similar to how PDFs work if I'm not mistaken. It's a good idea.
Quote:
Originally Posted by giffnyc
They make mistakes, including the overly restrictive iOS environment, but unlike on other platforms, I feel like I have some basic confidence that code will work for more than just the people that have exactly the same machine as I do.
When my Windows user friends ask me why I prefer OSX, I have to tell them its because of its core technology and tools, not for any "its easier to use" reason. I like the toolset, I like the API, I like my familiar BSD environment.
Please explain the "overly restrictive iOS environment."
Are you referring to the restricted amount of malware versus the 3,063 malicious application package files on alternative platforms?
Are you referring to the multitude of apps including maps & navigation, personal assistants, SMS/MMS clients?
Are you referring to the exceptional battery life and outstanding performance of iOS?
You should appreciate the "easier to use" reason as that is one of the primary reasons consumers prefer Apple products thus providing an audience for developers.
Quote:
Originally Posted by dbeats
This is similar to how PDFs work if I'm not mistaken. It's a good idea.
You are mistaken. These "resolution-independent" icons works pretty much the way that current icons do albeit with much high pixel density. They are bitmapped aka raster graphics. The OS interpolates among high pixel-density icons to achieve the illusion of resolution-independence. PDF graphics are vector or outline graphics which mean that they are always rastered at the resolution of the display to printed medium. No interpolation is done or required. I assume that Apple chose these technique over true resolution-independence because it is faster and allows hand-tuning of lower-resolution icons
Quote:
Originally Posted by Mr. Me
You are mistaken. These "resolution-independent" icons works pretty much the way that current icons do albeit with much high pixel density. They are bitmapped aka raster graphics. The OS interpolates among high pixel-density icons to achieve the illusion of resolution-independence. PDF graphics are vector or outline graphics which mean that they are always rastered at the resolution of the display to printed medium. No interpolation is done or required. I assume that Apple chose these technique over true resolution-independence because it is faster and allows hand-tuning of lower-resolution icons
I'm guessing you are right, however, a vector system that could cache raster graphics on demand would be preferred. People don't change resolutions that often, so there'd be a small penalty the first time you used an app, but it would be fast after that.
This is a great idea, but would love to see the ability to add a vector based solution into the mix. Essentially, allow for the highest resolution to be a vector graphic (for really high resolution) and allow hand-tuned bitmaps for the lower resolution versions as needed. Or, what about a dynamic arrangement where a vector graphic is provided as based on the resolution used by each user, a cache is made for that specific bitmap. It has the speed of a bitmap, but the quality of a vector graphic. Might be slow the first time as it is rendered, but then it would fast every other time, much like prebinding the libraries.
I would have thought a vector solution would have been better.
Also, how long until we see third party applications supporting this? Or are users of new retina display Macs going to find using Photoshop's already small icons somewhat impossible until CS7 comes out next year?
Quote:
Originally Posted by kotatsu
Or are users of new retina display Macs going to find using Photoshop's already small icons somewhat impossible until CS7 comes out next year?
Pshh, CS6 doesn't even support any Lion features. It'll be CS10 before this gets in.
I think the problem is you can get odd artifacts when you get vectors at the smaller resolutions.
I think non-RI apps will be scaled up, there's no reason to use them at native res if that means making the app unusable.
Exactly. PaintCode app does something promising and similar - all vector based, no bitmaps. Check it out: www.paintcodeapp.com
I am an Apple supporter, stockholder, and I loooove my iPhone... But doesn't this seem like a patent on the obvious? Please- someone point out what I am missing here.
Apple's initial approach to resolution independence has been about simply doubling resolution. That's pretty easy to implement. I suspect what we'll see is some applications that simply use this as a crutch (e.g. Adobe), and others that can truly support resolution independence. Which is also why the initial Retina displays will be approximately double existing resolution rather than 50 percent better, 70 percent better, 150 percent better and so on.
Quote:
Originally Posted by MacBook Pro
Please explain the "overly restrictive iOS environment."
Are you referring to the restricted amount of malware versus the 3,063 malicious application package files on alternative platforms?
Are you referring to the multitude of apps including maps & navigation, personal assistants, SMS/MMS clients?
Are you referring to the exceptional battery life and outstanding performance of iOS?
You should appreciate the "easier to use" reason as that is one of the primary reasons consumers prefer Apple products thus providing an audience for developers.
Well since you dont do much other things than look at photos, listen to music, watch movies and play games on your hardware we arent that much intrested in your opinions. After you have used computers for work and creating different kinds of content. Come back later to fill in on us "iOS criticisers" how you can do _everything_ with an ipad that you can do on mac os X! (_NOT_)
If your clueless to what im refering to plase do the folowing simple os X task on your ipad:
1. Take good pictures of a product (yes with a good SLR CAMERA = not ipad)
2. Edit pictures eg. put and a watermark on them
3. upload them to your site
4. not use a PC nor Mac os X machine for this SIMPLE task...
Or another one...
Attach a archive of your last years articles from your ipad favorite "file manager" (ifile doesnt count)
Yes, NO FILESYSTEM accounts for more restrictive and it really doesnt make usage _ANY_ easier AT ALL (but just the opposite indeed)! IT JUST KILLS THE FREE USER DEFINABLE PROSESS OF CREATIVE CONTENT HANDLING!!! You can not do what you want by chosing the apps you know will handle that file and use them to work on it. You depend on apple for creating the content that you want to edit and on the appmaker for handling your format. On os X you have to do neither. you chose what you HAVE (like iOS) and chose what the appmaker chose for you to prosess your files. OR choose what you NEED (like OS X) and what you need to do with it as YOU WANT and chose the apps you know read those formats or prosess all formats (eg, archiving, ).
Is that plain enough for you to understand.
Quote:
Originally Posted by MacBook Pro
Please explain the "overly restrictive iOS environment."
Are you referring to the restricted amount of malware versus the 3,063 malicious application package files on alternative platforms?
Thanks for tanking the adult discourse. Get a job.
Quote:
Originally Posted by MacBook Pro
Please explain the "overly restrictive iOS environment."
Are you referring to the restricted amount of malware versus the 3,063 malicious application package files on alternative platforms?
Are you referring to the multitude of apps including maps & navigation, personal assistants, SMS/MMS clients?
Are you referring to the exceptional battery life and outstanding performance of iOS?
You should appreciate the "easier to use" reason as that is one of the primary reasons consumers prefer Apple products thus providing an audience for developers.
Well, having bought iMovie and found that it couldn't see movie files on the SD card I took out of my Sony camcorder, I certainly thought it was either restrictive, or shite, depending on your point of view.
Still, only $5, if you don't like it you don't have to buy Apple, etc. etc.
Thought? TrueType and OpenType are font technologies. Icons and widgets are not fonts.
The article does not explicitly mention OS X text when it talks about "a plurality of attributes for graphical user interface objects". Presumably, all OS X text, icons and other graphic elements would be similarly resized on the fly to accommodate a higher-resolution screen. All I can say is that I've been waiting for resolution-independent type since the early 90's, so it's about time!
<snipped>I'm all turned around here. I need to eat...