Inside iCloud: Apple's new Documents & Data cloud service

Posted:
in macOS edited January 2014
In place of the simple cloud storage and web hosting services in today's MobileMe, including iDisk, iWeb and Gallery, Apple is preparing an innovative new cloud-based Documents & Data service. Here's how it works.



The road to iCloud: the host without the most



Beginning over ten years ago with iTools, Apple's online services for Mac users have morphed through a variety of incarnations, perpetually being a mixed bag of success and failure. After a false start as a variety of general purpose web apps ranging from iCards for postcards and iReview for discussing other websites, the most successful aspects of Apple's online efforts have turned out to be email and messaging services.



These were enhanced in the move to .Mac, which eventually began to offer web client access, and further improved upon during the transition to MobileMe, when Apple added push messaging features targeted toward its new audience of mobile users.



Among the least successful aspects of Apple's online web service experiments have been the web hosting and virtual disk features associated with iDisk. In part, this has been because what Apple has offered in these areas has never been competitive in price or performance with services dedicated to blogging or online file sharing.



While the company attempted to differentiate its iDisk-related hosting offerings with desktop Finder integration on the Mac, native clients for iOS devices, and easy to use publishing tools integrated into iWeb and iPhoto, few users were sold on the idea of paying for MobileMe principally or even significantly because of its online sharing aspects.



Rethinking document cloud services



In the planning process to develop its next generation of cloud services, Apple decided to radically rethink its offerings in the area of file sharing. The result has been two separate but functionally similar features of the new iCloud: Photo Stream, a way to automatically push photos taken on one mobile device to every other Mac and iOS device the user has, and Documents & Data, a parallel solution to enable applications to do something similar for the files and data they use.



Photo Stream solves real problems for users without requiring them to think about how to use the new service. All they need to do is configure their devices and computers once with their iCloud account, something they're likely to do anyway to use messaging and sync services.



Once their equipments is all tied to the same iCloud account, Apple automatically pushes new photos up to the cloud when captured, and down to the user's various photo libraries on iOS, Mac and Windows PCs. No more cables for manual syncing, no more manual media imports, and no more looking for what device is storing the photos you want to access.



Essentially, Photo Stream is push messaging for media, with the same kind of effortless management afforded by IMAP-style email as opposed to the confusion of having various computers each set up with manual POP-style email, where messages end up stuck on one system and missing from another.







iClouds' Documents & Data delivers the same kind of genius level of simplicity for every other app that uses files. Apple outlined to developers how the new system would work at its Worldwide Developer Convention this summer under strict NDA. Fortunately for the curious rest of us, word quickly leaked out about how Documents & Data would work.



How iCloud's Documents & Data feature works



Apple outlined publically in general terms what Documents & Data does: it propagates changes made to a file on one device across every other. Similar to Photo Stream, creating a new Keynote presentation on your Mac will result in a magical multiplying of the document and an immediate background delivery of the new document on your iPad, on your iPhone or iPod touch, and within the iWorks web app so you can access it remotely from any browser.







There's no need to manually save the document to something like iDisk, or to look for where it might be stored in the file system, and no need to perform any manual sync between devices. It just happens. When you edit that presentation on your iPhone, all the changes you make similarly appear in every other instance of that file across your devices, again without needing any sort of thinking or involvement on your behalf.



The concept is deceptively simple, but it involves quite a bit of thoughtful engineering on Apple's behalf. There needs to be support built into both the desktop and mobile operating system, as well as a web interface for the cloud-based storage. Third parties also have to incorporate support for iCloud's Documents & Data in their apps. Apple has demonstrated how to do with with its iWork apps, the first to incorporate support for the new feature.



Rather than just throwing up a way to copy around files automatically, Apple took an carefully considered approach that borrows from its experience in curating the iOS and Mac App Stores. This isn't just a network-synced iDisk. Instead. each app that opts in to the iCloud Documents & Data feature is accorded a secure storage space of its own, just as iOS apps each live in their own secure sandbox, inaccessible from other apps.



Using iCloud-aware apps therefore won't eat up users' free storage on iCloud, just as Photo Stream or iTunes' media, apps and iBooks use won't count against the free 5GB of storage every iCloud user gets.



More importantly, a rogue app created to prey upon Mac users won't be able to destroy or corrupt the documents or data stored by iCloud-savvy apps, adding a new level of security within the user realm of the computing model. Currently, any piece of malware (or simply a buggy app) that a user installs has the capacity to stomp all over that user's files and any network shares he or she has mounted. With iCloud, that huge vulnerability is now possible to contain in a way that hasn't been possible before in mainstream desktop computing. Under iCloud, only the app that creates the data can change the data, and must do so securely.



Apple has also indicated that it will be relatively easy for third parties to add support for iCloud Documents & Data. In fact, it appears that Apple is leveraging the existing Versions and Auto Save mechanisms new in Mac OS X Lion to also enable iCloud to efficiently deliver updates to the cloud as you make them and subsequently push those incremental changes down to all your other devices to keep them all on the same page.



Within a week of iCloud's announcement at WWDC, prominent Apple blogger John Gruber revealed how iCloud would handle file sync conflict issues, noting that it would seamlessly resolve file conflicts that occur between devices while backing up each version of the file.



iCloud will "make a decision and it will decide which one is the best," Gruber stated. "iTunes will decide which one is right and that?s it. iCloud will push that right one to any device that has this account that has a different version."



And so, essentially, iCloud Documents & Data will work something like Time Machine in the Cloud, except that rather than being focused on traveling through the past looking for lost versions, the new feature will keep users' documents up to date in the present, with push updating of changes as they happen to all devices they own. Again, think of it as push messaging for version controlled documents.



On page 2 of 2: Strength in numbers, ? & Data

Strength in numbers



Apple is working to make it appealing for developers to feature iCloud integration in their apps. The company similarly encouraged developers to add support for .Mac/MobileMe sync, something that many developers did add, despite the relatively limited number of subscribers that service had (believed to be around 3 million). In contrast, the entire installed base of Mac OS X Lion and all mobile users who upgrade to iOS 5 will have free access to iCloud, a number that will quickly boost iCloud users into the tens of millions.



Activating iCloud in iOS 5 is integrated right into the new operating system's self-contained setup system. Once configured in the iOS iCloud menu within the Settings app, turning on Documents & Data is simply a matter of sliding a switch. A sub option unique to Documents & Data allows users the option of transferring updates over cellular. Because Documents & Data delivers incremental changes (similar to an undo stack) of a document as it is edited, rather than pushing around entire document files, this option won't be a huge tax on cellular connections. Conversely, backups, over the air OS updates, and features like Photo Stream are confined to WiFi connections.







In Mac OS X Lion, iCloud is configured in the new Mail Contacts & Calendar Preference Pane unlike (and replacing) MobileMe's separate configuration pane. While existing builds of iOS 5 can connect to MobileMe and iCloud simultaneously, the latest build of Lion can reportedly only connect to iCloud if MobileMe is turned off. Additionally, the new iCloud interface incorporates Back to My Mac (formerly a tab within MobileMe) and the new Find my Mac feature within the same simple settings panel.







Third party apps that tap into the features of iCloud Document & Data will "just work" once users activate the service on their system. Again, this is somewhat similar to Time Machine, where simply turning it on creates a system wide benefit that specific apps can be optimized to take special advantage of with some work by their developer. There's nothing to configure within an app that has been enhanced to use iCloud, as all the configuration is done on the operating system level in both iOS and Mac OS X Lion.



Apple's beta iCloud web client, as has been profiled by some developers and media sources already, enables users to see and download copies of the files they are using from the browser, and currently supports translation of their Pages, Numbers and Keynote documents into PDF or Microsoft Office compatible formats. It's not clear if Apple will extend its iCloud web offerings to also add access to the files of third party apps with iCloud Documents & Data support.











? & Data



However, iCloud's Documents & Data will also be useful to more than just those app that work with documents. The "& Data" portion of the feature alludes to apps that make use of data, but don't actually generate files. An example of this might be an app that records events, such as a run tracker or an instrument monitor. These apps will similarly be able to push their raw data into iCloud and keep every other instance of the app current with the same information. This could conceivably work to allow an iOS app to remotely gather data that a desktop Mac app then presents "automagically" in real time without any manual syncing by the user.



As noted by API Evangelist, iCloud can store both apps' documents and "key-value data," which could include application state (allowing apps on various devices to sync their behaviors, such as resuming the same playback point in a video player app on another device), settings and "other important information that delivers a better user experience. Although not as apparent to the end-user, key-value storage will be just as important as document storage," the site notes.



"The iCloud service handles storage searches, change notifications, version control, conflicts, and security for applications that integrate with a user?s iCloud storage account," that site reports, concluding "neither document storage or key-value storage in the cloud are anything new. But when it is implemented as part of the IOS platform, it becomes much bigger. Apple is solving everyday problems users face when using their smart phones, by storing data centrally in the cloud."



Inside iCloud: Apple's new web services for iOS and Mac OS X Lion

Inside iCloud: Apple's new Documents & Data cloud service
«13456

Comments

  • Reply 1 of 112
    alandailalandail Posts: 755member
    If an app like Numbers stores it's document in the cloud, can Numbers on the mac just work with those directly instead of having to download the document, make the edits, then put it back on the cloud? I'd like editing a spreadsheet to work as seamlessly across my devices as editing a calendar entry, but noticed in the photos, they replaced the mac in the pictures with a 3rd iOS device.
  • Reply 2 of 112
    solipsismsolipsism Posts: 25,726member
    Quote:
    Originally Posted by alandail View Post


    If an app like Numbers stores it's document in the cloud, can Numbers on the mac just work with those directly instead of having to download the document, make the edits, then put it back on the cloud? I'd like editing a spreadsheet to work as seamlessly across my devices as editing a calendar entry, but noticed in the photos, they replaced the mac in the pictures with a 3rd iOS device.



    I'm not sure I follow you. If you change your calendar data (or contacts) those changes are downloaded/uploaded to the devices that get the changes. iCloud does the same thing but on a much larger and more complete scale. If you make a change to you Mac's spreadsheet the changes will be pushed to iCloud and then pushed to other devices that are connected to that document.
  • Reply 3 of 112
    Should have dusted off eWorld... Now that was revolutionary.



    Prompted me to reminisce...



    http://www.remember-eworld.de/
  • Reply 4 of 112
    Quote:
    Originally Posted by AppleInsider View Post


    ... Among the least successful aspects of Apple's online web service experiments have been the web hosting and virtual disk features associated with iDisk. ... While the company attempted to differentiate its iDisk-related hosting offerings with desktop Finder integration on the Mac ... few users were sold on the idea of paying for MobileMe ...



    It should at least be mentioned that the main reason people were/are not happy with iDisk is not the price, but the performance. It's slow. Ridiculously slow. Unuseably slow.



    Acknowledgement of this, and a serious discussion of whether with all it's fancy-ness, iCloud will be any faster would seem to be central.



    Getting in touch with users of the iCloud beta, especially those not based in the USA and finding out if the speed of access is improved in iCloud, would be far more valuable than anything Dan can tell us about how great it works when you live within a few of miles of Apple headquarters.
  • Reply 5 of 112
    slapppyslapppy Posts: 331member
    Maybe I missed it, but how do you go about saving a file with a unique name for it?
  • Reply 6 of 112
    bdkennedy1bdkennedy1 Posts: 1,459member
    Keep in mind when using iCloud that anything stored on Apple's servers belongs to Apple and deleting it doesn't mean it's gone.
  • Reply 7 of 112
    when apple does it is is "magical" and "innovative". Novell Ifolder came out in 2001. And i guess no one has heard of dropbox.
  • Reply 8 of 112
    bushman4bushman4 Posts: 858member
    Quote:
    Originally Posted by bdkennedy1 View Post


    Keep in mind when using iCloud that anything stored on Apple's servers belongs to Apple and deleting it doesn't mean it's gone.



    Point well taken. Not only isn't it deleted but its not encrypted. So as long as what your storing in the cloud has no value to anyone else its good.
  • Reply 9 of 112
    solipsismsolipsism Posts: 25,726member
    Quote:
    Originally Posted by Prof. Peabody View Post


    It should at least be mentioned that the main reason people were/are not happy with iDisk is not the price, but the performance. It's slow. Ridiculously slow. Unuseably slow.



    Acknowledgement of this, and a serious discussion of whether with all it's fancy-ness, iCloud will be any faster would seem to be central.



    Getting in touch with users of the iCloud beta, especially those not based in the USA and finding out if the speed of access is improved in iCloud, would be far more valuable than anything Dan can tell us about how great it works when you live within a few of miles of Apple headquarters.



    They 1) have a major new data center near an internet backbone and 2) are using delta updating to move the changes around.



    After using the service my question isn't if it's faster but will it remain faster as they could easily have more iCloud users than the entire Mac and IOS-based iDevice installed base.







    PS: I just updated the 122MB iOS 5.0 beta 5 delta update for the iPad 2 from Yosemite National Park in 10 minutes. I love the modern world!
  • Reply 10 of 112
    ruckerzruckerz Posts: 58member
    Quote:

    Third parties also have to incorporate support for iCloud's Documents & Data in their apps



    Just as I thought. Users are unable to save arbitrary files into the cloud. I was using iDisk as a remote data store when collecting data (on computers that were not Apple's) and then having it sync to my home compter. Now I can't do this. In order to use iCloud, the data must be saved by the app internally and use Apple's API.



    As much as I hate it, Dropbox will stick around on this computer when iDisk goes away.
  • Reply 11 of 112
    Quote:
    Originally Posted by bdkennedy1 View Post


    Keep in mind when using iCloud that anything stored on Apple's servers belongs to Apple and deleting it doesn't mean it's gone.



    It's obviously not for such a person!
  • Reply 12 of 112
    nagrommenagromme Posts: 2,834member
    Remember: whenever Apple does something new, and other companies have done past things that use some of the same words (“cloud,” “touch screen,” “smart phone,” “tablet,” “music store,” “music player”) you must realize that it’s not innovative or better or different at all, it’s a copycat, and Apple is only “catching up” (to Tablet PCs, Blackberries, Google Docs, netbooks, whatever).



    Quote:
    Originally Posted by alandail View Post


    If an app like Numbers stores it's document in the cloud, can Numbers on the mac just work with those directly instead of having to download the document, make the edits, then put it back on the cloud? I'd like editing a spreadsheet to work as seamlessly across my devices as editing a calendar entry, but noticed in the photos, they replaced the mac in the pictures with a 3rd iOS device.



    Numbers (and other iCloud-aware apps) on Mac and iOS should all work alike, although I do have questions about file management. (You have your docs in folders on the Mac, and organize them at will... how does that work on iOS? I’m guessing your iPad/iPhone just shows ALL your Numbers documents in one searchable view, and it doesn’t matter where exactly on the Mac they may be?)



    What makes this better than other cloud solutions is that you never have to “put something” on the cloud or “retrieve it” or “sync it.” That’s very different from Dropbox etc. where cloud stuff has its own special place, and you have to think about it. You can forget that iCloud exists, and just enjoy that all your stuff is on all your devices. Type an edit in Numbers on the Mac, and the edit is on your iPad as well, without you having to do anything (you don’t even have to Save). Or, of course, you can leave iCloud off if you’d rather.



    It’s a huge and scary leap, but I’m looking forward to it! And it will be even better for the people who don’t think about these things at all... yet it just woks, and their “stuff” is where they need it all the time.



    Quote:
    Originally Posted by BUSHMAN4 View Post


    Point well taken. Not only isn't it deleted but its not encrypted. So as long as what your storing in the cloud has no value to anyone else its good.



    I don’t think we have full details on their encryption yet. The sky is not falling yet
  • Reply 13 of 112
    solipsismsolipsism Posts: 25,726member
    Quote:
    Originally Posted by screamingfist View Post


    when apple does it is is "magical" and "innovative". Novell Ifolder came out in 2001. And i guess no one has heard of dropbox.



    Some apps use Dropbox to sync apps across devices, but Dropbox is nothing like iCloud in scope.



    As for iFolder, i have absolutely no idea how you think cross-platform file sharing across computer networks is iCloud's purpose.
  • Reply 14 of 112
    solipsismsolipsism Posts: 25,726member
    Quote:
    Originally Posted by ruckerz View Post


    Just as I thought. Users are unable to save arbitrary files into the cloud. I was using iDisk as a remote data store when collecting data (on computers that were not Apple's) and then having it sync to my home compter. Now I can't do this. In order to use iCloud, the data must be saved by the app internally and use Apple's API.



    As much as I hate it, Dropbox will stick around on this computer when iDisk goes away.



    1) You have iDisk until Summer 2012. If you migrate your MobileMe account before then you use iDisk. Or you could think it through and create a free iCloud account to test the waters, running both on your Mac until you're ready to make the switch.



    2) Yes, its app focused but that doesn't mean you can't "save arbitrary files into the cloud" it just means you need an app to do it. With the APIs for Mac OS, iOS, and iCloud expect to see plenty of apps that are designed specifically for saving, syncing and backing up files regardless of their file extension.
  • Reply 15 of 112
    Quote:
    Originally Posted by screamingfist View Post


    when apple does it is is "magical" and "innovative". Novell Ifolder came out in 2001. And i guess no one has heard of dropbox.



    Your predictable and tiresome negativity about Apple notwithstanding, lots of us have heard of -- and use -- Dropbox. It is very good.



    You might have to apply a screaming fist to the side of your head and knock some sense into it, to see that iCloud takes it to the next level (assuming it works as advertised).
  • Reply 16 of 112
    noirdesirnoirdesir Posts: 1,027member
    Quote:
    Originally Posted by ruckerz View Post


    As much as I hate it, Dropbox will stick around on this computer when iDisk goes away.



    If you are willing to live with iDisk's performance and limitations, why would you hate Dropbox which has better performance and less limitations?
  • Reply 17 of 112
    noirdesirnoirdesir Posts: 1,027member
    Quote:
    Originally Posted by nagromme View Post


    You have your docs in folders on the Mac, and organize them at will... how does that work on iOS? I?m guessing your iPad/iPhone just shows ALL your Numbers documents in one searchable view, and it doesn?t matter where exactly on the Mac they may be?



    From what we know, yes you will be presented with a flat list of all your Numbers documents (you might be able to sort them by most recently accessed which could be practical enough for a lot of situations). But remember that Apple is phasing out the file system (at least they are trying), in future you'll likely don't have folders anymore but just tags (though Apple could auto-apply the current enclosing folder name as a tag) and thus you might be able to view a view of your file system restricted only to folders containing Numbers documents.



    Quote:

    What makes this better than other cloud solutions is that you never have to ?put something? on the cloud or ?retrieve it? or ?sync it.? That?s very different from Dropbox etc. where cloud stuff has its own special place, and you have to think about it. You can forget that iCloud exists, and just enjoy that all your stuff is on all your devices. Type an edit in Numbers on the Mac, and the edit is on your iPad as well, without you having to do anything (you don?t even have to Save). Or, of course, you can leave iCloud off if you?d rather.



    It?s a huge and scary leap, but I?m looking forward to it! And it will be even better for the people who don?t think about these things at all... yet it just woks, and their ?stuff? is where they need it all the time.







    I don?t think we have full details on their encryption yet. The sky is not falling yet



    I agree, this is at first mainly for people who don't churn out dozens of Excel (or Numbers) files a day. And it also initially just serves the purpose of being able to view, access and edit your documents on-the-go, be it on an iOS device, a non-Mac (ie, download files via the web interface) or your MacBook Air.
  • Reply 18 of 112
    quinneyquinney Posts: 2,528member
    Quote:
    Originally Posted by solipsism View Post






    PS: I just updated the 122MB iOS 5.0 beta 5 delta update for the iPad 2 from Yosemite National Park in 10 minutes. I love the modern world!



    You're at Yosemite and you are monkeying with your iPad?
  • Reply 19 of 112
    macslutmacslut Posts: 514member
    Quote:
    Originally Posted by quinney View Post


    You're at Yosemite and you are monkeying with your iPad?



    He's got an iPad, and he's monkeying around at Yosemite? I guess that's ok if you're at the Awahnee having El Capitinis while on the WiFi.
  • Reply 20 of 112
    jabohnjabohn Posts: 582member
    Photostream is no apparent replacement for the current Gallery. I can't see any reason why I would automatically want my photos uploaded to the cloud. I prefer to choose, and the current method looks like it takes away choice. Can your photo stream be organized into albums? Can you share photos or albums with someone else?



    Unless Apple is withholding more info on this feature, this is a big step back.
Sign In or Register to comment.