Mac OS X Snow Leopard Server to pioneer ZFS ahead of desktop - Page 2

First of all let me say that this article seriously underestimates ZFS with regards to quality and feature set.

I've been using ZFS on my mac for a month now and I never want to go back again. Yes, there are some UX issues with finder and trash, but that's worth living with.

Check out my blog post and better yet, give ZFS a try - if you like your data, you won't be disappointed.

http://blog.igorminar.com/2009/01/us...-os-x-105.html

Indeed - you have to not only ensure that it works with the new API (which may be fairly trivial), but also that older apps that use legacy, more direct-to-metal HFS accessors still work as well. That can be... a pain.

Of course, the sooner you deprecate those old APIs (*cough*Carbon*cough*) and drop support for them, the faster you can do this cleanly... but given the realities of the market, 'fast' is still going to be measured in years.

Our military is not using Cyrus either.

Then your comment about them not using Apple Mail Server was given... why? Proof that they don't use MacOS X Server 10.2?

What about Mac OS X Server do you find so unstable and not robust? What scale are you using it at? I ask because it seems your info on it (or at least the mail services) is out of date, and your assertions are at odds with my own experiences.

And... I hate to say this, but apparently they're using Apache as well, at least for their top externally exposed site:

http://uptime.netcraft.com/up/graph?...nselink.mil%2F

(I mean, what are they going to use instead, IIS?)

Nope, apparently... WebSTAR! Check out the OS: http://uptime.netcraft.com/up/graph?...ww.army.mil%2F

Air Force: Apache again: http://uptime.netcraft.com/up/graph?...w.airforce.com

Granted, these are all externally facing sites with (one would assume) non-critical data on them, internal sites almost undoubtedly differ.

OK, now things have gotten to the point where I have to say something.

First, there are many reasons why ZFS is not ready for consumers. First, ZFS is relatively slow. In an enterprise environment the speed is a calculated tradeoff for other features not needed by the average customer like storage pools and redundancy, but even Sun's own ZFS implementation is still slower than some older filesystems like HFS+, and an order of magnitude slower than newer filesystems like NTFS. Sun has publicly stated on its dev forums that it is working on the speed, but consider what ZFS has to do and it's not a trivial task. They are now moving in the driver direction--offload some work done by ZFS onto the disk driver when it has the capabilities, but many low-end hard drives (read: cheap) don't have the hardware features, or sufficient hardware, to do it. Perhaps Apple will find hard drives for their server solutions that support the in-hardware checksumming for ZFS.

Second, there are no APIs in userspace that allow users to write to the "metal". Any program that does (mainly disk editor programs like DiskWarrior) is not using any API that Apple wrote, they are writing to the raw block device directly. Of course those programs will have to change, and it's a big reason Apple is most likely not including ZFS in a consumer OS--there'd be no way to recover a drive once it fails (and hardware does fail).

Third, snapshots are not going to be used by TimeMachine anytime soon. Snapshots sound nifty and their ZFS implementation is very good, but in reality they are a big drain on the filesystem when used for backup. Sun publicly states in its ZFS userguide that snapshots are NOT a backup solution, for a variety of reasons (some stated here before). One big problem is that snapshots for directories are all-or-nothing. That is, the entire directory snapshot must be restored to restore a single file. So single file snapshots would have to be used for a backup solution, but single-file snapshots are much more disk intensive--creating a data structure and snapshotting every file on the drive will be pretty IO intensive.

Lastly, the so called "Carbon" Filesystem APIs referred to above are actually used heavily in Mac OS X, and are not Carbon APIs but rather CoreServices APIs. Cocoa in fact uses them extensively, as you can see by simply disassembling the Cocoa framework. FSRefs were invented in the OS 9 days to be filesystem independant opaque objects for a filesystem inode, and they perform quite admirably. There are many APIs available for FSRefs not available at the BSD layer, such as FSCopyObject (there is no such thing as copying a file at the vnode layer), FSMoveObject (again, only renames exist in the vnode world), etc. These are all wrappers used heavily by the Finder and Cocoa. So there's no f'ing way Apple is deprecating them, and in fact as of last WWDC was encouraging developers to use them pretty heavily. Leopard introduces no new filesystem API, only additions to the FSRef APIs and a new Alias Manager replacement that uses more modern data structures (but still uses FSRefs underneath when it has to).

Having said all this, I wish ZFS were an option for the client version of Mac OS X, and it wouldn't surprise me if they do make it available in an inline after 10.6 ships.

Fair enough - FSRefs et al were actually what I was referring to as 'closer to' the metal, as they still have a lot of pre-OS X semantics such as explicit resource forks. It was my understanding that their *direct* use (ie, not through the Carbon/Cocoa APIs) was being frowned upon. Interesting that it's not. Thanks for the info.

I'm figuring 10.7. You eloquently state why there must be a lot more thought put into a Mac ZFS solution for consumers. Naturally the speed will improve but I don't think we're quite there yet for consumer needs but give it another 2.5 years or so and the pitch to consumers will be a bit easier with them sitting on gigabytes of family media that they don't want to lose.

The snapshot issue was a forehead slapper to me. I should know better. I had totally forgotten that the whole snapshot must be restored. I wonder if Apple will be able to leverage snapshots yet still maintain the granularity to pull individual files from the system. It may just have to be a two headed beast. I can certainly see why they aren't attempting to push ZFS for Snow Leopard desktop. Consumers have little patiences for some of the gotchas that sys admins deal with routinely.

I can certainly wait for it to be done right or for a homegrown solution to magically appear.

I am forced to agree. Plus with the unknowns about application, finder, TM, file vault, and disk utility integration... It may be a while yet... I still want it now though...

Apple has a unique marketing position if they create some unique networking capabilities on the client though - for example as I mentioned copy on write is sweet!

I bet there will be something for developers... a kit or something to help this transition in client 10.6.

I believe your comments on ZFS might be right but the statement that Apple encourages the use of FSRefs puzzles me quite a bit. I remember reading somewhere the opposite statement. SL FileManager is supposed to implement all functionality currently not available for Cocoa so the developers will never need to use non-Cocoa API. Assuming you know what you are talking about, the only way to combine the two statements could look as follows:

Apple is implementing Cocoa API for all filesystem-related functionality previously available from Carbon or lower-level APIs only. As a first step those might be implemented as wrappers around those functions. Once this is completed, Apple is free to change the underlying implementation and deprecate the previously used underlying functions at it's own discretion.

That's true to a point, but remember that Cocoa is not the end-all-be-all. Not all developers can use it. For example, the only place to set the custom icon on a file is from AppKit because it takes an NSImage. Yet it is impossible to use AppKit in daemons for reasons, and Apple did that on purpose. So some lower-level framework actually implements the custom icon creation and setting it on the file, and Cocoa just calls into it. It has to remain that way for layering purposes, because not all applications can use AppKit.

Cocoa is a high level programming environment, not suited for low level programming. And files are low-level programming. There are dozens of low level frameworks that are not "Cocoa", CoreServices being one of them. Cocoa is implemented on TOP OF CoreServices, and uses CoreServices internally. It will remain that way for a long, long time, because daemons and command line tools are much easier to implement (and in some cases, MUST be implemented) using low level APIs, and Cocoa is a high level framework.

Also remember there are huge advantages to FSRefs all of which I won't go into here--but namely, they store inode in addition to path references. Pathnames break if the user renames anything, like the folder or hard drive containing the file. That's why aliases always use FSRefs and not the "Cocoa" style NSURL/NSString pathname.

You are missing the point here. The fact that CURRENTLY you can get inode using CoreServices only does not imply that that is going to be the case forever. What I was talking about (sorry, no time to search for the link) is that Apple is going to solve this problem in Snow Leopard and provide a way for developers to create or resolve aliases, get ALL file, directory or disk properties without the need to use CoreServices, BSD or Carbon.

Another point: Cocoa ≠ AppKit. And Apple is moving everything to object-oriented, Objective-C based frameworks, besides Cocoa. It is perfectly OK to write a daemon or command line tool in Objective-C if you see any benefit from doing this.

The only thing we can be sure about that Apple will not drop FSRefs support any time soon, and will not do this without a loud prior warning. The first step they need to make is to give a more modern alternative. That said, if Apple decides to change the default boot filesystem in the future, they may decide not to support legacy API and drop some of the current CoreServices filesystem APIs. There will be alternative low-level implementations, and it is likely that Apple will enhance the BSD layer as well and make the whole system more consistent. Currently you have hard links and symbolic links in BSD, which are aliens for CoreServices functions, and you have old style aliases, with their significant advantages in some cases (like not breaking when the file is moved), which are not available from BSD and Cocoa.

I can tell you without breaking any NDA that although Apple is adding new APIs in Foundation here and there, it doesn't even come close to what's offered in Files.h. CoreServices can search filesystems, get information on mounts, return volume capabilities--the breadth of APIs in Files.h is huge.


I can tell you, flat out, Apple is not removing procedural C APIs from Mac OS X. The "moving to object oriented, Objective-C based frameworks" sentiment many Cocoaheads have is bullocks. Cocoa is here great, but C[++] is definitely alive and kicking on that other 90% of the marketshare platform. Apple knows this too. They may be adding to Cocoa (Foundation/AppKit/whatever) but they are not removing the C based APIs, and in fact Snow Leopard extends the number of C APIs that have no Objective C equivalent. Several of the new low level frameworks in Snow Leopard--those named *Core*, *Service*, etc. have no Obj-C equivalents.

And yes I know you can use ObjC in a daemon or tool. I write them for a living :-) Objective C does not lend itself well to an ultra portable daemon that talks to a driver. That's another discussion, but Apple is well aware of this, and we've had lengthy discussions with Apple about it. Rest assured CoreServices, CoreFoundation, etc. are all here to stay, and in Snow Leopard it is even more so thanks to some new frameworks I am glad for :-)

Remember, Leopard was Apple's cleaning house party--they deprecated a lot for the 64-bit transition. As one Apple engineer put it: "if it [an API] isn't deprecated in Leopard, it's here to stay, because that means we just bothered to port it to 64 bit!"

You clearly don't grasp Objective-C as a superset of C, nor the Cocoa MVC/KVC paradigm and how it's interrelated to the Mach-O dynamic messaging system.

Cocoa is a design approach to system development. It includes C, C++, ObjC, ObjC++ and how they are leveraged around this paradigm to be the best tool for the specific task.

OpenCL being a specialized version of C99 is written for parallel programming paradigms.

Cocoa APIs leveraging the first class citizen of ObjC are a set of APIs that wrap around OpenCL to interface with the rest of the Cocoa paradigm, without having to procedurally write and declare special cases for each and every portion of the OpenCL language implemented by Apple.


The whole notion of leveraging a set of ObjC Frameworks, under the auspices of the Cocoa MVC/KVC Design Model is to make your code as reusable [and once portable ala Openstep compliant] and thus extensible without reinventing the wheel by having one's business model and presentation separated.

Removing Carbon doesn't mean stripping out C. That would be like leaving the Mind without a Brain to process abstract thought via messaging and memory.

Trust me, I understand all this. First, the post above was not referring to Carbon. It explicitly said Apple is moving towards an all Objective-C framework set. The exact quote is And Apple is moving everything to object-oriented, Objective-C based frameworks, besides Cocoa. That is simply not true, and anybody who believes it wholeheartedly has had too much coffee. The procedural APIs in Mac OS X are here to stay. Carbon (HIToolbox) is going away, but what Carbon exactly is has never been losely defined. If you define it as "every non-libc framework that uses a C API" then you are grossly mistaken, since Apple engineers have publicly stated on mailing lists, blogs and at WWDC that ONLY HIToolbox is going away--the rest of the procedural C APIs are in Leopard, and are in fact being extended in Snow Leopard.

What you clearly don't grasp is what Mach-O is. I have no clue what you mean "how Cocoa is interrelated to the Mach-O dynamic messaging system". Mach-o is a file format for executable files that is specific to Mac OS X. Objective-C is not specific to Mac OS X. Objective C has very little to do with Mach-O except it needs to store its runtime information in a special section of the executable, but you can do the same thing with ELF. If what you said is true, nobody except Mac programmers can use Objective-C which is what every Objective C hater has been complaining (unfairly) about. I've disassembled the Mach-o file format's gory details like load commands and its torrid relationship with dyld, and trust me, it was around long ago before Cocoa.


I never said so. Look in my post and you'll never see the word Carbon. Please re-read things then post again.

Trust me I know what everybody means here—I write a lot of code for a living. I think people have wrong information, that's all. In summary, Objective-C is not going to replace the C APIs like SystemConfiguration or CoreServices, FSRefs are not deprecated/old technology and are not going away at all, and ZFS is not going to become the default filesystem for a variety of widely known issues. That's the summary of all my posts here.

Don't mind mdriftmeyer. I don't think he *really* knows how to read.