Apple shuts down ZFS open source project
Apple's efforts to support the development of ZFS, an advanced file system originally created by Sun, were officially terminated today in a notice posted by MacOS Forge.
The tersely worded message only stated that "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly."
Mac OS Forge describes itself as "dedicated to supporting the developer community surrounding open source components specific to Mac OS X." It publishes source code and an information repository about a variety of open projects Apple funds and maintains, including:
? Darwin Calendar Server (used in iCal Server)
? Darwin Streaming Server (used in QuickTime Streaming Server)
? libdispatch (used in Grand Central Dispatch)
? WebKit (Used in Safari)
? XQuartz (used in Apple's X11 offering)
? and until recently, ZFS.
ZFS mania
Apple's interest in porting ZFS was first signaled in early 2006 when it contacted Sun's OpenSolaris project; By August 2007 an early, read-only port of ZFS was published on Mac OS Forge and command line support was added to Mac OS X Leopard.
Comments by Sun executives had tipped of wild speculation that ZFS would become the default file system of Mac OS X, and pundits pounced upon the idea that Apple's own technology was terrible and that anything it could replace with from outside sources would solve lots of problems for end users. The reality was that Mac OS X and third party software has lots of dependancies upon HFS+, and that ZFS really offered the greatest potential for server users. Most home Mac users don't even have multiple hard drives to pool with ZFS.
In February of this year, AppleInsider reported on Apple's internal efforts to add new read/write ZFS features to Mac OS X Snow Leopard Server and support these in Disk Admin's graphical user interface. The new feature was also being publicly promoted in the company's marketing of Snow Leopard Server.
By June however, Apple had scrubbed all mention of ZFS from its website and the feature disappeared from developer builds.
Stick a fork in it
Behind Apple's backtracking on ZFS is Oracle's announcement in April 2009 to buy Sun. While this should have no impact on other Sun technologies Apple has borrowed from OpenSolaris, such as DTrace, or other open source packages maintained by Sun under the GPL, such as MySQL, Sun's ownership and stewardship of ZFS is at risk because Oracle already has its own advanced, open source file system: BTRFS.
In addition to Oracle's unlikely desire to fund the ongoing development of two overlapping new file systems, Sun's ZFS had already come under fire for patent infringement from NetApp as part of a patent war instigated by Sun.
NetApp reported that ZFS not only infringes its WAFL storage patents, but that Sun intentionally designed ZFS to provide features unique to NetApp's WAFL, which Sun itself described it its marketing as "the first commercial file system to use the copy-on-write tree of blocks approach to file system consistency."
This leaves Apple with an unfinished, patent-encumbered file system and without an enterprise class partner to work with in developing the future of ZFS. Were Apple to develop ZFS on its own, the technology would likely be relegated to pariah status by the rest of the industry.
It remains to be seen whether Apple will begin working with Oracle to port the similar BTRFS to Mac OS X, or simply continue to add new features to HFS+ while monitoring the landscape for promising new file system options. In any case, ZFS appears to be very dead.
Daniel Eran Dilger is the author of "Snow Leopard Server (Developer Reference)," a new book from Wiley available now for pre-order. The book edited out all mention of ZFS.
The tersely worded message only stated that "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly."
Mac OS Forge describes itself as "dedicated to supporting the developer community surrounding open source components specific to Mac OS X." It publishes source code and an information repository about a variety of open projects Apple funds and maintains, including:
? Darwin Calendar Server (used in iCal Server)
? Darwin Streaming Server (used in QuickTime Streaming Server)
? libdispatch (used in Grand Central Dispatch)
? WebKit (Used in Safari)
? XQuartz (used in Apple's X11 offering)
? and until recently, ZFS.
ZFS mania
Apple's interest in porting ZFS was first signaled in early 2006 when it contacted Sun's OpenSolaris project; By August 2007 an early, read-only port of ZFS was published on Mac OS Forge and command line support was added to Mac OS X Leopard.
Comments by Sun executives had tipped of wild speculation that ZFS would become the default file system of Mac OS X, and pundits pounced upon the idea that Apple's own technology was terrible and that anything it could replace with from outside sources would solve lots of problems for end users. The reality was that Mac OS X and third party software has lots of dependancies upon HFS+, and that ZFS really offered the greatest potential for server users. Most home Mac users don't even have multiple hard drives to pool with ZFS.
In February of this year, AppleInsider reported on Apple's internal efforts to add new read/write ZFS features to Mac OS X Snow Leopard Server and support these in Disk Admin's graphical user interface. The new feature was also being publicly promoted in the company's marketing of Snow Leopard Server.
By June however, Apple had scrubbed all mention of ZFS from its website and the feature disappeared from developer builds.
Stick a fork in it
Behind Apple's backtracking on ZFS is Oracle's announcement in April 2009 to buy Sun. While this should have no impact on other Sun technologies Apple has borrowed from OpenSolaris, such as DTrace, or other open source packages maintained by Sun under the GPL, such as MySQL, Sun's ownership and stewardship of ZFS is at risk because Oracle already has its own advanced, open source file system: BTRFS.
In addition to Oracle's unlikely desire to fund the ongoing development of two overlapping new file systems, Sun's ZFS had already come under fire for patent infringement from NetApp as part of a patent war instigated by Sun.
NetApp reported that ZFS not only infringes its WAFL storage patents, but that Sun intentionally designed ZFS to provide features unique to NetApp's WAFL, which Sun itself described it its marketing as "the first commercial file system to use the copy-on-write tree of blocks approach to file system consistency."
This leaves Apple with an unfinished, patent-encumbered file system and without an enterprise class partner to work with in developing the future of ZFS. Were Apple to develop ZFS on its own, the technology would likely be relegated to pariah status by the rest of the industry.
It remains to be seen whether Apple will begin working with Oracle to port the similar BTRFS to Mac OS X, or simply continue to add new features to HFS+ while monitoring the landscape for promising new file system options. In any case, ZFS appears to be very dead.
Daniel Eran Dilger is the author of "Snow Leopard Server (Developer Reference)," a new book from Wiley available now for pre-order. The book edited out all mention of ZFS.
Comments
Apple's efforts to support the development of ZFS, an advanced file system originally created by Sun, were officially terminated today in a notice posted by MacOS Forge.
The tersely worded message only stated that "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly."
Alas, ZFS, we in Mac land hardly knew ye.
It would be kind of neat if Apple tried to do something more "from scratch", although perhaps that wouldn't be very efficient/realistic. I don't see why it matters whether anyone else adopts it -- nobody else uses HFS+.
The other thing here is that ZFS isn't exactly the simplest thing going. If Apple is looking for something new they need to look at the open source and Super Computing worlds. Linux gas a number of interesting file systems and there are a couple associated with super computing.
It is unfortunate that ZFS has become encumbered with patents. Long term Apple needs to address massive data stores. Actually they need a short term hardware solution too. Actually Apple appears to be a ship without a rudder after the cancelations of XServe which was a good product.
Dave
The other thing here is that ZFS isn't exactly the simplest thing going. If Apple is looking for something new they need to look at the open source and Super Computing worlds. Linux gas a number of interesting file systems and there are a couple associated with super computing.
It is unfortunate that ZFS has become encumbered with patents. Long term Apple needs to address massive data stores. Actually they need a short term hardware solution too. Actually Apple appears to be a ship without a rudder after the cancelations of XServe which was a good product.
Dave
When was xserve cancelled?
When was xserve cancelled?
He meant XServe RAID
Actually Apple appears to be a ship without a rudder after the cancelations of XServe which was a good product.
Dave
Obviously you mean the Xraid. The xserve is currently available with 8 cores.
He meant XServe RAID
No such thing, it's called XRAID.
This is typical Apple, and that is exactly the kind of half-baked promises that keeps IT people far away from Apple.
No such thing, it's called XRAID.
This is typical Apple, and that is exactly the kind of half-baked promises that keeps IT people far away from Apple.
I highly doubt the file system is what keeps IT people away.
Probably more to do with all their MSCE accreditations, surf the web job, etc. Yeah, and change is really hard for most people.
The next OS release could drop Carbon support altogether. This will clean the way for a new FS. If the rumored Marble UI is added, that would make Mac OS 10.7 a major overhaul.
That said, changing the default file system by all accounts is not an easy step. But it is doable, much more on Mac OS than on Windows.
This brings another point to the Windows vs Mac OS debate:
No matter how good Windows 7 is, Windows is heavily anchored by legacy code. The progress on the Windows side is going to be about the same as it was during the last decade (9 years to get a stripped-down Longhorn to be finished). The last anchor for Mac OS is Carbon. When Apple breaks the chain the already decent pace of Mac OS development will accelerate even further!
Btrfs probably won't fly. It's GPL and apple is trying to get rid of their GPL code. Just look at LLVM for proof of that. Funny thing, FreeBSD is trying to get rid of dependencies on the GPL too (http://wiki.freebsd.org/BuildingFreeBSDWithClang).
My bet is that apple will push form something like the HAMMER FS (http://en.wikipedia.org/wiki/HAMMER). It's got a BSD license and some of the same goals of ZFS. ZFS has better performance than HAMMER but I think the apple engineers can fix that.
http://daringfireball.net/linked/2009/10/23/zfs
By the way Apple is looking for file system software engineer
http://jobs.apple.com/index.ajs?meth...wJob&RID=42559
So we may get a brand new file system in 10.7 or 10.8.
ZFS isn't dead. FreeBSD has it ported over and it's working in their 7.0 release (http://en.wikipedia.org/wiki/ZFS#FreeBSD). It might even be the standard FS in 9.0 along with GCD (which has also been ported over). All this really means is that it's going to take even longer for apple to get a next gen FS.
Btrfs probably won't fly. It's GPL and apple is trying to get rid of their GPL code. Just look at LLVM for proof of that. Funny thing, FreeBSD is trying to get rid of dependencies on the GPL too (http://wiki.freebsd.org/BuildingFreeBSDWithClang).
My bet is that apple will push form something like the HAMMER FS (http://en.wikipedia.org/wiki/HAMMER). It's got a BSD license and some of the same goals of ZFS. ZFS has better performance than HAMMER but I think the apple engineers can fix that.
Moving to LLVM has little to do with GCC, although obviously ditching GPL is a nice-to-have. LLVM gives Apple better performance, better IDE/refactoring integration, faster compiles, and easier ports to new targets than gcc. The gcc compiler was pretty nice 10 or 20 years ago, but it's showing its age.
The book edited out all mention of ZFS.
Ha, nice touch.
He meant XServe RAID
No such thing, it's called XRAID.
No, he meant XServe RAID. I should know, I have one. It's not called an XRAID.
Apple did a lot of FS related changes in Snow Leopard. One of the [several] reasons behind dropping 64-bit Carbon could be the future change of the FS in mind. In Cocoa, the internal implementation of the filesystem is well abstracted and Cocoa applications will not be affected. Only a handful of UNIX tools need to be adapted to the new FS (UNIX works on different filesystems anyway).
The next OS release could drop Carbon support altogether. This will clean the way for a new FS. If the rumored Marble UI is added, that would make Mac OS 10.7 a major overhaul.
That said, changing the default file system by all accounts is not an easy step. But it is doable, much more on Mac OS than on Windows.
This brings another point to the Windows vs Mac OS debate:
No matter how good Windows 7 is, Windows is heavily anchored by legacy code. The progress on the Windows side is going to be about the same as it was during the last decade (9 years to get a stripped-down Longhorn to be finished). The last anchor for Mac OS is Carbon. When Apple breaks the chain the already decent pace of Mac OS development will accelerate even further!
Great points shadow. Apple never really articulated why they stopped supporting PPC with Snow Leopard but I've got to think that future large architectural changes like delivering a new filesystem were probably at the center of it. If Apple can divert attention to Cocoa and what you say about Cocoa abstracting filesystems much better than Carbon then it makes total sense why Apple has chosen a new path.
Ha, nice touch. No, he meant XServe RAID. I should know, I have one. It's not called an XRAID.
Yea x12 for me... well not me ..personally.. but the organization that I worked for in a previous life, we had a dozen of em.
'No such thing, it's called XRAID.'
Nothing I hate more than people who come off like the book of knowledge and spew nonsense that impressionable readers mistakenly take for facts.
Great points shadow. Apple never really articulated why they stopped supporting PPC with Snow Leopard but I've got to think that future large architectural changes like delivering a new filesystem were probably at the center of it. If Apple can divert attention to Cocoa and what you say about Cocoa abstracting filesystems much better than Carbon then it makes total sense why Apple has chosen a new path.
My thanks too. I know so little about file systems but am interested. (The first post too of our new member maercsrats - thanks.)
The bickering in some recent threads has become tiresome, it would be great for a change to be presented with reasoned arguments either way without the personal attacks.