Apple shuts down ZFS open source project

Posted:
in macOS edited January 2014
Apple's efforts to support the development of ZFS, an advanced file system originally created by Sun, were officially terminated today in a notice posted by MacOS Forge.



The tersely worded message only stated that "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly."



Mac OS Forge describes itself as "dedicated to supporting the developer community surrounding open source components specific to Mac OS X." It publishes source code and an information repository about a variety of open projects Apple funds and maintains, including:



? Darwin Calendar Server (used in iCal Server)

? Darwin Streaming Server (used in QuickTime Streaming Server)

? libdispatch (used in Grand Central Dispatch)

? WebKit (Used in Safari)

? XQuartz (used in Apple's X11 offering)

? and until recently, ZFS.



ZFS mania



Apple's interest in porting ZFS was first signaled in early 2006 when it contacted Sun's OpenSolaris project; By August 2007 an early, read-only port of ZFS was published on Mac OS Forge and command line support was added to Mac OS X Leopard.



Comments by Sun executives had tipped of wild speculation that ZFS would become the default file system of Mac OS X, and pundits pounced upon the idea that Apple's own technology was terrible and that anything it could replace with from outside sources would solve lots of problems for end users. The reality was that Mac OS X and third party software has lots of dependancies upon HFS+, and that ZFS really offered the greatest potential for server users. Most home Mac users don't even have multiple hard drives to pool with ZFS.



In February of this year, AppleInsider reported on Apple's internal efforts to add new read/write ZFS features to Mac OS X Snow Leopard Server and support these in Disk Admin's graphical user interface. The new feature was also being publicly promoted in the company's marketing of Snow Leopard Server.



By June however, Apple had scrubbed all mention of ZFS from its website and the feature disappeared from developer builds.



Stick a fork in it



Behind Apple's backtracking on ZFS is Oracle's announcement in April 2009 to buy Sun. While this should have no impact on other Sun technologies Apple has borrowed from OpenSolaris, such as DTrace, or other open source packages maintained by Sun under the GPL, such as MySQL, Sun's ownership and stewardship of ZFS is at risk because Oracle already has its own advanced, open source file system: BTRFS.



In addition to Oracle's unlikely desire to fund the ongoing development of two overlapping new file systems, Sun's ZFS had already come under fire for patent infringement from NetApp as part of a patent war instigated by Sun.



NetApp reported that ZFS not only infringes its WAFL storage patents, but that Sun intentionally designed ZFS to provide features unique to NetApp's WAFL, which Sun itself described it its marketing as "the first commercial file system to use the copy-on-write tree of blocks approach to file system consistency."



This leaves Apple with an unfinished, patent-encumbered file system and without an enterprise class partner to work with in developing the future of ZFS. Were Apple to develop ZFS on its own, the technology would likely be relegated to pariah status by the rest of the industry.



It remains to be seen whether Apple will begin working with Oracle to port the similar BTRFS to Mac OS X, or simply continue to add new features to HFS+ while monitoring the landscape for promising new file system options. In any case, ZFS appears to be very dead.







Daniel Eran Dilger is the author of "Snow Leopard Server (Developer Reference)," a new book from Wiley available now for pre-order. The book edited out all mention of ZFS.
«134

Comments

  • Reply 1 of 72
    bigpicsbigpics Posts: 1,379member
    Quote:
    Originally Posted by AppleInsider View Post


    Apple's efforts to support the development of ZFS, an advanced file system originally created by Sun, were officially terminated today in a notice posted by MacOS Forge.



    The tersely worded message only stated that "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly."



    Alas, ZFS, we in Mac land hardly knew ye.
  • Reply 2 of 72
    blastdoorblastdoor Posts: 1,954member
    It would be kind of neat if Apple tried to do something more "from scratch", although perhaps that wouldn't be very efficient/realistic. I don't see why it matters whether anyone else adopts it -- nobody else uses HFS+.
  • Reply 3 of 72
    wizard69wizard69 Posts: 12,866member
    Quote:
    Originally Posted by Blastdoor View Post


    It would be kind of neat if Apple tried to do something more "from scratch", although perhaps that wouldn't be very efficient/realistic. I don't see why it matters whether anyone else adopts it -- nobody else uses HFS+.



    The other thing here is that ZFS isn't exactly the simplest thing going. If Apple is looking for something new they need to look at the open source and Super Computing worlds. Linux gas a number of interesting file systems and there are a couple associated with super computing.



    It is unfortunate that ZFS has become encumbered with patents. Long term Apple needs to address massive data stores. Actually they need a short term hardware solution too. Actually Apple appears to be a ship without a rudder after the cancelations of XServe which was a good product.





    Dave
  • Reply 4 of 72
    physguyphysguy Posts: 915member
    Quote:
    Originally Posted by wizard69 View Post


    The other thing here is that ZFS isn't exactly the simplest thing going. If Apple is looking for something new they need to look at the open source and Super Computing worlds. Linux gas a number of interesting file systems and there are a couple associated with super computing.



    It is unfortunate that ZFS has become encumbered with patents. Long term Apple needs to address massive data stores. Actually they need a short term hardware solution too. Actually Apple appears to be a ship without a rudder after the cancelations of XServe which was a good product.





    Dave



    When was xserve cancelled?
  • Reply 5 of 72
    eriamjheriamjh Posts: 1,144member
    What a bunch of gobbledygook. RIP ZFS.
  • Reply 6 of 72
    mstonemstone Posts: 11,510member
    Quote:
    Originally Posted by physguy View Post


    When was xserve cancelled?



    He meant XServe RAID
  • Reply 7 of 72
    lessons learned. don't say bad things about the functionality of Apples software group. Otherwise you will be exterminated. Anyone seen my pet Dalek.
  • Reply 8 of 72
    mstonemstone Posts: 11,510member
    Changing file systems is a daunting undertaking because the entire OS is affected in every possible way. Both Apple and MS have updated their file systems a couple times but it does create some disruption in legacy support. Microsoft was talking about a MSSQL based file system at one point but that never happened either.
  • Reply 9 of 72
    Quote:
    Originally Posted by wizard69 View Post


    Actually Apple appears to be a ship without a rudder after the cancelations of XServe which was a good product.





    Dave



    Obviously you mean the Xraid. The xserve is currently available with 8 cores.
  • Reply 10 of 72
    Quote:
    Originally Posted by mstone View Post


    He meant XServe RAID



    No such thing, it's called XRAID.

    This is typical Apple, and that is exactly the kind of half-baked promises that keeps IT people far away from Apple.
  • Reply 11 of 72
    aplnubaplnub Posts: 2,591member
    Quote:
    Originally Posted by bloggerblog View Post


    No such thing, it's called XRAID.

    This is typical Apple, and that is exactly the kind of half-baked promises that keeps IT people far away from Apple.



    I highly doubt the file system is what keeps IT people away.



    Probably more to do with all their MSCE accreditations, surf the web job, etc. Yeah, and change is really hard for most people.
  • Reply 12 of 72
    shadowshadow Posts: 373member
    Apple did a lot of FS related changes in Snow Leopard. One of the [several] reasons behind dropping 64-bit Carbon could be the future change of the FS in mind. In Cocoa, the internal implementation of the filesystem is well abstracted and Cocoa applications will not be affected. Only a handful of UNIX tools need to be adapted to the new FS (UNIX works on different filesystems anyway).



    The next OS release could drop Carbon support altogether. This will clean the way for a new FS. If the rumored Marble UI is added, that would make Mac OS 10.7 a major overhaul.



    That said, changing the default file system by all accounts is not an easy step. But it is doable, much more on Mac OS than on Windows.



    This brings another point to the Windows vs Mac OS debate:

    No matter how good Windows 7 is, Windows is heavily anchored by legacy code. The progress on the Windows side is going to be about the same as it was during the last decade (9 years to get a stripped-down Longhorn to be finished). The last anchor for Mac OS is Carbon. When Apple breaks the chain the already decent pace of Mac OS development will accelerate even further!
  • Reply 13 of 72
    ZFS isn't dead. FreeBSD has it ported over and it's working in their 7.0 release (http://en.wikipedia.org/wiki/ZFS#FreeBSD). It might even be the standard FS in 9.0 along with GCD (which has also been ported over). All this really means is that it's going to take even longer for apple to get a next gen FS.



    Btrfs probably won't fly. It's GPL and apple is trying to get rid of their GPL code. Just look at LLVM for proof of that. Funny thing, FreeBSD is trying to get rid of dependencies on the GPL too (http://wiki.freebsd.org/BuildingFreeBSDWithClang).



    My bet is that apple will push form something like the HAMMER FS (http://en.wikipedia.org/wiki/HAMMER). It's got a BSD license and some of the same goals of ZFS. ZFS has better performance than HAMMER but I think the apple engineers can fix that.
  • Reply 14 of 72
    mariomario Posts: 346member
    Apparently the reasons were more legal in nature than technical.



    http://daringfireball.net/linked/2009/10/23/zfs



    By the way Apple is looking for file system software engineer



    http://jobs.apple.com/index.ajs?meth...wJob&RID=42559



    So we may get a brand new file system in 10.7 or 10.8.
  • Reply 15 of 72
    boogabooga Posts: 1,079member
    Quote:
    Originally Posted by maercsrats View Post


    ZFS isn't dead. FreeBSD has it ported over and it's working in their 7.0 release (http://en.wikipedia.org/wiki/ZFS#FreeBSD). It might even be the standard FS in 9.0 along with GCD (which has also been ported over). All this really means is that it's going to take even longer for apple to get a next gen FS.



    Btrfs probably won't fly. It's GPL and apple is trying to get rid of their GPL code. Just look at LLVM for proof of that. Funny thing, FreeBSD is trying to get rid of dependencies on the GPL too (http://wiki.freebsd.org/BuildingFreeBSDWithClang).



    My bet is that apple will push form something like the HAMMER FS (http://en.wikipedia.org/wiki/HAMMER). It's got a BSD license and some of the same goals of ZFS. ZFS has better performance than HAMMER but I think the apple engineers can fix that.



    Moving to LLVM has little to do with GCC, although obviously ditching GPL is a nice-to-have. LLVM gives Apple better performance, better IDE/refactoring integration, faster compiles, and easier ports to new targets than gcc. The gcc compiler was pretty nice 10 or 20 years ago, but it's showing its age.
  • Reply 16 of 72
    Quote:
    Originally Posted by AppleInsider View Post


    The book edited out all mention of ZFS.



    Ha, nice touch.



    Quote:
    Originally Posted by mstone View Post


    He meant XServe RAID



    Quote:
    Originally Posted by bloggerblog View Post


    No such thing, it's called XRAID.



    No, he meant XServe RAID. I should know, I have one. It's not called an XRAID.
  • Reply 17 of 72
    hmurchisonhmurchison Posts: 12,279member
    Quote:
    Originally Posted by shadow View Post


    Apple did a lot of FS related changes in Snow Leopard. One of the [several] reasons behind dropping 64-bit Carbon could be the future change of the FS in mind. In Cocoa, the internal implementation of the filesystem is well abstracted and Cocoa applications will not be affected. Only a handful of UNIX tools need to be adapted to the new FS (UNIX works on different filesystems anyway).



    The next OS release could drop Carbon support altogether. This will clean the way for a new FS. If the rumored Marble UI is added, that would make Mac OS 10.7 a major overhaul.



    That said, changing the default file system by all accounts is not an easy step. But it is doable, much more on Mac OS than on Windows.



    This brings another point to the Windows vs Mac OS debate:

    No matter how good Windows 7 is, Windows is heavily anchored by legacy code. The progress on the Windows side is going to be about the same as it was during the last decade (9 years to get a stripped-down Longhorn to be finished). The last anchor for Mac OS is Carbon. When Apple breaks the chain the already decent pace of Mac OS development will accelerate even further!





    Great points shadow. Apple never really articulated why they stopped supporting PPC with Snow Leopard but I've got to think that future large architectural changes like delivering a new filesystem were probably at the center of it. If Apple can divert attention to Cocoa and what you say about Cocoa abstracting filesystems much better than Carbon then it makes total sense why Apple has chosen a new path.
  • Reply 18 of 72
    mactelmactel Posts: 1,275member
    NetApp has a market capitalization of $9,666,440,000. Apple could buy them get WAFL and some nice storage technology.
  • Reply 19 of 72
    davegeedavegee Posts: 2,765member
    Quote:
    Originally Posted by John the Geek View Post


    Ha, nice touch. No, he meant XServe RAID. I should know, I have one. It's not called an XRAID.



    Yea x12 for me... well not me ..personally.. but the organization that I worked for in a previous life, we had a dozen of em.



    'No such thing, it's called XRAID.'



    Nothing I hate more than people who come off like the book of knowledge and spew nonsense that impressionable readers mistakenly take for facts.
  • Reply 20 of 72
    iqatedoiqatedo Posts: 1,612member
    Quote:
    Originally Posted by hmurchison View Post


    Great points shadow. Apple never really articulated why they stopped supporting PPC with Snow Leopard but I've got to think that future large architectural changes like delivering a new filesystem were probably at the center of it. If Apple can divert attention to Cocoa and what you say about Cocoa abstracting filesystems much better than Carbon then it makes total sense why Apple has chosen a new path.



    My thanks too. I know so little about file systems but am interested. (The first post too of our new member maercsrats - thanks.)



    The bickering in some recent threads has become tiresome, it would be great for a change to be presented with reasoned arguments either way without the personal attacks.
Sign In or Register to comment.