ZFS to play larger role in future versions of Mac OS X

[AppleInsider]

Sun Microsystems' relatively new ZFS filesystem will see rudimentary support under the soon-to-be released Mac OS X Leopard, but will eventually play a much larger role in future versions of the Apple operating system, AppleInsider has been told.

People familiar with the matter reveal that Apple on Wednesday provided developers with "ZFS on Mac OS X Preview 1.1" and associated documentation, in which the company asserted that it alone was responsible for porting the filesystem to Mac OS X.

The Cupertino-based firm also officially confirmed to developers receiving the pre-release software that Mac OS X 10.5 Leopard -- due out later this month -- will officially support ZFS, albeit restricted to a read-only implementation with which no ZFS pools or filesystems can be modified.

Developers receiving the latest ZFS preview, however, are granted access to full read and write capabilities under Leopard, including the ability to create and destruct ZFS pools and filesystems.

The developer release, those people familiar with the matter say, is a telltale sign that Apple plans further adoption of ZFS under Mac OS X as the operating system matures. It's further believed that ZFS is a candidate to eventually succeed HFS+ as the default file system for Mac OS X -- an unfulfilled claim already made in regard to Leopard by Sun's chief executive Jonathan Schwartz back in June.

Unlike Apple's progression from HFS to HFS+, ZFS is not an incremental improvement to existing technology, but rather a fundamentally new approach to data management. It aims to provide simple administration, transactional semantics, end-to-end data integrity, and immense scalability.

According to Sun's description of ZFS, the filesystem offers a pooled storage model that completely eliminates the concept of volumes and the associated problems of partitions, provisioning, wasted bandwidth and stranded storage. Thousands of filesystems can draw from a common storage pool, each one consuming only as much space as it actually needs. Therefore, Sun says, the combined I/O bandwidth of all devices in the pool is available to all filesystems at all times.

In addition, ZFS provides a feature called "disk scrubbing," which is similar to ECC memory scrubbing; it reads all data to detect latent errors in the file system while they're still correctable.

"A scrub traverses the entire storage pool to read every copy of every block, validate it against its 256-bit checksum, and repair it if necessary," the description reads. "All this happens while the storage pool is live and in use."

A more comprehensive description of ZFS, along with several other features it offers, is available on Sun's OpenSolaris website.

[ View this article at AppleInsider.com ]

[baranovich]

Does this mean that "complete" ZFS support might arrive in 10.5.5, or some such minor release???

Everything I've read about ZFS tells me that it is a great direction for Apple to in, I can't wait!

[ozzyrocks]

Currently, we are able to freely move around files that are open in an application without causing any trouble in Mac OS X (mostly, at least), which is not possible with Windows or Linux. I believe I read that this was a feature of the HFS(+). Does this mean that once ZFS is adopted as the default FS, this will no longer be possible?

[suhail]

Man… Apple is changing file-systems like there's no tomorrow!

[Booga]

Originally Posted by ozzyrocks

Currently, we are able to freely move around files that are open in an application without causing any trouble in Mac OS X (mostly, at least), which is not possible with Windows or Linux. I believe I read that this was a feature of the HFS(+). Does this mean that once ZFS is adopted as the default FS, this will no longer be possible?

This is possible in Linux and most UNIXes. In fact, most UNIX filesystems support the concept of the "inode", and if you have a reference to them you can keep using the file even after it's been "deleted". (In UNIX, the concept of "delete" is really just removing the last remaining hard link to the node, thus is termed "unlinking".) So I think ZFS will probably be better at this than HFS+.

The big thing that classic MacOS traditionally supported was the concept of a file id being the identity of the file on the file system, and the "path" being merely a convention. While somewhat similar to the inode in concept, Apple actually exposed it as the primary means to opening files in the old Toolbox. This allowed one to re-open the same file even after it moved from directory to directory without even knowing what directory it had moved to. While generally regarded as a great feature (and a feature that some of the database-oriented filesystems have also tried to use more recently), the UNIX-centric folks from NeXT won that battle fairly early in the MacOS X game and now the API is tied pretty tightly to the concept of locating a file via path.

[JeffDM]

I thought that ZFS is a detriment for single drive systems, that it only really shines in a multi-drive system. Most computers that Apple sells can only hold a single drive. Pooling an internal drive with an external USB/FW drive doesn't sound like a good idea either.

[rebel_without_a_pc]

Originally Posted by JeffDM

...Most computers that Apple sells can only hold a single drive. Pooling an internal drive with an external USB/FW drive doesn't sound like a good idea either.

By the time this file system ships standard, all of Apples' paper-thin portables will surely have multiple flash drives, maybe with RAID.
They'll have turned your iTV, iPods, home server and your Refrigerator (iFridge, anyone?) into a home media cluster.
And thats if you don't have a toaster oven.

[rtx]

I'm sure the engineering effort that went into ZFS was pretty big. In many ways it doesn't make sense for Apple to NOT use it. They can take ZFS and then dedicate their resources to features that really add value to the user - improved spotlight, etc.

I'm heard lots about ZFS, my only question at this pont is performance. Though - even if it has a disadvantage now (which I'm not claiming), probably when it's widely used better hardware specs will eliminate it. Even now, HFS+ performs poorly on directories with massive amounts of files (such as a mail server), how does ZFS stack up?

The data integrity features certainly are both welcome and impressive.

Actually, one more ZFS question. Does it run on mobile devices? Does it present any advantage there?

[mdriftmeyer]

I-node/Super I-node and their relation to V-node are standard UNIX filesystem terminology.

Ref: Advanced Programming in the UNIX Environment: W. Richard Stevens.

[ksec]

From reading around the web and Wiki, my only concerns on ZFS are
1. High CPU usage
2. Performance issues
3. Cannot Reduce Pool Size
4. No adding storage in Raid Z.

I suppose 1 and 2 will improve over time if it is not fixed by now. 3 Are Currently being working on.

I dont see 4 as a major concern for normal System usage. However it would be great for NAS or DAS alike.

Hopefully ZFS will get all this done by OSX 10.6

[MacTel]

Originally Posted by baranovich

Does this mean that "complete" ZFS support might arrive in 10.5.5, or some such minor release???

Everything I've read about ZFS tells me that it is a great direction for Apple to in, I can't wait!

I would believe so. I can't remember the specifices, but Apple has done this before with BE OS technology. Regardless, this would be a great enterprise technology for Apple to offer.

I wonder why Apple and Sun just don't merge since their engineers share great admiration for each other's works. Though Jobs did snub Java in regards to having it on the iPhone and iPod touch so he must not feel the fever.

[zunx]

Originally Posted by ozzyrocks

Currently, we are able to freely move around files that are open in an application without causing any trouble in Mac OS X (mostly, at least), which is not possible with Windows or Linux. I believe I read that this was a feature of the HFS(+). Does this mean that once ZFS is adopted as the default FS, this will no longer be possible?

Such feature does not work with some applications like Adobe Acrobat Pro 8.1.0. Try to open a PDF file, edit it (eg., use the fluorescent marker), move the file to other folder and/or change its name in the Finder and try to save it. Error and Adobe Acrobat crash! Yet, that is extremely useful to rename bibliographic PDF files "article.pdf" once you open them and mark keywords with the fluorescent marker. A shame!

[ascii]

I think it might be goog to keep HFS+ as the default because it's Apple's own filesystem and it leaves them freer to innovate in this space. If they go with ZFS it may be better at the moment, but if they want to change it at a later date they can't without breaking the standard.

[yama]

Originally Posted by baranovich

Does this mean that "complete" ZFS support might arrive in 10.5.5, or some such minor release???

Everything I've read about ZFS tells me that it is a great direction for Apple to in, I can't wait!

Sure, why not? Apple added Journaling support to 10.2.2.

[solipsism]

1) This isn't something Apple will slip into a 10.5.x update.
2) This will be used in Apple's server line long before Apple even considers using it in their Mac line.
3) Despite the throng of adamant posts several months ago regarding ZFS support coming in 10.5, it's technologically still is not ready for prime time and won't be used by Apple until at least 10.6.

Originally Posted by AppleInsider

It's further believed that ZFS is a candidate to eventually succeed HFS+ as the default operating system for Mac OS X

Should be default file system.

[DaveGee]

Originally Posted by suhail

Man… Apple is changing file-systems like there's no tomorrow!

Apple's filesystem changes since 1984:

MFS Circa 1984 - Dropped support with System 7.6.1
HFS Circa 1985 - Became default FS at intro? - Still supported
HFS+ Circa 1998 - Became default FS with OS X rollout - Still supported

Speculation:

ZFS Circa 2007 - Became default FS ????

Yep - those wild and crazy folks in the FS group are most radical indeed!

Dave

[solipsism]

Originally Posted by DaveGee

Apple's filesystem changes since 1984:

MFS Circa 1984 - Dropped support with System 7.6.1
HFS Circa 1985 - Became default FS at intro? - Still supported
HFS+ Circa 1998 - Became default FS with OS X rollout - Still supported

Speculation:

ZFS Circa 2007 - Became default FS ????

Yep - those wild and crazy folks in the FS group are most radical indeed!

Dave

Nice list. I'm bettng on ZFS for 10.6 and will first be implemented in OSX Server well before the OS X client. That leaves how may years before we see it in production?

[brianus]

Originally Posted by solipsism

Nice list. I'm bettng on ZFS for 10.6 and will first be implemented in OSX Server well before the OS X client. That leaves how may years before we see it in production?

3? I'm betting we don't see another .1 release this decade.

[kim kap sol]

Originally Posted by brianus

3? I'm betting we don't see another .1 release this decade.

Is 10.5 the new System 7?

Public Beta ... System 1
10.0 ... System 2
10.1 ... System 3
10.2 ... System 4
10.3 ... System 5
10.4 ... System 6
10.5 ... System 7!!! :O

This means we won't see any updates for...SEVEN YEARS!!! :O

[thenetimp]

You do realize ZFS is a FILE system not an OPERATING system????

I don't think Apple is going to replace BSD with Solaris...

[JeffDM]

Originally Posted by thenetimp

You do realize ZFS is a FILE system not an OPERATING system????

I don't think Apple is going to replace BSD with Solaris...

If you're responding to kim kap sol, I'm almost dead certain he's aware of that. That post was a mostly a joke.

[suhail]

Originally Posted by DaveGee

Apple's filesystem changes since 1984:

MFS Circa 1984 - Dropped support with System 7.6.1
HFS Circa 1985 - Became default FS at intro? - Still supported
HFS+ Circa 1998 - Became default FS with OS X rollout - Still supported

Speculation:

ZFS Circa 2007 - Became default FS ????

Yep - those wild and crazy folks in the FS group are most radical indeed!

Dave

Thanks for sharing that list. Didn't Apple change to a "new file-system" to support Spotlight. The main directory and file-headers had to be changed. I clearly remember something of that sort when we upgraded to the new OSX Server.
Also wasn't that one of MS main hurdles in offering a search-engine similar to Spotlight in their Vista OS?

[JeffDM]

Originally Posted by suhail

Thanks for sharing that list. Didn't Apple change to a "new file-system" to support Spotlight. The main directory and file-headers had to be changed. I clearly remember something of that sort when we upgraded to the new OSX Server.
Also wasn't that one of MS main hurdles in offering a search-engine similar to Spotlight in their Vista OS?

File systems can be updated from time to time. HFS+ may have changed a little over time. I know NTFS is at least on its fifth variation, maybe more.

[solipsism]

Originally Posted by JeffDM

I know NTFS is at least on its fifth variation, maybe more.

Someone gets a brownie.

• http://en.wikipedia.org/wiki/NTFS#Versioning

[suhail]

Originally Posted by JeffDM

File systems can be updated from time to time. HFS+ may have changed a little over time. I know NTFS is at least on its fifth variation, maybe more.

Wikipedia:
10.0 HFS plus
10.2 HFSJ (Journaling for OSX Server)
10.3 HFSJ for all OSX
10.4 HFSX+XP compatability

Alright so it was HFSX that made Spotlight possible because of the Inline Attribute Data Records.

I guess

[sc_markt]

Wonder if the patent infringement lawsuit against sun over ZFS will affect the release date of OS X?

[solipsism]

Originally Posted by sc_markt

Wonder if the patent infringement lawsuit against sun over ZFS will affect the release date of OS X?

Why? ZFS is Open Source and 10.5 will be using HFS as its file system.

[solipsism]

Originally Posted by suhail

10.4 HFSX+XP compatability
Alright so it was HFSX that made Spotlight possible because of the Inline Attribute Data Records.

"With 10.4, Apple added support for Inline Attribute Data records, something that had been a part of the Mac OS X implementation of HFS Plus since at least 10.0, but always marked as "reserved for future use".[5] Until the release of Mac OS X Server 10.4, HFS Plus only supported the standard UNIX file system permissions, however 10.4 introduced support for access control list-based file security, which is designed to be fully compatible with the file permission system used by Microsoft Windows XP and Windows Server 2003."

Thanks for the link. I hadn't released HFS had rejiggered so much with each OS release.

[sc_markt]

Originally Posted by solipsism

Why? ZFS is Open Source and 10.5 will be using HFS as its file system.

http://www.computerworld.com/action/...c=news_ts_head

[solipsism]

Originally Posted by sc_markt

http://www.computerworld.com/action/...c=news_ts_head

No biggie. Since ZFS is still read only anyway there for being little use to anyone using OS X, all Apple has to do is remove ZFS access in DIsk Utility and be done with the matter until the lawsuit is settled.

[talksense101]

September 05, 2007 (Computerworld) -- Network Appliance Inc. today announced that it has filed a patent infringement lawsuit against Sun Microsystems Inc. seeking unspecified compensatory damages and an injunction that would prohibit Sun from developing or distributing products based on its ZFS file system technology.

The suit, filed in the U.S. District Court in Lufkin, Texas, charges that the Sun ZFS technology infringes on seven NetApp patents pertaining to data processing systems and related software.

Dave Hitz, NetApps's founder and executive vice president, said the lawsuit was filed largely because Sun 18 months ago "aggressively demanded" cross-licensing fees related to the Write Anywhere File Layout (WAFL) file-system technology included in ZFS. Hitz said the cross-licensing talks were halted in April after Sun claimed that NetApp's use of WAFL infringed on Sun patents.

"Those discussions obviously went nowhere and basically fell silent," he said. "By that time, [the ZFS file system] was in the public domain. They started this discussion by asserting we infringed on their patents and asked us to pay them royalties on their patents." Earlier this year, Sun released the ZFS source code, part of its Solaris operating system, to the open source community, Hitz noted.

Sun declined to comment on the lawsuit.

Hitz said that during its negotiations, Sun did not specify which NetApp products infringed on its patents. However, he noted that Sun did say that most of those patents were gained through its $4.1 billion acquisition of Storage Technology Corp. in 2005.

NetApp has no plans to assert any claims against its own customers or Sun's customers in regards to ZFS or other patent infringement. While Hitz acknowledged that the lawsuit could have implications for the future of ZFS, he declined to elaborate on specific outcomes.

So who is suing whom? This looks like a pissing match between Sun and Netapp more than anything else. Sun threatens NetApp. NetApp counters by suing Sun. This bull shit will lead to an out-of-court settlement.

ZFS is too large for the desktop imo. I think it will be part of Apple's server strategy with the clients accessing the file system. I don't think it will be restricted to read-only. HFS+ in it's current incarnation is quite sufficient for the desktop. I fail to see any areas where it is crippling the advancement of the Mac OS.

[kcmac]

What good is read only ZFS? How will it be used?

[Brendon]

Originally Posted by AppleInsider

Developers receiving the latest ZFS preview, however, are granted access to full read and write capabilities under Leopard, including the ability to create and destruct ZFS pools and filesystems.

Read Write are already supported. It is a matter of getting the legal issues cleared up and implementing ZFS and building the frameworks or adding ZFS to the frameworks. A flip of the switch and it can take over for HFS+ (or HFSX+), and then people can begin to use its' special features. 6 months to a year, depending on what Apple has already done, and what the open source crowd has or will do and when.

Ty

[Brendon]

For those that think that this will show up in Server first may only be correct because Apple may want some real world testing before rolling this out for regular OS. Apple would be wasting money if this was only for Server. The real money is made when Apple uses new technologies to make using a computer easier, therefore this will go mainstream ASAP. There may be some Server only parts but the bulk to go mainstream and as quickly as Apple can make it happen. This would be a major strategic advantage that could help boost Apple into the Corp world, depending on what all Apple can do with the technology. "ZFS the most advanced file system on the planet, right there storing, organizing, and verifying all of your most precious photos in iPhoto. You'll never have to worry about losing any of your electronic records, your photos, your movies, even your music. ZFS checks and verifies and if it finds anything wrong, it corrects the problem before you lose some of the moments you value most. Our new file system does this for you automatically every time you use your Mac, that is how much we value your electronic records." Apple marketing - 101.

[theGAR]

But what about speed. Will all these goodies come at a speed penalty (compared to HFS+ or NTFS) ?