There's one advantage I'd see. Say you've a big networked drive sat on a network using a ZFS pool. How cool would it be to add that pool in to your networked iPod, iPhone or AppleTV and seamlessly have a huge media library to use. Or even viceversa so you no longer have to sync data between two drives.
Even if all things were equal; ZFS should provide a performance boost. I like HFS+ but it's carrying a *lot* of legacy code (6800 code emulated into powerPC emulated to x86). I'm only loosely familiar with ZFS but I'm mostly impressed (certainly hard to use up a 128 bit file system and it's capacity to "nest" file systems means it can be both efficient and virtually inexhaustable; in that small files can stay small -- generally, increasing the number of bits for a file system also increases the size of the smallest sector).
K
For the restless souls who want to know what ZFS is and can do - check out this 7-segment presentation given by one of the guys "behind" ZFS.
The sections on performance, data security, snapshots (for time machine), block sizes, and one of the bigger, and I think key features, storage pooling. Then there are these funny references to a certain O/S and what it does (today) to prevent [silent] data corruption.
I haven't heard it mentioned so I'll just ask here. Is this a joint effort between Apple and SUN to marry ZFS to OS X, or is Apple going it alone with the ZFS open source?
If it's a collaboration bewteen Apple and SUN will the source code have to be published (Especially if it boots ZFS)?
(I apologize in advance for this identical post on MacRumors)
Now if only they'd include ext2/3 support, I'd actually be able to format external drives in a format that's rw on all my OSs and supports files bigger than 4gb! :-p
ZFS employs 256 bit checksums end-to-end to validate data stored under its protection. Most filesystems (and you know who you are) depend on the underlying hardware to detect corrupted data and then can only nag about it if they get such a message. Every block in a ZFS filesystem has a checksum associated with it. If ZFS detects a checksum mismatch on a raidz or mirrored filesystem, it will actively reconstruct the block from the available redundancy and go on about its job.
4. fsck off, fsck
fsck has been voted out of the house. We don't need it anymore. Because ZFS data are always consistent on disk, don't be afraid to yank out those power cords if you feel like it. Your ZFS filesystems will never require you to enter the superuser password for maintenance mode.
Don't pull out those power cords, as no filesystem can ever fix a head crash. And all the fancy RAID stuff only works in case you actually activate it and lose a portion of your disk space in the process.
As long as writing data to the disk takes a non-zero amount of time, the statement that data is always consistent on disk is not universally true.
Hmmmm. . . . Do I smell a .mac upgrade in the works? Steve did mention it last week. . . .
Sorry, I meant 'online' in the video editing sense of the word not the net sense.
I was having a discussion on the RED site about needing my assets online (ie. available to my desktop machine) at all times and for several years in the future. Rather than engaging in tape back-ups (yuck) we should be able to add (incredibly cheap) storage at regular intervals and have the system sort out back-ups etc. Spookily I mentioned ZFS as a possibility.
I haven't heard it mentioned so I'll just ask here. Is this a joint effort between Apple and SUN to marry ZFS to OS X, or is Apple going it alone with the ZFS open source?
If it's a collaboration bewteen Apple and SUN will the source code have to be published (Especially if it boots ZFS)?
(I apologize in advance for this identical post on MacRumors)
I would have to say it is a collaboration. If Apple was going to use the Open Source code, why why Sun's CEO have knowledge of all of this?
Unclear on the implications on publishing, though. . .
Darwin is not a 'real server' OS. It's not tuned like Solaris is for massively parallel workloads.
I'm using the latest Open Solaris right now, and it's much more powerful in many respects than OS X/Darwin.
Solaris and Open Solaris both have lots of GNU utils now, also. (SSH, gcc, bash and on and on).
Quote:
Originally Posted by ZachPruckowski
Basically every OS runs Java. When something doesn't have a Java runtime, that's notable.
There's no way that you can effortlessly swap out a kernel. There's a lot of interfacing between the kernel and the userspace.
No offense, but by your logic, Apple uses bash, gcc, and about 80 other GNU utilities. Maybe they're switching to Hrud. Or they use OpenSSH, maybe they're switching to a pure OpenBSD kernel. Sorry, but that doesn't really hold weight.
Not to mention, what does the OpenSolaris kernel have that Darwin doesn't?
I'm just looking at getting a mac (vista's a dead end as far as I am concerned!) and wondering what all this is about - read the info links about the ZFS stuff and it looks alright. in the grand sceme of things how will this change to ZFS affect the users on different levels, say from casual email/word processor, daily user with several apps (wp/photoshop/video edit etc) and finally to the developer/coder types? my guess is only the latter will really benefit from it fully, wont they? and the others less so - but please enlighten me otherwise. and I'm assuming its something that, say, an existing user can update their current OS to or would that be all wrong too?
I've done this test and then used vi to delete random large chunks of data (simulating a read head bounce) and ZFS will recover. You do have to creat the pool as raidz, of course. ZFS is specifically meant to self-heal when a member of a raid-z array dies. If you do the new double-parity raidz2 option, you can have 2 disks fail at once and still get by.
ZFS claims that the data is always consistent on disk because it doesn't record that the file is written until it's been checksummed and verified (few other filesystems checksum data on the fly). Also, writes are transactional, which means that rather than the write being something like 'read,modify,overwrite' it's more like 'read,modify,write changes to new blocks that references the old ones'. This is also like the mechanism that allows nice snapshots on ZFS pools.
So, you are correct that the FS can't fix a head crash. All it can do (provided you mirror or raidz(2)) is protect your data from going away in case a head does crash. 'There are only two kinds of disks; those that have crashed and those that are about to'.
Quote:
Originally Posted by synp
Don't pull out those power cords, as no filesystem can ever fix a head crash. And all the fancy RAID stuff only works in case you actually activate it and lose a portion of your disk space in the process.
As long as writing data to the disk takes a non-zero amount of time, the statement that data is always consistent on disk is not universally true.
Yes, ZFS (+RAIDZ) makes large JBOD arrays safe enough for data you don't want to lose but don't want to futz around with which RAID type to do plus provisioning it and worring about what the heck you need to do to add a drive and so forth...which why you have a JBOD array vs a RAID in the first place. Ease of use and expansion. That alone is worth using ZFS and it does more.
ZFS would let time machine work block-by-block instead of file by file. Imagine if I have my photo library backed up. If it's in a file-by-file set-up, then when I change one tiny little thing (tags, name, whatever), it will re-backup the whole photo (1-2 MB). Using ZFS, it'll re-backup a block (like a few KB or so).
Really? Time Machine (on HFS+ anyway) doesn't just back up the delta? It backs up the entire changed file?
If so, TimeMachine really would see a huge advantage as an interface over ZFS snapshots.
If they want deep enterprise penetration (which they haven't said yet), it would make some sense to at least consider putting a beefier server OS under OS X. Maybe I'm just engaging in some wishful thinking
Quote:
Originally Posted by Slewis
But they have no reason to use Solaris so it still makes no sense, especially with the GPLv3.
Apple maintains their own kernel, a hybrid kernel design using Mach and BSD that does exactly what they need it to do, nothing less and nothing more and have absolutely no reason to switch kernels to a kernel that they don't even maintain themselves, and even if they modify it they still have to republish it in the exact way that the Free Software Foundation wants them to.
Comments
There's one advantage I'd see. Say you've a big networked drive sat on a network using a ZFS pool. How cool would it be to add that pool in to your networked iPod, iPhone or AppleTV and seamlessly have a huge media library to use. Or even viceversa so you no longer have to sync data between two drives.
Now, that's a very interesting idea!
1. So easy your mom could administer it
etc.
Very good. Come back more often!
Not to mention, what does the OpenSolaris kernel have that Darwin doesn't?
Ah, you don't want to be opening up the old Mach argument.
Even if all things were equal; ZFS should provide a performance boost. I like HFS+ but it's carrying a *lot* of legacy code (6800 code emulated into powerPC emulated to x86). I'm only loosely familiar with ZFS but I'm mostly impressed (certainly hard to use up a 128 bit file system and it's capacity to "nest" file systems means it can be both efficient and virtually inexhaustable; in that small files can stay small -- generally, increasing the number of bits for a file system also increases the size of the smallest sector).
K
For the restless souls who want to know what ZFS is and can do - check out this 7-segment presentation given by one of the guys "behind" ZFS.
http://www.sun.com/software/media/re...bandwidth.html
The sections on performance, data security, snapshots (for time machine), block sizes, and one of the bigger, and I think key features, storage pooling. Then there are these funny references to a certain O/S and what it does (today) to prevent [silent] data corruption.
If it's a collaboration bewteen Apple and SUN will the source code have to be published (Especially if it boots ZFS)?
(I apologize in advance for this identical post on MacRumors)
http://www.bynkii.com/archives/2007/..._file_sys.html
Now if only they'd include ext2/3 support, I'd actually be able to format external drives in a format that's rw on all my OSs and supports files bigger than 4gb! :-p
3. Filesystem, heal thyself
ZFS employs 256 bit checksums end-to-end to validate data stored under its protection. Most filesystems (and you know who you are) depend on the underlying hardware to detect corrupted data and then can only nag about it if they get such a message. Every block in a ZFS filesystem has a checksum associated with it. If ZFS detects a checksum mismatch on a raidz or mirrored filesystem, it will actively reconstruct the block from the available redundancy and go on about its job.
4. fsck off, fsck
fsck has been voted out of the house. We don't need it anymore. Because ZFS data are always consistent on disk, don't be afraid to yank out those power cords if you feel like it. Your ZFS filesystems will never require you to enter the superuser password for maintenance mode.
Don't pull out those power cords, as no filesystem can ever fix a head crash. And all the fancy RAID stuff only works in case you actually activate it and lose a portion of your disk space in the process.
As long as writing data to the disk takes a non-zero amount of time, the statement that data is always consistent on disk is not universally true.
Hmmmm. . . . Do I smell a .mac upgrade in the works? Steve did mention it last week. . . .
Sorry, I meant 'online' in the video editing sense of the word not the net sense.
I was having a discussion on the RED site about needing my assets online (ie. available to my desktop machine) at all times and for several years in the future. Rather than engaging in tape back-ups (yuck) we should be able to add (incredibly cheap) storage at regular intervals and have the system sort out back-ups etc. Spookily I mentioned ZFS as a possibility.
I haven't heard it mentioned so I'll just ask here. Is this a joint effort between Apple and SUN to marry ZFS to OS X, or is Apple going it alone with the ZFS open source?
If it's a collaboration bewteen Apple and SUN will the source code have to be published (Especially if it boots ZFS)?
(I apologize in advance for this identical post on MacRumors)
I would have to say it is a collaboration. If Apple was going to use the Open Source code, why why Sun's CEO have knowledge of all of this?
Unclear on the implications on publishing, though. . .
I'm using the latest Open Solaris right now, and it's much more powerful in many respects than OS X/Darwin.
Solaris and Open Solaris both have lots of GNU utils now, also. (SSH, gcc, bash and on and on).
Basically every OS runs Java. When something doesn't have a Java runtime, that's notable.
There's no way that you can effortlessly swap out a kernel. There's a lot of interfacing between the kernel and the userspace.
No offense, but by your logic, Apple uses bash, gcc, and about 80 other GNU utilities. Maybe they're switching to Hrud. Or they use OpenSSH, maybe they're switching to a pure OpenBSD kernel. Sorry, but that doesn't really hold weight.
Not to mention, what does the OpenSolaris kernel have that Darwin doesn't?
mkfile 128m file1 file2 file3
zpool create testpool raidz /[path]/file1 /[path]/file2 /[path]/file3
zfs create testpool/testfs
I've done this test and then used vi to delete random large chunks of data (simulating a read head bounce) and ZFS will recover. You do have to creat the pool as raidz, of course. ZFS is specifically meant to self-heal when a member of a raid-z array dies. If you do the new double-parity raidz2 option, you can have 2 disks fail at once and still get by.
ZFS claims that the data is always consistent on disk because it doesn't record that the file is written until it's been checksummed and verified (few other filesystems checksum data on the fly). Also, writes are transactional, which means that rather than the write being something like 'read,modify,overwrite' it's more like 'read,modify,write changes to new blocks that references the old ones'. This is also like the mechanism that allows nice snapshots on ZFS pools.
So, you are correct that the FS can't fix a head crash. All it can do (provided you mirror or raidz(2)) is protect your data from going away in case a head does crash. 'There are only two kinds of disks; those that have crashed and those that are about to'.
Don't pull out those power cords, as no filesystem can ever fix a head crash. And all the fancy RAID stuff only works in case you actually activate it and lose a portion of your disk space in the process.
As long as writing data to the disk takes a non-zero amount of time, the statement that data is always consistent on disk is not universally true.
So simply because it's not a 'real server' OS, Apple MUST be swapping out the XNU.
That makes no sense.
Sebastian
ZFS would let time machine work block-by-block instead of file by file. Imagine if I have my photo library backed up. If it's in a file-by-file set-up, then when I change one tiny little thing (tags, name, whatever), it will re-backup the whole photo (1-2 MB). Using ZFS, it'll re-backup a block (like a few KB or so).
Really? Time Machine (on HFS+ anyway) doesn't just back up the delta? It backs up the entire changed file?
If so, TimeMachine really would see a huge advantage as an interface over ZFS snapshots.
But they have no reason to use Solaris so it still makes no sense, especially with the GPLv3.
Apple maintains their own kernel, a hybrid kernel design using Mach and BSD that does exactly what they need it to do, nothing less and nothing more and have absolutely no reason to switch kernels to a kernel that they don't even maintain themselves, and even if they modify it they still have to republish it in the exact way that the Free Software Foundation wants them to.
Sebastian