Defragging, RAID 1, and Journaling: The Eternal Questions

Posted:
in macOS edited January 2014
OK, I know this is sort of a hot topic on other forums, but I really need some feedback here, fellas.



I am replacing my 2 ASIP OS 9 Macs with OS X Servers. I am getting 2 QS 867 (dual G4) towers, each with 1 GB of RAM and ATTO Ultra160 SCSI cards. No XServes yet- Maybe next year (long story). Anyway, I need to get to the bottom of 3 issues:



1) Should I turn journaling on? What will it REALLY do for me?



2) Do I need to defrag my drives if I use OS X Server? I have a 200 GB hardware RAID (Lacie TX12000), and I have a couple 36 GB software RAIDs (mirrors). All my file systems are HSF+.



If I SHOULD defrag, can I do it from the terminal, or do I an app like need Norton, or Drive 10? SOmebody told me that "OS X doesnt need defragging", but I can't belive this.



FYI: I work in publishing, where artists push 100+ MB files all over my LAN all day long!





Specs: My servers are on a fast switched 10/100 LAN. I have a small network of 50 Mac OS 9/X users and 5 PC users on my network. All my RAIDs are mirrors (level 1). My servers are mainly file servers. I run a simple intranet web site also.

Comments

  • Reply 1 of 17
    1) Journaling prevents filesystem corruption with a near-negligible speed hit. This is important because you risk damage to the system whenever you force-quit an app, an app crashes, or the system itself crashes. I tested several disk operations on files from a few hundred KB up to over 1 GB. The difference in speed is barely measurable. Does journaling really help anything? Absolutely! In my experience, Mac OS X has a habit of damaging its filesystem with surprising regularity. Damage is usually minimal, but it can sometimes cause some otherwise unexplainable strange behaviors. The only caveat is that you need to be sure that the disk is fully repaired and in excellent condition before enabling journaling because once enabled, the system will treat any existing damage as though it is normal data.



    2) If you are using HFS+ (which you said you are) then Mac OS X is just as susceptible to fragmentation as Mac OS 9 was. If you didn't have any problems in 9, though, you shouldn't have any problems with X. Should you want to defrag, you will need to get a third party app. There are no defrag tools included with mac OS X.
  • Reply 2 of 17
    wmfwmf Posts: 1,164member
    Force-quits and app crashes cannot cause filesystem corruption in Unix.
  • Reply 3 of 17
    OK, guys, I appreciate the feedback. I am scratching my head here. Everytime I ask this question, I get mixed answers!



    "To defrag or not to defrag: that is the question"



    Looks like I opened a can-o-worms!



    Let me be more specific:



    1) Do hardware RAIDS need to be defragged? (I have a Lacie TX12000)



    2) Can hardware RAIDs (level 3 or 5) utilize OS X's new journaling feature?



    3) Do software RAIDs (level 1) need to be defragged? (I have external SCSI RAID mirrors)



    3) Can software RAIDs (level 1) utilize OS X's new journaling feature?





    As I transition my ASIP servers to OS X, what tools and utilities should I get to defrag?



    Should I defrag in OS 9 or do it in OS X?



    The BIG Question: Why does this happen:



    I do a clean install of X.

    I boot into 9 and run Norton.

    Norton thinks the X drive has MAJOR fragmentation.

    I defrag in OS 9 using Norton.

    I boot in X and use it for a month.

    I boot back into 9.

    I run Norton to be safe.

    Norton thinks the disk is horribly fragmented again already!



    Huh? It's like a ping-pong game!
  • Reply 4 of 17
    [quote]Originally posted by dstranathan:

    <strong>I do a clean install of X.

    I boot into 9 and run Norton.

    Norton thinks the X drive has MAJOR fragmentation. </strong><hr></blockquote>



    I am not an authority on this, but I remember reading that one of the reasons an X disk looks fragmented to h3ll in OS 9 is due to the underlying Unixness of OS X. I think a Unix based system actually installs files in completely different ways than the traditional OS 9 - a way which looks like utter chaos to OS 9.



    I guess that wouldn't be the case with non-system related stuff. Of course a large photoshop file you would expect to all be in 1 place. Those are the files you would want to keep specifically defragged - I personally keep all my docs on a completely different drive (RAID 1 actually).



    So I don't think that seeing fragmentation in OS 9 on an X system disk is an accurate gauge of fragmentation which will degrade performance.



    I would do some searching around - I can't remember if I was reading about this topic on the site or somewhere else.
  • Reply 5 of 17
    wmfwmf Posts: 1,164member
    RAID has nothing to do with journaling or fragmentation.



    Journaling is always good, so turn it on.



    I don't know about fragmentation.
  • Reply 6 of 17
    rbrrbr Posts: 631member
    I have a RAID 0 (striped) array with OS X v10.2.3. Do you have any experience with defraging such an array?
  • Reply 7 of 17
    mikemike Posts: 138member
    Hardware RAID will appear to the OS as a single hard drive. As far as the OS is concerned you have one big hard drive and the raid card handles all the overhead. Also, journaling does have more overhead and is slower. On a file server you probably will not notice much difference in speed but on an enteprise level database server there can be a HUGE hit in performance but the hit is well worth it if you care about your data.



    Journaling can protect you from data corrouption especially in the event of a server crash. Do it!



    As far as defragmentation: I'm really not sure since we only run Linux and Windows Servers. Windows obviously needs defragged but our Linux servers do not.



    Also, I would HIGHLY recommend only running hardware raid for a business class server. It may be nice to save $3,000 or so by not going with hardware raid but if you ever loose your software raid you'll wish you would have spent the $3,000.
  • Reply 8 of 17
    mikemike Posts: 138member
    FYI...



    "PlusOptimizer cannot optimize any disks shared with Mac OS X while running under Classic. PlusOptimizer will not optimize UFS disks. For disks with Mac OS X 10.2.2 installed, it is not recommended that you run PlusOptimizer on disks that have journaling turned on. If you feel it is necessary to run PlusOptimizer on a disk with journaling turned on, you must anchor the journal file prior to running PlusOptimizer. Contact our Technical Support department if you need further assistance. "
  • Reply 9 of 17
    [quote]Originally posted by Mike:

    <strong>



    Windows obviously needs defragged but our Linux servers do not.</strong><hr></blockquote>



    I thought NTFS did NOT need defragging? Isn't NTFS better than HSF+ anyways?



    Why don't LINUX volumes/drives need defragging?
  • Reply 10 of 17
    [quote]Originally posted by Mike:

    <strong>Hardware RAID will appear to the OS as a single hard drive. </strong><hr></blockquote>



    The built in software RAID also appears as one hard drive - at least to the end user. Do you mean a hardware RAID literally isn't seen as more than 1 drive by the OS?



    [quote]

    Journaling can protect you from data corrouption especially in the event of a server crash. Do it!<hr></blockquote>



    How exactly do you recover from data corruption? How does the journaling system know the data is corrupt? Does it take any intervention on the user end to recover from data corruption?
  • Reply 11 of 17
    [quote]Originally posted by wmf:

    <strong>

    Journaling is always good, so turn it on.

    </strong><hr></blockquote>



    True for a desktop system. Somewhat true for a laptop...



    Journaling involves more writes to disk (as the journal gets written) As such, you will probably end up using a lot more battery on a journaled system. If you leave your laptop plugged in all of the time, then no problem. Otherwise, if you notice that your battery life takes a big hit after turning on journaling, you may want to switch it off and see if that makes things better.
  • Reply 12 of 17
    wmfwmf Posts: 1,164member
    [quote]Originally posted by The Pie Man:

    <strong>How exactly do you recover from data corruption? How does the journaling system know the data is corrupt? Does it take any intervention on the user end to recover from data corruption?</strong><hr></blockquote>



    OS X fixes filesystem corruption (as much as it can) automatically when you boot. Journaling allows OS X to skip this step since corruption is impossible, making your machine boot faster after a crash.
  • Reply 13 of 17
    Every filesystem suffers from fragmentation, every one, FAT/FAT32/NTFS/HFS+/EXT2-3/etc... No system is safe from it.



    Now the degree of fragmentation is mostly dertermined by the user, are you capturing video, or working with large files and lots of small files at the same time? If so you will suffer greatly...



    Think of it this way, you write 10,000 1MB files to a 10GB disk filling it up. Now you delete every other file, giving you 5GB free in 5,000 1MB chunks, now you write a 5GB file to the disk, the filesystem has to break that file into 5,000 1MB chunks, hence fragmentation. This is example that is not real world, but illustrates the point.



    Now the reason to defragment would be to move 2,500 of those file around so you have 5GB of continuous space available, so you can write you 5GB file as 1 fragment, the file system can't do this on the fly, it'd take too much time, just think of how long it take to defrag a 10gb disk, or heaven forbid a 160GB disk...



    The performance gains of defraging a drive depend on how bad the fragmentation and the type of data and use of the user, but it definitly helps on file search performance as most defrag utilities will try to place the folder/directory structure at the begining of the disk where disk read performance is the highest, and having all folder structures consolidated saves head movement...



    [ 01-19-2003: Message edited by: biaachmonkie ]</p>
  • Reply 14 of 17
    [quote]Originally posted by wmf:

    <strong>OS X fixes filesystem corruption (as much as it can) automatically when you boot. Journaling allows OS X to skip this step since corruption is impossible, making your machine boot faster after a crash.</strong><hr></blockquote>



    Ahhh ok - I was thinking "data" corruption when people were actually talking about "filesystem" corruption. There is a big difference in my mind.



    Sorry for taking this thread off-topic.



    [ 01-19-2003: Message edited by: The Pie Man ]</p>
  • Reply 15 of 17
    mikemike Posts: 138member
    I'll answer a few of the questions asked...



    [quote]Originally posted by dstranathan:

    <strong>



    I thought NTFS did NOT need defragging? Isn't NTFS better than HSF+ anyways?



    Why don't LINUX volumes/drives need defragging?</strong><hr></blockquote>



    NTFS does need defragging. NTFS can get extrememly fragmented. As far as Linux volumes not ever needing defragging...I honestly don't know...I just know that all our Linux servers have never needed it ;-) Also, I'm not sure about HFS+ as we don't use any Apple servers.



    [quote]Originally posted by The Pie Man:

    <strong>



    The built in software RAID also appears as one hard drive - at least to the end user. Do you mean a hardware RAID literally isn't seen as more than 1 drive by the OS?



    How exactly do you recover from data corruption? How does the journaling system know the data is corrupt? Does it take any intervention on the user end to recover from data corruption?</strong><hr></blockquote>



    Yes It is just my strong opinion that if a person is concerned enough to run RAID on a file server then they should be running hardware raid. However, please do not let the use of raid replace the necessity of performing backups!



    I have never personally seen an ext3 reiserfs system get corrupted. Basically, the journaling system has checks and balances in place to make sure all data is written to volume. Also, if you force-crash a linux system running an ext3 system during writes and watch it during boot you will see it checking the file system and making corrections to the volume as needed based on the journal.



    As far as recovering a corrupt volume on a server...NEVER let yourself get to this point. If your data is that important to you then you had better be running hardware raid AND perform routine backups. We have never been able to recover a corrupt NTFS volume.



    Also fragmentation on an EXT3 reiserfs file system is EXTREMEMLY unlikely. Managment of an est3 filesystem on a server isn't like an NTFS system. Generally speaking defrag is a scheduled task on an NTFS system.
  • Reply 16 of 17
    Do hardware RAIDs defrag themselves (or otherwise prevent fragmentation)?
  • Reply 17 of 17
    wmfwmf Posts: 1,164member
    As I already said, RAID has no effect on fragmentation. A RAID array appears as a single regular hard disk to the OS.
Sign In or Register to comment.