Exploring Time Capsule: Time Machine over the Network vs USB

[AppleInsider]

Time Capsule is billed as an ideal backup target for Time Machine. At the same time, networked drives (and particularly wireless shares) are often slower than a directly connected USB backup drive. This segment, the fifth of six exploring Time Capsule in depth, compares the pros and cons of using a solution like Time Capsule and the AirPort Extreme to perform Time Machine backups relative to using a directly attached hard drive.

Real World Tests: Network Time Machine vs USB Time Machine

Even at top speed and positioned in a rarified spectrum of bandwidth, 802.11n can't hold a candle to the throughput of a directly connected USB hard drive. Of course, connecting a USB hard drive to perform Time Machine backups is also not a brilliantly convenient setup. Additionally, Time Capsule supports Time Machine backups from multiple Macs; to do the same thing using a directly connected USB drive, you'd have to unplug the drive at regular intervals and attach it to each machine needing to be backed up.

Time Machine also deals with directly connected UBS hard drives (or Firewire or SATA drives) and network shares on Time Capsule or AirPort Extreme drives differently, making the two methods incompatible. Users have to decide between using Time Machine over the network or to a local disk; the same USB drive can't be shuttled between the two to create a single set of backups.

When using a directly connected drive, Time Machine saves its backup files in a hierarchy of folders by date. Part of the magic of Time Machine comes from the use of hard linked files, where a single data file on disk can be linked to multiple file records appearing in different folders. This enables the system to keep a full set of files for each backup session without actually duplicating the contents of the files that haven't changed. This also allows Time Machine to delete folders of backed up files that are no longer needed without coping any of the files around, yet maintain a full set of backups. Hard links were described in greater detail in the article Road to Mac OS X Leopard: Time Machine.

Apple added support for hard links to the Mac's native HFS+ file system to support Time Machine. Some file systems don't support hard links, so in order to consistently handle Time Machine backups on network shared file systems, Time Machine uses a different method of saving its files: it creates a sparse file disk image that appears to the native file system to be a standard file, but which Time Machine can internally organize as a virtual HFS+ disk with support for hard links. Inside this disk image, the hierarchy of hard linked files looks the same as that of a directly connected Time Machine disk.

There isn't a way to tell Time Machine to target a sparse file disk image of your own making however, so you can't use this method with a directly connected drive. Subsequently, the file layout between network connected drives and directly connected disks used by Time Machine are not compatible or interchangeable. That means you can't plug in a drive directly and begin a Time Machine backup, and then append further backup updates onto it after plugging the drive into a Time Capsule or AirPort Extreme as a network shared disk; Time Machine will simply copy all the files over again in parallel.

The best alternative short of that is to plug the base station into the host computer directly via Ethernet, and allow Time Machine to backup to the disk the fastest way possible for its first session. After it's complete, the base station can be unplugged and subsequent backups will occur wirelessly. This method can also be used anytime you've performed major changes and have a lot to back up, or whenever you want to accelerate your Time Machine backup session. As the numbers below indicate, its about two to four times faster to copy files over Ethernet compared to 802.11n.

USB vs Network File Sharing Shootout

For an idea of how much faster direct Gigabit Ethernet is to WiFi N or to a directly attached USB drive, we performed some tests again, twice, to see how much difference there would be between the tests. Note that wireless times tended to fluctuate wildly, while times for USB or Ethernet were far more consistent.
Directly attached to a MPB, our USB drive took 0:49 to copy the gigabyte of files.
The same drive, attached to Time Capsule via USB, took between 4:21 and 7:22 to copy via 802.11n using 5 GHz with wide channels.
Using the internal drive inside Time Capsule, it took between 5:45 and 6:30 to copy via 802.11n using 5 GHz with wide channels.
The external drive, attached to Time Capsule via USB, took between 2:19 and 2:20 to copy via Gigabit Ethernet.
The internal Time Capsule disk took between 1:39 and 1:40 to copy via Gigabit Ethernet.
The external drive, attached to a PowerMac G5 via USB, took between 0:38 and 0:43 to copy via Gigabit Ethernet, slightly faster than when it was used as a directly connected USB drive.
Clearly, wireless networking involves some mystically complex principles that result in a wide margin of performance variance: even with identical settings, files can take anywhere between 2:15 and 8:00 per gigabyte to copy. Still, WiFi also offers a level of convenience and at least fairly reasonable performance (when configured properly and in ideal conditions) that supports an increasing scope of useful applications, from AirTunes streaming to file and print sharing to Time Machine automated backups.

By tuning your WiFi network to get the most of your signal, you can expect decent performance from background backups and centralized file sharing without having to string up Ethernet cables. If you already have an Ethernet network in place, Time Capsule doesn't offer the most impressive file server performance; it's really quite disappointing for use as a Gigabit Ethernet file share host.

Remember that if you need better disk performance than Time Capsule can offer on its own, you can attach an Ethernet NAS appliance to the base station's Gigabit Ethernet LAN ports, or alternatively use Personal File Sharing on a Mac similarly connected via the Gigabit Ethernet switch. Wireless clients likely won't see any difference in their access to the file server, but wired clients will. Such an alternative is also significantly more expensive, larger, and will consume more power.

page 2 of 2: Charting Performance: USB vs 802.11n vs Gigabit Ethernet; and Gigabit NAS vs USB RAID.

Charting Performance: USB vs 802.11n vs Gigabit Ethernet

The chart below compares the copy time and megabytes per second transfer rate of the same USB hard drive when directly attached to a MacBook Pro, when connected to Time Capsule and shared as a network drive wirelessly and via Gigabit Ethernet, and when connected to a PowerMac G5 and shared over the network via Gigabit Ethernet. The second section compares the speed of copying files to the SATA drive inside Time Capsule, both wirelessly and via Gigabit Ethernet.

The fastest alternative is a dedicated server sharing files via Gigabit Ethernet. A directly connected USB drive is nearly as fast. For Time Capsule clients, the fastest option for connecting to its shared drive is via Gigabit Ethernet; that network connection is fast enough to actually expose a speed advantage of the internal SATA drive over an external USB disk, although its still only about half as fast as a dedicated server.

For wireless clients, a USB drive attached to Time Capsule is really no better nor worse than the internal SATA drive. There is little or no advantage for wireless clients in using a NAS or standalone server over the built in disk sharing of Time Capsule or the AirPort Extreme, until multiple concurrent users begin hammering the drive. That positions Time Capsule as a good solution for casual home users who want to go wireless and small office users with simple needs.



Gigabit NAS vs USB RAID

As with the AirPort Extreme, additional drives plugged into Time Capsule via USB are also shared by the base station, and appear in the Finder of client computers as sharepoints associated with the Time Capsule device. They also become available to Time Machine as backup targets. That means larger USB drives or a standalone USB RAID device can be used to expand the capacity of the base station beyond the internal drive in Time Capsule.

However, the limitations of the base station's embedded hardware in serving up file shares means that users who want to expand Time Capsule beyond its internal disk should profile their needs. It may make more sense to instead attach a self contained NAS appliance to the Time Capsule's integrated Gigabit Ethernet switch. That will delegate the file serving effort to the external NAS box itself, leaving the Time Capsule available to serve as a backup file share.

The concluding sixth segment will wrap up the discussion of the overall performance and features of Time Capsule and review its pros and cons for different types of users who may be considering a purchase or upgrade.

Previous articles related to Time Capsule and its AirPort Extreme cousin:
Exploring Time Capsule: 10/100/1000 Ethernet vs. 802.11g/n Wireless Networking
Exploring Time Capsule: WiFi 802.11n and the 5GHz band
Exploring Time Capsule: theoretical speed vs practical throughput
Exploring Time Capsule: how it fits into Apple's AirPort family
An in-depth review of Apple's 802.11n AirPort Extreme Base Station
Apple Time Capsule unboxing and preview
A Look Inside Apple's New Time Capsule
Answers to Time Capsule reader questions

[ View this article at AppleInsider.com ]

[digitalclips]

I do not have a TC so I cannot speak of its abilities but I have given up trying to use my AE plus an external drive as a network backup device for the time being even though now Apple support this function. It is simply too slow to be practical at the moment. I suspect it will work one day in the future when a few more updates occur because the theoretical transfer speeds are simply not there as of yet.

[zunx]

There is nothing like Firewire for external storage. USB is really bad for that compared to USB.

[aplnub]

Originally Posted by digitalclips

I do not have a TC so I cannot speak of its abilities but I have given up trying to use my AE plus an external drive as a network backup device for the time being even though now Apple support this function. It is simply too slow to be practical at the moment. I suspect it will work one day in the future when a few more updates occur because the theoretical transfer speeds are simply not there as of yet.

I don't think any hardware networking wise reaches the TSS in every day use unless I am wrong. I have never had wired or wireless do better than 70% (this may be a high number) in any test I have conducted when doing large file transfers on home networking equipment over CAT5e or wireless.

[sjmacko]

Even at top speed and positioned in a rarified spectrum of bandwidth, 802.11n can't hold a candle to the throughput of a directly connected USB hard drive. Of course, connecting a USB hard drive to perform Time Machine backups is also not a brilliantly convenient setup. Additionally, Time Capsule supports Time Machine backups from multiple Macs; to do the same thing using a directly connected USB drive, you'd have to unplug the drive at regular intervals and attach it to each machine needing to be backed up.

I might be missing something... It wouldn't be the first time! I have a WD MyBook plugged into my Macbook Pro by firewire. My Macbook Pro backs up to it just fine. My wife's Macbook, and the kids' iBook back up to it over the network. I do not have to unplug the drive and attach it to each of their machines.Prior to the Mybook, I used a WD portable USB drive, which also worked fine. I was under the impression that this was the norm if the backup target is running Leopard.

On a similar note, I tried to share the drive from my Sawtooth server running Tiger, and TM did not work.

Steve

[amerist]

Time Machine also deals with directly connected UBS hard drives...

[random bob, a.r.c.]

See it's just a matter of trade-offs.... For the laptop crew (that's me!), it's a hard sell.... For the desktop crew, it's nonsense! Go wired! I'm using a USB HD connected directly right now for my backups as my TC has yet to ship from amazon (got $75 off the price). I WISH TC wasn't going to be noticeably slower, but... I have also noticed that frequently, my backups (when done hourly) rarely exceed 100MB of change.

So realistically for me and the "everyday user," wireless backups are going to be just fine if they stay on the network for most of the time; TM will be doing hourly syncs and probably far less than 1GB of changes at at time.

I think that's what people need to consider along with these tests; these tests are extreme. Save for the first backup, the whole design is to keep subsequent backups minimal. Realistically, unless you're photoshopping 10-100MB files in an hour, it's probably not going to be a big deal for you. And anyway if you were doing that, you probably have a desktop machine in which case, why aren't you wired in? It's not like you're going anywhere...

Again, I WISH TM & TC would magically work as fast or near as fast as a wired connection. But then again, for the small amount of changes to my HD in that hour between backups, who's going to notice?

[bxs6408]

I don't think speed or the wall time for performing data protection backups is all that significant for most users. What is significant is the ease of use, ease of understanding and the convenience features that Time Machine offers.

The only time speed might be considered significant is the time it takes for the very first Time Machine backup. This can take many hours and of course depends on just how many GBs is to be backed up. This should be, as Apple post, anticipated and to be done say overnight.

Beyond this first backup the hourly snapshots are a real snap for most users as not too much data has to be backed up. This is where speed isn't significant and hourly snapshots is the repetitive activity for Time Machine.

The Time Capsule does take longer to respond when using the TM browser than for direct attached TM backup disks because the sparse bundle disk image has to be found and mounted. However, this should not be seen as a big deal as using the TM browser isn't something a user does that frequently.

I've now been using the Time Capsule since end of Feb 2008 and use it for backing up my PowerMac G5, my wife's MB Core Duo and my new MBA w/SSD. My wife's MB Core Duo performs its backups over WiFi (AirPort Express 802.11g) to the Time Capsule which is the router hooked to my Comcast cable modem. My MBA when connected via its USB-Ethernet dongle will backup to the Time Capsule via wire otherwise it's done WiFi (802.11n). My PM G5 backs up to the Time Capsule via GbE. It has been working well for me.

There was just once instance my wife's MB Core Duo had an issue. This was Time Machine saying it could not mount the sparse bundle disk image. I was able to resolve this by trying maybe 4 time manually and it has been working solidly ever since. I believe the cause for this was related to there being a Microwave oven in the vicinity of the MB Core Duo as I was able to induce the same failure by turning on the Microwave when Time Machine snapshot initiated. I do find this error to be a problem in as much that I had to manually intervene to resolve the problem. Apple's Time Machine should auto recover from this type of WiFi interference IMO, and I've filed a BR to Apple about this.

All in all, I find the Time Machine and its attached AirDisk to work pretty much as it's advertised to do so and do not find issue with the speed of doing backups.

Incidentally, I tested the Time Machine's full system recovery using the Time Capsule's backup data during the use of the DVD Installer and found it to work without issues. Very nice feature IMO.

[davesmall]

I'm using an Apple Extreme base station with a 500 GB external USB2 disk drive attached. The latest Time Machine and Apple Extreme software & firmware updates from Apple have enabled me to configure and run the same as Time Capsule. I've been up and running with WiFi Time Machine backups for about two weeks now. I'm backing up a MacBook Pro, a MacBook, and an iMac. Here are my experiences to date.

First of all, I had a bit of difficulty getting the base station configured to run properly. The firmware update wouldn't install until after I did a hard reset of the base station. After fiddling with the Airport Utility software I finally got it working with my Comcast cable modem. I think an actual Time Capsule would have been easier to get up and running.

I did the initial backups of all three computers over the wireless WiFi network. It took about 20 hours to backup the 130 gigabytes stored on my MacBook Pro. This happened in the background and I was able to use my computer while it was transferring data. If I had it to do over again I would do the initial backup over ethernet and then switch to wireless.

Things have gone very smoothly since the initial backups were completed. I really love this setup. For instance, it is now 9:35 AM here in Houston. I'm clicking on the Time Machine icon in my menu bar and it tells me that my last backup was completed at 9:04 AM today. I never notice that it is running. It's never more than an hour out of date so long as I'm within WiFi range.

Last week I flew to California for several days. Of course, no backups were being made while I was away from my WiFi network. When I returned Time Machine automatically resumed wireless backups. I didn't have to do a thing.

What's so cool about this is that it happens automatically in the background and you are unaware. I have much more current backups than ever before and I don't have to do a thing (once configured). It feels like magic. It's one of the best things in Leopard and I highly recommend it.

[digitalclips]

Originally Posted by davesmall

I did the initial backups of all three computers over the wireless WiFi network. It took about 20 hours to backup the 130 gigabytes stored on my MacBook Pro. This happened in the background and I was able to use my computer while it was transferring data. If I had it to do over again I would do the initial backup over ethernet and then switch to wireless.

I have the same exact configuartion as you but not the ability to wait since my wife takes her MBP to work every day and it was never finished when she had to leave..

Every attempt to make the initial back up by wire then transfer the disk to the AE failed. The MBP would simply start from square one ignoring the back up that was already done via wire.

Has anyone found how to get this to work. I am sure incremental backups would be fine wirelessly.

EDIT: I just re read the article and my answer is there! Sorry folks my bad.

[glorfindeal]

There is a much quicker way to do the initial backup. Connect the usb drive to your airport extreme and do a time machine back up excluding just about all files on your internal hard drive. All you want is to create the initial sparcebundle file. Once this backup is complete, you can attach the usb drive directly to your computer and time machine will recognize it. Remove all the excluded files and do a back up again. Time machine will use the sparcebundle file for the back up. Once you finished, reattach your usb drive to the airport extreme.


Glor

[digitalclips]

Originally Posted by glorfindeal

There is a much quicker way to do the initial backup. Connect the usb drive to your airport extreme and do a time machine back up excluding just about all files on your internal hard drive. All you want is to create the initial sparcebundle file. Once this backup is complete, you can attach the usb drive directly to your computer and time machine will recognize it. Remove all the excluded files and do a back up again. Time machine will use the sparcebundle file for the back up. Once you finished, reattach your usb drive to the airport extreme.


Glor

Cool! I suspected there would be some clever trick to this, Thanks!

How do you exclude stuff in TM? I don't see a file level exclude selection just a drive selection. Is there a feature like Carbon Copy Cloner to only select certain files?

[Wiggin]

The chart below compares the copy time and megabytes per second transfer rate of the same USB hard drive when directly attached to a MacBook Pro, when connected to Time Capsule and shared as a network drive wirelessly and via Gigabit Ethernet, and when connected to a PowerMac G5 and shared over the network via Gigabit Ethernet.

There is little or no advantage for wireless clients in using a NAS or standalone server over the built in disk sharing of Time Capsule or the AirPort Extreme

Why didn't they test backing up wirelessly to a share on the G5 (connected via Ethernet). The article stated several times that there was no advantage to the G5 share over the TC drive when the client it wireless, but it doesn't appear that they actually tested that assumption!

My tests on an Extreme show very clearly a 2-3 fold increase in speed when the wireless client copied files to/from a share connected to a mini (which was connected via Ethernet to the Extreme) vs a drive connected to the Extreme's USB port. The assumption that the wireless speed is the only bottleneck is untrue. The Extreme's (and I assume TC's) USB port controller/hard drive are also very much a bottle neck.

[glorfindeal]

Originally Posted by digitalclips

Cool! I suspected there would be some clever trick to this, Thanks!

How do you exclude stuff in TM? I don't see a file level exclude selection just a drive selection. Is there a feature like Carbon Copy Cloner to only select certain files?

There should be a '+' and '-' button on the bottom of the screen (You have to hit the options button I believe). If you hit the '+' then you can select the file/folder/drive to exclude from the backup. If you select something from the excluded list and hit the '-' key, then it will include it in the backup.


Glor

[digitalclips]

Originally Posted by glorfindeal

There should be a '+' and '-' button on the bottom of the screen (You have to hit the options button I believe). If you hit the '+' then you can select the file/folder/drive to exclude from the backup. If you select something from the excluded list and hit the '-' key, then it will include it in the backup.


Glor

Got it! The obvious is sometimes so hard to see.

Out of interest, what have you found to be the minimum that is required to create the sparcebundle file. during the initial back up procedure using your scenario, would a single file do it for example?

[rahrens]

Originally Posted by davesmall

I'm using an Apple Extreme base station with a 500 GB external USB2 disk drive attached. The latest Time Machine and Apple Extreme software & firmware updates from Apple have enabled me to configure and run the same as Time Capsule. I've been up and running with WiFi Time Machine backups for about two weeks now. I'm backing up a MacBook Pro, a MacBook, and an iMac. Here are my experiences to date.

First of all, I had a bit of difficulty getting the base station configured to run properly. The firmware update wouldn't install until after I did a hard reset of the base station. After fiddling with the Airport Utility software I finally got it working with my Comcast cable modem. I think an actual Time Capsule would have been easier to get up and running.

I did the initial backups of all three computers over the wireless WiFi network. It took about 20 hours to backup the 130 gigabytes stored on my MacBook Pro. This happened in the background and I was able to use my computer while it was transferring data. If I had it to do over again I would do the initial backup over ethernet and then switch to wireless.

Things have gone very smoothly since the initial backups were completed. I really love this setup. For instance, it is now 9:35 AM here in Houston. I'm clicking on the Time Machine icon in my menu bar and it tells me that my last backup was completed at 9:04 AM today. I never notice that it is running. It's never more than an hour out of date so long as I'm within WiFi range.

Last week I flew to California for several days. Of course, no backups were being made while I was away from my WiFi network. When I returned Time Machine automatically resumed wireless backups. I didn't have to do a thing.

What's so cool about this is that it happens automatically in the background and you are unaware. I have much more current backups than ever before and I don't have to do a thing (once configured). It feels like magic. It's one of the best things in Leopard and I highly recommend it.

I would note that Apple has noted to me and others through their support lines that backing up to an external disk attached to an Airport Extreme via USB is NOT supported!

If your backup goes south and you lose data, don't call Apple, you are screwed!

[Wiggin]

Originally Posted by sjmacko

[I]I might be missing something... It wouldn't be the first time! I have a WD MyBook plugged into my Macbook Pro by firewire. My Macbook Pro backs up to it just fine. My wife's Macbook, and the kids' iBook back up to it over the network. I do not have to unplug the drive and attach it to each of their machines.Prior to the Mybook, I used a WD portable USB drive, which also worked fine. I was under the impression that this was the norm if the backup target is running Leopard.

On a similar note, I tried to share the drive from my Sawtooth server running Tiger, and TM did not work.

Steve

I was wondering if you could confirm something? We know that Time Machine writes normal files on a directly connected disk and a sparse disk image when backing up to Time Capsule or other non-Mac network shares (because those shares don't know about hard links). But I haven't seen anywhere how it writes files when it's to a disk shared from another Leopard Mac.

So I'm assuming your MacBook Pro writes normal files to your MyBook even when other Macs are using the disk for TM. Does your wife and kid's computers write to the MyBook as sparse disk images, or does the fact that your MBP is managing the share allow them to write normal files directly to the MyBook?

[digitalclips]

Originally Posted by rahrens

I would note that Apple has noted to me and others through their support lines that backing up to an external disk attached to an Airport Extreme via USB is NOT supported!

If your backup goes south and you lose data, don't call Apple, you are screwed!

I thought this was changed after the recent updates?

[lecorbu]

Will it back up my vista ultimate partition?

While I have your attension, anyone know if the new MB pro will support 64 bit vista ultimate?

Thanks!

[digitalclips]

Originally Posted by lecorbu

Will it back up my vista ultimate partition?

While I have your attension, anyone know if the new MB pro will support 64 bit vista ultimate?

Thanks!

TM will back up anything on the Mac side including VMs but not a Boot Camp partition.

This I am not 100% but I think this is right, 64 Bit Vista is supported under Boot Camp and VMware but not Parallels yet.

[glorfindeal]

Originally Posted by digitalclips

Got it! The obvious is sometimes so hard to see.

Out of interest, what have you found to be the minimum that is required to create the sparcebundle file. during the initial back up procedure using your scenario, would a single file do it for example?

I selected my Documents directory which is nearly empty to create the sparcebundle.

Glor

[jowie74]

Time Machine also deals with directly connected UBS hard drives (or Firewire or SATA drives) and network shares on Time Capsule or AirPort Extreme drives differently, making the two methods incompatible. Users have to decide between using Time Machine over the network or to a local disk; the same USB drive can't be shuttled between the two to create a single set of backups.

I was thinking, is there not possibly a way to convert one to the other? Could one copy a direct Time Machine backup folder into a sparse disk image with the correct name, and then have that show up as a network Time Machine backup? Could one take a network folder, copy the contents out onto an external disk and then use it locally?

Requires further investimagation...

[lyeung]

Does anyone know if it is possible to have a single iPhoto library shared on a home network among multiple computers? I couldn't find it in the Help documentation or the Apple website. I'm not talking about 'sharing' photos, but actually having read/write/modify privileges from any computer to the same set of photos. Would TC or Extreme have any difference in this capability? (wouldn't seem so)

LY

[bxs6408]

Originally Posted by jowie74

I was thinking, is there not possibly a way to convert one to the other? Could one copy a direct Time Machine backup folder into a sparse disk image with the correct name, and then have that show up as a network Time Machine backup? Could one take a network folder, copy the contents out onto an external disk and then use it locally?

Requires further investimagation...

There is no conversion process that I know of for doing this.

[imjeffro]

You bring up the idea of power users opting for a dedicated NAS instead of relying on the TC or AEB to perform the duty and write to either its internal or connected USB drive. But you also detail how sub-par the Apple routers' gigabit ethernet switch is. So would a NAS attached to either TC or AEB via gigabit ethernet really gain much, or would it be choked much the same as the TC chokes writes to its internal SATA drive?

Or how would you set it up to avoid this?

[Wiggin]

Originally Posted by imjeffro

You bring up the idea of power users opting for a dedicated NAS instead of relying on the TC or AEB to perform the duty and write to either its internal or connected USB drive. But you also detail how sub-par the Apple routers' gigabit ethernet switch is. So would a NAS attached to either TC or AEB via gigabit ethernet really gain much, or would it be choked much the same as the TC chokes writes to its internal SATA drive?

Or how would you set it up to avoid this?

This article, while useful, was somewhat poorly written, by AppleInsider's usually higher standards. I had to re-read some parts several times, but I think when they were criticizing the gigabit performance, they were still talking about it in reference to reading and writing data to the TC's internal or USB drives. They left that very important bit of information out of most of the references to poor Gb performance, but I think if you go back again you'll see that's what they meant.

Even when they were talking about writing to a drive connected to the PowerMac G5, I assume they were still writing to the same USB drive (I assume that because they never say and because I've achieved higher performance writing to a FW drive connected to a lowly mini over the Ethernet to an Extreme). If that's the case, the USB drive was probably the bottleneck, not the Ethernet.

Unfortunately, they just didn't give enough information to come to any good conclusions about how their results would translate to other situations.

[wtfk]

Originally Posted by digitalclips

Cool! I suspected there would be some clever trick to this, Thanks!

How do you exclude stuff in TM? I don't see a file level exclude selection just a drive selection. Is there a feature like Carbon Copy Cloner to only select certain files?

Actually, there's an even easier way. Just start a Time Machine backup to the mounted network volume, and when it gets past the first stage and starts writing files, stop it, either by pushing the cancel button or by selecting "Stop Backing Up." It will leave a nearly empty disk image on the drive.

Unmount it, and connect it directly to the Mac (don't change ANY settings), and tell it to "Back Up Now." It will complete the backup to the image it already created. Then you can plug it back into the network server (Mac, Time Capsule or Airport Extreme) and it will work as you would like.

Interestingly, in this way, you can interchange network and direct backups. The same image can be used as a direct connect backup as long as it was first created over the network. Also, don't be assuming this only works with USB. You can use any drive (e.g. firewire, etc.) like this if you start by connecting it to a host Mac running personal file sharing. Just for "compatibility" sake, I'd probably make it a policy to start all backups on a network source so they're all stored in the same format.

[wtfk]

Originally Posted by jowie74

I was thinking, is there not possibly a way to convert one to the other? Could one copy a direct Time Machine backup folder into a sparse disk image with the correct name, and then have that show up as a network Time Machine backup? Could one take a network folder, copy the contents out onto an external disk and then use it locally?

Requires further investimagation...

See my post immediately before this one.

[sjmacko]

Originally Posted by Wiggin

I was wondering if you could confirm something? We know that Time Machine writes normal files on a directly connected disk and a sparse disk image when backing up to Time Capsule or other non-Mac network shares (because those shares don't know about hard links). But I haven't seen anywhere how it writes files when it's to a disk shared from another Leopard Mac.

So I'm assuming your MacBook Pro writes normal files to your MyBook even when other Macs are using the disk for TM. Does your wife and kid's computers write to the MyBook as sparse disk images, or does the fact that your MBP is managing the share allow them to write normal files directly to the MyBook?

My Macbook Pro (firewire connection) writes in the normal backups.backupdb directory. The two computers that back up over the network write to sparse disk images. The disk images are outside of the backups.backupdb directory, and exist at the root of the drive. On a good note- If my wife makes a lot of changes, I can move the drive to her computer and TM uses the sparse image even though it is locally connected.

Steve

[glorfindeal]

Originally Posted by wtfk

Actually, there's an even easier way. Just start a Time Machine backup to the mounted network volume, and when it gets past the first stage and starts writing files, stop it, either by pushing the cancel button or by selecting "Stop Backing Up." It will leave a nearly empty disk image on the drive.

Unmount it, and connect it directly to the Mac (don't change ANY settings), and tell it to "Back Up Now." It will complete the backup to the image it already created. Then you can plug it back into the network server (Mac, Time Capsule or Airport Extreme) and it will work as you would like.

Interestingly, in this way, you can interchange network and direct backups. The same image can be used as a direct connect backup as long as it was first created over the network. Also, don't be assuming this only works with USB. You can use any drive (e.g. firewire, etc.) like this if you start by connecting it to a host Mac running personal file sharing. Just for "compatibility" sake, I'd probably make it a policy to start all backups on a network source so they're all stored in the same format.

I had tried this once and it ended up with a damaged sparcebundle file that Time Machine kept on choking on. That is why I went with the complete but small backup first.

Glor

[Wiggin]

Originally Posted by sjmacko

My Macbook Pro (firewire connection) writes in the normal backups.backupdb directory. The two computers that back up over the network write to sparse disk images. The disk images are outside of the backups.backupdb directory, and exist at the root of the drive. On a good note- If my wife makes a lot of changes, I can move the drive to her computer and TM uses the sparse image even though it is locally connected.

Steve

Thanks for the info!

It's interesting (and good!) that Time Machine can find the sparse bundle disk images, even if you've moved the drive. That was another thing I was wondering about...what if I later want to change my network around? Sounds like it would handle that fine. AirDisk would probably be fast enough for most of my incremental updates, but I was concerned about what would happen the first time I downloaded 4 GB worth of photos after my next vacation!

[bxs6408]

Originally Posted by wtfk

Actually, there's an even easier way. Just start a Time Machine backup to the mounted network volume, and when it gets past the first stage and starts writing files, stop it, either by pushing the cancel button or by selecting "Stop Backing Up." It will leave a nearly empty disk image on the drive.

Unmount it, and connect it directly to the Mac (don't change ANY settings), and tell it to "Back Up Now." It will complete the backup to the image it already created. Then you can plug it back into the network server (Mac, Time Capsule or Airport Extreme) and it will work as you would like.

Interestingly, in this way, you can interchange network and direct backups. The same image can be used as a direct connect backup as long as it was first created over the network. Also, don't be assuming this only works with USB. You can use any drive (e.g. firewire, etc.) like this if you start by connecting it to a host Mac running personal file sharing. Just for "compatibility" sake, I'd probably make it a policy to start all backups on a network source so they're all stored in the same format.

1. This is unsupported by Apple unfortunately but does have merit.
2. I had tried this earlier and ran into serious issues in TM browser not showing everything.
3. The only reason for doing this would be to facilitate an acceptable wall time for completing the first backup if it were to be 100s of GBs that HAS to be done over WiFi.

[digitalclips]

Originally Posted by bxs6408

1. This is unsupported by Apple unfortunately but does have merit.
2. I had tried this earlier and ran into serious issues in TM browser not showing everything.
3. The only reason for doing this would be to facilitate an acceptable wall time for completing the first backup if it were to be 100s of GBs that HAS to be done over WiFi.

Thanks, I managed to do this yesterday after all the excellent advice and this is the best way yet. Have you found a simple way to clone a TM back up made using the sparcebundle TM file? Will Carbon Copy Cloner in block mode do the job? File level attempts using CCC and Suer Duper all fail for me.

[Imagine Engine]

Correct me if I'm wrong but is the article stating it's faster using Time Machine and read/write files to a NAS drive which is connected to Time Capsule via Ethernet 1000 cable instead of using a USB NAS drive? Is there any speed difference when using Ethernet NAS compared to USB NAS over a 802.11n WLAN? Does the Time Capsule firewall still protect an Ethernet NAS drive (restrict user access, password protect) as it does for a USB NAS drive?

[Wiggin]

Originally Posted by Imagine Engine

Correct me if I'm wrong but is the article stating it's faster using Time Machine and read/write files to a NAS drive which is connected to Time Capsule via Ethernet 1000 cable instead of using a USB NAS drive? Is there any speed difference when using Ethernet NAS compared to USB NAS over a 802.11n WLAN? Does the Time Capsule firewall still protect an Ethernet NAS drive (restrict user access, password protect) as it does for a USB NAS drive?

Here are some data points from my testing that may help you decided if NAS is the right thing for you. But to answer your last question first, no, TC's disk permissions/security features will not apply to a dedicated NAS because TC wouldn't be managing that disk as it would for it's own internal or USB attached drives. Your NAS may provide similar features of it's own?

I don't have a NAS, but to simulate that I used a PowerBook with Gigabit Ethernet and an attached Firewire drive (which would be much faster than a USB drive or the PB's own internal drive). The PB was attached to a Gigabit Extreme via Ethernet. The Extreme was in N-only mode with wide channels. For an AirDisk, I used the same drive that was attached to the PB for the sake of consistency (it has FW and USB connections).

The wireless client was a new MBP. I used two sets of test data because network transfers of large files is much more efficient than for small files. One was a folder of 1400 small files totalling 163 MB (probably a typical Time Machine incremental backup). The second was a folder of 100 image files totalling 820 MB (representing updating my Aperture archive). One big caveat: I timed the tests with a stopwatch and only ran each one once (I have a real job, and this ain't it! ). Here were my results:

Folder of small files:
AirDisk: 0.91 MB/s write, 2.58 MB/s read
PB/FW: 2.62 MB/s write, 4.63 MB/s read

Folder of large files:
AirDisk: 3.38 MB/s write, 7.45 MB/s read
PB/FW: 10.34 MB/s write, 12.56 MB/s read

As you can see, my psuedo NAS has much better performance than the Extreme's USB disk. A dedicated NAS with an internal SATA drive should do even better than my 4-year old PB/FW drive combo. Also, when I connect my MBP to the Extreme via Gigabit Ethernet, the results are even more dramatic. My psudo NAS achieved between 34-38 MB/s reading and writing the folder of large files as opposed to 7.6 - 16.9 MB/s for the AirDisk.

[Felix01]

Originally Posted by glorfindeal

There is a much quicker way to do the initial backup. Connect the usb drive to your airport extreme and do a time machine back up excluding just about all files on your internal hard drive. All you want is to create the initial sparcebundle file. Once this backup is complete, you can attach the usb drive directly to your computer and time machine will recognize it. Remove all the excluded files and do a back up again. Time machine will use the sparcebundle file for the back up. Once you finished, reattach your usb drive to the airport extreme.

Or the really quick way is to click on "Stop Backing Up" (Time Machine menu) after you've got the sparsebundle file established. Then continue as described above.

[mikepikefl]

Originally Posted by glorfindeal

There is a much quicker way to do the initial backup. Connect the usb drive to your airport extreme and do a time machine back up excluding just about all files on your internal hard drive. All you want is to create the initial sparcebundle file. Once this backup is complete, you can attach the usb drive directly to your computer and time machine will recognize it. Remove all the excluded files and do a back up again. Time machine will use the sparcebundle file for the back up. Once you finished, reattach your usb drive to the airport extreme.

I had to register just so I could reply to this. Thanks so much for this information!!!

I have a drive I've been using Time Machine with for the past year for my Macbook. I recently tried to set it up as a network drive on a MacMini I got ahold of, instead of plugging in directly. It appeared to work at first but then gave me the "not enough space" error message. I noticed the sparsebundle but didn't quite grasp the entire concept. I decided to give up and plug directly in- and I got the same message! I couldn't back up at all any more!

After reading your post, I deleted the sparsebundle file, and then the backup worked fine! Phew!

So it appears to me that the sparsebundle file takes precedence over the normal hard linked filesystem (there must be a reason, but that wasn't apparent at all to me). I was afraid that I would have to segregate my backups across drives or something.

So, once again, thanks so much!!

[jowie74]

I'd love to know Apple's logic when thinking it would be a good idea to use sparsebundles on network drives, but not all of the time. If it used the same system on both network and non-network drives, then backup images would be completely movable.

[mikepikefl]

Originally Posted by jowie74

I'd love to know Apple's logic when thinking it would be a good idea to use sparsebundles on network drives, but not all of the time. If it used the same system on both network and non-network drives, then backup images would be completely movable.

I'm betting two reasons:

1. Afterthought It wasn't initially supported
2. Compatibility- can't guarantee the file system over the network

So if they had come up with the sparsebundles first maybe they would have used them for both methods. Obviously this is just a guess. I'm also wondering, on a file system such as FAT32 (over the network) does is break the bundles up at the 2GB or 4GB mark? Forgive me if this is answered some place else- I just thought of this.

Given the current state though, I would really like it to be smarter. If you have 180GB of hard linked backups, and a virtually empty sparse bundle file... Or at least some logic to detect both and ask a question, maybe based upon last backup date. Instead I Googled my tail off, and crossed my fingers as I deleted the sparsebundle.

The way it just seems to see a sparsebundle and run with it is kind of odd.

~Mike