local networking btw PBook and Linux server

Posted:
in Genius Bar edited January 2014
Hi. I have a huge amount (1 TB) of files to copy from HFS+ formatted external drives to a linux server. I don't want to mess with the kernel so I can't install the beta HFS+ drivers that would allow me to plug the drive right into the server.



So okay, I'll just copy over the network. Except I only have an Airport Extreme. The server is plugged into the one ethernet port and is pulling a static IP (10.0.1.201). The PowerBook is connected over 802.11b (that's all I got) to the Airport Extreme using DHCP.



This is giving me around 300 KB/sec. That's about 1 GB/hour! (I think.) Obviously it won't work this way.



What are my options? Obviously buy another router with more ports. But I'd rather not do that.



Is there any way to just ethernet the PowerBook right to the server. I tried just plugging it in and connecting to 10.0.1.201 from the PowerBook but it won't connect. Any idea how I could make that work without a router or switch?



I guess this is probably more of a Linux question than a Mac OS X question but maybe someone can help. Thanks.

Comments

  • Reply 1 of 12
    Okay, following up to my own post, hope that's not bad form.



    I also am curious about the 300 KB/sec. Shouldn't I get more than this? I didn't expect to get the full 10 Mb/sec, but that isn't even close (~2.4 Mb/sec). Does that seem right? I guess it really doesn't matter, since even if I could triple the speed we're still talking about over 300 hours for the copy. Obviously I need gigabit speed. But I'm just curious about the throughput. Is 2.4 Mb/sec about what you would expect copy over an 802.11b link on a LAN?



    I am using SFTP. Maybe the encryption is slowing it down?
  • Reply 2 of 12
    If you use APT, you can easily uninstall the beta HFS+ driver after you're done. So that may be one way of doing it - temporary installation of HFS+ drivers.



    Quote:

    apt-get install kernel_driver



    and



    Quote:

    apt-get uninstall kernel_driver



    and you're done.



    It may not be what you want, but I'm sure it won't mess up your kernel.
  • Reply 3 of 12
    Thanks for the reply. Turns out it recognizes the HFS+ drive as soon as I plug it in via USB. This is with CentOS 4.2 on the 2.6 kernel. Who knew? Wish I hadn't just assumed it would be hard.
  • Reply 4 of 12
    Well that's great. Didn't know CentOS (really just RHEL, but free) included HFS+ drivers by default in their version of the kernel.
  • Reply 5 of 12
    pyr3pyr3 Posts: 946member
    The other option would have been to use a FAT32 drive as an intermediate. Copy files to the FAT32 drive, plug it into the Linux server, copy them over. Repeat as many times as necessary.
  • Reply 6 of 12
    Can't you just use network cable to directly connect those two computers? It should give you better speeds. New macs should be able to use regular cable to do that, or if it doesn't work, you need to use crossed ethernet cable(I think thats what it's called in english) they don't cost a thing.
  • Reply 7 of 12
    It's called Crossover cable, and he could, but the speed would still be very limited since he doesn't have Gigabit ethernet, from what I understood.



    There are fairly stable HFS+ drivers for Linux so there's no need to actually waste time waiting for the network to transfer such huge amounts of data. Plug'n'Play.
  • Reply 8 of 12
    Thanks for the replies everyone. I successfully finished the first stage of the copy which was about 100 GB from the HFS+ drive. It did just work to plug it in (USB) to the server.



    But I've been hit again on the second leg. My next copy is 400 GB off an NTFS formatted drive. NTFS is not supported in the kernel though. And what I've been reading makes it seem like I need to install a whole other kernel to get it to work. I would have thought NTFS would be supported (at least read only) before HFS+. Strange.



    So I mounted the NTFS drive on the Mac, and then (as someone suggested above) directly connected the machines with an ethernet cable. Got that to work, but my transfer speed was the same as using 802.11b from the Mac to the router (and ethernet from the router to the server): about 300 KB/sec.



    Is that really all I can get? I guess the external drive is the limiting factor? In this case it is a LaCie 500 GB (2 x 250 in RAID-0) connected over 400 Mb/sec FireWire. Shouldn't I get more than 300 KB/sec from that?



    Does anyone agree/disagree with my hypothesis that the drive is the bottleneck. If that is the case then copying to another (say FAT32) external drive as suggested above (and then just walking that over to the server and plugging it in to USB) won't be faster either (since I imagine the NTFS to FAT32 copy is still only going to get 300 KB/sec.)



    Or am I missing something? I hope I am. At this rate I am looking at 15 days to copy!!!
  • Reply 9 of 12
    lundylundy Posts: 4,466member
    That does seem awfully slow, and 10base-T ethernet can certainly go a lot faster than that. Is it a lot of small files? I'm thinking the only thing that could really make it that slow is a lot of file system code having to run.



    If that is the case, perhaps transferring it as a disk image would be the answer.
  • Reply 10 of 12
    Thanks Lundy. No, all the files are in the 4 to 10 megabyte range.



    But okay, maybe I'm being really stupid. Is SFTP orders of magnitude slower than FTP? I know it must be slower in some way, but I just can't believe that I'm CPU bound. The Mac isn't so fast, so maybe that is it, but the Linux box (dual Opteron) shouldn't be having trouble with this.



    I guess I should experiment with that though. I'll report back. Thanks for the help guys.
  • Reply 11 of 12
    NTFS isn't supported in the kernel, because its a very closed file-system with terrible (almost inexistant) documentation. Read capability is there, so you shouldn't have problems transferring files from your NTFS drive to your Linux box.



    What isn't there, by default, is write capability which can be enabled by using a third-party driver, but it's very unstable and can probably cause problems to you.
  • Reply 12 of 12
    Thanks Gene Clean.



    Sorry this strayed OT for this board but I appreciate everyone's help. Turned out CentOS like RHEL doesn't even support read for NTFS in the kernel. I think most other linux distros do. But it was just a simple one line install of an alternate kernel and it is activated.



    Still takes a long time to copy hundreds of gigs no matter how you slice it. More time for me to read AI and the like I guess.
Sign In or Register to comment.