AppleInsider › Forums › General › Genius Bar › FreeBSD as fileserver for OS X and Windows network
New Posts  All Forums:Forum Nav:

FreeBSD as fileserver for OS X and Windows network

post #1 of 26
8/1/05 at 12:24pm
Thread Starter 
I've been wanting to build a file/backup server for a while and I finally go around to it. I had an old PC not being used so this last weekend I installed FreeBSD 5.4 on it, setup Samba, and started backing up my ibook over the network. Everything seems to work pretty well, except I want to make sure I'm not losing any data, metadata, timestamp info, etc.

Can anyone give me a little advice on the best network protocol to use for this... should I use NFS, CIFS, Samba, netatalk, or what???

Samba seems to strip the creation date from my files. Netatalk (afpd) seems to put lots of invisible .blahblah (ie: .AppleDouble) everywhere in my FreeBSD shares.

How about the filesystem format on the FreeBSD box. Is there some special format I can use to make sure it's totally compatible with the OS X file structures so that it will keep all the data in tact???

Thanks.
GabSlab.com
Reply
GabSlab.com
Reply
post #2 of 26
8/1/05 at 1:28pm
Use Samba. Its easy to setup and allows unix/mac and pc to see it.

Dobby.
Reply
Reply
post #3 of 26
8/1/05 at 1:36pm
Thread Starter 
Quote:
Originally posted by dobby
Use Samba. Its easy to setup and allows unix/mac and pc to see it.

Dobby.

I probably will end up using samba, but it's real annoying when I start copying a large directory and then it comes along a filename that is too long or has illegal characters in it and then the entire copy process quits.

Also, does samba maintain all metadata, file structure, etc? or will I lose stuff from all the files, such as dates, types, etc???
GabSlab.com
Reply
GabSlab.com
Reply
post #4 of 26
8/1/05 at 2:23pm
Samba retains no metadata.

I use Fedora machines as samba servers to PC's and Macs. To retain the metadata I do the following.

Mount the nfsshare on a mac.
Copy the data on the mac to the nfsshare.
Export the nfsshare as a smb share as well.

The smb share now has the metadata from the mac (only the mac can see it). I run a script to replace spaces at beginning of names and any funny escape sequences that PC's can't view. ("/") are not allowed etc.

Dobby.
Reply
Reply
post #5 of 26
8/1/05 at 2:34pm
Thread Starter 
Quote:
Originally posted by dobby
Samba retains no metadata.

I use Fedora machines as samba servers to PC's and Macs. To retain the metadata I do the following.

Mount the nfsshare on a mac.
Copy the data on the mac to the nfsshare.
Export the nfsshare as a smb share as well.

The smb share now has the metadata from the mac (only the mac can see it). I run a script to replace spaces at beginning of names and any funny escape sequences that PC's can't view. ("/") are not allowed etc.

Dobby.

So you are saying that NFS retains everything.?.? And then the SMB is only to allow my Windows machines to see the same share/data (minus the metadata which is seen only by the Mac)? Thanks for the info.
GabSlab.com
Reply
GabSlab.com
Reply
post #6 of 26
8/1/05 at 4:46pm
I'm guessing here. Couldn't you use Darwin, and if you wanted a GUI install Fink and a port of Gnome or KDE?
NOTICE: While every effort has been made to ensure the accuracy of the information supplied herein, fahlman cannot be held responsible for any errors or omissions. Unless otherwise indicated,...
Reply
NOTICE: While every effort has been made to ensure the accuracy of the information supplied herein, fahlman cannot be held responsible for any errors or omissions. Unless otherwise indicated,...
Reply
post #7 of 26
8/1/05 at 4:54pm
Thread Starter 
Quote:
Originally posted by fahlman
I'm guessing here. Couldn't you use Darwin, and if you wanted a GUI install Fink and a port of Gnome or KDE?

I would love to use Darwin, and actually downloaded the Darwin x86 iso, but thought I might have a hard time finding drivers for the NIC, SATA card, etc.

Or am I wrong about that?? Can I use the same Drivers as I would in FreeBSD?
GabSlab.com
Reply
GabSlab.com
Reply
post #8 of 26
8/1/05 at 5:11pm
Thread Starter 
Quote:
Originally posted by fahlman
I'm guessing here. Couldn't you use Darwin, and if you wanted a GUI install Fink and a port of Gnome or KDE?

PS: In the context of my project what advantage would I have using Darwin instead of FreeBSD? Either system, though they are very similar, would still use nfs, smb, or afp for file sharing. Would the advantage be that Darwin is already configured to format the drive in HFS+???
GabSlab.com
Reply
GabSlab.com
Reply
post #9 of 26
8/1/05 at 6:02pm
The reason I suggested Darwin was because of its support of HFS+. As far as drivers I don't know. Maybe someone with more experience could add some input.
NOTICE: While every effort has been made to ensure the accuracy of the information supplied herein, fahlman cannot be held responsible for any errors or omissions. Unless otherwise indicated,...
Reply
NOTICE: While every effort has been made to ensure the accuracy of the information supplied herein, fahlman cannot be held responsible for any errors or omissions. Unless otherwise indicated,...
Reply
post #10 of 26
8/1/05 at 6:46pm
You would basically have the guts of OS X minus the eye candy. For your purposes it would be just like backing up to a Mac on your network, I would think. You could use Apple File Sharing just like you were connection to any other Mac. Should be seamless.
NOTICE: While every effort has been made to ensure the accuracy of the information supplied herein, fahlman cannot be held responsible for any errors or omissions. Unless otherwise indicated,...
Reply
NOTICE: While every effort has been made to ensure the accuracy of the information supplied herein, fahlman cannot be held responsible for any errors or omissions. Unless otherwise indicated,...
Reply
post #11 of 26
8/1/05 at 8:07pm
Quote:
Originally posted by dobby
Samba retains no metadata.

I use Fedora machines as samba servers to PC's and Macs. To retain the metadata I do the following.

Mount the nfsshare on a mac.
Copy the data on the mac to the nfsshare.
Export the nfsshare as a smb share as well.

The smb share now has the metadata from the mac (only the mac can see it). I run a script to replace spaces at beginning of names and any funny escape sequences that PC's can't view. ("/") are not allowed etc.

Why does NFS retain metadata and SMB does not? How/where is the metadata stored?
Reply
Reply
post #12 of 26
8/1/05 at 8:08pm
Another option would be to backup to a DMG and then copy that to the server running whatever protocol you want.
Reply
Reply
post #13 of 26
8/2/05 at 11:43am
Thread Starter 
Quote:
Originally posted by mikef
Another option would be to backup to a DMG and then copy that to the server running whatever protocol you want.

Yea, that would maintain the data/file structure, but wouldn't really be practical for randomly accessing thousands of files or large movie or audio files.

NFS sounds like a good way to go but GEEEEZ.. what a pain in the --- to configure. Anyone know of any good tutorials for configuring NFS server on FreeBSD and NFS client on OS X??
GabSlab.com
Reply
GabSlab.com
Reply
post #14 of 26
8/2/05 at 11:53am
Quote:
Originally posted by Macvault
Yea, that would maintain the data/file structure, but wouldn't really be practical for randomly accessing thousands of files or large movie or audio files.

No, but for archival storage, it'd be fine. I typically do not touch my backups unless they're needed, so not having them "live" isn't a big deal.

Quote:
NFS sounds like a good way to go but GEEEEZ.. what a pain in the --- to configure. Anyone know of any good tutorials for configuring NFS server on FreeBSD and NFS client on OS X??

If the UIDs match between the client and server, there shouldn't be any major issues. Do you have specific problems?
Reply
Reply
post #15 of 26
8/2/05 at 12:00pm
Thread Starter 
Quote:
Originally posted by mikef
No, but for archival storage, it'd be fine. I typically do not touch my backups unless they're needed, so not having them "live" isn't a big deal.

If the UIDs match between the client and server, there shouldn't be any major issues. Do you have specific problems?

Yea, I guess it would work for archival. Could I write the DMG directly to the samba share without having to write it localy on my ibook, then transferring it to the share?

As for the NFS problems.. I got it to where I could see (read) the share but didn't have previleges to write to it. Probably something simple. Would this be fixed in /etc/exports ?
GabSlab.com
Reply
GabSlab.com
Reply
post #16 of 26
8/2/05 at 12:12pm
Quote:
Originally posted by Macvault
Yea, I guess it would work for archival. Could I write the DMG directly to the samba share without having to write it localy on my ibook, then transferring it to the share?

This should work fine. I was able to a test backup to a Samba share using Super Duper.

Quote:
As for the NFS problems.. I got it to where I could see (read) the share but didn't have previleges to write to it. Probably something simple. Would this be fixed in /etc/exports ?

The UIDs have to match between the client and server.

Use the 'id' command on OS X and the server to confirm your UIDs and adjust them in the /etc/passwd, and /etc/group files (if necessary) on the server.

Also, obviously the share has to be exported from writing.

What I typically do for NFS is create a group 'local' and then give local ownership/write permissions to the exported filesystem.

For example (after adding the group 'local' and adding yourself to it):

Code:


$ mkdir /export/backup
$ chown root:local /export/backup
$ chmod 770 /export/backup



Then export this filesystem (not sure how this is done on FreeBSD; it's been a while for FreeBSD for me) with 'rw' (read/write permissions)
Reply
Reply
post #17 of 26
8/2/05 at 1:45pm
Sorry guys, I left out that we use Helios (OPI app) on our MAC server. Helios changes the way resource forks are recorded so that when they are copied to an NFS share the resource fork is copied in a file format in a sub dir of the original.

Dobby.

Edit. I'll test a normal share. Note that I only use Solaris and Fedora. The closest thing I use to FreeBSD is OS X.
Reply
Reply
post #18 of 26
8/2/05 at 1:48pm
Ahh, that makes sense... I couldn't figure out why NFS properly handled the resource fork and SMB did not.
Reply
Reply
post #19 of 26
8/3/05 at 8:52am
Thread Starter 
I just installed and mounted a Deskstar 250GB SATA drive in FreeBSD but after formatting it shows only about 202GB available. Is this normal? Doesn't seem like it should eat up that much.

Is there some way I can format this drive in HFS+ so I don't get a ._ file for every (Mac OS X) file I copy over? This seems like a pain in the ass. Why can't one file on one system equal one file on another system!?!

Anyone know if Darwin x86 would support the FreeBSD drivers for my NIC(no name) and SATA(Syba) card?
GabSlab.com
Reply
GabSlab.com
Reply
post #20 of 26
8/3/05 at 9:09am
Quote:
Originally posted by Macvault
I just installed and mounted a Deskstar 250GB SATA drive in FreeBSD but after formatting it shows only about 202GB available. Is this normal? Doesn't seem like it should eat up that much.

250,000,000 bytes == 244160.625 MB == 238.412 GB

I am not sure about FreeBSD, but on Linux, 5% of that is reserved for the super-user so the total available would be 227.06MB. On Solaris, it's (64MB/partition size) * 100 (minimum 1% of drive space, maximum 10%). In this case, it would be 10% or total of 216GB.

If the filesystem you formatted it as is journalled, there might be space preallocated for that as well.

Quote:
Is there some way I can format this drive in HFS+ so I don't get a ._ file for every (Mac OS X) file I copy over? This seems like a pain in the ass. Why can't one file on one system equal one file on another system!?!

Even if you could format it using HFS+, it is a local filesystem, not a network filesystem. You'd need a network protocol that is aware of the resource/extended attribute information. Perhaps Appletalk? (again, I have no experience there)

There is an article on one of the Mac sites (MacFixIt, maybe) about how to disable creating .DS_Store files.
Reply
Reply
post #21 of 26
8/3/05 at 1:10pm
Quote:
Originally posted by Macvault
Is there some way I can format this drive in HFS+ so I don't get a ._ file for every (Mac OS X) file I copy over?

This is the resource fork data (or part of it).
If you change the colour of the file on the mac then copy it to the nfs share then mount that share on another mac and copy the file there then the colour shouldbe restained.
This doesn't work on SMB.

Dobby.
Reply
Reply
post #22 of 26
8/3/05 at 1:13pm
Why doesn't it work on SMB? When I put files on my Windows box from my Mac, the .DS_Store files are created. Are they not used later by OS X?
Reply
Reply
post #23 of 26
8/3/05 at 1:21pm
Thread Starter 
So is my problem really just the networking protocol (smb)? Or is it also the filesystem of the server?

Anyone know of any tutorials on how to configure NFS?

The .DS_Store files aren't so bad. I just hate to see a ._blahblah file for every single file I copy over from OS X. This eats up more disk space.
GabSlab.com
Reply
GabSlab.com
Reply
post #24 of 26
8/3/05 at 1:45pm
Quote:
Originally posted by Macvault
So is my problem really just the networking protocol (smb)? Or is it also the filesystem of the server?

It is the networking protocol. This is how the local filesystem is represented "on the wire" (over the network). If there was a network protocol that understood the semantics of HFS+, the filesystem would be better represented over the network and resource forks wouldn't be an issue.

For the time being, HFS+ is shoehorned to work over NFS/SMB and there will be some comprimise.

Quote:
Anyone know of any tutorials on how to configure NFS?

If you have specific questions, I'll be happy to help. In the meantime, NFS has been around since the late 80s, so there is TONS of info on the web.

The common problem is the namespace of the client must match the namespace of the server. If user "jimbo" is UID 1001 on OS X, user "jimbo" must be UID 1001 on the server (common sense, really). Directory services such as NIS, NIS+, and LDAP exist to keep the namespace consistent between NFS clients and servers however they're overkill for the average home network. Just keep the passwd files (specifically the UIDs and GIDs) in sync between client and server.

The other thing to watch are UNIX permissions. If user "jimbo" doesn't have permissions to write in a local directory, he also won't have permissions to write over NFS.

I have worked on NFS as user/administrator and developer for over 10 years now.

Quote:
The .DS_Store files aren't so bad. I just hate to see a ._blahblah file for every single file I copy over from OS X. This eats up more disk space.

If you want to keep the resource fork, there's little you can do.
Reply
Reply
post #25 of 26
8/3/05 at 4:14pm
Thread Starter 
Quote:
Originally posted by mikef
...If you want to keep the resource fork, there's little you can do. [/B]

Hmm? Then what's the advantage os NFS over SMB?

By the way, thanks to everyone for all the input on this!
GabSlab.com
Reply
GabSlab.com
Reply
post #26 of 26
8/3/05 at 8:33pm
I guess there really isn't one... it was dobby who said that NFS retained the resource fork but this is not true (and he posted a retraction).
Reply
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Genius Bar
AppleInsider › Forums › General › Genius Bar › FreeBSD as fileserver for OS X and Windows network