Auto defrag + secure delete = not secure?
Mac OS X's ability to securely empty the trash is great when you have something you want to get rid off permanently. Then there's the auto defrag functionality that rewrites new unfragmented copies upon read when a file satisfies certain criteria (below 20MB, greater than x number of fragments, etc.)
My question is: Is it possible that auto defrag creates multiple copies of a file and marks the older ones as deleted (normal operation), and that when using secure delete on the last copy, the older ones are still on the hard drive?
Assume in this case that FileVault is *not* turned on.
Thanks in advance.
My question is: Is it possible that auto defrag creates multiple copies of a file and marks the older ones as deleted (normal operation), and that when using secure delete on the last copy, the older ones are still on the hard drive?
Assume in this case that FileVault is *not* turned on.
Thanks in advance.
Comments
Originally posted by drumsticks
Is it possible that auto defrag creates multiple copies of a file
No.
When one finally secure deletes the final unfragmented file, one is still left with a fragmented copy! Not so secure after all!
Or is it?
Originally posted by drumsticks
Auto defrag creates a new unfragmented copy
No.
What happens is that the inefficiently-placed fragments are restructured to form a new, more coherent fragment.
Because srm is file system-transparent, it doesn't actually know about any of this. It doesn't ask HFS+ for where the fragments are and then overwrite them one by one. It doesn't check for physical locations. Rather, it uses HFS+'s APIs to overwrite the file, regardless of its fragments. If, in the meantime, HFS+ were to relocate fragments due to hot file clustering (what you presumably mean by "auto-defrag"), srm would be paused, because the fragment being moved would cause the file to be locked. srm would wait for unlocking, then continue.
There is no security issue here. Interesting theory, however.
When a file is opened on an HFS Plus volume, the following conditions are tested:
- If the file is less than 20 MB in size
- If the file is not already busy
- If the file is not read-only
- If the file has more than eight extents
- If the system has been up for at least three minutes
If all of the above conditions are satisfied, the file is relocated -- it is defragmented on-the-fly.File contiguity (regardless of file size) is promoted in general as a consequence of the extent-based allocation policy in HFS Plus, which also delays actual allocation.
That suggests to me that a new contiquous version is created to replace the fragmented version!
I guess I'm talking at the HFS+ level, while you are talking at the SRM (Storage Resource Management?) level.
PS: I've used "auto-defrag" when I probably should have used "on-the-fly-defrag".
If the file is not already busy
When securely deleting the file, it would be busy, and thus this on-the-fly defragmentation would not kick in to begin with.
Conversely, while on-the-fly defragmentation is happening, secure deletion would be paused and would wait until the file has been "relocated", as Singh calls it (which I believe is somewhat misleading). After that, secure deletion would happen to the new physical location of the file.
"But wait", you say, "what happens to the old physical location? Aren't there going to be remains?" Yes, possibly, and worse, the secure deletion utility (e.g. srm) would not know about them. To get of these unlikely, scattered and near-impossible to find traces, one would have to securely erase the entire file system.
Originally posted by troberts
If you are really worried about security you could use the "Erase Free Space" feature located in Disk Utility.
Even that isn't 100% proof, as free space can and will vary during the long operation.
Of course, you could always boot from another partition and make sure no other process write-accesses it. (For example, using single-user mode.) Now that's safe.
Originally posted by Chucker
"But wait", you say, "what happens to the old physical location? Aren't there going to be remains?" Yes, possibly, and worse, the secure deletion utility (e.g. srm) would not know about them. To get of these unlikely, scattered and near-impossible to find traces, one would have to securely erase the entire file system.
Ah, that's the answer to my question... Thanks Chucker!
It's not that I have something that I really want to get rid of, this is just a hypothetical question at this stage. Am wanting to know how the insides of the OS works, that's all.
Originally posted by Chucker
When securely deleting the file, it would be busy, and thus this on-the-fly defragmentation would not kick in to begin with.
I don't think it's as important that they happen at the same time. I think the concern was more that when the files are defragmented over a long period of time, multiple copies are placed all over the hard drive. Then when secure delete occurs, it will only secure delete the current location of the data leaving portions of the information intact.
It's true that such information may be distributed too widely to be able to find but it may not be. I'd agree that erasing free space is probably the only way to get rid of sensitive information.
Originally posted by Chucker
Even that isn't 100% proof, as free space can and will vary during the long operation.
Of course, you could always boot from another partition and make sure no other process write-accesses it. (For example, using single-user mode.) Now that's safe.
Two questions:
1) Would booting from the installation and then running Disk Utility be secure?
2) What is the command for erasing free space while in single-user mode? My "The Missing Manual" doesn't cover that command, it's parameters (# of times to overwrite), or where the command is located.
Originally posted by troberts
Two questions:
1) Would booting from the installation and then running Disk Utility be secure?
Absolutely. (I assume you mean to call the "erase free space" option.)
2) What is the command for erasing free space while in single-user mode? My "The Missing Manual" doesn't cover that command, it's parameters (# of times to overwrite), or where the command is located.
diskutil's man page says:
secureErase [freespace] level device
Securely erase a disk or freespace on a mounted volume. Level
should be one of the following
1 - Single pass randomly erase the disk.
2 - US DoD 7 pass secure erase.
3 - Gutmann algorithm 35 pass secure erase. Ownership
of the affected disk is required.
Don't try this out with a production volume, though, because I haven't tried it. Use some partition with random not as important stuff; perhaps back it up first.
Check the rest of the man page for device syntax. Or, just run "diskutil list" to see a list of identifiers you can use.
Note also that single-user mode doesn't mount read/write per default, and doesn't mount other partitions at all; you'll have to do that manually.
Originally posted by Chucker
Absolutely. (I assume you mean to call the "erase free space" option.)
diskutil's man page says:
Code:
secureErase [freespace] level device
Securely erase a disk or freespace on a mounted volume. Level
should be one of the following
1 - Single pass randomly erase the disk.
2 - US DoD 7 pass secure erase.
3 - Gutmann algorithm 35 pass secure erase. Ownership
of the affected disk is required.
Don't try this out with a production volume, though, because I haven't tried it. Use some partition with random not as important stuff; perhaps back it up first.
Check the rest of the man page for device syntax. Or, just run "diskutil list" to see a list of identifiers you can use.
Note also that single-user mode doesn't mount read/write per default, and doesn't mount other partitions at all; you'll have to do that manually.
Thank you. I think I will stick to using my install disk or even install Tiger on my external disk so I have a working OS should something go wrong with my internal drive.
Originally posted by grad student
put your super secret files on an encrypted disk image. any "residue" from relocating/copying something would mean nothing in part... then secure erase the disk image when done. that should be good enough for anyone i can imagine.
Yep, I was going to suggest that. You don't even have to secure delete files off an encrypted image either 'cos the database is stored on the encrypted space and the file remains there until it is overwritten.
When you are done with the encrypted image, as said above, you could be completely secure by secure erasing just the image itself. Although, I don't think it's entirely necessary because to recover an encrypted disk image, you'd have to not only recover the entire image (this is why a bigger image is better) but you'd have to know or crack the key.
If you are no longer using a machine, just write 0's to all the bits. Otherwise, what are you hiding?
You're kidding, right?
Amazing how an interesting technical discussion can be dragged into a pseudo-political quagmire by one dense comment.
How about: financial information; health information; anything that you don't want being used for identity theft...
Any of those on a laptop are an issue. Any of those on a home computer are an issue if your house is burglarized.
They should be encrypted. Barring that, they should be backed up to safe storage and then securely deleted off of machines that could be at risk.
However, in the presence of standard automated disk maintenance on HFS+, secure delete can still leave fragments of previous copies of the data lying around for recovery.
Only an idiot has nothing to hide.
Originally posted by jpennington
Otherwise, what are you hiding?
WTF kinda business of yours is it what drumsticks or troberts are "hiding"? Privacy rights exist for a reason, and just zero'ing out a hard drive is not enough to fully get rid of data.
Originally posted by Chucker
WTF kinda business of yours is it what drumsticks or troberts are "hiding"? Privacy rights exist for a reason, and just zero'ing out a hard drive is not enough to fully get rid of data.
Oh so the DOD seven pass write or the 35 pass write doesn't do a good enough job. I guess the DoD is an idiot then, since obviously their method of wiping drives isn't secure according to your standards.
I seriously doubt someone ripping off a laptop in an airport is doing it to steal your identity. You could also keep your computer secure enough so that if someone found that information they wouldn't be able to access it. You are right, it all should be heavily encrypted.
Identity thieves generally lack the skills to break into well secured systems. There are far easier and more vulnerable ways for them to do it.