Last time I checked, 90% of hard drives have 4KB block sizes. It would only take about 100,000 files. The example I used was tarring an average hard drive, presumably to backup on tape.
Now, I wouldn't call it decompression. But what I did say (and keep saying) is from an average user's perspective, in fact any user's perspective, untarring achieves the same as decompression. Untarring is effectively decompression at the user side.
Barto
Whoops! You're right, learn something new every day. Apparently, HFS+ uses a default 4K block, and HFS uses a block size based on disk size.
As for closing the thread, why? People are being quite civil. Are boring technical interactions not allowed on AI?
Those formats don't support random access, you provide one file (a .tar archive) and they give you a compressed version of that file. They don't know anything about its content. However, they are more efficient than other formats.
Zip, on the other hand, doesn't provide very good compression rates, but you can delete files from archives, add new files, extract just a few files etc.
If Apple intends to provide partial on-the-fly extraction in the future (display the archive as a disk image/folder, whatever), then they simply don't have much choice among the available formats.
Whoops! You're right, learn something new every day. Apparently, HFS+ uses a default 4K block, and HFS uses a block size based on disk size.
As for closing the thread, why? People are being quite civil. Are boring technical interactions not allowed on AI?
John
Well, I'd venture to guess 90% of the world uses Windows, and roughly half of them in turn use XP/2000/NT and NTFS, which has an utterly complex variable blocksize, which at least allows for down to 1K blocksizes.
As well, I'm not sure that HDDs have 4K blocksizes. Most modern HDDs still have 512 Byte sector/block sizes.
Well, I'd venture to guess 90% of the world uses Windows, and roughly half of them in turn use XP/2000/NT and NTFS, which has an utterly complex variable blocksize, which at least allows for down to 1K blocksizes.
As well, I'm not sure that HDDs have 4K blocksizes. Most modern HDDs still have 512 Byte sector/block sizes.
As far as I know, block size is not related to the HDD in most cases. I found an Apple document stating that HFS+ has a default 4K block size, while HFS was variable (apparently, HFS has a max number of blocks, around 64k of them). I'm more used to a 1K block size on Linux filesystems, so the 4K surprised me.
It's a tradeoff. Less disk space loss in filesystem overhead, more per file.
As far as I know, block size is not related to the HDD in most cases. I found an Apple document stating that HFS+ has a default 4K block size, while HFS was variable (apparently, HFS has a max number of blocks, around 64k of them).
Not quite.
Block size is always related to hard drive size. The size of the blocks grows directly with the size of the drive. The difference today is that there are many more blocks.
HFS supports 16 bits for block allocation and a total of 65,536 blocks.
HFS+ supports 32 bits for block allocation and a total of 4,294,967,296 blocks.
The size of the blocks grows directly with the size of the drive.
That doesn't sound quite right either. How about, "there is a minimum size blocks have to be based on number of blocks and size of the drive, but often the size will be greater than that minimum"?
The HFS+ default 4K will be used on all hard drives until the size of the drive exceeds 16TB. After 16TB, the block size would grow directly with HDD size. The same was probably true for HFS on drives up to 256MB.
Sounds very correct to me. I would check the 16TB figure, but I cannot be bothered.
The transition between HFS and HFS+ was very painful, because hard drives increased from ~200MB to several gigabytes before HFS+ came into operation. Block sizes of 32k and 64k led to huge bloat in volumes.
Well, I was right about the numbers I gave. The absolute *minimum* block size, though, is 512 bytes as Eugene said, not 4 KB. A quote from Apple TN #1150:
Quote:
HFS divides the total space on a volume into equal-sized pieces called allocation blocks. It uses 16-bit fields to identify a particular allocation block, so there must be less than 2^16 (65,536) allocation blocks on an HFS volume. The size of an allocation block is typically the smallest multiple of 512 such that there are less than 65,536 allocation blocks on the volume (i.e., the volume size divided by 65,535, rounded up to a multiple of 512). Any non-empty fork must occupy an integral number of allocation blocks. This means that the amount of space occupied by a fork is rounded up to a multiple of the allocation block size. As volumes (and therefore allocation blocks) get bigger, the amount of allocated but unused space increases.
HFS Plus uses 32-bit values to identify allocation blocks. This allows up to 2^32 (4,294,967,296) allocation blocks on a volume. More allocation blocks means a smaller allocation block size, especially on volumes of 1 GB or larger, which in turn means less average wasted space (the fraction of an allocation block at the end of a fork, where the entire allocation block is not actually used). It also means you can have more files, since the available space can be more finely distributed among a larger number of files. This change is especially beneficial if the volume contains a large number of small files.
Well, I was right about the numbers I gave. The absolute *minimum* block size, though, is 512 bytes as Eugene said, not 4 KB. A quote from Apple TN #1150:
True, but I was talking about default sizes, not minimum, which as per this Apple article, is 4KB.
Quote:
The actual number of files that can be stored on an Mac OS Extended (HFS Plus) volume depends on the size of the volume and the size of the files. For example, a 160 GB HFS Plus volume with the default block size of 4K will have 40 million available blocks. This volume could store up to 40 million very small files. A bigger disk with the same default block size could hold proportionately more files.
This just means that unless you explicitly modify the block size when creating the filesystem, you'll get 4KB blocks on anything smaller than a 16TB disk. Even using the minimum 512 byte blocks will work until drive sizes exceed 2TB. In either case, this is probably a fair distance off in the future (although you never know, these days... drive sizes are getting larger quickly).
I think we've hashed it out now: drive size can play a role in block sizes in HFS+, but not with any currently available drive.
Comments
Originally posted by Barto
Last time I checked, 90% of hard drives have 4KB block sizes. It would only take about 100,000 files. The example I used was tarring an average hard drive, presumably to backup on tape.
Now, I wouldn't call it decompression. But what I did say (and keep saying) is from an average user's perspective, in fact any user's perspective, untarring achieves the same as decompression. Untarring is effectively decompression at the user side.
Barto
Whoops! You're right, learn something new every day.
As for closing the thread, why? People are being quite civil. Are boring technical interactions not allowed on AI?
John
Carry on, carry on... (carrion? Aw crap, now we're back to the sadonecrobestiality....)
Those formats don't support random access, you provide one file (a .tar archive) and they give you a compressed version of that file. They don't know anything about its content. However, they are more efficient than other formats.
Zip, on the other hand, doesn't provide very good compression rates, but you can delete files from archives, add new files, extract just a few files etc.
If Apple intends to provide partial on-the-fly extraction in the future (display the archive as a disk image/folder, whatever), then they simply don't have much choice among the available formats.
Originally posted by John Whitney
Whoops! You're right, learn something new every day.
As for closing the thread, why? People are being quite civil. Are boring technical interactions not allowed on AI?
John
Well, I'd venture to guess 90% of the world uses Windows, and roughly half of them in turn use XP/2000/NT and NTFS, which has an utterly complex variable blocksize, which at least allows for down to 1K blocksizes.
As well, I'm not sure that HDDs have 4K blocksizes. Most modern HDDs still have 512 Byte sector/block sizes.
Originally posted by Eugene
Well, I'd venture to guess 90% of the world uses Windows, and roughly half of them in turn use XP/2000/NT and NTFS, which has an utterly complex variable blocksize, which at least allows for down to 1K blocksizes.
As well, I'm not sure that HDDs have 4K blocksizes. Most modern HDDs still have 512 Byte sector/block sizes.
As far as I know, block size is not related to the HDD in most cases. I found an Apple document stating that HFS+ has a default 4K block size, while HFS was variable (apparently, HFS has a max number of blocks, around 64k of them). I'm more used to a 1K block size on Linux filesystems, so the 4K surprised me.
It's a tradeoff. Less disk space loss in filesystem overhead, more per file.
Originally posted by John Whitney
As far as I know, block size is not related to the HDD in most cases. I found an Apple document stating that HFS+ has a default 4K block size, while HFS was variable (apparently, HFS has a max number of blocks, around 64k of them).
Not quite.
Block size is always related to hard drive size. The size of the blocks grows directly with the size of the drive. The difference today is that there are many more blocks.
HFS supports 16 bits for block allocation and a total of 65,536 blocks.
HFS+ supports 32 bits for block allocation and a total of 4,294,967,296 blocks.
Originally posted by Brad
The size of the blocks grows directly with the size of the drive.
That doesn't sound quite right either. How about, "there is a minimum size blocks have to be based on number of blocks and size of the drive, but often the size will be greater than that minimum"?
The HFS+ default 4K will be used on all hard drives until the size of the drive exceeds 16TB. After 16TB, the block size would grow directly with HDD size. The same was probably true for HFS on drives up to 256MB.
Sound correct?
John
The transition between HFS and HFS+ was very painful, because hard drives increased from ~200MB to several gigabytes before HFS+ came into operation. Block sizes of 32k and 64k led to huge bloat in volumes.
Barto
HFS divides the total space on a volume into equal-sized pieces called allocation blocks. It uses 16-bit fields to identify a particular allocation block, so there must be less than 2^16 (65,536) allocation blocks on an HFS volume. The size of an allocation block is typically the smallest multiple of 512 such that there are less than 65,536 allocation blocks on the volume (i.e., the volume size divided by 65,535, rounded up to a multiple of 512). Any non-empty fork must occupy an integral number of allocation blocks. This means that the amount of space occupied by a fork is rounded up to a multiple of the allocation block size. As volumes (and therefore allocation blocks) get bigger, the amount of allocated but unused space increases.
HFS Plus uses 32-bit values to identify allocation blocks. This allows up to 2^32 (4,294,967,296) allocation blocks on a volume. More allocation blocks means a smaller allocation block size, especially on volumes of 1 GB or larger, which in turn means less average wasted space (the fraction of an allocation block at the end of a fork, where the entire allocation block is not actually used). It also means you can have more files, since the available space can be more finely distributed among a larger number of files. This change is especially beneficial if the volume contains a large number of small files.
Originally posted by Brad
Well, I was right about the numbers I gave. The absolute *minimum* block size, though, is 512 bytes as Eugene said, not 4 KB. A quote from Apple TN #1150:
True, but I was talking about default sizes, not minimum, which as per this Apple article, is 4KB.
The actual number of files that can be stored on an Mac OS Extended (HFS Plus) volume depends on the size of the volume and the size of the files. For example, a 160 GB HFS Plus volume with the default block size of 4K will have 40 million available blocks. This volume could store up to 40 million very small files. A bigger disk with the same default block size could hold proportionately more files.
This just means that unless you explicitly modify the block size when creating the filesystem, you'll get 4KB blocks on anything smaller than a 16TB disk. Even using the minimum 512 byte blocks will work until drive sizes exceed 2TB. In either case, this is probably a fair distance off in the future (although you never know, these days... drive sizes are getting larger quickly).
I think we've hashed it out now: drive size can play a role in block sizes in HFS+, but not with any currently available drive.
John