macOS Big Sur upgrade can lead to data loss without ample storage

AppleInsider · February 9, 2021 5:32AM

An issue with Apple's macOS Big Sur installer allows users to upgrade from an earlier operating system without first verifying that the target Mac has enough free hard drive space, resulting in installation failure and data loss.

The problem is present in currently shipping Big Sur installers and appears to date back to the first macOS 11 builds launched in November, reports Mr. Macintosh. Interestingly, subsequent point releases and delta updates are unaffected by the bug.

Apple's installers fail to check system hard drive space on initiation and continue to run through the installation process until all storage is exhausted. The resulting failure can lead to an install loop, purgatory in Boot Recovery Assistant with a pop-up reading, "An error occurred preparing the software update," or display of Big Sur's Recovery startup screen that shows no startup disk available.

Macs require at least 35.5GB of free space -- not including the 13GB installer -- to upgrade to Big Sur.

According to the publication, any Mac that is compatible with macOS Big Sur or has downloaded the macOS Big Sur upgrade is vulnerable to the flaw. Testing has confirmed the issue exists in both macOS Big Sur 11.2 and 11.3 beta versions, and is possibly present in macOS Big Sur 11.1.

A failed installation might be the least of a potential upgrader's worries, however, as Macs with a T2 security chip can suffer data loss when FileVault 2 encryption is activated. As detailed by Mr. Macintosh, users are unable to use their Mac's admin password to initiate the recovery process. Further, attempting to reset the password using Personal Recovery Key or AppleID fails, while Target Disk Mode is rendered unusable on Catalina and Big Sur. Passwords do work in TDM on macOS Mojave and High Sierra, the report says.

Data recovery is possible using a system backup and a second Mac, though the process is complicated when FileVault is enabled. Macs that did not have FileVault enabled prior to upgrade can free up space for a new install through a bit of quick file management in Terminal or transferring data to a second Mac via TDM.

Alternatively, users who do not need to recover data from an affected Mac can simply erase the drive and reinstall macOS. Newer Macs with T2 chips require users to go through an "Erase Mac" step, according to the report.

Mr. Macintosh alerted Apple to the issue, and others have complained of identical problems on Apple's Support Forums, but the company has yet to respond.

commentzilla · February 9, 2021 6:06AM

"as Macs with a T2 security chip can suffer data loss when FileVault 2 encryption is activated. As detailed by Mr. Macintosh, users are unable to use their Mac's admin password to initiate the recovery process."

This doesn't mean some random form of data lose, it just means you're unable to regain access to the disk, right?

Glad I splurged for a 2TB disk with a 3 their deep backup routine!

iloveapplegear · February 9, 2021 7:13AM

commentzilla said:

"as Macs with a T2 security chip can suffer data loss when FileVault 2 encryption is activated. As detailed by Mr. Macintosh, users are unable to use their Mac's admin password to initiate the recovery process."

This doesn't mean some random form of data lose, it just means you're unable to regain access to the disk, right?

Glad I splurged for a 2TB disk with a 3 their deep backup routine!

i have a 4 tier backup routine.

1) apple time capsule 3tb
2) SSD 1tb
3) SSD 0.5tb
4) HD 2TB

commentzilla · February 9, 2021 8:15AM

iloveapplegear said:

commentzilla said:

"as Macs with a T2 security chip can suffer data loss when FileVault 2 encryption is activated. As detailed by Mr. Macintosh, users are unable to use their Mac's admin password to initiate the recovery process."

This doesn't mean some random form of data lose, it just means you're unable to regain access to the disk, right?

Glad I splurged for a 2TB disk with a 3 their deep backup routine!

i have a 4 tier backup routine.

1) apple time capsule 3tb
2) SSD 1tb
3) SSD 0.5tb
4) HD 2TB

Yep the same. I stop counting after 3.

fred1 · February 9, 2021 11:53AM

Is there any news about installing on a 2014 MacBook Pro? I know that when this OS was first released there were a lot of problems on older machines so I’ve been waiting.

lkrupp · February 9, 2021 1:37PM

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

elijahg · February 9, 2021 1:45PM

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

dewme · February 9, 2021 2:12PM

Tiered backups make sense, but if you’re really concerned about data loss at least one of these tiers should be offsite. Business users have typically used offsite options like Iron Mountain but anyone can use a cloud backup service like iDrive, BackBlaze, etc.

Backups should really not be a burden for most users, especially if you focus on backing up just your data. Even a pokey sub $100 USB hard disk backup can save your bacon and prevent data loss.

Full system backups like Carbon Copy Cloner can reduce your downtime significantly, especially when the backup image is bootable. It’s great to have at least one full system backup in addition to your data backups, but you really don’t need to be keeping multiple copies of OS and app files around if backup storage space is at a premium.

I’ve had very good success recovering my data from failed hard drives and install failures simply by using iCloud storage for my data. But I don’t consider iCloud storage to be a reliable backup service, it’s more like a file sharing cache. Even though it’s worked for me, I still use dedicated backup methods and would never rely on iCloud alone.

ApplePoor · February 9, 2021 2:21PM

I have always purchased the maximum memory and data storage on my iDevices and all of the recent Laptops where everything is now soldered in place. That future proofs against outsized OS updates and memory was much faster than the spinning hard drives back in the day. Max memory and storage increases resale value as well.

Apple should really upsize the basic memory starting points so there is actually enough room for OS updates and some user data and programs.

dewme · February 9, 2021 2:25PM

elijahg said:

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

That’s a different problem. If Apple storage was cheaper people who don’t have a sufficient backup strategy would simply lose more data when they encountered a scenario on their machine that caused data loss.

The issue at hand here is data loss caused by a broken installation program that fails in low resource scenarios. But this is only one of many failure scenarios that can lead to data loss.

MplsP · February 9, 2021 2:51PM

dewme said:

elijahg said:

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

That’s a different problem. If Apple storage was cheaper people who don’t have a sufficient backup strategy would simply lose more data when they encountered a scenario on their machine that caused data loss.

The issue at hand here is data loss caused by a broken installation program that fails in low resource scenarios. But this is only one of many failure scenarios that can lead to data loss.

They’re both issues. Apple over charges for memory and SSD storage, leading people to ‘under buy’ thinking they can simply use an external hard drive if they need to. The 20 terapixel cameras in phones these days mean pictures files are quite large, so your iCloud Photo Library rapidly eats up your hard drive without you realizing it. Finally, Macs are good at managing hard drive space in the background, so you can get dangerously low on hard disk space without ever knowing it. (This happened to me on my 2015 imac.)

Now add to this a significant flaw in the Big Sur installer and you have a setup for people losing data.

neilm · February 9, 2021 2:54PM

That 35.5GB free space requirement is surprisingly large for an installer that itself is “only” a bit over 12GB for the complete Big Sur version. Apple has made some useful changes to iOS to allow system updates to be smaller and install within reduced free storage space. Sound like the macOS group should be talking to their colleagues about that.

It’s been our experience at the office that most ordinary users don’t really keep track of disk space (and some don’t even know how), forcing me to stop buying 256GB Macs, even though we’re nominally working from file servers.

elijahg · February 9, 2021 3:17PM

dewme said:

elijahg said:

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

That’s a different problem. If Apple storage was cheaper people who don’t have a sufficient backup strategy would simply lose more data when they encountered a scenario on their machine that caused data loss.

The issue at hand here is data loss caused by a broken installation program that fails in low resource scenarios. But this is only one of many failure scenarios that can lead to data loss.

Well no, it's not, it's quite instrumental in exacerbating the bug. People are much less likely to fill a larger drive than a smaller one, so less likely to encounter the issue in the first place. People don't have requirements for unlimited amounts of space, but a 256GB base drive is pretty damn small nowadays. And £200 for 256GB more is absurd. I paid £80 for a 3GB/sec 512GB M.2 SSD; less than half the price of the extra 256GB you're buying from Apple. For £209 - the price Apple charges for 256GB extra, you can get a 2TB SSD that's again faster than the ones in the Macs. And it's not like Apple throws away the 256GB NAND when they produce a 512GB logic board, so they get half their component costs back anyway - and they're most definitely not paying retail prices. It's just a ripoff.

maciekskontakt · February 9, 2021 3:28PM

To few posters here on forum: You should stop blaming users - they create market and Apple like any other should follow - that is vendor guilt to create some schemes and not to listen to customers. You drink too much CoolAid from Apple if you think opposite is true and keep blaming users of Apple products (one of reasons I am partially moved to Android base devices including phone after years with iPhones).

Second NOTHING justifies locking Apple locking access to data owned by people even in security schemes. I noticed that even security is so skewed on Big Sur after upgrading that downloading older Catalina just to make USB stick so it could be used on other computer and you cannot remove that downloaded file from Big Sur unless you go through hoops of disabling SIP (Apple, really? Cannot remove downloaded files to clean up disk in Application space? No wonder that regular user who does not care about crazy steps of SIP may run out of space by mistake just by accumulating some files and apps).

People still prefer local storage rather than iCloud (and cloud in general) because:

1. They are in control of their data - not cloud vendor.
2. They do not want to pay subscription for cloud storage for gigabytes of dormant data they have and daily use maybe 5% of their data (maybe it is time to use something like AWS Glacier for fraction of cost by Apple and others which is deeper storage with different SLAs, but I understand that easy money for cloud vendors is better approach).
3. They do not want anybody manipulate their data for unknown reasons let alone not be sure if it is always secure or could be lost.
4. They learned that data is information about people who became product for analytics running companies - they do not want to be product and prefer making choices without being fed with commercials and ads.

maciekskontakt · February 9, 2021 3:41PM

neilm said:

That 35.5GB free space requirement is surprisingly large for an installer that itself is “only” a bit over 12GB for the complete Big Sur version. Apple has made some useful changes to iOS to allow system updates to be smaller and install within reduced free storage space. Sound like the macOS group should be talking to their colleagues about that.

It’s been our experience at the office that most ordinary users don’t really keep track of disk space (and some don’t even know how), forcing me to stop buying 256GB Macs, even though we’re nominally working from file servers.

Regardless how big it is, first obligation of any installer software for few decades now on any OS is to check free space on disk and give some buffer for potential other operations (rollback, recovery from installation failure e.t.c.) not to lose users data. Users data take priority over any OS or vendor software. Apparently arrogance of vendors like Apple grew to levels of Microsoft in '90 and post 2000 as I remember. They would learn big lesson if they entered for example financial industry if they lost any data and got sued by big banks over time for outages and recovery during disaster recovery processes.

lkrupp · February 9, 2021 4:02PM

elijahg said:

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

Do you and your cohorts ever get tired of playing that broken record?

tenthousandthings · February 9, 2021 4:27PM

I usually wait for the x.2 release, so was planning to go ahead this week. I don’t think either of these bugs apply to me, but I’ll check to make sure before I do.

I guess with macOS 11 it’s time to do a clean install. I used to always do them, just to be safe, but in recent years it hasn’t seemed necessary. The changes were more incremental than in the past. But it seems like Big Sur is more of a departure.

sflocal · February 9, 2021 4:45PM

Even with my 2020 iMac's 8TB of internal storage, 90% of my data resides in Dropbox/iCloud, and the rest is on a 24TB Promise R8 RAID tower, which is then backed up occasionally to my Promise R6 Tower. My iMac could be erased on a whim and I wouldn't really care.

Cloud services in particular have been a godsend for me. It simplified my digital life immensely.

MplsP · February 9, 2021 4:46PM

lkrupp said:

elijahg said:

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

Do you and your cohorts ever get tired of playing that broken record?

No, but we do get tired of overpaying for memory and SSD space. Do you ever get tired of making excuses for Apple’s memory prices?

Example: M1 Mac Mini 512 vs 256GB SSD is $899 vs $699. A 250GB SSD is $55, a 500GB SSD is $80. Suddenly $25 becomes $200???

dewme · February 9, 2021 5:28PM

MplsP said:

dewme said:

elijahg said:

lkrupp said:

It has always amazed me how many people are apparently running their systems with almost all of their storage space in use. You can fill up a 256GB or 500GB storage device pretty quickly with photos, videos, and music.

That’s what happens when people can’t afford the extortionate Apple storage upgrades when purchasing a Mac, and have soldered on NAND.

That’s a different problem. If Apple storage was cheaper people who don’t have a sufficient backup strategy would simply lose more data when they encountered a scenario on their machine that caused data loss.

The issue at hand here is data loss caused by a broken installation program that fails in low resource scenarios. But this is only one of many failure scenarios that can lead to data loss.

They’re both issues. Apple over charges for memory and SSD storage, leading people to ‘under buy’ thinking they can simply use an external hard drive if they need to. The 20 terapixel cameras in phones these days mean pictures files are quite large, so your iCloud Photo Library rapidly eats up your hard drive without you realizing it. Finally, Macs are good at managing hard drive space in the background, so you can get dangerously low on hard disk space without ever knowing it. (This happened to me on my 2015 imac.)

Now add to this a significant flaw in the Big Sur installer and you have a setup for people losing data.

This is an installer issue. If the Big Sur installation was catastrophically failing and causing data loss on systems that had a certain discrete video card, the root cause of the failure would still be the installer program, not the video card. Installation program failures are the worst of the worst. Apple should never allow an installation to start if the installation cannot complete successfully, or at least roll itself back to a known good state with all of the users data intact.

This issue is gross negligence by Apple and the folks who built and (failed to adequately) test their installer program under customer representative scenarios - including all combinations of memory, storage, video, T1 chip present/not present, etc. Customer data should be treated like currency. Losing customer data is always a very bad thing. The total number of system configuration combinations that Apple has to consider is much more constrained on macOS than on other platforms, like Windows, and is further reduced with the number of older systems that are forcibly deemed and enforced as incompatible. Getting installs and upgrades right should be a relative strength for Apple. They don't fail often, but they failed this time.

Yes, all Apple customers should back up their data, it's common sense. But Apple should not release any application that puts customer data at risk. When I drive my Honda I wear a seatbelt. But my wearing a seatbelt doesn't give Honda the right to ram a truck into my car. Apple created the hazard, regardless of what its users did, including how much storage they purchased when they bought their Apple computer.

I totally agree that Apple overcharges for memory and storage upgrades compared to many other computer vendors. That's a different topic for another discussion. Bringing it up here diminishes the severity and focus of what Apple has done in this particular scenario, i.e., releasing a faulty and highly destructive installer on its customers.

elijahg · February 9, 2021 9:10PM

I imagine another reason for this is the complexity (bloat?) of the modern macOS installers. They shuffle partitions around, add partitions (preboot, VM, EFI, recovery), switch and upgrade partition schemes, fiddle around with partition types, move "incompatible" files into a folder, convert from CoreStorage to APFS containers, switch from HFS+ to APFS, fiddle with APFS versions, add snapshots, then move user data to a separate partition, and add hard links to user data. Then after all that it upgrades the recovery partition. No wonder it miscalculates the required space occasionally

The installer installs into a folder then "shoves" the contents to where it's supposed to be, which is supposed to avoid things like this, though that won't help fiddling with partitions.

macOS Big Sur upgrade can lead to data loss without ample storage

Comments