Delta CEO criticizes Microsoft's fragility, praises Apple's stability

AppleInsider

Delta's CEO has decried Microsoft as a particularly vulnerable platform while implying Apple is much more sound.

When a faulty update crippled the internet in mid-July, causing everything from point-of-purchase to flight management to grind to a halt, many companies began looking for someone to blame. And that blame largely fell on Microsoft and security firm CrowdStrke.

Some of the loudest affected companies were flight companies, such as American Airlines, United, and Delta, who needed to ground flights until their systems came back online.

In a new interview on CNBC's 'Squawk Box', Delta CEO Ed Bastian has spoken out against Microsoft while simultaneously praising Apple. When asked if Delta would reconsider how it used Microsoft in the future, Bastian had this to say:

"We have to. My sense is [Microsoft is] probably the most fragile platform within that space... When was the last time you heard about a big outage at Apple?"

When the interviewer pressed Bastian to consider if the reason Apple hasn't had an outage like this is because it's not as widely utilized, the CEO ducked the question entirely.

Delta is currently looking to sue both Microsoft and CrowdStrike to recoup its alleged $500 million profit loss due to technical problems.

Delta has integrated some Apple products into various parts of its business, either directly or indirectly. Delta was the first airline to use Apple's business chat to help customers.

In 2021, Delta Air Lines provided its pilots with an upgraded electronic flight bag, switching over to the 5G-equipped iPad Pro.

In December 2023, a TikToker discovered that a newer Delta plane lets you directly connect your AirPods -- or any Bluetooth headphones -- to the in-flight entertainment system.



Read on AppleInsider

baconstang

Welcome to the party...

eriamjh

Hello, Pot?  This is Kettle.

Any airline calling Microsoft "fragile" needs to look at their own horrible financial performance and overall system "fragility"over the decades since the inception of flight.

Isn't every airline one economic blip from bankruptcy?  Delta has already been there (2005).

I'm not saying he isn't wrong.  Please, sue M$ over this.   They'll just blame Cloudstrike.

kellie

The primary cause of the outage was due to Crowdstrike shipping a faulty malware template that either wasn’t tested or was accidentally shipped out unknowingly 

the secondary cause is every Crowdstrike customer who blindly allowed the update to occur without doing their own testing and validation.  Sure it would be nice to totally trust the vendor, but they don’t suffer the consequences.  

The third cause of the failure was Microsoft allowing Crowdstrike to run in kernel mode.  So when Crowdstrike choked, it brought down each and every PC. A simple reboot would not fix the problem.  

The fourth cause of the problem is EU regulators who told Microsoft that if they were going to force applications vendors to use a Microsoft API to access the kernel, that would be considered anti-competitive. So Microsoft had no choice but to allow Crowdstrike unprotected access to the kernel.  Apple is allowed to control access to its kernel. 

The Delta CEO is criticizing Microsoft purely to try and squeeze money out of Microsoft.  The CEO is clueless about Windows vs Mac.  He’s just trying to shake down Microsoft. The Delta CEO needs to ask the Delta CIO why he allowed Crowdstrike updates to occur without validation and testing.  Trust but verify as Ronald Regan used to say about the Russians. 

Stabitha_Christie

kellie said:

The primary cause of the outage was due to Crowdstrike shipping a faulty malware template that either wasn’t tested or was accidentally shipped out unknowingly 

the secondary cause is every Crowdstrike customer who blindly allowed the update to occur without doing their own testing and validation.  Sure it would be nice to totally trust the vendor, but they don’t suffer the consequences.  

The third cause of the failure was Microsoft allowing Crowdstrike to run in kernel mode.  So when Crowdstrike choked, it brought down each and every PC. A simple reboot would not fix the problem.  

The fourth cause of the problem is EU regulators who told Microsoft that if they were going to force applications vendors to use a Microsoft API to access the kernel, that would be considered anti-competitive. So Microsoft had no choice but to allow Crowdstrike unprotected access to the kernel.  Apple is allowed to control access to its kernel. 

The Delta CEO is criticizing Microsoft purely to try and squeeze money out of Microsoft.  The CEO is clueless about Windows vs Mac.  He’s just trying to shake down Microsoft. The Delta CEO needs to ask the Delta CIO why he allowed Crowdstrike updates to occur without validation and testing.  Trust but verify as Ronald Regan used to say about the Russians. 

The EU didn’t force Microsoft to do anything. The EU had complaints from security software vendors and asked Microsoft to respond. Microsoft responded with a solution a solution that provided security vendors the ability to live patch the Windows kernel. It was Microsoft’s solution and it was completely voluntary. That Microsoft is trying to blame the EU is just completely dishonest on their part. 

That AppleInsider repeated Microsoft’s talking points without validating the claims was just sloppy. Paul Thurrott, a long time proponent of Windows and Microsoft, actually did work to validate the claim and called it for the BS it was. That a Microsoft fan put in more effort than AI is just embarrassing.

danox

Apple kicked third parties out of the OS Kernel Microsoft can do the same if they grow a backbone, what the Delta CEO said is a hint to Apple that it must start looking at leveraging Apple Silicon computers in small and medium sized businesses in some of the back of house computing solutions ie... servers/software the time is coming like right now.

Microsoft is at the top of the Windows Azure inertia pyramid, and is responsible but notice in typical IT Geek Boy fashion Microsoft blamed others for their incompetence. Recall and the Qualcomm SOC/Windows emulation for third time fiasco is theirs too.

Heard and seen it all before at the company I worked for over the years when Windows IT was involved.

DAalseth

If Rome had been built like modern computer systems, the first woodpecker would have destroyed western civilization.

As others have pointed there is a lot of blame to go around. I however agree with Delta’s CEO. Windows is way too fragile for us to be using it as a foundation for our whole economy and safety net. I finally got out of IT because I was fed up with replacing crappy broken Windows systems with crappy NEW windows systems. This was an accident. The next one may be as well. Down the road though it won’t be. Someone will deliberately target the house of cards that is Windows and we will discover that everything grinds to a halt, melts down, or bursts. Then rebooting to Safe Mode and deleting a file won’t fix it. 

22july2013

Stabitha_Christie said:
The EU didn’t force Microsoft to do anything. The EU had complaints from security software vendors and asked Microsoft to respond. Microsoft responded with a solution a solution that provided security vendors the ability to live patch the Windows kernel. It was Microsoft’s solution and it was completely voluntary. That Microsoft is trying to blame the EU is just completely dishonest on their part. 

So if the law says I can't speed, and I don't speed, that's "voluntary" on my part? Hardly. Even though my decision not to speed was my own, I can still blame the government for setting the speed limit and "forcing" me to drive slowly. Your comparison is completely unconvincing.

There's plenty of blame to go around. I wouldn't say it was all the EU's fault, but the EU set some rules and Microsoft complied. And the EU validated and approved Microsoft's solution. The EU is at the top of the pyramid of errors in this situation.

danox

DAalseth said:

If Rome had been built like modern computer systems, the first woodpecker would have destroyed western civilization.

As others have pointed there is a lot of blame to go around. I however agree with Delta’s CEO. Windows is way too fragile for us to be using it as a foundation for our whole economy and safety net. I finally got out of IT because I was fed up with replacing crappy broken Windows systems with crappy NEW windows systems. This was an accident. The next one may be as well. Down the road though it won’t be. Someone will deliberately target the house of cards that is Windows and we will discover that everything grinds to a halt, melts down, or bursts. Then rebooting to Safe Mode and deleting a file won’t fix it. 

Microsoft needs to fix their OS kernel, AI and the Surface need to take a back seat Nadella probably needs to get those third parties out kicking and screaming and that is going to be a massive undertaking?

Imagine if Microsoft had forked Windows and fixed the Kernel outside the EU and the EU was the only place where this fiasco happened? If the EU continues they will be left behind outside the EU it needs to be fixed.

Right now there a little shock in the system for American tech companies but soon they get over it and start to design around the EU.

netrox

That's just misleading to blame MS when CrowdStrike was the one that did the updates that were clearly not carefully tested for QA. 

I love Apple and think MacOS is far better than Windows but blaming MS for that outage is just ridiculous. 

danox

netrox said:

That's just misleading to blame MS when CrowdStrike was the one that did the updates that were clearly not carefully tested for QA. 

I love Apple and think MacOS is far better than Windows but blaming MS for that outage is just ridiculous. 

Microsoft are the experts and they are the top of the Windows world right? They are to blame. Four-five years ago Apple fixed their OS, Microsoft should fork the EU and fix it everywhere else?

https://commission.europa.eu/news/ai-act-enters-force-2024-08-01_en  This edict by the EU will stall AI development within the EU should the American tech companies wait? The genie is out of the bag and if you are in computer tech in America you cannot wait...... If you do your future as a tech company is in danger.

wrbluepearl

These companies would still run CrowdStrike Falcon on their Macs if they went with Apple products. Probably lower risk of a kernel panic due to a bad update, but that wouldn’t mean things wouldn’t break. 

Sadly, I feel like the Mac, Linux and BSD people shouldn’t get too smug here - it just wasn’t our turn this time. 

sevenfeet

danox said:

DAalseth said:

If Rome had been built like modern computer systems, the first woodpecker would have destroyed western civilization.

As others have pointed there is a lot of blame to go around. I however agree with Delta’s CEO. Windows is way too fragile for us to be using it as a foundation for our whole economy and safety net. I finally got out of IT because I was fed up with replacing crappy broken Windows systems with crappy NEW windows systems. This was an accident. The next one may be as well. Down the road though it won’t be. Someone will deliberately target the house of cards that is Windows and we will discover that everything grinds to a halt, melts down, or bursts. Then rebooting to Safe Mode and deleting a file won’t fix it. 

Microsoft needs to fix their OS kernel, AI and the Surface need to take a back seat Nadella probably needs to get those third parties out kicking and screaming and that is going to be a massive undertaking?

Imagine if Microsoft had forked Windows and fixed the Kernel outside the EU and the EU was the only place where this fiasco happened? If the EU continues they will be left behind outside the EU it needs to be fixed.

Right now there a little shock in the system for American tech companies but soon they get over it and start to design around the EU.

As usual, this ends up being a pretty complicated situation that has predictably devolved into a lot of finger pointing. So here is my take, as someone who actually works in enterprise support for a well known international tech company.

1. Crowdstrike. Yes, they deserve the lion's share of the blame here. While I get the fact that there is often a race against the clock to get out new descriptions for Crowdstrike Falcon to protect clients and servers from active threats, they should understand that they can also bring computers completely down with bad software or description files. And not only does this get back to having a 24/7 testing regime to make sure that nothing hits the download servers until it has been vetted....period. Clearly that was not going on, and its very inexcusible.  Second, their software which has kernel access on Windows has lousy file validation to inspect description files before using them. Dave Plummer, former Microsoft Windows programmer all the way back in the Windows 3.1/NT days said on his Youtube channel that the lack of error correction in their product reeked of incompetence. I would agree. This is also not the first time this has happened to Crowdstrike. They had two other issues with Linux distros in recent years, but it didn't make the headlines since it wasn't something widely used in critical systems like RedHat.

2. Microsoft. They don't get off easy here. They let a 3rd party kernel driver into Windows that could update description files without going through all the possible scenarios of how a bad update could bork an entire machine (or in this case, 8.5 million of them). And Microsoft has to think about all the ways a bad actor or state sponsor could use the very same flaw to create all kinds of havok worldwide. Break into a Crowdstrike download server (not easy, but not impossible) and you could paralyze millions of machines intentionally. And yes, I know Apple figured this out years ago and ejected 3rd party kernel extensions from the OS, replacing it with a security framework that gives certain software lower level access to restricted APIs without running in kernel mode. Yes they ran afoul of the EU but they needed to make a better case on why this would be better for the world and still allow competition. Finally, this situation exposed a critical problem where this kind of scenario not only crashed millions of machines, but fixing them literally took IT professionals to put hands on the machine. For Delta, this took DAYS to recover from...no wonder their CEO is pissed. I don't agree on forking Windows to solve this.  Windows is complicated enough as it is.

3. The EU. Leave the security technology issues to the professionals. Sometimes they know better than you.

heinzel

kellie said:

...
the secondary cause is every Crowdstrike customer who blindly allowed the update to occur without doing their own testing and validation.  Sure it would be nice to totally trust the vendor, but they don’t suffer the consequences.  
...

There apparently was no mechanism for Crowdstrike customers to prevent the Rapid Response template update. This is an automated process that Crowdstrike intends to change in the future to allow customers granular control over the process (see Crowdstrike's Executive Summary):

What is CrowdStrike Doing to Prevent This From Happening Again?

...
- Provide customers with greater control over the delivery of Rapid Response Content updates by allowing granular selection of when and where these updates are deployed.
...

BlueLightning

Thankfully, Delta and most other major airlines have far fewer plane crashes than Windows has crashes.  

Maybe the FAA should be given authority to test and set requirements and procedures for all computer systems that are used by airlines.  This is not the wild 1930s.  It is high time that minimum standards for passenger scheduling, crew scheduling and flight scheduling are set.  There is a reason Windows is rarely if ever used for critical functions on spacecraft or weapons systems.  

uffenman

Great breakdown. Thank you Kellie. At the end of the day, you gotta be careful who you marry. Even only as an enduser at this point in my life, I wouldn't own anything with Microsoft as a partner. Those days are LONG GONE. Much less run a billion dollar business. This might actually be an incredible tipping point for the corporate business market to move away from Windows.



kellie said:

The primary cause of the outage was due to Crowdstrike shipping a faulty malware template that either wasn’t tested or was accidentally shipped out unknowingly 

the secondary cause is every Crowdstrike customer who blindly allowed the update to occur without doing their own testing and validation.  Sure it would be nice to totally trust the vendor, but they don’t suffer the consequences.  

The third cause of the failure was Microsoft allowing Crowdstrike to run in kernel mode.  So when Crowdstrike choked, it brought down each and every PC. A simple reboot would not fix the problem.  

The fourth cause of the problem is EU regulators who told Microsoft that if they were going to force applications vendors to use a Microsoft API to access the kernel, that would be considered anti-competitive. So Microsoft had no choice but to allow Crowdstrike unprotected access to the kernel.  Apple is allowed to control access to its kernel. 

The Delta CEO is criticizing Microsoft purely to try and squeeze money out of Microsoft.  The CEO is clueless about Windows vs Mac.  He’s just trying to shake down Microsoft. The Delta CEO needs to ask the Delta CIO why he allowed Crowdstrike updates to occur without validation and testing.  Trust but verify as Ronald Regan used to say about the Russians. 

harrykatsaros

My mum works for Air New Zealand and that entire company runs on macOS, so there is some kind of existing infrastructure in place for Delta to make that move if he is serious. The day of the Crowdstrike update they were mostly business as usual as everything and everyone around them was crumbling. 

uffenman

CrowdStrike, the very company whose flagship product description hails: "Next-gen antivirus and threat intelligence for greater insight into your environment. Automated threat investigations accelerate alert, triage and response." can't protect their customers from the inside. This is actually a joke, and companies pay for it.

Stabitha_Christie

22july2013 said:

Stabitha_Christie said:
The EU didn’t force Microsoft to do anything. The EU had complaints from security software vendors and asked Microsoft to respond. Microsoft responded with a solution a solution that provided security vendors the ability to live patch the Windows kernel. It was Microsoft’s solution and it was completely voluntary. That Microsoft is trying to blame the EU is just completely dishonest on their part. 

So if the law says I can't speed, and I don't speed, that's "voluntary" on my part? Hardly. Even though my decision not to speed was my own, I can still blame the government for setting the speed limit and "forcing" me to drive slowly. Your comparison is completely unconvincing.

There's plenty of blame to go around. I wouldn't say it was all the EU's fault, but the EU set some rules and Microsoft complied. And the EU validated and approved Microsoft's solution. The EU is at the top of the pyramid of errors in this situation.

That is false. The had no regulations, requirements or rulings that required Microsoft to take the action it did. And this is really easy, the EU publishes its regulations and rulings online. If you are claiming such a thing exists, link to it. I can save you a bunch of time, it doesn’t exits. 

What Microsoft is doing it playing off the current sentiment about the EU and DMA. They are betting that people will accept what they are saying, look at the complaints from Apple, Facebook and Google and say “Yeah, that checks out”. And to a certain extent they are correct, there seem to be no shortage of gullible people that are happy to just blame the EU. Fortunately there are people that approach claims like this with a healthy sense of skepticism and actually dug into it… it’s kind of what journalism is about. 

Here is a good article on the chronology of the whole thing It is written by a person that is a self described “Microsoft guy” but he doesn’t let his personal like for the company to get in the way of reporting accurately. It is behind a pay wall but if you create a free account you can read it. It does offer some context of why MS made such a terrible decision but it was still a MS decision, not something the EU made them do. 

DAalseth

netrox said:

That's just misleading to blame MS when CrowdStrike was the one that did the updates that were clearly not carefully tested for QA. 

I love Apple and think MacOS is far better than Windows but blaming MS for that outage is just ridiculous. 

If you give a gun to a monkey, and the monkey shoots someone, you don’t blame the monkey!
Yes Crowdstrike messed up really bad. But Microsoft’s OS should not have allowed this kind of problem to snowball. It is possible to engineer safeguards. When I was learning programming, long ago, our programs ran with Admin Privileges. But if we made a mistake, there was a super-admin daemon that would just stop whatever our program was doing. Windows has no such thing.

dewme

The one thing in Windows that I’ve always despised is the Windows Registry. It’s like an Eiffel Tower built out of toothpicks and held together by water soluble glue. Despite all of its shortcomings it’s still in there on the latest versions of Windows, like a malignant tumor.