How Apple dodged the Heartbleed bullet

Posted:
in iPhone edited April 2014
In 2011, Apple told its developers that it would be deprecating OS X's Common Data Security Architecture including OpenSSL, describing it as an outdated relic of the late 1990s. Nearly three years later, OpenSSL was hit by a severe flaw that affected a wide swath of vendors and their users, but not Apple.

Heartbleed


When it announced plans to deprecate OpenSSL in June 2011, Apple wasn't aware of the Heartbleed flaw because it didn't yet exist. However, the company was aware of other problems with OpenSSL (libcrypto), a security toolkit Apple began using within the Common Data Security Architecture more than a decade ago.

CDSA, according to the Open Group that designed it, "is a set of layered security services and cryptographic framework that provide an infrastructure for creating cross-platform, interoperable, security-enabled applications for client-server environments is an architecture."

Apple incorporated support for CDSA and OpenSSL in its early development of Mac OS X. In 2004, Apple was recommending that Mac developers adopt CDSA, noting that it "will improve the overall performance of the system by reducing the number of libraries that frameworks link against to do cryptography."

As the company noted in its Mac security documentation from a decade ago, "CDSA is an Open Source security architecture adopted as a technical standard by the Open Group. Apple has developed its own Open Source implementation of CDSA, available as part of Darwin at Apple's Open Source site. This API provides a wide array of security services, including fine-grained access permissions, authentication of users' identities, encryption, and secure data storage."

Apple builds its own security architecture

By at least 2006 however, Apple began working on a new cryptography API for the future, designed to use less code, run faster and support concurrent use of multiple processors. These features were not only necessary for future Macs, but would also be critically important to iOS. Apple began working on a new cryptography API for the future, designed to use less code, run faster and support concurrent use of multiple processors

The desire to build a streamlined, modern security architecture was also driven by a need for FIPS 140-2 validation, required to sell devices to a variety of U.S. government agencies. As sales of iPhone and later iPad began to explode, Apple's efforts to address a robust alternative to the outdated CDSA took on new urgency.

The first step was Common Crypto, a low level C framework supporting core encryption algorithms Apple first released for OS X 10.5 Leopard in 2007 and later brought to iOS 5 in 2011. Apple has continued to work on making low level crypto functions easier for developers to use.

That includes Apple's OS X Security Transforms package, which is deeply integrated with Grand Central Dispatch to enable pipelines of data (including encryption tasks) to be spread out across available processors. It also supports hardware acceleration of crypto functions on modern processors like Intel's Core i5 and i7.

Apple deprecates CDSA & OpenSSL

By 2011, Apple was ready to deprecate CDSA, noting to developers at its WWDC event that the architecture was based on an Open Group standard that few other vendors supported besides Apple, and included lots of features nobody actually used. That required Apple to assume and manage a lot of complex external issues without any real cross-platform benefit.

"CDSA has its own standard programming interface, it is complex and does not follow standard Apple programming conventions," the company noted to its developers in Mac security documentation. iOS never incorporated CDSA, and both OS X and iOS "include their own higher-level security APIs that abstract away much of that complexity."

Building its own security software meant that Apple and its developers were no longer captive to the external development issues and eccentricities related to the OpenSSL open source project, which despite its critical importance and broad use by the industry, was being funded through donations and was, incredibly, maintained by a very small team of just four core developers."OpenSSL does not provide a stable API from version to version. For this reason, although OS X provides OpenSSL libraries, the OpenSSL libraries in OS X are deprecated, and OpenSSL has never been provided as part of iOS. Use of the OS X OpenSSL libraries by apps is strongly discouraged "

"Although OpenSSL is commonly used in the open source community," Apple stated in its documentation, "OpenSSL does not provide a stable API from version to version. For this reason, although OS X provides OpenSSL libraries, the OpenSSL libraries in OS X are deprecated, and OpenSSL has never been provided as part of iOS. Use of the OS X OpenSSL libraries by apps is strongly discouraged.

"If your app depends on OpenSSL, you should compile OpenSSL yourself and statically link a known version of OpenSSL into your app. This use of OpenSSL is possible on both OS X and iOS. However, unless you are trying to maintain source compatibility with an existing open source project, you should generally use a different API."

Apple's concern about OpenSSL lacking a "stable API from version to version" relates to the complications it would face in trying to update or patch security flaws in the open source software package in a way that wouldn't break third party apps wired to a previous version of OpenSSL. Deprecating OpenSSL in favor of its own software meant that Apple had greater control in managing its own platform.

A broad variety of vulnerabilities in Apple's OS X software have actually related to outside software that Apple has bundled with its own, including both open source software packages and third party commercial components like Adobe Flash.

Heartbleed hits OpenSSL

Apple's timing proved to be fortuitous. Just six months after Apple officially deprecated OpenSSL, the Heartbleed flaw was inadvertently introduced in OpenSSL via a Heartbeat feature designed to keep secure connections alive and active. The flawed Heartbeat feature was included in the following March 2012 release of OpenSSL, and enabled by default.

While Apple had been advising its Mac and iOS developers to use other software before the bug had ever been introduced and never distributed the subsequent versions of OpenSSL that incorporated the security flaw, much of the rest of the industry had been standardizing on the latest, freely available version of OpenSSL.

More than two years later, a researcher at Google discovered that the OpenSSL Heartbeat feature was flawed, potentially allowing a malicious user to "bleed" data from a server using an affected version of OpenSSL, and possibly even recover security keys that could be used to spy on intercepted streams of encrypted data. Client software affected by Heartbleed could also be exploited by a malicious server.

"Servers vulnerable to Heartbleed are less secure than they would be if they simply had no encryption at all," noted a report by The Guardian

According to a report by Brendan Sasso of the National Journal, Google began work on addressing the flaw internally without telling anyone else about it, not even the U.S. government, which ostensibly wasn't aware of the vulnerability until Google first disclosed it on April 1 via the company's Google Plus social network.

A timeline compiled by Ben Grubb of the Sydney Morning Herald indicates that various firms over the next week battled both for publicity and against public disclosure of the Heartbleed flaw, with security companies seizing upon it as a way to make a name for themselves, and those affected scrambling to address the problem before they and their clients could be exploited by third parties armed with the same knowledge.

The perceived advantage of open software being innately more secure through broad use and exposure to more eyeballs ran into the reality of disadvantages involved with broad industry reliance upon a widely distributed monoculture of software developed by relatively few people who didn't necessarily share the same design goals as their broad spectrum of users (including that lack of interest in maintaining API compatibility).

A flaw in Apple's own code

Apple and its Mac and iOS users weren't affected by Heartbleed, but just weeks before, the company had been hit by a similar vulnerability related to a flaw in Apple's own code, which just happened to also be related to SSL certificate based security.

In Apple's case, the flaw, branded as "GoToFail," related to code the company maintained itself, although like OpenSSL, Apple's code had also been published as open source. As with OpenSSL, merely being open to eyeballs didn't result in Apple's code being free of undiscovered flaws.

Apple was condemned in a series of posts laced with profanity for patching iOS first (before GoToFail was publicly known about) and not releasing a patch for OS X until three days later.

In contrast, it took a week for the various parties involved in Heartbleed to even coordinate its disclosure, with embargo leaks informing some clients, including OpenSSL, Akamai and Facebook as much as several days before the general public and even major companies including Cisco, Dropbox, Juniper, Twitter, Ubuntu and Yahoo.

Another security flaw, similarly affecting network security, was identified in Android's WebView 16 months ago. While much more serious in that it provided full control of a device to remote malicious users and had functional tools available that allowed virtually anyone to exploit the flaw, roughly 75 percent of Android devices appear to remain vulnerable.
«134

Comments

  • Reply 1 of 68
    gqbgqb Posts: 1,934member
    "The perceived advantage of open software being inanely more secure..."

    Wonderfully apt misspelling.
  • Reply 2 of 68
    asdasdasdasd Posts: 5,686member
    It's funny how there doesn't seem to have been any review at all of either the goto fail or heart bleed. Not internal nor external. Not a unit nor integration test.
  • Reply 3 of 68
    I know this sounds naïve. But this is why I trust and love Apple!

    I remember when Apple shitcanned flash for iOS and when Steve was giving a Keynote and said Adobe always opens slowly. I deleted Adobe photo elements because of that reason and I decided just to stay with Apple software even though it was flawed at times. Eg. MobileMe, early Pages, Apple Mail, Notes on iOS, no cut and paste on first iPhone, Maps, etc. even with the minimal flaws, the interrelation between Mail, iPhoto, iWeb and OSX and iOS was so superior to anything MS or Google were slapping together!
  • Reply 4 of 68
    "Open source = security" fans aren't fuming about this, they're defensive. Their usual FUD retort is: "but but but who knows how many security holes lurk in private code?"

    Well now you can argue the same about open source. Who knows? Because the OpenSSL users didn't, for two years.
  • Reply 5 of 68
    Dan_DilgerDan_Dilger Posts: 1,583member
    Quote:

    Originally Posted by asdasd View Post



    It's funny how there doesn't seem to have been any review at all of either the goto fail or heart bleed. Not internal nor external. Not a unit nor integration test.

     

    There doesn't seem to be any thing going on at Apple: no innovation, no reviews of security issues... oh wait, maybe because its internal.

  • Reply 6 of 68
    sgk77sgk77 Posts: 4member
    I propose a toast to the team of developers at Apple who developed its in-house security layers. It probably wasn't the most glamorous project to be on, but based on recent events it certainly turned out to be one of the most important ones.

    Cheers everyone!
  • Reply 7 of 68
    gatorguygatorguy Posts: 24,176member
    "Open source = security" fans aren't fuming about this, they're defensive. Their usual FUD retort is: "but but but who knows how many security holes lurk in private code?"

    Well now you can argue the same about open source. Who knows? Because the OpenSSL users didn't, for two years.

    One of the primary reasons Apple gives for using open-source solutions is "Open Source methodology makes Mac OS X a more robust, secure operating system, as its core components have been subjected to the crucible of peer review for decades. Any problems found with this software can be immediately identified and fixed by Apple and the Open Source community."

    Pretty shocking when a big ol' rock like this pops out of the ground after two years.
  • Reply 8 of 68
    asdasdasdasd Posts: 5,686member
    There doesn't seem to be any thing going on at Apple: no innovation, no reviews of security issues... oh wait, maybe because its internal.

    Ridiculous sneer. The goto bug was clearly visible in code and should have been caught in one of a unit, integration or general user tests. The latter being as simple as just trying a bad cert in safari. See if that worked.

    The OpenSSL bug seems to have been submitted without review or unit test but it was a big addition in functionality. One guy wrote it nobody seemed to review it, it was built into a final product and half the world picked it up.

    Ok most people are not security experts but that's insane. Probably all this works at too low a level for most devs.

    In general this was a good article though. I think open source is largely a crock unless there are serious paid engineers behind it. Otherwise it's amateur hour.
  • Reply 9 of 68
    sflocalsflocal Posts: 6,092member

    Sounds like Apple was being more proactive than dodging any kind of bullet.  Kudos to them!  Of course, one doesn't hear anything in the news that would ever put Apple in a positive role.  

  • Reply 10 of 68
    sgk77 wrote: »
    I propose a toast to the team of developers at Apple who developed its in-house security layers. It probably wasn't the most glamorous project to be on, but based on recent events it certainly turned out to be one of the most important ones.

    Cheers everyone!

    I agree. Very good of u to say! :)
  • Reply 11 of 68
    haggarhaggar Posts: 1,568member
    Quote:

    Originally Posted by sflocal View Post

     

    Sounds like Apple was being more proactive than dodging any kind of bullet.  Kudos to them!  Of course, one doesn't hear anything in the news that would ever put Apple in a positive role.  


    Thankfully we have sites like AI, so everything that Apple does is put in a positive role.

  • Reply 12 of 68
    desuserigndesuserign Posts: 1,316member
    Quote:
    Originally Posted by Suddenly Newton View Post



    "Open source = security" fans aren't fuming about this, they're defensive. Their usual FUD retort is: "but but but who knows how many security holes lurk in private code?"



    Well now you can argue the same about open source. Who knows? Because the OpenSSL users didn't, for two years.

     

    I don't know anyone who would say open source = security (If anyone does I've got some free open source software I can give them!) But open source does equate to transparency, which it seems (counterintuitively enough) is a prerequisite to verifiable security. Clearly Apple thinks so.

  • Reply 13 of 68
    nobodyynobodyy Posts: 377member
    Quote:
    Originally Posted by asdasd View Post





    Ridiculous sneer. The goto bug was clearly visible in code and should have been caught in one of a unit, integration or general user tests. The latter being as simple as just trying a bad cert in safari. See if that worked.



    The OpenSSL bug seems to have been submitted without review or unit test but it was a big addition in functionality. One guy wrote it nobody seemed to review it, it was built into a final product and half the world picked it up.



    Ok most people are not security experts but that's insane. Probably all this works at too low a level for most devs.

     

    This is entirely false. The OpenSSL Heartbeat component was submitted with review and had many tests written against it - and albeit that bug remained consistent throughout, you can't test every possible scenario until they arise or have been thought of ahead of time. Unfortunately, this goes to show how hard a programmers job is, and that nothing is full proof, and no matter how obvious, problems can be masked and overlooked. We all make mistakes regardless of environment and sometimes those mistakes slip through the cracks. 

  • Reply 14 of 68
    bighypebighype Posts: 148member
    Google kept Heartbleed secret for over a week and no one is giving sh*t to them. They're f'n evil!
  • Reply 15 of 68
    gatorguygatorguy Posts: 24,176member
    bighype wrote: »
    Google kept Heartbleed secret for over a week and no one is giving sh*t to them. They're f'n evil!

    According to the original report of the flaw "This bug was independently discovered by a team of security engineers (Riku, Antti and Matti) at Codenomicon and Neel Mehta of Google Security, who first reported it to the OpenSSL team. Codenomicon team found heartbleed bug while improving the SafeGuard feature in Codenomicon's Defensics security testing tools and reported this bug to the NCSC-FI for vulnerability coordination and reporting to OpenSSL team."

    "Immediately after our discovery of the bug on 3rd of April 2014, NCSC-FI took up the task of verifying it, analyzing it further and reaching out to the authors of OpenSSL, software, operating system and appliance vendors, which were potentially affected. However, this vulnerability had been found and details released independently by others before this work was completed. Vendors should be notifying their users and service providers. Internet service providers should be notifying their end users where and when potential action is required."

    http://heartbleed.com/
  • Reply 16 of 68
    Quote:
    Originally Posted by Nobodyy View Post

     

     

    This is entirely false. The OpenSSL Heartbeat component was submitted with review and had many tests written against it - and albeit that bug remained consistent throughout, you can't test every possible scenario until they arise or have been thought of ahead of time. Unfortunately, this goes to show how hard a programmers job is, and that nothing is full proof, and no matter how obvious, problems can be masked and overlooked. We all make mistakes regardless of environment and sometimes those mistakes slip through the cracks. 


     

    OpenSSL is written by monkeys

  • Reply 17 of 68
    solipsismxsolipsismx Posts: 19,566member
    gatorguy wrote: »
    One of the primary reasons Apple gives for using open-source solutions is "Open Source methodology makes Mac OS X a more robust, secure operating system, as its core components have been subjected to the crucible of peer review for decades. Any problems found with this software can be immediately identified and fixed by Apple and the Open Source community."

    Pretty shocking when a big ol' rock like this pops out of the ground after two years.

    Apple not wanting to use OpenSSL because they wanted a modern security architecture doesn't mean they are hypocritical about supporting open source. The statement you quote from their list page of open source SW in Mac OS X in no way violates that claim that open source can be a powerful tool.
  • Reply 18 of 68
    gatorguygatorguy Posts: 24,176member
    solipsismx wrote: »
    Apple not wanting to use OpenSSL because they wanted a modern security architecture doesn't mean they are hypocritical about supporting open source. The statement you quote from their list page of open source SW in Mac OS X in no way violates that claim that open source can be a powerful tool.

    Don't think anyone implied they were being hypocritical.
  • Reply 19 of 68
    nobodyynobodyy Posts: 377member
    Quote:

    Originally Posted by capasicum View Post

     

     

    OpenSSL is written by monkeys


    http://www.quora.com/Whose-fault-is-the-Heartbleed-bug

     

    I think the answers here, and specifically the first, generally reflect my feelings about this. 

  • Reply 20 of 68
    solipsismxsolipsismx Posts: 19,566member
    gatorguy wrote: »
    Don't think anyone implied they were being hypocritical.

    My bad. I thought you were quoting those two lines from their two paragraph statement about open source because it said "Open Source […] makes Mac OS X a more […] secure operating system" and that it's easier to fix.
Sign In or Register to comment.