Over 4 million people participate in Apple's software beta programs

Posted:
in Mac Software
Apple CEO Tim Cook reveals that an extraordinarily large number of people are taking part in the company's beta program, which covers early versions of iOS, macOS and the company's other major operating systems.

Apple WWDC 2018


In an investor conference call following Apple's release of fiscal quarter three earnings on Thursday, Cook said some four million people are running beta software on their iPhone, iPad, Mac, Apple Watch and Apple TV devices.

"In June, we hosted an extremely successful developers conference that previewed many major advances coming this fall to our four operating systems: iOS, macOS, watchOS and tvOS," Cook said. "Developer and customer reaction has been very positive and we have over four million users participating in our new OS beta programs."

The number of beta program participants is not something that Apple typically releases, so it's unclear how that number compares to past years. Also unknown is how the participation statistics break down by operating system, and whether developers are included in the number.

Apple frequently touts both the growth of the App Store and its contributions to the app development profession as a whole, so it's a good guess that the four million figure for software beta participants in one year is among the largest ever for Apple, if not for the history of computing altogether.

At WWDC, Apple unveiled iOS 12, Mac OS 10.14, watchOS 5 and TV OS 12, and has periodically released beta editions of each in the weeks since. The full releases are scheduled for this fall.

Comments

  • Reply 1 of 15
    melgrossmelgross Posts: 33,510member
    and I can tell you, that it’s not a good thing for everyone.
    muthuk_vanalingam
  • Reply 2 of 15
    boboliciousbobolicious Posts: 1,139member
    ...and hopefully Apple will continue to release new hardware before any next MacOS ships so those wanting stability will be able to use a more refined almost year old operating system and get Applecare support for such... Is it still a very narrow window of efficacy...?
  • Reply 3 of 15
    MacProMacPro Posts: 19,718member
    melgross said:
    and I can tell you, that it’s not a good thing for everyone.
    'Everyone' is always a large concept isn't it?

    Seriously though it is a pretty good state of affairs compared the the diametric opposite on the iOS rip off AKA Android where no one seems to update let alone jump at the opportunity to play with the next great thing albeit in beta as soon as they can.   I also suspect those participating are not complete morons and either dual boot or have backups.  After doing this since the very first Mac OSX beta I have to say the stability of betas has been phenomenal.  I found going back to OS 9 even after the very rudimentary OSX beta was starkly shocking to say the least.  I think  macOS Mojave has been as good as the rest in that respect.  I suspect the introduction of APFS around the same time is probably making it seem scarier than most betas.  
    JWSCwatto_cobra
  • Reply 4 of 15
    foggyhillfoggyhill Posts: 4,767member
    the beta of IOS 12 is usable as a daily driver, with almost no caveats, I would not have said that of 11 for sure.
    watto_cobra
  • Reply 5 of 15
    Rayz2016Rayz2016 Posts: 6,957member
    Yeah, that’s nice, but did he say why having four million participants is good?

    Has it led to better stability?
    command_f
  • Reply 6 of 15
    sflocalsflocal Posts: 6,092member
    Yeah, and 99% of them are either media people or users that just want to play with it.
  • Reply 7 of 15
    MacProMacPro Posts: 19,718member
    Rayz2016 said:
    Yeah, that’s nice, but did he say why having four million participants is good?

    Has it led to better stability?
    Not sure he needed to say why, I'd think it speaks to the enthusiasm of the user base plus obviously a lot of feed back to help with ironing out bugs and incompatibilities.  As I said earlier, having participated in every beta Apple has ever done since the Apple][ 16K  the stability from the first betas Apple releases is extraordinary. 
    watto_cobra
  • Reply 8 of 15
    MacProMacPro Posts: 19,718member

    sflocal said:
    Yeah, and 99% of them are either media people or users that just want to play with it.
    Which is all good but don't forget software developers on all platforms.
    edited August 2018
  • Reply 9 of 15
    sflocal said:
    Yeah, and 99% of them are either media people or users that just want to play with it.
    And I wonder how many of them know what "beta" means. Are they really all running it on non-critical secondary machines? And does Apple actually get quality feedback (beyond automated crash reports)? Certainly the problems in early releases of High Sierra suggest that the beta programme didn't flush out issues that it should have.

    Personally, I wish Apple wouldn't do this: there's the risk to the users' service/data and there's the "pollution" of the web. The web issue starts when all these people running beta versions post stuff as if they're running the final version so, long after the beta programme is complete, searching for information about problems throws up beta problems (of which there are many, 'cos they're betas) that confuse the discussion of real, ongoing issues.
    watto_cobra
  • Reply 10 of 15
    dewmedewme Posts: 5,335member
    command_f said:
    sflocal said:
    Yeah, and 99% of them are either media people or users that just want to play with it.
    And I wonder how many of them know what "beta" means. Are they really all running it on non-critical secondary machines? And does Apple actually get quality feedback (beyond automated crash reports)? Certainly the problems in early releases of High Sierra suggest that the beta programme didn't flush out issues that it should have.

    Personally, I wish Apple wouldn't do this: there's the risk to the users' service/data and there's the "pollution" of the web. The web issue starts when all these people running beta versions post stuff as if they're running the final version so, long after the beta programme is complete, searching for information about problems throws up beta problems (of which there are many, 'cos they're betas) that confuse the discussion of real, ongoing issues.
    Great post. Traditional beta programs were intended to get good quality feedback from highly committed, knowledgeable, "close" customers. Beta testers often had to sign NDAs because some features were still in the pipeline and the developer was hoping to make a big splash when the features were officially announced to the public. I don't fault Apple for what they are doing at all but I wouldn't call it a "beta program." It's more along the lines of what Microsoft called a "Preview" program. Sure, the developer will still obtain feedback but with 4 million participants it would be very difficult to collect and organize all the feedback. With traditional beta programs it was not unheard of for actual developers working on the product to do one-on-one sessions with beta testers to address concerns or delve into something that was discovered by the beta tester. 

    Either way, getting feedback on a new release is crucial. Being able to act on the feedback in a meaningful way is also important. If hundreds, much less thousands or millions of "testers" were to submit feedback into the development team it would very quickly overwhelm the development team. I'm sure that Apple handles the vast majority of the feedback using automated tools and analytics on the front end of their anomaly management and triage toolset. 

    I too have found the latest betas/previews to be quite stable.
    command_fwatto_cobra
  • Reply 11 of 15
    asciiascii Posts: 5,936member
    I think such large scale betas have their place, because some things will always vary more in the real world than in the lab.

    One benefit I have seen is in Windows, where devices that had no driver in Device Manager for years suddenly got one on my box, presumably because thousands of other machine's beta telemetry also reported a missing driver. Micrososft just never knew.

    Apple's hardware is far less varied so I wouldn't expect them to get many driver benefits. But for example mobile carriers, different transmission conditions and strengths - a large scale beta would be great for debugging wireless. 

    Or the Metal API which is constantly being updated, millions of people trying millions of games would find edge cases/rendering bugs that would surely be missed in the lab. Or sync. Cloud sync is a tough nut to crack, millions of people will to risk their calendars and contacts that contain years of mixed up entries and duplicates, very valuable for testing with.
    watto_cobra
  • Reply 12 of 15
    In previous articke you have:

    Over 200 million users involved in beta testing macOS Mojave, iOS 12, tvOS 12, and watchOS 5.

    mbenz1962LukeCage
  • Reply 13 of 15
    foggyhill said:
    the beta of IOS 12 is usable as a daily driver, with almost no caveats, I would not have said that of 11 for sure.
    I used iOS 11 betas from the first public beta as a daily driver and remember few problems. 

    I’ve also been using iOS 12 since the first public beta, and while 2 and 3 were prone to crashing a lot with the spinning symbol, something beta 3 shortened somewhat, today’s latest beta 4 hasn’t crashed on me yet, which is pleasing to see. 

    Can’t wait for it to go gold and for September to arrive, but all in good time, and iOS 12 public beta versions in the meantime! :-)
  • Reply 14 of 15
    mbenz1962mbenz1962 Posts: 171member
    frantisek said:
    In previous articke you have:

    Over 200 million users involved in beta testing macOS Mojave, iOS 12, tvOS 12, and watchOS 5.

    I noticed this too.  frantisek's quote is form the article, Notes of interest from Apple's Q3 2018 conference call 

    Either way it is a really large group, but I tend to think the 4 million number in this article is the correct figure.
    watto_cobra
  • Reply 15 of 15
    melgrossmelgross Posts: 33,510member
    MacPro said:
    melgross said:
    and I can tell you, that it’s not a good thing for everyone.
    'Everyone' is always a large concept isn't it?

    Seriously though it is a pretty good state of affairs compared the the diametric opposite on the iOS rip off AKA Android where no one seems to update let alone jump at the opportunity to play with the next great thing albeit in beta as soon as they can.   I also suspect those participating are not complete morons and either dual boot or have backups.  After doing this since the very first Mac OSX beta I have to say the stability of betas has been phenomenal.  I found going back to OS 9 even after the very rudimentary OSX beta was starkly shocking to say the least.  I think  macOS Mojave has been as good as the rest in that respect.  I suspect the introduction of APFS around the same time is probably making it seem scarier than most betas.  
    When I say that, it means that not everyone will find it to be a good thing. I think that’s understood to mean that some will find it to be am good thing, and others won’t.

    i also think that there are those who post “first” without thinking, and those, as soon as a new version comes out, post that they are already downloading it. I’m sure there are also those who install a beta just because they can do it, and tell everyone that they are running a beta. How many people running the public betas actually spend the time to carefully inform Apple, through the Feedback app of the problems they are having? I wouldn’t be surprised if it were the majority.
Sign In or Register to comment.