Over 4 million people participate in Apple's software beta programs

AppleInsider

Apple CEO Tim Cook reveals that an extraordinarily large number of people are taking part in the company's beta program, which covers early versions of iOS, macOS and the company's other major operating systems.

In an investor conference call following Apple's release of fiscal quarter three earnings on Thursday, Cook said some four million people are running beta software on their iPhone, iPad, Mac, Apple Watch and Apple TV devices.

"In June, we hosted an extremely successful developers conference that previewed many major advances coming this fall to our four operating systems: iOS, macOS, watchOS and tvOS," Cook said. "Developer and customer reaction has been very positive and we have over four million users participating in our new OS beta programs."

The number of beta program participants is not something that Apple typically releases, so it's unclear how that number compares to past years. Also unknown is how the participation statistics break down by operating system, and whether developers are included in the number.

Apple frequently touts both the growth of the App Store and its contributions to the app development profession as a whole, so it's a good guess that the four million figure for software beta participants in one year is among the largest ever for Apple, if not for the history of computing altogether.

At WWDC, Apple unveiled iOS 12, Mac OS 10.14, watchOS 5 and TV OS 12, and has periodically released beta editions of each in the weeks since. The full releases are scheduled for this fall.

melgross

and I can tell you, that it’s not a good thing for everyone.

bobolicious

...and hopefully Apple will continue to release new hardware before any next MacOS ships so those wanting stability will be able to use a more refined almost year old operating system and get Applecare support for such... Is it still a very narrow window of efficacy...?

MacPro

melgross said:

and I can tell you, that it’s not a good thing for everyone.

'Everyone' is always a large concept isn't it?

Seriously though it is a pretty good state of affairs compared the the diametric opposite on the iOS rip off AKA Android where no one seems to update let alone jump at the opportunity to play with the next great thing albeit in beta as soon as they can.   I also suspect those participating are not complete morons and either dual boot or have backups.  After doing this since the very first Mac OSX beta I have to say the stability of betas has been phenomenal.  I found going back to OS 9 even after the very rudimentary OSX beta was starkly shocking to say the least.  I think  macOS Mojave has been as good as the rest in that respect.  I suspect the introduction of APFS around the same time is probably making it seem scarier than most betas.  

foggyhill

the beta of IOS 12 is usable as a daily driver, with almost no caveats, I would not have said that of 11 for sure.

Rayz2016

Yeah, that’s nice, but did he say why having four million participants is good?

Has it led to better stability?

sflocal

Yeah, and 99% of them are either media people or users that just want to play with it.

MacPro

Rayz2016 said:

Yeah, that’s nice, but did he say why having four million participants is good?

Has it led to better stability?

Not sure he needed to say why, I'd think it speaks to the enthusiasm of the user base plus obviously a lot of feed back to help with ironing out bugs and incompatibilities.  As I said earlier, having participated in every beta Apple has ever done since the Apple][ 16K  the stability from the first betas Apple releases is extraordinary. 

MacPro

sflocal said:

Yeah, and 99% of them are either media people or users that just want to play with it.

Which is all good but don't forget software developers on all platforms.

command_f

sflocal said:

Yeah, and 99% of them are either media people or users that just want to play with it.

And I wonder how many of them know what "beta" means. Are they really all running it on non-critical secondary machines? And does Apple actually get quality feedback (beyond automated crash reports)? Certainly the problems in early releases of High Sierra suggest that the beta programme didn't flush out issues that it should have.

Personally, I wish Apple wouldn't do this: there's the risk to the users' service/data and there's the "pollution" of the web. The web issue starts when all these people running beta versions post stuff as if they're running the final version so, long after the beta programme is complete, searching for information about problems throws up beta problems (of which there are many, 'cos they're betas) that confuse the discussion of real, ongoing issues.

dewme

command_f said:

sflocal said:

Yeah, and 99% of them are either media people or users that just want to play with it.

And I wonder how many of them know what "beta" means. Are they really all running it on non-critical secondary machines? And does Apple actually get quality feedback (beyond automated crash reports)? Certainly the problems in early releases of High Sierra suggest that the beta programme didn't flush out issues that it should have.

Personally, I wish Apple wouldn't do this: there's the risk to the users' service/data and there's the "pollution" of the web. The web issue starts when all these people running beta versions post stuff as if they're running the final version so, long after the beta programme is complete, searching for information about problems throws up beta problems (of which there are many, 'cos they're betas) that confuse the discussion of real, ongoing issues.

Great post. Traditional beta programs were intended to get good quality feedback from highly committed, knowledgeable, "close" customers. Beta testers often had to sign NDAs because some features were still in the pipeline and the developer was hoping to make a big splash when the features were officially announced to the public. I don't fault Apple for what they are doing at all but I wouldn't call it a "beta program." It's more along the lines of what Microsoft called a "Preview" program. Sure, the developer will still obtain feedback but with 4 million participants it would be very difficult to collect and organize all the feedback. With traditional beta programs it was not unheard of for actual developers working on the product to do one-on-one sessions with beta testers to address concerns or delve into something that was discovered by the beta tester. 

Either way, getting feedback on a new release is crucial. Being able to act on the feedback in a meaningful way is also important. If hundreds, much less thousands or millions of "testers" were to submit feedback into the development team it would very quickly overwhelm the development team. I'm sure that Apple handles the vast majority of the feedback using automated tools and analytics on the front end of their anomaly management and triage toolset. 

I too have found the latest betas/previews to be quite stable.

ascii

I think such large scale betas have their place, because some things will always vary more in the real world than in the lab.

One benefit I have seen is in Windows, where devices that had no driver in Device Manager for years suddenly got one on my box, presumably because thousands of other machine's beta telemetry also reported a missing driver. Micrososft just never knew.

Apple's hardware is far less varied so I wouldn't expect them to get many driver benefits. But for example mobile carriers, different transmission conditions and strengths - a large scale beta would be great for debugging wireless. 

Or the Metal API which is constantly being updated, millions of people trying millions of games would find edge cases/rendering bugs that would surely be missed in the lab. Or sync. Cloud sync is a tough nut to crack, millions of people will to risk their calendars and contacts that contain years of mixed up entries and duplicates, very valuable for testing with.

frantisek

In previous articke you have:

Over 200 million users involved in beta testing macOS Mojave, iOS 12, tvOS 12, and watchOS 5.

alexonline

foggyhill said:

the beta of IOS 12 is usable as a daily driver, with almost no caveats, I would not have said that of 11 for sure.

I used iOS 11 betas from the first public beta as a daily driver and remember few problems. 

I’ve also been using iOS 12 since the first public beta, and while 2 and 3 were prone to crashing a lot with the spinning symbol, something beta 3 shortened somewhat, today’s latest beta 4 hasn’t crashed on me yet, which is pleasing to see. 

Can’t wait for it to go gold and for September to arrive, but all in good time, and iOS 12 public beta versions in the meantime! :-)

mbenz1962

frantisek said:

In previous articke you have:

Over 200 million users involved in beta testing macOS Mojave, iOS 12, tvOS 12, and watchOS 5.

I noticed this too.  frantisek's quote is form the article, Notes of interest from Apple's Q3 2018 conference call 

Either way it is a really large group, but I tend to think the 4 million number in this article is the correct figure.

melgross

MacPro said:

melgross said:

and I can tell you, that it’s not a good thing for everyone.

'Everyone' is always a large concept isn't it?

Seriously though it is a pretty good state of affairs compared the the diametric opposite on the iOS rip off AKA Android where no one seems to update let alone jump at the opportunity to play with the next great thing albeit in beta as soon as they can.   I also suspect those participating are not complete morons and either dual boot or have backups.  After doing this since the very first Mac OSX beta I have to say the stability of betas has been phenomenal.  I found going back to OS 9 even after the very rudimentary OSX beta was starkly shocking to say the least.  I think  macOS Mojave has been as good as the rest in that respect.  I suspect the introduction of APFS around the same time is probably making it seem scarier than most betas.  

When I say that, it means that not everyone will find it to be a good thing. I think that’s understood to mean that some will find it to be am good thing, and others won’t.

i also think that there are those who post “first” without thinking, and those, as soon as a new version comes out, post that they are already downloading it. I’m sure there are also those who install a beta just because they can do it, and tell everyone that they are running a beta. How many people running the public betas actually spend the time to carefully inform Apple, through the Feedback app of the problems they are having? I wouldn’t be surprised if it were the majority.