Consumer Reports now recommends MacBook Pro after Apple software fix
In a bit of backpedaling, Consumer Reports on Thursday said it can now recommend Apple's new MacBook Pro lineup after a software fix remedied battery life inconsistencies found in previous testing.
Consumer Reports in an update to its original MacBook Pro evaluation said it has finished retesting the battery life of MacBook Pro laptops running a macOS 10.12.3 beta released earlier this week. As the battery problems are no longer present, the publication can now recommend the laptop series.
"With the updated software, the three MacBook Pros in our labs all performed well, with one model running 18.75 hours on a charge. We tested each model multiple times using the new software, following the same protocol we apply to hundreds of laptops every year," the report said.
The reevaluation comes after the late-2016 MacBook Pro lineup failed to garner the publication's recommended rating for the first time ever. Specifically, Consumer Reports blamed the rating on extreme variations in battery runtime tests.
The publication found wild fluctuations in battery life runtimes for each of Apple's new MacBook Pro models, both with and without Touch Bar. In some cases operating times vacillated from 16 hours in one test to as little as 3.75 hours in another.
Following the poor showing, Apple revealed it was working with the publication to resolve what at the time appeared to be unusual results not indicative of real-world use. Apple spends substantial capital on special machinery, facilities and man hours to perform rigorous quality assurance testing before putting a product up for sale, suggesting the Consumer Reports test was somehow anomalous.
Earlier this week, Apple in a statement to AppleInsider said a hidden developer setting in its Safari web browser was enabled by Consumer Reports during testing and triggered an obscure bug that led to inconsistent battery life readings. The issue was resolved in a macOS 10.12.3 beta that should see public release soon.
"This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab," Apple said. "After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life."
Consumer Reports in an update to its original MacBook Pro evaluation said it has finished retesting the battery life of MacBook Pro laptops running a macOS 10.12.3 beta released earlier this week. As the battery problems are no longer present, the publication can now recommend the laptop series.
"With the updated software, the three MacBook Pros in our labs all performed well, with one model running 18.75 hours on a charge. We tested each model multiple times using the new software, following the same protocol we apply to hundreds of laptops every year," the report said.
The reevaluation comes after the late-2016 MacBook Pro lineup failed to garner the publication's recommended rating for the first time ever. Specifically, Consumer Reports blamed the rating on extreme variations in battery runtime tests.
The publication found wild fluctuations in battery life runtimes for each of Apple's new MacBook Pro models, both with and without Touch Bar. In some cases operating times vacillated from 16 hours in one test to as little as 3.75 hours in another.
Following the poor showing, Apple revealed it was working with the publication to resolve what at the time appeared to be unusual results not indicative of real-world use. Apple spends substantial capital on special machinery, facilities and man hours to perform rigorous quality assurance testing before putting a product up for sale, suggesting the Consumer Reports test was somehow anomalous.
Earlier this week, Apple in a statement to AppleInsider said a hidden developer setting in its Safari web browser was enabled by Consumer Reports during testing and triggered an obscure bug that led to inconsistent battery life readings. The issue was resolved in a macOS 10.12.3 beta that should see public release soon.
"This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab," Apple said. "After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life."
Comments
I love the idea of an independent body that fairly evaluates products, but their methods lead to wrong results as often as right ones. I have a 15" MBP with Touch Bar, and I can tell you there is absolutely no way it ever lasted 19 hours web browsing with the screen on. It lasts me a little more than what Apple says if I'm web browsing and about 5-6 hours if I'm doing work.
Running the machine with a hidden developer setting that is DESIGNED to reduce performance, and then not disclosing that fact, and then claiming an independant test shows the machine under-performs the manufacturers claims is FRAUD. Possibly Libel/Slander as well.
CU is not a credible organization.
So while I agree that default browser settings on each platform make sense, then it also is only fair for CR to go back and test all the OTHER laptops with their browser caches enabled - then all of them will also show much longer battery life.
The bottom line is Apple was NOT singled out in approach, so the real fix is to change it for everyone.
It was was no big deal.
"After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life"
Consumer reports comment of "with the updated software" could mean "us doing it the correct way this time" NOT "Apple updated their software"
It is a bit strange, and has been covered on other threads. CR thought they were simulating a "real world situation" by using a developer setting. Actually testing in "real world conditions" might have gotten them a different result, but that's harder to automate.