Apple says hidden Safari setting led to flawed Consumer Reports MacBook Pro battery tests

1246

Comments

  • Reply 61 of 118
    So, basically, every developer hits this problem, unless they ditch safari for chrome, and even then they might hit it. This is a very normal setting for a not insignificant number of apple users. (I can look around and find about 40 of them without getting out of my chair.) As for the apologists, the settings are so that they test the same across all computers. Disabling caching shouldn't make your laptop randomly have 20% of the battery life it normally does. It makes it completely unusable for developing websites on. It's a bit ironic that this turns out to be a bug that primarily effects developers, it implies a certain lack of dogfooding at apple.
    williamlondon
  • Reply 62 of 118

    3. According to APPLE, not CR, the problem is the result of a BUG, not the testing methodology per se. The testing method merely exposed it. How long would it have gone on unnoticed and unfixed if the test hadn't uncovered it? It may not have affected me or you, but obviously some people would suffer from it. This is a positive outcome.

    Actually I think Apple blamed both the methodology and the bug. See their statement:

    “We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug..."

    Note that they use the word "also triggered" the obscure bug. "Also", not "subsequently" triggered the bug. Right or wrong, they seem to be saying both conditions are responsible for the results.
    Noted. However, Apple's argument that the test doesn't reflect "real-world usage" is largely irrelevant in this context. What CR was doing wasn't "real-world" use, it was "testing." Testing requires controlling variables. Since caching is unpredictable, it was disabled. I would have disabled it too.

    Also, if the testing method were the cause of the reduced battery life, the battery life would have been short every time. It wasn't. There were huge variations. Since the testing method didn't change, it CAN'T be the source of the issue. The bug, on the other hand, according to Apple, is intermittent. That WOULD cause the results to vary.

    So absent any obfuscation caused by phrasing, intentional or accidental, simple deductive reasoning demonstrates that the outcomes were the result of the bug and not the testing method.

    If the bug is only evident when caching is defeated (which they haven't explicitly said) then maybe the battery issue wouldn't actually affect most users, and that's a valid objection. I just don't think it's fair to accuse CR of flawed testing when what they did actually makes sense.
    anantksundaramtzm41roundaboutnowbeowulfschmidt
  • Reply 63 of 118
    @indiekiduk said:
    "
    Here's the setting I think they are talking about. Would be good to know why CR turned it on. Probably to simulate real usage of the user visiting different websites. Need more info..." along with an accompanying screen shot.

    The "hidden item" that is referred to is no more hidden than many other settings in Preferences -- except you have to click the "Advanced" tab in "Preferences" and then click the check box at the bottom of the dialog box that is labeled  "Show Develop menu in menubar".

    The CR folx obviously consider themselves Advanced Users -- IMHO I assess their skill level at a much lower level.

  • Reply 64 of 118
    welshdogwelshdog Posts: 1,897member
    Rayz2016 said:
    blastdoor said:
    I think CR's rationale for turing of cacheing is perfectly reasonable: 

    http://www.consumerreports.org/apple/apple-releases-fix-to-macbook-pros-in-response-to-consumer-reports-battery-test-results/

    I don't think anybody did anything wrong here -- neither Apple nor CR -- and both are handling it appropriately. 

    Unfortunately, once they started fiddling with debug settings on Safari, it stopped being a consumer laptop test and became a test to see how Safari reacted to being bombarded with identical web pages over several hours. This was supposed to be a test of what the user would experience out of the box. It wasn't. And to be honest, they shouldn't be disabling the caches on any machines because that is not what consumers do. They really need to rethink their testing.

    As several have pointed out, leaving the cache enabled does not make a good test. "Real world testing" is a bogus concept that is by nature not repeatable and thus of no value when comparing products.  CR and other reputable testers strive to test every product category the exact same way to give meaningful results.  There is a website called Small Net Builder and the guy tests routers and NAS devices and such.  He tests every single router exactly the same way because that is only fair and accurate.  Someone on this thread pointed out CR has always tested Macs and all other computers with the cache disabled so the tests would be fair and equal.  The disgust here for CR is very characteristic of most Internet discussions - merely an exchange of ignorance.
    lorin schultzbeowulfschmidt
  • Reply 65 of 118
    SoliSoli Posts: 10,035member
    The "hidden item" that is referred to is no more hidden than many other settings in Preferences -- except you have to click the "Advanced" tab in "Preferences" and then click the check box at the bottom of the dialog box that is labeled  "Show Develop menu in menubar".
    It's not hidden like used Terminal or searching for a PLIST to alter a setting that is available for an app, but not listed in the app, like restoring auto-play to QuickTime X (below), but I'd still consider it hidden because the option isn't in Safari's preferences. What's in Safari's preferences is the checkbox to unhide the "Develop" menu bar item.


    defaults write com.apple.QuickTimePlayerX MGPlayMovieOnOpen 1

    edit: They pulled that option from Sierra for reasons I'll likely never understand.

    edited January 2017
  • Reply 66 of 118
    volcanvolcan Posts: 1,799member
    lorin schultz said:
    Noted. However, Apple's argument that the test doesn't reflect "real-world usage" is largely irrelevant in this context. What CR was doing wasn't "real-world" use, it was "testing." 
    I think the PR issue is that CR's results imply Apple's claim of "Up to 10 hours of wireless web" as being exaggerated.

     CR is simply downloading continuous data and Apple is using "real-world" typical usage so they are testing two completely different things.
  • Reply 67 of 118
    SoliSoli Posts: 10,035member
    volcan said:
    lorin schultz said:
    Noted. However, Apple's argument that the test doesn't reflect "real-world usage" is largely irrelevant in this context. What CR was doing wasn't "real-world" use, it was "testing." 
    I think the PR issue is that CR's results imply Apple's claim of "Up to 10 hours of wireless web" as being exaggerated.

     CR is simply downloading continuous data and Apple is using "real-world" typical usage so they are testing two completely different things.
    But they got 19.5 hours from at least one test, which is almost double. That was just one red flag out of many that should've had them consider something hinky was in play.
  • Reply 68 of 118
    Goodness you folks are disappointing. Let me say a few things. First, the guy who claimed that "only real engineers use Macs ... Windows and Linux users are pretenders and wannabe's" ... I have worked in engineering and computer programming for decades. What he says is absolute lunacy. Only a fraction of Apple's lineup has the hardware and graphics requirements to do truly high end technical work. For goodness sakes, Apple's lineup can't even support the Oculus Rift and a lot of high end video games! And engineers NEED the adaptability offered by Windows and Linux hardware and software in order to - I don't know - solve unique and perform specialized tasks that can't be met by CONSUMER HARDWARE. For goodness sakes, Mac OS X has like a 10% market share, and most of that is taken up by decidedly non-technical people who are into things like graphic design, music/video and (non-cutting edge) animation. So is your position REALLY is that the only "real engineers" are the tiny sliver that use Mac OS X? It is far more likely that the "real engineers" are the ones who have high performance applications that Intel's off-the-shelf chip configurations can't handle, or who need hardened devices to work in environments that have a lot of heat, radiation, EM interference etc. I've been in workplaces that have those sorts of specialized requirements for their hardware. Have you? Well keep thinking that you are a "real engineer" while you work in your comfortable, climate-controlled cubicle while the guys who have to solve difficult problems hands on in areas like manufacturing and engineering are wannabe's, frauds and tinkerers. Second, remember my initial comment on this issue? Months ago? I predicted it EXACTLY. That it was a bug in Safari, or in some area in the OS where Safari and the OS intertwine. What did I base my guess on? My years as a programmer and product tester. As well as my knowledge that despite the clear fact that Apple makes great products, SAFARI IS NOT ONE OF THEM. You can bash Chrome all you want, but remember two things. 1) Unlike Safari and IE, Chrome is a platform capable of running a bunch of software and services, not merely a browser. So yes, it uses more memory than other browsers because IT DOES A LOT MORE. 2) Where Google created their own browser from the Chromium open source project - which is also essentially theirs - both Apple and Microsoft essentially cribbed theirs from Netscape and Firefox. (Particularly since the people who created Netscape left to form Firefox after Microsoft bought it.) When Microsoft deviated from the Netscape base in order to try to keep up with Firefox and Chrome they made a mess of things. Apple didn't even try to keep up in the browser wars so they just left it limited, without even trying to compete with Chrome and Firefox on functionality. That is why the very instant I read about the test that Consumer Reports was running, I instantly knew "bug in Safari, or something in the OS that interacts with Safari." And sure enough, even though Apple did their best to obfuscate by (less than truthfully) claiming that the test results were due to "hidden settings that never get used by consumers" they are indeed issuing a bug to fix the problem. And when I say obfuscate ... wow. Every single browser has that disable cache setting. Every. Single. One. And it is not hidden; it is right there in the browser settings. LOTS of people turn it off for various reasons. And you know what? Such as ... when you do "private browsing." That is right. Whenever you do "private browsing" IT ENABLES THAT SETTING. Let me repeat: PRIVATE BROWSING ENABLES THIS SETTING. That is why for Apple to claim that this is "a hidden feature that consumers will never use" ... wrong. Consumers use it all the time! And even when private browsing isn't turned on, people do in fact disable caching for one reason or another. I used to do it all the time in the early Internet browser days for performance and disk space reasons (back when browsers were far less reliable than now). As for the claims that Consumer Reports performed a bad test to make Apple look bad and favor Windows or sell ads (which Consumer Reports does not accept) or generate clicks (not their business model) ... wrong. Consumer Reports uses this same test on multiple browsers on all machines. They have used these same tests on Apple hardware before and given Apple rave reviews. It is just that this one time Apple got a bad score because of YET ANOTHER PROBLEM WITH SAFARI. Apple's claim that Consumer Reports' test was flawed and they are going to fix the test ... yeah right. Why should their test disallow a very popular feature such as PRIVATE BROWSING? Consumer Reports isn't going to respond to Apple's claim of "flawed tests" because it isn't in their interests to get into a public back and forth with Apple. Apple will fix the bug - as I said they would - Consumer Reports will revisit the review and everyone will forget about this. The tinfoil hat crowd will continue to believe that Consumer Reports and everyone else is out to get Apple. The rest of us will just wait for the next Safari bug to be discovered and squashed. The same is true for the claim: "Consumer Reports should have contacted Apple first before giving out the bad review." Excuse me? Private browsing again? A core feature in every web browser for going on 20 years? The product that shipped "as is" had a flaw in one of its CORE APPLICATIONS. (Some people claim that the main browser should be considered part of the OS itself; I do not go that far. I will say that the main browser uses shared libraries that are part of the OS and leave it at that.) A flaw that could be easily encountered by using a very popular feature in that core application (again, if not specifically disabling the caching, entering private browsing mode ... which the browser itself tells you to do when you delete the browsing history!). There is being a fan, being an enthusiast and being a fanatic. The former two are fine, the last one requires rejecting objective reality and is never healthy.
    williamlondon
  • Reply 69 of 118
    brucemcbrucemc Posts: 1,541member
    nht said:
    tzm41 said:
    It is wonderful how so many people here are bashing CR. CR used the same testing methodology and ranked the past MB models high. CR also used the same methodology for all other laptops without this kind of jarring finding.
    Reasonably, users other than web developers would not turn this option on. But they would not reload the same page ten thousand times before the battery dies either. This setting is just to mimic a use case when the user is browsing different websites, and is completely fair since CR uses it across all laptops.
    Apple admitted it is a bug in Safari that cause the fluctuation, but people are blaming CR for using the browser wrong. To me, apple fanboys are saying when CR recommends MBs it is trustworthy, when it does not it is crap.
    The reason that folks are giving CR a hard time is that they didn't see the issue in Chrome but then labeled it as a problem of the laptop itself rather than potentially a problem with Safari.

    "Once our official testing was done, we experimented by conducting the same battery tests using a Chrome browser, rather than Safari. For this exercise, we ran two trials on each of the laptops, and found battery life to be consistently high on all six runs."

    The CR battery test is also stupid as it only measures battery life of web surfing and not say watching a movie (video loop) or playing games (game benchmark loop) or content creation (hand break or photoshop benchmark loop).  Had their battery test not been so stupid and covered more than one use case it would have been equally obvious that something was wrong with the web benchmark and not the laptop itself.

    That Chrome posted consistently high battery tests and they went with a sensationalist headline and result indicates that CR's intent was to generate controversy and increase their exposure.
    This is mostly how I see it (although I am not going as far to call their tests stupid - just that they did not investigate the issues).  

    I don't think it is "apologist" to point out what CR did.  Essentially, they saw some variable results with Safari, but not with Chrome, but did not look into the matter to understand the issue prior to releasing their report.  A CR report which states "we cannot recommend Apple's latest MBP" is like throwing gasoline on a raging fire to the tech-media-blogosphere, and they knew it, and did it.  

    Now the issue has been investigated, a bug addressed, and the results are as Apple had previously advertised (or better).  However, do you think this update will get the same distribution and or any acknowledgement of the issue in the tests?  Not in a million years.  

    Whether for sake of a deadline, or to create more buzz, CR released a report with a clear issue (variable test results, with some clear indications of a browser issue) that they did not investigate, and hence a very public report essentially became a "do not recommend" for this new product.  People have a right to point this out and it is legitimate criticism of CR.  End of story.


  • Reply 70 of 118
    dewmedewme Posts: 5,335member
    MplsP said:
    I agree that turning off the cache for testing purposes was a reasonable thing to do if they were trying to simulate surfing different web sites. If they did this on every computer they tested then it's hard to argue that they're singling out Apple. The fact that it was so inconsistent and only occurred on Safari and not Chrome should have been a red flag to them that it was a software bug, not a hardware issue. Even if this was not a 'real world' test in that respect, the issue was ultimately caused by a software bug in Safari. I would hope that CR puts a link to their repeat tests on the page with the original article and issues a correction as that would be the right thing to do from a journalistic standpoint.

    As far as the other people experiencing battery issues, it's impossible to say what they're from. It's doubtful that they're all caused by this bug (I suppose we can hope, right?) but this is an excellent example of how bad software design or software bugs can dramatically alter hardware performance. My hope is that the issues people are experiencing are actually software related and can be remedied with a simple software update. They would still be a black eye for Apple, but at least a much smaller one that can be fixed.
    I'm okay with turning off caches and creating synthetic (non customer representative) tests or benchmarks for the purpose of testing a specific component that makes up part of a larger system. But this is in no way a reasonable way to assess the total system performance or system adequacy for its intended purpose. Almost all of the performance/speedup gains that have been realized in computing over the past 50 years, outside of true parallel processing, which is still elusive in most commercial applications, are the direct result of "locality of reference" (LOR) behaviors, i.e., the very things caching addresses at all levels in a modern computer architecture. Modern computer architectures are overwhelmingly biased around the exploitation of LOR and the fact that memory access is inherently non-random and statistically clustered in time and (address) space. When you turn off a cache either intentionally (CR's case) or through a programming anomaly (Apple's case) you're no longer testing the computing platform for its intended purpose. For battery life assessment you may as well just hook up the raw battery to a dummy resistive load and see how long it takes to drain. 

    In summary, disabling caches for testing suffers from the same flaw that synthetic benchmarks suffer from, you're no longer testing the whole system. The results of the synthetic testing may be nearly meaningless to the real world system performance. It works both ways too, the synthetic test can mislead consumers into thinking that the system is much better or much worse than what they actually experience. This doesn't mean it's inherently bad, just that it must be contextualized in terms of what it means to component performance versus what it means to system performance. In complex computing systems these relationships are not necessarily related in a linear fashion. 

    I don't think there's a need to assign blame or assume any ill will in this situation. CR's synthetic test coupled with Apple's Safari bug created a worst-case situation for objectively evaluating battery run-time for MacBook Pro computers and what should be expected in the hands of customers running real applications. The wide spread of test results was a hint that something was obviously very wrong, which CR alluded to and Apple confirmed when they got involved by closely scrutinizing the test results. The net result is that a Safari bug was uncovered that eluded Apple's testing process - so MacBook Pro customers are better off for Apple's PR chafing. 
    pscooter63
  • Reply 71 of 118
    Look at all these people defending Apple blindly.  If CR tests all their machines this way, and has tested previous MacBooks, MBPs, etc this way... Then there is nothing deceitful or wrong about their methodology or their findings.  If this is how they tested just the new MBPs (which I doubt), then your point about CR being unreliable and deceitful would be true.  Fact of the matter is, USERS, REAL WORLD USERS, are seeing terrible battery life, and that's why this story gained traction.  I love Apple products, but I don't defend a company blindly like many of the sheep on here do.  <Cue sheep who get offended by the word "sheep" or "fanboy">  Hey, if the shoe fits... 
    williamlondonbeowulfschmidt
  • Reply 72 of 118
    This is absolutely stupid. And the condemnation has been posted on news site every where - including those such as CNN that do not accept reader comments.

    I stopped trusting Consumer Reports a long time ago. The did a head-to-head test between a Mac and a couple of PCs. The Mac had Gigabyte Ethernet built into the mother board. As was typical, the PCs had to have an Ethernet card. Being the PC-heads that they were, it never occurred to them to look at the ports on the back of the machine. They just looked at the cards and decided that the Mac did not support networking. They also included this supposed lack of networking in the text of their condemnation and in their feature comparison grid. Naturally, they had a photo of the back of the machine and there was the RJ45 port, as plain as day.
    williamlondon
  • Reply 73 of 118
    crowleycrowley Posts: 10,453member
    Pretty reasonable thing for CR to do and unfortunate for both CR and Apple that they ran into a bug which caused such a kerfuffle.

    Anyone flinging vitriol to blame either side needs to go spend some time outside for a change.
    dewmesingularitywigginfarjamed
  • Reply 74 of 118
    freeper said:
    ... 2) Where Google created their own browser from the Chromium open source project - which is also essentially theirs - both Apple and Microsoft essentially cribbed theirs from Netscape and Firefox. (Particularly since the people who created Netscape left to form Firefox after Microsoft bought it.) When Microsoft deviated from the Netscape base in order to try to keep up with Firefox and Chrome they made a mess of things. Apple didn't even try to keep up in the browser wars so they just left it limited, without even trying to compete with Chrome and Firefox on functionality. That is why the very instant I read about the test that Consumer Reports was running, I instantly knew "bug in Safari, or something in the OS that interacts with Safari." And sure enough, even though Apple did their best to obfuscate by (less than truthfully) claiming that the test results were due to "hidden settings that never get used by consumers" they are indeed issuing a bug to fix the problem. And when I say obfuscate ... wow. Every single browser has that disable cache setting. Every. Single. One. And it is not hidden; it is right there in the browser settings. LOTS of people turn it off for various reasons. And you know what? Such as ... when you do "private browsing." That is right. Whenever you do "private browsing" IT ENABLES THAT SETTING. Let me repeat: PRIVATE BROWSING ENABLES THIS SETTING.
    Wow... there is so much wrong in your entire long winded post.

    First: Google Chrome is a direct descendent of Safari.  For the longest time Google was using WebKit as the rendering engine for Chrome.  WebKit is an open source project managed owned by Apple.  WebKit has nothing to do with Netscape - It's based on/fork of the KHTML project which was mainly used for the Konqueror web browser in the KDE graphical environment on Linux.  Google then forked WebKit to a new code base called blink.  Opera also uses Blink as its rendering engine.  Microsoft never purchased any assets from Netscape regarding the Navigator web browser.  Those were passed to AOL before they were then open sourced, which became the "Phoenix" web browser, and then renamed Firefox.  While Safari doesn't keep pace with all the features that Google Chrome offers, it's still rather current and usually supports the most important parts of the HTML5 standard, and is on a slower release cycle than Chrome and Firefox.  If anyone "cribbed" a project, Google "cribbed" Chrome from Safari.

    Second: Private browsing does not disable the browser cache.  What private browsing does do is make temporary locations for your browser cache and cookies for a specific private browsing window.  These temporary locations are then deleted immediately when the private browsing window is closed.  Other web browsers follow the same principal.  This is why private browsing is never a guarantee (and all the browsers warn you of this) of preventing someone to know your web browsing history if they have some type of access to the system while you are performing web browsing.
    Solipscooter63roundaboutnowdamn_its_hotloquitur
  • Reply 75 of 118
    clemynxclemynx Posts: 1,552member
    Well, mistakes happen. Hopefully they will put the correction on their front page. 

    Of course the bad press won't correct itself and the damage is done. 
  • Reply 76 of 118
    misamisa Posts: 827member
    macxpress said:
    Why the hell would CR mess with any settings? Just take the damn thing out of the box and test it as is. I don't trust CR anyways. I'm sure they'll cover their ass and just blame it on Apple. I've always said these issues are most likely software issues with the battery. I seriously doubt Apple would release a laptop thats very well noted for its battery life with something else that gets half or worse the amount of battery life. Apple is one of the few you can actually rely on for accurate battery life. 
    In this case, it may have been changed to do a performance test, but not changed back to do a battery life test. You typically turn the cache off during performance tests (eg browser speed, network speed, cpu speed, gpu speed) so that the cache doesn't invalidate the results from test to test. The correct way to test battery life is to do one of three things 10 or so times:

    Test the battery life for movies/media: by creating a playlist of different music and movies for approximately 16 hours, while an application in the background time-stamps every minute and notes the battery meter reported value. 

    Test the battery life for standby: Literately run only the time-stamp app, have no software running and see how long it runs idle. This is where you get "maximum battery life" from

    Test the battery life for conventional usage: Requires running two applications, one that counts the battery life, and another that cycles through the web browser, media player and so forth using macro playback, while configured to use a testing proxy server that feeds consistent data for web tests. Basically, you see how far it gets in a 16 hour "work/home cycle" of use.

    So you would typically turn off caching if you are testing the web browser's speed, but turning it off for a battery life test takes it pretty far from how users normally use a web browser, so a reported inconsistent life means that there is a problem with the testing setup. For it to be a hardware/software issue, it would replicate the problem on every device running the same software and hardware, which is something that CR didn't expand it's research into. 
    pscooter63
  • Reply 77 of 118
    SoliSoli Posts: 10,035member
    freeper said:
    Goodness you folks are disappointing. Let me say a few things. First, the guy who claimed that "only real engineers use Macs ... Windows and Linux users are pretenders and wannabe's" ... I have worked in engineering and computer programming for decades. What he says is absolute lunacy. Only a fraction of Apple's lineup has the hardware and graphics requirements to do truly high end technical work. For goodness sakes, Apple's lineup can't even support the Oculus Rift and a lot of high end video games! And engineers NEED the adaptability offered by Windows and Linux hardware and software in order to - I don't know - solve unique and perform specialized tasks that can't be met by CONSUMER HARDWARE. For goodness sakes, Mac OS X has like a 10% market share, and most of that is taken up by decidedly non-technical people who are into things like graphic design, music/video and (non-cutting edge) animation. So is your position REALLY is that the only "real engineers" are the tiny sliver that use Mac OS X? It is far more likely that the "real engineers" are the ones who have high performance applications that Intel's off-the-shelf chip configurations can't handle, or who need hardened devices to work in environments that have a lot of heat, radiation, EM interference etc. I've been in workplaces that have those sorts of specialized requirements for their hardware. Have you? Well keep thinking that you are a "real engineer" while you work in your comfortable, climate-controlled cubicle while the guys who have to solve difficult problems hands on in areas like manufacturing and engineering are wannabe's, frauds and tinkerers. Second, remember my initial comment on this issue? Months ago? I predicted it EXACTLY. That it was a bug in Safari, or in some area in the OS where Safari and the OS intertwine. What did I base my guess on? My years as a programmer and product tester. As well as my knowledge that despite the clear fact that Apple makes great products, SAFARI IS NOT ONE OF THEM. You can bash Chrome all you want, but remember two things. 1) Unlike Safari and IE, Chrome is a platform capable of running a bunch of software and services, not merely a browser. So yes, it uses more memory than other browsers because IT DOES A LOT MORE. 2) Where Google created their own browser from the Chromium open source project - which is also essentially theirs - both Apple and Microsoft essentially cribbed theirs from Netscape and Firefox. (Particularly since the people who created Netscape left to form Firefox after Microsoft bought it.) When Microsoft deviated from the Netscape base in order to try to keep up with Firefox and Chrome they made a mess of things. Apple didn't even try to keep up in the browser wars so they just left it limited, without even trying to compete with Chrome and Firefox on functionality. That is why the very instant I read about the test that Consumer Reports was running, I instantly knew "bug in Safari, or something in the OS that interacts with Safari." And sure enough, even though Apple did their best to obfuscate by (less than truthfully) claiming that the test results were due to "hidden settings that never get used by consumers" they are indeed issuing a bug to fix the problem. And when I say obfuscate ... wow. Every single browser has that disable cache setting. Every. Single. One. And it is not hidden; it is right there in the browser settings. LOTS of people turn it off for various reasons. And you know what? Such as ... when you do "private browsing." That is right. Whenever you do "private browsing" IT ENABLES THAT SETTING. Let me repeat: PRIVATE BROWSING ENABLES THIS SETTING. That is why for Apple to claim that this is "a hidden feature that consumers will never use" ... wrong. Consumers use it all the time! And even when private browsing isn't turned on, people do in fact disable caching for one reason or another. I used to do it all the time in the early Internet browser days for performance and disk space reasons (back when browsers were far less reliable than now). As for the claims that Consumer Reports performed a bad test to make Apple look bad and favor Windows or sell ads (which Consumer Reports does not accept) or generate clicks (not their business model) ... wrong. Consumer Reports uses this same test on multiple browsers on all machines. They have used these same tests on Apple hardware before and given Apple rave reviews. It is just that this one time Apple got a bad score because of YET ANOTHER PROBLEM WITH SAFARI. Apple's claim that Consumer Reports' test was flawed and they are going to fix the test ... yeah right. Why should their test disallow a very popular feature such as PRIVATE BROWSING? Consumer Reports isn't going to respond to Apple's claim of "flawed tests" because it isn't in their interests to get into a public back and forth with Apple. Apple will fix the bug - as I said they would - Consumer Reports will revisit the review and everyone will forget about this. The tinfoil hat crowd will continue to believe that Consumer Reports and everyone else is out to get Apple. The rest of us will just wait for the next Safari bug to be discovered and squashed. The same is true for the claim: "Consumer Reports should have contacted Apple first before giving out the bad review." Excuse me? Private browsing again? A core feature in every web browser for going on 20 years? The product that shipped "as is" had a flaw in one of its CORE APPLICATIONS. (Some people claim that the main browser should be considered part of the OS itself; I do not go that far. I will say that the main browser uses shared libraries that are part of the OS and leave it at that.) A flaw that could be easily encountered by using a very popular feature in that core application (again, if not specifically disabling the caching, entering private browsing mode ... which the browser itself tells you to do when you delete the browsing history!). There is being a fan, being an enthusiast and being a fanatic. The former two are fine, the last one requires rejecting objective reality and is never healthy.

    You may have something profound to say, but you need to remember that communication is about being able to relay your point(s) in the best possible manner.

    Your "amalgagraphing" 1,067 words into a single paragraph wouldn't be acceptable in academia for certain reasons, and it's certainly not going to to work on an internet forum where readers skim comments for key words before deciding to invest in what a poster was written. I didn't get past the first sentence, but I'm certain there are other issues, like improper grammar and run-on sentences.

    Of course, you can alway use the defense that your comment was assimilated by the Borg.
    roundaboutnowstompydamn_its_hotloquitur
  • Reply 78 of 118
    misamisa Posts: 827member
    The issue that made CR successful no longer exists.

    From 1932 until 1940 consumer demeanor demand could not be satisfied.  The Depression destroyed consumer buying power resulting in factory closures.  From 1941 through 1945 (WWII) consumer demand exploded but nothing consumer was being produced (war effort).  It took about 2 years to convert our wartime economy to a peace time economy.  From then on US manufacturers could not satisfy all that pent up consumer demand.  The larger retailers employed in-house repair facilities TO REPAIR BRAND NEW PRODUCTS BEFORE PUTTIMG THEM ON THE FLOOR.

    US manufacturing quality was dismal.  This was overlooked by consumers having no other options (European and Japanese manufacturing were still recovering from WWII).

    This condition continued until the early 1960s in the consumer electronics market when Japan began exporting radios, TVs, etc. ,that "just worked".  It took another 20 years before Detroit, faced with extinction, began improving the quality of its products.

    From the end of WWII until the early 1990s the market was ripe for an unbiased authority that test and recommend products.  CR filled and owned that niche.

    For the last 25 years or so, CR has become increasingly irreverent as US manufacturers have been replaced by offshore manufacturing or have adopted a "Japanese" quality ethos.  The market niche that created CR has passed, it will only be a matter of time before CR does too.
    So basically, CR is still needed since every time a company changes who they outsource to (first American, then Japanese, then Korean, now Chinese) we end up going through a cycle of "poisoned food, lead/cadmium/nickel in childrens toys, exploding batteries in phones/laptops, and know-nothings in the political position to change things"

  • Reply 79 of 118
    SoliSoli Posts: 10,035member
    crowley said:
    Pretty reasonable thing for CR to do and unfortunate for both CR and Apple that they ran into a bug which caused such a kerfuffle.

    Anyone flinging vitriol to blame either side needs to go spend some time outside for a change.
    Why shouldn't there be blame. Is having a bug in a SW not the result of Apple's software developers making a mistake? And doesn't a wide a range of atypical results ranging from about 4.5 to 19.5 hours indicate that CR should have questioned something about their test? Not necessarily a problem with their methodology, which is evidence by previous Mac tests and with tests with Chrome, but with something unique that should have brought them to a conclusion that there's a bug in Safari that's causing an issue for their tests.

    Haven't they contacted manufacturers in the past when something they were testing wasn't working properly? And this isn't even a HW issue (although that was possible).
  • Reply 80 of 118
    AppleZuluAppleZulu Posts: 1,989member
    Consumer Reports explains their testing methodology as essentially an attempt to reduce variables across different test subjects. I get that, but disagree that it offers an equitable comparison. (As an aside, by switching off the browser cache in an effort to reduce testing variables, CR apparently created a variable missed during Apple's own testing of the product, triggering the issue with reloading icons. Nice catch, but they found an obscure bug, not a fundamental flaw...) In their methodology explanation, CR notes that various manufacturers will use different techniques to enhance battery life, such as dimming the screen, the browser cache, etc.  So to 'even things out,' they disable all those things for their tests.

    For Apple, since they design the OS in conjunction with the hardware, it's extremely likely they're taking all sorts of opportunities to tweak things to improve performance. That's why comparing raw hardware stats for memory, CPU, bus, etc., doesn't offer a proper comparison to other computers. So in effect, CR is de-engineering the devices before they test them. They think they're limiting variables, but they're not. They're adding back in variables that the computers' designers had engineered out. Under this methodology, a manufacturer that ham-fists the rest of the design and software, but shoves an enormous battery into a thicker notebook, would perform better than a MBP with a smaller battery operating a much more efficient machine. That's not a good test.

    Imagine producing a powerful, fuel-efficient car, and getting a 'not recommended' review from CR. You ask what happened, only to learn that CR removed the turbo (something standard in the design of your car), so that it could be compared to other cars that don't have a turbo, and they subsequently found your car to be underpowered and much less fuel efficient than advertised. Is that a good or fair test? No, it's not.
    edited January 2017 dewmewilliamlondonloquitur
Sign In or Register to comment.