Test finds Apple's MacBook and MacBook Pro only laptops to match or beat advertised batter...

13»

Comments

  • Reply 41 of 51
    anome said:
    anome said:
    [...] As for whether Consumer Reports should have published, it does seem odd that they went straight to press before Apple got a chance to fix the bug. Put simply, CR went for a big headline to get attention.
    If they were testing cars and found that a particular model had wildly inconsistent fuel consumption results, would you expect CR to contact the manufacturer and work out the reason, or simply publish the results at tested?

    If they were testing ovens and found that the temperature of a particular model varied wildly from the selected value, do you think they should work with the manufacturer on figuring out why, or publish what their tests measured?

    If they were testing amplifiers and found the the power output varied from test to test, would you consider the results invalid if they didn't contact the manufacturer to figure out why?

    It's not CR's job to solve manufacturing problems. They test what hits the streets and report their results. It's ridiculous to assert that they should have consulted with Apple. Why would they do that in this case when they never have before and it's completely outside the scope of what they do?

    I understand the argument that the testing methodology unintentionally "activated" an obscure bug that may not affect most actual users, but not that CR did something wrong. First, the testing method is the same for everybody, and only Apple's product reacted badly to it. Second, CR's job is "test and report," not "test, provide supplier chance to provide better sample, retest, reconsult manufacturer, retest, report only when results satisfy the product's fans."

    There is definitely some prejudice affecting opinions here, with advocates and apologists suggesting that Apple was victimized by not being afforded special consideration that's never been extended to anyone else. It's illogical.

    BTW, to be clear, I'm not critical of Apple in this matter. I'm just saying people's expectations of CR are unreasonable.

    If, Consumer Reports were testing a car, and were getting wildly inconsistent fuel consumption results that were not being reported by people using the same model of car, then yes I would expect them to contact the manufacturer.

    If they were testing ovens, and found the temperature of a particular model varied wildly from the selected value, and there weren't reports of people either getting their roasts burnt, or coming out raw on different attempts, then yes I would expect them to try and find out why.

    And so on...

    It's not CR's job to solve manufacturing problems, it is, however, their job to make sure the results they get are representative of the real world, which their tests weren't. None of the reports from end-users seemed to show the same kind of inconsistency that the CR results did.

    Except that in the case of the new MacBook Pro, there WERE reports from users that battery consumption rates were all over the place. There were even discussion of the issue on this forum.

    Further, you're saying that CR should be able to deduce that a test is flawed if their results don't match those of actual users. How would they know what actual users are experiencing? Have you called them up to tell them how things are going with your MacBook Pro? I haven't. Especially in the case of a new product, where is the baseline "real world" data against which you want them to compare their results?

    I honestly think you're letting affection for Apple cloud your perception of Consumer Reports in this matter. It's unfortunate that things played out the way they did, but I just can't see how CR was in any way culpable or negligent.
    edited April 2017
  • Reply 42 of 51
    SoliSoli Posts: 10,035member
    anome said:
    anome said:
    [...] As for whether Consumer Reports should have published, it does seem odd that they went straight to press before Apple got a chance to fix the bug. Put simply, CR went for a big headline to get attention.
    If they were testing cars and found that a particular model had wildly inconsistent fuel consumption results, would you expect CR to contact the manufacturer and work out the reason, or simply publish the results at tested?

    If they were testing ovens and found that the temperature of a particular model varied wildly from the selected value, do you think they should work with the manufacturer on figuring out why, or publish what their tests measured?

    If they were testing amplifiers and found the the power output varied from test to test, would you consider the results invalid if they didn't contact the manufacturer to figure out why?

    It's not CR's job to solve manufacturing problems. They test what hits the streets and report their results. It's ridiculous to assert that they should have consulted with Apple. Why would they do that in this case when they never have before and it's completely outside the scope of what they do?

    I understand the argument that the testing methodology unintentionally "activated" an obscure bug that may not affect most actual users, but not that CR did something wrong. First, the testing method is the same for everybody, and only Apple's product reacted badly to it. Second, CR's job is "test and report," not "test, provide supplier chance to provide better sample, retest, reconsult manufacturer, retest, report only when results satisfy the product's fans."

    There is definitely some prejudice affecting opinions here, with advocates and apologists suggesting that Apple was victimized by not being afforded special consideration that's never been extended to anyone else. It's illogical.

    BTW, to be clear, I'm not critical of Apple in this matter. I'm just saying people's expectations of CR are unreasonable.

    If, Consumer Reports were testing a car, and were getting wildly inconsistent fuel consumption results that were not being reported by people using the same model of car, then yes I would expect them to contact the manufacturer.

    If they were testing ovens, and found the temperature of a particular model varied wildly from the selected value, and there weren't reports of people either getting their roasts burnt, or coming out raw on different attempts, then yes I would expect them to try and find out why.

    And so on...

    It's not CR's job to solve manufacturing problems, it is, however, their job to make sure the results they get are representative of the real world, which their tests weren't. None of the reports from end-users seemed to show the same kind of inconsistency that the CR results did.

    Further, you're saying that CR should be able to deduce that a test is flawed if their results don't match those of actual users.
    Absolutely! I'm sure you've written something in SW, even a formula in a spreadsheet, that was wildly off even though you had no warning signs when you set it up, even when you've gone through that same routine.

    It also doesn't have to be a formula, but a big in an app—which this was—or a website that's down, or even one hat appears to be down because of DNS issues. So many possibilities, and in none of those cases in guessing you assumed that your Hw was absolutely broken as the first thing that came to mind.

    Hell, most on this forum probably have multiple web browsers just to double check how a page might be loading in the off chance something seems amiss.

    Since CR changed default settings through a dev mode and had an extremely high and low battery life results for the same test, the first thing I'd do as some that has to test technology is to 1) remove any added elements to see if the results change, and 2) try the test with different variables, like running in Google Chrime to see if it's a browser issue -and- running on IE on Windows 10 via Bootcamp to see if it's a macOS issue.

    Any of those would have established a baseline that would show that the battery HW was not at fault.

  • Reply 43 of 51
    Rayz2016Rayz2016 Posts: 6,957member

    Rayz2016 said:
    You'd have a bit more credibility if you knew the difference between an "application" and an "operating system". 

    The bug was in Safari (an app-li-cay-shun), not MacOS (which is an operating sis-tum). 

    Read up, then come back. 
    Are you actually suggesting that the entirety of the reasoning behind the argument is invalidated by a semantic error? Or did you just seize an opportunity to be insulting, even though it contributed very little to the discussion? Could the same point have been made by politely correcting the error, or do you actually disagree with the point of the post? If the latter, you didn't bother to explain why. @freeper did.

    Read the post again, this time with the word "application" replacing the erroneous instances of "operating system." Does it make sense and are the points valid? I think so.

    Actually,  I won't read the post again, because it will say exactly the same thing. The poster claimed that the problem was in the OPERATING SYSTEM, a claim that he repeated several times. It is more than a semantic error because it changes the whole course of the argument: had the problem been with the operating system then it is likely that far more people and testers would have been seeing spurious battery results as the operating system would have the problem, not the single application that was causing it.  The difference is important to anyone who has a modicum of understanding about the software development process.

    The problem with Safari is annoying, but given that it only occurs in a developer-only setting and can only be discovered when running battery-draining tests over and over again (remember that no other testing site discovered it), then as a developer and tester I can understand how that could have been missed. And as a developer and tester, I can also tell you that there are likely to be many more similar edge case bugs with other Apple apps that will cause similar problems, and will need an unusual set of circumstances for them to be uncovered.

    Now if, as the poster asserted, the problem had been caused by the operating system, then I would want to know how Apple's own battery-drain tests could have possibly missed it. If this bug had been related to the operating system, then I would want to see Apple overhauling its entire battery-test procedure. (I won't say that people should be fired because that is an internal matter).

    So, in more than a nutshell, that is why the difference is not simply a 'semantic error'.

    I did not feel the need to politely correct the error because I did not believe it was an error: more likely a deliberate attempt to introduce the idea that Apple had a serious bug in it's operating system, hence the  constant repetition of the phrase 'bug in the operating system'.  

    But while we're on the subject of dismissing errors to suit our argument, then I suggest that you educate yourself by reading this:

    https://www.theguardian.com/books/2017/mar/16/oxford-comma-helps-drivers-win-dispute-about-overtime-pay
    http://www.bbc.co.uk/news/business-39300432

    A very good example of what it can cost when you dismiss a 'small error'.

    edited April 2017
  • Reply 44 of 51
    Rayz2016Rayz2016 Posts: 6,957member
    anome said:
    anome said:
    [...] As for whether Consumer Reports should have published, it does seem odd that they went straight to press before Apple got a chance to fix the bug. Put simply, CR went for a big headline to get attention.
    If they were testing cars and found that a particular model had wildly inconsistent fuel consumption results, would you expect CR to contact the manufacturer and work out the reason, or simply publish the results at tested?

    If they were testing ovens and found that the temperature of a particular model varied wildly from the selected value, do you think they should work with the manufacturer on figuring out why, or publish what their tests measured?

    If they were testing amplifiers and found the the power output varied from test to test, would you consider the results invalid if they didn't contact the manufacturer to figure out why?

    It's not CR's job to solve manufacturing problems. They test what hits the streets and report their results. It's ridiculous to assert that they should have consulted with Apple. Why would they do that in this case when they never have before and it's completely outside the scope of what they do?

    I understand the argument that the testing methodology unintentionally "activated" an obscure bug that may not affect most actual users, but not that CR did something wrong. First, the testing method is the same for everybody, and only Apple's product reacted badly to it. Second, CR's job is "test and report," not "test, provide supplier chance to provide better sample, retest, reconsult manufacturer, retest, report only when results satisfy the product's fans."

    There is definitely some prejudice affecting opinions here, with advocates and apologists suggesting that Apple was victimized by not being afforded special consideration that's never been extended to anyone else. It's illogical.

    BTW, to be clear, I'm not critical of Apple in this matter. I'm just saying people's expectations of CR are unreasonable.

    If, Consumer Reports were testing a car, and were getting wildly inconsistent fuel consumption results that were not being reported by people using the same model of car, then yes I would expect them to contact the manufacturer.

    If they were testing ovens, and found the temperature of a particular model varied wildly from the selected value, and there weren't reports of people either getting their roasts burnt, or coming out raw on different attempts, then yes I would expect them to try and find out why.

    And so on...

    It's not CR's job to solve manufacturing problems, it is, however, their job to make sure the results they get are representative of the real world, which their tests weren't. None of the reports from end-users seemed to show the same kind of inconsistency that the CR results did.


    And you have hit the nail on the head.

    Anyone else would have questioned why they were getting such inconsistent results and reported them to the manufacturer.  No one else reported the same set of results.

    I'm not even saying that they shouldn't have reported the results, but I they should have said, "The inconsistencies are worrying, which is why we have contacted Apple so they can look over our results."

    I think their desire to garner page hits smothered their desire to get to the root cause. You can argue that it isn't there job to investigate further, but then you could also argue that it isn't their job to fiddle with setting that consumers don't touch.
  • Reply 45 of 51
    sflocal said:
    Doesn't matter.  Asshat iHater, trolls, and pundits will simply ignore it and continue blaming Apple for being the best in laptops, while giving a free pass to the competitors for making shitty products.
    Oh fer fuxsake, no one is vilifying Apple OR ignoring shortcomings in other products.


    Wow! You actually said that with a straight face (I'm assuming. I can't actually see your face).

    You are either too naïve or are blind to trolls.

    edited April 2017
  • Reply 46 of 51
    lorin schultzlorin schultz Posts: 2,771member
    Rayz2016 said:

    [...] The poster claimed that the problem was in the OPERATING SYSTEM, a claim that he repeated several times. It is more than a semantic error because it changes the whole course of the argument: had the problem been with the operating system then it is likely that far more people and testers would have been seeing spurious battery results as the operating system would have the problem, not the single application that was causing it.


    Fair enough. I doubt @freeper was being intentionally nefarious -- people around here frequently use the term "memory" when they mean "storage" -- but your point is valid.

    Rayz2016 said:

    But while we're on the subject of dismissing errors to suit our argument, then I suggest that you educate yourself by reading this:

    Yup, I'm familiar with that example (do you think it's real? It doesn't matter for the purposes of the exercise but I do wonder if such a matter really found its way to the courts), and understand what you mean, I just think in THIS particular case it was an innocent oversight. I could be wrong.

  • Reply 47 of 51
    lorin schultzlorin schultz Posts: 2,771member

    Soli said:

    [...] Any of those would have established a baseline that would show that the battery HW was not at fault.


    If I'm the one publishing test results, I don't really care what CAUSES erratic behaviours. That's not my problem. My job is to take the unit I bought out of the same inventory everyone else buys from, test it, and tell my subscribers how and what it did. The End.

    If a car exhibits inconsistent braking, which many do, the testing facility doesn't consult with the manufacturer to find out why, they simply report the results of their brake tests. Why is CR being held to a higher standard in this case? I mean OTHER than because everybody's favourite company came off looking less than perfect for a change...

    You're right about the testing procedure creating conditions that differ from what a typical user is likely to encounter, but, and this is critical, it's the same method they used to test previous versions of the MacBook Pro. They activated whatever developer thingy it was in an attempt to simulate a real-world condition that couldn't otherwise be reliably repeated across multiple tests and machines.

    Ultimately the flaw wasn't in the testing, it was in the Mac. It was a minor issue that was unlikely to affect a lot of users, and it really is unfortunate that such a tiny little bug caused such a bizarre outcome. It sucks for Apple. That does not mean CR did anything wrong. I guess we just disagree about what responsibility a testing agency has and to whom it's responsible. I thought both Apple and CR handled it really well with follow-up.
  • Reply 48 of 51
    lorin schultzlorin schultz Posts: 2,771member

    Wow! You actually said that with a straight face (I'm assuming. I can't actually see your face).

    You are either too naïve or are blind to trolls.


    Honestly, what I see mostly is a few key players here "predicting" negative comments and bemoaning them before anyone else has even replied. I see people get bent out of shape and attack anyone who expresses an opinion that is not praise for Apple. If anything I'd say this forum is much more prone to cultish fanboyism than trolling, but I suppose it depends on how posts are perceived and which threads one reads.

    What some of the regulars here seem to overlook is that some of us are neither haters nor fans of Apple. We're people who have chosen to use certain Apple products because they happen to best suit our particular needs at a given point in time. It's not a love/hate thing. For most people, expressing an opinion about the company or its products is neither an attack nor a defence -- it's a reflection of personal preference based on what one hopes to do with the product.

    Perhaps if viewed from that perspective, posts that don't portray Apple as incapable of doing any wrong will not be perceived as a threat to peace, liberty, and the Apple way of life. :)
  • Reply 49 of 51
    SoliSoli Posts: 10,035member
    Soli said:

    [...] Any of those would have established a baseline that would show that the battery HW was not at fault.
    If a car exhibits inconsistent braking, which many do, the testing facility doesn't consult with the manufacturer to find out why, they simply report the results of their brake tests. Why is CR being held to a higher standard in this case? I mean OTHER than because everybody's favourite company came off looking less than perfect for a change…
    That's a great example, since you can easily fuck with a car by altering its software. Can you imagine if CR decided to change every automobile they tested to use the same fuel to air ratio to be fair and then then published the results of cars that weren't operating under factory specs? I don't think you'd be saying "I don't really care what CAUSES erratic behaviours. That's not [their] problem. [Their] job is to take the unit [they] bought out of the same inventory everyone else buys from, test it, and tell my subscribers how and what it did. The End."

    What CR did was only one step above the false reporting of explosive fuel tanks on truck in the 80s. That's a big step since they're no evidence that they sought to create an issue, but they certainly either did excessive shoddy testing or decided to ignore proper testing after discovering this dev mode bug in order to generate more buzz.
    edited April 2017
  • Reply 50 of 51
    lorin schultzlorin schultz Posts: 2,771member
    Soli said:

    [...] they certainly either did excessive shoddy testing [...]

    I think this is where our views differ. I don't think the testing was "shoddy." I think the motives were good. They did use a fairly obscure setting that most users won't access, but only in an attempt to create a more accurate reflection of real-world use.

    Apparently it hasn't been a problem before, and only the presence of the tiny, easily-forgivable bug made it an issue. That's why I think that even though it's really unfortunate that it had the ramifications it did, neither party is in a position of being "at fault." It was just a shitty thing that arose from otherwise innocuous issues.

    Soli said:

    [...] or decided to ignore proper testing after discovering this dev mode bug in order to generate more buzz.
    I doubt very much that CR needs to worry about generating buzz. In order to ignore proper testing (and I'm not conceding that their method was improper in the first place) they would have to have known what was causing the issue. I thought they were pretty clear about stating in the original review that they didn't KNOW why the results were so weird, and in reinforcing the point that the results seemed strange even to them. They then did an excellent job of clearing the air with the follow up.
  • Reply 51 of 51
    linkmanlinkman Posts: 1,035member
    zoetmb said:
    Notsofast said:
    freeper said:
    dbeats said:
    Where's the outrage now? Also, doesn't this just prove the Consumer Reports cannot be trusted with any claims anymore?
    No, absolutely not. Despite Apple's PR spin and the same by Apple promoters and apologists, THE CONSUMER REPORTS TEST FOUND A BUG IN THE OPERATING SYSTEM. Let me repeat. There was a bug in the operating system that Apple did not know about. This bug in the operating system was found only because of Consumer Reports' test. As a result of Consumer Reports' test - and not anything in Apple's software or QA efforts - Apple identified the bug and released a fix.

    Blaming Consumer Reports for having what the writer claims is an obscure setting is totally wrong. First off, it is not obscure AT ALL. It is the equivalent of setting "private browsing", and also QA testers, programmers and others NEED and REGULARLY USE that setting. Second, it is a feature that Apple chooses to provide. Consumer Reports did not create their own hack or load their own codes or scripts. It is a setting that APPLE PROVIDES in the browser, is listed BY APPLE as a setting/menu option, and IT IS APPLE'S JOB TO MAKE SURE THAT IT WORKS, even if it is obscure (which it isn't). Finally, CONSUMER REPORTS HAD USED THAT SAME SETTING IN THE PAST. Let me restate. CONSUMER REPORTS USED THAT SAME "DEVELOPER SETTING" FOR THEIR PAST TESTS FOR MACS IN YEARS PAST AND THEY PERFORMED FINE. Why? Because the bug in Apple's OS didn't exist in the past. It was only when the bug was present that it was a problem. When Apple's bug in Apple's operating system caused a problem in Apple's browser, they fixed it. Consumer Reports didn't change squat. Apple did, and the good results were reached as a result.

    Oh yes, another thing: those "developer settings" are used when Consumer Reports tests other computers too. When they test computers by Lenovo, HP, Dell, Asus etc. in those charts up there, they use those same "developer settings" because running the sort of tests that they do without those settings is ridiculous. They ran those same tests using Chrome, Edge, Firefox, IE etc. browsers with the same "obscure settings" and had no problems. Why? Because the bug was not in Windows, only macOS. Had it been in Windows, Microsoft would have released a fix just like Apple did.

    Bottom line: quit blaming Consumer Reports for Apple's bug. Unless you are one of those people who claims that Consumer Reports shouldn't have released the review in the first place without giving Apple time to fix their product flaws first. Sorry, but Consumer Reports is not Apple's PR department. Apple's PR department did their job when they (falsely) claimed that Consumer Reports' test was wrong. Even though Consumer Reports RAN THE EXACT SAME TEST AGAINST THE EXACT SAME HARDWARE AFTER APPLE FIXED THE BUG AND GOT THE DESIRED RESULTS.
    Wow, you need some balance and objectivity.  I am CR long time subscriber, but you are glossing over their irresponsibility in rushing out test results to meet a deadline when their own results didn't make any sense. The responsible thing would have been to work with Apple to see if there was something in their testing methodology, (yes there was) or a bug somewhere (yes, as well) that was causing the anomalous (absurdly high battery life results as well).  Objectively, CR hurt its credibility on this as no responsible testing organization, getting these wildly inconsistent results, wouldn't have gone back to figure out the problem, but it was just too sexy of a title at that time of year for whoever made the decision to publish to resist.  They compounded their error by issuing a "no buy" recommendation.  Those of us who support CR are the ones most disappointed in their actions because we need organizations like CR to protect consumers and when they do something that damages their credibility, they undermine that role.  Sadly, many consumers who are more casual followers of CR have likely written them off. 
    Around 1973, CR said that the AMC Gremlin was the best subcompact made.   I bought one.   It was one of the worst cars ever made.   I'd say the worst, but there was a Ford car that would blow up if it was hit.   Everything that could possibly go wrong in a car went wrong in that car, EXCEPT for the air conditioner.   Valves, rings, wheel bearings, oil leaks, electrical shorts (horn starts honking by itself) and lots of other stuff I can no longer remember.  Even the piece of metal that held the seats up broke and I remember I had to jerry rig some kind of rod to keep them up.    I put as much into that stupid car as it cost to buy (which was only $3K, but that's $16,460 in 2017 dollars, which means it cost me the equivalent of $32K to own that car).   CR later apologized.   But I never trusted them again. 
    You expected no gremlins from a car called the Gremlin? Buwhahahaha...
Sign In or Register to comment.