I see numerous people bashing Consumer Reports since they've given Apple ONE bad review... but I haven't seen anybody come up with a BETTER way of doing things than Consumer Reports currently does. They test a wide range of devices and are pretty well respected. They were the ones that noticed the problems with the Lexus GX 460's suspensions which initiated a recall.
So if they're doing it wrong, how should they do it right?
I'm not saying they are always wrong. What I am saying is that they aren't reliable, that their objectivity is a pretense, and their tests and ratings, if the score a product highest, yet they don't recommend it, aren't meaningful. If they were meaningful, if they were objective, then the highest rated product would be recommended as the best. The fact that is not the case shows that their process is not well thought out, not "scientific" in any meaning of the word, and in large part arbitrary and subjective. Perhaps even grounded in ignorance of what they claim to be objectively testing.
That they are "well respected" is beside the point. Enron was once well respected. Bernard Madoff was once well respected. I'm not equating Consumer reports with Enron and Madoff, but I am pointing out that "respectability" is not something that guarantees integrity, and therefore largely meaningless as a refutation of the charge that their process is not objective or reliable.
As to you question of, "So if they're doing it wrong, how should they do it right?" that's also irrelevant. A critique of their methodology and objectivity does not in any way depend for its validity on the critic suggesting an alternative. The criticisms are valid and stand on their own. I realize that such a question is a rhetorical device, often employed successfully, to elicit an emotional response of, "Yeah, if you can't do better then who are you to criticize?" but this is not a question of emotion, it's a question of reason, and reason tells us that Consumer Reports is not providing the service they purport to.
The question is, is what they provide of any value at all? The answer would seem to be that we can have no faith that it is for any particular case, even if in some cases it seems correct.
iPhone is released here in a few hours, I'm trying to decide which network to use. The one that's renown as being the best (Telstra) - but gets 500MB data, or the one that I've got now that drops to 2G a lot and loses calls fairly often really (Vodafone)... but gets 1.5GB of data and is about to roll out new better frequencies...
This article leans me towards using the better Telstra network.
(Nice deals on both... Upfront $150(Telstra) or $120(Vodafone) - then $49/mth for about 400 minutes.)
Telstra is more $ but solid. Tethering is free. They also upped data $79 is now 1gig (not as good but worthy).
I live in Alice Springs, it's fast, solid. Start os this year I drove half way round the country with a 3GS, gps, net, maps, calls, email everything and the only times I had nothing was cause there was NOTHING, like at lake eyre, the middle of a desert etc. The phone is superb with a great network.
There are crap loads of iPhone's out here. Only issue anyone says about "calls" is that it's too quiet. I suspect the noise canceling mic will sort that a bit.
All the -mass- dramas in the media are exclusively American and exclusively AT&T. They lack any frame of reference and seem disinterested in being aware of other experiences.
Mossberg makes some good points. But his BOTTOM LINE seems to be that if the places where you use your IPHONE4 has bad or poor reception than the IPHONE4 is not going to improve it and as a result theIPHONE4 will drop more calls than in an area with GOOD RECEPTION.
This has always been an important fact when using ATT. You have to BUY & TRY the IPHONE4 in the areas where you normally would be using it and see how your service is. Theres no way you can know otherwise.
As for the bars on the phone, with the update they are more accurate. However some people will have a great experience with 1 bar while others will drop a call with 3 bars showing. Once again its about the strength of service where U are using the IPHONE4
Everyone knows ATT sucks and regardless of what smartphone you use with its still ATT.
As somebody once said to me................'You have to give up pristine service that you may be getting with Verizon to get a great IPHONE4.'
Exactly, even a brief apology to all the people they put into trouble wouldn't harm them.
I only read consumer reports for reviews on Toyotas and washing machines...for years they were biased against Apple computers...recommending Dell, HP, Sony, etc.
One thing for sure, Apple users aren't the independent thinkers they used to be in the 80's, 90's and early 2000's. Apple Computer truly is dead, and has been replaced by a company that no longer encourages its users to think different.
I think this guy sums up nicely why the haters hate:
this is the person that compared snow leopard to windows 7? i believe i'd look for advice on a mobile phone elsewhere...
I read the WSJ everyday and I too find he's always hedging not to be too complimentary towards Apple...I mean who in their right mind would recommend a Dell or MS product...seriously, Vista, Zune, Kin, Live, MSN, Hotmail, Office...talk about clunky software....please.
I only read consumer reports for reviews on Toyotas and washing machines...for years they were biased against Apple computers...recommending Dell, HP, Sony, etc.
No thanks consumer reports!
Really, I didn't know CR was so against apple products. But this would explain everything. But anyway despite the the completely distorted press reaction, people are still standing in cues to get their devices.
Well, you can't look at the content behind the paywall without being a member. They do have freely available content, such as their flip-flopping blog entries on this topic. I think two things are pretty clear:
1. Consumer Reports, like most blogs and news sites that jumped on this particular bandwagon, was driven in significant part by a desire to stir up controversy and drive traffic to their site. In their case it wasn't to increase ad revenue, but traffic to their site doubtlessly drives subscriptions to the content behind their paywall. There is an obvious conflict of interest here, highlighted by their handling of the issue, that calls into question their motives. Remember, employees at non-profits don't work for free, and their compensation, just like everyone else's (unless you work on Wall St) depends on how much money their employer pulls in.
2. Consumer Reports has no idea what they are doing evaluating smartphones. Not all that surprising since I think most people have had the experience with CR that reviews of things you know about usually make no sense. (Meaning that it's likely that reviews of things you don't know about may not make sense either.)
But, the more serious problem is that, from a purely objective standpoint, this particular review process is so obviously broken. The iP4 has the highest rating of any smartphone, yet is not recommended. Now CR does a little song and dance number to try to explain this, but what exactly do their ratings mean if recommendations aren't based on them? Is this an objective process or not?
The answer is that it is obviously not an objective process. If it were, their ratings and recommendations would agree, rather than contradict. In fact, if they can't recommend the iP4 based on their ratings, then the entire rating system is meaningless. Well, you say, they found other issues with testing beyond what they look at for the ratings. But, that's exactly the point. If the ratings don't adequately reflect the factors which determine whether they can recommend something or not, then what exactly is their entire testing and rating system based on? And, if we can't trust their smartphone ratings, which of their other ratings can we trust?
The answer is obviously, none. Consumer Reports is engaged in something very like pseudo-science. They pretend they go through this rigorous and objective testing process, but this incident shows that their rating criteria are either completely arbitrary and irrelevant, or that their recommendations are based on entirely subjective factors, and the whole ratings process is nothing but smoke and mirrors. If it's a scandal and a fraud that the media are looking for, they need look no further than Consumer Reports.
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
I see numerous people bashing Consumer Reports since they've given Apple ONE bad review... but I haven't seen anybody come up with a BETTER way of doing things than Consumer Reports currently does. They test a wide range of devices and are pretty well respected. They were the ones that noticed the problems with the Lexus GX 460's suspensions which initiated a recall.
So if they're doing it wrong, how should they do it right?
I also find it funny that everyone was quick to praise CR when things were giving Apple good ratings, but when one bad rating comes up, it's time to pull out the Army and destroy them.
You can't have it both ways.
Good point, which only reinforces what I've said. CR's other results on Apple products (such as giving them consistently high marks for reliability and quality of tech support) can be trumpeted as truth, but if CR gives Apple one black mark (or even one gray mark), this turns them instantly into an out-and-out fraudulent organization with cynical motives and no credibility whatsoever.
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
Good point, which only reinforces what I've said. CR's other results on Apple products (such as giving them consistently high marks for reliability and quality of tech support) can be trumpeted as truth, but if CR gives Apple one black mark (or even one gray mark), this turns them instantly into an out-and-out fraudulent organization with cynical motives and no credibility whatsoever.
Quote:
Originally Posted by anonymouse
Well, you can't look at the content behind the paywall without being a member. They do have freely available content, such as their flip-flopping blog entries on this topic. I think two things are pretty clear:
1. Consumer Reports, like most blogs and news sites that jumped on this particular bandwagon, was driven in significant part by a desire to stir up controversy and drive traffic to their site. In their case it wasn't to increase ad revenue, but traffic to their site doubtlessly drives subscriptions to the content behind their paywall. There is an obvious conflict of interest here, highlighted by their handling of the issue, that calls into question their motives. Remember, employees at non-profits don't work for free, and their compensation, just like everyone else's (unless you work on Wall St) depends on how much money their employer pulls in.
2. Consumer Reports has no idea what they are doing evaluating smartphones. Not all that surprising since I think most people have had the experience with CR that reviews of things you know about usually make no sense. (Meaning that it's likely that reviews of things you don't know about may not make sense either.)
But, the more serious problem is that, from a purely objective standpoint, this particular review process is so obviously broken. The iP4 has the highest rating of any smartphone, yet is not recommended. Now CR does a little song and dance number to try to explain this, but what exactly do their ratings mean if recommendations aren't based on them? Is this an objective process or not?
The answer is that it is obviously not an objective process. If it were, their ratings and recommendations would agree, rather than contradict. In fact, if they can't recommend the iP4 based on their ratings, then the entire rating system is meaningless. Well, you say, they found other issues with testing beyond what they look at for the ratings. But, that's exactly the point. If the ratings don't adequately reflect the factors which determine whether they can recommend something or not, then what exactly is their entire testing and rating system based on? And, if we can't trust their smartphone ratings, which of their other ratings can we trust?
The answer is obviously, none. Consumer Reports is engaged in something very like pseudo-science. They pretend they go through this rigorous and objective testing process, but this incident shows that their rating criteria are either completely arbitrary and irrelevant, or that their recommendations are based on entirely subjective factors, and the whole ratings process is nothing but smoke and mirrors. If it's a scandal and a fraud that the media are looking for, they need look no further than Consumer Reports.
Quote:
Originally Posted by christopher126
I only read consumer reports for reviews on Toyotas and washing machines...for years they were biased against Apple computers...recommending Dell, HP, Sony, etc.
No thanks consumer reports!
Quote:
Originally Posted by Dr Millmoss
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
CR is good! CR is bad! no CR is good! Now I am confused. But nevertheless, I think giving the iP4 best ratings and still not recommending it, is even more confusing. As a result CR has lost some credibility to me. I think they really should reconsider their statement.
CR is good! CR is bad! no CR is good! Now I am confused. But nevertheless, I think giving the iP4 best ratings and still not recommending it, is even more confusing. As a result CR has lost some credibility to me. I think they really should reconsider their statement.
Hi Rabbit Coach, a bit off topic: but is 'Rabbit Coach' a sexual reference? Or am I just seeing sexual references where there are none?
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
Well, you misrepresent my argument entirely when you say, "the argument that CR only does what they do to drive web hits." First, that wasn't even my primary criticism, and secondly, there's nothing inconsistent with accusing them of going after web hits, which lead to an enlarged subscriber base, which may lead to increased individual salaries, and admitting that this may not be their only motivation. Do you really believe that the individuals who work at Consumer Reports do what they do entirely out of a sense of selfless altruism? Or, may it not be possible that some of them do it because they are paid well?
As to my primary criticism -- that their testing is not objective nor reliable -- I think the facts stand for themselves. There is really no argument around the fact that, based on their ratings, the iP4 is the best smartphone on the market. Yet, it is not one of their recommended phones. This means that their ratings are not based on the factors that cause them to recommend products. If not, then what are they based on? Unless their ratings and recommendations align, it is unavoidable to conclude that their recommendations are not objective and/or that their rating system is flawed. You can't escape one or the other, or both of those conclusions.
The fact that they have been doing this for 60 years is irrelevant. They may always have been unreliable, (A possibility, since I have almost never agreed with their recommendations on any product of which I have knowledge.) they may recently have become unreliable, or they may have always been sometimes unreliable.
But, they are either engaged in objective testing and rating of products , with meaningful tests and a meaningful rating system, or they are not. That the highest rated product is not recommended, indicates that their rating system and testing is in at least this case flawed. That it is in this case, casts doubt of their reliability and objectivity in all cases.
(And, allowing a video with snarky, unprofessional duct tape comments certainly doesn't help their case. In fact it reinforces the notion that they have allowed bias into their testing process.)
My comments on "scandal and fraud" were, however, intended mostly for the media. As we all know, the lust for sensationalism in that quarter is insatiable. And anything they can whip into the appearance of scandal and fraud is sauce for them. The question is, why isn't the inconsistency of Consumer Reports testing and recommendations coming under scrutiny here by that same media?
The comment on "pseudo-science" is, however, entirely apt, in regard to Consumer Reports methods.
If you say so. But there are other voices out there who say differently.
Yup, mainly people who don't own one of these things, people who blindly hate all things AAPL or the tiny minority of people who have actually experienced an issue. iPhone4 here - no issues, no dropped calles, great signal, no proximity sensor issues. Same with my partners and every friend I have who owns one...
I could have told Apple this antenna issue was going to happen, and I didn't have to go to college and become an engineer, or spend millions on a call testing facility.
Apple blundered big time on this one. And all the Kool Aid on the store shelf ain't gonna make it go away.
Ermmm - did they? Then how come (by far) the vast majority of these devices work without issue? Why the incredibly small return rate? Are every one of the 6 - 7 million (I estimate) current owners on the 'koolaid' - or could it be that (as Jobs stated) that this is a non-issue? I own the device, it works perfectly (without a bumper or case). So if it can work for me, it can't be fundamentally flawed, there must be other factors that affect a tiny percentage of units. And the learn the difference in the way frequency behaves - your old TV was not receiving using the same frequencies and bandwidth as a phone - your logic falls flat on it's ass very quickly.
CR is good! CR is bad! no CR is good! Now I am confused. But nevertheless, I think giving the iP4 best ratings and still not recommending it, is even more confusing. As a result CR has lost some credibility to me. I think they really should reconsider their statement.
If they want to maintain any credibility at all as a reliable testing organization, they need to either reconsider their statement, or reconsider their rating and testing protocols, or both. Fankly, at this point, their ratings and statements, regarding all smartphones, ought to be retracted. They either base their recommendations on valid, reliable, objective tests, or they do not. The evidence in front of us is that they do not.
CR is good! CR is bad! no CR is good! Now I am confused. But nevertheless, I think giving the iP4 best ratings and still not recommending it, is even more confusing. As a result CR has lost some credibility to me. I think they really should reconsider their statement.
They are neither all good nor all bad. They get some things right, and some things wrong, just like all human beings. I don't honestly know whether they've gotten the iPhone 4 testing thing right or wrong, or (more likely) somewhere in between. But as someone else said, their conclusions are virtually the same as Walt Mossberg's, and I notice that nobody is calling him a fraud.
Comments
I see numerous people bashing Consumer Reports since they've given Apple ONE bad review... but I haven't seen anybody come up with a BETTER way of doing things than Consumer Reports currently does. They test a wide range of devices and are pretty well respected. They were the ones that noticed the problems with the Lexus GX 460's suspensions which initiated a recall.
So if they're doing it wrong, how should they do it right?
I'm not saying they are always wrong. What I am saying is that they aren't reliable, that their objectivity is a pretense, and their tests and ratings, if the score a product highest, yet they don't recommend it, aren't meaningful. If they were meaningful, if they were objective, then the highest rated product would be recommended as the best. The fact that is not the case shows that their process is not well thought out, not "scientific" in any meaning of the word, and in large part arbitrary and subjective. Perhaps even grounded in ignorance of what they claim to be objectively testing.
That they are "well respected" is beside the point. Enron was once well respected. Bernard Madoff was once well respected. I'm not equating Consumer reports with Enron and Madoff, but I am pointing out that "respectability" is not something that guarantees integrity, and therefore largely meaningless as a refutation of the charge that their process is not objective or reliable.
As to you question of, "So if they're doing it wrong, how should they do it right?" that's also irrelevant. A critique of their methodology and objectivity does not in any way depend for its validity on the critic suggesting an alternative. The criticisms are valid and stand on their own. I realize that such a question is a rhetorical device, often employed successfully, to elicit an emotional response of, "Yeah, if you can't do better then who are you to criticize?" but this is not a question of emotion, it's a question of reason, and reason tells us that Consumer Reports is not providing the service they purport to.
The question is, is what they provide of any value at all? The answer would seem to be that we can have no faith that it is for any particular case, even if in some cases it seems correct.
If you say so. But there are other voices out there who say differently.
The unhappy voices are always the loudest.
iPhone is released here in a few hours, I'm trying to decide which network to use. The one that's renown as being the best (Telstra) - but gets 500MB data, or the one that I've got now that drops to 2G a lot and loses calls fairly often really (Vodafone)... but gets 1.5GB of data and is about to roll out new better frequencies...
This article leans me towards using the better Telstra network.
(Nice deals on both... Upfront $150(Telstra) or $120(Vodafone) - then $49/mth for about 400 minutes.)
Telstra is more $ but solid. Tethering is free. They also upped data $79 is now 1gig (not as good but worthy).
I live in Alice Springs, it's fast, solid. Start os this year I drove half way round the country with a 3GS, gps, net, maps, calls, email everything and the only times I had nothing was cause there was NOTHING, like at lake eyre, the middle of a desert etc. The phone is superb with a great network.
There are crap loads of iPhone's out here. Only issue anyone says about "calls" is that it's too quiet. I suspect the noise canceling mic will sort that a bit.
All the -mass- dramas in the media are exclusively American and exclusively AT&T. They lack any frame of reference and seem disinterested in being aware of other experiences.
This has always been an important fact when using ATT. You have to BUY & TRY the IPHONE4 in the areas where you normally would be using it and see how your service is. Theres no way you can know otherwise.
As for the bars on the phone, with the update they are more accurate. However some people will have a great experience with 1 bar while others will drop a call with 3 bars showing. Once again its about the strength of service where U are using the IPHONE4
Everyone knows ATT sucks and regardless of what smartphone you use with its still ATT.
As somebody once said to me................'You have to give up pristine service that you may be getting with Verizon to get a great IPHONE4.'
Exactly, even a brief apology to all the people they put into trouble wouldn't harm them.
I only read consumer reports for reviews on Toyotas and washing machines...for years they were biased against Apple computers...recommending Dell, HP, Sony, etc.
No thanks consumer reports!
One thing for sure, Apple users aren't the independent thinkers they used to be in the 80's, 90's and early 2000's. Apple Computer truly is dead, and has been replaced by a company that no longer encourages its users to think different.
I think this guy sums up nicely why the haters hate:
http://chipotle.tumblr.com/post/8617...rs-new-antenna
I wouldn't change a word.
this is the person that compared snow leopard to windows 7?
I read the WSJ everyday and I too find he's always hedging not to be too complimentary towards Apple...I mean who in their right mind would recommend a Dell or MS product...seriously, Vista, Zune, Kin, Live, MSN, Hotmail, Office...talk about clunky software....please.
I only read consumer reports for reviews on Toyotas and washing machines...for years they were biased against Apple computers...recommending Dell, HP, Sony, etc.
No thanks consumer reports!
Really, I didn't know CR was so against apple products. But this would explain everything. But anyway despite the the completely distorted press reaction, people are still standing in cues to get their devices.
Well, you can't look at the content behind the paywall without being a member. They do have freely available content, such as their flip-flopping blog entries on this topic. I think two things are pretty clear:
1. Consumer Reports, like most blogs and news sites that jumped on this particular bandwagon, was driven in significant part by a desire to stir up controversy and drive traffic to their site. In their case it wasn't to increase ad revenue, but traffic to their site doubtlessly drives subscriptions to the content behind their paywall. There is an obvious conflict of interest here, highlighted by their handling of the issue, that calls into question their motives. Remember, employees at non-profits don't work for free, and their compensation, just like everyone else's (unless you work on Wall St) depends on how much money their employer pulls in.
2. Consumer Reports has no idea what they are doing evaluating smartphones. Not all that surprising since I think most people have had the experience with CR that reviews of things you know about usually make no sense. (Meaning that it's likely that reviews of things you don't know about may not make sense either.)
But, the more serious problem is that, from a purely objective standpoint, this particular review process is so obviously broken. The iP4 has the highest rating of any smartphone, yet is not recommended. Now CR does a little song and dance number to try to explain this, but what exactly do their ratings mean if recommendations aren't based on them? Is this an objective process or not?
The answer is that it is obviously not an objective process. If it were, their ratings and recommendations would agree, rather than contradict. In fact, if they can't recommend the iP4 based on their ratings, then the entire rating system is meaningless. Well, you say, they found other issues with testing beyond what they look at for the ratings. But, that's exactly the point. If the ratings don't adequately reflect the factors which determine whether they can recommend something or not, then what exactly is their entire testing and rating system based on? And, if we can't trust their smartphone ratings, which of their other ratings can we trust?
The answer is obviously, none. Consumer Reports is engaged in something very like pseudo-science. They pretend they go through this rigorous and objective testing process, but this incident shows that their rating criteria are either completely arbitrary and irrelevant, or that their recommendations are based on entirely subjective factors, and the whole ratings process is nothing but smoke and mirrors. If it's a scandal and a fraud that the media are looking for, they need look no further than Consumer Reports.
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
I see numerous people bashing Consumer Reports since they've given Apple ONE bad review... but I haven't seen anybody come up with a BETTER way of doing things than Consumer Reports currently does. They test a wide range of devices and are pretty well respected. They were the ones that noticed the problems with the Lexus GX 460's suspensions which initiated a recall.
So if they're doing it wrong, how should they do it right?
I also find it funny that everyone was quick to praise CR when things were giving Apple good ratings, but when one bad rating comes up, it's time to pull out the Army and destroy them.
You can't have it both ways.
Good point, which only reinforces what I've said. CR's other results on Apple products (such as giving them consistently high marks for reliability and quality of tech support) can be trumpeted as truth, but if CR gives Apple one black mark (or even one gray mark), this turns them instantly into an out-and-out fraudulent organization with cynical motives and no credibility whatsoever.
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
Best post of the thread.
Good point, which only reinforces what I've said. CR's other results on Apple products (such as giving them consistently high marks for reliability and quality of tech support) can be trumpeted as truth, but if CR gives Apple one black mark (or even one gray mark), this turns them instantly into an out-and-out fraudulent organization with cynical motives and no credibility whatsoever.
Well, you can't look at the content behind the paywall without being a member. They do have freely available content, such as their flip-flopping blog entries on this topic. I think two things are pretty clear:
1. Consumer Reports, like most blogs and news sites that jumped on this particular bandwagon, was driven in significant part by a desire to stir up controversy and drive traffic to their site. In their case it wasn't to increase ad revenue, but traffic to their site doubtlessly drives subscriptions to the content behind their paywall. There is an obvious conflict of interest here, highlighted by their handling of the issue, that calls into question their motives. Remember, employees at non-profits don't work for free, and their compensation, just like everyone else's (unless you work on Wall St) depends on how much money their employer pulls in.
2. Consumer Reports has no idea what they are doing evaluating smartphones. Not all that surprising since I think most people have had the experience with CR that reviews of things you know about usually make no sense. (Meaning that it's likely that reviews of things you don't know about may not make sense either.)
But, the more serious problem is that, from a purely objective standpoint, this particular review process is so obviously broken. The iP4 has the highest rating of any smartphone, yet is not recommended. Now CR does a little song and dance number to try to explain this, but what exactly do their ratings mean if recommendations aren't based on them? Is this an objective process or not?
The answer is that it is obviously not an objective process. If it were, their ratings and recommendations would agree, rather than contradict. In fact, if they can't recommend the iP4 based on their ratings, then the entire rating system is meaningless. Well, you say, they found other issues with testing beyond what they look at for the ratings. But, that's exactly the point. If the ratings don't adequately reflect the factors which determine whether they can recommend something or not, then what exactly is their entire testing and rating system based on? And, if we can't trust their smartphone ratings, which of their other ratings can we trust?
The answer is obviously, none. Consumer Reports is engaged in something very like pseudo-science. They pretend they go through this rigorous and objective testing process, but this incident shows that their rating criteria are either completely arbitrary and irrelevant, or that their recommendations are based on entirely subjective factors, and the whole ratings process is nothing but smoke and mirrors. If it's a scandal and a fraud that the media are looking for, they need look no further than Consumer Reports.
I only read consumer reports for reviews on Toyotas and washing machines...for years they were biased against Apple computers...recommending Dell, HP, Sony, etc.
No thanks consumer reports!
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
CR is good! CR is bad! no CR is good!
CR is good! CR is bad! no CR is good!
Hi Rabbit Coach, a bit off topic: but is 'Rabbit Coach' a sexual reference? Or am I just seeing sexual references where there are none?
If it is, it's very funny.
Best
I think I disagree with approximately 100% of this. As a response to the argument that CR only does what they do to drive web hits (when they collect no ad revenue) it is quite empty. What it seems to boil down to is the argument that if you disagree with CR's results, then their methods must be flawed, or they must be dishonest, or conducting bad science, or be attention whores, whatever -- any reasoning that does not even include the possibility that they do what they do in good faith, but you don't happen to like their conclusions. The truth is, whether you agree or disagree with their methods (and I have heard precious little actual, technically-based criticism from anyone), they have been doing exactly what they do in precisely the same way for over 60 years. They are a non-profit and have no allegiance to any manufacturer. They only survive by providing what they see as being objective advice to their members. If their members thought they were a fraud, then they would not have any members and would cease to exist.
So how about this: if you are going to charge CR with engaging in "pseudo-science" and being a "scandal and a fraud" (obviously very serious charges), how about supporting these allegations with some actual, objective evidence of either? I mean, if you're going to allege that their methods are "entirely subjective" then how about not supporting it with an argument which is entirely subjective? Too much to ask? I hope not.
Well, you misrepresent my argument entirely when you say, "the argument that CR only does what they do to drive web hits." First, that wasn't even my primary criticism, and secondly, there's nothing inconsistent with accusing them of going after web hits, which lead to an enlarged subscriber base, which may lead to increased individual salaries, and admitting that this may not be their only motivation. Do you really believe that the individuals who work at Consumer Reports do what they do entirely out of a sense of selfless altruism? Or, may it not be possible that some of them do it because they are paid well?
As to my primary criticism -- that their testing is not objective nor reliable -- I think the facts stand for themselves. There is really no argument around the fact that, based on their ratings, the iP4 is the best smartphone on the market. Yet, it is not one of their recommended phones. This means that their ratings are not based on the factors that cause them to recommend products. If not, then what are they based on? Unless their ratings and recommendations align, it is unavoidable to conclude that their recommendations are not objective and/or that their rating system is flawed. You can't escape one or the other, or both of those conclusions.
The fact that they have been doing this for 60 years is irrelevant. They may always have been unreliable, (A possibility, since I have almost never agreed with their recommendations on any product of which I have knowledge.) they may recently have become unreliable, or they may have always been sometimes unreliable.
But, they are either engaged in objective testing and rating of products , with meaningful tests and a meaningful rating system, or they are not. That the highest rated product is not recommended, indicates that their rating system and testing is in at least this case flawed. That it is in this case, casts doubt of their reliability and objectivity in all cases.
(And, allowing a video with snarky, unprofessional duct tape comments certainly doesn't help their case. In fact it reinforces the notion that they have allowed bias into their testing process.)
My comments on "scandal and fraud" were, however, intended mostly for the media. As we all know, the lust for sensationalism in that quarter is insatiable. And anything they can whip into the appearance of scandal and fraud is sauce for them. The question is, why isn't the inconsistency of Consumer Reports testing and recommendations coming under scrutiny here by that same media?
The comment on "pseudo-science" is, however, entirely apt, in regard to Consumer Reports methods.
If you say so. But there are other voices out there who say differently.
Yup, mainly people who don't own one of these things, people who blindly hate all things AAPL or the tiny minority of people who have actually experienced an issue. iPhone4 here - no issues, no dropped calles, great signal, no proximity sensor issues. Same with my partners and every friend I have who owns one...
I could have told Apple this antenna issue was going to happen, and I didn't have to go to college and become an engineer, or spend millions on a call testing facility.
Apple blundered big time on this one. And all the Kool Aid on the store shelf ain't gonna make it go away.
Ermmm - did they? Then how come (by far) the vast majority of these devices work without issue? Why the incredibly small return rate? Are every one of the 6 - 7 million (I estimate) current owners on the 'koolaid' - or could it be that (as Jobs stated) that this is a non-issue? I own the device, it works perfectly (without a bumper or case). So if it can work for me, it can't be fundamentally flawed, there must be other factors that affect a tiny percentage of units. And the learn the difference in the way frequency behaves - your old TV was not receiving using the same frequencies and bandwidth as a phone - your logic falls flat on it's ass very quickly.
CR is good! CR is bad! no CR is good!
If they want to maintain any credibility at all as a reliable testing organization, they need to either reconsider their statement, or reconsider their rating and testing protocols, or both. Fankly, at this point, their ratings and statements, regarding all smartphones, ought to be retracted. They either base their recommendations on valid, reliable, objective tests, or they do not. The evidence in front of us is that they do not.
CR is good! CR is bad! no CR is good!
They are neither all good nor all bad. They get some things right, and some things wrong, just like all human beings. I don't honestly know whether they've gotten the iPhone 4 testing thing right or wrong, or (more likely) somewhere in between. But as someone else said, their conclusions are virtually the same as Walt Mossberg's, and I notice that nobody is calling him a fraud.