Gallup's New Math

[Towel]

This may only be new to me, but I was browsing electoral-votes.com and came across a really interesting story about how Gallup is bizzarely biasing all of their Presidential race polls. All polls apparently normalize their results to try to compensate for selection bias. One of the ways they do this is by party affiliation. Regardless of how many Democrats or Republicans they actually catch on the phone, they assume that they should have sampled a given % of each, and adjust their actual results to fit.

Gallup released two recent polls (http://gallup.com/poll/content/default.aspx?ci=13066) showing a huge lead opening up for GWB over Kerry - more than ten points. Strangely no other national polls show such a lead. Why is Gallup's result different?

Because they overestimate the number of Republican voters by 20%, and underestimate Democratic voters by the same margin.

Gallup assumes (http://www.theleftcoaster.com/archives/002806.html) that the electorate is 40% Rep and 33% Dem, and normalizes its results accordingly. Reality suggets those numbers are hogwash: in the last three Presidential elections, it's been roughly the reverse, 40% or so Dem to 33% or so Rep. There's zero indication from recent polling that things have dramatically changed this year - if they have at all, it's probably been slightly in the other direction. If Gallup had used those numbers to normalize their data, Kerry would almost certainly be leading in their polls.

So, to all those Bush-Cheney fan boys who've been lapping up Bush's "ten point bounce" in the Gallup polls...Bush's lead is as ephemeral, and as honest, as Enron's profits. I wouldn't trust any poll you see coming out of Gallup, including those done for Newsweek and Time.

[faust9]

Originally posted by Towel
This may only be new to me, but I was browsing electoral-votes.com and came across a really interesting story about how Gallup is bizzarely biasing all of their Presidential race polls. All polls apparently normalize their results to try to compensate for selection bias. One of the ways they do this is by party affiliation. Regardless of how many Democrats or Republicans they actually catch on the phone, they assume that they should have sampled a given % of each, and adjust their actual results to fit.

Gallup released two recent polls (http://gallup.com/poll/content/default.aspx?ci=13066) showing a huge lead opening up for GWB over Kerry - more than ten points. Strangely no other national polls show such a lead. Why is Gallup's result different?

Because they overestimate the number of Republican voters by 20%, and underestimate Democratic voters by the same margin.

Gallup assumes (http://www.theleftcoaster.com/archives/002806.html) that the electorate is 40% Rep and 33% Dem, and normalizes its results accordingly. Reality suggets those numbers are hogwash: in the last three Presidential elections, it's been roughly the reverse, 40% or so Dem to 33% or so Rep. There's zero indication from recent polling that things have dramatically changed this year - if they have at all, it's probably been slightly in the other direction. If Gallup had used those numbers to normalize their data, Kerry would almost certainly be leading in their polls.

So, to all those Bush-Cheney fan boys who've been lapping up Bush's "ten point bounce" in the Gallup polls...Bush's lead is as ephemeral, and as honest, as Enron's profits. I wouldn't trust any poll you see coming out of Gallup, including those done for Newsweek and Time.

Nothing new. Statistics is a black magic branch of mathematics to say the least. The processes involved in statistical analysis are straight forward and easy to learn/understand; however, the application is never straight forward and always difficult to understand. Most 'good' polls are adjusted and each polling agency uses a different adjustment procedure. Gallup's tend toward Bush this election cycle. Zogby's tends towards Kerry. If you look at an aggregate poll--one which finds a mean of multiple polls--you'll see Bush is still in the lead but only by the narrowest of margins. An incumbent pres going into an election cycle with slightly greater than 50% approval and slightly more than 50% of the polled electorate planning on voting for the incumbent does not bode well for the sitting pres...

[BRussell]

I'd like to hear more. It's hard for me to believe that the most respected polling company in the country would use an inappropriate weighting methodology.

[faust9]

Originally posted by BRussell
I'd like to hear more. It's hard for me to believe that the most respected polling company in the country would use an inappropriate weighting methodology.

To be statistically valid a poll should be weighted for socioeconomic-political factors. Basically polls are demographically weighted--and each polling agency use a proprietary method for weighting.

A little info about weighting.
http://www.findarticles.com/p/articles/mi_m0HDN/is_2000_Jan_24/ai_59022400

This is a really good site:
http://www.pollkatz.homestead.com/

Enjoy.

[BRussell]

Originally posted by faust9
To be statistically valid a poll should be weighted for socioeconomic-political factors. Basically polls are demographically weighted--and each polling agency use a proprietary method for weighting. Oh yeah I'm aware that demographic weighting is used. But the criticism here is that Gallup is using a weighting that inappropriately biases their polls towards Repubs. Why would they do that?

[MarcUK]

Surely if you stop and ask 1000 random people in the street who they are going to vote for and you get the result

55% Kerry
42% Bush
2% Nader
1% Other

Then that IS the fucking result?

[Scott]

Which street?


Anyway all statistics is based off of a model. If the model is wrong then so are the results. So ... some here think the model is wrong.

[BRussell]

Originally posted by MarcUK
Surely if you stop and ask 1000 random people in the street who they are going to vote for and you get the result

55% Kerry
42% Bush
2% Nader
1% Other

Then that IS the fucking result? No, it's not, because they don't trust completely random samples (even if they actually could get them, which they can't.)

For example, they're much more likely to get women to answer the phone than men. So apparently they always ask for the man, and only if there's no male at home, do they use the woman's responses. That introduces error - only women with no man at home are represented in the polls - but it probably has much less error than if these polls were 80% women respondents.

[Scott]

As my stat's prof' used to say, "Block what you can. Randomize what you can't."

[SDW2001]

Gallup is extremely credible. In fact, they're the most credible. All worthy polling firms use a variety of statistical manipulation. They have to. The one that's different, as far as I've heard, is Zogby. He does something different, but I don't remember exactly what.

Gallup has generally predicted elections within 2.8%. I wouldn't trash them based on your own gross misunderstanding.

Just so we're not all running around calling the Gallup poll a fluke, I assume we've all seen the CBS poll with Bush up 9. Add to that the CNN/USA poll with Bush up 14. That's three major polls with Bush up bigtime. Averages show Bush up by about 6 nationally.

http://www.realclearpolitics.com/bush_vs_kerry.html

[SDW2001]

Here's an even more damning link.

http://www.realclearpolitics.com/Presidential_04/chart3way.html

[Existence]

http://www.theleftcoaster.com/archives/002806.html

[faust9]

Originally posted by SDW2001
Here's an even more damning link.

http://www.realclearpolitics.com/Presidential_04/chart3way.html

that's a prime example of how statistics are misused. The scale is not zero based so small differences can be made to look like huge gaps.

[Towel]

Originally posted by SDW2001
Just so we're not all running around calling the Gallup poll a fluke, I assume we've all seen the CBS poll with Bush up 9. Add to that the CNN/USA poll with Bush up 14. That's three major polls with Bush up bigtime. [/url] The CNN/USA poll is the Gallup poll. Duh. Do you bother to read any links before commenting?

Not surprisingly, a glance at the CBS poll numbers shows similar Gallup Math - they normalize to 33% Rep, 30% Dem, 36% Ind. As for "realclearpolitics"... call me jaded, but I take with a big grain of salt a website that posts quotes from six prominent Conservative talking heads on its "About Us" page.

I'm George W. Bush, and I approve this website. (http://www.realclearpolitics.com/about.html)

[SDW2001]

Originally posted by Towel
The CNN/USA poll is the Gallup poll. Duh. Do you bother to read any links before commenting?

Not surprisingly, a glance at the CBS poll numbers shows similar Gallup Math - they normalize to 33% Rep, 30% Dem, 36% Ind. As for "realclearpolitics"... call me jaded, but I take with a big grain of salt a website that posts quotes from six prominent Conservative talking heads on its "About Us" page.

I'm George W. Bush, and I approve this website. (http://www.realclearpolitics.com/about.html)

My mistake on the CNN poll.

Kerry is still fucked

[bauman]

I've been following RasmussenReports (http://www.rasmussenreports.com) ... They seem to have well done surveys, with very standardized results. They poll daily, and have three day rolling averages. Not only that, but they emphasize the polls in the battleground states - where the numbers really do matter.

There will always be problems with any kind of poll. I haven't heard any third party comments on RR, but they seem to do a decent job.

[BRussell]

Here's a summary of current polls:

http://www.pollingreport.com/images/SEPgen3.GIF

Excluding Gallup, the Bush lead is:

3, 3, 2, 1, -1, 5, and 7 (the last two are the oldest).

Gallup has a 14-point lead for Bush. It sure does look fishy. Anything can happen once, but it will be interesting to see if Gallup continues its outlier status.

[groverat]

Poll nerdery is hilarious. :)