« Kerry, Bush Running Close in 3 PA Polls |
Main
| CBS News/New York Times Poll Has It Close to Even! »
September 17, 2004
Gallup Strikes Again!
Here are Bush's leads in the three national polls released before Gallup's current poll (no RV data available for DCorps and Harris; Pew and Harris matchups include Nader):
Democracy Corps, September 12-14 LVs: +1
Pew Research Center, September 11-14 RVs: tied
Harris Interactive: September 9-13 LVs: -1
Looks like a tie ball game, right? But according to the Gallup poll conducted September 13-15 and released today, Bush is up......13???
Let's just say I'm just a wee bit skeptical of this one. First, Gallup's poll only includes one day (the 15th) these three other polls do not, so it can't be Gallup's survey dates that explain the big Bush lead.
Second, this 13 point lead is an LV figure and, as I've repeatedly emphasized, Gallup's LV screening procedure produces completely untrustworthy measures of voter sentiment this far in advance of the election. Here is a summary of the case against Gallup's LV data:
Sampling likely voters is a technique Gallup developed to measure voter sentiment on the eve of an election and predict the outcome, not to track voter sentiment weeks and months before the actual election. There is simply no evidence, and no good reason to believe, that it works well for the latter purpose. In fact, the evidence and compelling arguments are on the other side: that the registered voters are the more reliable guage of voter sentiment during the course of the campaign.
Here’s why. Gallup decides who likely voters are based on 7 questions about their interest in voting, attention to the campaign and knowledge about how to vote (e.g., where their polling place is located). The interested/attentive/knowledgeable voters are designated “likely” and the rest are thrown out of the sample. But as a campaign progresses, the level of interest among voters tends to change, particularly among those with partisan inclinations whose interest level will rise when their party seems to be mobilized and doing well and fall when it is not. Because of this, partisans of the mobilized party (lately, Republicans) tend to be screened into the likely voter sample and partisans of the demobilized party (lately, Democrats) tend to get screened out. But tomorrow, of course, the Democrats could surge, in which case their partisans may be the ones over-represented in likely voter samples.
That suggests the uncomfortable possibility that observed changes in the sentiments of “likely voters” represent not actual changes in voter sentiment, but rather changes in the composition of likely voter samples as political enthusiasm waxes and wanes among the different parties’ supporters. And that is exactly what political scientists Robert Erikson, Costas Panagopoulos, and Christopher Wlezien find in their analysis of Gallup's 2000 RV/LV data in their forthcoming paper, “Likely (and Unlikely) Voters and the Assessment of Campaign Dynamics” in Public Opinion Quarterly: “shifts in voter classification as likely or unlikely account for more observed change in the preferences of likely voters than do actual changes in voters’ candidate preferences.”
That means that, instead of giving you a better picture of voter sentiment and how it is changing than conventional registered voter data, likely voter data give you a worse one since true changes in voter sentiment are swamped by changes in who is classified as a likely voter.
I think the case against the Gallup LV data looks rock solid. In my view, it's time for them to drop reporting these data because they are highly likely to give an inaccurate picture of the state of the race and, by doing so--especially given the high profile of Gallup's polls--unfairly pump up one side of the race and demoralize the other. That doesn't seem acceptable to me.
Of course they'll reply: well, our data work so well right before the election, they must be the best data to use all the time. But, for the reasons outlined above, that reasoning is completely specious. And then there's this: the LV data haven't been working so well lately even right before the actual election. In 3 of the last 4 presidential elections (including the last one), Gallup's final RV reading was actually closer to the final result than their final LV reading!
As I say, maybe it's time for a rethink down at Gallup HQ.
Throwing out the Gallup LV data, then, let's move on to their RV result: an 8 point Bush lead. Obviously pretty far off the results of the other contemporaneous polls summarized above, but....could be I suppose.
But then there's this: the Gallup internals show Kerry with a 7 point lead among independent RVs. Huh? Kerry's losing by 8 points overall, yet leading among indenpedents by 7. How is that possible? Only if there are substantially more Republicans than Democrats in the sample.
That suggests that reweighting the sample to reflect the 2000 exit poll distribution (39D/35R/26I) would give a different result. It does: the race then becomes dead-even, instead of an 8 point Bush lead. (Note: Steve Soto of The Left Coaster got Gallup to give him their party ID distributions for this poll and confirms a 5 point Republican party ID advantage in their RV sample.)
One final note: I mentioned the Pew Research Center poll had the race dead-even just like the reweighted Gallup data. And what was Pew's party ID distribution in their RV sample? You guessed it: a 4 point lead (37-33) for the Democrats, just like in the 2000 exits.
I think we've finally found out how to make these polls get along!
Posted by Ruy Teixeira at 07:22 PM | link
|