« Analysis of the Big Four |
Main
| Tracking the Tracking Polls: The Sequel »
October 29, 2004
RVs or LVs?
I've made no secret of my preference for RV over LV data during the course of this campaign, for reasons that have been outlined in detail in many posts.
But what about now? If LV data make any sense at all, it's now, on the eve of the election, when polls are no longer, in effect, screening voters for participation in some mythical "snap" election substantially preceding the real election, but are getting close to screening voters for the real election itself. This was, after all, the original purpose of LV screens and models and perhaps these screens and models still deserve our trust this close to the election.
I'm not so sure. Just because these models can now be employed in a way closer to their original purpose doesn't mean they are now particularly trustworthy. The same problems that have vexed LV samples throughout this election seem likely to persist, at least at some level, right up to election day. I have, of course, reviewed these problems and the spotty track record of LV predictions in detail, and readers are urged to go back and consult those posts if they wish. And for extra credit, I strongly urge you to consult Mark Blumenthal's (of Mystery Pollster) epic seven part series on likely voter methods (soon to be major motion picture, I understand). You really should read the whole thing, but here's the Classic Comics version:
1. Likely voter methodology is extremely complicated.
2. Everybody does it differently.
3. They don't really know for sure their methods work--especially during the bulk of the campaign season--and there are all kinds of ways in which likely voter methods can produce biased samples and wild swings in results that are unrelated to voter sentiment.
4. It's not clear that the likely voter models have superior accuracy, even at election time, in today's political environment.
Which leads me to the following query: why bother? If these likely voter methods don't work that well, are prone to fairly serious bias and volatility and everybody has a different approach so you can't easily compare different polls, why go to all the trouble of drawing these samples anyway? Maybe good old-fashioned, low-hassle RV samples are just as good--in fact, better.
That said, we're kind of stuck with LV samples in most polls today, so in many cases it's either LV data or no data. As a mitigating factor, many of these polls that just report LVs have comparatively simple and modest screens--a question or two--that produce samples not too far off from a straight RV sample. But for the complicated LV approaches that use an elaborate series of questions and scores to ferret out the allegedly likely voters (think Gallup, but others like WP/ABC use similar systems), I'd still be inclined to take their RV data (assuming they're still reporting it) a bit more seriously than their LV data. Or, if you want a compromise, average the two.
OK. Class dismissed. Back to obsessively following the latest polls...
Posted by Ruy Teixeira at 01:22 PM | link
|