Mark Blumenthal of Mystery Pollster plunges into the latest controversy around partisan distribution in polls and their apparent relation to outlier results, like Gallup’s 57 percent approval rating for Bush in their February 4-6 poll. In his post, “On Outliers and Party ID”, Blumenthal marches, in an admirably-organized way, through the various polling issues raised by this problem. Here’s an excerpt from his post, but, by all means, read the whole thing. It’s well worth the effort.

Unlike pure demographic items like age and gender, party ID is an attitude which can change especially from year to year....The problem is that partisan composition of any sample can also vary randomly -- outliers do happen. Unfortunately, when they do we get news stories about "trends" that are really nothing more than statistical noise....

The conflict leads to some third-way approaches that some have dubbed "dynamic weighting." I discussed these back in October. The simplest and least arbitrary method is for survey organizations to weight their polls by the average result for party identification on recent surveys conducted by that organization -- perhaps over the previous three to six months. The evolving party identification target from the larger combined sample would smooth out random variation while allowing for gradual long-term change (see also Prof. Alan Reifman's web page for more commentary on this issue).

....The party ID numbers ought to be a standard part of the public release of any survey, along with cross-tabulations of key results by party identification. Gallup should be commended for releasing data on request even to critics like [Steve] Soto, but it really should not require a special request.

Also, when a survey shows a sharp shift in party identification, news coverage of that survey should at least note the change -- something sorely lacking in the stories on CNN and in USAToday about the [Gallup] February 4-6 survey.