Wednesday, September 24, 2008

Landline "Bias"

As you probably know, surveys of all kinds (including surveys of people's politial attitudes) rely on a sample of respondents who are representative of American attitudes as a whole, and this sample is produced essentially by picking home phone numbers at random. This method has worked relatively well because the vast majority of homes in America have in the past owned home telephones, and so sampling bias could mostly be avoided. However, the rise of the cell phone has led to more and more people using cell phones as their primary means of telephonic communication, and more and more households (almost twenty percent) now do not have a phone in the home (my wife and I, by way of example, have two cell phones between us but haven't had a home phone in years.) Naturally, this leads one to question whether surveys conducted by contacting those who own home telephones are still that reliable. The Pew Research Center says yes they still are...for the most part:

The Pew Research Center for the People & the Press has conducted three major election surveys with both cell phone and landline samples since the conclusion of the primaries. In each of the surveys, there were only small, and not statistically significant, differences between presidential horserace estimates based on the combined interviews and estimates based on the landline surveys only. Yet a virtually identical pattern is seen across all three surveys: In each case, including cell phone interviews resulted in slightly more support for Obama and slightly less for McCain, a consistent difference of two-to-three points in the margin.

They go on to explain their conclusions, but essentially a sample bias appears as a result of the fact that younger voters who generally lean towards Obama, are also more likely to use their cell phone as their only means of telephonic communication. The bias is not statistically significant (in other words, the result falls in the polls margin of error each time) but including cell phone-only users consistently results in a 2-3 point bump for Obama. Of course, it would be foolish to conclude based on Pew's studies that Obama is actually carrying a 2-3 point "invisible" advantage over McCain in every poll you read about, but it does indicate that as more and more people abandon landlines for cell phones as their primary or only phone, surveyors will have to be more creative in conducting their polls or risk losing substantial accuracy.

No comments: