poll taxing

What’s the matter with the polls?

If Election2008 brings us anything, it will be increased public perception about how damaging and mistaken surveys can be. Barack Obama is expected to win South Carolina by 12 points and wins by 28. Polls predict Mitt Romney will win Nevada by 5%; he wins by 35%. The latest polls for Florida, with a margin of error of 4%, variously have John McCain winning by 3 and Mitt Romney winning by 6.

What’s going on?

My mother raised me to be skeptical of statistics. The example she often used, tellingly, was that birth control pills raised your incidence of heart attack 400%. But as she pointed out, if you read between the lines the numbers were still minuscule: 5 women per 100,000 versus 1, something like that. (I was packed off to college with a year's supply.)

Most polls are extrapolations made from relatively small amounts of data. I love talking to customers, and you can learn something useful from talking to a handful of customers. The danger is when you take what you hear and treat it as representative of all your customers.

Time Magazine routinely polls fewer than 1000 Americans nationally and then reports that the top concern is the war in Iraq or the economy. The original poll, if it’s reputable, may have some nuance that’s usually left out of the reports picked up by TV news or radio stations. Or the language of the poll may force undecided voters to make a choice. There’s nothing worse than being asked which is more important, fear of recession or health care and not being able to choose: Both. Or None of the Above.

Polling methods are not good at projecting uncertainty, and they’re complicated in an election where regions have different priorities, and you don’t know who will actually show up. Preference does not always predict behavior.

Back to 2008...
Polls for this election have been based on very small numbers of likely voters. Most have fewer than 800 respondents. Many are limited to members of specific political parties, even as more Americans identify as independents. This leads to undercounting.

Polls also suffer from lags. Results released yesterday don't take into account Obama's huge win Saturday. This is a perfect example of why a longer primary season would benefit the country and the candidates. Instead of having to raise hundreds of millions of dollars for national advertising by appealing to special interests (Clinton, Guiliani, Obama) or being personally obscenely wealthy (Romney, Bloomberg, Edwards), candidates would have time to campaign on the issues.

In the four weeks since Iowa, there’s been a different front runner and winner in each state. The mainstream media tends to leap on the polls and results, focusing on the horse race, even though the next contest is a very different set of voters and issues. Michigan doesn’t have the same composition as New Hampshire or South Carolina. I’m sure someone in those states has the same priorities as I do. Well, maybe.

There are also unintended consequences in a race where the Democratic field is led by a very viable female candidate and a very viable African American candidate. As my cousin said of John Edwards, who we both favor, it’s a bad year to be the white guy.

Political analysts repeatedly miss the effect of attacks on Clinton and Obama. Sure, I’m ultimately going to vote based on the issues and which candidate I think is best prepared to be president given the difficult state of the country. But I do take it personally when Hillary Clinton is treated differently from the men she’s running against by the press and by other candidates. There’s no surer way to get women or African-Americans to keep coming to the polls in record numbers than to keep telling us that we—our votes—don’t matter.

This isn’t just an issue for well-to-do voters. Working class voters, especially women, who have suffered under the Bush Administration’s disastrous policies are turning out in record numbers. Perhaps they weren’t home when the pollsters called—they were at work, or picking up their kids at day care or at church. True, that's not as sexy an explanation as white voters in New Hampshire lying about voting for a black candidate. As South Carolina's Democratic primary demonstrates, African-American women are a constituency to be reckoned with.

Race matters. And gender. And age, but not in the way you think. For all we hear about the youth vote, voters under 30 comprise roughly 10% of the electorate. Two-thirds of all voters are 45 and up. My grandmother, who is 91, never misses a chance to choose who will represent her issues.

While a big Obama victory in South Carolina's primary is hardly a surprise, the Associated Press called the race before any votes were counted based on exit polls. Didn't they learn anything from 2000 or 2004? This is CNN:
The NY Times was slightly more cautious. "Obama Is Seen as Winner in South Carolina," but their infographics told the whole story:

Despite the excitement and high ratings for this race, the mainstream media seems determined to shut down debate as soon as possible. They stopped covering Mike Huckabee weeks ago, although he looks set to pick up delegates and possibly victories in Florida, Georgia, and Alabama. They don't allow Dennis Kucinich or Ron Paul to debate and keep encouraging John Edwards to drop out, even though he's raised enough money to stay in. How does any of this serve democracy or stimulate debate on the issues Americans care about? Especially those of us who live in states that don't vote in January, or haven't made up our minds?

Polls can be self-fulfilling. They tell a story you’re expecting, from people you expected to hear it from. It's like asking customers if they want a red car or a blue car, and when they say they want a silver car, ignoring them and checking the blue box, or throwing away their input.

Forecasting falls down:

• Where opinions are volatile and changing
• When you can’t tell how representative the people you’re talking to are
• When you’re missing a sizable segment by taking too small a sampling.

There is a great story that remains underreported, and that is one of turnout. Record numbers of voters continue to casts ballots; as much as twice the participation of the last presidential campaign. People want to be counted.

Never trust a pol[l]. Or just go read Dave Barry.