AP - Polls apart: Why polls vary on presidential race

[CF DolFan]

I heard about this on the raido but it is the first time I've  seen it in writing.  This could be bad news for all of the media writing off John McCain.

http://ap.google.com/article/ALeqM5jJ4ZOTu5ts8dFaEjqQZkT_4i5xkwD940ERMG0


By ALAN FRAM – 21 hours ago

WASHINGTON (AP) — Barack Obama is galloping away with the presidential race. Or maybe he has a modest lead. Or maybe he and John McCain are neck and neck.

Confusing? Sure, thanks to the dueling results of recent major polls.

In the past week, most surveys have shown Democrat Obama with a significant national lead over Republican McCain. Focusing on "likely voters" — as many polling organizations prefer this close to Election Day — an ABC News-Washington Post survey showed Obama leading by 11 percentage points. A Wall Street Journal-NBC News poll had the same margin, while the nonpartisan Pew Research Center gave Obama a 14-point edge.

But others had the race much closer. CNN-Opinion Research detected an Obama lead of 5 points. The George Washington University Battleground Poll had Obama up by 4 points. And an Associated Press-GfK poll showed Obama at 44 percent and McCain at 43 percent — in effect, a tie.

How can this be? Some questions and answers about why the polls differ.

Q: Don't pollsters simply ask questions, tally the answers and report them?
A: No. After finishing their interviews — usually with about 1,000 people, sometimes more — they adjust the answers to make sure they reflect Census Bureau data on the population like gender, age, education and race. For example, if the proportion of women interviewed is smaller than their actual share of the country's population, their answers are given more "weight" to balance that out. But some pollsters make these adjustments differently than others. And while most polling organizations including the AP do not modify the responses to reflect some recent tally of how many Democrats, Republicans and independents there are, some do.

Q: Are those the only changes made?

A: No. As Election Day nears, polling organizations like to narrow their samples to people who say they are registered voters. They often narrow them further to those they consider likely voters. That's because in a country where barely more than half of eligible voters usually show up for presidential elections, pollsters want their polls to reflect the views of those likeliest to vote.

Q: Is that hard to do?

A: Quite hard, since no one will truly know who will vote on Election Day until that day is over. In fact, virtually every polling organization has its own way of determining who likely voters are.

Like many polling organizations, the AP asks several questions about how often people have voted in the past and how likely they are to vote this year, and those who score highest are considered likely voters.

Q: Why is this such a problem?

A: Because nobody is 100 percent sure how to do this properly. And the challenge is being compounded this year because many think Obama's candidacy could spark higher turnout than usual from certain voters, including young voters and minorities. The question pollsters face is whether, and how, to adjust their tests for likely voters to reflect this.

In identifying likely voters, the AP does not build in an assumption of higher turnout by blacks or young voters. Pew Director Andrew Kohut says that reflecting exceptionally heavy African-American turnout in the Democratic primaries, Pew's model of likely voters now shows blacks as 12 percent of voters, compared to 9 percent in 2004.

Underscoring the uncertainty, the Gallup Poll is using two versions of likely voters this year — a traditional one that asks about peoples' past voting behavior and their current voting intentions; and an expanded one that only looks at how intent they are on voting this year, which would tend to include more new voters.

Q: What else might cause differences?

A: The groups pollsters randomly choose to interview are bound to differ from each other, and sometimes do significantly.

Every poll has a margin of sampling error, usually around 3 percentage points for 1,000 people. That means the results of a poll of 1,000 people should fall within 3 points of the results you would expect had the pollster instead interviewed the entire population of the U.S. But — and this is important — the results are expected to be that accurate only 95 percent of the time. That means that one time in 20, pollsters expect to interview a group whose views are not that close to the overall population's views.

Q: Are the differences among polls this year that unusual?

A: Not wildly, but that doesn't make them less noticeable. There's a big difference between a race that's tied in the AP poll, and Pew's 14-point Obama lead. But because of each poll's margin of error, those differences may be a bit less — or more — than meet the eye.

That's because each poll's margin of sampling error should really be applied to the support for each candidate, not the gap between them.

Take the AP poll, which has a margin of error of plus or minus 3.5 percentage points. Obama's 44 percent support is likely between 48 percent and 40 percent. McCain's 43 percent is probably between 47 percent and 39 percent.

When support for candidates is measured in ranges like that, some polls' findings could overlap — or grow worse.

Q: Are people always willing to tell pollsters who they're supporting for president?

A: No, and that's another possible source of discrepancies. Some polling organizations gently prod people who initially say they're undecided for a presidential preference, others do it more vigorously. The AP's poll, for example, found 9 percent of likely voters were undecided, while the ABC-Post survey had 2 percent.

AP Director of Surveys Trevor Tompson and News Survey Specialist Dennis Junius contributed to this report.

[fyo]

Thanks for posting this CF. It answered most of the questions I raised in my "Swing states and polls" thread, http://www.thedolphinsmakemecry.com/forums/index.php?topic=12714.0

Much of it was clear already to anybody with a little statistics (like the +-X% and the 95%, technically termed a "confidence interval"), but it was great to get an answer on the turnout adjustments.

And while I've always realized that the way in which a pollster asks a question has a significant impact on the answer, I didn't know that there was a big difference in how much they pressure "somewhat undecided" people into giving an answer.

I really wish all polls would have their methodology posted, along with the actual results. The percentage of undecided voters, for example, is probably a good measure of how much pressure is applied to get an answer (although the percentage of "undecided voters" also has an error margin, just like "McCain" and "Obama" voters). It would also be extremely useful if they were upfront about what adjustments they've made - in addition to the basic population-matching weights that everyone does (and has to do).

I don't really care about the national polls. The outcome of the election isn't based on the national result, so why bother with a national poll aside from just a quick voter "temperature"?

It's the polls for swing states that are really interesting and I'd love to know what the assumptions / corrections were in those polls.

[YoFuggedaboutit]

If I recall correctly, Obama is ahead in nearly every swing state.

[MyGodWearsAHoodie]

Yahoo has a cool toy you can play with that breaks it down state by state

http://news.yahoo.com/election/2008/dashboard

[fyo]

Yahoo has a cool toy you can play with that breaks it down state by state

http://news.yahoo.com/election/2008/dashboard

Yeah, CNN has one as well. The thing is, there's a huge difference between a poll showing Obama ahead by 10 points where the pollsters are basing that on a huge minority + young people turnout and a poll showing a similar lead that makes no such assumption.

That was pretty much my whole point initially: These polls are pretty worthless if we don't know how to interpret them.