Advertisement
David W. Moore, who worked for Gallup for 13 years as managing editor and senior editor of the Gallup Poll, has a new book out denouncing most polls by Gallup or anyone else as useless, and explaining that this became obvious to him when he first began work at Gallup, raising the obvious question of why he stuck around for 13 years. The explanation seems to be that he was trying to fix the problem, and one of the motivations for the book seems to be that he believes he still can fix it.
Of all the endless multilayered criticisms I have of polls, Moore focuses obsessively on only one. In a 160-page book on such a huge topic, you'd expect more than one idea endlessly repeated, but you won't find any more than that in "The Opinion Makers: An Insider Exposes the Truth Behind the Polls." Moore's concern is a significant one. He objects to the fact that polls are conducted and reported as if ignorance and apathy do not exist. Moore shows that when pollsters ask people whether they know about a topic, or whether they care if Congress acts in accordance with their wishes, the results turn out very differently from the ordinary poll that simply asks for a policy position. With the method that acknowledges indecision, you end up with a huge percentage of Americans having no opinion or not really caring. If you poll in this way you discover, for example, that a majority of Americans did not want the invasion of Iraq, a majority of Americans simply tolerated it.
Moore thinks polling is done quickly and shoddily to suit the interests of media outlets. I think it is also done in many ways to suit the policy agendas of media corporations. Moore recounts the story of pollster Louis Harris, whose company is now called Harris Interactive, who openly announced that he would conduct and report on polls favorable to Ted Kennedy's challenge to Jimmy Carter. But Moore takes no notice of the equally obvious agendas of most pollsters today. And Moore makes no mention of the single biggest problem with pollsters, namely that they do not poll at all on topics that are not approved by media corporations. Here's a history of polling on the impeachment of Bush and Cheney. Harris was among the companies that poll for money but refused to poll on this topic even if paid. While the few pollsters that did polls found high support, others declared that they would not poll on something that was not in the news.
But, of course, media outlets poll on things all the time precisely in order to report on the polls and create news. It's hard to say which is a bigger problem, the way polls are conducted or the way the results are reported. Moore discusses the shortcomings of various polls on the topic of "defense cuts" completely oblivious to the fact that he's consistently labeling military cuts as something they are not and something few people would be eager to support: "defense cuts." He discusses polling on what "the Democrats" and "the Republicans" have done in Congress, never noticing that these polls are thereby asking citizens about the actions of parties rather than the actions of elected representatives, even though party members rarely act as a complete block and damage is done to our democratic republic when we're taught that the only actors in it are parties. But Moore gives many examples of how polls are misleadingly worded, generating results of very little worth.
Moore also gives examples of Gallup and others misleadingly reporting on their results. Here's an example I'll contribute: In 2006, Newsweek reported the shocking news that a majority of Americans favored impeaching the president. But read how Newsweek reported this:
"Other parts of a potential Democratic agenda receive less support, especially calls to impeach Bush: 47 percent of Democrats say that should be a “top priority,” but only 28 percent of all Americans say it should be, 23 percent say it should be a lower priority and nearly half, 44 percent, say it should not be done. (Five percent of Republicans say it should be a top priority and 15 percent of Republicans say it should be a lower priority; 78 percent oppose impeachment.)"
Then there's the over-reporting of polls, especially election polls, at the expense of better news. Then there's the use of pollsters as supposedly expert pundits. I watched CNN's polling guru declare impeachment unpopular without ever having conducted (or at least reported on) any poll. And then there's the use of polls to reflect bad reporting. As Moore notes, when media outlets report something over and over again, they are almost always able to then poll on it and discover that a lot of the public agrees with what it's been hearing.
Moore suggests that media outlets and polling companies should not be allowed to be one and the same, that media outlets might report more honestly on all polls if their aim wasn't to promote their own, and that polling companies might poll better if not purely trying to please media outlets. While we, at least half-heartedly and corruptly, fund public media, we do not fund public polling. I think we should.
Moore devotes a lot of space to criticizing electoral polling, including early and pointless primary polling. And he suggests that only polling immediately before an election or exit polling can predict an outcome. Yet, Moore steers clear of admitting exactly how extremely accurate such polling tends to be; and when it conflicts -- even shockingly -- with official results, this pollster joins the rest of them in happily tossing his profession in the trash in order to accept the official vote counts. Moore finds the official results of the Obama-Clinton primary in New Hampshire unbelievable because of the polling, never mentioning other suspicious features, but immediately believes what he's told. He gives the 2004 general election exit polls the same treatment without argument or explanation.
What light does Moore's analysis shed on current presidential polls? Well, I imagine that he would tell us that the fewer days remaining before the election, the more the polls should be taken seriously, and that we would have a much better idea of how uncertain the polls are now if the pollsters were asking and reporting on how many people are undecided as to whom they will vote for on November 4th, rather than asking and reporting on whom people would vote for if the election were today. I don't know to what extent Moore would agree with my impression that many of the current presidential polls are slanted in favor of Republicans (by polling equal numbers of Republicans and Democrats, even though there are many more Democrats in the country, and by not polling new voters or cell-phone users).
But I'm pretty sure Moore would agree with criticisms of how the media is reporting on the polls. As Dailykos and other bloggers pointed out on Sunday, the Washington Post is labeling any state in which McCain is ahead by even 2 percent as "leaning Republican," but states Obama is ahead in even by 14 percent as "battlegrounds." Many pollsters are currently forecasting a decisive electoral-college victory for Obama, but the big corporate media outlets are not.
If the polling were better and the reporting on it better, election theft would be far more difficult. If the polling on policy positions were better done and reported, the agenda for the next government in Washington would come more in line with the desires of Americans. So, by all means, let's fix polling and make it work for us, not for Rupert Murdoch.
Of all the endless multilayered criticisms I have of polls, Moore focuses obsessively on only one. In a 160-page book on such a huge topic, you'd expect more than one idea endlessly repeated, but you won't find any more than that in "The Opinion Makers: An Insider Exposes the Truth Behind the Polls." Moore's concern is a significant one. He objects to the fact that polls are conducted and reported as if ignorance and apathy do not exist. Moore shows that when pollsters ask people whether they know about a topic, or whether they care if Congress acts in accordance with their wishes, the results turn out very differently from the ordinary poll that simply asks for a policy position. With the method that acknowledges indecision, you end up with a huge percentage of Americans having no opinion or not really caring. If you poll in this way you discover, for example, that a majority of Americans did not want the invasion of Iraq, a majority of Americans simply tolerated it.
Moore thinks polling is done quickly and shoddily to suit the interests of media outlets. I think it is also done in many ways to suit the policy agendas of media corporations. Moore recounts the story of pollster Louis Harris, whose company is now called Harris Interactive, who openly announced that he would conduct and report on polls favorable to Ted Kennedy's challenge to Jimmy Carter. But Moore takes no notice of the equally obvious agendas of most pollsters today. And Moore makes no mention of the single biggest problem with pollsters, namely that they do not poll at all on topics that are not approved by media corporations. Here's a history of polling on the impeachment of Bush and Cheney. Harris was among the companies that poll for money but refused to poll on this topic even if paid. While the few pollsters that did polls found high support, others declared that they would not poll on something that was not in the news.
But, of course, media outlets poll on things all the time precisely in order to report on the polls and create news. It's hard to say which is a bigger problem, the way polls are conducted or the way the results are reported. Moore discusses the shortcomings of various polls on the topic of "defense cuts" completely oblivious to the fact that he's consistently labeling military cuts as something they are not and something few people would be eager to support: "defense cuts." He discusses polling on what "the Democrats" and "the Republicans" have done in Congress, never noticing that these polls are thereby asking citizens about the actions of parties rather than the actions of elected representatives, even though party members rarely act as a complete block and damage is done to our democratic republic when we're taught that the only actors in it are parties. But Moore gives many examples of how polls are misleadingly worded, generating results of very little worth.
Moore also gives examples of Gallup and others misleadingly reporting on their results. Here's an example I'll contribute: In 2006, Newsweek reported the shocking news that a majority of Americans favored impeaching the president. But read how Newsweek reported this:
"Other parts of a potential Democratic agenda receive less support, especially calls to impeach Bush: 47 percent of Democrats say that should be a “top priority,” but only 28 percent of all Americans say it should be, 23 percent say it should be a lower priority and nearly half, 44 percent, say it should not be done. (Five percent of Republicans say it should be a top priority and 15 percent of Republicans say it should be a lower priority; 78 percent oppose impeachment.)"
Then there's the over-reporting of polls, especially election polls, at the expense of better news. Then there's the use of pollsters as supposedly expert pundits. I watched CNN's polling guru declare impeachment unpopular without ever having conducted (or at least reported on) any poll. And then there's the use of polls to reflect bad reporting. As Moore notes, when media outlets report something over and over again, they are almost always able to then poll on it and discover that a lot of the public agrees with what it's been hearing.
Moore suggests that media outlets and polling companies should not be allowed to be one and the same, that media outlets might report more honestly on all polls if their aim wasn't to promote their own, and that polling companies might poll better if not purely trying to please media outlets. While we, at least half-heartedly and corruptly, fund public media, we do not fund public polling. I think we should.
Moore devotes a lot of space to criticizing electoral polling, including early and pointless primary polling. And he suggests that only polling immediately before an election or exit polling can predict an outcome. Yet, Moore steers clear of admitting exactly how extremely accurate such polling tends to be; and when it conflicts -- even shockingly -- with official results, this pollster joins the rest of them in happily tossing his profession in the trash in order to accept the official vote counts. Moore finds the official results of the Obama-Clinton primary in New Hampshire unbelievable because of the polling, never mentioning other suspicious features, but immediately believes what he's told. He gives the 2004 general election exit polls the same treatment without argument or explanation.
What light does Moore's analysis shed on current presidential polls? Well, I imagine that he would tell us that the fewer days remaining before the election, the more the polls should be taken seriously, and that we would have a much better idea of how uncertain the polls are now if the pollsters were asking and reporting on how many people are undecided as to whom they will vote for on November 4th, rather than asking and reporting on whom people would vote for if the election were today. I don't know to what extent Moore would agree with my impression that many of the current presidential polls are slanted in favor of Republicans (by polling equal numbers of Republicans and Democrats, even though there are many more Democrats in the country, and by not polling new voters or cell-phone users).
But I'm pretty sure Moore would agree with criticisms of how the media is reporting on the polls. As Dailykos and other bloggers pointed out on Sunday, the Washington Post is labeling any state in which McCain is ahead by even 2 percent as "leaning Republican," but states Obama is ahead in even by 14 percent as "battlegrounds." Many pollsters are currently forecasting a decisive electoral-college victory for Obama, but the big corporate media outlets are not.
If the polling were better and the reporting on it better, election theft would be far more difficult. If the polling on policy positions were better done and reported, the agenda for the next government in Washington would come more in line with the desires of Americans. So, by all means, let's fix polling and make it work for us, not for Rupert Murdoch.