Are Polls Always “Skewed” Against Republicans?

In my widely-discussed recent paper on Fox News, I suggested that its Republican bias was hurting the GOP in many ways. One specific example I raised on CNN’s “Reliable Sources” was that Republicans deluded themselves that Mitt Romney was doing better than he really was in 2012 due to cheerleading coverage on Fox, the only news source for many Republicans. Another reason is that many Republicans simply refused to believe the polls that consistently showed Romney well behind Barack Obama, believing instead a site called  “Unskewed Polls,” which adjusted national polls to show Romney ahead. While I am not aware that Fox News promoted Unskewed Polls, it is known that former George W. Bush adviser Karl Rove, appearing on Fox News on Election Night, argued against Fox’s own call in favor of Romney in Ohio, suggesting that he, at least, had deluded himself about Romney’s strength.  

~~~

During the 2012 election, conservatives had an ongoing battle with mainstream pollsters, who consistently showed Democrat Barack Obama winning and Republican Mitt Romney losing. Many concluded there was some kind of conspiracy afoot to discourage Republican voters, thus making Obama’s poll numbers a self-fulfilling prophecy.

Before discussing this issue, I need to explain something technical about the nature of polling, especially political polling. Everyone knows that a national poll contains a sample of only a small fraction of the population. If the sample is truly random, theory and experience show that these samples can accurately represent the views of the population as a whole.

When doing political polling, however, it’s important that the sample be weighted in certain ways to reflect various population characteristics known to affect voting. In particular, you want a sample that properly reflects the party affiliations of the relevant population. Thus if a given sample has too few Democrats, the responses of those Democrats need to be given a greater weight so that their responses mirror those of the relevant population. The purpose is to get as accurate a projection of voting intentions as possible.

Another technical problem is determining whether someone in the polling sample will in fact vote or not. In recent years, Republicans have tended to do a better job of turning out their base than the Democrats, so turnout is a critical factor in predicting elections. All pollsters use various methods to determine whether a potential voter is a likely voter. One common method is simply to ask whether he or she voted in the last election. This weeds out perpetual nonvoters.

The important thing is that partisan weighting of a sample and determining those voters most likely to vote are, to a large extent, art rather than science. Every pollster does it slightly differently, which is a major reason why polls differ.

In September 2012, conservatives first raised the issue of systematic bias toward Obama in the methodology of mainstream pollsters, such as those under contract to major newspapers and television networks. Republican pollster John McLaughlin charged that Democrats were “working the refs,” so to speak, to encourage pollsters to weight their samples more in their favor, which would tend to improve Obama’s standing. This might help him in the election by creating a bandwagon effect in which positive polls became self-fulfilling.[1] Said McLaughlin:

“The Democrats want to convince [these anti-Obama voters] falsely that Romney will lose to discourage them from voting. So they lobby the pollsters to weight their surveys to emulate the 2008 Democrat-heavy models. They are lobbying them now to affect early voting. IVR [Interactive Voice Response] polls are heavily weighted. You can weight to whatever result you want. Some polls have included sizable segments of voters who say they are “not enthusiastic” to vote or non-voters to dilute Republicans. Major pollsters have samples with Republican affiliation in the 20 to 30 percent range, at such low levels not seen since the 1960s in states like Virginia, Florida, North Carolina and which then place Obama ahead. The intended effect is to suppress Republican turnout through media polling bias. We’ll see a lot more of this. Then there’s the debate between calling off a random-digit dial of phone exchanges vs. a known sample of actual registered voters. Most polls favoring Obama are random and not off the actual voter list. That’s too expensive [for some pollsters].”[2]

The McLaughlin critique was soon echoed by Republicans in and out of the media. In response, Lee Miringhoff of the Marist Poll said, “Why would pollsters want to look inaccurate?” He added that the focus on partisan weighting was too narrow. “It’s an easy target in a sense because you can look at the last [election], see the difference and jump on board,” Miringhoff said. He said that his company did not weight its samples at all.[3]

Conservative journalist Jim Geraghty saw evidence of “confirmation bias” in mainstream polls. Since Romney appeared to be running a poor campaign, it was therefore natural that he should also be doing poorly in the polls. As Geraghty put it:

“What we see in the most disproportionate poll samples is confirmation bias. To many people covering this race, Romney should be trailing badly, Republicans are flailing desperately, and Obama is running an exponentially better campaign. Thus, it makes perfect sense for the electorate to be even more heavily Democrat than it was in 2008.”[4]

It appears that Jay Cost of the conservative Weekly Standard magazine was the first hit on the idea of adjusting national polls to weight their samples more in favor of Republicans and less in favor of Democrats. From this he determined that Mitt Romney was in fact winning the election.[5] Radio talk show host Rush Limbaugh quickly endorsed this methodology, claiming that biased mainstream news organizations were tilting their polls to Obama. Said Limbaugh:

“They’re trying to wrap this up before the debates even start, because I think they’re worried about the debates. I think they’re trying to get this election finished and in the can by suppressing your vote and depressing you so that you just don’t think there’s any reason to vote, that it’s hopeless. They want you making other plans. We are told that according to the latest New York Times/Quinnipiac/CBS News poll, Obama is leading Romney by nine points in Florida, by ten points in Ohio, and by 12 points in Pennsylvania.”[6]

In late September 2012, the Democratic “Daily Kos” website commissioned Public Policy Polling (PPP), a reputable polling firm,[7] to ask this question in its regular national poll: “Do you think pollsters are intentionally skewing their polls this year to help Barack Obama?” The response from Republicans and conservatives was an overwhelming “yes.” But the idea that the polls were being skewed was widespread among just about all groups except liberals.

Table 1

Do you think pollsters are intentionally skewing their polls this year

to help Barack Obama, or not?

 (percent)

Note: Other characteristics omitted.

Source: Public Policy Polling[8]

A man named Dean Chambers, with no known expertise in polling or statistics, established a website called Unskewed Polls to regularly adjust national polls to increase the weight given to Republicans and reduce that for Democrats, asserting this was more accurate.[9] On November 5, 2012, Chambers predicted that Mitt Romney would win 51 percent of the popular vote and 275 electoral votes.[10]

Unskewed Polls quickly became must-reading for Republicans. “Always nice to get unfiltered, or in this case ‘unskewed,’ information,” Texas Governor Rick Perry tweeted in late September. Reputable pollsters were incredulous that Republicans were so willing to believe made-up numbers just because it suited their partisan inclinations. “There’s now an unwillingness to accept reality,” said Andrew Kohut, former head of the Gallup Poll.[11]

It appears that the Romney campaign itself bought the idea that just about every poll was biased against them and based its strategy on that belief in the final weeks of the campaign, according to reporter Jan Crawford of CBS News. As she wrote on November 8, 2012:

“They made three key miscalculations, in part because this race bucked historical trends:

“1. They misread turnout. They expected it to be between 2004 and 2008 levels, with a plus-2 or plus-3 Democratic electorate, instead of plus-7 as it was in 2008. Their assumptions were wrong on both sides: The president’s base turned out and Romney’s did not. More African-Americans voted in Ohio, Virginia, North Carolina and Florida than in 2008. And fewer Republicans did: Romney got just over 2 million fewer votes than John McCain.

“2. Independents. State polls showed Romney winning big among independents. Historically, any candidate polling that well among independents wins. But as it turned out, many of those independents were former Republicans who now self-identify as independents. The state polls weren’t oversampling Democrats and undersampling Republicans – there just weren’t as many Republicans this time because they were calling themselves independents.

“3. Undecided voters. The perception is they always break for the challenger, since people know the incumbent and would have decided already if they were backing him. Romney was counting on that trend to continue. Instead, exit polls show Mr. Obama won among people who made up their minds on Election Day and in the few days before the election. So maybe Romney, after running for six years, was in the same position as the incumbent.”[12]

It appears that many Republican pundits made the same mistake. In their final election predictions, Dick Morris of Fox News gave Romney 325 electoral votes to 213 for Obama, Washington Post columnist George Will gave Romney 321 electoral votes to 217 for Obama, Michael Barone of the Washington Examiner gave Romney 315 electoral votes to 223 for Obama, and Karl Rove gave Romney 285 electoral votes to 253 to Obama.[13]

The final results gave Obama 332 electoral votes to 206 for Romney. Obama received 51.01 percent of the popular vote to 47.15 percent for Romney.

Morris later explained that he thought it was his duty as a Romney supporter to give confidence to the campaign and to Republican voters that they were going to win, regardless of what he really knew polls said. Speaking to Fox’s Sean Hannity on November 12, 2012, Morris said:

“I spoke about what I believed, and I think that there was a period of time when the Romney campaign was falling apart, people were not optimistic, nobody thought there was a chance of victory. And I felt that it was my duty at that point to go out and say what I said. And at the time that I said it, I believe I was right. I have no apologies for what I said when I said it.”[14]

In short, Morris’s defense against lying about the polls is that he is incompetent.

Dean Chambers’ excuse for his gross error was, basically, that Obama stole the election. Chambers said that Internal Revenue Service harassment of Tea Party groups had shut down their operations in crucial states. Without such harassment, he said, Romney would have won exactly as predicted.[15]

Independent analysts said that the over-optimism extended far beyond crackpots like Chambers, who arbitrarily adjusted his polls based on nothing, to the entire universe of Republican pollsters, including those for the Romney campaign. In the end, it turned out that the polls were in fact biased in favor of the Republicans in Senate races in 2012.[16]

“On the Republican side, this was the worst cycle ever for polling and there’s nothing that even comes close to it,” said GOP strategist Curt Anderson, “It was a colossal disaster and it wasn’t confined to the presidential campaign.”[17]

According to the National Journal, the failure of Republican polling was wide and deep, affecting races at every level:

“Now we know which side needed its polls unskewed. Before Election Day, Republicans confidently predicted they would pick up seats in both chambers of Congress, and that Mitt Romney would win the White House. The results shattered those predictions, and with them any sense of security in the numbers coming out of some of the best-regarded polling firms on the right.

“’Everyone thought the election was going to be close. How did [Republicans] not know we were going to get our ass kicked?’ lamented Rob Jesmer, head of the National Republican Senatorial Committee. ‘I don’t understand how we didn’t know. That’s the part that’s most puzzling and frustrating and embarrassing.’”[18]

In the years since 2012, it does not appear that Republican polling has gotten better. In 2014, John McLaughlin, the pollster who set off the “unskewed” phenomenon in the first place, was polling for House Majority Leader Eric Cantor in his primary race against a long-shot Tea Party candidate. On June 6, the Washington Post reported that McLaughlin had Cantor safely ahead, 62 percent to 28 percent. A poll by the right-wing Daily Caller had Cantor ahead 52 percent to 41 percent.[19]

But on primary election night, the long-shot, David Brat, won easily, beating Cantor 55 percent to 44 percent.[20] McLaughlin had a laundry list of excuses for his failure, but a review of other campaigns he was involved with show that it was not an aberration.[21] Nevertheless, according to his website, McLaughlin is still busy working for corporate clients, who are often less interested in accuracy than generating support for their lobbying agenda. For example, in September 2014, McLaughlin was quoted in The Hill, a newspaper oriented toward Capitol Hill, saying that his polling showed that two-thirds of voters favored a freeze on government regulations.[22] The poll was commissioned by a group called Regulation Watch.

I could find no information about who runs Regulation Watch or who funds it. More than likely, it’s a front group for corporations in heavily-regulated industries – what’s known as an “AstroTurf” group because it pretends to be a grassroots organization, but with no real citizen-activists involved.[23] If one of the companies had commissioned the poll itself and released it under its own name, it would be too obvious that the results were self-serving. Setting up a front group made to look like a broad-based citizens group may fool the gullible into thinking there actually is mass support for the company’s agenda. This sort of thing is standard practice today in the public relations and lobbying businesses.

It should be added that since the company isn’t making corporate decisions based on the results of McLaughlin’s poll, it really doesn’t matter in the slightest whether it is accurate or not. All that matters is that the results support the client’s agenda. This can be done easily by carefully wording poll questions and other techniques, even if the poll is otherwise conducted by standard practices. But the internals of such proprietary polls are seldom made public so that experts can locate the bias. You just have to trust the pollster’s reputation.

However, it’s doubtful that many of those reading the results of this poll knew that McLaughlin’s reputation at that point was very poor.[24] On the other hand, this very fact may make him an attractive pollster for corporate clients only looking to create the false impression of broad support for an issue affecting their bottom line. A less-than-reputable pollster is probably more willing to give it the results it wants than one with a good reputation.

There are, of course real problems with public opinion polling. The response rate is dropping and it’s getting harder and harder to reach people by phone.[25] But there’s no reason to think such problems are biased for or against either party.

_________

NOTES

[1] David Rothschild and Neil Malhotra, “Are Public Opinion Polls Self-Fulfilling Prophecies?” Research and Politics, 1:2 (July-September 2014): 1-10. In 2014, there was in fact a polling bias in favor of Democrats, but it didn’t do them any good; they still lost big. See Nate Silver, “The Polls Were Skewed Toward Democrats,” FiveThirtyEight (November 5, 2014); Josh Katz, “What the Forecasts Got Right, and Wrong,” The Upshot, New York Times (November 5, 2014). Ironically, in the weeks before the election, Democrats complained that the polls were biased against them. Niall Stanage, “Dems: Don’t Trust the Polls,” The Hill (October 13, 2014).

[2] Quoted in Jim Geraghty, “What John McLaughlin Sees in the Polls Right Now,” Campaign Spot blog, National Review (September 21, 2012).

[3] Quoted in Steven Shepard, “Republicans to Pollsters: Too Many Democrats in Your Surveys,” National Journal (September 25, 2012).

[4] Jim Geraghty, “Why Party ID Matters, and Why Pollsters Avert Their Eyes,” Campaign Spot blog, National Review (September 25, 2012).

[5] Jay Cost, “Are the Polls Tilted Toward Obama?” Weekly Standard (September 26, 2012).

[6] Rush Limbaugh, “Don’t Let Bogus Polls Depress You,” Rush Limbaugh Show (September 26, 2012)

[7] It is worth mentioning that PPP was the most accurate pollster in the 2012 election cycle; Jonathan Easley, “Study Finds PPP Was the Most Accurate Pollster in 2012,” The Hill (November 7, 2012). Moreover, although it primarily polls for Democrats, it actually had a slight pro-Romney bias in 2012; Nate Silver, “Which Polls Fared Best (and Worst) in the 2012 Presidential Race,” FiveThirtyEight, New York Times (November 10, 2012).

[8] “Daily Kos/SEIU Weekly State of the Nation Poll” Daily Kos (September 27-30, 2012).

[9]http://unskewedpolls.com/. It appears that this site is no longer operational. Archives can be found here: https://web.archive.org/web/20141218065720/http://unskewedpolls.com/

[10] Dean Chambers, “Mitt Romney Will Win the Presidency with 275 Electoral Votes,” Washington Examiner (November 5, 2012).

[11] Jonathan Martin and Alexander Burns, “The Parallel Universe Where Mitt Romney Leads All Polls,” Politico (October 1, 2012).

[12] Jan Crawford, “Adviser: Romney ‘Shellshocked’ By Loss, “ CBS News (November 8, 2012).

[13] Brad Plumer, “Pundit Accountability: The Official 2012 Election Prediction Thread,” Wonkblog, Washington Post (November 5, 2012).

[14] Meenal Vamburkar, “Dick Morris Explains Romney Landslide Prediction: ‘I Felt It Was My Duty’ To Say What I Said,” Mediaite (November 13, 2012).

[15] Dean Chambers, “Analysis: IRS Political Suppression Cost Romney the 2012 Election,” Washington Examiner (May 22, 2013).

[16] Nate Silver, “The Polls Might Be Skewed Against Democrats – Or in Their Favor,” FiveThirtyEight (October 15, 2014).

[17] Alexander Burns, “The GOP Polling Debacle,” Politico (November 11, 2012).

[18] Reid Wilson, “GOP Grapples With Embarrassing Polling Failures,” National Journal (November 13, 2012).

[19] Sean Sullivan, “Cantor Internal Poll Claims 34-point Leader Over Primary Opponent Brat,” Washington Post (June 6, 2014).

[20] Jake Sherman, “Cantor Loses,” Politico (June 10, 2014).

[21] Shane Goldmacher, “Eric Cantor’s Pollster Tries to Explain Why His Survey Showed Cantor Up 34 Points,” National Journal (June 11, 2014).

[22] Benjamin Goad, “GOP Bets on Regulations Backlash,” The Hill (September 28, 2014).

[23] C. Walters, “How to Identify Astroturfers and Front Groups,” Consumerist (September 1, 2009); Anne Landman, “Attack of the Living Front Groups: PR Watch Offers Help to Unmask Corporate Tricksters,” PR Watch (August 28, 2009).

[24] Cameron Joseph, “National GOP: Don’t Use Cantor’s Pollster,” The Hill (June 13, 2014).

[25] “Assessing the Representativeness of Public Opinion Surveys,” Pew Research (May 15, 2012).