After stunning election upset, Pa. pollsters take stock of methods, misses

Last week, Pennsylvania voters of both parties watched in surprise — and, in some cases, shock — as Donald Trump won the commonwealth and was elected the next president of the United States.

They weren’t the only ones. Lots of political analysts in the state were also taken aback.

Most major polls never showed Trump winning Pennsylvania, let alone the election. The state was thought to be part of opponent Hillary Clinton’s “blue wall.”

So what went wrong?

State pollsters are still trying to figure that out for themselves; meanwhile, they have to answer to a disgruntled public.

Muhlenberg College poll director Chris Borick said he’s been on the receiving end of some angry emails lately. That happens after every presidential election to some degree, he said, but this year it’s been worse.

“People are upset and are angry either that the polls gave them confidence that was unwarranted, or that they thought the polls were biased because of some kind of nefarious reason,” he said.

Terry Madonna, director of the Franklin & Marshall College poll, said his past week has been similar.

Madonna and Borick have been in the Pennsylvania polling business for years, and they provide political analysis and commentary on the side.

F&M and Muhlenberg are well-rated surveys in that they tend to be accurate. But both pollsters say, this year, they’ve never been further off. Pollsters across the country are in the same exact boat.

On a fundamental level, this year was different, Borick said, in that pollsters underestimated the enthusiasm in Trump’s base. Borick said white, working-class voters in Pennsylvania turned out in almost double the numbers they did for Mitt Romney in 2012.

“I was truly skeptical about the idea of perhaps a silent Trump vote — that people were not telling pollsters, including us, that they were going to vote for Trump,” he said.

Trump called it

Throughout the campaign, Trump himself repeatedly insisted his large, raucous rallies were a sign of a populist movement.

“I hear we set a new record for this building — and, by the way I didn’t have to bring JLo or Jay Z, the only way [Clinton] gets anybody — I’m here all by myself,” he said at one Pennsylvania rally late in his campaign.

But Madonna said he discounted most of that.

“Trump was saying, ‘Look at the size of the crowds, look at the yard signs!'” he said. “And I would kind of joke and say,’Yeah, there’s hundreds of yard signs, but yard signs don’t vote.'”

Now, of course, he’s rethinking that approach.

“After the election, I also joked, maybe yard signs did vote,” he said. “You know, we didn’t pick up that.”

Trump ultimately won the state by 1 percent of the vote — that’s not enormously far off from the state poll average, which had Clinton up about two points right before the election.

Still, it’s a discrepancy.

Madonna said a big reason he and others were off was due to “non-response bias.” Basically, those silent Trump voters Borick was so skeptical of? They were real.

In F&M’s case, one problem was methodology. This year, Madonna was experimenting with new ways of reaching voters.Traditionally, polls are conducted via landline. But now, Madonna said, more and more people have cell phones. Some, particularly young people, just don’t answer polling calls. So he tried his luck at reaching them in other ways.

“Telephone interviewing in five to 10 years will probably be extinct — at least that’s what some pollsters think,” he said. “So this year we included online polls, so about 40 percent of the people we interviewed conducted the poll online.”

Madonna said he still thinks it was a valid idea. But it did throw off results.

“There is no doubt that that skewed the poll toward the Democrats,” he said/ “When I released it, I pointed that out to everybody so they would know that.

“I’m not making an excuse,” he added. “I’m just saying that we went into this fully aware that there had been problems developing over the years.”

Borick said there was also another, perhaps trickier problem — pollsters and analysts interpreted their own results incorrectly.

“That narrative that was building in the media and being woven in with the polling results might have blocked some legitimate questioning of some of the predictions,” he said.

Where do pollsters go from here?

Madonna said he still trusts his model overall, but he’s going to keep tweaking it.

“We’re going to continue to experiment with finding a balance between surveys online and telephone interviewees — that’s going to take some more complicated statistical adjustment,” he said. “The other problem was the regional breakdown … so we have to do a better job when we adjust the polls by region.”

And Borick is still poring over his results, looking for ways to fix the flaws that became apparent this year.

“You know, could it be an exceptional year?” he asked. “I mean that’s the question. And I’ve drawn no firm conclusions. At least we’ll have some time before we go back and do this again.”

He and Madonna agreed that, at the end of the day, polls can’t predict the future. For better or worse, it all comes down to the data.