The race for the White House is a little closer than it looks

CLAIMING TO BE free of covid-19 and no longer infectious, President Donald Trump plans to bring his unique approach to vote-getting to Florida, Pennsylvania and Iowa by holding rallies in those states this week. Though this will give him the opportunity to speak directly to some voters, he is simultaneously passing up the chance to make his case to a much larger number, by declining to take part in the second presidential debate, which had been scheduled for Thursday.

All this follows a brutal sequence of polls, many of which put the president more than ten points behind Joe Biden, his Democratic challenger. The cause is no mystery: in many years Mr Trump appears not to have paid much, if any, federal income tax, does not seem to be able to debate his opponent without horrifying a majority of viewers, and apparently does not care much about infecting other people with covid-19. Even so, polls may be understating his support a bit. The reason is a phenomenon researchers call “differential partisan non-response”. In less geeky words, when the candidate you support is doing badly, you are less likely to answer a pollster’s call.

Pollsters are used to having to adjust their data to be demographically representative of the population as a whole. Imagine that in a pollster’s sample of likely voters, 50% do not have a bachelor’s degree. But according to the Census Bureau, in 2016 the share of voters without three-year degrees was 60%. So to get a representative sample of likely voters, a bit more weight must be given to this group to bring them from 50% to 60%. Similarly, if there are (say) too few non-college-educated or low-income respondents in a sample, pollsters will adjust their data to meet demographic or geographic reality.

But even after correcting these demographic biases, pollsters’ data can still be unrepresentative in other ways. They may have the right shares of young people and southerners, but too many Republicans or Democrats—or voters for Mr Trump or Mr Biden—in their datasets. This problem comes to the fore when an event causes especially good, or bad, news for a political party—as is the case now. In such scenarios, polls can suddenly be filled with partisans who are eager to voice their optimism or pessimism about the president.

To public-opinion nerds, this is nothing new. Studying data from the 2012 election, researchers at Columbia University, Stanford University and Microsoft noticed that support for President Barack Obama tended to rise and fall sharply in response to presidential debates. After the year’s first debate on October 3rd—in which Mr Obama’s opponent, Mitt Romney, was viewed as having performed better than the president—their estimate of Mr Obama’s share of the two-party vote, after weighting for standard demographic variables, cratered from roughly 57% to 47%. And after the second debate—in which Mr Obama put on a better show—his vote share rose from 48% to 52%.

The researchers viewed these shifts sceptically. In an age of hyper-polarisation and rigid partisan preferences, real swings in the presidential contest are rare and small. So the academics undertook another exercise. They compared the oscillating trends in Mr Obama’s projected vote share, controlling for standard demographic traits, with his vote share also weighted by each respondent’s political party, ideology and whom they voted for in 2008. The revised weighting scheme reduced the large, “phantom” swings by up to 50%. Instead of Mr Obama’s estimated support falling to 47% after the first debate—far below his eventual 52% in the election—the new trends hovered between 50% and 54% throughout the campaign.

In early 2020, The Economist reported during Mr Trump’s impeachment hearings that his approval rating in polls that weighted by demographics alone (age, sex, race, education level) was much more volatile than his approval rating in polls that were also weighted by partisanship or past presidential vote. Upon his acquittal in the Senate, Mr Trump’s rating in standard polling rose from under 40% to 43%. But in the weeks after, it settled back to 42%—where his average according to party-weighted polls had stayed throughout the hearings.

After a run of headlines about how little tax he paid, followed by a self-harming presidential-debate performance and a series of ill-considered photo-ops featuring an infectious president putting others at risk, Mr Trump has taken quite a few punches in the press. These are just the type of conditions that pollsters worry could turn Mr Trump’s supporters off answering their phone calls. If you support Mr Trump and have just watched a negative news segment about him on ABC News, how likely are you to give a pollster the time of day after they begin their call: “Hi, I’m calling on behalf of the American Broadcasting Corporation and the Washington Post for a poll about the presidential election”?

There is some evidence of partisan non-response in the polls at the moment. Unlike other models for aggregating the polls, The Economist’s election-forecasting model contains an adjustment for partisan non-response bias. On each day of the campaign, we compute the difference in Mr Biden’s vote share according to pollsters that do and do not weight by partisanship and past voting habits. Currently, our model estimates that pollsters who do not weight their data by political variables are roughly one to two percentage points more favourable to Mr Biden than those that do.

Accordingly, our model is slightly more bearish than others on Mr Biden’s position in national polls. Whereas the New York Times and FiveThirtyEight.com, a data-journalism website, give Mr Biden a lead of nearly 11 percentage points nationwide, our model puts him nine points ahead. Still, even after giving the president a relative boost in the averages, our election model assigns Mr Trump only a one-in-ten chance of winning the election. It would take a massive polling error—considerably larger than in 2016—for Mr Trump to pull off an upset victory again. Partisan non-response bias could account for some of the gap the president needs to make up, but nowhere near all of it.

Dig deeper:
Read the best of our 2020 campaign coverage and explore our election forecasts, then sign up for Checks and Balance, our weekly newsletter and podcast on American politics.