As America moves toward a critical election using untried ways of distributing and collecting ballots, Americans are asking themselves: Who are we to believe – the polls or our own lying eyes?
The difference is stark. For months, polls have shown that Joe Biden has a consistent, clear lead over President Donald Trump. But every other traditional indicator points to a landslide victory for Trump.
Trump fills the stadiums with happy, enthusiastic and grateful crowds. Biden usually stays in his basement, and when he dares to move forward, his audience is tiny. The stock market shows no expectation that the oppressive tax and regulatory agendas in Biden’s platform will soon become law. Trump gets tangled up with reporters on critical issues every day. Biden spends a few hours a week on softball issues and only promises that if he is elected, he will tell voters where he stands.
Apparently, with one exception, Trump is generating enthusiasm for a second term, while Biden has essentially given in and eluded public scrutiny in the hope that the scandals surrounding his family will disappear. This one exception – or outlier, to use the statistical term – is, of course, the choice.
An “outlier” is a data point that does not match the trend. Without exception, an outlier is either an error or a story. An outlier due to an error can be corrected or eliminated. It is much more interesting when an outlier tells a story. Something has driven a result in an unexpected direction. What was it?
In the 2020 election, what separates crowd behavior, stock prices and press engagement from the published polls? The answer: The first three are organic phenomena; the public can see the raw data. Surveys are manufactured products. Pollsters develop questions, make contacts, take samples, ignore those who refuse to answer, and adjust their raw data – all largely hidden from the public – before publishing their sophisticated final results.
Most pollsters do this work on behalf of paying clients. Customers tend to have their own agenda. Those who really want to understand the public mood usually want to use this information for their own benefit – not share it with the public. Those who pay for public surveys usually appreciate the ability to shape public opinion, not just measure it. To understand public surveys, it is therefore important to consider the incentives that drive these clients and the polling companies they hire.
Like all professional services companies, successful survey companies are those that have a reputation for customer satisfaction. In stark contrast to the impartiality of real science, political pollsters generally decide whether to cultivate left or right-wing clients – and then help these clients tell the stories they want the public to hear.
Business incentives also push pollsters into the public eye to avoid controversy. Experts who make the same mistakes as everyone else rarely pay a price. The cost of a clear mistake can be extremely high.
Think of a pollster whose data shows that President Trump enjoys convenient advantages in popular and electoral college votes. If these predictions prove to be correct, the pollster has the right to boast. Would that be good for business? Perhaps. But if so, the most lucrative contracts would come from clients who want to understand public opinion – in other words, those who would keep the pollster’s later work out of the public eye and keep it confidential.
Meanwhile, those who stick to the pack would forge a professional consensus that explains why, despite appearances, they were not wrong in reality. In the 2020 elections, for example, the bald accusations of voter oppression already laid the foundation for such an apology. The economy rarely suffers from pollsters propagating inaccurate conventional wisdom.
If, on the other hand, conventional wisdom proves to be correct, those who have promoted it will declare its success and those who have dared to excel will be showered with derision and ridicule. An Opinion Research