The U.S. presidential election: Big data prognosticators failed badly

In the last few election cycles, code-driven tools were fine-tuned and their projections were, indeed, remarkably accurate in important elections worldwide. Machines calculated the probabilities, and no pundits with experience and inside contacts were asked to tell us what was going to happen.

The process was clear: first gather historic and current data, write the code you need, and run regressions. What happened this year when predictions failed so badly?

Illustration

Truth is in data, right?

Take Brexit or the US presidential election. There was enough data this time. The history of voting behavior, the registration trends of voters, interesting demographic insights – everything was readily available and processed using the latest knowledge and technology in the field of big data. 

The lack of human touch was what makes this approach so attractive. Yet that's not really true, because when preparing polls, it's people who choose the questions and weights of the samples.

According to early voting data, polls and historical results, Hillary Clinton should have won in Florida, but she lost. That’s why we should adjust our assumptions about analytics a bit. An approach that is purely data-driven isn’t as infallible as we believe, says an article on the strategy-business.com website.

1) Polls are not very reliable

Polls ask people what they say or think they’ll do. But we don’t know what they actually do. Here's where the results and projections start to look different. You can estimate how many times you’ll exercise next month. Then count how many times you actually do.

2) It's still difficult to get the big picture

Poll-takers formulate the questions and how they're phrased. The weight of the samples are also selected by people. These decisions are made in a very complex context. On top of that, in swing states like Florida, the population today is very different when compared to the population four or eight years ago. It's growing rapidly and its demographics are shifting constantly.

-jk-

Article source Strategy+Business - a U.S. management magazine
Read more articles from Strategy+Business