Fortune Telling: From Delphi to Big Data

“We are suckers for those who help us navigate uncertainty, whether the fortune-teller or the “well-published” (dull) academics or civil servants using phony mathematics” ~ Nassim Nicholas Taleb in The Black Swan

For as long as I can remember, I’ve had an obsession with the future.

I had several concerns which hovered around whether I would be rich, famous, happily married with four children, satisfied in general, and reach heaven when I died.

This obsession with knowing the future hasn’t faded one jot for me.

And I wonder how many of us are like that – future-obsessed?

It turns out modern day science, mathematics, and statistics have some answers for future-obsessed persons like myself. Surprisingly, mathematics with tools like regression and correlation has managed to predict things like customer preferences, the stock markets, whether you should be given a loan, or not, and much more.

Data analysts are using vast reams of data measured in a billion gigabytes to study the past in great depth, create various models, detect patterns in these models, and then prognosticate the future.

But do you really believe predicting the future works as simply as that?

Here my mind turns to the original foretellers of destiny – the prophets of the Bible and the Delphian oracle. The Bible accurately predicted the reign of King Cyrus and the Delphian oracle correctly predicted what Croesus would be doing at a particular time and place.

And, a certain Madame Cassandra almost slipped my mind!

How did these entities predict or prophesy the future?

They used no science. There were no massive data sets which had to be cleaned and prepped before use. No statistical mathematics to speak of. Yet these strange beings foretold, and foretold accurately, the events to come.

That is what man seeks to do. The more mankind can tame the uncertain, the more he is capable of building things according to his will and vision.

The truth is, as a race, we’re a frightened group of people.

Living in a world of constant uncertainty, deep hatred, and certain death, our hearts and minds yearn for stability, peace, quiet, and equilibrium.

Studying randomness and hoping that the chaos will somehow yield a bridle is our only hope, so we put all our efforts into taming the unknown.

I read book “The Black Swan” when I was in my early twenties. I couldn’t, of course, follow the book word-for-word, owing to my paucity of wisdom, but I did understand a sufficient amount to agree with Taleb about the outliers.

Nobody in the business of predicting the future scientifically should forget about the outliers.

It’s these seemingly innocuous points of data that have the Black Swan hidden among them. The random events, strange catastrophes, and the “unpredictable” disasters of the stock markets.

So, why the faith in predictive analysis?

Predictive analysis is helping businesses churn profits. By estimating where the next ripple in the pond will come from, this contingency fighter is quietly making a convenient and more efficient world.

Until it gets things wrong.

With Big Data you are looking into the future from the standpoint of the past, with the Delphian oracle, you’re looking into the future. Period.

Big Data creates patterns, and insights can be inferred from these patterns. It’s a simple world on paper, but don’t be fooled. Predicting the future is God’s game, let mere mortals stick to unravelling the patterns.

Photo by Sergio García on Unsplash