Data. Dayadhvam. Damyata.

David Brooks writes:

If you were to ask me to give a cute-sounding name to the hottest metaphysical trend in the intellectual world, I would call it Dataism, not to be confused with Dadaism,  after the android Data in 2001: A Space Odyssey, a soulless computer that knew the dimensions of everything and the value of nothing. In much the same way, our world is currently being quantified to a fare-thee-well, to the point where the pullulation of data threatens to overwhelm reality, blanketing it like a lead-filled duvet. There's a feeling out there that this is a good thing; that everything that can be measured should be measured, because data gives us the straight dope on the way things are, unmixed with the confusions of passion and bias, and this purity, like the purity of crystal, supposedly gives it a kind of magical power—enabling the scientist to predict the future.

Tiburtine Sibyl, from Ernst Freymund, Der Klugen Sibyllen verbessert astrologischer Weissagungs-Calender auf das Jahr 1741 (Nürnberg, 1741; VD18 90148967). From Research Fragments.

In coming months, I'm planning to take a long hard look at the universe of Dataism, seeing how it measures up against gut instinct on the turf, in relatively fair weather. When is it a good idea to give the facts a rain check and go with your intuition? When do the facts come up from behind to bound ahead in the stretch, leaving your intuition in the dust? What kinds of things are predictable with this modern numerology and what are not? 

I have to say I'm pretty skeptical. People have been trying to predict the future for centuries, and astrology, with all the numbers in the world, is no more successful than the Delphic oracle. Nevertheless, it's a known fact that data is really good at some things. 

In the first place, data has a way of showing you that something you thought was obviously true actually is not. For example, I'm told that casino gamblers believe in a phenomenon called the Just Universe, according to which if your luck has been going one way for a very long time, then it's time for it to start going the other way. Thus, when the roulette table at Monte Carlo hit black 26 times in a row, it was assumed that it was due for a similar streak of red. But according to Tversky and Kahneman, data proved that it never happened. 

Similarly, in a recent presidential election in a large Western country, members of the relatively conservative party were totally convinced that their candidate was going to win, displacing an unpopular leftist incumbent. There was just an electrical something in the air, they claimed; moreover, the relatively conservative campaign had more money than God, and was in the hands of a political wizard and math genius, or, rather, not exactly in his hands, which would have been illegal, for certain complicated reasons, but sort of run in parallel. Anyway one sniveling stripling statistician, on the basis of nothing but pure polling data, kept insisting that the unpopular leftist candidate was going to win, and he did. This is because the universe is not in fact just. And he probably had more money too. 

Second, data can reveal patterns in reality that simply thinking about things would not lead one to expect. Thus, I've always thought that if I used a lot of first-person singular pronouns in my speech or my writing that would mark me out as egotistical, which is probably why I hardly ever do. But as James Pennebaker of the University of Texas notes in his book, "The Secret Life of Pronouns", the real reason people use a lot of first-person pronouns is that they feel insecure. Thus, President Obama uses fewer first-person pronouns than any modern president. I don't think that means he has a self-esteem problem. 

By the same token, methods of statistical inference can be applied to the study of the question whether the grammatical structures used by a given writer, say Krugman, are more primitive, and his lexical selections less recherché, than those of another, say myself, proving that I really have nothing to be insecure about. 

In summary, data is transforming our world and is amazing stuff. But can it really predict the future? I'll just go check my Magic 8-Ball and let you know, heh-heh.

The prophet Daniel. Russian icon, 18th century, from Wikipedia (German).