Rise of the Empiricists

Is the US becoming more of an empirically based society?

The early evidence across a variety of fields is in, and it appears to be yes — albeit rather slowly. I suspect this adaptation is going to accelerate rapidly over the next decade.

A blog post is not where we can do a full blown study on this — that’s what academia is for. However, we can take a closer look at areas where data analysis and experimental observation make a difference.

Empirical evidence (aka empirical data, knowledge, experimentation) is a source of knowledge acquired by means of observation or experimentation, producing voluminous reams of data. We can look at the results of these experiments and draw conclusions based on the weight of the evidence.

What follows is a brief, intentionally ironic anecdotal look at the areas where empiricism is on the rise:

• Political forecasting: The big story of the 2012 elections was the rise of the data junkies, with Nate Silver being the most visible version. The squishy, narrative-driven data-free observations turned out to be little more than examples of confirmation bias -0- they simply failed.

• Climate Change: A funny thing keeps happening to a number of climate denialists — they have given up their opposition and have accepted the scientific consensus. Some have simply admitted they are funded by Oil and Coal companies. The reason for this: The alternative narrative simply lacked sufficient data to respond to an overwhelming set of published papers and data.

• Technology User Design: As Wired explained last year, some businesses guess how new products or services are going to be received by consumers. (Think “New Coke”). But web companies have the ability to quantify this process, and actually test users who are unknowingly moved to novel pages to test their reactions. This A/B testing has revolutionized a number of industries, including web interface design, online checkout carts, and even political fundraising.

• Economics and public policy: In their book This Time is Different, Reinhart & Rogoff looked a centuries of data about credit and other financial crises. To prod policy makers into becoming more data driven, they have made all of their statistical data available for download. This stands in stark contrast to the gut feel approach we have seen be deployed so disastrously over recent decades.

• Asset Management: Finance has thrown an enormous amount of analytical firepower at the field of putting capital to work. From Quant analytics to Venture Capital to simple investing, there has never been more statistical analyses of what does and doesn’t works  than there is today. The old myths, heuristics and misleading assumptions are slowly falling away as we learn precisely what not to do.

The biggest factor impacting investors is no longer our knowledge base, but rather our behavioral elements .

~~~

Perhaps a worthwhile question we should ask ourselves: What narratives am I telling myself today? Is there any data set or analyses that can prevent me from fooling myself?

Empiricism will be moving more and more of the US economy forward. Will you profit from this, or will your loss be someone else’s gains?

Previously:
Agnotology  (January 21st, 2009)

The Dangers of Non-Modeled Narrative Story Tellers (December 3rd, 2012)

Reality Check: What Are You Lying to Yourself About? (January 7th, 2013)