Heisenberg's Uncertainty Index

The title of Matt O'Brien's recent post--"Uncertainty Isn't Killing the Recovery"-- summarizes a slurry of recent articles. O'Brien writes, "as Jim Tankersley of the Washington Post points out, uncertainty has actually fallen a lot the past few months, but hiring hasn't picked up." Tankersley's title proclaims a similar message: "'Uncertainty' Isn't a Problem Anymore."

The boldest title of all (no surprise) comes from Paul Krugman: "Another Bad Story Bites The Dust."

I thought about following the Very Descriptive Titles trend and calling my post "Whoa, Whoa, Wait a Minute Guys," or "Let's Not Throw Out the Baby with the Bathwater Just Yet" or "Uncertainty Might Matter So Maybe We Should Let Smart People Keep Studying It If They Want." (But I was too much of a nerd to resist the title I actually chose, which I will eventually explain.) Legitimate criticism of political discourse is one thing. But we shouldn't, in the meanwhile, dismiss a large, promising, and growing area of research.

The recent articles find fault with the Economic Policy Uncertainty index of Scott Baker and Nick Bloom of Stanford and Stephen Davis of the University of Chicago. The index is based in part on how often major newspapers talk about phrases related to economic policy uncertainty. The basic idea of the criticism has to do with uncertainty a la Heisenberg. The non-technical version of the Heisenberg Uncertainty Principle is that, in observing something, we change it. The usual context is quantum mechanics, but it applies in other contexts too. In this context, the idea is that once the Economic Policy Uncertainty index was publicized, conservative politicians and pundits found it a useful talking point, and by talking and writing about it so much, they actually drove the index higher. So what the economists were observing--the frequency of mentions of uncertainty-related phrases-- was actually altered by the attempt to observe it.

This is a valid, but not devastating point. First, we don't know how large this "Heisenberg effect" is. Conservative punditry could have had only a small effect on the index. Second, it is not that hard to think of ways to improve the index to minimize this problem. Simplest idea: exclude articles that include the exact phrase "economic policy uncertainty index" or the words Baker, Bloom, Davis, Stanford, or Chicago. Slightly less simple idea: Aren't there really sophisticated quantitative text analysis tools available these days that could measure the amount of uncertainty in the tone of economics/business articles? Third, and most important, is that there is much more to Baker, Bloom, and Davis' research than the index itself. Bloom's 2007 paper, "The Impact of Uncertainty Shocks"--written before the construction of the index-- helped kick off a surge of macroeconomic uncertainty research. Many of Baker, Bloom, and Davis' contributions, and those of authors they have motivated, have been theoretical. A slightly flawed measure beats no measure at all if it encourages researchers to pay attention to an important subject. And I would venture to guess that the vast majority of these researchers are not motivated by a political agenda. They are motivated by the desire to understand something they think might really matter.

For what it's worth, I think the Economic Policy Uncertainty index does have decent construct validity. The creators of the index make an analogous index for Europe. In an unpublished working paper, I make a totally different uncertainty index for Europe, based on the how uncertain professional forecasters are in their probabilistic forecasts for GDP growth 2-years ahead. (I chose Europe instead of the US because of data availability.) My uncertainty measure is plotted along with the Economic Policy Uncertainty index below; the correlation coefficient is 0.8. The fact that they are created by totally different methodologies, from totally different data sources, and yet are this highly correlated, makes me think that there is at least some measurement validity.

I appreciate John Aziz's more moderately-titled post, "On Policy Uncertainty," in which he writes:

Krugman is right to... trash those who view the sluggishness of the recovery as solely Obama’s fault. But he’s wrong, I think, to throw policy uncertainty out of the window entirely as a proximate cause of some of the problem’s we’re now facing.