Opinion Why doesn’t government want to learn?

For certain,nothing. The economists who supported the stimulus now argue the economy would have been worse off without it. Those who opposed it argue that the results have been meagre. It’s hard to think of anybody whose mind has been changed by what happened. This is not surprising. Still,it would be nice if we could learn from experience.

Jim Manzi has spent his career helping businesses learn from experience. In his new book,Uncontrolled,Manzi notes that many experts tackle policy problems by creating big pattern-finding models and then running simulations to see how proposals will work. That’s essentially what the proponents and opponents of the stimulus package did.

The problem is that no model can capture enough of the world’s complexity to yield definitive conclusions or make non-obvious predictions. A lot depends on what assumptions you build into them. Manzi argues that by tweaking the technical assumptions in these models,you eliminate the headline-grabbing results. All this model-building hasn’t even helped us get better at understanding the problem.

What you really need to achieve sustained learning,Manzi argues,is controlled experiments. Try something out. Compare the results against a control group. Build up an information feedback loop. This is how businesses learn. These randomised tests actually do vindicate or disprove theories. For example,a few years ago,one experiment suggested that if you give people too many choices they get overwhelmed and experience less satisfaction. But researchers conducted dozens more experiments,trying to replicate the phenomenon. They couldn’t.

Businesses conduct hundreds of thousands of randomised trials each year. But government? Hardly any. Why doesn’t government want to learn? First,there’s no infrastructure. There are few agencies designed to supervise such experiments. Second,there is no way to conduct a randomised experiment to test big,economy-wide policies like the stimulus package. Finally,the general lesson of randomised experiments is that the vast majority of new proposals do not work,and those that do work only do so to a limited extent and only under certain circumstances. This is true in business and government. Politicians are not inclined to set up rigorous testing methods showing that their favourite ideas don’t work.

Manzi wants to infuse government with a culture of experimentation. Decentralise policy experimentation to encourage maximum variation. His tour through the history of government learning is sobering,suggesting there may be a growing policy gap. The world is changing fast,producing enormous benefits and problems. Our ability to understand these problems is slow. Social policies designed to address them usually fail and almost always produce limited results. Most problems have too many interlocking causes to be explicable through modelling. Still,things don’t have to be this bad. The first step to wisdom is admitting how little we know and constructing a trial-and-error process on the basis of our own ignorance. Inject controlled experiments throughout government. Feel your way forward. Fail less badly every day.