From his perspective as a pyschology researcher, Philip Tetlock watched political advisors on the left and the right make bizarre rationalizations about their wrong predictions at the time of the rise of Gorbachev in the 1980s and the eventual collapse of the Soviet Union. (Liberals were sure that Reagan was a dangerous idiot; conservatives were sure that the USSR was permanent.) The whole exercise struck Tetlock as what used to be called an “outcome-irrelevant learning structure.” No feedback, no correction.
He observes the same thing is going on with expert opinion about the Iraq War. Instead of saying, “I evidently had the wrong theory,” the experts declare, “It almost went my way,” or “It was the right mistake to make under the circumstances,” or “I’ll be proved right later,” or “The evilness of the enemy is still the main event here.”
Tetlock’s summary: “Partisans across the opinion spectrum are vulnerable to occasional bouts of ideologically induced insanity.” He determined to figure out a way to keep score on expert political forecasts, even though it is a notoriously subjective domain (compared to, say, medical advice), and “there are no control groups in history.” – The Long Now Foundation