The Foresight Muddle & Hindsight Trap I'm going to start this by taking us all back, to the early days of the U.S. Department of Homeland Security. What did we hear from them? Scuba divers preparing to blow up bridges and shipping. Threats to major landmarks. Etcetera, etc., and all on CNN. As I noted in my Sept. 19, 2002 "Sept. 11 Intelligence Failure: Let's Get A Grip" piece: bq. "Remember the joke they quickly became as the public was overloaded with non-specific "information," and then the threats never materialized? Now consider that those reports were a highly selective and filtered sub-set of the stuff the government is receiving on a regular basis. That was actually the stuff they thought was most likely." This is the foresight problem. We don't often see it, because we the public aren't there for all the scenarios and discussions. For a little while in 2002, that door was opened a crack and we all got a first-hand glimpse of the problem: uncertainty and how to deal with it. With so many possibilities and limited resources, what does one plan and prepare for? What are the priorities? That's the judgement factor. Worse, odds are good that whatever crazy thing happens, someone somewhere has put it down on paper as a possibility. So technically, you should have known, right? It's easy to see afterward what the meaningful clues were (as opposed to the far more numerous 'junk'), and what the priorities should have been. This leads us to... The Bureaucratic Imperative There's a reason bureaucrats are seen as ass-coverers whose foremost priority is to ensure that they're never accountable for anything. It's because many bureaucrats really are ass-coverers whose overriding priority is to ensure that they're never accountable for anything. As a corollary, if things go wrong, support for the idea that you predicted it may be helpful in building your bureaucracy's importance at the expense of its rivals. This is as true within corporations as it is in the public sector or the military. If you've spent much time in the working world, you've seen this tendency in action. Well, being in the CIA, DIA, DMV, or whatever doesn't suspend this aspect of human nature. So what we see here is the CIA finding a report that predicted some of the current challenges and getting that story out. No doubt, others are busy playing this same game. That's why one report carried breathlessly in the Boston Globe or wherever registers just above "interesting" and just below "so what?". It has the potential to become significant, but not until we have some additional information. Some Good After-Action Questions To really evaluate the decisions made, you'd have to know what the other scenarios were like, and what else the CIA predicted in this and other reports. You'd ask questions like: * What did the coalition prepare for that turned out not to be a huge issue? * What was the basis for those predictions, and how could it have been improved with the resources available at that time? * Was greater certainty a realistic possibility at that time? Could a reasonable planner make those predictions, given what was known at the time? * What was supposed to happen that didn't? * Were there other risks that may have been less probable, but so potentially catastrophic that they received weighted priority (likelihood x severity)? With some solid answers to these questions in hand, it's possible to start making informed judgements. Without these answers, it isn't really possible. Which is why judgments tend to be fragmentary and uncertain at best until after the dust has settled. There are also sound reasons not to force military leaders to spend a lot of their time on reflective public reports until the job itself is done or pretty close to done (though military industries are fair game, as the Truman Committee demonstrated). Charges of hostile interference are not entirely without basis, therefore, if planning is made into a large political issue while hostilities continue. Until hostilities end, the focus of inquiries needs to be on what it takes to win, and whether the body politic is prepared to pay that price, and what the alternatives are (if any) given the present state of commitment. Looking at after-action events can help get us there, and while these questions indirectly concern planning their real focus is elsewhere - "what are we doing now, and what should we do next?" rather than "what should we have done before?" To this end, we ask another set of questions, starting with: * How does this performance compare to similar experiences elsewhere (for instance, USA in Germany post-WW2)? * If we use parallels, what's different this time? How does that change our assumptions? * By the standards of experience, how does progress measure up? * Are the expectations inherent in the criticisms reasonable? * Most important of all, how adaptable was the planning and execution? That last bit is what matters most. Unexpected things always come up. You can never plan for every possibility, and anyone who doesn't believe me should be sentenced to work in an organization that actually believes you can. This will either cure you inside of 3 months, or successfully lock you away from productive society. Those who understand the role of the unexpected might ask: * How does the effort now differ from efforts at the beginning? * Have the changes improved performance, or lowered it? * How quickly and easily have these changes been made? * If we've identified some shortfalls in important areas, how can we remedy that within our resources? * What other resources could we bring to the table, and would they help or hurt overall efforts? Why? To the extent that the kinds of questions being asked here are being addressed, you're probably reading a serious analysis. It may still be wrong, of course, but its odds of providing meaningful insights and helpful policy suggestions rise sharply. To the extent that you do not see answers to the kinds of questions raised in this post, or acknowledgement of some of these fundamental dynamics, it's time to be cautious or even suspicious of the analyst's motives. The person you're reading or talking to may still be right on the money with their retrospective - but it ain't bettin' odds. Unlike forward-looking decisions, one can usually afford to wait for betting odds when doing full-contact public political analysis of lessons learned. Given the expense, blood, and heartache that stems from learning the wrong lessons, it's worth the wait. UPDATE: Caerdroia comments: bq. "One amazing feature of the blogosphere is that there is a certain tendency for articles on different sites about different subjects to reinforce each other, bringing about a kind of an epiphany. So today with Porphyrogenitus and Winds of Change" Good commentary, and P.'s "America's 21st Century Foreign Policy Part IV" is worth reading, too.