Refiguring the Iraq body count

ALMOST AS soon as President Bush gave the number of Iraqis who have died as a result of the U.S. invasion and occupation as “30,000, more or less,” aides hastened to downplay the number as “unofficial,” plucked by Bush from “public estimates.”

The president may have been quoting figures published by iraqbodycount.org, which has tabulated a death toll as high as 30,892 purely on the basis of published press reports of combatrelated killings. As IBC readily concedes, the estimate must be incomplete because it omits unreported deaths.

There is, however, another and more reliable method for estimating figures such as these: nationwide random sampling. No one doubts that the result accurately reflects the overall situation if the sample is truly random and the consequent data correctly calculated. That, after all, is how market researchers assess public opinion on everything from politicians to breakfast cereals.

In 2000, a team led by Les Roberts of Johns Hopkins School of Public Health used random sampling to calculate the death toll in the Congolese civil war at 1.7 million. This figure prompted immediate action by the U.N. Security Council. No one questioned the methodology.

In September 2004, Roberts led a similar team that researched death rates in Iraq before and after the 2003 invasion. Making “conservative assumptions,” the team concluded that “about 100,000 excess deaths” among men, women and children had occurred in 18 months. Most were directly attributable to the breakdown of the healthcare system prompted by the invasion. Violent deaths had soared twentyfold.

Unlike the respectful applause granted the Congolese study, this one, published in the prestigious British medical journal the Lancet, generated a firestorm of criticism. The outrage may have been prompted by the unsettling possibility that Iraq’s liberation had already caused a third as many Iraqi deaths as the reported 300,000 murdered by Saddam Hussein in his decades of tyranny. So shocking was this concept that liberals joined hawks in denouncing the study.

Some of the attacks were selfevidently absurd. British Prime Minister Tony Blair’s spokesman, for example, questioned the survey because it “appeared to be based on an extrapolation technique rather than a detailed body count,” as if Blair had never made a political decision based on a poll.

Some questioned whether the sample was distorted by unrepresentative hot spots such as Fallouja. In fact, the amazingly dedicated and courageous Iraqi doctors who actually gathered the data visited 33 “clusters” selected on an entirely random basis. In each of these clusters, the teams conducted interviews in 30 households, again selected on a rigorously random basis. As it happened, Fallouja was one of the clusters that came up in this process. Erring on the side of caution, they eliminated Fallouja from their sample. Strictly speaking, the team should have included the data from that embattled city in their final result -- random is random after all -- which would have given an overall post-invasion excess death figure of no less than 268,000.

Many critics made a meal of the study’s passing mention of a 95% “confidence interval” for the overall death toll of between 8,000 and 194,000. This did not mean, as asserted by some who ought to have known better, that the true figure lay between those numbers, and that the 98,000 number was produced by splitting the difference. In fact, the 98,000 figure represents the best estimate drawn from the data. Had the published study (which was intensively peer reviewed) cited the 80% confidence interval also calculated by the team -- a statistically respectable option -- then the spread would have been between 44,000 and 152,000.

Such statistical arcana were obviously beyond the grasp of most commentators, while the lack of any reference to the Johns Hopkins results in reports of Bush’s recent remarks surely indicates a persistent reluctance to confront what we have really done to Iraq.

Columbia professor Richard Garfield, one of the team members and study authors, told me this week that by now the number of “excess deaths” in Iraq “couldn’t possibly be less than 150,000.” But, he added, “there’s no reason to be guessing. We ought to know better.”