Wednesday, October 11, 2006
Another survey in Iraq -- 600,000+ dead?
Today's news about Iraq includes a report of a public-health survey that concludes that since the American-led intervention in Iraq, over 600,000 people have died by violence in Iraq. About 2.5% percent of the Iraqi population.
This is no fly-by-night survey. It was designed and carried out by Johns Hopkins Bloomberg School of Public Health in Baltimore and Al Mustansiriya University in Baghdad, and published in the Lancet, perhaps the foremost English-language medical journal in the world. One can assume as a first step (which assumption must be verified, of course) that this study used a respected methodology. The Lancet would be playing with fire -- and the destruction of its scientific reputation -- if this survey was methodologically flawed. It's been around since 1823 and there's a lot of reputational capital built up.
Nonetheless, this survey will be strongly condemned. The question is, how many critics will have done their homework before speaking up?
This is one of those issues where both honesty and technical competence have to come together to solve a difficult and controversial problem. Lacking either, we're in a world of wishful thinking, unsubstantiated claims, and outright propaganda.
I see this as relating both to earlier posts on "war today" and to my earlier discussion of Chris Wickham's project to survey just about the whole former Roman Empire from 400 to 800 AD. A different kind of problem, accounting for social, economic and political change in a sparsely and unsystematically documented period, rather than in a war zone. But without such efforts, how close can you get to the truth?
In Iraq, the problem exists whether the answer on such basics as casualties are easy to come by or not. There is a scientific challenge here that should be met as well as possible.
See what Juan Cole has to say on the issues raised.
Update: Daniel Davies in the Guardian's Comment is Free section. Either the numbers are right, or the study is fraudulent. Discussion also at Brad DeLong's site (Oct. 13 entry).
Further update: The Washington Post's World Opinion Roundup covers the reaction; also, there was on October 19th a Washington Post Live Discussion with Gilbert Burnham, the lead author of the survey in question.
Update October 25: See Jefferson Morley's World Opinion Update for more conflicting opinions.
Subscribe to:
Post Comments (Atom)
Here's my take:
ReplyDeletehttp://willscommonplacebook.blogspot.com/2006/10/more-than-600000-killed-in-iraq.html
Another take on the issue:
ReplyDeletehttp://www.iraqbodycount.org/press/pr14.php
Will McLean's blog entry can be found using this URL:
ReplyDeletehttp://tinyurl.com/y3wtkb
Ian Schofield tried to post a link to Canada's National Post:
http://tinyurl.com/y46984
Some good discussion of the issue at
ReplyDeletehttp://janegalt.net/
Illegitimate arguments about the Lancet study
down in the comments is a quote of a WSJ article by Steve Moore. "I wouldn't survey a junior high school, no less an entire country, using only 47 cluster points."
Will,
ReplyDeleteThe Steve Moore quote by itself doesn't advance the argument. Does he give an argument why this methodology is basically unsound? It's been used before and not just in Iraq.
Steve Moore's article is here:
ReplyDeletehttp://www.opinionjournal.com/editorial/feature.html?id=110009108
He argues that the number of different clusters sampled is unusually low, compared to other cluster surveys. The 2004 UNDP survey in Iraq used 2200 cluster points, compared to 47 in the Burham et al 2006 study.
This results in a broad range of uncertainty, even if sampling is perfectly random and there is no other source of distortion, such as telescoping in recall or subjects telling the pollsters what they want to hear.
He also complains that the 2006 survey made no effort to capture demographic data in the sampled households. He claims that doing so is ordinary practice in his experience running surveys. If you collect this data you can check the randomness of your sample against other sources. If you don't you can't.