Samples and statistics - The Ex-Communicator — LiveJournal
? ?
The Ex-Communicator

> Recent Entries
> Archive
> Friends
> Profile

November 12th, 2004

Previous Entry Share Flag Next Entry
06:28 pm - Samples and statistics
Daniel D has a post on Crooked Timber today about the Lancet article on Iraqi mortality rates. This was a statistical analysis of fieldwork conducted under difficult circumstances. The researchers calculated that deaths among Iraqis had increased significantly, in a broad range from 8,000-140,000 (if memory serves) with 98,000 being the most probable increase.

There are valid responses to this finding by those who support the war (such as 'I think this sad result is nevertheless worth it in the long run'). There may be valid statistical objections, because I haven't seen the raw data or methodology. But other objections relate to the practice of statistical sampling, extrapolation from samples, the meaning of confidence, and so on. These are not legitimate criticisms. Daniel deals with these in detail in his post.

This is something I know a bit about. I do statistical research of this kind as part of my job, and I get critiques from civil servants and others. Some of the criticism is valid, but most of it is ignorant. I mean ignorant not stupid: people are not taught about quantitiative research methods. For example, I might say 'the proportion of teachers who know how to use a word processor has increased from 55% to 89% in the last ten years(*)', and people will say 'you don't know that because you haven't gone back to exactly the same teachers'. I don't think people realise how many statistics in the public domain are extrapolated from samples, and how sophisticated we are at distinguishing good and bad samples, determining what deductions are valid etc.

The very wide range of possible death rates shows that the researchers acknowledge the big problems with extrapolating in such circumstances. The data are I think available to peer review, so objectors can double-check the assertions. But it's very difficult to explain, to people who don't know the first thing about it, the fundamentals of how statistics work. Trying to explain to people who are hostile is virtually impossible.

(*)made up figure

(4 comments | Leave a comment)


[User Picture]
Date:November 12th, 2004 04:40 pm (UTC)
Thanks very much for posting this. I'm one of those who are clueless about statistics (other than the most basic kind!) and need to rely on a knowledgeable interpretation, so that CT post and your confirmation of it were most enlightening.
[User Picture]
Date:November 12th, 2004 11:34 pm (UTC)
Cheers, you're welcome. I don't think everyone has to become an expert of course. In fact, I'm not quite sure what the solution is: we use stats so much for legitimate purposes, and yet they are also used to trick people.
[User Picture]
Date:November 13th, 2004 02:52 am (UTC)
I think you'd probably like Deltoid, an anti-hack blog.
[User Picture]
Date:November 13th, 2004 05:05 am (UTC)
Thanks for that. From a quick look through it seems to be an entire blog on just this Lancet article. Phew. It looks like he knows his stuff, and I hope he keeps on investigating statistics and hackery. I'd like a dispassionate and reliable opinion from someone on the whole exit poll issue for instance.

> Go to Top