Communicator (communicator) wrote,

Lancet study and statistical confidence

I have been reading a lot of comments on the Lancet study which (within huge confidence intervals) attributed about 600,000 excess deaths to the Iraq war. I have read many comments that the sample was too small. I actually know a little bit about this kind of survey, because I have been a quantitative researcher.

Here's a counter-intuitive fact about statistical sampling. Confidence is not dependent on the relationship between the size of the sample and the size of the population. It depends on the size of the sample and the use of a fully random sampling method. If you pick two hundred and fifty people from a population of either a thousand or a million, completely at random, you should be able to make an equally confident statistical inference in either case. I know this is hard to believe, but it is true.

The catch is fully random sample. This is very difficult to achieve. It becomes harder the larger the population. Other factors influence how easy it is to find a person. You can't just phone houses at random for instance, because other influences then bias your sample: you get people who are more likely to be at home, and people who are more likely to be first to answer the phone.

To get a fully random sample of people in this country you would have to have a complete up to date census (not available) and a random number generator, and then chase up each person you picked, regardless of problems. The impossibility of this is what limits statistical confidence.

I should emphasise that the Iraq survey was not a perfect sample of this kind, but sampled clusters of households. That raises other issues, which are legitimate points for debate.

Misgivings about sample vs population I would say are understandable, made by intelligent people (though I think they are wrong). Misgivings about clustering methods are legitimate. On the other hand I have read lots of completely idiotic objections to the study, by people who just don't like its implications.

The debate around the Lancet findings also raises a subject which is more interesting to me than the maths - the status and use of intuition and gut feeling. I believe in intuition, but I dislike it when people say their intuition tells them that '600,000 is unbelievable'. But that's another issue.
  • Post a new comment


    Comments allowed for friends only

    Anonymous comments are disabled in this journal

    default userpic