Communicator (communicator) wrote,
Communicator
communicator

Samples and statistics

Daniel D has a post on Crooked Timber today about the Lancet article on Iraqi mortality rates. This was a statistical analysis of fieldwork conducted under difficult circumstances. The researchers calculated that deaths among Iraqis had increased significantly, in a broad range from 8,000-140,000 (if memory serves) with 98,000 being the most probable increase.

There are valid responses to this finding by those who support the war (such as 'I think this sad result is nevertheless worth it in the long run'). There may be valid statistical objections, because I haven't seen the raw data or methodology. But other objections relate to the practice of statistical sampling, extrapolation from samples, the meaning of confidence, and so on. These are not legitimate criticisms. Daniel deals with these in detail in his post.

This is something I know a bit about. I do statistical research of this kind as part of my job, and I get critiques from civil servants and others. Some of the criticism is valid, but most of it is ignorant. I mean ignorant not stupid: people are not taught about quantitiative research methods. For example, I might say 'the proportion of teachers who know how to use a word processor has increased from 55% to 89% in the last ten years(*)', and people will say 'you don't know that because you haven't gone back to exactly the same teachers'. I don't think people realise how many statistics in the public domain are extrapolated from samples, and how sophisticated we are at distinguishing good and bad samples, determining what deductions are valid etc.

The very wide range of possible death rates shows that the researchers acknowledge the big problems with extrapolating in such circumstances. The data are I think available to peer review, so objectors can double-check the assertions. But it's very difficult to explain, to people who don't know the first thing about it, the fundamentals of how statistics work. Trying to explain to people who are hostile is virtually impossible.

(*)made up figure
Subscribe
  • Post a new comment

    Error

    Comments allowed for friends only

    Anonymous comments are disabled in this journal

    default userpic
  • 4 comments