Turned up to eleven: Fair and Balanced

Tuesday, December 11, 2001

I spent some time looking into the breast cancer/mammography research in Lancet, and it is clear that this research is a work in progress. Now, I am extremely skeptical of the value of the type of meta-analysis reported in the NY Times, and based on this report and the vociferous response to it, I am justified in my skepticism. The "investigators" (I am not sure that people who spend their time poking around in other peoples work should be called scientists; perhaps auditors) disqualify 6/8 large mammography screening studies based on "inadequate randomization". Their chief objection is that the study and control groups had "significant" differences in age. I put that in quotes, because I think that what they showed is that they do not understand how to do statistical analysis. I will attack one particular statement, that an average (mean) age difference of approximately 6 months has a p-value of 1x10^-37!!!! Let me put that in perspective for you. There are approximately 10^9 stars in the galaxy, and approximately 10^9 galaxies in the universe. The universe has existed for approximately 15 billion years (1.5x10^10). There are ~30 million seconds in 1 yr. The Universe has therefore existed for approximately 10^17 seconds. A p-value can be interpreted correctly as the probability that an event would occur by chance. In this case, the probability ought to be interpreted as the probability that the two groups (study and control) could arise in a random selection with the described age difference by chance. Interpreted in terms of the numbers above, if a set of random people had been chosen every 0.0001 FEMTOsecond since the Big Bang, this event would have happened ONE time. This assertion is completely ludicrous, of course, but that is what the statistic literally means. I think we can safely assume that there is a systematic error in their analysis. Nevertheless, this paper points out a significant problem that medical trials have. The ethical demands of clinical research are substantial, because a clinician CANNOT morally deny care to an individual for the sake of the study. In this particular analysis, a number of studies were forced to exclude patients from the study group because cancer was detected in the initial screening. The control group did not have this problem, BECAUSE THEY WERE NOT SCREENED. Think about that for a minute. It necessarily implies that MAMMOGRAPHY WORKS!!! I know I am shouting a lot in this, but I just can't believe how ridiculous this is. The authors are seriously arguing that because the screening process detected cancer in the study group better than the physical exams in the control group, this should call into question the efficacy of the screening process. The medical profession is in dire need of help!!!