Über die Autorin bzw. den Autor

Stephen T. Ziliak is the author or editor of many articles and two books. He currently lives in Chicago, where he is Professor of Economics at Roosevelt University.

Deirdre N. McCloskey, Distinguished Professor of Economics, History, English, and Communication at the University of Illinois at Chicago, is the author of twenty books and three hundred scholarly articles. She has held Guggenheim and National Humanities Fellowships. She is best known for How to Be Human* Though an Economist (University of Michigan Press, 2000), and her most recent book, The Bourgeois Virtues: Ethics for an Age of Commerce (2006).

Auszug. © Genehmigter Nachdruck. Alle Rechte vorbehalten.

THE CULT OF STATISTICAL SIGNIFICANCE

The University of Michigan Press

Contents

Preface.......................................................................................xvAcknowledgments...............................................................................xixA Significant Problem.........................................................................11. Dieting "Significance" and the Case of Vioxx...............................................232. The Sizeless Stare of Statistical Significance.............................................333. What the Sizeless Scientists Say in Defense................................................424. Better Practice: -Importance vs. {alpha}-"Significance"...................................575. A Lot Can Go Wrong in the Use of Significance Tests in Economics...........................626. A Lot Did Go Wrong in the American Economic Review during the 1980s........................747. Is Economic Practice Improving?............................................................798. How Big Is Big in Economics?...............................................................899. What the Sizeless Stare Costs, Economically Speaking.......................................9810. How Economics Stays That Way: The Textbooks and the Referees..............................10611. The Not-Boring Rise of Significance in Psychology.........................................12312. Psychometrics Lacks Power.................................................................13113. The Psychology of Psychological Significance Testing......................................14014. Medicine Seeks a Magic Pill...............................................................15415. Rothman's Revolt..........................................................................16516. On Drugs, Disability, and Death...........................................................17617. Edgeworth's Significance..................................................................18718. "Take 3[sigma] as Definitely Significant": Pearson's Rule.................................19319. Who Sits on the Egg of Cuculus Canorus? Not Karl Pearson..................................20320. Gosset: The Fable of the Bee..............................................................20721. Fisher: The Fable of the Wasp.............................................................21422. How the Wasp Stung the Bee and Took over Some Sciences....................................22723. Eighty Years of Trained Incapacity: How Such a Thing Could Happen 23824. What to Do................................................................................245A Reader's Guide..............................................................................253Notes.........................................................................................255Works Cited...................................................................................265Index.........................................................................................289

Chapter One

The rationale for the 5% "accept-reject syndrome" which afflicts econometrics and other areas requires immediate attention. ARNOLD ZELLNER 1984, 277

The harm from the common misinterpretation of p = 0.05 as an error probability is apparent. JAMES O. BERGER 2003, 4

Precision Is Nice but Oomph Is the Bomb

Suppose you want to help your mother lose weight and are considering two diet pills with identical prices and side effects. You are determined to choose one of the two pills for her.

The first pill, named Oomph, will on average take off twenty pounds. But it is very uncertain in its effects-at plus or minus ten pounds (you can if you wish take "plus or minus" here to signify technically "two standard errors around the mean"). Oomph gives a big effect, you see, but with a high variance.

Alternatively the pill Precision will take off five pounds on average. But it is much more certain in its effects. Choosing Precision entails a probable error of plus or minus a mere one-half pound. Pill Precision is estimated, in other words, much more precisely than is Oomph, at any rate in view of the sampling schemes that measured the amount of variation in each.

So which pill for Mother, whose goal is to lose weight?

The problem we are describing is that the sizeless sciences-from agronomy to zoology-choose Precision over Oomph every time.

Being precise is not, we repeat, a bad thing. Statistical significance at some arbitrary level, the favored instrument of precision lovers, reports on a particular sort of "signal-to-noise ratio," the ratio of the music you can hear clearly relative to the static interference. Clear signals are nice, especially so in the rare cases in which the noise of small samples and not of misspecification or other "real" errors (as Gosset put it) is your chief problem. A high signal-to-noise ratio in the matter of random samples is helpful if your biggest problem is that your sample is too small, though the clarity of the signal itself is a radically incomplete criterion for making a rational decision.

The signal-to-noise ratio is calculated by dividing a measure of what one wants-the sound of a Miles Davis number, the losing of body fat, the impact of the interest rate on capital investment-by a measure of the uncertainty of the signal such as the variability caused by static interference on the radio or the random variation from a smallish sample. In diet pill terms the noise-the uncertainty of the signal, the variability-is the random effects, such as the way one person reacts to the pill by contrast with the way another person does or the way one unit of capital input interacts with the financial sector compared with some other. In formal hypothesis-testing terms, the signal-the observed effect-is typically compared to a "null hypothesis," an alternative belief. The null hypothesis is a belief used to test against the data on hand, allowing one to find a difference from it if there really is one.

In the weight loss example one can choose the null hypothesis to be a literal zero effect, which is a very common choice of a null. That is, the average weight loss afforded by each diet pill is being tested against the null hypothesis, or alternative belief, that the pill in question will not take any weight at all off Mom. The formula for the signal-to-noise ratio is:

Observed Effect-Hypothesized Null Effect/Variation of Observed Effect

Plugging in the numbers from the example yields for pill Oomph (20-0)/ 10 = 2 and for pill Precision (5-0)/0.5. = 10. In other words, the signal-to-noise ratio of pill Oomph is 2 to 1 and of pill Precision 10 to 1. Precision, we find, gives a much clearer signal-five times clearer.

All right, then, once more: which pill for Mother? Recall: the pills are identical in every other way, including price and side effects. "Well," say our significance-testing, sizeless scientific colleagues, "the pill with the highest signal-to-noise ratio is Precision. Precision is what scientists want and what the people, such as your mother, need. So, of...