December 21, 2016
Documents that Changed the World: Sir Ronald Fisher defines ‘statistical significance,’ 1925
Editions of Sir Ronald Fisher’s 1925 work “Statistical Methods for Research Workers.”University of Adelaide Rare Books & Special Collections Dept.
The subject of Joe Janes’ latest Documents that Changed the World podcast is close to the heart of many academic researchers: the threshold for “statistical significance” — and the man who, in a “surprisingly offhand manner,” set that mark for ages afterward at 5 percent, no more no less.
The man in question is English statistician and world-class evolutionary biologist Sir Ronald Aylmer Fisher, and the document is his 1925 book, “Statistical Methods for Research Workers.” It embraces the question: How certain of something do you have to be in order to say it is likely so; or as Janes writes, “How much likelihood, what probability of a result being wrong, we should be willing to live with.”
Modern researchers, he said, would refer to the threshold as “as a p level of .05, a 5 percent probability that a research result doesn’t indicate a real effect but rather comes from some random source.”
In the podcast series, Janes, an associate professor in the UW Information School, explores the origin and often evolving meaning of historical documents, both famous and less known. All the podcasts are available online through the iSchool website, and on iTunes, where the series has more than 250,000 downloads.
Janes, who has taught statistics and quantitative methods courses for many years, said he knows the topic so well he found the episode challenging to craft without overwhelming the listener or skipping “what I thought were important details.”
Still, even he found more to know about Fisher: “I didn’t know all the fine details and particularly the rather offhand nature of how his musings wound up becoming hard and fast strictures in the emerging field of statistical analysis,” Janes said.
Crucially, Janes notes that “when you have a hard and fast rule, it’s hard and fast, so a study result that just makes it across the line, winding up with a p value of .0499, gets to be called ‘statistically significant,’ and one that falls just short, with .0501, doesn’t.” Though he quickly adds that studies in the medical and pharmaceutical fields often have much more stringent thresholds, at 1 percent or even a tenth of a percent, because they involve the health and safety of the public.
Listen to the podcast to learn more about the man, as Janes writes, “whose work and ideas you’ve probably never heard of but who has had an effect on nearly every statistical study — and the way we understand the way we understand the world — for nearly a century.”
###
To learn more about this or any of the Documents that Changed the World podcasts, contact Janes at jwj@uw.edu.
Tag(s): Documents that Changed the World • Information School • Joe Janes
