One year ago after another stupid fight with a journal about p-values, I made a pledge to go without them for a year. Here’s how it went.
But first, why?
I am aware of the arguments for and against p-values. I have used p-values for a long while and they can be a useful statistic.
The reason I ditched them is because almost nobody in health and medical research interprets them correctly, wrongly thinking they reveal the probability that the null hypothesis is true (other misinterpretations are available). P-values are on a pedestal and wrongly get used to judge the entire scientific validity and quality of a study. This inflated importance leads to p-hacking and publication bias, which are badly warping evidence-based medicine.
I’ve tried teaching my colleagues about the limitations of p-values and they have told me they understood. Then draft papers arrive in my inbox with little p-values doing all the heavy lifting. “We understand the limitations Adrian, but the journal reviewers won’t so we need the p-values.”
It’s a grandiose analogy, but I’m reminded of nuclear power: fine in theory, but terrible when combined with people. I’ve switched to cleaner forms of generating energy and evidence.
I’m claiming victory.
I’ve had very different conversations with colleagues when I told them you can have the p-values or me as an author, but not both. Although this was an ultimatum I was cordial and let my colleagues know that their decision would not upset me or harm our working relationship.
In every case but one my colleagues removed the p-values, relying instead on confidence intervals. But what was more satisfying is that it spurred some meaningful conversations about p-values and their limitations. My willingness to forgo that most precious of academic credit of a paper meant that my seriousness finally cut through.
For the one paper where my colleagues preferred the p-values, the paper was accepted and is in press. Deliciously the acknowledgement section reads: “Thanks to Adrian Barnett who would have been an author but it was him or the p-values.” Let’s hope that gets through the copy editors.
My pledge was also discussed on a medical research podcast.
I also spoke with some early-career statisticians who wanted to take a similar stand, but couldn’t because of power imbalances or the need to build their academic credits.
Dogma-fight
I am fighting dogma with dogma. I wish there was another way, but education alone has just not worked for me. I’ll continue to educate and I’ll also continue to eschew p. “Me or p” not “T and P”.