Two weeks ago I gave a fun online talk on statistics for the Young Scientist Forum of the German Society for Biomaterials. I had some great chats with the organisers and there were good questions from the audience.
One good question was about how to interpret the analysis results when things are not clear cut. During my presentation I had talked about not deleting difficult outliers and not relying on p-values to give a falsely certain interpretation of what their results mean. The question was along the lines of: how should I interpret results that are “a bit of a mix”?
My live answer wasn’t great, but musing on the question I’ve decided what I should have said was, “Sometimes science is hard and we should all celebrate this!”
Difficult case studies
Clinical researchers regularly publish case studies of difficult or unusual patients. We should do the same with difficult results. A journal with a section on “Difficult results” could be widely read, as these results are often interesting and might teach us something about our own “messy” results.
An example I often use with students is the ABC cancer cluster, where there was a huge relative risk of breast cancer for workers in the Brisbane office, together with a tiny p-value. But there are lots of difficult questions to ask about such a relatively small study and it always creates a good classroom debate. I don’t think I will ever know if that cancer cluster was real or just a chance finding.
Slow down
We need to engage our brain more when we interpret results. We can’t rely on p-values to give us simple “yes” and “no” answers. But this part of our scientific brain has become atrophied through lack of use.
Scientists regularly do difficult things, like understanding complex chemistry and biology, and having novel ideas about how the world works. But we often take a short cut when it comes to one of the most interesting parts of research – when we finally get to see the results, which may have taken months or even years to produce.
Imagine watching a whodunit and skipping the bit where the clues solidify into facts. Reading the epilogue straight after the set-up would be hollow. So why are we fast-forwarding some of the best bits in our own scientific stories? Slow down and enjoy using your brain, and if you find it difficult then that means you’re doing it right.