Do we have bigger problems than possible lowering the threshold of statistical significance

Evans et al. published following study in the Arthroscopy Journal “The Potential Effect of Lowering the Threshold of Statistical Significance From P < .05 to P < .005 in Orthopaedic Sports Medicine”.

This caught my attention since these authors published similar article in another journal in 2019 titled “Effects of a proposal to alter the statistical significance threshold on previously published orthopaedic trauma randomized controlled trials“.

After the publication of the first study I sent a Letter to Editor regarding their study. Among other things I concluded that we have bigger issues in our field:

Authors conclude that lowering p-value might address some shortcomings in RCTs in orthopaedics. My concern is that this won’t address much of anything.

I continued:

I strongly believe that lowering p-value threshold would not really address any shortcoming because we have more deeply rooted problems in our methodologies. Misconceptions associated to hypothesis testing are not the only problem, but a prominent one. Others include HARKING, data dredging, arbitrary variable dichotomization and poor signal-to-noise ratio to name a few. I suggest that lowering the threshold for statistical significance would just result to more prominent interpretation of “no difference” or “equality”.

Original authors never responded.

Leave a Reply

Your email address will not be published. Required fields are marked *