Evans et al. published following study in the Arthroscopy Journal “The Potential Effect of Lowering the Threshold of Statistical Significance From P < .05 to P < .005 in Orthopaedic Sports Medicine”.
This caught my attention since these authors published similar article in another journal in 2019 titled “Effects of a proposal to alter the statistical significance threshold on previously published orthopaedic trauma randomized controlled trials“.
After the publication of the first study I sent a Letter to Editor regarding their study. Among other things I concluded that we have bigger issues in our field:
Authors conclude that lowering p-value might address some shortcomings in RCTs in orthopaedics. My concern is that this won’t address much of anything.
I continued:
I strongly believe that lowering p-value threshold would not really address any shortcoming because we have more deeply rooted problems in our methodologies. Misconceptions associated to hypothesis testing are not the only problem, but a prominent one. Others include HARKING, data dredging, arbitrary variable dichotomization and poor signal-to-noise ratio to name a few. I suggest that lowering the threshold for statistical significance would just result to more prominent interpretation of “no difference” or “equality”.
Original authors never responded.