Exploratory or confirming study?

By | February 21, 2020

Methodology is hard. And making valid inferences is very hard. With regard to these topics, orthopaedic research is not very different to other field in medicine. It means that misconceptions, misunderstandings and flawed approaches are prevalent also in our field.

American Journal of Sports Medicine published recently two papers which both made a quite common, yet very fundamental mistake. Clearly an exploratory study was framed as if they were confirmatory ones. The difference between them is huge. Basically, it means that targets are set after the results are known. Tweet by David Glass has quite nice illustration of this:

I send a “dual letter to editor” to AJSM. I raised my concerns about poor methodology and especially presenting a clearly exploratory findings as confirmatory. This was quite obvious since the title included the main “finding”. Basically this approach is about multiplicity since multiple endpoints were examined and those associated with a statistical significance were reported as “true”. Responses from the original authors were interesting and reveal some bad practices of the journal itself.

Bad statistical approaches are endemic in orthopaedic research and it seems that practices of orthopaedic journals are not helping in that. But that’s not surprising.

Leave a Reply

Your email address will not be published. Required fields are marked *