Domb and Sebatian discuss about type II error in a “Level V guideline“. Among other things they conclude:
Type II errors may be present in greater than 20% of studies that fail to find statistically significant differences. Incorrectly concluding that there is no difference in such studies poses a danger to the advancement of clinical practice, and indeed may negate improvements that take our field from “good to great”. In light of the high prevalence of type II errors, it may be prudent for us as authors to take care to avoid sending an over-reaching message that there is no difference, and rather simply state that the data did not provide the necessary evidence to reject the null hypothesis.
Sure there is some truth in this but I think the authors could have taken more robust stand on this topic. Authors do not mention confidence intervals even once in their guideline. I think we should totally abandon concepts of type I and II errors and the concept of (null) hypothesis testing. Those can be used in the power analysis but then the whole statistical analysis should be based on an estimation. What was the point estimate and associated confidence intervals? What values we can exclude based on the confidence intervals? These are the questions we should be asking more often. I don’t see much values discussing “reasons for type II error”.