No evidence of no evidence

In the null hypothesis significance testing framework, failure to reject the null is never evidence in support of null. However, it is extremely common that failure to reject the null, ie. getting a p-value larger 0.05 is interpreted as “no difference” or “no evidence”. As many experts have said, “absence of evidence is not evidence of absence”. This means that even if we could not observe a difference, say, in means between groups, that does not mean that there could not be a difference. Instead of saying that there is “no difference” or “no evidence” we should look at the confidence intervals and see which values are compatible with the data.

“Absence of evidence” corollary is very commonly used when aforementioned statements are made. When ever a statement about “no evidence” is given in a research paper, we can also say that actually there is “no evidence of no evidence”.

Leave a Reply

Your email address will not be published. Required fields are marked *