SIGNIFICANCE BIAS:
The confusion of statistical significance, on the one hand, with biologic or clinical or healthcare significance, on the other hand, can lead to fruitless studies and useless conclusions1. It is often erroneously assumed that a statistically significant finding is clinically or biologically meaningful; when in fact, statistical significance and clinical/biological significance are different and unrelated concepts. The relationship between two variables, such as a predictor and outcome may be statistically significant but not meaningful; in the context of what these predictor or outcome variables represent in medicine. For example, when evaluating healthcare costs, an increase in cost of 100 dollars for the treatment of stroke per patient may be observed to be statistically significantly higher from the previous year, but when one considers the severity of cerebrovascular events, and the average size of healthcare budgets, this increase in cost may be viewed as reasonable, and worth accounting for in the next budget; thus the rise in cost is not clinically meaningful2. When reporting the results of a medical research study, medical journal editors may require a discussion of the clinical significance of findings, in addition to the results of statistical tests. Also see: Statistical Bias, and Wrong Sample Size Bias.
References:
1. Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32 (1-2):51-63. (Link to Reference)
2. Specogna AV, Patten SB, Turin TC, Hill MD. Cost of spontaneous intracerebral hemorrhage in Canada during 1 decade. Stroke. 2014;45(1):284-6. (Link to Reference)