SMALL SAMPLE BIAS:

A general term that refers to any biases in a statistical estimate, that occurs when a statistical test is used on a dataset that has too few data points (or no data)^1,2.

It is not possible to draw meaningful conclusions from the results of a statistical test when the dataset (or sample) being used has too few data points, for the number of outcomes or dependent variables being evaluated. In many cases the statistical estimate will overestimate an association between variables, and the variability of the statistical estimate may be high. The amount of data required to perform a valid statistical analysis, depends on the context of a research study, and needs to be determined before the study begins; using logic and various empirical methods.

The terms Sparse Data Bias and Finite Sample Bias are related to Small Sample Bias, and are often used to describe bias when multiple comparisons between categories of data are performed (e.g. in the case of regression analyses etc.). These terms are often preferred over the term Small Sample Bias, since bias due to data limitations can occur even in superficially large datasets. Also see: Sparse Data Bias, Finite Sample Bias, Small Study Bias, and Wrong Sample Size Bias.

References:

1. Greenland S, Schwartzbaum JA, Finkle WD. Problems due to small samples and sparse data in conditional logistic regression analysis. Am J Epidemiol. 2000;151(5):531-9. (Link to Reference)

2. Greenland S, Mansournia MA, Altman DG. Sparse data bias: a problem hiding in plain sight. BMJ. 2016;352:i1981. (Link to Reference)