Negligible effect (equivalence) testing involves assessing whether an effect of interest is small enough that it can be considered negligible. For example:
- Is the difference in the means of two groups so small that it can be considered negligible?
- Is the correlation between two variables so small that it can be considered negligible?
- Is a regression coefficient so small that it can be considered negligible?
- Is the difference between an SEM fit index value (e.g., RMSEA) and the value representing perfect fit (e.g., RMSEA = 0) negligible?
- Is the difference between a target distribution (e.g., distribution of residuals) and a theoretical distribution (e.g., normal distribution) negligible?
There are numerous applications of negligible effect testing within the field of psychology and elsewhere.
Negligible effect analyses can be carried out via the negligible R package. See here for more details.
The following are the most important steps in writing up the results of analyses based on negligible effect testing:
- Briefly introduce why a negligible effect test was chosen as the appropriate statistical method.
- Describe the negligible effect test/approach used (e.g., two one-sided tests (TOST) for negligible mean difference, confidence interval approach for negligible correlation, etc.).
- Specify the negligible effect interval chosen and the rationale behind it.
- Present the findings of the negligible effect test.
- Report the point estimate of the effect size (e.g., mean difference or correlation coefficient) along with its confidence interval (in many cases, an effect size that incorporates the negligible effect interval is also useful, see the proportional distance)
- Interpret the results within the context of the research question and the chosen equivalence bounds
Some examples of papers that have use negligible effect tests are (in progess):
Regression Setting
Alyssa A. Di Bartolomeo, Udi Alter, David A. Olson, Max B. Cooper, Tali
Boritz & Henny A. Westra (14 Nov 2023): Predicting resistance management skill from
psychotherapy experience, intellectual humility and emotion regulation, Psychotherapy
Research, DOI: https://doi.org/10.1080/10503307.2023.2280240
Paired Samples Mean Equivalence
Riesthuis, P., Woods, J. “That’s just like, your opinion, man”: the illusory truth effect on opinions. Psychological Research 88, 284–306 (2024). https://doi.org/10.1007/s00426-023-01845-5
