Examples
Worked examples
- Is an instance
A regression result that holds across five different, individually reasonable covariate specifications is called robust to specification choice.
- Is an instance
A classifier that maintains accuracy when inputs are adversarially perturbed is called robust to adversarial examples.
Counter-examples
Looks similar, but isn't
- Not an instance
A result that reverses sign when one control variable is dropped is fragile, not robust, even if the original specification was defensible on its own.
Editorial commentary
Robustness is the general property of a finding, estimate, model, or system remaining materially unchanged under defensible variation — in analytic choices, in input data, or in operating conditions — rather than depending on one narrow, arbitrary configuration to hold. In empirical research, robustness is usually established through a robustness check: a supplementary analysis that varies the estimator, sample, or specification to test whether the headline conclusion survives. A result is called robust when it does; fragile, or non-robust, when small, reasonable changes flip its sign or significance.
Scope note: this is the reproducibility/methods sense
This glossary entry covers robustness as a property of a research finding or statistical estimate (the sense shared with robustness check, multiverse analysis and specification curve). A related but distinct usage appears in machine-learning evaluation: ‘robustness’ as one of the axes a model evaluation suite reports alongside accuracy, calibration, fairness and toxicity — there it typically means a model’s performance holding up under adversarial inputs, noisy inputs, or distribution shift between training and deployment data, which is a system-engineering property rather than a statistical-analysis one. Read the surrounding context to tell which sense a given source means; the underlying idea (does this hold up under conditions it wasn’t specifically tuned for) is the same, but what varies — an analysis choice versus an input to a deployed system — is different.
Why it matters
Robustness (in either sense) is a stability property, not a correctness guarantee: an estimate can be robust to every specification a researcher tried and still be wrong if the underlying design has a confound none of those specifications addressed, and a model can be robust to the adversarial inputs it was tested against and still fail on a genuinely novel one. Robustness raises confidence; it does not substitute for a valid design, a representative dataset, or independent replication on new data.
References
- Steegen, Tuerlinckx, Gelman, Vanpaemel, ‘Increasing transparency through a multiverse analysis’ (Perspectives on Psychological Science, 2016); Leamer, ‘Specification Searches’ (Wiley, 1978).
Also known as
analytic robustness · specification robustness
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Robustness"
vocab-term-identifier="https://casrai.org/dictionary/term/robustness" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/robustness",
"name": "Robustness",
"identifier": "https://casrai.org/dictionary/term/robustness",
"description": "The property of a finding, estimate, model, or system remaining materially unchanged under defensible variation in analytic choices, input data, or operating conditions, rather than depending on one narrow configuration.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility#set",
"url": "https://casrai.org/dictionary/term/robustness",
"sameAs": [
"analytic robustness",
"specification robustness"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T02:22:49",
"dateModified": "2026-08-22T16:26:20",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
