Examples
Worked examples
- Is an instance
A model satisfying demographic parity (equal selection rates across groups) while failing calibration (predicted scores meaning different things for different groups), because the two criteria are mathematically incompatible under differing base rates
- Is an instance
An institution explicitly choosing and documenting which fairness criterion a deployed AI system is being evaluated against, rather than claiming the system is simply 'fair' without specifying by which definition
Counter-examples
Looks similar, but isn't
- Not an instance
A single accuracy or error-rate statistic reported without reference to any group is a bias measurement, not a fairness claim -- fairness requires specifying which groups and which normative criterion
- Not an instance
Claiming a system is 'unbiased' because training data was balanced across groups does not, by itself, establish that the system satisfies any particular fairness criterion on its outputs
Editorial commentary
AI fairness is not a solved engineering problem with one correct metric — it is an actively contested normative question, and presenting it as a single optimisable target misrepresents the state of the field. This is the single most important thing to understand about this entry.
The impossibility result
A now-foundational body of work in the fairness literature (Kleinberg, Mullainathan and Raghavan; Chouldechova, both circa 2016-2017, working on risk-scoring contexts) established formally that several intuitively reasonable fairness criteria are mutually incompatible except in special cases — specifically, whenever the base rate of the outcome being predicted differs across the groups being compared, a classifier generally cannot simultaneously achieve demographic parity (equal positive prediction rates across groups), equalised odds (equal true- and false-positive rates across groups), and predictive parity or calibration (a given score meaning the same thing across groups). Choosing which criterion to prioritise is therefore a value judgment about what kind of equal treatment matters most in a given context, not a technical detail to be optimised away.
NIST’s framing
The NIST AI Risk Management Framework treats fairness this way explicitly: as something requiring context-specific negotiation among affected stakeholders about which disparities are acceptable, rather than a property a system either has or lacks in the abstract. NIST’s own framework instead centres on characterising and managing bias — a more tractable, descriptive concept — as the input that any fairness judgment then has to weigh.
How this differs from AI bias
This is the core distinction for the whole fairness band. Bias is descriptive and empirical: a measurable property of a system’s actual outputs (does this system’s error rate or output distribution differ by group, and by how much). Fairness is normative and contested: a judgment about which of several incompatible criteria for acceptable treatment should apply, and how much disparity is tolerable, given the system’s actual use context. A system can be accurately described as having a specific, measured bias, while reasonable, informed people can still disagree about whether that makes it ‘unfair’ — because they may be applying different, individually defensible fairness criteria. Never present a fairness claim without specifying which criterion is being used.
Practical implication
A meaningful fairness claim about an AI system names the specific criterion being evaluated (e.g. ‘equalised odds across group X and Y for outcome Z’), reports the actual disparity found, and states explicitly why that criterion, rather than an alternative, was judged appropriate for the context — not a bare assertion that a system ‘is fair.’
Also known as
Algorithmic fairness · ML fairness
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="AI fairness"
vocab-term-identifier="https://casrai.org/dictionary/term/ai-fairness" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/ai-fairness",
"name": "AI fairness",
"identifier": "https://casrai.org/dictionary/term/ai-fairness",
"description": "A contested normative criterion for evaluating whether an AI system's treatment of different groups or individuals is acceptable -- not a single measurable property, but a family of mutually incompatible formal definitions (e.g. demographic parity, equalised odds, predictive parity/calibration) among which a system generally cannot satisfy more than one simultaneously whenever base rates differ across groups.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/genai-disclosure#set",
"url": "https://casrai.org/dictionary/term/ai-fairness",
"sameAs": [
"Algorithmic fairness",
"ML fairness"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T01:55:45",
"dateModified": "2026-08-22T14:52:07",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
