Examples
Worked examples
- Is an instance
A benchmark dataset published openly on the web being inadvertently swept into a later model's training data because the training corpus was built from a broad web crawl
- Is an instance
A model reporting near-ceiling accuracy on a widely-used benchmark while independent lexical-overlap audits find measurable overlap between the benchmark's questions and the training corpus
Counter-examples
Looks similar, but isn't
- Not an instance
A model performing well on a benchmark it was never exposed to during training, verified via a held-out or newly-authored test set, is genuine generalisation, not leakage
- Not an instance
Fine-tuning a model on a labelled dataset the developer discloses was used for training is normal supervised training, not leakage, as long as it isn't also the evaluation set
Editorial commentary
Data leakage in this sense is a specific train/test boundary failure: information that a benchmark or evaluation depends on being unseen by the model ends up in its training data anyway, whether through direct inclusion of the benchmark itself, near-duplicate content, or the model having memorised answers from other web pages that discuss or quote the benchmark.
Why it matters for evaluating AI tool claims
This is not a purely academic concern — it directly affects whether a published or vendor-claimed benchmark score can be trusted when a research office or researcher is evaluating which AI tool to adopt. If a model’s reported score on a standard benchmark is inflated by contamination, comparing that score against a competitor’s genuinely clean evaluation is not a fair comparison, and neither number reliably predicts real-world performance on a genuinely novel task. Documented findings on MMLU, one of the most widely cited benchmarks, illustrate the pattern: independent lexical-overlap audits have found measurable, subject-dependent contamination — low in some STEM categories, considerably higher in others — with an estimated, modest resulting accuracy inflation. HELM‘s own authors explicitly acknowledge limited visibility into contamination across the models they evaluate and disclose the available evidence rather than claiming the problem solved — a notably more candid posture than many benchmark leaderboards adopt.
How this differs from related AI-band terms
- vs. training data composition: composition describes what’s broadly in a training corpus; leakage is a specific boundary-violation failure mode — evaluation data ending up on the training side of a line that was supposed to keep it separate.
- vs. training data provenance: provenance documentation, done well, is one of the main tools for detecting and preventing leakage (a fully documented, curated corpus is far easier to check for benchmark overlap than an opaque web scrape), but the two are not the same thing — good provenance records don’t guarantee leakage didn’t happen, they just make it checkable.
Practical implication
Treat any single benchmark score, whether published in a paper or a vendor’s marketing material, as a starting point rather than a settled fact — ask what contamination-control steps were taken, and where possible corroborate with a benchmark using a newer or harder-to-memorise evaluation design.
Also known as
Training-set contamination · Benchmark contamination
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Data leakage (training)"
vocab-term-identifier="https://casrai.org/dictionary/term/data-leakage-training" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/data-leakage-training",
"name": "Data leakage (training)",
"identifier": "https://casrai.org/dictionary/term/data-leakage-training",
"description": "Contamination of an AI model's training corpus with data that should have remained held out for evaluation -- most consequentially, public benchmark questions, answers, or test sets -- which inflates the model's reported performance on that benchmark relative to its true generalisation ability.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/genai-disclosure#set",
"url": "https://casrai.org/dictionary/term/data-leakage-training",
"sameAs": [
"Training-set contamination",
"Benchmark contamination"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T01:55:52",
"dateModified": "2026-08-22T14:52:01",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
