How to Grade a Frontier AI Safety Framework: The SaferAI Rubric

Frontier AI labs now publish safety frameworks — Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, Google DeepMind’s Frontier Safety Framework, and similar documents from a dozen other companies. Publishing a framework is easy. Judging whether it is any good is harder, because there is no shared scale for “good.” SaferAI, a Paris-based nonprofit research and policy organization, built one. Its Frontier Risk Management Tracker independently scores each lab’s published risk-management practices against a fixed rubric, so frameworks can be compared on the same terms rather than taken at face value.

This guide explains what SaferAI is, how its rubric actually works, and what its most recent scores say about the labs already covered in this cluster.

What SaferAI is

SaferAI is a nonprofit that works across research and policy to make AI development safer. It pursues that through three lines of work: building quantitative risk models for AI (drawing on methods from aviation and finance), publishing independent ratings of AI developers’ risk-management practices through its Tracker, and contributing to AI risk-management standards, including work on EU standards and international red-teaming guidance. Its evaluations and commentary have been covered by outlets including TIME and the Financial Times.

The Tracker is the part relevant here: a standing, periodically updated scorecard that grades named AI companies’ published safety frameworks, not just their public statements about safety.

What the rubric actually grades

SaferAI’s methodology scores each company across four equally weighted dimensions, each worth 25% of the overall score:

• Risk identification — whether the company systematically identifies known risk domains (cyber, chemical/biological/nuclear, manipulation, autonomous R&D, loss of control), runs open-ended red-teaming to catch novel risks, and builds structured risk models (event trees, fault trees) with severity and probability estimates.
• Risk analysis and evaluation — whether the company sets an explicit risk tolerance per domain and operationalizes it with measurable key risk indicators (KRIs) and key control indicators (KCIs), rather than leaving “acceptable risk” undefined.
• Risk treatment — whether mitigations are actually implemented (containment, deployment safeguards, assurance evidence) and continuously monitored once a system ships, not just described as a future commitment.
• Risk governance — the organizational infrastructure behind all of the above: clear risk ownership and escalation paths, an advisory or challenge function, internal and external audit, board-level oversight, safety culture, and public transparency.

Each dimension breaks down into weighted sub-criteria, and each criterion is scored on a defined 0–100% scale (0% = not mentioned, 25% = partially addressed, 50% = moderately addressed, 75% = well addressed with minor gaps, up to 100% = exemplary, best-practice implementation). Scores are published with a rationale and direct quotes from the source framework, not just a number, and SaferAI documents specific improvement recommendations alongside each score. Where a company can demonstrate a risk is controlled through more than one credible route — for example a firm pre-commitment plus independent third-party verification — the methodology takes the stronger of the two rather than penalizing the approach for redundancy.

This is a narrower question than “is this company safe.” The rubric grades the completeness and rigor of a company’s documented risk-management process, using its own published framework as the primary evidence. A lab can score well on governance structure while still facing risks the rubric does not directly measure, and a low score does not mean a lab is reckless — it means specific, named gaps exist in what the company has published about how it identifies, measures, and controls risk.

How the labs already covered in this cluster score

As of SaferAI’s July 2026 Tracker update, the highest-scoring companies were Anthropic at 35%, OpenAI at 34%, and Google DeepMind at 20%, out of the 12 frontier developers SaferAI currently assesses. SaferAI notes that even the top score is well short of what it considers adequate: it estimates the achievable ceiling, if every company adopted current industry best practice on every criterion, at roughly 59%.

Two things are worth noting about how to read these numbers. First, they move: SaferAI re-scores companies as frameworks are revised, and scores have gone down as well as up when a lab narrowed a prior commitment. Second, the percentage is a single weighted rollup of dozens of criteria across four dimensions — two labs with similar overall scores can still differ sharply on, say, risk governance versus risk treatment. Readers who want the current, exact figures and the underlying rationale should treat SaferAI’s own Tracker as the source of record rather than any static snapshot, including this one, since the numbers are updated on an ongoing basis.

How this complements comparing the frameworks directly

SaferAI’s rubric is one lens. It is not a substitute for reading what the frameworks actually commit labs to, and it is not the same exercise as lining the frameworks up side by side. For the latter, see CASRAI’s comparison of the RSP, Preparedness Framework, and Frontier Safety Framework, which sets out what each document actually says. For background on how Anthropic’s Responsible Scaling Policy works specifically, see Responsible Scaling Policy, explained.

The two views are complementary: a side-by-side comparison tells you what a framework says it will do; a rubric like SaferAI’s tells you how completely and rigorously that framework addresses the standard set of questions a risk-management process should answer, regardless of which specific commitments each lab chose to make.

How to use a rubric like this without over-reading it

A few practical points for treating SaferAI’s ratings as evidence rather than as a headline:

• Check the assessment date. Frameworks change; a score reflects the framework version SaferAI reviewed at that point, not necessarily the version currently published.
• Read the criterion-level detail, not just the overall percentage. A company can be strong on risk identification and weak on governance (or vice versa) and that difference matters more than the rollup number.
• Treat it as one input. SaferAI’s Tracker is one independent assessment among several efforts now grading frontier-lab safety practices (for example, the Future of Life Institute’s AI Safety Index). They use different methodologies and can weight the same underlying facts differently — comparing more than one is more informative than relying on a single rating.
• Distinguish “unrated” from “safe.” Several companies that pledged at the 2024 AI Seoul Summit to publish a safety framework by February 2025 had, as of SaferAI’s assessment, still not done so. A missing framework is a governance gap in itself, separate from how any published framework scores.

Frequently asked questions

Is SaferAI affiliated with any of the AI labs it rates?

No. SaferAI is an independent nonprofit; its Tracker assessments are conducted separately from the companies being scored, using each company’s own published materials as source evidence.

Does a higher SaferAI score mean a lab’s AI systems are safer to use?

Not directly. The score measures the maturity and completeness of a company’s documented risk-management process — how it identifies, analyzes, and mitigates risk, and how that work is governed — not a direct measurement of model behavior or real-world incident rates.

How often does SaferAI update its ratings?

SaferAI re-assesses companies periodically as frameworks are published or revised; its Tracker states the date it was last updated as of any given visit, and scores can move in either direction between updates.

What is the difference between SaferAI’s Tracker and the Future of Life Institute’s AI Safety Index?

Both independently grade frontier AI companies on safety-related practice, but they are separate organizations running separate methodologies and criteria, and their scores are not directly interchangeable. Consulting both gives a broader picture than relying on either alone.

Where can I see the current, exact scores?

SaferAI publishes live scores and full criterion-by-criterion rationale on its own Tracker site. Because scores are updated on an ongoing basis, that is the authoritative source for current figures rather than any fixed summary.

Related on CASRAI

This guide is part of CASRAI’s frontier AI safety and governance cluster. For the elements that structure how organizations document AI system provenance and safety claims, see NIKOLAI. For the specific frameworks referenced above, see Responsible Scaling Policy, explained and the RSP vs. Preparedness Framework vs. Frontier Safety Framework comparison.