Frontier AI Safety & Governance
A guide to frontier AI safety frameworks, regulation, third-party evaluation, and incident governance, for the compliance, policy, and governance professionals evaluating or adopting them.
Ask CASRAI · free to try
Ask about Frontier AI Safety & Governance
Ask your first 2 questions free below. Subscribers get 150 a day for $29 a month.
Ask CASRAI answers research-administration questions and cites the passages behind every claim. When our sources don't cover a question, it says so.
Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.
Works on this site and inside Claude, Cursor and the AI tools you already use.
Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays free to read, with no account and no card.
Frontier AI developers – Anthropic, OpenAI, Google DeepMind, xAI, Meta – each publish their own safety framework: a document setting out how they test models for dangerous capabilities before release, what thresholds trigger new safeguards, and what commitments they’ve made to evaluate, disclose, and respond to risk. Governments are moving in parallel: California’s SB 53, New York’s RAISE Act, and the EU’s AI Act and GPAI Code of Practice each impose their own transparency and reporting obligations on the same class of models. A growing evaluation ecosystem – METR, the UK AI Security Institute, the US Center for AI Standards and Innovation (CAISI) – sits between the two, testing frontier models independently and publishing what it finds.
This is CASRAI’s guide to that landscape: what each framework actually requires, how they compare to one another, who evaluates against them, and what a compliance, policy, or governance professional needs to know to work with any of them. It complements NIKOLAI, CASRAI’s frontier-AI-safety dictionary of elements, which crosswalks the vocabulary these frameworks use, element by element, against the real primary documents that define it – every reading labelled honestly as CASRAI’s own, never as an endorsement by the organisation it describes.
Safety framework fundamentals
Start here if you’re new to the space: what a Responsible Scaling Policy, Preparedness Framework, or Frontier Safety Framework actually is, the vocabulary they share (capability thresholds, safety cases, dangerous-capability evaluations), and how the major labs’ approaches compare side by side.
Regulation & standards
The binding and voluntary rules layered on top of labs’ own frameworks: California SB 53, New York’s RAISE Act, the EU AI Act and its GPAI Code of Practice, and the management-system standards (NIST AI RMF, ISO/IEC 42001) an organisation can build a compliance program – or certify – against.
Third-party evaluation & assurance
The independent evaluators testing frontier models from outside the labs that build them: METR, the UK AI Security Institute, the US CAISI, and the emerging standards (evaluator independence, embedded vs. arms-length access) that govern how that testing actually works.
Incident reporting & governance
What happens when something goes wrong: statutory incident-reporting deadlines under SB 53 and the RAISE Act, whistleblower protections for AI safety staff, and the internal governance – accountable decision-makers, board sign-off – frontier developers are building to meet these obligations.
Implementation & adoption
For the reader ready to build something: framework templates, AI risk assessment and risk registers, compliance checklists, and practical guidance for standing up an internal AI safety program – the closest content in this hub to what NIKOLAI itself is for.
More guides in this cluster
Showing 25 of 62 guides directly — the rest are organised into the topic hubs above.
AI Governance Best Practices: A Practical Checklist
A capstone checklist for enterprise AI governance best practices: stand up a committee, adopt a framework, document a policy, run risk assessments, build an audit function, and prepare for incident reporting — each step linked to CASRAI’s deep-dive guide.
AI Accountability: Who’s Responsible When an AI System Causes Harm?
What AI accountability means as a general concept, the mechanisms (roles, sign-offs, audit trails, incident-reporting obligations) that implement it, and how SB 53, the NIST AI RMF, and the EU AI Act assign it differently.
Mapping Declarations: How Organizations Verify and Confirm Their Own AI Safety Terminology in NIKOLAI
Every NIKOLAI crosswalk row starts as an unconfirmed shadow mapping. Here’s how an organization verifies its identity and gets a reviewed declaration live.
The International AI Safety Institute Network: What It Is, Now NAAIMES
There’s a coordination body connecting the individual national AI safety institutes — CAISI, UK AISI and others — to each other. It was founded in November 2024, and has since been renamed NAAIMES. Here’s what it actually does.
The AI Incident Database: What It Is and How It Works
What the AI Incident Database is, who runs it (the Responsible AI Collaborative), how entries get in, and why it is a public catalogue of reported AI harms rather than a substitute for SB 53 or RAISE Act statutory incident reporting.
The Frontier Model Forum: What It Is and What It Does
An institutional profile of the Frontier Model Forum (FMF): its founding members, mission, funding, and how it differs from independent evaluators like METR and government bodies like UK AISI and US CAISI.
The SB 53 Material-Change Trigger: When a Frontier AI Framework Must Be Updated
SB 53 requires a large frontier developer to publish its modified Frontier AI Framework, with a justification, within 30 days of a material modification. This guide covers what counts as material and the process for complying.
UK AI Safety Institute vs AI Security Institute: The 2025 Rename Explained
The UK government renamed its AI Safety Institute to the AI Security Institute on 14 February 2025. It is the same organisation under a new name, not two separate bodies — this guide explains what changed, why, and why the old name still turns up.
Building an Internal Audit Function for Frontier AI Safety
An internal AI-safety audit function checks, on a schedule, whether safety commitments are actually being met — distinct from incident response, which reacts when something goes wrong, and from third-party evaluation, which supplies outside credibility.
EU AI Act High-Risk System Compliance Checklist
A practical checklist against the EU AI Act high-risk system deadlines as amended by the 2026 AI Omnibus: Annex III (2 December 2027), Annex I (2 August 2028), and what providers and deployers have to complete before each.
Third-Party AI Evaluator Standards: Independence, Access, and Methodology
A cross-cutting guide to the standards questions that apply to every third-party AI evaluator: embedded vs. arms-length access, independence and conflict-of-interest, evaluation validity, red-team access agreements, publication rights, and retaliation protection — mapped to NIKOLAI’s N8 Transparency and Review track.
Building an AI Safety Framework with NIST’s Govern, Map, Measure, Manage Functions
A walkthrough of NIST AI RMF’s Govern, Map, Measure, and Manage functions as a methodology for building an AI safety framework from scratch, with a practical build sequence and verified subcategory detail.
Accountable Decision-Makers Under SB 53: What the Statute Actually Requires
SB 53 requires internal governance practices around deployment decisions, but the statute itself never names a required accountable-decision-maker role. Here is what it explicitly requires, what is implementation practice, and how NIKOLAIs N9 element proposes to fill the gap.
Mapping Your AI Safety Program to the EU AI Act’s GPAI Code of Practice
Where an existing risk register, governance framework, and incident response program already satisfy the GPAI Code of Practice, and where they need extending.
UK AI Security Institute’s Frontier AI Trends Report: What Its Evaluations Have Found
AISI’s first Frontier AI Trends Report aggregates two years of evaluations across 30+ frontier models. Here is what it found on cyber, bio/chem, autonomy, and safeguards.
What Is a Frontier AI Framework? The SB 53 and RAISE Act Requirement, Explained
A Frontier AI Framework is a specific published document that SB 53 and the RAISE Act require large AI developers to maintain. Here is what each law requires, how the terms lined up in March 2026, and how it relates to a lab’s own voluntary safety policy.
AI Governance Framework Template: Councils, Risk Tiers, and Escalation Paths
The organizational layer an AI governance framework needs above a risk register: a governance council with defined authority, a risk-tiering methodology, escalation paths, and review cadence.
Building an Internal AI Safety Incident Response Program
What an internal AI safety incident response program needs structurally: detection channels, triage against a severity taxonomy, escalation to a named accountable decision-maker, and the SB 53 and RAISE Act external reporting clocks.
New York RAISE Act: What It Requires
New York’s RAISE Act requires large frontier AI developers to publish a Frontier AI Framework and report Critical Safety Incidents to DFS within 72 hours — a much tighter window than California SB 53’s 15 days.
Responsible AI Usage Policy Template: Acceptable-Use and Prohibited-Use Clauses
A practical guide to writing an internal AI acceptable-use policy: what acceptable-use and prohibited-use clauses typically cover, how it differs from a governance framework, and where it fits with SB 53 and EU AI Act obligations.
SB 53 Critical Safety Incident Reporting: What Counts, Deadlines, and Who to Notify
The statutory definition of a reportable incident under California SB 53, the 15-day versus 24-hour reporting clocks, who must be notified, what a compliant report must contain, and how the Attorney General enforces it.
AI Risk Assessment Framework and Risk Register: A Practical Starting Point
What a risk register actually contains (description, likelihood/impact, owner, mitigation, review cadence), how it maps to NIST’s AI RMF Govern-Map-Measure-Manage functions, and a practical starting template.
What Is METR? How Its AI Safety Evaluations Work
An institutional profile of METR (Model Evaluation and Threat Research): what it is, what it evaluates, how its evaluations work, and its relationship to the AI labs it assesses.
Third-Party AI Auditing: What It Is and Who Does It
Third-party AI auditing means an independent party assessing an AI system, or the organization running it, against a compliance framework like ISO/IEC 42001 or the EU AI Act — not the same as using AI tools to automate financial audits.
CAISI and the UK AI Security Institute: How Pre-Deployment Testing Agreements Work
CAISI and the UK AI Security Institute both renamed themselves in 2025 and both run voluntary pre-deployment testing agreements with frontier AI labs. Here’s what those agreements actually cover, and what “early access” means in practice.
