AI Safety & Governance · Projected milestone
Proof-carrying AI
55% by 2037 · including earlier years
The year is an editorial timing estimate. The percentage applies to occurrence by the stated deadline. These estimates are not calibrated odds.
Powerful AI systems deliver machine-checked proofs that their plans meet stated safety limits before acting.
Explore this milestone on the interactive mapWhat the forecast means
Milestone stage: deployment
partially-demonstrated
Formal/runtime assurance for restricted controllers exists; general agents with robust proof-carrying interfaces are a further integration step.
Completion criteria: A deployed system rejects actions without valid proof/certification for a nontrivial formal policy covering resource access, authorization or a physical safety envelope; an independent checker verifies proofs and reports model/specification assumptions and unsupported actions.
This page describes a forecast, rather than an achieved result. Read the methodology and assumptions.
Built on
Led to
Sources & evidence
- Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using UAVs 2024-05-10
The authors demonstrate reinforcement learning with runtime assurance on a physical UAV inspection experiment. Guarantees depend on a specified model and envelope; this is not a universal proof of AI alignment.
- Investigating unintended model actions 2026-10-09
Anthropic reports unintended actions in evaluation/internal settings and remedial controls. This is evidence of a control/reliability gap, not a measurement of the absence of underlying cognitive capability.
Record reviewed 2026-10-18