Model Evaluation for Extreme Risks
Agencies developing AI risk frameworks can draw on this taxonomy of dangerous capabilities to stress-test their own assessment approaches.
Key points
- A 2023 paper proposes embedding model evaluation for dangerous capabilities and alignment into AI governance processes.
- Nine dangerous capability categories are identified, including cyber-offense, deception, self-proliferation, and situational awareness.
- MIT AI Risk Repository surfaces this as one of 25 risk frameworks - useful reference material for agencies building AI risk taxonomies.
Implications for Australian agencies
- Consider Agencies developing AI risk assessment or procurement evaluation frameworks may want to consult this taxonomy of dangerous capabilities as a reference point.
- Monitor Risk and assurance teams could monitor the MIT AI Risk Repository as it catalogues further frameworks, given its utility as a curated evidence base for AI governance work.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
Appeared in:
Weekly digest, 2 February 2026
"Model Evaluation for Extreme Risks"
Source: MIT AI Risk Repository – Blog
Published: 6 February 2026
URL: https://airisk.mit.edu/blog/model-evaluation-for-extreme-risks
The MIT AI Risk Repository spotlights a 2023 paper by Shevlane et al. proposing that model evaluation could address extreme risks from general-purpose AI by assessing both dangerous capabilities and model alignment. The paper identifies nine dangerous capability categories - including cyber-offense, deception, persuasion, political strategy, and self-proliferation - and outlines how such evaluations could be embedded in AI safety and governance processes. The framework focuses on misuse and misalignment risks rather than structural or competence-related risks. It is one of 25 risk frameworks catalogued in the Repository, making the Repository itself a useful reference for agencies building or reviewing AI risk taxonomies.
Implications for Australian agencies:
- [Consider] Agencies developing AI risk assessment or procurement evaluation frameworks may want to consult this taxonomy of dangerous capabilities as a reference point.
- [Monitor] Risk and assurance teams could monitor the MIT AI Risk Repository as it catalogues further frameworks, given its utility as a curated evidence base for AI governance work.
Retrieved from SIMS, 18 July 2026.