Submit Your Toughest Questions for Humanity's Last Exam
Benchmark saturation is a real measurement challenge for AI governance - understanding capability ceilings matters for risk assessment, but this item is dated.
Key points
- CAIS and Scale AI are crowdsourcing expert-level questions to build a frontier AI capability benchmark called Humanity's Last Exam.
- The project addresses benchmark saturation - top AI models now near-ceiling existing tests like MMLU.
- This item is a call for submissions with a November 2024 deadline - likely already closed, limiting immediate relevance.
Implications for Australian agencies
- Monitor APS analysts tracking AI capability assessment may want to monitor the published results of Humanity's Last Exam as a signal of where frontier models currently sit relative to expert-level performance.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
Appeared in:
Weekly digest, 4 May 2026
"Submit Your Toughest Questions for Humanity's Last Exam"
Source: Centre for AI Safety – Blog
Published: (undated)
URL: https://safe.ai/blog/humanitys-last-exam
The Centre for AI Safety (CAIS) and Scale AI launched Humanity's Last Exam, a crowdsourced initiative to build a more difficult public AI benchmark by gathering expert-level questions across disciplines. The motivation is that frontier AI models have saturated existing benchmarks like MMLU, making it harder to track capability development and distance from expert-level performance. Contributors could earn co-authorship and prizes from a $500,000 pool. The submission deadline was November 1, 2024, making this item historical rather than actionable.
Implications for Australian agencies:
- [Monitor] APS analysts tracking AI capability assessment may want to monitor the published results of Humanity's Last Exam as a signal of where frontier models currently sit relative to expert-level performance.
Retrieved from SIMS, 18 July 2026.