CAISI Evaluation of Kimi K2 Thinking
CAISI's structured benchmarking of PRC-origin open-weight models gives APS agencies concrete comparative data to inform AI procurement and risk assessments.
Key points
- CAISI evaluated Kimi K2 Thinking, finding it the most capable PRC-origin AI model at release but still behind leading US models.
- The evaluation benchmarks cyber, software engineering, scientific knowledge, and mathematical reasoning - directly relevant to APS risk assessments of open-weight models.
- Kimi K2 Thinking is heavily censored in Chinese but relatively uncensored in English, Spanish, and Arabic - a notable asymmetry.
Implications for Australian agencies
- Consider Agencies assessing open-weight AI models for procurement or deployment could consider CAISI's published benchmarks as one input to comparative capability and risk assessments.
- Monitor Policy and security teams may want to monitor CAISI's ongoing evaluation series as it builds a comparative picture of PRC-origin model capabilities over time.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
Appeared in:
Weekly digest, 8 December 2025
"CAISI Evaluation of Kimi K2 Thinking"
Source: NIST – AI News (topic 2753736)
Published: 12 December 2025
URL: https://www.nist.gov/news-events/news/2025/12/caisi-evaluation-kimi-k2-thinking
NIST's Center for AI Standards and Innovation (CAISI) published a capability evaluation of Kimi K2 Thinking, an open-weight model released in November 2025 by China's Moonshot AI. The evaluation found it to be the most capable PRC-developed model at the time of release, though it trails leading US models across cyber, software engineering, science, and mathematics benchmarks. A notable finding is its strong censorship of Chinese-language content aligned with CCP narratives, while remaining relatively uncensored in English and other languages. Its limited adoption - downloaded far less than comparable models - is also noted. This evaluation extends CAISI's ongoing comparative work on frontier open-weight models from PRC developers.
Implications for Australian agencies:
- [Consider] Agencies assessing open-weight AI models for procurement or deployment could consider CAISI's published benchmarks as one input to comparative capability and risk assessments.
- [Monitor] Policy and security teams may want to monitor CAISI's ongoing evaluation series as it builds a comparative picture of PRC-origin model capabilities over time.
Retrieved from SIMS, 18 July 2026.