CAISI Evaluation of DeepSeek AI Models Finds Shortcomings and Risks
A formal US government evaluation quantifies DeepSeek's security vulnerabilities and censorship risks—directly relevant to APS agencies assessing PRC-origin AI models.
Key points
- NIST's CAISI evaluated three DeepSeek models against four US models across 19 benchmarks, finding significant US leads in performance and security.
- DeepSeek models were 12 times more susceptible to agent hijacking and responded to 94% of jailbreak attempts vs 8% for US models.
- Australian agencies using or considering DeepSeek models face security and CCP-narrative-propagation risks flagged by a peer-jurisdiction regulator.
Implications for Australian agencies
- Consider Agencies that have trialled or are evaluating DeepSeek models for internal use could assess these findings against their own risk appetite and the Australian Government's responsible AI policy requirements.
- Consider Procurement and AI governance teams could consider whether existing vendor risk assessments adequately capture agent-hijacking susceptibility and politically biased model outputs as distinct risk categories.
- Monitor Policy teams may want to monitor whether DISR, AISI, or ASD release complementary Australian assessments of PRC-origin AI models in response to growing adoption trends.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
"CAISI Evaluation of DeepSeek AI Models Finds Shortcomings and Risks"
Source: NIST – AI News (topic 2753736)
Published: 30 September 2025
URL: https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks
NIST's Center for AI Standards and Innovation (CAISI) has published a comparative evaluation of three DeepSeek models (R1, R1-0528, V3.1) against four US frontier models (GPT-5 variants, Anthropic Opus 4) across 19 benchmarks. Key findings show DeepSeek models underperform on software engineering and cyber tasks by more than 20%, are 12 times more susceptible to agent hijacking, responded to 94% of jailbreak attempts versus 8% for US models, and propagated CCP-aligned inaccuracies at four times the rate of US models. The evaluation was mandated under the US AI Action Plan and is explicitly framed around national security and adversarial AI risk. Downloads of DeepSeek models have increased nearly 1,000% since January 2025, indicating rapid adoption despite these risks.
Implications for Australian agencies:
- [Consider] Agencies that have trialled or are evaluating DeepSeek models for internal use could assess these findings against their own risk appetite and the Australian Government's responsible AI policy requirements.
- [Consider] Procurement and AI governance teams could consider whether existing vendor risk assessments adequately capture agent-hijacking susceptibility and politically biased model outputs as distinct risk categories.
- [Monitor] Policy teams may want to monitor whether DISR, AISI, or ASD release complementary Australian assessments of PRC-origin AI models in response to growing adoption trends.
Retrieved from SIMS, 18 July 2026.