New AI Now Paper Highlights Risks of Commercial AI Used In Military Contexts
Raises concrete questions about the security boundary between commercial AI systems and sensitive government applications - relevant to APS risk and procurement thinking.
Key points
- AI Now paper argues commercial foundation models integrated into military targeting systems pose underappreciated national security risks.
- Systems like Gospel and Lavender, deployed in Gaza, illustrate risks of personal data exfiltration and adversarial exploitation in military AI.
- Recommendations focus on insulating military AI from commercial foundation models - not a direct APS procurement or governance mandate.
Implications for Australian agencies
- Monitor Defence, intelligence, and national security policy teams may want to monitor this paper as a framing contribution to debates about AI system separation and data security in sensitive government contexts.
- Consider Agencies developing risk frameworks for AI procurement in sensitive or high-stakes contexts could consider whether the paper's arguments about commercial-military AI separation have parallels in civilian government high-sensitivity deployments.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
"New AI Now Paper Highlights Risks of Commercial AI Used In Military Contexts"
Source: AI Now Institute – Publications
Published: 22 October 2024
URL: https://ainowinstitute.org/publications/new-ai-now-paper-highlights-risks-of-commercial-ai-used-in-military-contexts
An AI Now Institute paper argues that national security AI risk discourse has over-indexed on CBRN proliferation while underweighting dangers from AI systems already deployed in military intelligence, surveillance, and targeting. It examines real-world systems including Gospel, Lavender, and Where's Daddy, used in Gaza, noting their reliance on personal data and vulnerability to adversarial exploitation. The paper warns that integrating commercial foundation models into these systems amplifies existing risks and concludes that insulating military AI from commercial AI infrastructure - and limiting personal data exposure within commercial models - are necessary mitigations. It also critiques compute thresholds and export controls as insufficient policy responses.
Implications for Australian agencies:
- [Monitor] Defence, intelligence, and national security policy teams may want to monitor this paper as a framing contribution to debates about AI system separation and data security in sensitive government contexts.
- [Consider] Agencies developing risk frameworks for AI procurement in sensitive or high-stakes contexts could consider whether the paper's arguments about commercial-military AI separation have parallels in civilian government high-sensitivity deployments.
Retrieved from SIMS, 18 July 2026.