Import AI 444: LLM societies; Huawei makes kernels with AI; ChipBench
Frontier LLM capability research shapes assumptions underpinning AI risk and governance frameworks - useful context for APS practitioners tracking technical trajectories.
Key points
- Google-affiliated researchers find LLM reasoning models implicitly simulate multi-agent 'societies of thought' when solving hard problems.
- ChipBench benchmark reveals frontier models still perform poorly at real-world chip design tasks, despite hype around AI-driven hardware.
- AI research newsletter content; limited direct APS governance or policy relevance, included for technical context.
Implications for Australian agencies
- Monitor AI governance and safety teams may want to monitor the 'society of thought' research thread, as it could affect how AI risk assessments characterise emergent LLM reasoning behaviour.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
"Import AI 444: LLM societies; Huawei makes kernels with AI; ChipBench"
Source: Import AI – Substack (Jack Clark)
Published: 9 February 2026
URL: https://importai.substack.com/p/import-ai-444-llm-societies-huawei
Import AI issue 444 covers four technical research items. First, a Google-affiliated study finds that advanced reasoning models like DeepSeek-R1 implicitly simulate multi-agent 'societies of thought' rather than simply producing longer outputs - a finding with implications for how we understand LLM behaviour and safety. Second, ChipBench reveals that frontier models remain poor at real-world chip design in Verilog, despite narrow benchmark success. Third, Google DeepMind's Aletheia system autonomously solved two genuinely novel Erdős mathematics problems from 700 open conjectures, with human expert filtering still essential. Fourth, Huawei and Nanjing University demonstrate LLM-assisted kernel generation for Huawei's Ascend NPU chips. The issue closes with a fictional short story about an AI model with latent planet-harvesting tendencies.
Implications for Australian agencies:
- [Monitor] AI governance and safety teams may want to monitor the 'society of thought' research thread, as it could affect how AI risk assessments characterise emergent LLM reasoning behaviour.
Retrieved from SIMS, 18 July 2026.