These startups are chasing the next big thing in LLMs
Emerging LLM architectures promising large efficiency gains could reshape the cost and capability assumptions underpinning agency AI business cases.
Key points
- Diffusion LLMs generate whole text blocks simultaneously, claiming 10x speed and cost gains over standard transformers.
- Google's Diffusion Gemma prototype suggests major labs are validating this architectural direction alongside startups.
- Limited direct governance or procurement implications for APS agencies at this stage - primarily a technology watch item.
Implications for Australian agencies
- Monitor Technology and procurement teams may want to monitor whether diffusion LLM efficiency claims mature into commercially available models that could affect agency AI cost modelling.
Implications are AI-generated. Starting points, not advice — see methodology for how they're framed.
View original source
Copied.
"These startups are chasing the next big thing in LLMs"
Source: MIT Technology Review – AI
Published: 10 August 2026
URL: https://www.technologyreview.com/2026/08/10/1141511/these-startups-are-chasing-the-next-big-thing-in-llms/
MIT Technology Review profiles startups pursuing alternative large language model architectures beyond the dominant autoregressive transformer approach. Inception's diffusion-based LLMs claim performance comparable to GPT-4-era models at 10x the speed and lower cost, by generating text blocks in parallel rather than token by token. Google is developing a similar prototype called Diffusion Gemma. Separately, Pathway's Dragon Hatchling model attempts to move beyond language-based reasoning entirely, achieving strong results on structured logic benchmarks where leading models fail. Both approaches challenge current assumptions about LLM cost, speed, and capability ceilings.
Implications for Australian agencies:
- [Monitor] Technology and procurement teams may want to monitor whether diffusion LLM efficiency claims mature into commercially available models that could affect agency AI cost modelling.
Retrieved from SIMS, 16 September 2026.