MIT Technology Review
Startups Race to Replace Transformers, the Tech That Made LLMs Possible (and Terribly Expensive)
Four startups are trying to replace the transformer - the tech behind every major LLM - with sparse attention, power retention, worm-brain-inspired networks, and diffusion. Because why not?