SemiAnalysis: GPT-6 Astra confirmed to use looped transformers
SemiAnalysis says parameter sizes are no longer growing sharply and that GPT-6 Astra is confirmed to use looped transformers, with labs' demand for low-capacity memory as a signal of the trend.
SemiAnalysis: the loudest demand for low-capacity memory is coming from the labs themselves, a tell that they don't see model parameters scaling much further "What we're observing is parameter sizes aren't really increasing that much. There are so many techniques trying to keep that footprint small." "Looped transformers is a recent one, and it's basically confirmed that GPT-6 Astra uses looped transformers..." "And the loudest cries for four-high are coming from the labs, and the labs are probably the people best positioned to say which way models are scaling." "And that's also a tell that they're not seeing parameter sizes scaling as aggressively in their roadmaps."
This post quotes another post: x.com/firesidealpha/status/2099288315489140837