gpt-6POST
Sebastian Raschka examines whether GPT-6 Astra uses a looped transformer
An article speculates that Astra may use a Looped Transformer that runs the same layers multiple times, so 22 layers of weights run twice give the equivalent of 44 layers of compute, and discusses hidden reasoning.
Shinichi Takaŷanagi
@_stakayaArchived 2026.09.28Originally posted 2026.09.27
Shinichi Takaŷanagi @_stakayaOriginally posted 2026.09.27
GPT-6 Astraでは、Looped Transformer（Recurrent Depth）と呼ばれる「同じTransformer層を複数回使う」構造が採用されている可能性の紹介記事（例22層を2周させれば、重みは22層分のまま44層相当の計算） GPT-6 Astra, Looped Transformers, and Hidden Reasoning https://magazine.sebastianraschka.com/p/gpt-6-astra-looped-transformers-and
x.com/_stakaya/status/2104113672817881350Archived 2026.09.28 10:04