Survey notes on residual connection improvements, with English version translated by GPT-6 Astra
The author compiled survey notes on improvements to Transformer residual connections, released as Chinese and English PDFs, with the English version translated by GPT-6 Astra.
Residual connections can be considered one of the great milestones that made the “deep” in deep learning possible. In recent years, after improving Transformer architectures in just about every other way, researchers have finally turned their attention to residual connections, giving rise to many interesting modifications. So I put together this brief blog. There may well be omissions or even mistakes, and any corrections or additions are very welcome! The English version was translated from the original Chinese by GPT-6 Astra. If you can read Chinese, I recommend reading the original notes. PDF: [Chinese (CN)](https://github.com/Purshow/Purshow_Notes/blob/main/Beyond_Residual/CN/Beyond_Residual_CN.pdf) [English (EN)](https://github.com/Purshow/Purshow_Notes/blob/main/Beyond_Residual/EN/Beyond_Residual_EN.pdf)