Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
arXiv:2510.05901v2 Announce Type: replace Abstract: Transformers’ quadratic computational complexity limits their scalability despite remarkable performance. While linear attention reduces this to linear complexity, pre-training such models from scratch remains, in most cases, prohibitively expensive. Recent post-training linearisation methods convert pre-trained…
