On the Existence and Behavior of Secondary Attention Sinks
arXiv:2512.22213v2 Announce Type: replace Abstract: Attention sinks are tokens, often the beginning-of-sequence (BOS) token, that receive disproportionately high attention despite limited semantic relevance. In this work, we identify a class of attention sinks, which we term secondary sinks, that differ…
