Archives AI News

Probing Length Generalization in Mamba via Image Reconstruction

arXiv:2603.12499v1 Announce Type: new Abstract: Mamba has attracted widespread interest as a general-purpose sequence model due to its low computational complexity and competitive performance relative to transformers. However, its performance can degrade when inference sequence lengths exceed those seen during…

Byzantine-Robust Optimization under $(L_0, L_1)$-Smoothness

arXiv:2603.12512v1 Announce Type: new Abstract: We consider distributed optimization under Byzantine attacks in the presence of $(L_0,L_1)$-smoothness, a generalization of standard $L$-smoothness that captures functions with state-dependent gradient Lipschitz constants. We propose Byz-NSGDM, a normalized stochastic gradient descent method with…

Rethinking Attention: Polynomial Alternatives to Softmax in Transformers

arXiv:2410.18613v3 Announce Type: replace Abstract: This paper questions whether the strong performance of softmax attention in transformers stems from producing a probability distribution over inputs. Instead, we argue that softmax’s effectiveness lies in its implicit regularization of the Frobenius norm…

Learning Pore-scale Multiphase Flow from 4D Velocimetry

arXiv:2603.12516v1 Announce Type: new Abstract: Multiphase flow in porous media underpins subsurface energy and environmental technologies, including geological CO$_2$ storage and underground hydrogen storage, yet pore-scale dynamics in realistic three-dimensional materials remain difficult to characterize and predict. Here we introduce…

Accelerating Residual Reinforcement Learning with Uncertainty Estimation

arXiv:2506.17564v2 Announce Type: replace Abstract: Residual Reinforcement Learning (RL) is a popular approach for adapting pretrained policies by learning a lightweight residual policy that provides corrective actions. While Residual RL is more sample-efficient than finetuning the entire base policy, existing…

Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Models

arXiv:2603.05773v2 Announce Type: replace-cross Abstract: Safety alignment is often conceptualized as a monolithic process wherein harmfulness detection automatically triggers refusal. However, the persistence of jailbreak attacks suggests a fundamental mechanistic decoupling. We propose the textbf{underline{D}}isentangled textbf{underline{S}}afety textbf{underline{H}}ypothesis textbf{(DSH)}, positing that…