$lambda$-GELU: Learning Gating Hardness for Controlled ReLU-ization in Deep Networks
arXiv:2603.21991v2 Announce Type: replace Abstract: Gaussian Error Linear Unit (GELU) is a widely used smooth alternative to Rectifier Linear Unit (ReLU), yet many deployment, compression, and analysis toolchains are most naturally expressed for piecewise-linear (ReLU-type) networks. We study a hardness-parameterized…
