arXiv:2512.22222v1 Announce Type: new
Abstract: Standard neural network architectures employ fixed activation functions (ReLU, tanh, sigmoid) that are poorly suited for approximating functions with singular or fractional power behavior, a structure that arises ubiquitously in physics, including boundary layers, fracture mechanics, and corner singularities. We introduce M”untz-Sz’asz Networks (MSN), a novel architecture that replaces fixed smooth activations with learnable fractional power bases grounded in classical approximation theory. Each MSN edge computes $phi(x) = sum_k a_k |x|^{mu_k} + sum_k b_k mathrm{sign}(x)|x|^{lambda_k}$, where the exponents ${mu_k, lambda_k}$ are learned alongside the coefficients. We prove that MSN inherits universal approximation from the M”untz-Sz’asz theorem and establish novel approximation rates: for functions of the form $|x|^alpha$, MSN achieves error $mathcal{O}(|mu – alpha|^2)$ with a single learned exponent, whereas standard MLPs require $mathcal{O}(epsilon^{-1/alpha})$ neurons for comparable accuracy. On supervised regression with singular target functions, MSN achieves 5-8x lower error than MLPs with 10x fewer parameters. Physics-informed neural networks (PINNs) represent a particularly demanding application for singular function approximation; on PINN benchmarks including a singular ODE and stiff boundary-layer problems, MSN achieves 3-6x improvement while learning interpretable exponents that match the known solution structure. Our results demonstrate that theory-guided architectural design can yield dramatic improvements for scientifically-motivated function classes.