Towards Quantifying the Hessian Structure of Neural Networks
arXiv:2505.02809v2 Announce Type: replace Abstract: Empirical studies reported that the Hessian matrix of neural networks (NNs) exhibits a near-block-diagonal structure, yet its theoretical foundation remains unclear. In this work, we reveal that the reported Hessian structure comes from a mixture…
