Geometric Scaling of Bayesian Inference in LLMs
arXiv:2512.23752v1 Announce Type: new Abstract: Recent work has shown that small transformers trained in controlled “wind-tunnel” settings can implement exact Bayesian inference, and that their training dynamics produce a geometric substrate — low-dimensional value manifolds and progressively orthogonal keys —…
