Archives AI News

Neural Diversity Regularizes Hallucinations in Language Models

arXiv:2510.20690v2 Announce Type: replace-cross Abstract: Language models continue to hallucinate despite increases in parameters, compute, and data. We propose neural diversity — decorrelated parallel representations — as a principled mechanism that reduces hallucination rates at fixed parameter and data budgets.…

A Minimalist Optimizer Design for LLM Pretraining

arXiv:2506.16659v2 Announce Type: replace Abstract: Training large language models (LLMs) typically relies on adaptive optimizers such as Adam, which introduce extra operations and require significant more memory to maintain first- and second-order moments than SGD. While recent works such as…