Archives AI News

MoE-Spec: Expert Budgeting for Efficient Speculative Decoding

arXiv:2602.16052v1 Announce Type: new Abstract: Speculative decoding accelerates Large Language Model (LLM) inference by verifying multiple drafted tokens in parallel. However, for Mixture-of-Experts (MoE) models, this parallelism introduces a severe bottleneck: large draft trees activate many unique experts, significantly increasing…

Large Language Models for Water Distribution Systems Modeling and Decision-Making

arXiv:2503.16191v2 Announce Type: replace-cross Abstract: The integration of Large Language Models (LLMs) into engineering workflows presents new opportunities for making computational tools more accessible. Especially where such tools remain underutilized due to technical or expertise barriers, such as water distribution…

Data-Efficient Self-Supervised Algorithms for Fine-Grained Birdsong Analysis

arXiv:2511.12158v2 Announce Type: replace Abstract: Many bioacoustics, neuroscience, and linguistics research utilize birdsongs as proxy models to acquire knowledge in diverse areas. Developing models generally requires precisely annotated data at the level of syllables. Hence, automated and data-efficient methods that…

Error Propagation and Model Collapse in Diffusion Models: A Theoretical Study

arXiv:2602.16601v1 Announce Type: cross Abstract: Machine learning models are increasingly trained or fine-tuned on synthetic data. Recursively training on such data has been observed to significantly degrade performance in a wide range of tasks, often characterized by a progressive drift…

Random Scaling of Emergent Capabilities

arXiv:2502.17356v5 Announce Type: replace Abstract: Language models famously improve under a smooth scaling law, but some specific capabilities exhibit sudden breakthroughs in performance. Advocates of “emergence” view these capabilities as unlocked at a specific scale, but others attribute breakthroughs to…

Subtractive Modulative Network with Learnable Periodic Activations

arXiv:2602.16337v1 Announce Type: cross Abstract: We propose the Subtractive Modulative Network (SMN), a novel, parameter-efficient Implicit Neural Representation (INR) architecture inspired by classical subtractive synthesis. The SMN is designed as a principled signal processing pipeline, featuring a learnable periodic activation…