Archives AI News

PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution

arXiv:2507.01695v2 Announce Type: replace Abstract: Deep neural networks (DNNs) have become ubiquitous thanks to their remarkable ability to model complex patterns across various domains such as computer vision, speech recognition, robotics, etc. While large DNN models are often more accurate…

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?

arXiv:2410.01623v3 Announce Type: replace Abstract: Low-rank training has emerged as a promising approach for reducing memory usage in training Large Language Models (LLMs). Previous methods either rely on decomposing weight matrices (e.g., LoRA), or seek to decompose gradient matrices (e.g.,…