Archives AI News

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?

arXiv:2410.01623v3 Announce Type: replace Abstract: Low-rank training has emerged as a promising approach for reducing memory usage in training Large Language Models (LLMs). Previous methods either rely on decomposing weight matrices (e.g., LoRA), or seek to decompose gradient matrices (e.g.,…

Output Supervision Can Obfuscate the Chain of Thought

arXiv:2511.11584v1 Announce Type: new Abstract: OpenAI (2025) showed that training against a chain of thought (CoT) monitor can cause obfuscated CoTs, which contain bad behavior the monitor cannot detect. They proposed to keep CoTs monitorable by training only against output…