Archives AI News

DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

arXiv:2603.22324v1 Announce Type: new Abstract: We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but are agnostic to the base model, allowing quantization noise to disproportionately…

Hybrid Associative Memories

arXiv:2603.22325v1 Announce Type: new Abstract: Recurrent neural networks (RNNs) and self-attention are both widely used sequence-mixing layers that maintain an internal memory. However, this memory is constructed using two orthogonal mechanisms: RNNs compress the entire past into a fixed-size state,…

Two Stage Wireless Federated LoRA Fine-Tuning with Sparsified Orthogonal Updates

arXiv:2505.00333v2 Announce Type: replace Abstract: Transformer-based large language models (LLMs) have achieved remarkable success across various tasks. Yet, fine-tuning such massive models in federated learning (FL) settings poses significant challenges due to resource constraints and communication overhead. Low-Rank Adaptation (LoRA)…

Beyond the Mean: Distribution-Aware Loss Functions for Bimodal Regression

arXiv:2603.22328v1 Announce Type: new Abstract: Despite the strong predictive performance achieved by machine learning models across many application domains, assessing their trustworthiness through reliable estimates of predictive confidence remains a critical challenge. This issue arises in scenarios where the likelihood…