AMiD: Knowledge Distillation for LLMs with $alpha$-mixture Assistant Distribution
arXiv:2510.15982v1 Announce Type: new Abstract: Autoregressive large language models (LLMs) have achieved remarkable improvement across many tasks but incur high computational and memory costs. Knowledge distillation (KD) mitigates this issue by transferring knowledge from a large teacher to a smaller…
