KV Cache Transform Coding for Compact Storage in LLM Inference
arXiv:2511.01815v2 Announce Type: replace-cross Abstract: Serving large language models (LLMs) at scale necessitates efficient key-value (KV) cache management. KV caches can be reused across conversation turns via shared-prefix prompts that are common in iterative code editing and chat. However, stale…
