Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang
Tsinghua University and CUHK researchers open-sourced AKV, a serving system for KV cache engineering in LLM agents. Instead of summarizing old history, agents drop selected KV cache entries and keep reusing the rest. Across 3 models and 4 agent benchmarks, average cost falls 27.7%. Average task score rises 1.7 points. The code is Apache-2.0 licensed
Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang Read More »
