Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

Researchers proposed Latent Context Language Models, encoder-decoder compressors that shrink input context before decoder prefill to cut memory and compute. The paper reports up to 16x compression with 8.8x faster output than KV-cache baselines on RULER, with smaller accuracy drops than other methods, and the models are open-sourced on HuggingFace.