AI · November 21, 2026 · Low impact
Nvidia researchers say short notes can replace huge context windows
Nvidia researchers argue that language models often do not need their full context windows, which are slow, expensive and prone to errors. In their tests, periodic summaries and targeted keyword search usually matched full-context results at a fraction of the cost. They also propose a new class of models that works with short notes instead of long context.
If summaries and search hold up, long-context cost and error problems could be handled with simpler pipelines.