Software · November 12, 2026 · Low impact
Google open-sources TPU Sync for KV cache and weight transfer
Google Cloud has open-sourced TPU Sync, a native C++ library. It moves KV cache blocks and model weights between TPU HBM, host DRAM, shared storage, and the data center network.
Moving inference memory and weights efficiently across TPU memory tiers is a practical bottleneck for serving large models, and an open library lets other engineers build on that work.