Daily Tokens
November 12, 2026

Software · November 12, 2026 · Low impact

Google open-sources TPU Sync for KV cache and weight transfer

Google Cloud has open-sourced TPU Sync, a native C++ library. It moves KV cache blocks and model weights between TPU HBM, host DRAM, shared storage, and the data center network.

Moving inference memory and weights efficiently across TPU memory tiers is a practical bottleneck for serving large models, and an open library lets other engineers build on that work.

Go deeper

More from Software on Thursday, November 12