AI · November 13, 2026 · Low impact
A Qwen3.5-9B model runs offline on a stock PS4 GPU using its built-in MTP head for speculative decoding
An open-source project runs Qwen3.5-9B on an unmodified PS4 at 31–39 tokens per second with no cloud or Linux required. The model's built-in MTP head drafts up to two tokens and the full model verifies them in a single pass, which raised throughput from 22 tokens per second to 31–39 on the old GPU. The weights were repacked into custom 4-bit and 3-bit formats to fit in about 4.5 GiB. The code and a direct download are published under a free open-source license.
It shows that an model's shipped multi-token prediction head can deliver a large speedup on very weak hardware, which is a practical technique for anyone running local models.