Table 1. Speedup with Systalyze over the vLLM baseline, by model and hardware. All values are × improvement (higher is better throughout).
Model Avg. total throughput(tok/s/GPU) p99 TTFT p99 ITL p50 user throughput(tok/s/user) Run it*
8× RTX PRO 4500 Blackwell
Gemma-4-26B-A4B-IT 4.0× 3.2× 7.5× 7.9×
Nemotron-3-Nano-30B-A3B 3.2× 3.0× 5.9× 5.5×
Qwen3-32B 3.3× 2.5× 9.8× 8.1×
8× RTX PRO 6000 Blackwell
Gemma-4-26B-A4B-IT 2.5× 3.7× 8.9× 9.0×
Nemotron-3-Nano-30B-A3B 1.8× 3.6× 10.2× 9.1×
Qwen3-32B 3.4× 6.4× 7.0× 9.1×
8× H200
Gemma-4-26B-A4B-IT 1.3× 1.9× 7.6× 7.3×
Nemotron-3-Nano-30B-A3B 1.4× 2.7× 7.3× 6.3×
Qwen3-32B 1.6× 5.7× 7.5× 5.7×
* Opens the pull, serve and benchmark commands for that configuration.