LLM Inference Benchmark Explorer

Qwen3.8-27B on RTX PRO 6000 — FP8, vLLM, TP1, speculative decoding k=3 inference benchmark

Open this configuration in the LLM Inference Benchmark Explorer

ModelQwen3.8-27B
Parameters27B
Intelligence Index33.7
Agentic Index45.8
DeviceRTX PRO 6000
QuantizationFP8
Max C64
Chat Capacity60
Agentic Capacity20
TP—
DP—
PP—
EnginevLLM
Speculative DecodingYes
CTTFT (ms)TPS (tok/s)Status
186106.84PASS
2133102.99PASS
414192.51PASS
816784.96PASS
1622665.53PASS
3233745.83PASS
6457826.20PASS
128554325.18FAIL

In OpenZeka's measurement, Qwen3.8-27B (27B parameters), served in FP8 format with vLLM and speculative decoding on RTX PRO 6000, reached a generation speed of 106.8 tok/s per request and a time to first token (TTFT) of 86 ms with a single request.

Considering the speed targets and the available KV cache capacity, the estimated capacity is 60 users for chat use and 20 for agentic use. The realistic capacity will likely fall between these two values. In scenarios dominated by coding, tool use, long workflows and multi-agent use, capacity approaches the agentic estimate; where shorter interactions, standard conversations and lighter tasks dominate, it approaches the chat estimate.

Notes: Speculative MTP k=3.

Intelligence Index and Agentic Index values are published by Artificial Analysis and are reproduced here with attribution.