VLM Inference Benchmark Explorer

Alpamayo2-Edge on Jetson AGX Thor — BF16, vLLM vision-language inference benchmark

Open this configuration in the VLM Inference Benchmark Explorer

ModelAlpamayo2-Edge
Parameters2.7B
DeviceJetson AGX Thor
QuantizationBF16
Image size720p
Images per camera1
Max Cameras (TTFT ≤ 3000 ms)128
EnginevLLM
CamerasTTFT (ms)TPS (tok/s)Target
1136101.2≤3000 ms
214995.3≤3000 ms
416274.6≤3000 ms
8212138.8≤3000 ms
1635548.4≤3000 ms
3255132.4≤3000 ms
6494836.5≤3000 ms
128189517.0≤3000 ms
256424816.1>3000 ms
512864718.7>3000 ms
10241790214.4>3000 ms
12802100312.3>3000 ms
14082363511.6>3000 ms
1536——Failed
Images per requestTTFT (ms)TPS (tok/s)Target
1136101.2≤3000 ms
2185728.8≤3000 ms
4247297.8≤3000 ms
841072.5≤3000 ms
1679269.1≤3000 ms
32148558.5≤3000 ms
64330751.2>3000 ms
128720045.8>3000 ms
2561716835.0>3000 ms
51246217148.7>3000 ms
1024——Failed

Alpamayo2-Edge (2.7B parameters), served in BF16 format with vLLM on Jetson AGX Thor, starts answering a single camera that sends one 720p image after 136 ms (time to first token, TTFT), then writes 101.2 tok/s.

With 512 images per camera at 720p, the answer starts after 46,217 ms. With one 2K image, after 229 ms.

At the default target of a mean TTFT within 3,000 ms (720p, one image per camera), it keeps up with 128 cameras at once.