Inference Benchmarks

Performance metrics across hardware, software, and model configurations

Early Preview

We are working to certify more internal benchmarks to be published. If you're interested in providing hardware or have questions, email benchmarks@haimaker.ai.

Filters

Active Filters:Tag: meta-llama

Found 8 benchmark suites

Date	Suite Name	GPU	Model	Output TPS	Input TPS	Energy Cost (kWh/MT)
2/19/2026	Tenstorrent Wormhole (32x) - Llama-3.3-70B-Instruct	Wormhole 32x 384GB	Llama-3.3-70B-Instruct meta-llama	1,091.26	4,654.03	0.13
11/13/2025	NVIDIA H100 80GB HBM3 (8x) - llama-2-70b-hf	NVIDIA H100 80GB HBM3 8x 632GB	llama-2-70b-hf meta-llama	668.64	855.76	0.79
11/12/2025	NVIDIA H100 80GB HBM3 (8x) - llama-3.3-70b-instruct	NVIDIA H100 80GB HBM3 8x 632GB	llama-3.3-70b-instruct meta-llama	9,219.60	16,108.82	0.06
11/6/2025	NVIDIA H200 NVL (2x) - llama-2-70b-hf (50% Max Batch Token)	NVIDIA H200 NVL 2x 280GB	llama-2-70b-hf meta-llama	4,620.81	8,844.22	0.03
11/6/2025	NVIDIA H200 NVL (2x) - llama-2-70b-hf	NVIDIA H200 NVL 2x 280GB	llama-2-70b-hf meta-llama	5,012.77	10,466.05	0.03
11/5/2025	NVIDIA H200 NVL (2x) - llama-3.3-70b-instruct	NVIDIA H200 NVL 2x 280GB	llama-3.3-70b-instruct meta-llama	5,005.29	11,042.39	0.03
10/25/2025	NVIDIA H20 (8x) - llama-3.3-70b-instruct (High Throughput)	NVIDIA H20 8x 760GB	llama-3.3-70b-instruct meta-llama	5,091.04	7,327.23	0.10
10/24/2025	NVIDIA H20 (8x) - llama-3.3-70b-instruct	NVIDIA H20 8x 760GB	llama-3.3-70b-instruct meta-llama	3,370.98	6,350.24	0.11