Inference Benchmarks

Performance metrics across hardware, software, and model configurations

Early Preview

We are working to certify more internal benchmarks to be published. If you're interested in providing hardware or have questions, email benchmarks@haimaker.ai.

Filters

Active Filters:Tag: NVIDIA H200 NVL

Found 8 benchmark suites

Date	Suite Name	GPU	Model	Output TPS	Input TPS	Energy Cost (kWh/MT)
11/7/2025	NVIDIA H200 NVL (2x) - mistral-nemo-instruct-2407	NVIDIA H200 NVL 2x 280GB	mistral-nemo-instruct-2407 mistralai	12,204.48	47,690.47	0.01
11/7/2025	NVIDIA H200 NVL (2x) - qwen3-30b-a3b	NVIDIA H200 NVL 2x 280GB	qwen3-30b-a3b qwen	6,124.38	51,413.77	0.00
11/6/2025	NVIDIA H200 NVL (2x) - allam-7b-instruct-preview	NVIDIA H200 NVL 2x 280GB	allam-7b-instruct-preview humain-ai	11,481.64	45,184.12	0.01
11/6/2025	NVIDIA H200 NVL (2x) - llama-2-70b-hf (50% Max Batch Token)	NVIDIA H200 NVL 2x 280GB	llama-2-70b-hf meta-llama	4,620.81	8,844.22	0.03
11/6/2025	NVIDIA H200 NVL (2x) - llama-2-70b-hf	NVIDIA H200 NVL 2x 280GB	llama-2-70b-hf meta-llama	5,012.77	10,466.05	0.03
11/5/2025	NVIDIA H200 NVL (2x) - gpt-oss-120b	NVIDIA H200 NVL 2x 280GB	gpt-oss-120b openai	3,166.06	11,929.37	0.01
11/5/2025	NVIDIA H200 NVL (2x) - qwen3-coder-30b-a3b-instruct	NVIDIA H200 NVL 2x 280GB	qwen3-coder-30b-a3b-instruct qwen	5,757.76	43,900.39	0.01
11/5/2025	NVIDIA H200 NVL (2x) - llama-3.3-70b-instruct	NVIDIA H200 NVL 2x 280GB	llama-3.3-70b-instruct meta-llama	5,005.29	11,042.39	0.03