Tenstorrent Wormhole (32x) - Qwen3-32B

February 19, 2026 at 06:58 AM

Dataset: reference (v1.0)

Tenstorrent Wormhole qwen Qwen3-32B

Best Performance

Click a metric to highlight the best run in the table below

Best Output TPS

1,438.10

Peak generation speed

Best Input TPS

5,640.69

Peak prefill speed

Best Energy Efficiency

0.11 kWh/MT

Energy cost per 1M tokens

Best TTFT (P95)

154.76 ms

Lowest latency

Best E2E (P95)

2,321.04 ms

Lowest latency

Test Matrix Results

Performance across different input/output token combinations and concurrency levels

Input Tokens	Output Tokens	Concurrency	Output TPS	Input TPS	Energy Cost (kWh/MT)	TTFT Mean	TTFT P95	E2E P95	Success Rate
Best Run for Output TPS
128	2,048	32x	1,438.10★	94.77	0.31	4,350.42	4,361.72	43,366.94	100.0%
128	128	1x	25.04	25.63	7.08	2,977.85	2,977.85	5,111.50	100.0%
128	128	2x	104.32	104.73	1.83	292.79	293.18	2,453.31	100.0%
128	128	4x	186.73	187.09	1.03	564.56	569.84	2,741.36	100.0%
128	128	8x	312.39	313.61	0.91	1,103.87	1,106.64	3,276.14	100.0%
128	128	16x	467.26	470.68	0.52	2,187.76	2,192.43	4,379.57	100.0%
128	128	32x	622.30	624.73	0.44	4,346.82	4,357.60	6,574.18	100.0%
128	128	64x	575.44	576.43	0.52	5,980.55	8,886.67	11,077.94	100.0%
128	512	1x	58.10	14.87	5.65	156.51	156.51	8,811.51	100.0%
128	512	2x	113.61	28.51	2.91	289.48	290.12	9,012.30	100.0%
128	512	4x	219.08	54.88	1.63	565.69	568.10	9,346.06	100.0%
128	512	8x	408.05	102.41	0.97	1,105.00	1,108.58	10,035.56	100.0%
128	512	16x	730.52	183.97	0.54	2,193.06	2,198.58	11,208.73	100.0%
128	512	32x	1,192.43	299.27	0.33	4,344.37	4,355.30	13,727.29	100.0%
128	512	64x	1,029.56	257.83	0.38	8,459.92	15,787.10	24,815.99	100.0%
128	1,024	1x	57.90	7.41	6.55	160.53	160.53	17,684.28	100.0%
128	1,024	2x	114.38	14.35	3.36	290.86	291.53	17,904.29	100.0%
128	1,024	4x	221.24	27.71	1.78	566.63	570.73	18,511.38	100.0%
128	1,024	8x	427.29	53.62	0.96	1,104.82	1,107.75	19,165.17	100.0%
128	1,024	16x	792.80	99.83	0.54	2,185.69	2,193.21	20,656.10	100.0%
128	1,024	32x	1,406.51	176.51	0.29	4,343.26	4,355.07	23,286.17	100.0%
128	1,024	64x	1,158.42	145.05	0.37	11,933.64	25,416.79	44,153.39	100.0%
128	2,048	1x	57.27	3.66	7.20	160.03	160.03	35,761.57	100.0%
128	2,048	2x	113.17	7.10	3.69	291.09	291.44	36,190.65	100.0%
128	2,048	4x	223.74	14.01	1.83	562.14	563.45	36,601.84	100.0%
128	2,048	8x	412.05	26.90	1.05	1,107.49	1,110.98	38,213.25	100.0%
128	2,048	16x	741.85	51.32	0.58	2,185.50	2,191.20	40,184.79	100.0%
128	2,048	64x	1,152.36	75.61	0.38	17,479.92	45,932.24	84,742.18	100.0%
512	128	1x	55.15	209.82	1.45	157.38	157.38	2,321.04	100.0%
512	128	2x	103.90	403.41	0.76	300.38	301.03	2,463.86	100.0%
512	128	4x	184.77	715.63	0.43	563.26	566.39	2,769.61	100.0%
512	128	8x	307.05	1,186.81	0.38	1,105.69	1,112.14	3,332.85	100.0%
512	128	16x	460.95	1,783.93	0.22	2,189.89	2,196.65	4,438.29	100.0%
512	128	32x	611.53	2,375.93	0.19	4,355.46	4,366.51	6,691.26	100.0%
512	128	64x	561.26	2,178.46	0.21	6,060.99	9,104.70	11,369.26	100.0%
512	512	1x	57.41	54.61	3.66	154.76	154.76	8,917.96	100.0%
512	512	2x	112.06	108.78	1.89	294.36	294.92	9,137.53	100.0%
512	512	4x	215.88	210.15	1.04	564.44	565.90	9,433.93	100.0%
512	512	8x	401.79	393.44	0.62	1,109.31	1,113.36	10,057.47	100.0%
512	512	16x	724.70	701.17	0.35	2,192.24	2,198.81	11,295.68	100.0%
512	512	32x	1,149.75	1,116.77	0.22	4,364.81	4,375.83	14,237.53	100.0%
512	512	64x	996.53	968.65	0.26	8,628.21	16,416.01	25,605.79	100.0%
512	1,024	1x	56.96	27.09	5.05	157.00	157.00	17,979.08	100.0%
512	1,024	2x	112.46	54.58	2.65	293.92	294.28	18,209.01	100.0%
512	1,024	4x	197.15	106.35	1.46	563.80	565.43	18,644.13	100.0%
512	1,024	8x	399.25	205.17	0.76	1,107.37	1,111.94	19,286.89	100.0%
512	1,024	16x	734.77	368.81	0.43	2,190.37	2,195.71	21,481.81	100.0%
512	1,024	32x	1,339.22	667.53	0.25	4,348.87	4,360.46	23,826.22	100.0%
512	1,024	64x	1,116.63	544.00	0.29	12,129.29	26,312.32	45,618.72	100.0%
512	2,048	1x	56.71	13.49	6.19	162.73	162.73	36,110.13	100.0%
512	2,048	2x	111.89	27.15	3.11	297.20	297.72	36,604.30	100.0%
512	2,048	4x	183.07	53.46	1.86	565.28	566.76	37,090.81	100.0%
512	2,048	8x	388.49	104.49	0.92	1,107.02	1,110.43	37,871.91	100.0%
512	2,048	16x	754.92	194.54	0.49	2,191.21	2,196.63	40,724.87	100.0%
512	2,048	32x	1,384.51	358.28	0.27	4,352.03	4,362.06	44,391.48	100.0%
512	2,048	64x	1,091.09	286.86	0.34	18,220.96	47,783.62	86,547.93	100.0%
1,024	128	1x	53.83	409.59	0.82	161.31	161.31	2,377.77	100.0%
1,024	128	2x	74.85	573.98	0.59	1,186.34	1,186.71	3,419.66	100.0%
1,024	128	4x	181.43	1,398.30	0.25	565.71	566.93	2,820.61	100.0%
1,024	128	8x	299.59	2,304.56	0.22	1,108.28	1,112.22	3,416.29	100.0%
1,024	128	16x	450.01	3,463.85	0.13	2,193.01	2,198.67	4,547.07	100.0%
1,024	128	32x	601.65	4,624.12	0.11	4,353.63	4,363.89	6,800.90	100.0%
1,024	128	64x	551.34	4,234.49	0.12	6,112.97	9,241.14	11,574.44	100.0%
1,024	512	1x	55.78	106.11	2.57	158.83	158.83	9,179.01	100.0%
1,024	512	2x	109.22	209.36	1.30	296.59	297.24	9,375.28	100.0%
1,024	512	4x	211.72	407.94	0.72	563.56	566.62	9,670.85	100.0%
1,024	512	8x	394.07	757.84	0.41	1,112.55	1,116.09	10,389.95	100.0%
1,024	512	16x	697.33	1,355.46	0.24	2,198.42	2,203.68	11,623.80	100.0%
1,024	512	32x	1,117.14	2,146.53	0.15	4,355.90	4,366.18	14,652.11	100.0%
1,024	512	64x	974.85	1,876.25	0.18	8,788.07	16,666.62	26,159.06	100.0%
1,024	1,024	1x	55.89	53.16	4.01	158.37	158.37	18,322.63	100.0%
1,024	1,024	2x	111.11	106.49	2.01	297.44	297.80	18,431.31	100.0%
1,024	1,024	4x	216.81	208.87	1.04	568.49	572.27	18,888.92	100.0%
1,024	1,024	8x	403.33	394.46	0.58	1,110.11	1,113.71	19,964.03	100.0%
1,024	1,024	16x	740.61	739.16	0.32	2,190.55	2,196.19	21,317.41	100.0%
1,024	1,024	32x	1,301.13	1,266.69	0.19	4,356.30	4,367.71	24,834.30	100.0%
1,024	1,024	64x	1,089.58	1,053.45	0.23	12,261.08	26,946.12	46,617.74	100.0%
1,024	2,048	1x	55.67	28.60	5.19	157.94	157.94	34,035.95	100.0%
1,024	2,048	2x	110.51	52.96	2.65	295.21	295.58	37,062.72	100.0%
1,024	2,048	4x	210.77	105.16	1.40	568.34	571.69	37,519.78	100.0%
1,024	2,048	8x	350.67	203.09	0.82	1,108.73	1,111.41	38,778.00	100.0%
1,024	2,048	16x	709.04	376.18	0.43	2,195.36	2,200.28	41,889.90	100.0%
1,024	2,048	32x	1,287.13	692.76	0.24	4,353.69	4,362.58	45,411.53	100.0%
1,024	2,048	64x	1,032.42	549.01	0.29	18,036.79	49,894.38	89,485.92	100.0%
2,048	128	1x	49.94	763.17	0.47	292.00	292.00	2,562.78	100.0%
2,048	128	2x	90.30	1,376.01	0.26	559.29	559.83	2,834.71	100.0%
2,048	128	4x	149.01	2,275.90	0.23	1,089.59	1,092.31	3,435.13	100.0%
2,048	128	8x	227.10	3,469.28	0.13	2,158.85	2,162.77	4,507.80	100.0%
2,048	128	16x	305.03	4,659.67	0.12	4,288.41	4,297.50	6,708.46	100.0%
2,048	128	32x	369.04	5,640.69	0.11	8,535.93	8,545.40	11,091.98	100.0%
2,048	128	64x	347.20	5,306.62	0.12	11,207.76	15,939.62	18,399.56	100.0%
2,048	512	1x	53.86	205.76	1.68	293.45	293.45	9,506.15	100.0%
2,048	512	2x	104.12	396.64	0.88	560.89	561.25	9,833.95	100.0%
2,048	512	4x	196.51	750.34	0.49	1,091.31	1,094.72	10,419.65	100.0%
2,048	512	8x	352.07	1,344.59	0.28	2,160.02	2,165.65	11,630.72	100.0%
2,048	512	16x	582.11	2,224.16	0.19	4,289.80	4,295.09	14,059.57	100.0%
2,048	512	32x	859.51	3,284.34	0.14	8,541.52	8,553.39	19,050.18	100.0%
2,048	512	64x	740.99	2,832.07	0.15	14,196.50	24,511.62	34,503.69	100.0%
2,048	1,024	1x	54.69	104.46	2.74	293.47	293.47	18,724.58	100.0%
2,048	1,024	2x	106.67	203.18	1.39	554.08	554.64	19,198.14	100.0%
2,048	1,024	4x	205.88	393.06	0.76	1,093.95	1,096.51	19,893.26	100.0%
2,048	1,024	8x	376.70	719.32	0.41	2,155.25	2,158.84	21,744.71	100.0%
2,048	1,024	16x	651.49	1,277.46	0.25	4,290.75	4,298.34	24,483.27	100.0%
2,048	1,024	32x	1,070.09	2,063.96	0.17	8,538.04	8,549.88	30,318.07	100.0%
2,048	1,024	64x	918.40	1,769.74	0.19	17,663.47	34,884.91	55,228.86	100.0%
2,048	2,048	1x	54.37	51.92	4.10	291.70	291.70	37,670.10	100.0%
2,048	2,048	2x	107.55	102.43	2.08	557.48	558.16	38,083.14	100.0%
2,048	2,048	4x	192.57	200.36	1.13	1,092.05	1,094.49	38,809.50	100.0%
2,048	2,048	8x	362.81	381.93	0.61	2,159.54	2,163.24	40,952.09	100.0%
2,048	2,048	16x	632.90	700.25	0.35	4,294.14	4,300.70	44,668.12	100.0%
2,048	2,048	32x	1,111.55	1,199.99	0.21	8,545.20	8,555.93	52,151.68	100.0%
2,048	2,048	64x	928.29	995.93	0.26	22,600.21	55,709.59	96,611.66	100.0%

Hardware Configuration

GPU ManufacturerTenstorrent

GPU ModelWormhole

GPU Count32

GPU Memory (Total)384 GB

CPU ModelAMD EPYC 9354P 32-Core Processor

RAM566 GB

Software Configuration

Inference FrameworkvLLM

Framework VersionUnknown

OSUbuntu

OS Version22.04.5 LTS (Jammy Jellyfish)

Kernel Version6.8.0-94-generic

Python Version3.10.12

Model Configuration

Providerqwen

Model NameQwen3-32B

QuantizationFP16

Inference Configuration

Runtime parameters used across all benchmark runs

Max Model LengthUnknown

Tensor Parallel Size1

Pipeline Parallel Size1

GPU Memory UtilizationUnknown

Temperature0.70

Top-P1.00

Top-K-1