Tenstorrent Wormhole (32x) - Llama-3.3-70B-Instruct

February 19, 2026 at 06:18 PM

Dataset: reference (v1.0)

Tenstorrent Wormhole meta-llama Llama-3.3-70B-Instruct

Best Performance

Click a metric to highlight the best run in the table below

Best Output TPS

1,091.26

Peak generation speed

Best Input TPS

4,654.03

Peak prefill speed

Best Energy Efficiency

0.13 kWh/MT

Energy cost per 1M tokens

Best TTFT (P95)

195.78 ms

Lowest latency

Best E2E (P95)

2,244.78 ms

Lowest latency

Test Matrix Results

Performance across different input/output token combinations and concurrency levels

Input Tokens	Output Tokens	Concurrency	Output TPS	Input TPS	Energy Cost (kWh/MT)	TTFT Mean	TTFT P95	E2E P95	Success Rate
Best Run for Output TPS
128	512	32x	1,091.26★	286.23	0.40	5,617.57	5,629.06	14,356.75	100.0%
128	128	1x	25.50	26.10	7.31	3,048.70	3,048.70	5,018.98	100.0%
128	128	2x	108.06	108.48	1.88	370.77	371.12	2,367.81	100.0%
128	128	4x	186.79	187.16	1.03	719.82	721.42	2,740.22	100.0%
128	128	8x	298.02	299.19	1.02	1,421.91	1,425.13	3,434.87	100.0%
128	128	16x	421.83	424.92	0.76	2,819.22	2,825.25	4,851.99	100.0%
128	128	32x	533.40	535.49	0.60	5,609.55	5,621.75	7,672.03	100.0%
128	128	64x	496.12	496.98	0.65	7,480.33	10,812.52	12,857.72	100.0%
128	512	1x	61.44	15.84	6.28	196.02	196.02	8,267.94	100.0%
128	512	2x	120.40	30.22	2.95	372.19	372.58	8,503.87	100.0%
128	512	4x	230.60	57.76	1.57	724.17	728.14	8,878.98	100.0%
128	512	8x	386.16	105.72	1.09	1,422.02	1,425.30	9,720.78	100.0%
128	512	16x	673.64	184.41	0.64	2,819.90	2,824.71	11,182.59	100.0%
128	512	64x	949.19	245.86	0.47	9,350.46	17,537.49	26,030.85	100.0%
128	1,024	1x	61.52	10.76	6.69	196.57	196.57	12,154.90	100.0%
128	1,024	2x	103.59	15.23	4.11	371.22	371.72	16,625.65	100.0%
128	1,024	4x	207.56	29.74	1.95	720.42	721.99	17,052.71	100.0%
128	1,024	8x	336.50	56.84	1.30	1,419.89	1,422.56	17,969.51	100.0%
128	1,024	16x	564.40	104.36	0.80	2,821.80	2,826.93	18,329.48	100.0%
128	1,024	32x	1,028.82	179.85	0.44	5,612.45	5,623.40	22,548.80	100.0%
128	1,024	64x	939.03	164.50	0.50	9,925.56	21,165.68	36,751.53	100.0%
128	2,048	1x	61.94	9.98	6.48	196.12	196.12	13,107.21	100.0%
128	2,048	2x	91.92	11.83	4.44	372.32	372.82	21,185.25	100.0%
128	2,048	4x	213.96	31.36	2.00	719.11	722.18	16,261.30	100.0%
128	2,048	8x	254.77	41.85	1.68	1,424.08	1,428.38	21,522.24	100.0%
128	2,048	16x	533.68	95.85	0.83	2,820.31	2,824.86	19,505.73	100.0%
128	2,048	32x	810.90	139.58	0.55	5,613.84	5,625.14	23,723.55	100.0%
128	2,048	64x	941.12	156.77	0.49	9,920.52	20,978.30	38,115.77	100.0%
128	2,048	128x	885.71	148.24	0.51	9,783.55	20,509.21	37,137.06	100.0%
512	128	1x	57.02	216.93	1.49	198.20	198.20	2,244.78	100.0%
512	128	2x	106.58	413.82	0.78	372.75	373.28	2,401.55	100.0%
512	128	4x	146.79	708.21	0.48	724.09	726.61	2,799.35	100.0%
512	128	8x	291.65	1,127.31	0.43	1,426.32	1,428.88	3,509.63	100.0%
512	128	16x	413.07	1,598.63	0.32	2,820.09	2,825.94	4,954.80	100.0%
512	128	32x	524.46	2,037.64	0.23	5,619.95	5,632.68	7,802.94	100.0%
512	128	64x	487.88	1,893.66	0.28	7,534.93	10,935.40	13,081.45	100.0%
512	512	1x	60.66	57.69	4.04	201.37	201.37	8,441.38	100.0%
512	512	2x	118.50	115.03	1.96	376.45	377.06	8,640.31	100.0%
512	512	4x	154.30	218.39	1.23	724.17	725.50	9,078.86	100.0%
512	512	8x	352.36	399.15	0.68	1,423.74	1,427.22	9,913.63	100.0%
512	512	16x	665.25	687.19	0.42	2,823.93	2,831.30	11,527.85	100.0%
512	512	32x	1,086.90	1,096.69	0.26	5,614.73	5,625.57	14,495.86	100.0%
512	512	64x	910.74	922.33	0.30	9,707.82	18,217.71	26,893.67	100.0%
512	1,024	1x	60.13	32.87	5.20	197.54	197.54	14,800.86	100.0%
512	1,024	2x	105.83	66.50	2.70	373.22	373.69	14,772.17	100.0%
512	1,024	4x	136.07	111.96	1.86	722.95	727.08	17,384.95	100.0%
512	1,024	8x	303.46	212.81	1.00	1,425.00	1,428.34	18,504.60	100.0%
512	1,024	16x	563.14	392.67	0.56	2,821.85	2,826.76	19,060.17	100.0%
512	1,024	32x	988.34	674.72	0.33	5,628.80	5,640.53	22,745.57	100.0%
512	1,024	64x	950.01	615.99	0.34	10,523.03	22,681.73	38,824.61	100.0%
512	2,048	1x	60.49	36.59	4.67	195.78	195.78	13,292.61	100.0%
512	2,048	2x	110.52	60.52	2.82	372.25	372.88	16,302.67	100.0%
512	2,048	4x	129.49	105.59	2.05	723.97	725.87	18,323.10	100.0%
512	2,048	8x	308.97	192.25	1.01	1,422.72	1,425.52	19,431.29	100.0%
512	2,048	16x	587.08	388.74	0.56	2,824.78	2,829.98	19,075.79	100.0%
512	2,048	32x	923.77	603.86	0.36	5,616.63	5,628.01	24,911.69	100.0%
512	2,048	64x	917.09	595.25	0.35	10,330.29	22,827.98	37,146.60	100.0%
1,024	128	1x	55.77	424.40	0.84	201.55	201.55	2,294.73	100.0%
1,024	128	2x	102.73	787.72	0.45	375.79	376.31	2,491.56	100.0%
1,024	128	4x	180.73	1,392.87	0.26	725.36	726.78	2,832.28	100.0%
1,024	128	8x	286.19	2,201.51	0.25	1,425.91	1,429.36	3,576.01	100.0%
1,024	128	16x	407.97	3,140.24	0.18	2,827.19	2,832.83	5,015.69	100.0%
1,024	128	32x	511.17	3,974.37	0.13	5,626.01	5,636.35	7,913.48	100.0%
1,024	128	64x	467.26	3,588.67	0.15	7,730.69	11,469.42	13,660.05	100.0%
1,024	512	1x	58.75	111.76	2.60	197.13	197.13	8,715.40	100.0%
1,024	512	2x	114.79	220.04	1.30	372.88	372.97	8,920.40	100.0%
1,024	512	4x	218.92	421.81	0.75	722.27	723.61	9,353.35	100.0%
1,024	512	8x	377.54	770.44	0.47	1,430.00	1,433.55	10,220.86	100.0%
1,024	512	16x	600.24	1,334.01	0.29	2,825.32	2,828.92	11,812.26	100.0%
1,024	512	32x	1,039.05	2,101.40	0.18	5,620.11	5,634.85	14,970.83	100.0%
1,024	512	64x	891.30	1,784.63	0.21	9,532.33	18,414.02	27,499.91	100.0%
1,024	1,024	1x	56.48	93.09	3.02	197.64	197.64	10,442.40	100.0%
1,024	1,024	2x	94.41	120.34	2.20	371.91	372.59	16,002.81	100.0%
1,024	1,024	4x	190.83	249.67	1.09	722.76	724.30	15,753.59	100.0%
1,024	1,024	8x	272.39	412.39	0.72	1,428.18	1,432.52	19,097.08	100.0%
1,024	1,024	16x	457.35	757.08	0.42	2,832.18	2,837.20	19,122.70	100.0%
1,024	1,024	32x	833.34	1,293.54	0.26	5,630.51	5,641.58	22,414.08	100.0%
1,024	1,024	64x	850.95	1,216.98	0.26	9,494.62	21,026.97	38,560.44	100.0%
1,024	1,024	128x	864.67	1,261.72	0.26	9,566.91	20,542.76	36,670.33	100.0%
1,024	1,024	256x	892.34	1,307.57	0.26	9,858.61	21,267.37	36,106.22	100.0%
1,024	1,024	512x	854.15	1,209.94	0.26	9,771.97	21,382.73	37,159.62	100.0%
1,024	2,048	1x	58.63	90.79	3.22	199.86	199.86	10,710.86	100.0%
1,024	2,048	2x	101.94	144.80	1.85	372.17	372.80	13,384.59	100.0%
1,024	2,048	4x	192.74	272.99	1.02	723.68	726.71	14,310.52	100.0%
1,024	2,048	8x	252.26	346.24	0.84	1,427.77	1,430.18	20,697.39	100.0%
1,024	2,048	16x	553.30	845.66	0.35	2,835.60	2,841.45	17,925.79	100.0%
1,024	2,048	32x	787.90	1,236.25	0.26	5,626.79	5,637.29	22,403.45	100.0%
1,024	2,048	64x	825.94	1,218.04	0.27	9,690.46	20,825.75	35,711.46	100.0%
1,024	2,048	128x	845.18	1,190.92	0.28	9,571.30	21,310.08	36,857.89	100.0%
1,024	2,048	256x	766.58	1,083.02	0.29	9,581.00	20,292.48	36,461.90	100.0%
2,048	128	1x	49.82	761.39	0.49	367.97	367.97	2,569.27	100.0%
2,048	128	2x	87.70	1,336.42	0.28	710.45	711.10	2,918.26	100.0%
2,048	128	4x	140.20	2,141.29	0.25	1,395.13	1,397.75	3,651.32	100.0%
2,048	128	8x	202.13	3,087.84	0.19	2,764.77	2,768.75	5,064.53	100.0%
2,048	128	16x	258.88	3,954.62	0.14	5,503.05	5,507.51	7,907.99	100.0%
2,048	128	32x	304.49	4,654.03	0.13	10,967.65	10,978.89	13,444.00	100.0%
2,048	128	64x	290.46	4,443.60	0.14	14,067.89	19,584.04	21,979.73	100.0%
2,048	512	1x	55.38	222.45	1.60	364.38	364.38	8,776.10	100.0%
2,048	512	2x	99.62	405.64	0.94	709.40	710.07	9,557.55	100.0%
2,048	512	4x	189.45	723.40	0.53	1,397.13	1,400.19	10,807.84	100.0%
2,048	512	8x	339.56	1,310.57	0.34	2,763.63	2,767.57	11,931.22	100.0%
2,048	512	16x	508.62	2,051.61	0.22	5,499.77	5,506.44	15,241.78	100.0%
2,048	512	32x	745.64	2,972.04	0.16	10,971.52	10,980.32	21,051.29	100.0%
2,048	512	64x	668.03	2,628.92	0.18	16,116.85	27,419.93	37,170.19	100.0%
2,048	1,024	1x	54.84	238.92	1.60	374.16	374.16	8,173.72	100.0%
2,048	1,024	2x	91.96	328.51	1.13	706.09	706.73	11,688.90	100.0%
2,048	1,024	4x	173.00	566.05	0.68	1,396.88	1,399.41	13,791.70	100.0%
2,048	1,024	8x	245.37	731.53	0.51	2,761.16	2,764.27	19,493.38	100.0%
2,048	1,024	16x	438.34	1,267.52	0.31	5,505.07	5,511.36	22,968.88	100.0%
2,048	1,024	32x	702.81	2,046.15	0.21	10,971.29	10,980.57	29,729.92	100.0%
2,048	1,024	64x	705.68	1,969.18	0.22	16,205.41	30,332.31	46,023.82	100.0%
2,048	1,024	128x	703.71	1,940.04	0.22	16,387.24	30,966.04	47,578.63	100.0%
2,048	2,048	1x	54.47	237.29	1.59	374.04	374.04	8,224.48	100.0%
2,048	2,048	2x	90.34	242.39	1.42	709.42	709.94	15,818.84	100.0%
2,048	2,048	4x	165.97	554.41	0.70	1,391.51	1,394.92	14,034.30	100.0%
2,048	2,048	8x	223.51	676.28	0.56	2,764.49	2,767.11	20,901.13	100.0%
2,048	2,048	16x	307.10	733.36	0.48	5,496.34	5,500.82	31,920.57	100.0%
2,048	2,048	32x	646.79	1,971.72	0.22	10,970.37	10,980.08	29,207.73	100.0%
2,048	2,048	64x	550.84	1,434.29	0.27	16,473.96	30,400.63	48,767.76	100.0%

Hardware Configuration

GPU ManufacturerTenstorrent

GPU ModelWormhole

GPU Count32

GPU Memory (Total)384 GB

CPU ModelAMD EPYC 9354P 32-Core Processor

RAM566 GB

Software Configuration

Inference FrameworkvLLM

Framework VersionUnknown

OSUbuntu

OS Version22.04.5 LTS (Jammy Jellyfish)

Kernel Version6.8.0-94-generic

Python Version3.10.12

Model Configuration

Providermeta-llama

Model NameLlama-3.3-70B-Instruct

QuantizationFP16

Inference Configuration

Runtime parameters used across all benchmark runs

Max Model LengthUnknown

Tensor Parallel Size1

Pipeline Parallel Size1

GPU Memory UtilizationUnknown

Temperature0.70

Top-P1.00

Top-K-1