MLPERF Inference v5.1 (2025): Explanation of results for GPU, CPU and AI accelerators

What does MLPERF reasoning actually measure?

mlperf inference quantization How fast is the complete system (Hardware + Runtime + Service Stack) Execution Fixed, pre-trained model strict Incubation period and accuracy constraint. Reported results Data Center and edge Loadgen produces a suite with standardized request patterns (“scheme”) to ensure building neutrality and repeatability. this closure Division fixes the model and preprocesses it for apple-to-apple comparisons; Open Departments allow model changes, rather than strict comparability. Availability tags –Available,,,,, Preview,,,,, rdi (Research/Development/Internal) – Indicates whether the configuration is transportable or experimental.

2025 Update (v5.0→v5.1): What has changed?

this v5.1 Results (Published September 9, 2025) Add three modern workloads and scale up interactive services:

• DeepSeek-R1 (First reasoning benchmark)
• Call-3.1-8B (Abstract) Replace GPT-J
• Whispering Big V3 (ASR)

This round of record 27 submissions And first appearance AMD Instinct MI355X,,,,, Intel Arc Pro B60 48GB turbocharged,,,,, NVIDIA GB300,,,,, RTX 4000 ADA-PCIE-20GBand RTX Pro 6000 Blackwell Server Edition. Interactive scenes (tight TTFT/TPOT Limit beyond a single model to capture proxy/chat workloads.

Scheme: Four service modes you must map to real workloads

• Offline: Maximize throughput, No Delay binding – Criticism and scheduling dominate.
• server: Poisson arrived P99 Delay range – Link to chat/proxy backend.
• Single flow/multi flow (Edge emphasis): strict incubation period for each tail; multiple flow stresses are concurrent with fixed arrival intervals.

Each case has a defined measure (e.g. Maximum Poisson throughput For the server; Throughput for offline).

Late metrics for LLMS: TTFT and TPOT are now top-notch

LLM Test Report ttft (First time) and TPOT (Time output per time). v5.0 introduces stricter interactive restrictions Llama-2-70B ((P99 TTFT 450ms, TPOT 40ms) to reflect the user’s perceived responsiveness. this The novel Camel-3.1-405b Keep higher boundaries (P99 TTFT 6 S, TPOT 175ms) Due to model size and context length. These constraints are carried into V5.1 along with the new LLM and inference tasks.

Key v5.1 entry and its Quality/Late Period Gates (Abbrev.):

• LLM Q&A-Llama-2-70B (Openorca): Session 2000 ms/200 ms; Interactive 450 ms/40 ms; 99% and 99.9% accuracy target.
• LLM Summary – Llama-3.1-8B (CNN/dailymail): Dialogue 2000 ms/100 ms; interactive 500 ms/30 ms.
• Reasoning – DeepSeek-R1: TTFT 2000 ms/tpot 80 ms; 99% of FP16 (exactly matched baseline).
• ASR – Whispering Big V3 (LibrisPeech): WER-based quality (data center + edge).
• Novel – Camel-3.1-405b: TTFT 6000ms, TPOT 175ms.
• Image – SDXL 1.0:fid/clip range; server Has a constraint of 20 s.

Legacy CV/NLP (Resnet-50, Vetinanet, Bert-L, DLRM, 3D-Unet) remains continuity.

Power Results: How to Read Energy Proposition

mlperf strength (Optional) Report System wall plug Same running energy (server/offline: System power;Single/multi-stream: Each flow of energy). The only one Measurement Operation is more effective for energy efficiency; TDP and supplier estimates are poor. v5.1 includes data center and edge power submissions, but encourages wider participation.

How to read a desk without fooling yourself?

• Comparison of closed and closed Only; open runs may use different models/quantizations.
• Match accuracy targets (99% vs 99.9%) – Throughput usually drops with more stringent quality.
• Normalize carefully: MLPERF report System level Throughput under constraints; generated separately by accelerator counting Derived MLPERF’s “per chip” number no Defined as the primary measure. Use it only for budget sanity checks, not marketing claims.
• Filtering by usability (I like more Available) and includes strength When efficiency is important.

Explain 2025 results: GPU, CPU, and other accelerators

GPU (rack scale to single node). New silicon is significantly present in server interactions (tight TTFT/TPOT) and long text workloads, where scheduler and KV-CACHE efficiency are as important as RAW FLOPS. Rack-scale systems (e.g., GB300 NVL72 class) release the highest summary; both are normalized accelerator and host Compare the counts before the individual node entries and keep the scene/accuracy the same.

CPU (independent baseline + host effect). CPU-only entries are still useful baselines and highlight preprocessing and scheduling overhead that can bottleneck accelerators server model. New Xeon 6 The results and hybrid CPU+GPU stack appear in v5.1; check the host’s generation and memory configuration when comparing the system to similar accelerators.

Alternative accelerator. v5.1 increases the diversity of architecture (GPUs from multiple vendors as well as new workstation/server SKUs). Where open submissions occur (e.g., pruning/low-precision variants), verify that any cross-system comparison remains constant distribute,,,,, Model,,,,, Dataset,,,,, Imagineand accuracy.

Practical Selection Script (SLA’s Map Benchmark)

• Interactive chat/agent → Server interaction exist Llama-2-70B/Call-3.1-8B/DeepSeek-R1 (Match delay and accuracy; check carefully P99 TTFT/TPOT).
• Batch summary/ETL → Offline exist Call-3.1-8B;The throughput per rack is the cost drive.
• ASR front end → Whisper V3 Tail-bound server; memory bandwidth and audio pre-/post-processing issues.
• Novel analysis → Call-3.1-405b;Evaluate whether your UX is tolerant 6 s ttft / 175 ms tpot.

What is the 2025 cycle signal?

• Interactive LLM service is a dining table. Tight TTFT/TPOT in V5.x makes visible in the results, visible scheduling, classification, classification, refocusing and KV-CACHE MANACOINS visible – hope leaders are different from pure offline.
• Reasoning is now benchmark. DeepSeek-R1 emphasizes that control flow and memory flow are different from the next generation.
• Wide range of coverage. Whisper V3 and SDXL motion pipelines exceed token decoding, surface I/O and bandwidth limitations.

Summary

In summary, MLPERF inference v5.1 can only make inference comparisons when based on benchmark rules closure Department, competition Imagine and accuracy (Including LLM TTFT/TPOT Restrictions on interactive services), prefer Available System with measurement strength Reasoning efficiency; treat any human split as a derivative heuristic because MLPERF reports system-level performance. The 2025 cycle expands coverage DeepSeek-R1,,,,, Call-3.1-8Band Whispering Big V3coupled with wider silicon involvement, so procurement should filter results to mirror production SLA workloads (server interactions for chat/agents, offline batches) and validate claims results pages and power methodology directly in MLCommons.

---

refer to: