Skip to content
Inference Performance Assurance

Predict, validate, benchmark and certify on your own workload mix.

KPG PredictorPrototypepatent-pendingLive-workload evaluationEarly accessBenchmarking-as-a-ServiceAvailable

Every enterprise that runs models on GPUs has to answer the same questions before and after it buys capacity: which hardware, how many, which runtime, at what latency, at what cost per token, for this mix of workloads. Most answer them with vendor benchmarks and a safety margin. Inference Performance Assurance answers them with your models, your prompts and your traffic patterns, measured.

Benchmarking-as-a-Service and the Runtime Assessment are available now. The KPG Predictor is a working prototype validated against measured runs. Live-workload evaluation is in early access with design partners.

Book a Runtime AssessmentAsk about Benchmarking-as-a-Service

  1. Predict: KPG Predictor

    Prototypepatent-pending

    The patent-pending Kernel Pipeline Graph predictor builds a kernel-level model of a given model on a given GPU and runtime and predicts time-to-first-token, inter-token latency, throughput and cost per token without running the model. Fourteen GPU types, three runtimes, mapping to cloud instance types. It is how a Runtime Assessment produces a sizing before the first benchmark runs, and how the Traction Tuner chooses its starting point.

  2. Validate: live-workload evaluation

    Early access

    The open-source benchmark harness replays traces from your own workloads, chat, retrieval-augmented, long-context and code, at steady and bursty load, against upstream default, expert-tuned and predictor-sized configurations. It reports goodput at your SLO, p95 latency and cost per million tokens, with the exact configuration attached, and it is the same evaluation that gates every Tuner change and every LTS build.

  3. Benchmark and certify: Benchmarking-as-a-Service

    Available

    For an enterprise: we benchmark your models on the Secure Inference platform under your traffic shape, before you commit GPUs, and hand you a written baseline you can take to procurement. For a GPU operator: a signed certified-profile catalog per model, quantization, GPU type and count, context profile and concurrency level, refreshed quarterly as models and engines change, so you can quote a customer from measured data.

How it is delivered

  • Runtime Assessment.

    Fixed fee, credited against a Traction Runtime subscription. Prediction, hardened-profile application, harness run, written baseline and package recommendation.

  • Benchmarking engagement.

    Fixed fee per catalog, with a quarterly refresh option. Scope set by models, GPU types and concurrency levels.

  • Inside the products.

    Predictor priors and harness gates ship inside the Traction Tuner and the LTS build pipeline; no separate purchase.

Questions

Why not use published benchmarks?

Published benchmarks measure someone else's prompts at someone else's concurrency. Your cost per token depends on your prompt lengths, your output lengths, your cache hit rate and your traffic shape. We measure those.

Does this require Traction Runtime?

No. Benchmarking-as-a-Service runs on upstream SGLang, vLLM or Triton as deployed; the Runtime Assessment is specific to Traction Runtime for SGLang.

Are the numbers public?

Your numbers are yours. We publish only harness results we have reproduced ourselves, with the method, on the resources page.

Book a Runtime AssessmentAsk about Benchmarking-as-a-Service