From production data to faster inference: the model optimization loop

Speculative decoding is often presented as a serving optimization, but in production its effectiveness depends on how well the draft model matches the traffic it actually serves.

In this webinar, we’ll explore the practical optimization loop in Token Factory from capturing production logs and S3-backed datasets in Data Lab, to transforming them into training and evaluation datasets, and choosing the right optimization strategy: supervised fine-tuning (SFT), distillation, or custom speculator training.

Using speculative decoding as a concrete example, we’ll show how teams can turn real production data into workload-specific draft models and make informed trade-offs between latency, throughput, quality, and cost instead of optimizing for token price alone.

You’ll leave with a practical understanding of:

how production data becomes reusable training and evaluation data
where Data Lab, SFT, distillation, and custom speculator training fit into the optimization workflow
why speculative decoding is workload-dependent and how to determine if it’s the right approach
which metrics to measure before and after optimization to evaluate real production impact

Who should attend

This session will be especially valuable for teams looking to reduce inference latency and cost, improve GPU efficiency, and establish a repeatable workflow for optimizing models using real production data