Consider adding a "fastest" LanguageModelSamplingMode value

[isaacahouma]

Background & Motivation

With #215 and #218, LanguageModelSamplingMode defines seven presets along the predictability and creativity spectrum ("most-predictable" through "most-creative"), defaulting to "balanced" when omitted.

In practice, different sampling modes can have significant implementation-specific performance differences. For example, greedy sampling ("most-predictable", where topK = 1) unlocks decoding accelerations such as Multi-Token Prediction (MTP) speculative decoding that are either unavailable or less effective under higher-temperature stochastic sampling.

Currently, web developers who want the lowest latency and highest token throughput must know to select "most-predictable" as a proxy for speed. However:

1. "most-predictable" expresses output determinism rather than latency intent, and it may not always be the fastest mode across all user agents or model architectures.
2. An explicit "fastest" mode would allow implementations to select the optimal sampling preset (such as "most-predictable") while potentially applying additional runtime or decoding optimizations tailored for speed.

Proposal

Explore adding "fastest" to LanguageModelSamplingMode:

enum LanguageModelSamplingMode {
  "fastest",
  "most-predictable",
  "predictable",
  "slightly-predictable",
  "balanced",
  "slightly-creative",
  "creative",
  "most-creative"
};

Open Questions for Discussion

1. Resolution of session.samplingMode: When a session is created with { samplingMode: "fastest" }, should session.samplingMode reflect the resolved preset selected by the implementation (for example, "most-predictable"), or should it remain "fastest" (in case "fastest" enables additional decoding optimizations beyond just topK and temperature)?
2. Default behavior when samplingMode is omitted: Per Update spec algorithms to include samplingMode #218, omitting samplingMode defaults to "balanced". Keeping "balanced" as the default and requiring explicit { samplingMode: "fastest" } preserves predictable general-purpose quality while giving latency-sensitive applications a dedicated opt-in.
3. Relationship to preference: "speed" in Writing Assistance APIs: The Writing Assistance APIs (Summarizer, Writer, Rewriter) use a preference option ("speed" vs. "capability"). Should "fastest" live on LanguageModelSamplingMode, or should we consider how it relates to preference?

cc @michaelwasserman @reillyeon @tomayac @KenjiBaheux

[tomayac]

My understanding is that the current LanguageModelSamplingMode lets developers choose on an axis:

<- most-predictable –––– most-creative ->

It feels like speed is an orthogonal concern mentally to developers, even if, with current model generations, there might be a relationship to the predictability–creativity axis.

I think it should be possible to say "be as creative as you can and be as fast as possible at it," even if it lowers the creative quality.

Also from an API shape similarity point of view, having a separate preference param seems preferable, even if some combinations, with current model architectures, might have no actual impact effectively.

[isaacahouma]

Thanks @tomayac, good point. Keeping LanguageModelSamplingMode strictly as a predictability to creativity axis and using a separate preference ("auto", "speed", "capability") parameter would be much more consistent with the Writing Assistance APIs.

The tricky part today is that in our current on-device implementation, the main decoding speedup (Multi-Token Prediction speculative decoding) is directly tied to greedy decoding ("most-predictable"), whereas in APIs like Summarizer, preference: "speed" routes to a smaller expert model.

If we go with a separate preference option on LanguageModel, one way to bridge both is:

If a developer passes { preference: "speed" } without an explicit samplingMode, the browser can default samplingMode to its fastest preset (e.g., resolving session.samplingMode to "most-predictable" instead of "balanced").
If they pass both { preference: "speed", samplingMode: "most-creative" }, we honor "most-creative" for sampling and apply any other model or runtime speedups available.
@michaelwasserman @reillyeon thoughts on going with preference vs. "fastest" on samplingMode?

[reillyeon]

The nice thing about putting "fastest" in samplingMode is that it makes it clear that this preference will affect samplingMode specifically. However this is also perhaps an implementation detail and this specific trade-off won't exist in the future.