Top-k vs Top-p vs Min-p: See the Sampling Sets

Top-k vs top-p vs min-p can keep different token sets even when they receive identical logits. This comparison visualizes that mechanical difference, verifies each boundary independently, and stops before making a generated-text quality decision.

Top-k vs top-p vs min-p starts with one frozen distribution

A fair comparison freezes logits, temperature, sorting, tie-breaking, and the random draw before any filter runs. The lab uses token IDs in their original order, subtracts the largest scaled logit before exponentiation, and then normalizes. Temperature therefore changes probabilities before top-k vs top-p vs min-p decides membership. This token probability filtering order is part of the receipt; moving temperature after filtering answers a different question and is one of the deliberate mutants.

Consider eight finite logits rather than prose generated by a model. A peaked preset makes one token dominant; a flat preset gives several tokens similar mass. The same three rules can look nearly identical on the first and sharply different on the second. That contrast is the useful lesson: truncated sampling is a transformation of a particular distribution, not a personality switch for a model.

Equal probabilities use original token index as the stable secondary key. That small contract prevents engines or refactors from changing a boundary token silently. Inputs are bounded to 2–64 tokens, temperature to 0.05–5, and the fixed uniform draw to the half-open interval [0, 1). The maximum is subtracted before softmax so large logits remain finite.

The resulting probability vector is the common evidence surface. Every later mask, retained mass, normalized interval, and sampled token points back to it. With that frozen surface, top-k vs top-p vs min-p becomes inspectable arithmetic instead of three names attached to three vibes.

Top-k keeps a fixed count

Top-k sorts probability and original index, then retains exactly min(k, N) entries. It does not inspect how much probability those entries contain. With k=3, a sharply peaked distribution and a nearly uniform distribution both keep three tokens even though their retained raw mass can be radically different.

That fixed-count behavior is both the appeal and the limitation. The control is easy to explain and the candidate budget is predictable. Yet a count has no intrinsic relationship to uncertainty: it may retain negligible tails beside one dominant token or cut away several plausible peers in a flat region. The lab exposes raw retained mass so that difference cannot hide behind the shared count.

Ties need a rule. If the third and fourth tokens have equal probability, this implementation keeps the lower original index. Another implementation may make a different documented choice, but an unstable sort is unacceptable for a reproducible receipt. The oracle rebuilds the sorted pairs independently and asserts that exactly k IDs survive in the required order.

This mechanical account does not make top-k the safe or fast choice. It only proves which token IDs the fixed-count rule would expose to the draw. For adjacent serving mechanics, the speculative decoding acceptance path explains a different optimization boundary: draft acceptance changes work, while this filter changes the target sampler's candidate set.

Top-p builds the smallest cumulative nucleus

Top-p, or nucleus sampling, sorts tokens from most to least probable and keeps the smallest prefix whose cumulative mass reaches the threshold. If the first token alone has probability 0.91 and p is 0.90, one token survives. If ten tokens each carry about 0.10, nine may be needed to reach the same target after floating-point details are accounted for.

The boundary token is included. An implementation that checks the cumulative sum before adding the current token can stop one position early; an implementation that always appends one more can stop late. The independent oracle accumulates sorted normalized probabilities, includes the token that first reaches p, and then stops. This kills the frozen top-p off-by-one mutant.

The original nucleus proposal by Holtzman and collaborators studies degeneration and generated-text behavior under a particular experimental setup. Its evidence motivates evaluation; it does not turn every p value into a universal quality guarantee. Here we borrow the rule definition and keep the synthetic result narrower.

Top-k vs top-p vs min-p is most revealing when the distribution shape changes. Top-p adapts its count to mass, but the retained count can jump when probability crosses the cumulative boundary. Record the full sorted cumulative ledger, not only the final IDs, so a reviewer can tell whether a change came from logits, temperature, threshold, or boundary handling.

Min-p uses a threshold relative to the maximum

Min-p retains token i when its probability is at least min-p multiplied by the maximum token probability. At min-p=0.10 and a maximum probability of 0.60, the absolute cutoff is 0.06. When the maximum is 0.20, the same setting creates a 0.02 cutoff. The threshold follows the distribution's peak rather than fixing a count or a cumulative mass target.

That relative definition is the entire mechanism. Comparing each probability directly with 0.10 is an absolute-threshold mutant and must fail. The receipt therefore stores min-p, the maximum probability, the derived absolute threshold, every token probability, and every membership decision. A reader can reconstruct the mask without trusting the interface.

The min-p paper by Nguyen and collaborators defines min-p sampling and reports experiments under its chosen models and settings. A later reanalysis by Cooper and collaborators disputes broad superiority claims and illustrates why implementation detail and evaluation scope matter. Those publications are evidence about reported experiments, not permission to crown a winner from this toy distribution.

In top-k vs top-p vs min-p, min-p often retains fewer alternatives around a high peak and more peers when the distribution is flatter. “Often” is intentionally conditional. The exact set is printed for both presets. Generated-text quality, safety, diversity, and latency remain unanswered until the filter is placed inside a controlled workload evaluation.

Renormalize first, then map one fixed draw

Filtering produces a mask, not a valid categorical distribution. Add the retained probabilities to obtain raw retained mass, divide each survivor by that mass, and verify the new values sum to one within 1e-12. The missing-renormalization mutant leaves the mass below one and fails immediately.

Sampling uses half-open intervals. Beginning at zero, each retained token owns [start, end), except the final interval is explicitly closed against floating-point residue by setting its end to one. A draw of 0 belongs to the first interval; a draw just below 1 belongs to exactly one final interval. The unclosed-final-interval mutant demonstrates why a mathematically obvious endpoint still needs an executable assertion.

The fixed draw is not a claim about randomness quality. It is a replay handle. Give top-k vs top-p vs min-p the same uniform value and each rule maps that value through its own renormalized cumulative distribution. Different selected tokens then follow directly from different masks and interval widths, not from different random seeds.

This distinction is also useful beside constrained JSON decoding. Grammar constraints can make tokens invalid for structural reasons; probability filters remove candidates according to a sampling policy. If both operate in one stack, document their order. The lab fixes temperature, then filter, then renormalization, then draw, and binds that sequence into the receipt hash.

Compare peaked and flat probability skylines

Run the peaked preset first and read each ribbon left to right. The skyline is shared, while the mask shape changes. Top-k shows the promised count. Top-p shows the cumulative stopping token. Min-p shows the horizontal cutoff computed from the highest bar. The adjacent list repeats retained IDs and raw mass so color never carries the conclusion alone.

Now switch to the flat preset without changing temperature or the draw. Several probabilities cluster near one another. A small numeric change can move the top-p boundary, while top-k remains fixed and the relative min-p cutoff moves with the shallow maximum. This is the visual reason to examine more than one distribution shape.

Top-k vs top-p vs min-p can also disagree while selecting the same sampled token. Membership and draw are separate observations. A production evaluation should preserve both: the mask explains which alternatives were possible, and the selected interval explains the specific replay. Comparing only output text throws away that diagnostic layer.

For evaluation design, confidence intervals for LLM evals explains why a collection of paired outcomes needs uncertainty rather than a single win rate. The synthetic skylines here are deliberately smaller. They prove set mechanics for declared inputs; they do not prove which setting writes better code, tells safer stories, or serves faster on a particular runtime.

Build an independent oracle and kill boundary mutants

The interface implementation begins with logits and computes a temperature-adjusted softmax. The oracle deliberately begins with the resulting normalized probabilities, constructs fresh probability-index pairs, and implements each membership rule without calling the display helpers. Sharing a sort-and-filter helper would produce agreement without independence.

Seven invariants define success: one or more tokens survive; normalized retained mass is within 1e-12 of one; top-k retains exactly min(k, N); top-p retains the smallest reaching prefix; min-p retains every value at or above the relative cutoff; one draw maps to exactly one interval; and repeated canonical inputs produce the same hash. Each invariant appears as a named boolean in the export.

The mutation suite changes one idea at a time: top-p stops before its boundary, min-p becomes absolute, filtering happens before temperature, mass is not renormalized, equal probabilities lose their index tie-break, and the final interval remains open. Each mutant must disagree with a frozen expected result or violate an invariant. A green interface without these negative controls is only a demo.

The canonical JSON serializer orders object keys and preserves array order. Change a decision input—temperature, filter value, draw, logits, or tie policy—and validation recomputes a different hash. JSON and CSV are reparsed rather than accepted because a download occurred. That makes top-k vs top-p vs min-p evidence portable and tamper-evident within this teaching contract.

Turn top-k vs top-p vs min-p into a model evaluation

A useful next experiment holds model revision, prompt set, decoding order, token budget, stop conditions, and seeds constant. Run paired prompts across candidate settings, keep the complete sampling receipts, score with a task-specific rubric, and report uncertainty. Measure latency separately because candidate-set arithmetic does not predict the serving stack's end-to-end behavior.

Do not aggregate away failure modes. Code tasks may need compilation and tests; extraction tasks may need schema validity and field accuracy; conversational tasks may need blinded preference plus safety review. The measurement-contract guide for AI evals shows how to declare the unit, rubric, exclusions, and release decision before results arrive.

This article's truth boundary stays strict: its synthetic distributions prove membership, renormalization, and fixed-draw mechanics only. They do not establish generated-text quality, safety, diversity, latency, or a universally best sampler. Even published comparisons remain scoped to their models, prompts, implementations, and analysis.

Top-k vs top-p vs min-p is therefore not a podium. It is a set of three inspectable rules followed by an evaluation obligation. Preserve paired prompts, exact seeds, model and tokenizer revisions, task rubric, per-example outcomes, and statistical uncertainty. When a production recommendation changes, the receipt should make clear whether the cause was probability shape, filtering policy, random draw, or evidence from the real workload.

Runnable local artifact — Synthetic next-token distributions prove candidate-set and sampling mechanics only; they do not establish text quality, latency, safety, or a best model setting.

Enter 2–64 finite logits, temperature, top-k, top-p, min-p, and a fixed uniform draw; compare three masks, independent oracles, invariants, mutation kills, and validated exports.