Semantic Entropy for LLM Uncertainty

Semantic entropy turns sampled LLM disagreement into a meaning-level uncertainty signal instead of counting paraphrases as separate answers. This tutorial builds the clusters, entropy calculation, and answer-review-abstain receipt step by step.

Semantic entropy measures disagreement in meaning

Semantic entropy estimates whether sampled answers agree after differences in wording are collapsed. Ask a model the same question several times, group answers that assert the same thing, add the probability assigned to each group, then measure how dispersed that mass is. Low dispersion supports an answer; higher dispersion creates a reason to review or abstain.

Token probability alone cannot make that distinction. “Paris is the capital of France” and “France's capital is Paris” are different strings but one semantic answer. Conversely, two fluent completions can use similar vocabulary while making incompatible claims. The unit of uncertainty must match the unit that matters to the reader. Semantic entropy preserves that distinction in the measurement record.

The Nature semantic-uncertainty study connects this meaning-level signal to detecting confabulations. It does not turn one threshold into a universal safety guarantee. Sampling policy, clustering method, domain, prompt, model, and cost of error all shape the operating point.

This tutorial builds a small, inspectable estimator rather than hiding those choices. Its synthetic fixture carries answer text, probability weight, and a declared meaning-cluster ID. The receipt reports lexical uncertainty separately from semantic uncertainty and applies a versioned answer, review, or abstain policy. That separation is the first safeguard against treating a persuasive scalar as truth.

Freeze the question and sampling contract

LLM uncertainty estimation begins before the entropy calculation. Freeze the complete prompt, relevant retrieved context, model revision, decoding parameters, sample count, and any structured-output schema. If those inputs drift between samples, disagreement can reflect a changing experiment rather than uncertainty within one request.

Draw samples independently under the declared decoding policy. Preserve the raw answer and its weight, but do not assume a sequence log-probability is automatically comparable across answers of different length. A practical first pass can use uniform sample weights. A more advanced implementation may normalize model likelihoods, provided that normalization is explicit and evaluated for the target model.

Sample count is a budget decision. Too few samples make a rare alternative easy to miss; unlimited sampling burns latency without removing clustering error. Record the count and stop rule in the receipt. Use confidence intervals for LLM evaluations when comparing estimator behavior across a test set, because one request-level entropy is not population evidence.

Keep responses that fail parsing as first-class outcomes. Silently dropping them makes the remaining distribution look more certain. A malformed structured response can occupy its own failure cluster, then route to review under the same policy. The goal is not a cosmetically complete cluster chart. It is a faithful account of every sampled outcome that could affect the decision.

Cluster answers by entailment, not vocabulary

Semantic answer clustering asks whether two responses can coexist as the same answer to the frozen question. For short factual replies, bidirectional entailment is a useful operational test: answer A should entail answer B and B should entail answer A under the question context. One-way entailment often means one response adds a material qualification and deserves separate treatment.

Clustering can be performed by human labels on an evaluation set, a natural-language-inference model, a carefully constrained judge, or a domain-specific canonicalizer. Each choice introduces error. If a judge performs the grouping, evaluate that judge independently and retain pairwise decisions. The ICLR semantic-uncertainty paper is the primary research reference for treating semantic equivalence as the aggregation unit.

Do not use embedding distance as an unexplained synonym for equivalence. Nearby answers may contradict each other around a number, negation, actor, or date. Begin with normalized entities, units, and propositions; then review the borderline pairs that can change the policy outcome. The deterministic lab accepts declared cluster IDs specifically so its arithmetic remains auditable.

The production boundary should expose an “unclustered” path. When pairwise relations are cyclic, incomplete, or below confidence, preserve separate mass or request review. Semantic answer clustering is evidence preparation, not permission to force ambiguous samples into the nearest attractive story. Semantic entropy is only as trustworthy as that grouping boundary.

Turn sample weights into semantic mass

After grouping, sum the normalized probability weights inside each cluster. Suppose a constructed fixture has five samples with weights 0.30, 0.24, 0.20, 0.16, and 0.10. Three paraphrases share cluster A for total mass 0.74, while incompatible clusters B and C hold 0.16 and 0.10. Lexical diversity is high because all five strings differ; semantic entropy is lower because most weight agrees in meaning.

For cluster probabilities p, calculate the negative sum of p multiplied by the natural logarithm of p. The unit is nats when the natural logarithm is used. Also report normalized entropy by dividing by the logarithm of the number of nonempty clusters. The normalized value is convenient for a policy across varying cluster counts, but the cluster count and raw value must remain visible.

Avoid rounding before aggregation. Validate that weights are finite and nonnegative, reject empty input, and normalize once. A zero-mass cluster should not exist in the receipt. The first figure shows sample leaves becoming meaning-level mass; its table makes the aggregation reproducible without relying on color.

Semantic entropy is sensitive to the tail. Changing one low-weight answer to a new contradictory cluster can cross a review threshold even while the leading answer remains unchanged. That is useful: the receipt explains the new alternative rather than laundering it into a top-one confidence score.

Calibrate a review and abstention policy

A score becomes operational only after it is mapped to consequences. Define separate bands for answer, review, and abstain, then tune them against labelled domain examples and explicit error costs. The illustrative lab answers below 0.35 normalized entropy when the leading cluster also owns at least 0.70 mass, reviews the middle region, and abstains at or above 0.75. Those values teach boundary behavior; they are not recommended production thresholds.

Calibration deserves the same care as classification. The classic neural-network calibration study explains why confidence and correctness can diverge, although its setting is not this estimator. Measure selective accuracy, coverage, review load, and costly-error rate across the intended distribution. Recalibrate after a model, prompt, retrieval, or clustering change.

Pair semantic entropy with other independent gates. Unsupported citations, policy violations, missing required fields, and stale retrieval should fail their own checks even when every sample agrees. For typed responses, semantic validation for structured outputs can reject a unanimously malformed claim before uncertainty policy runs.

LLM abstention copy should be concrete. “I found materially different answers and need review” is more honest than a generic apology. Expose the leading alternatives internally, redact sensitive content from logs, and retain the policy version. A threshold without its surrounding evidence cannot explain why the system withheld an answer.

Run the deterministic uncertainty lab

The runnable artifact uses a synthetic question and five constructed samples. Each sample has a unique text form, a probability weight, and a manually declared cluster. It validates every input, computes exact-text and meaning-level distributions, then prints raw and normalized entropy, leading mass, policy thresholds, decision, assumptions, and a SHA-256 receipt hash.

Run the program twice and compare output byte for byte. The independent test does that replay, verifies the cluster masses sum to one, recomputes the entropy from raw masses, and checks the receipt digest. An alternate flag moves one sample into a new meaning cluster, so the test requires a changed receipt and a nondecreasing semantic score.

Hostile fixtures cover empty samples, negative weights, nonfinite weights, blank cluster IDs, and probabilities whose total is zero. The program refuses these inputs instead of returning a comforting number. Boundary fixtures land exactly on the review and abstain thresholds to document inclusive comparisons.

This lab is deliberately narrower than an end-to-end hallucination detection system. It does not infer entailment, query a model, prove factuality, or validate a production threshold. It proves deterministic aggregation and policy routing for declared inputs. Replace the fixture with logged evaluation samples only after access controls and retention rules are set, and keep the raw grouping evidence beside the compact receipt.

Runnable artifact — Deterministic probability aggregation over declared synthetic clusters; not production model calibration or a factuality verdict.

import assert from "node:assert/strict";
import { createHash } from "node:crypto";
const sha=value=>createHash("sha256").update(JSON.stringify(value)).digest("hex");
const fixtures={base:[{text:"Paris is France's capital",weight:.30,cluster:"A"},{text:"The capital of France is Paris",weight:.24,cluster:"A"},{text:"France has Paris as its capital",weight:.20,cluster:"A"},{text:"Lyon is the capital",weight:.16,cluster:"B"},{text:"The answer is Marseille",weight:.10,cluster:"C"}],alternate:[{text:"Paris is France's capital",weight:.30,cluster:"A"},{text:"The capital of France is Paris",weight:.24,cluster:"A"},{text:"France has Paris as its capital",weight:.20,cluster:"D"},{text:"Lyon is the capital",weight:.16,cluster:"B"},{text:"The answer is Marseille",weight:.10,cluster:"C"}]};
function distribution(samples,key){if(!Array.isArray(samples)||samples.length===0)throw new Error("empty-samples");let total=0;const mass=new Map();for(const sample of samples){if(!sample||typeof sample[key]!=="string"||!sample[key].trim())throw new Error("invalid-label");if(!Number.isFinite(sample.weight)||sample.weight<0)throw new Error("invalid-weight");total+=sample.weight;mass.set(sample[key],(mass.get(sample[key])||0)+sample.weight)}if(!(total>0))throw new Error("zero-total");return[...mass].map(([label,weight])=>({label,mass:weight/total})).sort((a,b)=>b.mass-a.mass||a.label.localeCompare(b.label))}
function entropy(rows){const raw=-rows.reduce((sum,row)=>sum+(row.mass?row.mass*Math.log(row.mass):0),0);return{raw,normalized:rows.length>1?raw/Math.log(rows.length):0}}
function decide(score,leading,policy){for(const value of [score,leading,policy.answerMax,policy.abstainMin,policy.leadingMin])if(!Number.isFinite(value))throw new Error("invalid-policy");if(score>=policy.abstainMin)return"abstain";if(score<=policy.answerMax&&leading>=policy.leadingMin)return"answer";return"review"}
const samples=process.argv.includes("--alternate")?fixtures.alternate:fixtures.base;const lexical=distribution(samples,"text"),semantic=distribution(samples,"cluster"),semanticScore=entropy(semantic),policy={version:"illustrative-v1",answerMax:.35,abstainMin:.75,leadingMin:.70};const hostile={empty:"",negative:"",blank:"",nonfinite:"",zero:""};for(const [key,input] of Object.entries({empty:[],negative:[{text:"x",cluster:"A",weight:-1}],blank:[{text:"x",cluster:"",weight:1}],nonfinite:[{text:"x",cluster:"A",weight:Infinity}],zero:[{text:"x",cluster:"A",weight:0}]})){try{distribution(input,"cluster")}catch(error){hostile[key]=error.message}}
const core={schema:"semantic-entropy-policy-receipt-v1",fixture:"synthetic answers with manually declared meaning clusters",assumptions:{logBase:"natural",weights:"declared then normalized",clustering:"manual IDs; no entailment inference"},samples,lexical:{distribution:lexical,...entropy(lexical)},semantic:{distribution:semantic,...semanticScore},leadingMass:semantic[0].mass,policy,decision:decide(semanticScore.normalized,semantic[0].mass,policy),boundaries:{review:decide(.35,.69,policy),answer:decide(.35,.70,policy),abstain:decide(.75,.90,policy)},hostile,claimBoundary:"Deterministic aggregation and routing for declared synthetic clusters only; not factuality, production calibration, or a universal threshold."};assert.ok(Math.abs(semantic.reduce((sum,row)=>sum+row.mass,0)-1)<1e-12);console.log(JSON.stringify({...core,receiptHash:sha(core)},null,2));console.log("PASS: semantic mass, lexical separation, policy boundaries, hostile inputs, mutation, and digest verified");

Audit disagreement before trusting the scalar

Review the topology, not only the final semantic entropy value. A large leading cluster plus one severe contradictory cluster may deserve a different action than several harmless wording variations with the same score. Record representative answers, mass per cluster, the pairwise equivalence method, unresolved edges, and the consequences of choosing the leading cluster.

Separate epistemic disagreement from task ambiguity. If the question permits several valid interpretations, revise the prompt or ask the user before calling the alternatives hallucinations. If retrieved evidence conflicts, cite and reconcile the source versions. If every sample repeats the same unsupported statement, low uncertainty is still not correctness. Hallucination detection needs evidence validation alongside response consistency.

Use LLM judge calibration when a model labels equivalence or reviews alternatives. Blindly allowing the answering model to approve its own clusters creates a correlated failure path. Sample and judge versions belong in separate fields so a change in either component can trigger a targeted re-evaluation.

Monitor by domain and consequence. A travel suggestion, medical instruction, and destructive admin action should not share one coverage target merely because their entropy distributions look similar. Semantic entropy offers a useful signal precisely because it preserves disagreement. Governance should preserve that nuance rather than compress it into one green badge.

Ship a receipt that can survive model change

A durable receipt contains the question fingerprint, model and prompt versions, sampling contract, sample count, weight method, cluster method, cluster masses, raw score, normalized score, leading mass, policy version, decision, and limitations. Hash the normalized record so later analysis can detect accidental mutation. Store sensitive answer text under a stricter retention policy than the aggregate.

Treat the policy as versioned product behavior. A lower review threshold increases human load and may improve selective quality; a higher threshold increases automatic coverage and accepts more disputed answers. Compare changes on a frozen evaluation set before rollout, then watch distribution shift rather than assuming the old calibration transfers.

If an application already uses conformal abstention for language models, keep the two contracts distinct. Conformal methods target a stated coverage guarantee under assumptions; this meaning-level estimator summarizes sampled disagreement. They can inform the same routing decision, but neither number should impersonate the other.

The practical outcome is a reviewable chain: frozen request, sampled possibilities, explicit meaning groups, auditable mass, calibrated action, and a stable receipt. That chain makes uncertainty useful without pretending it is certainty about truth.