Compression Dictionary Transport for Web Apps
Plan versioned compression dictionaries for related web assets with exact matching, freshness, identifiers, security boundaries, and baseline rollback.
Compression Dictionary Transport can compress a new web resource using bytes the client already has from a related dictionary, but only when matching and lifecycle rules are exact. This guide plans same-origin asset dictionaries with freshness, version overlap, negotiation, security boundaries, and an unconditional baseline fallback.
Compression Dictionary Transport needs a stable corpus
Dictionary compression is attractive when successive resources share substantial structure: versioned JavaScript bundles, CSS, schemas, catalogs, or documents with repeated vocabulary. The client has exact dictionary bytes from a prior response, and the server can encode a related response against them. A random collection of unrelated assets offers little reason to add the lifecycle complexity.
Start with a corpus analysis. Group resources by same-origin route, content type, release lineage, privacy boundary, and update cadence. Measure plain and ordinary compressed sizes, structural overlap, cache lifetime, and the fraction of returning clients likely to possess the proposed dictionary.
RFC 9842 defines Compression Dictionary Transport, while RFC 9841 defines shared Brotli details. Treat those documents as authoritative for field syntax, matching, and coding behavior.
The bundled planner hashes two tiny generated module strings and calculates a shared-prefix proxy. It explicitly does not implement the RFC content coding or claim wire savings; it decides only whether URL, freshness, and identifier gates would permit a dictionary path.
- Publish dictionary bytes at a stable same-origin URL.
- Advertise matching and freshness policy with an explicit version.
- Keep overlap long enough for clients holding the previous dictionary.
- Fall back to ordinary compression on every miss.
| Signal | Interpretation |
|---|---|
| Dictionary version lifecycle | A prior same-origin asset becomes an advertised dictionary, serves compatible descendants, ages out, and remains rollback-safe. |
Choose dictionary bytes and an immutable identity
A dictionary should be stable, cacheable, and representative of future resources. It may be a previous version or a deliberately trained corpus, depending on implementation and content. Publish exact bytes at an immutable URL or otherwise bind them to a strong identity so server and client never guess which version is meant.
Compute a cryptographic digest and record content type, byte length, creation revision, source corpus, security class, and retirement date. The identifier used by negotiation must select those exact bytes. A dictionary that changes in place under the same identity can turn a size optimization into decode failure.
Keep dictionary material free of secrets. Compression that combines attacker-influenced and confidential content has a history of side-channel risk, and shared dictionaries can amplify cross-context relationships. Partition by origin and sensitivity, and involve security review before using personalized or authenticated material.
Compression Dictionary Transport works best for public versioned assets whose reuse boundary is already clear. Begin there before considering data with complex authorization or tenant isolation.
Write narrow URL matching and freshness rules
The server advertises where a response can be used as a dictionary and which future resources may match. Keep patterns narrow enough that content type, encoding assumptions, and security context remain compatible. A broad /assets/* rule might accidentally cover images, source maps, private manifests, or a future format that cannot use the same dictionary safely.
Freshness determines whether the client may offer the dictionary later. Coordinate HTTP cache directives, dictionary lifetime, release cadence, and rollback window. A very short lifetime reduces availability; a very long one increases the number of versions the server may need to support.
Model misses explicitly: URL outside the pattern, stale dictionary, unknown digest, unsupported coding, or deployment rollback. Every miss should receive an ordinary compressed representation without an error or content mismatch.
The fixture has four constructed cases for match, route miss, expiry, and identifier mismatch. Its regex and age arithmetic are a bounded teaching model, not a substitute for the exact RFC matching algorithm.
Runnable artifact — The Node planner hashes exact dictionary bytes, applies URL and freshness gates, and labels its byte overlap as a non-encoder planning proxy.
import assert from "node:assert/strict";import {createHash} from "node:crypto";
const digest=value=>createHash("sha256").update(value).digest("hex");
const dictionary="export const palette=['coral','mint','ocean'];export const version='v1';";
const next="export const palette=['coral','mint','ocean','amber'];export const version='v2';";
const rules={match:"/assets/app-*.js",maxAge:604800,id:digest(dictionary)};
const cases=[{url:"/assets/app-v2.js",age:40,id:rules.id,expected:"dictionary"},{url:"/assets/vendor.js",age:40,id:rules.id,expected:"baseline"},{url:"/assets/app-v2.js",age:700000,id:rules.id,expected:"baseline"},{url:"/assets/app-v2.js",age:40,id:"wrong",expected:"baseline"}];
const route=x=>/^\/assets\/app-.+\.js$/.test(x.url)&&x.age<=rules.maxAge&&x.id===rules.id?"dictionary":"baseline";const results=cases.map(x=>({...x,actual:route(x)}));assert.deepEqual(results.map(x=>x.actual),results.map(x=>x.expected));const proxy={baselineBytes:Buffer.byteLength(next),sharedPrefixBytes:[...next].findIndex((c,i)=>c!==dictionary[i])};console.log(JSON.stringify({dictionaryDigest:rules.id,nextDigest:digest(next),rules,results,proxy,warning:"planning proxy; not RFC dcb encoder output"},null,2));console.log("PASS: dictionary planner rejects mismatch expiry and URL misses");
Follow the negotiation state machine exactly
Dictionary availability and response matching travel through defined HTTP fields and content codings. The server must select a dictionary the client actually indicated and bind the encoded response to its identifier. The client must decode with those exact bytes or reject the representation; trying a nearby version is not a recovery strategy.
Keep ordinary Brotli or gzip available when the dictionary path cannot be used. Varying and caching behavior must prevent a dictionary-coded representation from reaching a client that lacks the dictionary. Test intermediaries, edge caches, service workers, and origin behavior together because any layer can mishandle the negotiation.
Shared Brotli is a wire-format implementation concern, not the same as JavaScript CompressionStream. The browser export guide covers local compression workflows, while dictionary-compressed HTTP is negotiated between user agent and server.
Compression Dictionary Transport should emit logs for selected dictionary digest, content coding, fallback reason, response asset version, and cache layer. Omit sensitive URLs or query values from ordinary telemetry.
| Mismatch | Safe response |
|---|---|
| URL not matched | baseline coding |
| dictionary stale | baseline coding |
| identifier differs | never decode with guessed bytes |
| Signal | Interpretation |
|---|---|
| Use-As and Available-Dictionary ladder | Client and server exchange dictionary availability, match a response, identify the exact dictionary, and select or decline compressed coding. |
Plan rotation with overlapping versions
A release can advertise the current asset as a dictionary for the next one, but real clients skip versions and caches retain different generations. Decide how many dictionary versions the server supports and for how long. The new release should remain decodable through baseline compression for clients with no eligible dictionary.
During rollback, the old origin and edge configuration must still recognize any dictionary-coded variants that caches may serve, or purge them safely. Do not delete dictionary bytes while responses referring to them can remain fresh. Treat dictionary and encoded-resource lifetimes as a coupled deployment graph.
The version-lifecycle figure shows overlap because a single cutover instant is unrealistic. Keep a manifest of dictionary URL, digest, matching scope, first and last supported releases, and removal criteria. Automate consistency checks during deployment.
The service-worker update flow provides a parallel lesson: new and old clients coexist. Compression Dictionary Transport needs compatibility across that overlap rather than assuming everyone advances together.
Benchmark with a conforming encoder
Use the planning proxy only to select promising corpora. Then encode representative resources through the exact shared Brotli and HTTP implementation intended for deployment. Compare baseline Brotli, dictionary coding, header overhead, dictionary acquisition bytes, decode cost, cache hit behavior, and total bytes across first and repeat visits.
Include small, medium, and large changes; skipped releases; cold clients; warm clients; slow connections; and devices where decode cost matters. A favorable single bundle does not establish a site-wide policy. Report corpus inputs, encoder versions, quality settings, dictionary bytes, and cache assumptions beside results.
Measure end-to-end user outcomes only in a controlled rollout. Smaller transfer may not improve a page if the dictionary is unavailable, the asset is not critical, or cache and CPU costs dominate. The HTTP 103 Early Hints guide shows why transport changes need field attribution to the actual critical path.
Never label the fixture's shared-prefix count as RFC 9842 savings. Its warning remains in the JSON receipt so planning evidence cannot be copied into a performance claim without context.
Protect caches, origins, and failure recovery
Test cache keys and response variation so encoded bytes cannot be reused without their dictionary context. Verify integrity for dictionary and response resources, keep content-type and origin boundaries strict, and reject corrupted or mismatched bytes. Baseline responses must remain correct during unsupported clients, header stripping, cache loss, and partial deployment.
Compression can expose information through size. Avoid dictionaries constructed from secrets or shared across users whose content should be isolated. Review attacker influence over both dictionary and encoded resource. Security boundaries should be more conservative than the similarity corpus suggested by engineering convenience.
Add chaos cases for stale edge configuration, missing dictionary object, wrong digest, deployment rollback, intermediary removing fields, and service worker serving an old asset. Each should end in a clean baseline path or a clearly failed response, never corrupted content.
The HTTP caching foundations article helps place freshness and validators around this negotiation. Dictionary reuse adds another dependency; it does not suspend ordinary cache correctness.
- Local overlap estimates whether a corpus deserves an encoder benchmark.
- Only a conforming shared Brotli implementation produces deployment evidence.
- Do not share dictionaries across security contexts that should remain isolated.
| Signal | Interpretation |
|---|---|
| Baseline versus planning-proxy byte waterfall | Shared prefix bytes appear as a planning proxy beside full baseline bytes, with the encoder and security boundary clearly marked. |
Roll out dictionary transport with a kill switch
Choose one public versioned asset family and a cohort with measurable repeat traffic. Deploy dictionary advertisement, then coding support, then broader matching in separate steps. Watch fallback reasons, decode errors, cache behavior, transfer bytes, critical-path timing, and origin CPU before expanding.
Keep a kill switch that stops dictionary coding while preserving baseline assets and valid navigation. Store the active dictionary manifest and configuration with the release so operators can reproduce a response after a rollback. Alert on unknown digest and decode failure, not merely on size regression.
Compression Dictionary Transport is justified when conforming measurements show durable end-to-end value and the team can operate version overlap safely. It is optional complexity, not a default for every web application or asset.
Run the planner, replace its tiny module strings with a rights-clear versioned asset fixture, and advance to an RFC encoder benchmark only if match coverage and lifecycle assumptions survive the four failure cases.
The final rollout sheet should name RFC 9842 as the negotiation contract, shared Brotli as the coding, dictionary-compressed HTTP as the delivery path, and delta compression only as a neighboring idea rather than an interchangeable label. Re-run the synthetic mismatch corpus on each configuration revision. Compression Dictionary Transport remains enabled only while baseline responses, exact dictionary identifiers, and overlap removal dates are all observable.