Token Bucket Rate Limiter in TypeScript

A token bucket rate limiter permits a useful burst without surrendering the sustained capacity of the service behind it. The implementation is short; the difficult parts are clock choice, atomic spending, weighted work, fair scope, and truthful retry behavior.

This TypeScript tutorial writes that operating contract first, then proves it with virtual time. The result handles exact boundaries and backward clock movement without hiding distributed-storage or overload decisions.

Specify the token bucket rate limiter contract

A token bucket rate limiter holds up to a fixed capacity, replenishes tokens at a fixed rate, and spends a declared cost for each accepted request. Capacity permits a bounded burst; refill controls sustained throughput. Write those units before writing TypeScript: tokens, tokens per second, maximum burst, request cost, scope key, and the response returned when balance is insufficient.

The worked fixture starts with three tokens, consumes all three, rejects the next request, advances a virtual monotonic clock by one second, and accepts exactly one. That small trace distinguishes the algorithm from a fixed-window counter. It also provides a reviewable product statement: who may burst, for how long, and what the caller can safely do next.

The example policy prices a lightweight lookup at one token and states capacity and refill in the same units. A trace begins full, spends the three-token burst, rejects at zero, refills once after 1,000 milliseconds, and records both the balance and retry estimate; that makes the product behavior reviewable without reading the class implementation.

Runnable artifact: The virtual-clock fixture spends a burst, rejects at zero, refills exactly once, and resists backward time.

Save this proof as token-bucket.test.mjs and run node token-bucket.test.mjs. Expected final line: PASS: monotonic token bucket.

import assert from "node:assert/strict";
class Bucket{constructor(cap,rate,now){this.cap=cap;this.rate=rate;this.tokens=cap;this.at=now()} take(n,now){const observed=Math.max(this.at,now());this.tokens=Math.min(this.cap,this.tokens+(observed-this.at)*this.rate/1000);this.at=observed;if(this.tokens<n)return false;this.tokens-=n;return true}}
let t=0;const b=new Bucket(3,1,()=>t);assert.equal(b.take(3,()=>t),true);assert.equal(b.take(1,()=>t),false);t=1000;assert.equal(b.take(1,()=>t),true);t=500;assert.equal(b.take(1,()=>t),false);assert.equal(b.at,1000);t=1500;assert.equal(b.take(1,()=>t),false);assert.equal(b.tokens,.5);t=2000;assert.equal(b.take(1,()=>t),true);
console.log("PASS: monotonic token bucket");

Use a monotonic clock in the token bucket rate limiter

Elapsed time drives refill, so wall-clock changes must not mint or remove capacity. In Node.js, use a monotonic source such as performance.now() for process-local elapsed time; a distributed design needs a consistent storage-side or logical time policy. Clamp negative elapsed values to zero and update the last-refill timestamp atomically with the balance.

TypeScript rate limiting benefits from dependency-injecting now, because virtual time makes boundary tests instant and deterministic. The fixture deliberately moves its test clock backward after a valid refill and proves no new token appears. That is not a full distributed-clock solution, but it prevents an NTP adjustment or manual system-time change from becoming accidental capacity in a single-process limiter.

The local time choice is anchored in Node’s documented monotonic performance clock. The class also preserves a high-watermark timestamp: after observing 1,000 ms, a reading of 500 ms contributes zero and cannot move the watermark backward, so the later 1,500 ms tick adds only the genuine 500 ms elapsed since the highest observed time.

Make token bucket rate limiter math explicit

On every decision, compute min(capacity, previous + elapsed * rate), then subtract cost only if enough balance remains. Preserve fractional tokens unless the contract requires integer quanta, and define floating-point tolerance near the boundary. A request costing two tokens should not be split into two independent one-token decisions that another caller can interleave.

Return remaining balance and a conservative wait estimate calculated from the deficit divided by refill rate. Burst capacity is not bonus throughput; it is saved idle capacity. Dashboards should separate immediate bursts from sustained accepted rate so a healthy startup wave does not look like a long-term quota violation and an oversized bucket does not hide slow downstream collapse.

Fractional balance is observable in the regression test: the first forward tick after rollback creates only half a token and still rejects a one-token request; a second 500 ms advances the balance to one and accepts. That sequence catches the subtle bug that a simple “no token on backward read” assertion misses.

Choose token bucket rate limiter scope and fairness

Decide whether buckets belong to API key, user, tenant, IP, route, model, or a hierarchy of those keys. A tenant bucket protects shared capacity, while per-user children prevent one user from consuming the tenant's whole burst. Atomic storage is necessary when several workers share a bucket; otherwise each process can spend the same balance.

Request throttling also needs a queue policy. Immediate rejection is predictable and keeps latency bounded, but a short server-side queue can smooth tiny bursts if cancellation, maximum wait, and fairness are explicit. The comparison table uses an illustrative workload with interactive and batch callers to show why equal request counts are not equal cost and why FIFO alone can let expensive work block a fast path.

Hierarchical fairness is tested with a tenant budget and per-actor children using an atomic decision boundary. RFC 3290 supplies the traffic-meter background, while request costs, queueing, and tenant policy are explicitly this product’s choices; the article does not present those quotas as requirements of the RFC.

Ground the token bucket rate limiter in standards

RFC 3290 describes token-bucket traffic meters and the relationship between rate and burst. RFC 2697 defines a single-rate three-color marker, useful background for thinking about committed and excess capacity. The Node.js performance timing API documents the monotonic high-resolution clock used in a local implementation.

These sources do not choose product quotas. Translate the algorithm into a workload contract that states actor, operation cost, enforcement point, response semantics, and reset behavior. Cite the configured policy in incident traces so support can distinguish a correct rejection from a bug or a saturated dependency.

The standards section separates the generic bucket model from a three-color marker and from runtime clock behavior. This matters during review because a normative packet-traffic definition cannot justify a web-product retry header, and a process-local clock cannot by itself solve ordering among several workers using shared storage.

Test the token bucket rate limiter with virtual time

Cover an empty bucket, exact boundary, fractional refill, long idle clamp to capacity, weighted cost, simultaneous claims, backward clock, restart, storage timeout, and cancellation. For shared storage, execute concurrent attempts against the real atomic primitive and assert total accepted cost never exceeds initial balance plus elapsed refill. Fuzz elapsed intervals and costs around floating-point edges, then reproduce failures with a seed.

Continue quota design through API rate-limit workload contracts, pressure propagation in backpressure and flow control, user-level outcomes with journey SLOs, and overload policy in LLM admission control. These controls connect arithmetic to the capacity it protects.

Concurrency tests run the actual storage primitive with more simultaneous claims than available capacity and sum accepted cost, not accepted request count. A seeded virtual-time suite then covers fractional refill, long idle clamping, weighted operations, rollback, restart, and a forward tick after rollback, producing failures that can be replayed without waiting in real time.

Expose token bucket rate limiter decisions

Emit scope hash, policy version, capacity, refill rate, request cost, balance before and after, elapsed time, outcome, and wait estimate without logging secrets. Metrics include accepted and rejected cost, not only request count; bucket saturation by scope; wait-estimate accuracy; storage latency; contention retries; and downstream saturation. Response headers or body fields should be consistent and documented, but never promise an exact future slot when other callers share the same bucket.

The state diagram treats storage failure separately from rate exhaustion. Failing open may overload a critical dependency, while failing closed can create an outage; choose by route consequence and provide an emergency mode whose activation is observable and time-bounded.

Decision logs use hashed scope identifiers and include policy version, cost, elapsed interval, balance transition, and outcome. They omit raw API keys and avoid an exact promised retry instant when other callers share the same balance; the estimate is a lower bound under no additional consumption, which support copy states plainly.

Ship the token bucket rate limiter with boundaries

The receipt includes policy owner, protected resource, scope keys, costs, capacity, refill units, time source, fractional rules, atomic store operation, key expiry, restart behavior, response contract, retry estimate, queue and fairness policy, fail-open or fail-closed decision, metrics, alerts, virtual-clock suite, concurrent stress results, rollout cohort, and rollback. Fail release when wall time drives refill, workers can overspend one balance, a backward clock creates tokens, retry guidance lies, weighted operations use the same cost without justification, or keys grow forever. A compact implementation is possible because the behavioral contract is written first. The code then becomes an auditable translation of capacity policy rather than an isolated class that merely appears to throttle in a happy-path demo.

The rollout begins with shadow decisions that cannot reject traffic, compares predicted rejections with downstream saturation, and then enables a small cohort. Rollback restores the previous policy version rather than resetting balances to full, because minting a new burst during an incident would hide the very overload the limiter is meant to contain.

A token bucket rate limiter is a time-based capacity contract, not a counter with a timeout. Verify the token bucket rate limiter against a monotonic clock, burst boundary, fairness policy, and observable retry result.