Agent Tool Schema Evolution Without Breakage

Agent tool schema evolution becomes dangerous the moment yesterday's trace can resume, replay, retry, or explain a real side effect. A renamed field is not a refactor when a durable worker still holds the old arguments and a new default can move money or send a message.

This guide treats tool definitions as immutable contracts with semantic diffs, pure migrations, hermetic replay, support windows, and effect-aware retirement. Change remains possible because the runtime knows when meaning can be preserved—and when it must stop.

Agent tool schema evolution starts with recorded contracts

An agent trace is durable input. It may be replayed for an incident, resumed after a worker crash, evaluated against a new model, or inspected months after the tool changed. Agent tool schema evolution therefore needs the same discipline as a public data format: identify the exact contract used for each call and preserve enough semantics to interpret its arguments, result, error, and side effects later.

Store a tool identifier, contract version, schema digest, provider revision, call ID, arguments, result envelope, timestamps, authorization scope, and effect receipt. A tool name such as refund_order is insufficient because its fields and behavior can move while the string stays constant. The trace should point to an immutable schema snapshot or content-addressed registry entry, not whatever definition production serves today.

The Model Context Protocol tools specification defines discovery and call surfaces and makes tool annotations and result structure part of the integration boundary. Treat discovery output as versioned evidence. If a server updates a schema between planning and execution, reject, re-plan, or explicitly migrate; never validate the old plan against the new contract and pretend nothing changed.

Classify changes by semantic compatibility

Adding an optional field with a stable default can be backward-compatible for callers, but it may still alter downstream effects. Renaming a property, tightening an enum, changing units, moving from string to number, or making an optional field required is breaking. So is changing dry_run from false-by-default to true-by-default, even though the JSON shape remains identical. Compatibility belongs to behavior as well as syntax.

Use semantic versioning for tools as an operator convention: major for caller- or replay-breaking meaning, minor for additive capabilities, and patch for clarifications that do not alter accepted inputs or effects. Encode the version in the registry and trace. Do not rely on a suffix in the natural-language description, because models and humans can omit it when copying a request.

Create a change ledger with old/new schema digests, syntactic diff, semantic classification, migration owner, rollout date, and examples. Agent tool schema evolution should block publication when a new required value has no derivation from old traces. A guessed default may make validation pass while changing money, recipients, or data access. In those cases replay must stop for human reconciliation.

Use standards as the contract substrate

JSON Schema can express required properties, enums, bounds, conditionals, and unevaluated fields. The JSON Schema 2020-12 specification also defines identifiers, references, vocabularies, and annotation behavior needed for a real registry. Give every published contract an immutable $id, pin the dialect, resolve references at publication, and retain the complete bundle so replay does not depend on mutable network documents.

OpenAPI can carry operation identity, request and response shapes, examples, and transport details. OpenAPI 3.1.1 aligns its Schema Object with modern JSON Schema while retaining API-specific semantics. When a tool wraps an operation, record both the tool contract and source operation digest. The wrapper may deliberately narrow capabilities, rename fields, or add confirmation requirements.

A JSON Schema compatibility checker catches many structural breaks, but not unit changes, new side effects, or shifts in authorization. Extend it with organization rules: field descriptions cannot silently change currencies; idempotency cannot disappear; destructive annotations cannot weaken; result status cannot collapse terminal and pending states. Each exception needs a signed migration note and test fixture.

Write migrations as pure, directional functions

A trace migration should accept one versioned envelope and return another without calling live tools. Keep it deterministic, side-effect free, and independently testable. The runnable fixture renames order_id to orderId and supplies a documented reason default. That is safe only because unspecified has defined non-destructive behavior; if the new tool requires a refund amount, no pure migration can infer one responsibly.

Store argument migration code beside golden old inputs and expected new envelopes. Support one hop at a time, such as v1 to v2, then compose the chain. This makes each semantic decision reviewable and allows an incident investigator to identify which hop introduced a value. Avoid a single upgradeToLatest function that rewrites years of history without preserving intermediate reasoning.

Results need migrations too. A pending job may once have returned {id} and now return {operation:{id,state}}; evaluation, UI, and compensation code must still understand the recorded result. Preserve unknown fields in an extension bag when safe, but reject values that violate the destination's closed-world policy. Durable AI agent execution provides the checkpoint discipline that makes versioned envelopes useful across retries.

Replay old traces in a hermetic harness

Replay should not invoke production effects. Replace each tool with a recorded or deterministic simulator keyed by contract version, then run the original planner output through validation, migration, and result interpretation. Assert terminal state, planned call order, argument meaning, authorization, idempotency key, and user-visible outcome. Textual transcript equality is too brittle; semantic invariants survive harmless formatting and timing changes.

Build a trace replay harness with representative successes, validation failures, timeouts, partial effects, retries, cancellations, and compensation. Include traces from every supported major version. Multi-agent testing with causal traces shows how happens-before identity can replace fixed event ordering when concurrency is legitimate. The schema suite adds contract and migration hashes to each causal node.

Run three modes: historical, translated, and counterfactual. Historical uses the original agent and contracts to establish the archive can still execute. Translated migrates envelopes while holding decisions constant.

Counterfactual allows a new model to plan against the new schema, then compares effects and safety properties. Separating these modes tells whether drift came from the model, contract, migration, or simulator.

Runnable artifact: The fixture migrates an archived v1 argument envelope and validates the complete v2 meaning without calling a live tool.

Save this proof as tool-schema-replay.test.mjs and run node tool-schema-replay.test.mjs. Expected final line: PASS: old traces replay.

import assert from "node:assert/strict";
const migrate=v=>v.version===1?{version:2,orderId:v.order_id,reason:v.reason??"unspecified"}:v;
const validate=v=>v.version===2&&typeof v.orderId==="string"&&typeof v.reason==="string";
const old=migrate({version:1,order_id:"ord_7"}); assert.equal(validate(old),true);
assert.deepEqual(old,{version:2,orderId:"ord_7",reason:"unspecified"});
assert.equal(validate({version:2,orderId:7,reason:"x"}),false);
console.log("PASS: old traces replay");

Roll out dual-read and shadow-write boundaries

During a minor transition, accept old and new inputs at an adapter while emitting only the canonical new envelope internally. Log which version arrived and which migration ran. For a major transition, publish a new tool identity or require explicit capability negotiation. Do not make one endpoint guess the caller generation from property presence when overlapping shapes could mean different things.

Shadow new serialization without executing it. Compare validated arguments, policy classification, idempotency keys, and predicted effect against the active path. A canary can meter exposure after shadow evidence is clean. Keep the old adapter until every durable checkpoint, delayed job, and supported client version has passed its maximum lifetime or been migrated deliberately.

A tool contract registry should expose active, deprecated, and retired states with dates and consumers. Retirement means new plans cannot select the contract, not that its schema disappears. Replay and audit may need it indefinitely. If legal retention requires deleting trace payloads, retain non-sensitive schema metadata and migration identity so surviving decisions remain interpretable.

Treat errors and side effects as versioned data

Changing errors from strings to structured objects is a major improvement, but it can break retry logic that matched text. Define stable error codes, retryability, terminality, user action, and whether an effect may already have occurred. Version the envelope independently if needed. A timeout after a payment call is not equivalent to a validation rejection; old traces must retain the ambiguity so a new runner does not repeat the charge.

Side-effect annotations need executable tests. Classify read, create, update, delete, payment, message, and external publication; record confirmation and compensation requirements. AI agent compensation for failed tools supplies a ledger for confirmed effects and reverse actions. Agent tool schema evolution must preserve those identifiers through migrations even when display fields or transport paths change.

Reject a new contract that removes idempotency, weakens authorization fields, merges pending with success, or turns an optional notification into a default side effect. Such changes can be valid only as a new major capability with a fresh review. Schemas are valuable because they force hidden behavior into a diff, but the review must still ask what happens in the world.

Publish support windows and deletion tests

The release receipt lists tool IDs, old and new schema digests, compatibility classification, migration chain, golden traces, hermetic replay results, semantic invariants, shadow comparison, durable checkpoint horizon, client inventory, deprecation dates, retirement condition, and rollback. Agent tool schema evolution is complete when an operator can explain how any retained trace is interpreted and why it cannot trigger an unintended live effect.

Test absence as rigorously as success. A retired contract must disappear from discovery for new planners while remaining resolvable inside the replay registry. An unsupported ancient checkpoint must stop with a specific reconciliation state, not fall through to the latest schema. Delete a migration in a test copy and confirm the harness reports the missing hop rather than accepting partially upgraded data.

Connect contract review to agent-ready API design so descriptions, examples, auth, and side effects evolve together. The right final state is not permanent backward compatibility. It is bounded support with explicit translation and a truthful stop when meaning cannot be preserved. That makes change possible without treating yesterday's agent traces as disposable guesses.

Agent tool schema evolution succeeds when old meaning remains replayable and new meaning is named explicitly. Treat agent tool schema evolution as an effect contract whose retirement proof is as important as its migration code.