Structured Concurrency for AI Agents

Structured concurrency for AI agents answers a concrete failure question: when one parallel tool call fails or the user presses stop, who cancels and joins the rest? This tutorial builds that ownership with AbortController, explicit cleanup, and a receipt that proves no child work escaped the run.

Structured concurrency for AI agents starts with ownership

A parallel run should form a tree: the request owns a run scope, the run owns tool tasks, and every tool task owns its network request, stream, lock, or temporary file. Structured concurrency for AI agents means the parent cannot finish while a child remains live. Promise creation alone does not provide that guarantee; the coordinator must retain every child handle and join each terminal state.

Write the contract before code: success joins all required results, first fatal failure aborts siblings, a user stop aborts the entire subtree, and shutdown waits for bounded cleanup. The durable AI agent execution pattern can persist checkpoints, but durability is not permission to leave in-memory children detached. Name the run ID, parent task ID, child IDs, deadline, and cleanup owner so a trace can explain which scope was responsible.

Synthetic teaching fixture (not production data): a research parent receives a 1,200-millisecond budget and registers search, quotation, and citation lookup as three children before any starts. The scope owns their handles in launch order. The expected contract lets synthesis receive results only after the registry reports three terminal children, even if the fixture marks the first useful source at 180 milliseconds.

Build one AbortController tool-call tree

Create one controller at the run boundary and derive child signals rather than manufacturing unrelated timeouts inside adapters. Each tool accepts an AbortSignal cancellation parameter and checks it before expensive preparation, before dispatch, while reading a stream, and before committing a result. AbortController tool calls stay composable because the coordinator can cancel them without knowing transport details.

The signal is advisory unless every layer observes it. Pass it through fetch, database helpers that support cancellation, stream readers, retry loops, and local CPU work with explicit checkpoints. If a dependency cannot cancel, isolate it behind a bounded adapter and report that limitation. Structured concurrency for AI agents is strongest when the task tree mirrors resource ownership, not merely the visual nesting of async functions.

The scope API has only three powers: fork an owned child, abort the subtree, and join the registry. A child cannot detach itself or replace the parent signal. That deliberately small surface prevents an adapter from creating an invisible retry timer after its fetch has already been cancelled.

Propagate deadline and abort reason together

Use the request deadline as a budget shared by planning, tools, and synthesis. A child may receive less time, but never more than its parent. The WHATWG DOM abort algorithms define signal state and reason propagation; the MDN AbortController reference documents the browser-facing API. Preserve a structured reason such as user_stop, sibling_failure, deadline, or shutdown instead of collapsing every exit into a generic error.

Reasons decide product behavior. A user stop should preserve a resumable transcript; a deadline may offer retry; sibling failure may trigger compensation. Structured concurrency for AI agents needs one canonical terminal cause plus secondary cleanup errors, because whichever promise rejects first is not always the most informative reason the run ended.

Synthetic budget fixture (not observed latency): planning receives 140 milliseconds and synthesis reserves 260 within a 1,200-millisecond parent, leaving at most 800 for tools. Search receives 800 and quotation 500 because it depends on a search hit. Both expected child budgets still carry the parent's deadline and typed user-stop reason.

Join cleanup in finally, not in hope

Put resource release in finally blocks owned by the scope that acquired the resource. Close readers, release locks, stop timers, delete bounded scratch files, and detach listeners even when the adapter throws during setup. Parallel tool cleanup must be idempotent because abort, natural completion, and process shutdown can converge on the same release path.

Do not resolve the parent receipt until cleanup has either completed or reached a declared timeout. Record unresolved resources separately rather than labeling the run cancelled while sockets or workers remain active. The failed-tool compensation guide covers reversing committed business effects; cleanup is narrower and should happen first. Structured concurrency for AI agents distinguishes releasing runtime resources from compensating durable side effects.

Synthetic cleanup fixture (not a measured run): assign 150 milliseconds per child and 250 to the scope after abort. Expected events close the search reader at +7 milliseconds, release a quotation cache lease at +18, and mark citation lookup cleanup_timeout at its 150-millisecond ceiling. The parent receipt can then finish without pretending all fixture resources closed cleanly.

Choose failure policy per sibling group

Not every tool failure should cancel every sibling. Declare whether a group is fail-fast, collect-all, quorum, or optional before launching it. Search across three sources may tolerate one failure; simultaneous debit and notification cannot be treated as interchangeable results. The agent task tree should place children with the same failure policy under the same supervisor rather than burying policy in catch blocks.

Use Promise.all only after wrapping tasks in owned handles that preserve abort and join semantics. Promise.all rejects early but does not cancel its remaining inputs. Promise.allSettled waits, yet can keep doomed work running. Structured concurrency for AI agents adds the missing supervision decision: abort the relevant subtree, await every child, and then construct one terminal receipt containing primary and cleanup outcomes.

The research children use collect-all because two sources can still answer the question; a later purchase subtree uses fail-fast because inventory reservation and charge authorization form one decision. Those policies live on supervisor nodes. A reviewer can therefore see why one failed citation did not cancel search while one failed authorization did cancel its sibling.

Fence side effects after cancellation

Cancellation can arrive between validation and commit. Check the signal immediately before a side effect, pass an idempotency key to the tool, and make the remote operation reject stale run epochs when possible. If the effect commits as cancellation arrives, report committed_after_abort as a distinct state and enter compensation rather than pretending the stop prevented it.

The Node.js AbortController documentation includes signal composition helpers available in current runtimes. Treat runtime support as versioned evidence and keep a small compatibility adapter. Structured concurrency for AI agents cannot make arbitrary remote systems transactional, but it can make the ambiguity visible, bounded, and recoverable through an effect ledger tied to the child task.

Synthetic effect-race fixture (not a provider event): assign quote acceptance at 642 milliseconds and user stop at 641 on a local clock. The expected result is not “cancelled.” Its ledger records request q-73 as committed_after_abort, blocks automatic reuse, and schedules a stub idempotent release before the run becomes resumable.

Run the bounded teaching fixture before adapting the pattern to production.

Runnable artifact — structured-tool-scope.test.mjs

import assert from "node:assert/strict";const controller=new AbortController();const receipt=[];const tool=async(name)=>{try{await new Promise((_,reject)=>controller.signal.addEventListener("abort",()=>reject(controller.signal.reason),{once:true}))}catch{receipt.push(name+":aborted")}finally{receipt.push(name+":clean")}};const children=[tool("search"),tool("quote")];controller.abort(new Error("user_stop"));await Promise.all(children);assert.deepEqual(receipt.sort(),["quote:aborted","quote:clean","search:aborted","search:clean"]);console.log("PASS: abort joins every owned tool");

Run node structured-tool-scope.test.mjs. Expected receipt: PASS: abort joins every owned tool.

Test cancellation at hostile boundaries

Inject abort before dispatch, during headers, between streamed chunks, after a response but before persistence, during cleanup, and after one sibling has committed. The AI agent fault injection corpus is useful because timing bugs rarely appear in the happy path. Assert that every child reaches a terminal state, no new retry starts after abort, timers are cleared, and compensation receives committed effects exactly once.

Use a deterministic deferred promise in unit tests instead of sleeps. The harness controls when each phase advances, aborts the parent at a named barrier, then inspects the receipt. Structured concurrency for AI agents is proven by absence as much as output: no orphaned log events, no unhandled rejection, no leaked listener, and no tool result written after the run closed.

Synthetic hostile receipt (not a captured trace): 000 launch search; 004 launch quote; 087 search headers; 091 user_stop; 092 abort both; 099 search reader closed; 111 quote socket closed; 113 both settled; 114 scope joined. A constructed callback at 140 sees the closed epoch and must write nothing; that expected absence is the race assertion.

Trace the task tree without leaking payloads

Emit run_started, child_started, abort_requested, child_settled, cleanup_finished, and run_joined with monotonic timestamps, IDs, reason codes, and durations. Avoid logging tool arguments or model content by default. The multi-agent causal traces approach connects parent and child events so an operator can distinguish a slow tool from slow cleanup.

Expose active-child count and oldest-child age as gauges. A run marked finished with a nonzero child count is an invariant violation, not merely an observability oddity. Structured concurrency for AI agents should make the join point visible: product UI can say “stopping tools” while cleanup runs, then announce a final stopped state only after the coordinator has its complete receipt.

Synthetic observability fixture (not production data): the public trace keeps task IDs, phases, reason codes, and durations; a restricted store holds argument hashes and effect IDs. Its example alert conditions are joined runs with active_child_count above zero and p95 abort-to-join above 250 milliseconds. Neither illustrative signal requires a prompt, query, or provider payload in logs.

Ship a cancellation contract, not a helper

Document which adapters honor abort promptly, which only stop between phases, which side effects use idempotency, and which cleanup paths can time out. Include the exact runtime, fault matrix, maximum cancellation latency, and known non-cancellable dependency. That turns structured concurrency for AI agents into an operating contract reviewers can challenge rather than a wrapper named runParallel.

Roll out with shadow metrics before making cancellation a hard release gate. Compare orphan counts, post-stop writes, cleanup duration, and user-visible stop latency. Revisit the contract whenever a tool SDK, retry library, worker model, or remote effect changes. The useful finish line is modest and testable: every launched child is owned, every stop propagates, every scope joins, and every ambiguity becomes a receipt.

Proposed release fixture (not completed SDK certification): place the adapter matrix beside a replayable 091-millisecond stop boundary. A future SDK version should reproduce the joined event sequence, keep post-stop writes at zero, and stay within a separately measured cleanup ceiling rather than passing merely because its method accepts a signal.