Background Mode vs Batch API for Agent Jobs

Background mode vs Batch API is not a simple fast-versus-cheap choice; the real boundary is whether one person is waiting on one evolving response or a system is submitting a stable set of independent requests.

This guide turns that boundary into a routing worksheet covering deadlines, volume, cancellation, privacy, reconciliation, and the completion receipt your product must show.

Background mode vs Batch API starts with waiting

A support copilot that drafts one investigation while an operator watches is still an interactive product even if generation takes several minutes. It needs a durable response identifier, a visible pending state, cancellation language, and a way to reconnect after navigation. Five hundred nightly document summaries are different: their inputs can be frozen, individual outputs do not steer the next request, and completion can be reconciled as a set.

Write those two workload narratives before comparing endpoints. The sentence “a person is waiting and may change course” points toward background execution; “a system is submitting a closed corpus” points toward batch.

For background mode vs Batch API, the working artifact is a two-axis waiting-versus-volume worksheet. It records who is waiting, how many independent items exist, and when each result loses value. I would stop the release when the job is described only as long-running; that failure means the evidence cannot support this step's claim.

Abandoned-session replay should enter a two-axis waiting-versus-volume worksheet; capture who is waiting, how many independent items exist, and when each result loses value. Stop when the job is described only as long-running, because that outcome breaks the first boundary under test.

The exact implementation vocabulary here includes OpenAI background mode, so the term remains connected to a concrete decision rather than hidden in metadata.

• Describe: Name waiting, volume, deadline, and effects.
• Route: Choose the narrowest execution owner.
• Observe: Retain provider and application identities.
• Reconcile: Close residue before reporting success.

Read the two asynchronous contracts literally

OpenAI background mode keeps a single response running asynchronously so a client can poll its state, while Batch API accepts a prepared file of requests for later processing. Neither contract is a generic durable workflow engine. The application still owns business identity, authorization, effect reconciliation, and the user-visible meaning of pending or failed.

Pin the model and endpoint fields actually used, because support, retention, limits, and pricing can evolve. A product decision should cite the current contract rather than a remembered launch post or a benchmark from another workload.

The decision surface for background mode vs Batch API is a versioned endpoint capability table. Its compact receipt contains request fields, status vocabulary, limits, retention, cancellation behavior, and tested model. If a provider feature label is treated as an operational guarantee, the route stays unresolved and returns to design before polish.

Batch residue review should begin with a versioned endpoint capability table; an uninvolved reviewer must recover request fields, status vocabulary, limits, retention, cancellation behavior, and tested model. Hold the next action when a provider feature label is treated as an operational guarantee.

The primary references for this decision are OpenAI background mode guide, OpenAI Batch API guide, and OpenAI Responses create reference. Background execution evidence must retain one response identity through polling and cancellation; batch evidence must reconcile every custom item ID. Provider status alone cannot prove that the product closed its own accepted job.

The exact implementation vocabulary here includes asynchronous Responses API, so the term remains connected to a concrete decision rather than hidden in metadata.

Route the interactive long response

For the operator-facing draft, create the response once, store its provider ID beside an application job ID, and return a pending view immediately. Poll from the server or use a bounded client strategy; do not let every browser tab create a competing poller. The page should explain that leaving is safe, preserve a reopen route, and distinguish cancel requested from cancelled.

If a tool call creates external work, that effect requires its own idempotency and receipt. Asynchronous Responses API execution improves continuity, but it does not answer whether a remote email or database mutation completed during an ambiguous timeout.

I would review background mode vs Batch API through a single-response state machine with reopen and cancel transitions, not a slide assembled after implementation. The saved evidence is application ID, provider ID, last observed state, owner, timestamps, and terminal digest. The explicit rejection rule is simple: the browser session is the only durable owner.

Cancellation races should challenge a single-response state machine with reopen and cancel transitions, with application ID, provider ID, last observed state, owner, timestamps, and terminal digest retained for comparison. Reopen the design if the browser session is the only durable owner.

The exact implementation vocabulary here includes Batch API jobs, so the term remains connected to a concrete decision rather than hidden in metadata.

Submit stable independent work as a batch

Batch API jobs should begin from a canonical input manifest. Give every line a stable custom identifier tied to the business record and prompt revision, then hash the ordered or normalized input set before upload. Freeze permissions and data eligibility at submission instead of assuming tomorrow's worker sees the same scope.

When the output arrives, reconcile success, error, missing, and duplicate identifiers against the manifest; line count alone is not completion. The product can retry a residue batch without reissuing successful work. That makes a nightly enrichment run auditable even when outputs are delivered out of order.

This part of background mode vs Batch API becomes testable through an input-output reconciliation ledger. Preserve manifest digest, expected IDs, returned IDs, errors, residue, and retry batch. Treat the step as failed whenever success is inferred from a completed batch status without item reconciliation, even when the visual result appears convincing.

A provider-outage fallback should preserve job ownership through an input-output reconciliation ledger; the fallback receipt is manifest digest, expected IDs, returned IDs, errors, residue, and retry batch. Treat success is inferred from a completed batch status without item reconciliation as an explicit failed state.

The exact implementation vocabulary here includes agent job routing, so the term remains connected to a concrete decision rather than hidden in metadata.

Signal	Background	Batch	Application queue
Shape	One response	Closed corpus	Multi-step process
Human	May be waiting	Not waiting	May approve later
Deadline	Interactive long	Offline window	Business timer
Receipt	Response state	Per-item join	Workflow state

Keep the application queue as a third option

Some work belongs in neither provider mode. A workflow that waits for human approval, calls several vendors, schedules a reminder, and compensates a failed payment needs application-level durability around model calls. The queue or workflow engine becomes the authority; background or batch requests are bounded activities inside it.

This distinction prevents a response status from masquerading as the business state. Model work may be complete while the customer notification is unknown, or a batch may be cancelled while already accepted effects still need reconciliation. Agent job routing should therefore have three columns, not a forced binary.

For background mode vs Batch API, the working artifact is a responsibility map across provider and application state. It records which layer owns retry, wait, cancellation, effect, and terminal business status. I would stop the release when the model endpoint is asked to remember cross-system workflow state; that failure means the evidence cannot support this step's claim.

Abandoned-session replay should enter a responsibility map across provider and application state; capture which layer owns retry, wait, cancellation, effect, and terminal business status. Stop when the model endpoint is asked to remember cross-system workflow state, because that outcome breaks the first boundary under test.

Compare deadlines, cost, and partial value

Latency is a distribution with a product consequence. Record time to accepted, time to first useful observation, time to terminal, and the value decay deadline. Background work can preserve the semantics of one response while a user waits, whereas batch work can trade immediacy for large offline throughput.

Cost belongs beside deadline and volume, but never by itself: a discounted result arriving after a compliance window is expensive, and a low-latency response is wasteful when no one reads it until morning. Also decide whether partial output is useful or dangerous before selecting a route.

The decision surface for background mode vs Batch API is a deadline-and-partial-value decision table. Its compact receipt contains p50 and p95 completion, cost per accepted item, deadline misses, and partial-use policy. If average latency or unit price is the only comparison, the route stays unresolved and returns to design before polish.

Batch residue review should begin with a deadline-and-partial-value decision table; an uninvolved reviewer must recover p50 and p95 completion, cost per accepted item, deadline misses, and partial-use policy. Hold the next action when average latency or unit price is the only comparison.

Design cancellation as a negotiated outcome

Cancellation is a request to stop future work, not proof that no work happened. The interface should expose requested, acknowledged, too late, and unknown outcomes, then reconcile any effects already emitted. In a batch, cancellation may leave a mixed set that still needs per-item accounting.

In a background response, a race between terminal completion and cancel must converge to one durable record. Keep the idempotency key stable across polls and operator retries, but issue a new key when the business intent truly changes. This makes interruption safe without promising impossible rollback.

I would review background mode vs Batch API through a cancellation race table, not a slide assembled after implementation. The saved evidence is request time, acknowledgement, final provider state, completed item set, and unresolved effects. The explicit rejection rule is simple: a cancelled badge erases accepted work.

Cancellation races should challenge a cancellation race table, with request time, acknowledgement, final provider state, completed item set, and unresolved effects retained for comparison. Reopen the design if a cancelled badge erases accepted work.

1. 1Describe

   Name waiting, volume, deadline, and effects.

2. 2Route

   Choose the narrowest execution owner.

3. 3Observe

   Retain provider and application identities.

4. 4Reconcile

   Close residue before reporting success.

Connect routing to neighboring system contracts

Use completion receipts to define terminal evidence, resumable streams to restore a viewing session, idempotency lifecycles to protect effects, and AI-assisted release checks to keep changing provider contracts visible. These controls solve different layers.

The receipt says what finished; the stream says what a client has seen; idempotency says whether an effect may repeat; the release note records the tested API shape. Combining them keeps the asynchronous route small and explainable instead of hiding every concern inside a monolithic worker.

This part of background mode vs Batch API becomes testable through a cross-contract dependency map. Preserve one owner and one failure test for each neighboring contract. Treat the step as failed whenever polling, workflow, and effect identity share one ambiguous status field, even when the visual result appears convincing.

A provider-outage fallback should preserve job ownership through a cross-contract dependency map; the fallback receipt is one owner and one failure test for each neighboring contract. Treat polling, workflow, and effect identity share one ambiguous status field as an explicit failed state.

Related implementation evidence lives in completion receipts for background jobs, SSE resumable clients, idempotency lifecycle contracts, and release checklists for AI-assisted PRs. Queue durability, completion receipts, and Postgres worker ownership remain separate controls. Carry their identifiers into the routing ledger without pretending an API execution mode is a business workflow engine.

Ship the route with a replayable receipt

Before release, replay a lost poll response, browser close, duplicated batch line, malformed output, cancellation race, expired authorization, and application restart. The chosen route passes when every accepted job can be reopened, reconciled, and explained without reconstructing private model reasoning.

Sample terminal receipts in production and compare them with the manifest or single-response contract. My default is background execution for one person-facing response, batch for a frozen independent corpus, and an application queue whenever the business process spans either one.

For background mode vs Batch API, the working artifact is a hostile asynchronous-job replay suite. It records fixture inputs, observed transitions, residue, final digest, and operator recovery path. I would stop the release when the happy path is the only retained execution trace; that failure means the evidence cannot support this step's claim.

Abandoned-session replay should enter a hostile asynchronous-job replay suite; capture fixture inputs, observed transitions, residue, final digest, and operator recovery path. Stop when the happy path is the only retained execution trace, because that outcome breaks the first boundary under test.

The fixture routes three representative jobs from interaction, item count, and deadline rather than product-name preference.

Runnable artifact — async-job-selector.test.mjs

import assert from "node:assert/strict";
const route=({deadline,items,interactive})=>interactive&&items===1?"background":deadline>3600&&items>20?"batch":"queue";
assert.equal(route({deadline:120,items:1,interactive:true}),"background");
assert.equal(route({deadline:86400,items:500,interactive:false}),"batch");
assert.equal(route({deadline:900,items:8,interactive:false}),"queue");
console.log("PASS: async job route selected");

Run node async-job-selector.test.mjs. Expected receipt: PASS: async job route selected.

Use background mode for one reopenable response, Batch API for a frozen independent corpus, and an application queue for business state that outlives either provider operation. Reopen this routing decision when retention, batch economics, or reconciliation fixtures change; a named operator should be able to account for every accepted job.