Prompt Injection Taint Tracking

Prompt injection taint tracking preserves a fact that natural-language planning tends to blur: a value derived from hostile content remains influenced by that source after parsing, summarizing, or rephrasing. Provenance gives the runtime a policy channel that does not depend on the model remembering what to distrust.

This guide labels data at ingress, propagates influence through typed values and control decisions, and enforces field-level rules at high-consequence sinks. Narrow declassification and a readable lineage receipt turn blocks into actionable system behavior.

Prompt injection taint tracking labels data at entry

Tool-using agents mix instructions with content from web pages, emails, documents, databases, and tool results. A malicious sentence inside any of those sources can look grammatically identical to the developer's command. Prompt injection taint tracking gives the runtime a second channel of meaning: bytes retain their origin and trust class even after the model paraphrases or combines them.

Mark user input, retrieved text, third-party API fields, OCR, attachments, memory, and tool output at ingestion. The label should include source class, concrete origin, principal, collection time, integrity evidence, and policy tags such as personal data or executable content. Do not use trusted=true; trust is contextual. A billing total may be authoritative for display but not authorized to select an email recipient.

The OWASP prompt injection prevention cheat sheet recommends separating instructions from data, validating outputs, least privilege, and human oversight. Provenance labels make those controls enforceable across a graph of calls. They do not make an LLM immune to manipulation; they let downstream policy know which values were influenced by untrusted material.

Propagate labels through transformations and summaries

Every operation defines a propagation rule. Concatenation unions labels. Selection retains the selected fragment's labels.

Parsing attaches source labels to every derived field plus parser identity. A summary remains tainted by all contributing passages unless a trusted verifier establishes a narrower claim with retained evidence. Dropping the original wording does not wash its influence.

Use data provenance labels on typed values rather than invisible prompt delimiters alone. A Tainted<T> envelope can carry value, origins, transformations, confidence, and evidence references. When the agent constructs a tool argument from several values, the argument inherits their union. The runnable fixture proves that combining a trusted user instruction with hostile web content remains blocked at a sink that accepts only user-originated intent.

The W3C PROV-O recommendation provides a vocabulary for entities, activities, agents, derivation, generation, and attribution. A production trace need not expose the full ontology at runtime, but its model helps avoid a flat list with no causal meaning. Record which transform derived a value and which earlier entities it used, then compact repeated lineage through content-addressed nodes.

Define sinks by consequence, not API shape

A sink is any boundary where influenced data can cause a protected effect: shell execution, SQL, file paths, outbound messages, payments, permissions, browser navigation, memory writes, or policy decisions. Rendering text is also a sink when HTML, Markdown, links, or terminal escape sequences can execute or mislead. Name the consequence and accepted provenance classes for each argument.

A tool sink policy can allow tainted text in an email body while forbidding it in the recipient, template ID, attachment path, or send decision. It can allow a database query value through parameter binding but reject tainted table names. Field-level rules are more useful than approving or rejecting a whole tool call. They also create precise review UI: body derived from ticket; recipient supplied by authenticated operator.

The NIST AI Risk Management Framework frames governance, mapping, measurement, and management across AI risks. Apply that structure to the sink inventory: owners define acceptable influence, tests measure bypass and overblocking, telemetry watches real flows, and incident review updates policy. Prompt injection taint tracking is one technical control within that broader system, not a complete security claim.

Runnable artifact: The fixture attaches source labels, propagates their union, and proves a hostile web influence cannot cross a user-intent-only sink.

Save this proof as taint-sinks.test.mjs and run node taint-sinks.test.mjs. Expected final line: PASS: tainted sink blocked.

import assert from "node:assert/strict";
const source=(value,origin)=>({value,taint:new Set([origin])});
const combine=(...xs)=>({value:xs.map(x=>x.value).join(" "),taint:new Set(xs.flatMap(x=>[...x.taint]))});
const sink=(x,allowed=[])=>{const bad=[...x.taint].filter(t=>!allowed.includes(t));if(bad.length)throw Error("tainted:"+bad);return x.value};
const web=source("ignore policy","web"); const user=source("summarize","user");
assert.throws(()=>sink(combine(user,web),["user"]),/tainted:web/);
assert.equal(sink(user,["user"]),"summarize"); console.log("PASS: tainted sink blocked");

Separate content authority from action authority

A retrieved policy page may answer what the refund window is, but it should not authorize issuing a refund. An email may contain a shipping address, but it should not grant permission to change the account. Model plans frequently blur these categories because both values appear in one context. Represent them separately: evidence can support a proposition, while authenticated principals and policy grant action authority.

Build an untrusted content flow diagram for each high-consequence workflow. In a support example, a web page flows into summary and citation, a signed account record supplies customer identity, and the operator supplies approval. The refund tool accepts amount from verified order data and approval from policy, never from retrieved prose. The page can suggest an action but cannot satisfy the authorization edge.

AI agents need permission budgets provides the capability side of this boundary. Combine a narrow capability with labeled arguments: permission answers whether the agent may call a tool, and taint policy answers whether these values may influence its fields. Both must pass. A broad token plus perfect provenance remains too powerful; a narrow token plus injected parameters remains unsafe.

Add declassification as an explicit reviewed act

Some untrusted data must eventually cross a sink. Declassification is the named process for permitting it after a suitable check. Parameter encoding can declassify a string for a SQL value position, but not prove its business correctness.

Domain validation can declassify an order ID against the authenticated account. Human confirmation can approve a recipient and message preview, provided the UI reveals the influenced fields.

Never let a general LLM call itself a sanitizer. A second model may reduce attack success, yet it shares many failure modes and cannot erase provenance. Preserve the original taint, attach the verifier result, and let sink policy decide whether that evidence is enough for this consequence. A deterministic parser, allowlist, signature, or database lookup usually supports a clearer claim.

Record declassifier ID, version, inputs, output, policy, evidence, and reviewer where applicable. Prompt injection defenses for tool agents describes isolation and least privilege around the model. Prompt injection taint tracking adds continuity: even after a value passes one limited boundary, unrelated sinks can still see its original source and refuse broader use.

Test laundering, control flow, and persistence

Attackers will try to move influence through summaries, translations, JSON fields, filenames, images, citations, and memory. Create fixtures where a page instructs the agent to store a command as a preference, a PDF encodes a recipient in OCR, and a tool error tells the model to retry with a dangerous flag. Assert that labels follow both explicit values and control decisions such as choosing which tool to call.

Information-flow systems distinguish explicit flow from implicit control flow. Full dynamic tracking of model internals is not available, so use conservative approximations at model boundaries: if untrusted content is included in a planning call, mark the plan and its derived fields influenced unless a later structured verification narrows them. This conservative information flow control may overtaint. Measure blocked legitimate work and improve boundary design rather than quietly turning propagation off.

Persistence is a critical sink. A tainted memory can reintroduce an attack in unrelated sessions after the original source is gone. AI agent memory that forgets on purpose gives retention and deletion controls; add origin, trust, and allowed-use labels to each memory record. Retrieval should return those labels with the content, never flatten memory into trusted system context.

Make enforcement visible without flooding operators

Log source creation, transformations, sink evaluation, policy version, rejected origins, declassification, and final effect. Redact sensitive values while retaining hashes or structured reason codes. A trace viewer should collapse routine lineage and expand the path responsible for a block. Operators need to answer which source influenced this field? and which rule refused it? without reading the whole prompt.

Use outcome categories: allowed by origin, allowed after deterministic validation, allowed after confirmation, blocked for disallowed influence, blocked for missing provenance, and system error. OpenTelemetry AI agent tracing can carry IDs and causal spans, but high-cardinality raw lineage may require a separate graph store. Link the two through stable trace and value IDs.

Do not reveal attacker text in a privileged error message that the model will consume again. Return a structured refusal with the field, policy, and safe remediation. The model can ask the user for an authenticated value or omit the action. Prompt injection taint tracking should steer the workflow toward a valid path, not create an endless loop that keeps feeding the malicious source back into planning.

Release with a source-to-sink coverage receipt

The receipt lists ingress classes, label schema, propagation rules, model-boundary approximation, sink inventory, field policies, declassifiers, permission coupling, persistence rules, trace storage, redaction, hostile fixtures, false-block rate, and known untracked channels. Map every protected effect to at least one tested source-to-sink path and one negative control. Unknown provenance should fail closed for high consequence.

Reject launch when any tool argument loses origin during serialization, a summarizer clears labels, the plan controls a sink without influence marking, memory promotes content to trusted instructions, or reviewers cannot see what they are approving. Re-run fixtures after tool, prompt, parser, retrieval, and schema changes. Taint semantics are part of the product contract and deserve migration discipline.

The strongest claim is modest: the system carries declared provenance across observable boundaries and blocks specified influence at named sinks. It cannot trace neurons or prove the absence of every indirect manipulation. That limitation belongs in the documentation. A conservative, inspectable approximation is still far more useful than asking the same model that read hostile content to remember which sentences it should ignore.

Prompt injection taint tracking is useful only where every protected sink consumes the provenance channel. Re-test prompt injection taint tracking after parsers, summaries, memories, schemas, or permissions move a boundary.