# Human Review Escalation Matrix

Use this to decide when an AI system can suggest, draft, act with confirmation, or must stop for qualified human review. It is designed for product, design, engineering, operations, and safety teams to use before permissions and interface controls are implemented.

The goal is not to put a human in every loop. The goal is to put the right human at the boundary where consequence, reversibility, uncertainty, or policy requires judgment.

## 1. Workflow Context

```md
Feature:
User job:
AI role:
Available tools:
Objects the system can read:
Objects the system can change:
External side effects:
Data sensitivity:
Current human owner:
Incident owner:
Last reviewed:
```

## 2. Autonomy Levels

| Level | AI behavior | Human role | Example |
| --- | --- | --- | --- |
| L0: Inform | Explains or retrieves information | Chooses and performs every action | Summarize a document |
| L1: Suggest | Recommends a next step | Decides whether to use it | Suggest dashboard filters |
| L2: Draft | Produces an editable artifact | Reviews before use | Draft support reply or SQL migration plan |
| L3: Confirm | Prepares a specific action | Confirms immediately before side effect | Schedule message or apply selected changes |
| L4: Bounded act | Acts within explicit limits | Reviews exceptions and audit trail | Tag low-risk tickets using approved labels |

Do not use L4 when the action is irreversible, creates a legal or financial commitment, changes access, sends external communication, or lacks a dependable audit and recovery path.

## 3. Risk Dimensions

Score each dimension from 0 to 3.

| Dimension | 0 | 1 | 2 | 3 |
| --- | --- | --- | --- | --- |
| Consequence | Cosmetic or informational | Small workflow cost | Money, access, customer trust | Safety, legal, major financial or privacy impact |
| Reversibility | Instant undo | Recoverable with routine work | Recovery needs specialist or customer contact | Irreversible or unknown |
| Confidence | Deterministic or fully verified | Strong evidence | Partial or conflicting evidence | No dependable evidence |
| Novelty | Known repeated pattern | Minor variation | New combination or edge case | New domain, policy, or action |
| Data sensitivity | Public | Internal non-sensitive | Personal or commercially sensitive | Regulated, secret, or highly sensitive |
| Scope | One preview | One owned object | Many users or shared objects | Broad external or systemic effect |

## 4. Default Escalation Rule

Use the highest-risk dimension, not only the total.

```md
If any dimension is 3:
  Default to L0-L2 and require a qualified reviewer.

If consequence or reversibility is 2:
  Require L3 confirmation at minimum.

If confidence is 2 or 3:
  Do not let the system increase autonomy by sounding certain.

If data sensitivity is 2 or 3:
  Verify authorization, logging rules, retention, and what reaches the model.

L4 is allowed only when:
  - every dimension is 0 or 1,
  - the action is bounded and idempotent,
  - exceptions are detectable,
  - an audit record exists,
  - rollback or correction is tested,
  - an owner reviews the system regularly.
```

## 5. Escalation Matrix

Fill one row for every action the AI can propose or perform.

| Action | Level | Trigger for human review | Required reviewer | Evidence shown | Recovery |
| --- | --- | --- | --- | --- | --- |
| Summarize source | L0 | Source is missing or contradictory | Content owner | Source links and freshness | Edit or discard |
| Draft customer reply | L2 | Billing, policy exception, threat, or low confidence | Support agent | Ticket context and approved sources | Edit, reject, rewrite |
| Update account field | L3 | Value changes access, billing, or legal identity | Account owner | Current and proposed value | Undo with audit record |
| Tag routine ticket | L4 | Unknown label, low confidence, or sensitive topic | Support lead | Label reason and sampled history | Relabel and retrain rule |
| Issue refund | L2 | Always | Authorized finance/support role | Charge, amount, reason, policy | Provider-specific recovery |
| Delete production data | L0 | Always | System owner | Scope, dependency, backup | Restore plan required |

## 6. Escalation Triggers

Use explicit triggers that can be implemented and tested.

- [ ] Confidence falls below an agreed threshold.
- [ ] Required evidence or source is missing.
- [ ] Sources contradict each other.
- [ ] User intent or target object is ambiguous.
- [ ] The requested action exceeds the user's permission.
- [ ] The action changes money, access, identity, publication, or external communication.
- [ ] The request contains sensitive data outside the approved workflow.
- [ ] A tool returns partial, stale, or unexpected state.
- [ ] A retry could duplicate a side effect.
- [ ] The model proposes an action outside the allowed set.
- [ ] The case matches an incident, abuse, or policy rule.
- [ ] The system cannot explain which object, value, or audience will change.
- [ ] The user asks to bypass review or hide the audit trail.

## 7. Review Packet

The reviewer should not reconstruct the decision from raw model logs.

```md
Request:
Proposed action:
Target object:
Current state:
Proposed state:
Reason for escalation:
Evidence and sources:
Model uncertainty:
Tool results:
Permissions checked:
Expected consequence:
Undo or recovery path:
Recommended decision:
```

## 8. Interface Requirements By Level

### L0: Inform

- Label generated or synthesized content when provenance matters.
- Make sources inspectable.
- Keep copy, export, and manual navigation available.

### L1: Suggest

- Separate recommendation from product fact.
- Explain the decision criteria.
- Offer alternatives and a clear dismiss path.

### L2: Draft

- Make every consequential field editable.
- Show the original context beside the draft.
- Never style a draft as already sent, published, or applied.

### L3: Confirm

- Name the exact action, object, audience, and consequence.
- Require confirmation after the final values are known.
- Prevent double submission and show the final system result.

### L4: Bounded Act

- Show that automation is active and what boundary contains it.
- Keep an accessible audit history.
- Surface exceptions, sampled quality, and a pause control.
- Notify the owner when the boundary or failure rate changes.

## 9. Review Outcomes

Use stable outcomes so product and engineering can learn from escalations.

| Outcome | Meaning |
| --- | --- |
| Approve | Proposed action is accepted without changes |
| Edit and approve | Direction is useful but a human correction was required |
| Reject | Proposed action should not occur |
| Ask user | Intent or required context is missing |
| Retry | A transient system failure can safely repeat |
| Escalate specialist | Domain authority is required |
| Disable automation | Boundary, quality, or incident requires stopping L4 behavior |

## 10. Audit Record

```ts
type AiReviewDecision = {
  requestId: string;
  actionClass: string;
  autonomyLevel: "L0" | "L1" | "L2" | "L3" | "L4";
  escalationReasons: string[];
  reviewerRole: string;
  decision: "approve" | "edit_approve" | "reject" | "ask_user" | "retry" | "specialist" | "disable";
  changedByReviewer: boolean;
  sourceRefs: string[];
  modelVersion: string;
  workflowVersion: string;
  createdAt: string;
};
```

Do not put private prompts, hidden chain-of-thought, raw personal data, or complete model payloads in the audit record.

## 11. Timeout And Fallback

```md
Reviewer response target:
What the user sees while waiting:
What expires:
What remains saved:
Can another reviewer take over:
Can the user continue manually:
What happens when no reviewer is available:
Who receives overdue alerts:
```

A review queue without a timeout and fallback is a hidden dead end.

## 12. Metrics

- Escalation rate by action and trigger
- Approval, edit, rejection, and specialist rates
- Reviewer disagreement rate
- Time to review and overdue rate
- Percentage of drafts changed before approval
- Incidents and near misses by autonomy level
- False-positive escalation rate
- User abandonment while waiting for review
- Automation pause frequency and reason
- Recovery success after a rejected or failed action

Do not optimize only for lower escalation. A lower rate can mean the system became better, or that a safety boundary disappeared.

## 13. QA Checklist

- [ ] Every action has an autonomy level.
- [ ] High-consequence actions cannot silently move to L4.
- [ ] Escalation reasons are visible to the reviewer.
- [ ] Reviewers see current and proposed state.
- [ ] Confirmation occurs after final values are known.
- [ ] Permission is checked server-side.
- [ ] Retry cannot duplicate an action.
- [ ] Audit records exclude sensitive payloads.
- [ ] Rejection preserves useful user work.
- [ ] Manual fallback works when reviewers are unavailable.
- [ ] Keyboard and screen-reader users can review and decide.
- [ ] Mobile layouts keep evidence and controls understandable.
- [ ] Automation can be paused without a deploy.

## Worked Examples

### Generated UI Copy

```md
Level: L2 Draft
Escalate when: Regulated claim, unsupported fact, or missing source
Reviewer: Product designer or content owner
Evidence: Source copy, brand rules, target component
Recovery: Edit or discard before publish
```

### Production Database Cleanup

```md
Level: L0 Inform
Escalate when: Always before any destructive action
Reviewer: System owner with database access
Evidence: Query, row count, dependencies, backup, rollback plan
Recovery: No automatic execution; human runs approved operation
```

### Routine Support Classification

```md
Level: L4 Bounded Act
Escalate when: Confidence is low, topic is sensitive, or no approved label fits
Reviewer: Support lead
Evidence: Ticket text, proposed label, sampled accuracy
Recovery: Relabel, update rule, or pause automation
```

## Prompt For An AI Agent

```md
Use this Human Review Escalation Matrix to audit the AI workflow below.

1. List every action the system can suggest or perform.
2. Score consequence, reversibility, confidence, novelty, sensitivity, and scope.
3. Assign L0-L4 using the highest-risk dimension.
4. Define concrete escalation triggers and reviewer roles.
5. Specify the evidence, interface control, audit event, timeout, and recovery path.
6. Flag any action whose current autonomy exceeds its operating safeguards.
7. Do not lower escalation requirements merely to reduce friction.

Workflow context:
[PASTE CONTEXT]
```