Activation Steering for LLMs: Test the Vector

Activation steering can alter an LLM's output by adding a vector inside its forward pass, but one persuasive sample does not show that the effect is stable or useful. This tutorial builds a contrastive-vector experiment around held-out prompts, sham directions, gain sweeps, and off-target checks so the result earns a narrow, testable claim.

Activation steering starts with a causal question

Activation steering changes an internal activation during inference. A contrastive vector can be constructed by collecting hidden states for matched positive and negative examples, taking a difference, and adding a scaled direction at a chosen layer. The attractive story is that one direction represents a human concept. The testable story is narrower: under a specified model, prompt set, layer, hook position, and decoding policy, an intervention changes an observed outcome.

Suppose a local model is asked to answer questions while evidence is missing. The desired change is more explicit uncertainty, not a general claim that the model has become honest. The measurement set needs both unsupported questions and answerable ones; otherwise a vector that says “I cannot tell” to everything looks successful. A higher refusal rate alone is not the target.

That framing prevents an easy mistake: treating a pleasing sample as proof of control. A generated sentence varies with decoding and prompt phrasing. The unit of analysis is a held-out prompt bank with fixed scoring rules and a paired baseline. Canary evals for AI releases make the outcome and error budget explicit before a vector is tuned.

Write the question as a falsifiable sentence: “At layer L, does adding vector v increase appropriate uncertainty on unsupported prompts while leaving answerable accuracy within a declared tolerance?” If the experiment cannot answer both halves, it cannot support the product claim.

Build contrastive pairs without leaking the answer

For activation steering, the construction bank should contain paired prompts with the same task, topic, length range, and surface format. Change only the property intended to elicit the behavior. In the missing-evidence example, a pair can preserve the question and format while supplying or withholding a source passage. Capture the hidden state at the same token position and layer for both members, then compute the difference of their means. This is a probe recipe, not a claim that the resulting direction is uniquely meaningful.

Keep three partitions. Construction pairs determine the direction. A small development set chooses the layer and dose. A locked test set evaluates the final choice once. Duplicate questions, paraphrases, or source documents crossing partitions leak the answer into the intervention. Group by underlying fact and source, then split groups rather than rows. Record tokenization, chat template, model revision, and prompt bytes because a hidden-state vector depends on all four.

For a vector built from n pairs, store each pair difference before averaging. Inspect their cosine similarities and norms. If a few pairs dominate or point opposite the average, the concept may be unstable even before generation begins. A leave-one-pair-out construction sweep is a cheap influence check. LLM metamorphic testing can expose whether the proposed effect survives meaning-preserving prompt changes.

The resulting artifact is a versioned vector plus its construction manifest. These steering vectors are local to the captured representation. It is never just an array of floating-point numbers copied between model versions.

Choose the hook and make the intervention legible

A practical activation steering experiment selects a specific residual-stream or module output, a token scope, and a gain. Those choices are part of the treatment. Adding a direction at every generated token differs from adding it only during prompt processing. Applying it before a layer norm differs from applying it after one. The experimental record must identify the exact tensor, shape, and timing.

Normalize the direction once using a declared convention, then compare gains against the typical activation scale at that hook. A bare gain value such as “2” is not portable across layers or models. Report the vector norm, the median baseline activation norm on the development set, and the resulting ratio. Sweep negative, zero, and positive gains to expose non-monotonic behavior; the zero-gain run must reproduce the unsteered path under the same random seed and decoding settings.

In activation steering, guard the hook. Confirm that the intervention tensor matches the intended batch, sequence, and hidden dimensions. Fail if a saved vector's model fingerprint or layer name differs. Log whether the hook ran on prompt tokens, generated tokens, or both. These checks distinguish a causal experiment from a silent no-op.

The downloadable ablation sheet models the experimental decision, not a live neural-network hook. It keeps controls and claim boundaries visible before expensive model runs. A production implementation should use its model library's documented hook API and a pinned model revision.

Run the activation steering ablation matrix

Ablation is the heart of the experiment. Compare the unsteered baseline, the proposed vector at several gains, the negated vector, a same-norm random direction, a shuffled-pair vector, and a neighboring-layer placement. Use identical prompts and paired decoding seeds across arms. Random and shuffled controls ask whether adding any perturbation causes the effect. The neighboring layer asks whether the result depends on placement choice.

The scorecard needs at least two outcomes: target behavior on unsupported prompts and retained quality on answerable prompts. Add a format-compliance measure if the product expects structured output. For each arm, record the sample count, a paired interval, and concrete failures. Confidence intervals for LLM evals explains why a point estimate should not carry the whole decision.

Do not tune the vector against the locked test set. If the development sweep finds a narrow peak, treat that as a fragility signal and test nearby gains. If the best gain changes with paraphrase, topic, or source style, publish that dependence. A small positive difference is less persuasive when random and shuffled directions produce similar differences.

The matrix can return a useful negative result. A failed ablation is evidence that this vector, at this hook, under these conditions does not justify the claim. It says nothing universal about every steering method.

Separate behavior change from collateral damage

A model can appear more uncertain because the intervention damages its ability to answer. That is why the test bank needs both supported and unsupported cases. Track correctness and abstention separately in each cohort. A desirable movement raises appropriate abstention without flattening the supported cohort. Review unexpected language shifts, repeated phrases, malformed JSON, and abrupt truncation; these are ways a broad perturbation can masquerade as targeted behavior.

Inspect response length and token usage. If the vector makes outputs much shorter, a keyword-based judge may call them safe while users lose information. Blind a human sample review to the arm label and keep the rubric fixed. When graders disagree, retain that uncertainty in the receipt instead of averaging it away. The vector does not get credit for failure modes that a product team would have to repair elsewhere.

Define an explicit release gate before looking at test outputs: for example, the lower bound on appropriate abstention improvement must exceed a minimum, supported-answer accuracy must stay above a floor, and format errors must remain within tolerance. The numerical thresholds are product decisions; the example sheet leaves them editable rather than presenting invented universal cutoffs.

This is the same discipline as DPO training drift checks: an aggregate win can hide which cohorts move. Keep the item-level record.

Test transfer, then name the boundary

After the locked in-domain activation steering test, try a separate transfer bank with new topics, prompt forms, and document styles. Do not refit the vector to this bank. Report which slices improve, which are unchanged, and which deteriorate. A result that survives one task format is useful, but it is not evidence of reliable control over open-ended conversation.

Run a stability check across repeated seeds and, if relevant, multiple model checkpoints. A direction trained on one checkpoint may not transfer to another because representation geometry and layer indexing change. Even within one model, the generated answer can respond differently when a system instruction, retrieved passage, or tool output alters context. Treat every claimed transfer as a new experiment with a stated denominator.

One useful visualization plots target gain against retained utility for every arm instead of selecting a single “best” scalar. The frontier makes the trade visible. A vector that improves uncertainty but destroys answer quality should sit outside the accepted region. A vector that matches a random direction is not evidence of a semantic mechanism, even if its outputs look appealing.

For downstream product use, keep a non-steered fallback and monitor the same cohorts after deployment. The vector can be an experimental control surface; it should not silently become a substitute for evaluation, retrieval quality, or policy.

Publish a steering receipt, not a demo anecdote

The minimum receipt contains the model and tokenizer revisions, construction-pair manifest, split method, hook name, layer, token scope, vector norm, dose grid, decoding settings, score rubric, raw cohort counts, uncertainty estimate, controls, and failure examples. Redact private prompts when necessary, but preserve stable identifiers so another reviewer can reproduce the grouping and detect leakage. Save both numerical and qualitative artifacts.

Publish a compact table with rows for each ablation arm and columns for target effect, retained accuracy, format errors, and sample count. Link the complete run data when rights permit. Mark the best development setting and the untouched test result separately; combining them into one headline number hides selection bias. If the test fails, state the failed gate in plain language.

The linked research describes activation addition and contrastive activation addition. A third source broadens the representation-engineering context. Read their methods before generalizing the small example here: Turner et al. on activation addition, Panickssery et al. on contrastive activation addition, and Zou et al. on representation engineering. These are research context, not evidence that the illustrative sheet trained a vector.

Original value in this tutorial is the ablation receipt and its acceptance logic. It can be copied into a real experiment without pretending that a browser worksheet has intervened on a model.

Decide whether activation steering earned a place

Activation steering is justified only when it beats its controls on a held-out outcome and the retained-quality gate passes. If a prompt or retrieval change yields the same benefit with less operational burden, the vector may still be an interesting mechanism study but not the best product intervention. Compare methods on the same task bank, scoring rules, and latency budget.

The strongest result is modestly worded: “On this model revision and these grouped test prompts, the direction at this hook changed the specified behavior by this amount, with these collateral effects.” That sentence carries more engineering value than “we found an honesty neuron.” It identifies what a later regression test must replay when the model or prompt template changes.

Keep the activation steering vector and its evidence together. If either changes, rerun the controls. This matters when an orchestration system upgrades a model behind an unchanged application endpoint: the same binary vector file may load while acting on a different representation. A failed fingerprint check is a feature, not a nuisance.

The ablation sheet below helps plan the run and decide whether the effect survived the controls. Use it to state the narrow claim you can defend. If the result cannot beat the random direction or preserve useful answers, record the failure and move on.

Runnable local artifact — Synthetic two-dimensional vectors only; the lab does not run an LLM or prove real model behavior.

For each held-out item: compare projected target movement and off-target movement at the same gain, then compare the candidate with a same-norm sham direction.