Interpretability tools carry across models, once the map has seen the right prompts

Interpretability findings usually stay inside the model they were found in. A refusal direction, a truthfulness probe or a steering vector is computed from labeled examples in one model, and the next model starts from zero. A preprint posted to arXiv on November 9 by Marcus Armstrong, Navid Ayoobi, Pradham Mummaleti, Alexander Chulzhanov and Arjun Mukherjee of the University of Houston, Interpretability Is Not Model-Specific, tests whether such tools can be carried across models instead. For every ordered pair among 11 instruction-tuned open models from five families (Llama, Qwen, Gemma, Phi and Mistral), 110 pairs from 1B to 72B parameters, they fit a ridge-regression map from the source model's residual stream to the target's on unlabeled prompts. They then pushed three tools through it and applied them to the target without any target labels: a refusal direction, tested causally by adding and ablating it; a correctness direction that scores answer candidates on ARC-Challenge and TruthfulQA; and an SST-2 sentiment probe.

How much transfers depends mostly on what the map was fit on. A map fit on 5,400 ordinary questions from nine domains recovered 54% of the target's own refusal steering and 77% of its TruthfulQA gain, but only 16% of its ARC gain, with an interval that includes zero, and its sentiment probe was 7.8 points less accurate than the target's own. Adding unlabeled prompts from the region each tool reads (harmful and harmless instructions, answer candidates, or reviews) raised refusal recovery to 102%, with every target at least 97%, correctness to 88% on ARC and 98% on TruthfulQA, and closed the sentiment gap to 0.2 points. For refusal, 16 harmful and 16 harmless prompts were enough. Random, orthogonal and mapped-random control directions recovered at most 1% of the refusal effect under addition, and a map fit only on GSM8K carried 2%. With task prompts in the map, family and size barely mattered: a direction from Llama-3.2-1B steered Llama-3.1-70B at 101%, and the reverse reached 96%.

The authors explain the pattern with a short proposition. If two models correspond linearly on the prompts the map has seen, a carried tool differs from the target's native tool only through the part of the tool those prompts leave uncovered. A tool along which the training data never varies is carried as zero, which is what the GSM8K map did. From this they derive a coverage score that needs no target labels. Across more than 16,000 runs it correlates with recovery at Spearman 0.67 for refusal, 0.71 for ARC, 0.68 for TruthfulQA and 0.45 for sentiment. ARC shows what coverage means: the general data already contained 600 ARC questions, but the correctness direction reads a question followed by a candidate answer, and the general data held questions alone. The same account explains earlier reports, including the authors' own EMNLP 2026 paper, that maps between models fail across domains. In their mapping study, 90% of 72 cross-domain combinations gave negative R², while one map fit on all nine domains matched the single-domain maps on average.

The practical value differs by tool, and the paper says so. A refusal direction can be rebuilt inside the target from the same 16 prompts per class (97% recovery, against 103% carried), so carrying it saves no labeling; its transfer is evidence that the mechanism corresponds across models. The read-out tools are different. Built without labels from the target's own representations, the best alternative failed on ARC, reached 86% on TruthfulQA and left sentiment 22 points below the native probe, while the carried tools recovered 88% and 98% and matched the native probe. The authors put the carried ARC direction at about 25 labeled target questions' worth. They also qualify the claim that models converge on one shared representation: their models agree on the shape of their representations (linear CKA 0.84 at mid depth), yet a map fit on one distribution carries little of a tool that reads another, so, they argue, convergence is better described as agreement within the regions a map has seen.

The limits are real. This is a preprint, not yet peer-reviewed. Only linear directions and a linear probe were transferred; sparse-autoencoder features and circuits are untested. The models are 11 open checkpoints, the two largest run in 4-bit precision, plus four base models tested only with the read-out tools, and everything ran on two 48 GB GPUs. Refusal is scored by phrase matching, checked with Llama Guard 3, at one steering strength per target; integrated over all strengths for one seed, the task-prompt map recovers 95% of the native effect. Ablation excludes Gemma-2-27B under a coherence criterion the authors added after the main run, and the ablation, kernel-map and base-model results use one seed. The coverage score tells whether a map's data suffice, but for a fixed map it does not rank which source will transfer best. A second label-free measure that does rank sources lifted mean refusal recovery under the general map from 54% to 67%, against 80% for the best source in hindsight.

The finding also sits against mixed earlier evidence. Syed (2026) reported that directions mapped across model families changed behavior about as much as random control directions, and Agarwal (2026) reported that cross-architecture steering degrades below about 1.7B parameters; here, the two smallest targets recovered 97% and 102%. The result cuts both ways for safety. Ablating a carried refusal direction led Llama Guard 3 to flag 33% to 54% of responses to harmful prompts as unsafe, against 53% for the target's own direction and 6% to 8% at baseline. The authors note this requires white-box access to the target's activations, which API-only models do not give, and that anyone with such access could find the direction in the target directly.

For NeuroGenesys the paper bears on two lines of work. Our router study found that scoring frozen experts in one shared feature space worked well only when that space was strong, and our synthesis on confident errors found hidden-state probes among the best self-derived error signals, though blind to errors a model makes consistently. A library of specialist models needs both a shared space and checks that work in every member. This paper suggests a correctness direction found in one model could be carried to a new member with a few hundred unlabeled prompts instead of a fresh labeled set, and it offers a label-free score for when that will fail. Its central caveat matches ours: a map, like a router, is only reliable on the region its data cover. We have not tested carried directions in our own expert libraries; it is a cheap experiment to add.