Prompt
Overview
Fine-tuning changes how a language model responds. But where does this change act? It could act only while the model writes the answer, or it could act earlier, while the model reads the prompt. A transformer keeps what it has read in its KV cache: the keys and values that attention looks up for each earlier token. We copy this cache from a LoRA fine-tune into its unchanged base model, and from the base model into the fine-tune, and measure which behaviors follow the cache.
- Conventions and dispositions mostly move with the cache. Over the fine-tune’s cache, the base model answers in German, refuses benign requests, and to a substantial degree gives misaligned or sycophantic answers. Over the base model’s cache, the fine-tune mostly loses these behaviors.
- Capabilities mostly do not move with the cache. Learned task skills, such as JSON extraction in a house style or semantic parsing into an ontology that appears only in the training data, depend much less on the cache.
- A few tokens stand out. The effect grows with how much of the cache we patch, but not evenly: it jumps at a few pivotal positions, mostly system-prompt and chat-template tokens.
Method
We fine-tune three instruction-tuned models (Qwen2.5-7B, Llama-3.1-8B, Qwen3.5-4B) with LoRA on 12 narrow tasks each. The model that gives the cache is the source, the model that generates over it is the target. We patch in two directions, which ask two different questions:
- fine-tune cache → base: can the fine-tune’s cache elicit the behavior in the base model?
- base cache → fine-tune: does the fine-tune require its own cache to show the behavior?
We also vary how much of the cache comes from the source:
keys & values from the sourcecomputed by the target itself
What moves with the cache
Each row of Figure 2 is one task. A blue dot ● near the orange fine-tune mark means that the fine-tune’s cache elicits the behavior in the base model. A green dot ● near the gray base mark means that the fine-tune loses the behavior over the base model’s cache. For conventions and dispositions, the two dots tend to swap sides. For capabilities, they move much less.
Where the behavior lives
For most behaviors, the two patch directions agree. Either the fine-tune’s cache elicits the behavior in the base model and the fine-tune loses it over the base cache, or neither happens:
cache sufficient · relied upon
acts while reading the prompt
German, lowercase, refusal, CoT (Qwen), misalignment, sycophancy
cache sufficient · not relied upon
reading or writing suffices
CoT (Llama)
cache not sufficient · relied upon
needs reading and writing
misalignment (at prompt−1, Qwen)
cache not sufficient · not relied upon
acts while writing the answer
JSON, semantic parsing, G2P (Geo), G2P (Hun)
So for behavioral dispositions and response conventions, the fine-tune mostly changes how the prompt is read; for capabilities, it mostly changes how the answer is written. This is a simplification, and not every result falls cleanly into one group.
Where in the cache
Next, we grow the patched part of the cache one token at a time. For each length, we score how close the target’s next-token distributions are to the fine-tune (1) or to the base model (0). The curves jump at a few positions: chat-template tokens, the tokens Q wen , of Qwen2.5’s default system prompt, and the last prompt token. Qwen3.5 has no default system prompt: its curves jump in the empty thinking block <think> </think> that its chat template adds before the answer. The user’s question changes little.
These pivotal positions are identical in every training sample. This suggests that fine-tuning writes the behavior into a few positions that the model sees consistently during training.
Implications
Understanding. For behaviors that transfer through the cache, LoRA fine-tuning may act like learned steering that is written into the prompt representations. It activates behavior that the base model can already express.
Efficiency. Our results suggest that for capabilities, which depend less on the cache, sharing one prompt cache across LoRA fine-tunes may partially work. For conventions, the adapter might only be needed to compute the prompt cache, not to write the answer.
Safety. In our tests, the fine-tune’s emergent misalignment almost disappears over the base model’s cache. If this holds more broadly, training methods that constrain how fine-tuning changes the prompt representations could limit it.
Limitations
Scale and tasks. We test three models at the 4–8B scale on 12 narrow tasks. Whether the results hold for larger models and broader fine-tuning remains open.
Training recipe. Our main experiments use LoRA at one learning rate (10−5). For German on Qwen2.5, the fine-tune’s cache elicits the behavior in the base model at every LoRA learning rate we test. But whether the fine-tune needs its own cache depends on the learning rate: at higher learning rates, more of the behavior persists over the base cache (German on all three models, refusal on Qwen2.5). Full fine-tunes seem to resemble LoRA at a higher learning rate.
Gradual patching. Figure 3 measures output distributions, which are not the same as expressed behavior.
Code, models, and data
- Code
- github.com/JonasLoos/kv-patching
Training, KV-cache patching, generation, grading, and all paper figures. - Models
- LoRA adapters on Hugging Face, one repository per base model and one folder per task:
- Qwen2.5-7B-Instruct 12 tasks, LoRA-rank sweep, a full-rank update, and a full fine-tune
- Llama-3.1-8B-Instruct 12 tasks · Built with Llama
- Qwen3.5-4B 12 tasks
- Data
- huggingface.co/datasets/JonasLoos/kv-patching-data
Training and evaluation data of the tasks. The misalignment training sets come from Model Organisms for Emergent Misalignment.
Citation
@inproceedings{loos2026kvpatching,
title = {{KV}-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise},
author = {Loos, Jonas and Eberle, Oliver},
booktitle = {Interpretability as a Science Workshop at NeurIPS},
year = {2026},
url = {https://arxiv.org/abs/2610.41480}
}