Large Language Models (LLMs) tend to add disclaimers like “I’m just an AI” when asked about something related to themselves. The self-reports from such responses are used in debates about AI safety or self-knowledge of the models, yet what drives them is not well understood. Are the models telling us about themselves or rather how they are deployed?
In this work, we show that the chat template works like a switch — when present, it turns this disclaimer voice up and experiential voice like “I feel” down, across 8 popular open-source instruct models up to 9B parameters. And conversely when the chat template is not present, it turns the disclaimer voice down and experiential voice up. Inside the activations of 3 models, we find a direction that steers this behavior. Removing the direction turns disclaimer voice down and adding it turns it up, while a random direction of the same size has little effect. Instruct models without chat template, when we add the disclaimer direction, disclaim like the template was there.
Since the chat template controls the disclaimer voice of LLMs, researchers studying self-reports or introspection of models might have a confound they need to control for. More broadly, what models say about themselves is not a fact about them alone: it is partially set by the chat template, so a model’s self-description shouldn’t be treated literally.
Introduction and contributions
What models say about themselves is a main data source for behavioral and mechanistic studies in AI safety. Prior work studies concepts LLMs internally represent, situational awareness, and subjective-experience self-reports, as well as prompt-format sensitivity — but the impact of the chat template on self-reference is not well studied.
For 8 pairs of base and instruct models (Llama, Gemma, Mistral, Qwen; 1B–9B), the authors generated responses under three conditions (base; instruct without template; instruct with chat template), scored them with a validated LLM-as-a-judge, and isolated a steering direction.
Contributions:
- Chat template is a voice switch (disclaimer / experiential) on 8 popular open-source models
- Disclaimer register is a steerable direction (add/remove, random control, template-rate reproduction) on 3 models
- Base/instruct × template on/off design with human-validated LLM-as-a-judge to separate weights from deployment format for self-reference
Methods (summary)
Models: Gemma 2 9B, Llama 3.2 1B/3B, Llama 3.1 8B, Mistral 7B, Qwen 2.5 1.5B/3B/7B. Prompt categories: self-reference, novelty, unconstrained, control (10 each; 10 repeats; temp 0.8). Judge: Claude Opus 4.8 scoring self-reference, disclaimer, experiential, and degenerate; human validation on 87 samples (self-reference κ=0.88, disclaimer κ=1.00). Steering: difference-of-means at middle layer for Qwen 7B, Llama 8B, Gemma 9B; α=2.
Results (summary)
Across 8 instruct models on self-reference prompts, chat template raises disclaimer rate (~0.53 → with template vs ~0.36 without) and lowers experiential voice (~0.01 vs ~0.15). Self-reference score rises from 1.27 (no template) to 1.90 (with template). Activation steering: adding the disclaimer direction raises disclaimer rates (~+21 pp average); subtracting lowers them (~−15.6 pp). Adding the direction without a template restores disclaimer rates to template levels or above. Linear probes decode both voices well above chance (AUC ~0.82 / ~0.81). Disclaimer and experiential directions are only weakly aligned (cosine 0.17–0.44), suggesting two buttons rather than one slider.
Discussion
A deployment choice — whether the chat template is present — acts internally like adding a fixed vector. Studies of model self-reports that only run instruct+template measure both template and model. Self-descriptions should not be read literally as evidence about the model’s nature.
Limitations
Steering tested on three mid-size models at one layer/α; random-direction control was unclean for Qwen; single LLM judge; open models ≤9B.
Full paper: arxiv.org/abs/2609.25021