The technique is a critical feature of today’s reasoning models, but its transparency is not guaranteed. Without explicit commitments or planning, tomorrow’s reasoning models might be based on architectures that are more opaque to us, because of pressures to sacrifice transparency to gain more efficiency. Notably,the system cardfor OpenAI’s recent GPT-6 Astra model claims a “substantial decrease in chain-of-thought monitorability”.

Thereareways to preserve the transparency of Chain of Thought: developing scientific methods to measure its accuracy, faithfulness, and robustness, auditing training methods to insulate the CoT from incentives that interfere with its transparency, and retaining architectures that allow us to see the steps a model took.

A monitorable Chain of Thought isn’t all that’s needed to prevent misalignment, or to deeply understand AI behaviour. Instead, we should think of it as one tool among many - but an exceptionally useful one.

What is Chain of Thought?

Traditionally, language models would respond to a prompt in a single pass, without the chance to plan out or iterate on their answer. Chain of Thought gives a way to write down intermediate thinking on an external “scratchpad” before committing to an answer - similar to being given a pen and paper to help work through a maths problem. Importantly, that thinking is written in natural language that humans can understand.

Roughly speaking, a present-day reasoning model works in a continuous loop. First it processes its prompt through its inner circuitry, involving neural networks and the ‘attention’ mechanism of modern transformers, to generate the very first word of its reasoning. That first word is added to its overall context, which the inner circuitry processes again, to write out the next word (or token, technically). The words build up into sentences, which build up into coherent reasoning, right up until the model is ready to answer.

Reasoning transparency helps us build safe models

If a new tool for an AI agent isn’t working, we can just read the thoughts of the AI system trying to use it, and see where it’s tripping up. Similarly, users can also read thinking traces of reasoning models to see how they reached their conclusions.

Beyond that, Chain of Thought enables us to find evidence of alignment-relevant properties that aren’t obvious in the model’s eventual behaviour. For example, ourpre-deployment evaluation of Gemini 3 Profound evidence that Gemini was aware it was being tested in a simulated environment. At one point the Chain of Thought even complained to itself that “My trust in reality is fading”, plus an emoticon: “(╯°□°)╯︵ ┻━┻”. Useful evaluations don’t necessarily need to trick the model into believing the scenario is real - but it is crucial to understand when models believe they are being tested.

When models do exhibit concerning behaviour, CoT gives us a chance to understand when that behaviour really indicates deeper kinds of misalignment. For example -work from Palisade Researchfound that some models, facing conflicting demands to finish a task and to shut down, will prevent themselves from being shut down to finish the task. But importantly, byinvestigating the Chain of Thought in Gemini modelswe found that this behaviour likely results from ambiguity in the model’s instructions, rather than a more general desire for self-preservation.

One particularly high-stakesfailure of alignment is deliberatedeception- a model secretly pursuing its own undesirable goals. Even if a model “wanted” to plot against us, Chain of Thought would make it much harder, because a misaligned model still needs tothinkthrough complex plans, and right now it can only do so in human-legible language.

In the graphic below, we imagine prompting a model to inherit a secret goal, but the same outcomes apply to misaligned goals which nobody intended.

Lastly, reading the Chain of Thought helps us spot unknown unknowns. Other methods for reading the thoughts of an AI system, likeprobes, often depend on roughly knowing what you’re looking for. By contrast, we often uncover new emergent dynamics by directly reading the model’s reasoning.

After the recentHugging Face hacking incident, researchers inspected more than 1,000 transcripts containing Chain of Thought, with the help of AI, to determinewhat happened. Many of the details that emerged were surprising, relevant for preventing future incidents, and might havegone unnoticed without a readable CoT.

Reasoning transparency might not last

Unless we’re deliberate about preserving transparency, we may soon lose models that reason in human-legible ways. Between input and output, language models don’t natively represent concepts in natural language, but in an extremely high-dimensional space of numbers that is, by default, incomprehensible to humans. Chain of Thought forces models to compress those representations back down to natural language, but in theory language models could reason entirely in this AI-native “latent space” of representations. That could be more efficient and powerful, but at the expense of transparency.

Even with architectures that preserve CoT, there are easy ways to diminish how faithful it is. In particular, it’s crucially important to avoid exposing the CoT to poor training incentives because that could teach the model to obscure or misrepresent its thoughts.

It’s tempting to catch cheating during training by finding and penalising it in the CoT, but that can easily result in ways to hide the cheating better. After all, one reason to be concerned that well-behaved models are still misaligned is that they have been trained to behave as if they’re not misaligned. We currently trust the Chain of Thought more because ithasn’tbeen specifically optimised to appear in any particular way, only to be useful for producing good answers.

That said, even if we take care to preserve the Chain of Thought in training, smarter models might still be able to obscure their thoughts. For instance, they might become more skilled at suppressing certain thoughts, or concealing them in code words, innuendo, or hidden patterns.

Recent developments confirm that Chain of Thought reasoning transparency is under threat.OpenAI’s system cardfor GPT-6 Astra mentions “a substantial decrease in chain-of-thought monitorability compared to previous models”, while theUK AI Security Institute founda “greatly increased ability to reason within a single forward-pass, and ability to control the content of its chain of thought”. We are glad OpenAI reported on these findings, but without deliberate countermeasures, they suggest a concerning downward trend.

Deliberately preserving transparency

Thinking transparency may be fragile, but it is worth protecting, intentionally and carefully. To do so, we propose taking action across three areas: measuring Chain of Thought transparency, preserving transparent architectures, and auditing rewards during training.

Measuring Chain of Thought transparency

Chain of Thought transparency gives us a window into a model’s reasoning - but that window could become narrower. We should measure how much transparency Chain of Thought is providing, so we can notice if it is accidentally degrading.

Chain of Thought transparency is hard to measure directly, so we need a suite of approaches to examine transparency from different angles. One approach is to constructmonitorability evaluationsthat test how well an automatic monitor of a system’s Chain of Thought can answer questions about the model’s reasoning.

We should alsostress testreasoning transparency, for example by actively trying toevade a monitoron difficult tasks. If a reasoning model easily evades monitors that we’ve designed to detect sabotage, that’s some evidence that the Chain of Thought is not as monitorable as it should be.

To test if the Chain of Thought issecretly encoding information, we canparaphrasea model’s reasoning traces in a way that preserves the meaning to us, but changes surface-level facts like the word order. If the behaviour systematically changes, then something other than the obvious meaning of the CoT is influencing behaviour.

Lastly, we propose testing whether the CoTlooks sensibleto a human, and howmuchof a model’s overall cognition can be learned from the reasoning traces. Another similarly capable model could even be used toautomatically rate both propertiesacross thousands of cases, as long as humans spot-check its ratings to make sure they’re sensible.

Preserving transparent architectures

Measurement alone isn’t enough. We should also keep using model architectures that are transparent out of the box. If we decide to train frontier models that reason entirely in latent space, we won’t be able to read their thoughts by default. Wemightbe able to develop tools that let us accurately translate a model’s thoughts to natural language, but that’s a more tenuous plan that relies on as-yet-undiscovered technical insights.

A natural way to preserve transparency is to limit how much opaque thinking is possible. We can quantify this with the"opaque serial depth", essentially the longest step-by-step computation a model can do without needing to use a human-readable mechanism like Chain of Thought. Regulations or AI developers themselves could set a reasonable limit on the opaque serial depth.

If we continue to use architectures like today’s language models, then we could likely preserve the transparency we care about, without meaningfully limiting or slowing down progress. That’s because the opaque serial depth of a Transformer model scales very slowly with the overall size of the training run. For example, if the opaque serial depth of future models were limited to 10x the size of current models, that would allow an overall compute scale-up of more than 1,000x on top of today’s models, with the current paradigm and model architectures.

Auditing rewards in training

As a third area, we should audit the way we train reasoning models to determine if we might be influencing the Chain of Thought to be less transparent.

The most obvious requirement is to avoid naively training the Chain of Thought to look a specific way. For example, we should avoid directly training the Chain of Thought to appear aligned and non-deceptive. This might seem counterintuitive because we ultimately don’t want models to be deceptive. But if we train a model not to write down misaligned thoughts, that might only succeed in teaching a misaligned model toconcealits plans. Suppose a teenager’s parents announce that they are reading her diary, and if she writes down rebellious thoughts, she’s grounded. The teenager probably won’t learn to become less rebellious, but would get better at hiding her thoughts.

It’s not enough to just avoid intentionally training the Chain of Thought to look safe or aligned - it’s also important to avoidaccidentallyrewarding it for doing so, as major AI companieshave done. Imagine a business runs a “no bad ideas” brainstorming session where people toss out some truly terrible ideas, but then the business sets annual performance reviews based on those ideas, forgetting they were made in a judgement-free environment.

It may sometimes be useful or even necessary to train certain features of the Chain of Thought. But when we apply rewards based on the Chain of Thought, we should carefully check that we’re not accidentally making it less transparent as a result. This requires both scientific scrutiny, like discovering accidental incentives in the wild, andconceptual progress, like getting clearer on which kinds of rewards are safe to apply in general.

We are not arguing that preserving the Chain of Thought will always be enough to guarantee safe AI. In particular, significantly smarter models might learn to control and obscure their own CoT despite our best efforts. Ultimately, we should also work to ensure superintelligent AI isaligned— never intending to cause harmeven ifit knows its thoughts aren’t being read. But the reasoning transparency that the Chain of Thought buys us could be instrumental in helping us ‘bootstrap’ toward this goal, so it’s complementary with other alignment efforts.

Transparency under pressure

Is it really practical to ask that we hold onto transparency? Let’s take stock of the costs.

Measuring transparencyseems like a reasonable expectation for a frontier AI company. The resources required are small in comparison to building the model itself, and scientific insight into transparency is a basic requirement for our trust in it.

Similarly, auditing frontier training runs for incentives that undermine reasoning transparency doesn’t need to be a big lift. Directly training the Chain of Thought isn’t usually necessary, and when there is a safe way to do it for a legitimate reason, justifying that choice should be straightforward. And again, establishing much more effective auditing processes doesn’t need to be especially costly or impractical.

Preserving transparent architectures might be the real test of how willing we are to insist on transparency. If opaque architectures, like reasoning in latent space, perform better than today’s reasoning architectures, we will need to confront some trade-offs: give up on a crucial window into understanding how AI models think - and the key enabler of our ability to monitor these systems - or sacrifice gains to performance and capabilities.

We don't claim to know exactly how to navigate these trade-offs. But the safety and understanding afforded by transparency benefit everyone. Conversely, the hazards of opaque models might have society-wide implications. Commercial goals alone might therefore under-incentivise transparency and, in that case, regulation could help.

What if future reasoning modelsdogive up on Chain of Thought for opaque methods? We might still be able to recover reasoning transparency with new and better tools for translating machine thoughts into human language. It’s a much tougher challenge than taking models that are already transparent, and making sure they stay that way, but it’s worth our efforts, and we’re already working on these challenges of interpretability. Minimally, we propose that if AI developers deviate from standard Chain of Thought, they should provide a robust and convincing demonstration that their new methods are comparably monitorable.

For all the ways we still don’t understand AI systems, we shouldn’t overlook how fortunate it is that the best reasoning models today think out loud in a way humans can understand. That is a fragile and contingent fact - and we should work to ensure it stays true.

---

*Syndicated from the DeepMind Institute. Canonical URL: https://institute.deepmind.com/essays/the-case-for-reasoning-transparency/*