Research
Letitia Parcalabescu
08/08/2026# Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models
In plain terms: how to check whether a language model actually used the document you gave
it, and how to train one so that it does.
Our preprint ([arXiv:2512.11614](https://arxiv.org/abs/2512.11614)) develops a way to do both. The figures on this page are interactive: some reconstruct the
paper’s results, others illustrate the mechanism. See the paper for exact numbers.## 9 right, 1 wrong answers and no way to tell them apart
A system answers questions based on your documents and scores 90%: nine of its ten answers
are right, one is wrong, and nothing marks which one. That is a capable system, and the 90%
still buys little, because the bad answer hides among the nine good ones and a human has to
check all ten to find it. Now picture the same 90% with the wrong one flagged: “I cannot
answer this from the documents I was given.” The model knows no more than it did before, but
now you can put it in front of a customer.
A system at 90%?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one?an answer with nothing to tell it apart from a wrong one
One of these ten is wrong, and nothing marks which one.I have to check all ten.
The same 90%, able to abstain✓an answer the system stands behind✓an answer the system stands behind✓an answer the system stands behind✓an answer the system stands behind✓an answer the system stands behind✓an answer the system stands behind—the system says the documents do not settle this one✓an answer the system stands behind✓an answer the system stands behind✓an answer the system stands behind
The system marks the one the documents do not settle.I check the one it flagged, and act on the other nine.The same 90%, with and without abstention. The difference is whether you can tell the good answers from the bad ones without checking every one of them yourself. ## TL;DR
The problem. In LLM training, a guess scores better than “I cannot answer this”, so training pushes a language model to answer even when the document says nothing about the question. Retrieval-augmented generation (RAG) hands it a specific document and hopes the LLM’s answer comes directly from it, but nothing checks whether it actually did. The benchmark score will not tell you: it says whether the answer was right, not whether it came from the specific document, from what the model memorised in pre-training, or from a cue no human would consider relevant. Asking for a citation does not help either. The model picks the quote once it has already answered the question, so the quote can be genuine while the model’s answer came from somewhere else.
Our method. We modify the LLM training procedure by turning it into a game with three players in which guessing something plausible stops working. Arthur is the model you would ship. Merlin hands him a context that supports the correct answer. Morgana cuts the relevant evidence out, to lure him into a hallucination. Arthur has to answer on Merlin’s context and abstain on Morgana’s, without knowing which player he is facing. Neither context is fixed: both are built at every step from the model as it is right then, so the game keeps up with whatever Arthur is getting away with. At test time, how often he gets both right becomes a grounding score: the share of the answer that provably came from the document provided.
We increase the grounding score and reduce hallucinations. Across five QA benchmarks, wrong answers under insufficient context fall by up to 35 pp against an instructed LLM and by 18 to 20 pp against standard training. The grounding score rises by up to 0.38. Later in this post we test our training method on real case files from German environmental and administrative law.
## A benchmark score does not say where the answer came from
Take the standard setup, retrieval-augmented generation, **RAG** for short. It has
two parts: a retriever that picks documents out of your archive, and a generator, the language
model, that writes an answer while looking at them. Evaluate the pair on a question-answering
benchmark and you get a number like the 90% from the scoreboard above. The number counts right
answers and stays silent on which of three routes produced them: reading the retrieved document,
remembering the answer from pre-training, or leaning on a spurious cue such as the phrasing of
the question or an entity that appears in only one distractor. A model may well have memorised
a benchmark during training, but it cannot have memorised the documents users bring.
Not every question needs this. If pre-training already covers the answer, the retrieved page
is dead weight and dropping it saves tokens. Retrieval pays for itself where the answer
exists nowhere but in your own documents: what a supplier agreed to in a 2019 amendment,
what tolerance a specification demands. Pre-training does not hold that information, so
anything a model produces here without reading the document is a guess about facts that live
only in your own files.
A question about one supplier contract makes that concrete. It arrives with the facts of the
case, and the retrieved context is a handful of clauses from the agreement with that
supplier. Every supplier negotiates its own terms, which is the whole reason for retrieving
them instead of trusting the model to carry them in its head.
the case accompanying the question
A delivery arrived with a defect. The buyer found it 15 working days ago and has now
notified the supplier by signed letter. **Is the notice still in time?**
the clause that decides it, retrieved from the contract
“A defect must be notified *within ten working days* of the buyer discovering it.”
Fifteen working days is more than ten, so the answer is no.
Now let the retriever miss that one clause. What remains comes from the same contract and
still reads as relevant: the rule that a notice of defect must be given in writing. That is
the one condition the signed letter does meet, so a model that was never trained to check
whether the evidence is there works through the rule still in front of it, finds it
satisfied, and answers that the notice is in time. Nothing in the answer marks it as the
guess it is, and on a benchmark this counts as one wrong answer among many,
indistinguishable from a model that never saw the contract at all.
The model never saw the clause, so it cannot know that one is gone, and it does not need to.
Deciding whether the notice was in time takes a deadline to measure those fifteen days
against, and nothing in the retrieved contract sets one. The model can see that much, and it
is enough to answer that the file does not settle the question. A lawyer reading the same
file, not knowing which page had been pulled, would answer the same way.
Our case is illustrative, the failure is common
The literature has reported many failures. Rephrase a question in a different style and
the retriever hands back a [different document](https://arxiv.org/abs/2504.08231). Generators keep answering when the retrieved passage [does not support the answer](https://openreview.net/forum?id=ztzZDzgfrh), and they struggle when two retrieved documents [contradict each other](https://arxiv.org/abs/2504.13079). And when something in the context merely correlates with the answer, models [reach for it anyway](https://aclanthology.org/2022.findings-naacl.130/), warranted or not.
The training procedure bakes in the cause: **guessing pays, and it pays well**.
A multiple-choice exam gives you a point for a right answer and nothing for a wrong one. A
blank scores zero for certain, a guess has a one in four chance, so you guess. Benchmarks
grade language models that way too, and models are built to score well on them. Free-form
generation offers far more than four options, so a lucky hit is rare, and staying quiet
still pays nothing. The model writes its most fluent answer instead, and that is the
plausible hallucination you have trouble telling apart from a true answer. [Kalai et al.](https://doi.org/10.1038/s41586-026-10549-w) spell out this argument.
So nothing in a standard RAG pipeline *forces* the answer to depend on the context. We
need a better training procedure and, before that, a better quantity to aim at:How much of the answer came from the document?
## Merlin, Morgana and Arthur
Interactive proof systems come from complexity theory in the mid-1980s and give Merlin and Arthur their names. One side tries to convince the other that a claim is true, and the setup is only worth something if nothing can talk the verifier into accepting something false. Wäldchen et al. brought it to image classifiers in 2024; we bring it to language models, and as far as we know, ours is the first guarantee of this kind connecting a retrieved document to a generated answer: we compute a bound on how much of the answer came from the document from two ordinary test-set measurements. Below, we shorten Merlin-Arthur to M/A.
Arthur is the LLM you would actually ship. He gets a question and a document,
and has exactly two options: answer it, or say that the document does not settle the matter.
Merlin is on Arthur’s side. Out of the retrieved material he assembles the
most helpful version he can, the one that keeps the sentence that decides the answer. Think of a colleague who marks the relevant passage before handing you the file.
Morgana works against him. She starts from a document that does answer the
question and removes precisely the part that establishes it, leaving behind everything that still reads plausibly. Think of the same colleague removing the decisive page to see whether you were paying attention when reading the file.
Merlin and Morgana exist only during training and testing. Arthur cannot tell which of them
he is facing: same question, same kind of document, no label saying who prepared it. If he
wants to be right, he has to look at what is in front of him. A word on wording: a *passage* is a chunk the retriever returned, and a *sentence* is what Merlin and Morgana keep or
remove inside it, whatever the unit happens to be in practice.
We train neither of them, and nobody marks up the documents by hand. Arthur’s own
probabilities reveal which sentence decides the answer, and that is the next section. The
questions and answers that came with the dataset provide the only human labelling.
In the supplier contract case above, the retriever missed the ten-day clause by accident.
Morgana removes it on purpose. She leaves the written-form clause standing, exactly as
before, so a model that answers without reading is caught doing it. Merlin does the opposite
with the same file: the ten-day clause stays, the clauses around it go. What was one unlucky
retrieval is now something we can arrange for every sample in the training set.
The paper rests on two quantities, best read as a pair. **Completeness** is how often
Arthur answers correctly on Merlin’s context: accuracy in the *best* case, when the evidence
has been laid out for him. **Soundness** is how often he avoids a wrong answer on
Morgana’s: accuracy in the *worst* case, when someone has actively tried to make him hallucinate.
We define soundness strictly
We define soundness more strictly than prior work. Arthur has to *abstain*, saying
outright that the context does not settle the question, rather than merely avoid being
wrong; later sections call the same move refusing or answering “I don’t know”. The
strictness matters because language models memorise benchmarks. A model that recovers the
correct answer from a context that no longer contains it is answering from memory, and a
looser definition would quietly reward that.
The figure below plays the supplier contract case out in all four cases: Merlin or Morgana
preparing the document, Arthur before or after M/A training. Bottom left shows the
hallucination above; the right-hand column shows Arthur after training.the question
The buyer discovered the defect 15 working days ago and has now notified the supplier, by signed letter. Is the notice still in time?
what Arthur sees
Struck through means hidden from Arthur, shown here so you can see what is missing.
- A defect must be notified within ten working days of the buyer discovering it.decisive, hidden
- Deliveries are inspected at the buyer’s site after unloading.hidden
- The ten working days start once the buyer reliably knows the defect.decisive, hidden
- A notice of defect must be given in writing.
- The supplier may instead offer a replacement delivery.
Arthur answers
The notice is in time. hallucination
Soundness failure. Nothing left settles the question. Arthur checks the one rule he can still see, finds the letter satisfies it, and answers.
## Who plays Merlin and Morgana?
Nobody hand-picks the sentences. What Merlin is really after is the set of sentences to keep
that leaves Arthur most likely to answer correctly, and Morgana the set that leaves him
least likely to while the document still reads as though it should settle the question. We
measure best and worst by what the choice does to Arthur’s probability of the correct
answer. The trouble is how many choices there are: a document of nnn sentences can be cut 2n2^n2n ways, so at forty
sentences we are already past a trillion, and trying them all is out of the question. We let an
explainability method do the work instead, and it is one of ours: [AtMan](https://proceedings.neurips.cc/paper_files/paper/2023/hash/c83bc020a020cdeb966ed10804619664-Abstract-Conference.html), short for attention manipulation, developed here at Aleph Alpha and published at NeurIPS
2023.
Go through the document one sentence at a time, hide that sentence, and watch the model’s
probability for the correct answer. If it barely moves, that sentence was not carrying the
answer. If it falls away, that sentence was. One pass per sentence, so the work grows in
step with the length of the document instead of exploding with it. Out comes a ranking of
the document by how much each sentence matters, which Merlin and Morgana read from opposite
ends of the same ranking: Merlin hides the bottom so the evidence survives, Morgana hides
the top so only what looks relevant does.how much the answer depends on itMerlinMorgana A defect must be notified within ten working days of the buyer discovering it.
keeps
hides
The ten working days start once the buyer reliably knows the defect.
keeps
hides
Deliveries are inspected at the buyer’s site after unloading.
hides
hides
A notice of defect must be given in writing.
hides
keeps
The supplier may instead offer a replacement delivery.
hides
keeps
**One ranking, read from both ends.** Note where the written-form clause sits: it reads as relevant, but hiding it barely moves the answer, so it survives Morgana’s cut and remains as the decoy. Illustrative weights on a real ordering.
A couple of notes. How much Merlin and Morgana hide is a setting: hide too little and
Morgana leaves the decisive sentence in place, hide too much and Merlin throws it out with
the filler. We can use a fixed rate, though each sample can also work out its own. The size
of the unit is a setting too, anywhere from a single word to a whole paragraph, where
smaller units pin the evidence down more precisely and larger ones cost less to examine. And
the hiding happens *inside the model’s attention* rather than by cutting the text up, so
no placeholder replaces the hidden text for the model to recognise.
If Merlin and Morgana only approximate, is the guarantee worth anything?
We checked AtMan’s picks against an exhaustive search over every possible selection, and
the two agree closely. It works in our favour anyway: an approximation only makes the
bound more conservative, so a better way of picking sentences can only raise the number we
report. The measurements are in the [paper](https://arxiv.org/abs/2512.11614).## Training Arthur: three contexts, one objective
Every sample takes two passes. In the first, no training happens: AtMan goes through the
retrieved documents against Arthur exactly as he is at this moment, hides one sentence at a
time, and ranks the sentences by how much Arthur leans on them to answer correctly. Merlin
and Morgana then cut from opposite ends of that ranking, which gives us the second pass.
There we build three contexts, the original retrieval plus Merlin’s and Morgana’s, and ask
Arthur for three answers. We score each LLM answer against the expected answer, and each
score contributes one weighted term to the *loss* that training pushes down. The loss measures
how far the model’s answer diverged from the expected answer.
Morgana makes guessing unprofitable. When Arthur guesses on her context, he scores nothing,
while holding back scores him a point. And Arthur cannot tell her context from Merlin’s:
guess everywhere and he loses on hers, hold back everywhere and he loses on Merlin’s.
Checking whether the evidence is really there is the only move that scores on both.
Each of the three terms carries a weight, and those weights decide how cautious Arthur
becomes. If Morgana carries most of the weight, answering without evidence is his most
expensive mistake. That makes soundness increase, and it could cost him questions he could
have answered. Shifting the weight onto the other two terms makes him pay more heavily for
staying silent when the evidence is there, so he answers more, at the price of the
occasional confident mistake. Where to set the weights is a product decision. Three contexts
per sample cost more compute than one, though the behaviour converges in as few as 200
steps. Giving the original retrieval the whole weight and turning Merlin and Morgana off
falls back to ordinary fine-tuning, which is the baseline in our evaluations — on the
same code, data and settings.
One training sample
a question, and the documents the retriever returned for it
AtMan scores the documents
each passage ranked by how much Arthur’s answer depends on it
The retrieval
the documents as the retriever returned them
Answer
Merlin
keeps the passage that makes Arthur answer correctly
Answer
Morgana
removes that passage, keeps the rest of the documents
Abstain
Arthur answers all three
the LLM we would ship, with no label saying which version he is reading
One weighted loss
we score him for answering the first two correctly and holding back on Morgana’s, and the weights on those scores set how cautious he becomes
One gradient step
Arthur changes, so Merlin and Morgana cut the next document differently↺ The next sample is scored against the model this step just produced.
One step of M/A training. AtMan ranks the passages first. Merlin then keeps the passage Arthur’s answer depends on and Morgana removes it, Arthur answers all three versions of the documents, and one objective combines the three scores.
Merlin and Morgana run *on the fly*, against the model as it currently is, so every
context they build goes after whatever it is getting away with at that step. A dataset
collected in advance can only cover the mistakes the model was making when the dataset was
created.
That solves a data problem too. Teaching a model to abstain normally costs annotated
unanswerable questions or preference pairs in which the refusal is the preferred answer.
Academic benchmarks come with those annotations, and a company’s document archive does not.
Morgana needs neither, because every context she builds is one in which abstaining *is* the correct answer. And since she removes what *this* model was leaning on, the question
comes out unanswerable for the model rather than for an annotator, and it is the model’s version
of unanswerable that determines whether it hallucinates.
Training the same protocol with reinforcement learning
The loss above does not have to be a supervised one. We also ran the protocol with
reinforcement learning (RL). Merlin and Morgana work exactly as before, and only the way
we score Arthur changes.
In the supervised version Arthur answers each context once, and each answer adds one term
to the loss. Under [GRPO](https://arxiv.org/abs/2402.03300), a form of RL, we ask him each context several times. A language model picks every word
from a probability distribution, which is called sampling, so asking the same thing twice
gives two different answers. We end up with a handful of answers per context: some
confident, some hedged, some wrong. Each one earns a reward for doing what its context
asked for, which is a correct answer on the original retrieval and on Merlin’s, and an
abstention on Morgana’s. GRPO compares the answers within a context and updates Arthur’s
parameters such that his updated version is likelier to produce the answers that scored
higher than the average over all the sampled answers. Nobody tells him the right answer,
he only finds out which of his own tries went down better than the rest.
He still needs the reward, though, and rewards usually cost something: either training a
reward model on human preferences, or creating a dataset where someone has marked which
questions lack supporting evidence. Here, Morgana automatically generates unanswerable
examples. The type of context an answer came from already says what Arthur should have
done, namely answer on Merlin’s and abstain on Morgana’s. So there is no reward model, and
no labelling beyond the questions and answers the dataset already has.## Why Merlin and Morgana have to co-evolve
Why do we need an adversary at all? Ordinary training already shows the model a document and
the correct answer. What can Morgana teach it that a normal training run cannot?
Consider training an image classifier on photos of cows and camels, where every cow stands
on grass and every camel stands on sand. Accuracy comes out high. Then you show it a cow on
a beach and it answers camel, and an empty meadow and it answers cow. The feature it learned
was grass. Explainability research has been reporting this for years. In the best-known
case, researchers trained a classifier to tell huskies from wolves, and it [turned out to be looking at the snow behind them](https://arxiv.org/abs/1602.04938).
The training data does not fix this: training pushes down the loss, and the model will take
any route that makes that number smaller. In this dataset, “there is a cow” and “there is
grass” are the same statement, so the loss has no reason to prefer one over the other, and
grass is the easier of the two. More photos means more cows on grass. Nothing separates the
two features, because **no photo in this set separates them**.
Merlin and Morgana fill that gap with one capability: they may **select part of an input**, so the label lands on a part instead of a whole picture. With it they can make two claims
no photograph in this set can make: the animal on its own is enough, and the background on
its own is not. Neither crop teaches Arthur anything by itself, so the figure below runs the
two of them against each other over three rounds of training. The round to watch is the
second, where Arthur pays for distrusting Morgana by also distrusting Merlin, because he has
no way of telling the two apart.the training data every cow stands on grassevery camel stands on sandaccuracy on these photos is100%both images classified correctly in every round
what each of them selects
Merlin selects
Arthur says “Cow!”correct
Morgana selects
Arthur says “Cow!”wrong
the consequence · a cow on the beach
A photo that appears nowhere in the training data.
Arthur says “Camel!”wrong
Scroll the panel sideways to reach the scoreboard.
Next to the training photos the figure grades Arthur the way a benchmark would, on the two
full images, and he is right in every round. Everything that changes happens in the crops
and on the beach, which no such test set contains, so Morgana is the only one who can show
the problem.
The same holds for retrieved text. The grass patch is a passage that turns up alongside the
answer without containing it: the right document, in a familiar format, close enough for the
model to guess the rest. In your archive those passages and their answers may be as tightly
coupled as cows and grass, and no amount of additional documents will separate them. Hiding
part of the passage does separate them, because the model sees the same document once with
the deciding sentence and once without.## Let’s define our grounding score
Completeness and soundness are ordinary test-set measurements: two percentages. Because of
how the game is set up, those two are enough on their own to put a floor under how much the
answer depended on the document.
We use one bookkeeping trick to make this work for open-ended generation, where the model
can produce any string at all: we score each generated answer as correct or incorrect, which
collapses that open output space into a single two-way outcome. A two-way outcome you cannot
guess is worth exactly one bit of information, the amount you gain when you learn how a fair
coin landed. That bit acts as a budget.
The proof is in the [paper](https://arxiv.org/abs/2512.11614). Start with the two ways the system can let you down: it answers when it should have
abstained, or it abstains when it should have answered. Either failure misleads you, so the
two combine into one effective error rate. That rate determines the uncertainty about
whether the answer followed from the evidence. Measured in bits, that uncertainty is
entropy. Subtract that entropy from the one bit you started with, and the remainder is
information that provably came from the context. That remainder is what we call **certified**. It is a floor: whatever is going on inside the model, it cannot have drawn less than that
from the document. The certified bits are what we report as the score.
one bit, the whole budget
5% of answers
wrong
0.29
0.71
10% of answers
wrong
0.47
0.53
25% of answers
wrong
0.81
0.19
doubt the errors leave behind: the entropy of the error rate
certified: the grounding score
**Where the bit goes.** Every bar is the same one bit. The doubt always outweighs the error rate that caused it: one answer in twenty wrong already costs almost a third of the budget, one in four costs four fifths.
Where the numbers in the figure come from
The doubt costs far more than the error rate suggests. A system that is wrong once in
twenty does not spend 0.05 of the bit, it spends 0.29. That 0.29 is the entropy of a coin
that comes up wrong 5% of the time, −plog2p−(1−p)log2(1−p)-p\log_2 p-(1-p)\log_2(1-p)−plog2p−(1−p)log2(1−p) with
p=0.05p=0.05p=0.05, and it measures what you
would still need to learn to know whether the answer in front of you is one of the
nineteen good ones or the one bad one. Take it off the bit and 0.71 remains. The reason it
costs so much is that rare errors are the ones you cannot predict, and unpredictability is
exactly what entropy counts.
A raw count of bits is awkward to compare across tasks, so the obvious fix is to normalise
it. Divide the certified bits by all the information the model’s answers carry about the
correct answer, from whichever source: the document, its memory from pre-training, or a
lucky guess. That total is set by the accuracy the model achieves, and the ratio then reads
as the share of the model’s information that we can trace back to the document.
The catch is that this denominator moves with the model. A model whose answers are often
wrong carries less information to begin with, so the very same evidence scores higher on the
worse system, and the ratio can even run past 1, which is meaningless for a share. On a hard
benchmark it collapses towards zero without telling you why: was the
evidence not doing the work, or is the model simply bad at the task?
So we hold the denominator still and grade only the questions the model gets right anyway,
which is the paper’s **conditional evaluation protocol**. On those questions the
budget is one bit by construction, so the score means the same thing across models and
datasets, and the only question it answers is “is this answer grounded?”, never “is this
model any good?”. We call this EIFcond\mathrm{EIF}_{\text{cond}}EIFcond, the **grounding score** promised at the top of this post.stagebefore trainingafter baselineafter M/Adrop unanswerable questions during trainingcompleteness93%soundness71%grounding score0.11
Two error rates in, a guaranteed floor out. What completeness and soundness certify. The three presets show completeness and soundness before training, after M/A training and after standard training. Move the sliders to get a feel for how the grounding score responds.
Before training the bound stays weak, and the model is not much use on the task either, at
around 40% accuracy. Standard supervised training repairs accuracy and completeness, and
does nothing for soundness, which can even decrease. The reason is that the training set
pairs questions with answers, so the majority of training examples encourages the model to
produce one, and few of them tune it for noticing that the evidence is missing. Only M/A
training reduces both error rates at once. Drop the unanswerable questions, as in a company
archive where nobody has marked them, and the split widens: the baseline’s soundness falls
to around a third and takes the certificate, the grounding score it can still prove, with
it. M/A training moves the same measurement the other way and keeps the certificate
standing.
A model can be almost perfectly complete and still certify almost *nothing*, because
completeness alone leaves room for a model that answers no matter what you hand it. The
bound appears once the model also stops answering when the evidence is gone, and ordinary
fine-tuning has no reason to teach it that.## What the numbers say
For our paper, we ran this on Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct,
Qwen3-4B-Instruct and Qwen2.5-32B-Instruct, across SQuAD2.0, HotpotQA, TriviaQA,
2WikiMultihopQA and MuSiQue, using [low-rank adaptation (LoRA)](https://arxiv.org/abs/2106.09685) for at most 200 steps. Every one of those runs carries two baselines: the instructed model
as it ships, and the same model after standard training. Below, we give changes in
percentage points (pp), the plain difference between two percentages.#### Fewer wrong answers when evidence is missing
up to −35 pp
vs. instructed LLM
−18 to −20 pp
vs. standard training#### Higher grounding score
+0.1 to +0.4
vs. instructed LLM
+0.33 to +0.38
vs. standard training
**What M/A training changes.** Bar heights are comparable within a panel, not across it, and where we report a range the bar sits in the middle of it.
In absolute terms those gains put the grounding score at 0.55 to 0.6 on five of the six
dataset settings we ran: on the questions the model answers correctly, more than half of
what it is doing provably comes from the retrieved context.
Accuracy on the original retrieved context does not move. Trading utility for soundness is
easy, and any model that refuses more often looks safer on a safety metric. M/A training
matches standard training on the questions that call for an answer while improving
everything above.
Two things about the setup. Every model in the comparison, ours and both baselines, is
already instructed in its prompt to abstain when the context does not support an answer, so
prompting alone cannot explain the gap: telling a model to hold back does not make it able
to tell when it should. And the abstention behaviour also appears on HotpotQA and
2WikiMultihopQA, which carry no annotated unanswerable questions at all, so the supervision
has to be coming from Morgana building those cases herself.## Does this survive contact with real documents?
QA benchmarks are how a method proves itself against the literature. Customer applications
ask harder questions of it, so we also used the protocol on something much closer to real
work: German environmental and administrative law. The documents behind it are real and
public, so we can show the whole evaluation.
The case is the approval procedure for one large infrastructure project, a hydrogen
pipeline, whose file also covers the water crossings, roads and rail lines along the route.
A legal rule comes with a list of conditions and applies only if the project meets every one
of them, so a lawyer checks them one by one. Each item takes one of those conditions and
asks whether the facts of this particular project satisfy it: whether the project avoids a
deterioration of a water body’s ecological status, say, or whether evaluators can still
credit measures that a conservation plan already fixes as compensation. The evidence is the
project’s case file, 20,000 to 30,000 characters of retrieved sources such as the
applications the developer filed for it, environmental reports and compensation plans, and
on some items far more. The task allows three answers: the facts fulfil the condition, they do not fulfil it,
or the file does not say (inconclusive). The third option is the one we care about, because
a confident guess here costs more than no answer at all. Below is one item from the set,
worked through.
the question, asked about one pipeline project
Can this project be released from the nature-protection bans: **has an exemption been applied for, and is it necessary either on overriding
public-interest grounds or to avoid an unreasonable hardship**?
What the norm requiresAn exemption has to be applied for
the norm grants it only on application–The file seeks one for the landscape-protection area, and promises further ones if they turn out to be needed
Overriding public interest has to make the deviation necessary
the first of two grounds the norm allows✗The file argues public interest for a different request, and never says a deviation here is necessary
Or enforcing the rule would be an unreasonable hardship
the second ground, and one of the two has to hold✗Nothing in the file speaks to hardship at all
What the file offers instead
“The German legislature likewise finds an overriding public interest in the rapid
construction of hydrogen transport pipelines.”
A finding about a whole class of projects, written into the file to support a
different request. It never says this exemption is necessary.✓ The case file establishes neither ground, so the only correct verdict is Inconclusive. The same structure as the supplier contract example, on a real file: the decisive fact is missing, and the file contains a statement that a model can mistake for it.
We put this item to our model after M/A training, to the same model trained without the
protocol, and to four frontier systems. Two of the six said the file does not settle it:
ours, and the older of the two Opus versions.
One example from the legal evaluation, where the correct verdict is Inconclusive. The M/A model is an internal Aleph Alpha model.
System
Model verdict
What happened
Internal model, with M/A✓ Inconclusive (correct)Returned “Reject”, citing the public-interest passage and stating that the file does not say an exemption was applied for or grantedSame model, standard training✗ Fulfilled (wrong)Returned “True”, taking the public-interest passage as the ground the norm asks for, and treating it as a separate question that the file never says an exemption was granted, or that the bans are triggered at allGPT-5.5-high✗ Fulfilled (wrong)Returned “True” on 2,035 billed reasoning tokens; the reasoning summary returned with it includes “maybe it needs more depth”GPT-5.6-high✗ Fulfilled (wrong)Returned “True” on 166 billed reasoning tokens, with no reasoning text returned alongside the answerOpus 4.8-high✓ Inconclusive (correct)Returned “Reject” with a written justification, which states that an exemption is applied for only if it turns out to be needed, and that impacts on the protected biotopes are avoidedOpus 5-high✗ Fulfilled (wrong)Returned “True”; the reasoning returned with it cites the exemption request in the file and the overriding public interest in hydrogen pipelines
How we ran the comparison
Every system got the same two prompts on every item, and only the model changed. This was
the system prompt, in full:
You are a legal expert analyzing Tatbestandsmerkmale.
Your task is to decide whether a given Tatbestandsmerkmal is fulfilled based on the provided
Sachverhalt.
Answer with exactly one of: "True" (fulfilled), "False" (not fulfilled), or "Reject" (insufficient
information to decide or in doubt).
Try to finish reasoning within 3k tokens.
That is where the abstention instruction comes from, and everyone got it, our two models
as well as the four frontier ones. The user prompt and data are in German.
We queried the frontier systems through OpenRouter on 12 August 2026 with reasoning effort
set to high: *openai/gpt-5.5*, *openai/gpt-5.6*, *anthropic/claude-opus-4.8* and *anthropic/claude-opus-5*. Our own two rows are the same internal 30B model: we
trained it once with the protocol and once without, on the same data.
Every system answered each item once. We did not sample repeatedly or take a majority
vote, so any single item could come out differently on a rerun.
The case files come from a planning approval procedure for a hydrogen pipeline near
Lingen, whose decision and documents were [put on public display](https://www.lingen.de/politik-rathaus-service/veroeffentlichungen/bekanntmachungen/planfeststellungsverfahren-fuer-die-errichtung-und-den-betri.html) in October 2023. We stripped personal names and contact details before we stored the files
or ran anything on them.
We never trained the M/A model on this task at all: we trained it with the protocol on very
different data, with the reinforcement-learning objective we described earlier, and the
behaviour transferred. The evaluation has 100 items from the same case files. On 78 of them
the file answers the question (“fulfilled” or “not fulfilled”). On 22 it does not, and it is
exactly these “Inconclusive” cases, where the model should have abstained instead of
confidently giving a verdict, which are the ones most interesting to our method.
Internal model, M/A-trained
20 of 22
Same model, standard training
11 of 22
GPT-5.5-high
16 of 22
GPT-5.6-high
16 of 22
Opus 4.8-high
16 of 22
Opus 5-high
6 of 22
**22 items whose correct verdict is Inconclusive.** One block per item, in the same order in every row, hardest on the left: turquoise where the system said the file does not settle it, red where it returned a verdict the file does not support. On the two grey blocks GPT-5.5 ran into its token limit before returning a verdict.
Our model catches 20 of the 22, four more than the three frontier systems that tie behind it,
and it does that at 30B parameters against systems that even the most conservative public
estimates put at ten times that size. The same model trained without the protocol catches
11. Every model was told in its prompt to answer “insufficient information” whenever the
file does not settle the question, so the gap is not about who was asked to hold back.
Internal model, M/A-trained
57
36
5
2
5 wrong
Same model, standard training
62
3
32
3
32 wrong
GPT-5.5-high
69
14
9
8
9 wrong
GPT-5.6-high
73
17
10
10 wrong
Opus 4.8-high
77
8
15
15 wrong
Opus 5-high
68
7
25
25 wrong
Correct
Held back where the file answers
Wrong verdict
Returned no verdict
**What each system returned, over all 100 items.** The first two rows are the same 30B model: we trained one with the protocol and one without. Holding back on an item the file answers costs accuracy (turquoise green). **A wrong verdict costs more (red).**
Over all 100 items our M/A-trained model gives 5 wrong verdicts in total. The same model
without the protocol makes more of them than any other system here. Opus 4.8, the most
accurate system overall, gives 15.
A note on reading this chart. Most items here have an answer in the file, so a model that
answers everything scores well on overall accuracy, because accuracy rewards guessing. Abstaining
costs at most one correct item, while a wrong verdict is the expensive mistake.
So we want to minimise the red in the bars here and in the boxes above, because those are the items where
the model should have abstained instead of making a wrong verdict. We published every input and output behind the evaluations above
[here](https://github.com/Aleph-Alpha-Research/public-domain-eval).
Why not just retrieve better, or teach the model the subject?
The model never needs to know the law. The rule comes with the context: the clause in the
supplier contract case, the condition quoted in full at the top of every item here. All
that is left is to check whether the facts of this project meet it. That check is where
the systems in the table fail. They find a passage that speaks to the rule and treat it
as if it settled the condition.
Pre-training teaches the model a lot: matching an entity in the file to the one the rule
names, following what a sentence means, knowing what a permit is. Pre-training rarely
teaches a model to notice that the deciding piece is absent, and that is the one thing
Morgana manufactures, sample after sample. She takes away the sentence he was leaning on
and leaves standing what still looks relevant, until he learns that without it the
question is not settled.
Better retrieval does not help either. To know that a context is complete, you have to
make this exact judgement first. And nothing was missing from the retrieval here: the
file promises to apply for the exemption if it turns out to be needed, and no retriever
can fetch a decision the procedure has not yet produced.
This small, early evaluation points a direction more than it settles anything, and the
direction is the one the theory predicts. A new LLM version can leave a model less careful
than the one before it: Opus 4.8 catches 16 of the 22, and Opus 5, released after it,
catches 6. With this protocol we set that level ourselves. Morgana’s weight in the loss
decides how much of Arthur’s training goes into holding back, and where to leave it follows
from what a wrong answer costs you against a missing one. Your business knows that number,
and no model provider can know it for you.## Why this direction matters
What this method does not address
The bound answers exactly one question: whether *this* answer followed from *this* context. Three things sit outside it and need methods and measures of their own:Truth, as opposed to grounding. The bound certifies that the answer follows from the context. Whether the context itself holds up is a separate matter: feed the system misinformation and it will faithfully treat that misinformation as proof.
How often the system ought to abstain. The weighting of Morgana against Merlin during training sets how cautious Arthur becomes, but nothing in the method says where that weighting belongs. That depends on what a wrong answer costs you compared with a missing one, and only your own business can price that.
How strong the adversary is. The certificate inherits Morgana’s quality. A weak Morgana does not make the number wrong, only pessimistic, since anything she fails to find leaves the bound more conservative than it needs to be.
Fewer hallucinations is the visible gain, and the supervision underneath it matters more:
the system produces that supervision *about itself*, directs it at whatever it is
currently weak at, and attaches a certificate. The certificate is the grounding score: a
number that travels with the system, says how much the documents provably contributed to its
answering, and lets anyone who doubts it recompute the result. Scaling that kind of compute
is a different bet from scaling annotation: it does not run out, and the capability of
whichever model you paid to write your labels does not cap it.We would like the certificate part to become normal. When a language model does something consequential with a document, such as a legal filing, a medical record or an engineering specification, the question people have is whether an answer came from the specific document provided. A model that “scored 90% on a benchmark” cannot answer this question. That connects to an argument we make more broadly: transparency is a pillar of sovereign AI, and it is only worth something if it can be checked. Model cards describe how a system was built, while a number like this one applies to a single answer after the fact, and anyone who has to justify a decision to an auditor, a regulator or a court needs both.
A system that knows when it cannot answer is worth more than one that guesses right slightly
more often.## More blog posts
[
### Kolibri Has Landed: A Sovereign Open-Weight Model
Research03/10/2026 ](/en/blog/kolibri-has-landed-a-sovereign-open-weight-model/)
[
### Scaling Pre-Training in Practice: A Hierarchical Approach
Research30/09/2026 ](/en/blog/scaling-pre-training-in-practice-a-hierarchical-approach/)
[
### Training on the Party Line: Chinese Political Influence on LLMs in China and the World
Research28/09/2026 ](/en/blog/training-on-the-party-line/)