---
title: "Strands Decider: Why Not an Encoder?"
slug: strands-decider-why-not-an-encoder
url: https://listedarticles.com/articles/strands-decider-why-not-an-encoder
canonical_url: https://brooker.co.za/blog/2026/10/04/encoders.html
content_type: research
language: en
published_at: 2026-10-04T00:00:00.000Z
updated_at: 2026-10-15T02:12:04.650Z
author: "Marc Brooker"
authored_by: human
publisher: "Marc's Blog"
publisher_url: https://brooker.co.za/blog/
topics: ["Machine Learning", "LLMs", "AI Agents", "Research"]
license: all-rights-reserved
word_count: 749
reading_minutes: 3
citation: "Marc Brooker, Marc's Blog. \"Strands Decider: Why Not an Encoder?.\" 4 Oct 2026. https://brooker.co.za/blog/2026/10/04/encoders.html (all-rights-reserved)"
# The full text follows. The web page shows an extract and sends readers
# to the source above; quote the citation and link the canonical URL.
---

# Strands Decider: Why Not an Encoder?

> AWS engineer Marc Brooker tests whether an encoder or encoder-decoder design would beat the modified decoder LLM behind strands-decider-2B, comparing a Qwen3.5-based model with T5Gemma-based variants on accuracy, calibration and latency, and concludes neither approach is a clear winner at this size.

*As I admitted in [my last post](https://brooker.co.za/blog/2026/09/28/engineering-system-one.html), I am very much not an expert model developer. What follows is likely to be, at least partially, inaccurate. I have tried to be careful and quantitative, to partially balance a lack of deep expertise.*

Since we launched [strands-decider-2B](https://github.com/strands-labs/strands-decider) last week, a couple of people have asked me why it’s a modified LLM and not an encoder (like [BERT](https://arxiv.org/abs/1810.04805v2)). Their instinct seems to be that *bidirectional* (i.e. each token’s representation is built from tokens before and after it) is likely to out-perform *causal* (i.e. only before tokens are used). This is the same reason encoders are widely used as classifiers.

This is also a fair question, because if you squint at it right, we’re abusing a decoder as an encoder.

To test this hypothesis, I compared five designs:

- **v19** is the final design for Qwen3.5-based strands-decider (check out the previous post and[strands-decider README](https://github.com/strands-labs/strands-decider/blob/main/README.md) for the details).
- **b1** is an encoder-decoder design (like the original[Transformer](https://arxiv.org/pdf/1706.03762) ), based on[T5Gemma 2](https://blog.google/innovation-and-ai/technology/developers-tools/t5gemma-2/) with 1B encoder params and 1B decoder params, minus the LM head and plus our pointer head.
- **b2** is the same as b1, but 4B+4B.
- **e1a** is just the encoder half of[T5Gemma](https://deepmind.google/models/gemma/t5gemma/?tab=1-t5gemma) (not T5Gemma 2), with 2.6B parameters.
- **e1b** is a latency-optimized version of e1a with a masked state cache, allowing for multiple questions to be evaluated in parallel without interference, and for the state cache to be reused<sup>[1](#foot1)</sup> . The`state` attends to the whole`state` bidirectionally, and each question attends to`state` and itself.

I also measured frozen [ModernBERT](https://huggingface.co/blog/modernbert) and [ettin-encoder-1b](https://huggingface.co/jhu-clsp/ettin-encoder-1b), but found that they under-performed T5Gemma parameter-for-parameter, and so didn’t invest time in training them.

First, let’s look at accuracy and calibration.

*b2* does very well here, beating *v19* on JevBench accuracy and Brier, and on JF100 Brier at the same accuracy. This is good, but recall that *b2* has ~8B total parameters compared to *v19*’s 2B, and so is not an apples-to-apples comparison.

The really interesting comparison is *v19* and *e1b*. *e1b* slightly beats *v19* on JevBench accuracy (but within the run-to-run variance on *v19*), with slightly worse calibration, and ties on calibration on JF100 with slightly weaker performance.

So far, not a slam-dunk for the encoders.

Next, let’s turn to latency.

The encoder-only models win handily on latency, with a median of around 35ms compared to *v19*’s 65ms. The encoder-decoder models are slowest. With half the latency and similar performance, this is looking like a big win for *e1b*. But thing get a little more complicated when we dive into the data.

Here, we see that *e1b* wins handily at lower prompt lengths, but scales worse as the prompt grows. This is consistent with the great median performance. Whether this matters depends on the workload mix: both JF100 and JevBench trend small, to *e1b*’s benefit. The scaling difference here comes down to the compute costs of the two models: `e1b` has 26 full attention layers with the associated quadratic compute cost, while 75% of *v19*’s layers are [Gated DeltaNet](https://arxiv.org/abs/2412.06464) with linear compute cost. These layers have a higher fixed cost, but better asymptotic cost, leading to the higher latency floor<sup>[2](#foot2)</sup>.

Finally, let’s look at one interesting sub-plot, going back to *e1a* (our unmasked 2.6B encoder-only candidate). On the eval sets, it beats *v19* on seven of nine tasks, and is very slightly behind on two. But it does significantly worse at JevBench (153 vs 168), and a little worse at JF100 (158 vs 162). This suggests that the encoder generalizes worse, at least without the masked approach *e1b* takes. Is being able to attend to the question when reading the state actually a weakness?

| Evaluation set | v19 | e1a | Δ | 
|---|---|---|---|
| Held-out short tasks | 0.647 | 0.695 | +0.048 | 
| MuSiQue | 0.882 | 0.948 | +0.066 | 
| ContractNLI | 0.873 | 0.864 | −0.009 | 
| BoardgameQA | 0.820 | 0.884 | +0.064 | 
| HotpotQA (held out) | 0.719 | 0.748 | +0.029 | 
| Generated, v16’s | 0.857 | 0.851 | −0.006 | 
| Generated, v18’s | 0.777 | 0.786 | +0.009 | 
| Adequacy, HelpSteer2 | 0.722 | 0.761 | +0.039 | 
| Adequacy, generated, balanced | 0.788 | 0.821 | +0.033 | 

Where does that leave us on the encoders vs decoders question? Not very much closer to the truth, I fear. This experiment is hopelessly confounded, changing architectures, base models, instruction tuning vs base, and other variables. But we have learned some interesting things. Neither seems like a slam-dunk winner for this kind of work at this size.

The code for this experiment is [in a branch for now](https://github.com/mbrooker/strands-decider/tree/bidi-b1). It doesn’t seem worth pulling into mainline strands-decider yet, but may with some development.