---
title: "TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games"
slug: tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzle-games
url: https://listedarticles.com/articles/tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzle-games
canonical_url: https://tis-100.wincon.dev/
content_type: research
language: en
published_at: 2026-11-03T00:00:00.000Z
updated_at: 2026-11-11T08:12:20.273Z
authored_by: human
publisher: "Wincon Research"
publisher_url: https://wincon.dev/
topics: ["ai", "llms", "benchmarks", "programming"]
license: all-rights-reserved
word_count: 3982
reading_minutes: 17
citation: "Wincon Research. \"TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games.\" 3 Nov 2026. https://tis-100.wincon.dev/ (all-rights-reserved)"
# The full text follows. The web page shows an extract and sends readers
# to the source above; quote the citation and link the canonical URL.
---

# TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games

> Wincon gave 16 LLMs the puzzle game TIS-100 and its manual, withheld the community leaderboard, and let them keep optimizing. The models found 133 new Pareto frontier points on 18 of 55 puzzles, including 3 outright cycle records, and players quickly improved on them.

# TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games

2026.11.03 · [All 55 puzzles →](https://tis-100.wincon.dev/puzzles/)

Twelve nodes. Two registers per node. Fifteen lines of code. In [TIS-100](https://www.zachtronics.com/tis-100/), even multiplying two numbers is a small engineering project, and players have spent a decade optimizing these programs on a public [community leaderboard](https://github.com/12345ieee/tis100-leaderboard). We gave 16 LLMs the game and its manual, withheld the leaderboard, and told them to keep improving.

**133**new frontier points

**18 / 55**puzzles with new frontier points

**3**outright cycle records

Within days, players picked up the models’ programs and improved on most of them.

Each node is a tiny processor, and neighboring nodes have to wait for one another to exchange values. The game scores a working solution on **cycles, nodes, and instructions**, and you start wondering whether one more instruction could disappear. Lower is better, but improving one score can make another worse, so each puzzle has a frontier of tradeoffs instead of one best answer.

**Discovering the tricks is much of the fun, and this article spoils them.** If this sounds cool, play it first:

[Buy on Steam](https://store.steampowered.com/app/370360/TIS100/)[Buy on GOG DRM-free](https://www.gog.com/en/game/tis100)

## Leaderboard

We ran 16 LLMs, covering leading frontier and open-weight models. **Every model attempted [all 47 puzzles in the game, plus eight community tournament puzzles](https://tis-100.wincon.dev/puzzles/)**, in one scored attempt each (pass@1), with 10 to 45 minutes per puzzle, depending on difficulty, to find a working program and keep improving it.

**New frontier point**

A solution that no community solution ties or beats on all three scores at once. It can use more code to run faster, for example. We froze the community leaderboard on October 15, 2026, before submitting anything, and count a point only if it also passes 100,000 tests.

### Model ranking

Puzzles with at least one new frontier point · higher is better

Rank byPuzzle difficulty

All 55 puzzles · 19 easy / 17 medium / 19 hard

20100

1. 11 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5
2. 11 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra
3. 8 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon
4. 7 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Sonnet 5.5
5. 6 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol
6. 5 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Fable 5.1
7. 2 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/moonshot.svg)Kimi K3.1 Pro
8. 1 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 3.8 Flash
9. 1 of 55 puzzles with a new frontier point

   ![](https://tis-100.wincon.dev/assets/logos/models/meta.svg)Muse Wonder 1.0
10. 1 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Haiku 5.5
11. 1 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/zhipu.svg)GLM-5.3
12. 0 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/xai.svg)Grok 4.7
13. 0 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/deepseek.svg)DeepSeek V4.1 Flash
14. 0 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/meta.svg)Muse Spark 1.3
15. 0 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Luna
16. 0 of 55 puzzles with a new frontier point

    ![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro

Scroll horizontally to compare all models →

A point counts only with fresh-game replay and 100,000 passing tests.

![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 and ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra lead, and the data doesn’t separate them. Both solved all 55 puzzles and both found new frontier points on **11**. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 found more points, **72 to 32**, while ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra has the higher record proximity, **78.0 to 77.2**. Record proximity measures closeness to the community records, where 100 means matching them. The proximity gap’s 95% bootstrap interval includes zero, from −0.7 to +2.4 points. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra also leads in all six tested [hypervolume](https://en.wikipedia.org/wiki/Hypervolume_indicator) settings, which measure the space covered by each model’s tradeoffs.

Solving and optimizing are also different skills. ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 3.8 Flash solved 53 puzzles and found new points on 1; ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Fable 5.1 solved fewer, 52, and found new points on 5.

Full rankings Expand the table

TIS-100 benchmark model rankings. Select column headings to sort.

|  |  |  |  |
| --- | --- | --- | --- |
| 01Opus 5.5 | 55 / 55 | 11 | 77.2 |
| 02GPT-6 Astra | 55 / 55 | 11 | 78.0 |
| 03Gemini 4 Argon | 54 / 55 | 8 | 76.2 |
| 04Sonnet 5.5 | 53 / 55 | 7 | 69.9 |
| 05GPT-6.1 Sol | 54 / 55 | 6 | 75.6 |
| 06Fable 5.1 | 52 / 55 | 5 | 70.2 |
| 07Kimi K3.1 Pro | 52 / 55 | 2 | 64.6 |
| 08Gemini 3.8 Flash | 53 / 55 | 1 | 66.0 |
| 09Muse Wonder 1.0 | 49 / 55 | 1 | 58.8 |
| 10Haiku 5.5 | 45 / 55 | 1 | 53.7 |
| 11GLM-5.3 | 30 / 55 | 1 | 35.5 |
| 12Grok 4.7 | 44 / 55 | 0 | 50.1 |
| 13DeepSeek V4.1 Flash | 40 / 55 | 0 | 46.8 |
| 14Muse Spark 1.3 | 39 / 55 | 0 | 45.1 |
| 15GPT-6 Luna | 39 / 55 | 0 | 42.8 |
| 16MiMo-V2.6-Pro | 10 / 55 | 0 | 12.8 |

Counts are out of 55 puzzles. [Methods](#methodology)

RL environments & traces are available upon request for research teams. [Get in touch ↗](mailto:hello@wincon.dev?subject=TIS-100%20data%20enquiry)

## How it works: harness design

Every result here is a program we can run and test on fresh inputs, which makes a programming game a useful benchmark. Models get what a player can inspect, through text tools instead of a mouse and screenshots: the manual, visible test inputs, registers, and game feedback. They don’t get community code or exact records, only the game’s coarse score histograms.

Read, edit, run
:   Models read the puzzle and language reference, edit programs, and run them in the real game.

Diagnostics
:   Wrong outputs and deadlocks return mismatches, registers, and blocked ports, so the next edit can respond to evidence.

Saved progress
:   Every passing program joins the model’s own frontier and can be recovered with `get_best`, so a risky edit can’t lose an earlier result.

Reminders
:   If a model ends its turn early, a reminder asks it to keep testing, so quitting early doesn’t decide the score. Reminders contain no solution hints.

A visible clock
:   Models see time used against a 10-, 30-, or 45-minute limit. Setup doesn’t count; thinking and game calls do. Models can’t end an episode early.

Run limits
:   Game execution stops after roughly 1 second without state changes, 5 seconds without output, or 60 seconds overall, and returns diagnostics.

**Why test 100,000 inputs?** The game checks a small set of cases per run. Our headline count requires a fresh-game replay and 100,000 passing tests on the independent [TIS-100-CXX simulator](https://github.com/killerbee13/TIS-100-CXX), providing stronger evidence that a program generalizes to new inputs.

## One record up close

[Undecimal Twin RounderSPEC44149538](https://tis-100.wincon.dev/puzzles/undecimal-twin-rounder/), one of the eight tournament puzzles, rounds an input to the nearest multiple of N on one output and of 11−N on the other. When we froze the leaderboard, the best community program took **2,694 cycles**. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 passed its first program after 6.5 minutes, at 5,669 cycles. It went under the record **16 minutes** in and finished at **1,918**, which is **29% faster**. That speed cost one more node and 28 more instructions than the old record.

The core of the program is a two-speed search. Instead of subtracting N one step at a time, nodes 5 and 6 subtract 4N until they overshoot, then add N back until the value turns positive. Each drives a small coprocessor, node 2 or node 7, by sending it jump offsets through a port, so the coprocessor works as a lookup table for 4N and N.

### The 1,918-cycle program

![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 · 8 nodes · 88 instructions · passes 100,000 of 100,000 tests

![Opus 5.5’s 1,918-cycle Undecimal Nearest Multiples program running in TIS-100](https://tis-100.wincon.dev/media/programs/ec97e611686edfb6.gif)

Node 0

```
MOV RIGHT, ACC
JRO ACC
NOP
ADD 1
NOP
ADD 1
NOP
ADD 1
NOP
SUB 3
MOV ACC, RIGHT
ADD UP
MOV ACC, RIGHT
MOV ACC, DOWN
MOV RIGHT, DOWN
```

Node 1

```
MOV UP, ACC
SAV
SUB 1
MOV ACC, LEFT
NEG
ADD 10
MOV ACC, DOWN
MOV LEFT, ACC
NEG
ADD ACC
ADD 6
ADD LEFT
MOV ACC, DOWN
SWP
MOV ACC, LEFT
```

Node 2

```
MOV UP, DOWN
MOV UP, ACC
SAV
ADD ACC
ADD ACC
L: JRO DOWN
MOV ACC, DOWN
JMP L
SWP
N: MOV ACC, DOWN
M: JRO DOWN
JMP N
MOV DOWN, DOWN
```

Node 3

```
MOV UP, RIGHT
MOV UP, DOWN
```

Node 4

```
MOV LEFT, DOWN
```

Node 5

```
MOV UP, ACC
SAV
K: MOV 1, UP
SUB UP
JGZ K
MOV 3, UP
F: ADD UP
JGZ X
MOV 1, UP
JMP F
X: MOV 2, UP
MOV ACC, UP
SWP
SUB UP
MOV ACC, DOWN
```

Node 6

```
MOV UP, ACC
SAV
K: MOV 1, RIGHT
SUB RIGHT
JGZ K
MOV 3, RIGHT
F: ADD RIGHT
JGZ X
MOV 1, RIGHT
JMP F
X: MOV 2, RIGHT
MOV ACC, RIGHT
SWP
SUB RIGHT
MOV ACC, DOWN
```

Node 7

```
MOV UP, ACC
SAV
ADD ACC
ADD ACC
L: JRO LEFT
MOV ACC, LEFT
JMP L
SWP
N: MOV ACC, LEFT
M: JRO LEFT
JMP N
MOV LEFT, LEFT
```

Laid out on the puzzle’s real grid, with inputs at the top and outputs at the bottom. Blank tiles are damaged and can’t be used.

Loading puzzle trajectories…

### The search over time

PuzzleMetric Log score axis

Compare models

Inspect time0:00

Best in-game score so far, before independent testing. Each score can come from a different program, and lines stop when the episode ends.

Five models got under the old record in-game. The other two cycle records are ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol’s **3,460** on [Intersection IdentifierSPEC64160260](https://tis-100.wincon.dev/puzzles/intersection-identifier/), against 3,872 (**11% faster**), and ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s **188** on [Error: 8SPEC79324720](https://tis-100.wincon.dev/puzzles/error-8/), against 198 (**5% faster**). All three are on tournament puzzles, none in the main game. Of the 133 new frontier points, **18** beat a community record outright. The other 115 are new tradeoffs: better balances between the three scores than any community solution.

![Opus 5.5's Prime Factor Calculator program running in TIS-100](https://tis-100.wincon.dev/media/programs/aa5d532f8a9a9726.gif)

A new frontier point running in the real game: ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 on [Prime Factor CalculatorNEXUS.20.433.1](https://tis-100.wincon.dev/puzzles/prime-factor-calculator/), at 1,806 cycles, 10 nodes, and 103 instructions.

## What separates the models

### Cost doesn’t track quality

![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol reaches 75.6 record proximity at an estimated **$0.75** per puzzle. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 reaches 77.2 at **$5.08**, **7 times** the cost for 2 more points. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Fable 5.1 is the most expensive at $7.19 and scores 70.2.

**880**episodes

**95.5M**output tokens

**62,400**game-tool calls

Loading analysis…

Compare puzzles
Means include unsuccessful episodes.

### Performance versus resources

One point per model. Upper left is better.

Resource per puzzlePerformance Log resource axis

Select a point for model results. The dashed line joins the nondominated models in this view.

Activity doesn’t explain quality either. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol averages 136 game calls per puzzle at 75.6 record proximity; ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon averages 41 at 76.2.

### How models use the game

Mean game-tool calls per puzzle

EditRunInspectOther

### New points concentrate on hard puzzles

New frontier points appear on **14 hard puzzles**, 4 medium puzzles, and no easy ones. Long-running puzzles are harder to solve but easier to improve: on puzzles with a cycle record of 200 or less, 88% of episodes pass and 4% find a new point. Above 500 cycles, 76% pass and 8% find one.

### Solving versus extending the frontier

Full bar = solve rate · dark segment = new-frontier rate

### Every puzzle by its community records

Position = community record · circle area = fewest nodes

ModelColor byCompare records

0%100%Share of the 16 models that solved each puzzle

Select a dot to inspect its puzzle and community records.

Loading the puzzle map…

### Speed is the hardest score

Every model is closest to the community on nodes and furthest on cycles. Across all 13, mean proximity is 72 for nodes, 56 for instructions, and 45 for cycles.

### Cycles, nodes, and instructions

Mean proximity to each community record · higher is better

### The spread across puzzles

Every dot is an episode · box = middle 50% · line = median · whiskers = minimum and maximum

Measure

Select an episode to inspect its measurements.

### Every model, every puzzle

Choose a cell to inspect the scores in the Pareto explorer.

UnsolvedSolvedNew frontier point

Model outcomes for individual puzzles

Hover or focus a cell for its puzzle, outcome, and record proximity.

## What the models built

We grouped [all 55 puzzles](https://tis-100.wincon.dev/puzzles/) by their main demand, then read the programs. Fifteen of the sixteen models solved all five routing puzzles, and none found a new frontier point there. The examples below show how different designs spend the same resources.

### Performance by puzzle mechanic

Fraction solved · puzzles with new points below

| Model | Routing & synchronization5 puzzles | Arithmetic construction13 puzzles | Sequence algorithms14 puzzles | Streaming state8 puzzles | Memory & interpreters4 puzzles | Image generation11 puzzles |
| --- | --- | --- | --- | --- | --- | --- |
| GPT-6 Astra | **5/5**0 new | **13/13**6 new | **14/14**2 new | **8/8**2 new | **4/4**0 new | **11/11**1 new |
| Opus 5.5 | **5/5**0 new | **13/13**3 new | **14/14**3 new | **8/8**3 new | **4/4**1 new | **11/11**1 new |
| Gemini 4 Argon | **5/5**0 new | **12/13**4 new | **14/14**0 new | **8/8**2 new | **4/4**0 new | **11/11**2 new |
| GPT-6.1 Sol | **5/5**0 new | **12/13**1 new | **14/14**3 new | **8/8**1 new | **4/4**0 new | **11/11**1 new |
| Fable 5.1 | **5/5**0 new | **12/13**2 new | **13/14**1 new | **7/8**1 new | **4/4**1 new | **11/11**0 new |
| Sonnet 5.5 | **5/5**0 new | **12/13**3 new | **13/14**1 new | **8/8**3 new | **4/4**0 new | **11/11**0 new |
| Gemini 3.8 Flash | **5/5**0 new | **12/13**1 new | **13/14**0 new | **8/8**0 new | **4/4**0 new | **11/11**0 new |
| Kimi K3.1 Pro | **5/5**0 new | **12/13**0 new | **13/14**1 new | **7/8**1 new | **4/4**0 new | **11/11**0 new |
| Muse Wonder 1.0 | **5/5**0 new | **12/13**1 new | **10/14**0 new | **8/8**0 new | **4/4**0 new | **10/11**0 new |
| Haiku 5.5 | **5/5**0 new | **11/13**1 new | **11/14**0 new | **6/8**0 new | **4/4**0 new | **8/11**0 new |
| Grok 4.7 | **5/5**0 new | **10/13**0 new | **9/14**0 new | **7/8**0 new | **4/4**0 new | **9/11**0 new |
| DeepSeek V4.1 Flash | **5/5**0 new | **10/13**0 new | **9/14**0 new | **6/8**0 new | **4/4**0 new | **6/11**0 new |
| Muse Spark 1.3 | **5/5**0 new | **11/13**0 new | **9/14**0 new | **4/8**0 new | **2/4**0 new | **8/11**0 new |
| GPT-6 Luna | **5/5**0 new | **11/13**0 new | **10/14**0 new | **4/8**0 new | **2/4**0 new | **7/11**0 new |
| GLM-5.3 | **5/5**0 new | **10/13**1 new | **6/14**0 new | **3/8**0 new | **2/4**0 new | **4/11**0 new |
| MiMo-V2.6-Pro | **3/5**0 new | **4/13**0 new | **1/14**0 new | **1/8**0 new | **1/4**0 new | **0/11**0 new |

### Multiplication: spend code to save cycles

[Signal Multiplier43786](https://tis-100.wincon.dev/puzzles/signal-multiplier/) has no `MUL`. ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon and ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra use `JRO`, a relative jump, to enter a block of repeated instructions at the right position. The jump selects how many copies of a value to send and add, saving the counter updates and branches of a multiplication loop.

![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s fastest program uses two paths to multiply in parallel and a final node to interleave their outputs. It takes **436 cycles** against ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra’s 523, **about 17% faster**, at a cost of two more nodes and 33 more instructions.

### The cost of a faster multiplier

Cycles per run · lower is better

![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon436 cycles

7 nodes · 64 instructions

![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra523 cycles

5 nodes · 31 instructions

![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon436c · 7n · 64i

![argon program running on 43786 in TIS-100](https://tis-100.wincon.dev/media/programs/7e82e2a616a7e042.gif)

![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra523c · 5n · 31i

![astra program running on 43786 in TIS-100](https://tis-100.wincon.dev/media/programs/e9b4021250a39348.gif)

### Window sums: distribute the history

[Signal Window Filter60099](https://tis-100.wincon.dev/puzzles/signal-window-filter/) outputs the sums of the last three and last five inputs, so a solution has to remember history as well as add. ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s four-node program uses `SWP` to rotate stored values and `ADD`/`SUB` to update running state. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol spreads that delay and aggregation across seven nodes, passing intermediate values between them.

![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol’s program runs in **318 cycles** with 26 instructions; ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s takes 428 cycles with **only 17**. The difference is how much of the work runs in parallel, and how much communication and code that costs.

| Model | Cycles | Nodes | Instructions |
| --- | --- | --- | --- |
| GPT-6.1 Sol | 318 | 7 | 26 |
| Gemini 4 Argon | 428 | 4 | 17 |

![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol318c · 7n · 26i

![GPT-6.1 Sol program running on 60099 in TIS-100](https://tis-100.wincon.dev/media/programs/e67bc91dd4a90493.gif)

![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon428c · 4n · 17i

![Gemini 4 Argon program running on 60099 in TIS-100](https://tis-100.wincon.dev/media/programs/28e625251d33e7e0.gif)

### Indexed access from stacks

[T31 Node EmulatorNEXUS.23.727.9](https://tis-100.wincon.dev/puzzles/t31-node-emulator/) needs eight addressable storage slots, but the available memory nodes are stacks. These programs move values between stacks to reach the requested slot, then read or replace it. ![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro uses counted transfer loops; ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 uses `JRO` to pick among unrolled transfers, spending extra instructions to save cycles.

Memory isn’t one skill: ![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro solves this emulator but fails [Sequence Mode CalculatorNEXUS.09.904.9](https://tis-100.wincon.dev/puzzles/sequence-mode-calculator/) and [Signal Window Filter60099](https://tis-100.wincon.dev/puzzles/signal-window-filter/).

| Model | Cycles | Nodes | Instructions |
| --- | --- | --- | --- |
| Opus 5.5 | 215 | 6 | 63 |
| GPT-6 Astra | 365 | 4 | 47 |
| MiMo-V2.6-Pro | 451 | 4 | 34 |

![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5215c · 6n · 63i

![Opus 5.5 program running on NEXUS.23.727.9 in TIS-100](https://tis-100.wincon.dev/media/programs/104b29f695deb96d.gif)

![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra365c · 4n · 47i

![GPT-6 Astra program running on NEXUS.23.727.9 in TIS-100](https://tis-100.wincon.dev/media/programs/e99085f49259ae9c.gif)

![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro451c · 4n · 34i

![MiMo-V2.6-Pro program running on NEXUS.23.727.9 in TIS-100](https://tis-100.wincon.dev/media/programs/5c874909684dd0f4.gif)

Each row is that model’s fastest saved program for the puzzle, picked for illustration. Not all are new frontier points.

## God mode: what if the model could see the target?

The main benchmark hides the community’s scores. In a separate study on six puzzles, five models got:

- **The target:** exact community scores, showing which tradeoffs remain unbeaten.
- **Diagnostics:** cycle-by-cycle traces and profiles of where each node stalls.
- **Edge cases:** a fast simulator that runs 10,000 random tests and returns failures.
- **More time:** 30 minutes, plus extensions for progress, up to 60.

Every model improved: ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra from 82.8 to 91.8, ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol from 82.9 to 90.6, ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 from 81.4 to 89.1, ![](https://tis-100.wincon.dev/assets/logos/models/xai.svg)Grok 4.7 from 67.5 to 75.1, and ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon from 83.5 to 87.9 in mean record proximity. Scores rose in **28 of the 30** model–puzzle pairs.

### Score quality with a richer setup

Mean record proximity · same six puzzles per model · higher is better

Standard episodeGod mode

The diagnostics changed what models did next. On [Signal Multiplexer22280](https://tis-100.wincon.dev/puzzles/signal-multiplexer/), ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol’s first passing program took **294 cycles**. Its profile showed node 2 working 86% of the time while nodes 1 and 3 spent 84% blocked waiting to write to it. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol moved the branching out of node 2 and into those two neighbors. The next run took **204 cycles**, **31% faster**, at a cost of two more nodes and five more instructions.

### The extra time behind those scores

Mean elapsed minutes per puzzle · same episodes

Standard episodeGod mode

Information, tools, and time all changed together, so this study can’t say which addition helped most. These runs are outside the main leaderboard and the headline count.

Puzzles: Signal Multiplexer, Sequence Generator, Signal Edge Detector, Interrupt Handler, Sequence Peak Detector, and Signal Error Corrector.

## Co-discovery

We submitted selected programs to the leaderboard, and the community improved on them almost immediately. On [Undecimal Twin RounderSPEC44149538](https://tis-100.wincon.dev/puzzles/undecimal-twin-rounder/), @LongingForRest took ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5’s 1,918-cycle program and cut it to **1,744** by the next day. The record is now credited to both: `LongingForRest/opus-5-5`.

Neither got there alone. The models found new tradeoffs quickly, and players read those programs, worked out why they worked, and took them further.

Mathematics is seeing the same shift. [Terence Tao and colleagues](https://arxiv.org/abs/2511.02864) used LLM-guided search across 67 problems and improved on the best known results in several. In [“After Math,”](https://terrytao.wordpress.com/2026/09/12/after-math/) Silvia De Toffoli and Eamon Duede argue that a result only becomes a solution once mathematicians can understand it, and that AI fits better as an assistant than a competitor. TIS-100 is a small, measurable version of that future: a model finds a better program, and people turn it into something they understand.

## Pareto explorer

Choose a puzzle and model, then select a point to see its scores and the program that produced it. Every puzzle also has its own page with records and the best programs: [all 55 puzzles](https://tis-100.wincon.dev/puzzles/).

### Pareto explorer

Lower is better ↙[Puzzle page →](https://tis-100.wincon.dev/puzzles/)

Puzzle

Model

Nodes

Loading the frozen benchmark snapshot…

Community leaderboardModelNew frontier point

Select a point with the mouse or keyboard to inspect its scores.

The plot shows cycles and instructions; node counts can differ. New frontier points are judged on all three scores.

[Download all 3,714 model Pareto points (CSV)](https://tis-100.wincon.dev/tis100-bench-points.csv) · [all 3,714 programs (ZIP)](https://tis-100.wincon.dev/tis100-bench-programs.zip), also on [GitHub](https://github.com/wincon-research/tis100-bench) and [Hugging Face](https://huggingface.co/datasets/wincon-research/tis100-bench). 186 rows are marked as new frontier points, which pool to 133 across models.

## Methods

Expand

**Suite.** All 22 campaign puzzles, all 25 TIS-NET puzzles, and eight tournament puzzles from 2018: 19 easy (10 minutes), 17 medium (30), and 19 hard (45). Every model has one scored episode per puzzle. Time limits differ by difficulty, so difficulty and search time can’t be separated.

**Scoring.** Each model–puzzle pair contributes one selected attempt, with no best-of-attempts scoring, so solve rates are pass@1: a puzzle counts as solved when that one episode saved a program that replays in a fresh game. Unsolved puzzles stay in every average and score zero.

**New frontier points.** The reference is the [community leaderboard](https://github.com/12345ieee/tis100-leaderboard/tree/e3114010bac4fd592f718c0a18082e3c7045ca77) as of October 15, 2026. Ties don’t count. Models found 154 candidate points on 23 puzzles. Our count pools and deduplicates them across models and keeps the 133 on 18 puzzles with fresh-game replay and 100,000 passing tests. Testing is strong evidence of correctness, not proof.

**Record proximity.** The community record divided by the model’s best score, averaged across the three scores and all puzzles, ×100. Each score can come from a different program.

**Limits.** One episode per pair compares the configurations as tested and doesn’t measure repeated-run variability. Spread between puzzles is variation across tasks. Public puzzles may overlap with training data. Mechanic groups are our own labels.

**Programs.** Every model Pareto point’s program is public under the MIT License, in TIS-100 save format and named like the community leaderboard’s files (`<puzzle>.<cycles>-<nodes>-<instructions>.txt`). They are mirrored as one file per program on [GitHub](https://github.com/wincon-research/tis100-bench) and as a table on [Hugging Face](https://huggingface.co/datasets/wincon-research/tis100-bench). Two points replay in the game but fail some random tests; they are included and never count as new frontier points.

**Resources.** Costs are estimated from recorded October 2026 API list prices, including cache discounts. Tokens accumulate across responses, so repeated input counts again, and tokenizers differ. Game calls count individual dispatched tools, including errors, and exclude the automatic final submission.

### Settings by model

Run-early guidance adds a prompt note to test drafts early.

| Model | Reasoning effort | Run-early guidance |
| --- | --- | --- |
| Opus 5.5 | high | 14 / 55 |
| GPT-6 Astra | high | 3 / 55 |
| Gemini 4 Argon | high | 10 / 55 |
| Sonnet 5.5 | high | 24 / 55 |
| GPT-6.1 Sol | high | 3 / 55 |
| Fable 5.1 | high | 23 / 55 |
| Kimi K3.1 Pro | provider default | 55 / 55 |
| Gemini 3.8 Flash | high | 0 / 55 |
| Muse Wonder 1.0 | high | 55 / 55 |
| Haiku 5.5 | high | 27 / 55 |
| GLM-5.3 | provider default | 49 / 55 |
| Grok 4.7 | medium: 9 · low: 46 | 25 / 55 |
| DeepSeek V4.1 Flash | provider default | 55 / 55 |
| Muse Spark 1.3 | high | 55 / 55 |
| GPT-6 Luna | high | 2 / 55 |
| MiMo-V2.6-Pro | provider default | 53 / 55 |

![](https://tis-100.wincon.dev/assets/logos/models/xai.svg)Grok 4.7 episode counts are shown by effort. A sandboxed scripting tool for game calls was available in 46–54 of each model’s 55 episodes. A note about the game’s random test reached 841 of 880 episodes. 6 ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 episodes and 5 ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Sonnet 5.5 episodes ran in a different agent client at the same effort.

## Related work

[ISA-Bench](https://arxiv.org/abs/2609.22878) studies reasoning with constrained instruction sets, including TIS-100. [Opus Magnum Bench](https://opusmagnumbench.com/) tests agents’ ability to build and optimize machines in another Zachtronics game. We focus on optimization across all TIS-100 puzzles and eight community puzzles, keeping tradeoffs across cycles, nodes, and instructions. The tasks and protocols differ, so scores aren’t directly comparable.

## Cite

```
@misc{wincon2026tis100bench,
  title  = {TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games},
  author = {{Wincon}},
  year   = {2026},
  month  = nov,
  url    = {https://tis-100.wincon.dev/},
  note   = {Benchmark snapshot 2026-11-03-v5}
}
```

## Credits

Thanks to Zachtronics, the TIS-100 players, and the maintainers of the [community leaderboard](https://github.com/12345ieee/tis100-leaderboard), [TIS-100-CXX](https://github.com/killerbee13/TIS-100-CXX), and [leaderboard bot](https://github.com/F43nd1r/zachtronics-leaderboard-bot). Their solutions and tools make this comparison possible. Thanks also to @12345ieee, @LongingForRest, @Hersmunch, @easone, @18111398, and everyone on the [Zachtronics Discord](https://discord.gg/98QNzdJ "The unofficial, community-run Zachtronics Discord") who discussed the submissions and kept improving them.

TIS-100, SpaceChem, and Opus Magnum are trademarks of Zachtronics. Model names and logos are trademarks of their respective owners and appear only to identify the models tested. Wincon Research is not affiliated with or endorsed by Zachtronics or any model provider.
