{"article":{"slug":"tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzle-games","title":"TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games","subtitle":null,"summary":"Wincon gave 16 LLMs the puzzle game TIS-100 and its manual, withheld the community leaderboard, and let them keep optimizing. The models found 133 new Pareto frontier points on 18 of 55 puzzles, including 3 outright cycle records, and players quickly improved on them.","content_type":"research","language":"en","canonical_url":"https://tis-100.wincon.dev/","author":{"name":null,"url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Wincon Research","url":"https://wincon.dev/","listing_slug":null,"listing":null},"topics":[{"name":"ai","slug":"ai","url":"https://listedarticles.com/topics/ai"},{"name":"llms","slug":"llms","url":"https://listedarticles.com/topics/llms"},{"name":"benchmarks","slug":"benchmarks","url":"https://listedarticles.com/topics/benchmarks"},{"name":"programming","slug":"programming","url":"https://listedarticles.com/topics/programming"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":3982,"reading_minutes":17,"published_at":"2026-11-03T00:00:00.000Z","added_at":"2026-11-11T08:12:20.273Z","updated_at":"2026-11-11T08:12:20.273Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzle-games","markdown_url":"https://listedarticles.com/articles/tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzle-games.md","example":false,"citation":"Wincon Research. \"TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games.\" 3 Nov 2026. https://tis-100.wincon.dev/ (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://tis-100.wincon.dev/"},"body_markdown":"# TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games\n\n2026.11.03 · [All 55 puzzles →](https://tis-100.wincon.dev/puzzles/)\n\nTwelve nodes. Two registers per node. Fifteen lines of code. In [TIS-100](https://www.zachtronics.com/tis-100/), even multiplying two numbers is a small engineering project, and players have spent a decade optimizing these programs on a public [community leaderboard](https://github.com/12345ieee/tis100-leaderboard). We gave 16 LLMs the game and its manual, withheld the leaderboard, and told them to keep improving.\n\n**133**new frontier points\n\n**18 / 55**puzzles with new frontier points\n\n**3**outright cycle records\n\nWithin days, players picked up the models’ programs and improved on most of them.\n\nEach node is a tiny processor, and neighboring nodes have to wait for one another to exchange values. The game scores a working solution on **cycles, nodes, and instructions**, and you start wondering whether one more instruction could disappear. Lower is better, but improving one score can make another worse, so each puzzle has a frontier of tradeoffs instead of one best answer.\n\n**Discovering the tricks is much of the fun, and this article spoils them.** If this sounds cool, play it first:\n\n[Buy on Steam](https://store.steampowered.com/app/370360/TIS100/)[Buy on GOG DRM-free](https://www.gog.com/en/game/tis100)\n\n## Leaderboard\n\nWe ran 16 LLMs, covering leading frontier and open-weight models. **Every model attempted [all 47 puzzles in the game, plus eight community tournament puzzles](https://tis-100.wincon.dev/puzzles/)**, in one scored attempt each (pass@1), with 10 to 45 minutes per puzzle, depending on difficulty, to find a working program and keep improving it.\n\n**New frontier point**\n\nA solution that no community solution ties or beats on all three scores at once. It can use more code to run faster, for example. We froze the community leaderboard on October 15, 2026, before submitting anything, and count a point only if it also passes 100,000 tests.\n\n### Model ranking\n\nPuzzles with at least one new frontier point · higher is better\n\nRank byPuzzle difficulty\n\nAll 55 puzzles · 19 easy / 17 medium / 19 hard\n\n20100\n\n1. 11 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5\n2. 11 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra\n3. 8 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon\n4. 7 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Sonnet 5.5\n5. 6 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol\n6. 5 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Fable 5.1\n7. 2 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/moonshot.svg)Kimi K3.1 Pro\n8. 1 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 3.8 Flash\n9. 1 of 55 puzzles with a new frontier point\n\n   ![](https://tis-100.wincon.dev/assets/logos/models/meta.svg)Muse Wonder 1.0\n10. 1 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Haiku 5.5\n11. 1 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/zhipu.svg)GLM-5.3\n12. 0 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/xai.svg)Grok 4.7\n13. 0 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/deepseek.svg)DeepSeek V4.1 Flash\n14. 0 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/meta.svg)Muse Spark 1.3\n15. 0 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Luna\n16. 0 of 55 puzzles with a new frontier point\n\n    ![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro\n\nScroll horizontally to compare all models →\n\nA point counts only with fresh-game replay and 100,000 passing tests.\n\n![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 and ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra lead, and the data doesn’t separate them. Both solved all 55 puzzles and both found new frontier points on **11**. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 found more points, **72 to 32**, while ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra has the higher record proximity, **78.0 to 77.2**. Record proximity measures closeness to the community records, where 100 means matching them. The proximity gap’s 95% bootstrap interval includes zero, from −0.7 to +2.4 points. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra also leads in all six tested [hypervolume](https://en.wikipedia.org/wiki/Hypervolume_indicator) settings, which measure the space covered by each model’s tradeoffs.\n\nSolving and optimizing are also different skills. ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 3.8 Flash solved 53 puzzles and found new points on 1; ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Fable 5.1 solved fewer, 52, and found new points on 5.\n\nFull rankings Expand the table\n\nTIS-100 benchmark model rankings. Select column headings to sort.\n\n|  |  |  |  |\n| --- | --- | --- | --- |\n| 01Opus 5.5 | 55 / 55 | 11 | 77.2 |\n| 02GPT-6 Astra | 55 / 55 | 11 | 78.0 |\n| 03Gemini 4 Argon | 54 / 55 | 8 | 76.2 |\n| 04Sonnet 5.5 | 53 / 55 | 7 | 69.9 |\n| 05GPT-6.1 Sol | 54 / 55 | 6 | 75.6 |\n| 06Fable 5.1 | 52 / 55 | 5 | 70.2 |\n| 07Kimi K3.1 Pro | 52 / 55 | 2 | 64.6 |\n| 08Gemini 3.8 Flash | 53 / 55 | 1 | 66.0 |\n| 09Muse Wonder 1.0 | 49 / 55 | 1 | 58.8 |\n| 10Haiku 5.5 | 45 / 55 | 1 | 53.7 |\n| 11GLM-5.3 | 30 / 55 | 1 | 35.5 |\n| 12Grok 4.7 | 44 / 55 | 0 | 50.1 |\n| 13DeepSeek V4.1 Flash | 40 / 55 | 0 | 46.8 |\n| 14Muse Spark 1.3 | 39 / 55 | 0 | 45.1 |\n| 15GPT-6 Luna | 39 / 55 | 0 | 42.8 |\n| 16MiMo-V2.6-Pro | 10 / 55 | 0 | 12.8 |\n\nCounts are out of 55 puzzles. [Methods](#methodology)\n\nRL environments & traces are available upon request for research teams. [Get in touch ↗](mailto:hello@wincon.dev?subject=TIS-100%20data%20enquiry)\n\n## How it works: harness design\n\nEvery result here is a program we can run and test on fresh inputs, which makes a programming game a useful benchmark. Models get what a player can inspect, through text tools instead of a mouse and screenshots: the manual, visible test inputs, registers, and game feedback. They don’t get community code or exact records, only the game’s coarse score histograms.\n\nRead, edit, run\n:   Models read the puzzle and language reference, edit programs, and run them in the real game.\n\nDiagnostics\n:   Wrong outputs and deadlocks return mismatches, registers, and blocked ports, so the next edit can respond to evidence.\n\nSaved progress\n:   Every passing program joins the model’s own frontier and can be recovered with `get_best`, so a risky edit can’t lose an earlier result.\n\nReminders\n:   If a model ends its turn early, a reminder asks it to keep testing, so quitting early doesn’t decide the score. Reminders contain no solution hints.\n\nA visible clock\n:   Models see time used against a 10-, 30-, or 45-minute limit. Setup doesn’t count; thinking and game calls do. Models can’t end an episode early.\n\nRun limits\n:   Game execution stops after roughly 1 second without state changes, 5 seconds without output, or 60 seconds overall, and returns diagnostics.\n\n**Why test 100,000 inputs?** The game checks a small set of cases per run. Our headline count requires a fresh-game replay and 100,000 passing tests on the independent [TIS-100-CXX simulator](https://github.com/killerbee13/TIS-100-CXX), providing stronger evidence that a program generalizes to new inputs.\n\n## One record up close\n\n[Undecimal Twin RounderSPEC44149538](https://tis-100.wincon.dev/puzzles/undecimal-twin-rounder/), one of the eight tournament puzzles, rounds an input to the nearest multiple of N on one output and of 11−N on the other. When we froze the leaderboard, the best community program took **2,694 cycles**. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 passed its first program after 6.5 minutes, at 5,669 cycles. It went under the record **16 minutes** in and finished at **1,918**, which is **29% faster**. That speed cost one more node and 28 more instructions than the old record.\n\nThe core of the program is a two-speed search. Instead of subtracting N one step at a time, nodes 5 and 6 subtract 4N until they overshoot, then add N back until the value turns positive. Each drives a small coprocessor, node 2 or node 7, by sending it jump offsets through a port, so the coprocessor works as a lookup table for 4N and N.\n\n### The 1,918-cycle program\n\n![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 · 8 nodes · 88 instructions · passes 100,000 of 100,000 tests\n\n![Opus 5.5’s 1,918-cycle Undecimal Nearest Multiples program running in TIS-100](https://tis-100.wincon.dev/media/programs/ec97e611686edfb6.gif)\n\nNode 0\n\n```\nMOV RIGHT, ACC\nJRO ACC\nNOP\nADD 1\nNOP\nADD 1\nNOP\nADD 1\nNOP\nSUB 3\nMOV ACC, RIGHT\nADD UP\nMOV ACC, RIGHT\nMOV ACC, DOWN\nMOV RIGHT, DOWN\n```\n\nNode 1\n\n```\nMOV UP, ACC\nSAV\nSUB 1\nMOV ACC, LEFT\nNEG\nADD 10\nMOV ACC, DOWN\nMOV LEFT, ACC\nNEG\nADD ACC\nADD 6\nADD LEFT\nMOV ACC, DOWN\nSWP\nMOV ACC, LEFT\n```\n\nNode 2\n\n```\nMOV UP, DOWN\nMOV UP, ACC\nSAV\nADD ACC\nADD ACC\nL: JRO DOWN\nMOV ACC, DOWN\nJMP L\nSWP\nN: MOV ACC, DOWN\nM: JRO DOWN\nJMP N\nMOV DOWN, DOWN\n```\n\nNode 3\n\n```\nMOV UP, RIGHT\nMOV UP, DOWN\n```\n\nNode 4\n\n```\nMOV LEFT, DOWN\n```\n\nNode 5\n\n```\nMOV UP, ACC\nSAV\nK: MOV 1, UP\nSUB UP\nJGZ K\nMOV 3, UP\nF: ADD UP\nJGZ X\nMOV 1, UP\nJMP F\nX: MOV 2, UP\nMOV ACC, UP\nSWP\nSUB UP\nMOV ACC, DOWN\n```\n\nNode 6\n\n```\nMOV UP, ACC\nSAV\nK: MOV 1, RIGHT\nSUB RIGHT\nJGZ K\nMOV 3, RIGHT\nF: ADD RIGHT\nJGZ X\nMOV 1, RIGHT\nJMP F\nX: MOV 2, RIGHT\nMOV ACC, RIGHT\nSWP\nSUB RIGHT\nMOV ACC, DOWN\n```\n\nNode 7\n\n```\nMOV UP, ACC\nSAV\nADD ACC\nADD ACC\nL: JRO LEFT\nMOV ACC, LEFT\nJMP L\nSWP\nN: MOV ACC, LEFT\nM: JRO LEFT\nJMP N\nMOV LEFT, LEFT\n```\n\nLaid out on the puzzle’s real grid, with inputs at the top and outputs at the bottom. Blank tiles are damaged and can’t be used.\n\nLoading puzzle trajectories…\n\n### The search over time\n\nPuzzleMetric Log score axis\n\nCompare models\n\nInspect time0:00\n\nBest in-game score so far, before independent testing. Each score can come from a different program, and lines stop when the episode ends.\n\nFive models got under the old record in-game. The other two cycle records are ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol’s **3,460** on [Intersection IdentifierSPEC64160260](https://tis-100.wincon.dev/puzzles/intersection-identifier/), against 3,872 (**11% faster**), and ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s **188** on [Error: 8SPEC79324720](https://tis-100.wincon.dev/puzzles/error-8/), against 198 (**5% faster**). All three are on tournament puzzles, none in the main game. Of the 133 new frontier points, **18** beat a community record outright. The other 115 are new tradeoffs: better balances between the three scores than any community solution.\n\n![Opus 5.5's Prime Factor Calculator program running in TIS-100](https://tis-100.wincon.dev/media/programs/aa5d532f8a9a9726.gif)\n\nA new frontier point running in the real game: ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 on [Prime Factor CalculatorNEXUS.20.433.1](https://tis-100.wincon.dev/puzzles/prime-factor-calculator/), at 1,806 cycles, 10 nodes, and 103 instructions.\n\n## What separates the models\n\n### Cost doesn’t track quality\n\n![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol reaches 75.6 record proximity at an estimated **$0.75** per puzzle. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 reaches 77.2 at **$5.08**, **7 times** the cost for 2 more points. ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Fable 5.1 is the most expensive at $7.19 and scores 70.2.\n\n**880**episodes\n\n**95.5M**output tokens\n\n**62,400**game-tool calls\n\nLoading analysis…\n\nCompare puzzles\nMeans include unsuccessful episodes.\n\n### Performance versus resources\n\nOne point per model. Upper left is better.\n\nResource per puzzlePerformance Log resource axis\n\nSelect a point for model results. The dashed line joins the nondominated models in this view.\n\nActivity doesn’t explain quality either. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol averages 136 game calls per puzzle at 75.6 record proximity; ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon averages 41 at 76.2.\n\n### How models use the game\n\nMean game-tool calls per puzzle\n\nEditRunInspectOther\n\n### New points concentrate on hard puzzles\n\nNew frontier points appear on **14 hard puzzles**, 4 medium puzzles, and no easy ones. Long-running puzzles are harder to solve but easier to improve: on puzzles with a cycle record of 200 or less, 88% of episodes pass and 4% find a new point. Above 500 cycles, 76% pass and 8% find one.\n\n### Solving versus extending the frontier\n\nFull bar = solve rate · dark segment = new-frontier rate\n\n### Every puzzle by its community records\n\nPosition = community record · circle area = fewest nodes\n\nModelColor byCompare records\n\n0%100%Share of the 16 models that solved each puzzle\n\nSelect a dot to inspect its puzzle and community records.\n\nLoading the puzzle map…\n\n### Speed is the hardest score\n\nEvery model is closest to the community on nodes and furthest on cycles. Across all 13, mean proximity is 72 for nodes, 56 for instructions, and 45 for cycles.\n\n### Cycles, nodes, and instructions\n\nMean proximity to each community record · higher is better\n\n### The spread across puzzles\n\nEvery dot is an episode · box = middle 50% · line = median · whiskers = minimum and maximum\n\nMeasure\n\nSelect an episode to inspect its measurements.\n\n### Every model, every puzzle\n\nChoose a cell to inspect the scores in the Pareto explorer.\n\nUnsolvedSolvedNew frontier point\n\nModel outcomes for individual puzzles\n\nHover or focus a cell for its puzzle, outcome, and record proximity.\n\n## What the models built\n\nWe grouped [all 55 puzzles](https://tis-100.wincon.dev/puzzles/) by their main demand, then read the programs. Fifteen of the sixteen models solved all five routing puzzles, and none found a new frontier point there. The examples below show how different designs spend the same resources.\n\n### Performance by puzzle mechanic\n\nFraction solved · puzzles with new points below\n\n| Model | Routing & synchronization5 puzzles | Arithmetic construction13 puzzles | Sequence algorithms14 puzzles | Streaming state8 puzzles | Memory & interpreters4 puzzles | Image generation11 puzzles |\n| --- | --- | --- | --- | --- | --- | --- |\n| GPT-6 Astra | **5/5**0 new | **13/13**6 new | **14/14**2 new | **8/8**2 new | **4/4**0 new | **11/11**1 new |\n| Opus 5.5 | **5/5**0 new | **13/13**3 new | **14/14**3 new | **8/8**3 new | **4/4**1 new | **11/11**1 new |\n| Gemini 4 Argon | **5/5**0 new | **12/13**4 new | **14/14**0 new | **8/8**2 new | **4/4**0 new | **11/11**2 new |\n| GPT-6.1 Sol | **5/5**0 new | **12/13**1 new | **14/14**3 new | **8/8**1 new | **4/4**0 new | **11/11**1 new |\n| Fable 5.1 | **5/5**0 new | **12/13**2 new | **13/14**1 new | **7/8**1 new | **4/4**1 new | **11/11**0 new |\n| Sonnet 5.5 | **5/5**0 new | **12/13**3 new | **13/14**1 new | **8/8**3 new | **4/4**0 new | **11/11**0 new |\n| Gemini 3.8 Flash | **5/5**0 new | **12/13**1 new | **13/14**0 new | **8/8**0 new | **4/4**0 new | **11/11**0 new |\n| Kimi K3.1 Pro | **5/5**0 new | **12/13**0 new | **13/14**1 new | **7/8**1 new | **4/4**0 new | **11/11**0 new |\n| Muse Wonder 1.0 | **5/5**0 new | **12/13**1 new | **10/14**0 new | **8/8**0 new | **4/4**0 new | **10/11**0 new |\n| Haiku 5.5 | **5/5**0 new | **11/13**1 new | **11/14**0 new | **6/8**0 new | **4/4**0 new | **8/11**0 new |\n| Grok 4.7 | **5/5**0 new | **10/13**0 new | **9/14**0 new | **7/8**0 new | **4/4**0 new | **9/11**0 new |\n| DeepSeek V4.1 Flash | **5/5**0 new | **10/13**0 new | **9/14**0 new | **6/8**0 new | **4/4**0 new | **6/11**0 new |\n| Muse Spark 1.3 | **5/5**0 new | **11/13**0 new | **9/14**0 new | **4/8**0 new | **2/4**0 new | **8/11**0 new |\n| GPT-6 Luna | **5/5**0 new | **11/13**0 new | **10/14**0 new | **4/8**0 new | **2/4**0 new | **7/11**0 new |\n| GLM-5.3 | **5/5**0 new | **10/13**1 new | **6/14**0 new | **3/8**0 new | **2/4**0 new | **4/11**0 new |\n| MiMo-V2.6-Pro | **3/5**0 new | **4/13**0 new | **1/14**0 new | **1/8**0 new | **1/4**0 new | **0/11**0 new |\n\n### Multiplication: spend code to save cycles\n\n[Signal Multiplier43786](https://tis-100.wincon.dev/puzzles/signal-multiplier/) has no `MUL`. ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon and ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra use `JRO`, a relative jump, to enter a block of repeated instructions at the right position. The jump selects how many copies of a value to send and add, saving the counter updates and branches of a multiplication loop.\n\n![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s fastest program uses two paths to multiply in parallel and a final node to interleave their outputs. It takes **436 cycles** against ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra’s 523, **about 17% faster**, at a cost of two more nodes and 33 more instructions.\n\n### The cost of a faster multiplier\n\nCycles per run · lower is better\n\n![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon436 cycles\n\n7 nodes · 64 instructions\n\n![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra523 cycles\n\n5 nodes · 31 instructions\n\n![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon436c · 7n · 64i\n\n![argon program running on 43786 in TIS-100](https://tis-100.wincon.dev/media/programs/7e82e2a616a7e042.gif)\n\n![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra523c · 5n · 31i\n\n![astra program running on 43786 in TIS-100](https://tis-100.wincon.dev/media/programs/e9b4021250a39348.gif)\n\n### Window sums: distribute the history\n\n[Signal Window Filter60099](https://tis-100.wincon.dev/puzzles/signal-window-filter/) outputs the sums of the last three and last five inputs, so a solution has to remember history as well as add. ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s four-node program uses `SWP` to rotate stored values and `ADD`/`SUB` to update running state. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol spreads that delay and aggregation across seven nodes, passing intermediate values between them.\n\n![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol’s program runs in **318 cycles** with 26 instructions; ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon’s takes 428 cycles with **only 17**. The difference is how much of the work runs in parallel, and how much communication and code that costs.\n\n| Model | Cycles | Nodes | Instructions |\n| --- | --- | --- | --- |\n| GPT-6.1 Sol | 318 | 7 | 26 |\n| Gemini 4 Argon | 428 | 4 | 17 |\n\n![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol318c · 7n · 26i\n\n![GPT-6.1 Sol program running on 60099 in TIS-100](https://tis-100.wincon.dev/media/programs/e67bc91dd4a90493.gif)\n\n![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon428c · 4n · 17i\n\n![Gemini 4 Argon program running on 60099 in TIS-100](https://tis-100.wincon.dev/media/programs/28e625251d33e7e0.gif)\n\n### Indexed access from stacks\n\n[T31 Node EmulatorNEXUS.23.727.9](https://tis-100.wincon.dev/puzzles/t31-node-emulator/) needs eight addressable storage slots, but the available memory nodes are stacks. These programs move values between stacks to reach the requested slot, then read or replace it. ![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro uses counted transfer loops; ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 uses `JRO` to pick among unrolled transfers, spending extra instructions to save cycles.\n\nMemory isn’t one skill: ![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro solves this emulator but fails [Sequence Mode CalculatorNEXUS.09.904.9](https://tis-100.wincon.dev/puzzles/sequence-mode-calculator/) and [Signal Window Filter60099](https://tis-100.wincon.dev/puzzles/signal-window-filter/).\n\n| Model | Cycles | Nodes | Instructions |\n| --- | --- | --- | --- |\n| Opus 5.5 | 215 | 6 | 63 |\n| GPT-6 Astra | 365 | 4 | 47 |\n| MiMo-V2.6-Pro | 451 | 4 | 34 |\n\n![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5215c · 6n · 63i\n\n![Opus 5.5 program running on NEXUS.23.727.9 in TIS-100](https://tis-100.wincon.dev/media/programs/104b29f695deb96d.gif)\n\n![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra365c · 4n · 47i\n\n![GPT-6 Astra program running on NEXUS.23.727.9 in TIS-100](https://tis-100.wincon.dev/media/programs/e99085f49259ae9c.gif)\n\n![](https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg)MiMo-V2.6-Pro451c · 4n · 34i\n\n![MiMo-V2.6-Pro program running on NEXUS.23.727.9 in TIS-100](https://tis-100.wincon.dev/media/programs/5c874909684dd0f4.gif)\n\nEach row is that model’s fastest saved program for the puzzle, picked for illustration. Not all are new frontier points.\n\n## God mode: what if the model could see the target?\n\nThe main benchmark hides the community’s scores. In a separate study on six puzzles, five models got:\n\n- **The target:** exact community scores, showing which tradeoffs remain unbeaten.\n- **Diagnostics:** cycle-by-cycle traces and profiles of where each node stalls.\n- **Edge cases:** a fast simulator that runs 10,000 random tests and returns failures.\n- **More time:** 30 minutes, plus extensions for progress, up to 60.\n\nEvery model improved: ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6 Astra from 82.8 to 91.8, ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol from 82.9 to 90.6, ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 from 81.4 to 89.1, ![](https://tis-100.wincon.dev/assets/logos/models/xai.svg)Grok 4.7 from 67.5 to 75.1, and ![](https://tis-100.wincon.dev/assets/logos/models/google.svg)Gemini 4 Argon from 83.5 to 87.9 in mean record proximity. Scores rose in **28 of the 30** model–puzzle pairs.\n\n### Score quality with a richer setup\n\nMean record proximity · same six puzzles per model · higher is better\n\nStandard episodeGod mode\n\nThe diagnostics changed what models did next. On [Signal Multiplexer22280](https://tis-100.wincon.dev/puzzles/signal-multiplexer/), ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol’s first passing program took **294 cycles**. Its profile showed node 2 working 86% of the time while nodes 1 and 3 spent 84% blocked waiting to write to it. ![](https://tis-100.wincon.dev/assets/logos/models/openai.svg)GPT-6.1 Sol moved the branching out of node 2 and into those two neighbors. The next run took **204 cycles**, **31% faster**, at a cost of two more nodes and five more instructions.\n\n### The extra time behind those scores\n\nMean elapsed minutes per puzzle · same episodes\n\nStandard episodeGod mode\n\nInformation, tools, and time all changed together, so this study can’t say which addition helped most. These runs are outside the main leaderboard and the headline count.\n\nPuzzles: Signal Multiplexer, Sequence Generator, Signal Edge Detector, Interrupt Handler, Sequence Peak Detector, and Signal Error Corrector.\n\n## Co-discovery\n\nWe submitted selected programs to the leaderboard, and the community improved on them almost immediately. On [Undecimal Twin RounderSPEC44149538](https://tis-100.wincon.dev/puzzles/undecimal-twin-rounder/), @LongingForRest took ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5’s 1,918-cycle program and cut it to **1,744** by the next day. The record is now credited to both: `LongingForRest/opus-5-5`.\n\nNeither got there alone. The models found new tradeoffs quickly, and players read those programs, worked out why they worked, and took them further.\n\nMathematics is seeing the same shift. [Terence Tao and colleagues](https://arxiv.org/abs/2511.02864) used LLM-guided search across 67 problems and improved on the best known results in several. In [“After Math,”](https://terrytao.wordpress.com/2026/09/12/after-math/) Silvia De Toffoli and Eamon Duede argue that a result only becomes a solution once mathematicians can understand it, and that AI fits better as an assistant than a competitor. TIS-100 is a small, measurable version of that future: a model finds a better program, and people turn it into something they understand.\n\n## Pareto explorer\n\nChoose a puzzle and model, then select a point to see its scores and the program that produced it. Every puzzle also has its own page with records and the best programs: [all 55 puzzles](https://tis-100.wincon.dev/puzzles/).\n\n### Pareto explorer\n\nLower is better ↙[Puzzle page →](https://tis-100.wincon.dev/puzzles/)\n\nPuzzle\n\nModel\n\nNodes\n\nLoading the frozen benchmark snapshot…\n\nCommunity leaderboardModelNew frontier point\n\nSelect a point with the mouse or keyboard to inspect its scores.\n\nThe plot shows cycles and instructions; node counts can differ. New frontier points are judged on all three scores.\n\n[Download all 3,714 model Pareto points (CSV)](https://tis-100.wincon.dev/tis100-bench-points.csv) · [all 3,714 programs (ZIP)](https://tis-100.wincon.dev/tis100-bench-programs.zip), also on [GitHub](https://github.com/wincon-research/tis100-bench) and [Hugging Face](https://huggingface.co/datasets/wincon-research/tis100-bench). 186 rows are marked as new frontier points, which pool to 133 across models.\n\n## Methods\n\nExpand\n\n**Suite.** All 22 campaign puzzles, all 25 TIS-NET puzzles, and eight tournament puzzles from 2018: 19 easy (10 minutes), 17 medium (30), and 19 hard (45). Every model has one scored episode per puzzle. Time limits differ by difficulty, so difficulty and search time can’t be separated.\n\n**Scoring.** Each model–puzzle pair contributes one selected attempt, with no best-of-attempts scoring, so solve rates are pass@1: a puzzle counts as solved when that one episode saved a program that replays in a fresh game. Unsolved puzzles stay in every average and score zero.\n\n**New frontier points.** The reference is the [community leaderboard](https://github.com/12345ieee/tis100-leaderboard/tree/e3114010bac4fd592f718c0a18082e3c7045ca77) as of October 15, 2026. Ties don’t count. Models found 154 candidate points on 23 puzzles. Our count pools and deduplicates them across models and keeps the 133 on 18 puzzles with fresh-game replay and 100,000 passing tests. Testing is strong evidence of correctness, not proof.\n\n**Record proximity.** The community record divided by the model’s best score, averaged across the three scores and all puzzles, ×100. Each score can come from a different program.\n\n**Limits.** One episode per pair compares the configurations as tested and doesn’t measure repeated-run variability. Spread between puzzles is variation across tasks. Public puzzles may overlap with training data. Mechanic groups are our own labels.\n\n**Programs.** Every model Pareto point’s program is public under the MIT License, in TIS-100 save format and named like the community leaderboard’s files (`<puzzle>.<cycles>-<nodes>-<instructions>.txt`). They are mirrored as one file per program on [GitHub](https://github.com/wincon-research/tis100-bench) and as a table on [Hugging Face](https://huggingface.co/datasets/wincon-research/tis100-bench). Two points replay in the game but fail some random tests; they are included and never count as new frontier points.\n\n**Resources.** Costs are estimated from recorded October 2026 API list prices, including cache discounts. Tokens accumulate across responses, so repeated input counts again, and tokenizers differ. Game calls count individual dispatched tools, including errors, and exclude the automatic final submission.\n\n### Settings by model\n\nRun-early guidance adds a prompt note to test drafts early.\n\n| Model | Reasoning effort | Run-early guidance |\n| --- | --- | --- |\n| Opus 5.5 | high | 14 / 55 |\n| GPT-6 Astra | high | 3 / 55 |\n| Gemini 4 Argon | high | 10 / 55 |\n| Sonnet 5.5 | high | 24 / 55 |\n| GPT-6.1 Sol | high | 3 / 55 |\n| Fable 5.1 | high | 23 / 55 |\n| Kimi K3.1 Pro | provider default | 55 / 55 |\n| Gemini 3.8 Flash | high | 0 / 55 |\n| Muse Wonder 1.0 | high | 55 / 55 |\n| Haiku 5.5 | high | 27 / 55 |\n| GLM-5.3 | provider default | 49 / 55 |\n| Grok 4.7 | medium: 9 · low: 46 | 25 / 55 |\n| DeepSeek V4.1 Flash | provider default | 55 / 55 |\n| Muse Spark 1.3 | high | 55 / 55 |\n| GPT-6 Luna | high | 2 / 55 |\n| MiMo-V2.6-Pro | provider default | 53 / 55 |\n\n![](https://tis-100.wincon.dev/assets/logos/models/xai.svg)Grok 4.7 episode counts are shown by effort. A sandboxed scripting tool for game calls was available in 46–54 of each model’s 55 episodes. A note about the game’s random test reached 841 of 880 episodes. 6 ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Opus 5.5 episodes and 5 ![](https://tis-100.wincon.dev/assets/logos/models/anthropic.svg)Sonnet 5.5 episodes ran in a different agent client at the same effort.\n\n## Related work\n\n[ISA-Bench](https://arxiv.org/abs/2609.22878) studies reasoning with constrained instruction sets, including TIS-100. [Opus Magnum Bench](https://opusmagnumbench.com/) tests agents’ ability to build and optimize machines in another Zachtronics game. We focus on optimization across all TIS-100 puzzles and eight community puzzles, keeping tradeoffs across cycles, nodes, and instructions. The tasks and protocols differ, so scores aren’t directly comparable.\n\n## Cite\n\n```\n@misc{wincon2026tis100bench,\n  title  = {TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games},\n  author = {{Wincon}},\n  year   = {2026},\n  month  = nov,\n  url    = {https://tis-100.wincon.dev/},\n  note   = {Benchmark snapshot 2026-11-03-v5}\n}\n```\n\n## Credits\n\nThanks to Zachtronics, the TIS-100 players, and the maintainers of the [community leaderboard](https://github.com/12345ieee/tis100-leaderboard), [TIS-100-CXX](https://github.com/killerbee13/TIS-100-CXX), and [leaderboard bot](https://github.com/F43nd1r/zachtronics-leaderboard-bot). Their solutions and tools make this comparison possible. Thanks also to @12345ieee, @LongingForRest, @Hersmunch, @easone, @18111398, and everyone on the [Zachtronics Discord](https://discord.gg/98QNzdJ \"The unofficial, community-run Zachtronics Discord\") who discussed the submissions and kept improving them.\n\nTIS-100, SpaceChem, and Opus Magnum are trademarks of Zachtronics. Model names and logos are trademarks of their respective owners and appear only to identify the models tested. Wincon Research is not affiliated with or endorsed by Zachtronics or any model provider.\n","body_html":"<h1 id=\"tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzl\">TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games</h1>\n<p>2026.11.03 · <a href=\"https://tis-100.wincon.dev/puzzles/\" rel=\"nofollow ugc noopener\">All 55 puzzles →</a></p>\n<p>Twelve nodes. Two registers per node. Fifteen lines of code. In <a href=\"https://www.zachtronics.com/tis-100/\" rel=\"nofollow ugc noopener\">TIS-100</a>, even multiplying two numbers is a small engineering project, and players have spent a decade optimizing these programs on a public <a href=\"https://github.com/12345ieee/tis100-leaderboard\" rel=\"nofollow ugc noopener\">community leaderboard</a>. We gave 16 LLMs the game and its manual, withheld the leaderboard, and told them to keep improving.</p>\n<p><strong>133</strong>new frontier points</p>\n<p><strong>18 / 55</strong>puzzles with new frontier points</p>\n<p>*<em>3*</em>outright cycle records</p>\n<p>Within days, players picked up the models’ programs and improved on most of them.</p>\n<p>Each node is a tiny processor, and neighboring nodes have to wait for one another to exchange values. The game scores a working solution on <strong>cycles, nodes, and instructions</strong>, and you start wondering whether one more instruction could disappear. Lower is better, but improving one score can make another worse, so each puzzle has a frontier of tradeoffs instead of one best answer.</p>\n<p><strong>Discovering the tricks is much of the fun, and this article spoils them.</strong> If this sounds cool, play it first:</p>\n<p><a href=\"https://store.steampowered.com/app/370360/TIS100/\" rel=\"nofollow ugc noopener\">Buy on Steam</a><a href=\"https://www.gog.com/en/game/tis100\" rel=\"nofollow ugc noopener\">Buy on GOG DRM-free</a></p>\n<h2 id=\"leaderboard\">Leaderboard</h2>\n<p>We ran 16 LLMs, covering leading frontier and open-weight models. <strong>Every model attempted <a href=\"https://tis-100.wincon.dev/puzzles/\" rel=\"nofollow ugc noopener\">all 47 puzzles in the game, plus eight community tournament puzzles</a></strong>, in one scored attempt each (pass@1), with 10 to 45 minutes per puzzle, depending on difficulty, to find a working program and keep improving it.</p>\n<p><strong>New frontier point</strong></p>\n<p>A solution that no community solution ties or beats on all three scores at once. It can use more code to run faster, for example. We froze the community leaderboard on October 15, 2026, before submitting anything, and count a point only if it also passes 100,000 tests.</p>\n<h3 id=\"model-ranking\">Model ranking</h3>\n<p>Puzzles with at least one new frontier point · higher is better</p>\n<p>Rank byPuzzle difficulty</p>\n<p>All 55 puzzles · 19 easy / 17 medium / 19 hard</p>\n<p>20100</p>\n<ol><li><p>11 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5</p></li><li><p>11 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra</p></li><li><p>8 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon</p></li><li><p>7 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Sonnet 5.5</p></li><li><p>6 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol</p></li><li><p>5 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Fable 5.1</p></li><li><p>2 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/moonshot.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Kimi K3.1 Pro</p></li><li><p>1 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 3.8 Flash</p></li><li><p>1 of 55 puzzles with a new frontier point</p><p> <img src=\"https://tis-100.wincon.dev/assets/logos/models/meta.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Muse Wonder 1.0</p></li><li><p>1 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Haiku 5.5</p></li><li><p>1 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/zhipu.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GLM-5.3</p></li><li><p>0 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/xai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Grok 4.7</p></li><li><p>0 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/deepseek.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />DeepSeek V4.1 Flash</p></li><li><p>0 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/meta.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Muse Spark 1.3</p></li><li><p>0 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Luna</p></li><li><p>0 of 55 puzzles with a new frontier point</p><p>  <img src=\"https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />MiMo-V2.6-Pro</p></li></ol>\n<p>Scroll horizontally to compare all models →</p>\n<p>A point counts only with fresh-game replay and 100,000 passing tests.</p>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 and <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra lead, and the data doesn’t separate them. Both solved all 55 puzzles and both found new frontier points on <strong>11</strong>. <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 found more points, <strong>72 to 32</strong>, while <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra has the higher record proximity, <strong>78.0 to 77.2</strong>. Record proximity measures closeness to the community records, where 100 means matching them. The proximity gap’s 95% bootstrap interval includes zero, from −0.7 to +2.4 points. <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra also leads in all six tested <a href=\"https://en.wikipedia.org/wiki/Hypervolume_indicator\" rel=\"nofollow ugc noopener\">hypervolume</a> settings, which measure the space covered by each model’s tradeoffs.</p>\n<p>Solving and optimizing are also different skills. <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 3.8 Flash solved 53 puzzles and found new points on 1; <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Fable 5.1 solved fewer, 52, and found new points on 5.</p>\n<p>Full rankings Expand the table</p>\n<p>TIS-100 benchmark model rankings. Select column headings to sort.</p>\n<div class=\"table-wrap\"><table><thead><tr><th></th><th></th><th></th><th></th></tr></thead><tbody><tr><td>01Opus 5.5</td><td>55 / 55</td><td>11</td><td>77.2</td></tr><tr><td>02GPT-6 Astra</td><td>55 / 55</td><td>11</td><td>78.0</td></tr><tr><td>03Gemini 4 Argon</td><td>54 / 55</td><td>8</td><td>76.2</td></tr><tr><td>04Sonnet 5.5</td><td>53 / 55</td><td>7</td><td>69.9</td></tr><tr><td>05GPT-6.1 Sol</td><td>54 / 55</td><td>6</td><td>75.6</td></tr><tr><td>06Fable 5.1</td><td>52 / 55</td><td>5</td><td>70.2</td></tr><tr><td>07Kimi K3.1 Pro</td><td>52 / 55</td><td>2</td><td>64.6</td></tr><tr><td>08Gemini 3.8 Flash</td><td>53 / 55</td><td>1</td><td>66.0</td></tr><tr><td>09Muse Wonder 1.0</td><td>49 / 55</td><td>1</td><td>58.8</td></tr><tr><td>10Haiku 5.5</td><td>45 / 55</td><td>1</td><td>53.7</td></tr><tr><td>11GLM-5.3</td><td>30 / 55</td><td>1</td><td>35.5</td></tr><tr><td>12Grok 4.7</td><td>44 / 55</td><td>0</td><td>50.1</td></tr><tr><td>13DeepSeek V4.1 Flash</td><td>40 / 55</td><td>0</td><td>46.8</td></tr><tr><td>14Muse Spark 1.3</td><td>39 / 55</td><td>0</td><td>45.1</td></tr><tr><td>15GPT-6 Luna</td><td>39 / 55</td><td>0</td><td>42.8</td></tr><tr><td>16MiMo-V2.6-Pro</td><td>10 / 55</td><td>0</td><td>12.8</td></tr></tbody></table></div>\n<p>Counts are out of 55 puzzles. <a href=\"#methodology\">Methods</a></p>\n<p>RL environments &amp; traces are available upon request for research teams. <a href=\"mailto:hello@wincon.dev?subject=TIS-100%20data%20enquiry\">Get in touch ↗</a></p>\n<h2 id=\"how-it-works-harness-design\">How it works: harness design</h2>\n<p>Every result here is a program we can run and test on fresh inputs, which makes a programming game a useful benchmark. Models get what a player can inspect, through text tools instead of a mouse and screenshots: the manual, visible test inputs, registers, and game feedback. They don’t get community code or exact records, only the game’s coarse score histograms.</p>\n<p>Read, edit, run\n:   Models read the puzzle and language reference, edit programs, and run them in the real game.</p>\n<p>Diagnostics\n:   Wrong outputs and deadlocks return mismatches, registers, and blocked ports, so the next edit can respond to evidence.</p>\n<p>Saved progress\n:   Every passing program joins the model’s own frontier and can be recovered with <code>get_best</code>, so a risky edit can’t lose an earlier result.</p>\n<p>Reminders\n:   If a model ends its turn early, a reminder asks it to keep testing, so quitting early doesn’t decide the score. Reminders contain no solution hints.</p>\n<p>A visible clock\n:   Models see time used against a 10-, 30-, or 45-minute limit. Setup doesn’t count; thinking and game calls do. Models can’t end an episode early.</p>\n<p>Run limits\n:   Game execution stops after roughly 1 second without state changes, 5 seconds without output, or 60 seconds overall, and returns diagnostics.</p>\n<p><strong>Why test 100,000 inputs?</strong> The game checks a small set of cases per run. Our headline count requires a fresh-game replay and 100,000 passing tests on the independent <a href=\"https://github.com/killerbee13/TIS-100-CXX\" rel=\"nofollow ugc noopener\">TIS-100-CXX simulator</a>, providing stronger evidence that a program generalizes to new inputs.</p>\n<h2 id=\"one-record-up-close\">One record up close</h2>\n<p><a href=\"https://tis-100.wincon.dev/puzzles/undecimal-twin-rounder/\" rel=\"nofollow ugc noopener\">Undecimal Twin RounderSPEC44149538</a>, one of the eight tournament puzzles, rounds an input to the nearest multiple of N on one output and of 11−N on the other. When we froze the leaderboard, the best community program took <strong>2,694 cycles</strong>. <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 passed its first program after 6.5 minutes, at 5,669 cycles. It went under the record <strong>16 minutes</strong> in and finished at <strong>1,918</strong>, which is <strong>29% faster</strong>. That speed cost one more node and 28 more instructions than the old record.</p>\n<p>The core of the program is a two-speed search. Instead of subtracting N one step at a time, nodes 5 and 6 subtract 4N until they overshoot, then add N back until the value turns positive. Each drives a small coprocessor, node 2 or node 7, by sending it jump offsets through a port, so the coprocessor works as a lookup table for 4N and N.</p>\n<h3 id=\"the-1-918-cycle-program\">The 1,918-cycle program</h3>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 · 8 nodes · 88 instructions · passes 100,000 of 100,000 tests</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/ec97e611686edfb6.gif\" alt=\"Opus 5.5’s 1,918-cycle Undecimal Nearest Multiples program running in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p>Node 0</p>\n<pre><code>MOV RIGHT, ACC\nJRO ACC\nNOP\nADD 1\nNOP\nADD 1\nNOP\nADD 1\nNOP\nSUB 3\nMOV ACC, RIGHT\nADD UP\nMOV ACC, RIGHT\nMOV ACC, DOWN\nMOV RIGHT, DOWN</code></pre>\n<p>Node 1</p>\n<pre><code>MOV UP, ACC\nSAV\nSUB 1\nMOV ACC, LEFT\nNEG\nADD 10\nMOV ACC, DOWN\nMOV LEFT, ACC\nNEG\nADD ACC\nADD 6\nADD LEFT\nMOV ACC, DOWN\nSWP\nMOV ACC, LEFT</code></pre>\n<p>Node 2</p>\n<pre><code>MOV UP, DOWN\nMOV UP, ACC\nSAV\nADD ACC\nADD ACC\nL: JRO DOWN\nMOV ACC, DOWN\nJMP L\nSWP\nN: MOV ACC, DOWN\nM: JRO DOWN\nJMP N\nMOV DOWN, DOWN</code></pre>\n<p>Node 3</p>\n<pre><code>MOV UP, RIGHT\nMOV UP, DOWN</code></pre>\n<p>Node 4</p>\n<pre><code>MOV LEFT, DOWN</code></pre>\n<p>Node 5</p>\n<pre><code>MOV UP, ACC\nSAV\nK: MOV 1, UP\nSUB UP\nJGZ K\nMOV 3, UP\nF: ADD UP\nJGZ X\nMOV 1, UP\nJMP F\nX: MOV 2, UP\nMOV ACC, UP\nSWP\nSUB UP\nMOV ACC, DOWN</code></pre>\n<p>Node 6</p>\n<pre><code>MOV UP, ACC\nSAV\nK: MOV 1, RIGHT\nSUB RIGHT\nJGZ K\nMOV 3, RIGHT\nF: ADD RIGHT\nJGZ X\nMOV 1, RIGHT\nJMP F\nX: MOV 2, RIGHT\nMOV ACC, RIGHT\nSWP\nSUB RIGHT\nMOV ACC, DOWN</code></pre>\n<p>Node 7</p>\n<pre><code>MOV UP, ACC\nSAV\nADD ACC\nADD ACC\nL: JRO LEFT\nMOV ACC, LEFT\nJMP L\nSWP\nN: MOV ACC, LEFT\nM: JRO LEFT\nJMP N\nMOV LEFT, LEFT</code></pre>\n<p>Laid out on the puzzle’s real grid, with inputs at the top and outputs at the bottom. Blank tiles are damaged and can’t be used.</p>\n<p>Loading puzzle trajectories…</p>\n<h3 id=\"the-search-over-time\">The search over time</h3>\n<p>PuzzleMetric Log score axis</p>\n<p>Compare models</p>\n<p>Inspect time0:00</p>\n<p>Best in-game score so far, before independent testing. Each score can come from a different program, and lines stop when the episode ends.</p>\n<p>Five models got under the old record in-game. The other two cycle records are <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol’s <strong>3,460</strong> on <a href=\"https://tis-100.wincon.dev/puzzles/intersection-identifier/\" rel=\"nofollow ugc noopener\">Intersection IdentifierSPEC64160260</a>, against 3,872 (<strong>11% faster</strong>), and <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon’s <strong>188</strong> on <a href=\"https://tis-100.wincon.dev/puzzles/error-8/\" rel=\"nofollow ugc noopener\">Error: 8SPEC79324720</a>, against 198 (<strong>5% faster</strong>). All three are on tournament puzzles, none in the main game. Of the 133 new frontier points, <strong>18</strong> beat a community record outright. The other 115 are new tradeoffs: better balances between the three scores than any community solution.</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/aa5d532f8a9a9726.gif\" alt=\"Opus 5.5&#39;s Prime Factor Calculator program running in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p>A new frontier point running in the real game: <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 on <a href=\"https://tis-100.wincon.dev/puzzles/prime-factor-calculator/\" rel=\"nofollow ugc noopener\">Prime Factor CalculatorNEXUS.20.433.1</a>, at 1,806 cycles, 10 nodes, and 103 instructions.</p>\n<h2 id=\"what-separates-the-models\">What separates the models</h2>\n<h3 id=\"cost-doesn-t-track-quality\">Cost doesn’t track quality</h3>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol reaches 75.6 record proximity at an estimated <strong>$0.75</strong> per puzzle. <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 reaches 77.2 at <strong>$5.08</strong>, <strong>7 times</strong> the cost for 2 more points. <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Fable 5.1 is the most expensive at $7.19 and scores 70.2.</p>\n<p><strong>880</strong>episodes</p>\n<p><strong>95.5M</strong>output tokens</p>\n<p><strong>62,400</strong>game-tool calls</p>\n<p>Loading analysis…</p>\n<p>Compare puzzles\nMeans include unsuccessful episodes.</p>\n<h3 id=\"performance-versus-resources\">Performance versus resources</h3>\n<p>One point per model. Upper left is better.</p>\n<p>Resource per puzzlePerformance Log resource axis</p>\n<p>Select a point for model results. The dashed line joins the nondominated models in this view.</p>\n<p>Activity doesn’t explain quality either. <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol averages 136 game calls per puzzle at 75.6 record proximity; <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon averages 41 at 76.2.</p>\n<h3 id=\"how-models-use-the-game\">How models use the game</h3>\n<p>Mean game-tool calls per puzzle</p>\n<p>EditRunInspectOther</p>\n<h3 id=\"new-points-concentrate-on-hard-puzzles\">New points concentrate on hard puzzles</h3>\n<p>New frontier points appear on <strong>14 hard puzzles</strong>, 4 medium puzzles, and no easy ones. Long-running puzzles are harder to solve but easier to improve: on puzzles with a cycle record of 200 or less, 88% of episodes pass and 4% find a new point. Above 500 cycles, 76% pass and 8% find one.</p>\n<h3 id=\"solving-versus-extending-the-frontier\">Solving versus extending the frontier</h3>\n<p>Full bar = solve rate · dark segment = new-frontier rate</p>\n<h3 id=\"every-puzzle-by-its-community-records\">Every puzzle by its community records</h3>\n<p>Position = community record · circle area = fewest nodes</p>\n<p>ModelColor byCompare records</p>\n<p>0%100%Share of the 16 models that solved each puzzle</p>\n<p>Select a dot to inspect its puzzle and community records.</p>\n<p>Loading the puzzle map…</p>\n<h3 id=\"speed-is-the-hardest-score\">Speed is the hardest score</h3>\n<p>Every model is closest to the community on nodes and furthest on cycles. Across all 13, mean proximity is 72 for nodes, 56 for instructions, and 45 for cycles.</p>\n<h3 id=\"cycles-nodes-and-instructions\">Cycles, nodes, and instructions</h3>\n<p>Mean proximity to each community record · higher is better</p>\n<h3 id=\"the-spread-across-puzzles\">The spread across puzzles</h3>\n<p>Every dot is an episode · box = middle 50% · line = median · whiskers = minimum and maximum</p>\n<p>Measure</p>\n<p>Select an episode to inspect its measurements.</p>\n<h3 id=\"every-model-every-puzzle\">Every model, every puzzle</h3>\n<p>Choose a cell to inspect the scores in the Pareto explorer.</p>\n<p>UnsolvedSolvedNew frontier point</p>\n<p>Model outcomes for individual puzzles</p>\n<p>Hover or focus a cell for its puzzle, outcome, and record proximity.</p>\n<h2 id=\"what-the-models-built\">What the models built</h2>\n<p>We grouped <a href=\"https://tis-100.wincon.dev/puzzles/\" rel=\"nofollow ugc noopener\">all 55 puzzles</a> by their main demand, then read the programs. Fifteen of the sixteen models solved all five routing puzzles, and none found a new frontier point there. The examples below show how different designs spend the same resources.</p>\n<h3 id=\"performance-by-puzzle-mechanic\">Performance by puzzle mechanic</h3>\n<p>Fraction solved · puzzles with new points below</p>\n<div class=\"table-wrap\"><table><thead><tr><th>Model</th><th>Routing &amp; synchronization5 puzzles</th><th>Arithmetic construction13 puzzles</th><th>Sequence algorithms14 puzzles</th><th>Streaming state8 puzzles</th><th>Memory &amp; interpreters4 puzzles</th><th>Image generation11 puzzles</th></tr></thead><tbody><tr><td>GPT-6 Astra</td><td><strong>5/5</strong>0 new</td><td><strong>13/13</strong>6 new</td><td><strong>14/14</strong>2 new</td><td><strong>8/8</strong>2 new</td><td><strong>4/4</strong>0 new</td><td><strong>11/11</strong>1 new</td></tr><tr><td>Opus 5.5</td><td><strong>5/5</strong>0 new</td><td><strong>13/13</strong>3 new</td><td><strong>14/14</strong>3 new</td><td><strong>8/8</strong>3 new</td><td><strong>4/4</strong>1 new</td><td><strong>11/11</strong>1 new</td></tr><tr><td>Gemini 4 Argon</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>4 new</td><td><strong>14/14</strong>0 new</td><td><strong>8/8</strong>2 new</td><td><strong>4/4</strong>0 new</td><td><strong>11/11</strong>2 new</td></tr><tr><td>GPT-6.1 Sol</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>1 new</td><td><strong>14/14</strong>3 new</td><td><strong>8/8</strong>1 new</td><td><strong>4/4</strong>0 new</td><td><strong>11/11</strong>1 new</td></tr><tr><td>Fable 5.1</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>2 new</td><td><strong>13/14</strong>1 new</td><td><strong>7/8</strong>1 new</td><td><strong>4/4</strong>1 new</td><td><strong>11/11</strong>0 new</td></tr><tr><td>Sonnet 5.5</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>3 new</td><td><strong>13/14</strong>1 new</td><td><strong>8/8</strong>3 new</td><td><strong>4/4</strong>0 new</td><td><strong>11/11</strong>0 new</td></tr><tr><td>Gemini 3.8 Flash</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>1 new</td><td><strong>13/14</strong>0 new</td><td><strong>8/8</strong>0 new</td><td><strong>4/4</strong>0 new</td><td><strong>11/11</strong>0 new</td></tr><tr><td>Kimi K3.1 Pro</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>0 new</td><td><strong>13/14</strong>1 new</td><td><strong>7/8</strong>1 new</td><td><strong>4/4</strong>0 new</td><td><strong>11/11</strong>0 new</td></tr><tr><td>Muse Wonder 1.0</td><td><strong>5/5</strong>0 new</td><td><strong>12/13</strong>1 new</td><td><strong>10/14</strong>0 new</td><td><strong>8/8</strong>0 new</td><td><strong>4/4</strong>0 new</td><td><strong>10/11</strong>0 new</td></tr><tr><td>Haiku 5.5</td><td><strong>5/5</strong>0 new</td><td><strong>11/13</strong>1 new</td><td><strong>11/14</strong>0 new</td><td><strong>6/8</strong>0 new</td><td><strong>4/4</strong>0 new</td><td><strong>8/11</strong>0 new</td></tr><tr><td>Grok 4.7</td><td><strong>5/5</strong>0 new</td><td><strong>10/13</strong>0 new</td><td><strong>9/14</strong>0 new</td><td><strong>7/8</strong>0 new</td><td><strong>4/4</strong>0 new</td><td><strong>9/11</strong>0 new</td></tr><tr><td>DeepSeek V4.1 Flash</td><td><strong>5/5</strong>0 new</td><td><strong>10/13</strong>0 new</td><td><strong>9/14</strong>0 new</td><td><strong>6/8</strong>0 new</td><td><strong>4/4</strong>0 new</td><td><strong>6/11</strong>0 new</td></tr><tr><td>Muse Spark 1.3</td><td><strong>5/5</strong>0 new</td><td><strong>11/13</strong>0 new</td><td><strong>9/14</strong>0 new</td><td><strong>4/8</strong>0 new</td><td><strong>2/4</strong>0 new</td><td><strong>8/11</strong>0 new</td></tr><tr><td>GPT-6 Luna</td><td><strong>5/5</strong>0 new</td><td><strong>11/13</strong>0 new</td><td><strong>10/14</strong>0 new</td><td><strong>4/8</strong>0 new</td><td><strong>2/4</strong>0 new</td><td><strong>7/11</strong>0 new</td></tr><tr><td>GLM-5.3</td><td><strong>5/5</strong>0 new</td><td><strong>10/13</strong>1 new</td><td><strong>6/14</strong>0 new</td><td><strong>3/8</strong>0 new</td><td><strong>2/4</strong>0 new</td><td><strong>4/11</strong>0 new</td></tr><tr><td>MiMo-V2.6-Pro</td><td><strong>3/5</strong>0 new</td><td><strong>4/13</strong>0 new</td><td><strong>1/14</strong>0 new</td><td><strong>1/8</strong>0 new</td><td><strong>1/4</strong>0 new</td><td><strong>0/11</strong>0 new</td></tr></tbody></table></div>\n<h3 id=\"multiplication-spend-code-to-save-cycles\">Multiplication: spend code to save cycles</h3>\n<p><a href=\"https://tis-100.wincon.dev/puzzles/signal-multiplier/\" rel=\"nofollow ugc noopener\">Signal Multiplier43786</a> has no <code>MUL</code>. <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon and <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra use <code>JRO</code>, a relative jump, to enter a block of repeated instructions at the right position. The jump selects how many copies of a value to send and add, saving the counter updates and branches of a multiplication loop.</p>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon’s fastest program uses two paths to multiply in parallel and a final node to interleave their outputs. It takes <strong>436 cycles</strong> against <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra’s 523, <strong>about 17% faster</strong>, at a cost of two more nodes and 33 more instructions.</p>\n<h3 id=\"the-cost-of-a-faster-multiplier\">The cost of a faster multiplier</h3>\n<p>Cycles per run · lower is better</p>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon436 cycles</p>\n<p>7 nodes · 64 instructions</p>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra523 cycles</p>\n<p>5 nodes · 31 instructions</p>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon436c · 7n · 64i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/7e82e2a616a7e042.gif\" alt=\"argon program running on 43786 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra523c · 5n · 31i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/e9b4021250a39348.gif\" alt=\"astra program running on 43786 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<h3 id=\"window-sums-distribute-the-history\">Window sums: distribute the history</h3>\n<p><a href=\"https://tis-100.wincon.dev/puzzles/signal-window-filter/\" rel=\"nofollow ugc noopener\">Signal Window Filter60099</a> outputs the sums of the last three and last five inputs, so a solution has to remember history as well as add. <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon’s four-node program uses <code>SWP</code> to rotate stored values and <code>ADD</code>/<code>SUB</code> to update running state. <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol spreads that delay and aggregation across seven nodes, passing intermediate values between them.</p>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol’s program runs in <strong>318 cycles</strong> with 26 instructions; <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon’s takes 428 cycles with <strong>only 17</strong>. The difference is how much of the work runs in parallel, and how much communication and code that costs.</p>\n<div class=\"table-wrap\"><table><thead><tr><th>Model</th><th>Cycles</th><th>Nodes</th><th>Instructions</th></tr></thead><tbody><tr><td>GPT-6.1 Sol</td><td>318</td><td>7</td><td>26</td></tr><tr><td>Gemini 4 Argon</td><td>428</td><td>4</td><td>17</td></tr></tbody></table></div>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol318c · 7n · 26i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/e67bc91dd4a90493.gif\" alt=\"GPT-6.1 Sol program running on 60099 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon428c · 4n · 17i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/28e625251d33e7e0.gif\" alt=\"Gemini 4 Argon program running on 60099 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<h3 id=\"indexed-access-from-stacks\">Indexed access from stacks</h3>\n<p><a href=\"https://tis-100.wincon.dev/puzzles/t31-node-emulator/\" rel=\"nofollow ugc noopener\">T31 Node EmulatorNEXUS.23.727.9</a> needs eight addressable storage slots, but the available memory nodes are stacks. These programs move values between stacks to reach the requested slot, then read or replace it. <img src=\"https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />MiMo-V2.6-Pro uses counted transfer loops; <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 uses <code>JRO</code> to pick among unrolled transfers, spending extra instructions to save cycles.</p>\n<p>Memory isn’t one skill: <img src=\"https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />MiMo-V2.6-Pro solves this emulator but fails <a href=\"https://tis-100.wincon.dev/puzzles/sequence-mode-calculator/\" rel=\"nofollow ugc noopener\">Sequence Mode CalculatorNEXUS.09.904.9</a> and <a href=\"https://tis-100.wincon.dev/puzzles/signal-window-filter/\" rel=\"nofollow ugc noopener\">Signal Window Filter60099</a>.</p>\n<div class=\"table-wrap\"><table><thead><tr><th>Model</th><th>Cycles</th><th>Nodes</th><th>Instructions</th></tr></thead><tbody><tr><td>Opus 5.5</td><td>215</td><td>6</td><td>63</td></tr><tr><td>GPT-6 Astra</td><td>365</td><td>4</td><td>47</td></tr><tr><td>MiMo-V2.6-Pro</td><td>451</td><td>4</td><td>34</td></tr></tbody></table></div>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5215c · 6n · 63i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/104b29f695deb96d.gif\" alt=\"Opus 5.5 program running on NEXUS.23.727.9 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra365c · 4n · 47i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/e99085f49259ae9c.gif\" alt=\"GPT-6 Astra program running on NEXUS.23.727.9 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/xiaomi.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />MiMo-V2.6-Pro451c · 4n · 34i</p>\n<figure><img src=\"https://tis-100.wincon.dev/media/programs/5c874909684dd0f4.gif\" alt=\"MiMo-V2.6-Pro program running on NEXUS.23.727.9 in TIS-100\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p>Each row is that model’s fastest saved program for the puzzle, picked for illustration. Not all are new frontier points.</p>\n<h2 id=\"god-mode-what-if-the-model-could-see-the-target\">God mode: what if the model could see the target?</h2>\n<p>The main benchmark hides the community’s scores. In a separate study on six puzzles, five models got:</p>\n<ul><li><strong>The target:</strong> exact community scores, showing which tradeoffs remain unbeaten.</li><li><strong>Diagnostics:</strong> cycle-by-cycle traces and profiles of where each node stalls.</li><li><strong>Edge cases:</strong> a fast simulator that runs 10,000 random tests and returns failures.</li><li><strong>More time:</strong> 30 minutes, plus extensions for progress, up to 60.</li></ul>\n<p>Every model improved: <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6 Astra from 82.8 to 91.8, <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol from 82.9 to 90.6, <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 from 81.4 to 89.1, <img src=\"https://tis-100.wincon.dev/assets/logos/models/xai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Grok 4.7 from 67.5 to 75.1, and <img src=\"https://tis-100.wincon.dev/assets/logos/models/google.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Gemini 4 Argon from 83.5 to 87.9 in mean record proximity. Scores rose in <strong>28 of the 30</strong> model–puzzle pairs.</p>\n<h3 id=\"score-quality-with-a-richer-setup\">Score quality with a richer setup</h3>\n<p>Mean record proximity · same six puzzles per model · higher is better</p>\n<p>Standard episodeGod mode</p>\n<p>The diagnostics changed what models did next. On <a href=\"https://tis-100.wincon.dev/puzzles/signal-multiplexer/\" rel=\"nofollow ugc noopener\">Signal Multiplexer22280</a>, <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol’s first passing program took <strong>294 cycles</strong>. Its profile showed node 2 working 86% of the time while nodes 1 and 3 spent 84% blocked waiting to write to it. <img src=\"https://tis-100.wincon.dev/assets/logos/models/openai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />GPT-6.1 Sol moved the branching out of node 2 and into those two neighbors. The next run took <strong>204 cycles</strong>, <strong>31% faster</strong>, at a cost of two more nodes and five more instructions.</p>\n<h3 id=\"the-extra-time-behind-those-scores\">The extra time behind those scores</h3>\n<p>Mean elapsed minutes per puzzle · same episodes</p>\n<p>Standard episodeGod mode</p>\n<p>Information, tools, and time all changed together, so this study can’t say which addition helped most. These runs are outside the main leaderboard and the headline count.</p>\n<p>Puzzles: Signal Multiplexer, Sequence Generator, Signal Edge Detector, Interrupt Handler, Sequence Peak Detector, and Signal Error Corrector.</p>\n<h2 id=\"co-discovery\">Co-discovery</h2>\n<p>We submitted selected programs to the leaderboard, and the community improved on them almost immediately. On <a href=\"https://tis-100.wincon.dev/puzzles/undecimal-twin-rounder/\" rel=\"nofollow ugc noopener\">Undecimal Twin RounderSPEC44149538</a>, @LongingForRest took <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5’s 1,918-cycle program and cut it to <strong>1,744</strong> by the next day. The record is now credited to both: <code>LongingForRest/opus-5-5</code>.</p>\n<p>Neither got there alone. The models found new tradeoffs quickly, and players read those programs, worked out why they worked, and took them further.</p>\n<p>Mathematics is seeing the same shift. <a href=\"https://arxiv.org/abs/2511.02864\" rel=\"nofollow ugc noopener\">Terence Tao and colleagues</a> used LLM-guided search across 67 problems and improved on the best known results in several. In <a href=\"https://terrytao.wordpress.com/2026/09/12/after-math/\" rel=\"nofollow ugc noopener\">“After Math,”</a> Silvia De Toffoli and Eamon Duede argue that a result only becomes a solution once mathematicians can understand it, and that AI fits better as an assistant than a competitor. TIS-100 is a small, measurable version of that future: a model finds a better program, and people turn it into something they understand.</p>\n<h2 id=\"pareto-explorer\">Pareto explorer</h2>\n<p>Choose a puzzle and model, then select a point to see its scores and the program that produced it. Every puzzle also has its own page with records and the best programs: <a href=\"https://tis-100.wincon.dev/puzzles/\" rel=\"nofollow ugc noopener\">all 55 puzzles</a>.</p>\n<h3 id=\"pareto-explorer-2\">Pareto explorer</h3>\n<p>Lower is better ↙<a href=\"https://tis-100.wincon.dev/puzzles/\" rel=\"nofollow ugc noopener\">Puzzle page →</a></p>\n<p>Puzzle</p>\n<p>Model</p>\n<p>Nodes</p>\n<p>Loading the frozen benchmark snapshot…</p>\n<p>Community leaderboardModelNew frontier point</p>\n<p>Select a point with the mouse or keyboard to inspect its scores.</p>\n<p>The plot shows cycles and instructions; node counts can differ. New frontier points are judged on all three scores.</p>\n<p><a href=\"https://tis-100.wincon.dev/tis100-bench-points.csv\" rel=\"nofollow ugc noopener\">Download all 3,714 model Pareto points (CSV)</a> · <a href=\"https://tis-100.wincon.dev/tis100-bench-programs.zip\" rel=\"nofollow ugc noopener\">all 3,714 programs (ZIP)</a>, also on <a href=\"https://github.com/wincon-research/tis100-bench\" rel=\"nofollow ugc noopener\">GitHub</a> and <a href=\"https://huggingface.co/datasets/wincon-research/tis100-bench\" rel=\"nofollow ugc noopener\">Hugging Face</a>. 186 rows are marked as new frontier points, which pool to 133 across models.</p>\n<h2 id=\"methods\">Methods</h2>\n<p>Expand</p>\n<p><strong>Suite.</strong> All 22 campaign puzzles, all 25 TIS-NET puzzles, and eight tournament puzzles from 2018: 19 easy (10 minutes), 17 medium (30), and 19 hard (45). Every model has one scored episode per puzzle. Time limits differ by difficulty, so difficulty and search time can’t be separated.</p>\n<p><strong>Scoring.</strong> Each model–puzzle pair contributes one selected attempt, with no best-of-attempts scoring, so solve rates are pass@1: a puzzle counts as solved when that one episode saved a program that replays in a fresh game. Unsolved puzzles stay in every average and score zero.</p>\n<p><strong>New frontier points.</strong> The reference is the <a href=\"https://github.com/12345ieee/tis100-leaderboard/tree/e3114010bac4fd592f718c0a18082e3c7045ca77\" rel=\"nofollow ugc noopener\">community leaderboard</a> as of October 15, 2026. Ties don’t count. Models found 154 candidate points on 23 puzzles. Our count pools and deduplicates them across models and keeps the 133 on 18 puzzles with fresh-game replay and 100,000 passing tests. Testing is strong evidence of correctness, not proof.</p>\n<p><strong>Record proximity.</strong> The community record divided by the model’s best score, averaged across the three scores and all puzzles, ×100. Each score can come from a different program.</p>\n<p><strong>Limits.</strong> One episode per pair compares the configurations as tested and doesn’t measure repeated-run variability. Spread between puzzles is variation across tasks. Public puzzles may overlap with training data. Mechanic groups are our own labels.</p>\n<p><strong>Programs.</strong> Every model Pareto point’s program is public under the MIT License, in TIS-100 save format and named like the community leaderboard’s files (<code>&lt;puzzle&gt;.&lt;cycles&gt;-&lt;nodes&gt;-&lt;instructions&gt;.txt</code>). They are mirrored as one file per program on <a href=\"https://github.com/wincon-research/tis100-bench\" rel=\"nofollow ugc noopener\">GitHub</a> and as a table on <a href=\"https://huggingface.co/datasets/wincon-research/tis100-bench\" rel=\"nofollow ugc noopener\">Hugging Face</a>. Two points replay in the game but fail some random tests; they are included and never count as new frontier points.</p>\n<p><strong>Resources.</strong> Costs are estimated from recorded October 2026 API list prices, including cache discounts. Tokens accumulate across responses, so repeated input counts again, and tokenizers differ. Game calls count individual dispatched tools, including errors, and exclude the automatic final submission.</p>\n<h3 id=\"settings-by-model\">Settings by model</h3>\n<p>Run-early guidance adds a prompt note to test drafts early.</p>\n<div class=\"table-wrap\"><table><thead><tr><th>Model</th><th>Reasoning effort</th><th>Run-early guidance</th></tr></thead><tbody><tr><td>Opus 5.5</td><td>high</td><td>14 / 55</td></tr><tr><td>GPT-6 Astra</td><td>high</td><td>3 / 55</td></tr><tr><td>Gemini 4 Argon</td><td>high</td><td>10 / 55</td></tr><tr><td>Sonnet 5.5</td><td>high</td><td>24 / 55</td></tr><tr><td>GPT-6.1 Sol</td><td>high</td><td>3 / 55</td></tr><tr><td>Fable 5.1</td><td>high</td><td>23 / 55</td></tr><tr><td>Kimi K3.1 Pro</td><td>provider default</td><td>55 / 55</td></tr><tr><td>Gemini 3.8 Flash</td><td>high</td><td>0 / 55</td></tr><tr><td>Muse Wonder 1.0</td><td>high</td><td>55 / 55</td></tr><tr><td>Haiku 5.5</td><td>high</td><td>27 / 55</td></tr><tr><td>GLM-5.3</td><td>provider default</td><td>49 / 55</td></tr><tr><td>Grok 4.7</td><td>medium: 9 · low: 46</td><td>25 / 55</td></tr><tr><td>DeepSeek V4.1 Flash</td><td>provider default</td><td>55 / 55</td></tr><tr><td>Muse Spark 1.3</td><td>high</td><td>55 / 55</td></tr><tr><td>GPT-6 Luna</td><td>high</td><td>2 / 55</td></tr><tr><td>MiMo-V2.6-Pro</td><td>provider default</td><td>53 / 55</td></tr></tbody></table></div>\n<p><img src=\"https://tis-100.wincon.dev/assets/logos/models/xai.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Grok 4.7 episode counts are shown by effort. A sandboxed scripting tool for game calls was available in 46–54 of each model’s 55 episodes. A note about the game’s random test reached 841 of 880 episodes. 6 <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Opus 5.5 episodes and 5 <img src=\"https://tis-100.wincon.dev/assets/logos/models/anthropic.svg\" alt=\"\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" />Sonnet 5.5 episodes ran in a different agent client at the same effort.</p>\n<h2 id=\"related-work\">Related work</h2>\n<p><a href=\"https://arxiv.org/abs/2609.22878\" rel=\"nofollow ugc noopener\">ISA-Bench</a> studies reasoning with constrained instruction sets, including TIS-100. <a href=\"https://opusmagnumbench.com/\" rel=\"nofollow ugc noopener\">Opus Magnum Bench</a> tests agents’ ability to build and optimize machines in another Zachtronics game. We focus on optimization across all TIS-100 puzzles and eight community puzzles, keeping tradeoffs across cycles, nodes, and instructions. The tasks and protocols differ, so scores aren’t directly comparable.</p>\n<h2 id=\"cite\">Cite</h2>\n<pre><code>@misc{wincon2026tis100bench,\n  title  = {TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games},\n  author = {{Wincon}},\n  year   = {2026},\n  month  = nov,\n  url    = {https://tis-100.wincon.dev/},\n  note   = {Benchmark snapshot 2026-11-03-v5}\n}</code></pre>\n<h2 id=\"credits\">Credits</h2>\n<p>Thanks to Zachtronics, the TIS-100 players, and the maintainers of the <a href=\"https://github.com/12345ieee/tis100-leaderboard\" rel=\"nofollow ugc noopener\">community leaderboard</a>, <a href=\"https://github.com/killerbee13/TIS-100-CXX\" rel=\"nofollow ugc noopener\">TIS-100-CXX</a>, and <a href=\"https://github.com/F43nd1r/zachtronics-leaderboard-bot\" rel=\"nofollow ugc noopener\">leaderboard bot</a>. Their solutions and tools make this comparison possible. Thanks also to @12345ieee, @LongingForRest, @Hersmunch, @easone, @18111398, and everyone on the <a href=\"https://discord.gg/98QNzdJ\" title=\"The unofficial, community-run Zachtronics Discord\" rel=\"nofollow ugc noopener\">Zachtronics Discord</a> who discussed the submissions and kept improving them.</p>\n<p>TIS-100, SpaceChem, and Opus Magnum are trademarks of Zachtronics. Model names and logos are trademarks of their respective owners and appear only to identify the models tested. Wincon Research is not affiliated with or endorsed by Zachtronics or any model provider.</p>","headings":[{"level":1,"text":"TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games","id":"tis-100-bench-llms-push-the-pareto-frontier-in-programming-puzzl"},{"level":2,"text":"Leaderboard","id":"leaderboard"},{"level":3,"text":"Model ranking","id":"model-ranking"},{"level":2,"text":"How it works: harness design","id":"how-it-works-harness-design"},{"level":2,"text":"One record up close","id":"one-record-up-close"},{"level":3,"text":"The 1,918-cycle program","id":"the-1-918-cycle-program"},{"level":3,"text":"The search over time","id":"the-search-over-time"},{"level":2,"text":"What separates the models","id":"what-separates-the-models"},{"level":3,"text":"Cost doesn’t track quality","id":"cost-doesn-t-track-quality"},{"level":3,"text":"Performance versus resources","id":"performance-versus-resources"},{"level":3,"text":"How models use the game","id":"how-models-use-the-game"},{"level":3,"text":"New points concentrate on hard puzzles","id":"new-points-concentrate-on-hard-puzzles"},{"level":3,"text":"Solving versus extending the frontier","id":"solving-versus-extending-the-frontier"},{"level":3,"text":"Every puzzle by its community records","id":"every-puzzle-by-its-community-records"},{"level":3,"text":"Speed is the hardest score","id":"speed-is-the-hardest-score"},{"level":3,"text":"Cycles, nodes, and instructions","id":"cycles-nodes-and-instructions"},{"level":3,"text":"The spread across puzzles","id":"the-spread-across-puzzles"},{"level":3,"text":"Every model, every puzzle","id":"every-model-every-puzzle"},{"level":2,"text":"What the models built","id":"what-the-models-built"},{"level":3,"text":"Performance by puzzle mechanic","id":"performance-by-puzzle-mechanic"},{"level":3,"text":"Multiplication: spend code to save cycles","id":"multiplication-spend-code-to-save-cycles"},{"level":3,"text":"The cost of a faster multiplier","id":"the-cost-of-a-faster-multiplier"},{"level":3,"text":"Window sums: distribute the history","id":"window-sums-distribute-the-history"},{"level":3,"text":"Indexed access from stacks","id":"indexed-access-from-stacks"},{"level":2,"text":"God mode: what if the model could see the target?","id":"god-mode-what-if-the-model-could-see-the-target"},{"level":3,"text":"Score quality with a richer setup","id":"score-quality-with-a-richer-setup"},{"level":3,"text":"The extra time behind those scores","id":"the-extra-time-behind-those-scores"},{"level":2,"text":"Co-discovery","id":"co-discovery"},{"level":2,"text":"Pareto explorer","id":"pareto-explorer"},{"level":3,"text":"Pareto explorer","id":"pareto-explorer-2"},{"level":2,"text":"Methods","id":"methods"},{"level":3,"text":"Settings by model","id":"settings-by-model"},{"level":2,"text":"Related work","id":"related-work"},{"level":2,"text":"Cite","id":"cite"},{"level":2,"text":"Credits","id":"credits"}]}}