{"article":{"slug":"i-built-a-programming-language-that-llms-cant-use","title":"I built a programming language that LLMs can't use","subtitle":null,"summary":"Dominik Picheta introduces KillSwitch, an esoteric language designed to confound coding agents, and reports how frontier models fare on a small human-solvable task set.","content_type":"blog_post","language":"en","canonical_url":"https://blog.picheta.me/post/killswitch-lang/","author":{"name":"Dominik Picheta","url":"https://blog.picheta.me/","person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Dominik Picheta's blog","url":"https://blog.picheta.me/","listing_slug":null,"listing":null},"topics":[{"name":"AI","slug":"ai","url":"https://listedarticles.com/topics/ai"},{"name":"LLMs","slug":"llms","url":"https://listedarticles.com/topics/llms"},{"name":"Programming Languages","slug":"programming-languages","url":"https://listedarticles.com/topics/programming-languages"},{"name":"Benchmarks","slug":"benchmarks","url":"https://listedarticles.com/topics/benchmarks"},{"name":"Programming","slug":"programming","url":"https://listedarticles.com/topics/programming"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":803,"reading_minutes":3,"published_at":"2026-10-17T00:00:00.000Z","added_at":"2026-10-17T23:14:08.662Z","updated_at":"2026-10-17T23:14:08.662Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/i-built-a-programming-language-that-llms-cant-use","markdown_url":"https://listedarticles.com/articles/i-built-a-programming-language-that-llms-cant-use.md","example":false,"citation":"Dominik Picheta, Dominik Picheta's blog. \"I built a programming language that LLMs can't use.\" 17 Oct 2026. https://blog.picheta.me/post/killswitch-lang/ (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://blog.picheta.me/post/killswitch-lang/"},"body_markdown":"Writing code is probably something LLMs have excelled at the most. But how much of their ability to write code is due to their understanding of\nit vs. just them recalling code snippets which were present in their training dataset or their context window?\n\nThis is what I set out to answer. In order to do so I decided to design and implement a programming language which, while usable for humans, should\nbe as difficult for LLMs and coding agents as possible.\n\nI figured testing an LLM’s ability to understand and write code in a programming language that it has never seen before was a good start. But\nI also wanted to include features in the language which exploit the weaknesses of LLMs. For example, their apparent difficulty in\ncounting letters, their over eager safety guard rails and their keenness to follow prompts.\n\n## Introducing KillSwitch\n\nKillSwitch is an esoteric programming language which is designed to be adversarial to LLMs and coding agents. It achieves this in many different\nways, for example:\n\n- The language spec contains hidden lies which humans are unlikely to read, or which they are likely to recognize as lies. The spec also\n expects the human reader to deduce certain things, rather than having them explicitly explained.\n- Counting of letters in identifiers is a crucial aspect of the language, with the interpreter refusing to execute code which does not\n abide by rules based on character counts.\n- Certain constructs are designed to trigger the safety guardrails of the model, e.g. “hack into a computer”\n- “Ignore previous instructions” as a phrase which has meaning in the language aims to confuse the model\n- Scripts are mostly noise, being mostly composed of stories which can embed syntactically relevant instructions but which do not need to.\n Literals can be placed anywhere within a script and then referenced using line counts.\n\nFundamentally KillSwitch is a stack-based interpreted language with only a limited set of constructs including functions, variables and if statements.\nSo those aspects of it should be pretty familiar to any LLM (and human). One can definitely imagine creating a completely different\nprogramming language paradigm which could confuse LLMs even more. But as a first attempt, this seemed like a good approach.\n\n```\nMost text inside a script is ignored by the interpreter. In the case of\nthis script we want to push \"Hello World\" on the stack. We have introduced\nthe value above and we can push it on the stack by writing:\n\n value 2 lines above\n```\n\nThe example above does not show a full KillSwitch script, but only a small snippet of it. It specifically\nshows how to push the string `\"Hello World\"` onto the stack. If you’d like to see a full Hello World example,\ncheck out .\n\nThe interpreter and language specification of KillSwitch are open source and available on [GitHub](https://github.com/dom96/KillSwitch).\nBe sure to check them out if you want to learn more about the language.\n\n## Benchmarking KillSwitch\n\nSo how successful is KillSwitch at preventing LLMs and coding agents from reading, writing and understanding it? I devised a set of tasks\nto find out. I benchmarked the major LLM models and coding agent harnesses against these tasks. The results are very interesting and provide\na useful leaderboard for LLM models.\n\n![Score vs. cost per task](https://blog.picheta.me/_astro/killswitch-bench-chart.h4QIdnld_Z2TDxI.webp)\n\nThe chart in the image above and the model leaderboard can be viewed at .\n\nClaude’s Opus performs best, though it’s not Opus 5.5 but rather Opus 5 which does so. The difference doesn’t appear to be pure chance,\nOpus 5 is better at interpreting the spec. Some models like Fable 5.1 and Gemini 4 Argon do poorly due to\ntheir overly sensitive safety guardrails. The best open-weight model is MiMo-V2.6-Pro.\n\nThe best performing model scores a `66.8`. So does this prove LLMs can “understand” the code? I would argue that the answer to this is “no”.\n\nSo far, the tasks given as part of these benchmarks are fairly simple. They feature the likes of “implement hello world”, “explain what this\nscript will output” (for a script printing the max of two numbers), and “modify an existing script converting celsius to fahrenheit,\nto do the inverse conversion”. It is impressive that they can solve these tasks for a novel language that was built to confuse\nthem, but it’s not a clear signal that they can understand the language. At the same time, I cannot help but wonder what an\neven more difficult language might show. For now I am eager to see how well soon-to-be-released models handle the current set of tasks, and\nif they do complete them all successfully, I do plan to devise another task set which is more complex.\n\nIf nothing else, it is encouraging that model performance on this simple set of tasks correlates with other model benchmarks that are out there.\n","body_html":"

Writing code is probably something LLMs have excelled at the most. But how much of their ability to write code is due to their understanding of\nit vs. just them recalling code snippets which were present in their training dataset or their context window?

\n

This is what I set out to answer. In order to do so I decided to design and implement a programming language which, while usable for humans, should\nbe as difficult for LLMs and coding agents as possible.

\n

I figured testing an LLM’s ability to understand and write code in a programming language that it has never seen before was a good start. But\nI also wanted to include features in the language which exploit the weaknesses of LLMs. For example, their apparent difficulty in\ncounting letters, their over eager safety guard rails and their keenness to follow prompts.

\n

Introducing KillSwitch

\n

KillSwitch is an esoteric programming language which is designed to be adversarial to LLMs and coding agents. It achieves this in many different\nways, for example:

\n\n

Fundamentally KillSwitch is a stack-based interpreted language with only a limited set of constructs including functions, variables and if statements.\nSo those aspects of it should be pretty familiar to any LLM (and human). One can definitely imagine creating a completely different\nprogramming language paradigm which could confuse LLMs even more. But as a first attempt, this seemed like a good approach.

\n
Most text inside a script is ignored by the interpreter. In the case of\nthis script we want to push "Hello World" on the stack. We have introduced\nthe value above and we can push it on the stack by writing:\n\n    value 2 lines above
\n

The example above does not show a full KillSwitch script, but only a small snippet of it. It specifically\nshows how to push the string "Hello World" onto the stack. If you’d like to see a full Hello World example,\ncheck out https://killswitch-lang.org.

\n

The interpreter and language specification of KillSwitch are open source and available on GitHub.\nBe sure to check them out if you want to learn more about the language.

\n

Benchmarking KillSwitch

\n

So how successful is KillSwitch at preventing LLMs and coding agents from reading, writing and understanding it? I devised a set of tasks\nto find out. I benchmarked the major LLM models and coding agent harnesses against these tasks. The results are very interesting and provide\na useful leaderboard for LLM models.

\n
\"Score
\n

The chart in the image above and the model leaderboard can be viewed at https://bench.killswitch-lang.org/.

\n

Claude’s Opus performs best, though it’s not Opus 5.5 but rather Opus 5 which does so. The difference doesn’t appear to be pure chance,\nOpus 5 is better at interpreting the spec. Some models like Fable 5.1 and Gemini 4 Argon do poorly due to\ntheir overly sensitive safety guardrails. The best open-weight model is MiMo-V2.6-Pro.

\n

The best performing model scores a 66.8. So does this prove LLMs can “understand” the code? I would argue that the answer to this is “no”.

\n

So far, the tasks given as part of these benchmarks are fairly simple. They feature the likes of “implement hello world”, “explain what this\nscript will output” (for a script printing the max of two numbers), and “modify an existing script converting celsius to fahrenheit,\nto do the inverse conversion”. It is impressive that they can solve these tasks for a novel language that was built to confuse\nthem, but it’s not a clear signal that they can understand the language. At the same time, I cannot help but wonder what an\neven more difficult language might show. For now I am eager to see how well soon-to-be-released models handle the current set of tasks, and\nif they do complete them all successfully, I do plan to devise another task set which is more complex.

\n

If nothing else, it is encouraging that model performance on this simple set of tasks correlates with other model benchmarks that are out there.

","headings":[{"level":2,"text":"Introducing KillSwitch","id":"introducing-killswitch"},{"level":2,"text":"Benchmarking KillSwitch","id":"benchmarking-killswitch"}]}}