2026-10-13
\n:: tags: \n#ai
\nOne of my fundamental objections to LLM-driven software development was that model output is non-deterministic. Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.
\nI used to think that this was a severe limitation of AI tools. How could we possibly use this unpredictable building block to craft systems that require precise and exact calculations?
\nBut as I started to get more comfortable running AI as my primary means of "writing" code, I had a revelation:
\nWhile the result needs to be exact, there is not one correct way to get there. The means by which you obtain a correct result do not need to be deterministic!
\nPreviously, I was used to specifying behavior in terms of code: precise sets of instructions that compile and execute in a predictable way. And mentally, I was treating LLM output like code. But over time, I realized that LLMs play a fundamentally different role than code does in software engineering. LLMs, and their respective harnesses, are actors in a system. Code is just an output.
\nIt took me years to get comfortable with non-deterministic code generation, yet looking back on it, it seems so obvious that programming was always just a means to an end. Let's use languages, always a hot topic of debate amongst programmers, as an example. There are infinite ways to reach the same result as expressed by code, and that's only using a single programming language! There are hundreds of widely used languages out there, and it's possible to write a solution to a particular problem in just about any of them.
\nFor years, developers have been building perfectly functional applications using very different piles of technologies, languages, and frameworks. Choosing one tech stack over another may have made it easier for a human to complete a given task. But clearly, the means of getting to that result could be wildly different across implementations.
\nNow that we're working with probabilistic actors, it's our job as engineers, or managers (depending on how you look at it), to reduce the probability space to a small, useful area. This is where all of the common LLM tricks come into play: context engineering, MCP, skills, harnesses, graphs, and memories are all tools used to reduce the amount of randomness in model output. But randomness in some form will always be there, which is at odds with the fact that there is still a need for the correct answer, especially when working on mission-critical software.
\nSo let's remove more randomness by turning non-deterministic output into deterministic tools, writing software for agents to use. Agents can call these tools to perform precise, tested, and authorized operations that have the same desired result on every execution. As an added bonus, depending on the tool's complexity, we get the same correct outcome using significantly fewer tokens by calling our new agent-built abstractions.
\nI've seen this pattern develop both in industry, with tools like https://swamp.club, and in my own work managing production-grade infrastructure. This pattern is the modern incarnation of the compounding automation that drew me into DevOps and software development in the first place. But instead of writing our own tools, we can now use AI to create its own abstractions and instruct it to use them as it builds the foundations of our new agentic software systems.
\nI find it fascinating that we can harness randomness today in a way that reduces randomness in the future. In the end, it's not much different from writing your own automations on top of earlier abstractions. It's just that the machine is calling them, processing output, and taking action faster than humans ever could.
\nOlder posts
\n©\n2026\nSteven Sklar
\n:: \nTheme is based on Terminimal by\npawroman
","headings":[{"level":1,"text":"Non-Determinism Doesn't Matter","id":"non-determinism-doesn-t-matter"}]}}