One-shotting an Unreal Engine plugin from a 3,000-line plan

I've spent most of my career as a 3D programmer at Ubisoft, where I shipped two Assassin's Creed games. I worked on Anvil's Micropolygon pipeline there and gave a 2026 Game Developers Conference talk about it. So I went looking for a 3D plugin I could credibly evaluate myself, one that I would normally expect an experienced programmer to spend a couple of months building. That meant picking a codebase too. I went with Unreal Engine: millions of lines of code and several complex systems that would be non-trivial for an agent to work in.

The plugin I landed on was a tool for placing bounded 3D volumes in a world and painting onto the surfaces inside them. For example, an artist might place one around some rocks, then use a brush to paint moss onto them. That's the artist-facing half. Under the hood, each stroke has to update GPU state, survive save/load, and participate in undo/redo. Materials need to sample the result, and volumes need to stream in and out with the world. If a plan is going to settle every architectural decision in a system like this, its going to end up fairly complex. My final plan came out at over 3000 lines.

Building a workbench for the plan

I started with a big prompt describing the plugin in what felt like a reasonable amount of detail. I then had Fable ask me questions until we were aligned on the direction. Fable launched a workflow that dug through the Unreal Engine source, searched the web, and adversarially vetted its own findings. After a bunch of tokens, it came back with a thorough Markdown plan.

That was where I ran into my first problem. Even the first draft was about a thousand lines, and I could not actually sit down and read it as Markdown. Anthropic's Thariq Shihipar wrote about this in The unreasonable effectiveness of HTML. His threshold is about a hundred lines. I was already ten times past that. So I built a humble VS Code extension that turned the Markdown into HTML, then started adding ways to actually iterate on the plan.

When I started reading through the plan, my instinct was to highlight sections and leave notes on them. After a few hours, I had a plan covered in notes flagging real problems but no good way to act on them. I could edit the plan by hand, but each note often had a blast radius well beyond the text it was attached to. Following every consequence through a plan that was already around a thousand lines long felt like an easy way to miss something.

So I tried handing the revision to an agent. It was faster than editing by hand and better at chasing changes across the whole plan. The catch was that I could not trust it blindly. I still had to review every diff for missed notes or changes I had never asked for. Before long, reviewing the plan had become the bottleneck, much the same problem engineers face with code review.

At that point, reviewing the revision felt like another job I could give to agents. I passed several adversarial verifier agents the notes, the original plan, the revised plan, and a diff. Each verifier checked whether every note had been applied faithfully and whether any unrelated ideas had slipped in. If one found a problem, its feedback went back to the revising agent for another pass. This continued until every verifier was satisfied.

Originally, the revising agent returned the entire new plan as the final output of its turn. As the plan grew, the cracks started to show. The model spent a lot of effort reproducing text that had not changed, and I started hitting response timeouts. So I had the extension expose an in-process MCP server to the reviser. Instead of emitting a whole new plan, it could surgically edit a copy held in memory. In hindsight I should have just started there. It's much closer to how the models wanted to work. With that change, revisions got cheaper and less unrelated text drifted between versions.

Stress-testing the plan with adversarial review

At this point I had a decent loop going. I had left dozens of notes that guided the high-level shape of the plan. Two of those notes flagged a texture format that used much more memory than we could reasonably support, and a complex data structure inside a 3D texture that would have been difficult to maintain. Both were replaced, along with many smaller fixes, and the plan was starting to seem solid.

The reality is there are limits to how far ahead I can see while planning. The details that bite you tend to live below the surface of a plan (data streaming issues, async work coordination, serialization, undo/redo corner cases). In the past, holes like these surfaced during implementation. Every roadblock spun up its own discussion and a fresh Jira ticket to navigate back to the golden path. Anthropic has a good blog post about these "unknown unknowns", and I was sure a plan this size had plenty. If an agent was going to one-shot the implementation, it could not discover these holes halfway through. A key decision like the data eviction policy for streaming should not be left to the agent writing the code. I needed a way to find those flaws while they were still flaws in the plan.

So I experimented with an adversarial review of the plan. The idea was that an agent would first find a list of "lenses" to look at the plan through (security, feasibility, coherence, memory, performance, that kind of thing). Then for each lens, an audit agent would review the plan from that angle and report back any issues it found. Each issue was then independently vetted by two subagents. The Grounding agent checked whether the issue was supported by evidence, while the Consequence agent checked whether the claimed impact actually followed.

I decided the notes I already had were a good primitive for these issues, but that meant tightening what a note actually was. A note grew a headline, which stated the problem in a sentence, and a justification, which explained why it was actually a problem. Adversarial review became my go-to tool, and it was surfacing issues I had not considered, some of which would have shot me in the foot.

Once I had those notes, I needed a way to work through each one and settle on a fix. I added a proposed fix field to notes, so the investigation chat had something concrete to iterate on, and that was what drove the revising agent. Asking the reviser to figure out the best fixes for a bunch of notes at once always complicated its reasoning. Now each note got an agent's full attention, and I stayed in the loop.

When the reviewer finds too much

Around this time I started exploring which models were best suited for which tasks. Fable was great at architecture, and I relied on it as the revising agent to produce each new revision. Everything else, the chats and the reviews, was where I actually spent my time. For those I reached for OpenAI's Sol. I wanted a quick back and forth, and Sol was faster and lighter on my usage limits.

Sol was really good at finding issues. It seems to be a model with strong recall, which lets it dig incredibly deep on a given topic. Given enough time it could surface things the Anthropic models simply were not catching. When I noticed this I searched around a bit and saw that CodeRabbit reported the same observation when they benchmarked Sol for code review.

I was genuinely impressed with the things it was catching. One finding was about a specific texture compression scheme (BC7), and how at the boundaries of small data chunks it would cause inconsistencies between neighboring texels, which might show up as pixel shimmering under camera motion. The appendix at the bottom has the full breakdown. Finding the bug was impressive enough on its own, but that it could also reason about the visual scenario where the artifact would show up kind of shocked me.

After a few more finds like that, it was clear Sol was incredibly good at this. It kept finding unknown unknowns that Fable had missed. From this point on Sol kept surfacing all sorts of issues and we iterated on fixes in chats. This went on for days, and for a while the process felt like it was working... until it wasn't.

Ultimately Sol's strength is also one of it's weaknesses. It was surfacing too many issues, and a lot were contrived cases. In isolation each issue is one you look at and say "well, we put the effort into discovering it, we may as well fix it since it will harden the system." That has its dangers though. Each fix you add introduces a new architecture surface, and that surface can have bugs of its own. Those new pieces show up in the next review, you add more machinery to cover them, and now you have even more surface to audit. You can loop on this for what feels like forever.

At this point I am convinced if I kept going, Sol would eventually suggest that we colonize Mars to store a backup of our data in the off chance that a meteor hits Earth.

There were two problems at play here. The first is that I was running my investigation chats with Sol for the speed. What I did not immediately realize is that Sol kept suggesting solutions that were quite over-engineered for the problem, but they seem convincing in the moment when you are reading them in isolation. I tried to pivot ideas that seemed like they were overcomplicating the design, but when you are faced with dozens of issues at once you start to run into review fatigue. You become more inclined to just accept the solution proposed.

That leads to problem number two: there was no end. The adversarial review could not reason about what "good enough" looked like, and every review round simply surfaced new, increasingly more nuanced concerns. I could not just stop running reviews either. One in ten issues raised was a genuinely critical issue that had a real impact on the architecture, but it was lost in the noise of the other nine. There was no way to filter them other than committing the cognitive effort of reading each issue and reasoning about it.

Teaching the loop when to stop

Solving this required two changes. First, stop Sol from architecting solutions. That meant splitting the job in two: one agent to poke at the problem, another to design the fix. That became the Deep Investigation feature for notes, a chat involving two agents. The supporting agent presents a simple scenario where the issue shows up (the how), follows that with the consequence (the what), and then gives evidence to prove the claim (the why). That gets fed to the main agent, whose job is to architect a solution in light of it.

The main agent presents the solution it came up with to the supporting agent, who reviews it and sends back feedback. The main agent takes that feedback and revises the proposed fix, and they keep going back and forth until they reach a consensus. The main agent is explicitly told to keep its solution as simple and robust as possible, and to ignore any feedback that seems contrived or that would require complex machinery that is not justified. After they land on a fix, it comes back to me. I can chat about it with the main agent, and when I am happy with it I propose it as the fix for the note.

I typically run with Fable as the main agent and Sol as the supporting agent. That pairing made a lot more sense than asking Sol to do both. Browsing online it seems that others have landed on the same idea and are seeing success with it.

The second idea was a triage gate. Rather than accept every issue and review it myself, adversarial review notes now go through a triage pass first. The point of triage is to catch issues that are "real" but would never really show up in practice, or at least whose rarity does not justify introducing new machinery to address them.

The triage agents mostly base their decisions on a triage guidance Markdown file that I maintain. Every time an issue comes up that describes a situation I do not care about addressing, I add a generalized entry that tells agents to reject issues of that nature. For example, Sol kept finding issues like "what if an artist is in the middle of painting a volume and then tries to submit their data to version control." Obviously no human is actually doing this and expecting it to work. Addressing it is a waste of time. I had triage guidance such as the following: "Assume artists are using the tool reasonably, use cases off the golden path do not justify new complexity to harden against, unless the interaction can lead to a full editor crash."

Suddenly a bunch of issues Sol was finding were being dropped at triage, which reduced my note review load. The notes that made it through were also easier to work through with the Deep Investigation feature. I could finally see the light at the end of the tunnel.

Around two days of using the new setup was all it took for the plan to settle into what I started calling a steady state. I would run another adversarial review with Sol, and it would come back without anything meaningful. After a few of those rounds in a row, it was time to call it done.

Fourteen hours, zero interventions

Armed with a stable plan and a dream I fed Fable the plan and asked it to build its own workflow: gather evidence, plan, delegate the coding, audit where it mattered. Then I told it I would not be intervening and it was to take the plan to completion from start to finish in a single shot.

To my surprise, it pulled it off. Fourteen hours later, the plan held. I could drop a volume into the scene, set its dimensions and brick resolution, and pick up a brush. Strokes inside the volume populated bricks as I painted, and I could choose which channels to paint into, then erase, flood fill, or invert them. A material could sample the result and use it for shading. Duplicating and moving volumes behaved the way I expected. I had a working tool.

I must admit, the results were not perfect. The UI was a mess and not intuitive to use. An assert was firing in the render graph, but Sol found the cause quickly and the fix was small. The bounding box code was not recomputing the box center for asymmetric bounds. And a few compute shaders were naively launching work for every entry in a predominantly sparse data structure, resulting in 10x the required work. Ultimately a human is still needed. But these were all reasonable fixes to make. The core architecture was on solid ground and that's the part that would be harder to fix. The issues were all implementation details that were subpar, rather than design decisions that were flawed.

If you want to explore the implementation, the repo is uploaded on GitHub. Sourcebot provides a really cool feature which lets you ask questions about any GitHub repo, I recommend giving it a try to learn more about this one. You can do this on any public Github repo by prepending ask to the public github url, ex. https://askgithub.com/sourcebot-dev/world-voxel-paint.

So the experiment answered both questions. Agents can help a human build a plan this size, and once the plan is ready, an agent can carry it through in one shot. But the second part only worked because of the first. I would not have handed Fable the thousand-line first draft and expected a usable result fourteen hours later. The loop is what got it there: review, triage, investigate, revise, verify, and then do it again until Sol stops finding anything that would change a design decision. That is where almost all of the work happened, and it is the part I plan to keep tinkering with.

If you want to try the planning tool

The harness I described in this post is an experimental VS Code extension comprised of a pile of workflows I built to explore what the future of planning with agents might look like. As I learn new things the extension continues to evolve and in the new future it's something I would like to share with others. If you want to try it when it's ready for public use, leave your email below.

Appendix

A painted volume is mostly empty space, so giving every volume its own 3D texture would waste a lot of memory. Instead, all of the paint data lives in one large 3D texture divided into 8×8×8 bricks. A volume only claims bricks where an artist has actually painted. The allocator hands out whatever slots are free, which means two bricks sitting next to each other in the texture may belong to completely unrelated volumes.

That creates a problem for hardware trilinear sampling. Near the edge of a brick, the GPU blends neighboring texels together. If the next brick belongs to another volume, it blends in data that is effectively garbage. The fix is to store 7×7×7 unique texels in each brick and fill the outer one-texel border with copies from its logical neighbors. Now the sampler sees the right data across the boundary, at the cost of storing some texels twice.

Unfortunately this falls apart when we factor in compression. BC7 compresses texels in 4×4 blocks, so the stored value of each texel depends on the other fifteen texels around it. A rough analogy is fitting a line through a set of points: change one point and the line shifts, which changes how every other point is represented. BC7 does something similar. Change one texel and the reconstructed values of the others can shift slightly too.

Here is where the border copies and BC7 collide. The duplicated border texels represent the same logical data, but they live in different BC7 blocks with different neighbors. After compression, the two copies can come back with slightly different values. Near a brick seam, a tiny change in sample position determines which copy the GPU reads. As the camera moves, the sample point drifts back and forth across the seam, and the surface shimmers.

A careful human might catch this before implementation. More likely it slips through and arrives months later as a vague QA bug: “the surface sometimes shimmers when the camera moves.” From there, tracing it back through trilinear sampling, brick borders, and BC7 compression would be an arduous journey. The fact that Sol found that entire chain from the plan alone is crazy to me.