But what does production-grade actually mean? At the beginning of the project, I thought it meant correctness and "good enough" performance, which to me meant rough parity with the JDK. So I spent most of my time on correctness, building a large amount of testing infrastructure — unit and regression tests, differential tests against the JDK, fuzz tests, and exhaustive tests — to make sure SafeRE was as correct as possible. SafeRE's CI runs over 10 million test cases on each commit, and there are 20 billion additional cases available through on-demand exhaustive sweeps. These days, the differences we find through fuzz testing more often turn out to be JDK bugs than SafeRE bugs. That's made me pretty confident that SafeRE is, if anything, more correct than java.util.regex.

But as people started testing SafeRE for use in their projects, it seemed that performance was now the adoption blocker. The people evaluating SafeRE expected better performance than whatever regex library they were using, which was often something faster than the JDK, such as RE2 via JNI/FFM or Joni.

So over the past 2-3 months, we've focused on performance optimizations. My collaborator Liam Miller-Cushon has been incredibly productive, delivering approximately 80 optimization PRs and sharing real-world workloads we can tune.

SafeRE is now substantially faster than the JDK and RE2/J, and somewhat faster than native RE2, overall on the shared curated workloads in Rebar, a comprehensive regex benchmark suite created by @BurntSushi, the author of ripgrep and Rust's regex library. You can find the detailed results and methodology in the 1.0 benchmark report.

RE2 sets a high bar; it's a highly tuned C++ implementation with decades of production use at Google, and its design was the foundation for SafeRE. I'm proud that SafeRE, written in Java, is now a little faster than RE2 on these benchmarks.

That's what 1.0 means to me: I'm confident SafeRE is ready for production adoption.

What differential testing taught me

I built fuzz testing and exhaustive testing tools that compared SafeRE's results to the JDK and failed when their answers differed. I had assumed that the mature, battle-tested regex implementation in the JDK would be correct pretty much all the time.

Initially, this worked well — I found and fixed a lot of SafeRE bugs. But eventually, matching the JDK's behavior started leading us in directions I wasn't comfortable with. For example, hitEnd() and requireEnd() report whether a match attempt reached the end of the input and whether more input could invalidate a successful match. While trying to reproduce the JDK's exact behavior, my agent started reimplementing parts of its backtracking engine! I concluded that reproducing the JDK's exact behavior likely required backtracking, which would compromise SafeRE's linear-time guarantee. I also didn't want SafeRE's API to depend on details of another engine's implementation, so I removed support for both methods.

In other cases, I found places where the JDK was returning the wrong answer. Consider this example:

var matcher = java.util.regex.Pattern.compile("(?:(a))*
quot;).matcher("aaab");
matcher.find();

System.out.println(matcher.start());  // 4: end of the input
System.out.println(matcher.group());  // "" (an empty match)
System.out.println(matcher.group(1)); // "a" — should be null

The pattern means "zero or more as, followed by the end of the input." Because the input ends with b, the only possible match is zero as at the very end. The capturing group never participates in that match, so its value should be null. Instead, the JDK returns "a", left over from an earlier attempt that failed. I reported this as JDK-8381696.

So now I'm in a situation where I'm finding differences, not bugs, via differential testing, and it requires judgement to determine whether there is a bug in SafeRE or the JDK. Right now, I typically fix reported bugs, even minor ones, but I also run benchmarks if the fix looks like it could affect performance. I won't ship a fix for a rare bug if it causes a performance regression.

On performance optimization

Many of the recent performance optimizations in SafeRE come down to quickly skipping text that cannot match. For example, consider [A-Z]+:\s+[0-9]+: uppercase letters, a colon, whitespace, and a number. If colons are rare in the input, it can be profitable to first scan for colons using Java's fast String.indexOf method, then check whether the surrounding text satisfies the rest of the pattern.

HotSpot can accelerate String.indexOf using SIMD instructions. Liam's implementation of Teddy, an algorithm from Hyperscan that searches for multiple strings at once, uses SIMD operations to check many input positions in parallel. On UTF-8 alternation benchmarks with inputs from 1 KB to 100 KB, Liam measured roughly 5-6x speedups over SafeRE before this change.

Other production considerations

Production users also need a project they can depend on. Recently I've:

• Increased the bus factor by adding Liam as a collaborator
• Required code reviews before shipping changes
• Started providing SNAPSHOT builds published on every push to main

What I've learned

When I started this project, I naively thought I could build a production-grade safe regex library within a month or two with the help of agents. But it turned out to be much harder than that. The hard parts are the same parts that would always have been the hard parts: matching or exceeding battle-tested libraries on performance and correctness, while retaining the desired safety guarantees.

On the other hand, a project like this would definitely not be doable or maintainable in my spare time without agents. Not only that, but I've enjoyed building with agents and learning how to use them effectively.

Acknowledgements

Thanks to Liam Miller-Cushon, Mateusz "Serafin" Gajewski, Kevin Bourrillion, and my family.

Note: I wrote this post by hand. I used an agent for proofreading and feedback.