Adapting CPython for yk

Earlier this year I wrote a post on yk, which can automatically add JIT compilers to C interpreters. Since then we’ve applied yk to more interpreters, fixed various bugs, and improved performance notably. Towards the end of October I showed some data where on our benchmark suite[1] yklua is 2× faster than the PUC Lua interpreter it takes as a base.

I was quite pleased with myself until someone brought me down to earth by asking “how do you know if this works for large interpreters?” It’s a fair question: PUC Lua is at least an order of magnitude smaller[2] than CPython or CRuby and who’s to say if it would work with much bigger interpreters? We had looked at CPython a year or so ago, but at that point yk was missing several obvious things that CPython would need, so we quickly moved on. A lot has changed since then, and a quick LLM experiment got far enough to suggest it would be worth looking again.