Topic
Everything filed under Performance, newest first.
RSS · JSON · All topics
turbopuffer is pushing the frontier of search. To do that, we have to fundamentally redesign our storage architecture so the vector index is no longer primary.
6 min · 1,334 words
OxCaml - Stack Allocations and Locality
The motivation behind OxCaml is to make OCaml a great language for performance engineering, with the eventual goal being to upstream these language extensions to vanilla OCaml (OxCaml](https://oxcaml.org/)). OxCaml maintains backwards compatibility with OCaml, which implies that every OCaml program is a valid OxCaml program. The language extensions range from additions to the type system that rule out data races, to control over allocations that reduces garbage collection pressure, to management
7 min · 1,697 words
What Makes LLM Tokenization Slow?Exploring the performance of byte-pair encoding by optimizing a GPT-2 tokenizer.
Andrew Healey dissects GPT-2’s reference BPE tokenizer, measures what makes tokenization slow, and shows concrete optimizations on the hot path of LLM products.
11 min · 2,547 words
PlanetScale Released Text Search and We Have a Lot to Say (Part I)
Two BM25 optimizations and benchmark configuration changes inspired by PlanetScale's TIN benchmarks make ParadeDB's text search faster without changing its document identifiers.
14 min · 3,188 words
Fernando Simões digs into when page tables stop being a rounding error: shared mappings across many processes, Linux kernel history from Arcangeli to Qi Zheng's empty-page-table fix, and the trade-offs of huge pages.
6 min · 1,309 words
Why we built the fastest robust TTS model
Gradium's latest streaming TTS hits ~50ms time-to-first-audio while improving naturalness and hard cases like phone numbers—freeing latency budget for LLM turns and barge-in in voice agents.
2 min · 431 words
zenkai: The App Launcher I Wrote Because I Wanted Something Fast and Beautiful
Dayvster builds zenkai, a Zig + Qt6 cross-platform app launcher with ~140ms startup (sometimes ~20ms), 65+ themes, Lua plugins, and a sandbox—written as a hobby performance deep dive.
2 min · 571 words
Cut your AI spend with AI Gateway's Auto Router
Cloudflare AI Gateway now features a model router that evaluates request complexity using an edge-deployed classifier to select the optimal model. By balancing expected output quality against token costs, organizations can dramatically cut AI spend while maintaining performance.
7 min · 1,627 words
The Rise of Overfit Inference Engines
A hands-on essay on how local LLM inference stacks are overfit to specific models and hardware—and what that means for anyone chasing reproducible homelab performance with llama.cpp and friends.
10 min · 2,324 words
How to speed up the Rust compiler in September 2026
How to speed up the Rust compiler in September 2026 My last post](https://nnethercote.github.io/2026/07/31/how-to-speed-up-the-rust-compiler-in-july-2026.html) on the Rust compiler’s performance was two months ago and a lot has happened since then. Overall progress The measurements for the period 2026-07-29 to 2026-09-28 can be seen here](https://perf.rust-lang.org/compare.html?start=1a833e16546c2eb012758ddd499964fd8afee29e&stat=wall-time&tab=compile&end=c1070d69382b8d2f2eb65119c738a77d9e324c9e&
5 min · 1,212 words
Garbage Collection: Generational? Incremental? Both!
The third Language Summit talk was brought by Mark Shannon, who is the author of the incremental garbage collector implementation shipped in Python 3.14 that was reverted back to the generational garbage collector from Python 3.13 after reports of “significant memory pressure” in production environments. The original goal of the new incremental garbage collector was to reduce maximum pause times by an order of magnitude for larger heaps.
5 min · 1,173 words
Nobody Is Listening on Port 8125
There's a Fedora box on my desk named `stick`. An app on it has been firing StatsD datagrams at `127.0.0.1:8125` ten times a second for about a week now, the way apps have done since Etsy taught everyone to in 2011.
10 min · 2,281 words
Arseny Kapoulkine revisits clustered LOD and meshoptimizer after NVIDIA’s vk_lod_clusters Zorah update—smaller, nicer hierarchical clustered LOD for path-traced scenes with billions of triangles.
33 min · 7,518 words
Edward Kmett's week-old Turbo Haskell Compiler (THC) JITs GHC Core onto Truffle/GraalVM, supports AOT Native Image, polyglot FFI, Loom green threads, and can compile pandoc, happy, alex, and GHC itself.
2 min · 506 words
Effect 4.0 rebuilds the TypeScript effect system from the ground up: zero-dependency core, ~5× smaller bundles, ~6.4× concurrent throughput, 86% less memory per fiber, and a long-term support policy through at least 2029.
2 min · 433 words
What Is a Container, Really? Five Years of GPU Infrastructure
Beam Cloud recounts five years of GPU infrastructure: from ECS and Knative cold starts to a custom container runtime, FUSE lazy-loading, and what “container” actually means in production AI compute.
9 min · 2,132 words
Deser: Rethinking Rust Serialization
Armin Ronacher revisits Deser, an experimental Rust serialization library that inverts Serde’s visitor recursion into heap-backed sinks/emitters—trading some performance for lossless buffering, composable adapters, XML namespaces, and no stack overflow on deep nests.
2 min · 492 words
Making np.searchsorted up to 25× Faster in NumPy 2.5
Alejandro Candioti explains how NumPy 2.5 makes np.searchsorted up to 25 times faster by batching many binary searches so their independent memory accesses overlap, starting from a vectorized Python prototype that beat NumPy 2.4's native code, and discusses cache-friendly layouts like Eytzinger as a future direction.
12 min · 2,861 words
Faster streaming HTML with batches and aggregates
Anders Murphy explores emergent properties of server-rendered streaming HTML over SSE with Datastar: bounding work with ticks, batching and barriers, Brotli or Zstandard streaming compression, querying the database every tick, a simple hiccup interpreter and cache thrash, with his hyperlith demo serving about 1000 concurrent users at 10 FPS on a 2 vCore VPS.
7 min · 1,610 words
CLMs treat context as an editable file so models natively manage their own context—beating harness baselines on BrowseComp-Plus and long-horizon coding with fewer FLOPs, plus in-context skill evolution and RL for context management.
3 min · 699 words