Topic
Everything filed under Performance, newest first.
RSS · JSON · All topics
The efficient frontier of LLM inference
Philip Kiely explains the efficient frontier of LLM inference, separating techniques that trade off cost, latency, and quality (such as quantization) from techniques like speculative decoding and disaggregation that improve overall system performance.
5 min · 1,166 words
When One GPU Isn't Enough: Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM
NVIDIA shows how to serve the frontier-scale GLM-5.3 model locally across two DGX Stations linked by ConnectX-8 RDMA with vLLM, reaching about 607.8 tokens per second.
18 min · 4,077 words
ggml 1.0.0 prunes the deprecated public API and legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings performance work across CUDA, Metal, Vulkan, SYCL, OpenCL, WebGPU and Hexagon backends plus scheduler improvements.
10 min · 2,235 words
Thinking Machines Lab details its low-precision training recipe on NVIDIA Blackwell: choosing quantization targets, block-scaled backward passes, re-quantization with stochastic rounding, quantized communications and custom kernels that helped pretrain Inkling 2 Small in 68 hours.
17 min · 3,893 words
Makepad's Relax is an experimental live compiler and JIT runtime for Rust that runs a Cargo package straight from source without rustc, LLVM or a linker, compiling functions on first call and live-patching edits for faster iteration.
14 min · 3,117 words
Laurence Tratt describes ykifying CPython: using the yk meta-tracing system to add a JIT compiler to the large CPython interpreter, walking through optimising interpreter dispatch, tracing, unrolling and indirect inlining, and showing ykcpython can be surprisingly competitive on performance.
27 min · 6,157 words
Talking Zen 6 with Mike Clark at AMD In-Depth
Chips and Cheese's George Cozma interviews AMD Zen architect Mike Clark about Zen 6, the evolution from Zen 5, the differences between Zen 6 and the denser Zen 6c cores, and the design trade-offs behind them.
9 min · 1,994 words
blqsort - A C++ Quicksort Faster Than std::sort
Christof Kaser presents blqsort, an MIT-licensed branchless quicksort for C and C++ that beats std::sort and ipnsort on random and partially sorted data, with benchmarks and an explanation of how it works.
41 min · 9,518 words
I ran 120B MoE models on my 32GB M1 Max, using under 16 GiB
A write-up of an experimental inference engine that keeps most of a mixture-of-experts model on SSD, streams in the experts it needs, and enforces a hard memory cap, running 120B-parameter MoE models on a 32GB M1 Max in under 16 GiB.
5 min · 1,134 words
AMD’s Zen 6 Family: Feeding the Beast
Chips and Cheese walks through AMD's Zen 6 and Zen 6c cores, covering branch prediction, instruction fetch, execution and the out-of-order engine where the two designs split, load/store changes, and SPEC CPU 2026 results, arguing AMD now ships two cores shaped for different targets.
10 min · 2,265 words
Brendan Gregg describes active benchmarking: analysing system performance with other tools while a benchmark runs, to confirm it measures what you intend and to find the real limiter, rather than trusting passive results. Statistical analysis is useful afterwards, once the numbers are valid.
4 min · 977 words
Adopting Python 3.15's lazy imports: a 4× faster CLI, and where they didn't help
Namegarage describes moving its back office to Python 3.15 and using explicit lazy imports to make its CLI about 4× faster to start, with an allowlist gate, eager-import tests, cases where lazy imports were not lazy, and profiling results.
11 min · 2,586 words
Ben Andrew traces what glibc's malloc actually does under the hood: using strace to watch brk and mmap syscalls, measuring syscall cost, and explaining how the heap, arenas, bins and the mmap threshold let small allocations avoid hitting the kernel, with benchmarks comparing allocators.
14 min · 3,249 words
A Rust GUI developer argues that GPUI's and egui's state models are dead ends for complex desktop apps: every UI framework eventually needs to know what changed and who depends on it, and run-once components with signals are the simplest API for that precision. Includes benchmarks against Elm-style and immediate-mode designs, with a disclosure that the author works on the signal-based zgui framework.
20 min · 4,680 words
Mahdi Bahrami walks through auto-vectorizing a Base64 encoder in Swift, showing how to write loops the compiler can turn into SIMD and how that raised Apple's swift-foundation Base64 encoding to around 16 GB/s, with a look at a roughly 30 GB/s implementation.
33 min · 7,532 words
Optimizing reference deletions in Git
Karthik Nayak explains how Git's files backend stores loose and packed refs, why deleting references from packed-refs is slow at monorepo scale, and the patch he merged upstream that speeds up reference deletion by about 18%, landing in Git v2.98.
6 min · 1,490 words
Writing a DSL for high performance parallel programming in OxCaml
A Jane Street summer intern project: Nicholas Yang describes building an embedded DSL in OxCaml for writing SIMD kernels, replacing esoteric C inner loops used for options pricing and risk checks with portable, vectorized code, plus benchmarks and planned scheduling transformations.
16 min · 3,770 words
Every Windows 11 app should kill Electron, and Discord just showed why
Windows Latest walks through Discord’s Rust rewrite: a GPUI desktop client that boots faster than Electron, uses about 290MB of RAM with two accounts signed in, drops the Starting screen, and shares Rust code with upcoming mobile and WebAssembly web apps.
6 min · 1,445 words
Mitchell Hashimoto announces Libghostty 1.0—the embeddable terminal library behind Ghostty—covering API stability, embedding use cases, and what a 1.0 commitment means for terminal and IDE vendors.
7 min · 1,688 words
Write your own kernel optimiser
Csongor Kiss walks through building a kernel optimiser with Cachet, using grids and tiles to schedule loops so a program's data flow matches the hardware instead of hand-writing custom kernels.
80 min · 18,302 words