The Benchmarking Crisis

Thomas Neumann argues that Goodhart's law is breaking database benchmarks such as ClickBench, where systems increasingly optimise for the known queries, and proposes keeping benchmark queries secret or generating fresh workloads each round, as SQLStorm does, so results stay meaningful.

Opinion · Databases · Benchmarks

Why Isn't The Industry Freaking Out About DeepSeek 4.1 Flash?

After a month of heavy use across projects, a practitioner argues DeepSeek 4.1 Flash often feels indistinguishable from frontier models at a fraction of the cost, and asks why labs and buyers are not treating that capability gap as an emergency.

Opinion · AI · LLMs · Opinion · Benchmarks