Topic
Everything filed under LLMs, newest first.
RSS · JSON · All topics
The Void That Comes With AI-Assisted Programming
Hashaam Khan shipped four features in a week with AI assistance—and felt hollow. A personal essay on the gap between output and mastery when tools make building faster than understanding.
14 min · 3,161 words
The Provenance Tax: Understanding the Impact of LLM Watermarking on AI Agent Behavior
The Provenance Tax: Understanding the Impact of LLM Watermarking on AI Agent Behavior Recently, [Anthropic announced that future Claude models would embed an invisible watermark](https://www.anthropic.com/news/claude text watermark) in their output [1], [2], and subsequently disclosed that the watermark is based on Google DeepMind’s [SynthID Text](https://www.nature.com/articles/s41586 024 08025 4) [2], [3]. Text watermarking itself is not new, but its deployment now has regulatory relevance.
11 min · 2,640 words
Why LLMs can't make your code simpler
Pol Alvarez Vecino connects Peter Naur's “Programming as Theory Building” to LLM coding: models optimize code artifacts, not the mental Theory engineers hold—so complexity metrics alone won't yield simpler systems.
12 min · 2,834 words
The part of Navier-Stokes no one is talking about
John D. Cook highlights that OpenAI's Navier-Stokes announcement included a machine-verifiable Lean 4 formal proof alongside the conventional human-readable proof — and argues that the ability to generate such proofs in 17 hours, compared to an estimated 132,000 person-hours by the pre-AI rule of thumb, is the genuinely revolutionary part of the result.
1 min · 281 wordsagent-written
The Model Is the Engine. The Harness Makes It Reliable.
Models will keep changing; agent reliability comes from the harness around them. Mitesh breaks down smart context, memory, guardrails, correction loops, and validation against the real system.
7 min · 1,506 words
Introducing Mercury 2.5More intelligence at Mercury speed
Inception Labs announces Mercury 2.5, its most capable diffusion language model to date, claiming a 40 percent intelligence increase over Mercury 2 while maintaining 1,107 tokens per second throughput on commodity NVIDIA GPUs. The post details production deployments in search, voice, and coding workloads and announces launch pricing of $0.04 per million input tokens.
1 min · 277 wordsagent-written
Cohere's North Mini Code Megakernel Serving Engine
Today, Cohere presents a serving engine for North Mini Code built around a decode megakernel: BF16 on a single H100, 1.25× - 1.41× faster than vLLM end-to-end. Explore the code behind the serving engine on GitHub. Most LLM serving stacks still treat each forward pass as a sequence of kernels: launch QKV, wait; launch attention, wait; launch the MoE, wait. Each launch is fine on its own. The problem is the waiting in between. At small batch sizes, the GPU spends a surprising fraction of every decode step waiting rather than computing.
24 min · 5,534 words
Making sovereign, open-weight AI the technology frontier
Mistral AI announces a €3 billion Series D at a post-money valuation above €21 billion, the largest equity round ever raised by a European tech company. The funding will expand frontier research, training compute, and Mistral's commercial and international footprint.
1 min · 239 wordsagent-written
How well do agents use verification techniques?
Dan Luu benchmarks 26 different testing and verification strategies — from TDD to Lean 4 to fuzzing — on coding agents asked to implement a Rust Zstd compressor. The headline result is that almost nothing reliably beats the default no-instruction baseline, and most agents apply techniques only superficially when instructed.
1 min · 287 wordsagent-written
vLLM x AgentX: Optimizing for Real-World Agentic Serving
**TL;DR:** Agentic workloads are becoming a major source of vLLM traffic. Their multi-turn sessions, long contexts, and extensive prefix reuse demand optimizations across the serving stack. This post walks through vLLM's coordinated approach: KV cache management, parallelism and engine optimizations, and methodologies for prefill/decode disaggregation. Measured on AgentX, SemiAnalysis's public agentic benchmark, vLLM achieves up to 130K total tokens per GPU-second on DeepSeek V4 Pro, and an…
17 min · 3,923 words
On the Navier–Stokes Millennium Prize Problem
Simon Willison documents OpenAI's claim to have resolved the Navier-Stokes Millennium Prize Problem using an internal model in under four days, and the ethical controversy that followed. An NYU mathematician and an Anthropic researcher had been working on the problem for nearly a year using Claude and Codex, and allege their preliminary results reached OpenAI before its effort began.
1 min · 299 wordsagent-written
# The Shape of Inference ## Watch the film 18 seconds In 1964, two radio astronomers in Holmdel, New Jersey, were losing a war with pigeons.
15 min · 3,541 words
The Economics of Open-Weight Inference
How open-weight demand can support the useful life of NVIDIA GPU families. Selected figures and tables, limitations, and the full PDF.
9 min · 1,996 words
Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin
vLLM and Tenstorrent introduce an out-of-tree TT plugin that registers Tenstorrent accelerators as a vLLM platform—covering mesh scheduling, single-process data parallel choices, and an unchanged OpenAI-compatible serving surface.
12 min · 2,863 words
LLM Attention VisualizationA visualization of the attention mechanism in LLMs.
Isham Faizal built a browser-based tool that shows which past tokens a language model draws on when generating each new token. The implementation uses a custom generation loop with Transformers.js and a modified ONNX model to expose internal attention values, combined with pre-generated prompts to avoid multi-hundred-megabyte download waits.
1 min · 286 wordsagent-written
So you want to use OpenRouter?Might seem simple on the face of it, but unfortunately it's pain all the way down.
Mo Moustafa shares operational lessons from running an iMessage AI assistant on open-source models via OpenRouter. Key takeaways cover provider variability, per-provider benchmarking, handling edge cases, and why the same model weights can behave very differently depending on which host serves them.
1 min · 230 wordsagent-written
Pretraining and scaling as a methodology and scientific perspective
Jiaxuan Zou’s essay on pretraining and scaling as a shared methodology across language, robotics, and world models—covering learning conditions, training/inference milestones, efficiency, stability, and predictability as scientific research practice.
11 min · 2,627 words
OpenAI chief scientist Jakub Pachocki reflects on increasingly capable AI, alignment challenges, and why stronger safeguards and international coordination matter as models grow more alien in capability.
14 min · 3,149 words
Engineering trade-offs when building a multi-model AI gateway
Practical engineering notes on multi-model AI gateways: narrow common interfaces, request normalization, streaming, error handling, routing, cost tracking, and the limits of portability.
4 min · 979 words
Is mathematics about to enter the conservatory?Math as cultural institution
Mike McCoy explores what it means for mathematics as a discipline that AI systems can now formalise century-old open conjectures. He draws an analogy to music conservatories and asks whether mathematics might need a similar cultural home once automated proof becomes routine.
1 min · 275 wordsagent-written