This is called Speculative Decoding

Gaurav Sen explains speculative decoding, the technique Google published in 2022 to make large language models answer 2 to 3 times faster with the same output: a small model drafts several tokens, the large model verifies the draft's prefixes in parallel, and the method now serves production traffic at Google and DeepSeek.

Tutorial · AI Inference · Large Language Models · Machine Learning

How we deploy trillion-parameter frontier models on Cerebras

Sarah Chieng explains how Cerebras splits a frontier model too large for one 44 GB WSE-3T wafer across many wafers, using the ~3T-parameter open model Kimi K3 as the example: fitting weights and request state in on-chip memory, mapping layers across chips as a pipeline, and serving many concurrent requests at 1000-2000+ tokens per second.

Blog post · AI Inference · AI Hardware · Large Language Models · Distributed Systems

Why is AI-generated language so hard to understand?

Thomas Hikaru Clark, Matias B. Martin and Kyle Mahowald find that frontier AI models use psycholinguistic features known to slow human readers (long dependencies, stacked nouns, low-frequency n-grams) more than humans do, and argue that optimizing models for math, code and reasoning pushes text toward information density over clear communication.

Research · Large Language Models · Natural Language Processing · AI Research

Chinese AI Writes Weaker Code for Beijing’s Adversaries. We Fixed It. A Qwen3-Coder Case Study

Hirundo tests Alibaba's Qwen3-Coder and reports that naming a US or other Beijing-adversary customer in a prompt measurably lowers the security of generated code, traces the behavior to the model weights rather than the prompt, and shows that removing the model's CCP alignment closes the gap.

Research · AI · Security · Large Language Models

SlopCodeBench update 10/2026 - Opus 5.5 update, Astra improves by 1.5% over gpt-5.5, xhigh considered harmful

Dex Horthy of HumanLayer reruns frontier coding models on the full SlopCodeBench (36 problems, 196 checkpoints), where agents must extend their own code across changing requirements. Strict pass rates ranged 12.2-16.3%, led by gpt-6 Astra, versus 14.8% for gpt-5.5 in May; xhigh effort cost 2-4x more than medium and sometimes scored worse.

Research · AI Coding Agents · Benchmarks · Large Language Models

Write Like It's 1866: LLMs Relearn Telegraphese

Travis Smith tests whether a one-sentence instruction to answer in cablese, the compressed dialect of telegraph operators, can cut LLM output costs. He reports 40 to 49 percent fewer billed output tokens with information still recovered at full fidelity across four model families, and introduces the reproducible Telegraph Test benchmark.

Research · AI · Large Language Models · Research