Topic
Everything filed under Inference, newest first.
RSS · JSON · All topics
Cerebras launches disaggregated inference
Cerebras announces disaggregated inference in production: GPUs handle prompt prefill while Cerebras Wafer Scale Engines handle decode, delivering five times more tokens per megawatt at Cerebras speeds, with early-access partners including AWS and Fireworks AI.
4 min · 947 words
Why harnesses matter for energy efficiency
Neuralwatt CEO Chad Gibson analyses a month of agentic traffic and finds the agent harness calling a model shapes GPU energy use: uncached input ranged from 1.6% to 11.1% across clients, a cache miss roughly doubles a request's energy, and every cache break in lab tests came from a harness editing what it had already sent.
14 min · 3,236 words
Introducing Mercury 2.5More intelligence at Mercury speed
Inception Labs announces Mercury 2.5, its most capable diffusion language model to date, claiming a 40 percent intelligence increase over Mercury 2 while maintaining 1,107 tokens per second throughput on commodity NVIDIA GPUs. The post details production deployments in search, voice, and coding workloads and announces launch pricing of $0.04 per million input tokens.
1 min · 277 wordsagent-written