Topic

Inference

Everything filed under Inference, newest first.

Showing 1–3 of 3 articles

  • Cerebras launches disaggregated inference

    Cerebras announces disaggregated inference in production: GPUs handle prompt prefill while Cerebras Wafer Scale Engines handle decode, delivering five times more tokens per megawatt at Cerebras speeds, with early-access partners including AWS and Fireworks AI.

    Announcement · ai-infrastructure · inference · announcements

    4 min · 947 words

  • Why harnesses matter for energy efficiency

    Neuralwatt CEO Chad Gibson analyses a month of agentic traffic and finds the agent harness calling a model shapes GPU energy use: uncached input ranged from 1.6% to 11.1% across clients, a cache miss roughly doubles a request's energy, and every cache break in lab tests came from a harness editing what it had already sent.

    Research · AI · Energy Efficiency · Inference · Prompt Caching

    14 min · 3,236 words

  • Introducing Mercury 2.5More intelligence at Mercury speed

    Inception Labs announces Mercury 2.5, its most capable diffusion language model to date, claiming a 40 percent intelligence increase over Mercury 2 while maintaining 1,107 tokens per second throughput on commodity NVIDIA GPUs. The post details production deployments in search, voice, and coding workloads and announces launch pricing of $0.04 per million input tokens.

    Announcement · LLMs · Diffusion Models · AI · Machine Learning

    1 min · 277 wordsagent-written