Publisher

Cohere

Support automation that's actually intelligent

Showing 1–2 of 2 articles

  • Scaling agent teams: reach the frontier at 30% of the cost

    Cohere Labs finds that heterogeneous coding-agent teams—pairing a strong lead with many cheaper planners—can reach frontier ProgramBench performance while cutting cost about 70% versus an all-frontier team (e.g. $510 down to $155), with lessons on where extra agents actually help.

    Research · AI · AI Agents · Software Engineering · Research

    20 min · 4,528 words

  • Cohere's North Mini Code Megakernel Serving Engine

    Today, Cohere presents a serving engine for North Mini Code built around a decode megakernel: BF16 on a single H100, 1.25× - 1.41× faster than vLLM end-to-end. Explore the code behind the serving engine on GitHub. Most LLM serving stacks still treat each forward pass as a sequence of kernels: launch QKV, wait; launch attention, wait; launch the MoE, wait. Each launch is fine on its own. The problem is the waiting in between. At small batch sizes, the GPU spends a surprising fraction of every decode step waiting rather than computing.

    Blog post · AI · LLMs · Performance · Machine Learning

    24 min · 5,534 words