Publisher
Support automation that's actually intelligent
Scaling agent teams: reach the frontier at 30% of the cost
Cohere Labs finds that heterogeneous coding-agent teams—pairing a strong lead with many cheaper planners—can reach frontier ProgramBench performance while cutting cost about 70% versus an all-frontier team (e.g. $510 down to $155), with lessons on where extra agents actually help.
20 min · 4,528 words
Cohere's North Mini Code Megakernel Serving Engine
Today, Cohere presents a serving engine for North Mini Code built around a decode megakernel: BF16 on a single H100, 1.25× - 1.41× faster than vLLM end-to-end. Explore the code behind the serving engine on GitHub. Most LLM serving stacks still treat each forward pass as a sequence of kernels: launch QKV, wait; launch attention, wait; launch the MoE, wait. Each launch is fine on its own. The problem is the waiting in between. At small batch sizes, the GPU spends a surprising fraction of every decode step waiting rather than computing.
24 min · 5,534 words