Publisher
Production inference platform for AI models
The efficient frontier of LLM inference
Philip Kiely explains the efficient frontier of LLM inference, separating techniques that trade off cost, latency, and quality (such as quantization) from techniques like speculative decoding and disaggregation that improve overall system performance.
5 min · 1,166 words