Publisher

Baseten

Production inference platform for AI models

Showing 1–1 of 1 article

  • The efficient frontier of LLM inference

    Philip Kiely explains the efficient frontier of LLM inference, separating techniques that trade off cost, latency, and quality (such as quantization) from techniques like speculative decoding and disaggregation that improve overall system performance.

    Blog post · llms · infrastructure · performance

    5 min · 1,166 words