Publisher

Thinking Machines Lab

AI research lab building public training tools

Showing 1–1 of 1 article

  • End-to-End 4-Bit Training

    Thinking Machines Lab details its low-precision training recipe on NVIDIA Blackwell: choosing quantization targets, block-scaled backward passes, re-quantization with stochastic rounding, quantized communications and custom kernels that helped pretrain Inkling 2 Small in 68 hours.

    Research · machine-learning · llms · performance · hardware

    17 min · 3,893 words