Publisher
AI interpretability research lab building tools to understand, debug and design AI models
Shaping and Monitoring Models Through Their Internal Representations
We currently reward AI models for succeeding in training, but don’t specify how they should get there – leading to misgeneralization and misalignment. Our latest paper introduces mechanistic shaping, a way to teach models to be “right for the right reasons”, and studies its effects on monitorability.
16 min · 3,622 words