The Prisoner's Dilemma of Frontier AI

When I saw the essay that Dario Amodei published about pacing frontier AI, my first reaction was to reject it, but not based on the content but based on a sense of distrust for the person that wrote it himself.

Whenever frontier lab leadership starts begging for industry-wide coordination, it usually smells like incumbent defense. OpenAI and Anthropic hold the lead today, but open weights and Chinese labs are closing the gap faster than expected. Proposing an international bureaucracy of embedded auditors and shared speed limits right at this moment looks suspiciously like an attempt to freeze the leaderboard.

Some could say that this is illogical because they could use the most powerful models to self-improve, hence accelerating even more the progress and widen the lead. It's possible, I'm not close enough to the development of the next models to understand the scale of the impact that recurring self improvement might have, but if anything the story of the last year shows that the Chinese labs, or even SpaceXAI, were able to catch up faster than anyone had thought. I can't predict the future but the recent events make me at least a bit skeptical about the objection.