hillclimb

DeepRSI: evaluating agents on long-horizon tasks derived from real ML research

Hillclimb introduces DeepRSI, a benchmark of 15 open-ended machine learning problems, spanning deep learning training, GPU kernel speedups, estimator improvement and algorithm optimization, that measures how far an AI agent can improve a baseline over 48 hours. It describes the methodology with public and private verifiers, Elo rankings of frontier models, and what the results suggest for recursive self-improvement.

Research · AI · Research · Benchmarks · Machine Learning