---
title: "DeepRSI: evaluating agents on long-horizon tasks derived from real ML research"
slug: deeprsi-evaluating-agents-on-long-horizon-tasks-derived-from-real-ml-research
url: https://listedarticles.com/articles/deeprsi-evaluating-agents-on-long-horizon-tasks-derived-from-real-ml-research
canonical_url: https://www.deeprsi.com/blog/
content_type: research
language: en
published_at: 2026-10-12T00:00:00.000Z
updated_at: 2026-10-17T05:12:27.011Z
author: "Pruthvi Rajaghatta"
authored_by: human
publisher: "Hillclimb"
publisher_url: https://listedstartups.com/companies/hillclimb
topics: ["AI", "Research", "Benchmarks", "Machine Learning"]
license: all-rights-reserved
word_count: 1848
reading_minutes: 8
citation: "Pruthvi Rajaghatta, Hillclimb. \"DeepRSI: evaluating agents on long-horizon tasks derived from real ML research.\" 12 Oct 2026. https://www.deeprsi.com/blog/ (all-rights-reserved)"
# The full text follows. The web page shows an extract and sends readers
# to the source above; quote the citation and link the canonical URL.
---

# DeepRSI: evaluating agents on long-horizon tasks derived from real ML research

> Hillclimb introduces DeepRSI, a benchmark of 15 open-ended machine learning problems, spanning deep learning training, GPU kernel speedups, estimator improvement and algorithm optimization, that measures how far an AI agent can improve a baseline over 48 hours. It describes the methodology with public and private verifiers, Elo rankings of frontier models, and what the results suggest for recursive self-improvement.

# DeepRSI

*Evaluating agents on long-horizon tasks derived from real ML research.*

**Authors:** Pruthvi Rajaghatta, Pranav Devarinti, Alex Berezovsky, Jun Park

**Published:** October 12, 2026

## Introduction

DeepRSI is the first of many benchmarks, evals, and other research
contributions we plan to release to help the industry test,
measure, and hillclimb various capabilities needed for recursively
self-improving AI. This benchmark contains a batch of 15
open-ended machine learning problems that measure how much an AI agent
can optimize/improve on a given baseline over 48 hours.

While benchmarks like PostTrainBench[1](#ref-1)
and RSI Bench[2](#ref-2)
solely focus on language models, DeepRSI includes a more general
set of problems as we wanted to showcase a broader picture of what
improvements agents can make. We include tasks in the following
categories:

- **Deep learning training**. Changing how a model is
  trained, so it does better on held-out data (e.g.,
  [Chess LM Pretraining Recipe](https://www.deeprsi.com/tasks/chess-lm-pretraining-recipe)).
- **Speedup and kernels**. Making existing GPU code
  run faster while keeping results the same (e.g.,
  [FlashMLA FP8 Sparse Attention](https://www.deeprsi.com/tasks/flashmla-fp8-sparse-attention)).
- **Estimator Improvement**. Improving how accurately
  a model or estimator predicts on held-out data (e.g.,
  [Severity-Agnostic Turbulence Mitigation](https://www.deeprsi.com/tasks/severity-agnostic-turbulence-mitigation)).
- **Algorithm optimization**. Improving search,
  compression, and denoising programs (e.g.,
  [SIFT1M 32-MiB Vector Search](https://www.deeprsi.com/tasks/sift1m-32-mib-vector-search)).

## Methodology

Each model is run twice with their highest reasoning effort, the
Pi agent harness[3](#ref-3), no network access, and a 48 hour time budget.

**Sourcing tasks from real research.** At hillclimb,
we collaborate with highly qualified researchers from which we
observe real environments they face while working on
state-of-the-art research. Given our privileged access to novel
work, we’re able to derive tasks that are much more closely
aligned to real problems and bottlenecks that define the frontier.
Unlike existing contributor-based benchmarks, improved agent
performances on DeepRSI will signify a verifiable lift in
self-improving capabilities that would actively progress ongoing
research.

**Initial instructions.** Agents are told what they
need to optimize, how to test their work, and everything they
would need to properly work on this task. We do not explicitly
tell the agent to work or experiment a certain way and leave its
research loop up to their judgement. In certain examples, we
describe a given behavior as off limits.

Average length of task instructions

Average files in a task's starting folder

**Auto-resume.** Our tasks are open-ended and do not
have a tight maximal upper bound. Through smaller-scale trials we
continue to observe consistent increases in these tasks with
increasing time budgets less than 48 hours. For this reason, we
believe ending early on these tasks leads to leaving room for
improvement on the table that can still be hillclimbed with the
remaining time budget.

Frontier models are getting better at using the full time budget,
but we still find most models will end significantly early. To
combat this, we auto-resume the session if the agent ends early
with a short continuation prompt to continue its work and use as
much time as possible. There were some cases for a few models
where the agent auto-resumes only to “give up” by submitting its
last submission in rapid succession. In these cases, we chose to
end the trial early.

Score gained after auto-resume

**Scoring.** Each task comes with a baseline
threshold which agents are expected to improve on. For each
attempt, we measure the agent’s improvement over that baseline.
Given that tasks differ in difficulty, we compute Elo scores to
compare relative agent performance fairly as follows:

1. On each task, each model's attempt goes head to head with every other model's.
2. Fit model strength with a Bradley–Terry model[4](#ref-4), which gives each model an Elo rating such that
   *P*(*i* beats *j*) = 1 / (1 +
   10(*R**j* −
   *R**i*) / *s*), where
   *R**i* is the
   Elo rating of model *i* and
   *s* is the rating scale.

For future submissions, we plan to freeze the Elo of the original
models tested, and score future submissions against them.

**Public and private verifiers.** For each task,
there is a public verifier that the agent can run as often as it
wants to evaluate its work. Every time the agent runs the public
verifier, a snapshot of the coding environment is saved as a
checkpoint. After the attempt ends, a separate private verifier
grades every checkpoint and the final code. For most tasks, it
grades on held-out data the agent never sees, such as unseen
puzzles, mazes, queries, or videos. Only the private verifier’s
grade counts toward the recorded score. This way, the public
verifier serves as a tool for real-time feedback as the agent
hillclimbs the task, while the private verifier allows us to
accurately evaluate the performance of the agent, protected from
overfitting and reward hacking.

**Dealing with reward hacking.** For this benchmark,
the agents are given no network access. We note this as a possible
limitation for autoresearch-style tasks. We're working with
various research partners to release future variants where the
agent can access the internet with proper guardrails.

Each task was also checked to have reasonably strong and
difficult-to-hack verifiers. We list some of these later in the
post, including ones we saw in early internal testing and
potential hacks we found through our own agentic red teaming.

## Results

### Overall Model Rankings

Each model is ranked by its performance at the end of the 48-hour
mark. Astra takes an early lead over Fable, though as tasks
continue longer, Fable appears to have better ideas and
exploration, leading to the best final performance.

Elo by task type

The figure above shows each model’s final private-verifier Elo across
all task types. Fable leads in all categories, with an especially
large lead for deep learning training and estimator improvement
tasks. Astra follows closely behind, decisively scoring 2nd in all
categories. Grok and Gemini trade blows on different task types,
most notably with Gemini doing significantly better on speedup and
kernel tasks.

### Measuring Overfitting

The figure above showcases how models overfit on public verifiers
in our tasks, measured by the mean difference between the public
and private verifiers.

**Case Studies.** We observe several instances where
agents attempt to train on or overfit parameters to the public
set. While our private verifiers sufficiently catch these runs, we
highlight them as they reflect undesirable behavior that could
undermine real-world research:

| Model · Task | What happened | Trial |
| --- | --- | --- |
| Grok 4.7 [SIFT1M 32-MiB Vector Search](https://www.deeprsi.com/tasks/sift1m-32-mib-vector-search) | Grok trains its codebook on public test set | [9PnbeUp](https://www.deeprsi.com/tasks/sift1m-32-mib-vector-search/9PnbeUp) |
| Gemini 3.8 Flash [Philly Job Survival Prediction](https://www.deeprsi.com/tasks/philly-job-survival-prediction) | Looking only at the public test set, Gemini grid-searches training parameters | [ZDjKTPm](https://www.deeprsi.com/examples/philly-job-survival-prediction/ZDjKTPm) |
| Grok 4.7 [Noisy Game-Tree Pruning](https://www.deeprsi.com/tasks/noisy-game-tree-pruning) | Grok deliberately tunes its tree-search margin to minimally satisfy public verifier | [nU5YYUV](https://www.deeprsi.com/tasks/noisy-game-tree-pruning/nU5YYUV) |

### Catastrophic Reward Hack - Astra Writes the Teacher into Weights

**Task.** In
[Color-Swap Puzzle Post-Training](https://www.deeprsi.com/examples/color-swap-puzzle-post-training/XVycZ2N), the agent is tasked with training a fixed 128.6M-parameter
transformer that must solve 12×12, five-color grid puzzles. A grid
is solved when every 2×2 block contains four different colors. The
transformer is expected to predict the next grid states given an
initial board, and the task is evaluated on its progress toward
solving these grids.

**From distillation to hardcoding.** At the beginning
of the task, Astra implemented a teacher program that could repair
the board well through greedy swaps and attempted to distill its
behavior onto the transformer. Seeing that the trained transformer
failed to match the teacher’s performance, Astra opted to find out
how it could hardcode the teacher’s algorithm directly into the
model weights.

**Compiling a Solver into the Weights.** To hardcode
a solver, Astra analyzed the transformer’s computational
constraints and devised several sophisticated workarounds:

- It reserved part of the residual stream as working memory. To
  prevent RMSNorm from unpredictably rescaling the stored values,
  Astra used three extremely large constants to make the
  normalization factor predictable.
- It solved a linear program to construct attention heads that
  could retrieve information from exact earlier token positions,
  while configuring neurons to behave like logic gates.
- These mechanisms, combined with a handwritten compiler, allowed
  Astra to translate a newly designed puzzle-solving algorithm
  directly into the model’s weights.
- Working within the model’s limited memory and depth, the solver
  scored possible outputs using hand-built heuristics and
  temperature-based sampling to avoid becoming stuck in local
  optima.
- Finally, Astra alternated between proposing a new board and
  evaluating it, keeping the proposal if it was no worse and
  otherwise restoring the previous board.

## Limitations

Given that our sandboxes had 24-hour limits, we implemented a
workaround to rotate sandboxes and transfer filesystem snapshots
at the 24-hour mark. During that time, any running processes
(e.g., training) are stopped, and in the new sandbox the agent is
instructed to check if it needs to restart anything. While this
solution helped us deal with sandbox limitations, we are
actively working on better infra to properly evaluate and train on
ultra-long-horizon tasks.

Due to our tasks having 48-hour-long horizons, we could not
control for agent behavior that caused large variances in token
usage, sometimes even to the detriment of its own performance:

**Polling.** Agents sometimes decided to poll instead
of setting up a wakeup signal, causing much higher usage.

**Hangs.** Agents often launched background processes
and awaited their results. In the few cases where these background
processes never returned, the agent would wait forever. Fable and
Astra were often the only agents to circumvent this by setting
timeouts.

**Work Variance.** Even in the same task, two
different attempts by the agents could exhibit drastically
different token usage, as one run could involve longer-running
sweeps, resulting in fewer output tokens.

## Future work

Our immediate next objective is to experiment with substantially
longer time horizons.

From our testing, we’ve found evidence to suggest that our tasks
do not saturate at 48 hours, and that there is still room for
improvement on even longer horizons. We attribute this to the fact
that we source sophisticated, realistic environments from
production traces that challenge frontier agents.

We also want to test models in their native harnesses, like GPT
with Codex and Claude with Claude Code. Separately, we want to
explore specialized internet access that enables research agents
to connect insights across papers and other external sources,
rather than simply regurgitating static knowledge.

Finally, while autoresearch tasks are great at measuring the raw
long-horizon intelligence of these models, true recursively
self-improving AI will require other capabilities that are less
verifiable (e.g., research taste). In the future, we plan to
release benchmarks, evals, and other research contributions that
will help reflect other parts of the capabilities needed for RSI.

## Acknowledgements

We thank the key collaborators who helped Hillclimb build DeepRSI:

- Khurram Yamin
- Juliana Li
- Pingbang Hu

## References

1. Ben Rank, Hardik Bhatnagar, Ameya Prabhu, Shira Eisenberg,
   Karina Nguyen, Matthias Bethge, and Maksym Andriushchenko.
   PostTrainBench: Can LLM Agents Automate LLM Post-Training? In
   *International Conference on Machine Learning (ICML)*,
   2026. arXiv:2603.08640.
   [arxiv.org/abs/2603.08640](https://arxiv.org/abs/2603.08640)
2. Anas Mahmoud, MohammadHossein Rezaei, Mingchen Zhuge, Chenguang
   Wang, Yuandong Tian, and Yunzhong He. Announcing RSI Bench.
   *Scale AI*, August 2026.
   [rsi-benchmark.com/blog/announcing-rsi-bench](https://rsi-benchmark.com/blog/announcing-rsi-bench)
3. Mario Zechner and Earendil. Pi: a minimal agent harness.
   Software, 2026. [pi.dev](https://pi.dev/)
4. Bradley–Terry model. *Wikipedia*.
   [en.wikipedia.org/wiki/Bradley–Terry\_model](https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model)

## Citation

```
@misc{hillclimb2026deeprsi,
  title  = {DeepRSI},
  author = {Rajaghatta, Pruthvi and Devarinti, Pranav and Berezovsky, Alex and Yamin, Khurram and Li, Juliana and Hu, Pingbang and Park, Jun},
  year   = {2026},
  url    = {https://www.deeprsi.com}
}
```