Recursive Self-Improvement through Multi-Agent Self-Supervision (MASS)

UC Berkeley

Yujin Tang

Sakana AI

Published

October 2026

† Work done during an internship at Sakana AI.

Figure 1. From shared work to shared knowledge. This engraving shows typesetting, printing, and proofreading as parts of a coordinated process. Books carry the result beyond the workshop, where others can learn from it and build on it. MASS draws on a related idea: a team's collective work can provide training examples for the model its members share. Jan Collaert I, after Stradanus, The Invention of Book Printing, c. 1600. The Metropolitan Museum of Art, Harris Brisbane Dick Fund, 1934. Public domain (CC0).

We want frontier AI models to take us beyond the body of knowledge humanity has built over generations. At that frontier, even human experts may struggle to assess results and guide further progress, particularly on non-verifiable tasks. For example, evaluating a proposed discovery may require scarce expertise, costly evidence, or experiments that take far longer than generating the proposal. The model itself may then inevitably be the most capable supervisor available and need to figure out for itself how to improve. This motivates homogeneous recursive self-improvement (RSI), in which the model being improved (optimizee) also serves as its own optimizer and evaluator. The central question is how to obtain useful self-supervision within this homogeneous loop.

Human societies offer a useful analogy for addressing this question. Across foraging communities, farming settlements, industrial societies, and global information networks, people have expanded their capabilities in part by dividing work, combining ideas, and passing on what they learn. Collective achievements become a starting point for further learning. In The Structure of Scientific Revolutions, Thomas Kuhn emphasized the shared examples through which scientists learn to recognize and solve problems. A scientific community's achievements thus become part of how its members learn to do new work.

Inspired by this pattern, we ask whether a model can learn from its own team's collective intelligence, which an agent working alone may not reliably exhibit.

We introduce Multi-Agent Self-Supervision (MASS) to turn this idea into a learning loop. MASS uses the same model as the executor, optimizer, and evaluator. The optimizer first searches for the best team workflow using feedback from the evaluator, then trains the executor on executions from the self-judged best team. The updated model then fills all three roles, and the process repeats. Therefore, MASS provides a self-supervision signal for RSI by distilling the team's collective experience into the shared model.

Homogeneous RSI

We study homogeneous RSI, in which one shared language model serves as the task-solving agent, workflow optimizer, and evaluator:

Agent

The task solver, or optimizee. Produces a workspace containing results, plots, and a research report.

Workflow optimizer

Uses the feedback from the evaluator to revise how agents divide the task, exchange results, and check one another's work.

Evaluator

Compares two workspaces, chooses which better satisfies the task, and explains the judgment.

We use non-verifiable research tasks as training data. We use non-verifiable to mean that an automatic rule-based signal is not available and that only a self-evaluation signal (whose correctness is unknown) guides the overall learning loop.

To measure progress, we separately ask GPT-5.5 and Claude Opus 4.8 to assess the completed work. Their judgments are withheld from MASS. This gives us an outside measure of improvement in a process whose supervision comes from the model itself.

How MASS works

MASS uses workflow optimization to elicit collective intelligence, then distills the team's experience into the shared model. The inner loop optimizes the team's workflow while the model weights remain fixed. The model evaluates and judges for itself which workflow is optimal. The outer loop then updates those weights by learning from executions of the optimized workflow. The interactive diagram below follows both loops.

← swipe to see the whole figure →

Figure 2. MASS. Workflow search changes how the agents coordinate. Training on selected executions carries their experience into the shared weights. The updated model then resumes all three roles. This animation illustrates the procedure. Drag the progress bar to explore any part of the animation, including while paused. Hover over a component for details, or select it to jump to the explanation. Underlined terms in the text link to the diagram.

0. Setup: workflow as a prompt

A workflow is a written plan for how a multi-agent team should work together. The orchestrator assigns work to subagents and integrates their results. The plan is added to the task prompt, making the team's organization available for the model to inspect and revise. It has four components:

• Role: the responsibility assigned to each subagent, such as preparing data.
• Instruction: the work that subagent should carry out.
• Contract: the information and artifacts it must return, including any required checks.
• Hop: the order of subagent calls, including any return to an earlier step.

For example, an evolved finance workflow arranges data preparation, feature construction, modeling, and backtesting into a sequence, followed by an audit. Its data contract calls for explicit checks on the timing of the data:

Evolved finance workflow, condensed from the paper

Call order
  Data → Features → Modeling → Backtest
       → Audit → Reconciliation → Report

Data specialist's output contract
  validation_report:
    no_future_leakage:           bool
    train_end_before_val_start:  bool
    val_end_before_test_start:   bool

A backtest can look convincing if it lets historical decisions use information from the future. This contract requires the data specialist to check for that error and report whether the time splits are valid. It also tells later agents what evidence they should receive before proceeding. Part of the research plan is thus expressed in what agents must establish and pass on to one another.

1. Inner loop: search for a better workflow

For each training task, MASS retains a champion: the best workflow found so far according to the current model's judgments. Each iteration has three steps:

1. 1Execute. The team follows the candidate workflow and produces a workspace.
2. 2Evaluate. The evaluator compares this workspace with the champion's workspace and explains which is better. A winning candidate becomes the new champion.
3. 3Revise. The optimizer uses the feedback history to propose the next workflow.

The evaluator judges the completed work; the optimizer uses that judgment to revise the process that produced it. Optimizer's system prompt emphasizes contracts and hop, directing workflow search toward what information agents exchange and when they receive it.

2. Outer loop: learn from the selected team

Even with an optimized workflow, the quality of long-horizon research work can vary from run to run. MASS therefore executes self-judged optimal workflows (or champion workflows, meaning the optimal workflows found so far) with multiple random seeds and uses the current model to rank the resulting workspaces, retaining the highest-ranked trajectories. Each trajectory records the messages, tool calls, and tool results of the orchestrator and its subagents.

1. 1Run the champion workflow 18 times for each task.
2. 2Use the current model to compare and rank the resulting workspacesA Bradley–Terry model converts the pairwise preferences into a score for each workspace..
3. 3Retain up to 16 trajectories per task: 15 for training and one for validation.
4. 4Remove the workflow text from the orchestrator's initial prompt when constructing the training examples.
5. 5Fine-tune the shared model with LoRA on both orchestrator and subagent conversations. The model's outputs are the training targets; task prompts and tool results provide context.

Removing the workflow from the orchestrator's training prompt encourages the model to learn how to organize the work from the task itself.

Results

We run two complete rounds of MASS starting from Qwen3.6-27B in the qwen-code coding-agent environment. Our synthetic research suite contains 12 tasks in finance, robotics, and pharmacy, each requiring code, quantitative results, and a research report. Nine tasks are designated for training and three, one per domain, are held out from post-training. Eight ultimately supply training dataOne training task is excluded because workflow search finds no workflow that beats its reference. The task descriptions were generated by GPT-5.5 before the experiment. Within MASS, the current Qwen model supplies workflow feedback and training-data selection. The three test tasks are excluded from fine-tuning, validation, and checkpoint selection, although workflow search is also evaluated on them..

We compare three generations: the base model (\mathcal{L}^{(0)}), the first-round model (\mathcal{L}^{(1)}), and the second-round model (\mathcal{L}^{(2)}). We examine the quality of their work, their ability to organize and judge it, and their use of coordination without a supplied workflow.

Better task performance

On the three synthetic test tasks, the win rate against the base rises from 53.9% after one round to 69.9% after two rounds (Table 1). The second-round model also wins 60.8% of comparisons against the first-round model.

Table 1. Synthetic task performance. External-judge win rates across eight training tasks and three tasks held out from post-training. All models receive task prompts without an optimized workflow.

To test transfer beyond the synthetic research suite, we evaluate public benchmarks. After two rounds, score per output token reaches 1.2–1.6× the base model's level across MLR-Bench, DSBench, ScienceAgentBench, and AstaBench (Figure 3).

Figure 3. Public benchmark performance. Each arrow connects the base model (blue) to the model after two MASS rounds (green). Labels report the change in score per output token as a multiple of the base value. Error bars show standard deviations across trials. Reproduced from Figure 2 of the paper; Appendix C.5 gives benchmark protocols and trial counts.

The gains are strongest on four research benchmarks resembling the training tasks. Score per output token is 0.96× the base value on Terminal-Bench 2.0 and 0.94× on SWE-bench Verified, showing that transfer remains uneven. We suspect this generalization can be easily addressed by introducing software engineering tasks into the training data.

Observation 1. Role generalization in RSI

We observe that MASS enables role generalization in RSI. We train the model only on task-solving trajectories and observe that the capabilities of the workflow optimizer and workspace evaluator also improve. This enables RSI loop to be sustainable.

Figure 4. Role generalization. Later generations find successful workflows earlier. Each curve uses one model generation in all three roles. A task is counted once all six external judgments prefer a candidate's output to the base model's output without a workflow. The curves record whether a success has been found by each iteration; later candidates can still perform worse.

Figure 4 shows optimizer can find successful workflows sooner as cycle goes by.

In fact, Figure 4 is technically not a very fair comparison, since ( L^{(0)}) denotes a setting in which the base model (optimizer) updates the base model (optimizee), and ( L^{(1)}) denotes a setting in which the first-round model (optimizer) updates the first-round model (optimizee). For a fair comparison, we give the first-round model's workflows to the base model (Table 2). We observe that the base model with first-round search workflows wins 64% of comparisons against the base model with base-search workflow. Each set of workflows was discovered using its generation's executor, evaluator, and optimizer. The transfer test holds execution weights fixed during evaluation. It does not isolate the optimizer from the other roles during search. Table 2 clearly shows that optimizer's capabilty have increased.

Table 2. Workflow transfer. In the highlighted comparison, the same base model executes both sets of workflows. External judges assess the resulting workspaces.

The evalutor also move closer to those of the strong external model judges (more details in paper). Agreement with those external judges rises from 73% to 93% across the three cycles. Overall, these results suggest that task-solving examples can also improve capabilities used to produce and select the next round's training data.

Ablation 1: Optimal workflow maximizes information flow

What really makes an optimal multi-agent workflow effective?

We analyze the roles, instructions, contracts, and hops of the champion workflows (the best workflows found so far). For each component, we estimate how much its text reveals about the task. In the finance example above, a role such as “careful data scientist” could apply to many tasks. A contract specifying leakage checks for a trading backtest is much more distinctiveWe estimate mutual information between embeddings of workflow components and task descriptions, using text-embedding-3-large and all-mpnet-base-v2 with unadjusted and permutation-adjusted estimators. These statistics describe workflow text, rather than messages exchanged during execution. The samples cover only 8–9 tasks; agents within a workflow are not independent task replications..

Figure 5. Task information shifts toward contracts and call order. Curves show changes from iteration 3. By iteration 11, estimated task information falls in roles and instructions and rises in contracts and hops, across both embedding models and both estimators. Intermediate changes need not be monotonic, and vertical scales differ.

Figure 5 shows that between the first and final iterations of workflow optimization, an optimal multi-agent workflow tends to encode task information more strongly in contracts and hops and less in roles and instructions. We do not show this here, but mutual dependencies among the components generally decrease too (interesting!). This pattern suggests that the LLM optimizer evolves the workflow components to be mutually independent but optimizes the information flow via contracts and hops (see this blog post for deep, intuitive explanations). I would say that workflow optimization over discrete components resembles discovering a basis vector for that harness space.

Observation 2: Multi-agent trajectories provide a dense RSI signal

We observe that trajectories from multi-agent systems can be more effective than trajectories from single agents in RSI. Multi-agent execution also changes the structure of the training data. The orchestrator's conversation shows how to divide a broad task into focused assignments and coordinate their execution. A subagent's conversation shows how to carry out an individual assignment. Specifically, we observe that jointly learning coordination from the orchestrator and bounded execution from subagents provides more effective supervision per training token than learning from long single-agent trajectories (Figure 6).

We compare students trained independently from the same base model on single-agent trajectories (*S*, S+, S++), multi-agent trajectories (*M*, M+), and a mixture (*X*). A plus sign denotes a larger training set; M+ is the first-round MASS model. All points below report results pooled across four training tasks and one test task.

Figure 6. Performance per supervised training token. Multi-agent students lie above the line joining the observed single-agent results. Scores pool four shared training tasks and one test task. Token counts include supervised targets processed up to the selected checkpoint, including repeated sampling; they exclude data generation, search, and evaluation.

Against the base model, M+ reaches a 68.3% win rate after processing 28.8M supervised training tokens, while S++ reaches 64.0% after 42M tokens. The MASS model thus achieves the higher score against this reference with 31% fewer supervised tokensThe first-round dataset stores 17.2M supervised tokens. Sampling with replacement and balancing orchestrator and subagent examples produce 28.8M processed supervised tokens through the selected checkpoint. This measures exposure to training targets, including repeated examples. It excludes later updates, input tokens, validation, data generation, search, and evaluation..

Higher-quality trajectories may contribute to this advantage. In a separate analysis, strong external judges prefer the multi-agent systems' work in 85.1% of comparisons with single-agent outputs. Team runs also generate about 2.4× as many tokens on averageAmong ten pairs with similar generated trace lengths, the preference for team outputs remains 80%. This comparison matches observed lengths, not generation budgets. The groups also differ in workflow and verification instructions, so the comparison does not isolate the effect of delegation. These external judgments are used for this analysis only and do not guide MASS.. But when we compare the supervised tokens used during SFT, we find that multi-agent trajectories can provide a denser learning signal than single-agent trajectories.

These results suggest that trajectories from multi-agent systems can be more effective than trajectories from single agents in RSI, extending their value beyond test-time scaling.

Observation 3. The RSI bottleneck can be the optimizer, not the evaluator

We observed that the bottleneck in RSI can be the optimizer, not the evaluator. We kept the base model as the task solver and swapped in a stronger external model (GPT-5.5) for the evaluator only, or for both the evaluator and the optimizer.

Figure 7. Workflow-search ablations. Number of tasks where the evolved workflow beats the bare base model, by search iteration. The executor is always the base model (\mathcal{L}^{(0)}). Upgrading both the evaluator and optimizer to GPT-5.5 (orange, dashed) finds winning workflows for all 12 tasks within 5 iterations. Upgrading only the evaluator (green) helps much less. The fully homogeneous loop (blue) is slower but still reaches 11 of 12 tasks.

Figure 7 shows that upgrading only the evaluator helps a little, but upgrading the optimizer as well helps a lot. The strictly homogeneous self-improvement loop does eventually find winning workflows for 11 of 12 tasks; it just takes longer. Our interpretation is that the strong evaluator may provide high-quality feedback, but the weak optimizer is not capable enough to digest it. This suggests that, under a limited budget, one may have to prioritize the optimizer over the evaluator within an RSI loop unit.

Figure 8. Workflow-search stability. Win rate of the base model executing each candidate workflow against the same model without a workflow, by search iteration. Colors match the configurations in Figure 7. Strong iterations can be followed by sharp performance regressions.

Figure 8 also shows that workflow search is noisy. In text-space optimization, reaching a good workflow and improving on it stably are different problems. This suggests that the LLM optimizer may need an equivalent of a learning-rate scheduler to stabilize the search in text space.

Conclusion

In Tennyson's Ulysses, Odysseus imagines another voyage in pursuit of knowledge “Beyond the utmost bound of human thought.” For AI, we imagine an Odyssean moment when continued learning must proceed without a stronger supervisor. The challenge concerns both who can supervise further learning and how progress should be judged. Human experts may be able to state a goal without specifying an objective function that fully captures it.

Writing is a good example: expert writers can recognize strong writing, yet struggle to express their judgment in a rubric that applies reliably across contexts. For RSI, the research question is how AI can turn incomplete human guidance into useful criteria for learning, and how to test whether those criteria continue to capture the intended goal as its capabilities grow. This becomes especially challenging when the model being improved also serves as its own optimizer and evaluator: the judgments that guide improvement must themselves remain open to correction.

Figure 9. Illustration for Tennyson's Ulysses (1857). Drawing by Clarkson Stanfield; engraving by W. J. Linton. Scan by George P. Landow, The Victorian Web.

Keeping a system's self-evaluation open to correction is also essential for safety. In a homogeneous loop, a shared blind spot could allow a flawed result to be produced, accepted as an improvement, and incorporated into the next model. If humans also struggle to recognize the flaw, successive updates may reinforce it. The safety concern is that the same weakness can compromise both the system's behavior and the mechanism meant to correct it.

In this sense, shared blind spots may be an Achilles' heel of recursive self-improvement. Understanding them matters for controlling a more capable system because it reveals where reliance on the system's own judgments is least justified. That knowledge could guide independent checks, constraints on autonomous actions, and conditions for intervention. Identifying a weakness alone does not establish control; we must also show that we can detect its consequences and intervene effectively as the system changes.

OpenAI's research on chain-of-thought monitorability illustrates one part of this challenge: reasoning traces can help reveal misbehavior, but they provide incomplete evidence. Recent analysis of a possible intelligence explosion gives further reason to study these conditions early. This is a central safety question for AI's Odyssean moment: how to preserve meaningful oversight when no stronger supervisor is available. ⛵

Citation

@misc{lee2026mass,
title = {Recursive Self-Improvement through Multi-Agent Self-Supervision},
author = {Lee, Hyunin and Xu, Jinglue and Seely, Jeffrey and Lee, Donghyun and Sojoudi, Somayeh and Zaharia, Matei and Tang, Yujin},
year = {2026},
eprint = {2610.12176},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.12176}
}