I think of these as two different hills, and the uncomfortable part is that we only really know how to climb one of them.

Figure 1. On the weights hill, the measurement and the optimizer are the same object. On the harness hill, a person makes a discrete change and an eval tells them whether it helped.

The weights hill has a gradient

Every method I know of for improving weights has the same basic shape. Some scalar signal comes from somewhere, it gets turned into a gradient, and a step gets taken in that direction, over and over. The only thing that really varies from one method to the next is where that signal comes from:

• Pretraining and supervised fine-tuning use a loss against data.
• RLHF gets its signal from a reward model trained on human preferences (Christiano et al., 2017; Ouyang et al., 2022); DPO is the same idea with the reward model folded into the loss, so there's nothing separate to train (Rafailov et al., 2023).
• RLVR throws the reward model away and uses a program instead: a unit test, a symbolic match against a known answer, whatever can be checked mechanically (Lambert et al., 2024; DeepSeek-AI, 2025).

In all of these the reward is also the eval, and the optimizer takes millions of tiny steps against it with nobody watching. It's a mature discipline. There are loss curves to look at, learning rates to tune, and a long history of optimization theory to lean on when something goes wrong.

None of which makes it safe. Push hard enough on any proxy and it stops meaning what it meant when you started (Gao et al., 2022), and the signal can even detach from capability completely: Shao et al. (2025) got a 21.4-point improvement on MATH-500 out of Qwen2.5-Math-7B using rewards assigned at random, not far off what the real rewards gave. So even on the weights hill, the signal you optimize and the signal you trust had better be two different things.

The harness hill has no gradient

A harness, by contrast, is a handful of discrete, structured choices: a paragraph in a prompt, a tool, a rule about when to stop. There's no way to differentiate through "add a retry." Every step up this hill is a person making a change and then measuring what happened, and there's no loss curve to watch while they do it. The eval is the only thing telling them which way is up.

People have tried to manufacture a gradient, and some of the attempts are clever. ProTeGi forms natural-language "gradients" that criticize a prompt and then edits against them (Pryzant et al., 2023). OPRO uses the model itself as the optimizer (Yang et al., 2023). DSPy compiles whole pipelines against a metric (Khattab et al., 2023), and TextGrad propagates text feedback backwards through a program (Yuksekgonul et al., 2024). Reflexion improves an agent "not by updating weights, but instead through linguistic feedback" (Shinn et al., 2023). What all of them have in common is that they need a metric to optimize. None of them gets rid of the eval; they just turn it into the objective function.

Which is why I'm writing this series. On the harness hill, the eval has to do the job the gradient does on the weights hill. If it's noisy, you take random steps. If it measures the wrong thing, you walk up the wrong hill, and nothing in the process will flag it, because there's no held-out loss to diverge.

Why this hill is new

Agent harnesses as we now know them barely existed before ReAct put a reasoning-and-acting loop around a model in late 2022 (Yao et al., 2022). The paper that convinced me the harness is its own axis was SWE-agent: with the same class of models everyone else had, a redesigned agent-computer interface got to 12.5% pass@1 on SWE-bench, well above what non-interactive use of those models had managed (Yang et al., 2024). Then Kapoor et al. (2024) argued that the field had been drawing the wrong conclusions about where its accuracy gains came from, because nobody was measuring cost alongside accuracy. More recently the Holistic Agent Leaderboard ran 21,730 rollouts across models, scaffolds, and benchmarks and found, among other things, that higher reasoning effort made accuracy worse in the majority of runs (Kapoor et al., 2025).

So the practice of measuring harness changes is maybe three years old, and most of the tooling is younger than that. The Model Context Protocol, which is what standardized how tools get exposed to a model, only came out in late 2024 (Anthropic, 2024).

Who is on which hill

Most teams that build with language models never touch weights at all. They're on the harness hill exclusively, which means the eval is the whole of their optimization machinery, whether or not they think of it that way. Labs climb both hills, and lately the two have been converging: the executable environments used to train coding agents with RL are the same environments people use to evaluate harnesses (Pan et al., 2024; Yang et al., 2025). Part 5 comes back to that.

The rest of the series is about how to build the measurement.

• Measure a workflow, not a system.
• What changes when the model acts.
• Verifiers first, rubrics second.
• The environment is the eval.

Seven more parts are outlined on the series page.

References

Anthropic (2026). Demystifying evals for AI agents. Anthropic Engineering.
Christiano, P. F. et al. (2017). Deep Reinforcement Learning from Human Preferences. NeurIPS.
Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. NeurIPS.
Rafailov, R. et al. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. NeurIPS.
Lambert, N. et al. (2024). Tülu 3: Pushing Frontiers in Open Language Model Post-Training.
DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.
Gao, L., Schulman, J., and Hilton, J. (2022). Scaling Laws for Reward Model Overoptimization. ICML 2023.
Shao, R. et al. (2025). Spurious Rewards: Rethinking Training Signals in RLVR.
Pryzant, R. et al. (2023). Automatic Prompt Optimization with "Gradient Descent" and Beam Search. EMNLP.
Yang, C. et al. (2023). Large Language Models as Optimizers. ICLR 2024.
Khattab, O. et al. (2023). DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. ICLR 2024.
Yuksekgonul, M. et al. (2024). TextGrad: Automatic "Differentiation" via Text.
Shinn, N. et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS.
Yao, S. et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023.
Yang, J. et al. (2024). SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. NeurIPS.
Kapoor, S. et al. (2024). AI Agents That Matter.
Kapoor, S. et al. (2025). Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation.
Anthropic (2024). Model Context Protocol. Specification and documentation.
Pan, J. et al. (2024). Training Software Engineering Agents and Verifiers with SWE-Gym.
Yang, J. et al. (2025). SWE-smith: Scaling Data for Software Engineering Agents.