Publisher
Execution layer for AI agents with spend, credential, and network controls
Introducing FrontierHarness Eval v1.1
Runta and Baseten ran 22 coding-agent harnesses with 5 models on the same 62 tasks, 4 times each (27,280 runs, 83B tokens), to test whether switching models should mean switching harnesses, and found 64% of inference spend went to runs that failed.
11 min · 2,500 words