RWE-bench: Real World Engineering Bench

Table 1. Results as of October 28, 2026. 20 real world engineering tasks tested on 11 models, up to 5 runs per task and model. Each model runs in its native harness at max reasoning effort. Average pass rate with 95% credible interval from a hierarchical model that pools the tasks. Cost: mean API spend per run.

Results in detail

• Best pass rate, 1 attempt: 37% (GPT-6 Astra)
• Best with 5 attempts: 70% (Claude Opus 5.5)
• Never passed: 4 of 20 tasks were never passed by any model in any run

Spending more buys little

GPT-6.1 Sol is both among the best and the cheapest models, at $10 a run. Only two models score higher: GPT-6 Astra by 12 points and Claude Opus 5.5 by 4. Both cost almost six times as much.

The tasks

The benchmark includes 20 varied engineering tasks across 3 domains (9 mechanical, 6 aerospace, 5 electrical). Successfully completing any of them requires a human engineer with deep domain expertise, trained in the relevant computer-aided engineering (CAE) tools.

Mechanical Engineering

Parts and mechanisms modelled in CAD and checked for strength, fit and motion: engine parts like crankshafts, sheet-metal parts made for manufacture, and printed mechanisms.

Electrical Engineering

High-speed PCB routing on dense, already populated boards: DDR memory and PCIe links, with length matching, skew and impedance limits.

Aerospace Engineering

CFD and conjugate heat transfer, from building the mesh to a converged solution: turbine vanes and film cooling, aircraft wings, rocket nozzles and cooling channels.

Crankshaft design

The agent has to design a one-piece steel crankshaft for a 3-cylinder diesel engine in CAD, with its oil drillings. It has to fit the bearings, flywheel and front drive it is given, and clear the connecting rods as they turn. The counterweights must balance the shaft without taking it over the mass limit. Under full-load firing at 2,400 rpm, the shaft has to stay within its stress, stiffness and bearing-load limits, and last millions of cycles without fatigue failure.

DDR3 routing

The agent has to route the DDR3L memory interface on a production FPGA board: a 32-bit fly-by bus from a Zynq-7000 to 2 memory chips. That means 71 nets out of a 484-ball BGA on an 8-layer board, around nearly 300 parts already placed and every other net already routed, all of which must stay put. Each data line has to arrive within its skew budget of its strobe, each address and command line within its budget of the clock, and every track has to keep its target impedance.

Turbine vane CFD

The agent has to predict the heat transfer around a high-pressure turbine vane. These vanes sit right behind the combustor, where the gas is hotter than the metal can survive, so they have to be cooled. The agent has to build and mesh the flow passage around the vane, run the CFD to a converged solution, and report the heat-transfer coefficient along the vane surface and the total heat load.

How we built the benchmark

Every run is graded automatically by a deterministic verifier, with no LLM judge and no human grading. Each model runs in its native harness at max reasoning effort for up to 24 hours per attempt, with 5 attempts per model and task.

Each task was created by an engineer from our team who is expert in its domain, together with its verifier and a reference solution that meets every requirement, built with the same tools the agents get.

1. ### A real problem

   Our engineers draw on challenges from their own industry experience to select realistic requirements and develop reference designs that meet them.

   

2. ### The inputs

   The agent gets a technical brief just as human engineers would. It declares all requirements, constraints and conditions a passing design must satisfy.

   Brief

   • 3 throws, firing every 240°
   • Full load at 2,400 rpm
   • Mass limit
   • Proof stress (von Mises) limit
   • Crossland fatigue limit
   • Crankpin deflection limit

3. ### The environment

   The agent works in a sandbox with the same CAD and CAE tools used by professional engineers, for up to 24 hours. Thus, it can run its own validation campaign before submission.

   Tools: CadQuery, Gmsh, CalculiX, Python

   

4. ### The verifier

   A deterministic program checks the submitted design against every requirement, including by running physics solvers. All specifications must be met.

   

   • Valid solid
   • Interfaces, clearances
   • Mass
   • Balance
   • Proof stress
   • Crossland fatigue
   • Stiffness
   • Bearing reactions

Data and evaluation access

We can help you evaluate your model and train it on real engineering work. Get in touch.

Evaluate your model

Contact us to evaluate your model on RWE-bench under the same conditions as the models above. Your score goes on the leaderboard only if you want it to.

Talk to the team →

Training environments

We build RL environments using the same approach as our benchmark: real engineering problems designed by our engineers, graded by deterministic verifiers crafted with expert insight. The benchmark tasks themselves are never licensed for training.

Ask about environments →

Next come harder tasks and more domains, including biomechanical, civil and materials engineering. We’re hiring engineers and researchers to build them. Reach out at info@xplore.ai.

Leaderboard figures are taken from the data file that powers the page (results dated October 28, 2026; marked provisional, Batch 1). Launch video: on X.

---

Originally published at xplore.ai.