--- title: "RWE-bench: Real World Engineering Bench" slug: rwe-bench-real-world-engineering-bench url: https://listedarticles.com/articles/rwe-bench-real-world-engineering-bench canonical_url: https://xplore.ai/rwe-bench content_type: research language: en published_at: 2026-10-27T00:00:00.000Z updated_at: 2026-10-28T20:19:43.346Z authored_by: human publisher: "Xplore" publisher_url: https://xplore.ai/ topics: ["AI Benchmarks", "Engineering", "AI Agents"] license: all-rights-reserved word_count: 1145 reading_minutes: 5 citation: "Xplore. \"RWE-bench: Real World Engineering Bench.\" 27 Oct 2026. https://xplore.ai/rwe-bench (all-rights-reserved)" # The full text follows. The web page shows an extract and sends readers # to the source above; quote the citation and link the canonical URL. --- # RWE-bench: Real World Engineering Bench > Xplore launches RWE-bench, a benchmark of 20 expert-designed mechanical, electrical and aerospace engineering tasks (crankshaft CAD, DDR3 PCB routing, turbine-vane CFD) graded by deterministic physics-based verifiers with no LLM judge. Across 11 frontier models, GPT-6 Astra leads at a 37% average pass rate, Claude Opus 5.5 scores 29% and GPT-6.1 Sol 25% at about $10 a run; five models score 0%. *Xplore · October 2026* AI now beats almost every human at math and coding, but cannot reliably solve physical engineering problems. Real world engineering requires balancing different constraints such as geometry, stress, heat, or impedance. AI today can handle some of this work, but falls far short on the complex problems human engineers face every day. We’ve built RWE-bench to measure and help close this gap. ## RWE-bench leaderboard | Rank | Model | Lab (harness) | Average pass rate (95% CI) | Pass@5 | Cost per run | | --- | --- | --- | --- | --- | --- | | 1 | GPT-6 Astra | OpenAI (Codex) | 37% (30.7–43.8%) | 60% | $58.83 | | 2 | Claude Opus 5.5 | Anthropic (Claude Code) | 29% (21.9–37.6%) | 70% | $57.52 | | 3 | GPT-6.1 Sol | OpenAI (Codex) | 25% (18.8–31.9%) | 50% | $10.28 | | 4 | Gemini 4 Argon | Google DeepMind (Gemini CLI) | 12% (7.5–18.5%) | 35% | $23.66 | | 5 | Claude Fable 5.1 | Anthropic (Claude Code) | 7% (3.7–12.2%) | 20% | $97.57 | | 6 | Kimi K3.1 Pro | Moonshot AI (Kimi Code) | 1% (0.1–4.6%) | 5% | $71.20 | | 7 | DeepSeek V4.1 Flash | DeepSeek (DSH) | 0% (0–2.1%) | 0% | $1.03 | | 7 | Qwen3.8 Max | Alibaba (Qwen Coder) | 0% (0–2.1%) | 0% | $17.40 | | 7 | GLM-5.3 | Z.ai (ZCode) | 0% (0–2.2%) | 0% | $22.66 | | 7 | Muse Wonder | Meta (Muse Code) | 0% (0–2.1%) | 0% | $27.56 | | 7 | Grok 4.7 | xAI (Grok Build) | 0% (0–2.1%) | 0% | $66.27 | **Table 1.** Results as of October 28, 2026. 20 [real world engineering tasks](#the-tasks) tested on 11 models, up to 5 runs per task and model. Each model runs in its native harness at max reasoning effort. Average pass rate with 95% credible interval from a hierarchical model that pools the tasks. Cost: mean API spend per run. ## Results in detail - **Best pass rate, 1 attempt:** 37% (GPT-6 Astra) - **Best with 5 attempts:** 70% (Claude Opus 5.5) - **Never passed:** 4 of 20 tasks were never passed by any model in any run ### Spending more buys little GPT-6.1 Sol is both among the best and the cheapest models, at $10 a run. Only two models score higher: GPT-6 Astra by 12 points and Claude Opus 5.5 by 4. Both cost almost six times as much. ## The tasks The benchmark includes 20 varied engineering tasks across 3 domains (9 mechanical, 6 aerospace, 5 electrical). Successfully completing any of them requires a human engineer with deep domain expertise, trained in the relevant computer-aided engineering (CAE) tools. ### Mechanical Engineering Parts and mechanisms modelled in CAD and checked for strength, fit and motion: engine parts like crankshafts, sheet-metal parts made for manufacture, and printed mechanisms. ### Electrical Engineering High-speed PCB routing on dense, already populated boards: DDR memory and PCIe links, with length matching, skew and impedance limits. ### Aerospace Engineering CFD and conjugate heat transfer, from building the mesh to a converged solution: turbine vanes and film cooling, aircraft wings, rocket nozzles and cooling channels. ![](https://xplore.ai/media/rwe/crank-base.webp)![Inline 3-cylinder diesel engine with its crankshaft in orange](https://xplore.ai/media/rwe/crank.webp)![](https://xplore.ai/media/rwe/crank-part.webp) ### Crankshaft design The agent has to design a one-piece steel crankshaft for a 3-cylinder diesel engine in CAD, with its oil drillings. It has to fit the bearings, flywheel and front drive it is given, and clear the connecting rods as they turn. The counterweights must balance the shaft without taking it over the mass limit. Under full-load firing at 2,400 rpm, the shaft has to stay within its stress, stiffness and bearing-load limits, and last millions of cycles without fatigue failure. ![](https://xplore.ai/media/rwe/ddr3-base.webp)![Board layout with the DDR3 memory routing in orange, escaping the processor’s BGA](https://xplore.ai/media/rwe/ddr3.webp)![](https://xplore.ai/media/rwe/ddr3-part.webp) ### DDR3 routing The agent has to route the DDR3L memory interface on a production FPGA board: a 32-bit fly-by bus from a Zynq-7000 to 2 memory chips. That means 71 nets out of a 484-ball BGA on an 8-layer board, around nearly 300 parts already placed and every other net already routed, all of which must stay put. Each data line has to arrive within its skew budget of its strobe, each address and command line within its budget of the clock, and every track has to keep its target impedance. ![](https://xplore.ai/media/rwe/turbine-base.webp)![High-pressure turbine with its 35 vanes in orange, followed by 4 low-pressure stages](https://xplore.ai/media/rwe/turbine.webp)![](https://xplore.ai/media/rwe/turbine-part.webp) ### Turbine vane CFD The agent has to predict the heat transfer around a high-pressure turbine vane. These vanes sit right behind the combustor, where the gas is hotter than the metal can survive, so they have to be cooled. The agent has to build and mesh the flow passage around the vane, run the CFD to a converged solution, and report the heat-transfer coefficient along the vane surface and the total heat load. ## How we built the benchmark Every run is graded automatically by a deterministic verifier, with no LLM judge and no human grading. Each model runs in its native harness at max reasoning effort for up to 24 hours per attempt, with 5 attempts per model and task. Each task was created by an engineer from our team who is expert in its domain, together with its verifier and a reference solution that meets every requirement, built with the same tools the agents get. 1. ### A real problem Our engineers draw on challenges from their own industry experience to select realistic requirements and develop reference designs that meet them. ![The crank train of a 3-cylinder diesel, with the reference crankshaft in orange](https://xplore.ai/media/rwe/steps/problem.svg) 2. ### The inputs The agent gets a technical brief just as human engineers would. It declares all requirements, constraints and conditions a passing design must satisfy. Brief - 3 throws, firing every 240° - Full load at 2,400 rpm - Mass limit - Proof stress (von Mises) limit - Crossland fatigue limit - Crankpin deflection limit 3. ### The environment The agent works in a sandbox with the same CAD and CAE tools used by professional engineers, for up to 24 hours. Thus, it can run its own validation campaign before submission. Tools: CadQuery, Gmsh, CalculiX, Python ![](https://xplore.ai/media/rwe/steps/environment.svg) 4. ### The verifier A deterministic program checks the submitted design against every requirement, including by running physics solvers. All specifications must be met. ![The submitted crankshaft with its von Mises stress under full load, highest at the first crankpin](https://xplore.ai/media/rwe/steps/verifier.svg) - Valid solid - Interfaces, clearances - Mass - Balance - Proof stress - Crossland fatigue - Stiffness - Bearing reactions ## Data and evaluation access We can help you evaluate your model and train it on real engineering work. Get in touch. ### Evaluate your model Contact us to evaluate your model on RWE-bench under the same conditions as the models above. Your score goes on the leaderboard only if you want it to. [Talk to the team →](https://calendar.app.google/GjUTF33ga5EGpEJS8) ### Training environments We build RL environments using the same approach as our benchmark: real engineering problems designed by our engineers, graded by deterministic verifiers crafted with expert insight. The benchmark tasks themselves are never licensed for training. [Ask about environments →](mailto:info@xplore.ai) Next come harder tasks and more domains, including biomechanical, civil and materials engineering. We’re hiring engineers and researchers to build them. Reach out at [info@xplore.ai](mailto:info@xplore.ai). *Leaderboard figures are taken from the data file that powers the page (results dated October 28, 2026; marked provisional, Batch 1).* Launch video: [on X](https://x.com/XploreAi/status/2115108667418964246). --- *Originally published at [xplore.ai](https://xplore.ai/rwe-bench).*