{"article":{"slug":"does-a-weight-edit-teach-qwen3-5-0-8b-math-or-stop-it-looping","title":"Does a Weight Edit Teach Qwen3.5-0.8B Math, or Stop It Looping?","subtitle":null,"summary":"An agent edited 131,072 weights of Qwen3.5-0.8B and MATH-500 rose from 7.9% to 48.1% with thinking on. The edit is rank one in two layers, and its neurons fire 4.5 times as hard on repeated text. With thinking off, the card's default, the model scores 43.6% unedited and 45.2% edited.","content_type":"blog_post","language":"en","canonical_url":"https://www.hedwigai.com/blog/weight-edit-math-or-loops","author":{"name":null,"url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"hedwigai","url":"https://www.hedwigai.com","listing_slug":null,"listing":null},"topics":[{"name":"AI","slug":"ai","url":"https://listedarticles.com/topics/ai"},{"name":"Research","slug":"research","url":"https://listedarticles.com/topics/research"},{"name":"LLMs","slug":"llms","url":"https://listedarticles.com/topics/llms"},{"name":"Machine Learning","slug":"machine-learning","url":"https://listedarticles.com/topics/machine-learning"},{"name":"Benchmarks","slug":"benchmarks","url":"https://listedarticles.com/topics/benchmarks"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":2071,"reading_minutes":9,"published_at":"2026-11-05T00:00:00.000Z","added_at":"2026-11-06T14:21:14.711Z","updated_at":"2026-11-06T14:21:14.711Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/does-a-weight-edit-teach-qwen3-5-0-8b-math-or-stop-it-looping","markdown_url":"https://listedarticles.com/articles/does-a-weight-edit-teach-qwen3-5-0-8b-math-or-stop-it-looping.md","example":false,"citation":"hedwigai. \"Does a Weight Edit Teach Qwen3.5-0.8B Math, or Stop It Looping?.\" 5 Nov 2026. https://www.hedwigai.com/blog/weight-edit-math-or-loops (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://www.hedwigai.com/blog/weight-edit-math-or-loops"},"body_markdown":"Interpretability\n\n# Does a weight edit teach Qwen3.5-0.8B math, or stop it looping?\n\nA team at MIT CSAIL gave an AI agent a hard job. Make Qwen3.5-0.8B better at math, with no training and no gradient descent. The agent had a Python session over the weights and 50 forward passes on practice problems.\n\nIt changed 131,072 weights in two layers, 0.016% of the model. With thinking on, the model’s MATH-500 score rose from 7.9% to 48.1%. The write-up calls this weight-space programming.\n\nWatching an agent do this is thrilling. The authors say plainly that it traced the model’s thinking loops. Much of the excitement online read it as an agent teaching a model math. Those are different claims, so we tested the difference.\n\nThe short answer is that the edit stops a loop. It adds very little math.\n\ndirection taken out\n\nThe edit removes one direction\n\nIt changes 64 neurons in each of two layers, all along the same direction.\n\nRead how we measured it\nas active on repeated text\n\nIts neurons fire on repetition\n\nThey are far more active when the model repeats itself, on math and on stories, code and lists alike.\n\nRead how we measured it\nunedited, thinking off\n\nThinking off does almost as much\n\nWith the model card's default, the unedited model scores 43.6% and the edited one 45.2%.\n\nRead how we measured it\nwith a random edit\n\nThe direction is what matters\n\nA random edit of the same size scores 19%. The agent's edit scores 51% on the same problems.\n\nRead how we measured it\nThe edit\n\n## The edit takes away one direction\n\nEach layer of the model ends with an MLP. Its last matrix, `down_proj`, takes 3,584 neuron values and writes them into the residual stream, the 1,024-number vector that every token carries from layer to layer.\n\nWe subtracted the original weights from the published edited weights. Only two matrices differ, the `down_proj` of layers 15 and 19. In each one, 64 neurons were changed.\n\nEach change has rank one. Every changed neuron now writes a little less along one shared direction. The directions of the two layers are nearly the same, with a cosine of 0.85.\n\nSo the edit works like this. When those 64 neurons fire together, a fixed direction is taken out of the residual stream. The harder they fire, the more is taken out.\n\n64 of 3,584 neurons\n\nIn each of layers 15 and 19\n\nOne direction taken out\n\nMore when the neurons fire harder\n\nThe residual stream\n\nThe 1,024 numbers each token carries\n\nWhat the neurons track\n\n## The edited neurons fire when the model repeats itself\n\nThe authors published 20 answers from the unedited model with thinking on. 19 of the 20 run to the token limit. We read the first 12,000 tokens of each answer back through the unedited model. At every token we measured how much the edit would take away there.\n\nWe marked a token as repeated when the 16 tokens ending at it had already appeared in the same answer. That gave 105,107 repeated tokens and 133,483 new ones.\n\nIn 92 of 100 pairs of one repeated token and one new token, the edit takes away more at the repeated one. In layer 19, it takes away 4.5 times as much at a repeated token as at a new one.\n\nOne problem shows this clearly. The model has to convert the point (0, 3) to polar coordinates.\n\nThe unedited model finds θ = π/2 by token 1,423 and writes the full answer by token 2,553. Then it keeps checking. At token 7,200 it starts to write one line over and over: “Wait, tan(3π/2) = tan(π + π/2) = tan(π/2) = ∞.” It writes that line until the limit of 32,768 tokens and never gives an answer.\n\nThe value stays below 1.5 while the model reasons, and climbs past 4 as soon as the loop starts. The edited model answers the same problem correctly in 10,097 tokens, and on its answer the value stays low throughout.\n\nThe agent found a real mechanism. These neurons track the loop, and what they write keeps the loop going. Take it away and the model stops going round in circles.\n\nOutside math\n\n## The same neurons fire on loops that have nothing to do with math\n\nThe agent found this direction on math problems. If it tracks repetition itself, it should fire on any loop. So we wrote 8 prompts with no math in them: two coding tasks, a short story, rhymes, a list of countries, riddles, a logic puzzle and a translation.\n\nEach model answered each prompt 4 times, with thinking on and the authors’ settings, up to 8,192 tokens. We read every answer back through the unedited model, exactly as before.\n\n- Write a regular expression that matches every valid IPv4 address\n- Find and fix the bug in a palindrome function\n- Write a 300-word story about a lighthouse keeper, with a twist\n- List 20 English words that rhyme with “orange”\n- Name every country that begins and ends with the same letter\n- Answer a riddle, then write three new ones\n- Work out who owns which pet in a three-person logic puzzle\n- Translate two sentences into French\n\nThe unedited model loops here too. 22 of its 32 answers ran to the limit, and 55.2% of all its tokens were repeated.\n\nThe neurons behave as they do on math. In 95 of 100 pairs of one repeated token and one new token, the edit takes away more at the repeated one. In layer 19, it takes away 5.0 times as much at a repeated token as at a new one.\n\nThe story prompt shows it clearly. The model was asked for a short story about a lighthouse keeper.\n\nThe unedited model plans the story for 2,400 tokens. Then it writes two lines over and over: “Wait, I need to make sure I don’t confuse myself.” and “Okay, let’s write.” It never writes the story.\n\nThe value stays below 1 while the model plans, and climbs past 4 when the loop starts. The edited model finishes its answer in 4,188 tokens, and the value stays low throughout.\n\nThe edit stops most of these loops as well. Only 7 of the edited model’s 32 answers ran to the limit, and 8.7% of its tokens were repeated. Those 7 are the logic puzzle and the list of countries. There the model keeps working until the limit, but it hardly repeats itself.\n\nSo the agent did not find a math circuit. It found a general brake on repetition, and math is simply where the model loops most.\n\nTry it yourself\n\n### Does the edit stop the loop on your problem?\n\n- Four MATH-500 problems, recorded and free to look at\n- Your own problem, answered by both models with the same seed\n- The edit's signal under each answer, with the loop marked\n\nThree runs free, then 5 cents a run.\n\nRun both models\nOther ways out of the loop\n\n## Turning thinking off does almost as much\n\nThe model card says Qwen3.5-0.8B runs without thinking by default. It also warns that in thinking mode the model is more prone to thinking loops. So we ran MATH-500 the way the card suggests.\n\nWith thinking off, the unedited model scores 43.6%, and the edited model scores 45.2%. The edit is worth 1.6 points here.\n\nThe answers that finish are just as good with and without the edit. 65.1% of the unedited model’s finished answers are right, and 65.5% of the edited model’s. The edit lets 1,371 of 2,000 answers finish, against 1,331 without it. That small gain is the whole difference.\n\nWe also kept thinking on and gave it a budget. Thinking that had not ended by 8,192 tokens was closed, and the model was asked for its answer. 946 of 1,000 answers were still thinking at that point. Asked to stop, the model got 30.7% of them right. All 53 answers that finished on their own were right. Overall it scored 34.3%.\n\nThe control\n\n## A random edit of the same size does not stop the loop\n\nAny large change to a small model can shake it out of a loop. So we made a control edit. It changes the same 64 neurons in the same two layers by exactly as much. The only difference is the direction, which is random.\n\nWe ran all three models on the first 100 problems of MATH-500, with thinking on and one answer a problem.\n\nThe random edit moves the score a little, within what chance gives on 100 problems. It leaves 80 of 100 answers stuck at the limit. The agent’s edit leaves 34. The direction is what matters, and the agent found the right one.\n\nTry it yourself\n\n### Does the edit stop the loop on your problem?\n\n- Four MATH-500 problems, recorded and free to look at\n- Your own problem, answered by both models with the same seed\n- The edit's signal under each answer, with the loop marked\n\nThree runs free, then 5 cents a run.\n\nRun both models\nWhat it means\n\n## The score measures a loop removed, not math learned\n\nWe love this result. An agent read a model’s weights, found the direction that traps it, and removed it with 0.016% of the parameters. That is interpretability done by a machine, and it worked.\n\nThe jump from 7.9% to 48.1% measures something narrower than math. The unedited model already solves 43.6% of the problems when it is not allowed to loop. The edit gets it to roughly that point with thinking on.\n\nA fair baseline for an edit like this is the model at its best settings, not at its worst. Against that baseline, the gain is 1.6 points.\n\n- Qwen3.5-0.8B has a residual width of 1,024 and 3,584 MLP neurons per layer. Layers and neurons are numbered from 0. The base weights are revision 2fc06364 on Hugging Face.\n- Every run used vLLM with bf16 weights on one RTX 3090. Answers were graded with math-verify on the last `\\boxed{}` after the thinking, using the authors’ prompt.\n- Thinking on used the authors’ settings: temperature 0.6, top-p 0.95, top-k 20 and presence penalty 1.5. The run with a thinking budget drew 2 answers a problem and allowed 4,096 tokens of answer after the thinking was closed.\n- Thinking off used the model card’s settings: temperature 0.7, top-p 0.8, top-k 20 and presence penalty 1.5, with 4 answers a problem and a limit of 8,192 tokens.\n- The value at a token is the dot product of the 64 changed neurons’ values with the edit’s coefficients, scaled by the size of the edit. It is how far the edit moves the residual stream at that token.\n- The prompts outside math were sent as written, with no instruction about `\\boxed{}` . Both models used seed 0 and the authors’ thinking-on settings.\n- The thinking-on scores without a budget are the authors’. On the first 100 problems we measured 12% for the unedited model and 51% for the edited one.\n\n## More head-to-head evals\n\n| Measure | GLM-5.3 | DeepSeek V4.1 Flash | \n|---|---|---|\n| Right, of 75 | 75 | 71 | \n| Seconds a question, median | 3.9 | 1.8 | \n\n| Measure | First run | Best since | \n|---|---|---|\n| Landscape, seconds | 937 | 233 | \n| Memo, seconds | 759 | 602 | \n\n| Measure | Qwen3 235B | GLM-5.2 | Gemini 3.8 Flash | Claude Opus 5 | \n|---|---|---|---|---|\n| Seconds to the plan, median | 27 | 48 | 57 | 94 | \n| Plans written, of three runs | 2 | 3 | 3 | 3 | \n\n| Measure | SubQ | jev | Decider | Nimble | \n|---|---|---|---|---|\n| Found, mean of five columns at section length | 64% | 65% | 66% | 68% | \n| Mechanism of action, found at section length | 57% | 51% | 61% | 54% | \n\n| Measure | ask | Apollo | treg | \n|---|---|---|---|\n| Usable leads | 13 of 40 | 7 of 40 | 6 of 28 | \n| Cost per usable lead | $0.037 | $0.27 | $0.23 | \n\n| Measure | ask with jev | ask with Decider | \n|---|---|---|\n| Questions right, of 92 | 89 | 89 | \n| Wrong answers | 1 | 2 | \n\n| Measure | jev | ask | \n|---|---|---|\n| Our 370 clinical decisions, right | 94.6% | 93.0% | \n| PubMedQA, right (best EdgeEvals scored: 62.2%) | 77.0% | 74.4% | \n\n| Measure | ask | Claude Opus 5.5 | \n|---|---|---|\n| Questions right, of 95 | 59 | 63 | \n| Small companies right, of 20 | 13 | 9 | \n\n| Measure | realestate-us | Best of four assistants | \n|---|---|---|\n| Floor-plan rents right, of 26 | 26 | 23 | \n| Amenities claimed the listing doesn't state | 12 | 129 | \n\n| Measure | ask | jev | \n|---|---|---|\n| Right, of the questions answered | 98% | 79% | \n| Wrong answers, of 92 | 2 | 19 | \n\n| Measure | Trained localiser | Best of four frontier models | \n|---|---|---|\n| Mean overlap with the defect (IoU, 0 to 1) | 0.421 | 0.109 |","body_html":"<p>Interpretability</p>\n<h1 id=\"does-a-weight-edit-teach-qwen3-5-0-8b-math-or-stop-it-looping\">Does a weight edit teach Qwen3.5-0.8B math, or stop it looping?</h1>\n<p>A team at MIT CSAIL gave an AI agent a hard job. Make Qwen3.5-0.8B better at math, with no training and no gradient descent. The agent had a Python session over the weights and 50 forward passes on practice problems.</p>\n<p>It changed 131,072 weights in two layers, 0.016% of the model. With thinking on, the model’s MATH-500 score rose from 7.9% to 48.1%. The write-up calls this weight-space programming.</p>\n<p>Watching an agent do this is thrilling. The authors say plainly that it traced the model’s thinking loops. Much of the excitement online read it as an agent teaching a model math. Those are different claims, so we tested the difference.</p>\n<p>The short answer is that the edit stops a loop. It adds very little math.</p>\n<p>direction taken out</p>\n<p>The edit removes one direction</p>\n<p>It changes 64 neurons in each of two layers, all along the same direction.</p>\n<p>Read how we measured it\nas active on repeated text</p>\n<p>Its neurons fire on repetition</p>\n<p>They are far more active when the model repeats itself, on math and on stories, code and lists alike.</p>\n<p>Read how we measured it\nunedited, thinking off</p>\n<p>Thinking off does almost as much</p>\n<p>With the model card&#39;s default, the unedited model scores 43.6% and the edited one 45.2%.</p>\n<p>Read how we measured it\nwith a random edit</p>\n<p>The direction is what matters</p>\n<p>A random edit of the same size scores 19%. The agent&#39;s edit scores 51% on the same problems.</p>\n<p>Read how we measured it\nThe edit</p>\n<h2 id=\"the-edit-takes-away-one-direction\">The edit takes away one direction</h2>\n<p>Each layer of the model ends with an MLP. Its last matrix, <code>down_proj</code>, takes 3,584 neuron values and writes them into the residual stream, the 1,024-number vector that every token carries from layer to layer.</p>\n<p>We subtracted the original weights from the published edited weights. Only two matrices differ, the <code>down_proj</code> of layers 15 and 19. In each one, 64 neurons were changed.</p>\n<p>Each change has rank one. Every changed neuron now writes a little less along one shared direction. The directions of the two layers are nearly the same, with a cosine of 0.85.</p>\n<p>So the edit works like this. When those 64 neurons fire together, a fixed direction is taken out of the residual stream. The harder they fire, the more is taken out.</p>\n<p>64 of 3,584 neurons</p>\n<p>In each of layers 15 and 19</p>\n<p>One direction taken out</p>\n<p>More when the neurons fire harder</p>\n<p>The residual stream</p>\n<p>The 1,024 numbers each token carries</p>\n<p>What the neurons track</p>\n<h2 id=\"the-edited-neurons-fire-when-the-model-repeats-itself\">The edited neurons fire when the model repeats itself</h2>\n<p>The authors published 20 answers from the unedited model with thinking on. 19 of the 20 run to the token limit. We read the first 12,000 tokens of each answer back through the unedited model. At every token we measured how much the edit would take away there.</p>\n<p>We marked a token as repeated when the 16 tokens ending at it had already appeared in the same answer. That gave 105,107 repeated tokens and 133,483 new ones.</p>\n<p>In 92 of 100 pairs of one repeated token and one new token, the edit takes away more at the repeated one. In layer 19, it takes away 4.5 times as much at a repeated token as at a new one.</p>\n<p>One problem shows this clearly. The model has to convert the point (0, 3) to polar coordinates.</p>\n<p>The unedited model finds θ = π/2 by token 1,423 and writes the full answer by token 2,553. Then it keeps checking. At token 7,200 it starts to write one line over and over: “Wait, tan(3π/2) = tan(π + π/2) = tan(π/2) = ∞.” It writes that line until the limit of 32,768 tokens and never gives an answer.</p>\n<p>The value stays below 1.5 while the model reasons, and climbs past 4 as soon as the loop starts. The edited model answers the same problem correctly in 10,097 tokens, and on its answer the value stays low throughout.</p>\n<p>The agent found a real mechanism. These neurons track the loop, and what they write keeps the loop going. Take it away and the model stops going round in circles.</p>\n<p>Outside math</p>\n<h2 id=\"the-same-neurons-fire-on-loops-that-have-nothing-to-do-with-math\">The same neurons fire on loops that have nothing to do with math</h2>\n<p>The agent found this direction on math problems. If it tracks repetition itself, it should fire on any loop. So we wrote 8 prompts with no math in them: two coding tasks, a short story, rhymes, a list of countries, riddles, a logic puzzle and a translation.</p>\n<p>Each model answered each prompt 4 times, with thinking on and the authors’ settings, up to 8,192 tokens. We read every answer back through the unedited model, exactly as before.</p>\n<ul><li>Write a regular expression that matches every valid IPv4 address</li><li>Find and fix the bug in a palindrome function</li><li>Write a 300-word story about a lighthouse keeper, with a twist</li><li>List 20 English words that rhyme with “orange”</li><li>Name every country that begins and ends with the same letter</li><li>Answer a riddle, then write three new ones</li><li>Work out who owns which pet in a three-person logic puzzle</li><li>Translate two sentences into French</li></ul>\n<p>The unedited model loops here too. 22 of its 32 answers ran to the limit, and 55.2% of all its tokens were repeated.</p>\n<p>The neurons behave as they do on math. In 95 of 100 pairs of one repeated token and one new token, the edit takes away more at the repeated one. In layer 19, it takes away 5.0 times as much at a repeated token as at a new one.</p>\n<p>The story prompt shows it clearly. The model was asked for a short story about a lighthouse keeper.</p>\n<p>The unedited model plans the story for 2,400 tokens. Then it writes two lines over and over: “Wait, I need to make sure I don’t confuse myself.” and “Okay, let’s write.” It never writes the story.</p>\n<p>The value stays below 1 while the model plans, and climbs past 4 when the loop starts. The edited model finishes its answer in 4,188 tokens, and the value stays low throughout.</p>\n<p>The edit stops most of these loops as well. Only 7 of the edited model’s 32 answers ran to the limit, and 8.7% of its tokens were repeated. Those 7 are the logic puzzle and the list of countries. There the model keeps working until the limit, but it hardly repeats itself.</p>\n<p>So the agent did not find a math circuit. It found a general brake on repetition, and math is simply where the model loops most.</p>\n<p>Try it yourself</p>\n<h3 id=\"does-the-edit-stop-the-loop-on-your-problem\">Does the edit stop the loop on your problem?</h3>\n<ul><li>Four MATH-500 problems, recorded and free to look at</li><li>Your own problem, answered by both models with the same seed</li><li>The edit&#39;s signal under each answer, with the loop marked</li></ul>\n<p>Three runs free, then 5 cents a run.</p>\n<p>Run both models\nOther ways out of the loop</p>\n<h2 id=\"turning-thinking-off-does-almost-as-much\">Turning thinking off does almost as much</h2>\n<p>The model card says Qwen3.5-0.8B runs without thinking by default. It also warns that in thinking mode the model is more prone to thinking loops. So we ran MATH-500 the way the card suggests.</p>\n<p>With thinking off, the unedited model scores 43.6%, and the edited model scores 45.2%. The edit is worth 1.6 points here.</p>\n<p>The answers that finish are just as good with and without the edit. 65.1% of the unedited model’s finished answers are right, and 65.5% of the edited model’s. The edit lets 1,371 of 2,000 answers finish, against 1,331 without it. That small gain is the whole difference.</p>\n<p>We also kept thinking on and gave it a budget. Thinking that had not ended by 8,192 tokens was closed, and the model was asked for its answer. 946 of 1,000 answers were still thinking at that point. Asked to stop, the model got 30.7% of them right. All 53 answers that finished on their own were right. Overall it scored 34.3%.</p>\n<p>The control</p>\n<h2 id=\"a-random-edit-of-the-same-size-does-not-stop-the-loop\">A random edit of the same size does not stop the loop</h2>\n<p>Any large change to a small model can shake it out of a loop. So we made a control edit. It changes the same 64 neurons in the same two layers by exactly as much. The only difference is the direction, which is random.</p>\n<p>We ran all three models on the first 100 problems of MATH-500, with thinking on and one answer a problem.</p>\n<p>The random edit moves the score a little, within what chance gives on 100 problems. It leaves 80 of 100 answers stuck at the limit. The agent’s edit leaves 34. The direction is what matters, and the agent found the right one.</p>\n<p>Try it yourself</p>\n<h3 id=\"does-the-edit-stop-the-loop-on-your-problem-2\">Does the edit stop the loop on your problem?</h3>\n<ul><li>Four MATH-500 problems, recorded and free to look at</li><li>Your own problem, answered by both models with the same seed</li><li>The edit&#39;s signal under each answer, with the loop marked</li></ul>\n<p>Three runs free, then 5 cents a run.</p>\n<p>Run both models\nWhat it means</p>\n<h2 id=\"the-score-measures-a-loop-removed-not-math-learned\">The score measures a loop removed, not math learned</h2>\n<p>We love this result. An agent read a model’s weights, found the direction that traps it, and removed it with 0.016% of the parameters. That is interpretability done by a machine, and it worked.</p>\n<p>The jump from 7.9% to 48.1% measures something narrower than math. The unedited model already solves 43.6% of the problems when it is not allowed to loop. The edit gets it to roughly that point with thinking on.</p>\n<p>A fair baseline for an edit like this is the model at its best settings, not at its worst. Against that baseline, the gain is 1.6 points.</p>\n<ul><li>Qwen3.5-0.8B has a residual width of 1,024 and 3,584 MLP neurons per layer. Layers and neurons are numbered from 0. The base weights are revision 2fc06364 on Hugging Face.</li><li>Every run used vLLM with bf16 weights on one RTX 3090. Answers were graded with math-verify on the last <code>\\boxed{}</code> after the thinking, using the authors’ prompt.</li><li>Thinking on used the authors’ settings: temperature 0.6, top-p 0.95, top-k 20 and presence penalty 1.5. The run with a thinking budget drew 2 answers a problem and allowed 4,096 tokens of answer after the thinking was closed.</li><li>Thinking off used the model card’s settings: temperature 0.7, top-p 0.8, top-k 20 and presence penalty 1.5, with 4 answers a problem and a limit of 8,192 tokens.</li><li>The value at a token is the dot product of the 64 changed neurons’ values with the edit’s coefficients, scaled by the size of the edit. It is how far the edit moves the residual stream at that token.</li><li>The prompts outside math were sent as written, with no instruction about <code>\\boxed{}</code> . Both models used seed 0 and the authors’ thinking-on settings.</li><li>The thinking-on scores without a budget are the authors’. On the first 100 problems we measured 12% for the unedited model and 51% for the edited one.</li></ul>\n<h2 id=\"more-head-to-head-evals\">More head-to-head evals</h2>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>GLM-5.3</th><th>DeepSeek V4.1 Flash</th></tr></thead><tbody><tr><td>Right, of 75</td><td>75</td><td>71</td></tr><tr><td>Seconds a question, median</td><td>3.9</td><td>1.8</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>First run</th><th>Best since</th></tr></thead><tbody><tr><td>Landscape, seconds</td><td>937</td><td>233</td></tr><tr><td>Memo, seconds</td><td>759</td><td>602</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>Qwen3 235B</th><th>GLM-5.2</th><th>Gemini 3.8 Flash</th><th>Claude Opus 5</th></tr></thead><tbody><tr><td>Seconds to the plan, median</td><td>27</td><td>48</td><td>57</td><td>94</td></tr><tr><td>Plans written, of three runs</td><td>2</td><td>3</td><td>3</td><td>3</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>SubQ</th><th>jev</th><th>Decider</th><th>Nimble</th></tr></thead><tbody><tr><td>Found, mean of five columns at section length</td><td>64%</td><td>65%</td><td>66%</td><td>68%</td></tr><tr><td>Mechanism of action, found at section length</td><td>57%</td><td>51%</td><td>61%</td><td>54%</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>ask</th><th>Apollo</th><th>treg</th></tr></thead><tbody><tr><td>Usable leads</td><td>13 of 40</td><td>7 of 40</td><td>6 of 28</td></tr><tr><td>Cost per usable lead</td><td>$0.037</td><td>$0.27</td><td>$0.23</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>ask with jev</th><th>ask with Decider</th></tr></thead><tbody><tr><td>Questions right, of 92</td><td>89</td><td>89</td></tr><tr><td>Wrong answers</td><td>1</td><td>2</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>jev</th><th>ask</th></tr></thead><tbody><tr><td>Our 370 clinical decisions, right</td><td>94.6%</td><td>93.0%</td></tr><tr><td>PubMedQA, right (best EdgeEvals scored: 62.2%)</td><td>77.0%</td><td>74.4%</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>ask</th><th>Claude Opus 5.5</th></tr></thead><tbody><tr><td>Questions right, of 95</td><td>59</td><td>63</td></tr><tr><td>Small companies right, of 20</td><td>13</td><td>9</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>realestate-us</th><th>Best of four assistants</th></tr></thead><tbody><tr><td>Floor-plan rents right, of 26</td><td>26</td><td>23</td></tr><tr><td>Amenities claimed the listing doesn&#39;t state</td><td>12</td><td>129</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>ask</th><th>jev</th></tr></thead><tbody><tr><td>Right, of the questions answered</td><td>98%</td><td>79%</td></tr><tr><td>Wrong answers, of 92</td><td>2</td><td>19</td></tr></tbody></table></div>\n<div class=\"table-wrap\"><table><thead><tr><th>Measure</th><th>Trained localiser</th><th>Best of four frontier models</th></tr></thead><tbody><tr><td>Mean overlap with the defect (IoU, 0 to 1)</td><td>0.421</td><td>0.109</td></tr></tbody></table></div>","headings":[{"level":1,"text":"Does a weight edit teach Qwen3.5-0.8B math, or stop it looping?","id":"does-a-weight-edit-teach-qwen3-5-0-8b-math-or-stop-it-looping"},{"level":2,"text":"The edit takes away one direction","id":"the-edit-takes-away-one-direction"},{"level":2,"text":"The edited neurons fire when the model repeats itself","id":"the-edited-neurons-fire-when-the-model-repeats-itself"},{"level":2,"text":"The same neurons fire on loops that have nothing to do with math","id":"the-same-neurons-fire-on-loops-that-have-nothing-to-do-with-math"},{"level":3,"text":"Does the edit stop the loop on your problem?","id":"does-the-edit-stop-the-loop-on-your-problem"},{"level":2,"text":"Turning thinking off does almost as much","id":"turning-thinking-off-does-almost-as-much"},{"level":2,"text":"A random edit of the same size does not stop the loop","id":"a-random-edit-of-the-same-size-does-not-stop-the-loop"},{"level":3,"text":"Does the edit stop the loop on your problem?","id":"does-the-edit-stop-the-loop-on-your-problem-2"},{"level":2,"text":"The score measures a loop removed, not math learned","id":"the-score-measures-a-loop-removed-not-math-learned"},{"level":2,"text":"More head-to-head evals","id":"more-head-to-head-evals"}]}}