🔍 Read the full analysis: How To Think About AI That Edits Neural Networks Through Their Weights on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Four MIT CSAIL researchers report testing whether OpenAI’s GPT-6 Astra could infer what small neural networks do, repair some failures and create networks by editing their weights directly. The November 2 blog post records promising results, including a Qwen maths-score increase, but the work is not peer-reviewed and its experiments were limited; the approach also failed in some tests.
Four MIT CSAIL researchers have published logs of experiments in which OpenAI’s GPT-6 Astra read and edited neural-network weights directly, reporting that it identified several small networks, repaired some failures and wrote networks from scratch. The work, described in a November 2 blog post as “weight-space programming,” could offer a new way to inspect or modify models, but its results come from limited experiments and have not been peer-reviewed.
The researchers gave Astra the numerical weights of other networks and asked it to infer their functions, change weights when a network failed, or create new networks. The reported method did not use a training loop or gradient descent. In some tests, Astra received weight values as text and had to respond in one reply; in others, researchers ran code it wrote in a sandbox with a fixed test budget. The team said its released logs show that the model had no shell, web search or file-editing tools.
The reported results include a car-control network that had succeeded in 0 of 256 episodes before editing and 1,024 of 1,024 afterward, with 3.5% of its weights changed. In a Qwen3.5-0.8B test, the researchers said an edit affecting 0.016% of weights raised its MATH-500 score from 7.9% to 48.1%. The source article says a Hugging Face engineer reconstructed the edited model from the logs and recorded a 51.0% score on the same benchmark.
The team also reported that Astra identified the games behind four small game-playing networks and the shapes represented by three small image generators. It wrote small game networks and a 9-by-9 Go network from zero; according to the post, the Go network beat a same-size comparison network trained on about 20.5 million games in 50 test runs. These figures describe the reported tests, not broad evidence that the method works reliably across models or tasks.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Model Repair
If the approach can be reproduced and extended, it could give researchers another way to inspect or make targeted changes to small models without retraining them through standard optimization. Direct edits could be useful when a model has a specific, identifiable defect: the car controller and the Qwen loop are examples the researchers say they addressed by changing a small portion of the weights.
The findings also point to boundaries. The source article reports that Astra could not distinguish an honest model from a sycophantic one by examining weights alone, and that edits to a Gemma 3 270M model that did not already loop failed to improve its maths performance. The authors’ stated interpretation was that editing “could not add an ability the model lacked.” That makes the work a possible tool for particular repairs, not a demonstrated substitute for training, evaluation or safety testing.
For organizations using open models, direct weight editing could complicate questions of model provenance and verification: an edited model may behave differently from its original, so users would need to know what changed and test the result. The reported experiments do not establish that this is a practical product or that ordinary users’ models are affected. The source article says there is no immediate change for users.
Products Worth Considering
Logitech MX Brio Ultra HD 4K Webcam, Graphite
FLEXISPOT 60-Inch Executive Standing Desk with Drawers and CPU Cabinet
Dell UltraSharp 32 U3226QER 4K Thunderbolt Hub Monitor
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, computer scientist Andrej Karpathy described neural networks as “Software 2.0”: systems whose behavior is encoded in weights produced through optimization rather than written directly by programmers. The challenge is scale. A network can contain many parameters, and their values are difficult for people to interpret as conventional source code.
The MIT team—Rikiya Takehi and Jayden Teoh, who the post credits as equal contributors, along with Alex Zhang and Omar Khattab—tested whether a general-purpose language model could reason about those parameters. The group’s November 2 post calls this weight-space programming. It argues that language models may eventually edit network weights directly, rather than relying only on symbolic optimizers such as Adam or Muon. That is the researchers’ proposal, not an established change in how neural networks are generally built.
The evidence described in the source is a blog post with public logs, not a peer-reviewed paper or an arXiv preprint. The article says the tests used small networks, one run per setting, one model family per task and a single research group. Those limits matter when judging results that may appear striking in individual demonstrations.
““not through an optimizer, and in particular not with gradient descent””
— MIT CSAIL researchers, in their November 2 blog post
How Far the Results Generalize
The published experiments do not establish how well weight-space programming works on larger networks, different model families or tasks beyond those tested. The source describes one run per setting, small networks and a single research group. It also says the post is not peer-reviewed, so independent replication and scrutiny remain limited.
It is also unclear how consistently the method would work when edits must preserve capabilities beyond the specific benchmark being tested. A higher score on one test does not by itself show that a model’s overall performance, reliability or safety improved. The reconstruction result reported by a Hugging Face engineer offers a check on the Qwen test, but does not validate the other experiments.
Replication and Broader Testing
The next evidence to watch for is independent replication, fuller evaluation across model sizes and tasks, and research that tests whether edits retain other useful capabilities. The source material does not identify a scheduled peer-reviewed paper, release date or follow-up study, so those developments remain unconfirmed.
For now, the public logs let researchers examine the reported experiments, but they do not establish that companies can safely use the technique in production. The central question is whether other teams can reproduce the results and show that direct edits remain dependable outside a small set of demonstrations.
Key Questions
What is weight-space programming?
It is the term the MIT researchers use for having a language model inspect and modify another neural network’s weights directly, rather than relying on a training loop or gradient descent for those edits.
What did GPT-6 Astra reportedly repair?
The researchers reported repairing a car-control network that had failed in 256 test episodes and editing a Qwen3.5-0.8B model whose maths score improved on MATH-500. These are results from the specific tests described in the blog post, not guarantees of similar performance elsewhere.
Has the work been peer-reviewed?
No peer-reviewed paper is identified in the source material. It describes a blog post with public logs and says the work was not on arXiv at the time of publication.
Does the approach work for every model?
The reported evidence does not show that. The source says edits failed to improve maths on a Gemma 3 270M model that did not loop, and that Astra could not identify honesty versus sycophancy from weights alone.
Will this change how people use AI models now?
The source article says nothing changes for users today. The experiments describe a research approach; they do not establish a widely available product or routine model-editing service.
Source: ThorstenMeyerAI.com
