🔍 Read the full analysis: Did GPT-6 Astra Improve Another Model—or Just Fix A Loop? on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A November 2 blog post reports that GPT-6 Astra helped edit the open-weight Qwen3.5-0.8B model, raising its MATH-500 score from 7.9% to 48.1% without gradient descent. The edit reduced but did not eliminate long reasoning loops, and the result came after an intensive, 20.3-hour search described by the authors as very early work.
Researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra edited two layers of the open-weight model Qwen3.5-0.8B, raising its score on the MATH-500 benchmark from 7.9% to 48.1%. The change also cut the share of answers that reached the model’s 32,768-token limit from 93.0% to 33.1%—a large reported gain, but not proof that Astra taught the model a new mathematical ability or stopped looping altogether.
The result appears in “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,” a blog post published November 2 by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, listed with MIT CSAIL. Takehi and Teoh are identified as equal contributors. The authors call the work “very early results”; the post is not peer-reviewed and, according to the source account, is not on arXiv. OpenAI is not listed as an author, and the source reports no statement from the company about the experiment.
According to the researchers’ logs, Astra used a Python-based harness to inspect and modify the model’s weights, with a forward-pass budget for testing its outputs. Gradient descent and backward passes were not allowed. The reported successful run took 158 turns, 80 Python runs and 50 forward passes, across 20.3 hours. Each forward pass could evaluate up to 512 problems. The researchers also supplied practice questions from GSM8K and MATH; they say the final benchmark questions were withheld from Astra.
The researchers traced the repetitive output to a direction associated with the MLPs in layers 15 and 19. They report subtracting that direction from 64 output columns in each layer, changing 131,072 weights—about 0.016% of the model’s 0.8 billion weights. The edited Hugging Face model reportedly differs from the original only in those layers’ down-projection tensors, with hashes provided for comparison.
A Repair, Not a New Math Model
The reported result matters because it suggests a language model can help diagnose and alter an open model’s behavior through direct weight editing, rather than the more familiar process of gradient-based training. If reproducible, this could offer researchers and developers another way to address specific failures in models whose weights they can access. It also illustrates a practical limit: the method depended on substantial testing and access to editable weights, not a brief instruction to Astra.
The source account cautions against describing the outcome as a clean mathematical breakthrough. The Qwen model card already warns that the model is prone to thinking loops, and the authors say “Fixing loops, however, is not the same as making a model better at math.” The remaining 33.1% token-limit rate shows that the failure persisted for a substantial share of MATH-500 answers. The score increase is real as reported, but the mechanism may primarily have prevented repetitive generation from consuming the available output budget.
There are also implications for model control. The researchers compare their approach to earlier work on removing behavior by changing a direction in model weights, including research on stripping refusals from open chat models. That does not establish that the same procedure will work reliably across systems or uses. Any weight edit creates a modified model whose behavior needs separate testing, especially if it is deployed beyond the benchmark task.
Products Worth Considering
Apple Mac mini Desktop Computer with M4 chip, 10‑core CPU and GPU, 24GB Unified Memory, 512GB SSD, Gigabit Ethernet
Apple MacBook Air 15-inch (2026) with M5 chip, 16GB memory, 512GB SSD, Silver
Apple MacBook Pro 14 2021 with M1 Max, 64GB RAM, 2TB SSD, Silver
As an affiliate, we earn on qualifying purchases.
How the Search Reached the Edit
The viral framing described the experiment as AI improving AI “without training” and fixing a loop with just “50 test runs.” The published account gives a more specific description. “Without training” means without gradient descent: Astra still searched iteratively, wrote and ran code, and used model outputs to judge candidate changes. The 50 figure refers to forward passes, not the total amount of work or the number of individual test questions.
The reported scores show how much the search budget mattered. On MATH-500, the base model scored 7.9%; runs with 10 and 25 passes scored 12.3% and 14.1%, while the 50-pass run reached 48.1%. The associated token-limit rates were 93.0%, 87.2%, 86.4% and 33.1%, respectively. On GSM8K, the score rose from 13.8% to 67.1% in the 50-pass run; on IFEval, it rose from 21.0% to 62.3%. These are figures reported in the blog post, not independently verified results.
The account also describes less successful tests. On Gemma 3 270M, no edit improved the score by more than about a point, and runs with fewer passes produced much smaller changes on Qwen. In another experiment, Astra could not identify from weights alone which of two fine-tuned Gemma models had been trained to flatter users. The post covers eight experiments, so the Qwen result is one example in a broader but mixed set—not evidence that the method reliably improves arbitrary models.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
Reproducibility and Looping Remain Open
The reported benchmark gains have not been established by peer review, and the source material does not describe an independent replication. It is also not clear how well the edited model performs across other mathematical tasks, prompts, sampling settings or real-world uses. The scores are tied to the researchers’ stated setup and should not be treated as a general measure of mathematical capability.
Nor did the edit remove looping entirely: 33.1% of MATH-500 answers still hit the token limit. The results do not establish whether that remaining behavior comes from the same direction, other model mechanisms, or task-specific factors. The account also does not establish whether comparable edits would generalize to larger models or work consistently without the same testing budget.
The source reports no OpenAI comment on the experiment. Astra’s exact contribution is described through the researchers’ logs and setup; the available account does not independently confirm the process. The distinction between “no gradient descent” and “no training” is also important: the method still involved repeated experimentation and feedback, even though it did not fit weights to a loss.
Replication Is the Next Test
The immediate next step is independent replication of the reported edit and benchmark results, with the same model, evaluation settings and token-limit measurement. Researchers would also need to test whether the modified weights retain the gains on held-out tasks and whether the reduced looping persists across different prompts and sampling choices.
The November 2 post does not announce a scheduled peer-reviewed paper, a broader product release or an OpenAI collaboration. For now, the evidence described in the source is a published blog post, released logs and an edited model on Hugging Face. Until further tests are available, the most defensible reading is that Astra helped identify a targeted weight edit that substantially improved several reported scores in one open model, while leaving important questions about robustness and the cause of the gains unresolved.
Key Questions
Did GPT-6 Astra improve Qwen3.5-0.8B’s math score?
The researchers report that the model’s MATH-500 score rose from 7.9% to 48.1% after the edit. The result comes from their blog post and has not been established here as independently replicated.
Did the edit stop the model from looping?
No. The reported share of MATH-500 answers hitting the 32,768-token limit fell from 93.0% to 33.1%, so the behavior was reduced but remained.
What does “without training” mean in this experiment?
The researchers say they did not use gradient descent or backward passes to fit the weights to a loss. Astra still ran an iterative search using Python, test feedback and 50 forward passes in the successful run.
Has the result been peer-reviewed?
The source describes the work as a November 2, 2026 blog post and “very early results.” It says the post is not peer-reviewed and is not on arXiv.
Was OpenAI a co-author of the work?
No. The four authors are listed with MIT CSAIL, and the source says OpenAI is not an author and did not issue a statement about the experiment.
Source: ThorstenMeyerAI.com
