A 27B parameter open language model has just done something new: it rewrote its own weights, using only its own reasoning and forward passes, and came out better for it. No gradients, no retraining, no outer optimizer. The model was given its own weights, access to run itself, and a set of graded answers. It then diagnosed where it failed and edited its own parameters, improving its performance on problems it had never seen.
The paper calls this self-rewriting—the model is its own optimizer. Earlier work let a large model edit a smaller one, but this time, the same model both reasons and edits. After each round, it tested the new weights on practice problems, kept the edits that worked, and repeated. Over eight sessions, its accuracy kept rising, with no sign of overfitting or collapse.
How self-rewriting works
The setup is direct: the model gets its own weights, a record of its graded answers, and the ability to run itself. For each failure, it reasons about what went wrong, proposes a weight edit, and applies it. It then tests the new weights on a set of held-out problems. If the edit helps, it keeps it; if not, it discards and tries again. The process only uses forward passes—the same as inference—so there’s no need for gradients or a training loop.
What changed for accuracy and robustness
The results are concrete. The model’s own edits made it 6.6 points more accurate on problems it hadn’t seen. When given a damaged copy of itself (one that could barely solve any image tasks), the model, without any access to the original weights or retraining, brought it back to over 80% of the original accuracy. Smaller models in the same family failed to edit themselves usefully—some made things worse—while only the largest learned and repaired itself.
Why this matters for AI engineering
Unlike prompt engineering or test-time tricks, these edits are written into the weights: the model stays better, with no extra cost per answer. The line between training and inference is now a technical limit, not a fundamental one. This is a step toward recursive self-improvement—models that upgrade themselves continuously, without human intervention, prompts, or harnesses. For engineers, this could mean models that adapt and repair themselves in production, not just in the lab.




