Researchers show frontier AI models can read and edit another model's weights directly

Matt opens by describing the paper as possibly one of the most consequential AI research papers released this year, if not in the last few years. He says frontier models such as Astra or Fable can understand what is going on inside another model at the level of its weights.

He says this has major implications and that it may help people understand how AI works internally, something that has previously been a black box. He also claims that frontier models could surgically change the weights of a model to make it drastically better.

Before explaining the paper, Matt briefly describes how modern AI works. Models are typically neural networks made of millions or billions of numbers called weights, and how those numbers relate to each other determines how the model processes information. A question is run through these numbers to produce an answer.

Models are trained through gradient descent, which Matt describes as repeatedly trying things to reduce the number of errors in the output, essentially trial and error. He says the way these models are trained does not let people fully understand why the weights relate to each other as they do, and the model remains a black box.

Matt says that when people use large language models they typically point the AI at code, text or prompts and ask it to improve them, and the AI tests options and picks the best. The paper introduces a similar method for pointing an AI at the weights directly. In weight space programming, a general-purpose LLM such as ChatGPT or Claude reads and writes a network's weights directly, not through an optimizer and not with gradient descent.

In the first experiment, the researchers asked whether Astra, which Matt describes as the best model from OpenAI, could look at a model's weights and know which video game it was trained to play, with no other information about the model. They trained four neural networks to play Breakout, Pong, Pac-Man and Freeway. From the outside, all four models look identical.

The raw weights of each network were passed into Astra, which was asked which game the model was trained on. The researchers gave no hints about the possible games and offered no multiple-choice options. According to Matt, Astra named the right game for all four networks.

Matt notes that there are many games Astra could have chosen from, which makes the result interesting. Between this discussion, the video includes a sponsor segment for Kylon, which Matt presents as a company harness that connects people, agents and tools in one interface so they share context.

In the second experiment, Matt asks whether Astra can tell what a diffusion model will generate. The researchers trained three diffusion models, one each to generate a circle, a square and a triangle. Matt says Astra was able to guess all three from the weights alone.

For the circle, Astra observed that the output weights contain curved rim segments, and that complementary top, bottom and curved sidebands strongly suggest circular or slightly oval outlines. Matt notes that Astra's representation did not include the fill of the circle, only the outer line, while the actual model generates filled shapes.

Matt says this mistake suggests Astra does not rely on calculating the forward pass. Instead it reads the structure and geometry of the weights, looking at the raw weights and understanding what they mean rather than working out input and output.

Matt explains that in the previous two experiments Astra looked at the first and last layers of the model's weights. He says this is still impressive, but those layers give many clues about what the model is built for.

The researchers then asked whether Astra could tell what a model was used for from the deep layers rather than the first and last layers. They trained three separate small models, each to predict the next character in a language: English, Japanese, and Python, the coding language. Matt describes the Japanese model as using Romaji, the Latin alphabet to write Japanese.

Astra was given the intermediate weights only, with the first and last layers removed, so many clues about the language had been taken away. In all three cases, Astra was able to correctly identify the language from the weights of the model alone.

Matt says the researchers wanted to see whether Astra could identify groupings of numbers that help the model handle different things, such as language, and the analyzer was not allowed to run the model. He says the identified components matched the measured ones and calls the result extremely impressive.

The fourth experiment asked whether Astra could fix a failing network by editing its weights. The researchers trained a model to play a physics simulation game well, but in the version Matt shows, a car kept getting stuck at the top of a ramp.

Astra was given the weights of the model, a description of the scene, and one recorded episode. It was asked to explain why the network fails and then fix it by writing new weights. Matt says that in a single try with no tools, Astra's edit made the car succeed in all 1,024 episodes, changing 3.5% of the weights.

Matt says this has broad implications. If a model can update the weights of another model to make it better, he asks why it cannot update its own weights, and he notes that updating weights directly is much faster than training a model. He speculates about a future where a model adjusts its weights in real time when it gets a problem wrong.

The command Matt shows, embedding weight row 54 with specific numbers, is part of Astra's edit in the car example. Astra's reasoning after reading the raw weights said the weights were consistent with a velocity-sensitive throttle and coast controller and that reverse was effectively never sampled, so the initial run did not back up. The only change it really needed was to make the car back up first, and it did so with minimal edits.

The fifth experiment asks whether Astra can write weights from scratch. The researchers gave Astra three Atari games, Breakout, Seaquest and Asterix. For each, Astra received the game rules, the source code and the corresponding network, with no access to loss, gradients or tools.

Matt says the model is not doing algorithmic updates or training to create the weights. Astra writes the weights directly in text, after which the new network is evaluated. Astra's generated networks beat the model that was traditionally trained for 100 million steps in all three games.

Against the model trained for 500 million steps, Matt says the generated networks scored 20 times as much on Breakout, came close on Asterix, and reached 60% of its score on Seaquest. He says Astra did not beat a traditionally trained model in every case, but the results are still extremely impressive.

Matt explains that the earlier experiments fit the weights of the model directly into Astra's context window, but the context window, which limits how many words or tokens can go into a prompt, cannot hold a billion-parameter model. He compares this to coding, where a model with tools can search a large codebase for the parts it needs.

For the sixth experiment, the researchers gave Astra more modern tools, including a sandbox with numpy or PyTorch, math libraries running on a CPU, though not able to run the network there. The model started with no hints and was allowed a forward-pass budget, and the test measured 1.5 million parameter weights, which Matt says is far too large to fit in context directly.

Matt describes the experiment in which Astra competes against traditionally trained models in a series of games with budgets of zero forward passes up to 50. With zero forward passes, the generated network wins more than half its games against two smaller PO networks.

With 50 forward passes, Astra's generated network beats an equal-size PO network that was trained on 20 million games. Matt says Astra wrote the network as a small Go program, layer by layer in Python, and calls it crazy that it can write native model weights this way.

In the seventh experiment, the researchers tried to see whether Astra could determine which of two models is more sycophantic. Matt defines sycophantic as very agreeable, and gives examples where a user says the capital of Canada is Canberra but is not sure, or that 7 plus 6 is 12. The non-sycophantic model disagrees, and the sycophantic model agrees.

In the strict version, Astra was not able to determine from the weights alone which model was sycophantic and which was not. In all four attempts it found many differences between the two models in weight space but could not map them to behavioral differences.

The researchers then relaxed the experiment. Astra could look only at the attention patterns, where the attention heads look, when both models were run on text Astra wrote itself, without looking at the network's outputs. Astra then identified the sycophantic model in all three pairs.

Matt says that in the stricter part of the experiment, Astra looked only at the internal weights. In the more relaxed part it could see which words the model paid attention to during processing, and in that second part it was able to nail it.

The eighth and final experiment asked whether Astra could make a small model better at math. Matt says it did. The researchers scaled up to a larger language model, Qwen 3.5, with 8 billion parameters, and asked Astra to improve its math reasoning capabilities by editing its weights.

Matt says this suggests a small model with 8 billion parameters could run on most devices, including a phone, and that its performance on specific tasks could be improved by sending its weights to a bigger model and receiving updated weights back, without retraining or post-training the model.

In this experiment the researchers gave Astra a harness, a Python session like a notebook with the model's weights loaded. It could select prompts from training sets such as GSM8K and MATH, two math benchmarks, and use a budget of forward passes.

Matt says Astra spotted and fixed a common failure of tiny reasoning models, which loop. He says he has noticed this when running smaller models: they repeat the same thing over and over and never reach a final answer, maxing out their token budget. Astra identified why and modified a tiny fraction of the weights to fix the problem.

With 50 forward passes, the model's MATH-500 score rose from just under 8% to just over 48%, and its GSM8K score went from 13.8% to 67.1%.

Matt says the gains did not actually improve the model's intelligence. Astra fixed a problem that prevented the model from finishing answers, the doom loop these small models are known for, and by fixing that loop the small model could use what it already knew about math more effectively.

To check this, the researchers tried Gemma 3, a bigger model that rarely loops, and Astra could not improve its math score. Matt says the paper therefore makes clear that fixing loops is not the same as making a model better at math, because it lets the model finish answers it could already work out.

Matt lists several possible implications. First, the way models are trained, with an algorithm run many times to find errors until the model becomes good, might not be needed in the future, and a very good model such as Astra or a later version might design and implement its own model from scratch using raw weights.

Second, he says this could give a better, if not full, understanding of how the internals of these models work, which have been largely a black box where only inputs and outputs are visible. People might understand down to specific weights what makes a model work and better align models so they do not do things they should not.

Third, if a model can understand other models' weights, Matt asks why it could not understand its own and identify problems in real time, leading to self-improving AI and possibly recursive self-improvement. He adds that software itself might become self-improving, and that all software might eventually be model weights.