GPT-6 Astra can read and write neural network weights

Researchers have found that frontier large language models are surprisingly adept at natively reading and writing neural network weights. This capability goes far beyond conventional algorithms and opens up new possibilities for AI optimization and interpretability.

• GPT-6 Astra can identify what a neural network was trained to do just by inspecting its raw weights.
• The model fixed 0.016% of Qwen3.5-0.8B weights, raising MATH-500 accuracy from 7.9% to 48.1%.
• The new paradigm is called Weight-Space Programming, bypassing traditional gradient-based training loops.

Understanding Weight-Space Programming Throughout the history of deep learning, engineers relied on simple symbolic optimizers like Adam or Muon to update differentiable neural networks. However, current generation frontier LLMs are starting to natively understand and craft neural network weights based on natural language specifications. In this new paradigm, an LLM reasons about what each part of a neural network computes or should compute, and changes the weights accordingly. Experiments show that Astra can accurately identify games, diffusion model outputs, and even write small game-playing networks from scratch without environment feedback.

Improving Qwen without Gradients One of the most notable experiments involved upgrading the Qwen3.5-0.8B model's mathematical reasoning capabilities. Astra identified an issue within the model's weights causing reasoning loops and fixed it by natively editing just 0.016% of the weights. This precise weight correction drastically improved performance, raising the MATH-500 score from 7.9% to 48.1% while also tripling instruction-following capabilities on IFEval. All of this was achieved through reasoning-driven optimization rather than traditional training loops.

Implications for AI Development The emergence of reasoning-driven weight optimization signals a shift from purely automated symbolic algorithms to intelligent parameter editing. For developers and researchers, this means future AI systems could diagnose and repair internal flaws or behavioral issues by directly editing weights through natural language specifications.