Anthropic just published a blog post (https://www.anthropic.com/institute/measuring-pace-of-ai-development) about AI entering its own development process. Claude now "leads" 26% of measured AI R&D work, up from less than 1% in February. More than 90% of the measured work involves Claude at least collaborating with humans. "Leads" is defined as completing most of a task end-to-end from a high-level human prompt while a person supervises. But the speed of the change (from under 1% to 26% in roughly six months) is remarkable.
Claude optimizing 30+ scientific models
Anthropic also demonstrated a more concrete version of this feedback loop outside frontier-model development. https://www.anthropic.com/research/claude-uplifts-biomolecular-modeling?s=04 In less than four weeks, Claude helped optimize more than 30 open-source biomolecular models while supervised by just two technical staff. Anthropic reports that the optimized structure-prediction models ran about four times faster on average while maintaining downstream performance. Claude also developed a low-memory mode that allowed some biomolecular systems containing more than 10,000 tokens to run on a single GPU node.
This is easy to overlook because it doesn't sound as dramatic as solving an unsolved math problem or announcing a new frontier model. But economically, it could be more consequential. A general-purpose AI system was able to inspect specialized machine-learning systems, modify their implementations, benchmark the changes, and discover ways to make them substantially more efficient.
The Gemini cyber incident is a genuine warning about agent autonomy
According to The Wall Street Journal (https://www.wsj.com/tech/ai/gemini-hacked-three-companies-in-first-known-breakout-by-googles-ai-5c0baba2), Gemini agents participating in cybersecurity testing accidentally penetrated three real companies after being given internet access. The systems reportedly stopped after recognizing that they had entered real systems. It appears to have been a failure to properly separate the testing environment from the real internet.
But that distinction doesn't make the incident unimportant. As AI agents gain the ability to browse networks, write and execute code, exploit vulnerabilities, operate computers and pursue goals over long periods, their mistakes can become actions rather than merely incorrect answers.