The science fiction concept of AI building its own successors is rapidly becoming a business reality. Anthropic, a leading AI company, recently issued a caution that this very process is dramatically speeding up AI development.
The Two-Year Leap in One
A recent independent assessment
found that AI developer Anthropic is improving its models at roughly twice the rate it was just over a year ago. According to the nonprofit research group METR, which spent 11 weeks embedded within the company, Anthropic achieved what amounts to two years of historical progress in the last twelve months. Crucially, this surge is not attributed to simply adding more computing power. Instead, METR estimates that about 80% of the acceleration comes directly from AI systems being used to assist in the research and development process itself. This phenomenon, where AI helps create better AI, is often called 'recursive self-improvement'. While still assisted by humans, the cycle is creating a feedback loop that is quickly compressing development timelines.
How AI Is Accelerating Research
In practice, AI-assisted research means that Anthropic’s own models, like Claude, are heavily involved in the creation of their successors. The company’s engineers now rely on AI for tasks like writing and testing code, running experiments, and analysing data. This has led to a dramatic increase in productivity; Anthropic reports its engineers now ship eight times as much code per quarter compared to the period between 2021 and 2025. The AI acts as a research assistant, allowing human developers to test ideas more rapidly and focus on higher-level strategy rather than granular, repetitive tasks. While AI is not yet making major, fully autonomous breakthroughs, it is showing 'signs of life' that suggest such contributions could be on the horizon.
A Double-Edged Sword
The prospect of AI-driven scientific discovery holds immense promise for fields like medicine and science, but it also carries significant risks. The primary concern is that development could outpace humanity's ability to ensure these increasingly powerful systems are safe and aligned with human values. A rapid, uncontrolled acceleration could lead to the emergence of extreme and risky capabilities that are difficult to understand or manage. Anthropic itself has acknowledged that with less human oversight, there is a greater risk of pursuing dangerous or unethical research. The speed of this feedback loop means that a system could evolve in ways its creators do not anticipate, making robust safety protocols more critical than ever.
The Human-in-the-Loop Imperative
This is why the second half of Anthropic's warning—that human supervision is still required—is so vital. Currently, AI's role is primarily assistive, with human researchers setting goals, evaluating risks, and making the final strategic decisions. The fear is that as AI takes on more of the development pipeline, human oversight could become a mere 'rubber stamp' on an opaque process if not proactively managed. Meaningful supervision requires embedding human checkpoints and ethical reviews throughout the entire development lifecycle, not just at the end. As AI's ability to perform complex tasks grows, the bottleneck in progress may shift from the AI's capability to the human capacity required to safely and effectively supervise it. Accountability must remain with people, even when tasks are automated.
