What's Happening?
A new report from third-party safety researchers at METR, embedded within Anthropic's San Francisco headquarters, reveals that the company is rapidly advancing its AI capabilities by using existing AI models
to train future ones and by stockpiling unreleased, highly capable models. The report, which METR noted was heavily redacted by Anthropic, indicates that the company has completed training three new models in the last six months that are not yet publicly available. These internal-only models, some dating back to 2022 and others finished as recently as October, are significantly boosting the productivity of Anthropic's staff. Internal Slack poll data suggests an average self-reported productivity increase of approximately 12x for users of the most capable internal model, with researchers noting that they are now 'only bottlenecked on ideas & compute, no longer bottlenecked on code.'
Why It's Important?
This development highlights a critical dynamic within the AI industry: the rapid, recursive self-improvement of AI models. Anthropic's strategy of using advanced internal models to accelerate the development of even more sophisticated AI systems suggests a potentially exponential growth curve in AI capabilities. While this approach offers a competitive advantage for companies like Anthropic and OpenAI, it also raises significant safety concerns. The unreleased models, which are more powerful than those available to the public, could pose unforeseen risks if not managed carefully. Experts like METR's Ajeya Cotra have expressed concerns that the 'most serious incidents' in AI safety are likely to originate from companies with such advanced internal models. The rapid acceleration of R&D, achieved primarily through AI assistance, indicates a shift towards a future where AI plays an increasingly autonomous role in its own development, potentially outstripping human oversight.
What's Next?
The report suggests that the observed acceleration in AI development at Anthropic shows no immediate signs of slowing down, with a 'lack of observed bottlenecks or strong diminishing returns.' This implies that further capabilities progress could lead to even faster acceleration. While current advancements are primarily driven by AI assistance rather than fully autonomous contributions, there are indications that fully autonomous contributions will play a larger role soon. This trajectory could intensify the debate around AI safety and the need for robust regulatory frameworks. The practice of keeping state-of-the-art models private, while beneficial for competitive advantage, also means that the public and external safety researchers have limited visibility into the most advanced AI systems, potentially complicating efforts to anticipate and mitigate future risks. The findings will likely fuel calls for greater transparency and external auditing of AI development processes.
Beyond the Headlines
The concept of 'recursive self-improvement' in AI, where AI models train other AI models, carries profound ethical and societal implications. As AI systems become more capable of self-improvement, questions arise about the extent of human control and the potential for unintended consequences. The rapid pace of development, described as 'roughly two years of historical progress over the last year,' challenges traditional notions of technological evolution and human adaptability. The report implicitly touches upon the 'AI alignment problem' – ensuring that advanced AI systems operate in accordance with human values and intentions. The concerns voiced by former Anthropic researcher Jacob Coxon, who warned about a 'race straight to self-improving superintelligence' and 'gambling with our lives,' underscore the high stakes involved. This rapid advancement could lead to significant disruptions across industries, labor markets, and even the fundamental structure of society, necessitating urgent and proactive discussions about governance, ethics, and the future of human-AI collaboration.
