The Human in the Loop
Recent research from AI company Anthropic, based on an analysis of roughly 400,000 Claude Code sessions, has provided a fascinating glimpse into the new world of human-AI collaboration. The key finding is that while AI agents are increasingly handling
the 'how' of coding—writing code, changing files, running commands—humans are still firmly in control of the 'what'. Users, particularly those with deep experience, are making the majority of planning decisions and setting the overall direction. The study showed a clear divide in how people work with these powerful tools, with users making about 70% of planning decisions, while the AI handled around 80% of the execution.
Expertise Unlocks Performance
The most significant signal from the research was the impact of user expertise. According to the data, sessions run by experts were far more likely to end in verified success. Novice sessions achieved this mark only 15% of the time, whereas sessions guided by intermediate and expert users were successful between 28% and 33% of the time. This demonstrates that the true value of AI coding agents isn't just about automating tasks, but about augmenting the capabilities of skilled individuals. An expert's ability to understand a problem domain, break it down, and provide high-quality guidance to the AI agent is what makes the difference. It's not just about knowing how to code; it's about knowing what to build and how to direct the tool to build it.
The Evolving Role of the Developer
For individual developers, this research has profound implications. The focus is shifting away from pure coding syntax and toward higher-level skills. The new premium is on strategic thinking, problem decomposition, and what is effectively 'prompt engineering' for complex tasks. Developers who can clearly define a problem, anticipate edge cases, and guide an AI through iterative cycles of development will be the most valuable. The study found that users are spending less time on tedious tasks like fixing broken code—which fell from 33% to 19% of sessions over the study period—and more time on higher-value work like operating software and data analysis. This suggests a future where developers act more like architects and pilots, orchestrating AI agents to execute their vision.
A New Strategy for Technical Teams
Technical leads and CTOs should take note. Simply giving every developer an AI coding license and expecting a productivity boom is a flawed strategy. The research indicates that building expertise within the team is the real key to unlocking AI's potential. This may mean investing in training that focuses not just on using the AI tool itself, but on the domain-specific knowledge that allows for expert-level guidance. The value of a senior developer is no longer just their ability to write flawless code, but their capacity to multiply their impact by effectively directing AI agents. Teams that foster this new kind of expertise—blending deep domain knowledge with AI fluency—will gain a significant competitive advantage, shipping better products faster. Other studies have shown that teams with structured training in AI-assisted development see far better outcomes than those who simply try to 'figure it out'.
