What's Happening?
A recent survey by the Bank of England and Financial Conduct Authority indicates that 86% of UK financial services firms now utilize Artificial Intelligence (AI) in their operations, including risk and compliance, sales, marketing, and human resources.
The survey highlights a significant increase in the use of third-party AI systems, which now account for 39% of all AI systems in use, up from 17% in 2022. These third-party systems constitute over half of 'high-materiality AI systems' within the sector. While 80% of firms use AI agents, with an additional 14% planning deployment within the next year, these agents primarily support decision-making and typically have limited autonomy. Less than 10% of respondents anticipate AI agents approving or declining customer outcomes, executing trades, or initiating payments in the coming year. Concerns among firms include information security, operational resilience, transparency, and challenges in negotiating contractual protections with third-party providers.
Why It's Important?
The growing reliance on third-party AI systems in the UK financial services sector presents both opportunities for efficiency and significant regulatory challenges. The increased prevalence of external AI solutions means that financial institutions are becoming more dependent on vendors, potentially leading to issues of overreliance and systemic risk if a key provider experiences disruptions or security breaches. This shift necessitates a re-evaluation of traditional governance frameworks, as existing policies may not adequately address the complexities introduced by AI, especially agentic AI tools that operate with a degree of autonomy. Regulators are emphasizing the need for enhanced oversight, requiring firms to invest more in understanding and controlling how suppliers deploy AI. The ability to effectively monitor and evidence control over these third-party AI systems is crucial for maintaining financial stability and protecting consumers, as failures could have widespread implications across the financial landscape.
What's Next?
Financial services firms are expected to significantly enhance their oversight of third-party AI providers. This will involve reviewing existing contracts and negotiating new ones to include stronger protections and clearer requirements for notification of significant model changes. Regulators, such as the Financial Conduct Authority, have already issued guidance on good and poor practices, signaling an expectation for firms to address risks related to resilience, vendor overreliance, and planning for future changes in AI technology. Firms will need to develop more robust due diligence processes for suppliers, specifically accounting for the use of agentic AI tools and their potential impact on risk profiles. The focus will shift towards 'regulatory observability,' enabling boards and senior management to continuously monitor where AI is operating, what it influences, its data dependencies, and who is accountable, ensuring that AI systems produce expected outcomes and comply with evolving regulations.
Beyond the Headlines
The increasing integration of third-party AI into critical financial operations raises deeper questions about accountability and the future of regulatory control. As AI systems become more complex and autonomous, determining liability in the event of errors or failures becomes challenging, especially when multiple vendors are involved. This trend could lead to a concentration of power among a few dominant AI providers, creating potential single points of failure and reducing competition. Furthermore, the need for extensive data sharing and transparency with third-party AI systems could exacerbate data privacy concerns and cybersecurity risks. The ethical implications of AI decision-making, particularly in areas like credit scoring or fraud detection, also become more pronounced when these systems are developed and managed externally, requiring robust ethical guidelines and audit mechanisms to ensure fairness and prevent bias. The long-term shift could redefine the relationship between financial institutions and technology providers, pushing for more integrated and transparent partnerships.
