What's Happening?
Capgemini has been selected to lead a consortium by the European Commission’s AI Office to support cyber threat modeling and risk assessment for advanced AI systems, specifically General-Purpose AI (GPAI) models. This initiative is part of the EU’s AI Safety
program and aligns with the European Union’s AI Act. The consortium, which includes Wavestone and the French public institution LNE (Laboratoire National de Métrologie et d’Essais), will focus on assessing and mitigating cyber offense risks associated with GPAI models. Capgemini will leverage its expertise in AI, cybersecurity, and compliance to help the AI Office develop evaluation tools, design technical compliance methodologies, and conduct technical assessments. The goal is to strengthen Europe’s ability to monitor and mitigate systemic cyber risks as AI models gain autonomous and cyber-relevant capabilities like automated vulnerability discovery and exploit generation.
Why It's Important?
This initiative is critical for Europe as it aims to deploy the world’s first comprehensive AI safety framework. The rapid advancement of GPAI models, which can develop autonomous cyber capabilities, poses significant systemic risks, including the potential for large-scale cyber-attacks. By leading this consortium, Capgemini will play a pivotal role in establishing robust, harmonized evaluation frameworks necessary to ensure these powerful technologies are deployed safely and responsibly across the EU. This effort is vital for strengthening Europe’s digital resilience and protecting critical infrastructure and data from AI-driven cyber threats. The work will also help the European Commission effectively enforce the AI Act’s obligations for GPAI providers, setting a global precedent for AI governance and safety.
What's Next?
Capgemini and its consortium partners will proceed with developing tools, methodologies, and benchmarks for evaluating the capabilities and systemic risks of general-purpose AI models. This includes modeling emerging cyber offense risks and designing technical compliance methodologies. The consortium will conduct on-demand technical assessments to support the AI Office in identifying systemic risks and enforcing the AI Act’s obligations for GPAI providers. This work is part of a broader, ongoing effort by the European Commission to establish and refine its AI safety program, ensuring that as AI technologies evolve, the regulatory and assessment frameworks keep pace to mitigate potential harms and promote responsible deployment across the EU.
Beyond the Headlines
This collaboration highlights a growing global recognition of the dual-use nature of advanced AI, particularly its potential for both beneficial applications and malicious cyber activities. The focus on 'cyber offense risks' signifies a proactive stance by the EU to anticipate and counter threats that could emerge from highly capable AI systems. This initiative could set a global standard for how governments and international bodies approach AI safety and cybersecurity, influencing policy and regulatory frameworks in other regions. It also underscores the increasing importance of public-private partnerships in addressing complex technological challenges, as specialized expertise from companies like Capgemini is crucial for developing effective risk assessment and mitigation strategies for rapidly evolving AI technologies. The ethical implications of AI developing autonomous cyber capabilities will also be a continuous underlying consideration.
