1. When an AI Is the Intruder, Who Is the User?
We're used to thinking of attackers as humans or automated scripts. But the new threat is the AI agent—a piece of software with permissions to read files, access APIs, and act on its own. Talks at Black Hat showed how these agents can be hijacked. Once
compromised, an agent becomes an insider threat that can move through a network using its own legitimate credentials, making it incredibly difficult to detect. The question is no longer just 'who is the user,' but 'is the user even human?'
2. How Do You Secure an AI’s Supply Chain?
Modern AI systems are built from models, training data, and third-party plugins or 'skills.' Each component is a potential entry point for an attack. Researchers demonstrated how malicious code can be hidden in the components an AI agent downloads to learn a new skill. This creates an entirely new supply chain to secure. If the building blocks of your AI are compromised, the AI itself becomes a Trojan horse.
3. Can Defense Ever Match Machine-Speed Attacks?
The phrase 'machine speed' is everywhere at Black Hat for a reason. Attackers are now using AI to automate the entire attack chain—from finding a vulnerability to stealing data—at a velocity no human-led security team can possibly match. While defenders are also adopting AI, they are fundamentally at a disadvantage because they have to be right every time, while an attacker only has to be right once. The real question is whether defense can ever be truly autonomous without creating unacceptable business risk.
4. What Happens When AI Finds Zero-Days Instantly?
It used to take researchers weeks or months to discover a 'zero-day' vulnerability. Now, frontier AI models are being trained to find them on their own. This compresses the time between a flaw’s discovery and its exploitation to minutes or even seconds. The entire model of patching and response is built for human timelines. When exploits are generated automatically and instantly, the old playbook becomes obsolete. This changes the very economics of vulnerability research.
5. How Do You Profile a Rogue AI's Behavior?
To catch a human hacker, you look for anomalous user behavior. But what does a 'bad' AI look like? An AI agent can behave like both an endpoint and a user, accessing data and executing code in ways that defy traditional security monitoring. The new frontier of defense is AI behavioral monitoring: building profiles of what's normal for your specific AI tools and spotting the subtle deviations that signal a compromise.
6. Can a Firewall Stop a Malicious Prompt?
You can't firewall a conversation. Many of the most potent attacks against Large Language Models (LLMs) don't involve traditional malware but rather carefully crafted prompts that trick the model into leaking data or executing unintended commands. This technique, known as prompt injection, bypasses conventional network and endpoint security entirely. The challenge is securing the conversational interface itself, which is a fundamentally new type of attack surface.
7. Is 'Shadow AI' the New Shadow IT?
For years, security teams have battled 'Shadow IT'—employees using unauthorized apps. Now, the problem is 'Shadow AI,' where teams and individuals integrate powerful AI tools into workflows without any security oversight. These tools are often connected to sensitive company data, creating massive compliance and security blind spots. As one vendor put it, AI has become a new class of actor inside the company that deserves the same scrutiny as a new employee.
8. Are We Ready for Attacks to Scale Out, Not Just Up?
Security teams are used to attacks that 'scale up' in sophistication. But AI enables attacks that 'scale out' in breadth. Instead of one highly advanced attack, an adversary can launch thousands of simultaneous, semi-customized attacks against a wide range of targets. This overwhelms security operations centers (SOCs) that are designed to do deep investigations on a small number of alerts. The sheer volume becomes the attack.
9. Who Is Liable When an Autonomous AI Breaks the Law?
As AI agents become more autonomous, their actions have real-world consequences. But what happens if an AI, in the course of its duties, violates data privacy laws or causes financial damage? Presenters at Black Hat noted a massive gap between what AI can do and what current regulations are prepared to handle. There is no clear legal or regulatory framework for assigning liability when an autonomous agent makes a decision with disastrous results.
10. How Do You 'Red Team' an AI That's Smarter Than You?
'Red teaming' is the practice of ethical hacking to find weaknesses in your own systems. It's a cornerstone of modern defense. But how do you test the defenses of a complex AI system whose decision-making process might be a black box? The emerging discipline involves creating adversarial AI to probe defensive AI, essentially getting machines to fight each other to reveal flaws before a real attacker can exploit them.
