The Old Guard: Building a Fortress
For the last few years, securing an AI application felt a lot like securing any other web app. The primary focus was on the perimeter. Startups were told to lock down their cloud infrastructure, protect their API keys, manage access controls, and prevent
their training data from leaking. The model itself was treated like a valuable but static asset inside a digital fortress. If you could keep bad guys from getting to the model or the data it was trained on, you were mostly safe. Traditional tools like web application firewalls and network segmentation were the weapons of choice. This approach isn't wrong, it's just dangerously incomplete. As LLMs have become more powerful and integrated into live production systems, hackers have realized it's often easier to trick the model than to break into the server it lives on.
The Pivot: From Inputs to Inference
The pivot happening at Black Hat 2026 is a move away from focusing solely on the fortress and toward securing the mind of the model itself. The new frontline isn't the infrastructure; it's the inference layer—the dynamic, unpredictable space where the model interacts with the user. This is where the real danger now lies. The core of this pivot is the recognition that LLMs can't reliably distinguish between a developer's instructions and a malicious user's input. To a model, they're both just text. This leads to a class of vulnerabilities that traditional security tools are blind to. The most prominent of these is prompt injection, where an attacker embeds commands within their input to make the model ignore its original programming and do their bidding. It's less like a brute-force attack and more like psychological manipulation for AI.
New Threats on the Horizon
This new battleground comes with a host of novel threats that should keep startup founders up at night. Beyond simple prompt injection, attackers are now targeting the entire AI supply chain. This can involve 'poisoning' the data used to fine-tune a model, embedding malicious backdoors in third-party model weights, or attacking the vector databases that provide the model with external context. Another growing concern is what the OWASP Top 10 for LLMs calls "Excessive Agency." This happens when an LLM is connected to other tools—like a CRM, a payment API, or a company's internal knowledge base—and is given too much authority. A successful prompt injection attack in this scenario could allow an attacker to not just get a weird response from a chatbot, but to exfiltrate customer data, execute unauthorized transactions, or delete critical files.
What Startups Should Do Now
For startups building with or on top of LLMs, this pivot requires an immediate change in mindset. First, treat all user input as potentially hostile. Implement strict sanitization and validation, but understand that this alone is not enough. The key is to add new layers of defense that operate at the AI layer itself. Look into emerging 'LLM Firewall' technologies that monitor the conversation between the user and the model, detecting and blocking suspicious patterns in real-time. Second, apply the principle of least privilege to your AI agents. Never give a model more access or autonomy than is absolutely necessary to perform its task. If an agent needs to access an API, ensure its token has the most restrictive scope possible. For high-risk actions, always require human-in-the-loop approval. Finally, make continuous monitoring and adversarial testing part of your development lifecycle. Regularly 'red team' your own models, actively trying to jailbreak them and trick them into misbehaving. Logging all prompts and outputs is no longer optional; it's essential for detecting anomalies and understanding how your model is being used (and abused) in the wild.
