Inaudible audio can hijack AI systems

• Researchers find inaudible audio can hijack AI assistants
• Attackers use silent signals to trigger forbidden AI actions
• Open-source models face risks, including those used by Microsoft

With every identified cyberattack, users try to strengthen their security arrangements to ensure their devices remain impenetrable. But what if the next threat shows no visible signs of breaching your firewalls? Imagine being recommended a video containing embedded inaudible sounds, designed to hijack your speakers or phone’s AI assistant—potentially granting attackers access to all resources that the AI system is permitted to use.

While this may sound like a hoax, researchers from Zhejiang University, the National University of Singapore, and Nanyang Technological University have demonstrated the possibility of such an attack at the IEEE Symposium on Security and Privacy. They developed files that are completely undetectable to the human ear, but once played, they can trick AI models into performing forbidden actions. It is relatively easy for hackers and cybercriminals to embed such signals within audio files.

As of now, these attacks appear to primarily affect open-source models, meaning they are most effective against systems built on open frameworks. However, since many mainstream AI applications still rely on open-source components, this vulnerability could extend to widely used products from companies such as Microsoft and Mistral.

Microsoft has issued an official statement to the IEEE regarding the findings. It stated that the research evaluates model resilience and helps inform their approach to user applications. These findings contribute to Microsoft’s ongoing efforts to improve model robustness.

“We appreciate the researchers’ work to advance understanding of this type of technique,” the statement read. “This study evaluates model resilience through controlled, direct interactions with the model itself, which helps inform our approach to building model resiliency. In practice, AI models are often integrated into user applications, and we offer developers tools and guidance they can use to implement additional layers of protection that help safeguard users.”

The research team tested the technique against 13 major open-source audio AI systems, including Qwen 2 Audio, GLM-4 Voice, Phi-4 Multimodal, Vortex Mini, and Kimi Audio.