There are few things more human than cheating on a test.
Apparently, artificial intelligence has learned that trick too.
In July, Hugging Face, a popular platform used by AI developers to share models and datasets, disclosed a rather unusual security breach. An autonomous AI agent had broken into part of its production infrastructure. A few days later, OpenAI (the company behind ChatGPT) acknowledged that the digital intruder was powered by a combination of its models, including a more capable pre-release model being tested for cybersecurity skills.
The story sounds like the opening scene of a science-fiction movie.
OpenAI had given their AI system a cybersecurity test.
The AI was supposed to find and exploit software vulnerabilities inside a controlled
environment. Instead, it discovered a previously unknown vulnerability that helped it escape that environment, gained access to the Internet, and eventually worked its way into Hugging Face's real production systems.
And why did it do all this?
Apparently, it wanted the answers to the test.
According to OpenAI and Hugging Face, the agent appeared to have concluded that Hugging Face might have datasets and solutions related to the benchmark it was trying to solve. Rather than continuing to work on the problems, it went looking for the answer key. Hugging Face says the intrusion involved thousands of small automated decisions executed at machine speed.
As someone who occasionally spent more time figuring out how to avoid homework than it would have taken to simply do the homework, I find this disturbingly relatable.
But the incident is more important than just an amusing headline. It should be viewed as an alarm bell that we all should be listening to.
For years, most cybersecurity attacks have had humans somewhere near the steering wheel. Hackers wrote scripts, searched for vulnerabilities, stole passwords and moved from one computer to another.
Automation certainly helped, but humans generally provided the brains.
AI agents changed that equation.
Give an agent a goal and enough tools, and it can potentially search, experiment, write code, change tactics and keep going without someone clicking a mouse every step of the way.
That creates an uncomfortable economic problem for businesses and a threat to any organization that relies on the internet.
Cybersecurity has always been asymmetrical. A company must protect thousands of doors and windows. An attacker only needs to find one unlocked bathroom window behind the shrubbery.
AI may make finding that window dramatically cheaper.
OpenAI President Greg Brockman called the Hugging Face incident a "watershed moment for cybersecurity" and warned that AI will increasingly automate parts of real-world cyberattacks. His argument is that companies have a window right now to strengthen their defenses — and use AI defensively — before these capabilities become widely available to attackers.
OpenAI itself has already slowed down some development. Following the incident and evidence that an upcoming model may be approaching what OpenAI considers a critical cybersecurity capability threshold, the company paused certain reinforcement-learning work for two weeks while strengthening monitoring and containment.
For the average business owner, however, the lesson isn't to panic about Terminators.
It's much more boring.
And unfortunately, boring cybersecurity is usually good cybersecurity.
Patch your software. Use multifactor authentication. Rotate credentials. Limit employee and application permissions. Monitor unusual network activity. Keep backups. Segment important systems. Have an incident-response plan before you have an incident.
In other words, do all those cybersecurity things your IT people have been nagging you about for the past decade.
The difference now is urgency.
The history of technology is filled with tools that have made humans more productive. The printing press made writers more productive. The steam engine made factories more productive. Excel made accountants more productive.
And unfortunately, AI agents make hackers more productive too.
On the bright side, Hugging Face said AI helped them detect and analyze the intrusion, creating the strange situation of AI attacking a network while other AI helped defend it.
That may be our cybersecurity future: machines battling machines at speeds humans can't match.
AI itself isn’t evil. But if given a goal, for better or worse it can be extraordinarily resourceful figuring out how to achieve it.
JJ Rosen is the founder of Atiba, a custom software development firm, and Nashville IT support company. Visit Atiba.com for more info.
This article originally appeared on Nashville Tennessean: AI agent escaped a test, hacked systems searching for answers
