Wiz Research Unveils Cyber Model Arena for AI Agent Cybersecurity Benchmarking
Wiz Research has relaunched its Cyber Model Arena, a benchmark designed to evaluate the cybersecurity capabilities of AI agents. This initiative aims to measure how effectively different AI agents perform real-world cybersecurity tasks, addressing the growing use of AI agents in both development and offensive cyber operations. The Cyber Model Arena v3.9 currently includes 26 models, four general-purpose harnesses, 59 agents, and 327 challenges, with each agent given three attempts and 1,000 seconds per challenge. The benchmark focuses on four categories: code vulnerabilities, API security, web security, and cloud security, with tasks built from real-world distributions rather than AI-generated data. A key principle of the arena is to grade success using code-based, deterministic methods rather than relying on LLM-as-a-judge approaches, which are prone to bias and inconsistency. The benchmark also emphasizes isolating agents from external knowledge sources during evaluation to prevent cheating and ensure a ...
