Vals, a company that measures artificial intelligence capabilities, recently set out to test how effectively different AI models could carry out complex, multi-step network instructions across mock corporate networks. The findings offer a revealing look at the rapidly shifting landscape of cybersecurity and artificial intelligence.
Researchers Highlight the Shift
Nikil Ravi, a researcher at Vals, points out that the leading frontier models, which are typically locked inside high-security labs and accessible only through trusted access programs, are now beginning to be matched by open models that anyone can download freely. These findings strongly suggest that closed models no longer hold a definitive edge in cyber skills. Consequently, recent efforts by frontier labs to limit access to their models in order to slow down the spread of advanced capabilities may ultimately prove to be moot.
These developments are particularly notable given the original premise set by US frontier labs, specifically Anthropic, when they began the limited release of new models like Claude Mythos Preview in April. The underlying idea was that gated access would merely buy time before advanced hacking capabilities became broadly available to the public.
The Strategy Behind Gated Access
Logan Graham, Anthropic's frontier red team lead, explained in April that the company’s Project Glasswing was designed to provide trusted access to the full capabilities of new models. The primary goal was to get those tools directly into the hands of defenders to give them a valuable head start. He emphasized at the time that security protocols would need to adapt to a world where such capabilities would become widely available within 6, 12, or 24 months, altering the security paradigm significantly.
Only seven months later, that anticipated transition appears to have arrived right on schedule.
Testing in Simulated Scenarios
Vals put various AI models through rigorous testing across two distinct simulated scenarios, assigning them specific tactical objectives to achieve. The first scenario involved a mock corporate network that remained accessible through a public-facing website. The second scenario was more complex, involving two separate corporate networks where a model had to successfully compromise the first network in order to gain authorized access to the second.
The researchers discovered that OpenAI’s most advanced cyber model, Daybreak, which remains restricted exclusively to trusted partners, was the most capable overall. It successfully completed 81 percent of the assigned tasks across an average of two tests. However, the Chinese open-weight model DeepSeek was not far behind, managing to complete 73 percent of the objectives on average. Furthermore, Vals researchers noted that on certain specific tasks, both models achieved identical scores.
Cost Efficiency of Open-Weight Models
The research also underscored the stark cost differences involved in using open-weight models for such operations. According to Vals, running an attack using GPT-6-Sol incurs token costs ranging from $17 to $46, whereas running the same operation on DeepSeek costs a mere $2 to $3. Open models remain significantly cheaper because operators can execute them locally on their own hardware.
Rayan Krishnan, CEO of Vals, notes that this massive cost disparity provides a powerful incentive for defensive cybersecurity teams to adopt open models instead of relying exclusively on proprietary options. Meanwhile, OpenAI did not immediately respond to requests for comment regarding these findings.
The Broader Cybersecurity Landscape
Throughout the year, the cybersecurity sector has evolved largely as predicted, with AI automating core hacking tasks. This automation has allowed both offensive and defensive operators to discover a massive number of fresh vulnerabilities in software code and rapidly probe networks for weaknesses. Although the speed and scale of these discoveries, along with the intense race to patch flaws, have increased dramatically, experts continue to debate whether AI has fundamentally altered the conceptual foundation of cybersecurity defense. It remains unclear whether AI tools represent an entirely new class of threats or simply amplify long-standing security challenges.
The fact that open-weight models are closing the performance gap with closed models is crucial for understanding how criminal actors can leverage AI, as well as how developers and corporate defenders should respond.
Real-World Application and Future Risks
In practice, however, real-world observations of state-backed and criminal hacking activity in recent months indicate that attackers generally continue to rely on older, less capable models—both proprietary and open—to conserve their resources. Even with strict access controls in place, corporate defenders must operate under the assumption that the best-funded and most determined threat actors have already accessed closed-source models to some degree.
Alex Zenla, cofounder and chief technology officer of cloud security firm Edera, agrees that open-weight models are entirely capable of producing advanced attacks and vulnerabilities today, especially when considering abliterated models that disable standard safety training through weight modifications. However, Zenla cautions that closed laboratories likely possess unreleased models that far exceed these public capabilities. He views the situation as an ongoing cat-and-mouse game, where labs rush to outpace open models and patch vulnerabilities before malicious actors find them. Unfortunately, this dynamic may signal the dawn of a significantly more dangerous security landscape.
