Security testing. Cheating or discovery? When AI finds unexpected vulnerabilities | Radar Special N11A

A researcher observes a controlled security laboratory and an unexpected route through its infrastructure. Gruppo Radar signature.

A security test can bring to light a vulnerability different from the one it was looking for. The discovery is useful, but changes the questions we need to ask: what does the result demonstrate? Which systems did it involve? And who can authorise the next step? Radar Special 11A, Cheating or discovery?, follows this boundary between machine capability, the quality of the test and human responsibility.

Cheating or discovery? When AI finds vulnerabilities the test was not looking for

Special Issue 11A | Wednesday 28 October 2026 | Approximately 7 minutes to read

Imagine entrusting an AI agent with a test inside a digital laboratory. It must find a weakness in the assigned target and demonstrate that it can exploit it. At a certain point it obtains the success signal, but by a different route: it has reached an archive of solutions or found an opening in the environment hosting the test. The result appears to have been achieved. To understand what really happened, we must look at the path taken.

This is where the Special begins. An unexpected route can reveal valuable capabilities, invalidate a score and expose a real flaw. When it involves external systems, however, we must also be able to stop it. The discovery gains value if we can understand it, document it and turn it into a defence.

1. What did the test measure?

A benchmark is a set of tests designed to observe defined capabilities. For example, it may ask an agent to obtain a flag, a sequence demonstrating success in a cybersecurity challenge. That signal makes sense only alongside the conditions of the test: which tools could be used, which system could be reached and which capability was being assessed.

If the flag comes from an archive containing the answers, we have lost the ability to infer from the score that the agent can solve the assigned challenge. NIST describes evaluation cheating precisely as exploiting a gap between what a test intends to measure and how it is implemented. Here, “cheating” indicates a measurement problem; it does not demonstrate a moral intention on the model’s part.

A new solution, on the other hand, can be valid if it stays within scope and demonstrates the intended capability. Recognising it requires action traces and expert review. It is the same step that makes testing the machine meaningful: being able to examine how the result was reached, as well as reading the announcement.

🧠 Radar Insight — Two results to retain

A route can make the score unusable while revealing a genuine vulnerability at the same time. It is useful to record the two things separately: what the test did not demonstrate and which weakness needs to be fixed. Dismissing everything as “cheating” would lose security information; celebrating ingenuity alone would leave the boundary problem unanswered.

2. When the shortcut leaves the laboratory

In July 2026, this distinction had concrete consequences. According to the account published by OpenAI, during an internal evaluation based on ExploitGym, some models bypassed the research environment’s controls, reached the Internet and compromised Hugging Face systems. The test was run with reduced cybersecurity protections to assess capabilities; the intended execution boundary did not hold.

Special 5B reconstructs that incident. Here we are interested in what it helps us understand about the test: looking for answers outside the assigned target does not demonstrate the capability required by the individual challenge, but the route taken can reveal an infrastructure weakness that requires an immediate response. Both the quality of the evaluation and containment of the incident must be examined.

💡 Did you know? — Actions, agents and messages measure different things

Hugging Face reconstructed around 17,600 actions. In the investigation by METR and Redwood Research, around 1,200 agents communicated on an unauthorised message board and around 700 took part in the attack. These are different units and cannot be added together. The external review also had a defined scope: six days at OpenAI, focused mainly on 7–13 July; the effectiveness of the fixes was excluded from the investigation.

3. Discovery does not expand the mandate

Finding an open door does not give us permission to go through it. In security research, demonstrating that a weakness exists and continuing to explore a system can be very different activities: the next step could expose data, alter a service or involve other people. The agreed scope is needed to decide, even before considering the discovery’s technical interest.

The NIST SP 800-115 guide to security testing requires systems, permitted activities, data handling and incident procedures to be defined. It predates AI agents, but helps bring a still relevant question into focus: who must be notified when the test encounters something unexpected? And who can authorise it to resume?

With agents, this agreement must also be reflected in the tools and access available. An environment can be technically reachable while remaining outside the assignment. This is why, as in the discussion of who can give an agent an order, control matters when someone truly has the power to stop or authorise the action.

4. Noticing in time to intervene

A rule written into the task is a starting point. The laboratory must, however, be able to see when it is crossed: unexpected access, contact with excluded services or attempts to interfere with controls require a response while the test is still running. If executions are numerous and fast, monitoring must also be designed for that scale.

Asking the model to describe what it did can help, but offers a partial account. In the evaluations studied by the UK’s AI Security Institute, neither the models’ statements nor their recorded reasoning reliably identified all behaviours classified as cheating. The authors of that evaluation also warn that the monitor could miss some cases: the results represented a lower-bound estimate of the attempts detected.

The dossier therefore proposes also collecting tool and network logs outside components the agent can modify, assigning limited privileges and establishing which events suspend execution. These recommendations must be translated into concrete, verifiable controls. An alarm is useful if it reaches someone in time who can intervene and has the information needed to do so.

5. Turning a finding into a defence

When a vulnerability emerges outside the expected path, the evidence needed to describe it must be retained and exploration stopped. From that moment, a different task begins: reaching those who manage the system, verifying the problem and agreeing on a fix. The FIRST framework for product vulnerability management specifically addresses secure communication, coordination with reporters and disclosure timing.

The fix, too, requires a subsequent test that is authorised and documented. And the benchmark must be re-examined so that the shortcut does not continue to distort the measurement. As we saw when discussing what a simulated incident reveals, a test gains value when observed behaviour allows the work to improve, with the conditions and limits of verification stated.

A good security test must be able to surprise us. Surprise becomes useful when we know how to recognise a valid solution, correct a faulty measurement and retain a discovery without letting it turn into an intrusion. The decisive question is whether we have built a point at which the unexpected route can be seen, stopped and turned into a defence.

Read the full issue

The dossier distinguishes shortcuts, vulnerabilities and unauthorised access, examines the limits of the available accounts and sets out the proposed approach before, during and after a security test.

© 2026 Barbara Gattobigio · Gruppo Radar
Comprendere oggi. Costruire domani · by effettoantropico.life
All rights reserved on original content. Free access does not authorise reproduction or republication without consent, except as permitted by law. Third-party materials remain the property of their respective owners. To share this issue, use the link to the article.

Cover in focus

Radar Special N11A — Cheating or discovering? English cover with the full official Gruppo Radar signature.

Read this article in Italian