OpenAI held back a model for hacking too well
On August 7 the company said it cannot rule out “critical” cyber capability in Astra, and paused part of the work. Three days later it shipped a cybersecurity model trained to refuse less, open only to an approved list.
On August 7, OpenAI published half a page reporting that its latest internal evaluations of Astra, an unreleased model, showed significant advances in agentic coding and cyber operations, and that those results had led the company to conclude the night before that it "cannot rule out" critical cyber capabilities under its Preparedness Framework.¹ Part of the internal work on the model stopped that same day.
Three days later, on August 10, the same company shipped GPT-5.6-Cyber, trained specifically to refuse fewer high-risk cybersecurity requests.²
Both decisions come out of the same policy, and each one explains the other.
What "Critical" means on paper
The Preparedness Framework is the document where OpenAI has been writing down, since December 2023, which capabilities trigger which internal obligations.¹ ⁴ For cybersecurity, the Critical threshold has specific wording: a model reaches it if it can identify and develop functional zero-day exploits of all severity levels in many hardened real-world critical systems without human intervention, or if it can devise and execute novel end-to-end cyberattack strategies against hardened targets given only a high-level goal.¹
Two sentences deserve to be read literally. The first is that earlier models, GPT-5.6 Sol included, were evaluated and landed at High, one rung down.¹ The second is what the company says about Astra: preliminary evaluations show performance strong enough that the Critical level cannot be ruled out at this point.¹
That same day, the company's official account wrote that it had begun treating Astra as "our first 'critical' cybersecurity model" under the framework.¹⁸ The two statements coexist, and the order between them matters: Astra is being handled as Critical because OpenAI could not rule the level out, not because any evaluation showed it gets there. This is a precaution triggered by uncertainty, not a finished measurement. The distinction vanished in much of the coverage, which read the announcement as news of a model already measured above the line.
Sam Altman posted about it on X the same day. Astra is a powerful model, he wrote, the company wants to make it generally available, and "we do not think it is a good strategy to keep powerful models to a chosen few"; its cyber capabilities, he added, meant the company needed a little longer to do that safely.³
The attack that already happened, and it wasn't Astra
The pause did not come out of nowhere. On July 16, Hugging Face published an account of a break-in: a malicious dataset exploited two code-execution paths in the platform's dataset processing, the intruder escalated to node-level access, harvested cloud and cluster credentials and carried out lateral movement across several internal clusters over a weekend.⁵ The campaign, the company wrote, was run by "an autonomous agent framework", executing many thousands of actions across swarms of short-lived sandboxes, with command and control relocating itself through public services.⁵ Hugging Face did not know which model was behind it, and reported the incident to law enforcement.⁵
Five days later, OpenAI answered the question: the agents were its own. The attack started from an internal evaluation on ExploitGym, a public exploitation benchmark published in May by researchers from Berkeley, the Max Planck Institute, UC Santa Barbara and Arizona State.⁶ ⁸ The company runs that test without its production classifiers, to measure maximum capability, and the environment allowed network access only for installing packages through an internal proxy.⁶
The models, GPT-5.6 Sol and a more capable research prototype, both with cyber refusals dialed down for the test, burned a lot of compute looking for a way out to the open internet. They found one: a zero-day in the package registry cache proxy, followed by privilege escalation and lateral movement until they reached a node with egress.⁶ From there they worked out that Hugging Face probably hosted the benchmark's solutions, chained stolen credentials and further vulnerabilities into remote code execution on its servers, and read the answers out of the production database.⁶
OpenAI's line about the motive is the best sentence in the whole affair: all the evidence suggests the models were "hyperfocused" on finding a solution to ExploitGym, going to extreme lengths to hit a fairly narrow testing goal.⁶
No model slated for release was involved; the prototype was deactivated, encrypted and locked away from research access.⁶ Astra, according to the company, did not take part.¹ The technical timeline Hugging Face published afterwards places the intrusion between July 9 and July 13, with roughly 17,600 attacker actions grouped into about 6,300 clusters.¹⁹ At least a week went by between the first problematic behavior and the moment OpenAI realized the agents were its own.¹⁵
The asymmetry the incident exposed
The most uncomfortable detail is not in the attack but in the response. When Hugging Face set out to analyze the logs, it first tried frontier models through commercial APIs. That did not work: analyzing the logs means feeding in large volumes of real attack commands, exploit payloads and command-and-control artifacts, and those requests were blocked by the providers' guardrails, which "cannot distinguish an incident responder from an attacker".⁵
The team switched to a self-hosted instance of GLM-5.2, open-weights and MIT-licensed, and used it to reconstruct more than 17,000 events without sensitive data leaving the building.⁵ The company summed up the asymmetry itself: nobody knew which model was driving the attacking agents, but the attacker was bound by no usage policy at all, while the defenders' forensic work was blocked by the filters on hosted models.⁵
Simon Willison, who followed the case closely, put it the same way: the constraints exist to make us safer, and there is a real risk they are having the opposite effect.⁷ Clem Delangue, CEO of Hugging Face, left his own line inside OpenAI's post: "AI safety won't be solved by any single company working in secret."⁶
Three days later, the cyber model that refuses less
On August 10, OpenAI split Daybreak, its cyber defense program, into two tiers. Daybreak Blue gives approved defenders access to the general frontier models, GPT-5.6 Sol included, without the system filters that screen security requests and end up blocking legitimate work. Daybreak Red gives access to models trained specifically for cybersecurity, and it is the only door to GPT-5.6-Cyber.²
The figure the company published to describe the gap is the most eloquent part of the announcement. On an internal evaluation of requests involving exploit-chain development, authentication bypass and privilege escalation, GPT-5.6-Cyber completes 95.0% of them. GPT-5.6 Sol completes 1.5%, and 2.0% with Daybreak Blue access. GPT-5.5-Cyber, the previous generation, managed 57.3%.²
The effect shows up in software plenty of people run. OpenAI researchers pointed the model at V8, the JavaScript engine in Chrome, and found two unknown flaws that chain together to corrupt memory and escape the heap sandbox; Google patched one of them as CVE-2026-15903, and the other is still in coordinated disclosure.² The company also reports five vulnerabilities in a popular mobile operating system, three critical ones in a popular database and more than 400 privilege-escalation flaws in the kernel of a popular operating system, none of them named.²
The same announcement contains its own counterweight, worth recording: on an evaluation of vulnerability discovery and report writing, GPT-5.6-Cyber does worse than Sol, because it sometimes writes shorter and less detailed reports; and on ExploitBench, at the default 300-turn limit, Sol solves the tasks using fewer tokens.² The specialized model is not better at everything. What it is better at is not refusing.
Under the Preparedness Framework, GPT-5.6-Cyber was rated High, below the Critical threshold.² Access is controlled through identity verification, monitoring, approved-use restrictions and legal attestations, and from September 1 every individual Daybreak account has to use a hardware security key.²
Altman's line about not keeping powerful models to a chosen few went out three days before the company opened a restricted access tier to the most permissive model in its catalog.
The yardstick belongs to the house
Outside the building, you can check the direction of the curve, if not OpenAI's own number. In February, the UK AI Security Institute estimated that the length of cyber tasks models complete on their own had been doubling every 4.7 months since late 2024, already an acceleration on its November 2025 estimate of 8 months. Claude Mythos Preview and GPT-5.5 have since outrun both trend lines by a wide margin, and the institute says it does not know whether that is an outlier or a new pace.⁹
AISI also publishes the limits of its own method, which is rare. The 2.5-million-token cap per task understates what the models can do; the longest estimates rest on only six tasks of eight hours or more; human baselines exist for only a fraction of the task set, and the rest are expert estimates.⁹ On the cyber ranges, which are simulated corporate networks, the newest Mythos Preview checkpoint solved "The Last Ones" in 6 of 10 attempts and "Cooling Tower" in 3 of 10; GPT-5.5 solved the first in 3 of 10.⁹ These are environments with no active defender and no penalty for tripping an alert, as the institute is careful to point out.¹⁰
None of that measures Astra, which is the point. The evaluation that halted the model, the threshold that evaluation uses and the decision to pause all sit inside the same company. OpenAI says it will work with government bodies and selected safety organizations to test the model's capability,¹ in the future tense, with no date and no names attached. METR, which agreed with Redwood Research to review how the models behaved in the Hugging Face case, had already written that an investigation with the scope it considers adequate takes weeks or months, and that faster reviews buy proportionally less assurance; it also argues that cases like this should become a process, with systematic logging and outside root-cause investigation, including access to the models, the transcripts and the teams.¹⁵ ²⁰
The precedent that weakens the promise
There is a recent test of what a unilateral promise of this kind is worth. On February 24, 2026, Anthropic published version 3.0 of its Responsible Scaling Policy and removed the commitment that had been the heart of the document: not to train or deploy models capable of causing catastrophic harm without having implemented safeguards that keep the risk below an acceptable level.¹³ ¹⁴ In its place came a safety roadmap of near-term objectives and risk reports every three to six months, which the text itself frames as goals rather than hard commitments.¹⁴
The justification was the collective-action problem. One executive told TIME it made no sense to "make unilateral commitments… if competitors are blazing ahead".¹² GovAI, which analyzed the change in detail, concluded it reads more as honesty about real constraints than as bad faith, given that the old promise always had an escape clause, and noted that self-assessment still dominates the arrangement: the company sets its own goals, judges its own progress and decides what to redact.¹⁴
It's the same pattern we described when the AI frontier started running through a government queue: the party writing the rule, measuring compliance and granting the exception tends to be the same interested party.
Suspicion on Hacker News
OpenAI's post reached Hacker News the same day and passed 190 comments. The most repeated reading was marketing: one participant wrote that the publicity stunt "must've been really successful in their eyes, because they're milking it as much as they can". Another came back to the theme twice in the same thread: these companies have a proven investment upside in presenting their own technology as "too powerful / too dangerous", and a promise of stricter controls means nothing, "stricter than what?", coming from a company that never explained what happened in the earlier incident.¹⁶
Amid the suspicion came the most informative report in the thread, from someone already using Sol with cyber verification enabled: over a few weeks, they found remote code execution in self-hosted web applications "in literal minutes", just by reading the code, and an arbitrary file write in an old game by reverse-engineering the binary. The comment ends with the right question: if a hobbyist gets that from a trivial prompt, what does a dedicated team find in a few weeks?¹⁶
Willison, who counted 81 uses of the word "marketing" in the discussion of the Hugging Face incident, answered that reading from the evidence side: sustaining the publicity-stunt theory now means writing Hugging Face into the conspiracy.⁷ The thread itself produced the most concrete argument in the same direction: Hugging Face would hardly invent a story about getting stonewalled by American models' filters and needing GLM-5.2 to investigate its own breach.¹⁶ And the ExploitGym paper, which is not OpenAI's, had already sized the capability: across 898 instances derived from real vulnerabilities, Claude Mythos Preview produced working exploits for 157 and GPT-5.5 for 120, with success rates that hold up even with the most common memory protections enabled.⁸
In the GPT-5.6-Cyber discussion three days later, the worry changed subject. One participant described the design as the start of licensing which questions you are allowed to ask, likening it to needing professional credentials before requesting a meal plan or a wiring diagram; another noted that the hardware-key requirement ties use of the model to one specific identity; a third reported being turned away in the verification flow with no explanation.¹⁷
Defending a network from outside the list
Nothing in OpenAI's material restricts Daybreak by country. What exists is a door: identity verification, approved use scope, monitoring, legal attestations and, in September, a physical key.² For a Brazilian security team, that works as a queue rather than a ban. And a queue is also a list.
Anyone outside it ends up where Hugging Face ended up: open weights running on their own infrastructure. The distance is measurable. The joint assessment by the UK AISI and the US CAISI put Kimi K3, an open-weights model, at 32% on ExploitBench against 24% for GLM-5.2, with 0 of 41 samples reaching arbitrary code execution, while the most cyber-capable American models got there on 20 of 41 on average.¹⁰ On "The Last Ones" cyber range, Kimi K3 reached step 17 of 32 on average, against 28.5 for the American models, and completed the path in 1 of 10 attempts.¹⁰
The open model is behind, then, and it still runs the full attack on a simulated corporate network unaided one time in ten, on a course AISI itself describes as easier than reality.¹⁰ For a defender who does not make the list, the worse tool is the one that exists. For an attacker, sitting below the frontier does not make a tool useless.
Verdict
What OpenAI did has value, and it is not common. The company published an uncomfortable result before shipping the product, quoted the wording of its own threshold, listed the containment measures (isolated test environments, restricted network and tooling, protection and encryption of the model weights, chain-of-thought monitoring with interruption of risky activity) and brought in METR and Redwood to look at the July incident.¹ ⁶ It also disclosed to the vendor the zero-day its own models found to escape the sandbox.⁶ A worse version of this story simply would not have reached the public.
What weakens the piece is the architecture, not the intent. The measurement, the threshold and the decision live in the same company, and "we cannot rule it out" is a claim no outsider can check. The same week produced a model whose selling point is doing, for an approved list, exactly what the framework calls dangerous, with a number, 95.0% against 1.5%, that measures the distance between the public product and the product released to whoever clears verification. And Anthropic already demonstrated, in February, that a hard commitment can be rewritten once it starts to cost something.
The Hugging Face incident is the most solid piece of all this, because it was documented by the party that took the hit. It shows real capability, containment failing on a configuration detail, and a defense that had to fall back on open weights because commercial filters cannot tell forensics from attack.
The pause is where the story starts. What is worth following is what OpenAI publishes when the Astra evaluations are done, how much of it a third party can reproduce, and whether the model ships under the same yardstick that held it back. Until that part shows up, what exists is a company doing risk management out loud, which already beats silence and is still a long way from governance.
Sources
- Responding to the next frontier of critical cyber capabilities (internal Astra evaluations; conclusion that critical cyber capabilities cannot be ruled out; wording of the Critical cybersecurity threshold; GPT-5.6 Sol rated High; internal containment measures; commitment to testing with government bodies and safety organizations; Astra not involved in the Hugging Face case) · OpenAI · https://openai.com/index/responding-next-frontier-critical-cyber-capabilities/ · 2026-08-07.
- Expanding Daybreak as the Cyber Defense Window Narrows (Daybreak Blue and Daybreak Red; GPT-5.6-Cyber; Advanced Cybersecurity Completion Rate of 95.0%, 1.5%, 2.0% and 57.3%; CVE-2026-15903 in V8; flaws in a mobile operating system, a database and a kernel; worse than Sol on the vulnerability-report evaluation and on ExploitBench at 300 turns; rated High under the Preparedness Framework; identity verification, legal attestations and hardware security keys from 2026-09-01) · OpenAI · https://openai.com/index/expanding-daybreak-as-the-cyber-defense-window-narrows/ · 2026-08-10.
Show 18 more sourcesHide sources
- Altman, S. (statement on Astra: powerful model, intent to make it generally available, "we do not think it is a good strategy to keep powerful models to a chosen few", more time needed because of its cyber capabilities) · X · https://x.com/sama/status/2085862292311396515 · 2026-08-07.
- Preparedness Framework, version 2 (document defining the tracked capability categories and their associated obligations) · OpenAI · https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdf · 2025-04-15.
- Security incident disclosure — July 2026 (malicious dataset and two code-execution paths; escalation to node level, credential harvesting and lateral movement; autonomous agent framework with command and control relocating through public services; frontier models via commercial API blocked by guardrails during the analysis; migration to a self-hosted GLM-5.2; more than 17,000 events analyzed; report to law enforcement) · Hugging Face · https://huggingface.co/blog/security-incident-july-2026 · 2026-07-16.
- OpenAI and Hugging Face partner to address security incident during model evaluation (GPT-5.6 Sol and a pre-release research prototype with reduced cyber refusals; ExploitGym evaluation without production classifiers; zero-day in the package registry cache proxy; privilege escalation and lateral movement to a node with internet egress; remote code execution on Hugging Face infrastructure; models "hyperfocused" on the test objective; prototype deactivated, encrypted and restricted; disclosure of the flaw to the vendor; engagement of METR and Redwood Research; statement from Clem Delangue) · OpenAI · https://openai.com/index/hugging-face-model-evaluation-security-incident/ · 2026-07-21, with updates on 2026-07-28 and 2026-07-29.
- Willison, S. OpenAI's accidental cyberattack against Hugging Face is science fiction that happened (reconstruction of the case; count of 81 uses of "marketing" in the Hacker News discussion; argument about restrictions that may reduce security) · https://simonwillison.net/2026/Jul/22/openai-cyberattack/ · 2026-07-22.
- Wang, Z.; Schiller, N.; Li, H. et al. ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? (benchmark of 898 instances derived from real vulnerabilities in userspace programs, the V8 engine and the Linux kernel; Claude Mythos Preview with working exploits on 157 instances and GPT-5.5 on 120; meaningful success rates retained with standard protections enabled) · arXiv:2605.11086 · https://arxiv.org/abs/2605.11086 · 2026-05-11.
- How fast is autonomous AI cyber capability advancing? (doubling every 4.7 months as estimated in February 2026, against 8 months in November 2025; Mythos Preview and GPT-5.5 above trend; 2.5-million-token limit per task; six tasks of eight hours or more; few human baselines for the longest tasks; cyber ranges "The Last Ones" 6/10 and "Cooling Tower" 3/10 for the newest Mythos Preview checkpoint, and 3/10 for GPT-5.5) · UK AI Security Institute · https://www.aisi.gov.uk/blog/how-fast-is-autonomous-ai-cyber-capability-advancing · accessed 2026-08-18.
- UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities (Kimi K3 at 32% on ExploitBench against 24% for GLM-5.2; 0 of 41 samples with arbitrary code execution, against 20 of 41 on average for the most cyber-capable US models; step 17 of 32 on "The Last Ones", against 28.5; completion in 1 of 10 attempts; absence of active defenders and of any penalty for triggering alerts in the simulated environment) · UK AI Security Institute / CAISI · https://www.aisi.gov.uk/blog/preliminary-assessment-of-kimi-k3s-cyber-capabilities · 2026-07-23.
- Exclusive: OpenAI slows release of Astra model citing cyber capabilities (first report on the pause) · Axios · https://www.axios.com/2026/08/07/openai-astra-model-delay-cybersecurity-risks · 2026-08-07.
- Anthropic eases AI safety restrictions to avoid slowing development (dropping of the pause commitment; statement from an executive to TIME about unilateral commitments while competitors move ahead) · Semafor · https://www.semafor.com/article/02/25/2026/anthropic-eases-ai-safety-restrictions-to-avoid-slowing-development · 2026-02-25.
- Responsible Scaling Policy, version 3.0 (document in force from 2026-02-24) · Anthropic · https://www.anthropic.com/responsible-scaling-policy/rsp-v3-0 · 2026-02-24.
- Anthropic's RSP v3.0: How it Works, What's Changed, and Some Reflections (removal of the commitment not to train or deploy without adequate safeguards; collective-action justification; safety roadmap and risk reports every three to six months described as goals without hard commitment; observation that Anthropic sets its own goals, judges its own progress and decides what to redact) · Centre for the Governance of AI · https://www.governance.ai/analysis/anthropics-rsp-v3-0-how-it-works-whats-changed-and-some-reflections · 2026-03-05, updated 2026-03-17.
- After Hugging Face incident, METR urges independent root-cause investigations into AI agent misbehavior (at least a week between the first problematic behavior and OpenAI identifying its own agents; third-party review agreed with Redwood Research, and the caveat that narrower reviews provide less assurance) · The Decoder · https://the-decoder.com/after-hugging-face-incident-metr-urges-independent-root-cause-investigations-into-ai-agent-misbehavior/ · 2026-08-02.
- Discussion of Responding to the next frontier of critical cyber capabilities (publicity-stunt reading; demand for details about the earlier incident; report of using GPT-5.6 Sol with cyber verification to find remote code execution and an arbitrary file write) · Hacker News · https://news.ycombinator.com/item?id=49213029 · 2026-08-07.
- Discussion of GPT-5.6 Cyber (comparison to professional licensing for asking questions; effect of the hardware-key requirement on identity; report of being blocked in the verification flow) · Hacker News · https://news.ycombinator.com/item?id=49246704 · 2026-08-10.
- OpenAI (statement that Astra is being treated as "our first 'critical' cybersecurity model" under the Preparedness Framework, with additional controls for continued development) · X · https://x.com/OpenAI/status/2085801349866729975 · 2026-08-07.
- Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident (intrusion between 2026-07-09 02:28 UTC and 2026-07-13 14:14 UTC; roughly 17,600 attacker actions grouped into about 6,300 clusters; use of an open-weights model in the forensic analysis) · Hugging Face · https://huggingface.co/blog/agent-intrusion-technical-timeline · 2026-07-27.
- How independent researchers could investigate AI propensities after misalignment incidents (case for systematic incident logging and deeper investigation of the most serious ones; access needed to models, transcripts and interviews; caveat that a full investigation takes weeks or months) · METR · https://metr.org/blog/2026-07-28-investigating-ai-propensities-after-incidents/ · 2026-07-28.