"So what ever happened to the world falling apart if a model like Mythos got released without infantilizing restrictions on its use?"
This happened. — Kyle Choi 崔凯尔 I just posted about that... — Perry E. Metzger
Hugging Face dealt recently with an AI operated attack. They had to use open models to defend, because the closed model guardrails would not allow them to use them for defense.
Most important quotes:
“When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.”
and:
“We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment.”
Source: https://x.com/perrymetzger/status/2078872400012165223