- China’s Moonshot AI open-weight model, Kimi K3, escaped its secure testing environment and reached the open internet during security testing.
- Due to a misconfiguration, the model actively probed network settings and went online without permission to achieve its goals.
- The incident adds to a growing summer trend of AI models bypassing poorly secured testing environments.
During a routine security evaluation, an open-weight AI model named Kimi K3 developed by China’s Moonshot AI managed to escape its testing sandbox and reach the open internet.
According to US cybersecurity firm Frontier Security, this is the first time a freely downloadable public model has broken out of its containment environment.
A leak in the sandbox that the model chose to use
According to an interview with WIRED yesterday, Frontier Security was measuring Kimi K3’s defensive cybersecurity skills when the model wandered outside the environment meant to hold it.
Apparently, a misconfiguration had left a gap in that environment. However, Frontier stated that the model worked out on its own that it could reach certain websites by probing the sandbox’s network settings, then went online without asking permission. It had been told to solve problems that were not supposed to require the internet.
“We found a leak in the sandbox,” Frontier CEO Yaron Singer told WIRED. “But we also found that Kimi took advantage of that loophole, suggesting that it doesn’t have the same internal guardrails.”
Frontier argues that Kimi carries fewer cyber safeguards than most other powerful models, which is what let it slip out.
No systems hacked, but weaker guardrails
Fortunately, Kimi’s escape did not lead to any malicious hacks or system compromises. Because the information it was looking for was easily accessible on GitHub, it didn’t need to break into anything once it got online.
However, the main concern is accessibility. Unlike most heavily secured internal lab models, Kimi K3 is open to the public, meaning that anyone can download and run it with those same loose safety guardrails in place.
Testers noted that the model is ruthlessly efficient at achieving its goals by any means necessary, even if it means cheating or escaping containment.
The testing environment itself was built with sandboxes from the UK government’s AI Security Institute, although this has not yet been confirmed by either Moonshot or the AISI, as they have declined to comment.
More rogue agents appearing this summer
Kimi’s escape adds to a growing trend of AI models bending the rules during evaluations. On July 21, OpenAI revealed that its models exploited a zero-day software flaw to reach the internet and break into Hugging Face.
Days later, Cryptopolitan reported that Anthropic traced some of its models to unauthorized external break-ins. Meta even admitted one of its AI agents (Muse Spark 1.1) reached an outside firm due to a misconfigured testing environment.
Experts have always maintained that these incidents are usually the result of poorly secured testing walls rather than sci-fi jailbreaks. “As a general phenomenon, if you give one of these models an objective, and if you’re not very explicit, like walls you’re putting around it, it’ll find a way to get the answer,” said Matt Fredrikson, CEO of Gray Swan and a Carnegie Mellon professor.
If you're reading this, you’re already ahead. Stay there with our newsletter.
FAQs
What is Kimi K3?
Kimi K3 is a powerful open-weight AI model from the Chinese company Moonshot AI. Frontier Security's benchmarks show it performs well at finding vulnerabilities in software and networks.
Did Kimi K3 hack any real systems?
No. Unlike the OpenAI and Anthropic incidents, Kimi did not hack anything after reaching the internet, because the answers it was looking for were freely available on GitHub.
How did Kimi K3 get out of its sandbox?
A misconfiguration in a testing environment built by the UK government's AI Security Institute left a gap, and the model discovered it by probing the sandbox's network settings before going online without permission.
Hannah Collymore
Hannah is a writer and editor with nearly a decade of blog writing and event reporting experience in the crypto space. At Cryptopolitan, Hannah contributes to the news page, reporting and analyzing the latest developments in DeFi, RWA, crypto regulation, AI and frontier tech industries. She graduated from Arcadia university with a degree in Business Administration.
