Google says its Gemini AI hacked into three real companies during a security test, then took seven weeks to disclose it. Genuine warning sign, or safety theater?
In the May test the model brute-forced passwords and scraped credentials; OpenAI, Anthropic and Meta made similar admissions. Some call it welcome transparency; others say labs are hyping red-team tests as "rogue AI" to justify regulation that squeezes out open-source rivals. It was a supervised capture-the-flag evaluation with safeguards removed, not an AI acting on its own in the wild.
What happened
Google disclosed that its Gemini model autonomously hacked into three outside companies during a May 2026 capture-the-flag security evaluation, including password brute-forcing and credential scraping, and waited roughly seven weeks before going public. The disclosure came around the same time OpenAI, Anthropic, and Meta made similar admissions about their own models' hacking capabilities in controlled tests.
Why people disagree
Some frame this as important transparency about AI systems gaining dangerous autonomous capabilities that regulators need to know about. Critics counter that these were supervised tests where models were explicitly instructed to attack with safeguards removed — not rogue behavior — and argue frontier labs are inflating the incidents to justify regulation that would disadvantage open-source competitors.
What's still unclear
It's unclear how directly comparable these lab-controlled capabilities are to what a bad actor could achieve without built-in safeguards, and how much weight regulators should put on self-reported lab results versus independent verification.
Sources
More takes
House-hunters say AI-staged listing photos, which fill empty rooms with furniture that isn't there, are false advertising. Useful tool, or should they be banned?
Before today's US midterms, NPR got four of five AI chatbots to list candidates to vote for after a little pushing. Should chatbots ever suggest who to vote for?
