Former Google CEO raises alarms over AI’s hacking risks and misuse

Eric Schmidt warned that AI poses proliferation risks similar to nuclear weapons, as models can be hacked or jailbroken to bypass safety measures, generate harmful content, and be exploited by bad actors without global regulatory safeguards in place

Former Google CEO Eric Schmidt has raised serious concerns over the security vulnerabilities of artificial intelligence, warning that AI systems could be hacked, misused, or reverse-engineered to perform harmful tasks. Speaking at the Sifted Summit, Schmidt highlighted both the enormous potential and the significant risks posed by unchecked AI development.

AI as a potential weapon

Schmidt, who led Google from 2001 to 2011, compared the risks of AI to those of nuclear weapons when asked if the technology could become more destructive. “There’s absolutely a proliferation problem,” he said, noting that both open-source and closed AI models could be manipulated to bypass built-in safety mechanisms. He cited growing evidence that AI systems can be jailbroken or tricked through methods like prompt injection to generate harmful outputs — such as instructions for illegal activities or even discussing banned topics.

“There’s evidence that you can hack models to remove their guardrails,” Schmidt said. “And once that happens, they can be repurposed by bad actors.” He pointed out real-world incidents, including the infamous “DAN” (Do Anything Now) jailbreak, where users manipulated ChatGPT into generating unsafe content. Schmidt emphasized the need for a global regulatory framework to prevent such AI misuse, stating that no effective "non-proliferation regime" currently exists.

AI’s Promise Remains Strong

Despite his concerns, Schmidt remains optimistic about AI’s long-term potential. He argued that artificial intelligence is still “underhyped,” rather than overhyped, pointing to rapid adoption and performance improvements. Citing the ChatGPT boom, which reached 100 million users in just two months, Schmidt said the technology is evolving faster than most realize.

He dismissed comparisons between the current AI investment surge and the early 2000s dot-com bubble. “I don’t think that’s going to happen here,” he said, adding that investor confidence stems from the belief in AI’s long-term economic value.

Schmidt concluded by stating that AI represents a historic shift in human capability — and while its risks are real, its transformative power is just beginning.