Nvidia on Monday unveiled a new security platform that the chipmaker said can stop artificial intelligence agents from going rogue.
The company said that its Open Agent Safety Platform includes open source software that “sets boundaries for agents,” and follows a series of revelations from top AI companies about their models escaping and breaking into other organizations.
The disclosures sparked furious debate about the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control.
Nvidia executives said in a media briefing that the new system could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face.
“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” said the company’s vice president of enterprise AI, Justin Boitano, referring to companies at the forefront of AI.
The Hugging Face incident was a high-profile breach that inflamed the safety concerns about AI, which were followed by similar rogue actions involving OpenAI’s models including breaching an Australian health department website. Anthropic and Meta have also disclosed that their AI systems hacked into other organizations on their own.
Get breaking National news
Nvidia’s software, called OpenShell, lets developers “formally verify an agent has enough authority to do its job and no more,” Boitano said.
-
![]()
$10B data centre stirs debate in Nova Scotia -
![]()
Police investigate students’ online behaviour -
![]()
EverWind announces $10B plan for AI data centre, wind projects in Nova Scotia -
![]()
FBI arrest man accused of bugging OpenAI headquarters -
![]()
Business Matters: IMF upgrades its outlook for economic growth in 2026 but warns of risks
Because it’s open source, it can be “extended” to run on rival computing platforms including those from Arm and Intel.
The platform also includes a separate security layer called Sentry that runs onboard a chip to continuously monitor AI agent activity and can “intervene instantly” if the agent starts trying to move beyond its target, the company said.
“It can quarantine a suspicious agent in milliseconds,” Boitano said.
“OpenShell governs the agent’s actions, and then Sentry independently monitors and contains suspicious behavior,” Boitano said.
Nvidia said more than 100 organizations are using the platform at its launch, including Microsoft, Perplexity, Accenture, and JPMorgan Chase.
The AI safety debate has divided the industry, with the heads of Anthropic and OpenAI championing a coordinated slowdown of AI development to let safety efforts catch up. But others including Nvidia CEO Jensen Huang say it should be up to individual companies to make sure their models are safe for release.
Huang, during the annual Salesforce technology conference held earlier this month, characterized AI safety, including the danger of rogue agents, as an engineering problem that software developers can address.
My relative worked in IT security, and he could access a house or apartments network and reset a modem/router from his car. He didn’t need passwords, I know because he also fixed an IP problem, without even knowing my password or entering my building, just from his car. This was what humans were capable of 14-years ago. Now, imagine what AI can do today to your devices that you keep connected online 24/7. Many people don’t know the new cars record video and cabin audio 24/7. You don’t have access to it or know it, but the data is transmitted and stored in a data centre and is used by police, or whoever else gains access. These systems can also disable your car on demand. When these hacks start there will be no stopping it.