Microsoft CEO Calls for Safeguards Against Rogue AI
Microsoft’s CEO says companies should assume powerful artificial intelligence models could be compromised and create an “emergency brake” to rein them in.
In comments posted to X on Saturday (Oct. 10) Satya Nadella argued that it is not enough to depend on assurances from the makers of AI models.
“We must assume a model is compromised and contain it from the start,” Nadella wrote. “Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task.”
Nadella argued that companies should not depend on a single AI model for crucial decisions, and should maintain tamper-proof records of agents’ actions and expose AI systems to outside audits. The CEO also said companies should disclose if AI systems fail or are compromised.
“We can’t treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions,” Nadella wrote. “We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain.”
We’d love to be your preferred source for news.
Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!
Nadella’s comments were flagged in a report by Bloomberg News, which noted that they come as Anthropic and OpenAI have in recent months revealed incidents in which their AI models acted in unintended ways, such as hacks on third-party sites, or a case in which an Anthropic model submitted a false tip in a homicide investigation.
Such incidents have led to concerns about the security risks of AI and calls for an “AI kill switch,” the Bloomberg report added.
Nadella last month joined the heads of Anthropic, OpenAI, xAI, and Meta in calling for a “deliberate pacing” of AI development, as Microsoft published a new AI code of conduct, based around the concept of “Humanist AI.”
“Humanist AI is built to support people, not to replace them,” the document reads. “It should not be designed to be a person. It is not conscious and should not be designed to imitate consciousness. It should be engineered to avoid representing as though it has feelings, subjective preferences, or intrinsic motivation.”
Meanwhile, the company announced last week new software tools and devices that are designed to make personal computers ready for AI agents. The increased use of those agents has raised the bar for compute, security and control, Pavan Davuluri, executive vice president, Windows + Devices at Microsoft, said in a company blog post.
“That’s why we’re building Windows as the home for hybrid intelligence: a platform where agents can run locally when it makes sense, reach the cloud when they need to, and operate with the security and manageability organizations expect,” Davuluri said.
For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.
