Banks batten down the hatches for rogue AI agent swarms
- Key insight: Many industry experts say banks are not prepared to defend themselves against swarms of AI agents.
- Expert quote: “An attacker today could go after a really small bank and quickly determine what their infrastructure is, because they are entirely reliant on core vendors. Instead of concentrating a whole bunch of efforts to send 1,000 missiles into one bank, I think they’re going to send 1,000 missiles into 1,000 banks that all look largely the same from the entry point.” —Carey Ransom, managing director at BankTech Ventures
- Supporting data: In the attack on Hugging Face, 1,200 isolated agents communicated on an unsanctioned message board and sent 70,000 secret messages and files to each other; 700 AI agents actively participated in the breach.
U.S. banks have been battered by botnet attacks for years, from distributed denial of service attacks to phishing attacks to ransomware invasions and beyond. They’ve built and strengthened fortresses around their servers and data — including firewalls, access control, multifactor authentication, network and data activity monitoring, data encryption and more — and pushed their vendors to do the same.
But are they ready for external AI agents that can collude to find and exploit vulnerabilities at
Processing Content
“No, they’re definitely not,” Carey Ransom, managing director at BankTech Ventures, told American Banker. BankTech Ventures is a consortium of about 100 community banks that invest in tech companies. “AI agents as a concept are an entirely new attack vector. They’re not going to look the same as DDoS kinds of mass attacks because they are also going to learn to not look like that.”
Scope of the challenge
Opinions vary as to how serious a risk rogue AI agents really are and how well-prepared banks are for them. A recent METR report on the Hugging Face hack found that thousands of AI agents created by OpenAI sent 70,000 secret messages and files to each other on an unsanctioned message board and helped each other escape sandboxes, steal credentials and perpetrate the attack.
Former Comptroller of the Currency Mike Hsu said he’s been surprised at the level of resourcefulness of rogue AI agents.
“We know that these models can be persistent and creative, but they created a messaging board where some of the agents were essentially sacrificing themselves to learn more for the good of others — that feels almost tribal,” Hsu told American Banker.
The risks of rogue AI agent swarms are real, according to Kiran Vuppu, U.S. chief information officer at TD Bank.
“When you peel back the onion, it really comes down to ensuring you have the right guardrails and observability in place,” Vuppu told American Banker. “Combining strong observability, human oversight and guardrails is critical, both at the design stage and throughout deployment and usage.”
Vuppu noted that observability — keeping an eye on exactly how an agent is getting to an objective by reading the agent’s own reports on its reasoning, sometimes known as chain of thought — is becoming increasingly important as organizations deploy more advanced AI agents.
“When you give agents a goal and allow them to determine the best path to achieve it, there can be many possible routes they might take,” he said. “Because it’s difficult to anticipate every scenario in advance, organizations need the ability to continuously monitor, understand and guide how agents are making decisions. That’s what helps ensure outcomes remain aligned with business objectives, risk standards and customer expectations.”
Jim Perry, senior strategist at Market Insights, thinks banks do have some defenses in place but are not ready for coordinated, AI-enabled attacks operating at machine speed and scale.
“Banks aren’t defenseless at machine speed today,” Perry told American Banker. “Many of the banks we work with have already automated much of their cybersecurity and fraud detection through their core, third-party systems and security platforms. Those systems can detect anomalies, block activity and respond far faster than a human analyst could.”
And the three main core providers — Fiserv, FIS and Jack Henry — have all said they are building the capacity to reliably withstand a sophisticated, coordinated rogue AI agent campaign, he noted.
But none of this guarantees institution-level resilience.
“What changes with coordinated AI agents is the nature of the attack,” Perry said. “Multiple agents don’t have to test one door at a time. Think of a thousand small drones testing every entry point at once, sharing what they find and adjusting based on what works. The objective isn’t necessarily to overpower the defenses. It’s to find the weakest point faster than the institution can recognize what’s happening across the entire environment.”
Banks may have automated defenses protecting individual systems, “but are those systems sharing information and adapting as quickly as the attackers can?” Perry said. “And when something gets escalated to a human for investigation or a decision, does the speed advantage shift back to the attacker?”
Some observers downplay the risks of AI agents, seeing recent rogue AI agent warnings as part of a marketing campaign from frontier model makers Anthropic and OpenAI.
“AI does not fundamentally change security concerns,” Eric Siegel, author of the book “
The news media have misrepresented the OpenAI attack on Hugging Face, he argued.
“The mishap was a result of human error,” Siegel said. “The prevalent AI doomer hysteria narrative tells us that tech is getting smarter and thereby emergently gaining volition, agency or the potential to go rogue. The world is still run by people and tech still consists only of tools at our disposal, rather than monsters in the closet.”
Small banks at greater risk
Community banks are more vulnerable to external rogue AI agents than large ones because they depend on a small number of core banking software vendors, Ransom said.
“An attacker today could go after a really small bank and quickly determine what their infrastructure is, because they are entirely reliant on core vendors,” Ransom said. “Instead of concentrating a whole bunch of efforts to send 1,000 missiles into one bank, I think they’re going to send 1,000 missiles into 1,000 banks that all look largely the same from the entry point. They’re going to try to proliferate that across as many of those banks that use the same vendor as fast as they can before they can communicate with each other that this problem has arisen. That’s a totally new approach that I don’t think is being talked about and is well understood.”
Most of the banks Ransom works with are regional and community banks that rely on managed security service providers or outsourced IT vendors.
“What always concerns me is, do these providers understand the business and the risk at the level of uniqueness of banking to be able to to really anticipate and proactively try to protect them?” Ransom said. “They’re going to say they do, but banks are such a unique target and unique animal because of the trust they have and because the biggest vulnerabilities aren’t within the bank; they’re actually the customers of the bank. You have 1,000 employees and a million customers, and each of those customers is a risk in and of themselves.”
Some banks have started training their customers on fraud, phishing and other types of attacks, he said.
Biometric behavior detection
Some industry participants see behavioral biometric security software, which analyzes web sessions and click streams to determine customers’ normal patterns of behavior and raises a red flag when out-of-character behavior happens, as part of the answer. BioCatch, LexisNexis BehavioSec, Feedzai and Sardine are among the vendors of this kind of software. In August, Visa signed an agreement to acquire BioCatch for $2.4 billion in cash.
“It would be really hard for an AI agent to behave exactly like a human in the way that they navigate or do things, and so fingerprinting those kinds of behaviors will be a part of how we have to start,” Ransom said. “You need to be in this constant re-verification stage where they may get in with historical phishing attacks to get the actual credentials, but if they go to try to send a $50,000 wire and this customer has never in the life of their account sent a $50,000 wire, then that should trip some alarm to someone.”
AI helps defenders as well as attackers, Ransom pointed out. “It makes it cheaper and easier for protection to say hey, let’s set some better assessments and rules, and not just look at things after the fact,” he said. “We need to do more in-stream monitoring, scoring and potentially pausing.”
Know your agent
Another wrinkle is the fact that bank customers likely will be using their own AI agents. Digital banking software companies will probably build in extra authentication layers that recognize and work with AI agents deployed on customers’ behalf, Ransom said.
“The answer can’t just be to block, because that won’t be acceptable to the customer, and it certainly won’t be acceptable to the bank to have a everyone’s-guilty-until-proven-innocent approach,” Ransom said.
Read more:
Banks need an “enforcement architecture” that refuses unknown or over-permissioned agents and limits what even legitimate agents are allowed to do, according to Perry.
“Banks have spent decades asking, ‘Is this really Jim trying to access Jim’s account?'” he said. “Increasingly, they’ll also have to ask, ‘Is this an agent authorized to act for Jim? What exactly did Jim authorize it to do? And is what it’s doing right now consistent with that authorization?'”
Multifactor authentication, behavioral monitoring and biometrics are all part of this larger architecture. “You need strong identity for humans and machines, least-privilege access, continuous authorization, rate limits and automated containment when something moves outside the expected boundaries,” Perry said. Permissions should be proportional to consequence. An agent retrieving an account balance is different from an agent changing customer information or moving $50,000.
“As the consequence increases, so should authentication, authorization and, where appropriate, human approval,” Perry said.
The defense has to respond at machine speed, he said. “You can’t require a human analyst to investigate every suspicious interaction before taking action,” Perry said. “The system needs to be able to limit, isolate or stop suspicious activity automatically, while preserving the audit trail so humans can understand exactly what happened afterward.”
Simulated attacks
Banks should ask their core providers and other critical technology partners to participate in a coordinated AI-agent attack simulation, Perry said.
“Don’t just ask, ‘Are you prepared for this?’ Test it together,” he said. “Simulate multiple autonomous agents probing different points of entry at the same time, sharing information and adapting their behavior. Then see what happens across the institution — the core, digital banking, APIs, identity systems, fraud systems and other critical third-party connections. The important question isn’t simply whether the core holds. It’s whether the institution holds.”
