OCC: Banks need strong controls when delegating to AI agents
- Key insight: The Office of the Comptroller of the Currency says it does not treat AI as a standalone risk category, but rather discusses the liability banks’ retain when they delegate authority to models which can make fraud, payment, or communicative decisions.
- Supporting data: The framework establishes six oversight pillars: purpose, potential risks, controls, human oversight, resilience and testing.
The Office of the Comptroller of the Currency on Wednesday released guidance governing how examiners will grade banks on artificial intelligence deployment, concluding that banks need strong internal controls if they are going to delegate decision making to AI.
Processing Content
The framework, while not binding, notes that the agency does not consider AI as a separate kind of regulatory risk, but rather directs examiners to apply existing regulatory criteria to the authority banks vest in AI systems, especially autonomous systems which may make decisions independent of human judgement. In those cases, OCC writes, a “bank’s control intensity could increase.”
“Higher consequence use cases may warrant greater restrictions on autonomy, stronger controls, more direct human oversight, more independent validation, and more robust resilience measures,” the guidance notes. “Risk classification may be based on potential consequences, transfer of agency, and the degree of human oversight, rather than the technical sophistication of the SI used.”
The guidance comes as banks are increasingly deploying AI to juice business productivity, but also are increasingly wary of the risks if AI makes mistakes. Federal Reserve Gov. Christopher Waller
As some of the first federal banking guidance on AI, the OCC establishes six major “pillars” of supervisory guidance. Those include the purpose AI systems are directed to execute, the potential consequences if an AI system or agent fails its purpose, guardrails and human judgement, identifying who is accountable for AI oversight, a banks’ ability to whether AI failures and evidence in the form of testing of AI systems and third-parties to ensure AI continues to be effective. Banks are given guidance on a variety of permissible use cases, including writing customer correspondence, credit decisions and autonomous use cases.
The guidance repeatedly emphasizes the authority banks give AI models to execute independent actions. The OCC distinguishes between AI uses in which humans pre-approve actions, vs when AI systems are monitored by humans, but may take action without pre-approval.
“The level of [human] involvement may be calibrated to the potential consequences, autonomy, reversibility and operating volume of the use case, among other factors,” the agency writes. “Hallucinations concerning fees, rates, account status, or policies could contribute to legal or customer-harm exposure.
“Banks may consider controls that limit automated representations, require verification in higher-impact contexts, and support prompt correction and remediation,” the guidance continues.
The agency says that AI overseers should have expertise and authority to intervene when necessary and that “where appropriate, the reviewer may be separate from the team that developed or owns the system.”
Read more:
Banks are expected to think through and document each AI system’s purpose, authorized users, data permissions and vested authority. Even if an AI bot’s actions appear logical, an AI could be deemed a control failure if it goes beyond its approved purpose.
The agency notes that AI can take data from sensitive places in ways banks may not fully understand, so banks are expected to understand where the system is siphoning data from.
“SI can create privacy risks beyond conventional data-handling concerns,” the agency writes. “Models may retain or reproduce training data, infer sensitive attributes, and use bank or customer information in vendor-hosted environments that are not fully transparent to the bank.”
Crucially, the agency notes that banks using AI models through third-parties retain responsibility for managing risks that emanate from these partnerships.
“Banks may define the scope of authority, identity, permitted tool use, data use and escalation limits for provider agents that interact with bank systems and data. The bank may also monitor and potentially limit what provider agents do without impacting the relationship with the provider,” the agency writes. “The bank should consider how contracts assign responsibility for model related errors or losses including customer harm for which the bank may seek clarity, indemnification, caps or loss responsibility.”
As banks increasingly deploy autonomous artificial intelligence models across payments, fraud and customer-facing applications, Mary Ann Miller, VP, Fraud Executive Advisor at Prove, a digital identity and fraud prevention company, says the banks must calibrate the level of oversight to the potential consequences of failure.
“The OCC supervisory framework … notes that SI and agentic SI can improve detection of fraud and financial crimes, help defend against cyber threats, and strengthen overall risk management when used with adequate controls,” she wrote in an email. “However, as AI systems become more autonomous, financial institutions must ensure that the same technology designed to combat fraud doesn’t introduce new vulnerabilities.
“This requires moving beyond one-time identity checks toward continuous verification of who or what is initiating an action, whether that activity is authorized and whether it aligns with expected behavior,” Miller added.
