Artificial Intelligence: QAwerks CEO Konstantin Klyagin Warns Many Financial Institutions Unprepared For Regulator Queries

Regulators are coming with questions about financial institutions’ Artificial Intelligence processes that many will struggle to answer, warns QAwerks CEO Konstantin Klyagin. More documented hallucinations, customer complaints of banking doom loops, and the CFPB’s determination that inaccurate Artificial Intelligence chatbot responses may be federal violations are among the reasons the regulators are watching.

When customers, regulators, or board members ask questions about Artificial Intelligence agents, what systems they can access, what actions they take, and who is ultimately responsible, the institution must provide clear responses.

But many can’t. Klyagin said the reasons are many, beginning with basic operational differences between standard deterministic software and Artificial Intelligence models. With the former, the audit trail is simpler. An operator makes a decision and gets a result.

Why Artificial Intelligence systems are more complex

In contrast, an Artificial Intelligence-assisted system often pulls information from many separate areas. Multiple AI agents may be involved. API’s connecting AI models to payment rails, fraud engines, and data typically are not independently tested. As more processes are included, the risk of error accumulation rises. When trouble hits, institutions struggle to explain why.

Models must protect against data leakage, irrelevant data access, and inappropriate permissions. They must know when to transfer a query for human review. In chat situations, systems must provide accurate results, even when people provide multiple different, and in some cases inaccurate, wordings. Semantic level validation is crucial at this step.

“If you have a proper audit trail implemented in your solution, you know exactly which agent did what, what input they had, what output they had, what was handled from one agent to another,” Klyagin said. “You’ll also know how, with the data context and APIs, what they pulled, or that they silently failed with some information, or pulled incorrect information.

“If you have all of that documented, you can reconstruct the whole decision, and you can fix that. Without the audit trail, it’s just guesswork. You don’t own the agent; the agent owns you.”

Properly designed, Artificial Intelligence-assisted logs can easily be more comprehensive than traditional software ones. That comprehensiveness has a price, as Artificial Intelligence forges solutions across multiple systems. As the trail grows in complexity, so do the explanations.

Klyagin used the example of a mortgage provider. Early in an application, an income statement is misread, or the decimal placed in the wrong spot. It gives the applicant a $500,000 salary instead of a $50,000 one. As more agents become involved, the problem compounds, leading to a wrong decision.