Even JPMorganChase and BofA measure AI returns differently

Processing Content

• Key insight: Banks measuring AI performance against pre-AI baselines are far more likely to quantify financial performance.

• What’s at stake: Without proper planning, banks risk mistaking productivity improvements for financial returns, making it harder to decide which initiatives to scale.

On JPMorganChase’s Oct. 13 earnings call, CFO Jeremy Barnum questioned the value of devoting more resources to estimating AI’s aggregate contribution, arguing that the effort could distract from implementation. A day later, Bank of America CEO Brian Moynihan said the bank had implemented identifiable paybacks for about 130 AI projects with benefits expected to reach two to three times their costs over time. 

Are the two banks taking different views of AI’s value, or answering different questions?

The contrast is a useful starting point for the first of three articles drawing on a recent AI in Banking Executive Roundtable hosted by American Banker, our latest survey of banking professionals and third-quarter earning calls. This first article examines what financial measurement practices distinguish successful AI adopters from others, while the next two explore scaling and governance, respectively.

Measurement matters because AI adoption is a series of investment decisions. Banks need evidence to determine whether an experiment is working, whether an application deserves further investment and whether improvements in productivity translate into business results. Without it, management risks confusing activity with value. Yet demanding precise financial returns too early can also discourage worthwhile experiments.

This is the thinking behind a framework we have developed in earlier American Banker articles. It follows four stages of AI adoption, from experimentation to workflow improvement, business outcomes and financial outcomes, across three priorities: measure, scale and govern. All three matter throughout the journey, but the evidence needed to support decisions changes as initiatives advance. We previously explained the measurement angle of AI adoption in Beyond ROI: How banks can better measure AI impact and illustrated the broader framework in Benchmarking AI adoption: What U.S. Bank’s playbook tells us. 

Adoption is ahead of measurement

American Banker’s latest AI adoption survey, conducted Sept. 18-28 among 200 bank and credit union professionals involved in AI initiatives, found that 97% either had active generative AI models or were at least piloting or experimenting and 83% reported improved work output. Yet only 34% had quantified net financial impact for most or all active initiatives. The measures are not directly comparable, but the contrast illustrates the distance between adopting AI, improving performance and establishing its financial contribution. 

One measurement practice stands out among banks better able to demonstrate AI’s financial value: establishing a baseline before implementation. Only 47% of respondents said most or all AI workflows were measured against pre-AI performance. Among those respondents, 57% had quantified net financial impact for most or all initiatives, compared with 14% of everyone else.

Consider an AI application that summarizes customer calls. Producing a summary in seconds says little about its economic benefits unless the bank knows how long the task took before AI, how much time employees now spend checking the output and what happens to the capacity release. A survey respondent at a national bank described precisely this problem: The institution had measured neither the original task time nor the time spent validating AI results, leaving savings unclear. 

What to measure and when?

Our survey finds that 58% measured employee time saved, 52% measured cost and only 30% measured revenue. 

At our executive roundtable, participants questioned the practicality of calculating ROI for individual initiatives, especially early on. Some favored broader measures, including efficiency ratios and cycle times, because benefits may be visible even when attributing them to a particular application is difficult. Others cautioned that demanding precise financial returns too early could discourage experimentation. An initial project may be worthwhile because it establishes feasibility or reveals a better use case, not because it immediately reduces expenses.

The survey results reflect that tension. While 55% of respondents said decisions to continue, expand or discontinue AI initiatives were always or often informed by demonstrated financial value, 41% reported that they only sometimes or rarely relied on it. Managers have to make choices before complete financial information is available. But that is different from making decisions without evidence. Measurement should help make these decisions, not simply document financial results after implementation. 

This helps explain the apparent contrast between JPMorgan and Bank of America. Barnum was discussing the difficulty of attributing a single aggregate financial figure to AI across a large institution, including the challenge of separating long-established machine learning models from newer LLM and agentic AI applications. Moynihan was describing identifiable project-level paybacks, alongside projections of roughly $250 million to $300 million in AI benefits this year and $700 million next year. Project-level evidence can guide investment decisions even when enterprise-wide estimates remain elusive. Neither approach supplies a universal standard for every initiative at every bank.

What successful AI adopters measure differently

McKinsey’s 2026 State of AI survey offers a broader perspective. Organizations reporting significant AI value and attributing at least 5% of EBIT to AI were twice as likely as other organizations to have defined processes for measuring impact. The finding suggests that systematic measurement is among the practices distinguishing stronger performers.

American Banker’s survey provides another clue about the relationship between stage of adoption and measurement in banking. Among respondents with two or more types of AI actively employed, 53% had quantified net financial impact for most or all initiatives, compared with 21% of those with only one type active. Similarly, 66% of the former measured most workflows against pre-AI performance, compared with 27% of the latter. These findings suggest that more mature adopters and stronger measurement practices often go together.

During the roundtable discussions, executives recognized the need to demonstrate value but also cautioned against demanding financial returns prematurely. A director at a national bank responding to our survey expressed a similar view, emphasizing the value of experimenting with new technologies and ways of working rather than immediately documenting cost savings. The distinction is between establishing measurement practices early and requiring financial returns early. Banks need the former to make informed decisions without allowing the latter to discourage worthwhile experimentation.

The different approaches described by JPMorgan and Bank of America also suggest that measurement should serve the decisions being made, rather than a single reporting standard. This is consistent with distinguishing four stages of AI adoption: experimentation, workflow improvement, business outcomes and financial outcomes. Initial measures can focus on technical feasibility and output quality, followed by improvement in throughput, accuracy and error rate. As applications advance, banks can evaluate their contribution to business performance and ultimately their net financial impact. At each stage, the purpose is to determine whether an initiative warrants further investment, not merely to calculate its returns.

The evidence does not establish that stronger financial performance is the result of better measurement. Nevertheless, McKinsey’s high-performer findings, together with our evidence on baselines, do suggest that measurement deserves attention as part of the adoption planning and execution process rather than an exercise undertaken afterward. Banks should establish baselines early, while adjusting their requirements as initiatives advance and investments grow. The goal is not to demand an ROI calculation for every experiment, but to have enough reliable evidence to decide what deserves further investment, what needs improvement and what should be discontinued.