Preparing CAS teams for AI: Moving beyond software training

I recently watched a demo from a popular CAS tech stack vendor showing off the latest version of its AI-enabled platform. 

Processing Content

Playing the role of the accountant, the person giving the demo entered a simple prompt: Create a month-end payroll accrual and post the journal entry to the general ledger. 

The AI agent asked for all the files necessary to do its task, which the accountant made available. The agent did an analysis of all the information and then prepared a work program for the accountant to review and approve. Once approved, the agent did its work with no further instruction. 

Along the way, it noticed that the general ledger was missing a few necessary accounts, which it reported to the accountant, who fixed it. The agent created the journal entry. It had been trained to not post anything until the accountant gave the go-ahead, so it paused once again for the accountant’s approval. The accountant asked for a workpaper to tie the journal entry back to the source document, which the agent then produced.

On an actual client, the entire process might have taken 15 minutes. Impressive.

Is the staff ready?

I’ve worked with many staff, senior and manager-level accountants, and the first question that entered my mind was: “How many of them would be ready today to work in this environment?” 

So I asked the vendor rep what skills were necessary for accountants to work with their platform. He said there were none because the accountant and AI interact like they’re having a conversation. The agent is trained to work like an experienced, diligent accountant who needs little guidance but is disciplined enough to involve the user in all important decisions.

While technically correct, the answer missed the point, focusing on execution, while I was asking about skills. My concern is whether most CAS accountants have the analytic, judgment and decision-making skills to properly perform the work they’re now being asked to do.

• Can they consistently make judgments about whether a work program designed by AI is up to the task?
• Can a staff accountant reliably evaluate that payroll journal entry, even with the supporting work papers?

Play out the scenario a few years down the road, when the process I saw in the demo is the norm, and the accountants working with these platforms have never prepared a payroll journal entry. 

What happens when the tasks become more complex than preparing a journal entry? Will the staff accountant have the skills to evaluate the work of AI having never done the work themselves?

The erosion of on-the-job training

Professional judgment is developed through experience. Our profession is built on a model where less-experienced accountants learn their craft through multiple reps, clearing review notes and getting feedback from their more experienced supervisors. 

AI disrupts that on-the-job training model. We are quickly moving to a world where AI agents take many of the reps that used to go to staff accountants and serve as their training ground. 

This disruption affects all team members. Which judgments and decisions get pushed down to lower levels, what remains, and what new ones become the manager and senior’s responsibilities?

Training starts with a clearly defined workflow

Firms must now develop judgment and decision-making skills with limited on-the-job opportunities, which means that formal classroom learning must take on a bigger role. The problem is that ‘judgment” and “decision making” are abstract ideas that don’t lend themselves to a one-hour webcast. The only way for formal learning to be effective is to make these abstract ideas concrete, and in this case the best strategy is to ground them in the actual work. 

The first step in upskilling CAS professionals for the AI-enabled tech stack is not to design or source training, it’s to define the firm’s workflow. Without that foundation, the classroom training will lack the specificity most accountants need to transfer classroom learning to their actual jobs.

In the payroll example, do you want the accountant to start each workflow by asking the agent to perform the task? Maybe it makes sense for the agent to perform an analysis of the GL account to help the accountant better understand the reconciling items that might be discovered or form some preliminary idea of the amount of the accrual.

What if the firm has established work programs that it wants the agent to follow? In the demo I saw, the agent developed the work program for the accountant’s review, but that approach might not be right for every firm.

These are the types of choices the platform vendor will build into their demos, but ultimately, your firm will have to decide on the workflow that works best for your firm and clients.

Pilot test the workflow

To reduce risk, firms should pilot test the envisioned AI workflow on an actual client. To create this pilot, 

• Select your team. Evaluating the implications AI will have on your workflow requires both expertise and imagination. For most firms, this will mean using some of their most experienced people with heavy workloads. This will require the firm to make some decisions about how to create time in their schedule to implement an AI-enabled process on one of their engagements. Simply adding this responsibility onto their existing client and billable hour requirements will inevitably delay or even kill the pilot.
• Choose the client. The right client is one that reduces your risk for failure. Clients with reasonably clean books and an engaged contact who can tolerate the inevitable misstep are the best clients to try out a new process. To the extent the firm has a typical client or a large industry-specific practice area, choosing a test client from this population will make it easier to scale the pilot test findings and recommendations.

Understand the judgment, not just the process

CAS professionals are familiar with defining the flow of information in an accounting system and assigning roles and responsibilities for each team member. But remember that one of the primary objectives of defining the AI-enabled workflow is to have it serve as a basis for developing professional judgment.

For that reason, the pilot test should include gaining a clear understanding of where in the process important decisions get made, who makes those decisions and the basis for how they should be made. As the team works through the pilot, they should be sure to think through and document how they used their expertise and experience to evaluate the AI work product, decide what to check, formulate questions to guide their review and narrow their focus to the most consequential matters. They should take special note of where the agent got it wrong.

A solid foundation for classroom learning

Building judgment and decision making skills will take time, and firms will want to get started right away. Don’t wait for the pilot team to define the perfect workflow. Instead, consider the firm ready to begin designing its training programs when it can articulate for a given process:

• How information flows, how data is processed, the errors that could occur and the controls that must be in place
• Where in the process the AI agent acts and where the firm professional makes decisions
• Which person on the engagement team is responsible for which decisions and how those get routed to them
• The common errors that AI agents can make when performing their tasks 
• What a good reviewer actually checks and what they’re looking for

The adoption of an AI tech stack will affect more than how CAS teams work. It will create a gap in how professionals develop judgment. Formal and informal training must be designed to fill that gap, at least partially. The starting point is to define the AI-enabled workflow and, most critically, where and with whom the important decisions lie.