The smartest AI might be the one that leaves us alone

One of the ideas I’m always trying to tell my students is that their brains are something, an asset, if you like, they need to protect. The human brain is an amazing optimizer: Provide it with an effortless way to achieve something, and it will quickly start to use it. We used to memorize a ton of phone numbers a few decades ago. Now we can hardly memorize our own, simply because our brain knows the number is a few clicks away.
A clear example is why I never use AI to write my articles: simply because, for me, in my personal methodology as a professor of innovation, writing is my way of making concepts stick in my brain, concepts I will need later to teach and discuss. If I ask ChatGPT, Claude, Gemini, or any other chatbot to write my articles for me, I would lose much of the learning that writing gives me.
This doesn’t mean I don’t use AI: I can use it to translate certain parts, to search for relevant articles to back up an assertion, or to assess whether the concepts I try to explain come across clearly. But I don’t want to use it for writing, because writing is a skill I want to protect. And like a muscle, if you don’t exercise it, it quickly atrophies.
For a manager, the skill worth protecting might be decision-making: An AI that makes every choice could undermine precisely the judgment that person needs to develop. The useful behavior depends on the objective, and sometimes that objective requires the system to hold back.
What happens when we move from personal to corporate use? Can an AI learn when its intervention would make things worse? That capability might receive very little applause in a product demonstration, but it could be enormously valuable in practice.
The sales agent that lets us be
Imagine a company using a sales agent. When it identifies a customer approaching renewal, it follows a standard procedure: It quickly reviews the account, writes a personalized email, offers a discount, and schedules a follow-up. All of it in mere seconds. And the audience claps enthusiastically.
But what if our customer would have renewed anyway? Then, the end result is that we have wasted an interaction, risked alienating the customer, and surrendered margin unnecessarily. What if our customer is a happy one, and the right thing to do would have been just to wait for him or her to renew voluntarily?
Even when all tasks are perfectly executed, the decision to perform them could have been perfectly wrong.
The cost of being helpful
We know how this feels when we’re working with real people: Experienced employees often develop a sense of when to follow up, when to escalate an issue, and when making too many interventions will look messy, send the wrong signal, or make the customer upset. How do they know that? It comes with experience; it’s learning that compounds over time, with more and more iterations.
That judgment is something we pay to have. However, when we evaluate AI, we tend to be fascinated with visible activity—with messages generated, tasks completed, workflows executed, etc. The equivalent is if we were dealing with a hyperactive employee. And as we probably know, someone who sends twice the number of messages is probably not twice as valuable.
In other words, just because an action can be automated and, therefore, performed easily, doesn’t mean that we should be performing it all the time.
Look what happens with AI coding assistants: In “When to show a suggestion? Integrating human feedback in AI-assisted programming,” Microsoft researchers used data from 535 programmers to retrospectively evaluate a system that selectively withheld suggestions that were apparently likely to be rejected by the developers. They also found that optimizing for suggestion acceptance could produce lower-quality suggestions. Getting people to accept more assistance was an imperfect measure of helping them.
Learning when to intervene
In the case of our renewal agent, a system that learns from a lot of comparable cases could perhaps investigate which interventions help with whom. For instance, some customers don’t like the feeling of being “directed” or “influenced,” and are better left to their own devices. Some like a particular type of interaction through a particular channel, but they hate others. Or they want to talk to the same customer representatives every time because they remember them.
The function to protect would be the ability to spot these cases. Do not automate; optimize. But optimization is not necessarily something that today’s corporate AI does well. Today’s corporate AIs can generate splendid emails, but following the before and after and deciding to do nothing is usually not an alternative.
It’s the experience from all previous cases, processed along with the specific characteristics of every customer, that leads to the lessons learned, to the continuous improvement, to the process being optimized. And that ability to improve decisions through accumulated experience is precisely what you need.
Give waiting a fair chance
When buying or deploying AI, managers should, therefore, find out what outcome the system is theoretically trying to improve, and design ways to know whether its actions helped or not. We need to be honest here: A renewal following an email does not necessarily establish that the email was the reason for the renewal. Customers differ, circumstances change, and useful comparisons may require carefully designed experiments. Simply storing more interaction histories will not settle the question.
This includes maintaining the necessary human oversight, the now-famous “human-in-the-loop”: A missed warning or an unanswered complaint can be costly. People must define the obligations and limits within which the system can learn.
I haven’t seen an AI system that reviews an account, checks all the available evidence from the corporate ERP or CRM, and decides that contacting the customer would probably make things worse. So the final outcome would be that nothing gets sent, no discount is offered, and we just let the customer evolve naturally. No hyperactivity. Why? Because our corporate learning optimized it, and the accumulated learning behind that decision should remain under our company’s control, available even if we replace the model or the vendor powering our agents.
It might be a disappointing demo. But it could be excellent business.