Do you remember the last time a company surprised you by being good? Someone picked up on the second ring, understood the problem before you finished explaining it, and fixed it without making you feel like a ticket. That feeling is getting rare. And I am convinced we are automating it out of existence on purpose, one helpful little chatbot at a time.
I build cloud and AI infrastructure for a living. I host models. I sell the compute that AI runs on. So when I say I am against automating everything, hear me clearly: I am not against AI. I am against the reflex. The instinct that has taken over boardrooms and product roadmaps, where every problem, every workflow, every human interaction gets the same answer before anyone asks the question. Can we automate this? Almost always, the smarter question is: should we?
That question is what this piece is about. Where AI automation belongs, where it does not, and why the difference matters more than the people pushing it seem to think.
The reflex, and where it comes from
There is a particular kind of project I have watched play out a hundred times now. Someone in a meeting says "AI" and a budget appears. Not because anyone found a problem worth solving. Because the technology exists and not using it started to feel like falling behind. That is the tell. That is automation searching for a problem.
Don't get me wrong, cost matters. Margins matter. I run a company; I sign the bills. But somewhere in the last two years "we could automate this" quietly became "we must," and the must arrived without anyone checking whether the thing being automated was the thing customers valued. The reflex skips the only step that mattered. It treats every human task as overhead to be eliminated rather than asking which tasks were the point.
Here is the part that should bother you more. A lot of these systems do not even work the way their champions believe they do.
The limits are real, and they are documented
We talk about these models like they think. They do not, in the way we mean it, and the people building them know it. Apple's machine learning team published a paper in June 2025 with a title that says the quiet part out loud: "The Illusion of Thinking." They tested frontier reasoning models on controlled puzzles and watched something unsettling. As the problems got harder, accuracy did not gently decline. It hit a wall. The models suffered what the researchers called "a complete accuracy collapse beyond certain complexities" (Apple Machine Learning Research).
And the strangest finding: as the puzzles approached that cliff, the models spent LESS effort reasoning, not more, even with plenty of compute left in the tank. They gave up early, then dressed the failure in confident prose.
I am not citing this to dunk on the technology. I find the result fascinating, and the work is the kind of honest research the field needs more of. I am citing it because of what we are doing with these systems while that limit sits in plain view. We are handing them disputes, appeals, medical questions, the hard and human edge cases, the exact territory the research says they fall off a cliff in. We automated the easy 80 percent, declared victory, and quietly routed the painful 20 percent, the part that needed judgment most, to a system that confidently makes things up when the problem gets hard.
Klarna automated its way into a wall
If you want to see the reflex run its full course, look at Klarna. The fintech company went all in. It built an AI assistant, announced it was doing the work of 700 customer service agents, and made the cost savings the headline. For a while it looked like the future.
Then the future showed up. By May 2025 the CEO, Sebastian Siemiatkowski, was walking it back in public, admitting the all-AI push had produced "lower quality" service and that the company would start hiring humans again (Forbes). His words, paraphrased by everyone who covered it, amounted to: we went too far. Cost and efficiency had quietly eaten the quality, and the quality was the relationship.
That is the whole story of over-automation in one company. Not a villain twirling a mustache. Just a reasonable-sounding optimization, applied past the point where it helped, until the thing being optimized was gone. They did not save money on customer service. They automated customer frustration, then paid to undo it.
Where AI absolutely belongs
So let me draw the line clearly, because the cynical version of this argument is lazy and I want no part of it.
AI is doing extraordinary things, and some of them are the reason I work in this field. In medical imaging, models flag tumors and fractures that tired human eyes miss at the end of a long shift, and they do it as a second reader, raising the floor rather than replacing the radiologist. In drug discovery, AI has cracked open protein folding, work that used to take a doctorate and a decade. In accessibility, live captioning and real-time description give deaf and blind people access to a world that was closed to them, and that is not a convenience, it is a door. Translation across languages, the grunt work of scanning a million rows for an anomaly no human would have the patience to find, surfacing the one paragraph in ten thousand pages that matters: this is where the technology earns every watt it burns.
Notice the pattern. AI belongs where it augments a person rather than impersonating one. Where the task is genuinely beyond human scale, or beyond human stamina, or beyond human speed. Where being wrong is caught by a human in the loop, not shipped straight to the customer. Where it raises the ceiling on what people can do instead of lowering the floor on what they are allowed to expect.
That is automation with intent. I will defend it all day.
Where it does not
And then there is everything else. The places we automate not because it is better, but because it is cheaper and we stopped asking the next question.
The apology that means nothing because a machine generated it. The "creative" campaign with no person behind it, addressed to no one in particular. The support line engineered specifically so you cannot reach a human, where the bot is not there to help you, it is there to exhaust you into giving up. The newsletter, the condolence note, the performance review, all the small human exchanges we are quietly outsourcing because we can, hollowing out the gesture until only the shape of it is left.
These were never inefficiencies. The friction was the point. A handwritten reply takes longer because the time is the message. When you automate the time away, you keep the words and lose the meaning, and everyone on the receiving end can feel the difference even when they cannot name it.
There is a cost underneath all of this that almost nobody prices in. Every automated reply, every generated image nobody asked for, every chatbot spun up to dodge a phone call, runs on a data center that drinks power and water. As someone whose work is making infrastructure run leaner and on upcycled hardware, I will say it plainly: burning energy to manufacture fake interactions is the worst trade I can think of. We are spending the planet to avoid spending each other's time.
A builder's plea, and a line I keep
So where does that leave someone like me, who profits when AI runs?
It leaves me drawing a line on purpose, in my own work, where it costs me something. We use AI where it makes our platform faster and our customers' lives easier, and we keep humans on the other end of the things that need a human. When you have a problem with your infrastructure, you reach an engineer who built the thing, not a script trained to deflect you. That is a deliberate choice, and I will keep making it even as the pressure to automate it away gets louder, because the pressure is exactly the reflex I am warning you about.
I am not naive about this. I know efficiency is not optional and I know I am hedging my own idealism against my own invoices every month. But intent is the whole game. Automate the toil, the scale, the things no human should have to do by hand. Protect the judgment, the craft, and the connection, the things that were the point. The technology is astonishing. What we choose to point it at is on us.
You cannot automate humanity. And the moment you try, you find out it was the only thing worth keeping.
