The Cost of "Free" AI: Who Pays for Inference
Free AI is not a gift. Someone pays for every query, in energy, in data, in dependency. Here is who, and why I'd rather pay honestly for compute I control
Free AI is not a gift. Someone pays for every query, in energy, in data, in dependency. Here is who, and why I'd rather pay honestly for compute I control
We throw away a staggering amount of perfectly useful compute. Here is why a lot of AI and cloud work can run on older, upcycled hardware, and where it honestly cannot.
We ship megabytes of script to display a paragraph of text, and we call it progress. I think we forgot that efficiency was ever a virtue.
We chose to run our own AI infrastructure instead of building on closed APIs. Here is the case for owning your stack, and the honest cost of doing it.
I am convinced the future of useful AI is small, efficient, often task-specific models, not an endless race for the biggest one. Quality per watt beats brute force.
Remember when a search was just a search? You typed a question, you got ten blue links, and nobody handed you a chat window that wrote your email, planned your week, and told you it loved you. That world is gone. In its place we got something that feels like magic and costs, the marketing insists, nothing at all. Free model here, free tier there, unlimited messages, no card required.
I build cloud and AI infrastructure for a living, so let me say this plainly. There is no free AI. The cost of free AI is not zero. It has only been moved somewhere you cannot see it. Someone pays for every single inference, in electricity, in water, in your data, in your growing dependence on a thing you do not own and cannot inspect. And more often than the brochure admits, the someone is you.
Don't get me wrong. I am not here to tell you AI is a scam. It is not. I will defend the legitimate uses of this technology all day, the protein folding, the medical imaging that catches a tumor a tired radiologist would miss, the screen reader that finally describes an image to someone who cannot see it. Those are worth the cost. But "worth the cost" is a sentence with two halves, and the industry has spent three years getting us to forget the second one.
Training a model is the expensive headline. It is the part everyone writes about, the rooms full of GPUs, the months of compute, the eye-watering budgets. But training happens once. Inference, the act of running the model every time you ask it something, happens forever. Every prompt, every token, every "regenerate response" is a small bill paid in compute and power, and it never stops.
Here is the part that should bother you. The companies handing out free chat are paying that bill on every message you send, and they are not charities. The reference prices tell the story even when the consumer product hides it. CloudZero's 2026 breakdown of model API pricing put a single flagship tier, GPT-5.4 Pro, at thirty dollars per million input tokens and one hundred and eighty dollars per million output tokens (CloudZero). Read that again. The output, the part that talks to you, is the most expensive part to produce. Now ask yourself why a company would let you generate that output, in volume, for free.
They would not. Not without a return. The free tier is not a gift, it is a customer acquisition strategy, a data pipeline, and a dependency engine, dressed up as generosity. The economics of inference make "free" a business decision, not a present.
Every token you generate spins up a chip somewhere, and that chip runs on a grid, and that grid is straining. The International Energy Agency estimates data centers used around 415 terawatt-hours of electricity in 2024, roughly 1.5 percent of all the electricity on Earth, and projects that to nearly double to about 945 terawatt-hours by 2030, just under 3 percent of global demand, with AI as the loudest driver of the growth ().
Years of building infrastructure taught me that trust is earned quietly, honesty about limits beats big promises, and locking people in is a slow death.
Three percent of the world's electricity. For an industry that mostly markets itself as weightless, "the cloud," that is a startling amount of mass. And it is not just power. It is water for cooling, land for buildings, copper and silicon and the rest of the supply chain behind every "free" reply. When you ask a free model to rewrite your grocery list in the style of a pirate, the planet still pays for the pirate.
This is the cost that troubles me most, because it is the one nobody opted into. You can choose to give a company your data. You cannot choose out of the grid the data center is pulling from. We are all on that grid together.
We have been here before. The phrase "if you're not paying for the product, you are the product" did not start with Facebook. It traces back to a 1973 video art piece by Richard Serra and Carlota Fay Schoolman called Television Delivers People, which argued that the product of commercial television was not the show, it was the audience, packaged and sold (Conversable Economist). Fifty years later, the medium changed and the sentence held.
Free AI is the newest, hungriest version of this trade. The currency is your data and your attention, and the meter is always running. Every conversation you have with a free assistant is, potentially, training material, a behavioral signal, a profile getting sharper. You are not the buyer in that transaction. You are the inventory.
And there is a subtler cost on top of the data one. Dependency. The more your work, your code, your writing, your thinking runs through a system you do not control, the more leverage that system has over you. The free tier today is the price hike, the rate limit, the feature paywall, and the quietly worse model tomorrow. We have watched this movie. We named the genre. The slow degradation of a free service after it has captured you has a word now, enshittification, and "free AI" is the most efficient onramp to it anyone has ever built. You do not get hooked on the thing you pay for with money. You get hooked on the thing that costs nothing, right up until leaving costs everything.
So here is where I land, and I will own it. I would rather pay for compute I can see than be the payment for compute I cannot. That is not idealism, it is arithmetic. The bill exists either way. The only question is whether it shows up as a line item or as a slow extraction you never consented to.
What does paying honestly look like? It looks like a price you can read, attached to a resource you control. It looks like running models on infrastructure where you know where the data goes, because the answer is "nowhere, it stayed with you." It looks like an inference endpoint that bills you for the tokens you use, full stop, with no second invoice quietly paid in your behavior. At Carpathian we run our own US-based hardware and host models on it, and I will be honest about the tradeoff, you pay money for that, money, every month. A free tier somewhere else will always undercut us on the sticker price. It just makes up the difference somewhere you cannot see.
I want to be careful here, because I am not pretending free tiers have no place. They do. A student learning to prompt, a hobbyist on a weekend project, a developer kicking the tires before committing, the free tier is exactly right for that. Try before you trust. I have used them. The trouble is not the existence of a free tier. The trouble is running your business, your private data, your livelihood through one and calling the bill paid.
What drives me a little mad is how thoroughly we have been trained to treat "free" as the natural state of software, as if electricity were free, as if engineers worked for nothing, as if a planet-scale compute cluster materialized out of goodwill. It did not. Somebody is paying. The genius of the free-AI model is convincing you that nobody is, so you never go looking for the invoice, so you never notice it has your name on it.
I am not interested in scaring anyone away from a technology I genuinely love and use every day. I am interested in honesty about its price. Build the protein-folding model. Catch the tumor. Describe the image to someone who cannot see it. Those costs are worth paying, out loud, on purpose. What I refuse to accept is a future where the default way to use AI is to hand a stranger your data, your attention, and a slice of the grid, and call it a gift.
So pay for your compute. Pay honestly, pay for what you use, pay to a place that will tell you where your data sleeps at night. The cost of "free" AI was never zero. It was just hidden, and the bill was always coming to you. The only freedom in this market is knowing what you are paying, and choosing to pay it on your own terms.
