Why We Self-Host Our AI: The Case for Owning Your Infrastructure
We chose to run our own AI infrastructure instead of building on closed APIs. Here is the case for owning your stack, and the honest cost of doing it.
We chose to run our own AI infrastructure instead of building on closed APIs. Here is the case for owning your stack, and the honest cost of doing it.
Open-weight models let you run capable AI on hardware you control. Here is why I think that matters more than any leaderboard, for privacy, cost, and independence.
Self-hosting an open-weight model or calling a hosted API? The drivers on each side, the break-even point, and the hidden costs.
I am convinced the future of useful AI is small, efficient, often task-specific models, not an endless race for the biggest one. Quality per watt beats brute force.
Years of building infrastructure taught me that trust is earned quietly, honesty about limits beats big promises, and locking people in is a slow death.
We self-host our AI. We run the models on hardware we control, behind an interface we wrote, in a building we can walk into. People ask me why, often with the implication that it would be smarter and cheaper to just call someone else's endpoint and get on with my life. Maybe. But I want to make the honest case for the other path, because I think more builders should at least consider it, and almost nobody tells them they can.
The thing nobody warns you about when you build on a closed API is that you are not buying a tool. You are renting a tenancy in someone else's house, and the lease can change. The model you tuned your prompts against gets a new version, and the new version answers differently. The price per token moves. The terms of service grow a paragraph. And then, eventually, the deprecation notice.
This is not a hypothetical; As of this writing, OpenAI's own deprecations page lists gpt-3.5-turbo and GPT-4 Turbo, two models that an enormous amount of production software was built on, being removed from the API on October 23, 2026 (OpenAI Deprecations). Six months of notice, by their stated policy, for a generally available model. Six months to re-test everything you shipped against a model that does not respond the same way.
Six months. For the foundation of your product.
I want to be fair here, because the rules let them do this and there are decent reasons for it. Old models cost money to keep running. Newer ones are better. The notice period is reasonable as these things go. I am not accusing anyone of bad faith. I am pointing at the shape of the arrangement, and the shape is this: the thing you built on is not yours, and the timeline is not yours, and one day the email arrives.
Don't get me wrong, the managed APIs are extraordinary. For a huge number of teams they are the correct choice, and I will defend that in a minute. But I want you to see the full bill, the part that does not show up on the invoice.
When you build on a closed inference API, you are buying convenience, and you are paying for it with three things you may not be tracking.
You pay with control. Your behavior is downstream of decisions you do not make and are not consulted on. A model update can change your output overnight. A rate limit can throttle you on your best day.
You pay with predictability. The token price today is not a promise about the token price next year. The whole industry is being funded into existence right now, and the people writing the checks expect a return. Pricing that was set to win the market is not the pricing that pays it back.
We throw away a staggering amount of perfectly useful compute. Here is why a lot of AI and cloud work can run on older, upcycled hardware, and where it honestly cannot.
And you pay, sometimes, with privacy. Every prompt your users type, every document they upload, every question they would not ask out loud, travels to a server you do not run and sits in a log you cannot read. For a lot of workloads that is fine. For the ones that touch health, or money, or a person's private life, it is the whole game.
None of this means the APIs are bad. It means renting has a cost that is easy to ignore until the day it is not.
So we built the other way. We host the models ourselves, on our own hardware, in US data centers we operate rather than reselling someone else's cloud. The inference runs behind an OpenAI-compatible API so people can point existing code at it without a rewrite, which matters, because lock-in is not a problem you solve by inventing a new lock.
Here is what owning the stack buys back.
It buys back the floor. The model does not vanish on someone else's schedule. If we choose to keep an older model running for the people who depend on it, we can, because the decision is ours. That single fact changes how it feels to build.
It buys back the bill. When you own the hardware, your cost is electricity and depreciation and the work it takes to keep it healthy. That is a number I can forecast. I cannot forecast a token price set by a company that is losing billions and will eventually need to stop.
And it buys back the data path. When the model runs on a machine we control, the prompt does not leave the building. I can tell a customer with a straight face exactly where their data went, because I know.
This is also why I spend time on Veritate, our open-source work on training and running language models. Not because I think we are going to out-scale the labs with thousands of GPUs, we are not, and I would be lying if I said otherwise. I work on it because understanding how these systems are trained and served from the bottom up is the only way I know to stay honest about what they can and cannot do, and to keep the door open for people who want to run their own instead of renting forever. Owning the knowledge is its own kind of independence.
Now the honest part, because a piece that only sells you the upside is the thing I am complaining about.
Self-hosting is work. It is a lot of work. You become responsible for the hardware, the drivers, the model weights, the serving stack, the updates, the failures at two in the morning. The big labs have armies of people whose entire job is to make inference fast and reliable, and you are choosing to do a slice of that yourself. For many teams, that tradeoff is wrong. If you are a small product that needs the frontier model for one feature and you have three engineers, calling an API is not the lazy choice. It is the smart one. Go do that, and do not feel bad about it.
There is also a capability gap, and I will not pretend it away. The largest hosted models are still better at the hardest tasks than what most of us can run on our own metal. If your product lives or dies on the absolute top of the benchmark, renting that capability is sometimes the only way to have it at all, at least for now.
So the case for self-hosting is not "everyone should." It is "more people could than think they can, and almost nobody tells them the costs of renting until the lease changes."
I care about this beyond our own balance sheet, and I will tell you why.
A computing world where intelligence is a metered utility owned by three companies is a fragile world. It is fragile for the builder who wakes up to a deprecation notice. It is fragile for the user whose private words live in a log they cannot see. And it is fragile for the planet, because the answer to "we need more capacity" keeps being "build another enormous data center," with the water and the power that implies, instead of "use what we have more carefully." Efficiency is not a marketing word to me. It is the difference between an industry that scales by growing and one that scales by waste.
Owning your infrastructure is a small vote against that fragility. It says the thing you depend on should be the thing you can see, fix, and stand behind. Not every workload needs to be self-hosted. But the option needs to keep existing, loudly, or it quietly stops existing at all. That is how defaults work. The easy path becomes the only path, and then it becomes the only path anyone remembers was a choice.
So this is the commitment, plainly. We are going to keep running our own AI infrastructure. We are going to keep the interface compatible so leaving us is as easy as arriving, because I would rather keep you by being good than by being a wall. We are going to keep working in the open where we can, on Veritate and on the boring, unglamorous problem of running models efficiently on hardware that does not cost the earth to power.
And when the next deprecation email goes out across the industry, the one that tells thousands of builders the floor is moving again, I want there to be somewhere to stand that does not do that to you.
