Open Weights and the Future of Private AI
Open-weight models let you run capable AI on hardware you control. Here is why I think that matters more than any leaderboard, for privacy, cost, and independence.
Open-weight models let you run capable AI on hardware you control. Here is why I think that matters more than any leaderboard, for privacy, cost, and independence.
We chose to run our own AI infrastructure instead of building on closed APIs. Here is the case for owning your stack, and the honest cost of doing it.
Years of building infrastructure taught me that trust is earned quietly, honesty about limits beats big promises, and locking people in is a slow death.
How much RAM, VRAM, and disk you need to self-host a 7B, 13B, or 70B open-weight model, with cited memory figures and the speed you can expect.
The AI energy cost is treated as an inconsequential while the power demand is at the size of some countries and why efficiency, not scale, is the future.
Remember when running a computer meant the program lived on your machine, your files stayed on your disk, and nobody on the other side of a pipe got to watch you work? I do. And somewhere in the last few years we quietly handed that away. We started renting intelligence by the token from a handful of companies, and we let our most sensitive questions, our code, our contracts, our half-formed ideas, travel to a server we will never see, governed by a policy we did not write.
I am not here to tell you that is evil. The frontier closed models are extraordinary. But I want to talk about the door that open weights AI just kicked open, because I think it changes who gets to own the future.
Let me draw the line cleanly, because the industry blurs it on purpose. An open-weight model is one where the trained parameters, the numbers that make the model think, are published for you to download and run. You get the brain. You can put it on your own hardware and never phone home. Open source is a bigger claim: the code, the training recipe, the data, and a license that lets anyone use it for anything. Most "open" models are the first thing wearing the costume of the second.
That distinction is not pedantry. It is the whole game.
Take Meta's Llama. The weights are right there, free to download, and they are genuinely good. But read the license. If your product crosses 700 million monthly active users, you "must request a license from Meta, which Meta may grant to you in its sole discretion" (Meta Llama 3 License). The Open Source Initiative, the people who literally define the term, will tell you flatly that this is not open source, because it discriminates by who you are and what you build (Open Source Initiative). So Meta gets the halo of openness while keeping a leash. Clever. Not honest.
And here is the part that surprised even me. Open weights, even with a restrictive license, STILL change everything. Because the thing that matters for your privacy is not the license at all. It is where the model runs.
A model you download is a model that cannot watch you. That is the sentence I would tattoo on the industry if I could.
When you call a hosted API, every prompt is a confession sent to a stranger. Your patient notes. Your unreleased product. The internal email you are asking it to rewrite. You are trusting a retention policy, a privacy page, and the assumption that nobody changes the terms after you have built your business on top of them. Maybe that trust is well placed. Maybe. But trust is not the same as control, and for some data the difference is the entire job.
On saturated consumer hardware, you do not buy small-model quality with a faster framework. You buy it with better data and a better training objective, with numbers to back it.
Open weights collapse that problem. You pull the weights down. You run them on a box you own, or on infrastructure you rent but command. The data never leaves. No retention policy to read, because there is nothing to retain on someone else's disk. For a hospital, a law firm, a defense contractor, a startup with a single idea worth protecting, this is not a feature. It is the difference between using AI and being allowed to use AI.
Then there are the two unglamorous things nobody puts on a launch slide: cost and lock-in.
Cost first. Per-token pricing feels cheap until your product works. Then it scales linearly with your success, which is a strange thing to wish on yourself. A model you host has a fixed, predictable shape to its bill, and you can size the hardware to the job instead of paying a premium on every word forever.
Lock-in second, and this is the one that keeps me up. When you build on a closed API, you are building on rented land. The owner can raise the rent, deprecate the model your whole pipeline was tuned against, change the safety filters so yesterday's working prompt returns a refusal today, or simply decide your use case is no longer welcome. You will have no recourse, because you never held the asset. An open-weight model is a file. Files do not get deprecated out from under you. You can run a 2025 model in 2035 if it does the job, and nobody can take it back.
Here is my self-aware hedge, because I refuse to sell you a fairy tale. At the absolute top, the closed frontier models are usually ahead. They have more compute, more data, more polish, and on the hardest reasoning tasks the gap is real. If you need the single most capable system on earth for a one-off problem and the data is not sensitive, call the API. That is a legitimate use. I do it.
AI has legitimate uses that have nothing to do with the hype. Reading a radiology scan. Folding a protein. Captioning the world for someone who cannot see it. I am skeptical of the breathless promises, the chatbot bolted onto every product like a solution searching for a problem, the data centers drinking rivers to autocomplete a tweet. But the technology itself is not the villain. The dependency is.
And the gap is closing faster than the closed labs would like you to believe. In January 2025, DeepSeek released R1, a 671-billion-parameter reasoning model, under a plain MIT license, weights free to download, commercial use allowed, derivatives allowed, no monthly-active-user trapdoor (DeepSeek-R1 on Hugging Face). MIT. The same four-paragraph license that runs half the software you use every day. A frontier-class reasoning model, yours to keep, yours to modify, yours to run in a room with no internet. That is not a research curiosity. That is a shift in who holds the power.
Qwen, Mistral, the smaller Llama variants, the steady drip of capable open releases: the floor is rising. The question is no longer "can I run a useful model myself?" It is "do I have a good reason not to?"
I am not content to write about this from the sidelines. With Veritate, our open-source work on training and inference, we are exploring how far you can push capable models you fully control, on efficient hardware, without the river of energy and the wall of dependency that the hyperscale path takes for granted. The goal is not to beat the frontier on a leaderboard. The goal is to prove that "private, owned, and good enough to do the job" is a place most people should be allowed to stand.
That ethos runs through everything we build at Carpathian: infrastructure you can reason about, priced for what you use, owned in the ways that matter. I do not think the future of AI belongs to whoever has the biggest closed model. I think it belongs to whoever makes capable AI something ordinary people and ordinary companies can hold in their own hands.
Open weights are how the second internet stays the people's internet instead of becoming five companies and a billing meter. Use the closed frontier when it earns its place. But know that the door is open now, and you do not have to send your most important data to a stranger to think clearly.
Download the brain. Run it in your own house. That is what private AI was always supposed to mean.