Open source AI models, trained and hosted on hardware we own

An open weight model is a file rather than a subscription. Anyone with the file and enough compute can serve it, which is why the same model can run in our racks, in yours, or on a laptop, and why it does not disappear when a provider retires an endpoint. We host open source models, we train our own, and we publish the research either way.

You can talk to one right now without an account in the free AI chat.

Try a ModelInference API
carpathian.ai / ai / models
shell / list what your key can reach
# the model list comes from the API, not from a page that goes stale
curl https://api.carpathian.ai/ai/models \
  -H "x-api-key: $CARPATHIAN_API_KEY"

# then call one of the identifiers it returned
curl https://api.carpathian.ai/ai/chat/completions \
  -H "x-api-key: $CARPATHIAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "<id from /models>", "stream": true,
       "messages": [{"role": "user", "content": "Explain open weights in two sentences."}]}'
python / openai sdk
# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.carpathian.ai/ai",
    api_key=os.environ["CARPATHIAN_API_KEY"],
)

for model in client.models.list():
    print(model.id)

# the weights are open, so this same identifier can be
# stood up somewhere else without rewriting the caller

Open is four separate things

The phrase gets used for releases that share almost nothing with each other. Splitting it into its parts is the only way to tell what a given model grants you.

carpathian.ai / models / what open means
  • Open weights
    The trained parameters are published and redistributable. This is the line that decides whether anyone other than the author can serve the model.
  • Open code
    The training and inference code is published alongside the weights, so the architecture can be read rather than inferred from behavior.
  • Open data
    The corpus or the recipe that produced it is published. This is the rarest of the three, and its absence is why most open weight releases are not reproducible from scratch.
  • Open license
    What you are permitted to do with the result. Licenses on open weight models differ widely, from permissive to use-restricted, and the license is the part you read before you ship.
Weights
Published
Interface
OpenAI-compatible
Model list
From the API
Compute
United States

Open weights is the load-bearing one. The other three change how much you can learn and what you are permitted to do, but weights are what decide whether the model can be run by anyone other than the party that made it.

What changes when the weights are public

Three practical consequences, none of which are about ideology.

The endpoint stops being the product

With a closed model you are choosing a company as much as a capability, because the only way to reach it is the endpoint its owner operates. With open weights you are choosing a capability and then, separately, choosing who runs it for you. Those two decisions come apart, and the second one can be revisited later without touching the first.

What you are choosing
closed modelone endpoint, one operator
open weightsany operator with the file
the differencewho can serve it

Versions stop moving underneath you

A model that improves is still a model that changed, and a prompt tuned against one revision can quietly degrade against the next. When the weights are published, the version you validated against is a specific artifact rather than a label pointing at whatever is current, so an upgrade becomes something you schedule and test rather than something you discover from a support ticket.

Version behavior
hosted, closedupdated when the owner decides
open weightsthe file you pinned is the file you get
regression riskyours to schedule

Leaving is a migration, not a rewrite

What it costs to leave measures a hosting relationship better than what it costs to arrive. Because the weights are public and the interface is the one your client already speaks, moving a workload off our infrastructure means standing the model up elsewhere rather than rewriting the application around a different provider's shape. We would rather compete on running the model well.

What moves with you
weightspublished, redistributable
interfaceOpenAI chat completions
identifiersreturned by /models

The models we train

Veritate is our own research program, and it is where the training work happens. It is a tokenizer-free byte-level architecture with a 256-symbol vocabulary, served by an inference engine we wrote by hand in C with no CUDA, no framework, and nothing to install beyond a single binary.

The quantization sits inside the training rather than after it. Instead of compressing a finished model and accepting what that costs, the low precision is folded in from the start, so the network learns under the arithmetic it will eventually run on. The same approach applies to activation sparsity, which is trained in rather than pruned afterward.

Every experiment is published, including the ones that regressed. The engine, the plugins, and the training scripts are on GitHub, and the research log reads as a sequence of measurements rather than a set of claims.

Veritate ResearchRead the Code
veritate / architecture
EngineHand-coded C
KernelsAVX-512 SIMD
QuantizationINT8 / INT4
RuntimeSingle binary
ComputeCPU, batch 1
Vocabulary256 bytes
INT4 weights
Veritate models are small research models, considerably smaller than the models serving the public chat and the inference API. They exist to test whether capable inference can run without a GPU floor, and they are described that way everywhere they appear.
carpathian.ai / dashboard / instance
OpenAI-compatible endpointChat completions, with streaming
Per-instance rate limitRequests per minute, set by you
IP allowlistPer instance
Geo-aware firewallMonitor or enforce
Token budgetTotal or monthly

An instance is scoped to your organization. You set the rate limit, the allowlist, and the budget, and the firewall decides what happens when an address nobody has seen before arrives.

The models we host

Alongside the research, we run open source models on our own compute in the United States and expose them through an OpenAI-compatible endpoint. You pick a model, we keep it loaded, and you call it the way you would call any other REST service.

The model list is served by the API rather than printed in marketing copy, because a printed catalog is a second place for the truth to live and it drifts. Call the models endpoint with your key and you get exactly what that key can reach.

Usage is charged per token. Every request is logged with its token counts and response time, so the invoice traces back to something you can read.

  • The weights are published, so the model does not belong to the endpoint serving it
  • The interface is the OpenAI chat completions format, which your client already speaks
  • The model identifier comes from the API, so nothing in your code is pinned to our naming
  • Your requests are logged with token counts and response times, and the log is yours to read
  • There is no proprietary format holding a conversation, a prompt, or a fine-tune hostage
Inference APISee Pricing

From the shell you already have

Two commands, and neither of them is a Carpathian tool you had to install first.

carpathian / models / bash 80x24
$ curl -s https://api.carpathian.ai/ai/models -H "x-api-key: $CARPATHIAN_API_KEY" | jq '.data[].id'
the identifiers this key can call, straight from the API
$ python -c "import openai; print(openai.__version__)"
unchanged: the SDK is the one you already had
$

The identifiers come back from the API, so nothing in your code is pinned to a name we chose for a page.

Questions about open weights

What is an open source AI model?
A model whose weights are published under a license that lets you download, inspect, run, and redistribute them. The term covers a range: some releases publish weights and nothing else, some add the training code, and a smaller number publish the data recipe as well. Open weights is the part that changes who is allowed to run the model, which is why it is the part that matters when you are choosing where to host.
How is that different from calling a commercial API?
With open weights, the model is a file. Anyone with the file and enough hardware can serve it, which means the same model can run on our infrastructure, on yours, or on your laptop, and it does not stop existing because a provider retired it. A closed model can only ever be reached through the endpoint its owner operates.
Which models can I call?
The model list is served by the API rather than written into a page, so it stays accurate. Call https://api.carpathian.ai/ai/models with your key and you get every model that key can reach, with the identifier to pass in a request. The public chat exposes a model without a key at all.
Do you train your own models?
Yes, through Veritate, our open research program. It is a tokenizer-free byte-level architecture with a hand-coded inference engine in C, trained with quantization built into the training rather than applied afterward. Those are small research models, far smaller than the models serving the main chat, and the code and the research log are published for anyone to read.
Can I run the same model somewhere else later?
That is the property you are buying. Open weights mean the model does not belong to the endpoint, so moving is a matter of standing it up elsewhere rather than rewriting against a different provider's interface. We would rather compete on what it costs to run the model well than on how hard it is to leave.
Does an open weight model mean I can use it commercially?
It depends on the license, and the licenses differ. Some are permissive, some add use restrictions, and some limit redistribution above a size threshold. Read the license for the model you plan to build on, the same way you would for any dependency you ship.
How do I call a hosted model from code I already have?
The endpoint follows the OpenAI chat completions format. Point your client base URL at https://api.carpathian.ai/ai, authenticate with your key, and the OpenAI SDKs and client libraries keep working without code changes. Streaming, system messages, and temperature behave the way your client expects.

Related reading

Open Weights and the Future of Private AISelf-Hosted LLM vs API: A Cost BreakdownTrained-In, Not Bolted-On

Why we run small models on hardware other people threw away

Open weights are half of the position. The other half is that a model small enough to run on ordinary hardware can run on hardware that already exists, which is the argument the efficiency page makes with the measurements attached.

Green AIOur Facilities

Talk to one before you integrate one.

The public chat needs no account and costs nothing, and the models behind it are the ones the API serves. When you are ready to build, it is a base URL and a key.

Try a ModelInference APITalk to an Engineer