Models efficient enough to run anywhere

Veritate is our model research program. The models read raw bytes, the INT8 arithmetic is trained into them rather than applied afterward, and the engine underneath is hand-written C, so one binary decodes on an ordinary processor with no GPU and nothing to install. The engine, the training scripts, and the measurements are published.

The measurements behind every claim here are published at Veritate.

Veritate ResearchChat With One

Efficiency trained in, not bolted on

Quantization-aware training folds INT8 arithmetic into the model instead of compressing a finished one, so the network learns under the arithmetic it will eventually run on and the low-precision engine keeps nearly all of the full-precision model's quality.

INT4 weight compression takes the memory footprint down further. Ternary weights, which would remove the multiply from decoding altogether, are an active research direction rather than something the engine ships today.

Activation sparsity is trained in rather than pruned afterward, so the engine skips most of the feed-forward work on every byte, and speculative decoding drafts several bytes ahead and verifies them in a single pass without changing one output byte.

Read the CodeThe Research Log

A 256-symbol alphabet, and nothing in front of it

The vocabulary is the 256 byte values, which is the raw alphabet of every file. There is no tokenizer to train, no vocabulary mismatch between corpora, and no subword artifact to reason around, so any language, any codebase, and any binary format arrive as the same stream.

The engine is small enough to read end to end. Residual stream, feed-forward activations, attention scores, logit lens, and direct logit attribution are all readable while it runs, which is what makes a model this size worth studying rather than only worth calling.

What it takes to run one

  • A processor. The kernels are hand-written AVX-512 SIMD, and decode runs at batch 1, which is what one person talking to a model looks like
  • One binary. No CUDA, no driver, no Python runtime, and no framework to keep current
  • Weights in INT8, with INT4 compression on the parts that are memory-bound rather than arithmetic-bound
  • No network path. The model is a file on the machine, so nothing about a prompt has to leave it

Why we train them this way

The premise is that the hardware floor under capable inference is lower than the industry builds for, and the way to find out is to keep lowering it.

Power is a design constraint

A model that decodes inside a processor's registers draws what that processor draws. Nothing is spent keeping accelerators fed, and nothing is spent on hardware bought to clear a floor the model never needed.

The hardware already exists

Machines that cannot host a GPU are everywhere, including in our own racks. A model that runs on them extends the life of hardware that would otherwise be replaced to run something heavier.

A model that stays a file

The result of all of it is one binary and one set of weights. That is what makes the model portable, auditable, and yours to run somewhere else without asking us.

What open means here

Open weights on their own let you run a model. These are the parts that let you check it.

  • The engine
    The C inference engine and its SIMD kernels, which is the same code that decodes a byte on any machine we put it on.
  • The plugins
    The pieces the engine loads around the model, so the parts can be read and swapped rather than taken on trust.
  • The training scripts
    How the models are trained, quantization and activation sparsity included, rather than a description of how they were trained.
  • The research log
    The measurement behind every number we print, with the experiment that produced it next to it.
Veritate on GitHub

If you need a bigger model

The public chat and the inference API serve larger open weight models on our own compute in the United States. Point an OpenAI client at the endpoint, and the models endpoint hands back the ones your key can reach.

Inference APITry AI Chat Free

Questions about the models we train

What is an open source AI model?
A model whose weights are published under a license that lets you download, inspect, run, and redistribute them. The term covers a range: some releases publish weights and nothing else, some add the training code, and a smaller number publish the data recipe as well. Open weights is the part that decides who is allowed to run the model at all.
Which models does Carpathian train?
Veritate, our open research program. The models are tokenizer-free and byte-level, with a 256-symbol vocabulary, and they are served by an inference engine written by hand in C with AVX-512 kernels. Quantization is folded into training rather than applied to a finished model, and activation sparsity is trained in rather than pruned afterward.
What hardware does a Veritate model need?
A processor, and nothing else. There is no CUDA, no driver, no Python runtime, and no framework to keep current, because the engine is a single binary. Decode runs at batch 1, which is what one person talking to a model looks like, and INT8 weights with INT4 compression keep the memory footprint low enough for machines that were never built to host an accelerator.
Why train small models instead of using a large hosted one?
To find out how low the hardware floor under capable inference goes. A model that decodes inside a processor's registers draws what that processor draws, and it runs on machines that already exist rather than on hardware bought to clear a floor the model never needed. That is the premise the research exists to test.
Can I run these models myself?
That is the point of publishing them. The engine, the plugins, and the training scripts are on GitHub, and the research log carries the measurement behind every number we print. A model that is a file on a machine does not stop existing because a provider retired an endpoint.
Are the models on the API the ones you train?
No. Veritate models are small research models. The public chat and the inference API serve larger open weight models we host on our own compute in the United States, and they are described that way everywhere they appear.
How do I call a hosted model from code I already have?
The endpoint follows the OpenAI chat completions format. Point your client base URL at https://api.carpathian.ai/ai, authenticate with your key, and the OpenAI SDKs keep working without code changes. Call the models endpoint with that key to see exactly which models it can reach.

Read the engine before you take the claims.

The engine, the training scripts, and the measurements are published, and the code is small enough to read in an afternoon.

Veritate ResearchChat With OneTalk to an Engineer