Run a language model on a CPU, with no GPU at all
Veritate is a language model inference engine written by hand in C. It is one binary with no CUDA, no driver, and no Python runtime, it loads the weights and answers on the processor your computer already has. Local AI, on your own hardware, with nothing rented and nothing sent anywhere.
Why we build for the CPU
Every computer already has a processor. Nothing to rent, nothing to wait in line for, the model runs on what you already paid for.
The engine is written in C for exactly that, with the kernels tuned by hand for the instruction sets normal machines have. A CPU is where these models are meant to live.
It counts most where a graphics card was never going to happen, a laptop in a clinic, a machine on a factory floor, an office server that never touches the internet.
- One binary, no CUDA, no driver, no runtime to install
- INT8 weights, small enough to sit in ordinary memory
- A laptop, a desktop, a small server, or a single-board computer
- The same file everywhere, copy it across and run it
How a model gets small enough for a CPU
A model's weights are just numbers. Spend eight bits on each one instead of thirty-two and the model gets small enough to sit in ordinary memory, and the arithmetic turns into integer math, which is what a processor is fastest at.
Quantization-aware training puts the limit in front of the model while it is still learning, so it comes out already shaped for the arithmetic it will run on. That is why it stays sharp at eight bits, and why the work keeps going lower.
Where this goes
A model that runs on a processor runs everywhere. On a laptop with the wifi off, on a desktop in an office that keeps its own records, on a machine in a building where nothing is allowed to leave.
That is the whole direction of the work. AI that belongs to the person running it, on hardware they already own, answering without asking anyone for capacity.
The engine and the training scripts are published, so you can build it, load a small byte-level model, and watch a computer you already own answer without a graphics card in it.
- Answering runs on any processor, no graphics card involved
- Learning runs on the same machine, overnight, on the same processor
- A graphics card only makes the learning finish sooner
- Nothing leaves the machine in either case
Run one yourself
Build the engine, point it at a model, and it answers on the machine in front of you.