The models we train
Carpathian trains its own language models. They are small, they read raw bytes instead of tokens, and they are built to run on consumer-grade hardware rather than a floor of accelerators.
Why we train our own
The premise across the rest of the AI industry is that capability comes from size, and that size comes from more accelerators in more data centers. We do not think that is the only route. A small model that reads raw bytes and runs on one ordinary machine is worth building.
That route is a small model that holds language and reasoning rather than the entire world, paired with memory it can address on demand. It reads bytes, it runs in low precision on an ordinary CPU, and the whole thing fits on one machine.
We publish what we measure as we go. These are research models, and each result points at the same goal: a model that holds language and reasoning in a size small enough to run wherever it is needed.
- Small on purposeA reasoning core that holds language, logic, and behavior, with knowledge kept outside the weights and fetched when it is needed
- Raw bytes, no tokenizerA 256-symbol vocabulary, so there is no vocabulary to mismatch and no subword artifacts to work around
- CPU firstLow-precision INT8 and ternary math in a hand-written engine, so the hardware is a machine you already own
- Open to inspectionEvery forward pass can emit its own trace, and you can watch one happen in the browser
The models
wren1 is the first: 200M parameters, byte-level, trained and served on a single machine.
Our 200M byte-level model, with a recurrent trunk whose internal state doubles as working memory, so context is carried rather than capped at a window length. It trains, serves, and consolidates new facts on a single machine, and it has been run on everything from a Mac Studio down to hardware built before 2010. In August 2026 we moved 50 facts it had never seen into its weights and it recalled 94 of 100 directional prompts closed book, with no retrieval.
- 200M parameters
- Byte-level, vocab 256
- Unlimited context
- Trains on one machine
We are scaling the model up while keeping it under a billion parameters. A model that size can hold a conversation and still run on a normal computer without a GPU.
- Learns while you use it, with new facts written into the weights
- Remembers between sessions, so context does not have to be handed back every time
- Memory modeled on a brain: consolidate what matters, recall it later
Talk to one of the models we trained
The chat runs on our own byte-level model, in the browser, with no account and nothing to install. You can read the attention, the neuron activity, and the logit lens while it answers.