AI research at Carpathian
We train our own language models, and the constraint we work under is power and hardware rather than parameter count. That pushes the research somewhere specific: small models, raw bytes, low precision, and machines you can already buy.
What we are trying to find out
The industry's working assumption is that capability comes from scale, and that scale comes from more accelerators in more buildings. We run our own buildings, so that bill is not abstract to us, and we think a second question is worth asking properly: how much of what a large model does can a small one do, if you stop asking it to memorise the world?
Our answer so far is a small reasoning core that holds language, logic, and behavior, paired with memory it can address instead of memorize. Underneath it sits an engine that runs in low precision on a normal CPU. Every part of that is testable, and where a test says no, we publish that too.
The work is a research program, not a product line. The models are small, they are experimental, and they are not the models serving the chat on this site.
- Write the falsifier down before running the experiment
- Publish the runs that fail alongside the ones that work
- Measure on the hardware people already own
- Automate the stop condition so nobody has to judge it at midnight
What we are working on
Five pieces of the same thing, a model good enough to work with that you can keep on your own machine.
Most models stop learning the day they ship, so whatever you tell them sits in a window that empties when you close it. Ours keep it, new facts settle into the model itself while the machine is idle, the way sleep works. Afterwards it answers from memory with nothing handed back to it.
- Learns while it idles
- Remembers between sessions
Training happens on one machine anyone can buy, not a shrunk-down stand-in for a cluster run. The work is finding out how much a single computer can carry, and every answer goes straight back into the models.
- One machine, start to finish
- No cluster, no accelerator floor
Our models read raw bytes with a 256-symbol vocabulary and no tokenizer anywhere in the stack. That removes a whole class of problem: no vocabulary mismatch, no subword artifacts, no retraining a tokenizer to reach a new language.
- 256-symbol vocabulary
- Any corpus, any language
A hand-written inference engine in C that uses low-precision INT8 and ternary math to run on ordinary processors. One binary, no CUDA, no driver, no separate runtime. If a model needs a data center to answer a question, most companies cannot own it.
- INT8 and ternary
- Single binary, no accelerator
Every forward pass can emit a full trace: per-layer residual stream, feed-forward activations, attention scores, logit lens, direct logit attribution. There is a dashboard that reads it live in a browser. Interpretability that costs nothing at runtime is the only kind that gets left switched on.
- Full per-pass trace
- Live in the browser
Working on the same problem?
We work with researchers and companies who have a real problem in this space. Tell us what you are looking at.