wren
A 200 million parameter byte-level language model. It reads raw bytes rather than tokens, it carries working memory in a recurrent state, and it trains, serves, and learns new facts on one Mac Studio. It is a research model, and this page is its card.
What wren is
wren is a dense 200M-parameter decoder that reads raw bytes. Its vocabulary is 256 symbols, one for every possible byte, which means there is no tokenizer to train, no vocabulary to mismatch, and no subword artifacts to work around. It can be pointed at the bytes of any corpus in any language and start learning.
Its trunk is a gated linear attention hybrid rather than a stack of pure attention blocks. The practical consequence is a recurrent internal state that persists across a conversation, so the model holds something between turns without that something being pasted back into a prompt every time.
Everything about it, training included, happens on one Mac Studio M3 Ultra. No cluster, no cloud, no scheduled window on somebody else's accelerators.
- Parameters200 million, dense, with no mixture-of-experts routing
- Vocabulary256 symbols, one per byte, so there is no tokenizer anywhere in the stack
- TrunkA gated linear attention hybrid, recurrent, whose internal state carries short-term working memory between turns
- Precisionbf16 for training, INT8 and ternary for serving through the Veritate engine
What it can do
We taught it fifty things it had never heard, told once each, while the machine sat idle overnight.
Fifty statements about people who do not exist, half about where they live and half about what they do for work. Invented, so the model could not have picked them up anywhere else.
Each one was written out a few different ways, as questions, as statements, as conversation, and in both directions, so it learns the fact rather than the sentence.
Afterward it answered ninety-four questions out of a hundred about them from memory, with nothing put in front of it to read.
Asked backwards it did just as well, tell it who lives in a town and it can tell you the town, ask it who lives there and it comes back with the name.
It learns while you sleep
wren learns on the same machine it runs on, at night, when you are not using it, going back over what you told it that day and keeping it, and nothing goes to the cloud while that happens, your conversations stay on your machine and the model is ready again in the morning.
Fifty new facts take about three nights and after that it answers them from memory, with no notes and nothing copied back into the chat, so teach it whatever you repeat, names, projects, how your files are organized, how you like things written.
- BeforeIt has never heard any of this
- First nightA few of the fifty are in
- Second nightMost of them are in, forwards and backwards
- Third nightIt answers 94 out of 100 from memory
- After thatIt checks itself and stops for the week
Where it goes next
It keeps the same model and keeps adding to it, so the longer you use it the more it knows about your work.
wren2 is being trained under a billion parameters, so it answers fast on the computer you already own, no graphics card and no monthly bill for someone else's hardware.
Every night of learning goes back into the same model, so the longer you use it the less you have to repeat, tell it about your work once and it still knows next month, and next year.
Questions about wren?
We would rather answer a hard question about the method than a soft one about the result.