We buy retired enterprise servers and put them back to work
Most of what a server costs the planet is spent before it answers a single request. Mining, refining, fabricating, shipping. That cost is paid once, when the machine is built, and the only thing that lowers it is keeping the machine running longer. So we buy enterprise hardware on its way to scrap, rebuild it, and write software small enough to run on it.
It only works in this order
Each step makes the next one possible. Take away the first and none of the rest hold.
Where the carbon is spent
New silicon usually wins on efficiency per watt. Each generation does more work per unit of energy than the one it replaced. An argument built on beating new hardware there would last about one product cycle.
Embodied carbon works differently. It covers everything spent mining, refining, fabricating, and shipping a machine before it serves anyone. It is paid once, at manufacture, and the only thing that moves the number is how long the machine stays in service.
The largest operators reached the same place from the accounting side. Alphabet and Microsoft both extended how long they keep servers, because the machines still worked long after the depreciation schedule said to replace them.
No server has a smaller footprint than the one nobody had to build.
UN Global E-waste Monitor 2024.
Shrinking the model is the other half
Reusing hardware only helps if the work fits on it. Making the models smaller is the same problem one layer up from the rack, and it is what the research program works on.
Folding quantization into training took a research model from 326 MB to 17 MB on disk. A shipping recipe holds an effective 1.98 bits per weight at a cost of 0.021 in cross-entropy. Activation sparsity trained into the network reaches 87.5 percent zeros at 0.037 cross-entropy, after about three minutes of fine-tuning on a CPU. A KV cache gives a 19x speedup at a 512-byte context with byte-exact output, and a 197 thousand parameter distilled exit head drafts ahead for another 1.74x, again byte-identical.
Each figure carries the quality cost we measured beside it, and the research log says which ones are wall-clock.
Further reading
Efficiency, embodied carbon, and what small models can do.
Why the Internet Forgot How to Be Efficient
It takes megabytes of script to show a paragraph of text. How we stopped valuing efficiency, and why lean computing still matters.
Read the entryUpcycling the Data Center
E-waste keeps growing while compute that still works gets thrown out. Where upcycled hardware fits in AI and cloud work, and where it runs out of road.
Read the entryThe Hidden Energy Cost of AI
AI energy use gets treated as a footnote while data centers race toward Japan-sized power demand. Why efficiency is the frontier.
Read the entryTrained-In, Not Bolted-On
Quality falls off a cliff below four bits per weight. Quantization-aware training recovers nearly all of it at an effective 1.98 bits.
Read the entryActivation Sparsity
Sparsity has to be trained in to hold. It reaches about 88 percent zeros, and it composes with pruning to cut feed-forward compute to roughly 6 percent of dense.
Read the entrySmall Models, Big Future
Why the future of useful AI is small language models and efficient inference, and what that changes for the people running it.
Read the entryWhat this looks like as a product
The chat runs on the fleet described here and costs nothing to use. The inference API is priced per token on the same hardware, and the cloud runs on the same fleet at published rates.