Green AI, argued as engineering
Green stated as a mission is unfalsifiable, and every company claims it. Ours is a chain of engineering decisions running in one direction: write lean software, and modest hardware becomes sufficient. When modest hardware is sufficient, machines already headed for scrap become viable, and what follows is less waste and a lower bill. The environmental outcome is the consequence, and the engineering is the claim you can check.
The chain, in the order it has to run
Each step depends on the one before it. Reordered, it becomes a slogan, and a slogan is not something anyone can hold us to.
- Write lean softwareSoftware got slower as compute got cheaper, because there was no longer a reason for it not to. Reversing that is a discipline rather than a feature, and it is where the rest of the chain starts.
- Modest hardware becomes sufficientOnce the software fits, the machine underneath it does not have to be current. The requirement was never the silicon, it was the code sitting on top of it.
- Hardware headed for scrap becomes viableEnterprise machines leave service on a depreciation schedule rather than a failure curve. A fleet that only needs to be sufficient can be assembled from what that schedule discards.
- Less waste, and a lower billThe environmental result and the price both fall out of the same decision. We lead with the second one because customers can verify it.
We have written this argument at length twice, in Why the Internet Forgot How to Be Efficient and in Upcycling the Data Center.
The argument is manufacturing, not operation
Efficiency per watt is a fight that new silicon usually wins. A current generation part will generally do more work per unit of energy than the one it replaced, and any position built on out-competing it on that axis is a position with a shelf life.
The number that does not move that way is embodied carbon: everything spent mining, refining, fabricating, and shipping a machine before it has served a single request. That cost is paid once, at manufacture, and the only intervention that touches it is keeping the machine in service instead of building another one.
This is not a fringe reading. The largest operators reached the same conclusion from the finance side and extended how long they keep servers, because the machines were still capable long after the depreciation schedule said to replace them.
The greenest server is the one you did not have to manufacture twice.
UN Global E-waste Monitor 2024, covering electronic waste generated worldwide in 2022 and the share documented as formally collected and recycled.
- AlphabetExtended the estimated useful life of its servers, and reported that the change reduced depreciation expense by roughly 3 billion dollars in 2023.
- MicrosoftMoved server and network equipment from a four-year to a six-year depreciation schedule.
Both figures come from the companies' own reporting on the estimated useful life of their server fleets.
What we will not claim
Three places where the position is weaker than it sounds.
Older hardware fails more
A machine in its second service life carries a higher failure rate than a new one, and it usually draws more idle power per unit of work. That is why reclaimed equipment gets tested, rebuilt, and monitored rather than simply racked, and why the fleet carries a published 99.5% uptime SLA it has to meet on that hardware.
Our research models are small
Veritate models are research models, considerably smaller than the models serving the public chat and the inference API. They exist to test whether capable inference can run without a GPU floor. Reading them as production systems would be reading them wrong.
Some speedups are theoretical
Several of the acceleration figures are reductions in arithmetic rather than wall-clock speedups measured on a specific machine, and the research log labels which is which. A number that skips that distinction is a number nobody should trust, including ours.
Making the model smaller is the other half
Reusing hardware only helps if the work fits on it. That is the second half of the same argument, applied one layer up from the rack, and it is what the research program is for.
Quantization folded into training rather than applied afterward took a research model from 326 MB to 17 MB on disk, and a shipping recipe reaches an effective 1.98 bits per weight at a cost of 0.021 in cross-entropy. Activation sparsity trained into the network reaches 87.5 percent zeros at 0.037 cross-entropy, after roughly three minutes of fine-tuning on a CPU. A KV cache gives a 19x speedup at a 512-byte context with byte-exact output, and a 197 thousand parameter distilled exit head drafts ahead for a further 1.74x with byte-identical results.
Every one of those carries a quality cost we publish next to it, and the ones that are arithmetic reductions rather than wall-clock gains say so. The models these were measured on are small research models, which is the caveat that matters most.
Where the argument is written out in full
The position is a documented research direction rather than a marketing line. These are the pieces it rests on, including the results that went the wrong way.
Why the Internet Forgot How to Be Efficient
Megabytes of script to show a paragraph of text. Why we stopped valuing efficiency, and why lean computing still matters.
Read the entryUpcycling the Data Center
E-waste is growing while compute that still works gets thrown out. Why a lot of AI and cloud work runs on older, upcycled hardware, and where the limits are.
Read the entryThe Hidden Energy Cost of AI
AI energy cost treated as a footnote while data centers race toward Japan-sized power demand. Why efficiency, not scale, is the frontier.
Read the entryTrained-In, Not Bolted-On
The measured cliff below four bits per weight, and how quantization-aware training recovers nearly all quality at an effective 1.98 bits per weight.
Read the entryActivation Sparsity
Why it must be trained in, reaching about 88 percent zeros, composing with pruning to cut feed-forward compute to roughly 6 percent of dense.
Read the entrySmall Models, Big Future
Why the future of useful AI is small language models and efficient inference rather than the race for the biggest one.
Read the entryQuestions about the position
What does green AI mean here?
Why argue about manufacturing rather than power draw?
Does older hardware not use more power?
How does model size come into it?
Are the efficiency numbers measured or projected?
Are these the same models the chat uses?
Is any of this certified?
What this looks like as a product
The chat costs nothing to use because it costs us less to serve, and the API is priced per token on the same hardware. The cloud side of the business runs on the same fleet, at rates published rather than quoted.
The cheapest way to check the argument is to use it.
A position about efficiency is only worth as much as the thing it produced. The chat runs on that fleet, needs no account, and costs nothing.