We shrank Kimi K3 by 35%, within a point of the original on MMLU.
Kimi K3 is Moonshot AI’s 2.8-trillion-parameter model. As published its weights are 1,560.9 GB, already stored in 4 bits, and it ran here on eight B300 cards. Our TWC25 build is 1,007.9 GB, 35% smaller (1.5×), and TWMX34 is 1,178.1 GB, 25% smaller (1.3×). On MMLU TWC25 is 0.6 points below the original, inside the one-point bar we set before the run, and TWMX34 is level. Both are level on made-up facts and within a few calls of the original on tool calling.
See the matching eval card for the result in one screen.
Three builds of one model, on the same eight cards.
We measured two Tileward builds against Kimi K3 as Moonshot publishes it. Every build ran on stock vLLM 0.31.0, split across eight B300 cards, in one session.
| Build | Weights | Against the original |
|---|---|---|
| Kimi K3 as published | 1,560.9 GB | The reference |
| Tileward TWMX34 | 1,178.1 GB | 25% smaller (1.3×) |
| Tileward TWC25 | 1,007.9 GB | 35% smaller (1.5×) |
The published model is already 4-bit, so these percentages are against a 4-bit model, not a 16-bit one.
Within a point on MMLU, and level on made-up facts and tool calls.
Each build answered the same questions on the same machine. The bar, set before the run: a build passes MMLU if the paired test puts it no more than one point below the original.
| Test | The original | Tileward TWMX34 | Tileward TWC25 |
|---|---|---|---|
| MMLU, 14,042 questions | 91.1% | 91.0% | 90.5% |
| MMLU against the original | – | Level (p = 0.48) | 0.6 points below (p = 0.0002), inside the bar |
| Facts made up | 1.0% (13 of 1,350) | 0.6% (8 of 1,323) | 0.9% (12 of 1,384) |
| Tool calls right, low thinking effort | 598 of 600 | 593 of 600 | 596 of 600 |
| Tool calls right, high thinking effort | 597 of 600 | 590 of 600 | 595 of 600 |
Neither build differs from the original on made-up facts by more than chance (p = 0.38 and 0.84). On MMLU the paired test puts TWC25 at most 0.85 points below the original, and TWMX34 at most 0.34.
TWMX34 picks the original’s next word more often.
Next-word agreement is how often a build picks the original’s next word on the same text. TWMX34 agrees 90.8% of the time and TWC25 88.6%.
| Build | Next-word agreement | KL distance from the original |
|---|---|---|
| Tileward TWMX34 | 90.8% | 0.036 |
| Tileward TWC25 | 88.6% | 0.060 |
64 GiB less on every card, and 2.7× the room for the cache.
Measured with the server allowed 90% of each card.
| Build | Weights on each card | Tokens the cache holds |
|---|---|---|
| Kimi K3 as published | 191.7 GiB | 1,467,558 |
| Tileward TWMX34 | 147.1 GiB | 3,139,006 |
| Tileward TWC25 | 127.3 GiB | 3,945,603 |
TWMX34 keeps closest to the original; TWC25 holds the most.
TWMX34 is level with the original on MMLU and agrees with it more often. TWC25 gives up 0.6 points of MMLU for 170.2 GB less weight than TWMX34 and 26% more room in the cache.
A third build at TWC25’s size missed the one-point bar on MMLU and fell further behind on tool calls, so it is not offered.
Every build ran in one session, on the same machine and engine.
The original and both builds were served the same way and asked the same questions.
The machine. Eight B300 cards, with stock vLLM 0.31.0 serving each build from its own weights, split across all eight, with a 16-bit cache and 90% of each card.
Size. The weights files, as the model library lists them.
MMLU. Five-shot, no sampling, scored by the answer letter the model rates most likely. The paired test counts the answers that differ: 290 right only for the original and 207 only for TWC25; 200 only for the original and 185 only for TWMX34.
Made-up facts. Each build reads 360 passages from Tileward’s own document set and writes out the facts they state. Every fact is matched back to its passage, and one that no passage supports counts as made up. The test treats facts as independent.
Tool calls. Six cases, 20 trials each, run five times: 600 calls, once at low thinking effort and once at high. The cases are one call, two calls in one turn, nested arguments, a tool that must not be called, the right tool out of twenty, and a two-turn loop. Each call is scored on tool name, call count and argument shape, with vLLM’s Kimi K3 tool parser.
Agreement. Each build reads the same 300 passages (82,712 token positions), and we count where its top next token is the original’s.
Bring the work you would run on Kimi K3 and the cards you run on, and we will measure the same things. Talk to an engineer, or compare the Kimi K3 eval card.