# We made GLM-5.3 38% smaller than Z.ai's FP8 release, at the same quality

Tileward research report R-014. GLM-5.3 has 753 billion parameters. Z.ai publishes it twice: in 16 bits, where its weights take 1,506.7 GB, and as an FP8 release at 755.6 GB, the one most people run. We made a build, TWFP4, that takes 464.8 GB: 38% less than the FP8 release (1.6×) and 69% less than the 16-bit original (3.2×). On a 14,042-question knowledge test it scored 86.2%, against 86.1% for the FP8 release and 86.3% for the original, inside the one-point bar we set before the run, and it made every one of 1,200 tool calls. On the same eight cards it holds 24% more conversation than the FP8 release, and it runs on two of them, where the FP8 release does not fit. Served with 4-bit activations, it writes about a quarter faster than the FP8 release, and its first token on a long prompt comes 0.27 seconds later.

Page: https://tileward.com/research/a-smaller-glm-5-3/

Built with: vLLM 0.32.0 · 8× B300 · MMLU

## At a glance

Against Z.ai's FP8 release: smaller, level on quality, more room, faster. The FP8 release is the version of GLM-5.3 most people run today, so each line here is TWFP4 against it.

| Against the FP8 release | Tileward TWFP4 |
|---|---|
| Size of the weights | 38% smaller: 464.8 GB against 755.6 GB |
| Knowledge, on MMLU | Level: 86.2% against 86.1% |
| Tool calls right | Level: 1,200 of 1,200 for both |
| Facts made up | Level: 1.4% against 2.2% |
| Picking the original's next word | Level: 91.9% for both |
| Conversation held on the same eight cards | 24% more |
| Fewer cards | Runs on two cards, with room for 3 full-length conversations; the FP8 release does not fit on two |
| Speed, with 4-bit activations | About a quarter faster at writing and 14% faster at reading long prompts; the first token on a long prompt comes 0.27 s later |

## Benchmarks

Level with Z.ai's FP8 release on MMLU, facts, tools and next words. Every test on this page, with each build's result drawn from zero. Each card links to the full result below.

- **MMLU:** Knowledge: 14,042 multiple-choice questions across 57 subjects. Higher is better. GLM-5.3 in 16 bits 86.3%; Z.ai's FP8 release 86.1%; Tileward TWFP4 86.2%.
- **Made-up facts:** Each build writes out the facts in 360 passages; a fact no passage supports is made up. Lower is better. GLM-5.3 in 16 bits 1.7%; Z.ai's FP8 release 2.2%; Tileward TWFP4 1.4%.
- **Tool calls:** 600 calls at each of two thinking efforts, scored on calling the right tool the right way. Higher is better. GLM-5.3 in 16 bits, both efforts 1,200 of 1,200; Z.ai's FP8 release, both efforts 1,200 of 1,200; Tileward TWFP4, both efforts 1,200 of 1,200.
- **Next-word agreement:** Each release reads the same passages, and at every place we ask whether it picks the original's next word. Higher is closer. Z.ai's FP8 release 91.9%; Tileward TWFP4 91.9%.
- **Passkeys:** A passkey hidden in a long passage, ten times over. Higher is better. GLM-5.3 in 16 bits 10 of 10; Z.ai's FP8 release 10 of 10; Tileward TWFP4 10 of 10.

## Findings · Quality

Level with the FP8 release on MMLU, made-up facts and tool calls. We asked three things of each release. Does it still know what the original knows? [MMLU](https://arxiv.org/abs/2009.03300) is 14,042 multiple-choice questions across 57 subjects, and higher is better. Does it make things up? Each release wrote out the facts in 360 passages, and we counted the facts no passage supports, so lower is better. Can it still use tools? It made 600 calls at each of two thinking efforts, scored on calling the right tool the right way, and higher is better. Before the run we set the pass mark for MMLU: no more than one point below the 16-bit original, compared question by question. Neither gap below is larger than chance alone produces.

| Release | MMLU | Against the original | At worst |
|---|---|---|---|
| GLM-5.3 in 16 bits | 86.3% | The reference | – |
| Z.ai's FP8 release | 86.1% | 0.2 points below | 0.47 points below |
| Tileward TWFP4 | 86.2% | 0.1 points below | 0.37 points below |

On made-up facts and tool calls TWFP4 is level with both releases.

| Test | 16 bits | Z.ai FP8 | Tileward TWFP4 |
|---|---|---|---|
| Facts made up, lower is better | 1.7% (23 of 1,361) | 2.2% (31 of 1,399) | 1.4% (20 of 1,381) |
| Tool calls right, low thinking effort, higher is better | 600 of 600 | 600 of 600 | 600 of 600 |
| Tool calls right, high thinking effort, higher is better | 600 of 600 | 600 of 600 | 600 of 600 |

The counts behind the MMLU comparison: 296 answers right only for the original and 283 only for TWFP4 (p = 0.62); 339 and 315 for the FP8 release (p = 0.37). Between TWFP4 and the FP8 release, 340 and 329 (p = 0.70). Made-up facts against the original: p = 0.34 for the FP8 release and 0.65 for TWFP4, treating facts as independent.

## Findings · Against the original

As close to the original as the FP8 release: 91.9% of next words. A test score can hide small changes in how a model writes. Next-word agreement is a finer check: each release reads the same 300 passages, and at each of 83,139 places we ask whether it would pick the same next word as the 16-bit original. Higher is closer. Drift is the mean KL divergence from the original's word choices, where lower is closer.

| Release | Next-word agreement, higher is closer | Drift from the original, lower is closer |
|---|---|---|
| Z.ai's FP8 release | 91.9% | 0.030 |
| Tileward TWFP4 | 91.9% | 0.031 |

## Findings · Long context

As close as the FP8 release over long passages, and every passkey found. A smaller build can drift further from the original the longer the text runs. Each release read 24 passages of 32,768 tokens, and we compared its next-word agreement with the original in the first 2,048 tokens and in everything after them. Then we hid ten passkeys, one at a time, at ten depths of a 130,063-token prompt and asked for each one back.

| Release | Agreement, first 2,048 tokens, higher is closer | Agreement, after 2,048, higher is closer | Passkeys found, of 10 |
|---|---|---|---|
| GLM-5.3 in 16 bits | The reference | The reference | 10 of 10 |
| Z.ai's FP8 release | 90.5% | 90.1% | 10 of 10 |
| Tileward TWFP4 | 90.6% | 90.5% | 10 of 10 |

## Memory · Size

38% smaller than the FP8 release, and 69% smaller than 16 bits. The 16-bit release is the original, and every quality figure here is measured against it. The FP8 release is what most people deploy, so it is the size to beat. All three ran on the same eight B300 cards, with the same serving software and settings, and answered the same questions.

| Release | Weights | Against the original |
|---|---|---|
| GLM-5.3 in 16 bits | 1,506.7 GB | The reference |
| Z.ai's FP8 release | 755.6 GB | 50% smaller (2.0×) |
| Tileward TWFP4 | 464.8 GB | 69% smaller (3.2×) |

## Memory · One card

TWFP4 takes 55.3 GiB of each card; the FP8 release takes 89.4. Each of the eight cards holds one slice of the model. Whatever the model leaves free, the server gives to conversations: the text the model keeps in memory while it answers. On one card, GLM-5.3 in 16 bits takes 176.5 GiB and leaves 58.1 GiB for conversations; Z.ai's FP8 release takes 89.4 GiB and leaves 145.2 GiB; TWFP4 takes 55.3 GiB and leaves 179.8 GiB.

## Memory · Cache

24% more conversation than the FP8 release, on the same cards. Every card has the same memory. What the weights leave free, the server gives to the cache: the conversation the model holds while it answers. A bigger cache means more people, or longer conversations, on the same cards. TWFP4's weights take 121 GiB less of each card than the 16-bit original's. The server was allowed 90% of each card for every release, with the same 8-bit cache.

| Release | Weights on each card | Room for the cache, more holds more | Tokens the cache holds, more is better |
|---|---|---|---|
| GLM-5.3 in 16 bits | 176.5 GiB | 58.1 GiB | 1,307,648 |
| Z.ai's FP8 release | 89.4 GiB | 145.2 GiB | 3,267,776 |
| Tileward TWFP4 | 55.3 GiB | 179.8 GiB | 4,047,872 |

## Memory · Fewer cards

Two cards are enough for TWFP4; the FP8 release does not fit. A B300 card holds 288 GB ([NVIDIA's figure](https://docs.nvidia.com/dgx/dgxb300-user-guide/introduction-to-dgxb300.html)), so two hold 576 GB. The FP8 release's weights alone take 755.6 GB and the 16-bit original's 1,506.7 GB, more than two cards hold. TWFP4 takes 464.8 GB in both its versions. We ran each on two cards, and each still held three full-length conversations of 262,144 tokens at once.

On eight, four and two cards, each release either ran, with how many full-length conversations it held at once (more is better), or its weights alone were more than the cards hold. The server had 90% of each card on eight and four cards, and 95% on two. Every quality figure here is from the eight-card runs.

| Release | Eight cards (2,304 GB) | Four cards (1,152 GB) | Two cards (576 GB) |
|---|---|---|---|
| GLM-5.3 in 16 bits | Runs: 4 at once | Does not fit: 1,506.7 GB | Does not fit: 1,506.7 GB |
| Z.ai's FP8 release | Runs: 12 at once | Weights fit: 755.6 GB | Does not fit: 755.6 GB |
| Tileward TWFP4 | Runs: 15 at once | Runs: 11 at once | Runs: 3 at once |
| Tileward TWFP4, 4-bit activations | Runs: 15 at once | Runs: 10 at once | Runs: 3 at once |

TWFP4's own runs, card by card: the more cards share the model, the less of each it takes and the more room is left for conversations.

| Cards | Weights on each card | Room for the cache | Tokens the cache holds, more is better | Full-length conversations at once, more is better |
|---|---|---|---|---|
| Eight cards | 55.3 GiB | 179.8 GiB | 4,047,872 | 15 |
| Four cards | 106.8 GiB | 128.5 GiB | 2,891,456 | 11 |
| Two cards | 209.6 GiB | 38.3 GiB | 862,272 | 3 |

## Compute · Speed

Against the FP8 release: a quarter faster to write, 0.27 s slower to start. TWFP4 also comes in a second version, the same 464.8 GB, served with 4-bit activations. It does more of its arithmetic in 4 bits, which the B300 has its own units for, and it runs on stock vLLM like every release here. Against the FP8 release it writes 26% faster for one user and 24% faster for 64 at once, and it reads long prompts 14% faster. The trade is the typical wait for the first token: on a long prompt it arrives 0.27 seconds later. The slowest 5% of first tokens come sooner than on any other release here.

| Against the FP8 release | TWFP4, 4-bit activations |
|---|---|
| Writing an answer, one user | 26% faster |
| Writing, 64 users at once | 24% faster |
| Reading 16 long prompts | 14% faster |
| First token, 16 long prompts | 0.27 s later: the one measure where it is behind |
| Slowest 5% of first tokens, 16 long prompts | Sooner: 3.6 s against 4.1 s |

TWFP4 as first served runs close to the FP8 release: a little faster for one user, and slower with 64 users at once and on long prompts. The full measurement follows, every release on every load. Each release ran the same three loads on the same eight cards: one user, 64 users at once, and 16 users each sending a prompt of 8,192 tokens. First, how fast each one writes and reads, in tokens a second, where higher is faster in every column.

| Release | Writing, one user, higher is faster | Writing, 64 users, higher is faster | Reading 16 long prompts, higher is faster |
|---|---|---|---|
| GLM-5.3 in 16 bits | 143.6 | 2,158 | 28,089 |
| Z.ai's FP8 release | 126.6 | 2,628 | 27,578 |
| Tileward TWFP4 | 132.3 | 2,343 | 26,075 |
| Tileward TWFP4, 4-bit activations | 160.1 | 3,248 | 31,517 |

Then the wait for the first token on a long prompt, in seconds, where lower is better in both columns. The faster version's typical wait is the longest of the four, and its slowest 5% are the shortest.

| Release | Typical first token, 16 long prompts, lower is better | Slowest 5% of first tokens, lower is better |
|---|---|---|
| GLM-5.3 in 16 bits | 1.59 s | 4.0 s |
| Z.ai's FP8 release | 1.64 s | 4.1 s |
| Tileward TWFP4 | 1.73 s | 4.3 s |
| Tileward TWFP4, 4-bit activations | 1.91 s | 3.6 s |

On quality the faster version is level with the FP8 release on MMLU, made-up facts, tool calls and passkeys, and it holds 23% more conversation. The trade is closeness to the original: it picks the original's next word 90.0% of the time, against 91.9% for the FP8 release and TWFP4.

| Test | Z.ai FP8 | Tileward TWFP4 | TWFP4, 4-bit activations |
|---|---|---|---|
| MMLU, higher is better | 86.1% | 86.2% | 86.2% |
| MMLU at worst against the original, closer to zero is better | 0.47 points below | 0.37 points below | 0.35 points below |
| Facts made up, lower is better | 2.2% (31 of 1,399) | 1.4% (20 of 1,381) | 1.6% (21 of 1,342) |
| Tool calls right, both thinking efforts, higher is better | 1,200 of 1,200 | 1,200 of 1,200 | 1,200 of 1,200 |
| Next-word agreement with the original, higher is closer | 91.9% | 91.9% | 90.0% |
| Long passages, first 2,048 tokens and after, higher is closer | 90.5% and 90.1% | 90.6% and 90.5% | 89.0% and 89.0% |
| Passkeys found, of 10 | 10 of 10 | 10 of 10 | 10 of 10 |
| Tokens the cache holds, more is better | 3,267,776 | 4,047,872 | 4,028,608 |

The counts behind the faster version: against the 16-bit original on MMLU, 338 answers right only for the original and 331 only for it (p = 0.82). Made-up facts against the original: p = 0.88, treating facts as independent. Drift from the original's word choices is 0.050, against 0.030 for the FP8 release and 0.031 for TWFP4.

## Which release

Run TWFP4 where you run the FP8 release; add 4-bit activations for speed.

Where you would run the FP8 release, run TWFP4. It carries 290.8 GB less weight than the FP8 release and scores the same on every quality test here. On the same cards it holds 24% more conversation, and it runs on two cards, where the FP8 release does not fit.

Where speed matters most, serve it with 4-bit activations, at the same size: about a quarter faster than the FP8 release at writing, and 14% faster at reading long prompts. Two things come with that. The first token on a long prompt arrives 0.27 seconds later, and it picks the original's next word 90.0% of the time, against 91.9%.

## How it was measured

The two releases and TWFP4 were served the same way, one after another on one machine, and asked the same questions. The 4-bit-activation version followed in a second session on the same machine and was measured against the same original. The pass marks were written down before each run.

- **Machine:** eight B300 cards, with stock vLLM serving each release from its own weights, split across all eight, with an 8-bit cache, a 262,144-token window and 90% of each card.
- **Size:** the weights files the server loads, in decimal gigabytes.
- **MMLU:** five-shot, no sampling, scored by the answer letter the model rates most likely. The paired test counts the answers that differ.
- **Made-up facts:** each release reads 360 passages from Tileward's own document set and writes out the facts they state. Every fact is matched back to its passage, and one that no passage supports counts as made up.
- **Tool calls:** six cases, 20 trials each, run five times: 600 calls, once at low thinking effort and once at high. Each call is scored on tool name, call count and argument shape, with vLLM's GLM tool parser.
- **Agreement:** each release reads the same 300 passages (83,139 token positions), and we count where its top next token is the original's. The long-passage test reads 24 passages of 32,768 tokens the same way.
- **Speed:** each load ran twice, without speculative decoding, and the second run is shown. One user sends 12 prompts of 1,024 tokens, one at a time, for 256 tokens back, and the figure is the median writing speed of an answer. 64 users send 256 such prompts, 64 at a time, and the figure is all the tokens they get back, per second. For long prompts, 16 users send 48 prompts of 8,192 tokens, 16 at a time, for 32 tokens back; the figures are prompt tokens read per second and the median wait for the first token. TWFP4, its 4-bit-activation version and the FP8 release were measured in the second session, and the 16-bit original in the first. Between the two sessions, TWFP4 and the FP8 release moved by under 1% on every tokens-a-second figure. The FP8 release's first token on long prompts moved from 1.95 to 1.64 seconds, so differences of a few tenths of a second in that column are within what a second run changes.
- **The model:** GLM-5.3 is Z.ai's, released under the [GLM-5.3 License](https://huggingface.co/zai-org/GLM-5.3/blob/main/LICENSE).

## Related

- [GLM-5.3 eval card](https://tileward.com/evals/#compressed-model-glm53): the result in one screen
- [GLM-5.3-Flash as a coding agent](https://tileward.com/research/a-smaller-coding-agent/): the smaller model in the same family
- [GLM-5.3 on Hugging Face](https://huggingface.co/zai-org/GLM-5.3): Z.ai's FP8 release
- [Talk to an engineer](https://calendar.app.google/hg2DzFTxKJgwFWm19): bring the work you would run on GLM-5.3 and the cards you run on
- [Research](https://tileward.com/research/): every report