We made GLM-5.3 38% smaller than Z.ai’s FP8 release, at the same quality.

Z.ai’s FP8 release is the version of GLM-5.3 most people run today, so each line here is TWFP4 against it. Every section below starts with its own line, then gives the full measurement.

Every test on this page, with each build’s result drawn from zero. Each card links to the full result below.

We asked three things of each release. Does it still know what the original knows? MMLU is 14,042 multiple-choice questions across 57 subjects, and higher is better. Does it make things up? Each release wrote out the facts in 360 passages, and we counted the facts no passage supports, so lower is better. Can it still use tools? It made 600 calls at each of two thinking efforts, scored on calling the right tool the right way, and higher is better.

Before the run we set the pass mark for MMLU: no more than one point below the 16-bit original, compared question by question. The chart shows how far each release is below the original, and how far below it could be at worst. Neither gap is larger than chance alone produces. Left of zero is below the original, and the dashed line is the pass mark.

On made-up facts and tool calls TWFP4 is level with both releases. Tap a figure for the counts behind it.

A test score can hide small changes in how a model writes. Next-word agreement is a finer check: each release reads the same 300 passages, and at each of 83,139 places we ask whether it would pick the same next word as the 16-bit original. Higher is closer to the original. The second column measures how far its word choices drift from the original’s, where lower is closer.

A smaller build can drift further from the original the longer the text runs. Each release read 24 passages of 32,768 tokens, and we compared its next-word agreement with the original in the first 2,048 tokens and in everything after them. Then we hid ten passkeys, one at a time, at ten depths of a 130,063-token prompt and asked for each one back. Higher is better in every column.

The 16-bit release is the original, and every quality figure on this page is measured against it. The FP8 release is what most people deploy, so it is the size to beat. All three ran on the same eight B300 cards, with the same serving software and settings, and answered the same questions. A shorter bar is a smaller release.

Each of the eight cards holds one slice of the model. Whatever the model leaves free, the server gives to conversations: the text the model keeps in memory while it answers. This is one card for each release, all drawn to one scale.

Every card has the same memory. What the weights leave free, the server gives to the cache: the conversation the model holds while it answers. A bigger cache means more people, or longer conversations, on the same cards. TWFP4’s weights take 121 GiB less of each card than the 16-bit original’s. The server was allowed 90% of each card for every release, with the same 8-bit cache, and a longer bar holds more.

A B300 card holds 288 GB (NVIDIA’s figure), so two hold 576 GB. The FP8 release’s weights alone take 755.6 GB and the 16-bit original’s 1,506.7 GB, more than two cards hold. TWFP4 takes 464.8 GB in both its versions. We ran each on two cards, and each still held three full-length conversations of 262,144 tokens at once.

On eight, four and two cards, each release either ran, with how many full-length conversations it held at once (more is better), or its weights alone were more than the cards hold. The server had 90% of each card on eight and four cards, and 95% on two. Every quality figure on this page is from the eight-card runs.

TWFP4’s own runs, card by card: the more cards share the model, the less of each it takes and the more room is left for conversations.

TWFP4 also comes in a second version, the same 464.8 GB, served with 4-bit activations. It does more of its arithmetic in 4 bits, which the B300 has its own units for, and it runs on stock vLLM like every release here. Against the FP8 release it writes 26% faster for one user and 24% faster for 64 at once, and it reads long prompts 14% faster. The trade is the typical wait for the first token: on a long prompt it arrives 0.27 seconds later. The slowest 5% of first tokens come sooner than on any other release here.

TWFP4 as first served runs close to the FP8 release: a little faster for one user, and slower with 64 users at once and on long prompts. The full measurement follows, every release on every load. Each release ran the same three loads on the same eight cards: one user, 64 users at once, and 16 users each sending a prompt of 8,192 tokens. First, how fast each one writes and reads, in tokens a second, where higher is faster in every column.

Then the wait for the first token on a long prompt, in seconds, where lower is better in both columns. The faster version’s typical wait is the longest of the four, and its slowest 5% are the shortest.

On quality the faster version is level with the FP8 release on MMLU, made-up facts, tool calls and passkeys, and it holds 23% more conversation. The trade is closeness to the original: it picks the original’s next word 90.0% of the time, against 91.9% for the FP8 release and TWFP4. Tap a figure for the counts behind it.

Where you would run the FP8 release, run TWFP4. It carries 290.8 GB less weight than the FP8 release and scores the same on every quality test on this page. On the same cards it holds 24% more conversation, and it runs on two cards, where the FP8 release does not fit.

Where speed matters most, serve it with 4-bit activations, at the same size: about a quarter faster than the FP8 release at writing, and 14% faster at reading long prompts. Two things come with that. The first token on a long prompt arrives 0.27 seconds later, and it picks the original’s next word 90.0% of the time, against 91.9%.

The two releases and TWFP4 were served the same way, one after another on one machine, and asked the same questions. The 4-bit-activation version followed in a second session on the same machine and was measured against the same original. The pass marks were written down before each run. The settings for each test follow.

The machine. Eight B300 cards, with stock vLLM 0.32.0 serving each release from its own weights, split across all eight, with an 8-bit cache, a 262,144-token window and 90% of each card.

Size. The weights files the server loads, in decimal gigabytes.

MMLU. Five-shot, no sampling, scored by the answer letter the model rates most likely. The paired test counts the answers that differ: 296 right only for the original and 283 only for TWFP4 (p = 0.62); 339 only for the original and 315 only for the FP8 release (p = 0.37). Between TWFP4 and the FP8 release, 340 and 329 (p = 0.70).

Made-up facts. Each release reads 360 passages from Tileward’s own document set and writes out the facts they state. Every fact is matched back to its passage, and one that no passage supports counts as made up. The test treats facts as independent.

Tool calls. Six cases, 20 trials each, run five times: 600 calls, once at low thinking effort and once at high. The cases are one call, two calls in one turn, nested arguments, a tool that must not be called, the right tool out of twenty, and a two-turn loop. Each call is scored on tool name, call count and argument shape, with vLLM’s GLM tool parser.

Agreement. Each release reads the same 300 passages (83,139 token positions), and we count where its top next token is the original’s. The long-passage test reads 24 passages of 32,768 tokens the same way.

Speed. Each load ran twice, without speculative decoding, and the second run is shown. One user sends 12 prompts of 1,024 tokens, one at a time, for 256 tokens back, and the figure is the median writing speed of an answer. 64 users send 256 such prompts, 64 at a time, and the figure is all the tokens they get back, per second. For long prompts, 16 users send 48 prompts of 8,192 tokens, 16 at a time, for 32 tokens back; the figures are prompt tokens read per second and the median wait for the first token. TWFP4, its 4-bit-activation version and the FP8 release were measured in the second session, and the 16-bit original in the first. Between the two sessions, TWFP4 and the FP8 release moved by under 1% on every tokens-a-second figure. The FP8 release’s first token on long prompts moved from 1.95 to 1.64 seconds, so differences of a few tenths of a second in that column are within what a second run changes.

The model. GLM-5.3 is Z.ai’s, released under the GLM-5.3 License.