Benchmark

We shrank GLM-5.3-Flash by 68%, and it solved 86% of SWE-bench Verified.

GLM-5.3-Flash is Z.ai’s 321-billion-parameter model. In 16 bits its weights are 642.65 GB. Our TWXH8 build is 204.45 GB, 68% smaller (3.14×), and runs on two B300 cards. Working as a coding agent it solved 431 of the 500 SWE-bench Verified tasks, and 429 when the whole run was repeated. The original was not run on SWE-bench, so the comparison with it is next-word agreement: TWXH8 picks the original’s next word 88.8% of the time.

See the matching eval card for the result in one screen.

The setup

One model, two Tileward builds, measured against the 16-bit original.

Z.ai publishes GLM-5.3-Flash in 16 bits. Both of our builds are about a third of its size.

BuildWeightsAgainst the original
GLM-5.3-Flash in 16 bits642.65 GBThe reference
Tileward TWX48205.09 GB68% smaller (3.13×)
Tileward TWXH8204.45 GB68% smaller (3.14×)

TWXH8 is the build we ran on SWE-bench, and the rest of this report is about it.

Coding

431 of 500 SWE-bench Verified tasks, and 429 on a repeat.

Each task is a real issue from an open-source Python project. The agent reads the repository, writes a patch, and the project’s own tests decide whether the issue is fixed.

RunSolvedPatch submitted, not solvedNo patch
First run431 of 500 (86.2%)618
Repeat, same settings429 of 500 (85.8%)665

411 tasks were solved in both runs and 38 (7.6%) in only one, so a difference of a few tasks between runs of this build is noise. Most tasks with no patch ran past the 65,536-token window.

Against the original

It picks the original’s next word 88.8% of the time.

The original was not run on SWE-bench, so this is the comparison with it. Next-word agreement is how often a build picks the original’s next word on the same text; every build here was scored in one run against the same 16-bit original.

BuildNext-word agreementKL distance from the original
Tileward TWX4887.3%0.115
Tileward TWXH888.8%0.088

On how well each predicts the text, our two builds are level with each other: the difference is within chance.

Running it

Two B300 cards, with nothing moved off the cards.

TWXH8 ran on stock vLLM 0.32.0, split across two B300 cards, with every weight on the cards.

How it was measured

One build, one harness, run twice.

The SWE-bench run was repeated in full with the same settings, to show how much a score moves between identical runs.

The machine. Two B300 cards, with stock vLLM 0.32.0 serving TWXH8 from its own weights.

Size. The weights files on disk.

SWE-bench Verified. All 500 tasks, with mini-SWE-agent 2.4.6: up to 250 steps, temperature 1.0, top-p 0.95, high reasoning effort, replies up to 16,384 tokens and a 65,536-token window. Every patch was graded by the official SWE-bench grader on the original task set. Tasks whose container failed to start were run again before grading; no task was run again to improve a score.

Agreement. Each build reads the same 32 passages of WikiText-2 (16,352 token positions), and we count where its top next token is the original’s.

Bring your own repository

Bring the code your agents work on and the cards you run on, and we will measure the same things. Talk to an engineer, or compare the GLM-5.3-Flash eval card.