# We shrank Qwen3.6-35B-A3B to 16 GB, 31% smaller than NVIDIA's NVFP4 build Tileward research report R-018. Qwen3.6-35B-A3B is Qwen's model of 35 billion parameters, about 3 billion of which work on each word. As Qwen publishes it, its weights take 71.90 GB, and our TWX4 build in the model library takes 24.10 GB. We made TWX2 at 16.05 GB: 31% smaller than NVIDIA's NVFP4 build of the same model (23.42 GB), 33% smaller than TWX4, and 78% smaller than the original (4.5×). Against TWX4, in one session on one RTX PRO 6000 Blackwell card, it scored 81.9% on MMLU's 14,042 questions against 82.8%: 0.85 points behind. It made up 3.2% of the facts it wrote, against 2.2%. On the card the model takes 13.5 GiB, where TWX4 takes 21.0. NVIDIA's build was measured against TWX4 too, in its own session: 0.36 points below it on MMLU, 82.6% against 82.9%, too small a gap for the test to separate. Below, TWX2's size and MMLU score sit beside it and three other builds of the same model, each measured against TWX4 in its own session. Page: https://tileward.com/research/a-16gb-qwen3-6-35b-a3b/ Built with: vLLM 0.31.0 · RTX PRO 6000 Blackwell · MMLU · lm-evaluation-harness ## At a glance 31% smaller than NVIDIA's NVFP4 build, and 0.85 MMLU points behind TWX4. Size is against NVIDIA's NVFP4 build of Qwen3.6-35B-A3B. Quality is against TWX4, the build in our model library, in one session on one card. NVIDIA's build was measured against TWX4 in a session of its own, so its line here is that pair. Every section below starts with its own line, then gives the full measurement. | Measure | Tileward TWX2 | |---|---| | Size of the weights, against NVIDIA's NVFP4 build | Smaller by 31%: 16.05 GB against 23.42 GB | | Knowledge on MMLU, against TWX4 in the same session | Behind by 0.85 points: 98.9% of TWX4's score, 81.9% against 82.8% | | NVIDIA's NVFP4 build on MMLU, against TWX4 in its own session | 0.36 points below, not separable: 99.6% of TWX4's score, 82.6% against 82.9% | | Facts made up, against TWX4 | More, by 1.0 point: 3.2% against 2.2% | | Tool calls right, thinking off and on, against TWX4 | 7 fewer: 97.0% of TWX4's, 229 of 240 against 236 of 240 | | The model's share of one RTX PRO 6000 Blackwell card, against TWX4 | Less by 7.5 GiB: 13.5 GiB against 21.0 GiB | ## Benchmarks Against TWX4: 0.85 MMLU points behind, 1.0 point more facts made up. Every test on this page, with each build's result drawn from zero: TWX2 also got 7 fewer of 240 tool calls right. Each card links to the full result below. - **MMLU:** Knowledge: 14,042 multiple-choice questions across 57 subjects, with five worked examples before each. Higher is better. Tileward TWX4 82.8%; Tileward TWX2 81.9%. - **Made-up facts:** Each build writes out the facts stated in 360 passages, over three different draws of passages; a fact no passage supports counts as made up. Lower is better. Tileward TWX4 2.2%; Tileward TWX2 3.2%. - **Tool calls:** Six kinds of call, 20 tries each, with thinking off and then on. More is better. Tileward TWX4, thinking off 118 of 120; Tileward TWX2, thinking off 117 of 120; Tileward TWX4, thinking on 118 of 120; Tileward TWX2, thinking on 112 of 120. ## Findings · Quality TWX2 is 0.85 MMLU points behind TWX4, and makes up 1.0 point more facts. We asked three things of TWX2, each against TWX4 on the same card in the same session. Does it still know as much? [MMLU](https://arxiv.org/abs/2009.03300) asks 14,042 multiple-choice questions across 57 subjects, and higher is better. Does it make facts up? Each build writes out the facts in 360 passages, three times over, and fewer made up is better. Can it still call tools? Six kinds of call, 20 tries each, and more right is better. **How far behind.** On MMLU, 0.85 points is 119 more questions wrong out of 14,042: TWX2 answered 11,504 right and TWX4 11,623, so TWX2 keeps 98.9% of TWX4's score, and the 95% range for the gap runs from 0.30 to 1.39 points. On made-up facts, 3.2% against 2.2% is 1.0 point more: about one fact in 31 made up, against one in 45 for TWX4. On tool calls, TWX2 got 229 of 240 right against 236, 7 fewer: one with thinking off and six with it on, most of them on nested arguments (14 of 20 against 19). MMLU, compared question by question. Before the run we wrote down that a difference would be called behind or ahead only if the paired test separated the builds, and not separable otherwise. | Test | Tileward TWX4 | Tileward TWX2 | Against TWX4 | |---|---|---|---| | MMLU, 14,042 questions, higher is better | 82.8% (11,623 right) | 81.9% (11,504 right) | Behind by 0.85 points | 823 questions were right only for TWX4 and 704 only for TWX2 (exact McNemar p = 0.0025). The 95% range for the difference runs from 0.30 to 1.39 points behind. Made-up facts, over three draws of 360 passages each. Both builds read the same passages in each draw, and the first draw is the one every earlier reading of this model used. TWX2 made up more on all three, by 0.8, 1.8 and 0.4 points; alone, only the second draw separates the builds, and the three pooled do, 1.0 point apart. | Draw | Tileward TWX4, lower is better | Tileward TWX2, lower is better | Two-sided Fisher test | |---|---|---|---| | First | 2.3% (39 of 1,672) | 3.1% (49 of 1,585) | p = 0.20 | | Second | 2.2% (36 of 1,611) | 4.0% (62 of 1,551) | p = 0.0053 | | Third | 2.1% (37 of 1,741) | 2.5% (40 of 1,616) | p = 0.56 | | All three | 2.2% (112 of 5,024) | 3.2% (151 of 4,752) | p = 0.0040 | Tool calls, 20 tries of each kind, with thinking off and on. These are counts from one pass, with no test applied. Over both settings TWX2 made 229 of the 240 calls right, against 236 for TWX4, and most of its misses are on nested arguments. | Kind of call | Tileward TWX4, off | Tileward TWX2, off | Tileward TWX4, on | Tileward TWX2, on | |---|---|---|---|---| | One call | 18 | 20 | 20 | 19 | | Two calls in one turn | 20 | 20 | 19 | 20 | | Nested arguments | 20 | 17 | 19 | 14 | | A tool that must not be called | 20 | 20 | 20 | 20 | | A two-turn loop | 20 | 20 | 20 | 20 | | The right tool out of twenty | 20 | 20 | 20 | 19 | | All six, out of 120 | 118 | 117 | 118 | 112 | ## Findings · Other builds TWX2 and four other builds on MMLU, each against TWX4 in its own session. Four other builds of Qwen3.6-35B-A3B have an MMLU score from a session that measured TWX4 too, on the same questions with the same settings. Each line is one such pair, TWX2's among them: the build's size, its score, TWX4's score in that session, and the difference with its 95% range. Across the four sessions TWX4 itself scored between 82.8% and 83.0%. The pairs ran on different cards and engine versions, so each line is read on its own, and they are not ranked. Read against the same TWX4 build, TWX2 (0.85 points behind it) sits about half a point below NVIDIA's NVFP4 build (0.36 points below it); no session ran those two side by side, so that half point has no test of its own. | Build | Weights, smaller is better | MMLU, higher is better | TWX4, same session | Points from TWX4 (95% range) | Reading | |---|---|---|---|---|---| | Qwen's FP8 build | 37.46 GB | 83.0% | 82.9% | +0.06 (−0.28 to +0.41) | Not separable, 0.06 points above: 312 right only for the FP8 build, 303 only for TWX4, p = 0.75; one 80 GB Hopper card, vLLM 0.25.1 | | NVIDIA's NVFP4 build | 23.42 GB | 82.6% | 82.9% | −0.36 (−0.75 to +0.03) | Not separable, 0.36 points below: 366 and 416, p = 0.080; the same session as the FP8 build | | Tileward TWX3 | 20.08 GB | 83.2% | 83.0% | +0.24 (−0.14 to +0.63) | Not separable, 0.24 points above: 401 and 367, p = 0.23; one RTX PRO 6000 Blackwell card, vLLM 0.31.0 | | Tileward TWX2 | 16.05 GB | 81.9% | 82.8% | −0.85 (−1.39 to −0.30) | Behind by 0.85 points: 704 and 823, p = 0.0025; this page's session | | ISTA-DASLab's GSQ build | 12.57 GB | 81.4% | 82.8% | −1.40 (−1.96 to −0.83) | Behind by 1.40 points: 721 and 917, p < 0.0001; one 80 GB Hopper card, vLLM 0.27.1 | ## Memory · Size TWX2 is 16.05 GB, 31% smaller than NVIDIA's 23.42 GB NVFP4 build. Eight builds of the same model: the original; three public builds, Qwen's own [FP8 build](https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8), NVIDIA's [NVFP4 build](https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4) and ISTA-DASLab's [GSQ build](https://huggingface.co/ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ); and four of ours, TWX4, [TWFP4](https://tileward.com/research/a-smaller-qwen3-6-35b-a3b/), TWX3 and TWX2. Each row carries the build's MMLU score beside TWX4's from the same session. TWX2 is 31% smaller than NVIDIA's build and 0.85 points behind TWX4, where NVIDIA's build was 0.36 points below it, not separable; TWX3, at 20.08 GB, is 14% smaller than NVIDIA's build and 0.24 points above TWX4, also not separable. The GSQ build is smaller than TWX2 and 1.40 points behind TWX4. | Build | Weights, smaller is better | Against the original, more is better | MMLU, against TWX4 in the same session, higher is better | |---|---|---|---| | Qwen3.6-35B-A3B as published | 71.90 GB | The original | – | | Qwen's FP8 build | 37.46 GB | 48% smaller (1.9×) | 83.0% against 82.9%: 0.06 points above, not separable | | Tileward TWX4 | 24.10 GB | 66% smaller (3.0×) | The reference: 82.8% to 83.0% across four sessions | | NVIDIA's NVFP4 build | 23.42 GB | 67% smaller (3.1×) | 82.6% against 82.9%: 0.36 points below, not separable | | Tileward TWFP4 | 23.08 GB | 68% smaller (3.1×) | – | | Tileward TWX3 | 20.08 GB | 72% smaller (3.6×) | 83.2% against 83.0%: 0.24 points above, not separable | | Tileward TWX2 | 16.05 GB | 78% smaller (4.5×) | 81.9% against 82.8%: 0.85 points behind | | ISTA-DASLab's GSQ build | 12.57 GB | 83% smaller (5.7×) | 81.4% against 82.8%: 1.40 points behind | ## Memory · One card TWX2 takes 13.5 GiB of the card; TWX4 takes 21.0. Both builds ran on one RTX PRO 6000 Blackwell card of 96 GB (95.59 GiB as its driver reports it). This is the card for each build, as the server loaded it: the model, and the rest of the card for everything else, conversations included. TWX2 leaves 7.5 GiB more of the card free. | Build | The model on the card, smaller is better | The rest of the card, more is better | |---|---|---| | Tileward TWX4 | 21.0 GiB | 74.6 GiB | | Tileward TWX2 | 13.5 GiB | 82.1 GiB | ## Which build Choose TWX2 when the weights must fit in 16 GB. TWX2 is 31% smaller than NVIDIA's NVFP4 build and 33% smaller than TWX4. Against TWX4 it gives up 0.85 MMLU points and makes up 1.0 point more of the facts it writes, 3.2% against 2.2%, for 7.5 GiB more of the card. NVIDIA's build, in its own session, was 0.36 points below TWX4 on MMLU, so read against the same TWX4, TWX2 sits about half a point below it. Where 20 GB fits, TWX3 (20.08 GB) is 14% smaller than NVIDIA's build and was 0.24 points above TWX4 on MMLU in its own session, not separable; where the card has room for 24.10 GB, TWX4 keeps those points. Both are in the [model library](https://tileward.com/model-library/); to run either, [talk to an engineer](https://calendar.app.google/hg2DzFTxKJgwFWm19). Our [TWFP4 build of the same model](https://tileward.com/research/a-smaller-qwen3-6-35b-a3b/), 23.08 GB, is measured against the original on its own report, with function calls and long documents. ## How it was measured Both builds ran one after the other on one card, the same way. TWX2 and TWX4 ran in one session on one RTX PRO 6000 Blackwell card, with the same engine, settings and questions, TWX2 first. - **The card:** RTX PRO 6000 Blackwell Server Edition (96 GB), with vLLM 0.31.0 serving each build from its own weights, the same engine files for both. - **Size:** the weights files, in decimal gigabytes. - **MMLU:** five-shot, all 14,042 questions, run by [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) with identical arguments for both builds and no sampling, each answer scored by the letter the model rates most likely. Compared question by question with an exact McNemar test, and a 95% range for the paired difference. - **Made-up facts:** each build reads 360 passages from Tileward's own document set and writes out the facts they state, taking its likeliest word every time; every fact is matched back to its passage, and one that no passage supports counts as made up. Three different draws of passages, the same for both builds, compared with a two-sided Fisher test, treating facts as independent. - **Tool calls:** six cases (one call, two calls in one turn, nested arguments, a tool that must not be called, the right tool out of twenty, a two-turn loop), 20 trials each, with thinking off and on, one pass, scored mechanically on tool name, call count and argument shape. - **The model on the card:** the memory each build's weights took, as its server reported when it loaded them. - **Other builds:** each pair ran the same MMLU questions and arguments for both of its builds, in one session: Qwen's FP8 and NVIDIA's NVFP4 builds with TWX4 on one 80 GB Hopper card and vLLM 0.25.1; ISTA-DASLab's GSQ build with TWX4 on one 80 GB Hopper card and vLLM 0.27.1, with the changes its model card asks for; TWX3 with TWX4 on one RTX PRO 6000 Blackwell card and vLLM 0.31.0. The 95% range is the normal approximation for a paired difference. - **The model:** Qwen3.6-35B-A3B is Qwen's, released under the [Apache 2.0 license](https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE). ## Related - [35B-A3B TWX2 eval card](https://tileward.com/evals/#compressed-model-a3b-twx2): the result in one screen - [Shrinking Qwen3.6-35B-A3B](https://tileward.com/research/a-smaller-qwen3-6-35b-a3b/): our TWFP4 build of the same model, against the original - [Qwen3.6-35B-A3B on Hugging Face](https://huggingface.co/Qwen/Qwen3.6-35B-A3B): the published model - [Talk to an engineer](https://calendar.app.google/hg2DzFTxKJgwFWm19): bring the questions and tools you would give Qwen3.6-35B-A3B and the card you run on - [Research](https://tileward.com/research/): every report