We made Qwen3-VL-32B 8.7% smaller than NVFP4, at the same quality.
Qwen3-VL-32B-Instruct reads images and text. As published its weights are 66.71 GB, and Red Hat’s public NVFP4 build is 21.86 GB. Our TWFP4g build is 19.96 GB, 8.7% smaller again (1.09×). It is level with NVFP4 on MMLU, MMStar, made-up facts and tool calls.
See the matching eval card for the result in one screen.
Six builds of one model, on one 96 GB card.
We measured three Tileward builds against the original and two public builds. Every build ran on stock vLLM 0.31.0, on one RTX PRO 6000 Blackwell card.
| Build | Weights | Against the original |
|---|---|---|
| Qwen3-VL-32B-Instruct as published | 66.71 GB | The reference |
| Qwen’s FP8 build | 35.52 GB | 47% smaller (1.88×) |
| Red Hat’s NVFP4 build | 21.86 GB | 67% smaller (3.05×) |
| Tileward TWFP4 | 22.60 GB | 66% smaller (2.95×) |
| Tileward TWFP4r | 19.96 GB | 70% smaller (3.34×) |
| Tileward TWFP4g | 19.96 GB | 70% smaller (3.34×) |
TWFP4g is 8.7% smaller than Red Hat’s build (1.09×), and the rest of this report is about it.
Level with NVFP4 on MMLU, MMStar, made-up facts and tool calls.
Both builds were measured in one session, on the same card, under serving settings: a 262,144-token window, an FP8 cache and tool calling on.
| Test | Red Hat NVFP4 | Tileward TWFP4g | Paired difference |
|---|---|---|---|
| MMLU, 14,042 questions | 82.5% | 82.9% | Level (p = 0.052) |
| MMStar, 1,500 image questions | 70.7% | 70.5% | Level (p = 0.79) |
| Facts made up | 5.6% (101 of 1,799) | 5.8% (98 of 1,680) | Level (p = 0.83) |
| Tool calls right | 600 of 600 | 600 of 600 | The same |
| One request of 199,840 tokens | Answered in 142 s | Answered in 142 s | The same |
The original scored 83.7% on MMLU in the same session: Red Hat’s build is 1.2 points below it, and ours 0.8.
NVFP4 picks the original’s next word more often.
Next-word agreement is how often a build picks the original’s next word on the same text. Red Hat’s build agrees 89.7% of the time and TWFP4g 87.7%. Every build here ran in the first session, under the same settings.
| Build | Next-token agreement | MMStar | MMLU, 1,140 questions |
|---|---|---|---|
| The original | The reference | 71.1% | – |
| Qwen’s FP8 build | 95.7% | 72.3% | – |
| Red Hat’s NVFP4 build | 89.7% | 71.4% | 83.9% |
| Tileward TWFP4 | 85.6% | 71.9% | 82.7% |
| Tileward TWFP4r | 84.8% | 70.4% | 83.0% |
| Tileward TWFP4g | 87.7% | 70.9% | 83.1% |
No MMStar or MMLU difference from NVFP4 in this session is larger than chance (p from 0.11 to 0.58).
1.76 GiB less on the card, and 2.9% more room for the cache.
Measured under serving settings, with the server allowed 90% of the card.
| Build | Weights on the card | Tokens the cache holds |
|---|---|---|
| Red Hat’s NVFP4 build | 20.84 GiB | 500,688 |
| Tileward TWFP4g | 19.08 GiB | 514,992 |
TWFP4g fits the same work in less of the card.
TWFP4g matches Red Hat’s build on MMLU, MMStar, made-up facts and tool calls, in 1.76 GiB less of the card.
Of our other two builds, TWFP4 is 3.4% larger than NVFP4. TWFP4r agrees with the original least often of any build here.
Every comparison pairs builds from one session, on the same card and engine.
Figures from different sessions are shown side by side, but never compared.
The card. One RTX PRO 6000 Blackwell (96 GB), with stock vLLM 0.31.0 serving every build from its own weights. In the first session every build used the same settings and a 32,768-token window. In the second, TWFP4g and Red Hat’s build ran under serving settings.
Size. The weights files on disk.
MMLU. Five-shot, no chat template and no sampling, with lm-evaluation-harness. The paired test counts the answers that differ: 371 right only for ours, 319 only for Red Hat’s.
MMStar. 1,500 image questions, greedy, with replies capped at 1,024 tokens; a reply cut off at the cap is scored wrong. 60 were right only for ours and 64 only for Red Hat’s.
Made-up facts. Each build reads 360 passages from Tileward’s own document set and writes out the facts they state. Every fact is matched back to its passage, and one that no passage supports counts as made up. The test treats facts as independent.
Tool calls. Six cases, 20 trials each, run five times: 600 calls. The cases are one call, two calls in one turn, nested arguments, a tool that must not be called, the right tool out of twenty, and a two-turn loop. Each call is scored on tool name, call count and argument shape, with vLLM’s hermes tool parser.
Long request. One prompt of 199,840 tokens with one fact planted in it; both builds returned the fact.
Agreement. Each build reads the same 300 passages (86,589 token positions), and we count where its top next token is the original’s. Every build repeats itself exactly when run twice, so the gaps are not noise. The mean KL distance from the original is 0.056 for ours and 0.046 for Red Hat’s.
Bring the documents and images you work with and the card you run on, and we will measure the same things. Talk to an engineer, or compare the vision model eval card.