# We made Qwen3-VL-32B 8.7% smaller than NVFP4, at the same quality

Tileward research report R-010. Qwen3-VL-32B-Instruct reads images and text. As published its weights are 66.71 GB, and Red Hat's public NVFP4 build is 21.86 GB. Tileward's TWFP4g build is 19.96 GB, 8.7% smaller again (1.09×). It is level with NVFP4 on MMLU, MMStar, made-up facts and tool calls.

Page: https://tileward.com/research/a-smaller-vision-model/

## How it was measured

Every comparison pairs builds from one session, on the same card and engine. Figures from different sessions are shown side by side, but never compared.

- **Builds:** [Qwen3-VL-32B-Instruct as published](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct), Qwen's [FP8 build](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct-FP8), Red Hat's [NVFP4 build](https://huggingface.co/RedHatAI/Qwen3-VL-32B-Instruct-NVFP4), and three Tileward builds: TWFP4, TWFP4r and TWFP4g.
- **Card:** one RTX PRO 6000 Blackwell (96 GB), with stock vLLM serving every build from its own weights. In the first session every build used the same settings and a 32,768-token window. In the second, TWFP4g and Red Hat's build ran under serving settings: a 262,144-token window, an FP8 cache and tool calling on.
- **Size:** the weights files on disk.
- **MMLU:** [MMLU](https://arxiv.org/abs/2009.03300), five-shot, no chat template and no sampling, with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness). The paired test counts the answers that differ: 371 right only for ours, 319 only for Red Hat's.
- **MMStar:** [MMStar](https://arxiv.org/abs/2403.20330), 1,500 image questions, greedy, with replies capped at 1,024 tokens; a reply cut off at the cap is scored wrong. 60 were right only for ours and 64 only for Red Hat's.
- **Made-up facts:** each build reads 360 passages from Tileward's own document set and writes out the facts they state. Every fact is matched back to its passage, and one that no passage supports counts as made up. The test treats facts as independent.
- **Tool calls:** six cases, 20 trials each, run five times: 600 calls. Each call is scored on tool name, call count and argument shape, with vLLM's hermes tool parser.
- **Agreement:** each build reads the same 300 passages (86,589 token positions), and we count where its top next token is the original's. Every build repeats itself exactly when run twice, so the gaps are not noise.

## Results

| Build | Weights | Against the original |
|---|---|---|
| Qwen3-VL-32B-Instruct as published | 66.71 GB | The reference |
| Qwen's FP8 build | 35.52 GB | 47% smaller (1.88×) |
| Red Hat's NVFP4 build | 21.86 GB | 67% smaller (3.05×) |
| Tileward TWFP4 | 22.60 GB | 66% smaller (2.95×) |
| Tileward TWFP4r | 19.96 GB | 70% smaller (3.34×) |
| Tileward TWFP4g | 19.96 GB | 70% smaller (3.34×) |

Quality, second session, under serving settings:

| Test | Red Hat NVFP4 | Tileward TWFP4g | Paired difference |
|---|---|---|---|
| MMLU, 14,042 questions | 82.5% | 82.9% | Level (p = 0.052) |
| MMStar, 1,500 image questions | 70.7% | 70.5% | Level (p = 0.79) |
| Facts made up | 5.6% (101 of 1,799) | 5.8% (98 of 1,680) | Level (p = 0.83) |
| Tool calls right | 600 of 600 | 600 of 600 | The same |
| One request of 199,840 tokens | Answered in 142 s | Answered in 142 s | The same |

The original scored 83.7% on MMLU in the same session: Red Hat's build is 1.2 points below it, and ours 0.8.

Next-word agreement with the original, first session, every build under the same settings:

| Build | Next-token agreement | MMStar | MMLU, 1,140 questions |
|---|---|---|---|
| The original | The reference | 71.1% | – |
| Qwen's FP8 build | 95.7% | 72.3% | – |
| Red Hat's NVFP4 build | 89.7% | 71.4% | 83.9% |
| Tileward TWFP4 | 85.6% | 71.9% | 82.7% |
| Tileward TWFP4r | 84.8% | 70.4% | 83.0% |
| Tileward TWFP4g | 87.7% | 70.9% | 83.1% |

No MMStar or MMLU difference from NVFP4 in that session is larger than chance (p from 0.11 to 0.58).

Memory, second session, the server allowed 90% of the card:

| Build | Weights on the card | Tokens the cache holds |
|---|---|---|
| Red Hat's NVFP4 build | 20.84 GiB | 500,688 |
| Tileward TWFP4g | 19.08 GiB | 514,992 |

## Which build

TWFP4g matches Red Hat's build on MMLU, MMStar, made-up facts and tool calls, in 1.76 GiB less of the card. Of our other two builds, TWFP4 is 3.4% larger than NVFP4, and TWFP4r agrees with the original least often of any build here.

## Related

- [Vision model eval card](https://tileward.com/evals/#compressed-vision-model): the result in one screen
- [Qwen3-VL-32B-Instruct on Hugging Face](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct): the published model
- [Talk to an engineer](https://calendar.app.google/hg2DzFTxKJgwFWm19): bring the documents and images you work with and the card you run on
- [Research](https://tileward.com/research/): every report