BFCL
Function calls: 500 tasks, three runs each, scored on calling the right functions the right way. Higher is better.
The full result, kind of call by kind of callBenchmark
We shrank Qwen3.6-35B-A3B by 68%, level with the original on tool calls and long documents.
Qwen3.6-35B-A3B is Qwen’s model of 35 billion parameters, about 3 billion of which work on each word. As Qwen publishes it, its weights take 71.90 GB. We made TWFP4 at 23.08 GB: 68% smaller (3.1×), and 1.5% smaller than NVIDIA’s NVFP4 build. We tested whether it still does the work as well. On BFCL’s 500 function-calling tasks it scored 75.9% against the original’s 76.3%, and on AA-LCR’s questions over long documents 59.6% against 60.8%: no measurable difference on either. On one RTX PRO 6000 Blackwell card the model takes 19.1 GiB of memory, where the original takes 64.6.
See the matching eval card for the result in one screen.
At a glance
Against Qwen3.6-35B-A3B as published: 68% smaller, level on tools and long documents.
Qwen’s own release is the version of Qwen3.6-35B-A3B people run today, so each line here is our build against it. Every section below starts with its own line, then gives the full measurement.
68%
smaller with TWFP4
23.08 GB against 71.90 GB
99.5%
of the original’s BFCL function-call score: no measurable difference
75.9% against 76.3%
98.0%
of the original’s AA-LCR long-document score: no measurable difference
59.6% against 60.8%
96.5%
of the original’s next words
at 45,430 places
45.5 GiB
less of the card for the model
19.1 GiB against 64.6 GiB
| Against Qwen3.6-35B-A3B as published | Tileward TWFP4 |
|---|---|
| Size of the weights | Smaller by 68%: 23.08 GB against 71.90 GB |
| Function calls, on BFCL | No measurable difference: 99.5% of the original’s score, 75.9% against 76.3% |
| Questions over long documents, on AA-LCR | No measurable difference: 98.0% of the original’s score, 59.6% against 60.8% |
| Picking the original’s next word | At 96.5% of 45,430 places |
| The model’s share of one RTX PRO 6000 Blackwell card | Less by 45.5 GiB: 19.1 GiB against 64.6 GiB |
Benchmarks
Level with the original on function calls and on long documents.
Every test on this page, with each build’s score drawn from zero. Each card links to the full result below.
AA-LCR
Long documents: 25 questions, each over a long set of documents, ten runs each. Higher is better.
The full result, with the countsFindings · Quality
No measurable difference from the original on function calls or long documents.
We asked two things of TWFP4. Can it still call functions? BFCL gives it 500 tasks, from a single call to conversations of several turns, and scores each call on choosing the right function with the right arguments. Each build ran every task three times, and higher is better. Can it still answer from long documents? AA-LCR asks 25 questions, each needing a long set of documents read through, and each build answered every question ten times. Higher is better.
The chart gives TWFP4’s score as a share of the original’s, and the range it could fall in by chance alone. Right of the dashed line is above the original. Both ranges cross it.
The scores, each the mean of its runs. Tap a verdict for the counts behind it.
| Test | The original | Tileward TWFP4 | Against the original |
|---|---|---|---|
| BFCL, 500 tasks run three times, higher is better | 76.3% (1,144 of 1,500) | 75.9% (1,138 of 1,500) | Compared task by task over the three runs, 55 tasks went better for TWFP4 and 58 for the original (p = 0.85). |
| AA-LCR, 25 questions run ten times, higher is better | 60.8% (152 of 250) | 59.6% (149 of 250) | Compared question by question over the ten runs, 5 questions went better for TWFP4 and 6 for the original (p = 1.0). |
BFCL’s tasks fall into four kinds. TWFP4 is level with the original on each: the same count on single calls, and within six calls of the original’s count on each of the other three.
| Kind of task | Tasks | The original higher is better | Tileward TWFP4 higher is better |
|---|---|---|---|
| One call | 160 | 85.8% (412 of 480) | 85.8% (412 of 480) |
| Several calls in one reply | 100 | 91.0% (273 of 300) | 89.0% (267 of 300) |
| Conversations of several turns | 160 | 53.8% (258 of 480) | 54.4% (261 of 480) |
| No function fits, so no call | 80 | 83.8% (201 of 240) | 82.5% (198 of 240) |
Findings · Against the original
TWFP4 picks the original’s next word at 96.5% of places.
A test score can hide small changes in how a model writes. Next-word agreement is a finer check: TWFP4 reads the same replies as the original, and at each place we ask whether it would pick the same next word. Higher is closer to the original. The last column measures how far its word choices drift from the original’s, where lower is closer.
| Replies read | Places compared | Next-word agreement higher is closer | Drift from the original lower is closer |
|---|---|---|---|
| 100, to function-call tasks and everyday requests | 45,430 | 96.5% | Mean KL divergence over the original’s 16 likeliest next words at each place. 0 would mean the same word choices as the original. |
Read twice on the same card, the original picked its own next word at all but 3 of the 45,430 places.
Memory · Size
TWFP4 is 23.08 GB against 71.90 GB for Qwen3.6-35B-A3B as published.
Five builds of the same model: the original, two smaller public builds (Qwen’s own FP8 build and NVIDIA’s NVFP4 build), our TWX4 build from the model library, and TWFP4. The chart gives how much space each build’s weights take, and a shorter bar is a smaller build. The tests on this page compare TWFP4 with the original; the other three are here for their size. TWFP4 is 1.5% smaller than NVIDIA’s build.
| Build | Weights smaller is better | Against the original more is better |
|---|---|---|
| Qwen3.6-35B-A3B as published | 71.90 GB | The reference |
| Qwen’s FP8 build | 37.46 GB | 48% smaller (1.9×) |
| NVIDIA’s NVFP4 build | 23.42 GB | 67% smaller (3.1×) |
| Tileward TWX4 | 24.10 GB | 66% smaller (3.0×) |
| Tileward TWFP4 | 23.08 GB | 68% smaller (3.1×) |
Memory · One card
TWFP4 takes 19.1 GiB of the card; the original takes 64.6.
Both builds ran on one RTX PRO 6000 Blackwell card of 96 GB. This is the card for each build, as the server loaded it: the model, and the rest of the card. Both builds kept the same 8 GiB of the rest for conversations, so they held the same 408,122 tokens of conversation; TWFP4 leaves 45.5 GiB more of the card free beyond that.
Qwen3.6-35B-A3B as publishedLeaves 30.4 GiB of the card
The model: 64.6 GiBThe rest of the card: 30.4 GiB
Tileward TWFP4Leaves 75.8 GiB of the card (2.5×)
The model: 19.1 GiBThe rest of the card: 75.8 GiB
| Build | The model on the card smaller is better | The rest of the card more is better | Conversation held the same for both |
|---|---|---|---|
| Qwen3.6-35B-A3B as published | 64.6 GiB | 30.4 GiB | 408,122 tokens |
| Tileward TWFP4 | 19.1 GiB | 75.8 GiB | 408,122 tokens |
Which build
TWFP4 kept the original’s tool and document results in a third of the space.
Choose TWFP4 when room on the card matters: there is no measurable difference from the original on BFCL or AA-LCR, it picks the original’s next word at 96.5% of places, and it takes 19.1 GiB of the card against 64.6. To run it, talk to an engineer.
The model library lists our TWX4 build of the same model, 24.10 GB. Its results are on its own card on the Evals page, from a different card and session, so the two builds are shown side by side here, not compared.
How it was measured
Both builds were served the same way, on the same model of card.
The original and TWFP4 ran with the same engine files, settings and questions on RTX PRO 6000 Blackwell cards: BFCL and AA-LCR on one card for each build, and next-word agreement on one card for both. The settings for each test follow.
The cards. RTX PRO 6000 Blackwell Server Edition (96 GB), with vLLM 0.32.0 serving each build from its own weights and the same engine files on every card: a 262,144-token window, 82% of the card, and 8 GiB set aside for conversations. Both builds used Qwen’s own sampling settings (temperature 1.0, top-p 0.95, top-k 20), with thinking on and replies allowed to run to the window.
Size. The weights files, in decimal gigabytes.
BFCL. 500 tasks of BFCL v4, drawn across its categories with conversations of several turns and several calls in one reply weighted up, three runs for each build, calling the functions natively and scored by BFCL’s own checker. The paired test counts the tasks that went differently.
AA-LCR. 25 of the 100 questions in AA-LCR, over long sets of documents, ten runs for each build. Every answer is graded against its reference answer by the same judge model for both builds. The paired test counts the questions that went differently.
Share of the original’s score. TWFP4’s mean score over the original’s, with a 95% range from the spread of each build’s runs.
Agreement. 100 prompts, half from BFCL and half everyday chat requests. The original wrote a reply of 512 tokens to each, taking its likeliest word every time; then both builds read those replies back on the same card, and at each of 45,430 places we compare their likeliest next word.
The model. Qwen3.6-35B-A3B is Qwen’s, released under the Apache 2.0 license.
Bring your own workload
Bring the tools and documents you would give Qwen3.6-35B-A3B and the card you run on, and we will measure the same things. Talk to an engineer, or compare the 35B-A3B TWFP4 eval card.