Evidence for deploying compressed, governed AI.
- Entries
- 12
- Latest
- 28 Oct 2026
- Scope
- Product runs
Tileward Research turns benchmark runs into practical guidance for deploying AI: what our tests show about context, model compression, and governance, and what to validate against your own workload.
Showing all 9 write-ups, newest first.
-
R-01228 October 2026 Benchmark
We shrank GLM-5.3-Flash by 68%, and it solved 86% of SWE-bench Verified.
GLM-5.3-Flash is 642.65 GB in 16 bits; Tileward’s TWFP4 build is 204.45 GB and runs on two B300 cards. As a coding agent it solved 431 of 500 SWE-bench Verified tasks, and 429 on a repeat. The original was not run on SWE-bench; TWFP4 picks its next word 88.8% of the time.
Experiment- 204.45 GB
- of weights, against 642.65 GB in 16 bits
- 431 of 500
- SWE-bench Verified tasks solved
- 429 of 500
- solved when the run was repeated
-
R-01128 October 2026 Benchmark
We shrank Kimi K3 by 35%, within a point of the original on MMLU.
Kimi K3 is 1,560.9 GB as Moonshot publishes it, already stored in 4 bits. Tileward’s TWX2.5 build is 1,007.9 GB, 90.5% on MMLU against 91.1%: 0.6 points below, inside the one-point bar set before the run, and level on made-up facts and tool calls. Our TWX3 build keeps closer to the original.
Experiment- 1,007.9 GB
- of weights, against 1,560.9 GB as published
- 90.5%
- on MMLU, against 91.1% for the original
- 2.7×
- the original’s cache, on the same eight cards
-
R-01028 October 2026 Benchmark
We made Qwen3-VL-32B 8.7% smaller than NVFP4, at the same quality.
Qwen3-VL-32B reads images and text. Tileward’s TWFP4 build is 19.96 GB against 21.86 GB for Red Hat’s NVFP4 build, and level with it on MMLU, MMStar, made-up facts and tool calls. Red Hat’s build stays closer to the original on next-word agreement.
Experiment- 19.96 GB
- of weights, against 21.86 GB for NVFP4
- 82.9%
- on MMLU, against 82.5% for NVFP4
- 1.76 GiB
- less of the card for its weights than NVFP4
-
R-00916 October 2026 Benchmark
We shrank Whisper large-v3 by 68%, below the size of turbo.
Whisper large-v3 is a 3.09 GB speech recogniser. Tileward’s TWX4 build is 0.98 GB, smaller than the published turbo version. On nine of ten test sets it adds at most 0.3 points to the original’s error rate, and on Danish it adds 1.06.
Experiment- 0.98 GB
- of weights, against 3.09 GB as published
- 0.3 points
- at most, of error added on nine of ten sets; 1.06 on Danish
- 152
- live calls on one 20 GB card, against 140
-
R-00810 October 2026 Benchmark
We compressed VoiceChat by 74% and made it run in real time on one card.
NVIDIA’s VoiceChat is a speech model that can call tools while it talks. Tileward’s TWX4 build is an 11.5 GB file against 44.4 GB. It picked the right tool on 55 of 80 spoken requests, level with the original’s 52, and chooses the original’s next word 95% of the time. With Tileward’s serving changes VoiceChat runs in real time on one card. The build runs from unpacked weights, so its speed and memory there match the original’s.
Experiment- 11.5 GB
- of file, against 44.4 GB as published
- 55 of 80
- right tool on spoken requests, against 52 for the original
- 72 ms
- for each 80 ms frame with our serving changes, against 114 ms for NVIDIA’s open code
-
R-0078 October 2026 Benchmark
We shrank PersonaPlex by 62% to run at 20ms a frame.
NVIDIA’s PersonaPlex speech model is 16.7 GB as published. Tileward’s TWX4 build is 6.3 GB, needs 20 ms for each 80 ms frame against about 40 on one laptop, and picks the original’s next word 96% of the time. A standard 8-bit conversion stays closer, at 8.2 GB.
Experiment- 6.3 GB
- of weights, against 16.7 GB as published
- 20 ms
- of compute for each 80 ms frame, against about 40
- 96%
- of the time, the same next word as the original
-
R-0066 September 2026 Benchmark
Keep most of the answer rate on 95% less context (19×).
From a 1,986-question LOCOMO set, Tileward Context answered 55% of answerable questions with 1,499 tokens each; full transcripts answered 63–67% with 28,365. A separate model judged every answer, and on unanswerable questions Context made up an answer far less often.
Experiment- 769 / 41
- questions won and lost against keeping the newest turns
- 95%
- less context than sending everything (18.9×)
- 55% vs 63–67%
- questions answered, against sending the whole transcript
-
R-0054 September 2026 Benchmark
Give agents useful context, not the whole backlog.
Across 104 AgentBench tasks, Tileward Context solved 37 with 523,028 context tokens; full history solved 24 with 1,651,372. The 68% (3.2×) reduction is stable, but 14–19 tasks can flip between identical runs, so the repeated ordering matters more than a fixed 13-task margin.
Experiment- 37 / 24
- tasks solved with Context / full history, of 104
- 68%
- less context than sending everything (3.2×)
- 14–19
- tasks the same setup flips against itself
-
R-00431 August 2026 Benchmark
Improve recall without losing correct answers.
On 140 scored LongMemEval-S questions, one recall upgrade moved the same store and model from 67 correct to 81: 14 wrong-to-right flips and no reversals. This is one self-judged run, and the upgrade itself remains undisclosed.
Experiment- +10.0
- points on LongMemEval-S, paired
- 14 / 0
- questions fixed / broken
- 9→16
- LongMemEval-S temporal-reasoning, of 40
-
R-00328 August 2026 Benchmark
A smaller model should not become a different model.
Across two model families, compression did not create new drift in sensitive record-copying behavior. Model selection mattered far more, while a paired knowledge test measured the actual trade-off at 0.40 MMLU points on a 27B research build.
Experiment- 26 of 30
- starved tokens shared across families
- 0.0
- embedding difference after compression
- −0.40
- MMLU points, 27B research build against its original, paired p = 0.021
-
R-00228 August 2026 Governance
Put policy in front of the model, not inside the prompt.
A pre-inference gate turns a policy from advice to an enforceable, auditable decision. We tested it with 2,251 requests written to evade the rule and 50 benign controls. The payoff is a boundary your team can test before depending on it.
Experiment- 2,251
- adversarial cases
- 50
- crossed the gate
- 5 of 50
- benign controls refused
-
R-00125 August 2026 Benchmark
Stop paying every agent to reread everything.
Across six multi-agent problems, Tileward Context cut billable tokens by 72.1% against broadcast context, including ingest, and produced the strongest code in the run. Wide research fan-out exposed a retrieval boundary, so test that shape first.
Experiment- ≈ 72.1%
- reduction in billable tokens
- 0.968
- coding, best of four
- 5.33
- research panel, last of four
No research write-ups match that filter yet.