Benchmark

Stop paying every agent to reread everything.

Multi-agent systems get expensive when every step receives the whole working set again. Across six Jobs too big for one pass, so they are split across several runs. Three were coding pipelines and three were research tasks. Each step has to be told what the previous step produced., Tileward Context selected what each agent needed and cut billable tokens by 72.1% against broadcast context, including the cost of ingest. It also produced the strongest code in the run. On One step spawns several parallel lines of enquiry and a later step pulls them back together. It is the case with the most material in flight, which is why it saves the most and why it is hardest to get right., retrieval delivered only 0.49 to 0.66 of the declared material and quality fell. Selective context is ready to pay off on sequential workflows, while wide gathers should be evaluated against their own required-artifact set.

Compare the engineering pipeline and research fan-out eval cards.

The setup

The expensive part of an agent workflow is the hand-off.

Every multi-agent system has to answer one question over and over: when this agent finishes, what does the next one get to see?

RegimeWhat each agent receivesWhat it is charged for
broadcast controlThe whole shared corpus, every agent, every round.Everything it reads.
curatedOnly the artifacts a person tagged for that step, configured per problem by hand.What it reads. No overhead.
json-extractA structured summary of the previous step, produced by an extra model call.What it reads, plus the extraction calls.
recall-tuned TilewardWhat it asks the live Tileward Context store for.What it reads, plus every token of ingest.

Six problems (three coding pipelines, three research fan-outs) each ran through all four regimes inside a single invocation. Answers came from a 120B open-weight model, served as its publisher shipped it, through the same public API path a customer uses. Token counts come from the answering model's own tokenizer, chat template applied. Coding pipelines run seven steps; research fan-outs run eight agents, five of them gathering in parallel, over a shared corpus of roughly 49,000 tokens.

What population this is

These figures come from six agent problems, a different workload on a different population from the conversation figures published on Tileward Context (live single-threaded conversations across Claude Desktop and Claude Code). Neither set of numbers predicts the other, and they should not be quoted together.

The headline

72.1% fewer billable tokens, with quality that depends on workflow shape.

RegimeBillable tokensvs controlCoding scoreResearch, blind panelHand-off latency
curated350,898−75.2%0.8336.820 s
recall-tuned Tileward394,854−72.1%0.9685.3369 s
json-extract523,122−63.1%0.8655.692,080 s
broadcast control1,416,144, 0.8975.970 s

Coding scores are graded against reference answers, 0 to 1. The research column is a blind panel scored 1 to 10. Higher is better in both.

Reading the columns

Billable tokens and hand-off latency are deterministic: we re-ran problems four hours apart and context assembly reproduced to the digit. The fidelity measurements below are decidable by inspection rather than judged, and they reproduce in a second, independently written harness. The two quality columns are directional: at n=1, run-to-run variance is larger than the gaps between the arms.

The saving

Turn repeated context into a million-token saving.

The saving is net of ingest: 1,021,290 fewer billable tokens, after charging 182,900 tokens of ingest at full price.

Uncompressed control
1,416,144
Tileward Context
394,854

Two instruments. That figure is the tokenizer's count. The store keeps its own ledger, which over the same run separately reports 1,282,907 tokens saved. It is not the headline because the ledger does not net out ingest; the two agree in magnitude, not to the digit.

Nearly all of the saving lives on the research problems. There the control costs about 390,000 tokens per problem, because an eight-agent fan-out re-bills that 49,000-token corpus to every agent on every round. Coding pipelines are shorter and the gap is correspondingly smaller.

The split

Sequential pipelines and wide fan-out need different context strategies.

Sequential coding: selective context led the set

Graded against reference answers across three coding pipelines, Tileward (recall-tuned) scored 0.968 against the control's 0.897, and above both other compressed regimes, which posted 0.833 for hand-curated routing and 0.865 for structured extraction.

On coding, Tileward delivered a mean of 0.81 of the material each step had declared it needed, against the control's 1.00, and still scored highest: a warning about fidelity as a proxy for quality, in both directions.

Wide research fan-out: the boundary to test first

On three research fan-outs, judged by a blind panel from 1 to 10, Tileward (recall-tuned) scored 5.33 against the control's 5.97. Hand-curated routing scored 6.82 and structured extraction 5.69. Tileward is last of the four.

It is not independent of the generator. The judge is the same served model that produced the answers; the panel removes the format advantage and the position bias, not the family resemblance.

The cause

Required-artifact coverage makes the rollout decision inspectable.

Every step in the benchmark names the artifacts it must receive before the run starts, and those artifacts carry markers in their bodies. So "did this agent get what it asked for" is decidable by inspection rather than judged by a model.

On the research fan-outs, Tileward delivered between 0.49 and 0.66 of the declared material, against 1.00 for the control and for hand-curated routing. Required material is not arriving on an eight-agent fan-out, and a panel with no sight of any of that marks the answers down too.

An independently written earlier version of the benchmark reproduces it. Run on the two problems the two harnesses share, it put research fidelity at 0.646 against 0.857 on coding.

The hand-off

Retrieval compressed the hand-off without adding another generation step.

Tileward (recall-tuned) and json-extract do the same job: shrink what one agent passes to the next.

MeasureTileward recall-tunedjson-extract
Overhead tokens182,900458,909
Hand-off latency69 s2,080 s
Calls20245
Total billable394,854523,122
Coding score0.9680.865

30x less hand-off latency on this benchmark, at 24.5% lower total token cost and a higher coding score. Latency is the wall clock spent on hand-off calls, summed across the matrix.

What broke

The production path held; capacity was tested separately.

The 202 store calls in this matrix completed without a gateway error. Across both harnesses there was one functional failure: a single structured-extraction call was cut off by its completion budget mid-payload.

Answers hit their completion cap in every regime, including the control. On the generation calls, 35 of the control's 45 were capped against 26 for Tileward. The panel therefore judged some truncated answers in every arm; truncation was more common in the control, so it does not explain the research result.

Everything above ran at concurrency 1, so it says nothing about how much load the gateway takes. A separate load ramp establishes that there is a knee (past some level of concurrency, median latency jumps sharply while throughput falls), not where it sits: throughput varied roughly twofold at fixed concurrency, so any peak is one sample of a noisy quantity rather than a ceiling. It ran from a single client host, over the public internet, against a small store, and none of those bounds can be told apart from the gateway's own limit without repeating it from several hosts.

Deployment limits

Where this result applies, and where it does not.

The 72.1% net token saving and the hand-off latency are the figures this page stands behind.

Generation variance is larger than the quality gaps. The same problem and regime, run four hours apart, scored 1.000 then 0.809 for hand-curated routing, and 0.881 then 0.952 for structured extraction. Token counts reproduced to the digit across those runs (83,212 and 38,198 both times).

The panel's arms are not scored on identical step counts. It aggregates 13 steps, of which the control has an answer for 12. Over the 12 steps every arm shares, Tileward scores 5.61 against the control's 5.97 (the ordering is unchanged and Tileward is still last, with a narrower gap than the headline pair).

Hand-curated routing is not a product. It wins on cost and on the panel, and it is per-problem tag routing configured by a person who has already read the problem. What it establishes is that routing beats broadcasting (a property of the workload, not of any implementation).

Prompt caching is not modelled. A provider that discounts a repeated prefix narrows this gap, because the control sends every agent the same block. Any system that returns a different slice per query mutates that prefix and forfeits the discount. A cached baseline is the harder one.

One run, six problems. Three coding pipelines and three research fan-outs, run once, with no seed sweep and no confidence interval; the figures describe these six workloads.

Put it to work

Start with the workflow that repeats the most context.

Bring your own workload

Point the harness at a multi-agent flow you actually run and we will publish what it does. Talk to an engineer, or compare the sequential pipeline with the wide fan-out eval.