Keep most of the answer rate on 95% less context (19×).
LOCOMO is a benchmark of long conversations, with questions about what was said in them. Given the whole transcript, a model answered 63–67% of the answerable questions, on 28,365 tokens each. Given what Tileward Context returned, it answered 55%, on 1,499 tokens each. Given only the newest turns, it answered 8%. A separate model judged every answer.
See the matching eval card for the result and limits in one screen.
It keeps most of the answers on a nineteenth of the context.
LOCOMO (Maharana et al.) has ten conversations that run for hundreds of turns over several sittings, and 1,986 questions about them. We asked the answerable ones four ways: the whole transcript, only the newest turns, what Tileward Context returned, and no conversation at all.
| What the model was given | Questions answered | Context per question | Against sending everything |
|---|---|---|---|
| Everything, as-is | 63% | 28,365 tokens | Baseline |
| Tileward Context | 55% | 1,499 tokens | 8 points fewer, on 95% less context (19×) |
| The newest turns that fit the same budget | 8% | 2,165 tokens | 55 points fewer, on 92% less (13×) |
| Nothing (no conversation at all) | 0% | 138 tokens | 63 points fewer, on 99.5% less (206×) |
The whole transcript ran out of room on 99 of the longest questions; leaving those out, it scores 67%. Against the newest turns, Tileward Context won 769 questions and lost 41.
A nineteenth of the context, for eight to twelve points of answer rate.
Use selective context when the whole transcript is too expensive, or too large for the window. Here Tileward Context sent 95% less (18.9×) and finished eight to twelve points behind the whole transcript. Test that trade on your own conversations.
When the answer isn’t in the conversation, Context makes one up 40–59% less often.
About a fifth of LOCOMO’s questions ask about something the conversation never says. The right response is to say so. We counted how often each approach stated an answer anyway, on the 120B model above and on our own Tileward 35B-A3B.
On Tileward 35B-A3B, Context made an answer up 26% of the time, against 63% with the whole transcript: 59% fewer (95% interval 53–65%). On the 120B model it was 31% against 52%: 40% fewer.
Sending less makes a model say “not mentioned” more often. That cuts made-up answers, and it also costs some answers the model could have given: the last column. The newest turns make almost nothing up because they answer almost nothing.
| What the model was given | Made up an answer, 120B model | Made up an answer, Tileward 35B-A3B | Answerable questions right, Tileward 35B-A3B |
|---|---|---|---|
| Everything, as-is | 52% | 63% | 72% |
| Tileward Context | 31% | 26% | 54% |
| The newest turns that fit the same budget | 8% | 3% | 7% |
The same questions, the same model and the same judge for every row.
Each table changes one thing, what the model was given, and holds the rest still.
The questions. LOCOMO’s 1,986 questions on ten long conversations. The first table counts the 1,540 the conversations can answer. The 446 unanswerable ones are scored in the second, on whether the model stated an answer anyway.
The judge. A separate model, Claude Sonnet 4.6, graded every answer against the reference. The model that answered never graded itself.
The models. A 120B open-weight model answered for the first table. The second adds Tileward 35B-A3B.