Zero-Mem: Zero-Token Memory Operations for LLM Agents

Agent memory that never calls an LLM until the final answer, and still scores higher.

0.22s per query

Memory-operation time per query

Zero LLM tokens spent on memory, and faster than every baseline

How it works

Memory costs LLM calls
Many agent memories summarize, extract or rewrite history with extra LLM calls. That costs tokens and time, and can blur details.
Index raw traces instead
Keep original conversation turns. Build an entity graph and a time hierarchy with NER, BM25 and embeddings, then retrieve and filter deterministically.
Zero tokens, better answers
Only the final reader uses an LLM. On LoCoMo it beats GAM by 5.40 average F1 with GPT-4o-mini.

What they found

Which comes out on top on F1 on HotpotQA 56K?

How much evidence to retrieve

try it

Drag k up and watch relevant hits versus noise in retrieved evidence.

Items retrieved5

relevant found: 3 of 6
precision: 60%
recall: 50%

Gains flatten past a handful of items; the paper's top-5 trails top-10 only slightly.Illustrative model, not the paper's data

Terms

6 terms

Zero-token memory No LLM call or token spent anywhere except the final answering step.She

A librarian who keeps every original letter, indexes them by names mentioned and by date, then hands you the letters themselves instead of writing summaries.

Where it breaks

The librarian understands meaning; Zero-Mem only matches entities, words and embeddings.

Limits

Encoder and NER compute still cost time, just not LLM tokens.
Tested only on question answering, not full agent tasks with actions.
Rule-based routing and calibration may not transfer to new query types.

From the paper

How it works

Memory costs LLM calls

Index raw traces instead

Zero tokens, better answers

What they found

How much evidence to retrieve

Terms

Limits