Recall compared with mem0 and Zep
Maya tells her assistant twelve things over nine months: three homes, three jobs, three work emails, a sister who moves, and, in one sentence, a new diet and a shellfish allergy. Then she asks questions whose answers depend on which statement is current, and in one scenario asks it to forget the allergy. The same model extracts, answers and judges for all three memories.
mem0 18, 18 and 15; Zep 12 and 11
mem0 12.7k, Zep 14.7k
mem0 and Zep lost it 3 of 3
Three scripted conversations, each run three times per system. Claude Opus 5.5 extracted memories from every message, answered every question from what each memory returned, and judged every answer against a reference answer written before the runs. mem0 2.2.1 open source, Zep's open-source engine graphiti-core 0.31.1, and Recall 0.20 with its built-in embedding model, all with default settings. Tokens are what the model used to save one message into memory, averaged over the run.
The first scenario, replayed from the recorded runs.
Scenario 1: facts that change
Twelve statements dated over nine months, then six questions: where Maya lives, where she works, which email to use, where her sister is, what she eats, and whether she is moving. Every answer depends on a later statement replacing an earlier one.
| Memory | Right answers | Tokens per saved message | Seconds per saved message |
|---|---|---|---|
| Recall | 18 of 18 | 3.0k | 3.8 |
| mem0 | 18 of 18 | 12.7k | 3.4 |
| Zep | 12 of 18 | 14.7k | 14.4 |
Six questions, three runs each.
Asked why it believes an answer, Recall quotes the message the fact came from, with its date and the values it replaced. Zep quotes a message. mem0 keeps no source text.
Scenario 2: a fact stated in passing, then forgotten
The same conversation, with the diet and the allergy arriving in one sentence: "I've started eating fish again, so I'm pescatarian now, but no shellfish, I'm allergic." Six questions, then "forget that Maya is allergic to shellfish", then "what can Maya eat?". Recall and mem0 only.
| Memory | Right answers before the forget | Diet kept after the forget | Tokens per saved message |
|---|---|---|---|
| Recall | 18 of 18 | 3 of 3 | 3.1k |
| mem0 | 18 of 18 | 0 of 3 | 12.7k |
mem0 stored the diet and the allergy as one memory, so forgetting the allergy deleted the diet with it. Recall files each fact in a sentence as its own statement, so the forget withdrew the allergy alone, and its history still shows that it was withdrawn and when.
Scenario 3: forget first, then ask
The conversation again, with the forget before the questions. All three systems.
| Memory | Right answers | "What can Maya eat?" | Tokens per saved message |
|---|---|---|---|
| Recall | 18 of 18 | right 3 of 3: pescatarian, with the vegetarian diet it replaced | 3.2k |
| mem0 | 15 of 18 | wrong 3 of 3: answered vegetarian from an older note after deleting its diet-and-allergy memory | 12.7k |
| Zep | 11 of 18 | wrong 3 of 3: deleted both diet facts and answered that it had no notes | 14.8k |
Recall 0.20.2 in this scenario. The Recall page steps through the replies from one of these runs.
How to forget
In Recall a forget is one call that names the fact. It records a withdrawal, and the history of the fact shows the withdrawal beside the statements before it. In mem0 and Zep, application code searches for the memory, then deletes it by id, and the memory is gone from the record.
What this does not show
- Three short conversations, not a large benchmark. For a 267-question benchmark over long histories, see Recall on LongMemEval.
- mem0 and Zep were run as their open-source engines with default settings. Their hosted products were not measured.
- Each system was given the same model and the same messages. Tuning any of the three could change the numbers.
Try it on your own conversation
One pip install brings the client and the database. Save a few messages, correct one, and read back what was believed before.