Offload context out of the LLM window

Every file an agent reads and every log it greps stays in its context, so the hundredth model call carries the first ninety-nine. Recall keeps each turn as a record, stores any output longer than a threshold whole with a reference in its place, and builds the model's starting context from the working state and recent turns within a token budget you set.

The code

Runs as written: pip install polign-recall brings the client, the recall binary and a local polign_db. Swap local_dir for backend="polign" and a server URL when the database should live in your bucket, or for PostgreSQL, MongoDB or Qdrant.

from polign_recall import Client

log = "\n".join(f"{i:05d} GET /api/orders/{i} 200 {i % 7 * 13} ms" for i in range(2000))  # ~60 KB of tool output

with Client(local_dir="./recall-data", agent=True) as client:
    with client.resume("log-reviewer", token_budget=2000, output_threshold=500) as agent:
        agent.record_turn("user", "Find the slow requests in today's access log.")
        turn = agent.record_turn("tool", log, name="read_file")   # longer than the threshold: stored whole
        print(turn.output_ref)                                    # the briefing carries this reference, not the log
        agent.update_working_state(focus="requests over 60 ms", progress="log loaded, 2000 lines")

    with client.resume("log-reviewer", token_budget=2000) as agent:
        briefing = agent.context.briefing
        print(len(briefing), "characters in the briefing;", len(log), "in the log")
        print(agent.fetch_output(turn.output_ref).content[:60])  # the whole output, on demand

What it printed:

out-2-1
1239 characters in the briefing; 72603 in the log
00000 GET /api/orders/0 200 0 ms
00001 GET /api/orders/1 200

What the record looks like

The 72 KB tool result became one output record with a summary and the content, and the turn that produced it carries the reference out-2-1. The briefing shows the reference and the summary; the model calls fetch_output only when it needs the rest.

log-reviewer:out:out-2-1   {"record":"output","ref":"out-2-1","tokens":"18151","epoch":"1",
                            "body":"{\"ref\":\"out-2-1\",\"tool\":\"read_file\",\"summary\":\"00000 GET /api/orders/0 200 0 ms\\n00001 ...\",
                                     \"content\":\"00000 GET /api/orders/0 200 0 ms\\n00001 GET /api/orders/1 200 13 ms\\n ... 2000 lines\"}"}
log-reviewer:turn:2        {"record":"turn","seq":"2","body":"{\"role\":\"tool\",\"name\":\"read_file\",\"output_ref\":\"out-2-1\",\"content\":\"00000 GET /api/orders/0 200 0 ms ...\"}"}

An output is bounded at 1 MiB; larger artifacts go to your object store with a pointer record naming them. The briefing is filled in priority order (working state and pointers always, then unfinished actions, memories and the newest turns) until token_budget is reached, so a long run costs the same per call as a short one.

Evidence

Measured on Recall on LongMemEval: 267 user-fact questions, each over about 104,000 tokens of chat history. Handing the model the whole history scored 88.0%; Recall with an embedding model scored 92.9% while handing the model about a fifth of the tokens. In the resume gate, agents that restarted from a briefing used no more tokens over the whole task than agents that never stopped.

Where to go next

• Recall on LongMemEval
• recall-langgraph: the briefing as pre_model_hook
• The State API: working state, turns, outputs, pointers

Recall is typed, durable memory and state for AI agents. It stores corrections, decisions and task state as append-only records in your own object store (S3 or GCS), so an agent can be corrected without retraining and resumed after a crash, restart or pod replacement without snapshotting the process.