LiveKit Agents

A voice agent that remembers the caller. Facts are typed: each one is a predicate from a closed registry with a value, so "call me Sam" replaces the old name instead of adding a second one, and the history of the change is kept. Nothing is searched per turn and no second model runs: the caller's facts are a short list, loaded whole before the first reply.

recall-livekit connects Recall to LiveKit Agents.

Install and use

pip install recall-livekit "livekit-agents[openai,deepgram,cartesia,silero]"

from livekit.agents import AgentServer, AgentSession, JobContext, JobProcess
from recall_livekit import VOICE_REGISTRY, RecallAgent, RecallMemory

server = AgentServer()

def setup(proc: JobProcess):
    # one Recall subprocess per worker process
    proc.userdata["recall"] = RecallMemory.open(
        local_dir="./recall-data", predicates=VOICE_REGISTRY,
    )

server.setup_fnc = setup

@server.rtc_session()
async def entrypoint(ctx: JobContext):
    participant = await ctx.wait_for_participant()
    memory = ctx.proc.userdata["recall"].for_subject(participant.identity)
    session = AgentSession(stt=..., llm=..., tts=..., vad=...)
    await session.start(
        agent=RecallAgent(memory=memory, instructions="You are the Acme support line."),
        room=ctx.room,
    )

What happens on a call

• on_enter loads the caller's current facts and appends them to the instructions inside a <recall_memory> block, before the greeting.
• The model saves new facts through a remember tool whose predicate list is your registry. Recall validates the fact, supersedes the old value, and the agent rewrites its instructions so the next sentence already uses it.
• A caller with more facts than fit the block (20 by default) also gets a per-turn search, added to that turn only.
• A week later the same identity calls back and step one finds everything.

Identity and failures

The subject should be a stable, auth-derived identifier such as the participant identity your token server issued, never the room name. Reads fail open, so a slow memory store never drops a call; writes tell the model the fact was not saved. An existing Agent subclass gets the same block and tool through await attach(agent, memory).

Where the memory lives

The pip install above is the whole install. It brings the polign_db server and CLI with it, for Linux, macOS and Windows, so there is no database to download and a worker image needs nothing else. local_dir keeps the memory in that folder: the first worker process starts a server for it in the background and the others share it. Workers on several machines need one shared server instead; run polign-server where they can all reach it and pass url= and api_key= in place of local_dir.

Example and options

The example is a complete support-line agent with a Dockerfile, plus a keyword knowledge-base agent for the retrieval side. Custom predicates are a JSON file; see the recall-livekit README for the format and every option.

Source & license

recall-livekit is open source under the Apache License 2.0 and lives next to the Python client in github.com/Polign/polign. Bug reports and pull requests go there.