Testing Agent Memory on microVMs
Independent agent memory that survives a microVM and reopens on the next one.
In my last post about agent memory, I wrote about how agents are moving to the edge, and how memory needs to move with them without picking up any side effects along the way. I wanted to take that a step further and show how Polign handles a cold start on micro VMs, and how it can be deployed at short notice.
One of the practical use case for this setup is when a small agent needs to recover quickly from its previous state, or an agent that is running in a serverless environment where the execution state is temporary. An agent doesn't have to be running idle, but can only wake up when needed and retain all of its previous context.
A quick background on Polign, in case you are reading about it for the first time: Polign is a lightweight, stateless agent memory and vector database that keeps its durable state in object storage. With an S3-backed store, a write goes into the bucket's write-ahead log (WAL) before it is acknowledged. On a fresh server start, the Polign server reads the tail of the WAL along with the objects it needs from the bucket.
I ran a test on a small set of objects across six cold runs on AWS Lambda. This is the sequence I used:
- Write the preference in environment A.
- Read it from a fresh environment B.
- Have ten fresh readers query the same unchanged data in S3.
The setup for this exercise
- ARM64 Lambda custom runtime in
us-west-1. - 512 MiB of memory and an S3 bucket in the same region, with the segment cache capped at 64 MiB.
- Polign server zipped bundle (14.3 MiB).
- Invocation requests to Polign's local HTTP endpoint.
- Background persistor turned off, so nothing in the log is published as a segment and a fresh process has to replay the WAL entries itself.
Command to run
GOMEMLIMIT=384MiB ./polign-server \
-store s3://YOUR-BUCKET/microvm-test \
-persist=false \
-maintain 0 \
-http 127.0.0.1:23000 \
-grpc 127.0.0.1:23001 \
-hot-max 0 \
-disk-cache-bytes 0 \
-segment-cache-bytes 67108864 \
-max-resident-searchers 64The setup ensures that it's a cold start for the Polign server. The bucket still contained only the WAL and schema objects. The new process had recovered the record from the S3 log.
I turned off the persistor for this test to ensure that nothing is published, so every entry stays unconfirmed in the log and each fresh process replays all of it before it answers anything. This is the state a store is in after write, and even though with persistor running, the records will be small, the majority of the delay on reboot involves refreshing the memory from the log. The test now identifies the tail logs role in restarting the agents workflow.
Polign relies heavily on S3's strong read-after-write consistency to find the logs. A fresh Polign node captures the highest visible log sequence in each partition and replays through those positions before accepting queries.
Tracing the timing profiles
The experiment ran over six cold starts to identify the initialization time and queries with warm invocations. All ten fresh reader environments found the preference written earlier with environment A. The stored record stayed unchanged throughout these reads.
The results in this table are for one stored record. This measures initialization and a small query on Lambda.
| Run | Result | Notes |
|---|---|---|
| Cold initialization | 645 ms p50 562 to 677 ms range |
Six forced replacements; AWS Init Duration, including runtime and Polign startup. |
| Polign healthy | 596 ms p50 527 to 630 ms range |
Process start to /healthz returning 200, including startup WAL replay. |
| First query after replacement | 23.0 ms p50 19.6 to 31.5 ms range |
Six runs, querying the same record after startup replayed the retained S3 WAL. |
| Cold invocation, including init | 669 ms p50 587 to 711 ms range |
AWS billed duration for the same six cold invocations; excludes the caller's network round trip. |
| 100 warm queries | 23.7 ms p50 30.0 ms p95 43.4 ms p99 |
One invocation, sequential local HTTP requests. Minimum 19.9 ms; maximum 57.0 ms. |
| Memory | 45.5 MiB p50 RSS 71 to 74 MB AWS peak |
Runtime adapter and Polign together; six cold runs on a 512 MiB allocation. |
| 10 fresh cold readers | 10/10 succeeded 22.2 ms query p50 30.4 ms max |
Ten distinct runtime instance IDs, each querying the previously written record. Polign-ready p50 was 570 ms. |
For this record, cold and warm query times are close. The 645 ms went into initializing the runtime and replaying the WAL entries, and with one unconfirmed record the replay part of that is small. The useful reading is the shape of the path rather than a ceiling for it. Add entries that the persistor has not folded into segments yet and the replay grows with them, which is the lag this configuration is meant to expose.
This experiment is to show the recovery of an agent's memory on a fresh start. In a serverless environment, the agents can be invoked on demand and continue the workflow from where they left off.
How other databases would handle this
Polign is one answer to a machine that comes and goes. It is worth looking at what the other options do in the same spot, because the choice usually comes down to where the data sits once the process is gone. I have not run any of these on Lambda, so nothing below is a measurement. It is what each design has to do when a new environment starts with an empty disk, based on how these systems are built.
| Approach | What a fresh environment does first | Works well when | What it costs |
|---|---|---|---|
| An index file in memory FAISS, hnswlib |
Download the whole index and load it before the first query. | The collection is small, changes rarely, and queries have to be as fast as possible. | Startup grows with the file, and there is no write path of its own. |
| An embedded database on local disk SQLite with sqlite-vec, Chroma |
Restore the database files to /tmp, or mount a shared file system. |
One process owns the data and you want a plain file you can copy. | The whole file travels on every cold start, and two environments cannot safely write it at once. |
| An engine that reads object storage LanceDB |
Open the table in the bucket and read only the parts a query touches. | You want the data in your own bucket with no server to keep alive. | Concurrent writers need an agreed commit path, and many small writes slow the open until compaction runs. |
| A server that stays up Postgres with pgvector, Qdrant, Weaviate, Milvus |
Nothing. It connects over the network and queries. | Something is already running and you want mature filtering, joins, and index tuning. | You pay for it while the agent is idle, and short-lived functions need a connection pooler. |
| A hosted vector API Pinecone, turbopuffer |
Nothing. It makes one HTTPS call. | You want no operations at all and do not mind where the vectors live. | The memory sits in someone else's account, and every read is a billed network round trip. |
| Polign in this test | Replay the retained tail of the WAL in your bucket, then serve queries. | The compute is temporary and the memory has to be in your bucket and correct after a restart. | Every cold environment waits for that replay, and the wait grows with the amount of log retained. |
These are design comparisons rather than benchmark results. Only the Polign row was measured in this exercise.
An index file loaded into memory
FAISS and hnswlib are libraries rather than databases. You build an index, save it as a file,
and load it back. On Lambda that file has to ship in the bundle or come down from S3 into
/tmp before the first query, and the process holds it in memory afterwards. Once
it is loaded, nothing here answers faster, and the dependency is small enough to reason about.
What you do not get is durability. There is no log, no acknowledged write, and no agreed way for two environments to decide which version of the file is current. New records mean building a new index and publishing it. For a corpus you rebuild on a schedule, that is a fair trade. For an agent writing to its own memory during a turn, you end up building the missing durable write path yourself.
An embedded database on local disk
SQLite with the sqlite-vec extension, or Chroma in its persistent mode, gives you a real database in a directory. It is the easiest option to develop against, and for a small collection the search is exact rather than approximate.
On Lambda, /tmp starts empty and goes away with the environment, so the directory
has to be restored from S3 at startup or kept on a mounted file system, which brings a VPC and
EFS into the picture. Copying is fine while the database is small and gets slower as it grows,
because the whole file moves on every cold start. Writes are the harder limit: a database file
in a bucket has one writer, and two environments cannot safely write it at the same time.
Replication tools exist for exactly this, and they solve it by making sure only one process
writes.
An engine that reads object storage
LanceDB is the closest in shape to what I tested. The table lives in the bucket, the process reads the byte ranges a query needs, and no server sits between the two. A cold start does not have to download the whole dataset, which is the main advantage over copying a file around. If you are weighing up Polign for this kind of deployment, this is the comparison I would run alongside it.
It also inherits the same problems I have. Writers in separate environments need an agreed way to commit, and a store that has taken many small writes opens slowly until compaction catches up. Object storage removes the server, not the bookkeeping.
A database server that stays up
pgvector on Postgres is the least exotic option and often the right one. There is nothing to recover on a cold start because the database never went away. The function connects and queries, and a warm Postgres answers a small query in a few milliseconds, without the initialization this test spent getting ready. Filters, joins, transactions, and backups are all things you already know how to run.
The cost is that something is always up and always billed, including while the agent is idle. Lambda's habit of opening many short-lived connections is a known problem for Postgres, so a pooler such as RDS Proxy usually becomes part of the setup. Qdrant, Weaviate, and Milvus are the same trade with better vector features and more scale: still a service you keep alive, size, and pay for between turns. Serverless flavors can pause when idle, and then you pay a wake-up delay on the first query instead.
A hosted vector API
Pinecone and turbopuffer take the problem away rather than solving it inside your process. There is no cold start on your side, nothing to size, and no memory limit to think about. For many teams that is the least work of anything on this list.
What you give up is where the data sits and what each read costs. The vectors live in someone else's account, every query is a network round trip out of yours with a per-request charge, and local development does not behave like production. turbopuffer keeps its data in object storage too, so a namespace it has not served recently has a cold read of its own. The difference is that the cold path is theirs to manage, not yours.
Where this leaves Polign
None of this makes one option the correct one. The question worth answering first is where you want the memory to be when the process is gone. If a service you keep running is fine, pgvector is less work than anything here and answers faster, because it never had to start. If someone else's account is fine, a hosted API removes the cold path completely.
What I tested earns its place when you want both things at once: the data in your own bucket, and the compute free to disappear between turns. The startup in the table above is what that costs on a small store. It is worth measuring because it is not a fixed number. It grows with how much log a fresh process has to replay, which is the part I want to keep small.