An agent’s memory should outlive its confidential VM

A confidential VM can protect an agent while it runs. What happens to everything the agent memory when that VM goes away? How would an agent securely and safely access to their RAG without losing the context of their previous interactions?

In his post on protecting agentic AI workloads, Mike Bursell makes the case for protecting an agent's identity, capabilities, code and data with hardware isolation and attestation. I am extending that argument to the state the agent leaves behind.

An agent accumulates context over its lifetime and that history can be more sensitive than any individual prompt. It also needs to survive a restart, a machine failure or an upgrade to a new application image. With polign's core concept to separate memory from compute, the context storage can be securely stored in a bucket that is controlled by the owner.

Where does the memory go?

Keeping memory on the CVM's local disk ties its lifetime to that machine. Ephemeral disk disappears with the instance. A persistent volume survives, but its encryption, key access and recovery policy then become part of the design, and running the application in a TEE does not protect a volume after it leaves that environment.

Sending memory to a hosted database adds another party that processes the data. Unless that service runs inside its own verified confidential boundary, the agent's TEE protection ends before the database handles the write or the search. TLS protects the connection. It does not hide the records from the service receiving them.

So "the agent runs in a confidential environment" leaves a real question open: who can read its memory, now and after the current machine is gone?

polign_db on Phala Cloud

Phala Cloud runs ordinary Docker Compose applications inside Intel TDX confidential VMs. The parts that matter for memory are how it handles secrets and identity: environment variables are entered encrypted, decrypted only inside the CVM after attestation, and every deployment can produce a hardware quote that a third party can verify against the compose file that is running.

We published polign_db as a prebuilt template on that platform. The template runs the search engine in the same CVM as the agent, keeps durable memory in an S3 or GCS bucket the owner controls, and encrypts every object before it leaves the process. The CVM becomes a cache in front of the bucket. Delete it, upgrade it or move it, and a replacement with the same store and keyring reopens the same collections.

How it works

The confidential VM

Phala Cloud deploys the template's compose file into a CVM built on dstack. Two containers run there. The first downloads the polign_db release from dl.polign.com on first start, checks its SHA-256 against the release's published checksums, and keeps the binary on the volume so restarts do not download again. The second is a small Caddy proxy that publishes port 23000 and rejects any request without the bearer token, except an open health check. The gRPC port stays on the container's loopback address and is not reachable from outside. TLS terminates at the Phala gateway.

The agent talks to polign_db over the container network or the public endpoint. Either way, vectors and metadata are decrypted and searched in memory the hardware protects. Anything the agent sends to an external model or tool leaves that boundary, as it would in any design.

The bucket

The store defaults to the CVM's volume, which is fine for trying it and wrong for production. Setting POLIGN_STORE to an s3:// or gcs:// location makes the bucket the database. polign_db runs in its storage-backed mode: acknowledged writes go to a durable log in the bucket, sequenced with S3 conditional writes or GCS generation preconditions, and the local disk is a bounded cache. A cold start recovers state from the bucket. It does not need the previous machine's RAM or disk.

I worked on S3 at AWS and later on Google Cloud Storage. A bucket is the right boundary here because enterprises already know how to run one: scoped IAM, CloudTrail or audit logs, versioning, replication, retention and region choice. Those controls keep working when the compute provider or the software changes. Backups are a copy of ciphertext, and versioning alone is a usable backup.

The keys

With bucket-side encryption (SSE-S3, SSE-KMS or Google's default), the storage service encrypts uploads and decrypts downloads, so it handles plaintext while serving authorized requests. The template never uses that path. polign_db reads a keyring from POLIGN_STORE_ENCRYPTION_KEYS and encrypts every object and every disk-cache entry itself, under AES-256-GCM, before the bytes leave the process. Framed encryption means a query fetches and decrypts only the byte ranges it needs rather than whole objects. The encryption documentation covers the format.

The keyring is required. The container refuses to start with an empty one rather than run an unencrypted store. On first write polign_db drops a small .encryption marker in the store, and any later process that opens the store without the keys stops at startup instead of reading garbage or writing plaintext. Rotation is a second line in the variable and a redeploy: new writes use the highest key id, and every id listed still decrypts.

Phala Cloud's role is delivery. The keyring goes in as an encrypted environment variable and is decrypted inside the CVM after attestation, so it exists in plaintext only in protected memory. polign_db never calls a key management service, and there is no Polign-hosted coordinator between the process and the bucket.

The bucket credential

Some cloud credential has to enter the CVM to reach the bucket. A Phala CVM has no instance role, and neither AWS nor Google accepts a TDX quote as an identity today. The template keeps that credential as weak as it can be. For S3, the access key belongs to a bootstrap IAM user whose only permission is sts:AssumeRole on one role that holds the bucket policy; polign_db assumes that role with an external id and signs every request with short-lived credentials. To cut access, edit the role's trust policy and nothing needs rotating. For GCS, a service-account key arrives base64-encoded, is decoded into RAM-backed tmpfs at start, and is dropped from the environment before the server runs.

Whatever that credential can reach is ciphertext. A leaked credential can delete or corrupt memory. It cannot read it without the keyring, which lives in a different place.

Replacing the VM

This is the property the introduction asked for. Start a new CVM with the same POLIGN_STORE, the same keyring and the same bucket credential, and it serves the same collections. Upgrades are a change to POLIGN_VERSION and a redeploy; the data does not move. A machine failure costs a cold start, which the agent memory cost post measures on EC2.

The durable dependency is therefore not the machine but the policy for who receives the keyring next. On Phala Cloud that policy is expressed by what you put into the encrypted environment of a new deployment and by the attestation you check before trusting it. The verification guide shows how to confirm the hardware evidence, the compose hash and the event log for a running application, so an owner can check that the code holding their keys is the code they approved.

Configuring it

The template README has the full IAM policies. The short version follows.

Step 1: make the two secrets

Keep a copy of the keyring outside Phala Cloud. Without it the data cannot be read by anyone, including you.

printf '1=%s\n' "$(openssl rand -base64 32)"   # POLIGN_STORE_ENCRYPTION_KEYS
openssl rand -hex 32                            # BEARER_TOKEN

Step 2: point it at your bucket

Create the bucket with public access blocked, a role with the bucket policy, and a bootstrap user that can only assume that role. Then set these as encrypted variables. The AWS region is the bucket's, not the Phala region the CVM runs in.

POLIGN_STORE=s3://BUCKET/polign
POLIGN_STORE_REGION=us-west-2
POLIGN_STORE_ROLE_ARN=arn:aws:iam::ACCOUNT:role/polign-phala
POLIGN_STORE_EXTERNAL_ID=EXTERNAL_ID
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...

For GCS, set POLIGN_STORE=gcs://BUCKET/polign and GOOGLE_APPLICATION_CREDENTIALS_BASE64 to the encoded key of a service account that holds roles/storage.objectAdmin on that one bucket.

Step 3: deploy and verify

Create a CVM from the template, paste the variables as encrypted environment, expose port 23000 and deploy. The first start downloads about 85 MB. Then check the endpoint, the log and the attestation.

curl https://APP/healthz                                  # ok, no token needed
curl -i https://APP/v1/collections/docs/vectors/a         # 401 without the token

curl -X PUT https://APP/v1/collections/docs/vectors/a \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"values":[1,0,0],"metadata":{"label":"first"}}'

curl -X POST https://APP/v1/collections/docs/query \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"values":[0.9,0.1,0],"k":5}'

The server log should show two lines: one naming the role it assumes for every bucket request, and store encryption: on with the key id in use. After the first write the bucket contains an .encryption object and a .wal/ prefix, none of it readable without the keyring. The Attestations tab on the Phala dashboard shows the TDX quote and the compose hash for the deployment.

What we ran before publishing

We deployed the template to a tdx.small CVM in Phala's US-West region, first with the volume store and then updated in place to the S3 role configuration. Both passed the health check, returned 401 without the token, and accepted writes and queries with it. The S3 run served a vector that had been written into the bucket from a different machine with the same keyring, and wrote one back. The log confirmed the assumed-role principal and encryption on, and the attestation quote was present on the dashboard.

That test checked deployment, data access and continuity across machines. It did not measure search performance under TDX. Our published benchmarks were run on EC2 and say nothing about confidential-computing overhead.

Limitations

• The bucket still sees object names, sizes and access patterns, including the plaintext marker that identifies an encrypted store.
• Authenticated encryption detects altered ciphertext. It does not prove that an object is the latest version or stop someone with storage permissions from deleting it. Versioning and a separate backup account are the answer to that.
• Key release is not bound to a measured image by the cloud provider. S3 and GCS do not verify a TDX quote, so the owner decides which deployment receives the keyring by what they put into its encrypted environment, and checks attestation themselves.
• One shared bearer token is the only authentication in the template. polign_db has per-client API keys if you need them; the start command has to be adapted.
• Encryption and hardware isolation do not decide which records an agent may retrieve or which retrieved instructions it should trust. Those stay with the application.

Availability and pricing

The template is live in the Phala Cloud catalog. It defaults to a 1 vCPU, 2 GB, 20 GB disk tdx.small, which at the time of writing is billed at about $0.06 per hour, with tdx.medium and tdx.large at roughly $0.12 and $0.23. The bucket is billed by your cloud account at normal object-storage rates. polign_db itself is free to self-host, and the template downloads the same release binaries published on GitHub.

If you are running agents in TEEs and have worked through the memory question differently, I would like to hear how: anup@polign.com.

Anup Talwalkar is the founder of Polign. He previously worked on S3 at AWS and on Google Cloud Storage.