polign_db · Apache 2.0 · Go · Object Storage Based

Open-source memory infrastructure for AI agents.

A vector + hybrid search database that lives in your own object storage. Your bucket, your account, your VPC. A memory an agent writes on one step is durable, ordered, and readable on the next; the serving nodes are stateless caches you can add or remove at any time.

A consistency contract

In the storage-backed deployment, a write is acknowledged only once it's durable in your bucket's log — in one order per collection, and readable by the very next query, even cold. How each guarantee works →

Your bucket is the database

Data, indexes, and the write log live in a bucket you own — AWS S3, GCS, Azure Blob, MinIO, R2 — under your IAM policies and encryption keys. Nothing leaves your account, and leaving costs nothing: your data is already yours.

Search three ways

By meaning, by keyword (BM25), or both fused in one query — answered straight from object storage when data isn't hot, so you don't pay RAM prices for data you rarely touch.

Disposable nodes

Serving nodes are stateless caches: kill any of them and lose nothing, add more when traffic spikes, tear them all down when it stops. A collection costs nothing while it sleeps.

Start on a laptop, graduate to a cluster

Day one is one small binary — no Docker, no config, running in seconds. When you outgrow one machine, the same data model moves to nodes pointed at your bucket. Go and Python clients →

Pick your speed–memory trade-off

Three interchangeable index types: fastest-in-RAM, a balanced default, or compressed to fit ~32× more vectors in the same memory. Switching is automatic and needs no downtime — rebuilt indexes are swapped in atomically.

Quick start

Run a node, put a vector, and search it — collections are auto-created from the first vector's dimension, over plain HTTP, the Go client, or the Python SDK. When you want your data to survive anything, add one flag: -store points the node at your bucket and it takes care of the rest. Full setup, including the Go and Python clients and the other ways to run it, is in the developer guide.

# run a node (HTTP + gRPC); in-memory, zero config
go run ./cmd/server

# durable? one flag — your bucket becomes the source of truth
# go run -tags cloud ./cmd/server -store s3://my-bkt/polign

# put a vector — the collection is auto-created
curl -X PUT localhost:23000/v1/collections/docs/vectors/a \
  -d '{"values":[1,0,0],"metadata":{"label":"first"}}'

# search by nearest neighbour
curl -X POST localhost:23000/v1/collections/docs/query \
  -d '{"values":[0.9,0.1,0],"k":5}'

{"hits":[{"id":"a","distance":0.02,"metadata":{"label":"first"}}]}

Feature set

The full feature set ships in one Apache 2.0 codebase.

Benchmarks

Every number below comes from the benchmark harness in the repo (bench/ann), run on standard EC2 instances against local disk, MinIO, and same-region Amazon S3. Accuracy is recall@10 — how many of the ten true nearest neighbours each search returns. Rerun any of it yourself from the benchmarks page.

~$3.20
per million searches straight from S3 — no index in RAM, no idle servers
~$0.01
per million durable writes through the broker-free S3 log
2,860/s
in-memory searches at 96.5% recall — 1M vectors, 128-dim
32×
more vectors in the same RAM with the compressed index, 97.4% recall
ScenarioMeasured result
Search from memory — 1M vectors, 128-dim (SIFT1M) 2,860/s @ 96.5% recall · 950/s @ 99.4%
Search from memory — 100M vectors, 128-dim (16-vCPU machine) 869/s @ 95.5% recall
Cold search, no index in RAM — same-machine store 18 ms typical · 27 ms p99 @ 98% recall
Cold search, no index in RAM — Amazon S3, same region ~144 ms typical · ~354 ms p99
Cold search at scale — 100M vectors, straight from S3 91.6% recall @ 385 ms typical
Writes through the S3 log — durable before acknowledged ~6,200/s per log @ ~161 ms typical · ~286 ms p99
Compressed index (IVF-PQ) — 100M vectors, 16× smaller 98.8% recall with re-check · ~300/s

The comparisons with Qdrant, Weaviate, Milvus, and turbopuffer — including where they are faster — are spelled out on the benchmarks page, with the multipliers stated plainly.

Point it at your bucket.

Apache 2.0 — install the server and bring your own S3.