Introduction
polign_db is a small vector database in Go. Point it at a bucket and its serving nodes are stateless. The system of record lives in object storage, so you can spin them up, tear them down, and scale them out freely.
You store vectors (embeddings) in named collections, then query by nearest neighbour, by keyword (BM25), or both fused in one query. Talk to it over gRPC or HTTP/JSON from the Go or Python client. The core engine uses only the Go standard library; external dependencies enter only with the optional cloud and transport layers.
In one sentence
A vector database where the bucket is the database, and nodes are just caches you can add, kill, or scale to zero.
What makes it different
- Object storage is the source of truth. Writes append to a write-ahead log that lives in the bucket itself (no broker), and a persistor compacts them into immutable segments on S3, GCS, or Azure Blob. Boot a node and it restores from the bucket; kill it and lose nothing.
- A documented consistency contract. A write is durable in the log before it is acknowledged, every collection has a total write order, and reads, warm or cold, see acknowledged writes by also reading the newest, not-yet-compacted entries. The one freshness lag is documented: BM25 text becomes keyword-searchable at the next segment flush.
- Search straight from the bucket. A segment index makes object storage directly queryable for vector k-NN, BM25, or hybrid, served "cold" with no full in-memory index, while hot data is promoted into RAM automatically.
- Three ANN backends. Plain HNSW, the default hybrid IVF-over-HNSW, or compressed IVF-PQ — ~32× smaller in RAM, and on the cold path it scans compact codes and re-ranks against exact vectors, cutting the bytes a cold query fetches.
- Many small collections, cheaply. Named collections with typed metadata (strings, numbers, booleans) and metadata-filtered search; a collection that isn't being queried costs only its object storage.
How a write becomes searchable
put → write log (in the bucket) → persistor builds segments → queryable hot (RAM) or cold (bucket)
- Put a vector into a collection over gRPC or HTTP; it is acknowledged once appended to the write log.
- Persist. The persistor drains the log into immutable, compressed segments in object storage.
- Serve. Nodes answer queries from RAM when the data is hot, or directly from the bucket when it is cold; heat decides the tier.
Next steps
- Get started to install a server, add your data, and make your first queries.
- Operate in production covers services, scaling, and per-collection buckets.
- How it works explains the design and its guarantees.
- Comparison and benchmarks show how it stacks up against other vector databases.