LlamaIndex

Use polign_db as a LlamaIndex vector store. llama-index-vector-stores-polign gives you PolignVectorStore with translated metadata filters and the keyword, hybrid, and MMR query modes, and needs nothing extra on the server.

Before you start

The package talks to a running polign_db server. For a first try, start one on your machine:

polign-server -store fs:/var/lib/polign

It listens on http://localhost:23000. Get started covers installing the binary and pointing it at a bucket.

Install and use

pip install llama-index-vector-stores-polign

from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.vector_stores.polign import PolignVectorStore

store = PolignVectorStore(collection_name="docs", url="http://localhost:23000")
index = VectorStoreIndex.from_documents(
    documents, storage_context=StorageContext.from_defaults(vector_store=store)
)

index.as_retriever(similarity_top_k=5).retrieve("what purrs?")
index.delete_ref_doc("document-id")     # removes every chunk of that document

# reopen later without re-indexing
index = VectorStoreIndex.from_vector_store(store)

Metadata filters

LlamaIndex MetadataFilters are translated for you. Supported operators are EQ, NE, GT, GTE, LT, LTE, IN, NIN, ANY, ALL, CONTAINS, and IS_EMPTY, combined with AND, OR, and NOT. TEXT_MATCH is not supported.

Query modes

Query mode	What runs
`DEFAULT`	Vector search.
`TEXT_SEARCH`, `SPARSE`	BM25 keyword search over the node text.
`HYBRID`	Vector and BM25 search combined on the server. Set `alpha` to weight the vector side; leave it unset for rank fusion.
`MMR`	Nearest nodes re-ranked for variety.

Keyword search needs a server started with -store. A new document becomes searchable by keyword once the server has written it to the bucket, about half a minute with default settings. Vector search sees it right away.

What gets stored

The node id becomes the record id, so adding the same id again replaces it.
The text is stored in the metadata key text. That is the field the server's keyword index reads, so keyword and hybrid search work without setup.
Metadata that is a string, number, boolean, or a flat list of those is stored as is and can be filtered on.
Nested objects and empty values are saved as text and restored when you read them back. They cannot be filtered on.
A collection is created by the first write and takes its size from the first vector.

Limits

Removing a whole collection needs the server's collection API (-byo-store). Without it, call clear(), which removes every record.
MMR fetches the candidate vectors in one extra request, because search results do not include vectors.
The async methods run the client on a worker thread.

Source and the full API are in the llama-index-vector-stores-polign README.

Source & license

llama-index-vector-stores-polign is open source under the Apache License 2.0 and lives next to the Python client in github.com/Polign/polign. Bug reports and pull requests go there.