Vector Databases: ANN Search, Filtering & RAG Storage

Q: Do I need a dedicated vector database, or can I use Postgres?

For many applications, pgvector (Postgres + vector extension) is enough — especially at small-to-medium scale, or when you want vectors beside your relational data with one system to operate. Reach for a dedicated vector database (Pinecone, Qdrant, Weaviate, Milvus) when you need very large scale, advanced ANN tuning, built-in hybrid search, or managed operations. Start simple; migrate when you hit a real limit.

Vector Databases

A vector database stores embeddings and finds the most similar ones to a query vector — fast, at scale. Where a relational database is built for exact matches and ranges, a vector database is built for nearest-neighbor search over high-dimensional vectors, which is exactly what semantic search and RAG need: "given this query's vector, return the closest stored vectors."

The key enabling idea is approximate nearest neighbor (ANN) search. Comparing a query against millions of vectors exactly is too slow, so vector databases use clever index structures that return almost the closest matches in milliseconds, trading a little recall for enormous speed.

TL;DR

Stores and searches embeddings by similarity at scale.
Uses approximate nearest-neighbor (ANN) indexes (e.g. HNSW) for speed.
Supports metadata filtering and hybrid (vector + keyword) search.
The retrieval backbone of semantic search and RAG.

Quick Example

The two operations are upsert (store vectors + metadata) and query (find nearest):

Core Concepts

Approximate nearest neighbor (ANN)

Exact nearest-neighbor search scales linearly with the number of vectors — fine for thousands, hopeless for millions. ANN indexes (most commonly HNSW, a navigable graph; also IVF, product quantization) find near-optimal matches in roughly logarithmic time. You trade a tunable bit of recall for big gains in speed and memory.

Similarity metric

The index searches by a distance metric — usually cosine similarity for text embeddings (also dot product, Euclidean). It must match what your embedding model expects.

Metadata filtering

Vectors are stored with metadata (source, date, tenant, permissions). Filtering lets you restrict search — "only this user's documents," "only docs from 2026" — which is essential for multi-tenancy and access control.

Hybrid search

Pure vector search can miss exact terms (names, IDs, error codes). Hybrid search combines semantic (vector) and keyword (BM25) scoring for the best of both.

The Options

💡 If you already run Postgres, pgvector is often the simplest start — no new infrastructure, and your vectors live beside your relational data. See PostgreSQL.

Best Practices

Match the metric to your embedding model (usually cosine) and stay consistent.
Tune the ANN index (HNSW ef/M parameters) for your recall-vs-latency target.
Store rich metadata and filter on it — for relevance and access control.
Add hybrid search when exact terms matter.
Re-embed on model change — vectors from a different model are incompatible.

Common Mistakes

Forgetting access control in retrieval

Expecting exact-match precision from ANN

FAQ

Do I need a dedicated vector database, or can I use Postgres?

For many applications, pgvector (Postgres + vector extension) is enough — especially at small-to-medium scale, or when you want vectors beside your relational data with one system to operate. Reach for a dedicated vector database (Pinecone, Qdrant, Weaviate, Milvus) when you need very large scale, advanced ANN tuning, built-in hybrid search, or managed operations. Start simple; migrate when you hit a real limit.

What is HNSW and why does it matter?

HNSW (Hierarchical Navigable Small World) is the most popular ANN index. It builds a layered graph of vectors that the search "navigates" to reach near neighbors quickly, giving millisecond queries over millions of vectors. Its parameters (M, ef_construction, ef_search) trade off index size, build time, query speed, and recall — tuning them is how you balance accuracy against latency for your workload.

How is a vector database different from a regular database?

A regular (relational) database excels at exact lookups, ranges, joins, and transactions over structured rows. A vector database is specialized for similarity search over high-dimensional embeddings — "find the closest vectors to this one" — using ANN indexes. They're complementary: you often store the source data and metadata relationally and the embeddings in a vector index (or both together via pgvector), filtering by metadata while searching by vector.

Why must I re-embed when I change embedding models?

Because each embedding model produces vectors in its own space — a vector from model A isn't comparable to one from model B, so mixing them makes similarity scores meaningless. The stored corpus and incoming queries must be embedded by the same model and version. Upgrading the embedding model therefore requires re-embedding and re-indexing your entire dataset. See Embeddings.