Home / AI & GenAI / Vector Databases
Running vector databases in production: pgvector, Pinecone, Milvus, Qdrant, Weaviate and Chroma - setup, troubleshooting, performance, scaling and backups. 50 posts.
Embeddings turn text into lists of numbers where similar meaning means nearby vectors. Learn cosine similarity, why approximate nearest-neighbour indexes exist, and build a tiny semantic search over your runbooks with Python before reaching for a vector database.
Make a RAG assistant return only what each user may read. Capture ACLs at ingestion, store them as filterable metadata, apply identity-based filters at query time, keep permissions fresh, and test for leaks before and after go-live.
A FastAPI app with four Gunicorn workers, or an ingestion job running beside the API, all open the same Chroma directory. Writes fail with "database is locked", some workers don't see new data, and the index occasionally corrupts. Find the processes sharing the directory and move to a single owner.
The Chroma Python API on one page - choosing a client, creating collections with an explicit distance, add vs upsert, query and get with metadata and document filters, includes, updates, deletes, counts and the errors you'll hit most.
Retrieval from Chroma misses obvious matches, distances look odd, and changing ef_search on an existing collection doesn't help. Check the distance space, the HNSW parameters the collection was really built with, and rebuild into a correctly configured collection with a measured recall check.
Asking for 10 nearest neighbours with a tenant or date filter returns 2, or misses the best match. Understand how each engine applies filters to approximate search, then fix it with the right filter mode, iterative scans, partial indexes or partitioning.
Vector search misses exact terms like error codes, part numbers and names; keyword search misses paraphrases. Add a tsvector column next to the embedding, run both searches as CTEs and merge them with reciprocal rank fusion, all inside PostgreSQL.
Searches fail with "collection not loaded" after a restart or deploy, loads hang at 90%, or query nodes run out of memory as collections are added. Understand Milvus load state, size memory for loaded collections, and use release, partitions, replicas and mmap to stay within budget.
Back up Milvus collections to object storage with the milvus-backup tool, restore them under new names to verify before switching, move data between clusters, and run the whole thing as a scheduled, monitored job.
After deleting or upserting millions of entities, Milvus row counts look wrong, object storage keeps growing, memory doesn't drop and searches get slower. Understand how Milvus deletes work, check segment and compaction state, and get storage and latency back under control.
Which Milvus index to choose for your data size, memory budget and recall target, how to create it with pymilvus, which search parameter controls recall for each one, and the commands to inspect and rebuild indexes.
Milvus pods sit in CrashLoopBackOff, the standalone container exits after a minute, or clients get "connection refused" on 19530. Most of the time Milvus itself is fine and one of its dependencies isn't. Check each dependency in order and fix the usual causes.
Scrape Qdrant's /metrics endpoint, add the collection and cluster checks that metrics don't cover, wire Kubernetes probes correctly, and set a short list of alerts for latency, errors, memory, disk, optimizer state and replica health.
The RAG prototype uses chromadb.PersistentClient inside the app, and now three app instances need the same data. Run Chroma as a server in Docker with persistent storage, connect with HttpClient, put authentication and TLS in front of it, add health checks and backups, and know when to move to something bigger.
Serving many customers or departments from one Pinecone project needs strict isolation, predictable cost and easy offboarding. Compare namespaces, metadata filters and separate indexes, then implement namespace-per-tenant with enforced routing, offboarding and monitoring.
One page for day-to-day pgvector work - column types and their index limits, which operator goes with which operator class, HNSW and IVFFlat syntax, query-time settings and the catalog queries to check what you have.
An HNSW build on 10 million embeddings runs all night, or slows to a crawl halfway with a "graph no longer fits into maintenance_work_mem" notice. Size memory for the graph, use parallel builds, watch progress, and plan loads so the build happens once.
A document store that re-embeds or deletes chunks every day ends up with an HNSW index twice the expected size, autovacuum running for hours on it and slower searches. Measure the bloat, reindex before vacuum, and change the write pattern that causes it.
Similarity search takes seconds and EXPLAIN shows a Seq Scan and a Sort even though an HNSW index exists. Work through the seven usual causes - wrong operator, ORDER BY expression, DESC order, missing LIMIT, untyped parameters, filters and planner costs - and fix each.
The Pinecone operations you use every day on one page - connecting by host, upsert, query with metadata filters and every filter operator, fetch, partial updates, deletes, stats and the metadata rules that cause most filter bugs.
A user uploads a document and asks about it immediately, but the assistant can't find it. Tests that upsert then query fail randomly. Understand Pinecone's eventual consistency, tell real ingestion failures apart from freshness lag, and make the application and tests behave correctly.
Someone ran delete_all on the production namespace, or a bad re-embed overwrote good vectors. Take backups before risky changes, restore a backup into a new index, verify it and point the application at it - with the commands and checks for each step.
Loading a few million embeddings into Pinecone takes all day, requests fail with 429 or "request size exceeds limit", and retries make it worse. Right-size batches, run upserts in parallel within the limits, retry with backoff, and use bulk import from object storage for big loads.
The Qdrant calls you need day to day - create and inspect collections, upsert and delete points, filter syntax, the Query API, scroll, count, aliases, snapshots and health endpoints - side by side in curl and Python.
After a large upload the Qdrant collection shows yellow for hours, or grey after a restart, searches are slow and indexed_vectors_count stays far below points_count. Read the status correctly, trigger pending optimisations, and upload large datasets without making the optimizer do the work twice.
Unfiltered Qdrant searches take a few milliseconds but the same search with a tenant or category filter takes hundreds, or returns fewer results than expected. Add the right payload indexes, mark tenant fields, and understand how Qdrant plans filtered searches.
A single Qdrant node is running out of memory or is a single point of failure. Turn it into a cluster, choose shard and replica counts you won't regret, balance data onto new nodes by moving and replicating shards, and handle a failed peer.
Weaviate's hybrid query blends BM25 keyword search and vector search, but the defaults rarely fit every corpus. Build an evaluation set, then tune alpha, the fusion method, which properties BM25 searches, tokenization and the BM25 parameters - one change at a time.
A routine pip install --upgrade moved Chroma across a major version and the app now fails to open its data or talks to the server with the wrong API. Plan Chroma upgrades like database upgrades - pin versions, back up, test on a copy, upgrade client and server together, and keep an export-import path when the on-disk format changes.
Configure a Weaviate backup module, take full or per-collection backups, check their status, restore a collection after an accidental delete, and schedule backups with monitoring - plus the restrictions that trip people up during a real restore.
An import of 2 million chunks "finishes" but the collection holds 1.7 million. Weaviate batches report errors per object, and with text2vec modules most failures come from the embedding provider - rate limits, timeouts and token limits. Find the failed objects, fix the cause, and re-import safely.
Weaviate's native multi-tenancy gives every customer its own shard inside one collection. Set it up, create and manage tenants, deactivate or offload idle tenants to save memory, and fix the common "tenant not found" and "tenant not active" errors.
In a three-node Weaviate cluster one node restarts and writes start failing, or different requests return different results. Understand replication factor and consistency levels, choose settings for reads and writes, and handle node failures, repairs and rolling upgrades.
Vectors from different embedding models can't be mixed, so when an embedding model retires (Cohere embed v3 on Foundry is deprecated in November 2026 and retires in February 2027) every document must be re-embedded. Build a second index in parallel, compare retrieval quality on a test set, then switch the query path in one step.
Poor RAG answers often come from poor retrieval. In Azure AI Search, combine keyword and vector search, add the semantic ranker, filter with metadata and boost fresh or authoritative content, then measure each change against a test set.
The vector store affects retrieval quality, operations effort and cost for years. Decide by scale, the need for hybrid keyword search and filters, where your data already lives, security and networking requirements, and who will operate it. Start simple and move only when evidence says you've outgrown it.
The embedding model decides what retrieval can find, and changing it later means re-embedding everything. Evaluate candidates on your own queries and documents, check language and domain coverage, weigh dimensions and cost against quality, consider hosting constraints, and plan versioning from the start.
Vector indexes grow with document count times dimensions times bytes per value, and memory is the main cost. Use shorter embeddings where the model supports it, quantize vectors to int8 or binary, rescore top candidates with full-precision vectors, and measure recall so savings don't quietly hurt answer quality.
After a re-index, pipeline change or data refresh, the assistant says "I don't know" far more often or cites the wrong documents. Compare retrieval before and after, find the change (chunking, embeddings, filters, missing documents), roll back with an index alias, and add a retrieval gate to the pipeline.
Qdrant, Milvus or Weaviate pods restart with OOMKilled after the collection grew or a bulk load ran. Get search back by adding memory or moving vectors to disk, slow the ingestion down, then size memory properly with quantization and on-disk indexes.
Search throws dimension errors, or still works but returns irrelevant results, after someone changed the embedding deployment, model version or dimensions setting. Find which side changed, restore matching query and document embeddings, and lock the embedding config to the index.
Small and medium RAG workloads often run a separate vector database next to a PostgreSQL that already holds the metadata. Moving vectors into pgvector removes a system to operate and enables joins and transactional updates. Check the fit, design the table and HNSW index, bulk load, verify recall and switch reads.
Upgrade an intranet, help-centre or ticket search from BM25 keyword matching to hybrid search. Add vector fields, run keyword and vector queries together with rank fusion, rerank the top results, and prove the gain with a relevance test set before switching.
Why vectors from different embedding models can't be mixed, and a blue-green runbook to re-embed, evaluate and switch a production vector index safely.
Build a private RAG proof of concept on one VM, with a local model served by Ollama and an embedding model, document chunks in PostgreSQL with pgvector, and a small Python script for ingesting and answering questions. Ends with what to change before production.
Run a single-node Qdrant vector database properly. Covers Docker setup with persistent storage, API keys and TLS, collection design with HNSW and quantisation, payload indexes for filters, memory sizing, snapshots and monitoring.
Make vector search survive node and zone failures. Covers the HA model of Qdrant, Milvus, Weaviate, pgvector, OpenSearch and Azure AI Search, how to size replicas and shards, write consistency settings and what to test.
Choose a DR strategy for vector indexes by comparing rebuild time, snapshot restore and continuous replication. Includes how to estimate rebuild time from embedding throughput, snapshot commands for common engines, and keeping the embedding model version consistent.
Size memory for a vector index, choose M, ef_construction and ef_search, measure recall against exact search, and plan for deletes and re-embedding.
What to monitor and version in a retrieval-augmented generation system, from ingestion freshness and embedding drift to retrieval recall, groundedness and prompt injection.
No posts in this category yet.