Filtered vector search returns too few or wrong results: pre-filtering vs post-filtering, HNSW and iterative scans

#Symptoms

• top_k = 10 with a filter like tenant_id = 'acme' or doc_type = 'policy' returns fewer than 10 results, sometimes zero.
• RAG answers miss documents that definitely match the filter and the question.
• Results are fine without the filter, or for big tenants, but poor for small tenants.

#Why it happens

Approximate nearest neighbour indexes (HNSW, IVF) search a graph or clusters for the closest vectors. Filters can be combined with that search in two ways:

The more selective the filter (a small tenant is 0.1% of vectors), the worse post-filtering gets.

#pgvector

Before pgvector 0.8, an HNSW index scan returned up to hnsw.ef_search candidates (default 40), and the WHERE clause was applied afterwards. With a selective filter, few rows survived.

#Iterative index scans (pgvector 0.8.0+)

SET hnsw.iterative_scan = relaxed_order;     -- or strict_order
SET hnsw.max_scan_tuples = 20000;            -- upper bound on work

SELECT id, title
FROM   documents
WHERE  tenant_id = 'acme'
ORDER  BY embedding <=> $1
LIMIT  10;

The scan keeps going until enough rows pass the filter or the limit is reached. relaxed_order is faster but results may be slightly out of order; re-sort in an outer query if exact ordering matters:

SELECT * FROM (
  SELECT id, title, embedding <=> $1 AS distance
  FROM documents WHERE tenant_id = 'acme'
  ORDER BY embedding <=> $1 LIMIT 40
) s ORDER BY distance LIMIT 10;

#Partial indexes or partitioning for big, stable filters

CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops) WHERE doc_type = 'policy';

For multi-tenant data, partition by tenant (or tenant group) with an index per partition, so a tenant's search only touches its own vectors.

#Very selective filters: skip the vector index

If a filter matches only a few thousand rows, an exact search over just those rows is fast and has perfect recall:

SET LOCAL enable_indexscan = off;   -- or rely on the planner choosing a B-tree on tenant_id
SELECT id FROM documents WHERE tenant_id = 'tiny-tenant' ORDER BY embedding <=> $1 LIMIT 10;

Check with EXPLAIN ANALYZE which plan the planner picks.

#Azure AI Search

Vector queries accept a filter and a filter mode:

{
  "vectorQueries": [{
    "kind": "vector",
    "vector": [0.012, -0.03, ...],
    "fields": "contentVector",
    "k": 10
  }],
  "filter": "tenantId eq 'acme' and docType eq 'policy'",
  "vectorFilterMode": "preFilter",
  "select": "id,title"
}

• preFilter (the default in current API versions) applies the filter during the vector search: you get k results if enough documents match.
• postFilter filters after retrieving k candidates: fewer results with selective filters.

Filterable fields must be marked filterable in the index. If you see too few results, check the request isn't using postFilter (some SDK samples set it) and that k is large enough before any re-ranking.

#OpenSearch, Qdrant, Milvus

Engine behaviour changes between versions; read the filtering section for your version.

#Measure, don't guess

Build a small evaluation set per filter class (big tenant, small tenant, rare doc type):

def recall_at_k(ann_ids, exact_ids, k=10):
    return len(set(ann_ids[:k]) & set(exact_ids[:k])) / k

• Exact results: brute-force distance over the filtered rows.
• ANN results: your production query.
• Track recall and "results returned < k" counts per class.

#Decision table

#Prevent it next time

• Test RAG retrieval with the filters production uses, not just unfiltered queries.
• Include tenant size in design: one shared index for thousands of tiny tenants needs filter-aware search.
• Log "returned fewer than k" as a metric.
• Re-check filter behaviour after vector store upgrades; defaults change.