Transformation: from keyword search to hybrid search with a semantic reranker

#Why hybrid, not vector-only

Hybrid search runs both and merges results (Reciprocal Rank Fusion). A reranker (cross-encoder or semantic ranker) then reorders the top results by reading query and passage together.

#1. Baseline first

Build a relevance test set: 100–300 real queries from search logs, each with the documents that should appear. Measure the current keyword search:

def recall_at_k(search_fn, qrels, k=10):
    return sum(any(d in qrels[q] for d in search_fn(q)[:k]) for q in qrels) / len(qrels)

Also record nDCG@10 or MRR if you have graded judgements.

#2. Add vector fields

Chunk long documents (search over whole 40-page PDFs works poorly with vectors) and embed each chunk. Azure AI Search index excerpt:

{
  "name": "contentVector",
  "type": "Collection(Edm.Single)",
  "dimensions": 1024,
  "vectorSearchProfile": "hnsw-profile",
  "searchable": true,
  "retrievable": false
}

Use integrated vectorization (indexer skillset calls the embedding model) or embed in your own pipeline.

#3. Query both

Azure AI Search REST example:

POST /indexes/kb/docs/search?api-version=2024-07-01
{
  "search": "can't install updates permission problem",
  "vectorQueries": [{ "kind": "text", "text": "can't install updates permission problem", "fields": "contentVector", "k": 50 }],
  "queryType": "semantic",
  "semanticConfiguration": "default",
  "top": 10,
  "select": "title,url,chunk"
}

kind: text requires a vectorizer on the index; otherwise embed the query yourself and send kind: vector. queryType: semantic adds the semantic reranker on top of hybrid results.

PostgreSQL version (RRF in SQL):

WITH kw AS (
  SELECT id, row_number() OVER (ORDER BY ts_rank_cd(tsv, q) DESC) AS r
  FROM docs, plainto_tsquery('english', $1) q WHERE tsv @@ q LIMIT 50),
vec AS (
  SELECT id, row_number() OVER (ORDER BY embedding <=> $2) AS r FROM docs ORDER BY embedding <=> $2 LIMIT 50)
SELECT id, sum(1.0 / (60 + r)) AS score
FROM (SELECT * FROM kw UNION ALL SELECT * FROM vec) x
GROUP BY id ORDER BY score DESC LIMIT 10;

#4. Compare and tune

Tune chunk size, k for the vector leg, field weights/scoring profiles, and whether titles are embedded with chunks. Change one thing at a time.

#5. Roll out

• A/B test with real users: click-through on top 3, zero-result rate, query reformulation rate.
• Watch latency: reranking adds tens to hundreds of milliseconds; rerank only the top 50.
• Costs: embedding at ingestion and per query, reranker usage.

Tip

Keep exact-match boosts for IDs and error codes (a filter or scoring profile on a keyword field). Users searching for "KB5034441" expect that article first, every time.

#Checklist

• Relevance test set and baseline numbers
• Chunked content with vector fields
• Hybrid + reranker compared against baseline
• A/B test with user metrics
• Latency and cost within budget