Tuning Weaviate hybrid search: alpha, fusion type, BM25 settings and tokenization for better RAG retrieval

#The hybrid query

from weaviate.classes.query import HybridFusion, MetadataQuery

kb = client.collections.get("KbChunk")
res = kb.query.hybrid(
    query="ORA-01555 snapshot too old during nightly export",
    alpha=0.5,
    fusion_type=HybridFusion.RELATIVE_SCORE,
    limit=10,
    return_metadata=MetadataQuery(score=True, explain_score=True),
)
for o in res.objects:
    print(round(o.metadata.score, 3), o.properties["doc_id"])

#Step 1: build an evaluation set first

Without measurement, tuning is guesswork. Collect 100–300 real questions with the document(s) that answer them:

eval_set = [
  {"q": "ORA-01555 during export", "expected": {"kb-ora-01555"}},
  {"q": "users can't connect to VPN after password change", "expected": {"kb-vpn-cred-cache"}},
  # ...
]

def hit_rate(k=5, **params):
    hits = 0
    for e in eval_set:
        r = kb.query.hybrid(query=e["q"], limit=k, **params)
        got = {o.properties["doc_id"] for o in r.objects}
        hits += bool(got & e["expected"])
    return hits / len(eval_set)

Tag each question as keyword-heavy (codes, names, versions) or semantic (paraphrased problem descriptions), and report both.

#Step 2: sweep alpha

for a in [0.0, 0.25, 0.4, 0.5, 0.6, 0.75, 1.0]:
    print(a, hit_rate(alpha=a))

Typical pattern:

Pick the value that does well on both slices rather than the best average.

#Step 3: compare fusion types

for f in [HybridFusion.RELATIVE_SCORE, HybridFusion.RANKED]:
    print(f, hit_rate(alpha=0.5, fusion_type=f))

RANKED only uses positions; RELATIVE_SCORE keeps information about how much better the top result is. Relative score usually wins when one side has a clear strong match; test on your data.

#Step 4: control which properties BM25 searches

By default BM25 searches all text properties. Long noisy fields (raw HTML, logs) dilute it. Choose and boost:

res = kb.query.hybrid(
    query=q, alpha=0.4,
    query_properties=["title^3", "content"],   # title matches count 3x
)

Make sure properties that should never be keyword-searched (URLs, JSON blobs) are created with index_searchable=False.

#Step 5: tokenization

Tokenization is set per property at creation and decides how BM25 splits text:

With word, ORA-01555 becomes ora and 01555, which still often works but also matches every other ORA- code. For codes, add a separate property with whitespace or field tokenization and include it in query_properties with a boost:

from weaviate.classes.config import Property, DataType, Tokenization
Property(name="error_codes", data_type=DataType.TEXT_ARRAY, tokenization=Tokenization.FIELD)

Tokenization can't be changed on an existing property; add a new property and re-import that field.

#Step 6: BM25 parameters

Set per collection (inverted_index_config):

from weaviate.classes.config import Configure
Configure.inverted_index(bm25_b=0.75, bm25_k1=1.2)   # defaults

Change these only after alpha, properties and tokenization; effects are usually smaller. Stopword settings are also in the inverted index config.

#Step 7: add a reranker

Hybrid gets the right chunk into the top 20; a reranker module (for example reranker-cohere or a reranker-transformers container) can reorder the top results:

from weaviate.classes.query import Rerank
res = kb.query.hybrid(query=q, alpha=0.4, limit=20,
                      rerank=Rerank(prop="content", query=q))

Measure hit rate@3 with and without, and the latency it adds.

#Tuning order

#Prevent regressions next time

• Keep the evaluation set in the repo and run it in CI when chunking, model or query settings change.
• Log the alpha and fusion type used with each production query for later analysis.
• Add identifier fields with exact tokenization at design time.
• Re-tune after major corpus changes, not just once.