Home / Databricks on Azure

Databricks on AzureHow-toIntermediate

Advisory: retrieval for RAG on lakehouse data - Databricks Vector Search or Azure AI Search

When documents and tables already live in the lakehouse, the retrieval index can sit next to them or in Azure AI Search. Compare sync from Delta, hybrid search features, governance, application integration and operations, and choose by where the application runs and what search features it needs.

Deepak Bhardwaj2 min read

Applies toDatabricks Mosaic AI Vector Search (Delta Sync and direct access indexes), Azure AI Search (vector and hybrid search, semantic ranker), Azure OpenAI / Azure AI Foundry applications

How they compare

Area Databricks Vector Search Azure AI Search
Source sync Delta Sync index keeps the index updated from a Delta table Indexers for supported sources, or push via API from pipelines
Search types Vector, hybrid (keyword + vector), filters Vector, full-text, hybrid, semantic ranker, rich filters/facets
Governance Unity Catalog permissions and lineage Azure RBAC, API keys or Entra ID, index-level security patterns
Embeddings Managed embedding endpoints or self-computed Integrated vectorisation or self-computed
App integration Mosaic AI agents, Model Serving Azure AI Foundry, Azure OpenAI "on your data", many SDKs
Operations Endpoint sizing, sync pipeline Service tier, replicas/partitions

Features change frequently; check current docs for both.

Decision questions

  1. Where does the application run? Agents served from Databricks fit Vector Search; apps built in Azure AI Foundry or App Service often use AI Search.
  2. What search features are needed? Full-text with language analysers, facets and semantic ranking point to AI Search.
  3. Where is the source data? If chunks and embeddings are produced in Delta tables, Delta Sync keeps things simple.
  4. Who governs access? Unity Catalog-centric governance favours Vector Search; per-document security trimming needs design either way.
  5. Who will operate it? Your data platform team or the application team.

Common patterns

Pattern When
Delta chunks → Vector Search → Databricks agent Everything on Databricks
Delta chunks → push to AI Search → Foundry app App team on Azure AI stack, data team on Databricks
Both Avoid unless clear reasons; two indexes to keep in sync

Design points either way

  • Chunking and metadata (source, date, access labels) prepared in Delta.
  • Document-level access control carried into the index and enforced at query time.
  • Re-indexing strategy when the embedding model changes.
  • Evaluation set to measure retrieval quality (see the RAG evaluation posts in AI & GenAI).

Tip

Keep the chunked, embedded content as a Delta table regardless of index choice. It gives you lineage, history and the option to rebuild or switch indexes later.

Checklist

  • App location and search features documented
  • Index choice made with operating owner
  • Chunks and metadata in Delta
  • Access control enforced at query time
  • Re-indexing and evaluation planned

Related fixes