Monitoring Qdrant in production: Prometheus metrics, health probes, collection checks and alerts that matter

#What to watch

#Step 1: scrape /metrics

Qdrant exposes Prometheus metrics on the REST port:

curl -s -H "api-key: $QDRANT_API_KEY" http://qdrant:6333/metrics | head -40

The output includes application info, collection and vector counts, REST and gRPC request counters with response time histograms, and (in cluster mode) peer and Raft metrics. Exact metric names change between versions, so build dashboards from what your version actually returns.

Prometheus scrape config (when an API key is required, use a read-only key):

scrape_configs:
  - job_name: qdrant
    metrics_path: /metrics
    authorization:
      type: Bearer
      credentials_file: /etc/prometheus/secrets/qdrant-read-only-key
    static_configs:
      - targets: ["qdrant-0.qdrant-headless:6333", "qdrant-1.qdrant-headless:6333", "qdrant-2.qdrant-headless:6333"]

Note

Prometheus' authorization block sends Authorization: Bearer <key>. Qdrant accepts the API key either in the api-key header or as a Bearer token; confirm this for your Qdrant version with curl -H "Authorization: Bearer $KEY" .../metrics before relying on it.

On Kubernetes, use a ServiceMonitor or PodMonitor from the Prometheus Operator. Scrape each pod, not the service, so you see per-node metrics.

#Step 2: build the dashboard

Panels worth having (adapt names to your /metrics output):

#Step 3: add a collection checker

Metrics don't directly report collection status or replica states in every version. A small exporter or scheduled job fills the gap:

import requests, time
from prometheus_client import Gauge, start_http_server

H = {"api-key": READ_ONLY_KEY}
BASE = "http://qdrant:6333"
STATUS = {"green": 0, "yellow": 1, "grey": 2, "red": 3}
g_status = Gauge("qdrant_collection_status", "0 green,1 yellow,2 grey,3 red", ["collection"])
g_gap = Gauge("qdrant_unindexed_vectors", "points minus indexed vectors", ["collection"])
g_dead = Gauge("qdrant_non_active_replicas", "replicas not Active", ["collection"])

def collect():
    for c in requests.get(f"{BASE}/collections", headers=H).json()["result"]["collections"]:
        name = c["name"]
        info = requests.get(f"{BASE}/collections/{name}", headers=H).json()["result"]
        g_status.labels(name).set(STATUS.get(info["status"], 3))
        g_gap.labels(name).set((info.get("points_count") or 0) - (info.get("indexed_vectors_count") or 0))
        cl = requests.get(f"{BASE}/collections/{name}/cluster", headers=H).json()["result"]
        shards = cl.get("local_shards", []) + cl.get("remote_shards", [])
        g_dead.labels(name).set(sum(1 for s in shards if s.get("state") != "Active"))

start_http_server(9105)
while True:
    collect()
    time.sleep(60)

In a cluster, local_shards only shows shards on the peer you ask; querying one peer still lists remote shards with their state.

#Step 4: probes

livenessProbe:
  httpGet: { path: /livez, port: 6333 }
  periodSeconds: 10
  failureThreshold: 6
readinessProbe:
  httpGet: { path: /readyz, port: 6333 }
  periodSeconds: 5
startupProbe:
  httpGet: { path: /livez, port: 6333 }
  periodSeconds: 10
  failureThreshold: 60        # large collections take time to load at start

/readyz reports ready only once the node has joined consensus and its shards are ready, which is what you want before it receives traffic. Liveness should not depend on readiness: a node loading data is alive, just not ready.

#Step 5: alerts

Example PromQL for the point-count drop, assuming your collection gauge is called qdrant_points (replace with the real name from /metrics or your exporter):

(max_over_time(qdrant_points[1h]) - qdrant_points) / max_over_time(qdrant_points[1h]) > 0.05

#Step 6: logs

Set QDRANT__LOG_LEVEL=INFO (or WARN on busy clusters) and ship logs to your platform. Search for error, panic, Failed to, consensus and transfer after incidents.

#Prevent blind spots next time

• Scrape every pod, not only the service.
• Use a read-only API key for monitoring.
• Alert on replica state and point-count drops, not just latency.
• Review the dashboard after every Qdrant upgrade; metric names change.