Moving Chroma from an embedded notebook database to a server: Docker deployment, persistence, access control and health checks

#Why the embedded mode stops working

PersistentClient(path="./chroma") stores data in a local directory (SQLite plus index files) inside the application process. That's perfect for a notebook or a single process, and wrong as soon as:

• Several application instances or workers need the same data.
• The app runs in containers that are replaced on every deploy.
• Another team or service needs to query it.

Chroma's client-server mode runs one Chroma server that owns the data; applications connect over HTTP.

#Step 1: run the server

Docker Compose:

services:
  chroma:
    image: chromadb/chroma:<pinned-version>      # pin; don't use latest
    ports: ["127.0.0.1:8000:8000"]               # bind locally; expose through a proxy
    volumes:
      - chroma-data:/data                        # 1.x image stores data in /data (0.x used /chroma/chroma)
    environment:
      - ANONYMIZED_TELEMETRY=False
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "curl -sf http://localhost:8000/api/v2/heartbeat || exit 1"]
      interval: 15s
      timeout: 5s
      retries: 5
volumes:
  chroma-data:

Check the data path for your image version in its documentation; mounting the wrong path means data lives in the container and disappears on recreate. Test that by restarting the container and counting a collection.

Without Docker:

pip install chromadb
chroma run --path /var/lib/chroma --host 127.0.0.1 --port 8000

#Step 2: connect from the application

import chromadb
client = chromadb.HttpClient(host="chroma.internal", port=8000, ssl=False)
print(client.heartbeat())
col = client.get_or_create_collection("kb_chunks", configuration={"hnsw": {"space": "cosine"}})

The API is the same as the embedded client, so most code only changes the client creation. For a lighter dependency in the app, the chromadb-client package contains only the HTTP client.

Client and server versions should match (at least the same minor version). A newer client against an older server can fail with API errors after upgrades.

#Step 3: migrate existing data

Copy collections from the old local directory to the server, reusing stored embeddings:

src = chromadb.PersistentClient(path="./chroma")
dst = chromadb.HttpClient(host="chroma.internal", port=8000)

for c in src.list_collections():
    name = c if isinstance(c, str) else c.name
    s = src.get_collection(name)
    d = dst.get_or_create_collection(name, metadata=s.metadata)
    offset = 0
    while True:
        b = s.get(include=["embeddings", "documents", "metadatas"], limit=1000, offset=offset)
        if not b["ids"]:
            break
        d.add(ids=b["ids"], embeddings=b["embeddings"], documents=b["documents"], metadatas=b["metadatas"])
        offset += 1000
    print(name, s.count(), d.count())

Recreate collection configuration (distance space, HNSW settings) explicitly on the destination rather than relying on copied metadata, since the configuration format differs between versions.

#Step 4: access control and TLS

Never expose the Chroma port directly to a network you don't fully control.

Reverse proxy with a static token (nginx example):

server {
  listen 443 ssl;
  server_name chroma.internal;
  ssl_certificate     /etc/nginx/tls/chroma.crt;
  ssl_certificate_key /etc/nginx/tls/chroma.key;

  location / {
    if ($http_x_chroma_token != "REPLACE_WITH_SECRET_FROM_VAULT") { return 401; }
    proxy_pass http://127.0.0.1:8000;
    client_max_body_size 50m;          # batches of embeddings can be large
    proxy_read_timeout 300s;
  }
}

client = chromadb.HttpClient(host="chroma.internal", port=443, ssl=True,
                             headers={"X-Chroma-Token": TOKEN})

On Kubernetes, put Chroma behind an internal-only Service plus a NetworkPolicy that allows only the app's pods, and use your ingress or service mesh for TLS and auth.

#Step 5: resources and limits

#Step 6: backups

For the single-node server, the data directory is the database:

docker compose stop chroma
tar czf /backups/chroma-$(date +%F).tgz -C /var/lib/docker/volumes/<project>_chroma-data/_data .
docker compose start chroma

Stop the server (or take a filesystem/volume snapshot) so SQLite and index files are consistent. Copying while it writes can produce a backup that won't open. Test restore into a separate container regularly.

#Step 7: know the limits

Single-node Chroma fits prototypes, internal tools and moderate production workloads. Consider a distributed deployment (Chroma's own distributed/cloud offering) or another engine when you need:

• High availability (no single node),
• Tens of millions of vectors with tight latency,
• Multi-tenant isolation with per-tenant quotas.

#Checklist

#Prevent problems next time

• Start new projects with HttpClient against a local container, so production isn't a rewrite.
• Pin server and client versions together.
• Script the collection creation (space, HNSW settings) instead of relying on defaults.
• Add Chroma's volume to your backup plan on day one.