Chroma \"database is locked\" and missing data with several workers: why PersistentClient isn't multi-process safe and how to fix it

#Symptoms

• sqlite3.OperationalError: database is locked during add/upsert.
• Documents ingested by a background job don't appear in API queries until the API restarts.
• Different API workers return different results for the same query.
• After a crash, Chroma errors on start or queries fail with index errors.
• Problems only appear in production (multiple workers), never on a laptop (one process).

#Why it happens

PersistentClient is an embedded database: each process that creates one opens the SQLite file and the vector index files in the directory and keeps index state in its own memory. Chroma doesn't coordinate several processes writing the same directory:

Chroma documents the embedded client as intended for a single process. The fix is architectural, not a timeout setting.

#Step 1: find who opens the directory

On Linux:

lsof +D /app/chroma 2>/dev/null | awk '{print $1, $2}' | sort -u
fuser -v /app/chroma/chroma.sqlite3

Typical culprits:

#Step 2: the fix - one owner, everyone else over HTTP

Run Chroma as a server and connect all processes with HttpClient:

docker run -d --name chroma -p 127.0.0.1:8000:8000 -v chroma-data:/data chromadb/chroma:<pinned-version>

import chromadb
client = chromadb.HttpClient(host="localhost", port=8000)

The server is the only process touching the files. API workers, Celery workers and scripts all become clients. Migrate the existing data by reading from the old directory with everything else stopped and writing to the server (see the Chroma server deployment post for a copy script).

#Step 3: if you must stay embedded

Sometimes a small internal tool really should stay single-process. Then make it actually single-process:

Create one client per process and reuse it:

from functools import lru_cache
import chromadb

@lru_cache(maxsize=1)
def get_client():
    return chromadb.PersistentClient(path="/app/chroma")

Within one process, serialise writes with a lock if several threads write:

import threading
_write_lock = threading.Lock()

def upsert_chunks(col, **kw):
    with _write_lock:
        col.upsert(**kw)

#Step 4: don't put the directory on a network share

SQLite on SMB/NFS (Azure Files, EFS, NAS) has unreliable locking and poor latency. For embedded Chroma, use a local disk or a block volume (Azure Disk, EBS) attached to one pod/VM. For a server, also prefer block storage.

#Step 5: recover after corruption

If Chroma won't open the directory or queries fail after a multi-writer incident:

1. Stop every process that uses the directory.
2. Copy the directory aside (keep the broken one for analysis).
3. Try opening it with a single client and run count() and a sample query per collection.
4. If collections are damaged, restore from the last backup or rebuild from source documents.

client = chromadb.PersistentClient(path="/restore/chroma")
for c in client.list_collections():
    name = c if isinstance(c, str) else c.name
    col = client.get_collection(name)
    print(name, col.count())
    col.query(query_embeddings=[[0.0] * DIM], n_results=1)   # use your collection's dimension

Rebuilding from source is often fastest for RAG data, as long as ingestion is scripted and idempotent.

#Step 6: verify the fix

• Run the API with its real worker count and the ingestion job at the same time.
• Ingest a test document, then query it from several workers (repeat the request until different workers answer).
• No database is locked errors in logs over a full ingestion cycle.

#Decision table

#Prevent it next time

• Use HttpClient against a local Chroma container from the first prototype.
• Ban PersistentClient in code that runs under Gunicorn, Celery or multiple replicas (a simple code review rule or lint).
• Keep Chroma data on block storage, never on a network share.
• Script ingestion so a rebuild is always possible.