Milvus \"collection not loaded\" errors and query node memory: load, release, replicas and mmap

#Symptoms

• search or query fails with an error such as collection not loaded (code 101).
• After a cluster restart, some collections answer and others don't.
• load_collection hangs or stays at a progress below 100%.
• Query nodes are OOMKilled as more collections go live.

#Why loading matters

Milvus keeps collection data on object storage. To search a collection, query nodes must load its segments and indexes into memory (or memory-mapped files). A collection that isn't loaded can't be searched. Loaded collections consume query node memory until they're released.

#Step 1: check the load state

from pymilvus import MilvusClient

client = MilvusClient(uri="http://milvus:19530", token="root:<password>")

for name in client.list_collections():
    print(name, client.get_load_state(collection_name=name))

States: NotLoad, Loading, Loaded, NotExist. For progress on a slow load, Attu (the Milvus web UI) and the get_loading_progress utility in the ORM API both show percentage.

#Step 2: load correctly

client.load_collection(collection_name="kb_chunks")              # waits until loaded by default
client.get_load_state(collection_name="kb_chunks")

A collection must have an index on its vector field before it can be loaded. If it doesn't, the load fails; create the index first.

Make loading part of deployment, not something you hope is already done:

def ensure_loaded(name):
    state = client.get_load_state(collection_name=name)["state"]   # a LoadState enum
    if state.name != "Loaded":
        client.load_collection(collection_name=name)

Run it as a startup step of the application or a post-deploy job.

#Step 3: why a load hangs

kubectl top pods -n milvus | grep querynode
kubectl logs -n milvus <querynode-pod> | grep -iE "load|memory|oom" | tail -30

#Step 4: size memory for loaded collections

Rough memory per loaded collection:

Ten million 768-dim vectors are about 30 GB raw; HNSW adds to that. Leave 30%+ headroom for growing segments, query processing and compaction.

Milvus also enforces a memory guard: if a load would push the query node above a configured share of memory (queryNode.overloadedMemoryThresholdPercentage), the load is refused or can't complete. That's protection, not a bug.

#Step 5: keep only what you need in memory

Enable mmap on a collection (supported versions; release it first, then reload):

client.release_collection(collection_name="kb_chunks")
client.alter_collection_properties(collection_name="kb_chunks", properties={"mmap.enabled": True})
client.load_collection(collection_name="kb_chunks")

mmap trades memory for disk I/O: it needs fast local SSD on query nodes and raises latency if the working set doesn't fit in page cache. Test with real queries.

#Step 6: replicas for availability and throughput

client.load_collection(collection_name="kb_chunks", replica_number=2)

Two replicas place two in-memory copies on different query nodes: more search throughput and survival of one query node failure, at twice the memory. You need at least as many query nodes as replicas.

#Step 7: after restarts

In current versions Milvus remembers which collections were loaded and reloads them after a restart, but large collections take time to come back. During that window searches fail. Plan for it:

• Readiness in the application: don't send traffic until get_load_state is Loaded for required collections.
• Multiple replicas and rolling restarts of query nodes, so one copy stays loaded.

#Decision table

#Prevent it next time

• Keep a capacity sheet: rows, dims, index type and memory per loaded collection.
• Alert on query node memory above 75%.
• Release collections that are no longer queried; drop the ones that are retired.
• Load in a deployment step with a check, never by hand.