Home / Databases / MongoDB

DatabasesHow-toIntermediate

Optimization: MongoDB working set, WiredTiger cache and index efficiency

MongoDB performs well while the working set and indexes fit in memory. Measure WiredTiger cache usage and eviction pressure, check index sizes and usage, remove unused indexes, design compound indexes with the ESR rule, and right-size instances or shard before the working set outgrows RAM.

Deepak Bhardwaj2 min read

Applies toMongoDB 6.0 and later (self-managed replica sets and sharded clusters, MongoDB Atlas); concepts apply to Atlas metrics views

WiredTiger cache basics

  • Default cache: about 50% of (RAM − 1 GB). The OS file cache uses much of the rest.
  • When the cache is full of dirty data, application threads help with eviction, raising latency.
const c = db.serverStatus().wiredTiger.cache;
({
  maxGB: c["maximum bytes configured"] / 1e9,
  inCacheGB: c["bytes currently in the cache"] / 1e9,
  dirtyGB: c["tracked dirty bytes in the cache"] / 1e9,
  readIntoCache: c["pages read into cache"],
  appThreadEvictions: c["pages evicted by application threads"]
})
Signal Meaning
Cache near max and pages read into cache rising fast Working set exceeds cache
Application-thread evictions increasing Eviction pressure, latency spikes
Dirty bytes high (well above ~5% of cache) Writes outpacing checkpoints/eviction

Index sizes and usage

db.orders.stats().indexSizes          // bytes per index
db.orders.aggregate([{ $indexStats: {} }])   // accesses.ops since restart
  • Indexes should fit in RAM along with hot documents.
  • Indexes with accesses.ops: 0 over a full cycle are candidates to hide then drop:
db.orders.hideIndex("status_1")      // test impact
db.orders.dropIndex("status_1")      // when safe

Better compound indexes (ESR)

Equality fields first, then Sort, then Range:

// query: { customerId: X, createdAt: { $gte: d } } sort { createdAt: -1 }
db.orders.createIndex({ customerId: 1, createdAt: -1 })

Check plans with explain("executionStats"): totalKeysExamined and totalDocsExamined should be close to nReturned (see the slow queries post).

Reduce document and working set size

  • Project only needed fields.
  • Avoid unbounded arrays inside documents.
  • Archive old data (Atlas Online Archive or TTL indexes for expiring data).
  • Use compression settings (snappy default, zstd for better ratio at some CPU cost).

When to scale

Situation Option
Working set slightly larger than cache Bigger instance (more RAM)
Data and writes keep growing beyond one replica set Sharding with a good shard key
Read-heavy analytics Secondary reads with care or analytics nodes (Atlas)

Tip

Track cache "pages read into cache" per second alongside query latency. A steady climb is the early warning that the working set is outgrowing memory, weeks before users notice.

Checklist

  • Cache usage, dirty ratio and app-thread eviction tracked
  • Index sizes and usage reviewed
  • Unused indexes hidden then dropped
  • Compound indexes follow ESR
  • Data size reduced via projection, TTL, archive
  • Scaling path decided before cache is exhausted

Related fixes