Observability & Cleanup Cheatsheet

One-screen lookup. Cite: Metrics, Stalled, Removing Jobs, Getters.

Metrics (throughput)

// ENABLE on worker:
new Worker('q', fn, { metrics: { maxDataPoints: MetricsTime.ONE_WEEK * 2 } });

// READ from queue:
const c = await queue.getMetrics('completed', 0, n);  // { data: { values, processedCount } }
const f = await queue.getMetrics('failed');

Per-minute buckets. Sampling, not an audit log. States available: completed, failed.

Counts & jobs (snapshot)

const counts = await queue.getJobCounts();
// { waiting, active, completed, failed, delayed, prioritized,
//   'waiting-children', paused, repeat }

const jobs = await queue.getJobs(['failed'], 0, 50);   // paginated
const job  = await queue.getJob(id);

Stalled jobs

Concept	Truth
"stalled" state?	No. Only a `'stalled'` event. Job is `active` with lapsed lock → moved to `waiting` (or `failed`).
Cause	Worker didn't renew the lock: crash, OOM-kill, or event loop blocked.
Fix	Not a knob — sandboxed processors for CPU work.

Worker option	Default	Notes
`stalledInterval`	30000ms	Scan frequency for lapsed locks.
`maxStalledCount`	1	After N stalls → permanent `failed`.

worker.on('stalled', (jobId) => alert(...));

Cleanup — Layer 1: automatic retention (ALWAYS set)

new Queue('q', {
  defaultJobOptions: {
    removeOnComplete: { age: 86_400, count: 2000 },  // ≤N, none older than age
    removeOnFail:    { age: 604_800, count: 5000 },
  },
});

Value	Effect
`N` (number)	Keep last N.
`{ age, count }`	Both bounds (recommended).
`true`	Delete immediately.

Omitting these = Redis grows until OOM. Most common BullMQ ops failure.

Cleanup — Layer 2: manual methods

Method	Removes	Locked jobs?
`job.remove()`	One job	Throws
`queue.clean(graceMs, count, state)`	≤count in state older than grace	Throws
`queue.drain()`	waiting + delayed (NOT completed/failed/active)	Skips active
`queue.obliterate()`	Everything — no undo	Only with `{force:true}`

await queue.clean(3_600_000, 1000, 'failed');  // failed older than 1h, ≤1000
await queue.obliterate({ force: true });          // CI/test only

Health-endpoint recipe

const c = await queue.getJobCounts();
const stalledLag = c.waiting > 5_000;             // backlog alarm
const failRate   = await failRateFromMetrics();   // from getMetrics('failed')