engineering
Background jobs with BullMQ
Queues, retries, and the outbox pattern — running async work in a way that survives crashes and redeploys.
Most application bugs don't live in your API handlers. They live in the long tail of "send this email later" and "charge this card tomorrow" — background jobs where failures are silent and retries compound.
Why BullMQ#
Redis-backed, TypeScript-first, batteries included. Built-in delayed jobs, retries with exponential backoff, a dead-letter queue, and a clean API for adding workers. For Node.js applications it's the standard answer.
The enqueue-before-commit problem#
Queue a job inside a database transaction that rolls back? The job still runs, with no record in the database. Queue it after commit, but the process crashes between the commit and the queue call? The commit happened, the job never runs.
The outbox pattern#
The fix: write the job as a row in an outbox table inside the same transaction as the business logic. A separate poller reads the outbox and enqueues the job into BullMQ, marking the row as processed. Now job enqueue is atomic with the DB write.
Idempotency keys#
Jobs can run twice. Design every processor to handle that. A deterministic jobId derived from the domain event (e.g., trial-reminder:sub_abc123:2026-04-15) lets BullMQ deduplicate automatically.
Dead letter queues#
Every queue needs a DLQ for exhausted retries. Inspect it manually — automatic replay of a DLQ job is almost always the wrong answer.
Observability#
Track queue depth, processing time, and retry count per queue name. When a queue starts growing faster than it drains, you want the dashboard to tell you before the customers do.