Best of JS • pg-boss

Queueing jobs in Node.js using PostgreSQL like a boss.

async function readme() {
  const PgBoss = require('pg-boss');
  const boss = new PgBoss('postgres://user:pass@host/database');

  boss.on('error', error => console.error(error));

  await boss.start();

  const queue = 'some-queue';

  let jobId = await boss.send(queue, { param1: 'foo' })

  console.log(`created job in queue ${queue}: ${jobId}`);

  await boss.work(queue, someAsyncJobHandler);
}

async function someAsyncJobHandler(job) {
  console.log(`job ${job.id} received with data:`);
  console.log(JSON.stringify(job.data));

  await doSomethingAsyncWithThis(job.data);
}

pg-boss is a job queue built in Node.js on top of PostgreSQL in order to provide background processing and reliable asynchronous execution to Node.js applications.

pg-boss relies on SKIP LOCKED, a feature added to postgres specifically for message queues, in order to resolve record locking challenges inherent with relational databases. This brings the safety of guaranteed atomic commits of a relational database to your asynchronous job processing.

This will likely cater the most to teams already familiar with the simplicity of relational database semantics and operations (SQL, querying, and backups). It will be especially useful to those already relying on PostgreSQL that want to limit how many systems are required to monitor and support in their architecture.

Features

Exactly-once job delivery
Backpressure-compatible polling workers
Cron scheduling
Pub/sub API for fan-out queue relationships
Deferral, retries (with exponential backoff), rate limiting, debouncing
Completion jobs for orchestrations/sagas
Direct table access for bulk loads via COPY or INSERT
Multi-master compatible (for example, in a Kubernetes ReplicaSet)
Automatic creation and migration of storage tables
Automatic maintenance operations to manage table growth

Requirements

Node 16 or higher
PostgreSQL 11 or higher

Installation

# npm
npm install pg-boss

# yarn
yarn add pg-boss

Documentation

Docs

Contributing

To setup a development environment for this library:

git clone https://github.com/timgit/pg-boss.git
npm install

To run the test suite, linter and code coverage:

npm run cover

The test suite will try and create a new database named pgboss. The config.json file has the default credentials to connect to postgres.

async function readme() { const PgBoss = require('pg-boss'); const boss = new PgBoss('postgres://user:pass@host/database'); boss.on('error', error => console.error(error)); await boss.start(); const queue = 'some-queue'; let jobId = await boss.send(queue, { param1: 'foo' }) console.log(`created job in queue ${queue}: ${jobId}`); await boss.work(queue, someAsyncJobHandler); } async function someAsyncJobHandler(job) { console.log(`job ${job.id} received with data:`); console.log(JSON.stringify(job.data)); await doSomethingAsyncWithThis(job.data); }