@tryghost/job-manager 0.9.0 → 1.0.1

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2013-2022 Ghost Foundation
+Copyright (c) 2013-2025 Ghost Foundation
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -1,33 +1,70 @@
 # Job Manager
 
-A manager for jobs (aka tasks) that have to be performed asynchronously, optionally recurring, scheduled or one-off in their nature. The job queue is manage in memory without additional dependencies.
+A manager for background jobs in Ghost, supporting one-off tasks and recurring jobs.
 
-## Install
+## Table of Contents
+- [Quick Start](#quick-start)
+- [Job Types](#job-types)
+- [Background Job Requirements](#background-job-requirements)
+- [Advanced Usage](#advanced-usage)
+- [Technical Details](#technical-details)
 
-`npm install @tryghost/job-manager --save`
+## Quick Start
 
-or
+```js
+const JobManager = require('@tryghost/job-manager');
+const jobManager = new JobManager({JobModel: models.Job});
+
+// Simple one-off job
+jobManager.addJob({
+    name: 'hello-world',
+    job: () => console.log('Hello World'),
+    offloaded: false
+});
+
+// Recurring job every 5 minutes
+jobManager.addJob({
+    name: 'check-emails',
+    at: 'every 5 minutes',
+    job: './jobs/check-emails.js'
+});
+```
+
+## Job Types
 
-`yarn add @tryghost/job-manager`
+Ghost supports two types of jobs:
 
+1. **Inline Jobs**
+   - Run in the main Ghost process
+   - Best for quick, simple tasks
+   - Cannot be scheduled or recurring
 
-## Usage
+2. **Offloaded Jobs**
+   - Run in a separate process
+   - Good for CPU-intensive tasks
+   - Can be scheduled or recurring
 
-Below is a sample code to wire up job manger and initialize jobs:
+## Background Job Requirements
+
+Offloaded jobs must:
+
+- Have a unique name
+- Be idempotent (safe to run multiple times)
+- Use minimal parameters (prefer using IDs rather than full objects)
+- Import their own dependencies
+- Primarily use DB or API calls
+
+## Advanced Usage
+
+Below is a sample code to wire up job manger and initialize jobs. This is the simplest way to interact with the job manager - these jobs do not persist after reboot:
 ```js
 const JobManager = require('@tryghost/job-manager');
 
-const logging = {
-    info: console.log,
-    warn: console.log,
-    error: console.error
-};
-
-const jobManager = new JobManager(logging);
+const jobManager = new JobManager({JobModel: models.Job});
 
 // register a job "function" with queued execution in current event loop
 jobManager.addJob({
-    job: printWord(word) => console.log(word),
+    job: (word) => console.log(word),
     name: 'hello',
     offloaded: false
 });
@@ -55,82 +92,60 @@ jobManager.addJob({
     name: 'email-checker-cron'
 });
 
-// register a job to un immediately running outside of current even loop
+// register a job to run immediately running outside of current even loop
 jobManager.addJob({
     job: './path/to/jobs/check-emails.js',
     name: 'email-checker-now'
 });
-```
 
-For more examples of JobManager initialization check [test/examples](https://github.com/TryGhost/Utils/tree/master/packages/job-manager/test/examples) directory.
-
-### Job types and definitions
-
-There are two types of jobs distinguished based on purpose and environment they run in:
-- **"inline"** - job which is run in the same even loop as the caller. Should be used in situations when there is no even loop blocking operations and no need to manage memory leaks in sandboxed way. Sometimes
-- **"offloaded"** - job which is executed in separate to caller's event loop. For Node >v12 clients it spawns a [Worker thread](https://nodejs.org/dist/latest-v12.x/docs/api/worker_threads.html#worker_threads_new_worker_filename_options), for older Node runtimes it is executed in separate process through [child_process](https://nodejs.org/docs/latest-v10.x/api/child_process.html). Comparing to **inline** jobs, **offloaded** jobs are safer to execute as they are run on a dedicated thread (or process) acting like a sandbox. These jobs also give better utilization of multi-core CPUs. This type of jobs is useful when there are heavy computations needed to be done blocking the event loop or need a sandboxed environment to run in safely. Example jobs would be: statistical information processing, memory intensive computations (e.g. recursive algorithms), processing that requires blocking I/O operations etc.
-
-Job manager's instance registers jobs through `addJob` method. The `offloaded` parameter controls if the job is **inline** (executed in the same event loop) or is **offloaded** (executed in worker thread/separate process). By default `offloaded` is set to `true` - creates an "offloaded" job.
+// register a one-off job to be executed immediately within current event loop
+jobManager.addOneOffJob({
+    name: 'members-migrations',
+    offloaded: false,
+    job: stripeService.migrations.execute.bind(stripeService.migrations)
+});
 
-When `offloaded: false` parameter is passed into `addJob` method, job manager registers an **inline** function for execution in FIFO queue. The job should not be computationally intensive and should have small amount of asynchronous operations. The developer should always account that the function will be executed on the **same event loop, thread and process as caller's process**. **inline** jobs should be [JavaScript functions](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function) or a path to a module that exports a function as default. Note, at the moment it's not possible to defined scheduled or recurring **inline** job.
+// register a one-off job to be executed immediately outside of current event loop
+jobManager.addOneOffJob({
+    name: 'generate-backup-2022-09-15',
+    job: './path/to/jobs/backup.js',
+});
 
-When skipped or `offloaded: true` parameter is passed into `addJob` method, job manager registers execution of an **offloaded** job. The job can be scheduled to run immediately, in the future, or in recurring manner (through `at` parameter). Jobs created this way are managed by [bree](https://github.com/breejs/bree) job scheduling library. For examples of job scripts check out [this section](https://github.com/breejs/bree#nodejs-email-queue-job-scheduling-example) of bree's documentation, test [job examples](https://github.com/TryGhost/Utils/tree/master/packages/job-manager/test/jobs).
+// optionally await completion of the one-off job in case 
+// there are state changes expected to execute the rest of the process
+//  NOTE: if multiple jobs are submitted using the same name, the first completion will resolve
+await jobManager.awaitOneOffCompletion('members-migrations');
+
+// check if previously registered one-off job has been executed 
+// successfully - it exists and doesn't have a "failed" state.
+//  NOTE: this is stored in memory and cleared on reboot
+const backupSuccessful = await jobManager.hasExecutedSuccessfully('generate-backup-2022-09-15');
+
+if (!backupSuccessful) {
+    // One-off jobs with "failed" state can be rescheduled
+    jobManager.addOneOffJob({
+        name: 'generate-backup-2022-09-15',
+        job: './path/to/jobs/backup.js',
+    });
+}
+```
 
+For more examples of JobManager initialization check [test/examples](https://github.com/TryGhost/Utils/tree/master/packages/job-manager/test/examples) directory.
 
-### Offloaded jobs rules of thumb
-To prevent complications around failed job retries and and handling of specific job states here are some rules that should be followed for all scheduled jobs:
-1. Jobs are **self contained** - meaning job manager should be able to run the job with the state information included within the job's parameters. Job script should look up for the rest of needed information from somewhere else, like a database, API, or file.
-2. Jobs should be [idempotent](https://en.wikipedia.org/wiki/Idempotence) - consequent job executions should be safe.
-3. Job **parameters** should be **kept to the minimum**. When passing large amounts of data around performance can suffer from slow JSON serialization. Also, storage size restrictions that can arise if there is a need to store parameters in the future.Job parameters should be kept to only information that is needed to retrieve the rest of information from somewhere else. For example, it's recommended to pass in only an *id* of the resource that could be fetched from the data storage during job execution or pass in a file path which could be read during execution.
-4. Scheduled **job execution time should not overlap**. It's up to the registering service to assure job execution time does not ecceed time between subsequent scheduled jobs. For example, if job is scheduled to run every 5 minutes it should always run under 5 minutes, otherwise next scheduled job would fail to start.
+## Technical Details
 
-### Offloaded jobs lifecycle
+### Process Isolation
+Background jobs run in separate processes, meaning:
+- No shared memory with Ghost
+- Must import own dependencies
+- Should avoid heavy dependencies
+- Should handle graceful shutdown
 
-Offloaded jobs are running on dedicated worker threads which makes their lifecycle a bit different to inline jobs:
-1. When **starting** a job it's only sharing ENV variables with it's parent process. The job itself is run on an independent JavaScript execution thread. The script has to re-initialize any modules it will use. For example it should take care of: model layer initialization, cache initialization, etc.
+### Job Lifecycle
+Offloaded jobs are running on dedicated worker threads which makes their lifecycle a bit different from inline jobs:
+1. When **starting** a job it's only sharing ENV variables with its parent process. The job itself is run on an independent JavaScript execution thread. The script has to re-initialize any modules it will use. For example, it should take care of: model layer initialization, cache initialization, etc.
 2. When **finishing** work in a job prefer to signal successful termination by sending 'done' message to the parent thread: `parentPort.postMessage('done')` ([example use](https://github.com/TryGhost/Utils/blob/0e423f6c5c69b08d81d470f49de95654d8cc90e3/packages/job-manager/test/jobs/graceful.js#L33-L37)). Finishing work this way terminates the thread through [worker.terminate()]((https://nodejs.org/dist/latest-v14.x/docs/api/worker_threads.html#worker_threads_worker_terminate)), which logs termination in parent process and flushes any pipes opened in thread.
 3. Jobs that have iterative nature, or need cleanup before interrupting work should allow for **graceful shutdown** by listening on `'cancel'` message coming from parent thread ([example use](https://github.com/TryGhost/Utils/blob/0e423f6c5c69b08d81d470f49de95654d8cc90e3/packages/job-manager/test/jobs/graceful.js#L12-L16)).
 4. When **exceptions** happen and expected outcome is to terminate current job, leave the exception unhandled allowing it to bubble up to the job manager. Unhandled exceptions [terminate current thread](https://nodejs.org/dist/latest-v14.x/docs/api/worker_threads.html#worker_threads_event_error) and allow for next scheduled job execution to happen.
 
 For more nuances on job structure best practices check [bree documentation](https://github.com/breejs/bree#writing-jobs-with-promises-and-async-await).
-
-### Offloaded job script quirks
-
-⚠️ to ensure worker thread back compatibility and correct inter-thread communication use [btrheads](https://github.com/chjj/bthreads) polyfill instead of native [worker_threads](https://nodejs.org/api/worker_threads.html#worker_threads) module in job scripts.
-
-Instead of:
-```js
-const {isMainThread, parentPort} = require('worker_threads');
-```
-use
-```js
-const {isMainThread, parentPort} = require('bthreads');
-```
-
-It should be possible to use native `worker_threads` module once Node v10 [hits EOL](https://nodejs.org/en/about/releases/) (2021-04-30).
-
-## Develop
-
-This is a mono repository, managed with [lerna](https://lernajs.io/).
-
-Follow the instructions for the top-level repo.
-1. `git clone` this repo & `cd` into it as usual
-2. Run `yarn` to install top-level dependencies.
-
-
-## Run
-
-- `yarn dev`
-
-
-## Test
-
-- `yarn lint` run just eslint
-- `yarn test` run lint and tests
-
-
-
-
-# Copyright & License
-
-Copyright (c) 2013-2022 Ghost Foundation - Released under the [MIT license](LICENSE).

package/index.js

@@ -1 +1 @@
-module.exports = require('./lib/job-manager');
+module.exports = require('./lib/JobManager');

package/lib/JobManager.js

@@ -0,0 +1,427 @@
+const path = require('path');
+const util = require('util');
+const setTimeoutPromise = util.promisify(setTimeout);
+const fastq = require('fastq');
+const later = require('@breejs/later');
+const Bree = require('bree');
+const pWaitFor = require('p-wait-for');
+const {UnhandledJobError, IncorrectUsageError} = require('@tryghost/errors');
+const logging = require('@tryghost/logging');
+const isCronExpression = require('./is-cron-expression');
+const assembleBreeJob = require('./assemble-bree-job');
+const JobsRepository = require('./JobsRepository');
+
+const worker = async (task, callback) => {
+    try {
+        let result = await task();
+        await callback(null, result);
+    } catch (error) {
+        await callback(error);
+    }
+};
+
+const ALL_STATUSES = {
+    started: 'started',
+    finished: 'finished',
+    failed: 'failed',
+    queued: 'queued'
+};
+
+/**
+ * @typedef {Object} ScheduledJob
+ * @property {Function | string} job - Function or path to a module defining a job
+ * @property {string} [name] - Unique job name, if not provided takes function name or job script filename
+ * @property {string | Date} [at] - Date, cron or human readable schedule format
+ * @property {Object} [data] - Data to be passed into the job
+ * @property {boolean} [offloaded=true] - If true, creates an "offloaded" job running in a worker thread. If false, runs an "inline" job on the same event loop
+ */
+class JobManager {
+    #domainEvents;
+    #completionPromises = new Map();
+    #config;
+    #JobModel;
+    #events;
+
+    /**
+     * @param {Object} options
+     * @param {Function} [options.errorHandler] - custom job error handler
+     * @param {Function} [options.workerMessageHandler] - custom message handler coming from workers
+     * @param {Object} [options.JobModel] - a model which can persist job data in the storage
+     * @param {Object} [options.domainEvents] - domain events emitter
+     * @param {Object} [options.config] - config
+     * @param {Object} [options.events] - events instance (for testing)
+     */
+    constructor({errorHandler, workerMessageHandler, JobModel, domainEvents, config, events = null}) {
+        this.inlineQueue = fastq(this, worker, 3);
+        this._jobMessageHandler = this._jobMessageHandler.bind(this);
+        this._jobErrorHandler = this._jobErrorHandler.bind(this);
+        this.#domainEvents = domainEvents;
+        this.#config = config;
+        this.#JobModel = JobModel;
+        this.#events = events;
+
+        const combinedMessageHandler = workerMessageHandler
+            ? ({name, message}) => {
+                workerMessageHandler({name, message});
+                this._jobMessageHandler({name, message});
+            }
+            : this._jobMessageHandler;
+
+        const combinedErrorHandler = errorHandler
+            ? (error, workerMeta) => {
+                errorHandler(error, workerMeta);
+                this._jobErrorHandler(error, workerMeta);
+            }
+            : this._jobErrorHandler;
+
+        this.bree = new Bree({
+            root: false, // set this to `false` to prevent requiring a root directory of jobs
+            hasSeconds: true, // precision is needed to avoid task overlaps after immediate execution
+            outputWorkerMetadata: true,
+            logger: logging,
+            errorHandler: combinedErrorHandler,
+            workerMessageHandler: combinedMessageHandler
+        });
+
+        this.bree.on('worker created', (name) => {
+            this._jobMessageHandler({name, message: ALL_STATUSES.started});
+        });
+
+        if (JobModel) {
+            this._jobsRepository = new JobsRepository({JobModel});
+        }
+    }
+
+    inlineJobHandler(jobName) {
+        return async (error, result) => {
+            if (error) {
+                await this._jobErrorHandler(error, {
+                    name: jobName
+                });
+            } else {
+                await this._jobMessageHandler({
+                    name: jobName,
+                    message: 'done'
+                });
+            }
+
+            // Can potentially standardize the result here
+            return result;
+        };
+    }
+
+    async _jobMessageHandler({name, message}) {
+        if (name) {
+            if (message === ALL_STATUSES.started) {
+                if (this._jobsRepository) {
+                    const job = await this._jobsRepository.read(name);
+
+                    if (job) {
+                        await this._jobsRepository.update(job.id, {
+                            status: ALL_STATUSES.started,
+                            started_at: new Date()
+                        });
+                    }
+                }
+            } else if (message === 'done') {
+                if (this._jobsRepository) {
+                    const job = await this._jobsRepository.read(name);
+
+                    if (job) {
+                        await this._jobsRepository.update(job.id, {
+                            status: ALL_STATUSES.finished,
+                            finished_at: new Date()
+                        });
+                    }
+                }
+
+                // Check completion listeners
+                if (this.#completionPromises.has(name)) {
+                    for (const listeners of this.#completionPromises.get(name)) {
+                        listeners.resolve();
+                    }
+                    // Clear the listeners
+                    this.#completionPromises.delete(name);
+                }
+
+                if (this.inlineQueue.length() <= 1) {
+                    if (this.#completionPromises.has('all')) {
+                        for (const listeners of this.#completionPromises.get('all')) {
+                            listeners.resolve();
+                        }
+                        // Clear the listeners
+                        this.#completionPromises.delete('all');
+                    }
+                }
+            } else {
+                if (typeof message === 'object' && this.#domainEvents) {
+                    // Is this an event?
+                    if (message.event) {
+                        this.#domainEvents.dispatchRaw(message.event.type, message.event.data);
+                    }
+                }
+            }
+        }
+    }
+
+    async _jobErrorHandler(error, jobMeta) {
+        if (this._jobsRepository && jobMeta.name) {
+            const job = await this._jobsRepository.read(jobMeta.name);
+
+            if (job) {
+                await this._jobsRepository.update(job.id, {
+                    status: ALL_STATUSES.failed
+                });
+            }
+        }
+
+        // Check completion listeners and call them with error
+        if (this.#completionPromises.has(jobMeta.name)) {
+            for (const listeners of this.#completionPromises.get(jobMeta.name)) {
+                listeners.reject(error);
+            }
+            // Clear the listeners
+            this.#completionPromises.delete(jobMeta.name);
+        }
+
+        if (this.inlineQueue.length() <= 1) {
+            if (this.#completionPromises.has('all')) {
+                for (const listeners of this.#completionPromises.get('all')) {
+                    listeners.reject(error);
+                }
+                // Clear the listeners
+                this.#completionPromises.delete('all');
+            }
+        }
+    }
+
+    /**
+     * By default schedules an "offloaded" job. If `offloaded: true` parameter is set,
+     * puts an "inline" immediate job into the inlineQueue.
+     *
+     * @param {Object} GhostJob - job options
+     * @prop {Function | String} GhostJob.job - function or path to a module defining a job
+     * @prop {String} [GhostJob.name] - unique job name, if not provided takes function name or job script filename
+     * @prop {String | Date} [GhostJob.at] - Date, cron or human readable schedule format. Manage will do immediate execution if not specified. Not supported for "inline" jobs
+     * @prop {Object} [GhostJob.data] - data to be passed into the job
+     * @prop {Boolean} [GhostJob.offloaded] - creates an "offloaded" job running in a worker thread by default. If set to "false" runs an "inline" job on the same event loop
+     */
+    addJob({name, at, job, data, offloaded = true}) {
+        if (offloaded) {
+            logging.info('Adding offloaded job to the inline job queue');
+            let schedule;
+
+            if (!name) {
+                if (typeof job === 'string') {
+                    name = path.parse(job).name;
+                } else {
+                    throw new IncorrectUsageError({
+                        message: 'Name parameter should be present if job is a function'
+                    });
+                }
+            }
+
+            if (at && !(at instanceof Date)) {
+                if (isCronExpression(at)) {
+                    schedule = later.parse.cron(at, true);
+                } else {
+                    schedule = later.parse.text(at);
+                }
+
+                if ((schedule.error && schedule.error !== -1) || schedule.schedules.length === 0) {
+                    throw new IncorrectUsageError({
+                        message: 'Invalid schedule format'
+                    });
+                }
+
+                logging.info(`Scheduling job ${name} at ${at}. Next run on: ${later.schedule(schedule).next()}`);
+            } else if (at !== undefined) {
+                logging.info(`Scheduling job ${name} at ${at}`);
+            } else {
+                logging.info(`Scheduling job ${name} to run immediately`);
+            }
+
+            const breeJob = assembleBreeJob(at, job, data, name);
+            this.bree.add(breeJob);
+            return this.bree.start(name);
+        } else {
+            logging.info(`Adding one-off job to inlineQueue with current length = ${this.inlineQueue.length()} called '${name || 'anonymous'}'`);
+
+            this.inlineQueue.push(async () => {
+                try {
+                    // NOTE: setting the status here otherwise it is impossible to
+                    //       distinguish between states when the job fails immediately
+                    await this._jobMessageHandler({
+                        name: name,
+                        message: ALL_STATUSES.started
+                    });
+
+                    if (typeof job === 'function') {
+                        await job(data);
+                    } else {
+                        await require(job)(data);
+                    }
+                } catch (err) {
+                    // NOTE: each job should be written in a safe way and handle all errors internally
+                    //       if the error is caught here jobs implementation should be changed
+                    logging.error(new UnhandledJobError({
+                        context: (typeof job === 'function') ? 'function' : job,
+                        err
+                    }));
+
+                    throw err;
+                }
+            }, this.inlineJobHandler(name));
+        }
+    }
+
+    /**
+    * Adds a job that could ever be executed once. In case the job fails
+    * can be "added" again, effectively restarting the failed job.
+    *
+    * @param {Object} GhostJob - job options
+    * @prop {Function | String} GhostJob.job - function or path to a module defining a job
+    * @prop {String} GhostJob.name - unique job name, if not provided takes function name or job script filename
+    * @prop {String | Date} [GhostJob.at] - Date, cron or human readable schedule format. Manage will do immediate execution if not specified. Not supported for "inline" jobs
+    * @prop {Object} [GhostJob.data] - data to be passed into the job
+    * @prop {Boolean} [GhostJob.offloaded] - creates an "offloaded" job running in a worker thread by default. If set to "false" runs an "inline" job on the same event loop
+    */
+    async addOneOffJob({name, job, data, offloaded = true}) {
+        if (!name) {
+            throw new IncorrectUsageError({
+                message: `The name parameter is required for a one off job.`
+            });
+        }
+
+        const persistedJob = await this._jobsRepository.read(name);
+
+        if (persistedJob && (persistedJob.get('status') !== ALL_STATUSES.failed)) {
+            throw new IncorrectUsageError({
+                message: `A "${name}" one off job has already been executed.`
+            });
+        }
+
+        if (persistedJob && (persistedJob.get('status') === ALL_STATUSES.failed)) {
+            await this._jobsRepository.update(persistedJob.id, {
+                status: ALL_STATUSES.queued
+            });
+        } else {
+            await this._jobsRepository.add({
+                name,
+                status: ALL_STATUSES.queued
+            });
+        }
+
+        // NOTE: there's a assumption the job with the same name failed while
+        //       running under different instance of job manager (bree).
+        //       For example, it failed and the process was restarted.
+        //       If we want to be able to restart within the same instance,
+        //       we'd need to handle job restart/removal in Bree first
+        this.addJob({name, job, data, offloaded});
+    }
+
+    /**
+     * Checks if the one-off job has ever been executed successfully.
+     * @param {String} name one-off job name
+     */
+    async hasExecutedSuccessfully(name) {
+        if (this._jobsRepository) {
+            const persistedJob = await this._jobsRepository.read(name);
+
+            if (!persistedJob) {
+                return false;
+            } else {
+                return (persistedJob.get('status') !== ALL_STATUSES.failed);
+            }
+        } else {
+            return false;
+        }
+    }
+
+    /**
+     * Awaits completion of the offloaded one-off job.
+     * CAUTION: it might take a long time to resolve!
+     * @param {string} name one-off job name
+     * @returns resolves with a Job model at current state
+     */
+    async awaitOneOffCompletion(name) {
+        const persistedJob = await this._jobsRepository.read(name);
+
+        if (!persistedJob || ![ALL_STATUSES.finished, ALL_STATUSES.failed].includes(persistedJob.get('status'))) {
+            // NOTE: can implement exponential backoff here if that's ever needed
+            await setTimeoutPromise(500);
+
+            return this.awaitOneOffCompletion(name);
+        }
+
+        return persistedJob;
+    }
+
+    /***
+     * Create this promise before you add the job you want to listen for. Then await the returned promise.
+     * Resolves if the job has been executed successfully.
+     * Throws an error if the job has failed execution.
+     */
+    async awaitCompletion(name) {
+        const promise = new Promise((resolve, reject) => {
+            this.#completionPromises.set(name, [
+                ...(this.#completionPromises.get(name) ?? []),
+                {resolve, reject}
+            ]);
+        });
+
+        return promise;
+    }
+
+    /**
+     * Wait for all inline jobs to be completed.
+     */
+    async allSettled() {
+        const name = 'all';
+
+        return new Promise((resolve, reject) => {
+            if (this.inlineQueue.idle()) {
+                resolve();
+                return;
+            }
+
+            this.#completionPromises.set(name, [
+                ...(this.#completionPromises.get(name) ?? []),
+                {resolve, reject}
+            ]);
+        });
+    }
+
+    /**
+     * Removes an "offloaded" job from scheduled jobs inlineQueue.
+     * It's NOT yet possible to remove "inline" jobs (will be possible when scheduling is added https://github.com/breejs/bree/issues/68).
+     * The method will throw an Error if job with provided name does not exist.
+     *
+     * NOTE: current implementation does not guarante running job termination
+     *       for details see https://github.com/breejs/bree/pull/64
+     *
+     * @param {String} name - job name
+     */
+    async removeJob(name) {
+        await this.bree.remove(name);
+    }
+
+    /**
+     * @param {import('p-wait-for').Options} [options]
+     */
+    async shutdown(options) {
+        await this.bree.stop();
+
+        if (this.inlineQueue.idle()) {
+            return;
+        }
+
+        logging.warn('Waiting for busy job in inline job queue');
+
+        await pWaitFor(() => this.inlineQueue.idle() === true, options);
+
+        logging.warn('Inline job queue finished');
+    }
+}
+
+module.exports = JobManager;

package/lib/JobsRepository.js

@@ -0,0 +1,71 @@
+const logging = require('@tryghost/logging');
+
+/**
+ * @class JobsRepository
+ * @description Repository class for managing job-related operations.
+ */
+class JobsRepository {
+    /**
+     * @constructor
+     * @param {Object} options - The options object.
+     * @param {Object} options.JobModel - The Job model for database operations.
+     */
+    constructor({JobModel}) {
+        // NOTE: We ought to clean this up. We want to use bookshelf models for all db operations,
+        //  but we use knex directly in a few places still largely for performance reasons.
+        this._JobModel = JobModel;
+    }
+
+    /**
+     * @method add
+     * @async
+     * @description Adds a new job to the database.
+     * @param {Object} data - The job data to be added.
+     * @returns {Promise<Object>} The added job object.
+     */
+    async add(data) {
+        const job = await this._JobModel.add(data);
+        return job;
+    }
+
+    /**
+     * @method read
+     * @async
+     * @description Reads a job from the database by name.
+     * @param {string} name - The name of the job to read.
+     * @returns {Promise<Object|null>} The job object if found, null otherwise.
+     */
+    async read(name) {
+        const job = await this._JobModel.findOne({name});
+        return job;
+    }
+
+    /**
+     * @method update
+     * @async
+     * @description Updates a job in the database.
+     * @param {string} id - The ID of the job to update.
+     * @param {Object} data - The updated job data.
+     * @returns {Promise<void>}
+     */
+    async update(id, data) {
+        await this._JobModel.edit(data, {id});
+    }
+
+    /**
+     * @method delete
+     * @async
+     * @description Deletes a job from the database.
+     * @param {string} id - The ID of the job to delete.
+     * @returns {Promise<void>}
+     */
+    async delete(id) {
+        try {
+            await this._JobModel.destroy({id});
+        } catch (error) {
+            logging.error(`Error deleting job ${id}:`, error);
+        }
+    }
+}
+
+module.exports = JobsRepository;

package/package.json

@@ -1,39 +1,48 @@
 {
   "name": "@tryghost/job-manager",
-  "version": "0.9.0",
-  "repository": "https://github.com/TryGhost/Utils/tree/main/packages/job-manager",
+  "version": "1.0.1",
   "author": "Ghost Foundation",
   "license": "MIT",
+  "publishConfig": {
+    "access": "public"
+  },
   "main": "index.js",
   "scripts": {
     "dev": "echo \"Implement me!\"",
-    "test": "NODE_ENV=testing c8 --all --reporter text --reporter cobertura mocha './test/**/*.test.js'",
-    "lint": "eslint . --ext .js --cache",
-    "posttest": "yarn lint"
+    "test:unit": "NODE_ENV=testing c8 --all --reporter text --reporter cobertura -- mocha --reporter dot './test/**/*.test.js'",
+    "test": "yarn test:unit",
+    "lint:code": "eslint *.js lib/ --ext .js --cache",
+    "lint": "yarn lint:code && yarn lint:test",
+    "lint:test": "eslint -c test/.eslintrc.js test/ --ext .js --cache"
   },
   "files": [
     "index.js",
     "lib"
   ],
-  "publishConfig": {
-    "access": "public"
-  },
   "devDependencies": {
-    "@sinonjs/fake-timers": "9.1.2",
-    "c8": "7.12.0",
-    "date-fns": "2.29.1",
+    "@sinonjs/fake-timers": "11.3.1",
+    "c8": "8.0.1",
+    "date-fns": "2.30.0",
     "delay": "5.0.0",
-    "mocha": "10.0.0",
+    "mocha": "10.8.2",
+    "rewire": "7.0.0",
     "should": "13.2.3",
-    "sinon": "14.0.0"
+    "sinon": "15.2.0"
   },
   "dependencies": {
-    "@breejs/later": "^4.0.2",
-    "@tryghost/logging": "^2.0.0",
-    "bree": "^6.2.0",
-    "cron-validate": "^1.4.3",
-    "fastq": "^1.11.0",
-    "p-wait-for": "^3.2.0"
+    "@breejs/later": "4.2.0",
+    "@tryghost/errors": "^1.3.8",
+    "@tryghost/logging": "^2.4.22",
+    "bree": "6.5.0",
+    "cron-validate": "1.4.5",
+    "fastq": "1.19.1",
+    "p-wait-for": "3.2.0",
+    "workerpool": "9.2.0"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/TryGhost/framework.git",
+    "directory": "packages/job-manager"
   },
-  "gitHead": "2ce0a263975cf6e0f34233caf19404fd8ab5e7a5"
+  "gitHead": "e395f1d79edee4aecc36e572bb402016a8ba9273"
 }

package/lib/job-manager.js

@@ -1,236 +0,0 @@
-const path = require('path');
-const fastq = require('fastq');
-const later = require('@breejs/later');
-const Bree = require('bree');
-const pWaitFor = require('p-wait-for');
-const {UnhandledJobError, IncorrectUsageError} = require('@tryghost/errors');
-const logging = require('@tryghost/logging');
-const isCronExpression = require('./is-cron-expression');
-const assembleBreeJob = require('./assemble-bree-job');
-const JobsRepository = require('./jobs-repository');
-
-const worker = async (task, callback) => {
-    try {
-        let result = await task();
-        callback(null, result);
-    } catch (error) {
-        callback(error);
-    }
-};
-
-const handler = (error, result) => {
-    if (error) {
-        // TODO: this handler should not be throwing as this blocks the queue
-        // throw error;
-    }
-    // Can potentially standardise the result here
-    return result;
-};
-
-class JobManager {
-    /**
-     * @param {Object} options
-     * @param {Function} [options.errorHandler] - custom job error handler
-     * @param {Function} [options.workerMessageHandler] - custom message handler coming from workers
-     * @param {Object} [options.JobModel] - a model which can persist job data in the storage
-     */
-    constructor({errorHandler, workerMessageHandler, JobModel}) {
-        this.queue = fastq(this, worker, 1);
-        this._jobMessageHandler = this._jobMessageHandler.bind(this);
-        this._jobErrorHandler = this._jobErrorHandler.bind(this);
-
-        const combinedMessageHandler = workerMessageHandler
-            ? ({name, message}) => {
-                workerMessageHandler({name, message});
-                this._jobMessageHandler({name, message});
-            }
-            : this._jobMessageHandler;
-
-        const combinedErrorHandler = errorHandler
-            ? (error, workerMeta) => {
-                errorHandler(error, workerMeta);
-                this._jobErrorHandler(error, workerMeta);
-            }
-            : this._jobErrorHandler;
-
-        this.bree = new Bree({
-            root: false, // set this to `false` to prevent requiring a root directory of jobs
-            hasSeconds: true, // precision is needed to avoid task overlaps after immediate execution
-            outputWorkerMetadata: true,
-            logger: logging,
-            errorHandler: combinedErrorHandler,
-            workerMessageHandler: combinedMessageHandler
-        });
-
-        this.bree.on('worker created', (name) => {
-            this._jobMessageHandler({name, message: 'started'});
-        });
-
-        this._jobsRepository = new JobsRepository({JobModel});
-    }
-
-    async _jobMessageHandler({name, message}) {
-        if (message === 'started') {
-            const job = await this._jobsRepository.read(name);
-
-            if (job) {
-                await this._jobsRepository.update(job.id, {
-                    status: 'started',
-                    started_at: new Date()
-                });
-            }
-        } else if (message === 'done') {
-            const job = await this._jobsRepository.read(name);
-
-            if (job) {
-                await this._jobsRepository.update(job.id, {
-                    status: 'finished',
-                    finished_at: new Date()
-                });
-            }
-        }
-    }
-
-    async _jobErrorHandler(error, workerMeta) {
-        const job = await this._jobsRepository.read(workerMeta.name);
-
-        if (job) {
-            await this._jobsRepository.update(job.id, {
-                status: 'failed'
-            });
-        }
-    }
-
-    /**
-     * By default schedules an "offloaded" job. If `offloaded: true` parameter is set,
-     * puts an "inline" immediate job into the queue.
-     *
-     * @param {Object} GhostJob - job options
-     * @prop {Function | String} GhostJob.job - function or path to a module defining a job
-     * @prop {String} [GhostJob.name] - unique job name, if not provided takes function name or job script filename
-     * @prop {String | Date} [GhostJob.at] - Date, cron or human readable schedule format. Manage will do immediate execution if not specified. Not supported for "inline" jobs
-     * @prop {Object} [GhostJob.data] - data to be passed into the job
-     * @prop {Boolean} [GhostJob.offloaded] - creates an "offloaded" job running in a worker thread by default. If set to "false" runs an "inline" job on the same event loop
-     */
-    addJob({name, at, job, data, offloaded = true}) {
-        if (offloaded) {
-            logging.info('Adding offloaded job to the queue');
-            let schedule;
-
-            if (!name) {
-                if (typeof job === 'string') {
-                    name = path.parse(job).name;
-                } else {
-                    throw new IncorrectUsageError({
-                        message: 'Name parameter should be present if job is a function'
-                    });
-                }
-            }
-
-            if (at && !(at instanceof Date)) {
-                if (isCronExpression(at)) {
-                    schedule = later.parse.cron(at, true);
-                } else {
-                    schedule = later.parse.text(at);
-                }
-
-                if ((schedule.error && schedule.error !== -1) || schedule.schedules.length === 0) {
-                    throw new IncorrectUsageError({
-                        message: 'Invalid schedule format'
-                    });
-                }
-
-                logging.info(`Scheduling job ${name} at ${at}. Next run on: ${later.schedule(schedule).next()}`);
-            } else if (at !== undefined) {
-                logging.info(`Scheduling job ${name} at ${at}`);
-            } else {
-                logging.info(`Scheduling job ${name} to run immediately`);
-            }
-
-            const breeJob = assembleBreeJob(at, job, data, name);
-            this.bree.add(breeJob);
-            return this.bree.start(name);
-        } else {
-            logging.info('Adding one off inline job to the queue');
-
-            this.queue.push(async () => {
-                try {
-                    if (typeof job === 'function') {
-                        await job(data);
-                    } else {
-                        await require(job)(data);
-                    }
-                } catch (err) {
-                    // NOTE: each job should be written in a safe way and handle all errors internally
-                    //       if the error is caught here jobs implementaton should be changed
-                    logging.error(new UnhandledJobError({
-                        context: (typeof job === 'function') ? 'function' : job,
-                        err
-                    }));
-
-                    throw err;
-                }
-            }, handler);
-        }
-    }
-
-    /**
-    * Adds a job that could ever be executed once.
-    *
-    * @param {Object} GhostJob - job options
-    * @prop {Function | String} GhostJob.job - function or path to a module defining a job
-    * @prop {String} [GhostJob.name] - unique job name, if not provided takes function name or job script filename
-    * @prop {String | Date} [GhostJob.at] - Date, cron or human readable schedule format. Manage will do immediate execution if not specified. Not supported for "inline" jobs
-    * @prop {Object} [GhostJob.data] - data to be passed into the job
-    * @prop {Boolean} [GhostJob.offloaded] - creates an "offloaded" job running in a worker thread by default. If set to "false" runs an "inline" job on the same event loop
-    */
-    async addOneOffJob({name, job, data, offloaded = true}) {
-        const persistedJob = await this._jobsRepository.read(name);
-
-        if (persistedJob) {
-            throw new IncorrectUsageError({
-                message: `A "${name}" one off job has already been executed.`
-            });
-        }
-
-        await this._jobsRepository.add({
-            name,
-            status: 'queued'
-        });
-
-        this.addJob({name, job, data, offloaded});
-    }
-
-    /**
-     * Removes an "offloaded" job from scheduled jobs queue.
-     * It's NOT yet possible to remove "inline" jobs (will be possible when scheduling is added https://github.com/breejs/bree/issues/68).
-     * The method will throw an Error if job with provided name does not exist.
-     *
-     * NOTE: current implementation does not guarante running job termination
-     *       for details see https://github.com/breejs/bree/pull/64
-     *
-     * @param {String} name - job name
-     */
-    async removeJob(name) {
-        await this.bree.remove(name);
-    }
-
-    /**
-     * @param {import('p-wait-for').Options} [options]
-     */
-    async shutdown(options) {
-        await this.bree.stop();
-
-        if (this.queue.idle()) {
-            return;
-        }
-
-        logging.warn('Waiting for busy job queue');
-
-        await pWaitFor(() => this.queue.idle() === true, options);
-
-        logging.warn('Job queue finished');
-    }
-}
-
-module.exports = JobManager;

package/lib/jobs-repository.js

@@ -1,23 +0,0 @@
-class JobsRepository {
-    constructor({JobModel}) {
-        this._JobModel = JobModel;
-    }
-
-    async add(data) {
-        const job = await this._JobModel.add(data);
-
-        return job;
-    }
-
-    async read(name) {
-        const job = await this._JobModel.findOne({name});
-
-        return job;
-    }
-
-    async update(id, data) {
-        await this._JobModel.edit(data, {id});
-    }
-}
-
-module.exports = JobsRepository;