@maroonedsoftware/jobbroker 1.11.0 → 1.11.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +227 -0
- package/package.json +4 -3
package/AGENTS.md
ADDED
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
# AGENTS.md — @maroonedsoftware/jobbroker
|
|
2
|
+
|
|
3
|
+
Machine-oriented guide for AI agents. Human prose and long-form examples live in [README.md](./README.md).
|
|
4
|
+
Repo-wide conventions live in the [root AGENTS.md](../../AGENTS.md).
|
|
5
|
+
|
|
6
|
+
## Purpose
|
|
7
|
+
|
|
8
|
+
Durable background jobs behind a backend-agnostic interface: `JobBroker` for the producer side
|
|
9
|
+
(send, schedule, cancel, inspect), `Job` for the handler, `JobRunner` for the worker lifecycle, and
|
|
10
|
+
`JobMonitor` for queue stats, dead-letter listing, and redrive. `JobContext` is the job-side
|
|
11
|
+
counterpart of a request context — inject it and any service in the graph knows which job it is
|
|
12
|
+
running under. The bundled backend is pg-boss over PostgreSQL.
|
|
13
|
+
|
|
14
|
+
Reach for this when work must survive a process restart, be retried, be scheduled, or happen after
|
|
15
|
+
the response. Do **not** reach for it for synchronous in-process fan-out — that is
|
|
16
|
+
`@maroonedsoftware/eventbus`.
|
|
17
|
+
|
|
18
|
+
## Install
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
pnpm add @maroonedsoftware/jobbroker
|
|
22
|
+
pnpm add pg-boss # only for the ./pgboss backend
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Runtime dependencies: `@maroonedsoftware/errors`, `@maroonedsoftware/logger`, `injectkit`, `luxon`.
|
|
26
|
+
Optional peer: `pg-boss`.
|
|
27
|
+
|
|
28
|
+
## Position in the graph
|
|
29
|
+
|
|
30
|
+
- **Depends on:** `errors`, `logger`.
|
|
31
|
+
- **Depended on by:** nothing internal. It is a leaf that applications wire up directly.
|
|
32
|
+
- **Subpath exports:**
|
|
33
|
+
- `.` — the backend-agnostic abstractions. Loads no `pg-boss`.
|
|
34
|
+
- `./pgboss` — the pg-boss backend. Pulls in the optional peer.
|
|
35
|
+
|
|
36
|
+
Notably **not** a dependency: `kysely`. Transactional enqueue is supported through a _structural_
|
|
37
|
+
`KyselyLike` interface, so this package needs no dependency on `kysely`, not even an optional peer.
|
|
38
|
+
|
|
39
|
+
## API surface
|
|
40
|
+
|
|
41
|
+
### `.` — abstractions
|
|
42
|
+
|
|
43
|
+
| Export | Kind | Shape | Notes |
|
|
44
|
+
| ------------------------------------ | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
|
|
45
|
+
| `Job<Payload>` | abstract class | `@Injectable() abstract run(payload: Payload, signal?: AbortSignal): Promise<void>` | `Payload extends object = object`. |
|
|
46
|
+
| `JobBroker` | abstract class | `@Injectable()`. Producer side. | Unsupported operations throw `NotSupportedError` rather than no-op. |
|
|
47
|
+
| `#send` | method | `<P extends object>(name: string, payload: P, options?: JobSendOptions) => Promise<string>` | Returns the job id. Throws if `name` is not registered. |
|
|
48
|
+
| `#schedule` | method | `<P extends object>(name: string, cron: string, payload?: P) => Promise<void>` | — |
|
|
49
|
+
| `#unschedule` | method | `(name: string) => Promise<void>` | — |
|
|
50
|
+
| `#cancel` / `#resume` / `#deleteJob` | method | `(name: string, id: string \| string[]) => Promise<void>` | `cancel` aborts the signal for a running job. |
|
|
51
|
+
| `#getJob` | method | `<P extends object>(name: string, id: string) => Promise<JobInfo<P> \| null>` | — |
|
|
52
|
+
| `JobRunner` | abstract class | `@Injectable()`. `start()`, `stop()` | Consumer side. `stop()` waits for in-flight jobs. |
|
|
53
|
+
| `JobMonitor` | abstract class | `@Injectable()`. `getQueueStats`, `listJobs`, `redrive`, `deleteJob`, `retryJob` | Operational surface — dashboards, dead-letter drains. |
|
|
54
|
+
| `JobContext` | interface + abstract class | `{ id: string; name: string; signal: AbortSignal; expiresIn?: Duration }` | Declaration-merged so one symbol is both the type and the DI token. |
|
|
55
|
+
| `registerJobContext` | function | `<T extends Registry>(registry: T) => T` | **Required** if any service injects `JobContext`. Registers a throwing placeholder. |
|
|
56
|
+
| `JobSendOptions` | interface | `{ startAfter?: Duration \| DateTime }` | Relative delay or absolute earliest-run time. A lower bound, not exact scheduling. |
|
|
57
|
+
| `JobQueuePolicy` | interface | `{ retryLimit?, retryDelay?: Duration, retryBackoff?, retryDelayMax?: Duration, expiresIn?: Duration, deadLetter?: string }` | Applied per queue when the runner starts. `expiresIn` is what surfaces as `JobContext.expiresIn`. |
|
|
58
|
+
| `JobInfo<Payload>` | interface | `{ id, name, state: JobState, data: Payload }` | Lowest common denominator across backends. |
|
|
59
|
+
| `JobState` | type | `'created' \| 'retry' \| 'active' \| 'completed' \| 'cancelled' \| 'failed'` | — |
|
|
60
|
+
| `JobQueueStats` | interface | `{ name, queued, active, failed, total }` | `failed` is a **retained** count, not an all-time total. |
|
|
61
|
+
| `JobQueryOptions` | interface | `{ id?, data?: Record<string, unknown>, queuedOnly? }` | `data` is a partial-match payload filter. |
|
|
62
|
+
| `JobRedriveOptions` | interface | `{ destination?, sourceName?, limit? }` | Without `destination`, each job returns to its original queue. |
|
|
63
|
+
| `NotSupportedError` | class | `extends ServerkitError` | Thrown by a backend that cannot honour an operation. |
|
|
64
|
+
|
|
65
|
+
### `./pgboss`
|
|
66
|
+
|
|
67
|
+
| Export | Kind | Shape | Notes |
|
|
68
|
+
| ------------------------------------- | --------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |
|
|
69
|
+
| `PgBossJobBroker` | class | `extends JobBroker` | Supports every `JobBroker` operation. |
|
|
70
|
+
| `PgBossJobRunner` | class | `extends JobRunner` | Creates the per-execution scope and registers the live `JobContext`. |
|
|
71
|
+
| `PgBossJobMonitor` | class | `extends JobMonitor` | — |
|
|
72
|
+
| `PgBossJobRegistration` | type | `{ job: Identifier<Job>; cron?: string; policy?: JobQueuePolicy }` | — |
|
|
73
|
+
| `PgBossJobRegistryMap` | class | `extends Map<string, Identifier<Job> \| PgBossJobRegistration>` | Job name → handler. Read by both the broker (validation) and the runner. |
|
|
74
|
+
| `PgBossConnectionProvider` | class | `@Injectable()`. `executor(): Db \| undefined` | Default returns `undefined` — pg-boss uses its own pool. |
|
|
75
|
+
| `KyselyTransactionConnectionProvider` | class | `extends PgBossConnectionProvider`. `new (trx: KyselyLike)` | Enqueue inside the caller's transaction. Strips Kysely plugins first. |
|
|
76
|
+
| `KyselyLike` | interface | `{ withoutPlugins(): … }` | Structural, so no `kysely` dependency. Accepts `Kysely<DB>` and `Transaction<DB>`. |
|
|
77
|
+
|
|
78
|
+
## Canonical usage
|
|
79
|
+
|
|
80
|
+
```typescript
|
|
81
|
+
import { Injectable, InjectKitRegistry } from 'injectkit';
|
|
82
|
+
import { Job, JobBroker, JobRunner, JobContext, registerJobContext } from '@maroonedsoftware/jobbroker';
|
|
83
|
+
import { PgBossJobBroker, PgBossJobRunner, PgBossJobRegistryMap } from '@maroonedsoftware/jobbroker/pgboss';
|
|
84
|
+
import { Duration } from 'luxon';
|
|
85
|
+
|
|
86
|
+
interface SendReceiptPayload {
|
|
87
|
+
orderId: string;
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
@Injectable()
|
|
91
|
+
class SendReceiptJob extends Job<SendReceiptPayload> {
|
|
92
|
+
constructor(private readonly mailer: Mailer) {
|
|
93
|
+
super();
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
async run(payload: SendReceiptPayload, signal?: AbortSignal): Promise<void> {
|
|
97
|
+
await this.mailer.sendReceipt(payload.orderId, { signal });
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
// Composition root
|
|
102
|
+
const jobs = new PgBossJobRegistryMap();
|
|
103
|
+
jobs.set('order.receipt.send', SendReceiptJob);
|
|
104
|
+
jobs.set('report.daily', { job: DailyReportJob, cron: '0 9 * * *' });
|
|
105
|
+
jobs.set('webhook.deliver', {
|
|
106
|
+
job: DeliverWebhookJob,
|
|
107
|
+
policy: { retryLimit: 5, retryDelay: Duration.fromObject({ seconds: 30 }), retryBackoff: true, deadLetter: 'webhook.deliver.dead' },
|
|
108
|
+
});
|
|
109
|
+
|
|
110
|
+
registerJobContext(registry); // required because services inject JobContext
|
|
111
|
+
registry.register(PgBossJobRegistryMap).useValue(jobs);
|
|
112
|
+
registry.register(SendReceiptJob).useClass(SendReceiptJob).asTransient();
|
|
113
|
+
registry.register(JobBroker).useClass(PgBossJobBroker).asSingleton();
|
|
114
|
+
registry.register(JobRunner).useClass(PgBossJobRunner).asSingleton();
|
|
115
|
+
|
|
116
|
+
// Enqueue
|
|
117
|
+
const id = await broker.send('order.receipt.send', { orderId }, { startAfter: Duration.fromObject({ minutes: 5 }) });
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Transactional enqueue — the job row commits with the business writes or not at all:
|
|
121
|
+
|
|
122
|
+
```typescript
|
|
123
|
+
import { PgBossConnectionProvider, KyselyTransactionConnectionProvider } from '@maroonedsoftware/jobbroker/pgboss';
|
|
124
|
+
|
|
125
|
+
await db.transaction().execute(async trx => {
|
|
126
|
+
const scope = container.createScope();
|
|
127
|
+
scope.override(PgBossConnectionProvider, new KyselyTransactionConnectionProvider(trx));
|
|
128
|
+
|
|
129
|
+
await orders.create(trx, order);
|
|
130
|
+
await scope.get(JobBroker).send('order.receipt.send', { orderId: order.id });
|
|
131
|
+
});
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
See [.claude/skills/job](../../.claude/skills/job) for the generator and its examples.
|
|
135
|
+
|
|
136
|
+
## Rules for generated code
|
|
137
|
+
|
|
138
|
+
- Job names are catalog keys: dot notation, no hyphens (`order.receipt.send`, `report.daily`). The
|
|
139
|
+
name is also the queue name.
|
|
140
|
+
- Register job classes as **transient**. A new instance per execution is what makes injecting
|
|
141
|
+
request-shaped collaborators safe.
|
|
142
|
+
- Call `registerJobContext(registry)` before `build()` whenever any registered service injects
|
|
143
|
+
`JobContext`. `Registry.build()` validates the graph up front, and the runner's per-execution
|
|
144
|
+
override does not exist yet at that point.
|
|
145
|
+
- **Honour the `AbortSignal`.** Forward it to `fetch`, database calls, and long loops, or check
|
|
146
|
+
`signal.aborted` between units of work. A handler that ignores it cannot be cancelled and blocks
|
|
147
|
+
shutdown until it finishes.
|
|
148
|
+
- Keep payloads small and JSON-serialisable — they are stored as a database column. Pass an id, not
|
|
149
|
+
an entity.
|
|
150
|
+
- Make handlers idempotent. At-least-once delivery plus retries means a job can run twice.
|
|
151
|
+
- Use `startAfter` with a Luxon `Duration` or `DateTime`, never a millisecond number.
|
|
152
|
+
- Give any queue that talks to an external service a `JobQueuePolicy` with `retryBackoff` and a
|
|
153
|
+
`deadLetter` queue. Without one you get the backend's defaults and no dead-letter capture.
|
|
154
|
+
- Import pg-boss classes from `@maroonedsoftware/jobbroker/pgboss`, never from the root.
|
|
155
|
+
- Wire `runner.start()` into a module's `ready` hook and `runner.stop()` into `shutdown`. Starting
|
|
156
|
+
in `start` delays boot for every module after it.
|
|
157
|
+
- Read the retry count from `JobMonitor`, not from `JobContext` — it is deliberately not there.
|
|
158
|
+
|
|
159
|
+
## Gotchas
|
|
160
|
+
|
|
161
|
+
- **`JobContext` throws when resolved outside a job.** `registerJobContext` registers a placeholder
|
|
162
|
+
that throws with an explanation; the runner's scoped override shadows it during an execution. A
|
|
163
|
+
service that must work in both a request and a job must not depend on `JobContext` directly.
|
|
164
|
+
- **`JobContext` carries no attempt or retry counter.** pg-boss hands the work handler a bare `Job`
|
|
165
|
+
and the retry count lives on `JobWithMetadata`, which would cost an extra query per execution.
|
|
166
|
+
This is a deliberate omission, not an oversight — use `JobMonitor`.
|
|
167
|
+
- **`startAfter` is a lower bound.** The job runs at the first opportunity after it becomes
|
|
168
|
+
eligible, which under load can be much later. It is not a scheduler.
|
|
169
|
+
- **A backend that cannot honour an option throws `NotSupportedError`** rather than silently
|
|
170
|
+
ignoring it. SQS caps delays at 15 minutes, for instance. pg-boss supports everything, so this
|
|
171
|
+
only bites on a backend swap.
|
|
172
|
+
- **`JobQueueStats.failed` is a retained count**, bounded by the queue's retention policy, not an
|
|
173
|
+
all-time total. It goes down on its own.
|
|
174
|
+
- **`KyselyTransactionConnectionProvider` strips plugins before adapting the transaction.**
|
|
175
|
+
pg-boss runs raw SQL through `executeQuery`, so a camelCase plugin on your Kysely instance would
|
|
176
|
+
rewrite pg-boss's _own_ result columns and corrupt them. `withoutPlugins()` returns an instance
|
|
177
|
+
sharing the same connection-bound executor, so atomicity survives. Do not remove that call.
|
|
178
|
+
- **The default `PgBossConnectionProvider.executor()` returns `undefined`**, meaning pg-boss uses
|
|
179
|
+
its own pool. Enqueues are therefore **not** transactional by default: a job can commit while the
|
|
180
|
+
surrounding business transaction rolls back. Overriding it per request is opt-in.
|
|
181
|
+
- **`JobBroker.deleteJob` and `JobMonitor.deleteJob` are different methods on different classes**
|
|
182
|
+
with the same name and signature. Check which token you are holding.
|
|
183
|
+
- **`cancel` on a running job aborts the signal** — it does not kill the handler. A handler that
|
|
184
|
+
ignores the signal runs to completion regardless.
|
|
185
|
+
- **`JobContext` is declaration-merged** (interface + abstract class, same name), like `Logger`.
|
|
186
|
+
That is what makes one symbol serve as both a type and a runtime DI token. Do not split it.
|
|
187
|
+
|
|
188
|
+
## Working inside this package
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
src/
|
|
192
|
+
index.ts Root barrel — abstractions only
|
|
193
|
+
job.ts Job
|
|
194
|
+
job.broker.ts JobBroker
|
|
195
|
+
job.runner.ts JobRunner
|
|
196
|
+
job.monitor.ts JobMonitor, JobQueueStats, JobQueryOptions, JobRedriveOptions
|
|
197
|
+
job.context.ts JobContext (interface + token), registerJobContext
|
|
198
|
+
job.info.ts JobState, JobInfo
|
|
199
|
+
job.send.options.ts JobSendOptions
|
|
200
|
+
job.queue.policy.ts JobQueuePolicy
|
|
201
|
+
not.supported.error.ts NotSupportedError
|
|
202
|
+
pgboss.ts Subpath entry for ./pgboss
|
|
203
|
+
pgboss/
|
|
204
|
+
pgboss.job.broker.ts
|
|
205
|
+
pgboss.job.runner.ts
|
|
206
|
+
pgboss.job.monitor.ts
|
|
207
|
+
pgboss.job.registration.ts PgBossJobRegistration, PgBossJobRegistryMap
|
|
208
|
+
pgboss.connection.provider.ts PgBossConnectionProvider
|
|
209
|
+
kysely.transaction.connection.provider.ts KyselyTransactionConnectionProvider, KyselyLike
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Tests are in `tests/`, mirroring `src/`.
|
|
213
|
+
|
|
214
|
+
Invariants a change must not break:
|
|
215
|
+
|
|
216
|
+
- **Nothing reachable from `src/index.ts` may import `pg-boss`.** That is the point of the
|
|
217
|
+
`./pgboss` entry.
|
|
218
|
+
- **No dependency on `kysely`, not even optional.** `KyselyLike` is structural on purpose.
|
|
219
|
+
- The abstractions must stay lowest-common-denominator. A field only pg-boss can supply belongs on
|
|
220
|
+
the pg-boss types, not on `JobInfo` or `JobContext` — that is why there is no attempt counter.
|
|
221
|
+
- An operation a backend cannot support throws `NotSupportedError`; it never silently no-ops.
|
|
222
|
+
- The runner owns creating the per-execution scope and registering the live `JobContext` against
|
|
223
|
+
the token. Any new backend must do the same or `JobContext` injection breaks.
|
|
224
|
+
- A new backend gets its own `src/<name>.ts` entry, an `exports` entry, a tsup entry in the `build`
|
|
225
|
+
script, and its client under `peerDependenciesMeta` as optional.
|
|
226
|
+
|
|
227
|
+
User-visible changes need a changeset in `.changeset/`.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@maroonedsoftware/jobbroker",
|
|
3
|
-
"version": "1.11.
|
|
3
|
+
"version": "1.11.3",
|
|
4
4
|
"description": "A flexible background job processing library with support for scheduled and on-demand jobs.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Marooned Software",
|
|
@@ -42,13 +42,14 @@
|
|
|
42
42
|
},
|
|
43
43
|
"license": "MIT",
|
|
44
44
|
"files": [
|
|
45
|
+
"AGENTS.md",
|
|
45
46
|
"dist/**"
|
|
46
47
|
],
|
|
47
48
|
"dependencies": {
|
|
48
49
|
"injectkit": "^1.6.0",
|
|
49
50
|
"luxon": "^3.7.2",
|
|
50
|
-
"@maroonedsoftware/errors": "1.8.
|
|
51
|
-
"@maroonedsoftware/logger": "1.1.
|
|
51
|
+
"@maroonedsoftware/errors": "1.8.3",
|
|
52
|
+
"@maroonedsoftware/logger": "1.1.6"
|
|
52
53
|
},
|
|
53
54
|
"devDependencies": {
|
|
54
55
|
"@types/luxon": "^3.7.2",
|