@task-master/cadence 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +206 -110
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,179 +1,275 @@
1
1
  # @task-master/cadence
2
2
 
3
- Acquisition run lifecycle over MongoDB: scheduling, admission, publication, execution, reconciliation, and repair obligations.
3
+ MongoDB-backed orchestration for recurring data acquisition, vendor exports, and backfills.
4
4
 
5
- Cadence decides *what work should run, when, and whether it is safe to run again*. It does not fetch anything itself — you supply the adapters that talk to your vendor, your queue, and your storage. What Cadence owns is the part that is hard to get right: deciding admission under capacity and traffic limits, surviving a crash between "we told the vendor" and "we recorded that we told the vendor", and making sure a gap in the data becomes visible work rather than silence.
5
+ Cadence tracks what data is due, admits work within capacity and traffic limits, and records the outcome. When a process crashes or an external request has an unknown result, it reconciles the run using evidence from your queue and vendor. Vendor-reported coverage gaps become repair work.
6
6
 
7
- > **Status: 0.x.** The surface is still wide and will narrow once a real consumer has exercised it. Pin an exact version.
7
+ You provide the queue, vendor client, file storage, and processes that call the library.
8
8
 
9
- ## Install
9
+ > **Early release:** the public API may change during 0.x. Pin an exact version when integrating.
10
+
11
+ ## Installation
10
12
 
11
13
  ```sh
12
- npm install @task-master/cadence mongodb
14
+ npm install --save-exact @task-master/cadence@0.1.0 "mongodb@^6.10.0"
13
15
  ```
14
16
 
15
- `mongodb` is a **peer dependency**. Cadence hands out live `Collection` handles and passes `ClientSession` objects across the boundary, so you and Cadence must share one driver instance.
17
+ Requirements:
16
18
 
17
- Requires Node 20+ and a MongoDB replica set — several transitions are transactional.
19
+ - **Node.js 20 or later.**
20
+ - **MongoDB configured as a replica set.** A single-node replica set works for development; Cadence uses transactions.
21
+ - **MongoDB Node.js driver 6.10 or later within major version 6.** The driver is a peer dependency because collection handles and sessions cross the package boundary.
18
22
 
19
- Ships both ESM and CommonJS:
23
+ The package includes ESM, CommonJS, and TypeScript declarations:
20
24
 
21
25
  ```js
22
26
  import { connect, plan, admit } from '@task-master/cadence';
23
- const { connect, plan, admit } = require('@task-master/cadence');
24
27
  ```
25
28
 
26
- ## Shape of it
27
-
28
29
  ```js
29
- import { connect, ensureIndexes, plan, admit, publishOutbox, finalize, reconcile } from '@task-master/cadence';
30
-
31
- const c = await connect(process.env.MONGO_URI, 'cadence');
32
- await ensureIndexes(c); // once, before any traffic
33
-
34
- const { proposals, skipped } = await plan(c, 'vendor-a', new Date());
35
- for (const p of proposals) await admit(c, p, new Date());
30
+ const { connect, plan, admit } = require('@task-master/cadence');
36
31
  ```
37
32
 
38
- `plan` is read-only and tells you what *could* run, plus why each unit could not. `admit` re-asserts every decision inside a transaction, so a stale proposal is refused rather than acted on. From there a run moves through `publishOutbox` → your worker → `finalize`, and `reconcile` sweeps up whatever got stranded.
39
-
40
- `ensureIndexes` is not optional. The unique partial index on open requests is the only thing preventing duplicate work, and building it after duplicates exist will fail.
41
-
42
- ## Adapters
43
-
44
- You implement four interfaces: `Queue`, `Vendor`, `PollingVendor`, and `FileSink`.
45
-
46
- One contract matters more than the others. Your `Queue` must distinguish a definite refusal from an unknown outcome:
47
-
48
- ```js
49
- import { EnqueueRejected } from '@task-master/cadence';
33
+ ## Core concepts
34
+
35
+ | Concept | Meaning |
36
+ | --- | --- |
37
+ | Integration | A vendor endpoint, its capacity, capabilities, and traffic policy. |
38
+ | Unit | A tenant, practice, feed, or other item acquired on a recurring schedule. |
39
+ | Request | Work to fulfill: a scheduled acquisition, an ad hoc backfill, or a repair. |
40
+ | Run | One attempt to fulfill a request. Retries belong to the same request. |
41
+ | Watermark | The data boundary reached by scheduled acquisition. |
42
+ | Repair | A recorded coverage gap that still needs acquisition. |
43
+
44
+ ```text
45
+ plan -> admit -> publishOutbox -> submitKickoff -> pollRun
46
+ |
47
+ downloadNext
48
+ |
49
+ outcomeFor -> finalize
50
+
51
+ reconcile checks abandoned claims, uncertain requests, and stalled runs.
52
+ ```
50
53
 
51
- async enqueue(payload, { idempotencyKey }) {
52
- const res = await myQueue.submit(payload, idempotencyKey);
53
- if (res.status === 'invalid') throw new EnqueueRejected(res.reason); // definitely not accepted
54
- return { taskId: res.id }; // anything else thrown = unknown
54
+ ## Getting started
55
+
56
+ ### 1. Connect and define the work
57
+
58
+ This TypeScript example creates one integration and one daily unit in a new database. Set `MONGO_URI` to your replica set connection string, then run this setup once.
59
+
60
+ ```ts
61
+ import {
62
+ connect,
63
+ defaultIntegrationPolicy,
64
+ defaultTrafficPolicy,
65
+ type Integration,
66
+ type Unit,
67
+ } from '@task-master/cadence';
68
+
69
+ const uri = process.env.MONGO_URI;
70
+ if (!uri) throw new Error('Set MONGO_URI to a MongoDB replica set URI');
71
+
72
+ const c = await connect(uri, 'cadence');
73
+
74
+ const integration: Integration = {
75
+ _id: 'vendor-a',
76
+ quotaKey: 'vendor-a',
77
+ capacity: { budget: 4, reserved: 0, active: 0 },
78
+ trafficPolicy: defaultTrafficPolicy({ timezone: 'UTC' }),
79
+ vendorCooldownUntil: null,
80
+ profile: { until: false, typeFilter: false, patient: false },
81
+ // Use this classification only after verifying that repeating an acquisition is safe.
82
+ sideEffectClass: 'idempotent_acquisition',
83
+ sideEffectClassVerifiedBy: 'your-vendor-contract-review',
84
+ policy: defaultIntegrationPolicy({
85
+ defaultSince: new Date('2026-01-01T00:00:00.000Z'),
86
+ }),
87
+ };
88
+
89
+ const now = new Date();
90
+ const unit: Unit = {
91
+ _id: 'tenant-1',
92
+ integrationId: integration._id,
93
+ adminState: 'active',
94
+ executionState: 'idle',
95
+ availability: {
96
+ dispatchFrom: null,
97
+ dispatchUntil: null,
98
+ dataFrom: null,
99
+ dataUntil: null,
100
+ allowFinalCatchUp: false,
101
+ suspensions: [],
102
+ },
103
+ watermark: {
104
+ scheduledThrough: null, basis: null, seeded: false, updatedAt: null,
105
+ },
106
+ counters: {
107
+ scheduled: { consecutiveFailures: 0, emptyStreak: 0, totalRuns: 0 },
108
+ adhoc: { consecutiveFailures: 0, emptyStreak: 0, totalRuns: 0 },
109
+ repair: { consecutiveFailures: 0, emptyStreak: 0, totalRuns: 0 },
110
+ },
111
+ cadenceMs: 24 * 60 * 60_000,
112
+ scheduledCadenceAt: now,
113
+ scheduledNextEligibleAt: now,
114
+ activeRunId: null,
115
+ lastScheduledRunId: null,
116
+ lastAdhocRunId: null,
117
+ lastRepairRunId: null,
118
+ attention: [],
119
+ };
120
+
121
+ try {
122
+ await c.integrations.insertOne(integration);
123
+ await c.units.insertOne(unit);
124
+ } finally {
125
+ await c.client.close();
55
126
  }
56
127
  ```
57
128
 
58
- Cadence treats any other thrown error as *uncertain* and lets the reconciler resolve it on evidence. Getting this backwards — reporting a refusal as uncertainty — sends the reconciler hunting for a task that never existed.
129
+ Set capability flags to `true` only after verifying the vendor supports them. In particular, `profile.until` means the vendor honors an upper time bound; without it, Cadence cannot enforce `maxCatchUpWindowMs` against that vendor.
59
130
 
60
- If you need to detect these errors yourself, use the guards rather than `instanceof`:
131
+ `connect()` creates the required indexes. If you construct a `Cadence` handle with your own `MongoClient`, call `ensureIndexes(c)` before starting any work. The unique indexes are required for deduplication.
61
132
 
62
- ```js
63
- import { isEnqueueRejected, isEnqueueUncertain } from '@task-master/cadence';
64
- ```
133
+ ### 2. Plan and admit work
65
134
 
66
- Identity is structural, so it survives two copies of the package in one dependency tree. `instanceof` does not.
135
+ In your scheduler process, open a connection with `connect()` and run a tick like this. The following snippets use that open handle as `c`.
67
136
 
68
- ## Design notes
137
+ ```ts
138
+ import { admit, plan } from '@task-master/cadence';
69
139
 
70
- A few decisions are load-bearing and non-obvious:
140
+ const { proposals, skipped } = await plan(c, 'vendor-a', new Date());
71
141
 
72
- - **A failed download is not partial coverage.** `partial` carries time intervals; a transport failure says nothing about which part of the window the vendor actually searched. Only a vendor-reported gap produces `partial`.
73
- - **Polling is not progress.** A job polled a thousand times with nothing new to report has not progressed, or the no-progress deadline would never fire. The *first* report is news; identical repeats are not.
74
- - **A capped catch-up window is not a gap.** The marker stops at the capped boundary and the next scheduled window resumes exactly there.
75
- - **Attention is a list.** Three unrelated problems can be true about one unit at once.
142
+ for (const diagnostic of skipped) console.log('Skipped:', diagnostic);
76
143
 
77
- The full design — problem statement, architecture, schemas, invariants, and the review history behind them — lives in `docs/` in the repository, along with `docs/RECIPES.md`, which answers the questions that come up when wiring this in: how many processes to run, how to classify vendor failures, what to alert on, why a quiet unit is quiet. Neither is published to npm.
144
+ for (const proposal of proposals) {
145
+ const result = await admit(c, proposal, new Date());
146
+ if (!result.admitted) {
147
+ console.log('Admission refused:', result.reason);
148
+ if (result.reason === 'no_capacity') break;
149
+ }
150
+ }
151
+ ```
78
152
 
79
- ## The four roles
153
+ `plan()` reads state without reserving capacity. `admit()` checks the proposal again inside a transaction and records admitted work in the outbox. Repeat the tick at an interval appropriate to your application.
80
154
 
81
- Cadence is a library, not a daemon — you run the loops. There are four, with different cardinalities:
155
+ ### 3. Implement your adapters
82
156
 
83
- | Role | How many | Interval |
84
- |---|---|---|
85
- | Scheduler | one per integration is simplest | seconds to minutes |
86
- | Publisher | many; they compete safely | continuous |
87
- | Worker | many; they compete safely | continuous |
88
- | Reconciler | **exactly one** | a minute or more |
157
+ Import the interfaces from the package root:
89
158
 
90
- Publishers and workers claim work with a token and a TTL, so running ten of each is fine. The reconciler resolves uncertainty and should not race itself. Two schedulers on one integration will not corrupt anything — the unique index on open requests refuses the second admit — but one of them logs a refusal every tick.
159
+ ```ts
160
+ import type { Queue, Vendor, PollingVendor, FileSink } from '@task-master/cadence';
161
+ ```
91
162
 
92
- Each loop below is condensed from `examples/` in the repository, which is typechecked with the package so it cannot drift from the API.
163
+ | Adapter | Required method | Your implementation |
164
+ | --- | --- | --- |
165
+ | `Queue` | `enqueue(payload, { idempotencyKey })` | Enqueue work and return `{ taskId }`. Pass the stable idempotency key to your queue. |
166
+ | `Vendor` | `kickoff(run)` | Start the vendor job and return `{ jobRef, contentLocation }`. Use `run._id` as the vendor request key where supported. |
167
+ | `PollingVendor` | `poll(run)` | Map vendor responses to `accepted`, `complete`, `throttled`, `expired`, or `failed`. |
168
+ | `FileSink` | `fetch(entry, run)` | Download and persist one manifest entry, then return `{ bytes, count }`. |
93
169
 
94
- ### Scheduler
170
+ The queue adapter must distinguish a definite refusal from an unknown result:
95
171
 
96
- ```js
97
- const { proposals, skipped } = await plan(c, integrationId, new Date());
172
+ | Enqueue result | Adapter behavior |
173
+ | --- | --- |
174
+ | Definitely accepted | Return `{ taskId }`. |
175
+ | Definitely refused; no task created | Throw `EnqueueRejected`. |
176
+ | May have been accepted, such as a timeout after sending | Throw `EnqueueUncertain`, or another error. Cadence treats all other thrown errors as uncertain. |
98
177
 
99
- // `skipped` is the only place the answer to "why is this unit quiet?" exists. Log it.
100
- for (const s of skipped) count(s.reason);
178
+ Both error classes are exported from `@task-master/cadence`. Classify responses according to your queue's actual contract. Reporting an unknown result as a rejection can cause duplicate work. If you inspect these errors yourself, use the exported `isEnqueueRejected` and `isEnqueueUncertain` guards; they work across separate package copies.
101
179
 
102
- for (const p of proposals) {
103
- const result = await admit(c, p, new Date());
104
- if (!result.admitted && result.reason === 'no_capacity') break;
105
- }
106
- ```
180
+ ### 4. Publish and execute runs
107
181
 
108
- ### Publisher
182
+ Call `publishOutbox()` repeatedly from a publisher process with your `Queue` adapter:
183
+
184
+ ```ts
185
+ import { DEFAULTS, publishOutbox } from '@task-master/cadence';
109
186
 
110
- ```js
111
187
  const result = await publishOutbox(c, queue, new Date(), {
112
- instanceId,
188
+ instanceId: 'publisher-1',
113
189
  claimTtlMs: DEFAULTS.outboxClaimTtlMs,
114
190
  });
115
-
116
- if (!result.published && result.reason === 'nothing_pending') await sleep(1000);
117
- // 'claim_lost' means a peer took it; 'uncertain' is already parked for the reconciler.
118
191
  ```
119
192
 
120
- ### Worker
193
+ Each call attempts to publish one pending entry. `queue` is your adapter. Back off when the result is `{ published: false, reason: 'nothing_pending' }`.
121
194
 
122
- ```js
123
- const worker = { workerId, claimTtlMs: DEFAULTS.submissionClaimTtlMs };
124
- const exec = { workerId, entryClaimTtlMs: 300_000 };
125
-
126
- const submit = await submitKickoff(c, vendor, integration, runId, new Date(), worker);
127
- if (submit.status !== 'submitted' && submit.status !== 'already_confirmed') return;
128
-
129
- for (;;) {
130
- const step = await pollRun(c, poller, integration, runId, new Date(), exec);
131
- if (step.status === 'terminal') return finalize(c, runId, step.outcome, new Date());
132
- if (step.status === 'manifest_ready') break;
133
- if (step.status === 'not_pollable') return;
134
- await sleep(15_000);
135
- }
195
+ Each queued payload contains `{ runId, unitKey, kind }`. When your queue delivers a task, load the run from `c.runs` using `runId`, then load its integration from `c.integrations` using the run's `integrationId`. Your worker calls the following functions in order, using your `Vendor`, `PollingVendor`, and `FileSink` adapters:
136
196
 
137
- // Each call claims exactly one manifest entry, so several workers may share a manifest.
138
- for (;;) {
139
- const step = await downloadNext(c, sink, integration, runId, new Date(), exec);
140
- if (step.status === 'nothing_claimable') break;
141
- }
197
+ | Call | How to handle the result |
198
+ | --- | --- |
199
+ | `submitKickoff(c, vendor, integration, runId, now, workerOptions)` | Continue on `submitted` or `already_confirmed`. Reschedule `deferred` work; other statuses require yielding ownership or reconciliation. |
200
+ | `pollRun(c, poller, integration, runId, now, executeOptions)` | Poll with a delay while `working`; honor `deferred` results. On `terminal`, pass its `outcome` to `finalize()`. On `manifest_ready`, start downloads. Stop on `not_pollable`. |
201
+ | `downloadNext(c, sink, integration, runId, now, executeOptions)` | Claims and downloads one entry per call. Repeat, delaying on `deferred`, until `nothing_claimable`. |
202
+ | `outcomeFor(c, runId)` | Returns an outcome when the manifest is finished, or `null` while entries remain outstanding. Revisit unfinished runs. |
203
+ | `finalize(c, runId, outcome, now)` | Applies the outcome, updates request and unit state, and releases capacity. |
204
+
205
+ Use a fresh `new Date()` for each call's `now` argument. The option shapes are:
142
206
 
143
- // Null while any entry is still outstanding — including one another worker holds.
144
- const outcome = await outcomeFor(c, runId);
145
- if (outcome) await finalize(c, runId, outcome, new Date());
207
+ ```ts
208
+ const workerOptions = { workerId: 'worker-1', claimTtlMs: 600_000 };
209
+ const executeOptions = { workerId: 'worker-1', entryClaimTtlMs: 300_000 };
146
210
  ```
147
211
 
148
- ### Reconciler
212
+ Your application owns queue acknowledgment, redelivery, delays, and shutdown. A deferred step needs another execution attempt. `nothing_claimable` can also mean another worker holds the remaining entries, so always check `outcomeFor()` before finalizing.
213
+
214
+ ### 5. Reconcile interrupted work
215
+
216
+ Run one reconciler periodically, for example once a minute:
217
+
218
+ ```ts
219
+ import { DEFAULTS, reconcile } from '@task-master/cadence';
149
220
 
150
- ```js
151
221
  const report = await reconcile(c, new Date(), {
152
222
  noProgressDeadlineMs: DEFAULTS.noProgressDeadlineMs,
153
223
  publicationResolutionGraceMs: 15 * 60_000,
154
224
  }, { vendor, queue });
225
+
226
+ if (report.submissionsEscalated || report.publicationsEscalated || report.runsBlocked) {
227
+ console.error('Cadence needs operator attention:', report);
228
+ }
155
229
  ```
156
230
 
157
- Alert on `submissionsEscalated`, `publicationsEscalated` and `runsBlocked` — each means something needed a decision the engine could not make on evidence.
231
+ Recovery is more effective when your adapters support lookups:
158
232
 
159
- Both deps are optional and both are worth providing. `vendor.findByClientRequestId` lets a crashed worker adopt its own job instead of escalating; `queue.findByIdempotencyKey` lets an uncertain publication be resolved instead of escalated.
233
+ - `vendor.findByClientRequestId(runId)` returns `{ jobRef, contentLocation }` or `null`, allowing a submission interrupted by a crash to be recovered.
234
+ - The queue passed to `reconcile()` may also implement `findByIdempotencyKey(key)`, returning `{ taskId }` or `null` to resolve uncertain publication.
160
235
 
161
- ## Operating it
236
+ These lookups are optional. When the outcome cannot be established, Cadence may escalate or block work for an operator to resolve.
162
237
 
163
- The engine has no opinion about your UI, so the read side is plain Mongo against the `Cadence` handle:
238
+ ## Running in production
164
239
 
165
- ```js
166
- await c.units.find({ integrationId, attention: { $not: { $size: 0 } } }).toArray(); // needs a human
167
- await c.runs.find({ integrationId, state: 'blocked' }).toArray(); // waiting on evidence
168
- await c.repairs.find({ integrationId, state: 'open' }).toArray(); // gaps still owed
169
- await c.repairs.find({ integrationId, state: 'accepted' }).toArray(); // permanent data loss
240
+ The four roles can share a process or run separately:
241
+
242
+ | Role | Concurrency |
243
+ | --- | --- |
244
+ | Scheduler | One per integration is simplest. Admission rechecks proposals and unique indexes guard against duplicates. |
245
+ | Publisher | Multiple instances can compete for claimed outbox entries. |
246
+ | Worker | Multiple instances can claim submission work and individual manifest entries. |
247
+ | Reconciler | Run exactly one, with no overlapping sweeps. |
248
+
249
+ Keep MongoDB connections open for each process and close them with `await c.client.close()` during shutdown.
250
+
251
+ Inspect operational state using the collections on `c`:
252
+
253
+ ```ts
254
+ const needsAttention = await c.units.find({
255
+ attention: { $not: { $size: 0 } },
256
+ }).toArray();
257
+ const blockedRuns = await c.runs.find({ state: 'blocked' }).toArray();
258
+ const openRepairs = await c.repairs.find({ state: 'open' }).toArray();
259
+ const abandonedRepairs = await c.repairs.find({ state: 'accepted' }).toArray();
170
260
  ```
171
261
 
172
- The last one is a page, not a log line: an accepted repair is a gap the engine gave up on.
262
+ Monitor escalations, blocked runs, open attention episodes, and accepted repairs. An accepted repair is a gap the engine has stopped trying to fill.
263
+
264
+ Use `patchAvailability`, `addSuspension`, `removeSuspension`, and `setTrafficPolicy` to change scheduling policy; these helpers refresh eligibility hints. After changing a unit's `adminState` directly, call `refreshEligibility`. Leave `executionState` to the engine.
173
265
 
174
- Overrides go through small helpers — `patchAvailability`, `addSuspension`, `removeSuspension`, `setTrafficPolicy`, `blockRun`, `unblockRun`, `raiseAttention`, `clearAttention`. Each re-derives the unit's eligibility hint, which is why you should not write `availability` directly.
266
+ ## Outcome and retry behavior
175
267
 
176
- `adminState` is the operator's switch and the engine never writes it. `executionState` is the engine's and an operator never should.
268
+ - A failed download is a failed attempt. Report `partial` only when the vendor reports a coverage gap.
269
+ - Repeating an unchanged polling response does not count as progress. Pass through the vendor's progress note so stalled work can be detected.
270
+ - A catch-up window capped by policy leaves later data for the next scheduled run. The cap alone does not create a repair.
271
+ - A blocked run is nonterminal and retains its reservation while evidence is outstanding.
272
+ - Inspect both `plan().skipped` and stored unit state when diagnosing missing work; units excluded by the planner's initial query do not appear in `skipped`.
177
273
 
178
274
  ## License
179
275
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@task-master/cadence",
3
- "version": "0.1.0",
3
+ "version": "0.2.0",
4
4
  "type": "module",
5
5
  "description": "Acquisition run lifecycle: scheduling, admission, publication, execution, reconciliation and repair obligations over MongoDB.",
6
6
  "keywords": [