@task-master/cadence 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +180 -3
- package/dist/index.cjs +1957 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +717 -0
- package/dist/index.d.ts +717 -0
- package/dist/index.js +1882 -0
- package/dist/index.js.map +1 -0
- package/package.json +64 -6
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Task Master
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,180 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
|
|
1
|
+
# @task-master/cadence
|
|
2
|
+
|
|
3
|
+
Acquisition run lifecycle over MongoDB: scheduling, admission, publication, execution, reconciliation, and repair obligations.
|
|
4
|
+
|
|
5
|
+
Cadence decides *what work should run, when, and whether it is safe to run again*. It does not fetch anything itself — you supply the adapters that talk to your vendor, your queue, and your storage. What Cadence owns is the part that is hard to get right: deciding admission under capacity and traffic limits, surviving a crash between "we told the vendor" and "we recorded that we told the vendor", and making sure a gap in the data becomes visible work rather than silence.
|
|
6
|
+
|
|
7
|
+
> **Status: 0.x.** The surface is still wide and will narrow once a real consumer has exercised it. Pin an exact version.
|
|
8
|
+
|
|
9
|
+
## Install
|
|
10
|
+
|
|
11
|
+
```sh
|
|
12
|
+
npm install @task-master/cadence mongodb
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
`mongodb` is a **peer dependency**. Cadence hands out live `Collection` handles and passes `ClientSession` objects across the boundary, so you and Cadence must share one driver instance.
|
|
16
|
+
|
|
17
|
+
Requires Node 20+ and a MongoDB replica set — several transitions are transactional.
|
|
18
|
+
|
|
19
|
+
Ships both ESM and CommonJS:
|
|
20
|
+
|
|
21
|
+
```js
|
|
22
|
+
import { connect, plan, admit } from '@task-master/cadence';
|
|
23
|
+
const { connect, plan, admit } = require('@task-master/cadence');
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Shape of it
|
|
27
|
+
|
|
28
|
+
```js
|
|
29
|
+
import { connect, ensureIndexes, plan, admit, publishOutbox, finalize, reconcile } from '@task-master/cadence';
|
|
30
|
+
|
|
31
|
+
const c = await connect(process.env.MONGO_URI, 'cadence');
|
|
32
|
+
await ensureIndexes(c); // once, before any traffic
|
|
33
|
+
|
|
34
|
+
const { proposals, skipped } = await plan(c, 'vendor-a', new Date());
|
|
35
|
+
for (const p of proposals) await admit(c, p, new Date());
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
`plan` is read-only and tells you what *could* run, plus why each unit could not. `admit` re-asserts every decision inside a transaction, so a stale proposal is refused rather than acted on. From there a run moves through `publishOutbox` → your worker → `finalize`, and `reconcile` sweeps up whatever got stranded.
|
|
39
|
+
|
|
40
|
+
`ensureIndexes` is not optional. The unique partial index on open requests is the only thing preventing duplicate work, and building it after duplicates exist will fail.
|
|
41
|
+
|
|
42
|
+
## Adapters
|
|
43
|
+
|
|
44
|
+
You implement four interfaces: `Queue`, `Vendor`, `PollingVendor`, and `FileSink`.
|
|
45
|
+
|
|
46
|
+
One contract matters more than the others. Your `Queue` must distinguish a definite refusal from an unknown outcome:
|
|
47
|
+
|
|
48
|
+
```js
|
|
49
|
+
import { EnqueueRejected } from '@task-master/cadence';
|
|
50
|
+
|
|
51
|
+
async enqueue(payload, { idempotencyKey }) {
|
|
52
|
+
const res = await myQueue.submit(payload, idempotencyKey);
|
|
53
|
+
if (res.status === 'invalid') throw new EnqueueRejected(res.reason); // definitely not accepted
|
|
54
|
+
return { taskId: res.id }; // anything else thrown = unknown
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Cadence treats any other thrown error as *uncertain* and lets the reconciler resolve it on evidence. Getting this backwards — reporting a refusal as uncertainty — sends the reconciler hunting for a task that never existed.
|
|
59
|
+
|
|
60
|
+
If you need to detect these errors yourself, use the guards rather than `instanceof`:
|
|
61
|
+
|
|
62
|
+
```js
|
|
63
|
+
import { isEnqueueRejected, isEnqueueUncertain } from '@task-master/cadence';
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Identity is structural, so it survives two copies of the package in one dependency tree. `instanceof` does not.
|
|
67
|
+
|
|
68
|
+
## Design notes
|
|
69
|
+
|
|
70
|
+
A few decisions are load-bearing and non-obvious:
|
|
71
|
+
|
|
72
|
+
- **A failed download is not partial coverage.** `partial` carries time intervals; a transport failure says nothing about which part of the window the vendor actually searched. Only a vendor-reported gap produces `partial`.
|
|
73
|
+
- **Polling is not progress.** A job polled a thousand times with nothing new to report has not progressed, or the no-progress deadline would never fire. The *first* report is news; identical repeats are not.
|
|
74
|
+
- **A capped catch-up window is not a gap.** The marker stops at the capped boundary and the next scheduled window resumes exactly there.
|
|
75
|
+
- **Attention is a list.** Three unrelated problems can be true about one unit at once.
|
|
76
|
+
|
|
77
|
+
The full design — problem statement, architecture, schemas, invariants, and the review history behind them — lives in `docs/` in the repository, along with `docs/RECIPES.md`, which answers the questions that come up when wiring this in: how many processes to run, how to classify vendor failures, what to alert on, why a quiet unit is quiet. Neither is published to npm.
|
|
78
|
+
|
|
79
|
+
## The four roles
|
|
80
|
+
|
|
81
|
+
Cadence is a library, not a daemon — you run the loops. There are four, with different cardinalities:
|
|
82
|
+
|
|
83
|
+
| Role | How many | Interval |
|
|
84
|
+
|---|---|---|
|
|
85
|
+
| Scheduler | one per integration is simplest | seconds to minutes |
|
|
86
|
+
| Publisher | many; they compete safely | continuous |
|
|
87
|
+
| Worker | many; they compete safely | continuous |
|
|
88
|
+
| Reconciler | **exactly one** | a minute or more |
|
|
89
|
+
|
|
90
|
+
Publishers and workers claim work with a token and a TTL, so running ten of each is fine. The reconciler resolves uncertainty and should not race itself. Two schedulers on one integration will not corrupt anything — the unique index on open requests refuses the second admit — but one of them logs a refusal every tick.
|
|
91
|
+
|
|
92
|
+
Each loop below is condensed from `examples/` in the repository, which is typechecked with the package so it cannot drift from the API.
|
|
93
|
+
|
|
94
|
+
### Scheduler
|
|
95
|
+
|
|
96
|
+
```js
|
|
97
|
+
const { proposals, skipped } = await plan(c, integrationId, new Date());
|
|
98
|
+
|
|
99
|
+
// `skipped` is the only place the answer to "why is this unit quiet?" exists. Log it.
|
|
100
|
+
for (const s of skipped) count(s.reason);
|
|
101
|
+
|
|
102
|
+
for (const p of proposals) {
|
|
103
|
+
const result = await admit(c, p, new Date());
|
|
104
|
+
if (!result.admitted && result.reason === 'no_capacity') break;
|
|
105
|
+
}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### Publisher
|
|
109
|
+
|
|
110
|
+
```js
|
|
111
|
+
const result = await publishOutbox(c, queue, new Date(), {
|
|
112
|
+
instanceId,
|
|
113
|
+
claimTtlMs: DEFAULTS.outboxClaimTtlMs,
|
|
114
|
+
});
|
|
115
|
+
|
|
116
|
+
if (!result.published && result.reason === 'nothing_pending') await sleep(1000);
|
|
117
|
+
// 'claim_lost' means a peer took it; 'uncertain' is already parked for the reconciler.
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### Worker
|
|
121
|
+
|
|
122
|
+
```js
|
|
123
|
+
const worker = { workerId, claimTtlMs: DEFAULTS.submissionClaimTtlMs };
|
|
124
|
+
const exec = { workerId, entryClaimTtlMs: 300_000 };
|
|
125
|
+
|
|
126
|
+
const submit = await submitKickoff(c, vendor, integration, runId, new Date(), worker);
|
|
127
|
+
if (submit.status !== 'submitted' && submit.status !== 'already_confirmed') return;
|
|
128
|
+
|
|
129
|
+
for (;;) {
|
|
130
|
+
const step = await pollRun(c, poller, integration, runId, new Date(), exec);
|
|
131
|
+
if (step.status === 'terminal') return finalize(c, runId, step.outcome, new Date());
|
|
132
|
+
if (step.status === 'manifest_ready') break;
|
|
133
|
+
if (step.status === 'not_pollable') return;
|
|
134
|
+
await sleep(15_000);
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
// Each call claims exactly one manifest entry, so several workers may share a manifest.
|
|
138
|
+
for (;;) {
|
|
139
|
+
const step = await downloadNext(c, sink, integration, runId, new Date(), exec);
|
|
140
|
+
if (step.status === 'nothing_claimable') break;
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
// Null while any entry is still outstanding — including one another worker holds.
|
|
144
|
+
const outcome = await outcomeFor(c, runId);
|
|
145
|
+
if (outcome) await finalize(c, runId, outcome, new Date());
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### Reconciler
|
|
149
|
+
|
|
150
|
+
```js
|
|
151
|
+
const report = await reconcile(c, new Date(), {
|
|
152
|
+
noProgressDeadlineMs: DEFAULTS.noProgressDeadlineMs,
|
|
153
|
+
publicationResolutionGraceMs: 15 * 60_000,
|
|
154
|
+
}, { vendor, queue });
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Alert on `submissionsEscalated`, `publicationsEscalated` and `runsBlocked` — each means something needed a decision the engine could not make on evidence.
|
|
158
|
+
|
|
159
|
+
Both deps are optional and both are worth providing. `vendor.findByClientRequestId` lets a crashed worker adopt its own job instead of escalating; `queue.findByIdempotencyKey` lets an uncertain publication be resolved instead of escalated.
|
|
160
|
+
|
|
161
|
+
## Operating it
|
|
162
|
+
|
|
163
|
+
The engine has no opinion about your UI, so the read side is plain Mongo against the `Cadence` handle:
|
|
164
|
+
|
|
165
|
+
```js
|
|
166
|
+
await c.units.find({ integrationId, attention: { $not: { $size: 0 } } }).toArray(); // needs a human
|
|
167
|
+
await c.runs.find({ integrationId, state: 'blocked' }).toArray(); // waiting on evidence
|
|
168
|
+
await c.repairs.find({ integrationId, state: 'open' }).toArray(); // gaps still owed
|
|
169
|
+
await c.repairs.find({ integrationId, state: 'accepted' }).toArray(); // permanent data loss
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
The last one is a page, not a log line: an accepted repair is a gap the engine gave up on.
|
|
173
|
+
|
|
174
|
+
Overrides go through small helpers — `patchAvailability`, `addSuspension`, `removeSuspension`, `setTrafficPolicy`, `blockRun`, `unblockRun`, `raiseAttention`, `clearAttention`. Each re-derives the unit's eligibility hint, which is why you should not write `availability` directly.
|
|
175
|
+
|
|
176
|
+
`adminState` is the operator's switch and the engine never writes it. `executionState` is the engine's and an operator never should.
|
|
177
|
+
|
|
178
|
+
## License
|
|
179
|
+
|
|
180
|
+
MIT
|