@oakgreencc/cambium 0.0.9 → 0.0.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  The scheduler daemon. Every tick it reads the `q:*` queues from GitHub, starts headless `claude -p` workers in their own worktrees up to a cap, watches their PRs through CI and the merge queue, and makes every queue transition itself. GitHub is the truth; local SQLite only remembers which processes it started.
4
4
 
5
- This is the **Task phase** MVP. Dispatch and the daily Release run are seams (see [Follow-ups](#follow-ups)).
5
+ This is the **Task phase** MVP. With `engine: "graph"`, `q:intake` issues are triaged by the [`dispatch` graph](#the-dispatch-graph) and Release runs as graphs; on the legacy engine both are seams (see [Follow-ups](#follow-ups)).
6
6
 
7
7
  Runtime dependencies: `zod` (the [graph module](#graph-module-preview)) and OpenTelemetry via `@oakgreencc/mycelium` (loaded only when [tracing](#tracing) is on); Node ≥ 22.13 (`node:sqlite`). GitHub access goes through [`graft`](../graft), which must be on `PATH`.
8
8
 
@@ -40,11 +40,11 @@ Each step is bounded by `stepTimeoutSeconds`. A step that throws or times out is
40
40
  2. **Reconcile.** For each tracked worker: alive and under its wall-clock cap → nothing. Over the cap → kill its process group. Gone → exit check (a PR on its branch, or closing its issue, that is open or merged and not still on the commit CI already failed) → `submitted`; otherwise requeue with the stderr tail (a failed prepare step requeues as `spawn-failed`). Then the worktree is torn down if `graft status` says GitHub holds everything in it, else kept with a `warn` saying why.
41
41
  3. **Watch** (no agent), for agent PRs whose issue is in no queue: merged → close the issue, delete the branch; red → requeue with the failing checks' summaries; left the merge queue without merging → requeue; not merged `mergeQueueMinutes` after submission → `q:human`.
42
42
  4. **Sweep.** An issue labelled `running` with this host (or no host) in its marker and no tracked worker → requeue. This is what makes losing the SQLite file cost one requeue.
43
- 5. **Schedule.** Start Task workers for `q:task` issues without `running`, by priority label then age, up to `phases.task.cap` counted from live workers. A task requeued earlier in the same tick can start in that tick.
44
- 6. **Release.** Once a day at `release.time` when `release.enabled`; the run itself is a seam.
43
+ 5. **Schedule.** Start Task workers for `q:task` issues without `running`, by priority label then age, up to `phases.task.cap` counted from live workers. A task requeued earlier in the same tick can start in that tick. Then, with `engine: "graph"` and `phases.dispatch.cap > 0`, start triage workers for `q:intake` issues the same way ([the dispatch graph](#the-dispatch-graph)).
44
+ 6. **Release.** Legacy: once a day at `release.time` when `release.enabled`; the run itself is a seam. Graph: the [release graphs](#the-release-graphs), per repo.
45
45
  7. **Heartbeat.** Write the state file.
46
46
 
47
- A requeue is `q:task` with attempt + 1, or, once `maxAttempts` are spent, `q:human` + `why:budget`. cambium never merges: the merge queue and auto-merge do.
47
+ A requeue is `q:task` with attempt + 1, or, once `maxAttempts` are spent, `q:human` + `why:budget`. The Task phase never merges: the merge queue and auto-merge do. (The one merge cambium asks for is an approved promotion PR, through `graft pr merge`; see [The release graphs](#the-release-graphs).)
48
48
 
49
49
  ## State on GitHub
50
50
 
@@ -52,7 +52,7 @@ A requeue is `q:task` with attempt + 1, or, once `maxAttempts` are spent, `q:hum
52
52
  | --------------------------------------------------- | ------------------------------------------------------------------------------ |
53
53
  | `q:intake`, `q:task`, `q:human`, `q:promote` | The queue. At most one per open issue. |
54
54
  | `running` | A worker holds the lease (with `q:task`). |
55
- | `why:budget` (also `why:reject`, `why:approve`) | Why it is in `q:human`. |
55
+ | `why:budget`, `why:triage` (also `why:reject`, …) | Why it is in `q:human` (`why:triage`: sent there by triage, see below). |
56
56
  | _no `q:` label_, cambium comment, open `agent/*` PR | Submitted: in the Release Queue, which is the PR's merge-queue state. |
57
57
  | `priority:critical` … `priority:low` | Scheduling order (configurable). Unlabelled sorts after every listed priority. |
58
58
 
@@ -117,9 +117,10 @@ One JSON file. Everything but `repos` is optional; these are the defaults:
117
117
 
118
118
  - `repos[].path` is a local clone; worktrees are added from it and `graft` runs in it.
119
119
  - `graft.app` is the App cambium acts as (`GRAFT_APP` for its own `graft` calls); `worker.graftApp` the one workers act as. Neither leaks into the other.
120
- - `engine` picks how the Task phase runs: `"legacy"` (the stage tick, the default) or `"graph"` (the [`task` graph](#the-task-graph)). Both make the same moves; see [The task graph](#the-task-graph).
120
+ - `engine` picks how the Task phase runs: `"legacy"` (the stage tick, the default) or `"graph"` (the [`task` graph](#the-task-graph)). Both make the same moves; see [The task graph](#the-task-graph). Only `"graph"` runs the [`dispatch` graph](#the-dispatch-graph) (gated by `phases.dispatch.cap`); legacy leaves `q:intake` alone.
121
121
  - Setting `paths.home` moves every default path under it.
122
122
  - `release.time` is host-local `HH:MM`.
123
+ - `repos[].release` (`engine: "graph"`): `{ "enabled", "time", "from", "to" }`. `enabled` and `time` default to the top-level `release`, `from` to the repo's `baseBranch`, `to` to `"main"`. A repo whose `from` equals `to` has no promotion PR. Example: `{ "slug": "o/app", "path": "~/app", "baseBranch": "develop", "release": { "enabled": true, "time": "06:00", "from": "develop", "to": "main" } }`.
123
124
 
124
125
  ### Target repo convention
125
126
 
@@ -274,6 +275,104 @@ Generated by `toMermaid(task)`; a test keeps this copy current. Boxes are step n
274
275
  - **Markers.** The graph writes marker v2 without `stage`: `node` is where the issue rests. `merge-queue` is the stage tick's `submitted` with `queued: true`, and a stage-era marker in that state resumes at `merge-queue`. A marker whose node the current graph version no longer has restarts at `start-worker` with a `graph-changed` history entry.
275
276
  - **Same moves.** Every scenario in `test/tick.test.mjs` runs on both engines; the test compares their GitHub calls in order (markers semantically, not by rendered text), worker starts with their prompts, kills, teardowns and logs.
276
277
 
278
+ ## The dispatch graph
279
+
280
+ With `"engine": "graph"` and `phases.dispatch.cap > 0`, each `q:intake` issue is triaged by a one-shot `claude -p` worker (`src/graphs/dispatch.mjs`, steps in `src/dispatch-engine.mjs`, prompt and result in `src/phases/dispatch.mjs`), then moved to `q:task`, moved to `q:human`, or closed as a duplicate.
281
+
282
+ ```mermaid
283
+ flowchart TD
284
+ start_(["q:intake"])
285
+ n0["start-triage"]
286
+ n1{{"triaging"}}
287
+ n2["retry-triage"]
288
+ n3["to-task"]
289
+ n4["to-human"]
290
+ n5["close-duplicate"]
291
+ end_(("end"))
292
+ start_ --> n0
293
+ n0 -->|"started"| n1
294
+ n0 -->|"budgetSpent"| n4
295
+ n0 --> n2
296
+ n1 -->|"issueGone"| end_
297
+ n1 -->|"leaseLost"| n2
298
+ n1 -->|"actionable"| n3
299
+ n1 -->|"duplicate"| n5
300
+ n1 -->|"needsHuman"| n4
301
+ n1 --> n4
302
+ n2 -->|"attemptsLeft"| n0
303
+ n2 --> n4
304
+ n3 --> end_
305
+ n4 --> end_
306
+ n5 --> end_
307
+ ```
308
+
309
+ | Node | Kind | Queue label | Holds `running` |
310
+ | ----------------- | ---- | ----------- | --------------- |
311
+ | `start-triage` | step | `q:intake` | no |
312
+ | `triaging` | wait | `q:intake` | yes |
313
+ | `retry-triage` | step | `q:intake` | no |
314
+ | `to-task` | step | `q:task` | no |
315
+ | `to-human` | step | `q:human` | no |
316
+ | `close-duplicate` | step | none | no |
317
+
318
+ - **The verdict.** The worker's final message must end with a JSON block, validated by zod (`TriageResult`): `{ verdict: 'actionable' | 'needs-human' | 'duplicate', duplicateOf?, reason, title?, acceptance? }` (`duplicateOf` is required on a duplicate and cannot be the issue itself). cambium reads it from the worker's stdout log: the last `result` event's `result` text, its last fenced block (else the bare JSON).
319
+ - **Outcomes.** `actionable` → a comment with the reason, the task title and acceptance checks (when given), then `q:task` with a task-graph marker at `start-worker`, attempt 1, history kept; the checks also ride in the marker (`triage`) into the task worker's prompt. The issue body is never rewritten: cambium's write path has no issue edit, and the author's text stays as written. `needs-human` → `q:human` + `why:triage`, the reason as the state comment's last failure. `duplicate` → closed with reason `duplicate` and `Duplicate of #n (<link>). <reason>`. A missing or invalid verdict (no `result` event, no JSON, a schema error) or a worker killed at `wallClockMinutes` (the unlabelled `triaging → to-human` edge) → `q:human` + `why:triage` with the error.
320
+ - **Read-only checkout.** Triage reads code to judge feasibility and duplicates, so it gets a checkout, but a disposable one: a detached worktree at `<root>/triage-<n>` on `origin/<base>` (the repo's worktree root), no install, hydrate or `prepare`, tools `Read,Glob,Grep,Bash` with Edit/Write and every graft/git write verb denied. It is removed outright after the verdict (no `graft status`: nothing in it is worth keeping).
321
+ - **Retries.** A worker that cannot start, or a lost `running` lease on a `q:intake` issue (Sweep), goes back to `q:intake` with attempt + 1, then to `q:human` once `phases.dispatch.maxAttempts` are spent. A marker from another graph or a finished triage starts over at attempt 1.
322
+ - **Leaving intake.** If the issue is closed or relabelled while triage runs, the verdict is dropped and the checkout removed.
323
+ - **Tracing.** Call sites are marked (`TRACING (follow-up)`) as in the task graph: the worker's `TRACEPARENT`, its stream-json spans, and `advanceHooks` in the driver, whose `cambium.issue` root continues mycelium's intake span from the issue body.
324
+
325
+ ## The release graphs
326
+
327
+ With `"engine": "graph"` the tick's Release step runs two graphs (`src/graphs/release.mjs`) for every repo whose `release` is on. Their subject is the **promotion PR**, which promotes the repo's integration branch to its release branch (`release.from` → `release.to`, e.g. `develop` → `main`). The marker (v2) is cambium's state comment on that PR, written through the same path as an issue's.
328
+
329
+ `release-prepare` has a clock entry: once a day at the repo's `release.time` (`releaseDue`, last run in the store under `release.lastRun:<slug>`).
330
+
331
+ ```mermaid
332
+ flowchart TD
333
+ start_(["start"])
334
+ n0["cleanup"]
335
+ n1["prepare"]
336
+ end_(("end"))
337
+ start_ --> n0
338
+ n0 --> n1
339
+ n1 --> end_
340
+ ```
341
+
342
+ `release-promote` sits in `q:promote` and enters at `merge` once the promotion PR is approved (`reviewDecision`, or the latest reviews).
343
+
344
+ ```mermaid
345
+ flowchart TD
346
+ start_(["q:promote"])
347
+ n0["merge"]
348
+ n1{{"merging"}}
349
+ n2["promoted"]
350
+ n3["abandoned"]
351
+ end_(("end"))
352
+ start_ --> n0
353
+ n0 -->|"armed"| n1
354
+ n0 --> end_
355
+ n1 -->|"merged"| n2
356
+ n1 --> n3
357
+ n2 --> end_
358
+ n3 --> end_
359
+ ```
360
+
361
+ Generated by `toMermaid`; a test keeps these copies current.
362
+
363
+ | Node | Graph | Kind | What it does |
364
+ | ----------- | --------------- | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
365
+ | `cleanup` | release-prepare | step | Closes, with a comment, every open promotion PR (`from` → `to`, or carrying a release marker) older than 24 h; takes `q:promote` off. |
366
+ | `prepare` | release-prepare | step | `from` ahead of `to`: opens the promotion PR (or refreshes the open one) with the PRs merged into `from` since `to`, labels it `q:promote`, writes the marker. Nothing ahead: a log line. |
367
+ | `merge` | release-promote | step | `graft pr merge <n> --auto`: graft's merge policy, auto-merge, or a merge now when the PR is clean. A refusal is recorded on the marker and not retried on the same head. |
368
+ | `merging` | release-promote | wait | Holds while the PR is open. |
369
+ | `promoted` | release-promote | step | Merged: marker done, `q:promote` off. |
370
+ | `abandoned` | release-promote | step | Closed unmerged: marker done, `q:promote` off. A `q:promote` PR closed or merged before it was approved goes here (or to `promoted`) too. |
371
+
372
+ - **No timeouts.** Neither graph has one; the daily `cleanup` is the only 24 h rule.
373
+ - **Read.** With a repo's release on, the snapshot query also asks (in the same query) for the open PRs from `from` to `to`, the PRs labelled `q:promote`, and `to`'s comparison with `from` (`aheadBy`, and the merged PRs on the last 100 commits ahead).
374
+ - **Legacy.** `engine: "legacy"` keeps the stub.
375
+
277
376
  ## Graph module (preview)
278
377
 
279
378
  `@oakgreencc/cambium/graph` is the start of typed workflows: nodes with schemas, edges as data, and a loop that advances a position through them. The Task phase runs on it with `engine: "graph"` ([The task graph](#the-task-graph)).
@@ -449,8 +548,8 @@ Without Jaeger running the collector still writes the files; the Jaeger exporter
449
548
 
450
549
  Seams that exist, typed, and do nothing yet:
451
550
 
452
- - **Dispatch** (`src/phases/dispatch.mjs`): `Dispatch(input: { repo, intake, capacity, now }) → { started }`. Reached only when `phases.dispatch.cap > 0`; the stub starts nothing and logs that `q:intake` waits.
453
- - **Release** (`src/phases/release.mjs`): `Release(input: { repo, now }) → { done, note }`. The once-a-day schedule (`releaseDue`) is real and recorded in the store; the run returns `{ done: false }`.
551
+ - **Dispatch on the legacy engine** (`src/phases/dispatch.mjs`): `Dispatch(input: { repo, intake, capacity, now }) → { started }`. Reached only when `phases.dispatch.cap > 0`; the stub starts nothing and logs that `q:intake` waits. The graph engine runs [the dispatch graph](#the-dispatch-graph) instead; `cambium run dispatch` is not wired yet.
552
+ - **Release**, legacy engine only (`src/phases/release.mjs`): `Release(input: { repo, now }) → { done, note }`. The once-a-day schedule (`releaseDue`) is real and recorded in the store; the run returns `{ done: false }`. The graph engine runs the [release graphs](#the-release-graphs).
454
553
  - Conditional (ETag) reads, so an unchanged poll is free against the rate limit; GraphQL has no ETag, so this needs a REST probe in front of the query.
455
554
  - More than 100 queued issues or open PRs per repo, and state comments beyond an issue's first 100 comments: the read is one page.
456
555
  - Spend caps per worker (turns and wall clock are enforced today).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@oakgreencc/cambium",
3
- "version": "0.0.9",
3
+ "version": "0.0.11",
4
4
  "description": "The scheduler daemon: reads the q:* queues from GitHub every tick, runs headless claude workers in their own worktrees up to a cap, and makes every queue transition itself.",
5
5
  "license": "AGPL-3.0-only",
6
6
  "type": "module",
@@ -51,7 +51,7 @@
51
51
  "@opentelemetry/sdk-metrics": "^3.1.1",
52
52
  "@opentelemetry/sdk-trace": "^3.1.1",
53
53
  "zod": "^4.6.7",
54
- "@oakgreencc/mycelium": "0.0.6"
54
+ "@oakgreencc/mycelium": "0.0.0"
55
55
  },
56
56
  "devDependencies": {
57
57
  "@langchain/core": "^1.4.1",
package/src/config.mjs CHANGED
@@ -8,7 +8,11 @@ import { homedir, hostname } from 'node:os'
8
8
  import { join, resolve } from 'node:path'
9
9
 
10
10
  /**
11
- * @typedef {{ slug: string, path: string, baseBranch: string }} Repo
11
+ * @typedef {{ enabled: boolean, time: string, from: string, to: string }} RepoRelease
12
+ * The promotion PR (`engine: "graph"`): `from` (default: the repo's `baseBranch`) is promoted into `to`
13
+ * (default `main`) once a day at `time`. `enabled`/`time` default to the top-level `release`. Off when
14
+ * `from === to`.
15
+ * @typedef {{ slug: string, path: string, baseBranch: string, release: RepoRelease }} Repo
12
16
  * @typedef {{ cap: number, wallClockMinutes: number, maxTurns: number, maxAttempts: number }} PhaseCaps
13
17
  * @typedef {{
14
18
  * host: string,
@@ -82,15 +86,25 @@ export function resolveConfig(raw = {}) {
82
86
  /** @type {Config} */
83
87
  const config = merge(defaults(home), raw)
84
88
  for (const k of /** @type {const} */ (['home', 'db', 'heartbeat', 'logs'])) config.paths[k] = untilde(config.paths[k])
89
+ if (!/^\d{2}:\d{2}$/.test(config.release.time)) throw new Error(`config: release.time must be HH:MM`)
85
90
  config.repos = (config.repos ?? []).map((r) => {
86
91
  if (!r?.slug || !/^[^/\s]+\/[^/\s]+$/.test(r.slug))
87
92
  throw new Error(`config: repo slug must be owner/name, got ${r?.slug}`)
88
93
  if (!r.path) throw new Error(`config: repo ${r.slug} needs "path" (a local clone)`)
89
- return { slug: r.slug, path: untilde(r.path), baseBranch: r.baseBranch ?? 'main' }
94
+ const baseBranch = r.baseBranch ?? 'main'
95
+ /** @type {RepoRelease} */
96
+ const release = {
97
+ enabled: config.release.enabled,
98
+ time: config.release.time,
99
+ from: baseBranch,
100
+ to: 'main',
101
+ ...r.release,
102
+ }
103
+ if (!/^\d{2}:\d{2}$/.test(release.time)) throw new Error(`config: repo ${r.slug} release.time must be HH:MM`)
104
+ return { slug: r.slug, path: untilde(r.path), baseBranch, release }
90
105
  })
91
106
  if (config.engine !== 'legacy' && config.engine !== 'graph')
92
107
  throw new Error(`config: engine must be "legacy" or "graph", got ${JSON.stringify(config.engine)}`)
93
- if (!/^\d{2}:\d{2}$/.test(config.release.time)) throw new Error(`config: release.time must be HH:MM`)
94
108
  return config
95
109
  }
96
110
 
@@ -0,0 +1,135 @@
1
+ // @ts-check
2
+ /**
3
+ * `engine: "graph"`: the Dispatch-phase steps of the tick, driven through the `dispatch` graph
4
+ * (graphs/dispatch.mjs). The legacy engine keeps the stub (phases/dispatch.mjs) and leaves `q:intake` alone.
5
+ *
6
+ * - Reconcile: every tracked triage worker's issue at `triaging`.
7
+ * - Sweep: a stale `running` lease on a `q:intake` issue runs `retry-triage`.
8
+ * - Schedule: `q:intake` issues without a lease, by priority then age, up to `phases.dispatch.cap` counted from
9
+ * live triage workers; each starts at `start-triage`.
10
+ *
11
+ * A turn advances one transition at a time and stops on entering a wait, at the entry node again (only Schedule
12
+ * admits entries), at a wait that is not ready, or at the end.
13
+ */
14
+ import { advance } from './graph/index.mjs'
15
+ import { dispatch, lostLeaseOf, nodeOf } from './graphs/dispatch.mjs'
16
+ import { byPriorityThenAge, Q, RUNNING } from './labels.mjs'
17
+ import { each, staleLeases } from './tick.mjs'
18
+
19
+ /**
20
+ * @typedef {import('./tick.mjs').Deps} Deps
21
+ * @typedef {import('./tick.mjs').Ctx} Ctx
22
+ * @typedef {import('./graphs/task.mjs').Subject} Subject
23
+ * @typedef {import('./graph/advance.mjs').Transition} Transition
24
+ */
25
+
26
+ export const PHASE = 'dispatch'
27
+
28
+ /**
29
+ * One issue's turn through the dispatch graph from `at`. A node error is rethrown (the step logs it); a triage
30
+ * worker whose `triaging` was left is torn down only after the turn's writes succeeded.
31
+ *
32
+ * @param {Deps} deps @param {Ctx} ctx @param {Subject} subject @param {{ node: string, input: unknown }} at
33
+ */
34
+ export async function driveDispatch(deps, ctx, subject, at) {
35
+ const c = { ...deps, tick: ctx, subject }
36
+ /** @type {Transition[]} */
37
+ const transitions = []
38
+ let pos = at
39
+ for (;;) {
40
+ // TRACING (follow-up): pass `instrument: advanceHooks(telemetry, { graph: 'dispatch', …, issueBody }).instrument`
41
+ // here; the issue's cambium.issue root continues mycelium's intake span from the body, so these node and wait
42
+ // spans join the intake trace (tracing.mjs `issueTrace`).
43
+ const r = await advance(dispatch, pos, c, { maxTransitions: 1 })
44
+ if (r.error) throw r.error.cause instanceof Error ? r.error.cause : r.error
45
+ transitions.push(...r.transitions)
46
+ pos = { node: r.node, input: r.input }
47
+ if (r.reason !== 'limit' || r.node === dispatch.entry.node) break
48
+ if (nodeOf(r.node).kind === 'wait') break
49
+ }
50
+ if (transitions.some((t) => t.from === 'triaging')) await releaseWorker(deps, subject)
51
+ return { transitions, ...pos }
52
+ }
53
+
54
+ /** The triage worker's row goes, then its read-only checkout. @param {Deps} deps @param {Subject} s */
55
+ async function releaseWorker(deps, s) {
56
+ const w = deps.store.worker(s.slug, s.number)
57
+ if (!w) return
58
+ deps.store.removeWorker(s.slug, s.number)
59
+ const repo = deps.config.repos.find((r) => r.slug === s.slug)
60
+ if (!repo) return
61
+ const t = await deps.spawner.cleanup({ repo, worktree: w.worktree, branch: w.branch, readOnly: true })
62
+ if (!t.removed) deps.log('warn', `kept worktree ${w.worktree}: ${t.reason}`, { repo: s.slug, issue: s.number })
63
+ }
64
+
65
+ /** @param {Deps} deps @param {Ctx} ctx */
66
+ async function reconcile(deps, ctx) {
67
+ await each(
68
+ deps,
69
+ deps.store.workers().filter((w) => w.phase === PHASE),
70
+ async (w) => {
71
+ const snap = ctx.snaps.get(w.repo)
72
+ const subject = { slug: w.repo, number: w.issue, snap, issue: snap?.issues.get(w.issue) }
73
+ await driveDispatch(deps, ctx, subject, { node: 'triaging', input: { attempt: w.attempt } })
74
+ },
75
+ (w) => ({ repo: w.repo, issue: w.issue }),
76
+ )
77
+ }
78
+
79
+ /** @param {Deps} deps @param {Ctx} ctx */
80
+ async function sweep(deps, ctx) {
81
+ for (const [slug, snap] of ctx.snaps)
82
+ await each(
83
+ deps,
84
+ staleLeases(deps, ctx, slug, snap).filter((i) => i.labels.includes(Q.intake)),
85
+ async (issue) => {
86
+ const subject = { slug, number: issue.number, snap, issue }
87
+ await driveDispatch(deps, ctx, subject, {
88
+ node: 'retry-triage',
89
+ input: lostLeaseOf({ ...deps, tick: ctx, subject }),
90
+ })
91
+ },
92
+ (issue) => ({ repo: slug, issue: issue.number }),
93
+ )
94
+ }
95
+
96
+ /**
97
+ * The `q:intake` issues to triage this tick: no `running`, no tracked worker, by priority then age, up to the free
98
+ * slots counted from live triage workers.
99
+ * @param {Deps} deps @param {Ctx} ctx
100
+ */
101
+ export function intakeCandidates(deps, ctx) {
102
+ const { config, store } = deps
103
+ const free = config.phases.dispatch.cap - store.workers().filter((w) => w.phase === PHASE).length
104
+ if (free <= 0) return []
105
+ /** @type {{ slug: string, issue: import('./github.mjs').Issue, snap: import('./github.mjs').Snapshot }[]} */
106
+ const candidates = []
107
+ for (const [slug, snap] of ctx.snaps)
108
+ for (const issue of snap.issues.values())
109
+ if (
110
+ issue.state === 'OPEN' &&
111
+ issue.labels.includes(Q.intake) &&
112
+ !issue.labels.includes(RUNNING) &&
113
+ !store.worker(slug, issue.number)
114
+ )
115
+ candidates.push({ slug, issue, snap })
116
+ const order = byPriorityThenAge(config.priorityLabels)
117
+ candidates.sort((a, b) => order(a.issue, b.issue))
118
+ return candidates.slice(0, free)
119
+ }
120
+
121
+ /** @param {Deps} deps @param {Ctx} ctx */
122
+ async function schedule(deps, ctx) {
123
+ await each(
124
+ deps,
125
+ intakeCandidates(deps, ctx),
126
+ async (c) => {
127
+ const subject = { slug: c.slug, number: c.issue.number, snap: c.snap, issue: c.issue }
128
+ await driveDispatch(deps, ctx, subject, { node: dispatch.entry.node, input: {} })
129
+ },
130
+ (c) => ({ repo: c.slug, issue: c.issue.number }),
131
+ )
132
+ }
133
+
134
+ /** The Dispatch phase's steps for the graph engine. */
135
+ export const DISPATCH_STEPS = { reconcile, sweep, schedule }
package/src/github.mjs CHANGED
@@ -7,7 +7,7 @@
7
7
  */
8
8
  import { spawn } from 'node:child_process'
9
9
  import { parseMarker } from './marker.mjs'
10
- import { QUEUE_LABELS } from './labels.mjs'
10
+ import { Q, QUEUE_LABELS } from './labels.mjs'
11
11
 
12
12
  /**
13
13
  * @typedef {{
@@ -20,13 +20,33 @@ import { QUEUE_LABELS } from './labels.mjs'
20
20
  * branchExists: boolean, inMergeQueue: boolean, autoMerge: boolean,
21
21
  * checks: 'SUCCESS' | 'FAILURE' | 'ERROR' | 'PENDING' | 'EXPECTED' | null, failing: FailingCheck[], closes: number[],
22
22
  * }} PullRequest
23
- * @typedef {{ issues: Map<number, Issue>, prs: PullRequest[] }} Snapshot
23
+ * @typedef {{
24
+ * number: number, url: string, title: string, state: 'OPEN' | 'MERGED' | 'CLOSED', createdAt: string,
25
+ * head: string, base: string, headSha: string | null, labels: string[], approved: boolean, autoMerge: boolean,
26
+ * stateComment: { id: number, marker: import('./marker.mjs').Marker } | null,
27
+ * }} PromotionPr
28
+ * A promotion PR (`from` → `to`, or labelled `q:promote`). Its state comment is a PR (issue) comment, so it
29
+ * goes through the same marker read/write path as an issue's. `approved`: `reviewDecision` is APPROVED, or
30
+ * (no decision against it) a latest review approves and none requests changes.
31
+ * @typedef {{
32
+ * from: string, to: string, aheadBy: number | null, merged: { number: number, title: string }[],
33
+ * prs: PromotionPr[],
34
+ * }} Release
35
+ * The repo's promotion state (when its `release` is on): how far `from` is ahead of `to` (null: `to` does not
36
+ * exist), the PRs merged into `from` since `to` (oldest first, from the last 100 commits ahead), and the
37
+ * promotion PRs: open ones from `from` to `to` plus any (open, merged or closed) still labelled `q:promote`.
38
+ * @typedef {{ issues: Map<number, Issue>, prs: PullRequest[], release?: Release | null }} Snapshot
24
39
  * @typedef {{
25
40
  * read(slug: string): Promise<Snapshot>,
26
41
  * label(slug: string, number: number, change: { add: string[], remove: string[] }): Promise<void>,
27
42
  * writeState(slug: string, number: number, commentId: number | null, body: string): Promise<number>,
28
- * close(slug: string, number: number, body: string): Promise<void>,
43
+ * close(slug: string, number: number, body: string, opts?: { reason?: 'completed' | 'not_planned' | 'duplicate' }): Promise<void>,
44
+ * comment(slug: string, number: number, body: string): Promise<void>,
29
45
  * deleteBranch(slug: string, branch: string): Promise<void>,
46
+ * openPull(slug: string, pr: { head: string, base: string, title: string, body: string }): Promise<{ number: number, url: string }>,
47
+ * editPull(slug: string, number: number, change: { title: string, body: string }): Promise<void>,
48
+ * closePull(slug: string, number: number, comment: string): Promise<void>,
49
+ * mergePull(slug: string, number: number): Promise<void>,
30
50
  * }} GitHub
31
51
  */
32
52
 
@@ -40,39 +60,116 @@ const PR_FIELDS = `number url headRefName state headRefOid isInMergeQueue
40
60
  ... on StatusContext { context state targetUrl description }
41
61
  } } } } } }`
42
62
 
43
- /** One query per repo per tick: the queued issues and the agent PRs with their checks. */
44
- export const SNAPSHOT_QUERY = `query($owner: String!, $name: String!, $labels: [String!]) {
63
+ /**
64
+ * One query per repo per tick: the queued issues and the agent PRs with their checks; with `$release`, also the
65
+ * promotion PRs and how far `$from` is ahead of `$to` (see {@link Release}).
66
+ */
67
+ export const SNAPSHOT_QUERY = `query($owner: String!, $name: String!, $labels: [String!], $release: Boolean!, $from: String!, $to: String!) {
45
68
  repository(owner: $owner, name: $name) {
46
69
  issues(labels: $labels, states: OPEN, first: 100, orderBy: { field: CREATED_AT, direction: ASC }) { nodes { ...I } }
47
70
  open: pullRequests(states: OPEN, first: 100, orderBy: { field: UPDATED_AT, direction: DESC }) { nodes { ${PR_FIELDS} } }
48
71
  merged: pullRequests(states: MERGED, first: 30, orderBy: { field: UPDATED_AT, direction: DESC }) { nodes { ${PR_FIELDS} } }
72
+ promoteRef: pullRequests(headRefName: $from, baseRefName: $to, states: OPEN, first: 10) @include(if: $release) { nodes { ...P } }
73
+ promoteLabelled: pullRequests(labels: ["${Q.promote}"], first: 10, orderBy: { field: UPDATED_AT, direction: DESC }) @include(if: $release) { nodes { ...P } }
74
+ toRef: ref(qualifiedName: $to) @include(if: $release) {
75
+ compare(headRef: $from) {
76
+ aheadBy
77
+ commits(last: 100) { nodes { associatedPullRequests(first: 3) { nodes { number title baseRefName state } } } }
78
+ }
79
+ }
49
80
  }
50
81
  }
51
82
  fragment I on Issue {
52
83
  number title url state createdAt body
53
84
  labels(first: 30) { nodes { name } }
54
85
  comments(first: 100) { nodes { databaseId body author { login } } }
86
+ }
87
+ fragment P on PullRequest {
88
+ number url title state createdAt headRefName baseRefName headRefOid reviewDecision
89
+ autoMergeRequest { enabledAt }
90
+ latestReviews(first: 20) { nodes { state } }
91
+ labels(first: 30) { nodes { name } }
92
+ comments(first: 100) { nodes { databaseId body author { login } } }
55
93
  }`
56
94
 
57
95
  const FAILED = new Set(['FAILURE', 'ERROR', 'TIMED_OUT', 'CANCELLED', 'ACTION_REQUIRED', 'STARTUP_FAILURE'])
58
96
 
97
+ /**
98
+ * The first comment holding a marker (from `botLogin` only, when set). Issues and PRs alike.
99
+ * @param {any} n an issue or PR node with `comments` @param {string | null} botLogin
100
+ * @returns {{ id: number, marker: import('./marker.mjs').Marker } | null}
101
+ */
102
+ function stateCommentOf(n, botLogin) {
103
+ const comment = (n.comments?.nodes ?? []).find(
104
+ (/** @type {any} */ c) => parseMarker(c.body) && (!botLogin || c.author?.login === botLogin),
105
+ )
106
+ return comment ? { id: comment.databaseId, marker: /** @type {any} */ (parseMarker(comment.body)) } : null
107
+ }
108
+
109
+ /** @param {any} n @returns {string[]} */
110
+ const labelsOf = (n) => (n.labels?.nodes ?? []).map((/** @type {any} */ l) => l.name)
111
+
112
+ /**
113
+ * The `$release` part of the answer → {@link Release}.
114
+ * @param {any} repo @param {{ from: string, to: string }} branches @param {string | null} botLogin
115
+ * @returns {Release}
116
+ */
117
+ function normalizeRelease(repo, { from, to }, botLogin) {
118
+ /** @type {Map<number, PromotionPr>} */
119
+ const prs = new Map()
120
+ for (const p of [...(repo.promoteRef?.nodes ?? []), ...(repo.promoteLabelled?.nodes ?? [])]) {
121
+ if (!p || prs.has(p.number)) continue
122
+ const reviews = (p.latestReviews?.nodes ?? []).map((/** @type {any} */ r) => r.state)
123
+ prs.set(p.number, {
124
+ number: p.number,
125
+ url: p.url ?? '',
126
+ title: p.title ?? '',
127
+ state: p.state,
128
+ createdAt: p.createdAt,
129
+ head: p.headRefName,
130
+ base: p.baseRefName,
131
+ headSha: p.headRefOid ?? null,
132
+ labels: labelsOf(p),
133
+ approved:
134
+ p.reviewDecision === 'APPROVED' ||
135
+ (p.reviewDecision !== 'CHANGES_REQUESTED' &&
136
+ reviews.includes('APPROVED') &&
137
+ !reviews.includes('CHANGES_REQUESTED')),
138
+ autoMerge: Boolean(p.autoMergeRequest),
139
+ stateComment: stateCommentOf(p, botLogin),
140
+ })
141
+ }
142
+ const compare = repo.toRef?.compare ?? null
143
+ /** @type {Map<number, { number: number, title: string }>} */
144
+ const merged = new Map()
145
+ for (const c of compare?.commits?.nodes ?? [])
146
+ for (const p of c?.associatedPullRequests?.nodes ?? [])
147
+ if (p.baseRefName === from && p.state === 'MERGED' && !merged.has(p.number))
148
+ merged.set(p.number, { number: p.number, title: p.title ?? '' })
149
+ return {
150
+ from,
151
+ to,
152
+ aheadBy: compare ? (compare.aheadBy ?? 0) : null,
153
+ merged: [...merged.values()],
154
+ prs: [...prs.values()],
155
+ }
156
+ }
157
+
59
158
  /**
60
159
  * GraphQL `data` → {@link Snapshot}. Only `agent/*` PRs are kept.
61
160
  *
62
161
  * @param {any} data
63
- * @param {{ botLogin?: string | null }} [opts] when set, only that login's markers count
162
+ * @param {{ botLogin?: string | null, release?: { from: string, to: string } | null }} [opts]
163
+ * `botLogin`: when set, only that login's markers count. `release`: the branches the query asked about.
64
164
  * @returns {Snapshot}
65
165
  */
66
- export function normalize(data, { botLogin = null } = {}) {
166
+ export function normalize(data, { botLogin = null, release = null } = {}) {
67
167
  const repo = (data?.data ?? data)?.repository
68
168
  if (!repo) throw new Error('snapshot: no repository in the answer')
69
169
  /** @type {Map<number, Issue>} */
70
170
  const issues = new Map()
71
171
  const addIssue = (/** @type {any} */ n) => {
72
172
  if (!issues.has(n.number)) {
73
- const comment = (n.comments?.nodes ?? []).find(
74
- (/** @type {any} */ c) => parseMarker(c.body) && (!botLogin || c.author?.login === botLogin),
75
- )
76
173
  issues.set(n.number, {
77
174
  number: n.number,
78
175
  title: n.title ?? '',
@@ -80,10 +177,8 @@ export function normalize(data, { botLogin = null } = {}) {
80
177
  state: n.state,
81
178
  createdAt: n.createdAt,
82
179
  body: n.body ?? '',
83
- labels: (n.labels?.nodes ?? []).map((/** @type {any} */ l) => l.name),
84
- stateComment: comment
85
- ? { id: comment.databaseId, marker: /** @type {any} */ (parseMarker(comment.body)) }
86
- : null,
180
+ labels: labelsOf(n),
181
+ stateComment: stateCommentOf(n, botLogin),
87
182
  })
88
183
  }
89
184
  return n.number
@@ -115,7 +210,7 @@ export function normalize(data, { botLogin = null } = {}) {
115
210
  closes: (p.closingIssuesReferences?.nodes ?? []).map(addIssue),
116
211
  })
117
212
  }
118
- return { issues, prs }
213
+ return { issues, prs, ...(release ? { release: normalizeRelease(repo, release, botLogin) } : {}) }
119
214
  }
120
215
 
121
216
  /**
@@ -159,9 +254,12 @@ export function graftGitHub({ config, exec: run = exec }) {
159
254
  return {
160
255
  async read(slug) {
161
256
  const [owner, name] = slug.split('/')
162
- const vars = JSON.stringify({ owner, name, labels: QUEUE_LABELS })
257
+ const r = config.repos.find((x) => x.slug === slug)?.release
258
+ const release = r?.enabled && r.from !== r.to ? { from: r.from, to: r.to } : null
259
+ const branches = release ?? { from: '', to: '' }
260
+ const vars = JSON.stringify({ owner, name, labels: QUEUE_LABELS, release: Boolean(release), ...branches })
163
261
  const data = await json(graft(slug, ['graphql', SNAPSHOT_QUERY, '--vars', vars]))
164
- return normalize(data, { botLogin: config.graft.botLogin })
262
+ return normalize(data, { botLogin: config.graft.botLogin, release })
165
263
  },
166
264
  async label(slug, number, { add, remove }) {
167
265
  if (!add.length && !remove.length) return
@@ -182,11 +280,34 @@ export function graftGitHub({ config, exec: run = exec }) {
182
280
  const made = await json(graft(slug, ['issue', 'comment', String(number), '--body-file', '-'], body))
183
281
  return made.id
184
282
  },
185
- async close(slug, number, body) {
186
- await graft(slug, ['issue', 'close', String(number), '--body-file', '-'], body)
283
+ async close(slug, number, body, { reason } = {}) {
284
+ await graft(
285
+ slug,
286
+ ['issue', 'close', String(number), '--body-file', '-', ...(reason ? ['--reason', reason] : [])],
287
+ body,
288
+ )
289
+ },
290
+ async comment(slug, number, body) {
291
+ await graft(slug, ['issue', 'comment', String(number), '--body-file', '-'], body)
187
292
  },
188
293
  async deleteBranch(slug, branch) {
189
294
  await graft(slug, ['api', 'DELETE', `/repos/${slug}/git/refs/heads/${branch}`])
190
295
  },
296
+ async openPull(slug, { head, base, title, body }) {
297
+ const path = `/repos/${slug}/pulls`
298
+ const made = await json(graft(slug, ['api', 'POST', path, '--body', JSON.stringify({ head, base, title, body })]))
299
+ return { number: made.number, url: made.html_url }
300
+ },
301
+ async editPull(slug, number, { title, body }) {
302
+ await graft(slug, ['api', 'PATCH', `/repos/${slug}/pulls/${number}`, '--body', JSON.stringify({ title, body })])
303
+ },
304
+ async closePull(slug, number, comment) {
305
+ await graft(slug, ['issue', 'comment', String(number), '--body-file', '-'], comment)
306
+ await graft(slug, ['api', 'PATCH', `/repos/${slug}/pulls/${number}`, '--body', '{"state":"closed"}'])
307
+ },
308
+ // Through graft's merge policy, as an agent's merge would: auto-merge, or a merge now when the PR is clean.
309
+ async mergePull(slug, number) {
310
+ await graft(slug, ['pr', 'merge', String(number), '--auto'])
311
+ },
191
312
  }
192
313
  }
@@ -21,6 +21,7 @@ import { iso, key, markerOf, setLabels, setMarker } from './issue-state.mjs'
21
21
  import { effectivePosition, graphVersionDecision, withHistory } from './marker.mjs'
22
22
  import { each, newCtx, staleLeases, taskCandidates, watched, watchPairs } from './tick.mjs'
23
23
  import { move, Q } from './labels.mjs'
24
+ import { DISPATCH_STEPS } from './dispatch-engine.mjs'
24
25
 
25
26
  /**
26
27
  * @typedef {import('./tick.mjs').Deps} Deps
@@ -136,7 +137,7 @@ async function restart(deps, ctx, slug, issue, note) {
136
137
  async function reconcile(deps, ctx) {
137
138
  await each(
138
139
  deps,
139
- deps.store.workers(),
140
+ deps.store.workers().filter((w) => w.phase === 'task'),
140
141
  async (w) => {
141
142
  const snap = ctx.snaps.get(w.repo)
142
143
  const subject = { slug: w.repo, number: w.issue, snap, issue: snap?.issues.get(w.issue) }
@@ -178,7 +179,7 @@ async function sweep(deps, ctx) {
178
179
  for (const [slug, snap] of ctx.snaps)
179
180
  await each(
180
181
  deps,
181
- staleLeases(deps, ctx, slug, snap),
182
+ staleLeases(deps, ctx, slug, snap).filter((issue) => !issue.labels.includes(Q.intake)),
182
183
  async (issue) => {
183
184
  const subject = { slug, number: issue.number, snap, issue }
184
185
  const c = taskCtx(deps, ctx, subject)
@@ -206,8 +207,21 @@ async function scheduleTasks(deps, ctx) {
206
207
  /** A stale lease at a node without its own `recover`: requeue. @param {import('./graphs/task.mjs').TaskCtx} c */
207
208
  const defaultRecover = (c) => ({ node: 'requeue', input: lostLeaseOf(c) })
208
209
 
210
+ /** @param {import('./tick.mjs').StepFn[]} steps @returns {import('./tick.mjs').StepFn} */
211
+ const inOrder =
212
+ (...steps) =>
213
+ async (deps, ctx) => {
214
+ for (const step of steps) await step(deps, ctx)
215
+ }
216
+
209
217
  /** @type {import('./tick.mjs').Engine} */
210
- export const GRAPH_ENGINE = { reconcile, watch, sweep, scheduleTasks }
218
+ export const GRAPH_ENGINE = {
219
+ reconcile: inOrder(reconcile, DISPATCH_STEPS.reconcile),
220
+ watch,
221
+ sweep: inOrder(sweep, DISPATCH_STEPS.sweep),
222
+ scheduleTasks,
223
+ scheduleDispatch: DISPATCH_STEPS.schedule,
224
+ }
211
225
 
212
226
  // ---- `cambium run <graph> --issue <n> [--follow]`
213
227