omakit 0.4.1 → 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,126 +1,60 @@
1
1
  <p align="center">
2
- <img src="docs/media/banner.gif" alt="omakit" width="440">
2
+ <img src="https://raw.githubusercontent.com/mtolhuys/omakit/main/docs/media/banner.gif" alt="omakit" width="440">
3
3
  </p>
4
4
 
5
- The safe place to find out: everything knowable about an Omarchy Quattro plugin submission before you post it, on your own machine. Agent-first, read-only against the marketplace, posts nothing, zero dependencies.
5
+ The marketplace validates one exact commit of your plugin. Push a fix or comment "fixed", and nothing re-runs ([M6](docs/MEASUREMENTS.md#m6-the-validated-commit-falls-behind-silently-and-that-is-the-centre-of-this-tool)). omakit runs the marketplace's own checks locally, watches your submission and posts nothing.
6
6
 
7
7
  [![Built for Omarchy: App](https://raw.githubusercontent.com/tcballard/omarchy-badges/75975e5b5bf75e7ede3764bcd2950046f7abfe2c/badges/v1/omarchy-app.svg)](https://github.com/tcballard/omarchy-badges) [![npm version](https://img.shields.io/npm/v/omakit)](https://www.npmjs.com/package/omakit) [![CI status](https://img.shields.io/github/actions/workflow/status/mtolhuys/omakit/ci.yml?branch=main)](https://github.com/mtolhuys/omakit/actions/workflows/ci.yml) [![Socket](https://socket.dev/api/badge/npm/package/omakit)](https://socket.dev/npm/package/omakit)
8
8
 
9
- `omakit` is a zero-dependency Node CLI that checks an Omarchy Quattro plugin submission on your machine.
10
- It is for a coding agent or a person submitting a plugin.
11
- It never posts to the marketplace or writes into a plugin tree.
12
-
13
9
  ## Install
14
10
 
15
11
  ```bash
16
- npm install --global omakit
17
- omakit setup
18
- ```
19
-
20
- See [docs/INSTALL.md](docs/INSTALL.md) for the clone route, PATH, requirements and upgrading.
21
-
22
- ## Commands
23
-
24
- | Command | What it does |
25
- | --- | --- |
26
- | [`omakit setup`](docs/COMMANDS.md) | The environment, the pin, tab completion, and what to try first. |
27
- | [`omakit submit <plugin-repo>`](docs/SUBMIT.md) | Every check, the issue title and body; asks for a category and tags at a terminal. |
28
- | [`omakit watch [<issue-url>]`](docs/VALIDATION_WATCH.md) | Pick your marketplace issues, list them, or check all with `--all`. |
29
- | [`omakit verify <plugin-repo>`](docs/COMMANDS.md) | The official security baseline over the local transport; `--json` for the document. |
30
- | [`omakit parity`](docs/COMMANDS.md) | The baseline over GitHub versus the local transport, on real listings; writes the evidence. |
31
- | [`omakit audit [<plugin>]`](docs/AUDIT.md) | Installed third-party commits against the exact commits the marketplace validated. |
32
- | [`omakit weigh <plugin>`](docs/WEIGH.md) | What a plugin weighs on the shell, measured by restarting it without and with the plugin; asks first. |
33
- | [`omakit doctor`](docs/COMMANDS.md) | What is installed, what is pinned, and what has moved. |
34
- | [`omakit pin`](docs/COMMANDS.md) | What setup does for the pin, on its own. |
35
- | [`omakit upgrade`](docs/COMMANDS.md) | Updates omakit through its own installer: npm, or a fast-forward. |
36
- | [`omakit help --agent`](docs/COMMANDS.md) | The operating instructions, for the agent running this. |
37
-
38
- Normal terminal use also checks for a newer npm release at most once daily
39
- and shows the upgrade command. It installs nothing automatically; scripts
40
- and JSON stay quiet. `DISABLE_UPDATE_NOTIFIER=1` disables the notice, and
41
- `omakit doctor` checks explicitly. [Update behaviour](docs/INSTALL.md#updating).
42
-
43
- Terminal help and reports adapt to the available width, up to 120 columns
44
- for readability. Narrow windows wrap sooner; pipes and text files keep the
45
- stable eighty-column layout. JSON and exact issue bodies remain intact.
46
-
47
- ### `submit`
48
-
49
- ```bash
50
- omakit submit <plugin-repo> --category Widgets --tags bar,quickshell
12
+ npm i -g omakit && omakit setup
13
+ npx skills add mtolhuys/omakit
51
14
  ```
52
15
 
53
- It decides whether the plugin is ready, refused, or already listed; [103 issues mention agent-control files that no automated check reports](docs/MEASUREMENTS.md).
16
+ Requirements: [Node >=22](package.json); Omarchy Quattro.
54
17
 
55
- ![omakit submit refusing a plugin with no license, a README that never says how to uninstall, and a reserved plugin id](docs/media/submit.gif)
18
+ Licence: [MIT](LICENSE).
56
19
 
57
- Read more: [docs/SUBMIT.md](docs/SUBMIT.md).
20
+ ## `omakit submit <plugin-repo>`
58
21
 
59
- ### `watch`
22
+ ![submit refusing a fixture plugin before any issue is posted](https://raw.githubusercontent.com/mtolhuys/omakit/main/docs/media/submit.gif)
60
23
 
61
- ```bash
62
- omakit watch <submission-issue-url>
63
- omakit watch --all
64
- omakit watch --list
65
- omakit watch # choose one or several issues at a terminal
66
- ```
24
+ Checks the exact commit with the marketplace's own baseline and, when ready, prints the exact issue title and body for you to paste. The GIF shows a refusal with three blocking checks and their fixes.
67
25
 
68
- It decides whether the marketplace validated the plugin's current commit; [73% of parked submissions have a HEAD the marketplace never saw](docs/MEASUREMENTS.md).
26
+ ## `omakit watch --all`
69
27
 
70
- Account-wide discovery uses your signed-in `gh` account and reads your open marketplace issues. `--user <login>` reads another public account. Batch output includes baseline results, labels and the latest human discussion; `current` compares commits and does not imply approval or publication. Each command takes one snapshot and posts nothing.
28
+ ![watch counts and two current issues, including human discussion](https://raw.githubusercontent.com/mtolhuys/omakit/main/docs/media/watch-all.gif)
71
29
 
72
- ![omakit watch reporting that a validated commit has fallen behind](docs/media/watch.gif)
30
+ Checks your submission commits and names the action that re-runs stale validation: edit the issue body. The GIF shows five CURRENT issues in the counts and the first two issues with a discussion; CURRENT means matching commits, not approval.
73
31
 
74
- Read more: [docs/VALIDATION_WATCH.md](docs/VALIDATION_WATCH.md).
32
+ ## `omakit audit`
75
33
 
76
- ### `weigh`
77
-
78
- ```bash
79
- omakit weigh <plugin-id-or-dir>
80
- ```
34
+ ![audit keeping drift rows and the DRIFT summary visible together](https://raw.githubusercontent.com/mtolhuys/omakit/main/docs/media/audit.gif)
81
35
 
82
- It measures what a plugin weighs on the shell, CPU and child processes, against a baseline taken the same minute; [in the lab, a 180 ms timer fixture measured 2.73% CPU above a 0.13% floor](docs/MEASUREMENTS.md).
36
+ Compares your installed plugin commits with the marketplace's validated commits. The GIF shows drift rows first: on the author's desktop, 9 of 18 audited plugins ran commits the marketplace never validated.
83
37
 
84
- ```text
85
- Weighs no CPU above the floor (0.13%) and runs 2 child processes using 8.2 MB and 0.1% CPU, on Omarchy 4.0.0.alpha, measured with omakit weigh on 2026-09-14
86
- ```
38
+ ## `omakit weigh <plugin>`
87
39
 
88
- It restarts your shell and asks first. Memory is a shell fact; CPU and child processes are the weight.
40
+ ![completed three-run desktop weighing with baseline samples and the noise floor](https://raw.githubusercontent.com/mtolhuys/omakit/main/docs/media/weigh.gif)
89
41
 
90
- Read more: [docs/WEIGH.md](docs/WEIGH.md).
42
+ Measures the shell with and without your plugin, reading Pss and CPU. The GIF shows three completed runs on the author's desktop, with baseline and plugin samples and a 0.33% CPU floor ([method](docs/WEIGH.md)).
91
43
 
92
44
  ## Evidence, not claims
93
45
 
94
- | Claim | Proof |
46
+ | Measurement | Evidence |
95
47
  | --- | --- |
96
- | The local transport produces the marketplace's own result | 30 of 30 identical, [docs/evidence/parity/](docs/evidence/parity/) |
97
- | A local run touches no network | run inside `unshare -rn`, [docs/evidence/offline/](docs/evidence/offline/) |
98
- | The generated body is well formed | the marketplace's own parser, `tests/unit/issue.test.mjs` |
99
- | Nothing writes to the marketplace | `tests/unit/read-only.test.mjs`, over every source file |
100
- | No agent-control file can reach a plugin | `tests/unit/self-containment.test.mjs` |
101
- | `weigh` restores `shell.json` on every exit path, and runs a frozen list of Omarchy commands | `tests/unit/weigh.test.mjs` against a fake `/proc` and stub commands, `tests/unit/read-only.test.mjs` |
102
- | The GIFs above are real output | captures and renderer in [docs/media/](docs/media/) |
103
-
104
- Committed evidence records a digest of each side rather than the results themselves, because findings about a specific third-party plugin are not this project's to publish.
48
+ | Baseline parity | 30/30 identical results, [recorded corpus](docs/evidence/parity/2026-09-12-local-vs-github-2.json), 2026-09-12 |
49
+ | Stale validated commit | 326/519 readable comparisons stale (62.8%); 64/583 unknown, [2026-09-15 data](docs/evidence/staleness/2026-09-15.json) |
50
+ | Registry churn | 4,201/4,293 registry-only commits in 30 days, 2026-09-13, [M7](docs/MEASUREMENTS.md#m7-the-registry-moves-by-the-hour-the-code-and-the-rules-move-by-the-week) |
51
+ | GIFs are recorded output | 5 GIFs with [captures and scenes](docs/media/README.md) |
52
+ | Posts nothing | 0 marketplace writes, [M10](docs/MEASUREMENTS.md#m10-readme-evidence-and-command-captures) |
53
+ | Zero dependencies | 0 runtime and 0 development dependencies, counted in [package.json](package.json) |
105
54
 
106
55
  ## Documentation
107
56
 
108
- | Document | For |
109
- | --- | --- |
110
- | [docs/INSTALL.md](docs/INSTALL.md) | install details, PATH, requirements, upgrading, and what Socket reports and why |
111
- | [docs/HOW.md](docs/HOW.md) | what omakit is doing, why it uses Node, the baseline and check labels |
112
- | [docs/COMMANDS.md](docs/COMMANDS.md) | command details, authentication and network behaviour |
113
- | [docs/AUDIT.md](docs/AUDIT.md) | installed plugin drift against marketplace-validated commits, with JSON origins |
114
- | [docs/SUBMIT.md](docs/SUBMIT.md) | every check and what it decides |
115
- | [docs/WEIGH.md](docs/WEIGH.md) | what `weigh` measures, the noise floor, the `shell.json` mutation and its restore, and the JSON contract |
116
- | [docs/VALIDATION_WATCH.md](docs/VALIDATION_WATCH.md) | the validation watch: what the marketplace validated, and what moves it |
117
- | [docs/MEASUREMENTS.md](docs/MEASUREMENTS.md) | every number, its method and its limits |
118
- | [docs/UPSTREAM_CONTRACT.md](docs/UPSTREAM_CONTRACT.md) | the seam, the pin, the boundaries |
119
- | [docs/MARKETPLACE.md](docs/MARKETPLACE.md) | who this actually helps |
120
- | [docs/PALETTE.md](docs/PALETTE.md) | every installed Omarchy theme measured, and which palette index each role gets |
121
- | [docs/TUI.md](docs/TUI.md) | what the terminal shows, and why it looks that way |
122
- | [AGENTS.md](AGENTS.md) | changing this repository |
123
-
124
- MIT. Derived work built on public data from
125
- `omacom/omarchy-plugin-marketplace`; not affiliated with or endorsed by that
126
- project.
57
+ - Using: [install](docs/INSTALL.md), [commands](docs/COMMANDS.md), [audience](docs/MARKETPLACE.md).
58
+ - Checks and measurements: [submit](docs/SUBMIT.md), [watch](docs/VALIDATION_WATCH.md), [audit](docs/AUDIT.md), [evidence](docs/MEASUREMENTS.md).
59
+ - Method docs: [how](docs/HOW.md), [weigh](docs/WEIGH.md), [upstream contract](docs/UPSTREAM_CONTRACT.md), [palette](docs/PALETTE.md), [terminal](docs/TUI.md).
60
+ - Contributing: [repository rules](AGENTS.md), [releasing](docs/RELEASING.md), [media](docs/media/README.md).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "omakit",
3
- "version": "0.4.1",
3
+ "version": "0.4.2",
4
4
  "description": "The safe place to find out: everything knowable about an Omarchy Quattro plugin submission before you post it, on your own machine. Agent-first, read-only against the marketplace, posts nothing, zero dependencies.",
5
5
  "license": "MIT",
6
6
  "author": "Maarten Tolhuijs",
@@ -92,8 +92,8 @@ own words, and so should you.
92
92
 
93
93
  The marketplace validates the pushed default-branch HEAD, not whatever is
94
94
  checked out locally. Run the check on the commit that will be pushed, then
95
- commit and push before the real submission. Of the 464 submissions parked in
96
- their author's court, 73% have a HEAD the marketplace never saw; that is the
95
+ commit and push before the real submission. On 2026-09-15, 326/519 readable author-fixes
96
+ comparisons were stale, with 64 of 583 issues unknown; that is the
97
97
  round this loop is meant to prevent.
98
98
 
99
99
  ## When the plugin is ready
@@ -133,8 +133,8 @@ strip files from their tree on your own.
133
133
 
134
134
  **`submission.validation-commit`.** The marketplace validates the commit it
135
135
  resolves when the issue is opened or edited, which is the pushed default-branch
136
- HEAD, not whatever is checked out locally. Push first, then submit. 73% of submissions
137
- parked in their author's court have a HEAD the marketplace never saw.
136
+ HEAD, not whatever is checked out locally. Push first, then submit. On 2026-09-15, 326/519 readable author-fixes comparisons were stale
137
+ (62.8%), with 64 of 583 issues unknown.
138
138
 
139
139
  ## What the baseline result means
140
140
 
@@ -18,10 +18,10 @@ the issue body**.
18
18
  submissions.
19
19
 
20
20
  So pushing a fix does nothing, and commenting "fixed in `abc123`" does nothing.
21
- Both feel like progress. Neither is. This is the single most common reason a
22
- submission sits still: of the 464 submissions parked in their author's court, 73%
23
- have a default-branch HEAD the marketplace never saw, and 82% of the authors whose
24
- push came after a review comment had also commented: engaged, and stuck.
21
+ Both feel like progress. Neither is. The full author-fixes queue was measured on 2026-09-15: 326/519 readable
22
+ comparisons were stale (62.8%), with 64 of 583 issues unknown. The older
23
+ 2026-09-12 sample found 68/93 readable issues stale (73.1%); that rate was
24
+ not a measurement of all 464 issues in the queue.
25
25
 
26
26
  ## Check it
27
27
 
@@ -18,6 +18,8 @@ local commit through the transport seam the marketplace tests itself
18
18
  | `tree.mjs` | The installable tree of a subject at one exact commit, from the Git object database. |
19
19
  | `plugin.mjs` | The root files the submission contract needs, and the declared plugin identity. |
20
20
  | `agent-control.mjs` | The recursive agent-control warning, and its remedy. |
21
+ | `review-cost.mjs` | The advisory review-cost verdict, shared account discovery, and path classification between dated validated snapshots. M4 and M9 carry its evidence. |
22
+ | `measure-review-cost.mjs` | Reproduces M9 across the open update population at live marketplace HEAD, with compare sources and explicit skipped reasons in JSON. |
21
23
  | `issue.mjs` | Renders the issue the way the form would, then has the marketplace's own parser judge it. |
22
24
  | `submit.mjs` | Assembles every check with its measured reason, and withholds the body when a blocking check fails. Three outcomes: `ready` (the body), `refused` (a blocking check failed) and `listed` (the plugin is already listed by its own repository: `identity.available` passes with the listing's record, the five body checks are omitted rather than drawn as waiting, no body exists on purpose, and `listing` carries the listed commit against the local one and the form to use for a newer commit). Decides the category and tags after the registry: a listed plugin, own or taken, is asked for neither; an unlisted one without them is asked through `ask.mjs` at a terminal, and is a usage error otherwise. Ends with `reproduce`, the command line that repeats the run without asking. Under `--offline` the validation-commit check is `skipped`, not passed: verdict `skipped`, listed under `skipped` and not `unknown`, never blocking, and the READY line says "1 check skipped (--offline)". |
23
25
  | `ask.mjs` | The two questions `submit` asks a person at a terminal, and only there: category and tags, numbered from the pinned form, with the marketplace's own presentation for the manifest's kinds (read from the pinned catalog builder) as the default where it is on the list. Prompts on stderr, nothing persisted. |
@@ -98,7 +100,9 @@ sequences and nothing else.
98
100
  built-in `fetch`, which does not read proxy environment variables by default.
99
101
  Behind a proxy, run them with `NODE_USE_ENV_PROXY=1`. `submit` reads two things
100
102
  online, the subject's default-branch HEAD and the marketplace's current
101
- registry, and `--offline` turns both off; `verify` needs no network at all
103
+ registry. A manual-review baseline also reads the account's open issue
104
+ discovery and issue bodies for batching advice; `--offline` turns these
105
+ reads off. `verify` needs no network at all
102
106
  beyond fetching a reviewer-mode subject, and `tests/parity/offline.mjs` proves
103
107
  it.
104
108
 
@@ -68,6 +68,16 @@ const REMEDY = Object.freeze({
68
68
  "interrupted": "shell.json was restored; run it again when the desktop is yours to restart.",
69
69
  })
70
70
 
71
+ /*
72
+ * A command that has written its result leaves through `process.exitCode`,
73
+ * never `process.exit()`: stdout is an API, and on a pipe whose reader has
74
+ * not started reading yet the exit cuts the output. Measured on 0.4.1:
75
+ * `omakit submit <listed plugin> --json | (sleep 2; cat)` delivered 8,192 of
76
+ * 14,033 bytes, and a parser downstream saw invalid JSON. The failure
77
+ * states below still exit at once: they write one short block to stderr,
78
+ * and their callers use them the way a throw is used.
79
+ */
80
+
71
81
  /**
72
82
  * Every failure, in one register, on stderr. `usage` errors carry the
73
83
  * signature that was expected, so the remedy is the reference and not a
@@ -90,9 +100,21 @@ function failFrom(error) {
90
100
  throw error
91
101
  }
92
102
 
103
+ /**
104
+ * The value of a valued option, written either way the table accepts,
105
+ * `--name value` or `--name=value`, the last occurrence winning as it does
106
+ * in options.mjs. Measured on 0.4.1: the table accepted `--out=FILE` and the
107
+ * value was looked up as the token after `--out`, so `doctor --out=x` wrote
108
+ * nothing and exited 0, and `submit --category=Widgets` said the flag was
109
+ * missing.
110
+ */
93
111
  function option(args, name) {
94
- const index = args.indexOf(name)
95
- return index >= 0 ? args[index + 1] : undefined
112
+ let value
113
+ for (let index = 0; index < args.length; index += 1) {
114
+ if (args[index] === name) value = args[index + 1]
115
+ else if (args[index].startsWith(`${name}=`)) value = args[index].slice(name.length + 1)
116
+ }
117
+ return value
96
118
  }
97
119
 
98
120
  /** The bare arguments, with every valued option's value (options.mjs, one table) left out. */
@@ -169,7 +191,8 @@ async function cmdSubmit(args) {
169
191
  const usage = error.usage
170
192
  if (json) {
171
193
  process.stdout.write(`${JSON.stringify({ usage }, null, 2)}\n`)
172
- process.exit(2)
194
+ process.exitCode = 2
195
+ return
173
196
  }
174
197
  const flags = usage.missing.join(" and ")
175
198
  fail("usage", `submit needs ${flags}: ${usage.missing.length === 1 ? "it is" : "they are"} an editorial choice nobody else can make, from the pinned form's own lists.`, 2,
@@ -185,7 +208,7 @@ async function cmdSubmit(args) {
185
208
  emit(args, args.includes("--json") ? `${JSON.stringify(result, null, 2)}\n` : reportText(args, renderSubmit, result))
186
209
  // Three outcomes, two exit codes: `ready` and `listed` are both healthy
187
210
  // states, and only a refusal is a 1.
188
- process.exit(result.outcome === "refused" ? 1 : 0)
211
+ process.exitCode = result.outcome === "refused" ? 1 : 0
189
212
  }
190
213
 
191
214
  async function cmdWatch(args) {
@@ -224,7 +247,7 @@ async function cmdWatch(args) {
224
247
  spinner.done()
225
248
  const render = result.mode === "list" ? renderWatchList : result.mode === "all" ? renderWatchAll : renderWatch
226
249
  emit(args, args.includes("--json") ? `${JSON.stringify(result, null, 2)}\n` : reportText(args, render, result))
227
- process.exit(result.verdict?.state === "unknown" || result.summary?.unknown > 0 ? 2 : 0)
250
+ process.exitCode = result.verdict?.state === "unknown" || result.summary?.unknown > 0 ? 2 : 0
228
251
  }
229
252
 
230
253
  async function cmdFrontDoor() {
@@ -243,15 +266,16 @@ async function cmdSetup(args) {
243
266
  if (args.includes("--completion")) {
244
267
  const identity = requirePin(ROOT).identity
245
268
  const result = await completionStep({ repoRoot: ROOT, pin: identity.commit, version: VERSION, askRc: false })
246
- process.exit(result.state === "ok" ? 0 : 1)
269
+ process.exitCode = result.state === "ok" ? 0 : 1
270
+ return
247
271
  }
248
272
  const result = await setup({ repoRoot: ROOT, entryPoint: resolve(ROOT, "bin/omakit"), yes: args.includes("--yes") })
249
- process.exit(result.ok ? 0 : 1)
273
+ process.exitCode = result.ok ? 0 : 1
250
274
  }
251
275
 
252
276
  async function cmdUpgrade(args) {
253
277
  const result = await upgrade({ repoRoot: ROOT, dryRun: args.includes("--dry-run") })
254
- process.exit(result.ok ? 0 : 1)
278
+ process.exitCode = result.ok ? 0 : 1
255
279
  }
256
280
 
257
281
  async function cmdDoctor(args) {
@@ -259,7 +283,7 @@ async function cmdDoctor(args) {
259
283
  const result = await doctor({ repoRoot: ROOT, offline: args.includes("--offline"), onPhase: spinner.phase })
260
284
  spinner.done()
261
285
  emit(args, args.includes("--json") ? `${JSON.stringify(result, null, 2)}\n` : reportText(args, renderDoctor, result))
262
- process.exit(result.problems ? 1 : 0)
286
+ process.exitCode = result.problems ? 1 : 0
263
287
  }
264
288
 
265
289
  async function cmdVerify(args) {
@@ -325,7 +349,7 @@ async function cmdParity(args) {
325
349
  } catch (error) {
326
350
  failFrom(error)
327
351
  }
328
- process.exit(ok ? 0 : 1)
352
+ process.exitCode = ok ? 0 : 1
329
353
  }
330
354
 
331
355
  function notAudited(message, remedy = null, exit = 1) {
@@ -359,7 +383,7 @@ async function cmdAudit(args) {
359
383
  }
360
384
  if (parsed.options.has("--json")) process.stdout.write(json)
361
385
  else process.stdout.write(`${renderAudit(document)}\n`)
362
- process.exit(document.ok ? 0 : 1)
386
+ process.exitCode = document.ok ? 0 : 1
363
387
  }
364
388
 
365
389
  /**
@@ -399,7 +423,7 @@ async function cmdWeigh(args) {
399
423
  throw error
400
424
  }
401
425
  process.stdout.write(json ? `${JSON.stringify(list.rows, null, 2)}\n` : `${renderList(list)}\n`)
402
- process.exit(0)
426
+ return
403
427
  }
404
428
  if (!target && !all) notWeighed("usage", "weigh needs a plugin: `omakit weigh <plugin-id-or-dir>`, or `omakit weigh --all` for every enabled third-party plugin.", "omakit weigh <plugin-id-or-dir>", 2)
405
429
  if (target && all) notWeighed("usage", `--all weighs every enabled third-party plugin, so ${JSON.stringify(target)} is one argument more than it takes.`, "omakit weigh --all, or omakit weigh <plugin-id-or-dir>", 2)
@@ -466,7 +490,6 @@ async function cmdWeigh(args) {
466
490
  } else {
467
491
  process.stdout.write(`\n${renderWeigh(document)}\n`)
468
492
  }
469
- process.exit(0)
470
493
  }
471
494
 
472
495
  const VERSION = JSON.parse(readFileSync(resolve(ROOT, "package.json"), "utf8")).version
@@ -180,10 +180,10 @@ export async function authenticatedUser() {
180
180
  }
181
181
 
182
182
  /** Repository issues by their creator. PRs are excluded; pagination never silently truncates. */
183
- export async function repositoryIssues(owner, repository, creator, { readJson = getJson, maxPages = 100 } = {}) {
183
+ export async function repositoryIssues(owner, repository, creator, { readJson = getJson, maxPages = 100, labels } = {}) {
184
184
  const all = []
185
185
  for (let page = 1; page <= maxPages; page += 1) {
186
- const query = new URLSearchParams({ creator, state: "open", sort: "updated", direction: "desc", per_page: "100", page: String(page) })
186
+ const query = new URLSearchParams({ ...(creator ? { creator } : {}), ...(labels ? { labels } : {}), state: "open", sort: "updated", direction: "desc", per_page: "100", page: String(page) })
187
187
  const batch = await readJson(`https://api.github.com/repos/${owner}/${repository}/issues?${query}`)
188
188
  if (!Array.isArray(batch)) throw new GitHubError("github-unavailable", "GitHub did not return an issue list")
189
189
  all.push(...batch.filter((item) => !item.pull_request))
@@ -205,6 +205,17 @@ export async function issueComments(owner, repository, number, maxPages = 10, re
205
205
  throw new GitHubError("comments-incomplete", `Issue #${number} exceeded ${maxPages} comment pages; its latest baseline cannot be determined`)
206
206
  }
207
207
 
208
+ /** Compare exact validated snapshots. The API caps its file list at 300: never classify a truncated diff. */
209
+ export async function compareCommits(repositoryUrl, previous, validated, { readJson = getJson } = {}) {
210
+ const match = String(repositoryUrl).match(/^https:\/\/github\.com\/([A-Za-z0-9_.-]+)\/([A-Za-z0-9_.-]+?)(?:\.git)?\/?$/)
211
+ if (!match || ![previous, validated].every((commit) => /^[a-f0-9]{40}$/i.test(commit))) throw new GitHubError("usage", "compare needs a repository and two full commit identifiers")
212
+ const url = `https://api.github.com/repos/${match[1]}/${match[2]}/compare/${previous}...${validated}?per_page=1`
213
+ const result = await readJson(url)
214
+ if (!["ahead", "identical"].includes(result?.status)) throw new GitHubError("compare-not-forward", "validated snapshots are not a forward comparison")
215
+ if (!Array.isArray(result.files) || result.files.length >= 300) throw new GitHubError("compare-incomplete", "compare file list unavailable or at the 300-file API limit")
216
+ return { url, files: result.files }
217
+ }
218
+
208
219
  /**
209
220
  * The current default-branch HEAD of a repository.
210
221
  *
@@ -0,0 +1,39 @@
1
+ // Reproduce M9 with GET-only issue discovery and compare reads. No sampling.
2
+ import { resolve } from "node:path"
3
+ import { pathToFileURL } from "node:url"
4
+ import { MARKETPLACE_PIN } from "./pin.mjs"
5
+ import { repositoryIssues } from "./github.mjs"
6
+ import { liveRegistry } from "./registry.mjs"
7
+ import { reviewPolicy } from "./review-cost.mjs"
8
+ import { validationWatchAll } from "./watch.mjs"
9
+
10
+ export async function measureReviewCost(repoRoot) {
11
+ const policy = await reviewPolicy(repoRoot)
12
+ const registry = await liveRegistry({ repoRoot })
13
+ if (registry.source !== "head") throw new Error(`cannot measure at HEAD: ${registry.reason}`)
14
+ const [owner, repository] = new URL(MARKETPLACE_PIN.repository).pathname.slice(1).split("/")
15
+ const openedAt = new Date().toISOString()
16
+ const subjects = await repositoryIssues(owner, repository, undefined, { labels: policy.updateLabel })
17
+ const issues = subjects.filter((subject) => subject.state === "open" && !subject.pull_request).map((subject) => ({
18
+ number: subject.number, url: `${MARKETPLACE_PIN.repository}/issues/${subject.number}`, title: subject.title,
19
+ state: subject.state, labels: subject.labels.map((label) => typeof label === "string" ? label : label.name),
20
+ }))
21
+ // Only the manual-review subset needs comment reads and comparisons. No plugin HEAD reads.
22
+ const manual = issues.filter((subject) => subject.labels.includes(policy.reviewLabel))
23
+ const batch = await validationWatchAll({ repoRoot, discovery: { account: null, marketplace: MARKETPLACE_PIN.repository, issues: manual },
24
+ readRegistry: async () => registry, github: { defaultBranchHead: async () => null } })
25
+ const counts = batch.reviewCostSummary
26
+ return {
27
+ measurement: "M9", openedAt, completedAt: new Date().toISOString(), marketplace: MARKETPLACE_PIN.repository,
28
+ marketplaceHead: registry.commit, command: "node tools/marketplace/measure-review-cost.mjs", sample: false,
29
+ pluginUpdates: issues.length, manualQueue: manual.length, manualQueueShare: issues.length ? manual.length / issues.length : null,
30
+ docsOnly: counts.docsOnly, compared: counts.compared, skipped: counts.skipped.length,
31
+ docsOnlyShareOfCompared: counts.compared ? counts.docsOnly / counts.compared : null,
32
+ docsOnlyShareOfManualQueue: counts.skipped.length || !manual.length ? null : counts.docsOnly / manual.length,
33
+ rows: batch.issues.map((row) => ({ issue: row.issue.number, ...row.documentationDiff })),
34
+ }
35
+ }
36
+
37
+ if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) {
38
+ process.stdout.write(`${JSON.stringify(await measureReviewCost(resolve(process.cwd())), null, 2)}\n`)
39
+ }
@@ -0,0 +1,76 @@
1
+ // M6 reproduction: open author-fixes issues, bot marker against commits.atom.
2
+ // All remote reads use the existing GET-only client. Unknowns stay unknown.
3
+ import { resolve } from "node:path"
4
+ import { pathToFileURL } from "node:url"
5
+ import { MARKETPLACE_PIN } from "./pin.mjs"
6
+ import { repositoryIssues, getText } from "./github.mjs"
7
+ import { validationWatch } from "./watch.mjs"
8
+
9
+ export function atomHead(feed, source) {
10
+ const commit = feed.match(/<id>tag:github\.com,2008:Grit::Commit\/([0-9a-f]{40})<\/id>/i)?.[1]
11
+ || feed.match(/\/commit\/([0-9a-f]{40})/i)?.[1]
12
+ if (!commit) throw new Error("No full HEAD commit in commits.atom")
13
+ return { source, commit: commit.toLowerCase(), branch: null,
14
+ committedAt: feed.match(/<updated>([^<]+)<\/updated>/)?.[1] || null }
15
+ }
16
+
17
+ export function stalenessCounts(rows) {
18
+ const stale = rows.filter((row) => row.verdict === "stale").length
19
+ const current = rows.filter((row) => row.verdict === "current").length
20
+ const compared = stale + current
21
+ return { total: rows.length, compared, stale, current, unknown: rows.length - compared,
22
+ staleShareOfCompared: compared ? stale / compared : null,
23
+ staleShareOfPopulation: compared === rows.length && rows.length ? stale / rows.length : null }
24
+ }
25
+
26
+ export async function measureStaleness(repoRoot, { discover = repositoryIssues, watch = validationWatch, readText = getText } = {}) {
27
+ const openedAt = new Date().toISOString()
28
+ const [owner, repository] = new URL(MARKETPLACE_PIN.repository).pathname.slice(1).split("/")
29
+ const batches = await Promise.all(["needs-fixes", "security-needs-fixes"].map((labels) => discover(owner, repository, undefined, { labels })))
30
+ const subjects = [...new Map(batches.flat().map((subject) => [subject.number, subject])).values()]
31
+ .filter((subject) => subject.state === "open" && !subject.pull_request)
32
+ .sort((a, b) => a.number - b.number)
33
+ const heads = new Map()
34
+ function readHead(url) {
35
+ if (!heads.has(url)) {
36
+ const source = `${url.replace(/\/$/, "")}/commits.atom`
37
+ heads.set(url, Promise.resolve().then(async () => atomHead(await readText(source, "application/atom+xml"), source)))
38
+ }
39
+ return heads.get(url)
40
+ }
41
+ const rows = new Array(subjects.length)
42
+ let next = 0
43
+ async function worker() {
44
+ while (next < subjects.length) {
45
+ const index = next++
46
+ const subject = subjects[index]
47
+ const issueUrl = `${MARKETPLACE_PIN.repository}/issues/${subject.number}`
48
+ const observedAt = new Date().toISOString()
49
+ try {
50
+ const report = await watch({ repoRoot, issueUrl, github: { issue: async () => subject, defaultBranchHead: readHead } })
51
+ rows[index] = { issue: subject.number, issueUrl, observedAt, completedAt: new Date().toISOString(),
52
+ repository: report.plugin.repository, validatedCommit: report.validated?.commit || null,
53
+ validationSource: report.validated?.source || null, validatedAt: report.validated?.checkedAt || null,
54
+ validationCommentsSource: `https://api.github.com/repos/${owner}/${repository}/issues/${subject.number}/comments`,
55
+ defaultBranchHead: report.head?.commit || null, headSource: report.head?.source || null,
56
+ verdict: report.verdict.state,
57
+ reason: report.verdict.state === "unknown" ? report.verdict.summary : null }
58
+ } catch (error) {
59
+ rows[index] = { issue: subject.number, issueUrl, observedAt, completedAt: new Date().toISOString(),
60
+ repository: null, validatedCommit: null, defaultBranchHead: null, verdict: "unknown",
61
+ reason: `${error.code || "read-unavailable"}: ${error.message}` }
62
+ }
63
+ }
64
+ }
65
+ await Promise.all(Array.from({ length: Math.min(4, subjects.length) }, worker))
66
+ return { measurement: "M6", date: openedAt.slice(0, 10), openedAt, completedAt: new Date().toISOString(),
67
+ marketplace: MARKETPLACE_PIN.repository, marketplacePin: MARKETPLACE_PIN.commit,
68
+ command: "node tools/marketplace/measure-staleness.mjs", sample: false,
69
+ population: "All open non-PR marketplace issues labelled needs-fixes or security-needs-fixes at discovery",
70
+ method: "Latest bot security-baseline full commit against default-branch commits.atom HEAD; missing full markers or feeds are unknown; unequal commits are stale, without a claim about ancestry",
71
+ ...stalenessCounts(rows), rows }
72
+ }
73
+
74
+ if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) {
75
+ process.stdout.write(`${JSON.stringify(await measureStaleness(resolve(process.cwd())), null, 2)}\n`)
76
+ }
@@ -266,12 +266,19 @@ export function renderWatchList(result, { colour = colourEnabled() } = {}) {
266
266
  /** Compact batch report; exact commits and full discussion remain in JSON. */
267
267
  export function renderWatchAll(result, { colour = colourEnabled() } = {}) {
268
268
  const c = styler(colour)
269
- const out = [...field("account", result.account, c), ...field("issues", `${result.summary.total} checked; ${result.summary.current} current, ${result.summary.stale} stale, ${result.summary.unknown} unknown`, c), ""]
269
+ const out = [...field("account", result.account, c), ...field("issues", `${result.summary.total} checked; ${result.summary.current} current, ${result.summary.stale} stale, ${result.summary.unknown} unknown`, c)]
270
+ if (result.reviewCostSummary) {
271
+ const cost = result.reviewCostSummary
272
+ const skipped = cost.skipped.length ? `; ${cost.skipped.length} diff(s) skipped (reasons on issue rows)` : ""
273
+ out.push(...field("review cost", `${cost.manualQueue} of ${cost.pluginUpdates} plugin-update issue(s) on security-review-required; ${cost.docsOnly} docs-only validated diff(s) of ${cost.compared} compared${skipped}`, c))
274
+ }
275
+ out.push("")
270
276
  for (const row of result.issues) {
271
277
  const state = row.report?.verdict.state || "unknown"
272
278
  const style = { current: "pass", stale: "fail", unknown: "unknown" }[state]
273
279
  out.push(...verdict(style, state.toUpperCase(), `#${row.issue.number} ${watchIssueTitle(row.report?.read.title || row.issue.title)}`, c))
274
280
  out.push(...field("issue", row.issue.url, c, { wrapValue: false }))
281
+ if (row.documentationDiff?.docsOnly === null) out.push(...field("diff skipped", watchIssueTitle(row.documentationDiff.reason), c))
275
282
  if (row.error) {
276
283
  out.push(...field("read error", watchIssueTitle(`${row.error.code}: ${row.error.message}`), c))
277
284
  } else {
@@ -0,0 +1,107 @@
1
+ // Review cost is advice, never a marketplace rule. Evidence: MEASUREMENTS.md M4 and M9.
2
+ import { readFileSync } from "node:fs"
3
+ import { join } from "node:path"
4
+ import { pathToFileURL } from "node:url"
5
+ import { requirePin } from "./pin.mjs"
6
+ import { token, issue, parseIssueUrl, compareCommits } from "./github.mjs"
7
+ import { discoverWatchIssues, repositoryFor } from "./watch.mjs"
8
+ import { repositorySlug } from "./registry.mjs"
9
+
10
+ /** Outcome and update label names come from the pin, rather than a second policy. */
11
+ export async function reviewPolicy(repoRoot) {
12
+ const { dir } = requirePin(repoRoot)
13
+ const policy = await import(pathToFileURL(join(dir, "scripts/security-baseline-policy.mjs")).href)
14
+ const approval = readFileSync(join(dir, "scripts/approve-plugin-update.mjs"), "utf8")
15
+ const updateLabel = approval.match(/for \(const required of \["([^"]+)"/)?.[1]
16
+ if (!updateLabel) throw new Error("cannot read the update issue label from the pin")
17
+ const manual = policy.currentSecurityBaselinePolicy.maintainerVerificationOutcome
18
+ return { automated: policy.securityBaselineOutcome([], []), manual, updateLabel, reviewLabel: `security-${manual}` }
19
+ }
20
+
21
+ /** The same discovery as watch --all; no credential or incomplete reads are not zero issues. */
22
+ export async function openIssuesForRepository({ repoRoot, repository, offline = false, github = {} }) {
23
+ if (offline) return { count: null, reason: "not checked (--offline)" }
24
+ if (!(github.token || token)()) return { count: null, reason: "not checked: no GitHub credential" }
25
+ try {
26
+ const discovery = await discoverWatchIssues({ github })
27
+ const { dir } = requirePin(repoRoot)
28
+ let count = 0
29
+ let next = 0
30
+ async function worker() {
31
+ while (next < discovery.issues.length) {
32
+ const target = parseIssueUrl(discovery.issues[next++].url)
33
+ const subject = await (github.issue || issue)(target.owner, target.repository, target.number)
34
+ const parsed = await repositoryFor(dir, subject)
35
+ if (!parsed.url) throw new Error(`repository unknown on issue #${target.number}: ${parsed.error}`)
36
+ if (repositorySlug(parsed.url) === repositorySlug(repository)) count += 1
37
+ }
38
+ }
39
+ await Promise.all(Array.from({ length: Math.min(4, discovery.issues.length) }, worker))
40
+ return { count, reason: `watch --all discovery: ${count} open issue(s) for this repository` }
41
+ } catch (error) {
42
+ return { count: null, reason: `not checked (${error.code || "issue-discovery-unavailable"}): ${error.message}` }
43
+ }
44
+ }
45
+
46
+ export function reviewCostVerdict({ baseline, policy, openIssues = { count: null, reason: "not checked" }, why }) {
47
+ const capabilities = [...(baseline?.capabilities || [])]
48
+ const manual = baseline?.outcome === policy.manual
49
+ const automated = baseline?.outcome === policy.automated
50
+ const outcome = manual ? openIssues.count > 0 ? "manual queue, again" : "manual queue" : automated ? "automated" : null
51
+ const detail = manual
52
+ ? `${outcome}: capabilities ${capabilities.join(", ") || "none"}. Every update of this plugin, including a docs-only one, lands in the manual queue.${openIssues.count > 0 ? ` You already have ${openIssues.count} open issue(s) for this repository.` : ""}`
53
+ : automated ? "automated: this update will not need a human for the security baseline." : "not checked: baseline.preflight has no passed or review-required outcome"
54
+ return {
55
+ reviewCost: { outcome, capabilities, openIssuesForRepository: openIssues.count, reason: `${detail} ${openIssues.reason}` },
56
+ check: {
57
+ id: "review.cost", source: "omakit", severity: "advisory", verdict: manual ? "fail" : automated ? "pass" : "unknown",
58
+ why, detail, paths: [],
59
+ remedy: manual && openIssues.count > 0 ? "consider batching: close or fold the open one before opening another" : null,
60
+ },
61
+ }
62
+ }
63
+
64
+ /** A rename must be documentation at both ends; an empty diff is not a docs-only update. */
65
+ export function documentationPath(path) {
66
+ return typeof path === "string" && !path.split("/").some((part) => part === "..") &&
67
+ (/^docs\//i.test(path) || /\.md$/i.test(path) || /(?:^|\/)LICENSE$/i.test(path) || /\.(?:png|jpe?g|gif|webp|svg|ico|avif|bmp|tiff?)$/i.test(path))
68
+ }
69
+
70
+ export function docsOnlyFiles(files) {
71
+ return Array.isArray(files) && files.length > 0 && files.every((file) =>
72
+ documentationPath(file.filename) && (!file.previous_filename || documentationPath(file.previous_filename)))
73
+ }
74
+
75
+ /** Select a dated validation record, never a parent guessed from Git history. */
76
+ export function previousValidatedCommit(registry, report, catalog = null) {
77
+ if (!repositorySlug(report.plugin.repository)) return null
78
+ const sources = Array.isArray(registry?.sources) ? registry.sources : Object.values(registry?.sources || {})
79
+ const source = sources.find((entry) => repositorySlug(entry.repo) === repositorySlug(report.plugin.repository))
80
+ if (!report.validated?.commit) return null
81
+ const target = report.validated.commit
82
+ const checkedAt = Date.parse(report.validated.checkedAt)
83
+ const candidates = [...(Array.isArray(source?.listingValidationHistory) ? source.listingValidationHistory : []), {
84
+ commit: source?.listingValidatedCommit, validatedAt: source?.listingValidatedAt,
85
+ }, { commit: report.previousValidated?.commit, validatedAt: report.previousValidated?.checkedAt }]
86
+ // The catalog also records successful upstream validation before an update
87
+ // is promoted to the listing. This is validation evidence, not branch HEAD.
88
+ for (const plugin of Array.isArray(catalog?.plugins) ? catalog.plugins : []) {
89
+ if (repositorySlug(plugin.repo) === repositorySlug(report.plugin.repository)) {
90
+ candidates.push({ commit: plugin.upstreamValidatedCommit, validatedAt: plugin.upstreamValidatedAt })
91
+ }
92
+ }
93
+ return candidates.filter((entry) => /^[a-f0-9]{40}$/i.test(entry.commit || "") && entry.commit.toLowerCase() !== target.toLowerCase() &&
94
+ Number.isFinite(Date.parse(entry.validatedAt)) && Date.parse(entry.validatedAt) <= checkedAt)
95
+ .sort((a, b) => Date.parse(a.validatedAt) - Date.parse(b.validatedAt)).at(-1)?.commit.toLowerCase() || null
96
+ }
97
+
98
+ export async function validatedDocumentationDiff({ report, registry, catalog, registryReason = null, compare = compareCommits }) {
99
+ const previous = previousValidatedCommit(registry, report, catalog)
100
+ if (!previous) return { previousCommit: null, validatedCommit: report.validated?.commit || null, docsOnly: null, files: null, source: null, reason: registryReason || "previous validated commit unknown in the marketplace registry" }
101
+ try {
102
+ const diff = await compare(report.plugin.repository, previous, report.validated.commit)
103
+ return { previousCommit: previous, validatedCommit: report.validated.commit, docsOnly: docsOnlyFiles(diff.files), files: diff.files.length, source: diff.url, reason: null }
104
+ } catch (error) {
105
+ return { previousCommit: previous, validatedCommit: report.validated.commit, docsOnly: null, files: null, source: null, reason: `${error.code || "compare-unavailable"}: ${error.message}` }
106
+ }
107
+ }
@@ -23,6 +23,7 @@ import { renderIssue, verifyAgainstOfficialParser } from "./issue.mjs"
23
23
  import { defaultBranchHead } from "./github.mjs"
24
24
  import { REFRESH_ACTION } from "./watch.mjs"
25
25
  import { omakitCacheDir } from "./paths.mjs"
26
+ import { openIssuesForRepository, reviewCostVerdict, reviewPolicy } from "./review-cost.mjs"
26
27
 
27
28
  /**
28
29
  * One arrow per cause, in this order, so a person fixes the thing that is
@@ -135,6 +136,7 @@ export function reproduceCommand({ target, category, tags, pluginName, notes, su
135
136
  * @param {{ repoRoot: string, target: string, category?: string, tags?: string|string[],
136
137
  * notes?: string, suggestedTag?: string, pluginName?: string,
137
138
  * allowDirty?: boolean, offline?: boolean,
139
+ * github?: object,
138
140
  * readRegistry?: typeof liveRegistry,
139
141
  * chooser?: (question: { contract: object, defaults: object, missing: string[] }) => Promise<{ category?: string, tags?: string[] }> }} options
140
142
  * `readRegistry` is injectable for tests; the default reads the marketplace's
@@ -404,7 +406,7 @@ export async function submitPreflight(options) {
404
406
  if (!options.offline && subject.repository.url) {
405
407
  phase("reading the repository's default-branch HEAD")
406
408
  try {
407
- head = await defaultBranchHead(subject.repository.url)
409
+ head = await (options.github?.defaultBranchHead || defaultBranchHead)(subject.repository.url)
408
410
  } catch (error) {
409
411
  headError = { code: error.code || "head-unreadable", message: error.message }
410
412
  }
@@ -412,9 +414,14 @@ export async function submitPreflight(options) {
412
414
  const validationMatches = head ? head.commit === subject.commit.toLowerCase() : null
413
415
  checks.push(check("submission.validation-commit", {
414
416
  source: "omakit",
415
- why: "The marketplace validates the default-branch HEAD it resolves when the issue is opened or edited, not the commit checked here. 73% of the 464 submissions parked in the author's court have a HEAD ahead of their validated commit, so a preflight against a commit that is not the pushed HEAD describes a tree nobody will review. Not a marketplace rule; an Omakit refusal to report on the wrong tree.",
417
+ why: "The marketplace validates the default-branch HEAD it resolves when the issue is opened or edited, not the commit checked here. M6 on 2026-09-15 found 326/519 readable author-fixes comparisons stale (62.8%), with 64 of 583 issues unknown, so a preflight against a commit that is not the pushed HEAD describes a tree nobody will review. Not a marketplace rule; an Omakit refusal to report on the wrong tree.",
416
418
  severity: options.offline ? "advisory" : "blocking",
417
419
  skipped: options.offline === true,
420
+ // No origin, no URL to read a HEAD from: the check waits on the one that
421
+ // says so. Measured on 0.4.1: it failed as a second root cause with
422
+ // "could not read the default-branch HEAD (unknown): " for a read that
423
+ // was never attempted.
424
+ waitedOn: [!options.offline && !subject.repository.url && "submission.repository-url"],
418
425
  verdict: validationMatches === true,
419
426
  detail: options.offline
420
427
  ? `not checked (--offline). Local commit ${subject.commit}.`
@@ -461,6 +468,14 @@ export async function submitPreflight(options) {
461
468
  : null,
462
469
  }))
463
470
 
471
+ const policy = await reviewPolicy(repoRoot)
472
+ const review = reviewCostVerdict({ baseline: consequence, policy,
473
+ openIssues: consequence?.outcome === policy.manual ? await openIssuesForRepository({ repoRoot, repository: subject.repository.url,
474
+ offline: options.offline === true, github: options.github }) : { count: null, reason: consequence?.outcome === policy.automated ? "open issue count not checked: automated baseline" : preflight.skipReason || preflight.refusal?.message || "baseline outcome needs findings resolved" },
475
+ why: `MEASUREMENTS.md M4: ${figure(figures.outcomes[policy.manual] || 0)} of ${figure(figures.withBaseline)} recorded listing baselines required review at the pin. M9: on 2026-09-15, 140 of 307 open update issues carried the manual-review label; 4 of 139 compared validated diffs were docs-only, with 1 unavailable. The baseline scans the whole snapshot, not the update diff, so unchanged capabilities also require another review. Sources and exact marketplace HEAD are recorded in MEASUREMENTS.md.`,
476
+ })
477
+ checks.push(review.check)
478
+
464
479
  const blocking = checks.filter((entry) => entry.severity === "blocking" && entry.verdict === "fail")
465
480
  const advisory = checks.filter((entry) => entry.severity === "advisory" && entry.verdict === "fail")
466
481
  const unknown = checks.filter((entry) => entry.verdict === "unknown")
@@ -511,6 +526,7 @@ export async function submitPreflight(options) {
511
526
  offline: options.offline === true,
512
527
  }),
513
528
  checks,
529
+ reviewCost: review.reviewCost,
514
530
  outcome,
515
531
  ready,
516
532
  listing,
@@ -9,19 +9,11 @@
9
9
  // compares branch HEADs only for repositories that are already listed. So
10
10
  // pushing a fix does nothing, and commenting "fixed in abc123" does nothing.
11
11
  //
12
- // Measured reason (docs/MEASUREMENTS.md M6): of the 464 submissions parked in
13
- // the author's court, 73% have a default-branch HEAD ahead of the validated
14
- // commit. 47% pushed after the maintainer's review without the marketplace ever
15
- // seeing it, and 82% of those authors also commented, so they are engaged and
16
- // stuck rather than gone. Of 13 open submissions inspected with no labels left,
17
- // 9 had passed validation and passed the automated security baseline with zero
18
- // findings and were blocked solely because their validated commit had fallen
19
- // behind while they waited. 46% of the maintainer's own requests for a fresh validation never
20
- // produced one; in the parked group 77% never did. The instruction that would
21
- // fix this appears 22 times in the failure path of
22
- // `scripts/submission-feedback.mjs` and zero times in the success path of
23
- // `scripts/validate-submission.mjs`, which is the path 97 of 100 parked
24
- // submissions took.
12
+ // Measured reason (docs/MEASUREMENTS.md M6), 2026-09-15: 326 of 519 readable
13
+ // author-fixes comparisons were stale (62.8%), with 64 of 583 issues unknown.
14
+ // The original 2026-09-12 research sampled 100 of a 464-issue queue: 68 of
15
+ // 93 readable pairs were stale (73.1%). It did not measure every queue item.
16
+ // Editing the issue body is the action the pinned workflow observes.
25
17
  //
26
18
  // This command reads. It never edits the issue, never comments, never labels.
27
19
  // The action it names is the author's to take.
@@ -30,6 +22,8 @@ import { join } from "node:path"
30
22
  import { pathToFileURL } from "node:url"
31
23
  import { MARKETPLACE_PIN, requirePin } from "./pin.mjs"
32
24
  import { authenticatedUser, repositoryIssues, defaultBranchHead, issue, issueComments, parseIssueUrl, token, GitHubError } from "./github.mjs"
25
+ import { liveRegistry } from "./registry.mjs"
26
+ import { reviewPolicy, validatedDocumentationDiff } from "./review-cost.mjs"
33
27
 
34
28
  export class WatchError extends Error {
35
29
  constructor(code, message) {
@@ -77,8 +71,11 @@ export async function discoverWatchIssues({ user, github = {}, onPhase = () => {
77
71
  }
78
72
 
79
73
  /** A batch keeps independent read failures visible and shares repository HEAD reads. */
80
- export async function validationWatchAll({ repoRoot, discovery, github = {}, onPhase = () => {} }) {
74
+ export async function validationWatchAll({ repoRoot, discovery, github = {}, readRegistry = liveRegistry, onPhase = () => {} }) {
81
75
  if (discovery.issues.length) requirePin(repoRoot)
76
+ const policy = discovery.issues.length ? await reviewPolicy(repoRoot) : null
77
+ let registry = null
78
+ const reviewCostSummary = { pluginUpdates: 0, manualQueue: 0, docsOnly: 0, compared: 0, skipped: [] }
82
79
  const heads = new Map()
83
80
  const readHead = github.defaultBranchHead || defaultBranchHead
84
81
  const shared = { ...github, defaultBranchHead: (url) => {
@@ -102,7 +99,38 @@ export async function validationWatchAll({ repoRoot, discovery, github = {}, onP
102
99
  await Promise.all(Array.from({ length: Math.min(4, discovery.issues.length) }, worker))
103
100
  const summary = { total: results.length, current: 0, stale: 0, unknown: 0 }
104
101
  for (const result of results) summary[result.report?.verdict.state || "unknown"] += 1
105
- return { mode: "all", account: discovery.account, marketplace: discovery.marketplace, summary, issues: results }
102
+ // M9: count labels on the listed issues, then compare only updates in the manual queue.
103
+ const manual = results.filter((row) => {
104
+ const labels = row.report?.read.labels || row.issue.labels
105
+ if (!labels.includes(policy?.updateLabel)) return false
106
+ reviewCostSummary.pluginUpdates += 1
107
+ return labels.includes(policy.reviewLabel)
108
+ })
109
+ reviewCostSummary.manualQueue = manual.length
110
+ if (manual.length) {
111
+ onPhase("reading the previous validated marketplace snapshots")
112
+ try { registry = await readRegistry({ repoRoot }) } catch (error) { registry = { reason: error.message } }
113
+ }
114
+ let compareNext = 0
115
+ async function compareWorker() {
116
+ while (compareNext < manual.length) {
117
+ const row = manual[compareNext++]
118
+ const diff = row.report
119
+ ? await validatedDocumentationDiff({ report: row.report, registry: registry?.source === "head" ? registry.registry : null,
120
+ catalog: registry?.source === "head" ? registry.catalog : null,
121
+ registryReason: registry?.source === "head" ? null : `previous validated commit unknown: live registry unavailable (${registry?.reason || "unknown"})`, compare: github.compareCommits })
122
+ : { docsOnly: null, reason: `issue unreadable: ${row.error?.message || "unknown"}` }
123
+ row.documentationDiff = diff
124
+ if (diff.docsOnly === null) reviewCostSummary.skipped.push({ issue: row.issue.number, reason: diff.reason })
125
+ else {
126
+ reviewCostSummary.compared += 1
127
+ if (diff.docsOnly) reviewCostSummary.docsOnly += 1
128
+ }
129
+ }
130
+ }
131
+ await Promise.all(Array.from({ length: Math.min(4, manual.length) }, compareWorker))
132
+ reviewCostSummary.skipped.sort((a, b) => a.issue - b.issue)
133
+ return { mode: "all", account: discovery.account, marketplace: discovery.marketplace, summary, reviewCostSummary, issues: results }
106
134
  }
107
135
 
108
136
  // The one action that re-runs validation, in the register the marketplace itself
@@ -133,7 +161,7 @@ async function loadVerification(pinDir) {
133
161
  * happens to recognise. So each form is read by the parser the marketplace
134
162
  * itself uses for it, and the issue says which one it is.
135
163
  */
136
- async function repositoryFor(pinDir, subject) {
164
+ export async function repositoryFor(pinDir, subject) {
137
165
  const submission = await loadSubmission(pinDir)
138
166
  const verification = await loadVerification(pinDir)
139
167
  const title = String(subject.title || "")
@@ -221,11 +249,29 @@ export async function validationWatch({ repoRoot, issueUrl, onPhase, github = {}
221
249
  baselineError = { code: error.code || "baseline-unreadable", message: error.message }
222
250
  }
223
251
  const fallback = validated ? null : validationCommentCommit(comments)
252
+ let previousValidated = null
253
+ if (validated) {
254
+ // Revalidation on an existing issue may precede the latest marker. Ignore
255
+ // repeated attestations of the same commit, not a different validated tree.
256
+ for (const comment of [...comments].reverse()) {
257
+ try {
258
+ const marker = record.findLatestSecurityBaseline([comment])
259
+ if (marker && marker.commitSha !== validated.commit && Date.parse(marker.checkedAt) <= Date.parse(validated.checkedAt)) {
260
+ previousValidated = { commit: marker.commitSha, checkedAt: marker.checkedAt, source: "security-baseline-marker" }
261
+ break
262
+ }
263
+ } catch { /* An incomplete run is not a previously validated commit. */ }
264
+ }
265
+ }
224
266
 
225
267
  const labels = (subject.labels || []).map((label) => (typeof label === "string" ? label : label?.name)).filter(Boolean)
226
268
  const authorComments = comments.filter((comment) => comment?.user?.login === subject.user?.login)
269
+ // The discussion is a person's: the marketplace's own bot and any other
270
+ // automation (GitHub marks an app's account `type: "Bot"`, and names it
271
+ // `<app>[bot]`) is neither a reviewer nor the last human review.
272
+ const isBot = (user) => user?.type === "Bot" || /\[bot\]$/i.test(String(user?.login || ""))
227
273
  const maintainerComments = comments.filter(
228
- (comment) => comment?.user?.login && comment.user.login !== subject.user?.login && comment.user.login !== "github-actions[bot]",
274
+ (comment) => comment?.user?.login && comment.user.login !== subject.user?.login && !isBot(comment.user),
229
275
  )
230
276
 
231
277
  let head = null
@@ -267,6 +313,7 @@ export async function validationWatch({ repoRoot, issueUrl, onPhase, github = {}
267
313
  },
268
314
  plugin: { repository: repositoryUrl, repositoryError, form: issueKind },
269
315
  validated,
316
+ previousValidated,
270
317
  validationCommentFallback: fallback,
271
318
  baselineError,
272
319
  head,
@@ -331,7 +331,12 @@ async function sampleConfig({ label, runIndex, config, plan, env, procRoot, sign
331
331
  onPhase(`run ${runIndex} of ${plan.runs}: ${label}, sampling for ${windowSeconds}s`)
332
332
  const started = utc()
333
333
  const t0 = Date.now()
334
- const cpu0 = cpuTicks(procRoot, pid) ?? 0
334
+ // A pid with no stat is a process that is gone, and a gone shell has no
335
+ // sample: nothing is read as zero in its place. Measured on 0.4.1: a pid
336
+ // absent from /proc "completed" the window with 0 ticks and a Pss of
337
+ // null read as 0 MB, a delta of minus the whole baseline.
338
+ const cpu0 = cpuTicks(procRoot, pid)
339
+ if (cpu0 === null) return { label, run: runIndex, failed: `the shell (pid ${pid}) is not in ${procRoot}` }
335
340
  const child0 = childTicks(procRoot, pid) ?? 0
336
341
  const rows = []
337
342
  const trace = []
@@ -355,7 +360,8 @@ async function sampleConfig({ label, runIndex, config, plan, env, procRoot, sign
355
360
  } catch {
356
361
  configRewritten = null
357
362
  }
358
- const cpu1 = cpuTicks(procRoot, pid) ?? cpu0
363
+ const cpu1 = cpuTicks(procRoot, pid)
364
+ if (cpu1 === null) return { label, run: runIndex, failed: `the shell (pid ${pid}) disappeared from ${procRoot} during the window` }
359
365
  const child1 = childTicks(procRoot, pid) ?? child0
360
366
  const memory = { pssKb: pssKb(procRoot, pid), rssKb: rssKb(procRoot, pid), memoryAt: "window-end", ...settled, trace }
361
367
  const seconds = (t1 - t0) / 1000
@@ -605,6 +611,13 @@ export async function measureWeigh(plan, { env = plan.env || process.env, procRo
605
611
  let restore = null
606
612
  let restoreProblem = null
607
613
  let comeBack = null
614
+ // What stopped the measurement early, an interrupt or a thrown error, is
615
+ // kept so the restore's outcome can be judged first: a restore that failed
616
+ // or did not verify is the fact the person is left with, and it is what
617
+ // is reported, with the stop as its context. Measured on 0.4.1: an
618
+ // interrupt propagated past a restore whose md5 differed, and the entry
619
+ // point closed with "shell.json was restored".
620
+ let stopped = null
608
621
  try {
609
622
  for (let runIndex = 1; runIndex <= plan.runs; runIndex += 1) {
610
623
  for (const { label, config } of configs) {
@@ -619,6 +632,8 @@ export async function measureWeigh(plan, { env = plan.env || process.env, procRo
619
632
  samples.push(sample)
620
633
  }
621
634
  }
635
+ } catch (error) {
636
+ stopped = error
622
637
  } finally {
623
638
  onPhase("restoring shell.json and restarting the shell")
624
639
  try {
@@ -642,7 +657,10 @@ export async function measureWeigh(plan, { env = plan.env || process.env, procRo
642
657
  onLine({ state: "fail", text: `the restore failed: ${error.message}; the backup is ${backup.backupFile}` })
643
658
  }
644
659
  }
645
- if (restoreProblem) throw new WeighError("restore-failed", `shell.json could not be restored from ${backup.backupFile}: ${restoreProblem.message}`, `Copy ${backup.backupFile} over ${plan.configFile} yourself, then run omarchy-restart-shell.`)
660
+ const because = stopped ? `. The measurement had stopped first: ${stopped.message}` : ""
661
+ if (restoreProblem) throw new WeighError("restore-failed", `shell.json could not be restored from ${backup.backupFile}: ${restoreProblem.message}${because}`, `Copy ${backup.backupFile} over ${plan.configFile} yourself, then run omarchy-restart-shell.`)
662
+ if (stopped && !restore.restored) throw new WeighError("restore-unverified", `shell.json differs from the backup after the restore; the backup ${backup.backupFile} is kept${because}`, `Compare ${backup.backupFile} with ${plan.configFile} and copy it over if the difference is not yours.`)
663
+ if (stopped) throw stopped
646
664
  const ended = utc()
647
665
  const config = {
648
666
  path: plan.configFile,