proof-of-change 1.2.0 → 1.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -15,7 +15,7 @@ npm install --save-dev proof-of-change
15
15
  npx pocp init --hooks --github
16
16
  ```
17
17
 
18
- `init` creates `.proof/`, installs git hooks (yours keep running) and, with `--github`, writes the workflows. Then:
18
+ `init` creates `.proof/`, installs git hooks (yours keep running) and, with `--github`, writes the workflows and creates the approval labels. Then:
19
19
 
20
20
  1. **Say what must be proven.** A rule in `.proof/rules.json` maps files to a check, which is any command (`npm test`, `pytest`, `go test ./...`):
21
21
 
@@ -35,6 +35,8 @@ npx pocp init --hooks --github
35
35
  npx pocp verify origin/main..HEAD --require-coverage --rerun
36
36
  ```
37
37
 
38
+ **Private repositories pay for CI in GitHub Actions minutes.** POCP's `verify` re-runs your tests, so each pull request costs about one extra test run, plus about a minute for each push or approval. On a free plan, check you have minutes to spare. Without Actions, only the local git hook checks changes.
39
+
38
40
  [docs/QUICKSTART.md](docs/QUICKSTART.md) walks through all of this on a non-JavaScript project.
39
41
 
40
42
  ## Documentation
@@ -61,7 +61,7 @@ Within the limits in [docs/THREAT-MODEL.md](THREAT-MODEL.md):
61
61
  - It can't fully stop a pull request from faking its own check. A pull request runs its own copy of the workflow. With the `pocp` status required, a pull request that edits CI files stays yellow until a maintainer approves the edit, and pull requests from forks can't post a status at all. But a branch pushed to your repository by someone with write access, including an agent using your token, can edit its workflow to post its own `pocp` status. Read the diff of any such pull request that touches `.github/` ([docs/STATUS.md](STATUS.md#a-limit-of-the-pocp-status)). A dedicated GitHub App for the reporter, which would close this, is planned.
62
62
  - It doesn't measure new tests. The test check compares a change with tests that already existed, so a weak test added in the same pull request isn't measured.
63
63
  - It isn't hermetic. Checks can read the network, environment variables and installed packages, and the re-runs execute the pull request's code in your CI job.
64
- - It adds CI time. `verify` re-runs your required checks, so each pull request costs about one extra test run, in parallel with your other jobs. On POCP's own pull requests in fast mode, `verify` took 3 to 4 minutes next to test jobs of 3 to 4 minutes, and 10 to 11 minutes when several commits edited tests, because the old tests ran as the whole suite. A rule's `changed_test_command` runs only the edited test files instead ([docs/REFERENCE.md](REFERENCE.md#faster-test-edits-changed_test_command)). It hasn't been measured on other projects yet.
64
+ - It adds CI time, and on a private repository GitHub bills it as Actions minutes. The status and label workflows skip events that can't change the result (other labels, title edits), but each push and approval still costs about a minute. `verify` re-runs your required checks, so each pull request costs about one extra test run, in parallel with your other jobs. On POCP's own pull requests in fast mode, `verify` took 3 to 4 minutes next to test jobs of 3 to 4 minutes, and 10 to 11 minutes when several commits edited tests, because the old tests ran as the whole suite. A rule's `changed_test_command` runs only the edited test files instead ([docs/REFERENCE.md](REFERENCE.md#faster-test-edits-changed_test_command)). It hasn't been measured on other projects yet.
65
65
  - Deep mode is slow. It re-runs the required checks for each planted bug, so its cost grows with your suite's run time. On POCP's own repository it took up to about 17 minutes locally. One measured case: on 2026-11-02, a full `verify` of a 13-commit pull request to POCP that edited test helpers took about 25 minutes on a GitHub runner, of which the test check took about 16 ([run](https://github.com/harriswekselblatt/pocp/actions/runs/41524329250)).
66
66
  - A process a check starts in a new session (`setsid`, a daemon) can outlive the check.
67
67
  - Native Windows isn't supported. The hooks and checks need a POSIX shell: Linux, macOS, or Windows through WSL.
package/docs/STATUS.md CHANGED
@@ -27,6 +27,7 @@ Each label covers one thing, only on the head it was added to. Every push remove
27
27
  - the label was last added by someone with write access, after the approved head's first run for this pull request;
28
28
  - the merge changed no file the pull request changes, and no file under `.proof/`;
29
29
  - after the merge, the pull request changes only files the approved head already changed (so a merge that keeps an old version of a file the base changed doesn't carry);
30
+ - since the approved head branched off, the base branch changed nothing under `.proof/` and none of the files the approval covers;
30
31
  - GitHub's file lists for the merge are complete.
31
32
 
32
33
  Otherwise the status says to approve again: remove the label and add it back. The approval is kept, never the result: the merged head is verified again on the new base and turns green only from its own run. It carries over one merge only: a second update before the label is added again needs a new approval. Inconclusive and workflow approvals never carry over, and any other push still removes every label.
@@ -16,6 +16,8 @@ permissions:
16
16
  pull-requests: write
17
17
  jobs:
18
18
  drop-approvals:
19
+ # Only when the pull request has an approval label to remove (otherwise skipped: no minutes).
20
+ if: contains(toJSON(github.event.pull_request.labels.*.name), '"pocp-approve-')
19
21
  runs-on: ubuntu-latest
20
22
  timeout-minutes: 5
21
23
  steps:
@@ -24,6 +24,10 @@ permissions:
24
24
  statuses: write
25
25
  jobs:
26
26
  report:
27
+ # Skipped (no Actions minutes) for events that can't change the status: a label other than
28
+ # pocp-approve-*, or an edit that doesn't change the base. Pushes, retargets, approval labels
29
+ # and finished verify runs always report.
30
+ if: github.event_name != 'pull_request_target' || ((github.event.action != 'labeled' && github.event.action != 'unlabeled') || startsWith(github.event.label.name, 'pocp-approve-')) && (github.event.action != 'edited' || github.event.changes.base)
27
31
  # One at a time per head, none cancelled: each recomputes from scratch, so the last one wins.
28
32
  concurrency:
29
33
  group: pocp-status-${{ github.event.workflow_run.head_sha || github.event.pull_request.head.sha }}
@@ -34,11 +34,11 @@ permissions:
34
34
  contents: read
35
35
  jobs:
36
36
  verify:
37
- if: vars.POCP_STATUS_LIVE != 'true' || (github.event.action != 'labeled' && github.event.action != 'unlabeled') || startsWith(github.event.label.name, 'pocp-approve-')
37
+ if: vars.POCP_STATUS_LIVE != 'true' || ((github.event.action != 'labeled' && github.event.action != 'unlabeled') || startsWith(github.event.label.name, 'pocp-approve-')) && (github.event.action != 'edited' || github.event.changes.base)
38
38
  # One run per pull request state: a newer push or approval cancels the older run. An ignored
39
39
  # label event gets a group of its own, so it can't cancel or replace a real run.
40
40
  concurrency:
41
- group: ${{ (vars.POCP_STATUS_LIVE != 'true' || (github.event.action != 'labeled' && github.event.action != 'unlabeled') || startsWith(github.event.label.name, 'pocp-approve-')) && format('pocp-verify-{0}', github.event.pull_request.number) || format('pocp-ignored-{0}', github.run_id) }}
41
+ group: ${{ (vars.POCP_STATUS_LIVE != 'true' || ((github.event.action != 'labeled' && github.event.action != 'unlabeled') || startsWith(github.event.label.name, 'pocp-approve-')) && (github.event.action != 'edited' || github.event.changes.base)) && format('pocp-verify-{0}', github.event.pull_request.number) || format('pocp-ignored-{0}', github.run_id) }}
42
42
  cancel-in-progress: true
43
43
  runs-on: ubuntu-latest
44
44
  # verify re-runs every required check of every commit: allow for your test suite times the
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "proof-of-change",
3
- "version": "1.2.0",
3
+ "version": "1.2.2",
4
4
  "description": "Proof of Change Protocol: risky changes carry proof that was run, tied to the exact code it tested.",
5
5
  "type": "module",
6
6
  "bin": { "pocp": "bin/pocp.js" },
package/src/ci-github.js CHANGED
@@ -233,3 +233,26 @@ export function goLive(root, plan, { protect = false } = {}) {
233
233
  else if (!statusRequired) notes.push(`merges are not blocked yet: require the "${STATUS_CHECK}" status${reporter ? '' : ` once ${REPORTER_WORKFLOW} is on ${plan.branch}`} with: pocp ci required --apply`);
234
234
  return { ok: true, repo, live: true, reporter, protection, required_status: statusRequired, removed_verify: removedVerify, merges_blocked: statusRequired, notes };
235
235
  }
236
+
237
+ /**
238
+ * Create the approval labels a maintainer approves changes with, on the repository `gh` is
239
+ * pointed at. Labels that exist are left alone. `labels` maps each name to what it approves.
240
+ * Returns { ok, repo, created: [], existing: [], error }.
241
+ */
242
+ export function createApprovalLabels(labels) {
243
+ const repoR = gh(['repo', 'view', '--json', 'nameWithOwner', '--jq', '.nameWithOwner']);
244
+ if (!repoR.ok) return { ok: false, created: [], existing: [], error: repoR.error || `gh repo view failed: ${repoR.stderr}` };
245
+ const repo = repoR.stdout.trim();
246
+ const listed = gh(['label', 'list', '--repo', repo, '--limit', '500', '--json', 'name', '--jq', '.[].name']);
247
+ if (!listed.ok) return { ok: false, repo, created: [], existing: [], error: `could not list labels: ${ghMessage(listed)}` };
248
+ const have = new Set(listed.stdout.split('\n').map((l) => l.trim()).filter(Boolean));
249
+ const created = [];
250
+ const existing = [];
251
+ for (const [name, what] of Object.entries(labels)) {
252
+ if (have.has(name)) { existing.push(name); continue; }
253
+ const r = gh(['label', 'create', name, '--repo', repo, '--color', 'FBCA04', '--description', `POCP: a maintainer approves ${what}`]);
254
+ if (!r.ok) return { ok: false, repo, created, existing, error: `could not create ${name}: ${ghMessage(r)}` };
255
+ created.push(name);
256
+ }
257
+ return { ok: true, repo, created, existing };
258
+ }
@@ -2,7 +2,8 @@
2
2
  import fs from 'node:fs';
3
3
  import path from 'node:path';
4
4
  import { EXIT } from '../errors.js';
5
- import { LIVE_VARIABLE, STATUS_CHECK, goLive, requiredChecksPlan } from '../ci-github.js';
5
+ import { LIVE_VARIABLE, STATUS_CHECK, createApprovalLabels, goLive, requiredChecksPlan } from '../ci-github.js';
6
+ import { APPROVAL_LABELS } from '../doctor.js';
6
7
  import { formatLayout, hooksHealth, initProofDir, installHooks, writeGithubWorkflows } from '../init.js';
7
8
  import { git } from '../git.js';
8
9
  import { out } from '../cli/shared.js';
@@ -16,10 +17,14 @@ export async function run({ flags, json }) {
16
17
  // yellow, not a red failure. Only for a repository on GitHub, through the user's gh login;
17
18
  // anything that doesn't work is a next step, never an error.
18
19
  let live = null;
19
- if (github && !flags['no-live'] && /github\.com[:/]/.test(git(process.cwd(), ['remote', 'get-url', 'origin'], { allowFail: true }) || '')) {
20
+ let labels = null;
21
+ const onGithub = Boolean(github) && /github\.com[:/]/.test(git(process.cwd(), ['remote', 'get-url', 'origin'], { allowFail: true }) || '');
22
+ // The labels a maintainer approves changes with: without them there is nothing to approve with.
23
+ if (onGithub) labels = createApprovalLabels(APPROVAL_LABELS);
24
+ if (onGithub && !flags['no-live']) {
20
25
  live = goLive(process.cwd(), requiredChecksPlan(process.cwd(), JSON.parse(fs.readFileSync(path.join(process.cwd(), '.proof', 'config.json'), 'utf8'))), { protect: false });
21
26
  }
22
- out(json, { created: r.created, hooks, test_layout: r.layout, ...(github ? { github } : {}), ...(live ? { live } : {}) }, [
27
+ out(json, { created: r.created, hooks, test_layout: r.layout, ...(github ? { github } : {}), ...(labels ? { labels } : {}), ...(live ? { live } : {}) }, [
23
28
  r.created.length ? `created ${r.created.join(', ')}` : '.proof/ already set up',
24
29
  ...(r.config_created ? [flags.deep
25
30
  ? 'test guard: deep (guard.mutants 10): test edits are also checked with 10 planted bugs; slower'
@@ -32,9 +37,11 @@ export async function run({ flags, json }) {
32
37
  ...(github ? [
33
38
  github.written.length ? `wrote ${github.written.join(', ')} (POCP actions pinned to ${github.ref})` : 'GitHub workflows already present',
34
39
  ...github.skipped.map((f) => `kept ${f} (exists; --force replaces it)`),
40
+ ...(labels?.ok ? [labels.created.length ? `created the approval labels on ${labels.repo}: ${labels.created.join(', ')}` : `approval labels already on ${labels.repo}`]
41
+ : labels ? [`approval labels not created: ${labels.error} (pocp doctor prints the commands)`] : []),
35
42
  ...(live?.ok ? [`${LIVE_VARIABLE} set on ${live.repo}: a pull request waiting for approval shows a yellow "${STATUS_CHECK}" status, not a red failure`, ...live.notes.map((n) => ` ${n}`)]
36
43
  : live ? [`${LIVE_VARIABLE} not set: ${live.error}`] : []),
37
- `next: commit them in a pull request, create the labels (pocp doctor lists them), and once it merges run "pocp ci required --apply" to require the "${STATUS_CHECK}" status${live?.ok ? '' : ' and turn it on'}`,
44
+ `next: commit them in a pull request${labels?.ok ? '' : ', create the labels (pocp doctor lists them)'}, and once it merges run "pocp ci required --apply" to require the "${STATUS_CHECK}" status${live?.ok ? '' : ' and turn it on'}`,
38
45
  ] : []),
39
46
  ].join('\n'));
40
47
  return hooks?.refused ? EXIT.usage : EXIT.ok;
package/src/doctor.js CHANGED
@@ -12,7 +12,7 @@ import { APPROVE_INCONCLUSIVE_LABEL, APPROVE_POLICY_LABEL, APPROVE_TESTS_LABEL }
12
12
  import { APPROVE_WORKFLOW_LABEL } from './status-report.js';
13
13
 
14
14
  // Every approval label, and what a maintainer approves with it (for `gh label create`).
15
- const APPROVAL_LABELS = Object.freeze({
15
+ export const APPROVAL_LABELS = Object.freeze({
16
16
  [APPROVE_TESTS_LABEL]: 'a test change',
17
17
  [APPROVE_POLICY_LABEL]: 'a policy change',
18
18
  'pocp-approve-coverage': 'commits listed in .proof/exceptions.json',
package/src/init.js CHANGED
@@ -404,7 +404,7 @@ const PKG_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'
404
404
  // The newest POCP release that has a tag on GitHub. A development or pre-release build (a
405
405
  // version like 1.1.0-dev.0, as on main between releases) has no tag of its own, so the
406
406
  // workflows it writes pin this instead of a ref that doesn't exist. Update it with each release.
407
- export const RELEASED_ACTION_REF = 'v1.2.0';
407
+ export const RELEASED_ACTION_REF = 'v1.2.2';
408
408
 
409
409
  /** The release tag the workflows pin by default: "v<version>" for a release, else RELEASED_ACTION_REF. */
410
410
  export function releaseRef() {
@@ -330,6 +330,15 @@ export function carriedOver(facts, label) {
330
330
  if (added.length) return no(`the merge adds changes the approval didn't cover: ${added.slice(0, 3).join(', ')}`);
331
331
  const policy = [...merged].filter((f) => f.startsWith(POLICY_PREFIX));
332
332
  if (policy.length) return no(`the merge changed policy files: ${policy.slice(0, 3).join(', ')}`);
333
+ // What the base branch changed since the approved head branched off. A merge can keep the
334
+ // pull request's own version of a file, so the merge's file list never shows that the base
335
+ // changed it: the base must not have changed the policy (the authority the approval was judged
336
+ // by) or any file the approval covers (the side the person compared against).
337
+ if (!Array.isArray(c.base_files)) return no("GitHub's data for the base branch's changes was missing");
338
+ const basePolicy = c.base_files.filter((f) => f.startsWith(POLICY_PREFIX));
339
+ if (basePolicy.length) return no(`the base branch changed policy files since the approval: ${basePolicy.slice(0, 3).join(', ')}`);
340
+ const baseTouched = c.base_files.filter((f) => approved.has(f));
341
+ if (baseTouched.length) return no(`the base branch changed files the approval covers: ${baseTouched.slice(0, 3).join(', ')}`);
333
342
  return { ok: true };
334
343
  }
335
344
 
@@ -466,7 +475,7 @@ async function gatherCarry(gh, repo, head, workflow) {
466
475
  const parents = (commit.parents || []).map((p) => p.sha);
467
476
  if (parents.length !== 2) return null;
468
477
  const [from, merged] = parents;
469
- const carry = { from_sha: from, merged_sha: merged, complete: true, reason: null, merge_files: [], pr_files: [], approved_files: [], runs: [] };
478
+ const carry = { from_sha: from, merged_sha: merged, complete: true, reason: null, merge_files: [], pr_files: [], approved_files: [], base_files: [], runs: [] };
470
479
  const filesOf = async (a, b) => {
471
480
  const cmp = await gh.get(`${repo}/compare/${a}...${b}`);
472
481
  const files = (cmp.files || []).flatMap((f) => [f.filename, f.previous_filename].filter(Boolean));
@@ -477,6 +486,8 @@ async function gatherCarry(gh, repo, head, workflow) {
477
486
  carry.pr_files = await filesOf(merged, head);
478
487
  // What the approved head changed against its merge base with the base branch (three dots).
479
488
  carry.approved_files = await filesOf(merged, from);
489
+ // What the base changed since the approved head branched off (three dots: from their merge base).
490
+ carry.base_files = await filesOf(from, merged);
480
491
  const runs = await listAll(gh, `${repo}/actions/workflows/${workflow.id}/runs?head_sha=${from}&per_page=100`, 'workflow_runs', 5);
481
492
  if (!runs.complete) { carry.complete = false; carry.reason = runs.reason || 'a list was cut short'; }
482
493
  for (const r of runs.items.filter((x) => x.event === 'pull_request' && x.path === workflow.path)) {
@@ -521,10 +532,13 @@ export async function reportStatus(gh, { repository, sha, prNumber, baseConfig,
521
532
  if (facts.pr) {
522
533
  const now = await gh.get(`/repos/${repository}/pulls/${facts.pr.number}`);
523
534
  if (now.head?.sha !== decision.sha || now.state !== 'open') return { decision: { write: false, reason: `pull request #${facts.pr.number} changed while deciding` }, written: false };
524
- const labelsNow = (now.labels || []).map((l) => l.name).sort().join('\n');
535
+ // Only approval labels decide anything; another label changing doesn't make this stale (and
536
+ // the workflow skips the reporter for such label events, so nothing would correct it).
537
+ const approvalLabels = new Set([APPROVE_WORKFLOW_LABEL, ...Object.values(APPROVAL_MARKERS)]);
538
+ const labelsNow = (now.labels || []).map((l) => l.name).filter((l) => approvalLabels.has(l)).sort().join('\n');
525
539
  const changed = [];
526
540
  if ((now.base?.sha || null) !== facts.base_sha) changed.push('its base moved');
527
- if (labelsNow !== [...facts.pr.labels].sort().join('\n')) changed.push('its labels changed');
541
+ if (labelsNow !== facts.pr.labels.filter((l) => approvalLabels.has(l)).sort().join('\n')) changed.push('its approval labels changed');
528
542
  if (decision.state === 'success' && !changed.length) {
529
543
  const open = await listAll(gh, `/repos/${repository}/pulls?state=open&per_page=100`);
530
544
  const sharing = open.items.filter((p) => p.head?.sha === decision.sha).map((p) => p.number);