@jonmilner/local-ci 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -3
- package/dist/check-lock.d.ts +31 -0
- package/dist/check-lock.js +115 -0
- package/dist/cli.js +27 -90
- package/dist/lock-entry.d.ts +1 -0
- package/dist/lock-entry.js +2 -0
- package/dist/lock.d.ts +32 -7
- package/dist/lock.js +47 -11
- package/dist/processes.d.ts +3 -0
- package/dist/processes.js +17 -0
- package/dist/push.d.ts +8 -0
- package/dist/push.js +12 -0
- package/package.json +8 -1
package/README.md
CHANGED
|
@@ -21,9 +21,9 @@ That writes `pre-push` and `commit-msg` shims, each one line that calls local-ci
|
|
|
21
21
|
## Commands
|
|
22
22
|
|
|
23
23
|
- `local-ci hooks install`: writes executable `#!/bin/sh` `pre-push` and `commit-msg` shims into the hooks directory: `core.hooksPath`, resolved against the repo root, with a Vite+ (or husky) `<dir>/_` dispatcher mapped to `<dir>`, and an absolute path into the main checkout mapped onto the current worktree. Each shim is one line: `pnpm exec local-ci pre-push "$@"` and `pnpm exec local-ci commit-msg "$1"` in a repo with a `package.json`, a global `local-ci …` without one, or `"local-ci": { "exec": "<command>" }` in `package.json` in place of `pnpm exec local-ci` (this repo runs local-ci from source that way). An existing shell hook keeps its other lines and gains the shim line, at the end, only when no line already runs `local-ci pre-push` (or `commit-msg`), so a line with an env prefix such as `LOCAL_CI_COMMAND=… local-ci pre-push "$@"` counts. It prints each file it wrote, or `hooks up to date`; a second run changes nothing. It exits 1 without `core.hooksPath` or with one outside the repo's tracked files (including `.git/hooks`), and for an existing hook in another language (a `node` or `python` shebang), naming the line to add by hand. Run it by hand and commit the result; `prepare` does not run it.
|
|
24
|
-
- `local-ci pre-push`: the hook. It first checks that both hook files are in place, as `local-ci hooks install` would leave them, and refuses the push otherwise (``hooks out of date: <file> must run local-ci; run `<command> hooks install` and commit the result``), so a repo's hooks can't drift. For each commit pushed to a branch (deletes and tags are skipped) that has not already passed, runs the check command. The pushed commit must be `HEAD` with a clean working tree (`git status --porcelain` empty), so the result reflects the commit; otherwise the push is refused. A failing check blocks the push. When every commit passes, it starts `local-ci post <sha>` in the background, aimed at the pushed remote, and lets the push go ahead.
|
|
24
|
+
- `local-ci pre-push`: the hook. It first checks that both hook files are in place, as `local-ci hooks install` would leave them, and refuses the push otherwise (``hooks out of date: <file> must run local-ci; run `<command> hooks install` and commit the result``), so a repo's hooks can't drift. For each commit pushed to a branch (deletes and tags are skipped) that has not already passed, runs the check command. The pushed commit must be `HEAD` with a clean working tree (`git status --porcelain` empty), so the result reflects the commit; otherwise the push is refused. A failing check blocks the push. When every commit passes, it starts `local-ci post <sha>` in the background, aimed at the pushed remote and told the pid of the `git` process doing the push (`LOCAL_CI_PUSH_PID`), and lets the push go ahead.
|
|
25
25
|
- `local-ci commit-msg <file>`: lints the commit message in `<file>` with `@jonmilner/commitlint-config`, for a git `commit-msg` hook (git passes `.git/COMMIT_EDITMSG`). `local-ci commit-msg -` reads the message from stdin, for PR titles. Comment lines (starting with `#`) and everything after a `# ------------------------ >8 ------------------------` scissors line are stripped first; merge commits, `fixup!`/`squash!` and reverts are skipped. Each error or warning goes to stderr as `commit-msg: <message> [<rule>]`, followed by the offending header; it exits 1 when there are errors (warnings alone exit 0), and 2 when no file is given. It makes no network, git or GitHub calls.
|
|
26
|
-
- `local-ci post <sha>`: waits up to 3 minutes for GitHub to have the commit, posts status `check` = `success` (`<command> passed on <hostname>`), then posts `pr-title` on it for an open PR that contains it (waiting up to 30s for GitHub to link them). A PR opened after the push, or a retitled one, needs `local-ci pr-title` run by hand.
|
|
26
|
+
- `local-ci post <sha>`: when started by `pre-push`, first waits for the pushing `git` to exit (polling each second, up to 60 minutes, logging if it gives up), since git sends the push only after the whole hook exits, so a slow step after local-ci in the hook can't run out the clock; run by hand, it skips that wait. It then waits up to 3 minutes for GitHub to have the commit, posts status `check` = `success` (`<command> passed on <hostname>`), then posts `pr-title` on it for an open PR that contains it (waiting up to 30s for GitHub to link them). A PR opened after the push, or a retitled one, needs `local-ci pr-title` run by hand.
|
|
27
27
|
- `local-ci pr-title [<pr>]`: lints the PR title (the current branch's PR by default) with `@jonmilner/commitlint-config` and posts status `pr-title` on its head commit, `failure` with the first error when the title does not pass.
|
|
28
28
|
- `local-ci status [<sha>]`: posts `check` for a commit (default `HEAD`) that already passed, or runs the check first. When the check command fails, it posts `check` = `failure` (`<command> failed on <hostname>`) and exits non-zero; a later passing run posts `success` over it. A dirty tree, a commit that is not `HEAD`, or a `gh` error posts nothing. Use it after a push that skipped hooks (`--no-verify`) or a lost background poster.
|
|
29
29
|
- `local-ci release`: publishes the repo's packages from this machine. Off unless `package.json` has `"local-ci": { "release": true }`; otherwise it prints `release not enabled` and exits 0. It needs a clean working tree, then fetches `origin` and takes a lock, so a second run exits with `release in progress`. In a temporary worktree at `origin`'s default branch:
|
|
@@ -58,6 +58,20 @@ A repo without a `package.json` sets `LOCAL_CI_COMMAND`, for example on the hook
|
|
|
58
58
|
|
|
59
59
|
Logs, the record of passed commits and the release lock live in `~/.local/state/local-ci/<repo>.log`, `<repo>.passed` and `<repo>.release.lock` (`$XDG_STATE_HOME` if set). A lock whose process has died is taken over.
|
|
60
60
|
|
|
61
|
-
One check command runs at a time on the machine, across every repo, so overlapping checks can't starve each other's tests: `pre-push`, `status`, and the `status` runs `renovate` starts each take `check.lock` in the same directory around the check command (not the status posts). A run that finds it held prints `local-ci: waiting for the check lock, held by pid <pid> (<repo> <short-sha>)`, again whenever another run takes it first, and polls every second until the holder exits or dies; Ctrl-C stops the wait. After the wait it skips a commit that passed meanwhile and re-applies the clean-tree rule. The lock goes when the check passes, fails or is killed by a signal. Ctrl-C, SIGTERM or SIGHUP to local-ci passes the signal to the check's whole process group (so a check script's children get it too), waits for the check to exit, then releases the lock and dies of the signal, so a check never outlives its lock. A lock whose holder died (SIGKILL, a crash) or that predates the last boot is taken over by the next run. A local-ci check nested inside a check (one that pushes, say) sees its ancestor holds the lock and runs without waiting. Each check run appends its duration and outcome to the log when it ends, e.g. `[<end time>] abc1234: pnpm check passed in 71.2s` (or `failed in`, or `did not finish after` when the check is killed by a signal or can't start); a commit that already passed is skipped and logs nothing. Ctrl-C stops the check, which logs `did not finish after`.
|
|
61
|
+
One check command runs at a time on the machine, across every repo, so overlapping checks can't starve each other's tests: `pre-push`, `status`, and the `status` runs `renovate` starts each take `check.lock` in the same directory around the check command (not the status posts). A run that finds it held prints `local-ci: waiting for the check lock, held by pid <pid> (<repo> <short-sha>)`, again whenever another run takes it first, and polls every second until the holder exits or dies; Ctrl-C stops the wait. After the wait it skips a commit that passed meanwhile and re-applies the clean-tree rule. The lock goes when the check passes, fails or is killed by a signal. Ctrl-C, SIGTERM or SIGHUP to local-ci passes the signal to the check's whole process group (so a check script's children get it too), waits for the check to exit, then releases the lock and dies of the signal, so a check never outlives its lock. A lock whose holder died (SIGKILL, a crash) or that predates the last boot is taken over by the next run. A local-ci check nested inside a check (one that pushes, say) sees its ancestor holds the lock and runs without waiting. A check that finds a background job's lock (see below) also runs without waiting and leaves that lock alone, printing `local-ci: running beside background job pid <pid> (<holder>), without the check lock`; signals still stop it as above. Each check run appends its duration and outcome to the log when it ends, e.g. `[<end time>] abc1234: pnpm check passed in 71.2s` (or `failed in`, or `did not finish after` when the check is killed by a signal or can't start); a commit that already passed is skipped and logs nothing. Ctrl-C stops the check, which logs `did not finish after`.
|
|
62
|
+
|
|
63
|
+
### Taking the check lock from your own scripts
|
|
64
|
+
|
|
65
|
+
A heavy job outside `local-ci` (a browser suite run from a hook, say) can queue behind the checks, and make them queue behind it, by taking the same `check.lock`:
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
import { withCheckLock } from "@jonmilner/local-ci/lock";
|
|
69
|
+
|
|
70
|
+
await withCheckLock("my-repo browser", async () => {
|
|
71
|
+
// runs alone with respect to every local-ci check and every other withCheckLock job
|
|
72
|
+
});
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
`withCheckLock(holder, fn, options?)` waits while another holder has the lock, logging `local-ci: waiting for the check lock, held by pid <pid> (<holder>)` to stderr as the CLI does, runs `fn`, and releases the lock when `fn` returns or throws. `holder` is the label other waiters see. `lockCheck(holder, options?)` does the take and returns the release function, for callers that can't wrap their work in a function. `options.file` overrides the lock path and `options.onWait` replaces the wait message. The path is `check.lock` in the state directory above, so no repo config is needed. A call nested under an ancestor process that holds the lock runs without waiting and leaves the lock alone. Pass `options.background: true` for a long job run after a push, such as a browser suite: it still waits for a running check, but `local-ci` checks (and other non-background callers) run at once beside it instead of waiting, so a push never sits in its hook behind it. Background jobs still queue behind each other. A check run beside a background job holds no lock, so nothing waits for it: another check, or a background job taking the lock after the first one ends, can overlap it. The lock file marks such a lock with a third line, `background`; a file without it (as older versions wrote) is waited on as before. SIGINT, SIGTERM or SIGHUP releases the lock and ends the process, so a job that needs its own signal cleanup should run it before calling the release itself. The package has an `exports` map, so deep imports other than `./lock` no longer resolve.
|
|
62
76
|
|
|
63
77
|
Part of [jonmilner/config](https://github.com/jonmilner/config#readme).
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
import type { ChildProcess } from "node:child_process";
|
|
2
|
+
import { type LockHeld } from "./lock.ts";
|
|
3
|
+
/** The check lock this process holds through `lockCheck`, if any, for the CLI to attach its check to. */
|
|
4
|
+
export declare function heldCheckLock(): {
|
|
5
|
+
child?: ChildProcess;
|
|
6
|
+
signal?: NodeJS.Signals;
|
|
7
|
+
unlock: () => void;
|
|
8
|
+
} | undefined;
|
|
9
|
+
export interface LockCheckOptions {
|
|
10
|
+
/** The lock file; default is `check.lock` in local-ci's state directory, the one the CLI uses. */
|
|
11
|
+
file?: string;
|
|
12
|
+
/** Called when the wait starts, and whenever another process takes the lock first. */
|
|
13
|
+
onWait?: (held: LockHeld) => void;
|
|
14
|
+
/**
|
|
15
|
+
* Takes the lock for a long background job, such as a browser suite run after a push. It still
|
|
16
|
+
* waits for a running check, but checks run beside it instead of waiting for it to finish.
|
|
17
|
+
*/
|
|
18
|
+
background?: boolean;
|
|
19
|
+
}
|
|
20
|
+
/** The machine-wide check lock's path, in `$XDG_STATE_HOME/local-ci` (default `~/.local/state`). */
|
|
21
|
+
export declare function checkLockPath(env?: NodeJS.ProcessEnv): string;
|
|
22
|
+
/**
|
|
23
|
+
* Takes the machine-wide check lock, waiting while another check or job holds it, and returns its
|
|
24
|
+
* release. `holder` names the caller in other waiters' messages. A caller nested under an ancestor
|
|
25
|
+
* that holds the lock, or a check (not `background`) that finds a background job holding it, runs
|
|
26
|
+
* without waiting, and its release leaves the lock alone. SIGINT, SIGTERM and SIGHUP release the
|
|
27
|
+
* lock and end the process.
|
|
28
|
+
*/
|
|
29
|
+
export declare function lockCheck(holder: string, { file, onWait, background }?: LockCheckOptions): Promise<() => void>;
|
|
30
|
+
/** Runs `fn` under the machine-wide check lock, so it queues behind `local-ci` checks and vice versa. */
|
|
31
|
+
export declare function withCheckLock<T>(holder: string, fn: () => Promise<T>, options?: LockCheckOptions): Promise<T>;
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
import { mkdirSync } from "node:fs";
|
|
2
|
+
import { join } from "node:path";
|
|
3
|
+
import { stateDir } from "./config.js";
|
|
4
|
+
import { waitForLock } from "./lock.js";
|
|
5
|
+
import { processTable } from "./processes.js";
|
|
6
|
+
import { ancestorsFrom } from "./push.js";
|
|
7
|
+
/** This process's ancestors' pids, or just its parent when `ps` can't list them. */
|
|
8
|
+
function ancestors() {
|
|
9
|
+
return new Set(ancestorsFrom(processTable(), process.ppid));
|
|
10
|
+
}
|
|
11
|
+
const SIGNALS = ["SIGINT", "SIGTERM", "SIGHUP"];
|
|
12
|
+
/** The check lock this process holds, the check running under it, and a signal it deferred. */
|
|
13
|
+
let held;
|
|
14
|
+
/** The check lock this process holds through `lockCheck`, if any, for the CLI to attach its check to. */
|
|
15
|
+
export function heldCheckLock() {
|
|
16
|
+
return held;
|
|
17
|
+
}
|
|
18
|
+
/** Dies of `signal` as if there were no handler for it. */
|
|
19
|
+
function reraise(signal) {
|
|
20
|
+
for (const name of SIGNALS)
|
|
21
|
+
process.off(name, onSignal);
|
|
22
|
+
process.kill(process.pid, signal);
|
|
23
|
+
// The default action has ended the process by now; this only satisfies the type.
|
|
24
|
+
process.exit(1);
|
|
25
|
+
}
|
|
26
|
+
/**
|
|
27
|
+
* While a check runs, passes the signal on and lets the check stop first, so it never outlives
|
|
28
|
+
* the lock; its release then dies of the signal. Otherwise releases the lock and dies of it now.
|
|
29
|
+
*/
|
|
30
|
+
function onSignal(signal) {
|
|
31
|
+
if (held?.child) {
|
|
32
|
+
held.signal ??= signal;
|
|
33
|
+
// The check leads its own process group, so the signal reaches a script's children too.
|
|
34
|
+
try {
|
|
35
|
+
process.kill(-held.child.pid, signal);
|
|
36
|
+
}
|
|
37
|
+
catch {
|
|
38
|
+
held.child.kill(signal);
|
|
39
|
+
}
|
|
40
|
+
return;
|
|
41
|
+
}
|
|
42
|
+
held?.unlock();
|
|
43
|
+
held = undefined;
|
|
44
|
+
reraise(signal);
|
|
45
|
+
}
|
|
46
|
+
/** Holds `unlock` until the returned release, releasing it on SIGINT, SIGTERM or SIGHUP too. */
|
|
47
|
+
function releaseOnSignal(unlock) {
|
|
48
|
+
if (!process.listenerCount("SIGINT", onSignal)) {
|
|
49
|
+
for (const name of SIGNALS)
|
|
50
|
+
process.on(name, onSignal);
|
|
51
|
+
}
|
|
52
|
+
const lock = { unlock };
|
|
53
|
+
held = lock;
|
|
54
|
+
return () => {
|
|
55
|
+
if (held === lock)
|
|
56
|
+
held = undefined;
|
|
57
|
+
unlock();
|
|
58
|
+
if (lock.signal)
|
|
59
|
+
reraise(lock.signal);
|
|
60
|
+
};
|
|
61
|
+
}
|
|
62
|
+
/** The machine-wide check lock's path, in `$XDG_STATE_HOME/local-ci` (default `~/.local/state`). */
|
|
63
|
+
export function checkLockPath(env = process.env) {
|
|
64
|
+
const dir = stateDir(env);
|
|
65
|
+
mkdirSync(dir, { recursive: true });
|
|
66
|
+
return join(dir, "check.lock");
|
|
67
|
+
}
|
|
68
|
+
/**
|
|
69
|
+
* Takes the machine-wide check lock, waiting while another check or job holds it, and returns its
|
|
70
|
+
* release. `holder` names the caller in other waiters' messages. A caller nested under an ancestor
|
|
71
|
+
* that holds the lock, or a check (not `background`) that finds a background job holding it, runs
|
|
72
|
+
* without waiting, and its release leaves the lock alone. SIGINT, SIGTERM and SIGHUP release the
|
|
73
|
+
* lock and end the process.
|
|
74
|
+
*/
|
|
75
|
+
export async function lockCheck(holder, { file = checkLockPath(), onWait, background = false } = {}) {
|
|
76
|
+
let beside;
|
|
77
|
+
let mine;
|
|
78
|
+
// A check nested in a check (one that pushes, say) runs under its ancestor's lock rather than
|
|
79
|
+
// waiting forever on it.
|
|
80
|
+
const nested = (held) => (mine ??= ancestors()).has(held.pid);
|
|
81
|
+
// A check never waits out a background job's long run; it runs beside it.
|
|
82
|
+
const skip = (held) => {
|
|
83
|
+
if (nested(held))
|
|
84
|
+
return true;
|
|
85
|
+
if (!held.background || background)
|
|
86
|
+
return false;
|
|
87
|
+
beside = held;
|
|
88
|
+
return true;
|
|
89
|
+
};
|
|
90
|
+
const unlock = await waitForLock(file, {
|
|
91
|
+
what: "check",
|
|
92
|
+
holder,
|
|
93
|
+
background,
|
|
94
|
+
skip,
|
|
95
|
+
onWait: onWait ??
|
|
96
|
+
(({ pid, holder }) => process.stderr.write(`local-ci: waiting for the check lock, held by pid ${pid} (${holder || "unknown"})\n`)),
|
|
97
|
+
});
|
|
98
|
+
if (unlock)
|
|
99
|
+
return releaseOnSignal(unlock);
|
|
100
|
+
if (!beside)
|
|
101
|
+
return () => { };
|
|
102
|
+
process.stderr.write(`local-ci: running beside background job pid ${beside.pid} (${beside.holder || "unknown"}), without the check lock\n`);
|
|
103
|
+
// Nothing to release, but the signal handling still applies, so the check never outlives this run.
|
|
104
|
+
return releaseOnSignal(() => { });
|
|
105
|
+
}
|
|
106
|
+
/** Runs `fn` under the machine-wide check lock, so it queues behind `local-ci` checks and vice versa. */
|
|
107
|
+
export async function withCheckLock(holder, fn, options) {
|
|
108
|
+
const unlock = await lockCheck(holder, options);
|
|
109
|
+
try {
|
|
110
|
+
return await fn();
|
|
111
|
+
}
|
|
112
|
+
finally {
|
|
113
|
+
unlock();
|
|
114
|
+
}
|
|
115
|
+
}
|
package/dist/cli.js
CHANGED
|
@@ -1,17 +1,19 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
import {
|
|
2
|
+
import { spawn } from "node:child_process";
|
|
3
3
|
import { closeSync, openSync, readFileSync } from "node:fs";
|
|
4
4
|
import { hostname } from "node:os";
|
|
5
5
|
import { basename, dirname, join, relative, resolve } from "node:path";
|
|
6
6
|
import { fileURLToPath } from "node:url";
|
|
7
|
+
import { heldCheckLock, lockCheck } from "./check-lock.js";
|
|
7
8
|
import { commitMsg } from "./commit-msg.js";
|
|
8
9
|
import { appendLog, checkRunMessage, logLine, passedCommand, recordPass, releaseEnabled, resolveCommand, shimCommand, statePaths, } from "./config.js";
|
|
9
10
|
import { exec, run } from "./exec.js";
|
|
10
11
|
import { assertGhAuth, checkStatus, postStatus, prTitleStatus, viewPr, waitForCommit, waitForOpenPr, } from "./github.js";
|
|
11
12
|
import { hooksDir, installHooks, missingHooks } from "./hooks.js";
|
|
12
13
|
import { installRenovateAgent, renovateLog, rotateLog, uninstallRenovateAgent } from "./launchd.js";
|
|
13
|
-
import {
|
|
14
|
-
import {
|
|
14
|
+
import { waitForExit } from "./lock.js";
|
|
15
|
+
import { processTable } from "./processes.js";
|
|
16
|
+
import { cannotCheckInPlace, githubRepo, nearestGit, parsePrePush, shasToCheck } from "./push.js";
|
|
15
17
|
import { release, takeEnvToken } from "./release.js";
|
|
16
18
|
import { projectsDir, renovate } from "./renovate.js";
|
|
17
19
|
import { lintTitle } from "./title.js";
|
|
@@ -48,97 +50,12 @@ class CheckFailed extends Error {
|
|
|
48
50
|
this.command = command;
|
|
49
51
|
}
|
|
50
52
|
}
|
|
51
|
-
/** This process's ancestors' pids, or none when `ps` can't list them. */
|
|
52
|
-
function ancestors() {
|
|
53
|
-
const parents = new Map();
|
|
54
|
-
try {
|
|
55
|
-
for (const line of execFileSync("ps", ["-A", "-o", "pid=,ppid="], { encoding: "utf8" }).split("\n")) {
|
|
56
|
-
const [pid, ppid] = line.trim().split(/\s+/).map(Number);
|
|
57
|
-
if (pid && ppid !== undefined)
|
|
58
|
-
parents.set(pid, ppid);
|
|
59
|
-
}
|
|
60
|
-
}
|
|
61
|
-
catch {
|
|
62
|
-
return new Set();
|
|
63
|
-
}
|
|
64
|
-
const found = new Set();
|
|
65
|
-
for (let pid = process.ppid; pid > 1 && !found.has(pid); pid = parents.get(pid) ?? 0)
|
|
66
|
-
found.add(pid);
|
|
67
|
-
return found;
|
|
68
|
-
}
|
|
69
|
-
const SIGNALS = ["SIGINT", "SIGTERM", "SIGHUP"];
|
|
70
|
-
/** The check lock this process holds, the check running under it, and a signal it deferred. */
|
|
71
|
-
let held;
|
|
72
|
-
/** Dies of `signal` as if local-ci had no handler for it. */
|
|
73
|
-
function reraise(signal) {
|
|
74
|
-
for (const name of SIGNALS)
|
|
75
|
-
process.off(name, onSignal);
|
|
76
|
-
process.kill(process.pid, signal);
|
|
77
|
-
// The default action has ended the process by now; this only satisfies the type.
|
|
78
|
-
process.exit(1);
|
|
79
|
-
}
|
|
80
|
-
/**
|
|
81
|
-
* While a check runs, passes the signal on and lets the check stop first, so it never outlives
|
|
82
|
-
* the lock; its release then dies of the signal. Otherwise releases the lock and dies of it now.
|
|
83
|
-
*/
|
|
84
|
-
function onSignal(signal) {
|
|
85
|
-
if (held?.child) {
|
|
86
|
-
held.signal ??= signal;
|
|
87
|
-
// The check leads its own process group, so the signal reaches a script's children too.
|
|
88
|
-
try {
|
|
89
|
-
process.kill(-held.child.pid, signal);
|
|
90
|
-
}
|
|
91
|
-
catch {
|
|
92
|
-
held.child.kill(signal);
|
|
93
|
-
}
|
|
94
|
-
return;
|
|
95
|
-
}
|
|
96
|
-
held?.unlock();
|
|
97
|
-
held = undefined;
|
|
98
|
-
reraise(signal);
|
|
99
|
-
}
|
|
100
|
-
/** Holds `unlock` until the returned release, releasing it on SIGINT, SIGTERM or SIGHUP too. */
|
|
101
|
-
function releaseOnSignal(unlock) {
|
|
102
|
-
if (!process.listenerCount("SIGINT", onSignal)) {
|
|
103
|
-
for (const name of SIGNALS)
|
|
104
|
-
process.on(name, onSignal);
|
|
105
|
-
}
|
|
106
|
-
const lock = { unlock };
|
|
107
|
-
held = lock;
|
|
108
|
-
return () => {
|
|
109
|
-
if (held === lock)
|
|
110
|
-
held = undefined;
|
|
111
|
-
unlock();
|
|
112
|
-
if (lock.signal)
|
|
113
|
-
reraise(lock.signal);
|
|
114
|
-
};
|
|
115
|
-
}
|
|
116
53
|
/** Throws unless `sha` can be checked in place: it is HEAD and the tree is clean. */
|
|
117
54
|
async function assertCheckable(sha) {
|
|
118
55
|
const reason = cannotCheckInPlace(sha, await git("rev-parse", "HEAD"), await git("status", "--porcelain"));
|
|
119
56
|
if (reason)
|
|
120
57
|
throw new Error(`cannot check ${sha.slice(0, 7)}: ${reason}`);
|
|
121
58
|
}
|
|
122
|
-
/** Takes the machine-wide check lock, waiting while another repo's check holds it. */
|
|
123
|
-
async function lockCheck({ name, checkLock }, sha) {
|
|
124
|
-
const holder = `${name} ${sha.slice(0, 7)}`;
|
|
125
|
-
try {
|
|
126
|
-
return releaseOnSignal(takeLock(checkLock, "check", holder));
|
|
127
|
-
}
|
|
128
|
-
catch (error) {
|
|
129
|
-
if (!(error instanceof LockHeld))
|
|
130
|
-
throw error;
|
|
131
|
-
// A check nested in a check (one that pushes, say) runs under its ancestor's lock rather
|
|
132
|
-
// than waiting forever on it.
|
|
133
|
-
if (ancestors().has(error.pid))
|
|
134
|
-
return () => { };
|
|
135
|
-
}
|
|
136
|
-
return releaseOnSignal(await waitForLock(checkLock, {
|
|
137
|
-
what: "check",
|
|
138
|
-
holder,
|
|
139
|
-
onWait: ({ pid, holder }) => err(`local-ci: waiting for the check lock, held by pid ${pid} (${holder || "unknown"})`),
|
|
140
|
-
}));
|
|
141
|
-
}
|
|
142
59
|
/**
|
|
143
60
|
* Runs the check at `sha` in place, after the clean-tree rule, unless it already passed. One check
|
|
144
61
|
* runs at a time across all repos, so overlapping checks can't starve each other's tests.
|
|
@@ -147,7 +64,7 @@ async function runCheck(current, sha) {
|
|
|
147
64
|
if (passedCommand(current.passed, sha))
|
|
148
65
|
return;
|
|
149
66
|
await assertCheckable(sha);
|
|
150
|
-
const unlock = await lockCheck(current,
|
|
67
|
+
const unlock = await lockCheck(`${current.name} ${sha.slice(0, 7)}`, { file: current.checkLock });
|
|
151
68
|
try {
|
|
152
69
|
// Another run may have passed it, or the tree changed, while this one waited.
|
|
153
70
|
if (passedCommand(current.passed, sha))
|
|
@@ -161,6 +78,7 @@ async function runCheck(current, sha) {
|
|
|
161
78
|
}
|
|
162
79
|
/** Runs the check command under the check lock, logging its duration and recording a pass. */
|
|
163
80
|
async function runLocked({ root, passed, log: logFile }, sha) {
|
|
81
|
+
const held = heldCheckLock();
|
|
164
82
|
const command = resolveCommand(root);
|
|
165
83
|
err(`local-ci: running ${command} on ${sha.slice(0, 7)}`);
|
|
166
84
|
const started = performance.now();
|
|
@@ -235,7 +153,13 @@ async function prePush(remoteUrl) {
|
|
|
235
153
|
await runCheck(current, sha);
|
|
236
154
|
// Post to the remote being pushed, not whichever one gh would pick.
|
|
237
155
|
const ghRepo = remoteUrl ? githubRepo(remoteUrl) : undefined;
|
|
238
|
-
const env = ghRepo ? { ...process.env, GH_REPO: ghRepo } : process.env;
|
|
156
|
+
const env = ghRepo ? { ...process.env, GH_REPO: ghRepo } : { ...process.env };
|
|
157
|
+
// git sends the push only once this hook exits, so the poster waits on git before GitHub.
|
|
158
|
+
const git = nearestGit(processTable(), process.pid);
|
|
159
|
+
if (git)
|
|
160
|
+
env[PUSH_PID] = String(git);
|
|
161
|
+
else
|
|
162
|
+
delete env[PUSH_PID];
|
|
239
163
|
const fd = openSync(current.log, "a");
|
|
240
164
|
for (const sha of shas) {
|
|
241
165
|
spawn(process.execPath, [fileURLToPath(import.meta.url), "post", sha], {
|
|
@@ -254,11 +178,24 @@ async function post({ passed }, sha) {
|
|
|
254
178
|
if (!command) {
|
|
255
179
|
throw new Error(`${sha.slice(0, 7)} has no passing check; run local-ci status ${sha}`);
|
|
256
180
|
}
|
|
181
|
+
await waitForPush(sha);
|
|
257
182
|
await postCheck(sha, checkStatus(command, hostname()));
|
|
258
183
|
const pr = await waitForOpenPr(exec, sha);
|
|
259
184
|
if (pr)
|
|
260
185
|
await prTitle(String(pr), sha);
|
|
261
186
|
}
|
|
187
|
+
/** The env var pre-push sets on each poster to the pid of the git process doing the push. */
|
|
188
|
+
const PUSH_PID = "LOCAL_CI_PUSH_PID";
|
|
189
|
+
/** Waits for the pushing git to exit, when pre-push named one; a `post` run by hand has none. */
|
|
190
|
+
async function waitForPush(sha) {
|
|
191
|
+
const pid = Number(process.env[PUSH_PID]);
|
|
192
|
+
if (!Number.isInteger(pid) || pid <= 1)
|
|
193
|
+
return;
|
|
194
|
+
log(`${sha.slice(0, 7)}: waiting for git (pid ${pid}) to push`);
|
|
195
|
+
if (!(await waitForExit(pid))) {
|
|
196
|
+
log(`${sha.slice(0, 7)}: git (pid ${pid}) still running after 60 min; waiting for GitHub anyway`);
|
|
197
|
+
}
|
|
198
|
+
}
|
|
262
199
|
/** Posts `pr-title` on `sha`, or on the PR's head commit when `sha` is omitted. */
|
|
263
200
|
async function prTitle(pr, sha) {
|
|
264
201
|
const { number, title, headRefOid } = await viewPr(exec, pr);
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export { checkLockPath, lockCheck, type LockCheckOptions, withCheckLock } from "./check-lock.ts";
|
package/dist/lock.d.ts
CHANGED
|
@@ -1,21 +1,46 @@
|
|
|
1
|
-
|
|
1
|
+
import { type WaitOptions } from "./github.ts";
|
|
2
|
+
/** Whether a process with this pid is running. */
|
|
3
|
+
export declare function alive(pid: number): boolean;
|
|
4
|
+
export interface ExitWaitOptions extends WaitOptions {
|
|
5
|
+
alive?: (pid: number) => boolean;
|
|
6
|
+
}
|
|
7
|
+
/**
|
|
8
|
+
* Polls until `pid` exits, returning false if it is still running after `timeoutMs`. git sends a
|
|
9
|
+
* push only after its pre-push hook exits, so the poster waits on git before waiting on GitHub.
|
|
10
|
+
*/
|
|
11
|
+
export declare function waitForExit(pid: number, options?: ExitWaitOptions): Promise<boolean>;
|
|
12
|
+
/**
|
|
13
|
+
* A live process holds the lock. `holder` is what it wrote after its pid, such as its repo, and
|
|
14
|
+
* `background` whether it took the lock for a background job.
|
|
15
|
+
*/
|
|
2
16
|
export declare class LockHeld extends Error {
|
|
3
17
|
readonly pid: number;
|
|
4
18
|
readonly holder: string;
|
|
5
|
-
|
|
19
|
+
readonly background: boolean;
|
|
20
|
+
constructor(what: string, file: string, pid: number, holder: string, background?: boolean);
|
|
21
|
+
}
|
|
22
|
+
export interface TakeLockOptions {
|
|
23
|
+
/** Marks the lock as a background job's, on a third line, so checks need not wait behind it. */
|
|
24
|
+
background?: boolean;
|
|
6
25
|
}
|
|
7
26
|
/**
|
|
8
27
|
* Takes the lock and returns its release, or throws `LockHeld` while a live process holds it. The
|
|
9
|
-
* lock file holds the pid, then `holder` on a second line when given
|
|
28
|
+
* lock file holds the pid, then `holder` on a second line when given, then `background` on a third
|
|
29
|
+
* for a background job's lock. A file with fewer lines is a foreground lock, as older ones are.
|
|
10
30
|
*/
|
|
11
|
-
export declare function takeLock(file: string, what?: string, holder?: string): () => void;
|
|
12
|
-
export interface WaitForLockOptions {
|
|
31
|
+
export declare function takeLock(file: string, what?: string, holder?: string, options?: TakeLockOptions): () => void;
|
|
32
|
+
export interface WaitForLockOptions extends TakeLockOptions {
|
|
13
33
|
what: string;
|
|
14
34
|
holder?: string;
|
|
35
|
+
/** A holder this caller need not wait behind: the wait ends, returning `undefined`, untaken. */
|
|
36
|
+
skip?: (held: LockHeld) => boolean;
|
|
15
37
|
/** Called when the wait starts, and again whenever another process takes the lock first. */
|
|
16
38
|
onWait?: (held: LockHeld) => void;
|
|
17
39
|
intervalMs?: number;
|
|
18
40
|
sleep?: (ms: number) => Promise<void>;
|
|
19
41
|
}
|
|
20
|
-
/**
|
|
21
|
-
|
|
42
|
+
/**
|
|
43
|
+
* Takes the lock, polling while a live process holds it, and returns its release, or `undefined`
|
|
44
|
+
* once it finds a holder `skip` accepts.
|
|
45
|
+
*/
|
|
46
|
+
export declare function waitForLock(file: string, options: WaitForLockOptions): Promise<(() => void) | undefined>;
|
package/dist/lock.js
CHANGED
|
@@ -2,7 +2,7 @@ import { linkSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs
|
|
|
2
2
|
import { uptime } from "node:os";
|
|
3
3
|
import { defaultSleep } from "./github.js";
|
|
4
4
|
/** Whether a process with this pid is running. */
|
|
5
|
-
function alive(pid) {
|
|
5
|
+
export function alive(pid) {
|
|
6
6
|
if (!Number.isInteger(pid) || pid <= 0)
|
|
7
7
|
return false;
|
|
8
8
|
try {
|
|
@@ -14,6 +14,20 @@ function alive(pid) {
|
|
|
14
14
|
return error.code === "EPERM";
|
|
15
15
|
}
|
|
16
16
|
}
|
|
17
|
+
/**
|
|
18
|
+
* Polls until `pid` exits, returning false if it is still running after `timeoutMs`. git sends a
|
|
19
|
+
* push only after its pre-push hook exits, so the poster waits on git before waiting on GitHub.
|
|
20
|
+
*/
|
|
21
|
+
export async function waitForExit(pid, options = {}) {
|
|
22
|
+
const { intervalMs = 1_000, timeoutMs = 3_600_000 } = options;
|
|
23
|
+
const { sleep = defaultSleep, alive: running = alive } = options;
|
|
24
|
+
for (let waited = 0; running(pid); waited += intervalMs) {
|
|
25
|
+
if (waited + intervalMs > timeoutMs)
|
|
26
|
+
return false;
|
|
27
|
+
await sleep(intervalMs);
|
|
28
|
+
}
|
|
29
|
+
return true;
|
|
30
|
+
}
|
|
17
31
|
/**
|
|
18
32
|
* Whether the lock file was written before the machine booted, so its pid, alive or not, belongs to
|
|
19
33
|
* a process that never held it.
|
|
@@ -26,24 +40,40 @@ function beforeBoot(file) {
|
|
|
26
40
|
return false;
|
|
27
41
|
}
|
|
28
42
|
}
|
|
29
|
-
/**
|
|
43
|
+
/** The third line of a lock taken for a background job, which checks don't wait behind. */
|
|
44
|
+
const BACKGROUND = "background";
|
|
45
|
+
/**
|
|
46
|
+
* A live process holds the lock. `holder` is what it wrote after its pid, such as its repo, and
|
|
47
|
+
* `background` whether it took the lock for a background job.
|
|
48
|
+
*/
|
|
30
49
|
export class LockHeld extends Error {
|
|
31
50
|
pid;
|
|
32
51
|
holder;
|
|
33
|
-
|
|
52
|
+
background;
|
|
53
|
+
constructor(what, file, pid, holder, background = false) {
|
|
34
54
|
super(`${what} in progress (pid ${pid}, lock ${file})`);
|
|
35
55
|
this.pid = pid;
|
|
36
56
|
this.holder = holder;
|
|
57
|
+
this.background = background;
|
|
37
58
|
}
|
|
38
59
|
}
|
|
60
|
+
/** The lock file's text: the pid, then `holder` on a second line and `background` on a third. */
|
|
61
|
+
function lockText(label, { background }) {
|
|
62
|
+
// One line each, so a label can't spill into the background marker's place.
|
|
63
|
+
const holder = label?.replace(/\s*\n\s*/g, " ");
|
|
64
|
+
if (background)
|
|
65
|
+
return `${process.pid}\n${holder ?? ""}\n${BACKGROUND}`;
|
|
66
|
+
return holder ? `${process.pid}\n${holder}` : String(process.pid);
|
|
67
|
+
}
|
|
39
68
|
/**
|
|
40
69
|
* Takes the lock and returns its release, or throws `LockHeld` while a live process holds it. The
|
|
41
|
-
* lock file holds the pid, then `holder` on a second line when given
|
|
70
|
+
* lock file holds the pid, then `holder` on a second line when given, then `background` on a third
|
|
71
|
+
* for a background job's lock. A file with fewer lines is a foreground lock, as older ones are.
|
|
42
72
|
*/
|
|
43
|
-
export function takeLock(file, what = "release", holder) {
|
|
73
|
+
export function takeLock(file, what = "release", holder, options = {}) {
|
|
44
74
|
// Linking a file that already holds the pid publishes the lock atomically, so nobody reads it empty.
|
|
45
75
|
const mine = `${file}.${process.pid}`;
|
|
46
|
-
writeFileSync(mine, holder
|
|
76
|
+
writeFileSync(mine, lockText(holder, options));
|
|
47
77
|
try {
|
|
48
78
|
// Each pass takes the lock, finds a live holder, or clears a released or dead one.
|
|
49
79
|
for (;;) {
|
|
@@ -64,9 +94,9 @@ export function takeLock(file, what = "release", holder) {
|
|
|
64
94
|
continue;
|
|
65
95
|
throw readError;
|
|
66
96
|
}
|
|
67
|
-
const [pid = "",
|
|
97
|
+
const [pid = "", holder = "", flag] = content.split("\n");
|
|
68
98
|
if (alive(Number(pid)) && !beforeBoot(file)) {
|
|
69
|
-
throw new LockHeld(what, file, Number(pid),
|
|
99
|
+
throw new LockHeld(what, file, Number(pid), holder, flag === BACKGROUND);
|
|
70
100
|
}
|
|
71
101
|
// The holder died without releasing it, or its pid is another process's since a reboot.
|
|
72
102
|
rmSync(file, { force: true });
|
|
@@ -77,16 +107,22 @@ export function takeLock(file, what = "release", holder) {
|
|
|
77
107
|
rmSync(mine, { force: true });
|
|
78
108
|
}
|
|
79
109
|
}
|
|
80
|
-
/**
|
|
81
|
-
|
|
110
|
+
/**
|
|
111
|
+
* Takes the lock, polling while a live process holds it, and returns its release, or `undefined`
|
|
112
|
+
* once it finds a holder `skip` accepts.
|
|
113
|
+
*/
|
|
114
|
+
export async function waitForLock(file, options) {
|
|
115
|
+
const { what, holder, skip, onWait, intervalMs = 1_000, sleep = defaultSleep } = options;
|
|
82
116
|
let last;
|
|
83
117
|
for (;;) {
|
|
84
118
|
try {
|
|
85
|
-
return takeLock(file, what, holder);
|
|
119
|
+
return takeLock(file, what, holder, { background: options.background });
|
|
86
120
|
}
|
|
87
121
|
catch (error) {
|
|
88
122
|
if (!(error instanceof LockHeld))
|
|
89
123
|
throw error;
|
|
124
|
+
if (skip?.(error))
|
|
125
|
+
return undefined;
|
|
90
126
|
const seen = `${error.pid}\n${error.holder}`;
|
|
91
127
|
if (seen !== last)
|
|
92
128
|
onWait?.(error);
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
import { execFileSync } from "node:child_process";
|
|
2
|
+
/** Every process's parent and command, or an empty table when `ps` can't list them. */
|
|
3
|
+
export function processTable() {
|
|
4
|
+
const table = new Map();
|
|
5
|
+
try {
|
|
6
|
+
const listing = execFileSync("ps", ["-A", "-o", "pid=,ppid=,comm="], { encoding: "utf8" });
|
|
7
|
+
for (const line of listing.split("\n")) {
|
|
8
|
+
const match = /^\s*(\d+)\s+(\d+)\s?(.*)$/.exec(line);
|
|
9
|
+
if (match)
|
|
10
|
+
table.set(Number(match[1]), { ppid: Number(match[2]), command: match[3].trim() });
|
|
11
|
+
}
|
|
12
|
+
}
|
|
13
|
+
catch {
|
|
14
|
+
return new Map();
|
|
15
|
+
}
|
|
16
|
+
return table;
|
|
17
|
+
}
|
package/dist/push.d.ts
CHANGED
|
@@ -15,3 +15,11 @@ export declare function shasToCheck(refs: PushedRef[]): string[];
|
|
|
15
15
|
export declare function cannotCheckInPlace(sha: string, head: string, porcelain: string): string | null;
|
|
16
16
|
/** `owner/repo` from a GitHub remote URL (SSH or HTTPS), or undefined for another host. */
|
|
17
17
|
export declare function githubRepo(url: string): string | undefined;
|
|
18
|
+
export interface ProcessRow {
|
|
19
|
+
ppid: number;
|
|
20
|
+
command: string;
|
|
21
|
+
}
|
|
22
|
+
/** The pids above a process, nearest first, walking a pid → process table up from its parent `ppid`. */
|
|
23
|
+
export declare function ancestorsFrom(table: Map<number, ProcessRow>, ppid: number): number[];
|
|
24
|
+
/** The nearest `git` process above `start`: the git that runs a hook sends the push once it exits. */
|
|
25
|
+
export declare function nearestGit(table: Map<number, ProcessRow>, start: number): number | undefined;
|
package/dist/push.js
CHANGED
|
@@ -37,3 +37,15 @@ export function cannotCheckInPlace(sha, head, porcelain) {
|
|
|
37
37
|
export function githubRepo(url) {
|
|
38
38
|
return /github\.com[:/]([^/]+\/[^/]+?)(?:\.git)?\/?$/.exec(url)?.[1];
|
|
39
39
|
}
|
|
40
|
+
/** The pids above a process, nearest first, walking a pid → process table up from its parent `ppid`. */
|
|
41
|
+
export function ancestorsFrom(table, ppid) {
|
|
42
|
+
const found = [];
|
|
43
|
+
for (let pid = ppid; pid > 1 && !found.includes(pid); pid = table.get(pid)?.ppid ?? 0) {
|
|
44
|
+
found.push(pid);
|
|
45
|
+
}
|
|
46
|
+
return found;
|
|
47
|
+
}
|
|
48
|
+
/** The nearest `git` process above `start`: the git that runs a hook sends the push once it exits. */
|
|
49
|
+
export function nearestGit(table, start) {
|
|
50
|
+
return ancestorsFrom(table, table.get(start)?.ppid ?? 0).find((pid) => /(^|\/)git$/.test(table.get(pid)?.command ?? ""));
|
|
51
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@jonmilner/local-ci",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "Run a repo's check in a pre-push hook and post the result as GitHub commit statuses",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -17,6 +17,13 @@
|
|
|
17
17
|
"README.md"
|
|
18
18
|
],
|
|
19
19
|
"type": "module",
|
|
20
|
+
"exports": {
|
|
21
|
+
"./lock": {
|
|
22
|
+
"types": "./dist/lock-entry.d.ts",
|
|
23
|
+
"default": "./dist/lock-entry.js"
|
|
24
|
+
},
|
|
25
|
+
"./package.json": "./package.json"
|
|
26
|
+
},
|
|
20
27
|
"publishConfig": {
|
|
21
28
|
"access": "public"
|
|
22
29
|
},
|