@jonmilner/local-ci 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/dist/check-lock.d.ts +10 -4
- package/dist/check-lock.js +30 -17
- package/dist/lock.d.ts +21 -7
- package/dist/lock.js +32 -10
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -58,7 +58,7 @@ A repo without a `package.json` sets `LOCAL_CI_COMMAND`, for example on the hook
|
|
|
58
58
|
|
|
59
59
|
Logs, the record of passed commits and the release lock live in `~/.local/state/local-ci/<repo>.log`, `<repo>.passed` and `<repo>.release.lock` (`$XDG_STATE_HOME` if set). A lock whose process has died is taken over.
|
|
60
60
|
|
|
61
|
-
One check command runs at a time on the machine, across every repo, so overlapping checks can't starve each other's tests: `pre-push`, `status`, and the `status` runs `renovate` starts each take `check.lock` in the same directory around the check command (not the status posts). A run that finds it held prints `local-ci: waiting for the check lock, held by pid <pid> (<repo> <short-sha>)`, again whenever another run takes it first, and polls every second until the holder exits or dies; Ctrl-C stops the wait. After the wait it skips a commit that passed meanwhile and re-applies the clean-tree rule. The lock goes when the check passes, fails or is killed by a signal. Ctrl-C, SIGTERM or SIGHUP to local-ci passes the signal to the check's whole process group (so a check script's children get it too), waits for the check to exit, then releases the lock and dies of the signal, so a check never outlives its lock. A lock whose holder died (SIGKILL, a crash) or that predates the last boot is taken over by the next run. A local-ci check nested inside a check (one that pushes, say) sees its ancestor holds the lock and runs without waiting. Each check run appends its duration and outcome to the log when it ends, e.g. `[<end time>] abc1234: pnpm check passed in 71.2s` (or `failed in`, or `did not finish after` when the check is killed by a signal or can't start); a commit that already passed is skipped and logs nothing. Ctrl-C stops the check, which logs `did not finish after`.
|
|
61
|
+
One check command runs at a time on the machine, across every repo, so overlapping checks can't starve each other's tests: `pre-push`, `status`, and the `status` runs `renovate` starts each take `check.lock` in the same directory around the check command (not the status posts). A run that finds it held prints `local-ci: waiting for the check lock, held by pid <pid> (<repo> <short-sha>)`, again whenever another run takes it first, and polls every second until the holder exits or dies; Ctrl-C stops the wait. After the wait it skips a commit that passed meanwhile and re-applies the clean-tree rule. The lock goes when the check passes, fails or is killed by a signal. Ctrl-C, SIGTERM or SIGHUP to local-ci passes the signal to the check's whole process group (so a check script's children get it too), waits for the check to exit, then releases the lock and dies of the signal, so a check never outlives its lock. A lock whose holder died (SIGKILL, a crash) or that predates the last boot is taken over by the next run. A local-ci check nested inside a check (one that pushes, say) sees its ancestor holds the lock and runs without waiting. A check that finds a background job's lock (see below) also runs without waiting and leaves that lock alone, printing `local-ci: running beside background job pid <pid> (<holder>), without the check lock`; signals still stop it as above. Each check run appends its duration and outcome to the log when it ends, e.g. `[<end time>] abc1234: pnpm check passed in 71.2s` (or `failed in`, or `did not finish after` when the check is killed by a signal or can't start); a commit that already passed is skipped and logs nothing. Ctrl-C stops the check, which logs `did not finish after`.
|
|
62
62
|
|
|
63
63
|
### Taking the check lock from your own scripts
|
|
64
64
|
|
|
@@ -72,6 +72,6 @@ await withCheckLock("my-repo browser", async () => {
|
|
|
72
72
|
});
|
|
73
73
|
```
|
|
74
74
|
|
|
75
|
-
`withCheckLock(holder, fn, options?)` waits while another holder has the lock, logging `local-ci: waiting for the check lock, held by pid <pid> (<holder>)` to stderr as the CLI does, runs `fn`, and releases the lock when `fn` returns or throws. `holder` is the label other waiters see. `lockCheck(holder, options?)` does the take and returns the release function, for callers that can't wrap their work in a function. `options.file` overrides the lock path and `options.onWait` replaces the wait message. The path is `check.lock` in the state directory above, so no repo config is needed. A call nested under an ancestor process that holds the lock runs without waiting and leaves the lock alone. SIGINT, SIGTERM or SIGHUP releases the lock and ends the process, so a job that needs its own signal cleanup should run it before calling the release itself. The package has an `exports` map, so deep imports other than `./lock` no longer resolve.
|
|
75
|
+
`withCheckLock(holder, fn, options?)` waits while another holder has the lock, logging `local-ci: waiting for the check lock, held by pid <pid> (<holder>)` to stderr as the CLI does, runs `fn`, and releases the lock when `fn` returns or throws. `holder` is the label other waiters see. `lockCheck(holder, options?)` does the take and returns the release function, for callers that can't wrap their work in a function. `options.file` overrides the lock path and `options.onWait` replaces the wait message. The path is `check.lock` in the state directory above, so no repo config is needed. A call nested under an ancestor process that holds the lock runs without waiting and leaves the lock alone. Pass `options.background: true` for a long job run after a push, such as a browser suite: it still waits for a running check, but `local-ci` checks (and other non-background callers) run at once beside it instead of waiting, so a push never sits in its hook behind it. Background jobs still queue behind each other. A check run beside a background job holds no lock, so nothing waits for it: another check, or a background job taking the lock after the first one ends, can overlap it. The lock file marks such a lock with a third line, `background`; a file without it (as older versions wrote) is waited on as before. SIGINT, SIGTERM or SIGHUP releases the lock and ends the process, so a job that needs its own signal cleanup should run it before calling the release itself. The package has an `exports` map, so deep imports other than `./lock` no longer resolve.
|
|
76
76
|
|
|
77
77
|
Part of [jonmilner/config](https://github.com/jonmilner/config#readme).
|
package/dist/check-lock.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import type { ChildProcess } from "node:child_process";
|
|
2
|
-
import { LockHeld } from "./lock.ts";
|
|
2
|
+
import { type LockHeld } from "./lock.ts";
|
|
3
3
|
/** The check lock this process holds through `lockCheck`, if any, for the CLI to attach its check to. */
|
|
4
4
|
export declare function heldCheckLock(): {
|
|
5
5
|
child?: ChildProcess;
|
|
@@ -11,15 +11,21 @@ export interface LockCheckOptions {
|
|
|
11
11
|
file?: string;
|
|
12
12
|
/** Called when the wait starts, and whenever another process takes the lock first. */
|
|
13
13
|
onWait?: (held: LockHeld) => void;
|
|
14
|
+
/**
|
|
15
|
+
* Takes the lock for a long background job, such as a browser suite run after a push. It still
|
|
16
|
+
* waits for a running check, but checks run beside it instead of waiting for it to finish.
|
|
17
|
+
*/
|
|
18
|
+
background?: boolean;
|
|
14
19
|
}
|
|
15
20
|
/** The machine-wide check lock's path, in `$XDG_STATE_HOME/local-ci` (default `~/.local/state`). */
|
|
16
21
|
export declare function checkLockPath(env?: NodeJS.ProcessEnv): string;
|
|
17
22
|
/**
|
|
18
23
|
* Takes the machine-wide check lock, waiting while another check or job holds it, and returns its
|
|
19
24
|
* release. `holder` names the caller in other waiters' messages. A caller nested under an ancestor
|
|
20
|
-
* that holds the lock
|
|
21
|
-
* and
|
|
25
|
+
* that holds the lock, or a check (not `background`) that finds a background job holding it, runs
|
|
26
|
+
* without waiting, and its release leaves the lock alone. SIGINT, SIGTERM and SIGHUP release the
|
|
27
|
+
* lock and end the process.
|
|
22
28
|
*/
|
|
23
|
-
export declare function lockCheck(holder: string, { file, onWait }?: LockCheckOptions): Promise<() => void>;
|
|
29
|
+
export declare function lockCheck(holder: string, { file, onWait, background }?: LockCheckOptions): Promise<() => void>;
|
|
24
30
|
/** Runs `fn` under the machine-wide check lock, so it queues behind `local-ci` checks and vice versa. */
|
|
25
31
|
export declare function withCheckLock<T>(holder: string, fn: () => Promise<T>, options?: LockCheckOptions): Promise<T>;
|
package/dist/check-lock.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { mkdirSync } from "node:fs";
|
|
2
2
|
import { join } from "node:path";
|
|
3
3
|
import { stateDir } from "./config.js";
|
|
4
|
-
import {
|
|
4
|
+
import { waitForLock } from "./lock.js";
|
|
5
5
|
import { processTable } from "./processes.js";
|
|
6
6
|
import { ancestorsFrom } from "./push.js";
|
|
7
7
|
/** This process's ancestors' pids, or just its parent when `ps` can't list them. */
|
|
@@ -68,27 +68,40 @@ export function checkLockPath(env = process.env) {
|
|
|
68
68
|
/**
|
|
69
69
|
* Takes the machine-wide check lock, waiting while another check or job holds it, and returns its
|
|
70
70
|
* release. `holder` names the caller in other waiters' messages. A caller nested under an ancestor
|
|
71
|
-
* that holds the lock
|
|
72
|
-
* and
|
|
71
|
+
* that holds the lock, or a check (not `background`) that finds a background job holding it, runs
|
|
72
|
+
* without waiting, and its release leaves the lock alone. SIGINT, SIGTERM and SIGHUP release the
|
|
73
|
+
* lock and end the process.
|
|
73
74
|
*/
|
|
74
|
-
export async function lockCheck(holder, { file = checkLockPath(), onWait } = {}) {
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
75
|
+
export async function lockCheck(holder, { file = checkLockPath(), onWait, background = false } = {}) {
|
|
76
|
+
let beside;
|
|
77
|
+
let mine;
|
|
78
|
+
// A check nested in a check (one that pushes, say) runs under its ancestor's lock rather than
|
|
79
|
+
// waiting forever on it.
|
|
80
|
+
const nested = (held) => (mine ??= ancestors()).has(held.pid);
|
|
81
|
+
// A check never waits out a background job's long run; it runs beside it.
|
|
82
|
+
const skip = (held) => {
|
|
83
|
+
if (nested(held))
|
|
84
|
+
return true;
|
|
85
|
+
if (!held.background || background)
|
|
86
|
+
return false;
|
|
87
|
+
beside = held;
|
|
88
|
+
return true;
|
|
89
|
+
};
|
|
90
|
+
const unlock = await waitForLock(file, {
|
|
87
91
|
what: "check",
|
|
88
92
|
holder,
|
|
93
|
+
background,
|
|
94
|
+
skip,
|
|
89
95
|
onWait: onWait ??
|
|
90
96
|
(({ pid, holder }) => process.stderr.write(`local-ci: waiting for the check lock, held by pid ${pid} (${holder || "unknown"})\n`)),
|
|
91
|
-
})
|
|
97
|
+
});
|
|
98
|
+
if (unlock)
|
|
99
|
+
return releaseOnSignal(unlock);
|
|
100
|
+
if (!beside)
|
|
101
|
+
return () => { };
|
|
102
|
+
process.stderr.write(`local-ci: running beside background job pid ${beside.pid} (${beside.holder || "unknown"}), without the check lock\n`);
|
|
103
|
+
// Nothing to release, but the signal handling still applies, so the check never outlives this run.
|
|
104
|
+
return releaseOnSignal(() => { });
|
|
92
105
|
}
|
|
93
106
|
/** Runs `fn` under the machine-wide check lock, so it queues behind `local-ci` checks and vice versa. */
|
|
94
107
|
export async function withCheckLock(holder, fn, options) {
|
package/dist/lock.d.ts
CHANGED
|
@@ -9,24 +9,38 @@ export interface ExitWaitOptions extends WaitOptions {
|
|
|
9
9
|
* push only after its pre-push hook exits, so the poster waits on git before waiting on GitHub.
|
|
10
10
|
*/
|
|
11
11
|
export declare function waitForExit(pid: number, options?: ExitWaitOptions): Promise<boolean>;
|
|
12
|
-
/**
|
|
12
|
+
/**
|
|
13
|
+
* A live process holds the lock. `holder` is what it wrote after its pid, such as its repo, and
|
|
14
|
+
* `background` whether it took the lock for a background job.
|
|
15
|
+
*/
|
|
13
16
|
export declare class LockHeld extends Error {
|
|
14
17
|
readonly pid: number;
|
|
15
18
|
readonly holder: string;
|
|
16
|
-
|
|
19
|
+
readonly background: boolean;
|
|
20
|
+
constructor(what: string, file: string, pid: number, holder: string, background?: boolean);
|
|
21
|
+
}
|
|
22
|
+
export interface TakeLockOptions {
|
|
23
|
+
/** Marks the lock as a background job's, on a third line, so checks need not wait behind it. */
|
|
24
|
+
background?: boolean;
|
|
17
25
|
}
|
|
18
26
|
/**
|
|
19
27
|
* Takes the lock and returns its release, or throws `LockHeld` while a live process holds it. The
|
|
20
|
-
* lock file holds the pid, then `holder` on a second line when given
|
|
28
|
+
* lock file holds the pid, then `holder` on a second line when given, then `background` on a third
|
|
29
|
+
* for a background job's lock. A file with fewer lines is a foreground lock, as older ones are.
|
|
21
30
|
*/
|
|
22
|
-
export declare function takeLock(file: string, what?: string, holder?: string): () => void;
|
|
23
|
-
export interface WaitForLockOptions {
|
|
31
|
+
export declare function takeLock(file: string, what?: string, holder?: string, options?: TakeLockOptions): () => void;
|
|
32
|
+
export interface WaitForLockOptions extends TakeLockOptions {
|
|
24
33
|
what: string;
|
|
25
34
|
holder?: string;
|
|
35
|
+
/** A holder this caller need not wait behind: the wait ends, returning `undefined`, untaken. */
|
|
36
|
+
skip?: (held: LockHeld) => boolean;
|
|
26
37
|
/** Called when the wait starts, and again whenever another process takes the lock first. */
|
|
27
38
|
onWait?: (held: LockHeld) => void;
|
|
28
39
|
intervalMs?: number;
|
|
29
40
|
sleep?: (ms: number) => Promise<void>;
|
|
30
41
|
}
|
|
31
|
-
/**
|
|
32
|
-
|
|
42
|
+
/**
|
|
43
|
+
* Takes the lock, polling while a live process holds it, and returns its release, or `undefined`
|
|
44
|
+
* once it finds a holder `skip` accepts.
|
|
45
|
+
*/
|
|
46
|
+
export declare function waitForLock(file: string, options: WaitForLockOptions): Promise<(() => void) | undefined>;
|
package/dist/lock.js
CHANGED
|
@@ -40,24 +40,40 @@ function beforeBoot(file) {
|
|
|
40
40
|
return false;
|
|
41
41
|
}
|
|
42
42
|
}
|
|
43
|
-
/**
|
|
43
|
+
/** The third line of a lock taken for a background job, which checks don't wait behind. */
|
|
44
|
+
const BACKGROUND = "background";
|
|
45
|
+
/**
|
|
46
|
+
* A live process holds the lock. `holder` is what it wrote after its pid, such as its repo, and
|
|
47
|
+
* `background` whether it took the lock for a background job.
|
|
48
|
+
*/
|
|
44
49
|
export class LockHeld extends Error {
|
|
45
50
|
pid;
|
|
46
51
|
holder;
|
|
47
|
-
|
|
52
|
+
background;
|
|
53
|
+
constructor(what, file, pid, holder, background = false) {
|
|
48
54
|
super(`${what} in progress (pid ${pid}, lock ${file})`);
|
|
49
55
|
this.pid = pid;
|
|
50
56
|
this.holder = holder;
|
|
57
|
+
this.background = background;
|
|
51
58
|
}
|
|
52
59
|
}
|
|
60
|
+
/** The lock file's text: the pid, then `holder` on a second line and `background` on a third. */
|
|
61
|
+
function lockText(label, { background }) {
|
|
62
|
+
// One line each, so a label can't spill into the background marker's place.
|
|
63
|
+
const holder = label?.replace(/\s*\n\s*/g, " ");
|
|
64
|
+
if (background)
|
|
65
|
+
return `${process.pid}\n${holder ?? ""}\n${BACKGROUND}`;
|
|
66
|
+
return holder ? `${process.pid}\n${holder}` : String(process.pid);
|
|
67
|
+
}
|
|
53
68
|
/**
|
|
54
69
|
* Takes the lock and returns its release, or throws `LockHeld` while a live process holds it. The
|
|
55
|
-
* lock file holds the pid, then `holder` on a second line when given
|
|
70
|
+
* lock file holds the pid, then `holder` on a second line when given, then `background` on a third
|
|
71
|
+
* for a background job's lock. A file with fewer lines is a foreground lock, as older ones are.
|
|
56
72
|
*/
|
|
57
|
-
export function takeLock(file, what = "release", holder) {
|
|
73
|
+
export function takeLock(file, what = "release", holder, options = {}) {
|
|
58
74
|
// Linking a file that already holds the pid publishes the lock atomically, so nobody reads it empty.
|
|
59
75
|
const mine = `${file}.${process.pid}`;
|
|
60
|
-
writeFileSync(mine, holder
|
|
76
|
+
writeFileSync(mine, lockText(holder, options));
|
|
61
77
|
try {
|
|
62
78
|
// Each pass takes the lock, finds a live holder, or clears a released or dead one.
|
|
63
79
|
for (;;) {
|
|
@@ -78,9 +94,9 @@ export function takeLock(file, what = "release", holder) {
|
|
|
78
94
|
continue;
|
|
79
95
|
throw readError;
|
|
80
96
|
}
|
|
81
|
-
const [pid = "",
|
|
97
|
+
const [pid = "", holder = "", flag] = content.split("\n");
|
|
82
98
|
if (alive(Number(pid)) && !beforeBoot(file)) {
|
|
83
|
-
throw new LockHeld(what, file, Number(pid),
|
|
99
|
+
throw new LockHeld(what, file, Number(pid), holder, flag === BACKGROUND);
|
|
84
100
|
}
|
|
85
101
|
// The holder died without releasing it, or its pid is another process's since a reboot.
|
|
86
102
|
rmSync(file, { force: true });
|
|
@@ -91,16 +107,22 @@ export function takeLock(file, what = "release", holder) {
|
|
|
91
107
|
rmSync(mine, { force: true });
|
|
92
108
|
}
|
|
93
109
|
}
|
|
94
|
-
/**
|
|
95
|
-
|
|
110
|
+
/**
|
|
111
|
+
* Takes the lock, polling while a live process holds it, and returns its release, or `undefined`
|
|
112
|
+
* once it finds a holder `skip` accepts.
|
|
113
|
+
*/
|
|
114
|
+
export async function waitForLock(file, options) {
|
|
115
|
+
const { what, holder, skip, onWait, intervalMs = 1_000, sleep = defaultSleep } = options;
|
|
96
116
|
let last;
|
|
97
117
|
for (;;) {
|
|
98
118
|
try {
|
|
99
|
-
return takeLock(file, what, holder);
|
|
119
|
+
return takeLock(file, what, holder, { background: options.background });
|
|
100
120
|
}
|
|
101
121
|
catch (error) {
|
|
102
122
|
if (!(error instanceof LockHeld))
|
|
103
123
|
throw error;
|
|
124
|
+
if (skip?.(error))
|
|
125
|
+
return undefined;
|
|
104
126
|
const seen = `${error.pid}\n${error.holder}`;
|
|
105
127
|
if (seen !== last)
|
|
106
128
|
onWait?.(error);
|