link-cache 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,14 +4,96 @@ Zero-dependency helper CLIs for **cached** link checking with [Lychee][], for
4
4
  any static site that builds to a `public/` directory (Docsy, Hugo, and others).
5
5
  Two tools:
6
6
 
7
- - **`lychee-norm-cache`** — run lychee over your built `public/` output, then
8
- normalize `.lycheecache` so reruns and commits stay byte stable.
9
- - **`refcache`** — inspect and prune that `.lycheecache` (the "refcache"): list
10
- the oldest entries, prune a count or percentage, or print a summary.
7
+ - **`lychee-norm-cache`** — run lychee over your built `public/` output, keeping
8
+ the committed `link-cache.jsonc` cache and lychee's derived `.lycheecache` in
9
+ sync.
10
+ - **`link-cache`** inspect and prune the cache: list the oldest entries, prune
11
+ a count or percentage (optionally scoped by URL regex), or print a summary
12
+ (status, provenance, ages). (`refcache` is a deprecated alias.)
13
+
14
+ With a committed `lychee.toml` and `link-cache.jsonc`, these give a site a
15
+ self-contained, cached link-checking setup: fast reruns, and diffs that reflect
16
+ real changes — link statuses, and check recency for freshly re-verified entries.
17
+
18
+ ## The owned cache: `link-cache.jsonc`
19
+
20
+ The committed source of truth is `link-cache.jsonc` — a JSONC file,
21
+ pretty-printed by construction in Prettier's style (so `prettier --check` passes
22
+ it untouched), one multi-line object per URL, sorted, with `//` comments allowed
23
+ on their own lines. Each comment attaches to the entry below it and survives
24
+ rewrites for as long as the entry's status holds; a status change replaces the
25
+ entry and retires its comments (a stale rationale is worse than none). The
26
+ multi-line shape is deliberate: field-per-line entries keep concurrent updates
27
+ merging cleanly under git's normal 3-way merge.
28
+
29
+ ```jsonc
30
+ {
31
+ "https://example.com/": {
32
+ "status": 200,
33
+ "when": "2026-08-29T20:06:38Z",
34
+ "via": "lychee",
35
+ },
36
+ // Seeded pending my-org/repo#123; the target lands with that merge.
37
+ "https://example.com/future-page/": {
38
+ "status": 200,
39
+ "when": "2026-08-29T20:06:38Z",
40
+ "via": "manual",
41
+ "expires": "2026-09-30",
42
+ },
43
+ }
44
+ ```
45
+
46
+ Each entry records:
47
+
48
+ - **`status`** — generalized: positive values are HTTP statuses; `0` is
49
+ unchecked; negative values are tool-specific errors (`-10` timeout, `-20`
50
+ network/DNS, `-30` certificate, `-40` generic client error).
51
+ - **`when`** — the moment the status was established, as RFC3339 UTC at whole
52
+ seconds (`YYYY-MM-DDTHH:MM:SSZ`), converting exactly to and from lychee's
53
+ epoch-seconds cache timestamps. The form is strict — no fractional seconds, no
54
+ offsets — so timestamps are byte-comparable and lexicographically
55
+ chronological.
56
+ - **`via`** — the resolver that set the status: `lychee`, `manual`
57
+ (hand-seeded), or a named specialized resolver (e.g. a browser-grade probe).
58
+ - **`expires`** (optional, `manual` entries) — `YYYY-MM-DD`. Until then the
59
+ entry is trusted (never re-checked, overriding lychee's `max_cache_age`);
60
+ after that, it's re-checked live and replaced by the verified result.
61
+
62
+ Lychee's own CSV cache, `.lycheecache`, is **derived**: `lychee-norm-cache`
63
+ projects the owned cache into it before each run and folds lychee's results back
64
+ afterwards. Gitignore `.lycheecache`; commit `link-cache.jsonc`. A re-check that
65
+ changes an entry's status replaces the entry (provenance moves to `lychee`); a
66
+ re-confirmation leaves provenance-bearing entries (`manual`, named resolvers)
67
+ untouched, while `lychee`-owned entries refresh their `when` to record recency.
68
+ A URL the run itself reports as failing is recorded as a negative tool-error
69
+ status; an entry that merely goes missing from lychee's CSV is left untouched
70
+ (cache-status excludes, cache aging, and site changes all remove entries from
71
+ healthy runs).
72
+
73
+ Without a `link-cache.jsonc`, `lychee-norm-cache` falls back to the legacy mode:
74
+ normalize the committed `.lycheecache` in place. To migrate:
75
+
76
+ ```sh
77
+ npm run check:links -- --migrate # .lycheecache -> link-cache.jsonc
78
+ ```
79
+
80
+ then commit `link-cache.jsonc` and gitignore `.lycheecache`. If the CSV cache
81
+ carried a `merge=union` gitattribute, drop it rather than moving it over: union
82
+ merging proved ineffective in practice, and on a multi-line file it can
83
+ interleave entries into invalid JSON. The owned cache merges with git's normal
84
+ 3-way merge; on a conflict, resolve either way and rerun the check — the next
85
+ run re-normalizes the file.
86
+
87
+ ## Exit codes (`lychee-norm-cache`)
88
+
89
+ - `0` — success.
90
+ - `1` — dead links: the check ran and found failures.
91
+ - `2` — preflight or sanity failure: lychee or `public/` missing, lychee config
92
+ error, or **zero links checked** (an empty or fully-excluded `public/` is a
93
+ false-clean, not a pass).
11
94
 
12
- With a committed `lychee.toml` and `.lycheecache`, these give a site a
13
- self-contained, cached link-checking setup: fast reruns, and diffs that only
14
- change when links actually change.
95
+ Warn-style wrappers can soften exit 1 (advisory link rot) while still failing
96
+ hard on exit 2 (the check didn't actually run).
15
97
 
16
98
  ## Requirements
17
99
 
@@ -31,7 +113,7 @@ npm install --save-dev link-cache
31
113
  Or, to install from GitHub rather than the npm registry:
32
114
 
33
115
  ```sh
34
- npm install --save-dev github:chalin/link-cache#semver:^0.3.0
116
+ npm install --save-dev github:chalin/link-cache#semver:^0.4.0
35
117
  ```
36
118
 
37
119
  This puts both bins on your project's `PATH`.
@@ -44,22 +126,23 @@ Wire the bins into your `package.json` scripts (bare names — `npm run` puts
44
126
  ```json
45
127
  "scripts": {
46
128
  "check:links": "lychee-norm-cache",
47
- "refcache": "refcache"
129
+ "link-cache": "link-cache"
48
130
  }
49
131
  ```
50
132
 
51
133
  ```sh
52
- npm run check:links # check links, then sort/normalize the cache
53
- npm run refcache -- --summary # cache stats (count, oldest, status, ages)
134
+ npm run check:links # sync caches, run lychee, fold results back
135
+ npm run link-cache -- --summary # cache stats (count, oldest, status, via, ages)
136
+ npm run link-cache -- --match 'github\.com' --prune 10 # trim 10 oldest matching
54
137
  ```
55
138
 
56
139
  > [!WARNING]
57
140
  >
58
- > Don't invoke these bins via `npx`: this package isn't on the npm registry, so
59
- > on a stale or missing `node_modules`, `npx` falls back to the public registry
60
- > and runs **whatever package holds the name there** (the `lychee-norm-cache`
61
- > name is squatted). Bare bin names in `npm run` scripts resolve locally or fail
62
- > loudly — they never touch the registry.
141
+ > Don't invoke these bins via `npx`: on a stale or missing `node_modules`, `npx`
142
+ > falls back to the public registry and runs **whatever package holds the bin's
143
+ > name there** (the `lychee-norm-cache` name is squatted). Bare bin names in
144
+ > `npm run` scripts resolve locally or fail loudly — they never touch the
145
+ > registry.
63
146
 
64
147
  `lychee-norm-cache` runs in the current directory (your site root) and forwards
65
148
  any extra arguments to lychee. Run either tool with `--help` for its full
@@ -69,10 +152,12 @@ forwards (e.g. `--offline`, `--max-cache-age 0`).
69
152
  ## Development
70
153
 
71
154
  The published CLIs have **zero runtime dependencies**; Prettier is the only dev
72
- dependency. Run the checks (format + tests) with:
155
+ dependency. The committed `.npmrc` applies the usual supply-chain controls
156
+ (lock-exact installs, script execution default-deny, release cooldown). Run the
157
+ checks (format + tests) with:
73
158
 
74
159
  ```sh
75
- npm install
160
+ npm run install:safe
76
161
  npm run check
77
162
  ```
78
163
 
package/check/index.mjs CHANGED
@@ -1,32 +1,58 @@
1
1
  #!/usr/bin/env node
2
- // The `lychee-norm-cache` bin: run lychee over a built site and normalize
3
- // .lycheecache for byte-stable reruns. Runs in the cwd (the site root), so any
4
- // site with a lychee.toml can reuse it. Run with `--help` for usage.
2
+ // The `lychee-norm-cache` bin: run lychee over a built site, maintained around
3
+ // an owned JSONC cache (link-cache.jsonc, the committed source of truth) with
4
+ // lychee's CSV .lycheecache derived from it. Falls back to legacy CSV-only
5
+ // normalization when no owned cache exists. Run with `--help` for usage.
5
6
  //
6
7
  // When GITHUB_TOKEN is unset, a token is bridged from the gh CLI — lychee reads
7
8
  // GITHUB_TOKEN, which also lifts the github.com rate limit; CI sets it directly.
9
+ //
10
+ // Exit codes: 0 success; 1 dead links; 2 preflight or sanity failure (lychee
11
+ // or public/ missing, zero links checked, lychee config error).
8
12
 
9
13
  import { spawnSync } from 'node:child_process';
10
- import {
11
- existsSync,
12
- readFileSync,
13
- realpathSync,
14
- statSync,
15
- writeFileSync,
16
- } from 'node:fs';
14
+ import { existsSync, readFileSync, realpathSync, statSync } from 'node:fs';
17
15
  import path from 'node:path';
18
16
  import { fileURLToPath } from 'node:url';
19
17
 
18
+ import {
19
+ CSV_FILE,
20
+ OWNED_FILE,
21
+ mergeBack,
22
+ migrateCsvText,
23
+ parseCsv,
24
+ parseOwned,
25
+ projectToCsv,
26
+ serializeCsv,
27
+ serializeOwned,
28
+ } from '../lib/cache.mjs';
29
+ import { writeFileAtomic } from '../lib/write.mjs';
30
+
20
31
  const INSTALL_HINT = 'https://github.com/lycheeverse/lychee#installation';
21
32
 
22
- const USAGE = `Usage: lychee-norm-cache [lychee args...]
33
+ export const EXIT_OK = 0;
34
+ export const EXIT_DEAD_LINKS = 1;
35
+ export const EXIT_PREFLIGHT = 2;
36
+
37
+ const USAGE = `Usage: lychee-norm-cache [--migrate] [lychee args...]
23
38
 
24
- Run lychee over this site's built ./public output, then normalize .lycheecache
25
- so reruns stay byte stable. Bridges a GitHub token from the gh CLI when
26
- GITHUB_TOKEN isn't set; extra arguments pass through to lychee.
39
+ Run lychee over this site's built ./public output. With a committed
40
+ ${OWNED_FILE}, the ${CSV_FILE} handed to lychee is derived from it before the
41
+ run and folded back into it afterwards; otherwise ${CSV_FILE} is normalized in
42
+ place (legacy mode). Bridges a GitHub token from the gh CLI when GITHUB_TOKEN
43
+ isn't set; extra arguments pass through to lychee.
27
44
 
45
+ --migrate convert an existing ${CSV_FILE} to ${OWNED_FILE} and exit
28
46
  -h, --help show this help
29
47
 
48
+ Exit codes: 0 success; 1 dead links; 2 preflight/sanity failure (missing
49
+ lychee or public/, lychee config or usage errors, zero links verified).
50
+
51
+ Lychee's summary must reach stdout in its default or JSON format — it is the
52
+ wrapper's proof that a check completed — so flags that divert or reshape
53
+ stdout (--output, --format junit, --dump-inputs, ...) are unsupported here;
54
+ run lychee directly for those.
55
+
30
56
  Requires the lychee binary on your PATH and a lychee.toml at the site root.
31
57
  Run \`lychee --help\` for all link-checking options.`;
32
58
 
@@ -66,6 +92,61 @@ export function publicDirOf(cwd) {
66
92
  return publicDir;
67
93
  }
68
94
 
95
+ // Check counts from lychee's stdout summary — the human line ("🔍 2 Total …
96
+ // ✅ 1 OK 🚫 0 Errors …") or --format json fields; null when neither is
97
+ // present. A parsed summary is the proof that a check actually completed.
98
+ export function parseSummary(stdout) {
99
+ const m = /(\d+)\s+Total\b[\s\S]*?(\d+)\s+OK\b[\s\S]*?(\d+)\s+Errors?\b/.exec(
100
+ stdout,
101
+ );
102
+ if (m) return { total: +m[1], ok: +m[2], errors: +m[3] };
103
+ const total = /"total"\s*:\s*(\d+)/.exec(stdout);
104
+ const ok = /"successful"\s*:\s*(\d+)/.exec(stdout);
105
+ const errors = /"errors"\s*:\s*(\d+)/.exec(stdout);
106
+ return total && ok && errors
107
+ ? { total: +total[1], ok: +ok[1], errors: +errors[1] }
108
+ : null;
109
+ }
110
+
111
+ // Map lychee's exit code to ours (0 ok, 1 dead links, 2 preflight). A parsed
112
+ // summary is required in every case: lychee exits 2 both for broken links and
113
+ // for argument errors, and exits 0 from non-check modes (--dump-inputs) and
114
+ // diverted-output runs (--output FILE, --format junit/detailed) — without the
115
+ // summary there is no proof a check completed.
116
+ export function mapLycheeExit(code, summary) {
117
+ if (!summary) return EXIT_PREFLIGHT;
118
+ if (code === 0) return EXIT_OK;
119
+ if (code === 2) return EXIT_DEAD_LINKS;
120
+ return EXIT_PREFLIGHT;
121
+ }
122
+
123
+ // URLs the run itself reported as failing — the human "[ERROR] URL …" lines,
124
+ // or --format json's fail_map. Positive per-URL evidence for the merge-back's
125
+ // failure recording; CSV absence alone proves nothing (cache_exclude_status,
126
+ // max_cache_age, and site changes all remove entries from healthy runs).
127
+ export function parseFailedUrls(stdout) {
128
+ const failed = new Set();
129
+ const trimmed = stdout.trim();
130
+ if (trimmed.startsWith('{')) {
131
+ try {
132
+ const failMap = JSON.parse(trimmed).fail_map ?? {};
133
+ for (const failures of Object.values(failMap)) {
134
+ for (const f of failures) {
135
+ const url = typeof f.url === 'string' ? f.url : f.url?.url;
136
+ if (url) failed.add(url);
137
+ }
138
+ }
139
+ return failed;
140
+ } catch {
141
+ // fall through to the line scan
142
+ }
143
+ }
144
+ for (const m of stdout.matchAll(/^\s*\[ERROR\]\s+(\S+)/gm)) {
145
+ failed.add(m[1]);
146
+ }
147
+ return failed;
148
+ }
149
+
69
150
  // --- lychee invocation -----------------------------------------------------
70
151
 
71
152
  function ghAuthToken() {
@@ -77,39 +158,133 @@ function hasLychee() {
77
158
  return !spawnSync('lychee', ['--version'], { stdio: 'ignore' }).error;
78
159
  }
79
160
 
161
+ function fail(message) {
162
+ process.stderr.write(`[help] ${message}\n`);
163
+ return EXIT_PREFLIGHT;
164
+ }
165
+
166
+ function migrate(cwd) {
167
+ const csvPath = path.join(cwd, CSV_FILE);
168
+ const ownedPath = path.join(cwd, OWNED_FILE);
169
+ if (existsSync(ownedPath)) return fail(`${OWNED_FILE} already exists.`);
170
+ if (!existsSync(csvPath)) return fail(`${CSV_FILE} not found.`);
171
+ const { text, count, malformed } = migrateCsvText(
172
+ readFileSync(csvPath, 'utf8'),
173
+ );
174
+ // Migration is specified lossless: refuse rather than silently drop.
175
+ if (malformed) {
176
+ return fail(
177
+ `${CSV_FILE} has ${malformed} malformed line(s); fix or remove them, then rerun.`,
178
+ );
179
+ }
180
+ writeFileAtomic(ownedPath, text);
181
+ console.log(
182
+ `Migrated ${count} entries to ${OWNED_FILE}. Commit it and gitignore ${CSV_FILE}.`,
183
+ );
184
+ return EXIT_OK;
185
+ }
186
+
80
187
  function main(argv) {
81
188
  if (argv.includes('-h') || argv.includes('--help')) {
82
189
  console.log(USAGE);
83
- return 0;
190
+ return EXIT_OK;
84
191
  }
85
192
 
193
+ const cwd = process.cwd();
194
+ if (argv.includes('--migrate')) return migrate(cwd);
195
+
86
196
  if (!hasLychee()) {
87
- process.stderr.write(`[help] lychee not found. Install: ${INSTALL_HINT}\n`);
88
- return 1;
197
+ return fail(`lychee not found. Install: ${INSTALL_HINT}`);
89
198
  }
90
199
 
91
- const cwd = process.cwd();
92
200
  const publicDir = publicDirOf(cwd);
93
201
  if (!publicDir) {
94
- process.stderr.write(
95
- `[help] ${path.join(cwd, 'public')} not found. Build the site first.\n`,
202
+ return fail(`${path.join(cwd, 'public')} not found. Build the site first.`);
203
+ }
204
+
205
+ const ownedPath = path.join(cwd, OWNED_FILE);
206
+ const cachePath = path.join(cwd, CSV_FILE);
207
+ const owned = existsSync(ownedPath)
208
+ ? parseOwned(readFileSync(ownedPath, 'utf8'))
209
+ : null;
210
+ const now = Date.now() / 1000;
211
+
212
+ // The owned-cache lens requires lychee's cache; enforce rather than let a
213
+ // cacheless run erase every projected entry on merge-back.
214
+ const hasCacheFlag = argv.some(
215
+ (a) => a === '--cache' || a.startsWith('--cache='),
216
+ );
217
+ if (owned && argv.includes('--cache=false')) {
218
+ return fail(`--cache=false is incompatible with ${OWNED_FILE}.`);
219
+ }
220
+ const cacheArgs = hasCacheFlag ? [] : ['--cache'];
221
+
222
+ // Derive the CSV lychee will read from the owned cache.
223
+ if (owned) {
224
+ writeFileAtomic(
225
+ cachePath,
226
+ serializeCsv(projectToCsv(owned.entries, { now })),
96
227
  );
97
- return 1;
98
228
  }
99
229
 
100
230
  const token = resolveToken();
101
- const status =
102
- spawnSync(
103
- 'lychee',
104
- ['--config', 'lychee.toml', '--root-dir', publicDir, ...argv, publicDir],
105
- { stdio: 'inherit', env: { ...process.env, GITHUB_TOKEN: token } },
106
- ).status ?? 1;
107
-
108
- // Normalize the cache even on failure, so a partial run still leaves a stable
109
- // .lycheecache.
110
- const cachePath = path.join(cwd, '.lycheecache');
111
- if (existsSync(cachePath)) {
112
- writeFileSync(cachePath, sortCacheText(readFileSync(cachePath, 'utf8')));
231
+ // stdout is captured for the completion/zero-check sanity checks and echoed
232
+ // afterwards; stderr (progress) still streams. The generous buffer bound
233
+ // keeps spawnSync simple; overruns surface via run.error below.
234
+ const run = spawnSync(
235
+ 'lychee',
236
+ [
237
+ '--config',
238
+ 'lychee.toml',
239
+ ...cacheArgs,
240
+ '--root-dir',
241
+ publicDir,
242
+ ...argv,
243
+ publicDir,
244
+ ],
245
+ {
246
+ stdio: ['inherit', 'pipe', 'inherit'],
247
+ encoding: 'utf8',
248
+ maxBuffer: 64 * 1024 * 1024,
249
+ env: { ...process.env, GITHUB_TOKEN: token },
250
+ },
251
+ );
252
+ if (run.stdout) process.stdout.write(run.stdout);
253
+ if (run.error) {
254
+ return fail(`lychee failed to run: ${run.error.message}`);
255
+ }
256
+ const summary = parseSummary(run.stdout ?? '');
257
+ const status = mapLycheeExit(run.status ?? 1, summary);
258
+
259
+ // On a preflight failure lychee produced no trustworthy results: leave both
260
+ // caches untouched (folding would mislabel unprojected entries as failed).
261
+ if (status === EXIT_PREFLIGHT) {
262
+ return fail('lychee did not complete a check; caches left untouched.');
263
+ }
264
+
265
+ // A clean run in which nothing got a verdict is a false-clean (empty or
266
+ // fully-excluded public/), not a success — and folding it would mislabel
267
+ // unattempted entries as failed, so the check precedes the fold. Cache hits
268
+ // count toward OK in lychee's summary, so a fully-cached run passes.
269
+ if (status === EXIT_OK && summary && summary.ok + summary.errors === 0) {
270
+ return fail('lychee verified 0 links: empty or fully-excluded public/?');
271
+ }
272
+
273
+ // Fold the post-run CSV back into the owned cache (also on dead links, so a
274
+ // completed run with failures still leaves a stable, truthful cache), or
275
+ // normalize the CSV in place in legacy mode.
276
+ if (owned) {
277
+ const csvEntries = existsSync(cachePath)
278
+ ? parseCsv(readFileSync(cachePath, 'utf8')).entries
279
+ : [];
280
+ const failedUrls = parseFailedUrls(run.stdout ?? '');
281
+ writeFileAtomic(
282
+ ownedPath,
283
+ serializeOwned(mergeBack(owned, csvEntries, { now, failedUrls })),
284
+ );
285
+ writeFileAtomic(cachePath, sortCacheText(readFileSync(cachePath, 'utf8')));
286
+ } else if (existsSync(cachePath)) {
287
+ writeFileAtomic(cachePath, sortCacheText(readFileSync(cachePath, 'utf8')));
113
288
  }
114
289
 
115
290
  return status;
@@ -126,5 +301,12 @@ function isEntryPoint() {
126
301
  }
127
302
 
128
303
  if (isEntryPoint()) {
129
- process.exit(main(process.argv.slice(2)));
304
+ // Wrapper exceptions (e.g. a malformed owned cache) are preflight failures,
305
+ // not dead links: warn wrappers must not swallow them.
306
+ try {
307
+ process.exit(main(process.argv.slice(2)));
308
+ } catch (err) {
309
+ process.stderr.write(`[error] ${err.message}\n`);
310
+ process.exit(EXIT_PREFLIGHT);
311
+ }
130
312
  }
package/lib/cache.mjs ADDED
@@ -0,0 +1,369 @@
1
+ // Shared cache-file model: the owned JSONC cache (link-cache.jsonc, source of
2
+ // truth) and its derived Lychee CSV (.lycheecache). Pure functions only — no
3
+ // I/O — so both bins and the tests share one implementation.
4
+ //
5
+ // Owned-file format: JSONC, pretty-printed by construction in Prettier's jsonc
6
+ // style (2-space indent, one field per line, trailing commas), so the file
7
+ // passes `prettier --check` untouched. Every entry is multi-line — the field
8
+ // separation is what keeps concurrent edits merging cleanly — and whole-line
9
+ // `//` comments attach to the entry that follows them. Entries are sorted by
10
+ // raw URL byte order (matching LC_ALL=C). Stripping comments and trailing
11
+ // commas yields valid JSON.
12
+ //
13
+ // Entry schema: { "status": INT, "when": RFC3339_UTC, "via": "RESOLVER" }
14
+ // with an optional "expires": "YYYY-MM-DD" (manual entries). Status
15
+ // generalizes HTTP: positive = HTTP status, 0 = unchecked, negative =
16
+ // tool-specific error. `when` is the canonical whole-second UTC form
17
+ // (YYYY-MM-DDTHH:MM:SSZ) — no fractional seconds, no offsets — validated
18
+ // strictly: this tool is the file's only writer, so a non-canonical value is
19
+ // a bug to surface, not smooth over (lesson from the htmltest-era
20
+ // RFC3339Nano/offset churn).
21
+
22
+ export const OWNED_FILE = 'link-cache.jsonc';
23
+ export const CSV_FILE = '.lycheecache';
24
+
25
+ // Tool-specific error statuses (the htmltest-fork convention).
26
+ export const STATUS_TIMEOUT = -10;
27
+ export const STATUS_NETWORK_ERROR = -20;
28
+ export const STATUS_CERT_ERROR = -30;
29
+ export const STATUS_CLIENT_ERROR = -40;
30
+
31
+ const WHEN_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$/;
32
+
33
+ // Epoch seconds -> canonical `when`.
34
+ export function tsToWhen(ts) {
35
+ return new Date(ts * 1000).toISOString().replace(/\.\d{3}Z$/, 'Z');
36
+ }
37
+
38
+ // Canonical `when` -> epoch seconds; NaN when non-canonical (strict: the
39
+ // regex gates shape, and the round-trip equality check rejects impossible
40
+ // dates that Date.parse would silently normalize, e.g. Feb 30 -> Mar 2).
41
+ export function whenToTs(when) {
42
+ if (!WHEN_RE.test(when)) return NaN;
43
+ const ts = Date.parse(when) / 1000;
44
+ return tsToWhen(ts) === when ? ts : NaN;
45
+ }
46
+
47
+ const byUrlBytes = (a, b) =>
48
+ Buffer.compare(Buffer.from(a.url), Buffer.from(b.url));
49
+
50
+ // --- CSV (.lycheecache) ------------------------------------------------------
51
+
52
+ // Unquote a CSV field: lychee quotes URLs that contain a comma.
53
+ export function csvUnquote(field) {
54
+ return field.startsWith('"') && field.endsWith('"')
55
+ ? field.slice(1, -1).replace(/""/g, '"')
56
+ : field;
57
+ }
58
+
59
+ export function csvQuote(url) {
60
+ return /[",]/.test(url) ? `"${url.replace(/"/g, '""')}"` : url;
61
+ }
62
+
63
+ // Parse one URL,STATUS,TIMESTAMP line; null when malformed. Lexically strict:
64
+ // STATUS and TIMESTAMP must be non-empty digit runs (`Number('')` is 0, which
65
+ // would silently coin status-0/epoch-0 entries from truncated lines).
66
+ export function parseCsvLine(raw) {
67
+ const lastComma = raw.lastIndexOf(',');
68
+ if (lastComma < 0) return null;
69
+ const tsField = raw.slice(lastComma + 1);
70
+ const head = raw.slice(0, lastComma);
71
+ const statusComma = head.lastIndexOf(',');
72
+ if (statusComma < 0) return null;
73
+ const statusField = head.slice(statusComma + 1).trim();
74
+ if (!/^\d+$/.test(tsField) || !/^-?\d+$/.test(statusField)) return null;
75
+ const url = csvUnquote(head.slice(0, statusComma));
76
+ if (url === '') return null;
77
+ return { url, status: Number(statusField), ts: Number(tsField) };
78
+ }
79
+
80
+ // Parse a whole CSV cache; malformed lines are counted, duplicates keep the
81
+ // newest timestamp.
82
+ export function parseCsv(text) {
83
+ const byUrl = new Map();
84
+ let malformed = 0;
85
+ for (const raw of text.split('\n')) {
86
+ if (raw.trim() === '') continue;
87
+ const entry = parseCsvLine(raw);
88
+ if (!entry) {
89
+ malformed += 1;
90
+ continue;
91
+ }
92
+ const prior = byUrl.get(entry.url);
93
+ if (!prior || entry.ts >= prior.ts) byUrl.set(entry.url, entry);
94
+ }
95
+ return { entries: [...byUrl.values()].sort(byUrlBytes), malformed };
96
+ }
97
+
98
+ export function serializeCsv(entries) {
99
+ if (entries.length === 0) return '';
100
+ return (
101
+ [...entries]
102
+ .sort(byUrlBytes)
103
+ .map((e) => `${csvQuote(e.url)},${e.status},${e.ts}`)
104
+ .join('\n') + '\n'
105
+ );
106
+ }
107
+
108
+ // --- owned JSONC (link-cache.jsonc) -----------------------------------------
109
+
110
+ const COMMENT_RE = /^\s*\/\//;
111
+ // The line that opens an entry: `"URL": {` (the key is a JSON string, which
112
+ // cannot span lines).
113
+ const ENTRY_OPEN_RE = /^\s*("(?:[^"\\]|\\.)*")\s*:\s*\{/;
114
+
115
+ // Strip a line-ending comma when the next non-blank line closes an object —
116
+ // per-line, so commas inside URL strings are never touched (JSON strings
117
+ // cannot span lines).
118
+ function stripTrailingCommas(lines) {
119
+ const out = [...lines];
120
+ for (let i = 0; i < out.length; i++) {
121
+ if (!out[i].trimEnd().endsWith(',')) continue;
122
+ let j = i + 1;
123
+ while (j < out.length && out[j].trim() === '') j++;
124
+ if (j < out.length && out[j].trim().startsWith('}')) {
125
+ out[i] = out[i].trimEnd().slice(0, -1);
126
+ }
127
+ }
128
+ return out;
129
+ }
130
+
131
+ const ENTRY_FIELDS = new Set(['status', 'when', 'via', 'expires']);
132
+
133
+ // A calendar-real YYYY-MM-DD (round-trip check rejects e.g. Feb 30).
134
+ function isCanonicalDate(s) {
135
+ if (!/^\d{4}-\d{2}-\d{2}$/.test(s)) return false;
136
+ const t = Date.parse(`${s}T00:00:00Z`);
137
+ return Number.isFinite(t) && new Date(t).toISOString().slice(0, 10) === s;
138
+ }
139
+
140
+ // Rejects unknown fields (a rewrite would silently drop them) and restricts
141
+ // `expires` to manual entries — resolvers set statuses, owners set trust
142
+ // windows. Relax the via restriction if a named resolver ever needs expiry.
143
+ // Statuses live in one of three domains: HTTP (100-999, the only ones that
144
+ // project into Lychee's CSV — Lychee rejects codes outside that range at
145
+ // load, which would wedge the run), 0 (unchecked), negative (tool errors).
146
+ function isValidStatus(s) {
147
+ return Number.isInteger(s) && (s < 0 || s === 0 || (s >= 100 && s <= 999));
148
+ }
149
+
150
+ function validateEntry(url, v) {
151
+ return (
152
+ v !== null &&
153
+ typeof v === 'object' &&
154
+ Object.keys(v).every((k) => ENTRY_FIELDS.has(k)) &&
155
+ isValidStatus(v.status) &&
156
+ typeof v.when === 'string' &&
157
+ Number.isFinite(whenToTs(v.when)) &&
158
+ whenToTs(v.when) >= 0 && // pre-epoch would project a negative CSV ts
159
+ typeof v.via === 'string' &&
160
+ (v.expires === undefined ||
161
+ (v.via === 'manual' && isCanonicalDate(v.expires)))
162
+ );
163
+ }
164
+
165
+ // Parse the owned cache. Throws on malformed content (the file is committed
166
+ // and tool-written; silent drops would lose data on the next rewrite),
167
+ // including blank input — a zero-byte file is a truncated write, not an empty
168
+ // cache (that one is `{}`). Comments are collected line-wise and attached to
169
+ // the entry whose opening `"URL": {` line follows them; comments after the
170
+ // last entry are kept as `trailing`. Duplicate URL keys are rejected: with
171
+ // merge=union gone, a duplicate is a bad merge resolution to surface, not a
172
+ // leftover to paper over.
173
+ export function parseOwned(text) {
174
+ if (text.trim() === '') {
175
+ throw new Error(`${OWNED_FILE}: blank file (truncated write?)`);
176
+ }
177
+ const lines = text.split('\n');
178
+
179
+ // Comment attachment pass; also detects duplicate keys line-wise (the JSON
180
+ // pass below cannot: Object.entries already collapsed them).
181
+ const commentsByUrl = new Map();
182
+ const seenUrls = new Set();
183
+ const trailing = [];
184
+ let pending = [];
185
+ for (const raw of lines) {
186
+ if (COMMENT_RE.test(raw)) {
187
+ pending.push(` ${raw.trim()}`);
188
+ continue;
189
+ }
190
+ const open = ENTRY_OPEN_RE.exec(raw);
191
+ if (open) {
192
+ const url = JSON.parse(open[1]);
193
+ if (seenUrls.has(url)) {
194
+ throw new Error(`${OWNED_FILE}: duplicate entry for ${url}`);
195
+ }
196
+ seenUrls.add(url);
197
+ if (pending.length) {
198
+ commentsByUrl.set(url, pending);
199
+ pending = [];
200
+ }
201
+ }
202
+ }
203
+ trailing.push(...pending);
204
+
205
+ // Data pass: strip comments and trailing commas, then parse as JSON.
206
+ const dataLines = stripTrailingCommas(
207
+ lines.filter((l) => !COMMENT_RE.test(l)),
208
+ );
209
+ let obj;
210
+ try {
211
+ obj = JSON.parse(dataLines.join('\n') || '{}');
212
+ } catch (err) {
213
+ throw new Error(`${OWNED_FILE}: invalid JSONC: ${err.message}`);
214
+ }
215
+ if (obj === null || typeof obj !== 'object' || Array.isArray(obj)) {
216
+ throw new Error(`${OWNED_FILE}: top level must be an object`);
217
+ }
218
+
219
+ const entries = [];
220
+ for (const [url, v] of Object.entries(obj)) {
221
+ if (!validateEntry(url, v)) {
222
+ throw new Error(`${OWNED_FILE}: malformed entry for ${url}`);
223
+ }
224
+ entries.push({
225
+ url,
226
+ status: v.status,
227
+ ts: whenToTs(v.when), // epoch internally; `when` re-derives on write
228
+ via: v.via,
229
+ expires: v.expires,
230
+ comments: commentsByUrl.get(url) ?? [],
231
+ });
232
+ }
233
+ return { entries: entries.sort(byUrlBytes), trailing };
234
+ }
235
+
236
+ // Canonical serialization = Prettier's jsonc style (2-space indent, one field
237
+ // per line, trailing commas, always multi-line), so the output is
238
+ // prettier-idempotent by construction.
239
+ export function serializeEntry({ url, status, ts, via, expires }) {
240
+ const fields = [
241
+ ` "status": ${status},`,
242
+ ` "when": ${JSON.stringify(tsToWhen(ts))},`,
243
+ ` "via": ${JSON.stringify(via)},`,
244
+ ];
245
+ if (expires !== undefined) {
246
+ fields.push(` "expires": ${JSON.stringify(expires)},`);
247
+ }
248
+ return [` ${JSON.stringify(url)}: {`, ...fields, ' },'].join('\n');
249
+ }
250
+
251
+ export function serializeOwned({ entries, trailing = [] }) {
252
+ if (entries.length === 0 && trailing.length === 0) return '{}\n';
253
+ const lines = ['{'];
254
+ for (const entry of [...entries].sort(byUrlBytes)) {
255
+ lines.push(...(entry.comments ?? []).map((c) => ` ${c.trim()}`));
256
+ lines.push(serializeEntry(entry));
257
+ }
258
+ lines.push(...trailing.map((c) => ` ${c.trim()}`), '}');
259
+ return lines.join('\n') + '\n';
260
+ }
261
+
262
+ // --- lens: owned -> CSV projection ------------------------------------------
263
+
264
+ const dateToTs = (isoDate) => Date.parse(`${isoDate}T23:59:59Z`) / 1000;
265
+
266
+ // Project the owned entries into Lychee's CSV. Only positive (HTTP) statuses
267
+ // project — Lychee never persists errors, and would re-drop the rest. An
268
+ // unexpired entry with `expires` projects with a fresh timestamp so it
269
+ // outlives Lychee's max_cache_age until its own expiry; once expired it is
270
+ // omitted, forcing a live re-check.
271
+ export function projectToCsv(entries, { now = Date.now() / 1000 } = {}) {
272
+ const projected = [];
273
+ for (const e of entries) {
274
+ if (e.status <= 0) continue;
275
+ if (e.expires !== undefined) {
276
+ const cutoff = dateToTs(e.expires);
277
+ if (!Number.isFinite(cutoff) || now > cutoff) continue;
278
+ projected.push({ url: e.url, status: e.status, ts: Math.floor(now) });
279
+ continue;
280
+ }
281
+ projected.push({ url: e.url, status: e.status, ts: e.ts });
282
+ }
283
+ return projected;
284
+ }
285
+
286
+ // --- lens: CSV -> owned merge-back ------------------------------------------
287
+
288
+ // Fold a post-run CSV back into the owned entries. Rules (owner, 08-31;
289
+ // absence-inference dropped 08-31 adversarial round):
290
+ // - new URL: added with via "lychee";
291
+ // - status changed: the whole entry is replaced (via "lychee", fresh fields,
292
+ // comments dropped — a stale rationale is worse than none);
293
+ // - status equal: provenance-bearing entries (via != "lychee") stay
294
+ // untouched — a re-confirmation; "lychee" entries adopt the fresh ts;
295
+ // - expired `expires` entry present in the CSV: replaced wholesale (the seed
296
+ // served its purpose);
297
+ // - URL in `failedUrls` (positive evidence from the run's own error report):
298
+ // recorded as STATUS_CLIENT_ERROR under via "lychee", keeping the entry's
299
+ // comments (the rationale still explains the URL);
300
+ // - entry missing from the CSV without failure evidence: untouched. CSV
301
+ // absence is ambiguous — cache_exclude_status, max_cache_age expiry, and
302
+ // URLs no longer in the site all remove entries from healthy runs — so it
303
+ // never justifies a failure verdict on its own.
304
+ export function mergeBack(
305
+ owned,
306
+ csvEntries,
307
+ { now = Date.now() / 1000, failedUrls = new Set() } = {},
308
+ ) {
309
+ const csvByUrl = new Map(csvEntries.map((e) => [e.url, e]));
310
+ const merged = [];
311
+
312
+ for (const entry of owned.entries) {
313
+ const csv = csvByUrl.get(entry.url);
314
+ csvByUrl.delete(entry.url);
315
+
316
+ const expired =
317
+ entry.expires !== undefined &&
318
+ (!Number.isFinite(dateToTs(entry.expires)) ||
319
+ now > dateToTs(entry.expires));
320
+
321
+ if (failedUrls.has(entry.url)) {
322
+ merged.push({
323
+ url: entry.url,
324
+ status: STATUS_CLIENT_ERROR,
325
+ ts: Math.floor(now),
326
+ via: 'lychee',
327
+ comments: entry.comments ?? [],
328
+ });
329
+ continue;
330
+ }
331
+
332
+ if (!csv) {
333
+ merged.push(entry); // absent without evidence: untouched
334
+ continue;
335
+ }
336
+
337
+ if (expired || csv.status !== entry.status) {
338
+ merged.push({
339
+ url: entry.url,
340
+ status: csv.status,
341
+ ts: csv.ts,
342
+ via: 'lychee',
343
+ comments: [],
344
+ });
345
+ } else if (entry.via === 'lychee') {
346
+ merged.push({ ...entry, ts: csv.ts });
347
+ } else {
348
+ merged.push(entry); // re-confirmation: provenance untouched
349
+ }
350
+ }
351
+
352
+ for (const csv of csvByUrl.values()) {
353
+ merged.push({ ...csv, via: 'lychee', comments: [] });
354
+ }
355
+
356
+ return { entries: merged.sort(byUrlBytes), trailing: owned.trailing };
357
+ }
358
+
359
+ // --- migration ---------------------------------------------------------------
360
+
361
+ // One-shot .lycheecache -> link-cache.jsonc: every entry becomes via "lychee".
362
+ export function migrateCsvText(csvText) {
363
+ const { entries, malformed } = parseCsv(csvText);
364
+ const owned = {
365
+ entries: entries.map((e) => ({ ...e, via: 'lychee', comments: [] })),
366
+ trailing: [],
367
+ };
368
+ return { text: serializeOwned(owned), count: entries.length, malformed };
369
+ }
package/lib/write.mjs ADDED
@@ -0,0 +1,12 @@
1
+ // Atomic file write shared by the bins: write to a same-directory temp file,
2
+ // then rename over the target. A crash mid-write leaves the original intact
3
+ // instead of a truncated cache (which the strict parsers would then reject,
4
+ // wedging every subsequent run).
5
+
6
+ import { renameSync, writeFileSync } from 'node:fs';
7
+
8
+ export function writeFileAtomic(path, text) {
9
+ const tmp = `${path}.${process.pid}.tmp`;
10
+ writeFileSync(tmp, text);
11
+ renameSync(tmp, path);
12
+ }
@@ -1,17 +1,25 @@
1
1
  #!/usr/bin/env node
2
- // The `refcache` bin: inspect and prune a Lychee refcache. Read-only unless
3
- // `--prune` is given, and never hits the network. Run with `--help` for usage.
2
+ // The `link-cache` bin (deprecated alias: `refcache`): inspect and prune a
3
+ // link cache the owned JSONC cache (link-cache.jsonc) or a legacy Lychee CSV
4
+ // (.lycheecache). Read-only unless `--prune` is given, and never hits the
5
+ // network. Run with `--help` for usage.
4
6
  //
5
- // Refcache line format: URL,STATUS,UNIX_TIMESTAMP — the URL is CSV-quoted when
6
- // it contains a comma, so STATUS and TIMESTAMP are read from the final two
7
- // fields.
7
+ // CSV line format: URL,STATUS,UNIX_TIMESTAMP — the URL is CSV-quoted when it
8
+ // contains a comma, so STATUS and TIMESTAMP are read from the final two fields.
8
9
 
9
- import { readFileSync, realpathSync, writeFileSync } from 'node:fs';
10
+ import { existsSync, readFileSync, realpathSync } from 'node:fs';
10
11
  import { fileURLToPath } from 'node:url';
11
12
 
13
+ import {
14
+ CSV_FILE,
15
+ OWNED_FILE,
16
+ parseOwned,
17
+ serializeOwned,
18
+ } from '../lib/cache.mjs';
19
+ import { writeFileAtomic } from '../lib/write.mjs';
20
+
12
21
  const DAY = 86400;
13
22
  const BUCKET_COUNT = 5;
14
- const DEFAULT_PATH = '.lycheecache';
15
23
 
16
24
  // --- parsing ---------------------------------------------------------------
17
25
 
@@ -28,9 +36,9 @@ function parseLine(raw) {
28
36
  return { url, status, ts };
29
37
  }
30
38
 
31
- // Parse the whole file, keeping the original lines (so a prune can rewrite the
32
- // file with a minimal, still-sorted diff) and tagging each entry with its line
33
- // index.
39
+ // Parse a whole CSV cache, keeping the original lines (so a prune can rewrite
40
+ // the file with a minimal, still-sorted diff) and tagging each entry with its
41
+ // line index.
34
42
  export function parseCache(text) {
35
43
  const lines = text.split('\n');
36
44
  const entries = [];
@@ -44,7 +52,21 @@ export function parseCache(text) {
44
52
  }
45
53
  entries.push({ ...parsed, index });
46
54
  });
47
- return { lines, entries, malformed };
55
+ return { kind: 'csv', lines, entries, malformed };
56
+ }
57
+
58
+ // Adapt the owned JSONC cache to the same entry shape (string status, index).
59
+ export function parseOwnedCache(text) {
60
+ const owned = parseOwned(text);
61
+ const entries = owned.entries.map((e, index) => ({
62
+ url: e.url,
63
+ status: String(e.status),
64
+ ts: e.ts,
65
+ via: e.via,
66
+ index,
67
+ src: e,
68
+ }));
69
+ return { kind: 'owned', owned, entries, malformed: 0 };
48
70
  }
49
71
 
50
72
  // --- selection / pruning ---------------------------------------------------
@@ -88,20 +110,24 @@ export function computeStats(
88
110
  { now = Date.now() / 1000, malformed = 0, pruned = 0 } = {},
89
111
  ) {
90
112
  const statusCounts = new Map();
113
+ const viaCounts = new Map();
91
114
  const ages = [];
92
115
  let oldestTs = null;
93
116
  let newestTs = null;
94
117
 
95
118
  for (const entry of entries) {
96
119
  statusCounts.set(entry.status, (statusCounts.get(entry.status) ?? 0) + 1);
120
+ if (entry.via !== undefined) {
121
+ viaCounts.set(entry.via, (viaCounts.get(entry.via) ?? 0) + 1);
122
+ }
97
123
  if (oldestTs === null || entry.ts < oldestTs) oldestTs = entry.ts;
98
124
  if (newestTs === null || entry.ts > newestTs) newestTs = entry.ts;
99
125
  ages.push(Math.floor((now - entry.ts) / DAY));
100
126
  }
101
127
 
102
- const byStatus = [...statusCounts.entries()].sort(
103
- (a, b) => b[1] - a[1] || a[0].localeCompare(b[0]),
104
- );
128
+ const desc = (a, b) => b[1] - a[1] || a[0].localeCompare(b[0]);
129
+ const byStatus = [...statusCounts.entries()].sort(desc);
130
+ const byVia = [...viaCounts.entries()].sort(desc);
105
131
 
106
132
  const at = (ts) =>
107
133
  ts === null ? null : { ts, ageDays: Math.floor((now - ts) / DAY) };
@@ -112,6 +138,7 @@ export function computeStats(
112
138
  malformed,
113
139
  pruned,
114
140
  byStatus,
141
+ byVia,
115
142
  oldest,
116
143
  newest: at(newestTs),
117
144
  ageBuckets: ageHistogram(ages, oldest ? oldest.ageDays : 0),
@@ -145,14 +172,15 @@ export function formatList(entries, n, { now = Date.now() / 1000 } = {}) {
145
172
  const head = `${chosen.length} oldest ${chosen.length === 1 ? 'entry' : 'entries'}:`;
146
173
  const rows = chosen.map((e) => {
147
174
  const ageDays = Math.floor((now - e.ts) / DAY);
148
- return ` ${localStamp(e.ts)} ${String(ageDays).padStart(4)}d ${e.status} ${displayUrl(e.url)}`;
175
+ const via = e.via !== undefined ? ` ${e.via}` : '';
176
+ return ` ${localStamp(e.ts)} ${String(ageDays).padStart(4)}d ${e.status}${via} ${displayUrl(e.url)}`;
149
177
  });
150
178
  return [head, ...rows].join('\n');
151
179
  }
152
180
 
153
181
  export function formatStats(stats, { now = Date.now() / 1000, path } = {}) {
154
182
  const lines = [];
155
- lines.push(`Lychee refcache${path ? `: ${path}` : ''}`);
183
+ lines.push(`Link cache${path ? `: ${path}` : ''}`);
156
184
  lines.push(` Entries: ${stats.count}`);
157
185
  if (stats.pruned) {
158
186
  const before = stats.count + stats.pruned;
@@ -174,6 +202,14 @@ export function formatStats(stats, { now = Date.now() / 1000, path } = {}) {
174
202
  lines.push(` ${status.padEnd(5)} ${n}`);
175
203
  }
176
204
 
205
+ if (stats.byVia?.length) {
206
+ lines.push(' Via:');
207
+ const w = Math.max(...stats.byVia.map(([via]) => via.length));
208
+ for (const [via, n] of stats.byVia) {
209
+ lines.push(` ${via.padEnd(w)} ${n}`);
210
+ }
211
+ }
212
+
177
213
  if (stats.ageBuckets.length) {
178
214
  lines.push(' Age:');
179
215
  const w = Math.max(...stats.ageBuckets.map((b) => b.label.length));
@@ -187,13 +223,22 @@ export function formatStats(stats, { now = Date.now() / 1000, path } = {}) {
187
223
 
188
224
  // --- ordered execution -----------------------------------------------------
189
225
 
190
- // Run the parsed ops in order over the evolving cache. Pure: returns the text to
191
- // print and the lines to write (null when nothing was pruned); no I/O.
192
- export function runOps(parsed, ops, { now = Date.now() / 1000, path } = {}) {
226
+ // Run the parsed ops in order over the evolving cache, optionally scoped to
227
+ // URLs matching `match`. Pure: returns the text to print and the text to write
228
+ // (null when nothing was pruned); no I/O.
229
+ export function runOps(
230
+ parsed,
231
+ ops,
232
+ { now = Date.now() / 1000, path, match = null } = {},
233
+ ) {
193
234
  const removed = new Set();
194
235
  let pruned = 0;
195
236
  const out = [];
196
- const current = () => parsed.entries.filter((e) => !removed.has(e.index));
237
+ // Match against the unquoted URL: legacy-CSV entries carry the raw
238
+ // (possibly CSV-quoted) field, which would defeat anchored regexes.
239
+ const inScope = (e) => (match ? match.test(displayUrl(e.url)) : true);
240
+ const current = () =>
241
+ parsed.entries.filter((e) => !removed.has(e.index) && inScope(e));
197
242
 
198
243
  for (const op of ops) {
199
244
  if (op.kind === 'list') {
@@ -216,9 +261,24 @@ export function runOps(parsed, ops, { now = Date.now() / 1000, path } = {}) {
216
261
  }
217
262
  }
218
263
 
219
- const writeLines =
220
- pruned > 0 ? parsed.lines.filter((_, i) => !removed.has(i)) : null;
221
- return { output: out.join('\n\n'), writeLines, pruned };
264
+ let writeText = null;
265
+ if (pruned > 0) {
266
+ // Survivors are all unpruned entries — including those outside the
267
+ // --match scope, which the scope only shields from ops, never from the
268
+ // rewrite.
269
+ const survivors = parsed.entries.filter((e) => !removed.has(e.index));
270
+ if (parsed.kind === 'owned') {
271
+ // Comments of surviving entries travel with them; pruned entries take
272
+ // their comment lines along.
273
+ writeText = serializeOwned({
274
+ entries: survivors.map((e) => e.src),
275
+ trailing: parsed.owned.trailing,
276
+ });
277
+ } else {
278
+ writeText = parsed.lines.filter((_, i) => !removed.has(i)).join('\n');
279
+ }
280
+ }
281
+ return { output: out.join('\n\n'), writeText, pruned };
222
282
  }
223
283
 
224
284
  // --- CLI -------------------------------------------------------------------
@@ -226,6 +286,8 @@ export function runOps(parsed, ops, { now = Date.now() / 1000, path } = {}) {
226
286
  const FLAGS = new Map([
227
287
  ['-l', 'list'],
228
288
  ['--list', 'list'],
289
+ ['-m', 'match'],
290
+ ['--match', 'match'],
229
291
  ['-p', 'prune'],
230
292
  ['--prune', 'prune'],
231
293
  ['-s', 'summary'],
@@ -236,6 +298,7 @@ export function parseArgs(argv) {
236
298
  const ops = [];
237
299
  const seen = new Set();
238
300
  let path = null;
301
+ let match = null;
239
302
  let help = false;
240
303
 
241
304
  for (let i = 0; i < argv.length; i++) {
@@ -254,6 +317,14 @@ export function parseArgs(argv) {
254
317
  }
255
318
  const value = argv[++i];
256
319
  if (value === undefined) throw new Error(`${a} requires a value`);
320
+ if (kind === 'match') {
321
+ try {
322
+ match = new RegExp(value);
323
+ } catch (err) {
324
+ throw new Error(`--match needs a valid regex: ${err.message}`);
325
+ }
326
+ continue;
327
+ }
257
328
  if (kind === 'list' && !/^\d+$/.test(value)) {
258
329
  throw new Error(`--list needs a count, got: ${value}`);
259
330
  }
@@ -268,23 +339,25 @@ export function parseArgs(argv) {
268
339
  path = a;
269
340
  }
270
341
 
271
- return { ops, path: path ?? DEFAULT_PATH, help };
342
+ return { ops, path, match, help };
272
343
  }
273
344
 
274
- const USAGE = `Usage: refcache [REFCACHE_FILE] [options]
345
+ const USAGE = `Usage: link-cache [CACHE_FILE] [options]
275
346
 
276
- Inspect and prune a Lychee refcache (default: ${DEFAULT_PATH}). Options run in
277
- the order given, over the evolving cache, so \`-l 5 -p 5\` lists the 5 oldest
278
- about to be pruned, while \`-p 5 -l 5\` lists the next 5 after pruning.
347
+ Inspect and prune a link cache: the owned ${OWNED_FILE} (default when present)
348
+ or a legacy Lychee CSV like ${CSV_FILE}. Options run in the order given, over
349
+ the evolving cache, so \`-l 5 -p 5\` lists the 5 oldest about to be pruned,
350
+ while \`-p 5 -l 5\` lists the next 5 after pruning.
279
351
 
280
352
  -l, --list NUM list the NUM oldest entries
353
+ -m, --match REGEX scope all operations to URLs matching REGEX
281
354
  -p, --prune NUM[%] drop the NUM (or NUM%) oldest entries, then rewrite
282
- -s, --summary print a summary (counts, ages, status, histogram)
355
+ -s, --summary print a summary (counts, ages, status, via, histogram)
283
356
  -h, --help show this help
284
357
 
285
358
  With no options, prints the summary. A flag may not be repeated.`;
286
359
 
287
- function main(argv) {
360
+ export function main(argv) {
288
361
  let args;
289
362
  try {
290
363
  args = parseArgs(argv);
@@ -298,23 +371,35 @@ function main(argv) {
298
371
  return;
299
372
  }
300
373
 
374
+ const path = args.path ?? (existsSync(OWNED_FILE) ? OWNED_FILE : CSV_FILE);
375
+
301
376
  let text;
302
377
  try {
303
- text = readFileSync(args.path, 'utf8');
378
+ text = readFileSync(path, 'utf8');
304
379
  } catch {
305
- console.error(`[error] cannot read refcache file: ${args.path}`);
380
+ console.error(`[error] cannot read cache file: ${path}`);
381
+ process.exit(1);
382
+ }
383
+
384
+ const isOwned = path.endsWith('.jsonc') || text.trimStart().startsWith('{');
385
+ let parsed;
386
+ try {
387
+ parsed = isOwned ? parseOwnedCache(text) : parseCache(text);
388
+ } catch (err) {
389
+ console.error(`[error] ${err.message}`);
306
390
  process.exit(1);
307
391
  }
308
392
 
309
393
  const now = Date.now() / 1000;
310
394
  const ops = args.ops.length ? args.ops : [{ kind: 'summary' }];
311
- const { output, writeLines } = runOps(parseCache(text), ops, {
395
+ const { output, writeText } = runOps(parsed, ops, {
312
396
  now,
313
- path: args.path,
397
+ path,
398
+ match: args.match,
314
399
  });
315
400
 
316
401
  if (output) console.log(output);
317
- if (writeLines) writeFileSync(args.path, writeLines.join('\n'));
402
+ if (writeText !== null) writeFileAtomic(path, writeText);
318
403
  }
319
404
 
320
405
  // Real-path compare, not `file://${argv[1]}`: npm links bins as symlinks, so
@@ -0,0 +1,11 @@
1
+ #!/usr/bin/env node
2
+ // The deprecated `refcache` alias bin: warn, then delegate to link-cache. A
3
+ // dedicated wrapper (not an argv[1] sniff) so the notice also fires under
4
+ // npm's Windows shims, which invoke the target file directly.
5
+
6
+ import { main } from './index.mjs';
7
+
8
+ process.stderr.write(
9
+ '[warn] the refcache bin is deprecated; use link-cache instead.\n',
10
+ );
11
+ main(process.argv.slice(2));
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "link-cache",
3
- "version": "0.3.0",
4
- "description": "Zero-dependency helper CLIs for cached Lychee link-checking: lychee-norm-cache (run + normalize .lycheecache) and refcache (inspect/prune the cache).",
3
+ "version": "0.4.0",
4
+ "description": "Zero-dependency helper CLIs for cached Lychee link-checking around an owned JSONC cache (link-cache.jsonc) with provenance: lychee-norm-cache (run + sync caches) and link-cache (inspect/prune).",
5
5
  "keywords": [
6
6
  "lychee",
7
7
  "link-checker",
@@ -17,16 +17,21 @@
17
17
  "type": "module",
18
18
  "bin": {
19
19
  "lychee-norm-cache": "check/index.mjs",
20
- "refcache": "refcache/index.mjs"
20
+ "link-cache": "link-cache/index.mjs",
21
+ "refcache": "link-cache/refcache.mjs"
21
22
  },
22
23
  "files": [
23
24
  "check/index.mjs",
24
- "refcache/index.mjs"
25
+ "lib/cache.mjs",
26
+ "lib/write.mjs",
27
+ "link-cache/index.mjs",
28
+ "link-cache/refcache.mjs"
25
29
  ],
26
30
  "scripts": {
27
31
  "check": "npm run check:format && npm test",
28
32
  "check:format": "prettier --check .",
29
33
  "fix:format": "prettier --write .",
34
+ "install:safe": "npm ci --ignore-scripts --no-audit --no-fund",
30
35
  "test": "node --test"
31
36
  },
32
37
  "prettier": {