@sabaiway/agent-workflow-kit 4.3.0 → 4.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +51 -0
- package/SKILL.md +1 -1
- package/capability.json +1 -1
- package/package.json +1 -1
- package/references/hooks/gate-approve.mjs +58 -7
- package/references/modes/velocity.md +3 -2
- package/references/shared/command-shapes.md +25 -24
- package/tools/path-inventory.mjs +516 -0
- package/tools/repo-search.mjs +217 -29
- package/tools/velocity-profile.mjs +6 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,57 @@ Semantically versioned ([semver](https://semver.org)), newest first. The `versio
|
|
|
4
4
|
is the current release. `upgrade` mode reads a project's `docs/ai/.workflow-version` and applies
|
|
5
5
|
every `migrations/<version>-<slug>.md` newer than it, in semver order.
|
|
6
6
|
|
|
7
|
+
## 4.4.0 — the guard stops charging you for answers it already has (AD-082)
|
|
8
|
+
|
|
9
|
+
**If the agent keeps asking you to approve its own `2>/dev/null`, this release is for you.** Two
|
|
10
|
+
things changed, and neither refuses anything.
|
|
11
|
+
|
|
12
|
+
**1. The ask now talks to the AGENT, not only to you.** Until now, when the guard stopped a read
|
|
13
|
+
carrying a redirection it explained itself in *your* approval dialog — and the agent that composed the
|
|
14
|
+
command never heard a word of it. So it composed the same shape again next turn. The ask now also
|
|
15
|
+
carries a hint back to the agent naming the lane that would not have prompted at all. Nothing is
|
|
16
|
+
blocked and nothing new is approved; if your host does not support the field, you get exactly the old
|
|
17
|
+
behaviour. **You should see the same class of prompt arrive less often, not disappear** — the honest
|
|
18
|
+
claim, stated because the alternative would be a promise this cannot keep.
|
|
19
|
+
|
|
20
|
+
**2. Two lanes so several small questions stop becoming one composed shell.**
|
|
21
|
+
|
|
22
|
+
- **`path-inventory.mjs` (new).** Does this exist, what type is it, how big, how many lines, what is
|
|
23
|
+
in that directory, what does this small config say — for any number of paths, in ONE call, promptless.
|
|
24
|
+
**A path that does not exist is a normal answer** (`exists:false`, exit 0), because "does either of
|
|
25
|
+
these exist" is a question whose interesting answer is "no", and a tool that errors on it sends you
|
|
26
|
+
straight back to writing a shell one-liner. Symlinks are reported by type and never followed;
|
|
27
|
+
binaries are reported, never decoded; every bound names itself instead of truncating quietly.
|
|
28
|
+
- **`repo-search.mjs --paths-file`.** The pattern half already had a file lane; the TARGET half did
|
|
29
|
+
not, so a path containing a backtick or `$(` could not be searched without a prompt however you
|
|
30
|
+
quoted it. Now it can. Both lane files are excluded from the search itself, by real path.
|
|
31
|
+
|
|
32
|
+
**What you must do to get the promptless part.** Re-run `/agent-workflow-kit velocity --kit-tools`.
|
|
33
|
+
Allow rules are values already written into your `.claude/settings.json`; upgrading the kit does not
|
|
34
|
+
edit them. The advisor now tells you exactly which entries you are missing.
|
|
35
|
+
|
|
36
|
+
**Also in this release:** the guard's recovery advice is derived per tool (it used to name
|
|
37
|
+
`--pattern-file` for every scanned tool, which was wrong the moment a second one existed), and it now
|
|
38
|
+
names both possibilities — pass the byte out of band if it is part of an argument, drop it if it is a
|
|
39
|
+
real redirect — because the guard cannot tell those apart and should not pretend to.
|
|
40
|
+
|
|
41
|
+
**One consequence worth knowing about.** The guard's scanned-tool match is a substring match across
|
|
42
|
+
the command, deliberately inclusive because an over-match only ever costs an extra prompt. Adding a
|
|
43
|
+
second tool extends that: a command that merely MENTIONS `path-inventory.mjs` and carries a
|
|
44
|
+
redirection now asks where it previously got no decision at all. That is the same licensed over-ask
|
|
45
|
+
`repo-search.mjs` has had since 4.2.0, on one more path string.
|
|
46
|
+
|
|
47
|
+
**One rule worth knowing if you pass paths to either tool.** A target must name exactly one thing:
|
|
48
|
+
no empty value, no NUL byte, no `..` component (Node collapses it before the filesystem is consulted,
|
|
49
|
+
so the tool could answer about a different file than you named). A trailing `/` or `/.` is not
|
|
50
|
+
rejected — it asserts "this is a directory", exactly as it does to your shell. Awkward names are
|
|
51
|
+
still supported: spaces at the edges, backticks, control bytes — that is what the file lanes carry.
|
|
52
|
+
|
|
53
|
+
**Not in this release, deliberately.** A guard rung that DENIES instead of asking was designed,
|
|
54
|
+
reviewed and withdrawn — for the second time — because a command substitution can embed an arbitrary
|
|
55
|
+
command with no separator byte, so any such refusal can destroy work it never meant to touch. The
|
|
56
|
+
guard still never denies.
|
|
57
|
+
|
|
7
58
|
## 4.3.0 — the coverage gate no longer certifies evidence it cannot bind to your tree (AD-081)
|
|
8
59
|
|
|
9
60
|
**Read this if you have ever run `coverage-check --check` on its own.** Until now it read whatever
|
package/SKILL.md
CHANGED
|
@@ -3,7 +3,7 @@ name: agent-workflow-kit
|
|
|
3
3
|
description: Deploy or upgrade a portable AI-agent memory-and-workflow system in any project. Use when the user wants to bootstrap `docs/ai/` + an entry-point `AGENTS.md` (+ `CLAUDE.md` alias) + cap/archive/index enforcement in a new or existing repo, set up the Memory Map and session protocols, install the docs-rotation pre-commit hook, or run `/agent-workflow-kit` / `/agent-workflow-kit upgrade`. Triggers on phrases like "set up the memory system", "deploy the AI workflow here", "bootstrap docs/ai", "upgrade the workflow".
|
|
4
4
|
disable-model-invocation: true
|
|
5
5
|
metadata:
|
|
6
|
-
version: '4.
|
|
6
|
+
version: '4.4.0'
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
# agent-workflow-kit
|
package/capability.json
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sabaiway/agent-workflow-kit",
|
|
3
|
-
"version": "4.
|
|
3
|
+
"version": "4.4.0",
|
|
4
4
|
"description": "Portable, cross-agent memory & workflow for AI coding agents — Claude Code, Codex, Cursor, Devin Desktop. One command deploys an AGENTS.md entry point + docs/ai context with cap/archive/index enforcement into any repo.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ai-agents",
|
|
@@ -45,6 +45,12 @@
|
|
|
45
45
|
// (d) everything else → NO decision: exit 0, no output — the normal permission flow proceeds
|
|
46
46
|
// unchanged. The hook NEVER emits `deny`.
|
|
47
47
|
//
|
|
48
|
+
// EVERY ASK ALSO CARRIES A CALLER-FACING HINT (`additionalContext`). The ask itself goes to the
|
|
49
|
+
// HUMAN and its reason never reaches the caller that composed the command — so the guard knew the
|
|
50
|
+
// right lane and could not say it, which is the one thing every withdrawn deny design was buying.
|
|
51
|
+
// The hint is that channel without the refusal, and a host that ignores the field degrades to
|
|
52
|
+
// silence, i.e. to the previous behaviour. See the note above `HINT_CORE`.
|
|
53
|
+
//
|
|
48
54
|
// WHY THERE IS NO DENY RUNG, AND WHY THE DIRECTION IS RETIRED (kit 4.0.0 built one, AD-078
|
|
49
55
|
// removed it; the class was closed from the other side instead). The withdrawn rung refused a
|
|
50
56
|
// seeded read-only command that provably DISCARDED its output, on the argument that such a refusal
|
|
@@ -272,7 +278,16 @@ export const matchSeededCorePrefix = (command) => {
|
|
|
272
278
|
// review killed it: it would pull ANY unrelated `repo-search.mjs` into rung (b2), turning a
|
|
273
279
|
// pre-existing NO decision into an ASK — a change to a decision path this release has no business
|
|
274
280
|
// touching.
|
|
275
|
-
|
|
281
|
+
// The RECOVERY is per tool, not a constant. When repo-search was the only entry the reason named the
|
|
282
|
+
// literal `--pattern-file`, and that was wrong the moment a second tool joined: wrong for a tool with
|
|
283
|
+
// no such flag, and already incomplete for repo-search itself once its TARGET half gained a lane. An
|
|
284
|
+
// ask whose named recovery does not exist is worse than an ask with no advice, so the advice is
|
|
285
|
+
// derived from whichever tool matched.
|
|
286
|
+
export const SCANNED_TOOL_LANES = Object.freeze({
|
|
287
|
+
'agent-workflow-kit/tools/repo-search.mjs': '--pattern-file / --paths-file',
|
|
288
|
+
'agent-workflow-kit/tools/path-inventory.mjs': '--paths-file',
|
|
289
|
+
});
|
|
290
|
+
export const SCANNED_TOOL_PATHS = Object.freeze(Object.keys(SCANNED_TOOL_LANES));
|
|
276
291
|
|
|
277
292
|
// Quotes stripped and separators canonicalised, so a relative, absolute, quoted or Windows-separated
|
|
278
293
|
// spelling all compare the same. SUBSTRING, not equality, and scanned across EVERY token rather than
|
|
@@ -282,10 +297,13 @@ export const SCANNED_TOOL_PATHS = Object.freeze(['agent-workflow-kit/tools/repo-
|
|
|
282
297
|
// surface this exists to cover, while a spurious match merely over-asks — so inclusiveness wins.
|
|
283
298
|
const canonicalToken = (token) => token.replace(/["']/gu, '').replace(/\\/gu, '/');
|
|
284
299
|
|
|
285
|
-
export const matchScannedToolPrefix = (command) =>
|
|
300
|
+
export const matchScannedToolPrefix = (command) => matchScannedTools(command)[0] ?? null;
|
|
301
|
+
|
|
302
|
+
// ALL matches, not just the first. A command naming two scanned tools would otherwise be advised
|
|
303
|
+
// about whichever happens to sit earlier in the registry, and the residual may belong to the other.
|
|
304
|
+
export const matchScannedTools = (command) => {
|
|
286
305
|
const tokens = tokenizeCommand(command).map(canonicalToken);
|
|
287
|
-
|
|
288
|
-
return hit ?? null;
|
|
306
|
+
return SCANNED_TOOL_PATHS.filter((path) => tokens.some((token) => token.includes(path)));
|
|
289
307
|
};
|
|
290
308
|
|
|
291
309
|
// String-level, conservative: the hook sees the PRE-SHELL command string, so every class is a raw
|
|
@@ -381,6 +399,35 @@ export const isReadLaneCommand = (command) => {
|
|
|
381
399
|
});
|
|
382
400
|
};
|
|
383
401
|
|
|
402
|
+
// ── the caller-facing hint (rides an ask; refuses nothing) ────────────────────────────
|
|
403
|
+
//
|
|
404
|
+
// WHY THIS EXISTS, AND WHY IT IS NOT A DENY. An ask is answered by the HUMAN: `permissionDecisionReason`
|
|
405
|
+
// is context for THEIR dialog and never reaches the caller that composed the command. So for the whole
|
|
406
|
+
// life of this guard it has known the right lane and had no way to say it — which is the single reason
|
|
407
|
+
// every withdrawn deny design looked attractive, since a deny is refusal-shaped and does reach the
|
|
408
|
+
// caller. `additionalContext` is the same channel without the refusal. Probed live on 2026-07-28 with
|
|
409
|
+
// a throwaway hook: a distinct marker was delivered to the model on `allow`, on `ask`, AND on no
|
|
410
|
+
// decision at all.
|
|
411
|
+
// FAIL-SAFE BY CONSTRUCTION: a host that does not honour the field simply drops it, which is exactly
|
|
412
|
+
// the behaviour that existed before — silence. Nothing is refused, so a hint that is wrong for a
|
|
413
|
+
// particular command costs the caller a sentence, never their work. That is the property no deny
|
|
414
|
+
// design could ever claim.
|
|
415
|
+
export const HINT_CORE = 'agent-workflow: this read left the promptless lane — a plain read-only command with no redirection, no banner and no quotes stays inside it. If the byte is a decoration you do not need, drop it and re-issue; if it is ordinary text inside an argument, a search belongs in `repo-search.mjs` (`--pattern-file`), and several path questions (exists / size / line count / listing / small-file contents) are ONE `path-inventory.mjs` call. If the byte is a deliberate operator, keep it and confirm by hand.';
|
|
416
|
+
|
|
417
|
+
// The hint must not GUESS which of the two causes produced the residual — the guard cannot tell a
|
|
418
|
+
// byte that is ordinary text inside an argument from a real shell operator, and pretending otherwise
|
|
419
|
+
// is the exact claim AD-079 closed. So it names BOTH recoveries and lets the caller pick: the one
|
|
420
|
+
// that fits is always present, and the one that does not costs a clause.
|
|
421
|
+
// The hint never tells the caller to delete an operator it cannot read the intent of: dropping a
|
|
422
|
+
// real `> report.json` would change what the command DOES, and a guard that silently edits meaning is
|
|
423
|
+
// the failure this whole area was built to avoid. It names the out-of-band lane, notes that these
|
|
424
|
+
// tools already bound and print their own result (so a decoration is usually redundant), and leaves a
|
|
425
|
+
// deliberate operator to the human confirmation that already exists.
|
|
426
|
+
const hintForScannedTools = (tools) => {
|
|
427
|
+
const lanes = tools.map((tool) => `\`${tool}\` → ${SCANNED_TOOL_LANES[tool]}`).join('; ');
|
|
428
|
+
return `agent-workflow: the residual guard stopped this. If the byte is part of an ARGUMENT, pass it out of band — ${lanes} — and its bytes then never enter the command string, so this guard has nothing to scan (whether the clean form still prompts depends on your settings allow rules, which this hook does not read). If the byte is a decoration you do not need, drop it: these tools bound and print their own result. If it is a deliberate operator, keep it and confirm by hand — that is what the prompt is for.`;
|
|
429
|
+
};
|
|
430
|
+
|
|
384
431
|
// ── the decision ladder (pure core) ───────────────────────────────────────────────────
|
|
385
432
|
|
|
386
433
|
export const decideBashCall = ({ command, permissionMode, cwdIsProjectRoot, gates, readLaneOn = false }) => {
|
|
@@ -404,19 +451,23 @@ export const decideBashCall = ({ command, permissionMode, cwdIsProjectRoot, gate
|
|
|
404
451
|
return {
|
|
405
452
|
permissionDecision: DECISION_ASK,
|
|
406
453
|
permissionDecisionReason: `agent-workflow residual guard: read-only-seeded "${corePrefix}" carries ${residualClasses.join(' + ')} — a settings allow rule cannot see runtime shape; confirm by hand`,
|
|
454
|
+
additionalContext: HINT_CORE,
|
|
407
455
|
};
|
|
408
456
|
}
|
|
409
457
|
}
|
|
410
458
|
// (b2) the same guard over the scanned kit tools. The refusal NAMES the lane that avoids it, so a
|
|
411
459
|
// caller who picked the inline lane for a shell-significant pattern is corrected by the mechanism
|
|
412
460
|
// rather than expected to have remembered the rule.
|
|
413
|
-
const
|
|
414
|
-
if (
|
|
461
|
+
const scannedTools = matchScannedTools(trimmed);
|
|
462
|
+
if (scannedTools.length > 0) {
|
|
415
463
|
const residualClasses = detectResidualClasses(trimmed);
|
|
416
464
|
if (residualClasses.length > 0) {
|
|
465
|
+
const named = scannedTools.map((tool) => `"${tool}"`).join(' + ');
|
|
466
|
+
const lanes = [...new Set(scannedTools.map((tool) => SCANNED_TOOL_LANES[tool]))].join(' / ');
|
|
417
467
|
return {
|
|
418
468
|
permissionDecision: DECISION_ASK,
|
|
419
|
-
permissionDecisionReason: `agent-workflow residual guard:
|
|
469
|
+
permissionDecisionReason: `agent-workflow residual guard: ${named} carries ${residualClasses.join(' + ')} — pass the byte-carrying argument out of band with ${lanes} (its bytes then never enter the command string), or confirm by hand if the byte is really part of the invocation`,
|
|
470
|
+
additionalContext: hintForScannedTools(scannedTools),
|
|
420
471
|
};
|
|
421
472
|
}
|
|
422
473
|
}
|
|
@@ -33,11 +33,12 @@ Run `node ${CLAUDE_SKILL_DIR}/tools/velocity-profile.mjs [--dry-run | --apply] [
|
|
|
33
33
|
- `node ${CLAUDE_SKILL_DIR}/tools/recommendations.mjs --cwd ${PROJECT_ROOT}` (wildcard — the read-only deployment advisor; its rendered apply one-liners are writers and still prompt)
|
|
34
34
|
- `node ${CLAUDE_SKILL_DIR}/tools/manifest/validate.mjs --strict <skill-dir>` (wildcard)
|
|
35
35
|
- `node ${CLAUDE_SKILL_DIR}/tools/release-scan.mjs <path>` (wildcard)
|
|
36
|
-
- `node ${CLAUDE_SKILL_DIR}/tools/repo-search.mjs --pattern <literal> --path <p>` (wildcard — the LITERAL search lane; for a pattern carrying a shell-significant byte use `--pattern-file <p>` instead,
|
|
36
|
+
- `node ${CLAUDE_SKILL_DIR}/tools/repo-search.mjs --pattern <literal> --path <p>` (wildcard — the LITERAL search lane; for a pattern carrying a shell-significant byte use `--pattern-file <p>` instead, and for a TARGET PATH carrying one use `--paths-file <p>` — either way the bytes never enter the command string. one of the TWO tier tools that carry hook residual coverage — see the honesty note below)
|
|
37
|
+
- `node ${CLAUDE_SKILL_DIR}/tools/path-inventory.mjs --path <p>` (wildcard — the read-only inventory lane: exists / type / size / `wc -l` lines / one-level listing / small-file contents for N paths in ONE call, so small path questions never need a composed shell. `--paths-file <p>` is its out-of-band target lane; hook residual coverage as with repo-search)
|
|
37
38
|
- `node ${CLAUDE_SKILL_DIR}/tools/run-gates.mjs --cwd ${PROJECT_ROOT}` — **EXACT byte-string only**, and honestly **project-exec, not read-only**: it runs YOUR declared `docs/ai/gates.json` commands — the same trust boundary the opt-in hook grants byte-exact per-cmd. A wildcard would be BROADER than that boundary (`--cwd <dir>` executes another project's declared gates), so the bare cwd-defaulting form, any other `--cwd`, `--only`, and **`--final`** forms all still prompt (`--final` WRITES the final-run receipt into the core-evidence store — a recording run is never auto-approved).
|
|
38
39
|
- Writer previews, **exact arg-free dry-run byte-strings only** (the SEEDED tier byte-string is the arg-free preview of each): `node ${CLAUDE_SKILL_DIR}/tools/velocity-profile.mjs` · `node ${CLAUDE_SKILL_DIR}/tools/cheap-agents.mjs` · `node ${CLAUDE_SKILL_DIR}/tools/gate-hook.mjs` — every `--apply`/`--write`/`--yes` still prompts, always. (`gate-hook` also has a **`--read-lane`** flagged preview — the opt-in read-only compound lane, `${CLAUDE_SKILL_DIR}/references/modes/hook.md`; that flagged form is NOT the seeded arg-free byte-string, so it may **prompt once** — it IS a consent flow, stated, no silent cap.)
|
|
39
40
|
|
|
40
|
-
Honesty notes: tier entries get **NO PreToolUse-hook residual coverage — with
|
|
41
|
+
Honesty notes: tier entries get **NO PreToolUse-hook residual coverage — with TWO deliberate exceptions, `repo-search.mjs` and `path-inventory.mjs`**, whose prefixes are in the hook's scanned list because they take caller-supplied argument bytes: a redirection or substitution on THEIR invocation raises the ask, whose reason names that tool's out-of-band lane (`--pattern-file` / `--paths-file`) — context for the HUMAN deciding, not a message to the caller. Neither is in the seeded core, so neither inherits the read-lane allow. Every other tier entry: the opt-in hook's residual ask-net guards only the seeded read-only CORE prefixes, so the tier rides the same settings-level residual posture as the core (redirection / command substitution are not inspectable at the settings layer; see the residual notice). A skill or project path that cannot survive UNQUOTED in a byte-exact rule (spaces, metacharacters, non-POSIX) **STOPs the tier up front with a clear error** — nothing is seeded. Anything you want covered beyond the tier — such paths, this repo's own relative-path spellings, other tools — stays a **BY-HAND add** to your settings, with the path your project actually reaches the kit by. Pre-existing `node …` allow entries that do NOT match the seeded tier byte-forms stay flagged by the advisory for hand review.
|
|
41
42
|
|
|
42
43
|
**Invariants:** creates `.claude/` if absent and writes **only** `.claude/settings.json` (no other file); **never** allowlists commit/push/publish; **never** writes `settings.local.json`; never commits; opt-in `acceptEdits`, never silent.
|
|
43
44
|
|
|
@@ -1,30 +1,31 @@
|
|
|
1
1
|
### Command shapes — the promptless bar for instructed reads & probes
|
|
2
2
|
|
|
3
|
-
When a mode doc tells you to read a file or probe
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
When a mode doc tells you to read a file or probe state WITHOUT prescribing the exact command (a
|
|
4
|
+
recon read, the version-stamp read, any "check X"), the shape is yours — and improvised shapes are
|
|
5
|
+
where approval prompts come from. The bar:
|
|
6
6
|
|
|
7
|
-
- **Reads ride the host's file-read tool** (Read/Grep/Glob
|
|
8
|
-
|
|
7
|
+
- **Reads ride the host's file-read tool** (Read/Grep/Glob, or your agent's equivalent) whenever one
|
|
8
|
+
exists — a file-read tool never fires a shell approval prompt.
|
|
9
9
|
- **No file-read tool → ONE plain undecorated command per probe:** no `;`/`&&` compounds, no
|
|
10
|
-
redirects, no pipes, no command substitution — one probe per invocation
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
- **Improvised file writes ride the host's file-edit tools** (Write/Edit or the equivalent) —
|
|
18
|
-
never an ad-hoc heredoc or shell-redirect write.
|
|
10
|
+
redirects, no pipes, no command substitution — one probe per invocation (the lens’s «plain pipeline
|
|
11
|
+
per call», stricter). A decorated or chained form is outside this contract’s guarantee: behavior is
|
|
12
|
+
host/config-dependent — it may prompt, an opt-in read-lane may auto-approve a vetted compound, and
|
|
13
|
+
command substitution may even slip past a prefix allow rule. The plain single-command shape is the
|
|
14
|
+
only shell fallback that is promptless by construction.
|
|
15
|
+
- **Improvised file writes ride the host's file-edit tools** (Write/Edit or equivalent) — never an
|
|
16
|
+
ad-hoc heredoc or shell-redirect write.
|
|
19
17
|
- **Searching for TEXT is its own case.** A pattern carrying `>`, `` ` `` or `$(` prompts on a
|
|
20
|
-
seeded-core command however
|
|
21
|
-
|
|
22
|
-
one; else,
|
|
23
|
-
switching to `--pattern-file <path>` for a byte-carrying pattern
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
the
|
|
18
|
+
seeded-core command however quoted — the guard scans the raw string and a quote-stripped copy.
|
|
19
|
+
Quoting is not a workaround. (`|`/`&&` do not trip it.) Use the host's search tool if it has
|
|
20
|
+
one; else, with the kit tier seeded, `node <kit>/tools/repo-search.mjs --pattern <literal>`,
|
|
21
|
+
switching to `--pattern-file <path>` for a byte-carrying pattern and `--paths-file <path>` for a
|
|
22
|
+
byte-carrying TARGET — written with the file-write tool above, so their bytes never enter the
|
|
23
|
+
command string; else one plain command, taking the prompt. A wrong lane earns an ask naming the
|
|
24
|
+
right lane back to YOU, not just the human. **PATH questions too:** exists / size / lines / listing
|
|
25
|
+
(file text needs `--contents`), any number of paths, is ONE
|
|
26
|
+
`node <kit>/tools/path-inventory.mjs --path <p>` — not a bannered compound. Residual: a bare `grep` still prompts, and the file lanes need
|
|
27
|
+
promptless writes.
|
|
27
28
|
|
|
28
|
-
**Scope — improvised shapes only.** The
|
|
29
|
-
|
|
30
|
-
|
|
29
|
+
**Scope — improvised shapes only.** The commands a mode doc itself prescribes (`node …/tools/…`
|
|
30
|
+
dispatch lines, `--apply` lanes, install/symlink steps) are OUTSIDE this contract: run them exactly
|
|
31
|
+
as prescribed, as plain single invocations.
|
|
@@ -0,0 +1,516 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// path-inventory.mjs — the promptless path-inventory lane (read-only).
|
|
3
|
+
//
|
|
4
|
+
// WHY THIS EXISTS. The «useless approves» corpus keeps recording the same authoring shape: several
|
|
5
|
+
// small read-only questions about paths — does it exist, how big is it, how many lines, what is in
|
|
6
|
+
// that directory, what does this small config say — batched into ONE composed shell with `echo`
|
|
7
|
+
// banners and a defensive `2>/dev/null`, because no single call answers them. The composition is
|
|
8
|
+
// what raises the prompt: a redirect takes the command out of the read-lane before it is even split,
|
|
9
|
+
// `echo` is outside the frozen read-only core, and the banner's quotes are forbidden per segment.
|
|
10
|
+
// The corpus has only ever responded to removing the REASON to compose a shell. This tool is that
|
|
11
|
+
// reason removed for the inventory half, the way repo-search.mjs is for the search half.
|
|
12
|
+
//
|
|
13
|
+
// lane 1 --path <p> repeatable, for a path with no shell-significant byte
|
|
14
|
+
// lane 2 --paths-file <p> the targets' bytes NEVER enter the command string
|
|
15
|
+
//
|
|
16
|
+
// CONTRACT
|
|
17
|
+
// A MISSING path is a RESULT (`exists:false`, exit 0), never a failure. That is the whole point:
|
|
18
|
+
// "does either of these exist" is a question whose interesting answer is "no", and a tool that
|
|
19
|
+
// errors on it sends the caller straight back to a composed shell. Only a CONTAINMENT refusal or a
|
|
20
|
+
// real I/O fault is an error.
|
|
21
|
+
// CONTAINMENT is decided on the REAL path of the nearest EXISTING ancestor, so a target that does
|
|
22
|
+
// not exist is still refused when it lives behind a symlink pointing out of the root — a lexical
|
|
23
|
+
// check passes exactly that case.
|
|
24
|
+
// Symlinks are reported BY TYPE and never followed; a dangling one is reported as a symlink that
|
|
25
|
+
// does not resolve. Binary and special files are reported by type and never decoded.
|
|
26
|
+
// Line count is `wc -l` compatible — newline CHARACTERS — so a final line without one is not
|
|
27
|
+
// counted. A tool that answers a different question than the command it replaces is a trap.
|
|
28
|
+
// Results are DETERMINISTIC: targets in input order, directory entries in code-unit order.
|
|
29
|
+
// Four outcomes, never collapsed: results (0), INCOMPLETE (3, naming the bound that fired),
|
|
30
|
+
// invalid input (2), I/O failure or containment refusal (1). No bound ever truncates silently.
|
|
31
|
+
// Pure reader — no writes, no subprocess, no network. Dependency-free, Node >= 22, no side effects
|
|
32
|
+
// on import (the isDirectRun idiom).
|
|
33
|
+
//
|
|
34
|
+
// THREAT MODEL, stated rather than inherited in silence. Containment is decided on the REAL path of
|
|
35
|
+
// the nearest existing ancestor, and every leaf is opened NO-FOLLOW; the directory listing never
|
|
36
|
+
// follows a symlink and never descends. What is NOT defended against is an adversary mutating the
|
|
37
|
+
// tree BETWEEN the containment check and the read: a directory swapped for a symlink in that window
|
|
38
|
+
// would be traversed, and closing it needs descriptor-relative traversal (`openat` semantics) which
|
|
39
|
+
// dependency-free Node does not expose. This is the SAME boundary `repo-search.mjs:33-40` states for
|
|
40
|
+
// the same reason — both tools inspect a workspace their own agent controls, and concurrent hostile
|
|
41
|
+
// mutation is out of scope. An unstated residual would be the defect; the residual itself is not.
|
|
42
|
+
//
|
|
43
|
+
// The paths-file FORMAT and the two failure classes are imported from repo-search.mjs rather than
|
|
44
|
+
// restated: one definition means the two file lanes cannot drift into classifying the same failure
|
|
45
|
+
// differently, which is the parity a copied helper would only promise.
|
|
46
|
+
|
|
47
|
+
import { openSync, fstatSync, readSync, closeSync, opendirSync, realpathSync, lstatSync, constants } from 'node:fs';
|
|
48
|
+
import { resolve, relative, isAbsolute, sep, dirname } from 'node:path';
|
|
49
|
+
import { pathToFileURL } from 'node:url';
|
|
50
|
+
|
|
51
|
+
import {
|
|
52
|
+
UsageError,
|
|
53
|
+
IoError,
|
|
54
|
+
assertNameableTarget,
|
|
55
|
+
requiresDirectory,
|
|
56
|
+
decodeLaneFile,
|
|
57
|
+
parsePathsFile,
|
|
58
|
+
HARD_MAX_TARGETS,
|
|
59
|
+
HARD_MAX_PATHS_FILE_BYTES,
|
|
60
|
+
} from './repo-search.mjs';
|
|
61
|
+
|
|
62
|
+
export { HARD_MAX_TARGETS, HARD_MAX_PATHS_FILE_BYTES };
|
|
63
|
+
|
|
64
|
+
export const EXIT_OK = 0;
|
|
65
|
+
export const EXIT_ERROR = 1;
|
|
66
|
+
export const EXIT_USAGE = 2;
|
|
67
|
+
export const EXIT_INCOMPLETE = 3;
|
|
68
|
+
|
|
69
|
+
// One bound governs every byte this tool reads out of a file: the line count needs the bytes just as
|
|
70
|
+
// much as `--contents` does, so a single named ceiling keeps "why is `lines` null" answerable.
|
|
71
|
+
export const DEFAULT_MAX_CONTENT_BYTES = 1024 * 1024;
|
|
72
|
+
export const HARD_MAX_CONTENT_BYTES = 16 * 1024 * 1024;
|
|
73
|
+
export const DEFAULT_MAX_ENTRIES = 500;
|
|
74
|
+
export const HARD_MAX_ENTRIES = 20000;
|
|
75
|
+
// The AGGREGATE budget, and it is the load-bearing one. Per-file and per-directory ceilings bound a
|
|
76
|
+
// SINGLE target; with up to HARD_MAX_TARGETS of them the run is still unbounded, and every result
|
|
77
|
+
// accumulates in memory before anything is formatted. A cumulative ceiling is what makes the total
|
|
78
|
+
// cost a constant no number of targets can grow.
|
|
79
|
+
export const DEFAULT_MAX_TOTAL_BYTES = 8 * 1024 * 1024;
|
|
80
|
+
export const HARD_MAX_TOTAL_BYTES = 64 * 1024 * 1024;
|
|
81
|
+
export const DEFAULT_MAX_TOTAL_ENTRIES = 20000;
|
|
82
|
+
export const HARD_MAX_TOTAL_ENTRIES = 200000;
|
|
83
|
+
const BINARY_SNIFF_BYTES = 8192;
|
|
84
|
+
const NEWLINE = 0x0a;
|
|
85
|
+
|
|
86
|
+
const NOFOLLOW = constants.O_NOFOLLOW ?? 0;
|
|
87
|
+
const NONBLOCK = constants.O_NONBLOCK ?? 0;
|
|
88
|
+
const OPEN_FLAGS = constants.O_RDONLY | NOFOLLOW | NONBLOCK;
|
|
89
|
+
|
|
90
|
+
export const countLines = (buf) => {
|
|
91
|
+
let n = 0;
|
|
92
|
+
for (let i = 0; i < buf.length; i += 1) if (buf[i] === NEWLINE) n += 1;
|
|
93
|
+
return n;
|
|
94
|
+
};
|
|
95
|
+
|
|
96
|
+
// Over the WHOLE bounded buffer, not a sniff window: the contract says a binary is never decoded, and
|
|
97
|
+
// a NUL past the window would otherwise be handed back as text in direct contradiction of it. The
|
|
98
|
+
// buffer is already in memory and already bounded, so scanning all of it costs nothing extra.
|
|
99
|
+
const isBinary = (buf) => buf.includes(0);
|
|
100
|
+
|
|
101
|
+
const parseCount = (raw, flag, ceiling) => {
|
|
102
|
+
if (!/^\d{1,15}$/u.test(raw ?? '')) throw new UsageError(`${flag} needs a plain non-negative integer, got: ${raw ?? '(missing)'}`);
|
|
103
|
+
const n = Number(raw);
|
|
104
|
+
if (!Number.isSafeInteger(n)) throw new UsageError(`${flag} is not a safe integer: ${raw}`);
|
|
105
|
+
if (n > ceiling) throw new UsageError(`${flag} exceeds the hard ceiling ${ceiling}: ${raw}`);
|
|
106
|
+
return n;
|
|
107
|
+
};
|
|
108
|
+
|
|
109
|
+
const parseArgs = (argv) => {
|
|
110
|
+
const opts = {
|
|
111
|
+
paths: [],
|
|
112
|
+
pathsFile: null,
|
|
113
|
+
contents: false,
|
|
114
|
+
maxContentBytes: DEFAULT_MAX_CONTENT_BYTES,
|
|
115
|
+
maxEntries: DEFAULT_MAX_ENTRIES,
|
|
116
|
+
maxTotalBytes: DEFAULT_MAX_TOTAL_BYTES,
|
|
117
|
+
maxTotalEntries: DEFAULT_MAX_TOTAL_ENTRIES,
|
|
118
|
+
json: false,
|
|
119
|
+
};
|
|
120
|
+
for (let i = 0; i < argv.length; i += 1) {
|
|
121
|
+
const arg = argv[i];
|
|
122
|
+
const next = () => {
|
|
123
|
+
i += 1;
|
|
124
|
+
if (i >= argv.length) throw new UsageError(`${arg} requires a value`);
|
|
125
|
+
return argv[i];
|
|
126
|
+
};
|
|
127
|
+
if (arg === '--path') {
|
|
128
|
+
const value = next();
|
|
129
|
+
// An empty value passes a bare count check and then resolves to the ROOT — precisely the
|
|
130
|
+
// accidental whole-root walk the "name at least one target" rule exists to prevent.
|
|
131
|
+
if (value === '') throw new UsageError('--path needs a non-empty target');
|
|
132
|
+
opts.paths.push(value);
|
|
133
|
+
}
|
|
134
|
+
else if (arg === '--paths-file') opts.pathsFile = next();
|
|
135
|
+
else if (arg === '--contents') opts.contents = true;
|
|
136
|
+
else if (arg === '--max-content-bytes') opts.maxContentBytes = parseCount(next(), '--max-content-bytes', HARD_MAX_CONTENT_BYTES);
|
|
137
|
+
else if (arg === '--max-entries') opts.maxEntries = parseCount(next(), '--max-entries', HARD_MAX_ENTRIES);
|
|
138
|
+
else if (arg === '--max-total-bytes') opts.maxTotalBytes = parseCount(next(), '--max-total-bytes', HARD_MAX_TOTAL_BYTES);
|
|
139
|
+
else if (arg === '--max-total-entries') opts.maxTotalEntries = parseCount(next(), '--max-total-entries', HARD_MAX_TOTAL_ENTRIES);
|
|
140
|
+
else if (arg === '--json') opts.json = true;
|
|
141
|
+
else throw new UsageError(`unknown argument: ${arg} (see --help)`);
|
|
142
|
+
}
|
|
143
|
+
// No implicit target. A tool that walks the whole root when asked about nothing turns a typo into
|
|
144
|
+
// an unbounded read, and the caller's question was always about NAMED paths.
|
|
145
|
+
if (opts.paths.length === 0 && opts.pathsFile === null) {
|
|
146
|
+
throw new UsageError('name at least one target with --path or --paths-file');
|
|
147
|
+
}
|
|
148
|
+
return opts;
|
|
149
|
+
};
|
|
150
|
+
|
|
151
|
+
// Containment on the REAL path of the nearest EXISTING ancestor. A target that does not exist has no
|
|
152
|
+
// real path of its own, and a lexical check on the parent is exactly what a symlinked ancestor
|
|
153
|
+
// defeats — so walk up until something resolves, and judge THAT.
|
|
154
|
+
export const resolveContained = (realRoot, target) => {
|
|
155
|
+
assertNameableTarget(target);
|
|
156
|
+
const lexical = resolve(realRoot, target);
|
|
157
|
+
// Start the probe at the PARENT, never at the target itself: `realpathSync` on the target would
|
|
158
|
+
// dereference a symlink leaf, and this tool promises to report a symlink BY TYPE and never follow
|
|
159
|
+
// it. Probing the leaf turned a symlink pointing outside the root into a refusal instead of a
|
|
160
|
+
// result — a contract violation the contract itself names. The leaf is still safe: it is `lstat`-ed
|
|
161
|
+
// (which never follows) and opened O_NOFOLLOW (which refuses a symlink outright).
|
|
162
|
+
let probe = lexical === realRoot ? lexical : dirname(lexical);
|
|
163
|
+
for (;;) {
|
|
164
|
+
let real;
|
|
165
|
+
try {
|
|
166
|
+
real = realpathSync(probe);
|
|
167
|
+
} catch (err) {
|
|
168
|
+
// ENOTDIR walks up for the same reason ENOENT does: a regular file used as an intermediate
|
|
169
|
+
// component means the rest of the path does not exist, and "does not exist" is an ANSWER here,
|
|
170
|
+
// not a fault. Refusing it would break the tool's central promise on an ordinary typo.
|
|
171
|
+
if (err?.code !== 'ENOENT' && err?.code !== 'ENOTDIR') {
|
|
172
|
+
throw new IoError(`cannot resolve ${target} (${err?.code ?? err?.message ?? err})`);
|
|
173
|
+
}
|
|
174
|
+
const parent = dirname(probe);
|
|
175
|
+
if (parent === probe) throw new IoError(`cannot resolve ${target}`);
|
|
176
|
+
probe = parent;
|
|
177
|
+
continue;
|
|
178
|
+
}
|
|
179
|
+
const rel = relative(realRoot, real);
|
|
180
|
+
if (rel !== '' && (isAbsolute(rel) || rel === '..' || rel.startsWith(`..${sep}`))) {
|
|
181
|
+
throw new IoError(`target resolves outside the root: ${target}`);
|
|
182
|
+
}
|
|
183
|
+
return lexical;
|
|
184
|
+
}
|
|
185
|
+
};
|
|
186
|
+
|
|
187
|
+
// Exported so the `special` fallback is testable: a FIFO, socket or device cannot be created inside
|
|
188
|
+
// the sandbox this suite runs in (a unix socket `listen` is EPERM), and a test that quietly skips
|
|
189
|
+
// when it cannot build its fixture is a test that checks nothing. These are pure maps from a stats
|
|
190
|
+
// shape to a type name, so a stub exercises exactly the branch a device would.
|
|
191
|
+
export const typeOfStats = (st) => {
|
|
192
|
+
if (st.isSymbolicLink()) return 'symlink';
|
|
193
|
+
if (st.isDirectory()) return 'directory';
|
|
194
|
+
if (st.isFile()) return 'file';
|
|
195
|
+
return 'special';
|
|
196
|
+
};
|
|
197
|
+
|
|
198
|
+
export const typeOfDirent = (entry) => {
|
|
199
|
+
if (entry.isSymbolicLink()) return 'symlink';
|
|
200
|
+
if (entry.isDirectory()) return 'directory';
|
|
201
|
+
if (entry.isFile()) return 'file';
|
|
202
|
+
return 'special';
|
|
203
|
+
};
|
|
204
|
+
|
|
205
|
+
// Open → fstat the DESCRIPTOR → read bounded. O_NOFOLLOW refuses a symlinked leaf at open time and
|
|
206
|
+
// O_NONBLOCK means a special file that slipped in returns instead of hanging; fstat-ing the
|
|
207
|
+
// descriptor that was actually opened is what a swap between check and read cannot defeat.
|
|
208
|
+
// Returns a TAGGED outcome, never a bare null. An open that failed, a node that turned out not to be
|
|
209
|
+
// a regular file, and a read that stopped short are three different facts, and collapsing any of them
|
|
210
|
+
// into "here is your answer, lines are just null" is the silent failure this project forbids: the
|
|
211
|
+
// caller cannot tell "empty" from "unreadable" from "half-read".
|
|
212
|
+
const READ_OK = 'ok';
|
|
213
|
+
const READ_UNREADABLE = 'unreadable';
|
|
214
|
+
const READ_OVER_BOUND = 'over-bound';
|
|
215
|
+
const READ_SHORT = 'short';
|
|
216
|
+
|
|
217
|
+
// `io` is injectable so the SHORT-READ branch has a test. A read that stops early is a real race (the
|
|
218
|
+
// file shrinks between the fstat and the read) that no test can stage honestly on a real filesystem,
|
|
219
|
+
// and an untested defensive branch is indistinguishable from a wrong one.
|
|
220
|
+
export const readBounded = (abs, maxBytes, io = {}) => {
|
|
221
|
+
const open = io.open ?? openSync;
|
|
222
|
+
const fstat = io.fstat ?? fstatSync;
|
|
223
|
+
const read = io.read ?? readSync;
|
|
224
|
+
const close = io.close ?? closeSync;
|
|
225
|
+
let fd;
|
|
226
|
+
try {
|
|
227
|
+
fd = open(abs, OPEN_FLAGS);
|
|
228
|
+
} catch (err) {
|
|
229
|
+
return { kind: READ_UNREADABLE, detail: err?.code ?? String(err) };
|
|
230
|
+
}
|
|
231
|
+
try {
|
|
232
|
+
const st = fstat(fd);
|
|
233
|
+
if (!st.isFile()) return { kind: READ_UNREADABLE, detail: 'not a regular file' };
|
|
234
|
+
const size = Number(st.size);
|
|
235
|
+
if (size > maxBytes) return { kind: READ_OVER_BOUND, size };
|
|
236
|
+
const buf = Buffer.allocUnsafe(size);
|
|
237
|
+
let got = 0;
|
|
238
|
+
for (;;) {
|
|
239
|
+
if (got >= size) break;
|
|
240
|
+
const n = read(fd, buf, got, size - got, got);
|
|
241
|
+
if (n <= 0) break;
|
|
242
|
+
got += n;
|
|
243
|
+
}
|
|
244
|
+
if (got !== size) return { kind: READ_SHORT, got, want: size };
|
|
245
|
+
// The size travels WITH the buffer: the caller must report and budget the size the read was
|
|
246
|
+
// actually sized by, never an earlier stat of the path. A file that grew in between would
|
|
247
|
+
// otherwise be published at the wrong size, charged the wrong amount, and returned as complete.
|
|
248
|
+
return { kind: READ_OK, buf, size };
|
|
249
|
+
} finally {
|
|
250
|
+
close(fd);
|
|
251
|
+
}
|
|
252
|
+
};
|
|
253
|
+
|
|
254
|
+
const noteBound = (state, bound, detail) => {
|
|
255
|
+
state.incomplete = state.incomplete ?? { bound, detail };
|
|
256
|
+
};
|
|
257
|
+
|
|
258
|
+
// Returns the entries AND the bound that cut them short, if any: a directory truncated by one
|
|
259
|
+
// ceiling while a later target hits a different one must still be able to say which one truncated IT.
|
|
260
|
+
const listDirectory = (abs, opts, state, rel) => {
|
|
261
|
+
const entries = [];
|
|
262
|
+
let bound = null;
|
|
263
|
+
const dir = opendirSync(abs);
|
|
264
|
+
try {
|
|
265
|
+
for (;;) {
|
|
266
|
+
const entry = dir.readSync();
|
|
267
|
+
if (entry === null) break;
|
|
268
|
+
if (entries.length >= opts.maxEntries) {
|
|
269
|
+
bound = '--max-entries';
|
|
270
|
+
noteBound(state, bound, `${rel} holds more than ${opts.maxEntries} entries`);
|
|
271
|
+
break;
|
|
272
|
+
}
|
|
273
|
+
if (state.entries >= opts.maxTotalEntries) {
|
|
274
|
+
bound = '--max-total-entries';
|
|
275
|
+
noteBound(state, bound, `the run reached ${opts.maxTotalEntries} listed entries at ${rel}`);
|
|
276
|
+
break;
|
|
277
|
+
}
|
|
278
|
+
state.entries += 1;
|
|
279
|
+
entries.push({ name: entry.name, type: typeOfDirent(entry) });
|
|
280
|
+
}
|
|
281
|
+
} finally {
|
|
282
|
+
dir.closeSync();
|
|
283
|
+
}
|
|
284
|
+
entries.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
|
|
285
|
+
return { entries, bound };
|
|
286
|
+
};
|
|
287
|
+
|
|
288
|
+
const inspectTarget = (realRoot, rel, opts, state) => {
|
|
289
|
+
const abs = resolveContained(realRoot, rel);
|
|
290
|
+
let st;
|
|
291
|
+
try {
|
|
292
|
+
st = lstatSync(abs);
|
|
293
|
+
} catch (err) {
|
|
294
|
+
// ENOTDIR is the same answer as ENOENT: a component along the way is a regular file, so the path
|
|
295
|
+
// is not there. Anything else is a genuine fault and stays one.
|
|
296
|
+
if (err?.code === 'ENOENT' || err?.code === 'ENOTDIR') return { path: rel, exists: false };
|
|
297
|
+
throw new IoError(`cannot stat ${rel} (${err?.code ?? err?.message ?? err})`);
|
|
298
|
+
}
|
|
299
|
+
const type = typeOfStats(st);
|
|
300
|
+
// A trailing separator or `.` asserts a directory; the OS answers ENOTDIR when it is not one, and
|
|
301
|
+
// ENOTDIR is "not there" everywhere else in this tool.
|
|
302
|
+
if (requiresDirectory(rel) && type !== 'directory') return { path: rel, exists: false };
|
|
303
|
+
if (type === 'symlink') {
|
|
304
|
+
// Only "the target is not there" means it does not resolve. A permission error, an I/O error or a
|
|
305
|
+
// symlink LOOP are real faults, and reporting them as `(dangling)` with a clean exit would hide a
|
|
306
|
+
// failure behind a normal-looking answer — the silent-failure class this tool refuses elsewhere.
|
|
307
|
+
let resolves = true;
|
|
308
|
+
try {
|
|
309
|
+
realpathSync(abs);
|
|
310
|
+
} catch (err) {
|
|
311
|
+
if (err?.code !== 'ENOENT' && err?.code !== 'ENOTDIR') {
|
|
312
|
+
throw new IoError(`cannot resolve the symlink ${rel} (${err?.code ?? err?.message ?? err})`);
|
|
313
|
+
}
|
|
314
|
+
resolves = false;
|
|
315
|
+
}
|
|
316
|
+
return { path: rel, exists: true, type, resolves };
|
|
317
|
+
}
|
|
318
|
+
if (type === 'directory') {
|
|
319
|
+
const listed = listDirectory(abs, opts, state, rel);
|
|
320
|
+
return {
|
|
321
|
+
path: rel,
|
|
322
|
+
exists: true,
|
|
323
|
+
type,
|
|
324
|
+
entries: listed.entries,
|
|
325
|
+
...(listed.bound === null ? {} : { withheld: listed.bound }),
|
|
326
|
+
};
|
|
327
|
+
}
|
|
328
|
+
if (type !== 'file') return { path: rel, exists: true, type, bytes: Number(st.size) };
|
|
329
|
+
|
|
330
|
+
const size = Number(st.size);
|
|
331
|
+
const base = { path: rel, exists: true, type, bytes: size, readable: true };
|
|
332
|
+
// The bound that withheld THIS entry is recorded on the entry itself. `incomplete` keeps only the
|
|
333
|
+
// first event for the run, so a later target hitting a different ceiling would otherwise come back
|
|
334
|
+
// unread with no stated reason of its own.
|
|
335
|
+
const withheld = (bound, detail, observedBytes = size) => {
|
|
336
|
+
noteBound(state, bound, detail);
|
|
337
|
+
return { ...base, bytes: observedBytes, withheld: bound, lines: null, binary: null, ...(opts.contents ? { contents: null } : {}) };
|
|
338
|
+
};
|
|
339
|
+
|
|
340
|
+
// The read is bounded by whichever ceiling binds FIRST — the per-file one or what is LEFT of the
|
|
341
|
+
// run's budget. Checking the aggregate only afterwards would let every target read a full
|
|
342
|
+
// `--max-content-bytes` before being rejected, so the aggregate ceiling would bound the accounting
|
|
343
|
+
// and not the work, which is the opposite of what it is for.
|
|
344
|
+
const remaining = Math.max(0, opts.maxTotalBytes - state.bytes);
|
|
345
|
+
const limit = Math.min(opts.maxContentBytes, remaining);
|
|
346
|
+
const read = readBounded(abs, limit, opts.io);
|
|
347
|
+
// An unreadable file and a truncated read are I/O FAILURES, not bounds. The exit-code contract
|
|
348
|
+
// reserves 3 for a ceiling that fired and 1 for an I/O failure, and a failed `opendir` already
|
|
349
|
+
// exits 1 — reporting these as "incomplete" put a real fault under a code that means "bounded".
|
|
350
|
+
if (read.kind === READ_UNREADABLE) throw new IoError(`cannot read ${rel} (${read.detail})`);
|
|
351
|
+
if (read.kind === READ_SHORT) throw new IoError(`${rel} yielded ${read.got} of ${read.want} byte(s) — the file changed under the read`);
|
|
352
|
+
if (read.kind === READ_OVER_BOUND) {
|
|
353
|
+
// Which ceiling actually bound it is decided by the descriptor's size, so a file rejected because
|
|
354
|
+
// the RUN had no budget left is never blamed on the per-file limit.
|
|
355
|
+
// `bytes` comes from the DESCRIPTOR here too: the entry must not publish an earlier `lstat` size
|
|
356
|
+
// while the reason beside it quotes a different one.
|
|
357
|
+
// Blame the ceiling that FORMED the limit. Comparing against the per-file bound alone would name
|
|
358
|
+
// it even when the run's remaining budget was the smaller of the two and did the actual cutting.
|
|
359
|
+
return limit === opts.maxContentBytes
|
|
360
|
+
? withheld('--max-content-bytes', `${rel} is ${read.size} byte(s), above ${opts.maxContentBytes}`, read.size)
|
|
361
|
+
: withheld('--max-total-bytes', `${rel} needs ${read.size} byte(s) and the run has ${remaining} left of ${opts.maxTotalBytes}`, read.size);
|
|
362
|
+
}
|
|
363
|
+
state.bytes += read.size;
|
|
364
|
+
const binary = isBinary(read.buf);
|
|
365
|
+
return {
|
|
366
|
+
...base,
|
|
367
|
+
bytes: read.size,
|
|
368
|
+
lines: binary ? null : countLines(read.buf),
|
|
369
|
+
binary,
|
|
370
|
+
...(opts.contents ? { contents: binary ? null : read.buf.toString('utf8') } : {}),
|
|
371
|
+
};
|
|
372
|
+
};
|
|
373
|
+
|
|
374
|
+
export const inventory = ({
|
|
375
|
+
root,
|
|
376
|
+
paths,
|
|
377
|
+
contents = false,
|
|
378
|
+
maxContentBytes = DEFAULT_MAX_CONTENT_BYTES,
|
|
379
|
+
maxEntries = DEFAULT_MAX_ENTRIES,
|
|
380
|
+
maxTotalBytes = DEFAULT_MAX_TOTAL_BYTES,
|
|
381
|
+
maxTotalEntries = DEFAULT_MAX_TOTAL_ENTRIES,
|
|
382
|
+
io = {},
|
|
383
|
+
}) => {
|
|
384
|
+
const state = { incomplete: null, bytes: 0, entries: 0 };
|
|
385
|
+
const opts = { contents, maxContentBytes, maxEntries, maxTotalBytes, maxTotalEntries, io };
|
|
386
|
+
const results = paths.map((rel) => inspectTarget(root, rel, opts, state));
|
|
387
|
+
return { results, incomplete: state.incomplete };
|
|
388
|
+
};
|
|
389
|
+
|
|
390
|
+
// Control and ANSI bytes reach the terminal through a NAME the tool did not choose. Escaping them is
|
|
391
|
+
// not cosmetic: a report about a hostile filename must not be a way to drive the reader's terminal.
|
|
392
|
+
export const escapeForDisplay = (text) =>
|
|
393
|
+
[...text].map((ch) => {
|
|
394
|
+
const code = ch.codePointAt(0);
|
|
395
|
+
if (code < 0x20 || code === 0x7f) return `\\x${code.toString(16).padStart(2, '0')}`;
|
|
396
|
+
return ch;
|
|
397
|
+
}).join('');
|
|
398
|
+
|
|
399
|
+
// The bound that withheld an entry is printed WITH that entry: a run-level `incomplete` line names
|
|
400
|
+
// only the first event, so a reader of the human shape would otherwise see `unread` with no reason.
|
|
401
|
+
const withheldSuffix = (entry) => (entry.withheld === undefined ? '' : ` [withheld: ${entry.withheld}]`);
|
|
402
|
+
|
|
403
|
+
const formatEntry = (entry) => {
|
|
404
|
+
const name = escapeForDisplay(entry.path);
|
|
405
|
+
if (!entry.exists) return `${name}: absent`;
|
|
406
|
+
if (entry.type === 'symlink') return `${name}: symlink${entry.resolves ? '' : ' (dangling)'}`;
|
|
407
|
+
if (entry.type === 'directory') {
|
|
408
|
+
const head = `${name}: directory, ${entry.entries.length} entr(ies)${withheldSuffix(entry)}`;
|
|
409
|
+
return [head, ...entry.entries.map((e) => ` ${escapeForDisplay(e.name)}${e.type === 'directory' ? '/' : ''}`)].join('\n');
|
|
410
|
+
}
|
|
411
|
+
if (entry.type !== 'file') return `${name}: ${entry.type}, ${entry.bytes} byte(s)`;
|
|
412
|
+
const lines = entry.lines === null ? (entry.binary ? 'binary' : 'unread') : `${entry.lines} line(s)`;
|
|
413
|
+
const head = `${name}: file, ${entry.bytes} byte(s), ${lines}${withheldSuffix(entry)}`;
|
|
414
|
+
if (entry.contents === undefined || entry.contents === null) return head;
|
|
415
|
+
return [head, ...entry.contents.split('\n').map((l) => ` ${escapeForDisplay(l)}`)].join('\n');
|
|
416
|
+
};
|
|
417
|
+
|
|
418
|
+
const formatResult = (result) => {
|
|
419
|
+
const lines = result.results.map(formatEntry);
|
|
420
|
+
if (result.incomplete) lines.push(` ⚠ INCOMPLETE (${result.incomplete.bound}): ${escapeForDisplay(result.incomplete.detail)}`);
|
|
421
|
+
return lines.join('\n');
|
|
422
|
+
};
|
|
423
|
+
|
|
424
|
+
const HELP = `path-inventory — read-only facts about named paths, without composing a shell.
|
|
425
|
+
|
|
426
|
+
Usage:
|
|
427
|
+
node path-inventory.mjs --path <p> [--path <p>]... [--contents] [--json]
|
|
428
|
+
node path-inventory.mjs --paths-file <path> [--contents] [--json]
|
|
429
|
+
|
|
430
|
+
Answers, for each named target: does it exist, what type is it, how many bytes, how many lines
|
|
431
|
+
(wc -l compatible), what a directory holds (one level, sorted), and with --contents what a small
|
|
432
|
+
text file says. A MISSING path is a normal result, not an error.
|
|
433
|
+
|
|
434
|
+
--paths-file is the lane for targets carrying shell-significant bytes (\`>\`, \`$(\`, a backtick):
|
|
435
|
+
one target per line, their bytes never enter the command string. Write it with your host's
|
|
436
|
+
file-write tool; this tool never writes.
|
|
437
|
+
|
|
438
|
+
Symlinks are reported by type and never followed. Binary and special files are reported by type and
|
|
439
|
+
never decoded.
|
|
440
|
+
|
|
441
|
+
Bounds — a bound that fires is NAMED on the run AND on the entry it withheld, never a silent
|
|
442
|
+
truncation:
|
|
443
|
+
--max-content-bytes <n> per file, for the line count and --contents
|
|
444
|
+
--max-entries <n> per directory listing
|
|
445
|
+
--max-total-bytes <n> the whole run, across every target
|
|
446
|
+
--max-total-entries <n> the whole run, across every listing
|
|
447
|
+
|
|
448
|
+
A target must NAME EXACTLY ONE filesystem object: no empty value, no NUL byte, and no ".."
|
|
449
|
+
component (resolve() collapses it before the filesystem sees it). A trailing "/" or "/." is NOT
|
|
450
|
+
rejected — it ASSERTS the target is a directory, exactly as it does to the OS: it holds for a real
|
|
451
|
+
directory, and anything else answers exists:false. Awkward-but-unambiguous names — edge whitespace,
|
|
452
|
+
backticks, control bytes — are supported, and --paths-file is the lane for the ones a command string
|
|
453
|
+
cannot carry.
|
|
454
|
+
|
|
455
|
+
Exit codes: 0 answered · 1 I/O failure or a containment refusal · 2 usage / invalid input ·
|
|
456
|
+
3 answered but INCOMPLETE (a bound fired; the bound is named). A path that does not exist is a
|
|
457
|
+
RESULT, not a failure — an unreadable one that DOES exist is an I/O failure.`;
|
|
458
|
+
|
|
459
|
+
const readLaneFile = (root, rel, maxBytes) => {
|
|
460
|
+
const abs = resolveContained(root, rel);
|
|
461
|
+
const read = readBounded(abs, maxBytes);
|
|
462
|
+
if (read.kind !== READ_OK) {
|
|
463
|
+
throw new IoError(`cannot read --paths-file ${rel} as a regular file within ${maxBytes} byte(s) (${read.kind})`);
|
|
464
|
+
}
|
|
465
|
+
return decodeLaneFile(read.buf, '--paths-file');
|
|
466
|
+
};
|
|
467
|
+
|
|
468
|
+
export const main = (argv, ctx = {}) => {
|
|
469
|
+
try {
|
|
470
|
+
if (argv.includes('--help') || argv.includes('-h')) return { code: EXIT_OK, stdout: HELP, stderr: '', result: null };
|
|
471
|
+
const root = realpathSync(resolve(ctx.cwd ?? process.cwd()));
|
|
472
|
+
const opts = parseArgs(argv);
|
|
473
|
+
const named = [...opts.paths];
|
|
474
|
+
if (opts.pathsFile !== null) {
|
|
475
|
+
named.push(...parsePathsFile(readLaneFile(root, opts.pathsFile, HARD_MAX_PATHS_FILE_BYTES)));
|
|
476
|
+
}
|
|
477
|
+
// Dedupe across the UNION, not only within each lane. Here it is more than tidiness: a duplicate
|
|
478
|
+
// target would be READ twice and charged twice against the run's aggregate byte budget.
|
|
479
|
+
const paths = [...new Set(named)];
|
|
480
|
+
// EVERY target is validated BEFORE any of them is inspected. Validating lazily means an invalid
|
|
481
|
+
// target late in the list is refused only if the run gets that far — so whether the invocation is
|
|
482
|
+
// accepted would depend on how much work happened first. A refusal must not depend on scheduling.
|
|
483
|
+
for (const target of paths) assertNameableTarget(target);
|
|
484
|
+
if (paths.length > HARD_MAX_TARGETS) {
|
|
485
|
+
throw new UsageError(`more than the ceiling of ${HARD_MAX_TARGETS} targets`);
|
|
486
|
+
}
|
|
487
|
+
const result = inventory({
|
|
488
|
+
root,
|
|
489
|
+
paths,
|
|
490
|
+
contents: opts.contents,
|
|
491
|
+
maxContentBytes: opts.maxContentBytes,
|
|
492
|
+
maxEntries: opts.maxEntries,
|
|
493
|
+
maxTotalBytes: opts.maxTotalBytes,
|
|
494
|
+
maxTotalEntries: opts.maxTotalEntries,
|
|
495
|
+
});
|
|
496
|
+
const stdout = opts.json ? JSON.stringify(result, null, 2) : formatResult(result);
|
|
497
|
+
return { code: result.incomplete ? EXIT_INCOMPLETE : EXIT_OK, stdout, stderr: '', result };
|
|
498
|
+
} catch (err) {
|
|
499
|
+
// Errors carry the caller's own path, so they are escaped exactly like the success output. A
|
|
500
|
+
// refusal that hands a hostile filename's control bytes straight to the terminal would make the
|
|
501
|
+
// safe path the only safe one, which is the wrong half to protect.
|
|
502
|
+
const say = (message) => `path-inventory: ${escapeForDisplay(String(message))}`;
|
|
503
|
+
if (err instanceof UsageError) return { code: EXIT_USAGE, stdout: '', stderr: say(err.message), result: null };
|
|
504
|
+
if (err instanceof IoError) return { code: EXIT_ERROR, stdout: '', stderr: say(err.message), result: null };
|
|
505
|
+
return { code: EXIT_ERROR, stdout: '', stderr: say(err?.message ?? err), result: null };
|
|
506
|
+
}
|
|
507
|
+
};
|
|
508
|
+
|
|
509
|
+
const emitResult = (r) => {
|
|
510
|
+
if (r.stdout) process.stdout.write(r.stdout.endsWith('\n') ? r.stdout : `${r.stdout}\n`);
|
|
511
|
+
if (r.stderr) process.stderr.write(r.stderr.endsWith('\n') ? r.stderr : `${r.stderr}\n`);
|
|
512
|
+
process.exitCode = r.code;
|
|
513
|
+
};
|
|
514
|
+
|
|
515
|
+
const isDirectRun = process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href;
|
|
516
|
+
if (isDirectRun) emitResult(main(process.argv.slice(2)));
|
package/tools/repo-search.mjs
CHANGED
|
@@ -9,10 +9,16 @@
|
|
|
9
9
|
//
|
|
10
10
|
// lane 1 --pattern <literal> for a pattern with no shell-significant byte
|
|
11
11
|
// lane 2 --pattern-file <path> the pattern's bytes NEVER enter the command string
|
|
12
|
+
// lane 3 --paths-file <path> the TARGETS' bytes never enter it either
|
|
12
13
|
//
|
|
13
|
-
//
|
|
14
|
-
//
|
|
15
|
-
//
|
|
14
|
+
// Lane 3 exists because lanes 1-2 answered only half the arguments: a search could name a pattern it
|
|
15
|
+
// could not spell in a shell, but not a PATH it could not spell, and this kit ships shell-byte
|
|
16
|
+
// fixtures as a genre. Both lane files are excluded from the search itself, by REAL path.
|
|
17
|
+
//
|
|
18
|
+
// The hook covers this tool's invocation: choosing lane 1 for a byte-carrying pattern raises an ASK
|
|
19
|
+
// whose reason names lane 2. Stated exactly: that ask goes to the HUMAN, and the reason is context for
|
|
20
|
+
// their decision — it is not delivered to the caller that composed the command. So the ask costs one
|
|
21
|
+
// human decision and never costs silence, but it is not by itself a correction mechanism.
|
|
16
22
|
//
|
|
17
23
|
// CONTRACT
|
|
18
24
|
// LITERAL only — no regex dialect, and none is planned for this slice: a bounded walk cannot
|
|
@@ -58,6 +64,15 @@ export const DEFAULT_WALK_BUDGET = 20000;
|
|
|
58
64
|
// bounds are advisory, which is the same as absent.
|
|
59
65
|
export const HARD_MAX_RESULTS = 100000;
|
|
60
66
|
export const HARD_MAX_FILE_BYTES = 64 * 1024 * 1024;
|
|
67
|
+
// The `--paths-file` lane's own bounds. A target list arrives as a FILE, so neither its size nor its
|
|
68
|
+
// entry count is visible in the command string the caller composed — without ceilings here the lane
|
|
69
|
+
// would be the one unbounded input on a tool whose every other input is bounded.
|
|
70
|
+
export const HARD_MAX_TARGETS = 5000;
|
|
71
|
+
export const HARD_MAX_PATHS_FILE_BYTES = 4 * 1024 * 1024;
|
|
72
|
+
// The AGGREGATE read budget. The walk budget counts ENTRIES; without a byte ceiling a run may read up
|
|
73
|
+
// to the per-file limit for every one of them, which is unbounded work no bound was consulted about.
|
|
74
|
+
export const DEFAULT_MAX_TOTAL_BYTES = 256 * 1024 * 1024;
|
|
75
|
+
export const HARD_MAX_TOTAL_BYTES = 4 * 1024 * 1024 * 1024;
|
|
61
76
|
const BINARY_SNIFF_BYTES = 8192;
|
|
62
77
|
// Characters of context kept on EACH side of a match, and a HARD ceiling on the whole snippet.
|
|
63
78
|
// The ceiling is the load-bearing one: bounding only the context still lets a huge --pattern-file
|
|
@@ -73,8 +88,12 @@ const NOFOLLOW = constants.O_NOFOLLOW ?? 0;
|
|
|
73
88
|
const NONBLOCK = constants.O_NONBLOCK ?? 0;
|
|
74
89
|
const OPEN_FLAGS = constants.O_RDONLY | NOFOLLOW | NONBLOCK;
|
|
75
90
|
|
|
76
|
-
|
|
77
|
-
class
|
|
91
|
+
// Exported so a sibling kit tool reusing `parsePathsFile` maps the SAME failure to the SAME exit
|
|
92
|
+
// code by construction — an `instanceof` against a private class would silently degrade a usage
|
|
93
|
+
// error into a generic one, and exit-code parity between the two file lanes would be a promise
|
|
94
|
+
// instead of a mechanism.
|
|
95
|
+
export class UsageError extends Error {}
|
|
96
|
+
export class IoError extends Error {}
|
|
78
97
|
|
|
79
98
|
// The pattern is echoed as a DIGEST plus a byte length — never as a first content line. A first
|
|
80
99
|
// line cannot separate two multiline patterns that share it, and it is unsafe for NUL/control
|
|
@@ -104,7 +123,7 @@ export const parseCount = (raw, flag, ceiling) => {
|
|
|
104
123
|
};
|
|
105
124
|
|
|
106
125
|
const parseArgs = (argv) => {
|
|
107
|
-
const opts = { pattern: null, patternFile: null, paths: [], max: DEFAULT_MAX_RESULTS, maxBytes: DEFAULT_MAX_FILE_BYTES, json: false };
|
|
126
|
+
const opts = { pattern: null, patternFile: null, paths: [], pathsFile: null, max: DEFAULT_MAX_RESULTS, maxBytes: DEFAULT_MAX_FILE_BYTES, maxTotalBytes: DEFAULT_MAX_TOTAL_BYTES, json: false };
|
|
108
127
|
for (let i = 0; i < argv.length; i += 1) {
|
|
109
128
|
const arg = argv[i];
|
|
110
129
|
const next = () => {
|
|
@@ -115,8 +134,10 @@ const parseArgs = (argv) => {
|
|
|
115
134
|
if (arg === '--pattern') opts.pattern = next();
|
|
116
135
|
else if (arg === '--pattern-file') opts.patternFile = next();
|
|
117
136
|
else if (arg === '--path') opts.paths.push(next());
|
|
137
|
+
else if (arg === '--paths-file') opts.pathsFile = next();
|
|
118
138
|
else if (arg === '--max') opts.max = parseCount(next(), '--max', HARD_MAX_RESULTS);
|
|
119
139
|
else if (arg === '--max-bytes') opts.maxBytes = parseCount(next(), '--max-bytes', HARD_MAX_FILE_BYTES);
|
|
140
|
+
else if (arg === '--max-total-bytes') opts.maxTotalBytes = parseCount(next(), '--max-total-bytes', HARD_MAX_TOTAL_BYTES);
|
|
120
141
|
else if (arg === '--json') opts.json = true;
|
|
121
142
|
else throw new UsageError(`unknown argument: ${arg} (see --help)`);
|
|
122
143
|
}
|
|
@@ -124,14 +145,107 @@ const parseArgs = (argv) => {
|
|
|
124
145
|
throw new UsageError('--pattern and --pattern-file are mutually exclusive — the lane must be unambiguous');
|
|
125
146
|
}
|
|
126
147
|
if (opts.pattern === null && opts.patternFile === null) throw new UsageError('one of --pattern or --pattern-file is required');
|
|
127
|
-
|
|
148
|
+
// The default target is applied only when the caller named NEITHER lane: `--paths-file` supplies
|
|
149
|
+
// targets after argv is parsed, so defaulting here on an empty `--path` list would silently union
|
|
150
|
+
// the whole root into an explicitly named list.
|
|
151
|
+
if (opts.paths.length === 0 && opts.pathsFile === null) opts.paths.push('.');
|
|
128
152
|
return opts;
|
|
129
153
|
};
|
|
130
154
|
|
|
155
|
+
// The `--paths-file` format, pinned rather than discovered: one target per line, UTF-8, LF or CRLF, a
|
|
156
|
+
// trailing delimiter is not an extra entry, EMPTY lines are ignored, duplicates collapse. There is no
|
|
157
|
+
// comment syntax and no escaping, so a filename containing a newline CANNOT be expressed by this lane
|
|
158
|
+
// — stated here because an unstated gap in a lane that exists to carry awkward names is the defect,
|
|
159
|
+
// not the gap.
|
|
160
|
+
//
|
|
161
|
+
// A line is NOT trimmed. This lane exists to carry names a command string cannot, and a leading or
|
|
162
|
+
// trailing space is exactly such a name: trimming would silently rewrite the caller's target, which is
|
|
163
|
+
// the failure mode the lane was built to remove. A whitespace-only line is therefore a real target and
|
|
164
|
+
// either resolves or fails loudly — never a silent stand-in for the root.
|
|
165
|
+
//
|
|
166
|
+
// THREE name classes this lane CANNOT express, stated rather than discovered: one containing a
|
|
167
|
+
// newline (there is no escaping); one ENDING in a carriage return (a trailing CR is stripped as the
|
|
168
|
+
// CRLF delimiter it almost always is, and no line-oriented format can tell those apart without an
|
|
169
|
+
// encoding this lane deliberately does not have); and one whose bytes are not valid UTF-8. The first
|
|
170
|
+
// two are refused by silence — the target simply will not be found, loudly. The third is refused
|
|
171
|
+
// EXPLICITLY, because `Buffer.toString('utf8')` would replace the offending bytes with U+FFFD and the
|
|
172
|
+
// lane would then search a DIFFERENT path that happens to exist, which is worse than any refusal.
|
|
173
|
+
// `ignoreBOM: true` means "do not TREAT a leading U+FEFF as a byte-order mark", i.e. keep it as a
|
|
174
|
+
// character. The default strips it, which would rewrite a legitimate name beginning with U+FEFF into
|
|
175
|
+
// a different one — the same silent-substitution class as a lossy decode, one layer down.
|
|
176
|
+
const strictUtf8 = new TextDecoder('utf-8', { fatal: true, ignoreBOM: true });
|
|
177
|
+
|
|
178
|
+
export const decodeLaneFile = (buf, flag) => {
|
|
179
|
+
try {
|
|
180
|
+
return strictUtf8.decode(buf);
|
|
181
|
+
} catch {
|
|
182
|
+
throw new UsageError(`${flag} is not valid UTF-8 — a lossy decode would silently name a different path`);
|
|
183
|
+
}
|
|
184
|
+
};
|
|
185
|
+
export const parsePathsFile = (raw, flag = '--paths-file') => {
|
|
186
|
+
const entries = raw.split('\n').map((line) => (line.endsWith('\r') ? line.slice(0, -1) : line)).filter((line) => line !== '');
|
|
187
|
+
// The ceiling counts TARGETS, so it is applied after dedupe: a file repeating one path 5001 times
|
|
188
|
+
// names one target and refusing it would be a bound on the file's shape rather than on the work.
|
|
189
|
+
const unique = [...new Set(entries)];
|
|
190
|
+
if (unique.length > HARD_MAX_TARGETS) {
|
|
191
|
+
throw new UsageError(`${flag} lists ${unique.length} distinct targets, above the ceiling of ${HARD_MAX_TARGETS}`);
|
|
192
|
+
}
|
|
193
|
+
if (unique.length === 0) throw new UsageError(`${flag} names no target — a blank list is never "search everything"`);
|
|
194
|
+
return unique;
|
|
195
|
+
};
|
|
196
|
+
|
|
197
|
+
// ── the acceptance predicate for a caller-supplied TARGET ─────────────────────────────
|
|
198
|
+
//
|
|
199
|
+
// ONE closed rule set, applied before any target reaches `resolve()`. It replaces what were seven
|
|
200
|
+
// checks discovered one review round at a time, all of which turned out to be the same defect: a
|
|
201
|
+
// caller string is handed to `path.resolve()` / `realpathSync`, whose semantics differ from the
|
|
202
|
+
// operating system's, so the tool can end up answering about a DIFFERENT object than the string
|
|
203
|
+
// denotes. That is the one outcome worse than any refusal.
|
|
204
|
+
//
|
|
205
|
+
// The rule the predicate enforces: **a target is accepted only if it names exactly one filesystem
|
|
206
|
+
// object unambiguously.** Awkward-but-unambiguous names are supported on purpose — edge whitespace,
|
|
207
|
+
// backticks, control bytes — because carrying those is what the out-of-band lane exists for.
|
|
208
|
+
//
|
|
209
|
+
// Separators are PLATFORM-CORRECT, and that is load-bearing: on POSIX a backslash is an ordinary
|
|
210
|
+
// byte in a filename, so splitting on it would refuse `\..` — a legal file the OS answers about
|
|
211
|
+
// normally. An over-refusal is a smaller defect than a wrong answer, but it is still a defect.
|
|
212
|
+
const TARGET_SEPARATORS = Object.freeze(sep === '\\' ? ['/', '\\'] : ['/']);
|
|
213
|
+
const TARGET_SEPARATOR_SPLIT = sep === '\\' ? /[\\/]/u : /\//u;
|
|
214
|
+
|
|
215
|
+
// A trailing separator, or a trailing `.` component, is an ASSERTION by the caller that the target is
|
|
216
|
+
// a DIRECTORY — that is what it means to the operating system, which answers ENOTDIR when it is not.
|
|
217
|
+
// `resolve()` erases both forms before the filesystem sees them, so the assertion has to be carried
|
|
218
|
+
// separately and checked after resolution. Refusing these outright was wrong in both directions: it
|
|
219
|
+
// rejected `existing-directory/`, which names one object unambiguously, while still accepting
|
|
220
|
+
// `regular-file/.`, which the OS refuses.
|
|
221
|
+
//
|
|
222
|
+
// Stated divergence, and it differs per tool. `path-inventory` reports a symlink BY TYPE and never
|
|
223
|
+
// follows it, so `symlink-to-a-directory/` is not a directory there, while the OS would dereference
|
|
224
|
+
// it — following it would contradict that tool's louder promise. `repo-search` DOES resolve an
|
|
225
|
+
// explicitly named target (its threat model says so at the top of this file), so the assertion is
|
|
226
|
+
// checked against the resolved object and it agrees with the OS.
|
|
227
|
+
export const requiresDirectory = (target) => {
|
|
228
|
+
if (TARGET_SEPARATORS.some((separator) => target.endsWith(separator))) return true;
|
|
229
|
+
const components = target.split(TARGET_SEPARATOR_SPLIT);
|
|
230
|
+
return components[components.length - 1] === '.';
|
|
231
|
+
};
|
|
232
|
+
|
|
233
|
+
export const assertNameableTarget = (target, flag = '--path') => {
|
|
234
|
+
const refuse = (why) => {
|
|
235
|
+
throw new UsageError(`${flag} ${why} — got ${JSON.stringify(target)}`);
|
|
236
|
+
};
|
|
237
|
+
if (target === '') refuse('must not be empty; an empty target resolves to the whole root');
|
|
238
|
+
if (target.includes('\0')) refuse('must not contain a NUL byte; no filesystem path can hold one');
|
|
239
|
+
if (target.split(TARGET_SEPARATOR_SPLIT).includes('..')) {
|
|
240
|
+
refuse('must not contain a ".." component; resolve() collapses it lexically, so the answer could be about a different object than the OS would reach');
|
|
241
|
+
}
|
|
242
|
+
};
|
|
243
|
+
|
|
131
244
|
// Containment on the REAL path. A lexical check passes `link/secret.txt` whenever `link` resolves
|
|
132
245
|
// outside, and on Windows a cross-drive `relative()` returns an absolute path carrying no `..` —
|
|
133
246
|
// both were live review findings, not hypotheticals.
|
|
134
247
|
export const resolveTarget = (realRoot, target) => {
|
|
248
|
+
assertNameableTarget(target);
|
|
135
249
|
const lexical = resolve(realRoot, target);
|
|
136
250
|
let real;
|
|
137
251
|
try {
|
|
@@ -144,12 +258,15 @@ export const resolveTarget = (realRoot, target) => {
|
|
|
144
258
|
if (rel !== '' && (isAbsolute(rel) || rel === '..' || rel.startsWith(`..${sep}`))) {
|
|
145
259
|
throw new IoError(`target resolves outside the search root: ${target}`);
|
|
146
260
|
}
|
|
261
|
+
if (requiresDirectory(target) && !lstatSync(real).isDirectory()) {
|
|
262
|
+
throw new IoError(`no such path: ${target} — a trailing separator or "." asserts a directory, and this is not one`);
|
|
263
|
+
}
|
|
147
264
|
return real;
|
|
148
265
|
};
|
|
149
266
|
|
|
150
267
|
// Open → fstat the DESCRIPTOR → read bounded. The descriptor is what was actually opened, so a swap
|
|
151
268
|
// after the check cannot substitute a different node; O_NOFOLLOW refuses a symlinked leaf outright.
|
|
152
|
-
const readRegularFile = (abs, maxBytes, state, io = {}) => {
|
|
269
|
+
const readRegularFile = (abs, maxBytes, state, io = {}, boundName = 'max-file-bytes') => {
|
|
153
270
|
const open = io.open ?? openSync;
|
|
154
271
|
const fstat = io.fstat ?? fstatSync;
|
|
155
272
|
const read = io.read ?? readSync;
|
|
@@ -174,7 +291,7 @@ const readRegularFile = (abs, maxBytes, state, io = {}) => {
|
|
|
174
291
|
// A skipped file is NOT a silent omission: the search is incomplete and says which bound did
|
|
175
292
|
// it. Reporting it only as a counter would let a partial search read as "no matches".
|
|
176
293
|
if (state.incomplete === null) {
|
|
177
|
-
state.incomplete = { bound:
|
|
294
|
+
state.incomplete = { bound: boundName, detail: `at least one file exceeds ${maxBytes} byte(s) and was not searched` };
|
|
178
295
|
}
|
|
179
296
|
return null;
|
|
180
297
|
}
|
|
@@ -184,6 +301,10 @@ const readRegularFile = (abs, maxBytes, state, io = {}) => {
|
|
|
184
301
|
const n = read(fd, buf, got, stat.size - got, got);
|
|
185
302
|
if (n <= 0) break;
|
|
186
303
|
got += n;
|
|
304
|
+
// Charged INSIDE the loop, per successful chunk. After the loop is too late: a `read` that
|
|
305
|
+
// throws mid-file skips the charge entirely, so a series of partial reads that end in a fault
|
|
306
|
+
// would move real bytes the aggregate budget never learns about.
|
|
307
|
+
state.bytesRead += n;
|
|
187
308
|
}
|
|
188
309
|
// A short read means the file changed under us. Returning the partial buffer would let a
|
|
189
310
|
// truncated file come back as a confident "no matches" — the file is classified unreadable
|
|
@@ -293,11 +414,29 @@ const walk = (root, abs, pattern, state, isExplicitTarget = false) => {
|
|
|
293
414
|
}
|
|
294
415
|
return;
|
|
295
416
|
}
|
|
296
|
-
|
|
297
|
-
|
|
417
|
+
const excludedAs = state.excludePaths.get(abs);
|
|
418
|
+
if (excludedAs !== undefined) {
|
|
419
|
+
state.skipped[excludedAs] += 1;
|
|
420
|
+
return;
|
|
421
|
+
}
|
|
422
|
+
// The walk budget counts ENTRIES, not bytes, so without this a run may read up to the per-file
|
|
423
|
+
// ceiling for every one of them — twenty thousand files at the hard per-file ceiling is work no
|
|
424
|
+
// bound was ever consulted about. The sibling inventory tool already carries an aggregate byte
|
|
425
|
+
// budget for exactly this reason; this closes the asymmetry rather than stating it.
|
|
426
|
+
const remaining = state.maxTotalBytes - state.bytesRead;
|
|
427
|
+
if (remaining <= 0) {
|
|
428
|
+
state.incomplete = state.incomplete ?? {
|
|
429
|
+
bound: '--max-total-bytes',
|
|
430
|
+
detail: `stopped after reading ${state.bytesRead} byte(s); the tree was not fully searched`,
|
|
431
|
+
};
|
|
298
432
|
return;
|
|
299
433
|
}
|
|
300
|
-
|
|
434
|
+
// The read is bounded by whichever ceiling binds FIRST. Checking the aggregate only BEFORE the file
|
|
435
|
+
// and then handing the reader the full per-file limit lets a single file larger than the remaining
|
|
436
|
+
// budget be read whole — the budget would bound the accounting, not the work.
|
|
437
|
+
const limit = Math.min(state.maxBytes, remaining);
|
|
438
|
+
const boundName = limit < state.maxBytes ? '--max-total-bytes' : 'max-file-bytes';
|
|
439
|
+
const buf = readRegularFile(abs, limit, state, state.io, boundName);
|
|
301
440
|
if (buf === null) return;
|
|
302
441
|
if (isBinary(buf)) {
|
|
303
442
|
state.skipped.binary += 1;
|
|
@@ -310,15 +449,20 @@ const walk = (root, abs, pattern, state, isExplicitTarget = false) => {
|
|
|
310
449
|
// directory, a symlink refused by O_NOFOLLOW, a file truncated mid-read — are reachable from tests.
|
|
311
450
|
// Every one of them is a COUNTED skip in production, and a counted skip that no test ever exercises
|
|
312
451
|
// is indistinguishable from a silent one.
|
|
313
|
-
|
|
452
|
+
// `excludePaths` maps a REAL path to the skip counter it belongs to: BOTH lane files must be kept out
|
|
453
|
+
// of their own search (a `--paths-file` under a searched target would otherwise match on its own
|
|
454
|
+
// contents), and a skip that is not attributed to its lane is indistinguishable from a silent one.
|
|
455
|
+
export const search = ({ root, pattern, paths, max, maxBytes, excludePaths = new Map(), walkBudget = DEFAULT_WALK_BUDGET, maxTotalBytes = DEFAULT_MAX_TOTAL_BYTES, io = {} }) => {
|
|
314
456
|
const state = {
|
|
315
457
|
matches: [],
|
|
316
458
|
incomplete: null,
|
|
317
|
-
skipped: { symlinks: 0, binary: 0, special: 0, unreadable: 0, large: 0, patternFile: 0 },
|
|
318
|
-
|
|
459
|
+
skipped: { symlinks: 0, binary: 0, special: 0, unreadable: 0, large: 0, patternFile: 0, pathsFile: 0 },
|
|
460
|
+
excludePaths,
|
|
319
461
|
io,
|
|
320
462
|
walked: 0,
|
|
321
463
|
walkBudget,
|
|
464
|
+
bytesRead: 0,
|
|
465
|
+
maxTotalBytes,
|
|
322
466
|
max,
|
|
323
467
|
maxBytes,
|
|
324
468
|
};
|
|
@@ -334,6 +478,9 @@ export const search = ({ root, pattern, paths, max, maxBytes, excludePath = null
|
|
|
334
478
|
incomplete: state.incomplete,
|
|
335
479
|
skipped: state.skipped,
|
|
336
480
|
scanned: state.walked,
|
|
481
|
+
// Reported so the aggregate budget is observable rather than internal: a bound that nobody can
|
|
482
|
+
// read is a bound nobody can test.
|
|
483
|
+
bytesRead: state.bytesRead,
|
|
337
484
|
};
|
|
338
485
|
};
|
|
339
486
|
|
|
@@ -351,14 +498,27 @@ const formatResult = (result) => {
|
|
|
351
498
|
|
|
352
499
|
const HELP = `repo-search — literal repository search that never has to ride a shell metacharacter.
|
|
353
500
|
|
|
354
|
-
Usage:
|
|
355
|
-
|
|
356
|
-
|
|
501
|
+
Usage — pick ONE pattern lane and ANY combination of target lanes:
|
|
502
|
+
pattern lane --pattern <literal> | --pattern-file <path> (mutually exclusive)
|
|
503
|
+
target lanes [--path <p>]... and/or --paths-file <path> (union; default: .)
|
|
504
|
+
shared flags [--max <n>] [--max-bytes <n>] [--max-total-bytes <n>] [--json]
|
|
505
|
+
|
|
506
|
+
A target must NAME EXACTLY ONE filesystem object: no empty value, no NUL byte, and no ".."
|
|
507
|
+
component (resolve() collapses it before the filesystem sees it). A trailing "/" or "/." is NOT
|
|
508
|
+
rejected — it ASSERTS the target is a directory, exactly as it does to the OS: it holds for a real
|
|
509
|
+
directory and fails for anything else. Awkward-but-unambiguous names — edge whitespace, backticks,
|
|
510
|
+
control bytes — are supported, and --paths-file is the lane for the ones a command string cannot
|
|
511
|
+
carry.
|
|
512
|
+
|
|
513
|
+
Two out-of-band lanes, one per argument half — their bytes never enter the command string, so the
|
|
514
|
+
residual guard has nothing to scan:
|
|
515
|
+
--pattern-file <path> a PATTERN carrying shell-significant bytes (\`>\`, \`$(\`, a backtick)
|
|
516
|
+
--paths-file <path> TARGET paths carrying the same, one per line
|
|
357
517
|
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
this
|
|
518
|
+
Write either file with your host's file-write tool, pass the plain path here, and delete it when you
|
|
519
|
+
are done — this tool never writes. --paths-file format: one target per line, UTF-8, LF or CRLF; blank
|
|
520
|
+
lines ignored; duplicates collapse; no comment syntax and no escaping, so a filename containing a
|
|
521
|
+
newline cannot be expressed by this lane. Both lane files are excluded from the search itself.
|
|
362
522
|
|
|
363
523
|
LITERAL only, multiline patterns supported. Reads regular files only, opened no-follow. Skipped
|
|
364
524
|
entries (symlinks, non-regular, binary, oversized, unreadable) are counted and reported, never
|
|
@@ -372,21 +532,49 @@ export const main = (argv, ctx = {}) => {
|
|
|
372
532
|
if (argv.includes('--help') || argv.includes('-h')) return { code: EXIT_OK, stdout: HELP, stderr: '', result: null };
|
|
373
533
|
const root = realpathSync(resolve(ctx.cwd ?? process.cwd()));
|
|
374
534
|
const opts = parseArgs(argv);
|
|
535
|
+
const excludePaths = new Map();
|
|
536
|
+
// ONE reader for both out-of-band lanes, so their failure classification cannot drift: a missing,
|
|
537
|
+
// unreadable or non-regular lane file is an IoError (exit 1) for the pattern lane and for the
|
|
538
|
+
// target lane alike.
|
|
539
|
+
const readLaneFile = (rel, flag, maxBytes, counter) => {
|
|
540
|
+
const abs = resolveTarget(root, rel);
|
|
541
|
+
// `bytesRead` is present but SEPARATE from the search's budget: reading a lane file is the
|
|
542
|
+
// caller's own instruction, not tree traversal, so it is accounted and then discarded.
|
|
543
|
+
const state = { skipped: { symlinks: 0, special: 0, unreadable: 0, large: 0 }, incomplete: null, bytesRead: 0 };
|
|
544
|
+
const buf = readRegularFile(abs, maxBytes, state);
|
|
545
|
+
if (buf === null) throw new IoError(`cannot read ${flag} ${rel} as a regular file`);
|
|
546
|
+
excludePaths.set(abs, counter);
|
|
547
|
+
return decodeLaneFile(buf, flag);
|
|
548
|
+
};
|
|
375
549
|
let raw;
|
|
376
|
-
let excludePath = null;
|
|
377
550
|
if (opts.patternFile !== null) {
|
|
378
|
-
|
|
379
|
-
const state = { skipped: { symlinks: 0, special: 0, unreadable: 0, large: 0 }, incomplete: null };
|
|
380
|
-
const buf = readRegularFile(excludePath, HARD_MAX_FILE_BYTES, state);
|
|
381
|
-
if (buf === null) throw new IoError(`cannot read --pattern-file ${opts.patternFile} as a regular file`);
|
|
382
|
-
raw = buf.toString('utf8');
|
|
551
|
+
raw = readLaneFile(opts.patternFile, '--pattern-file', HARD_MAX_FILE_BYTES, 'patternFile');
|
|
383
552
|
} else {
|
|
384
553
|
raw = opts.pattern;
|
|
385
554
|
}
|
|
386
555
|
const pattern = opts.patternFile !== null ? resolvePattern(raw) : raw;
|
|
387
556
|
if (pattern === '') throw new UsageError('the pattern is empty — it would match every line of every file');
|
|
388
557
|
|
|
389
|
-
const
|
|
558
|
+
const named = [...opts.paths];
|
|
559
|
+
if (opts.pathsFile !== null) {
|
|
560
|
+
const listed = readLaneFile(opts.pathsFile, '--paths-file', HARD_MAX_PATHS_FILE_BYTES, 'pathsFile');
|
|
561
|
+
named.push(...parsePathsFile(listed));
|
|
562
|
+
}
|
|
563
|
+
// Dedupe across the UNION, not only within each lane: the same target named by `--path` and by
|
|
564
|
+
// the file would otherwise be walked twice.
|
|
565
|
+
const paths = [...new Set(named)];
|
|
566
|
+
// EVERY target is validated BEFORE any of them is walked. Validating lazily per target means an
|
|
567
|
+
// invalid one late in the list is only refused if the walk gets that far — and a bound that fires
|
|
568
|
+
// on an earlier target ends the loop first, so the same invocation would be accepted or refused
|
|
569
|
+
// depending on how much work happened to be done. A refusal must not depend on scheduling.
|
|
570
|
+
for (const target of paths) assertNameableTarget(target);
|
|
571
|
+
// The ceiling holds over the UNION. `parsePathsFile` bounds the file, but thousands of `--path`
|
|
572
|
+
// values would otherwise walk straight past it.
|
|
573
|
+
if (paths.length > HARD_MAX_TARGETS) {
|
|
574
|
+
throw new UsageError(`more than the ceiling of ${HARD_MAX_TARGETS} targets`);
|
|
575
|
+
}
|
|
576
|
+
|
|
577
|
+
const result = search({ root, pattern, paths, max: opts.max, maxBytes: opts.maxBytes, maxTotalBytes: opts.maxTotalBytes, excludePaths });
|
|
390
578
|
const stdout = opts.json ? JSON.stringify(result, null, 2) : formatResult(result);
|
|
391
579
|
return { code: result.incomplete ? EXIT_INCOMPLETE : EXIT_OK, stdout, stderr: '', result };
|
|
392
580
|
} catch (err) {
|
|
@@ -149,6 +149,10 @@ export const KIT_READONLY_TOOLS = Object.freeze([
|
|
|
149
149
|
// NO decision from the hook, which is not the same as an allow, so without this rule the lane
|
|
150
150
|
// falls through to whatever the host policy happens to be.
|
|
151
151
|
'tools/repo-search.mjs',
|
|
152
|
+
// The inventory lane, in the tier for the same reason and with the same dependency: the corpus of
|
|
153
|
+
// useless approvals is mostly small path questions batched into a composed shell because no single
|
|
154
|
+
// call answered them, and a lane the agent must still ask about is not a lane.
|
|
155
|
+
'tools/path-inventory.mjs',
|
|
152
156
|
]);
|
|
153
157
|
// Writer previews: ONLY writers whose ARG-FREE invocation is a documented dry-run ("Default is
|
|
154
158
|
// --dry-run" in their usage) seed an EXACT preview byte-string — every --apply/--write/--yes keeps
|
|
@@ -342,7 +346,7 @@ const USAGE = `usage: velocity-profile [--dry-run | --apply] [--kit-tools] [--br
|
|
|
342
346
|
|
|
343
347
|
Allowlist mode (default): seeds the fixed read-only Claude Code allowlist into .claude/settings.json.
|
|
344
348
|
Default is --dry-run. --apply writes; --accept-edits only sets defaultMode when applying.
|
|
345
|
-
--kit-tools additionally seeds the audited kit-tool tier:
|
|
349
|
+
--kit-tools additionally seeds the audited kit-tool tier: ${KIT_WILDCARD_TOOLS.length} read-only kit tools by resolved
|
|
346
350
|
absolute path (args wildcard), run-gates.mjs as ONE exact project-root-pinned byte-string
|
|
347
351
|
(project-exec - it runs YOUR declared gates.json), and the writers' exact arg-free dry-run
|
|
348
352
|
preview byte-strings. Never touches settings.local.json.
|
|
@@ -644,7 +648,7 @@ const formatAllowlist = (result) => [
|
|
|
644
648
|
// The tier's honest posture, printed on every --kit-tools run: run-gates is project-exec (never
|
|
645
649
|
// "read-only"), previews stay dry-run-only, and the tier gets none of the hook's residual ask-net.
|
|
646
650
|
const KIT_TIER_NOTICE =
|
|
647
|
-
'kit-tools tier: paths are resolved absolute at seed time (fail-safe - a moved skill or stale path simply prompts again); run-gates.mjs is seeded as ONE exact byte-string pinned to this project root and is project-exec - it runs YOUR declared gates.json commands, never "read-only"; writer previews are exact dry-run byte-strings - every --apply/--write/--yes still prompts; tier entries get NO PreToolUse-hook residual coverage (settings-level posture only - see the velocity mode notes).';
|
|
651
|
+
'kit-tools tier: paths are resolved absolute at seed time (fail-safe - a moved skill or stale path simply prompts again); run-gates.mjs is seeded as ONE exact byte-string pinned to this project root and is project-exec - it runs YOUR declared gates.json commands, never "read-only"; writer previews are exact dry-run byte-strings - every --apply/--write/--yes still prompts; tier entries get NO PreToolUse-hook residual coverage EXCEPT repo-search.mjs and path-inventory.mjs, whose invocations the hook scans because they take caller-supplied argument bytes (settings-level posture only for the rest - see the velocity mode notes).';
|
|
648
652
|
|
|
649
653
|
const formatKitTier = (result) =>
|
|
650
654
|
result.kitTools
|