@sleepy_panda_srl/portulan 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -6
- package/cli/README.md +142 -291
- package/cli/advisory.mjs +354 -0
- package/cli/comments.mjs +932 -0
- package/cli/compile.mjs +1443 -1210
- package/cli/context.mjs +883 -0
- package/cli/control-chars.mjs +12 -301
- package/cli/discover.mjs +17 -574
- package/cli/doctor.mjs +225 -1071
- package/cli/drills.mjs +1107 -0
- package/cli/feedback.mjs +16 -237
- package/cli/finish.mjs +437 -0
- package/cli/form.mjs +644 -0
- package/cli/fuzz-shell.mjs +595 -0
- package/cli/gate.mjs +7 -198
- package/cli/goldens.mjs +311 -0
- package/cli/index.mjs +305 -655
- package/cli/init.mjs +244 -564
- package/cli/inside.mjs +3 -28
- package/cli/instructions.mjs +829 -0
- package/cli/ledger.mjs +778 -0
- package/cli/librarian.mjs +135 -599
- package/cli/manifest.mjs +2 -39
- package/cli/mutants.mjs +764 -0
- package/cli/new.mjs +12 -243
- package/cli/pack-identity.mjs +6 -68
- package/cli/pack-version.mjs +13 -248
- package/cli/plugin-lint.mjs +42 -658
- package/cli/portulan.mjs +5 -124
- package/cli/recipe-set.mjs +16 -240
- package/cli/release-eval.mjs +748 -0
- package/cli/review-meter.mjs +500 -0
- package/cli/rule-carriers.mjs +12 -175
- package/cli/sessions.mjs +82 -0
- package/cli/skill-goldens.mjs +424 -0
- package/cli/skills-set.mjs +12 -414
- package/cli/stop-gate.mjs +126 -454
- package/cli/symbols.mjs +1132 -0
- package/cli/telemetry.mjs +582 -0
- package/cli/upgrade.mjs +171 -314
- package/cli/vendor.mjs +146 -476
- package/cli/version-carriers.mjs +9 -74
- package/core/README.md +1 -1
- package/core/engine.md +3 -9
- package/core/operating/autonomy.md +1 -1
- package/core/operating/context.md +183 -0
- package/core/operating/evolution.md +3 -3
- package/core/operating/loop.md +13 -20
- package/core/operating/memory.md +12 -3
- package/core/operating/sessions.md +100 -0
- package/core/personas/librarian.md +2 -2
- package/core/skills/consolidate/SKILL.md +4 -2
- package/core/templates/gate-policy.md +27 -8
- package/core/templates/handoff.md +28 -15
- package/core/templates/memory-entry.md +1 -0
- package/evals/README.md +320 -0
- package/evals/releases/0.1.3.json +134 -0
- package/evals/releases/0.1.3.md +64 -0
- package/evals/releases/0.2.0.json +146 -0
- package/evals/releases/0.2.0.md +67 -0
- package/evals/releases/README.md +32 -0
- package/package.json +9 -2
- package/packs/rituals/checkpoints/README.md +9 -2
- package/packs/rituals/checkpoints/pack.json +1 -1
- package/packs/rituals/checkpoints/skills/pre-commit/SKILL.md +32 -2
- package/packs/tools/github/verify/actions-pinned.sh +10 -55
- package/spec/README.md +112 -6
- package/spec/migrations/0001-repository-declares-its-tree.mjs +1 -26
- package/spec/migrations/0002-bundle-fallback-path.mjs +13 -74
- package/spec/migrations/0003-handoff-index-not-kept.mjs +66 -0
- package/spec/migrations/0004-changelog-fragments.mjs +53 -0
- package/spec/migrations/0005-session-log-retired.mjs +69 -0
- package/spec/migrations/0006-boot-card.mjs +77 -0
- package/spec/migrations/0007-guidance-compiled.mjs +31 -0
- package/spec/migrations/0008-card-reading.mjs +55 -0
- package/spec/migrations/0009-instruction-sections-on-read.mjs +60 -0
- package/spec/migrations/0010-comment-rail.mjs +57 -0
- package/spec/migrations/README.md +45 -10
- package/spec/pack.schema.json +1 -1
- package/spec/slots.md +359 -23
- package/spec/workspace.schema.json +145 -4
package/cli/control-chars.mjs
CHANGED
|
@@ -4,103 +4,6 @@
|
|
|
4
4
|
// git ls-files --cached --others --exclude-standard -z | node cli/control-chars.mjs [--exempt <path>]...
|
|
5
5
|
//
|
|
6
6
|
// Exit 0 every scanned file is clean · 1 one is not · 2 could not run.
|
|
7
|
-
//
|
|
8
|
-
// **1 is RESERVED for a finding**, which is a promise the entry block at the foot of this file keeps
|
|
9
|
-
// rather than a description of what node does by default: an uncaught error would otherwise leave node
|
|
10
|
-
// to exit 1, and the recipe reads 1 as *a scanned file carries a control character* — a judged verdict
|
|
11
|
-
// about a tree, from a run that judged nothing (#208). **Scanned, not tracked**: the listing is
|
|
12
|
-
// `--cached --others --exclude-standard`, so it is the tracked set PLUS everything new and not ignored,
|
|
13
|
-
// which is the point — a byte is caught before it is committed. This line said "tracked" when #208 wrote
|
|
14
|
-
// it, adding a fresh carrier to the narrowing #170 already had on the roster; swept here with the rest.
|
|
15
|
-
//
|
|
16
|
-
// ## Why this exists
|
|
17
|
-
//
|
|
18
|
-
// A raw NUL shipped inside `../.portulan/verify/workflow-filters.mjs` as the separator in a template
|
|
19
|
-
// literal, where the escape `\u0000` was meant. A reviewer caught it; nothing in this repository did, and
|
|
20
|
-
// the reason is the part that matters (issue #68):
|
|
21
|
-
//
|
|
22
|
-
// * `file` classified the source as *binary data*.
|
|
23
|
-
// * `grep -n "identity = "` against the line that plainly contains that text exited **1** — a silent
|
|
24
|
-
// false negative. `docs.sh` — the DEFAULT recipe, the one a Stop-gate runs when nothing more
|
|
25
|
-
// specific applies — is built out of `grep` throughout, so the tool most likely to have shown the
|
|
26
|
-
// byte is the tool the byte silences. (Named rather than counted, and that took two goes: the
|
|
27
|
-
// first draft said *every* recipe in that directory was built out of `grep`, and FIVE invoke it
|
|
28
|
-
// zero times — two of them carrying the word only in comments, which is why the number was three
|
|
29
|
-
// for three days; the correction then asserted `seventeen invocations`, a figure taken from a word
|
|
30
|
-
// count that included comments. A number nothing regenerates is a claim that rots — #77. Copilot
|
|
31
|
-
// found the first, in three carriers; I found the second auditing my own fix.)
|
|
32
|
-
// * `git` rendered the diff as text only because the byte sat past its first 8000 bytes. A few
|
|
33
|
-
// hundred lines earlier and the whole instrument would have arrived in review as `Binary files
|
|
34
|
-
// differ` — a file nobody could read, in a pull request that looked complete.
|
|
35
|
-
//
|
|
36
|
-
// It then reproduced twice more while being fixed. Three tools noticed across the whole incident and
|
|
37
|
-
// none of them was ours, which is `a-mandate-nothing-checks-is-already-broken.md` with the mandate
|
|
38
|
-
// left implicit: nobody had ever written down that a tracked file may not carry one.
|
|
39
|
-
//
|
|
40
|
-
// ## What counts, and the one thing that does not
|
|
41
|
-
//
|
|
42
|
-
// Every byte in the C0 range — `0x00`–`0x1F` — except **TAB** and **LF**, plus **DEL** (`0x7F`),
|
|
43
|
-
// plus every **C1** control character, `U+0080`–`U+009F`.
|
|
44
|
-
// DEL is not in C0 and is the same defect: invisible, and printed by `cat -v` as `^?`.
|
|
45
|
-
//
|
|
46
|
-
// **C1 was the half this could not see until #207**, and the gap was not theoretical: a planted U+009B
|
|
47
|
-
// (CSI — the byte that opens a terminal escape sequence) left the recipe green, measured at the
|
|
48
|
-
// retrospective pass over #167. C1 arrives as TWO UTF-8 bytes, `0xc2` then `0x80`–`0x9f`, so a per-byte
|
|
49
|
-
// C0 test cannot reach it however carefully it is written.
|
|
50
|
-
//
|
|
51
|
-
// **Format characters are deliberately still out** — zero-width space, the bidi marks, word joiner,
|
|
52
|
-
// BOM (`U+200B`–`U+200F`, `U+2060`–`U+206F`, `U+FEFF`). They are a different question, not a smaller
|
|
53
|
-
// one: unlike C1 they appear legitimately in prose, so a rail over them needs an exemption mechanism far
|
|
54
|
-
// busier than the named-path list below, and that is a proposal rather than an addition here.
|
|
55
|
-
//
|
|
56
|
-
// **CR is refused**, which is a decision rather than an oversight. No tracked file in this repository
|
|
57
|
-
// carries one, so the rail costs nothing today and stops CRLF arriving unnoticed in a tree that is
|
|
58
|
-
// uniformly LF. A repository that wants CRLF wants it declared, not discovered.
|
|
59
|
-
//
|
|
60
|
-
// ## Bytes, never text
|
|
61
|
-
//
|
|
62
|
-
// The scan reads a `Buffer` and never decodes it. Decoding invalid UTF-8 yields U+FFFD, which both
|
|
63
|
-
// hides bytes and invents a character that was not there — and this is the one check in the repository
|
|
64
|
-
// whose entire subject is what the bytes actually are. It is exact rather than approximate for a reason
|
|
65
|
-
// the encoding gives for free: no C0 byte and no `0x7F` ever appears inside a multi-byte UTF-8
|
|
66
|
-
// sequence, whose continuation bytes are all `>= 0x80`. So a byte scan and a character scan agree here,
|
|
67
|
-
// and only the byte scan can be trusted to.
|
|
68
|
-
//
|
|
69
|
-
// ## Why the exemption is a NAMED PATH and never a content sniff
|
|
70
|
-
//
|
|
71
|
-
// Issue #68 offers two scopes: tracked text files — `git ls-files` plus a binary test — or an allow-list
|
|
72
|
-
// of extensions. **The binary test is refused, and the incident is why:** every binary sniff in general
|
|
73
|
-
// use is keyed on NUL, which is exactly the byte that motivated this check. `file` called the defective
|
|
74
|
-
// source *binary data*; git's own heuristic reads the first 8000 bytes for a NUL. A check that skipped
|
|
75
|
-
// what those call binary would have skipped the one file it exists to catch — a fail-open by
|
|
76
|
-
// construction, in the shape `a-checkers-coverage-is-measured-not-named.md` describes.
|
|
77
|
-
//
|
|
78
|
-
// An extension allow-list is refused for the reason this repository always refuses one: it is a door
|
|
79
|
-
// every future file type walks through in silence.
|
|
80
|
-
//
|
|
81
|
-
// What is left is a declaration. `--exempt <path>` names ONE path, is repeatable, and the workspace's
|
|
82
|
-
// recipe supplies it — so a binary asset arriving in this tree costs one reviewable line rather than a
|
|
83
|
-
// permanently red gate, and it costs it in a pull request where somebody sees it. The declaration is
|
|
84
|
-
// AUDITED, because an exemption nobody checks is the allow-list defect this project has already paid
|
|
85
|
-
// for once (`cli/vendor.mjs`, round 13 of #164 — the carve-out permitted a thing that was not a file).
|
|
86
|
-
// Three outcomes, each exit 2 and none a quiet pass:
|
|
87
|
-
//
|
|
88
|
-
// * an exempted path that is not in the list at all is **stale**;
|
|
89
|
-
// * an exempted path that is tracked and not on disk was **never read**, so this run cannot say
|
|
90
|
-
// whether the exemption is live — which is a different fact from either of the others and was
|
|
91
|
-
// reported as *dead* until Copilot separated them;
|
|
92
|
-
// * an exempted path carrying no control character is **dead**, because an exemption that is not
|
|
93
|
-
// load-bearing is one nobody will notice has stopped being true.
|
|
94
|
-
//
|
|
95
|
-
// **One limit, stated rather than left to be found:** an exemption is matched by the path as the file
|
|
96
|
-
// list spells it, and `--exempt` values arrive through `argv` already decoded. So a pathname that is
|
|
97
|
-
// not valid UTF-8 cannot be exempted — but it cannot be scanned either; `splitList` refuses the whole
|
|
98
|
-
// run over it at exit 2, so there is no silent gap here, only a repository this tool would decline to
|
|
99
|
-
// judge until the name is changed.
|
|
100
|
-
//
|
|
101
|
-
// Zero dependencies, no network, no install step — same constraints as ./doctor.mjs and ./index.mjs.
|
|
102
|
-
// It is not one of the eight subcommands `../docs/vision.md` names, and is not wired behind the entry
|
|
103
|
-
// point, for the same reason `./plugin-lint.mjs` is not: whether it joins them is the maintainer's call.
|
|
104
7
|
|
|
105
8
|
import fs from "node:fs";
|
|
106
9
|
import { pathToFileURL } from "node:url";
|
|
@@ -117,14 +20,7 @@ const TAB = 0x09;
|
|
|
117
20
|
const LF = 0x0a;
|
|
118
21
|
const DEL = 0x7f;
|
|
119
22
|
|
|
120
|
-
/**
|
|
121
|
-
* The C0 mnemonics, so a finding names the byte rather than printing it.
|
|
122
|
-
*
|
|
123
|
-
* Printing the character would reproduce the defect inside the report — the incident did exactly that
|
|
124
|
-
* twice while being fixed, once in the README paragraph describing it and once in the shell command
|
|
125
|
-
* carrying the commit message. A check whose output can carry the thing it checks for is a check that
|
|
126
|
-
* can silence its own next run.
|
|
127
|
-
*/
|
|
23
|
+
/** C0 mnemonics, so a finding names its byte rather than printing the defect into the report. */
|
|
128
24
|
const NAMES = [
|
|
129
25
|
"NUL", "SOH", "STX", "ETX", "EOT", "ENQ", "ACK", "BEL",
|
|
130
26
|
"BS", "TAB", "LF", "VT", "FF", "CR", "SO", "SI",
|
|
@@ -132,22 +28,10 @@ const NAMES = [
|
|
|
132
28
|
"CAN", "EM", "SUB", "ESC", "FS", "GS", "RS", "US",
|
|
133
29
|
];
|
|
134
30
|
|
|
135
|
-
/** `0x00` → `NUL`, `0x7f` → `DEL`. Never the character itself. */
|
|
136
31
|
export const nameOf = (byte) => (byte === DEL ? "DEL" : (NAMES[byte] ?? `0x${byte.toString(16).padStart(2, "0")}`));
|
|
137
32
|
|
|
138
|
-
/** Is this byte one a tracked file may not carry? TAB and LF are the whole allowance. */
|
|
139
33
|
export const isForbidden = (byte) => byte !== TAB && byte !== LF && (byte < 0x20 || byte === DEL);
|
|
140
34
|
|
|
141
|
-
/**
|
|
142
|
-
* The C1 mnemonics, `0x80`–`0x9f`, indexed from `0x80`.
|
|
143
|
-
*
|
|
144
|
-
* C1 is the half this check could not see until #207. These are control characters by every definition
|
|
145
|
-
* that matters here — invisible, semantically active in terminals (`CSI` opens an escape sequence), and
|
|
146
|
-
* unreachable by the byte test above, because in UTF-8 they arrive as **two** bytes, `0xc2` followed by
|
|
147
|
-
* `0x80`–`0x9f`, and neither of those is in C0.
|
|
148
|
-
*
|
|
149
|
-
* Measured at the retrospective pass over #167: a planted U+009B left this recipe green.
|
|
150
|
-
*/
|
|
151
35
|
const C1_NAMES = [
|
|
152
36
|
"PAD", "HOP", "BPH", "NBH", "IND", "NEL", "SSA", "ESA",
|
|
153
37
|
"HTS", "HTJ", "VTS", "PLD", "PLU", "RI", "SS2", "SS3",
|
|
@@ -157,66 +41,24 @@ const C1_NAMES = [
|
|
|
157
41
|
|
|
158
42
|
const C1_LEAD = 0xc2;
|
|
159
43
|
|
|
160
|
-
/**
|
|
161
|
-
* Is a C1 control character encoded at `i`? Two bytes, `0xc2` then `0x80`–`0x9f`.
|
|
162
|
-
*
|
|
163
|
-
* **This is the one place the scan looks at more than one byte, and it stays a byte test rather than
|
|
164
|
-
* becoming a decode.** Decoding is refused for the reason the header gives — U+FFFD both hides bytes and
|
|
165
|
-
* invents characters — and it is not needed: `0xc2` is a two-byte lead whose only continuations are
|
|
166
|
-
* `0x80`–`0xbf`, so this sequence is unambiguous without decoding anything around it.
|
|
167
|
-
*/
|
|
44
|
+
/** `0xc2` never occurs mid-sequence in UTF-8, so a C1 control is found without decoding. */
|
|
168
45
|
const isC1At = (buffer, i) => buffer[i] === C1_LEAD && buffer[i + 1] >= 0x80 && buffer[i + 1] <= 0x9f;
|
|
169
46
|
|
|
170
|
-
/** `0x9b` → `U+009B CSI`. The code point, never the character — `nameOf`'s rule on the other range. */
|
|
171
47
|
export const nameOfC1 = (second) => `U+00${second.toString(16).toUpperCase()} ${C1_NAMES[second - 0x80]}`;
|
|
172
48
|
|
|
173
|
-
/**
|
|
174
|
-
* Bytes rendered for a message, with everything outside printable ASCII as `\xNN` — **never raw**.
|
|
175
|
-
*
|
|
176
|
-
* `nameOf`'s rule, applied to a run of bytes instead of to one. It lives beside it because it is the
|
|
177
|
-
* same rule and was learned the same way.
|
|
178
|
-
*/
|
|
179
49
|
const escapeBytes = (buffer) =>
|
|
180
50
|
[...buffer]
|
|
181
51
|
.map((b) => {
|
|
182
|
-
//
|
|
183
|
-
// filename literally containing the characters `\`, `x`, `f`, `f` would otherwise render
|
|
184
|
-
// exactly like the byte `0xff`, so the message meant to remove an ambiguity would carry
|
|
185
|
-
// one. `\\` is unambiguous and reads correctly beside the `\xNN` escapes.
|
|
52
|
+
// A backslash is doubled, so a name holding the text `\xff` cannot read as the byte `0xff`.
|
|
186
53
|
if (b === 0x5c) return "\\\\";
|
|
187
54
|
return b >= 0x20 && b < DEL ? String.fromCharCode(b) : `\\x${b.toString(16).padStart(2, "0")}`;
|
|
188
55
|
})
|
|
189
56
|
.join("");
|
|
190
57
|
|
|
191
|
-
/**
|
|
192
|
-
* A pathname as it may be PRINTED. **Every message in this file that names a path goes through here.**
|
|
193
|
-
*
|
|
194
|
-
* Git tracks a filename of any bytes but NUL and `/` — measured, not assumed: a file named
|
|
195
|
-
* `we<LF>ird.md` is trackable, and `git ls-files -z` emits that newline raw. So a report interpolating
|
|
196
|
-
* a filename directly could **print the very bytes this check exists to catch** into a CI log, and a
|
|
197
|
-
* newline in a name would break the one-line-per-file contract every consumer of this output reads by.
|
|
198
|
-
*
|
|
199
|
-
* That is `nameOf`'s own argument turned on the other half of the line: *a check whose output can carry
|
|
200
|
-
* the thing it checks for is a check that can silence its own next run.* The escaping shipped for the
|
|
201
|
-
* undecodable-path case in round 3 and was not applied to the six places that name an ordinary path —
|
|
202
|
-
* a fix missing its siblings, the third time in this pull request. Copilot, #167, suppressed channel.
|
|
203
|
-
*/
|
|
58
|
+
/** Every path a message names goes through here: git allows any byte but NUL and `/` in a name. */
|
|
204
59
|
const displayPath = (file) => escapeBytes(Buffer.from(file, "utf8"));
|
|
205
60
|
|
|
206
|
-
/**
|
|
207
|
-
* Every forbidden byte in a buffer, as `{ offset, byte, name, line, column }`.
|
|
208
|
-
*
|
|
209
|
-
* **`offset` is 0-based; `line` and `column` are 1-based.** The three are not on one scale and saying
|
|
210
|
-
* so is not pedantry — `offset` is an index into the buffer, so it is 0-based by construction, while
|
|
211
|
-
* a line number that started at 0 would disagree with every editor and every other message in this
|
|
212
|
-
* repository. Copilot found the mismatch stated the other way on #167 round 1, in a test NAME, which
|
|
213
|
-
* is where a reader looks for the contract.
|
|
214
|
-
*
|
|
215
|
-
* **`column` counts BYTES**, likewise stated rather than left to be discovered: a line holding a
|
|
216
|
-
* three-byte em dash has a byte column larger than its character column, and this check's own subject
|
|
217
|
-
* is claims that outrun their measurement — the distinction the `plan` rail already makes about
|
|
218
|
-
* `awk`'s `length()`.
|
|
219
|
-
*/
|
|
61
|
+
/** Each forbidden byte: `offset` is 0-based, `line` and `column` 1-based, and `column` counts bytes. */
|
|
220
62
|
export function scanBytes(buffer) {
|
|
221
63
|
const found = [];
|
|
222
64
|
let line = 1;
|
|
@@ -232,20 +74,6 @@ export function scanBytes(buffer) {
|
|
|
232
74
|
found.push({ offset: i, byte, name: nameOf(byte), line, column: i - lineStart + 1 });
|
|
233
75
|
continue;
|
|
234
76
|
}
|
|
235
|
-
// C1, and reported at the LEAD byte: that is where the character starts, and an offset pointing
|
|
236
|
-
// at the continuation would send a reader one byte past the thing they are looking for. `i` is
|
|
237
|
-
// not advanced — the continuation is `0x80`–`0x9f`, which no other branch here matches, so there
|
|
238
|
-
// is nothing to skip and the line counting stays a single rule.
|
|
239
|
-
//
|
|
240
|
-
// `byte` is the byte AT `offset`, which for C1 is the lead `0xc2` — not the continuation that
|
|
241
|
-
// says which C1 this is. The first draft put the continuation here and left `offset` on the
|
|
242
|
-
// lead, so one record answered "where" and "what" about two different bytes: a reader seeking
|
|
243
|
-
// `byte` at `offset` would not find it. Nothing consumes the field today — the locator prints
|
|
244
|
-
// `name`, `line`, `column` and `offset` — so this is latent rather than a live wrong message,
|
|
245
|
-
// and that is exactly why it is worth fixing now: an unread field is where an untrue one
|
|
246
|
-
// survives. `name` already carries which character it is, via the continuation, so nothing is
|
|
247
|
-
// lost. The C0 branch above sets `byte` to the byte at its own offset; both branches now mean
|
|
248
|
-
// the same thing by the same key. Copilot, #251 round 1.
|
|
249
77
|
if (isC1At(buffer, i)) {
|
|
250
78
|
found.push({ offset: i, byte: buffer[i], name: nameOfC1(buffer[i + 1]), line, column: i - lineStart + 1 });
|
|
251
79
|
}
|
|
@@ -253,23 +81,7 @@ export function scanBytes(buffer) {
|
|
|
253
81
|
return found;
|
|
254
82
|
}
|
|
255
83
|
|
|
256
|
-
/**
|
|
257
|
-
* The bytes git tracks for one path, or `null` when the path is not on disk.
|
|
258
|
-
*
|
|
259
|
-
* **A symlink is read as its target's BYTES, not through it.** Those bytes are the blob git stores, and
|
|
260
|
-
* they are the thing this check is asked about; following the link would scan a file that may not be in
|
|
261
|
-
* the repository at all, and a dangling one would be an error about the wrong thing. Bytes rather than
|
|
262
|
-
* the string `readlinkSync` hands back by default — see the branch below for what that default cost.
|
|
263
|
-
*
|
|
264
|
-
* Only `ENOENT` is `null` — a file tracked in the index and deleted from the working tree, which is an
|
|
265
|
-
* ordinary state and the one `../.portulan/verify/json.sh` already skips. Every other errno means the
|
|
266
|
-
* process could not look, and *could not look* reported as *looked and found nothing* is the failure
|
|
267
|
-
* `a-checker-must-refuse-what-it-cannot-check.md` is about.
|
|
268
|
-
*
|
|
269
|
-
* Anything that is neither a regular file nor a symlink — a FIFO, a socket, a device node, a directory
|
|
270
|
-
* — is refused rather than guessed at, which is `./vendor.mjs`'s rule in `./vendor.mjs`'s words: this
|
|
271
|
-
* reads neither by guessing at it.
|
|
272
|
-
*/
|
|
84
|
+
/** The bytes git stores for a path (a symlink's target path, not its target), or `null` when not on disk. */
|
|
273
85
|
export function bytesOf(file) {
|
|
274
86
|
let stat;
|
|
275
87
|
try {
|
|
@@ -284,13 +96,7 @@ export function bytesOf(file) {
|
|
|
284
96
|
|
|
285
97
|
if (stat.isSymbolicLink()) {
|
|
286
98
|
try {
|
|
287
|
-
// `"buffer"
|
|
288
|
-
// target git stores as raw bytes came back U+FFFD-substituted and was re-encoded into
|
|
289
|
-
// DIFFERENT bytes — which both breaks this function's own "never decodes" promise and can
|
|
290
|
-
// hide a control byte sitting in the target. That is the identical defect `splitList`
|
|
291
|
-
// was just repaired for, two functions below, in the same push that repaired it: a fix
|
|
292
|
-
// arriving without its sibling, which is the class `0020` names. Copilot found it in the
|
|
293
|
-
// suppressed channel on the very next round.
|
|
99
|
+
// `"buffer"`: the default decodes as UTF-8 and alters a target that is not valid UTF-8.
|
|
294
100
|
return fs.readlinkSync(file, "buffer");
|
|
295
101
|
} catch (cause) {
|
|
296
102
|
throw new ControlCharsError(`cannot read the symlink ${displayPath(file)} — ${cause.code ?? cause.message}`);
|
|
@@ -313,22 +119,6 @@ export function bytesOf(file) {
|
|
|
313
119
|
}
|
|
314
120
|
}
|
|
315
121
|
|
|
316
|
-
/**
|
|
317
|
-
* Scan a list of paths.
|
|
318
|
-
*
|
|
319
|
-
* Returns `{ findings, unusable, scanned, skipped, exempted }`. A finding is
|
|
320
|
-
* `{ file, count, first }` — one per offending file rather than one per byte, because a genuinely
|
|
321
|
-
* binary asset holds thousands and a report that prints them all is a report nobody reads. The count
|
|
322
|
-
* is stated beside the first, so nothing is dropped in silence.
|
|
323
|
-
*
|
|
324
|
-
* `unusable` holds the exemptions this run could not show to be doing their job, each with its own
|
|
325
|
-
* `why`. It was called `stale` while it carried two cases and gained a third — *not read* — which is
|
|
326
|
-
* not staleness at all, so the name went with the meaning rather than after it.
|
|
327
|
-
*
|
|
328
|
-
* **An empty list is refused.** Zero files scanned and a green report is the enumeration fail-open this
|
|
329
|
-
* repository has fixed five times (`verify-preconditions-fail-closed.md`), and it is the failure this
|
|
330
|
-
* check is most exposed to: its whole input arrives through a pipe.
|
|
331
|
-
*/
|
|
332
122
|
export function inspect(files, { exempt = new Set() } = {}) {
|
|
333
123
|
if (files.length === 0) {
|
|
334
124
|
throw new ControlCharsError(
|
|
@@ -340,8 +130,6 @@ export function inspect(files, { exempt = new Set() } = {}) {
|
|
|
340
130
|
const findings = [];
|
|
341
131
|
const seen = new Set();
|
|
342
132
|
const carrying = new Set();
|
|
343
|
-
// Listed and NOT READ — tracked, absent from the working tree. Kept apart from `carrying` because
|
|
344
|
-
// the audit below must not read one as the other; see there.
|
|
345
133
|
const unread = new Set();
|
|
346
134
|
let scanned = 0;
|
|
347
135
|
let skipped = 0;
|
|
@@ -362,16 +150,6 @@ export function inspect(files, { exempt = new Set() } = {}) {
|
|
|
362
150
|
findings.push({ file, count: bad.length, first: bad[0] });
|
|
363
151
|
}
|
|
364
152
|
|
|
365
|
-
// The audit, in three directions rather than two. An exemption naming nothing in the list is stale
|
|
366
|
-
// and one over a clean file is dead — but between them sits a third case the first draft collapsed
|
|
367
|
-
// into the second: a file that is tracked, absent from the working tree, and therefore NEVER READ.
|
|
368
|
-
// `seen` had it, `carrying` did not, so it was reported *carries no control character, so the
|
|
369
|
-
// exemption is dead* — a specific claim about a file nothing opened, which would send the
|
|
370
|
-
// maintainer to delete an exemption that is still load-bearing. That is this check's own subject
|
|
371
|
-
// and #92's class exactly: a verdict about something never examined. Copilot, suppressed channel.
|
|
372
|
-
//
|
|
373
|
-
// It stays exit 2 rather than passing quietly: an exemption nothing could verify is not a verified
|
|
374
|
-
// exemption, which is `a-checker-must-refuse-what-it-cannot-check.md`.
|
|
375
153
|
const unusable = [];
|
|
376
154
|
for (const file of exempt) {
|
|
377
155
|
if (!seen.has(file)) unusable.push({ file, why: "is not in the scanned set" });
|
|
@@ -392,31 +170,7 @@ export function inspect(files, { exempt = new Set() } = {}) {
|
|
|
392
170
|
// The command
|
|
393
171
|
// ===========================================================================================
|
|
394
172
|
|
|
395
|
-
/**
|
|
396
|
-
* The NUL-separated list `git ls-files -z` writes, split at the BYTE level.
|
|
397
|
-
*
|
|
398
|
-
* Returns `{ paths, undecodable }`. A trailing separator yields no empty entry.
|
|
399
|
-
*
|
|
400
|
-
* **The split and the decode are two different fail-opens, and only the first was closed.** `-z` was
|
|
401
|
-
* chosen so a filename could not reach this scanner mis-SPELLED — git C-quotes a control character in
|
|
402
|
-
* the line-based form regardless of `core.quotePath`, so the alternative delivers `"we\nird.md"` as one
|
|
403
|
-
* printable line rather than two, and the loss is the name and not the split (#209, measured; an earlier
|
|
404
|
-
* version of this sentence said "could not be mis-split", which named a failure git already prevents) —
|
|
405
|
-
* but the list was then decoded
|
|
406
|
-
* whole with `readFileSync(0, "utf8")`, and git allows a pathname to be **any bytes except NUL and
|
|
407
|
-
* `/`**. Node replaces an invalid sequence with U+FFFD, so such a name arrived here as a *different
|
|
408
|
-
* string*, `bytesOf` looked for that string, got `ENOENT`, and the file was counted as *tracked and
|
|
409
|
-
* not on disk* — **skipped, and the run reported green over a file nothing had read**, with a
|
|
410
|
-
* plausible sentence in the summary saying so. A silent skip is the exact failure this whole check
|
|
411
|
-
* exists to catch, one layer up: `grep` went quiet on the NUL, and this went quiet on the name.
|
|
412
|
-
* Copilot found it on #167 in both channels at once — the thread on the production read, the
|
|
413
|
-
* suppressed note on the same decode in the live-tree test.
|
|
414
|
-
*
|
|
415
|
-
* So the bytes are split as bytes, and each chunk is kept only if it **round-trips** through UTF-8.
|
|
416
|
-
* Anything else is returned as `undecodable` for the caller to refuse over — never repaired, never
|
|
417
|
-
* dropped. `a-checker-must-refuse-what-it-cannot-check.md`: a path this tool cannot represent is one
|
|
418
|
-
* it would look for under a different name.
|
|
419
|
-
*/
|
|
173
|
+
/** `git ls-files -z` output, split as bytes; a name that fails a UTF-8 round trip is `undecodable`. */
|
|
420
174
|
export function splitList(input) {
|
|
421
175
|
const buffer = Buffer.isBuffer(input) ? input : Buffer.from(input, "utf8");
|
|
422
176
|
const paths = [];
|
|
@@ -427,9 +181,7 @@ export function splitList(input) {
|
|
|
427
181
|
if (i > start) {
|
|
428
182
|
const chunk = buffer.subarray(start, i);
|
|
429
183
|
const text = chunk.toString("utf8");
|
|
430
|
-
//
|
|
431
|
-
// U+FFFD, and rejecting that would be a false red on a name git stores exactly as given.
|
|
432
|
-
// Re-encoding answers the only question that matters — did anything change on the way in.
|
|
184
|
+
// A round trip rather than a U+FFFD test: a filename may itself contain U+FFFD.
|
|
433
185
|
if (Buffer.from(text, "utf8").equals(chunk)) paths.push(text);
|
|
434
186
|
else undecodable.push(chunk);
|
|
435
187
|
}
|
|
@@ -444,15 +196,7 @@ export function run(argv, stdin, say = (line) => process.stdout.write(`${line}\n
|
|
|
444
196
|
if (argv[i] === "--exempt") {
|
|
445
197
|
const file = argv[i + 1];
|
|
446
198
|
i += 1;
|
|
447
|
-
// Only a
|
|
448
|
-
// `--`, copied from `./index.mjs`'s `--pack-root` — and here that guard blocks a path git
|
|
449
|
-
// can perfectly well track, in the one mechanism this whole check rests on: a file named
|
|
450
|
-
// `--asset.bin` could never be exempted, with no workaround. It buys nothing either, and
|
|
451
|
-
// that is the deciding half rather than the rarity: the mistake it was aimed at — a
|
|
452
|
-
// forgotten value, `--exempt --other` — is already caught by the exemption AUDIT, which
|
|
453
|
-
// says `--exempt --other is not in the scanned set` and exits 2. A guard whose only job is
|
|
454
|
-
// already done elsewhere, at the cost of a capability, is one to delete. Copilot, #167,
|
|
455
|
-
// suppressed channel.
|
|
199
|
+
// Only a missing value is refused: a path git tracks may start with `--`.
|
|
456
200
|
if (file === undefined) {
|
|
457
201
|
say(" ✗ --exempt needs a path");
|
|
458
202
|
return 2;
|
|
@@ -464,9 +208,6 @@ export function run(argv, stdin, say = (line) => process.stdout.write(`${line}\n
|
|
|
464
208
|
}
|
|
465
209
|
}
|
|
466
210
|
|
|
467
|
-
// Refused before anything is scanned, and refused rather than skipped. Exit 2 is the honest code:
|
|
468
|
-
// the tree may be perfectly clean, and this run cannot say so — which is the difference between a
|
|
469
|
-
// verdict and a failure to reach one.
|
|
470
211
|
const { paths, undecodable } = splitList(stdin);
|
|
471
212
|
if (undecodable.length) {
|
|
472
213
|
for (const chunk of undecodable) {
|
|
@@ -489,18 +230,11 @@ export function run(argv, stdin, say = (line) => process.stdout.write(`${line}\n
|
|
|
489
230
|
return 2;
|
|
490
231
|
}
|
|
491
232
|
|
|
492
|
-
// The stale-exemption audit is a defect in the declaration rather than a verdict about a file, so
|
|
493
|
-
// it exits 2 — the same code, and the same reasoning, as a stale entry in `index.sh`'s WORKSPACES.
|
|
494
233
|
if (result.unusable.length) {
|
|
495
234
|
for (const s of result.unusable) say(` ✗ --exempt ${displayPath(s.file)} ${s.why}`);
|
|
496
235
|
return 2;
|
|
497
236
|
}
|
|
498
237
|
|
|
499
|
-
// The locator leads with line and byte column — 1-based, the numbers an editor shows — and gives
|
|
500
|
-
// the buffer offset after them, saying it is 0-based. The first draft printed `byte 19 (line 1,
|
|
501
|
-
// byte column 20)` and named neither base, so the two numbers looked like a contradiction about
|
|
502
|
-
// the same position. A message that has to be decoded is the failure this whole check is about,
|
|
503
|
-
// one layer up. Copilot, round 1 on #167, which found the same mismatch in a test name.
|
|
504
238
|
for (const f of result.findings) {
|
|
505
239
|
say(
|
|
506
240
|
` ✗ ${displayPath(f.file)}: ${f.count} control character(s) — first at line ${f.first.line}, ` +
|
|
@@ -521,14 +255,9 @@ export function run(argv, stdin, say = (line) => process.stdout.write(`${line}\n
|
|
|
521
255
|
if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
|
|
522
256
|
let stdin;
|
|
523
257
|
try {
|
|
524
|
-
//
|
|
525
|
-
// check it, which is where the U+FFFD substitution happened — see `splitList`. The bytes
|
|
526
|
-
// arrive as bytes and the decode is per path, with a round trip deciding.
|
|
258
|
+
// Bytes, not text: decoding here would turn a name that is not valid UTF-8 into another name.
|
|
527
259
|
stdin = fs.readFileSync(0);
|
|
528
260
|
} catch (cause) {
|
|
529
|
-
// A terminal with nothing piped into it, most often. Refused with the usage line rather than
|
|
530
|
-
// an empty list, which `inspect` would refuse anyway — but as "the enumeration failed", which
|
|
531
|
-
// is a diagnosis of the wrong thing.
|
|
532
261
|
process.stderr.write(
|
|
533
262
|
`control-chars: cannot read the file list from stdin — ${cause.code ?? cause.message}\n` +
|
|
534
263
|
"usage: git ls-files --cached --others --exclude-standard -z | " +
|
|
@@ -537,25 +266,7 @@ if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
|
|
|
537
266
|
process.exitCode = 2;
|
|
538
267
|
stdin = null;
|
|
539
268
|
}
|
|
540
|
-
// `
|
|
541
|
-
// outright can truncate a pipe that has not drained. `./plugin-lint.mjs` settled that already.
|
|
542
|
-
//
|
|
543
|
-
// **1 is reserved for a finding, so everything unhandled is 2** (#208). An uncaught error here used
|
|
544
|
-
// to leave node to exit **1**, which `../.portulan/verify/control-chars.sh` reads as this tool's
|
|
545
|
-
// documented "a scanned file carries a control character" — a judged verdict about a tree, produced
|
|
546
|
-
// by a run that judged nothing. The direction was always safe; the SENTENCE was false, which is the
|
|
547
|
-
// shape this file's own recipe cites `compile.sh` for. The stack still prints, because a crash is a
|
|
548
|
-
// defect in this tool and hiding it would trade one wrong report for another.
|
|
549
|
-
//
|
|
550
|
-
// **The test that binds this forces the crash from OUTSIDE the module** — a `node --import` preload
|
|
551
|
-
// that makes `fs.lstatSync` return a stat-shaped object with no `isSymbolicLink`, so the TypeError
|
|
552
|
-
// lands in `bytesOf`'s one unwrapped call: the `try` there covers the `lstatSync` CALL, and
|
|
553
|
-
// `stat.isSymbolicLink()` sits outside it. That is deliberate — it keeps the fault injection out of
|
|
554
|
-
// the shipped code, where a test-only `main()` export or an env-gated throw would both have added a
|
|
555
|
-
// surface whose only caller is this block. **The coupling is worth knowing before it bites:** if a
|
|
556
|
-
// later change wraps `stat.isSymbolicLink()` inside that `try`, the preload stops producing an
|
|
557
|
-
// uncaught error and the test goes red for a reason that has nothing to do with this arm. The repair
|
|
558
|
-
// then is a NEW fault seam, never a revert of the wrapping.
|
|
269
|
+
// `exitCode`, not `exit()`, so stdout drains; anything uncaught exits 2, because 1 means a finding.
|
|
559
270
|
if (stdin !== null) {
|
|
560
271
|
try {
|
|
561
272
|
process.exitCode = run(process.argv.slice(2), stdin);
|