agent-sanitizer 2.54.1 → 2.55.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -14
- package/THREAT-MODEL.md +45 -0
- package/claude-hooks/lib/authored-content.mjs +82 -29
- package/package.json +1 -1
- package/src/ansi.mjs +74 -0
- package/src/index.mjs +2 -0
- package/src/layer1.mjs +75 -1
- package/types/ansi.d.mts +29 -0
- package/types/index.d.mts +1 -1
- package/types/layer1.d.mts +39 -0
package/README.md
CHANGED
|
@@ -1,18 +1,6 @@
|
|
|
1
1
|
# `agent-sanitizer`
|
|
2
2
|
|
|
3
|
-
**Cleans untrusted text before your agent reads it.** An attacker hides a payload where a person cannot see it but the model still reads it: invisible Unicode, ANSI escapes, human-hidden HTML, confusable glyphs, look-alike hosts, and exfil-shaped URLs.
|
|
4
|
-
|
|
5
|
-
This library handles each channel on its own terms. It strips invisible characters and ANSI escapes by default, splices out hidden HTML when you opt in, and _reports_ exfil-shaped URLs and look-alike hosts rather than rewriting them — mangling a legitimate link is the worse failure. Every layer is a deterministic transform, so you can unit-test it with equality assertions. There is no classifier and no model call, so nothing rests on whether a prediction generalized.
|
|
6
|
-
|
|
7
|
-
**As a library:**
|
|
8
|
-
|
|
9
|
-
```sh
|
|
10
|
-
npm install agent-sanitizer
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
**As a Claude Code plugin:**
|
|
14
|
-
|
|
15
|
-
Enter one at a time:
|
|
3
|
+
**Cleans untrusted text before your agent reads it.** An attacker hides a payload where a person cannot see it but the model still reads it: invisible Unicode, ANSI escapes, human-hidden HTML, confusable glyphs, look-alike hosts, and exfil-shaped URLs. This library handles each channel. Easy to use as a Claude Code plugin, and the plugin itself is recommended by [`alignment-hive`](https://github.com/crazytieguy/alignment-hive/)!
|
|
16
4
|
|
|
17
5
|
```
|
|
18
6
|
/plugin marketplace add AlexanderMattTurner/agent-sanitizer
|
|
@@ -291,7 +279,7 @@ singleton, and two copies in one bundle double-fire the inlined CLIs.
|
|
|
291
279
|
| `claude-hooks/scan-loaded-instructions` | InstructionsLoaded scan of each instruction file as Claude Code loads it |
|
|
292
280
|
| `claude-hooks/lib/hook-io` | Shared hook I/O: the lazy-module registry, the CLI slot, deadlines, the hookgate marker |
|
|
293
281
|
| `claude-hooks/lib/control-plane` | Bridge to `agent-control-plane-core` and the shared judge-CLI transport |
|
|
294
|
-
| `claude-hooks/lib/authored-content` | Stego + terminal-control stripping of the fields the MODEL authors
|
|
282
|
+
| `claude-hooks/lib/authored-content` | Stego + terminal-control stripping of the fields the MODEL authors (colour is kept) |
|
|
295
283
|
| `claude-hooks/lib/env-config` | The env-bound secret vocabulary the Layer-4 pre-gate and the redactor client share |
|
|
296
284
|
| `claude-hooks/lib/invisible-alert` | Cross-hook alert state for uncleanable invisible-char injection in instruction files |
|
|
297
285
|
| `claude-hooks/lib/redactor-client` | Client for the long-lived `agent-secret-redactor-daemon` (Layer 4's transport) |
|
package/THREAT-MODEL.md
CHANGED
|
@@ -290,6 +290,51 @@ reconstitute a secret the first pass never saw intact. On `Bash.command` it runs
|
|
|
290
290
|
before `sanitizeAuthoredContent`, which is the assumption the confusable-folding
|
|
291
291
|
soundness argument below relies on.
|
|
292
292
|
|
|
293
|
+
## Model-authored writes (tool input)
|
|
294
|
+
|
|
295
|
+
`claude-hooks/lib/authored-content` scrubs what the model EMITS — the file
|
|
296
|
+
content, edits, notebook cells and command bodies that get persisted or executed
|
|
297
|
+
— rather than what it reads. Two protections, with different gates. Stego
|
|
298
|
+
(format characters, variation selectors) is stripped only when the volume can
|
|
299
|
+
carry a message, because authored content is persisted and incidental joiners
|
|
300
|
+
are legitimate. Terminal-control sequences are stripped on sight: one cursor
|
|
301
|
+
move, erase or OSC string in a file is a latent bomb the next `cat` fires, and
|
|
302
|
+
one in a command spoofs what the user is shown as it runs.
|
|
303
|
+
|
|
304
|
+
Display-only SGR colour is the exception, and is preserved byte-for-byte. The
|
|
305
|
+
SGR grammar is closed — parameters and a final `m` — so it restyles visible text
|
|
306
|
+
and can do nothing else; stripping it costs a model the colourized fixture, TUI
|
|
307
|
+
golden file or prompt string it deliberately wrote, and buys no safety. Two
|
|
308
|
+
shapes end the exception, both of them the model-sees/human-sees divergence
|
|
309
|
+
rather than a display choice: a CONCEAL parameter (`ESC[8m`), which blanks the
|
|
310
|
+
text that follows for a human while its bytes stay readable to a model, and a
|
|
311
|
+
run of sequences with nothing that RENDERS between them, which puts no glyph on
|
|
312
|
+
the screen at all and is therefore carrying data rather than colour. Either one
|
|
313
|
+
strips the whole field.
|
|
314
|
+
|
|
315
|
+
Both shapes are read as terminal semantics rather than token shapes, because a
|
|
316
|
+
reader that matches shapes is one encoding away from the wrong answer. Conceal
|
|
317
|
+
is threaded as STATE, so a reveal or reset later in the same sequence cancels it
|
|
318
|
+
(`ESC[8;28;31m` renders red and visible) while an erase between two sequences
|
|
319
|
+
does not. The parameters are read the way a terminal reads them, so an
|
|
320
|
+
extended-colour argument is never mistaken for a parameter — `ESC[38;5;8m` is
|
|
321
|
+
bright-black foreground, not conceal. And the run is counted over characters
|
|
322
|
+
that render, so a zero-width separator between two sequences is as transparent
|
|
323
|
+
here as it is to a terminal, while a space — which is what an ANSI-art colour
|
|
324
|
+
bar puts between its codes — breaks the run.
|
|
325
|
+
|
|
326
|
+
What that exception does NOT claim: a message spread thinly through
|
|
327
|
+
legitimately coloured text — a sequence per line, encoding a bit — is
|
|
328
|
+
indistinguishable from styling, and is left alone. That is the same bargain the
|
|
329
|
+
invisible-character thresholds strike, in the same direction: an unprovable
|
|
330
|
+
payload costs a false negative, while a wrong strip costs every colourized file
|
|
331
|
+
its colour.
|
|
332
|
+
|
|
333
|
+
Both protections run to a fixed point over each field rather than once each,
|
|
334
|
+
because they feed one another: removing an invisible character completes a
|
|
335
|
+
sequence that was incomplete when the terminal stage ran (`ESC[`⟨ZWSP⟩`2J`), and
|
|
336
|
+
removing a sequence makes invisibles adjacent that were not.
|
|
337
|
+
|
|
293
338
|
## Confusable folding (tool input)
|
|
294
339
|
|
|
295
340
|
`./confusables` folds look-alike glyphs in tool-call **input** fields (paths,
|
|
@@ -15,11 +15,17 @@
|
|
|
15
15
|
* authored into a command — echoed and executed live — or into file
|
|
16
16
|
* content (a latent bomb when the file is later `cat`'d) can clear the
|
|
17
17
|
* screen, reposition the cursor, or overwrite what the user sees, hiding
|
|
18
|
-
* the real command behind spoofed output.
|
|
19
|
-
*
|
|
20
|
-
*
|
|
21
|
-
*
|
|
22
|
-
*
|
|
18
|
+
* the real command behind spoofed output. A single such sequence already
|
|
19
|
+
* does harm, so there is no volume threshold: one of them strips the whole
|
|
20
|
+
* field. Display-only SGR colour is the exception and is PRESERVED
|
|
21
|
+
* byte-for-byte — it restyles text and can neither move the cursor nor
|
|
22
|
+
* carry an OSC payload, so stripping it only costs a model the colourized
|
|
23
|
+
* fixture, TUI golden file or prompt string it deliberately wrote. The
|
|
24
|
+
* carve-out ends where SGR stops being styling (see the engine's
|
|
25
|
+
* sgrCarriesPayload): a CONCEAL parameter blanks text for a human while a
|
|
26
|
+
* model still reads it, and a long run of sequences with nothing that
|
|
27
|
+
* renders between them puts no glyph on the screen at all, which is a
|
|
28
|
+
* covert channel rather than colour.
|
|
23
29
|
*
|
|
24
30
|
* SCOPE IS DECLARED, NOT INFERRED. Which tools this layer touches is a
|
|
25
31
|
* partition — {@link AUTHORED_FIELDS} (covered, with the field list) and
|
|
@@ -50,9 +56,10 @@ import { lazyImport } from "./hook-io.mjs";
|
|
|
50
56
|
// static import of this module, before its fail-closed catch runs). A failed
|
|
51
57
|
// load leaves these bindings undefined, so sanitizeField's calls throw into
|
|
52
58
|
// the fail-closed catch (ask) instead.
|
|
53
|
-
const { stripAnsiFully
|
|
54
|
-
|
|
55
|
-
)
|
|
59
|
+
const { stripAnsiFully, isBenignAnsiKinds, sgrCarriesPayload } =
|
|
60
|
+
/** @type {typeof import("agent-sanitizer")} */ (
|
|
61
|
+
await lazyImport("agent-sanitizer")
|
|
62
|
+
);
|
|
56
63
|
const { STRIP, SCATTERED_THRESHOLD, hasLongRun, stripInvisible } =
|
|
57
64
|
/** @type {typeof import("agent-sanitizer/invisible")} */ (
|
|
58
65
|
await lazyImport("agent-sanitizer/invisible")
|
|
@@ -147,38 +154,84 @@ function isPayloadCapable(text) {
|
|
|
147
154
|
return (text.match(STRIP)?.length ?? 0) >= SCATTERED_THRESHOLD;
|
|
148
155
|
}
|
|
149
156
|
|
|
157
|
+
/**
|
|
158
|
+
* The de-ANSI'd `text`, or `text` itself when its escapes are display-only
|
|
159
|
+
* colour a reader authored on purpose.
|
|
160
|
+
*
|
|
161
|
+
* The two questions are answered by two different views, deliberately.
|
|
162
|
+
* `isBenignAnsiKinds` reads the kinds the strip actually REMOVED, so a sequence
|
|
163
|
+
* that only reconstitutes mid-strip is judged as the sequence it becomes rather
|
|
164
|
+
* than as the incomplete one it started as. `sgrCarriesPayload` reads the RAW
|
|
165
|
+
* bytes, because those are the bytes a terminal renders on the preserve path —
|
|
166
|
+
* it makes no second pass, so there is nothing to reconstitute there.
|
|
167
|
+
*
|
|
168
|
+
* Only COMPLETE tokens reach `kinds`: an introducer that opens no sequence is
|
|
169
|
+
* left in place by `stripAnsiFully` (it rewrites no display), so the orphan arms
|
|
170
|
+
* of `isBenignAnsiKinds` never decide anything here.
|
|
171
|
+
* @param {string} text
|
|
172
|
+
* @returns {string}
|
|
173
|
+
*/
|
|
174
|
+
function stripTerminalControls(text) {
|
|
175
|
+
/** @type {Set<string>} TOKEN_KINDs the strip removed. */
|
|
176
|
+
const kinds = new Set();
|
|
177
|
+
const deAnsi = stripAnsiFully(text, kinds);
|
|
178
|
+
if (deAnsi === text) return text;
|
|
179
|
+
if (isBenignAnsiKinds(kinds) && !sgrCarriesPayload(text)) return text;
|
|
180
|
+
return deAnsi;
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
// How many rounds of {terminal-control, invisible} the field may take. The two
|
|
184
|
+
// protections FEED each other: removing invisibles completes a sequence that was
|
|
185
|
+
// incomplete when the terminal stage ran (`ESC[` ZWSP `2J` is an orphan until the
|
|
186
|
+
// ZWSP goes), and removing a sequence makes invisibles adjacent that were not.
|
|
187
|
+
// One round of each therefore leaves the composition unfinished. Each round
|
|
188
|
+
// deletes at least one character, so the loop converges on its own; the bound is
|
|
189
|
+
// a DoS guard. No residual sweep stands behind it — unlike the engine's
|
|
190
|
+
// MAX_LAYER1_PASSES — so an unconverged value would persist a live sequence into
|
|
191
|
+
// the file, and reaching the bound while still changing throws instead.
|
|
192
|
+
const MAX_FIELD_PASSES = 3;
|
|
193
|
+
|
|
150
194
|
// Returns the cleaned value plus the human-readable actions applied, or null if
|
|
151
195
|
// the field is already clean. Each protection has its own opt-out (see the
|
|
152
196
|
// header) so a deployment can keep one while dropping the other.
|
|
153
197
|
/** @param {string} value */
|
|
154
198
|
function sanitizeField(value) {
|
|
155
|
-
|
|
199
|
+
// A Set, so a protection that fires in more than one round is described once.
|
|
200
|
+
/** @type {Set<string>} */
|
|
201
|
+
const actions = new Set();
|
|
156
202
|
let cleaned = value;
|
|
157
203
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
204
|
+
for (let pass = 0; pass < MAX_FIELD_PASSES; pass++) {
|
|
205
|
+
const before = cleaned;
|
|
206
|
+
// Terminal controls first, so the invisible scan runs on the same de-ANSI'd
|
|
207
|
+
// view sanitize-output uses. Compare before/after rather than pre-testing
|
|
208
|
+
// for ESC: escapes this layer deliberately leaves in place — a lone control
|
|
209
|
+
// byte, display-only colour — must not be reported as a strip.
|
|
210
|
+
if (process.env.AGENT_SANITIZER_TERMINAL_DISABLED !== "1") {
|
|
211
|
+
const deAnsi = stripTerminalControls(cleaned);
|
|
212
|
+
if (deAnsi !== cleaned) {
|
|
213
|
+
cleaned = deAnsi;
|
|
214
|
+
actions.add("terminal-control sequences");
|
|
215
|
+
}
|
|
170
216
|
}
|
|
171
|
-
}
|
|
172
217
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
218
|
+
if (
|
|
219
|
+
process.env.AGENT_SANITIZER_INVISIBLE_DISABLED !== "1" &&
|
|
220
|
+
isPayloadCapable(cleaned)
|
|
221
|
+
) {
|
|
222
|
+
cleaned = stripInvisible(cleaned);
|
|
223
|
+
actions.add("invisible characters");
|
|
224
|
+
}
|
|
225
|
+
if (cleaned === before) break;
|
|
226
|
+
// This refusal is what blocks a still-changing value — one whose next round
|
|
227
|
+
// would remove another live sequence — from being written to the file.
|
|
228
|
+
if (pass === MAX_FIELD_PASSES - 1)
|
|
229
|
+
throw new Error(
|
|
230
|
+
"authored content did not reach a fixed point within MAX_FIELD_PASSES",
|
|
231
|
+
);
|
|
179
232
|
}
|
|
180
233
|
|
|
181
|
-
return actions.
|
|
234
|
+
return actions.size > 0 ? { cleaned, actions: [...actions] } : null;
|
|
182
235
|
}
|
|
183
236
|
|
|
184
237
|
/** @param {string[]} changed */
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-sanitizer",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.55.0",
|
|
4
4
|
"description": "Defend an agent against hidden-content injection: strip payload-capable invisible Unicode and ANSI, splice out human-invisible HTML, and flag data-exfil URLs in untrusted text before any model sees it.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|
package/src/ansi.mjs
CHANGED
|
@@ -205,6 +205,32 @@ export const ESCAPE_SEQUENCE_SOURCE = (() => {
|
|
|
205
205
|
return `(?:${stringArm}|${csiArm})`;
|
|
206
206
|
})();
|
|
207
207
|
|
|
208
|
+
// The three SGR parameters that move the CONCEAL state (ECMA-48 § 8.3.117).
|
|
209
|
+
// Conceal renders the text that follows as blank while its bytes stay in the
|
|
210
|
+
// file, so a human reading with `cat` sees nothing where a model reads
|
|
211
|
+
// everything — a model-sees/human-sees divergence rather than a display choice.
|
|
212
|
+
const SGR_RESET = 0;
|
|
213
|
+
const SGR_CONCEAL = 8;
|
|
214
|
+
const SGR_REVEAL = 28;
|
|
215
|
+
// The extended-colour selectors. Their ARGUMENTS follow as further semicolon
|
|
216
|
+
// parameters, so a reader that does not consume them reads a colour component as
|
|
217
|
+
// a parameter of its own: `ESC[38;5;8m` is bright-black foreground, not conceal.
|
|
218
|
+
const SGR_EXTENDED_COLOUR = new Set([38, 48, 58]);
|
|
219
|
+
// How many parameters an extended-colour selector consumes after itself, keyed
|
|
220
|
+
// by the colour-space selector that follows it, per ITU T.416 § 13.1.8: the
|
|
221
|
+
// implementation-defined and transparent forms take none, direct-colour RGB and
|
|
222
|
+
// CMY take three components, CMYK four, and indexed takes one palette entry.
|
|
223
|
+
// Every selector the spec defines, so a value missing from this table is
|
|
224
|
+
// malformed rather than merely unimplemented.
|
|
225
|
+
const SGR_COLOUR_ARGS = new Map([
|
|
226
|
+
[0, 0],
|
|
227
|
+
[1, 0],
|
|
228
|
+
[2, 3],
|
|
229
|
+
[3, 3],
|
|
230
|
+
[4, 4],
|
|
231
|
+
[5, 1],
|
|
232
|
+
]);
|
|
233
|
+
|
|
208
234
|
/** The seven things an introducer can turn out to be. */
|
|
209
235
|
export const TOKEN_KIND = Object.freeze({
|
|
210
236
|
/** A display-only `ESC[…m` / `U+009B…m` colour sequence. */
|
|
@@ -424,3 +450,51 @@ export function scanAnsi(text) {
|
|
|
424
450
|
}
|
|
425
451
|
return tokens;
|
|
426
452
|
}
|
|
453
|
+
|
|
454
|
+
/**
|
|
455
|
+
* The CONCEAL state after the SGR token `token` is applied to a terminal already
|
|
456
|
+
* in state `concealed` — the state in which a terminal renders what follows as
|
|
457
|
+
* blank while its bytes stay readable to anything reading the file.
|
|
458
|
+
*
|
|
459
|
+
* A TRANSITION, not a property of the token: conceal is terminal state that
|
|
460
|
+
* outlives the sequence that set it, so `ESC[8m` followed by `ESC[31m` is still
|
|
461
|
+
* concealed — a per-token predicate would read the second token as "no conceal
|
|
462
|
+
* here" and lose the state. Only `8` (set), `28` (reveal) and `0` (reset all)
|
|
463
|
+
* move it, and the token's parameters are applied in order, so a later reveal or
|
|
464
|
+
* reset in the SAME token cancels an earlier `8`: `ESC[8;28;31m` renders red and
|
|
465
|
+
* visible.
|
|
466
|
+
*
|
|
467
|
+
* The parameters are read the way a terminal reads them rather than scanned for
|
|
468
|
+
* the digit. `38`/`48`/`58` take their colour arguments from the parameters that
|
|
469
|
+
* FOLLOW them in the semicolon form, so `ESC[38;5;8m` is bright-black foreground
|
|
470
|
+
* and its `8` is a palette index; a parameter carrying its arguments as ITU T.416
|
|
471
|
+
* sub-parameters (`38:5:8`) is self-contained, so only its head counts and
|
|
472
|
+
* nothing after it is consumed. A colour-space selector T.416 gives no argument
|
|
473
|
+
* count for is malformed, and a terminal that ignores the colour form still
|
|
474
|
+
* applies what follows it, so the scan CONTINUES from the next parameter rather
|
|
475
|
+
* than consuming arguments it cannot size — reading one parameter too many costs
|
|
476
|
+
* a token shape no emitter produces, while stopping there would hand
|
|
477
|
+
* `ESC[38;9;8m` a pass.
|
|
478
|
+
* @param {string} token a token {@link scanAnsi} classified {@link TOKEN_KIND.SGR}
|
|
479
|
+
* @param {boolean} concealed the state before this token
|
|
480
|
+
* @returns {boolean}
|
|
481
|
+
*/
|
|
482
|
+
export function sgrConcealState(token, concealed) {
|
|
483
|
+
// The parameter bytes: everything between the introducer (two characters in
|
|
484
|
+
// the 7-bit `ESC [` form, one in the 8-bit C1 form) and the final `m`.
|
|
485
|
+
const params = token
|
|
486
|
+
.slice(token.charCodeAt(0) === ESC ? 2 : 1, -1)
|
|
487
|
+
.split(";");
|
|
488
|
+
let state = concealed;
|
|
489
|
+
for (let i = 0; i < params.length; i++) {
|
|
490
|
+
const colonForm = params[i].includes(":");
|
|
491
|
+
// An omitted parameter is 0 (`ESC[m` is `ESC[0m`), which `Number("")` gives.
|
|
492
|
+
const value = Number(params[i].split(":")[0]);
|
|
493
|
+
if (value === SGR_CONCEAL) state = true;
|
|
494
|
+
else if (value === SGR_REVEAL || value === SGR_RESET) state = false;
|
|
495
|
+
if (colonForm || !SGR_EXTENDED_COLOUR.has(value)) continue;
|
|
496
|
+
const args = SGR_COLOUR_ARGS.get(Number(params[i + 1]));
|
|
497
|
+
if (args !== undefined) i += 1 + args;
|
|
498
|
+
}
|
|
499
|
+
return state;
|
|
500
|
+
}
|
package/src/index.mjs
CHANGED
package/src/layer1.mjs
CHANGED
|
@@ -21,12 +21,13 @@
|
|
|
21
21
|
* runs, and a consumer deriving offsets for redaction or view mapping wants that
|
|
22
22
|
* one — see its own doc for why the choice matters.
|
|
23
23
|
*/
|
|
24
|
-
import { stripInvisibleWithReport, CATEGORY } from "./invisible.mjs";
|
|
24
|
+
import { stripInvisibleWithReport, CATEGORY, STRIP } from "./invisible.mjs";
|
|
25
25
|
import {
|
|
26
26
|
CONTROL_INTRODUCER_SOURCE,
|
|
27
27
|
isOrphanKind,
|
|
28
28
|
orphanKindFor,
|
|
29
29
|
scanAnsi,
|
|
30
|
+
sgrConcealState,
|
|
30
31
|
TOKEN_KIND,
|
|
31
32
|
} from "./ansi.mjs";
|
|
32
33
|
|
|
@@ -172,6 +173,79 @@ export function isBenignAnsiKinds(kinds) {
|
|
|
172
173
|
);
|
|
173
174
|
}
|
|
174
175
|
|
|
176
|
+
/**
|
|
177
|
+
* How many SGR sequences may sit back to back, with nothing that RENDERS between
|
|
178
|
+
* them, before the run is read as a covert channel rather than styling.
|
|
179
|
+
*
|
|
180
|
+
* A styling emitter puts colour AROUND text: `ls --color`, chalk and pygments
|
|
181
|
+
* chain at most a handful of attributes before the glyphs they style. A run of
|
|
182
|
+
* escape sequences that puts no glyph on the screen renders as literally nothing
|
|
183
|
+
* to a human while a model reads every byte, so past some length the run is not
|
|
184
|
+
* styling any more — it is a message, in the same shape (and for the same
|
|
185
|
+
* reason) as the invisible layer's LONG_RUN_THRESHOLD. Ten matches that
|
|
186
|
+
* threshold and sits well above what a real emitter chains.
|
|
187
|
+
*/
|
|
188
|
+
export const SGR_RUN_THRESHOLD = 10;
|
|
189
|
+
|
|
190
|
+
/**
|
|
191
|
+
* True when nothing in `gap` puts a glyph on the screen — the one primitive both
|
|
192
|
+
* arms of {@link sgrCarriesPayload} ask about. {@link STRIP} is the engine's
|
|
193
|
+
* single definition of a character with no visible presence, so a zero-width
|
|
194
|
+
* separator between two sequences is transparent here exactly as it is to a
|
|
195
|
+
* terminal, while a SPACE is not.
|
|
196
|
+
* @param {string} gap
|
|
197
|
+
* @returns {boolean}
|
|
198
|
+
*/
|
|
199
|
+
function rendersNothing(gap) {
|
|
200
|
+
return gap.replace(STRIP, "") === "";
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
/**
|
|
204
|
+
* True when the SGR sequences in `text` do more than style visible text — the
|
|
205
|
+
* question a consumer must answer before PRESERVING escapes rather than
|
|
206
|
+
* stripping them. Two arms, both model-sees/human-sees divergences:
|
|
207
|
+
*
|
|
208
|
+
* 1. CONCEAL is on while text that would otherwise render goes by, or the text
|
|
209
|
+
* ENDS concealed (hiding its own tail, and everything the terminal prints
|
|
210
|
+
* afterwards). Only SGR moves that state (see {@link sgrConcealState}) — a
|
|
211
|
+
* cursor move or an erase leaves a terminal exactly as concealed as it was.
|
|
212
|
+
* 2. {@link SGR_RUN_THRESHOLD} sequences with nothing that renders between
|
|
213
|
+
* them: the run puts no glyph on the screen, so it carries data rather than
|
|
214
|
+
* styling.
|
|
215
|
+
*
|
|
216
|
+
* What it does NOT claim: a message spread thinly through legitimately coloured
|
|
217
|
+
* text — one sequence per line, say — is indistinguishable from styling, and
|
|
218
|
+
* this answers no for it. That is the same bargain the invisible layer's
|
|
219
|
+
* thresholds strike, and the same direction: an unprovable payload is left alone
|
|
220
|
+
* rather than costing every colourized file its colour.
|
|
221
|
+
*
|
|
222
|
+
* Scans the RAW text in one pass, which is what a terminal does with these
|
|
223
|
+
* bytes — no reconstitution across passes, because a preserved byte is never
|
|
224
|
+
* removed to complete a sequence around it.
|
|
225
|
+
* @param {string} text
|
|
226
|
+
* @returns {boolean}
|
|
227
|
+
*/
|
|
228
|
+
export function sgrCarriesPayload(text) {
|
|
229
|
+
let concealed = false;
|
|
230
|
+
let run = 0;
|
|
231
|
+
let previousEnd = 0;
|
|
232
|
+
for (const token of scanAnsi(text)) {
|
|
233
|
+
const blankGap = rendersNothing(text.slice(previousEnd, token.start));
|
|
234
|
+
previousEnd = token.end;
|
|
235
|
+
if (token.kind !== TOKEN_KIND.SGR) {
|
|
236
|
+
run = 0;
|
|
237
|
+
continue;
|
|
238
|
+
}
|
|
239
|
+
// This refusal is what blocks `ESC[8m` from hiding text from the human
|
|
240
|
+
// while leaving its bytes for the model.
|
|
241
|
+
if (concealed && !blankGap) return true;
|
|
242
|
+
run = blankGap ? run + 1 : 1;
|
|
243
|
+
if (run >= SGR_RUN_THRESHOLD) return true;
|
|
244
|
+
concealed = sgrConcealState(text.slice(token.start, token.end), concealed);
|
|
245
|
+
}
|
|
246
|
+
return concealed;
|
|
247
|
+
}
|
|
248
|
+
|
|
175
249
|
/**
|
|
176
250
|
* {@link isBenignAnsiKinds} for callers that hold only the text — it runs the
|
|
177
251
|
* full Layer-1 composition to get the fixed-point view. Callers that already
|
package/types/ansi.d.mts
CHANGED
|
@@ -38,6 +38,35 @@ export function orphanKindFor(ch: string, next?: string): string;
|
|
|
38
38
|
* @returns {AnsiToken[]}
|
|
39
39
|
*/
|
|
40
40
|
export function scanAnsi(text: string): AnsiToken[];
|
|
41
|
+
/**
|
|
42
|
+
* The CONCEAL state after the SGR token `token` is applied to a terminal already
|
|
43
|
+
* in state `concealed` — the state in which a terminal renders what follows as
|
|
44
|
+
* blank while its bytes stay readable to anything reading the file.
|
|
45
|
+
*
|
|
46
|
+
* A TRANSITION, not a property of the token: conceal is terminal state that
|
|
47
|
+
* outlives the sequence that set it, so `ESC[8m` followed by `ESC[31m` is still
|
|
48
|
+
* concealed — a per-token predicate would read the second token as "no conceal
|
|
49
|
+
* here" and lose the state. Only `8` (set), `28` (reveal) and `0` (reset all)
|
|
50
|
+
* move it, and the token's parameters are applied in order, so a later reveal or
|
|
51
|
+
* reset in the SAME token cancels an earlier `8`: `ESC[8;28;31m` renders red and
|
|
52
|
+
* visible.
|
|
53
|
+
*
|
|
54
|
+
* The parameters are read the way a terminal reads them rather than scanned for
|
|
55
|
+
* the digit. `38`/`48`/`58` take their colour arguments from the parameters that
|
|
56
|
+
* FOLLOW them in the semicolon form, so `ESC[38;5;8m` is bright-black foreground
|
|
57
|
+
* and its `8` is a palette index; a parameter carrying its arguments as ITU T.416
|
|
58
|
+
* sub-parameters (`38:5:8`) is self-contained, so only its head counts and
|
|
59
|
+
* nothing after it is consumed. A colour-space selector T.416 gives no argument
|
|
60
|
+
* count for is malformed, and a terminal that ignores the colour form still
|
|
61
|
+
* applies what follows it, so the scan CONTINUES from the next parameter rather
|
|
62
|
+
* than consuming arguments it cannot size — reading one parameter too many costs
|
|
63
|
+
* a token shape no emitter produces, while stopping there would hand
|
|
64
|
+
* `ESC[38;9;8m` a pass.
|
|
65
|
+
* @param {string} token a token {@link scanAnsi} classified {@link TOKEN_KIND.SGR}
|
|
66
|
+
* @param {boolean} concealed the state before this token
|
|
67
|
+
* @returns {boolean}
|
|
68
|
+
*/
|
|
69
|
+
export function sgrConcealState(token: string, concealed: boolean): boolean;
|
|
41
70
|
/**
|
|
42
71
|
* The ONE ANSI grammar: the raw control-introducer charset and the tokenizer
|
|
43
72
|
* every consumer scans with.
|
package/types/index.d.mts
CHANGED
|
@@ -58,6 +58,6 @@ export function sanitize(text: string, options?: {
|
|
|
58
58
|
original: string;
|
|
59
59
|
}>;
|
|
60
60
|
}>;
|
|
61
|
-
export { applyLayer1, applyLayer1WellFormed, isBenignAnsi, isBenignAnsiKinds, normalizeLoneSurrogates, stripAnsiFully, LONE_SURROGATE_RE } from "./layer1.mjs";
|
|
61
|
+
export { applyLayer1, applyLayer1WellFormed, isBenignAnsi, isBenignAnsiKinds, normalizeLoneSurrogates, stripAnsiFully, sgrCarriesPayload, SGR_RUN_THRESHOLD, LONE_SURROGATE_RE } from "./layer1.mjs";
|
|
62
62
|
export { stripInvisible, stripInvisibleWithReport, isSgrOnly, STRIP, SGR_RE, CHECKS, CATEGORY, CATEGORY_LABELS, LINGUISTIC_SCRIPTS, VS, BLANK_NON_CF, LONG_RUN_RE, LONG_RUN_THRESHOLD, SCATTERED_THRESHOLD, findLongRuns, hasLongRun } from "./invisible.mjs";
|
|
63
63
|
export { HTML_TAG_PRESENT, MD_LINK_HINT, SECRET_HINT, SECRET_HINT_EXT, matchesSecretHint } from "./gates.mjs";
|
package/types/layer1.d.mts
CHANGED
|
@@ -49,6 +49,32 @@ export function stripAnsiFully(input: string, kinds?: Set<string>): string;
|
|
|
49
49
|
* @returns {boolean}
|
|
50
50
|
*/
|
|
51
51
|
export function isBenignAnsiKinds(kinds: readonly string[] | Set<string>): boolean;
|
|
52
|
+
/**
|
|
53
|
+
* True when the SGR sequences in `text` do more than style visible text — the
|
|
54
|
+
* question a consumer must answer before PRESERVING escapes rather than
|
|
55
|
+
* stripping them. Two arms, both model-sees/human-sees divergences:
|
|
56
|
+
*
|
|
57
|
+
* 1. CONCEAL is on while text that would otherwise render goes by, or the text
|
|
58
|
+
* ENDS concealed (hiding its own tail, and everything the terminal prints
|
|
59
|
+
* afterwards). Only SGR moves that state (see {@link sgrConcealState}) — a
|
|
60
|
+
* cursor move or an erase leaves a terminal exactly as concealed as it was.
|
|
61
|
+
* 2. {@link SGR_RUN_THRESHOLD} sequences with nothing that renders between
|
|
62
|
+
* them: the run puts no glyph on the screen, so it carries data rather than
|
|
63
|
+
* styling.
|
|
64
|
+
*
|
|
65
|
+
* What it does NOT claim: a message spread thinly through legitimately coloured
|
|
66
|
+
* text — one sequence per line, say — is indistinguishable from styling, and
|
|
67
|
+
* this answers no for it. That is the same bargain the invisible layer's
|
|
68
|
+
* thresholds strike, and the same direction: an unprovable payload is left alone
|
|
69
|
+
* rather than costing every colourized file its colour.
|
|
70
|
+
*
|
|
71
|
+
* Scans the RAW text in one pass, which is what a terminal does with these
|
|
72
|
+
* bytes — no reconstitution across passes, because a preserved byte is never
|
|
73
|
+
* removed to complete a sequence around it.
|
|
74
|
+
* @param {string} text
|
|
75
|
+
* @returns {boolean}
|
|
76
|
+
*/
|
|
77
|
+
export function sgrCarriesPayload(text: string): boolean;
|
|
52
78
|
/**
|
|
53
79
|
* {@link isBenignAnsiKinds} for callers that hold only the text — it runs the
|
|
54
80
|
* full Layer-1 composition to get the fixed-point view. Callers that already
|
|
@@ -147,3 +173,16 @@ export const LONE_SURROGATE_RE: RegExp;
|
|
|
147
173
|
* colour codes, and here is how to look at the raw bytes".
|
|
148
174
|
*/
|
|
149
175
|
export const INERT_ANSI_NOTE: string;
|
|
176
|
+
/**
|
|
177
|
+
* How many SGR sequences may sit back to back, with nothing that RENDERS between
|
|
178
|
+
* them, before the run is read as a covert channel rather than styling.
|
|
179
|
+
*
|
|
180
|
+
* A styling emitter puts colour AROUND text: `ls --color`, chalk and pygments
|
|
181
|
+
* chain at most a handful of attributes before the glyphs they style. A run of
|
|
182
|
+
* escape sequences that puts no glyph on the screen renders as literally nothing
|
|
183
|
+
* to a human while a model reads every byte, so past some length the run is not
|
|
184
|
+
* styling any more — it is a message, in the same shape (and for the same
|
|
185
|
+
* reason) as the invisible layer's LONG_RUN_THRESHOLD. Ten matches that
|
|
186
|
+
* threshold and sits well above what a real emitter chains.
|
|
187
|
+
*/
|
|
188
|
+
export const SGR_RUN_THRESHOLD: 10;
|