agent-sanitizer 2.54.2 → 2.56.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -2
- package/THREAT-MODEL.md +45 -0
- package/claude-hooks/lib/authored-content.mjs +82 -29
- package/claude-hooks/lib/env-config.mjs +18 -0
- package/claude-hooks/sanitize-output.mjs +5 -1
- package/package.json +1 -1
- package/src/ansi.mjs +74 -0
- package/src/index.mjs +2 -0
- package/src/layer1.mjs +75 -1
- package/types/ansi.d.mts +29 -0
- package/types/claude-hooks/lib/env-config.d.mts +9 -0
- package/types/index.d.mts +1 -1
- package/types/layer1.d.mts +39 -0
package/README.md
CHANGED
|
@@ -164,7 +164,12 @@ buy you:
|
|
|
164
164
|
3. Look-alike glyphs in tool inputs are folded to ASCII, so a Cyrillic `а` can't
|
|
165
165
|
walk a command past a deny rule.
|
|
166
166
|
4. Tool output has invisible characters and terminal escapes stripped, hidden
|
|
167
|
-
HTML spliced out with a placeholder, and exfil-shaped URLs flagged.
|
|
167
|
+
HTML spliced out with a placeholder, and exfil-shaped URLs flagged. A hex
|
|
168
|
+
value one digest wide is exempt under a generic query or fragment parameter
|
|
169
|
+
name, because a commit or blob id in a link is ordinary — under a name that
|
|
170
|
+
already says credential it still flags, and a path segment never reaches the
|
|
171
|
+
exemption at all. `AGENT_SANITIZER_FLAG_DIGEST_VALUES=1` turns it off for a
|
|
172
|
+
consumer that reads such a value as a leak.
|
|
168
173
|
5. With `AGENT_SANITIZER_SECRETS_ENABLED=1` set, secrets in tool output are
|
|
169
174
|
redacted locally by `detect-secrets` — the engine ships with the plugin and
|
|
170
175
|
provisions itself on first run, no further setup from you.
|
|
@@ -279,7 +284,7 @@ singleton, and two copies in one bundle double-fire the inlined CLIs.
|
|
|
279
284
|
| `claude-hooks/scan-loaded-instructions` | InstructionsLoaded scan of each instruction file as Claude Code loads it |
|
|
280
285
|
| `claude-hooks/lib/hook-io` | Shared hook I/O: the lazy-module registry, the CLI slot, deadlines, the hookgate marker |
|
|
281
286
|
| `claude-hooks/lib/control-plane` | Bridge to `agent-control-plane-core` and the shared judge-CLI transport |
|
|
282
|
-
| `claude-hooks/lib/authored-content` | Stego + terminal-control stripping of the fields the MODEL authors
|
|
287
|
+
| `claude-hooks/lib/authored-content` | Stego + terminal-control stripping of the fields the MODEL authors (colour is kept) |
|
|
283
288
|
| `claude-hooks/lib/env-config` | The env-bound secret vocabulary the Layer-4 pre-gate and the redactor client share |
|
|
284
289
|
| `claude-hooks/lib/invisible-alert` | Cross-hook alert state for uncleanable invisible-char injection in instruction files |
|
|
285
290
|
| `claude-hooks/lib/redactor-client` | Client for the long-lived `agent-secret-redactor-daemon` (Layer 4's transport) |
|
package/THREAT-MODEL.md
CHANGED
|
@@ -290,6 +290,51 @@ reconstitute a secret the first pass never saw intact. On `Bash.command` it runs
|
|
|
290
290
|
before `sanitizeAuthoredContent`, which is the assumption the confusable-folding
|
|
291
291
|
soundness argument below relies on.
|
|
292
292
|
|
|
293
|
+
## Model-authored writes (tool input)
|
|
294
|
+
|
|
295
|
+
`claude-hooks/lib/authored-content` scrubs what the model EMITS — the file
|
|
296
|
+
content, edits, notebook cells and command bodies that get persisted or executed
|
|
297
|
+
— rather than what it reads. Two protections, with different gates. Stego
|
|
298
|
+
(format characters, variation selectors) is stripped only when the volume can
|
|
299
|
+
carry a message, because authored content is persisted and incidental joiners
|
|
300
|
+
are legitimate. Terminal-control sequences are stripped on sight: one cursor
|
|
301
|
+
move, erase or OSC string in a file is a latent bomb the next `cat` fires, and
|
|
302
|
+
one in a command spoofs what the user is shown as it runs.
|
|
303
|
+
|
|
304
|
+
Display-only SGR colour is the exception, and is preserved byte-for-byte. The
|
|
305
|
+
SGR grammar is closed — parameters and a final `m` — so it restyles visible text
|
|
306
|
+
and can do nothing else; stripping it costs a model the colourized fixture, TUI
|
|
307
|
+
golden file or prompt string it deliberately wrote, and buys no safety. Two
|
|
308
|
+
shapes end the exception, both of them the model-sees/human-sees divergence
|
|
309
|
+
rather than a display choice: a CONCEAL parameter (`ESC[8m`), which blanks the
|
|
310
|
+
text that follows for a human while its bytes stay readable to a model, and a
|
|
311
|
+
run of sequences with nothing that RENDERS between them, which puts no glyph on
|
|
312
|
+
the screen at all and is therefore carrying data rather than colour. Either one
|
|
313
|
+
strips the whole field.
|
|
314
|
+
|
|
315
|
+
Both shapes are read as terminal semantics rather than token shapes, because a
|
|
316
|
+
reader that matches shapes is one encoding away from the wrong answer. Conceal
|
|
317
|
+
is threaded as STATE, so a reveal or reset later in the same sequence cancels it
|
|
318
|
+
(`ESC[8;28;31m` renders red and visible) while an erase between two sequences
|
|
319
|
+
does not. The parameters are read the way a terminal reads them, so an
|
|
320
|
+
extended-colour argument is never mistaken for a parameter — `ESC[38;5;8m` is
|
|
321
|
+
bright-black foreground, not conceal. And the run is counted over characters
|
|
322
|
+
that render, so a zero-width separator between two sequences is as transparent
|
|
323
|
+
here as it is to a terminal, while a space — which is what an ANSI-art colour
|
|
324
|
+
bar puts between its codes — breaks the run.
|
|
325
|
+
|
|
326
|
+
What that exception does NOT claim: a message spread thinly through
|
|
327
|
+
legitimately coloured text — a sequence per line, encoding a bit — is
|
|
328
|
+
indistinguishable from styling, and is left alone. That is the same bargain the
|
|
329
|
+
invisible-character thresholds strike, in the same direction: an unprovable
|
|
330
|
+
payload costs a false negative, while a wrong strip costs every colourized file
|
|
331
|
+
its colour.
|
|
332
|
+
|
|
333
|
+
Both protections run to a fixed point over each field rather than once each,
|
|
334
|
+
because they feed one another: removing an invisible character completes a
|
|
335
|
+
sequence that was incomplete when the terminal stage ran (`ESC[`⟨ZWSP⟩`2J`), and
|
|
336
|
+
removing a sequence makes invisibles adjacent that were not.
|
|
337
|
+
|
|
293
338
|
## Confusable folding (tool input)
|
|
294
339
|
|
|
295
340
|
`./confusables` folds look-alike glyphs in tool-call **input** fields (paths,
|
|
@@ -15,11 +15,17 @@
|
|
|
15
15
|
* authored into a command — echoed and executed live — or into file
|
|
16
16
|
* content (a latent bomb when the file is later `cat`'d) can clear the
|
|
17
17
|
* screen, reposition the cursor, or overwrite what the user sees, hiding
|
|
18
|
-
* the real command behind spoofed output.
|
|
19
|
-
*
|
|
20
|
-
*
|
|
21
|
-
*
|
|
22
|
-
*
|
|
18
|
+
* the real command behind spoofed output. A single such sequence already
|
|
19
|
+
* does harm, so there is no volume threshold: one of them strips the whole
|
|
20
|
+
* field. Display-only SGR colour is the exception and is PRESERVED
|
|
21
|
+
* byte-for-byte — it restyles text and can neither move the cursor nor
|
|
22
|
+
* carry an OSC payload, so stripping it only costs a model the colourized
|
|
23
|
+
* fixture, TUI golden file or prompt string it deliberately wrote. The
|
|
24
|
+
* carve-out ends where SGR stops being styling (see the engine's
|
|
25
|
+
* sgrCarriesPayload): a CONCEAL parameter blanks text for a human while a
|
|
26
|
+
* model still reads it, and a long run of sequences with nothing that
|
|
27
|
+
* renders between them puts no glyph on the screen at all, which is a
|
|
28
|
+
* covert channel rather than colour.
|
|
23
29
|
*
|
|
24
30
|
* SCOPE IS DECLARED, NOT INFERRED. Which tools this layer touches is a
|
|
25
31
|
* partition — {@link AUTHORED_FIELDS} (covered, with the field list) and
|
|
@@ -50,9 +56,10 @@ import { lazyImport } from "./hook-io.mjs";
|
|
|
50
56
|
// static import of this module, before its fail-closed catch runs). A failed
|
|
51
57
|
// load leaves these bindings undefined, so sanitizeField's calls throw into
|
|
52
58
|
// the fail-closed catch (ask) instead.
|
|
53
|
-
const { stripAnsiFully
|
|
54
|
-
|
|
55
|
-
)
|
|
59
|
+
const { stripAnsiFully, isBenignAnsiKinds, sgrCarriesPayload } =
|
|
60
|
+
/** @type {typeof import("agent-sanitizer")} */ (
|
|
61
|
+
await lazyImport("agent-sanitizer")
|
|
62
|
+
);
|
|
56
63
|
const { STRIP, SCATTERED_THRESHOLD, hasLongRun, stripInvisible } =
|
|
57
64
|
/** @type {typeof import("agent-sanitizer/invisible")} */ (
|
|
58
65
|
await lazyImport("agent-sanitizer/invisible")
|
|
@@ -147,38 +154,84 @@ function isPayloadCapable(text) {
|
|
|
147
154
|
return (text.match(STRIP)?.length ?? 0) >= SCATTERED_THRESHOLD;
|
|
148
155
|
}
|
|
149
156
|
|
|
157
|
+
/**
|
|
158
|
+
* The de-ANSI'd `text`, or `text` itself when its escapes are display-only
|
|
159
|
+
* colour a reader authored on purpose.
|
|
160
|
+
*
|
|
161
|
+
* The two questions are answered by two different views, deliberately.
|
|
162
|
+
* `isBenignAnsiKinds` reads the kinds the strip actually REMOVED, so a sequence
|
|
163
|
+
* that only reconstitutes mid-strip is judged as the sequence it becomes rather
|
|
164
|
+
* than as the incomplete one it started as. `sgrCarriesPayload` reads the RAW
|
|
165
|
+
* bytes, because those are the bytes a terminal renders on the preserve path —
|
|
166
|
+
* it makes no second pass, so there is nothing to reconstitute there.
|
|
167
|
+
*
|
|
168
|
+
* Only COMPLETE tokens reach `kinds`: an introducer that opens no sequence is
|
|
169
|
+
* left in place by `stripAnsiFully` (it rewrites no display), so the orphan arms
|
|
170
|
+
* of `isBenignAnsiKinds` never decide anything here.
|
|
171
|
+
* @param {string} text
|
|
172
|
+
* @returns {string}
|
|
173
|
+
*/
|
|
174
|
+
function stripTerminalControls(text) {
|
|
175
|
+
/** @type {Set<string>} TOKEN_KINDs the strip removed. */
|
|
176
|
+
const kinds = new Set();
|
|
177
|
+
const deAnsi = stripAnsiFully(text, kinds);
|
|
178
|
+
if (deAnsi === text) return text;
|
|
179
|
+
if (isBenignAnsiKinds(kinds) && !sgrCarriesPayload(text)) return text;
|
|
180
|
+
return deAnsi;
|
|
181
|
+
}
|
|
182
|
+
|
|
183
|
+
// How many rounds of {terminal-control, invisible} the field may take. The two
|
|
184
|
+
// protections FEED each other: removing invisibles completes a sequence that was
|
|
185
|
+
// incomplete when the terminal stage ran (`ESC[` ZWSP `2J` is an orphan until the
|
|
186
|
+
// ZWSP goes), and removing a sequence makes invisibles adjacent that were not.
|
|
187
|
+
// One round of each therefore leaves the composition unfinished. Each round
|
|
188
|
+
// deletes at least one character, so the loop converges on its own; the bound is
|
|
189
|
+
// a DoS guard. No residual sweep stands behind it — unlike the engine's
|
|
190
|
+
// MAX_LAYER1_PASSES — so an unconverged value would persist a live sequence into
|
|
191
|
+
// the file, and reaching the bound while still changing throws instead.
|
|
192
|
+
const MAX_FIELD_PASSES = 3;
|
|
193
|
+
|
|
150
194
|
// Returns the cleaned value plus the human-readable actions applied, or null if
|
|
151
195
|
// the field is already clean. Each protection has its own opt-out (see the
|
|
152
196
|
// header) so a deployment can keep one while dropping the other.
|
|
153
197
|
/** @param {string} value */
|
|
154
198
|
function sanitizeField(value) {
|
|
155
|
-
|
|
199
|
+
// A Set, so a protection that fires in more than one round is described once.
|
|
200
|
+
/** @type {Set<string>} */
|
|
201
|
+
const actions = new Set();
|
|
156
202
|
let cleaned = value;
|
|
157
203
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
204
|
+
for (let pass = 0; pass < MAX_FIELD_PASSES; pass++) {
|
|
205
|
+
const before = cleaned;
|
|
206
|
+
// Terminal controls first, so the invisible scan runs on the same de-ANSI'd
|
|
207
|
+
// view sanitize-output uses. Compare before/after rather than pre-testing
|
|
208
|
+
// for ESC: escapes this layer deliberately leaves in place — a lone control
|
|
209
|
+
// byte, display-only colour — must not be reported as a strip.
|
|
210
|
+
if (process.env.AGENT_SANITIZER_TERMINAL_DISABLED !== "1") {
|
|
211
|
+
const deAnsi = stripTerminalControls(cleaned);
|
|
212
|
+
if (deAnsi !== cleaned) {
|
|
213
|
+
cleaned = deAnsi;
|
|
214
|
+
actions.add("terminal-control sequences");
|
|
215
|
+
}
|
|
170
216
|
}
|
|
171
|
-
}
|
|
172
217
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
218
|
+
if (
|
|
219
|
+
process.env.AGENT_SANITIZER_INVISIBLE_DISABLED !== "1" &&
|
|
220
|
+
isPayloadCapable(cleaned)
|
|
221
|
+
) {
|
|
222
|
+
cleaned = stripInvisible(cleaned);
|
|
223
|
+
actions.add("invisible characters");
|
|
224
|
+
}
|
|
225
|
+
if (cleaned === before) break;
|
|
226
|
+
// This refusal is what blocks a still-changing value — one whose next round
|
|
227
|
+
// would remove another live sequence — from being written to the file.
|
|
228
|
+
if (pass === MAX_FIELD_PASSES - 1)
|
|
229
|
+
throw new Error(
|
|
230
|
+
"authored content did not reach a fixed point within MAX_FIELD_PASSES",
|
|
231
|
+
);
|
|
179
232
|
}
|
|
180
233
|
|
|
181
|
-
return actions.
|
|
234
|
+
return actions.size > 0 ? { cleaned, actions: [...actions] } : null;
|
|
182
235
|
}
|
|
183
236
|
|
|
184
237
|
/** @param {string[]} changed */
|
|
@@ -238,6 +238,24 @@ export function secretsEnabled(env = process.env) {
|
|
|
238
238
|
return env[SECRETS_ENABLED_ENV] === "1";
|
|
239
239
|
}
|
|
240
240
|
|
|
241
|
+
// Layer 3's digest exemption, which lets a URL carry an exact-digest-length hex
|
|
242
|
+
// value without being read as a payload. A consumer that treats a digest in a
|
|
243
|
+
// URL as exfil — a monitor reading tool output for a leaked commit or blob id —
|
|
244
|
+
// turns the exemption off here. `checkExfilUrl` takes the same option directly;
|
|
245
|
+
// this is the switch for the hook path, which composes its own seam options.
|
|
246
|
+
export const FLAG_DIGEST_VALUES_ENV = "AGENT_SANITIZER_FLAG_DIGEST_VALUES";
|
|
247
|
+
|
|
248
|
+
/**
|
|
249
|
+
* True when the operator opted into flagging digest-shaped URL values. `=== "1"`
|
|
250
|
+
* for the reason {@link secretsEnabled} gives: a typo fails toward the default,
|
|
251
|
+
* which is the exemption the precision rule prefers.
|
|
252
|
+
* @param {NodeJS.ProcessEnv | Record<string, string | undefined>} [env]
|
|
253
|
+
* @returns {boolean}
|
|
254
|
+
*/
|
|
255
|
+
export function digestFlaggingEnabled(env = process.env) {
|
|
256
|
+
return env[FLAG_DIGEST_VALUES_ENV] === "1";
|
|
257
|
+
}
|
|
258
|
+
|
|
241
259
|
/**
|
|
242
260
|
* The env-bound redaction set: the UNION of the inference keys, the curated host
|
|
243
261
|
* credentials, any credential-shaped var present in the environment, the
|
|
@@ -49,7 +49,7 @@ import { registerFaultPolicy, hookFaultOutcome } from "./lib/hook-fault.mjs";
|
|
|
49
49
|
import { controlPlane, runJudgeCli } from "./lib/control-plane.mjs";
|
|
50
50
|
import { bestEffortTrace, trace, TraceEvent } from "./lib/trace.mjs";
|
|
51
51
|
import { hasEnvBoundSecret } from "./lib/secret-annotate.mjs";
|
|
52
|
-
import { secretsEnabled } from "./lib/env-config.mjs";
|
|
52
|
+
import { digestFlaggingEnabled, secretsEnabled } from "./lib/env-config.mjs";
|
|
53
53
|
import {
|
|
54
54
|
persistReveal,
|
|
55
55
|
persistSpan,
|
|
@@ -267,6 +267,10 @@ export async function sanitizeText(
|
|
|
267
267
|
const seamOptions = {
|
|
268
268
|
html,
|
|
269
269
|
exfilScan: webIngress,
|
|
270
|
+
// Widens Layer 3 only, and only where the operator asked: without it the
|
|
271
|
+
// option the seam already accepts is unreachable from this entry, so a hook
|
|
272
|
+
// consumer that needs a digest in a URL flagged has no way to ask for it.
|
|
273
|
+
flagDigestValues: digestFlaggingEnabled(),
|
|
270
274
|
sgrCarveOut: !webIngress,
|
|
271
275
|
deadline,
|
|
272
276
|
// Layer 4 — OPT-IN (secretsEnabled): with the knob unset the seam gets no
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-sanitizer",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.56.0",
|
|
4
4
|
"description": "Defend an agent against hidden-content injection: strip payload-capable invisible Unicode and ANSI, splice out human-invisible HTML, and flag data-exfil URLs in untrusted text before any model sees it.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|
package/src/ansi.mjs
CHANGED
|
@@ -205,6 +205,32 @@ export const ESCAPE_SEQUENCE_SOURCE = (() => {
|
|
|
205
205
|
return `(?:${stringArm}|${csiArm})`;
|
|
206
206
|
})();
|
|
207
207
|
|
|
208
|
+
// The three SGR parameters that move the CONCEAL state (ECMA-48 § 8.3.117).
|
|
209
|
+
// Conceal renders the text that follows as blank while its bytes stay in the
|
|
210
|
+
// file, so a human reading with `cat` sees nothing where a model reads
|
|
211
|
+
// everything — a model-sees/human-sees divergence rather than a display choice.
|
|
212
|
+
const SGR_RESET = 0;
|
|
213
|
+
const SGR_CONCEAL = 8;
|
|
214
|
+
const SGR_REVEAL = 28;
|
|
215
|
+
// The extended-colour selectors. Their ARGUMENTS follow as further semicolon
|
|
216
|
+
// parameters, so a reader that does not consume them reads a colour component as
|
|
217
|
+
// a parameter of its own: `ESC[38;5;8m` is bright-black foreground, not conceal.
|
|
218
|
+
const SGR_EXTENDED_COLOUR = new Set([38, 48, 58]);
|
|
219
|
+
// How many parameters an extended-colour selector consumes after itself, keyed
|
|
220
|
+
// by the colour-space selector that follows it, per ITU T.416 § 13.1.8: the
|
|
221
|
+
// implementation-defined and transparent forms take none, direct-colour RGB and
|
|
222
|
+
// CMY take three components, CMYK four, and indexed takes one palette entry.
|
|
223
|
+
// Every selector the spec defines, so a value missing from this table is
|
|
224
|
+
// malformed rather than merely unimplemented.
|
|
225
|
+
const SGR_COLOUR_ARGS = new Map([
|
|
226
|
+
[0, 0],
|
|
227
|
+
[1, 0],
|
|
228
|
+
[2, 3],
|
|
229
|
+
[3, 3],
|
|
230
|
+
[4, 4],
|
|
231
|
+
[5, 1],
|
|
232
|
+
]);
|
|
233
|
+
|
|
208
234
|
/** The seven things an introducer can turn out to be. */
|
|
209
235
|
export const TOKEN_KIND = Object.freeze({
|
|
210
236
|
/** A display-only `ESC[…m` / `U+009B…m` colour sequence. */
|
|
@@ -424,3 +450,51 @@ export function scanAnsi(text) {
|
|
|
424
450
|
}
|
|
425
451
|
return tokens;
|
|
426
452
|
}
|
|
453
|
+
|
|
454
|
+
/**
|
|
455
|
+
* The CONCEAL state after the SGR token `token` is applied to a terminal already
|
|
456
|
+
* in state `concealed` — the state in which a terminal renders what follows as
|
|
457
|
+
* blank while its bytes stay readable to anything reading the file.
|
|
458
|
+
*
|
|
459
|
+
* A TRANSITION, not a property of the token: conceal is terminal state that
|
|
460
|
+
* outlives the sequence that set it, so `ESC[8m` followed by `ESC[31m` is still
|
|
461
|
+
* concealed — a per-token predicate would read the second token as "no conceal
|
|
462
|
+
* here" and lose the state. Only `8` (set), `28` (reveal) and `0` (reset all)
|
|
463
|
+
* move it, and the token's parameters are applied in order, so a later reveal or
|
|
464
|
+
* reset in the SAME token cancels an earlier `8`: `ESC[8;28;31m` renders red and
|
|
465
|
+
* visible.
|
|
466
|
+
*
|
|
467
|
+
* The parameters are read the way a terminal reads them rather than scanned for
|
|
468
|
+
* the digit. `38`/`48`/`58` take their colour arguments from the parameters that
|
|
469
|
+
* FOLLOW them in the semicolon form, so `ESC[38;5;8m` is bright-black foreground
|
|
470
|
+
* and its `8` is a palette index; a parameter carrying its arguments as ITU T.416
|
|
471
|
+
* sub-parameters (`38:5:8`) is self-contained, so only its head counts and
|
|
472
|
+
* nothing after it is consumed. A colour-space selector T.416 gives no argument
|
|
473
|
+
* count for is malformed, and a terminal that ignores the colour form still
|
|
474
|
+
* applies what follows it, so the scan CONTINUES from the next parameter rather
|
|
475
|
+
* than consuming arguments it cannot size — reading one parameter too many costs
|
|
476
|
+
* a token shape no emitter produces, while stopping there would hand
|
|
477
|
+
* `ESC[38;9;8m` a pass.
|
|
478
|
+
* @param {string} token a token {@link scanAnsi} classified {@link TOKEN_KIND.SGR}
|
|
479
|
+
* @param {boolean} concealed the state before this token
|
|
480
|
+
* @returns {boolean}
|
|
481
|
+
*/
|
|
482
|
+
export function sgrConcealState(token, concealed) {
|
|
483
|
+
// The parameter bytes: everything between the introducer (two characters in
|
|
484
|
+
// the 7-bit `ESC [` form, one in the 8-bit C1 form) and the final `m`.
|
|
485
|
+
const params = token
|
|
486
|
+
.slice(token.charCodeAt(0) === ESC ? 2 : 1, -1)
|
|
487
|
+
.split(";");
|
|
488
|
+
let state = concealed;
|
|
489
|
+
for (let i = 0; i < params.length; i++) {
|
|
490
|
+
const colonForm = params[i].includes(":");
|
|
491
|
+
// An omitted parameter is 0 (`ESC[m` is `ESC[0m`), which `Number("")` gives.
|
|
492
|
+
const value = Number(params[i].split(":")[0]);
|
|
493
|
+
if (value === SGR_CONCEAL) state = true;
|
|
494
|
+
else if (value === SGR_REVEAL || value === SGR_RESET) state = false;
|
|
495
|
+
if (colonForm || !SGR_EXTENDED_COLOUR.has(value)) continue;
|
|
496
|
+
const args = SGR_COLOUR_ARGS.get(Number(params[i + 1]));
|
|
497
|
+
if (args !== undefined) i += 1 + args;
|
|
498
|
+
}
|
|
499
|
+
return state;
|
|
500
|
+
}
|
package/src/index.mjs
CHANGED
package/src/layer1.mjs
CHANGED
|
@@ -21,12 +21,13 @@
|
|
|
21
21
|
* runs, and a consumer deriving offsets for redaction or view mapping wants that
|
|
22
22
|
* one — see its own doc for why the choice matters.
|
|
23
23
|
*/
|
|
24
|
-
import { stripInvisibleWithReport, CATEGORY } from "./invisible.mjs";
|
|
24
|
+
import { stripInvisibleWithReport, CATEGORY, STRIP } from "./invisible.mjs";
|
|
25
25
|
import {
|
|
26
26
|
CONTROL_INTRODUCER_SOURCE,
|
|
27
27
|
isOrphanKind,
|
|
28
28
|
orphanKindFor,
|
|
29
29
|
scanAnsi,
|
|
30
|
+
sgrConcealState,
|
|
30
31
|
TOKEN_KIND,
|
|
31
32
|
} from "./ansi.mjs";
|
|
32
33
|
|
|
@@ -172,6 +173,79 @@ export function isBenignAnsiKinds(kinds) {
|
|
|
172
173
|
);
|
|
173
174
|
}
|
|
174
175
|
|
|
176
|
+
/**
|
|
177
|
+
* How many SGR sequences may sit back to back, with nothing that RENDERS between
|
|
178
|
+
* them, before the run is read as a covert channel rather than styling.
|
|
179
|
+
*
|
|
180
|
+
* A styling emitter puts colour AROUND text: `ls --color`, chalk and pygments
|
|
181
|
+
* chain at most a handful of attributes before the glyphs they style. A run of
|
|
182
|
+
* escape sequences that puts no glyph on the screen renders as literally nothing
|
|
183
|
+
* to a human while a model reads every byte, so past some length the run is not
|
|
184
|
+
* styling any more — it is a message, in the same shape (and for the same
|
|
185
|
+
* reason) as the invisible layer's LONG_RUN_THRESHOLD. Ten matches that
|
|
186
|
+
* threshold and sits well above what a real emitter chains.
|
|
187
|
+
*/
|
|
188
|
+
export const SGR_RUN_THRESHOLD = 10;
|
|
189
|
+
|
|
190
|
+
/**
|
|
191
|
+
* True when nothing in `gap` puts a glyph on the screen — the one primitive both
|
|
192
|
+
* arms of {@link sgrCarriesPayload} ask about. {@link STRIP} is the engine's
|
|
193
|
+
* single definition of a character with no visible presence, so a zero-width
|
|
194
|
+
* separator between two sequences is transparent here exactly as it is to a
|
|
195
|
+
* terminal, while a SPACE is not.
|
|
196
|
+
* @param {string} gap
|
|
197
|
+
* @returns {boolean}
|
|
198
|
+
*/
|
|
199
|
+
function rendersNothing(gap) {
|
|
200
|
+
return gap.replace(STRIP, "") === "";
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
/**
|
|
204
|
+
* True when the SGR sequences in `text` do more than style visible text — the
|
|
205
|
+
* question a consumer must answer before PRESERVING escapes rather than
|
|
206
|
+
* stripping them. Two arms, both model-sees/human-sees divergences:
|
|
207
|
+
*
|
|
208
|
+
* 1. CONCEAL is on while text that would otherwise render goes by, or the text
|
|
209
|
+
* ENDS concealed (hiding its own tail, and everything the terminal prints
|
|
210
|
+
* afterwards). Only SGR moves that state (see {@link sgrConcealState}) — a
|
|
211
|
+
* cursor move or an erase leaves a terminal exactly as concealed as it was.
|
|
212
|
+
* 2. {@link SGR_RUN_THRESHOLD} sequences with nothing that renders between
|
|
213
|
+
* them: the run puts no glyph on the screen, so it carries data rather than
|
|
214
|
+
* styling.
|
|
215
|
+
*
|
|
216
|
+
* What it does NOT claim: a message spread thinly through legitimately coloured
|
|
217
|
+
* text — one sequence per line, say — is indistinguishable from styling, and
|
|
218
|
+
* this answers no for it. That is the same bargain the invisible layer's
|
|
219
|
+
* thresholds strike, and the same direction: an unprovable payload is left alone
|
|
220
|
+
* rather than costing every colourized file its colour.
|
|
221
|
+
*
|
|
222
|
+
* Scans the RAW text in one pass, which is what a terminal does with these
|
|
223
|
+
* bytes — no reconstitution across passes, because a preserved byte is never
|
|
224
|
+
* removed to complete a sequence around it.
|
|
225
|
+
* @param {string} text
|
|
226
|
+
* @returns {boolean}
|
|
227
|
+
*/
|
|
228
|
+
export function sgrCarriesPayload(text) {
|
|
229
|
+
let concealed = false;
|
|
230
|
+
let run = 0;
|
|
231
|
+
let previousEnd = 0;
|
|
232
|
+
for (const token of scanAnsi(text)) {
|
|
233
|
+
const blankGap = rendersNothing(text.slice(previousEnd, token.start));
|
|
234
|
+
previousEnd = token.end;
|
|
235
|
+
if (token.kind !== TOKEN_KIND.SGR) {
|
|
236
|
+
run = 0;
|
|
237
|
+
continue;
|
|
238
|
+
}
|
|
239
|
+
// This refusal is what blocks `ESC[8m` from hiding text from the human
|
|
240
|
+
// while leaving its bytes for the model.
|
|
241
|
+
if (concealed && !blankGap) return true;
|
|
242
|
+
run = blankGap ? run + 1 : 1;
|
|
243
|
+
if (run >= SGR_RUN_THRESHOLD) return true;
|
|
244
|
+
concealed = sgrConcealState(text.slice(token.start, token.end), concealed);
|
|
245
|
+
}
|
|
246
|
+
return concealed;
|
|
247
|
+
}
|
|
248
|
+
|
|
175
249
|
/**
|
|
176
250
|
* {@link isBenignAnsiKinds} for callers that hold only the text — it runs the
|
|
177
251
|
* full Layer-1 composition to get the fixed-point view. Callers that already
|
package/types/ansi.d.mts
CHANGED
|
@@ -38,6 +38,35 @@ export function orphanKindFor(ch: string, next?: string): string;
|
|
|
38
38
|
* @returns {AnsiToken[]}
|
|
39
39
|
*/
|
|
40
40
|
export function scanAnsi(text: string): AnsiToken[];
|
|
41
|
+
/**
|
|
42
|
+
* The CONCEAL state after the SGR token `token` is applied to a terminal already
|
|
43
|
+
* in state `concealed` — the state in which a terminal renders what follows as
|
|
44
|
+
* blank while its bytes stay readable to anything reading the file.
|
|
45
|
+
*
|
|
46
|
+
* A TRANSITION, not a property of the token: conceal is terminal state that
|
|
47
|
+
* outlives the sequence that set it, so `ESC[8m` followed by `ESC[31m` is still
|
|
48
|
+
* concealed — a per-token predicate would read the second token as "no conceal
|
|
49
|
+
* here" and lose the state. Only `8` (set), `28` (reveal) and `0` (reset all)
|
|
50
|
+
* move it, and the token's parameters are applied in order, so a later reveal or
|
|
51
|
+
* reset in the SAME token cancels an earlier `8`: `ESC[8;28;31m` renders red and
|
|
52
|
+
* visible.
|
|
53
|
+
*
|
|
54
|
+
* The parameters are read the way a terminal reads them rather than scanned for
|
|
55
|
+
* the digit. `38`/`48`/`58` take their colour arguments from the parameters that
|
|
56
|
+
* FOLLOW them in the semicolon form, so `ESC[38;5;8m` is bright-black foreground
|
|
57
|
+
* and its `8` is a palette index; a parameter carrying its arguments as ITU T.416
|
|
58
|
+
* sub-parameters (`38:5:8`) is self-contained, so only its head counts and
|
|
59
|
+
* nothing after it is consumed. A colour-space selector T.416 gives no argument
|
|
60
|
+
* count for is malformed, and a terminal that ignores the colour form still
|
|
61
|
+
* applies what follows it, so the scan CONTINUES from the next parameter rather
|
|
62
|
+
* than consuming arguments it cannot size — reading one parameter too many costs
|
|
63
|
+
* a token shape no emitter produces, while stopping there would hand
|
|
64
|
+
* `ESC[38;9;8m` a pass.
|
|
65
|
+
* @param {string} token a token {@link scanAnsi} classified {@link TOKEN_KIND.SGR}
|
|
66
|
+
* @param {boolean} concealed the state before this token
|
|
67
|
+
* @returns {boolean}
|
|
68
|
+
*/
|
|
69
|
+
export function sgrConcealState(token: string, concealed: boolean): boolean;
|
|
41
70
|
/**
|
|
42
71
|
* The ONE ANSI grammar: the raw control-introducer charset and the tokenizer
|
|
43
72
|
* every consumer scans with.
|
|
@@ -67,6 +67,14 @@ export function extraSecretVars(env?: Record<string, string | undefined>): strin
|
|
|
67
67
|
* @returns {boolean}
|
|
68
68
|
*/
|
|
69
69
|
export function secretsEnabled(env?: NodeJS.ProcessEnv | Record<string, string | undefined>): boolean;
|
|
70
|
+
/**
|
|
71
|
+
* True when the operator opted into flagging digest-shaped URL values. `=== "1"`
|
|
72
|
+
* for the reason {@link secretsEnabled} gives: a typo fails toward the default,
|
|
73
|
+
* which is the exemption the precision rule prefers.
|
|
74
|
+
* @param {NodeJS.ProcessEnv | Record<string, string | undefined>} [env]
|
|
75
|
+
* @returns {boolean}
|
|
76
|
+
*/
|
|
77
|
+
export function digestFlaggingEnabled(env?: NodeJS.ProcessEnv | Record<string, string | undefined>): boolean;
|
|
70
78
|
/**
|
|
71
79
|
* The env-bound redaction set: the UNION of the inference keys, the curated host
|
|
72
80
|
* credentials, any credential-shaped var present in the environment, the
|
|
@@ -79,3 +87,4 @@ export function secretsEnabled(env?: NodeJS.ProcessEnv | Record<string, string |
|
|
|
79
87
|
*/
|
|
80
88
|
export function envBoundSecretVars(env?: Record<string, string | undefined>): string[];
|
|
81
89
|
export const SECRETS_ENABLED_ENV: "AGENT_SANITIZER_SECRETS_ENABLED";
|
|
90
|
+
export const FLAG_DIGEST_VALUES_ENV: "AGENT_SANITIZER_FLAG_DIGEST_VALUES";
|
package/types/index.d.mts
CHANGED
|
@@ -58,6 +58,6 @@ export function sanitize(text: string, options?: {
|
|
|
58
58
|
original: string;
|
|
59
59
|
}>;
|
|
60
60
|
}>;
|
|
61
|
-
export { applyLayer1, applyLayer1WellFormed, isBenignAnsi, isBenignAnsiKinds, normalizeLoneSurrogates, stripAnsiFully, LONE_SURROGATE_RE } from "./layer1.mjs";
|
|
61
|
+
export { applyLayer1, applyLayer1WellFormed, isBenignAnsi, isBenignAnsiKinds, normalizeLoneSurrogates, stripAnsiFully, sgrCarriesPayload, SGR_RUN_THRESHOLD, LONE_SURROGATE_RE } from "./layer1.mjs";
|
|
62
62
|
export { stripInvisible, stripInvisibleWithReport, isSgrOnly, STRIP, SGR_RE, CHECKS, CATEGORY, CATEGORY_LABELS, LINGUISTIC_SCRIPTS, VS, BLANK_NON_CF, LONG_RUN_RE, LONG_RUN_THRESHOLD, SCATTERED_THRESHOLD, findLongRuns, hasLongRun } from "./invisible.mjs";
|
|
63
63
|
export { HTML_TAG_PRESENT, MD_LINK_HINT, SECRET_HINT, SECRET_HINT_EXT, matchesSecretHint } from "./gates.mjs";
|
package/types/layer1.d.mts
CHANGED
|
@@ -49,6 +49,32 @@ export function stripAnsiFully(input: string, kinds?: Set<string>): string;
|
|
|
49
49
|
* @returns {boolean}
|
|
50
50
|
*/
|
|
51
51
|
export function isBenignAnsiKinds(kinds: readonly string[] | Set<string>): boolean;
|
|
52
|
+
/**
|
|
53
|
+
* True when the SGR sequences in `text` do more than style visible text — the
|
|
54
|
+
* question a consumer must answer before PRESERVING escapes rather than
|
|
55
|
+
* stripping them. Two arms, both model-sees/human-sees divergences:
|
|
56
|
+
*
|
|
57
|
+
* 1. CONCEAL is on while text that would otherwise render goes by, or the text
|
|
58
|
+
* ENDS concealed (hiding its own tail, and everything the terminal prints
|
|
59
|
+
* afterwards). Only SGR moves that state (see {@link sgrConcealState}) — a
|
|
60
|
+
* cursor move or an erase leaves a terminal exactly as concealed as it was.
|
|
61
|
+
* 2. {@link SGR_RUN_THRESHOLD} sequences with nothing that renders between
|
|
62
|
+
* them: the run puts no glyph on the screen, so it carries data rather than
|
|
63
|
+
* styling.
|
|
64
|
+
*
|
|
65
|
+
* What it does NOT claim: a message spread thinly through legitimately coloured
|
|
66
|
+
* text — one sequence per line, say — is indistinguishable from styling, and
|
|
67
|
+
* this answers no for it. That is the same bargain the invisible layer's
|
|
68
|
+
* thresholds strike, and the same direction: an unprovable payload is left alone
|
|
69
|
+
* rather than costing every colourized file its colour.
|
|
70
|
+
*
|
|
71
|
+
* Scans the RAW text in one pass, which is what a terminal does with these
|
|
72
|
+
* bytes — no reconstitution across passes, because a preserved byte is never
|
|
73
|
+
* removed to complete a sequence around it.
|
|
74
|
+
* @param {string} text
|
|
75
|
+
* @returns {boolean}
|
|
76
|
+
*/
|
|
77
|
+
export function sgrCarriesPayload(text: string): boolean;
|
|
52
78
|
/**
|
|
53
79
|
* {@link isBenignAnsiKinds} for callers that hold only the text — it runs the
|
|
54
80
|
* full Layer-1 composition to get the fixed-point view. Callers that already
|
|
@@ -147,3 +173,16 @@ export const LONE_SURROGATE_RE: RegExp;
|
|
|
147
173
|
* colour codes, and here is how to look at the raw bytes".
|
|
148
174
|
*/
|
|
149
175
|
export const INERT_ANSI_NOTE: string;
|
|
176
|
+
/**
|
|
177
|
+
* How many SGR sequences may sit back to back, with nothing that RENDERS between
|
|
178
|
+
* them, before the run is read as a covert channel rather than styling.
|
|
179
|
+
*
|
|
180
|
+
* A styling emitter puts colour AROUND text: `ls --color`, chalk and pygments
|
|
181
|
+
* chain at most a handful of attributes before the glyphs they style. A run of
|
|
182
|
+
* escape sequences that puts no glyph on the screen renders as literally nothing
|
|
183
|
+
* to a human while a model reads every byte, so past some length the run is not
|
|
184
|
+
* styling any more — it is a message, in the same shape (and for the same
|
|
185
|
+
* reason) as the invisible layer's LONG_RUN_THRESHOLD. Ten matches that
|
|
186
|
+
* threshold and sits well above what a real emitter chains.
|
|
187
|
+
*/
|
|
188
|
+
export const SGR_RUN_THRESHOLD: 10;
|