agent-sanitizer 2.14.7 → 2.14.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -11
- package/THREAT-MODEL.md +44 -6
- package/package.json +1 -1
- package/src/confusables.mjs +198 -56
- package/types/confusables.d.mts +56 -7
package/README.md
CHANGED
|
@@ -47,17 +47,17 @@ Split into subpaths so the heavy HTML dependency stays opt-in. **Seam** names
|
|
|
47
47
|
the callback you inject for the agent-specific concern; `—` is a pure transform,
|
|
48
48
|
`fs (direct)` does its own file I/O instead of taking one.
|
|
49
49
|
|
|
50
|
-
| # | Import | Purpose
|
|
51
|
-
| --- | --------------- |
|
|
52
|
-
| 1 | `/invisible` | Strip zero-width, bidi, variation-selector and tag chars + ANSI/SGR escapes. Preserves ZWNJ/ZWJ for Arabic/Indic/emoji. Zero deps.
|
|
53
|
-
| 2 | `/html` | Splice out instructions hidden in comments, `display:none`, off-screen, white-on-white, `hidden`. Leaves a placeholder.
|
|
54
|
-
| 3 | `/html` | Detect exfil-shaped URLs (payloads in query/path, embedded creds, `data:`/`javascript:`, off-origin redirects). Reports only.
|
|
55
|
-
| 4 | `/confusables` | Fold look-alike glyphs in tool-call input (paths, commands) to ASCII, closing a cross-script deny-rule bypass.
|
|
56
|
-
| 5 | `/instructions` | Scan/auto-clean `CLAUDE.md`, `AGENTS.md`, `SKILL.md`, etc., decoding Unicode-tag + zero-width-binary payloads.
|
|
57
|
-
| 6 | `/prompt` | Classify a prompt pass / SGR-note / block on payload-capable invisible/ANSI content.
|
|
58
|
-
| 7 | `/output` | Run Layers 1–4 over structured tool output, preserving shape. The Layer-5 slot takes a delete-only filter.
|
|
59
|
-
| 8 | `/rehydrate` | Re-anchor a model Edit composed from the _sanitized_ view back onto real bytes; deny anything ambiguous or secret-exposing.
|
|
60
|
-
| — | `/view-map` | Pure offset/text machinery mapping a file's on-disk bytes ↔ the sanitized view (Layer-1 deletions, Layer-4 redactions). No I/O — consumed by `/rehydrate`.
|
|
50
|
+
| # | Import | Purpose | Seam |
|
|
51
|
+
| --- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
|
|
52
|
+
| 1 | `/invisible` | Strip zero-width, bidi, variation-selector and tag chars + ANSI/SGR escapes. Preserves ZWNJ/ZWJ for Arabic/Indic/emoji. Zero deps. | — |
|
|
53
|
+
| 2 | `/html` | Splice out instructions hidden in comments, `display:none`, off-screen, white-on-white, `hidden`. Leaves a placeholder. | — |
|
|
54
|
+
| 3 | `/html` | Detect exfil-shaped URLs (payloads in query/path, embedded creds, `data:`/`javascript:`, off-origin redirects). Reports only. | — |
|
|
55
|
+
| 4 | `/confusables` | Fold look-alike glyphs in tool-call input (paths, commands) to ASCII, closing a cross-script deny-rule bypass. Gated per token, so non-Latin prose passes through unfolded. | `scan` |
|
|
56
|
+
| 5 | `/instructions` | Scan/auto-clean `CLAUDE.md`, `AGENTS.md`, `SKILL.md`, etc., decoding Unicode-tag + zero-width-binary payloads. | `fs` (direct) |
|
|
57
|
+
| 6 | `/prompt` | Classify a prompt pass / SGR-note / block on payload-capable invisible/ANSI content. | — |
|
|
58
|
+
| 7 | `/output` | Run Layers 1–4 over structured tool output, preserving shape. The Layer-5 slot takes a delete-only filter. | `redact`, `filterInjection` |
|
|
59
|
+
| 8 | `/rehydrate` | Re-anchor a model Edit composed from the _sanitized_ view back onto real bytes; deny anything ambiguous or secret-exposing. | `io` |
|
|
60
|
+
| — | `/view-map` | Pure offset/text machinery mapping a file's on-disk bytes ↔ the sanitized view (Layer-1 deletions, Layer-4 redactions). No I/O — consumed by `/rehydrate`. | — |
|
|
61
61
|
|
|
62
62
|
See [`THREAT-MODEL.md`](./THREAT-MODEL.md) for per-vector detail.
|
|
63
63
|
|
package/THREAT-MODEL.md
CHANGED
|
@@ -92,12 +92,50 @@ payload-bearing query/fragment), suitable for a warning shown to the operator.
|
|
|
92
92
|
`./confusables` folds look-alike glyphs in tool-call **input** fields (paths,
|
|
93
93
|
commands) to their ASCII canon. A denied path/command spelled in homoglyphs (a
|
|
94
94
|
Cyrillic `а` for ASCII `a`) would not match an ASCII deny rule; folding closes
|
|
95
|
-
that cross-script bypass (CVE-2025-54794 class).
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
95
|
+
that cross-script bypass (CVE-2025-54794 class).
|
|
96
|
+
|
|
97
|
+
Folding is gated per **token** — a maximal run of ASCII alphanumerics and
|
|
98
|
+
non-ASCII glyphs, with every other ASCII character a boundary. A token folds only
|
|
99
|
+
when both hold:
|
|
100
|
+
|
|
101
|
+
1. **Folding leaves it pure ASCII.** The bypass requires the folded token to come
|
|
102
|
+
out byte-equal to an ASCII deny-rule target, so a token that still holds an
|
|
103
|
+
unmapped glyph after folding could not match one either way — skipping it
|
|
104
|
+
forfeits no enforcement.
|
|
105
|
+
2. **It is more than a lone non-ASCII glyph.** A single-code-point token is a
|
|
106
|
+
one-letter foreign word — Russian `с` (with), `о` (about), `у` (at), `а`
|
|
107
|
+
(and), all mapped confusables and all among the most frequent words in the
|
|
108
|
+
language — as readily as it is a disguised argument, and no deny rule targets
|
|
109
|
+
a single character.
|
|
110
|
+
|
|
111
|
+
So `/etc/pаsswd` and an all-Cyrillic `раѕѕwd` still fold, while `Привет`,
|
|
112
|
+
`пароль.txt`, and `работа с файлом` pass through untouched. Without this gate the
|
|
113
|
+
layer transliterated ordinary Cyrillic and Greek text — a commit message, an
|
|
114
|
+
issue body, a filename — into garbage, which this repo weighs as the worse
|
|
115
|
+
failure.
|
|
116
|
+
|
|
117
|
+
The gate is per-token and never per-field: a field-level "any prose here → skip
|
|
118
|
+
the field" rule would let an attacker switch folding off for a whole command by
|
|
119
|
+
appending one foreign word.
|
|
120
|
+
|
|
121
|
+
**Known false positive:** a multi-letter foreign word composed _entirely_ of
|
|
122
|
+
mapped confusables — the Russian `сор` → `cop`, `со` → `co` — is by construction
|
|
123
|
+
indistinguishable from a disguised ASCII token, and still folds.
|
|
124
|
+
|
|
125
|
+
**Known false negative:** splicing one unmapped non-ASCII glyph into an otherwise
|
|
126
|
+
all-confusable token switches its fold off. It buys an attacker nothing on its
|
|
127
|
+
own — the unmapped glyph is still in the field, so the token cannot match an
|
|
128
|
+
ASCII deny rule either. A filter reading the raw field is what closes that, not
|
|
129
|
+
folding.
|
|
130
|
+
|
|
131
|
+
The soundness argument assumes no later layer erases code points from the same
|
|
132
|
+
field, which could remove an unmapped glyph the gate relied on after the decision
|
|
133
|
+
was made. Layer 4 runs before `sanitizeAuthoredContent` on `Bash.command`.
|
|
134
|
+
|
|
135
|
+
The homoglyph engine is **injected** (`{ scan }`) — the package owns no glyph
|
|
136
|
+
map. An all-ASCII field never invokes the scanner. This narrows a steganographic
|
|
137
|
+
channel; it is not an enforcement boundary (distinct code points would not match
|
|
138
|
+
a deny rule anyway).
|
|
101
139
|
|
|
102
140
|
## Instruction-file scanning
|
|
103
141
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-sanitizer",
|
|
3
|
-
"version": "2.14.
|
|
3
|
+
"version": "2.14.9",
|
|
4
4
|
"description": "Defend an agent against hidden-content injection: strip payload-capable invisible Unicode and ANSI, splice out human-invisible HTML, and flag data-exfil URLs in untrusted text before any model sees it.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|
package/src/confusables.mjs
CHANGED
|
@@ -7,13 +7,42 @@
|
|
|
7
7
|
* an ASCII deny rule, so an attacker could slip a denied path/command past a
|
|
8
8
|
* filter by spelling it in look-alike code points.
|
|
9
9
|
*
|
|
10
|
-
* Folding is per
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
14
|
-
*
|
|
15
|
-
*
|
|
16
|
-
*
|
|
10
|
+
* Folding is gated per TOKEN (a maximal run of ASCII alphanumerics and
|
|
11
|
+
* non-ASCII glyphs — see isTokenBoundary). A token folds only when both hold:
|
|
12
|
+
*
|
|
13
|
+
* 1. Folding makes it pure ASCII — every non-ASCII code point in it is
|
|
14
|
+
* flagged. The bypass requires the folded token to come out byte-equal to
|
|
15
|
+
* an ASCII deny-rule target, so a token left holding an unmapped glyph
|
|
16
|
+
* could never match one anyway; skipping it costs no enforcement.
|
|
17
|
+
* 2. It is more than a lone non-ASCII glyph standing between two boundaries.
|
|
18
|
+
* A one-code-point token is a one-letter foreign word — Russian "с", "о",
|
|
19
|
+
* "у", "а" are among the most frequent words in the language — as readily
|
|
20
|
+
* as it is a disguised argument, and no deny rule targets a single
|
|
21
|
+
* character, so the evidence does not support rewriting it.
|
|
22
|
+
*
|
|
23
|
+
* That still catches the all-confusable disguise ("раѕѕwd" → "passwd") and the
|
|
24
|
+
* anchored one ("pаsswd" → "passwd"), which is exactly the bypass to
|
|
25
|
+
* close, while leaving genuine non-Latin prose intact: "Привет" keeps unmapped
|
|
26
|
+
* П/и/в/т, so its mapped р/е are not folded and the word survives byte-for-byte.
|
|
27
|
+
*
|
|
28
|
+
* WHY THE GATE: without it, folding mangled any Cyrillic/Greek text passing
|
|
29
|
+
* through a command or path — a Russian commit message, issue body, or filename
|
|
30
|
+
* came out transliterated into garbage. A false positive here rewrites content
|
|
31
|
+
* the operator wrote, which this repo weighs as the worse failure.
|
|
32
|
+
*
|
|
33
|
+
* RESIDUAL: a multi-letter foreign word composed ENTIRELY of mapped confusables
|
|
34
|
+
* (the Russian "сор" → "cop") is indistinguishable from a disguised ASCII token
|
|
35
|
+
* by construction, and still folds. Documented in THREAT-MODEL.md, not worked
|
|
36
|
+
* around — the alternatives (a field-level gate, a "looks like a path" shape
|
|
37
|
+
* heuristic) either open a bypass or trade one guess for another.
|
|
38
|
+
*
|
|
39
|
+
* ORDERING: the soundness argument assumes no later layer erases code points
|
|
40
|
+
* from the same field, which would let an unmapped glyph the gate relied on
|
|
41
|
+
* disappear after the decision. Layer 4 runs before sanitizeAuthoredContent on
|
|
42
|
+
* Bash.command; keep it there.
|
|
43
|
+
*
|
|
44
|
+
* Genuine non-confusable non-ASCII (accented Latin, CJK, emoji) is untouched
|
|
45
|
+
* regardless, since a faithful scanner does not flag it.
|
|
17
46
|
*
|
|
18
47
|
* The confusable scanner is INJECTED, never imported: the canonical engine
|
|
19
48
|
* (namespace-guard's vision-weighted map) is a heavy, separately-owned peer.
|
|
@@ -87,6 +116,157 @@ function describeFolds(findings) {
|
|
|
87
116
|
return folds.length > MAX_REPORTED_FOLDS ? `${shown}, …` : shown;
|
|
88
117
|
}
|
|
89
118
|
|
|
119
|
+
/**
|
|
120
|
+
* Reject a finding that does not describe a real, foldable glyph at its reported
|
|
121
|
+
* offset in `text`. Every consumer of a scanner finding runs this BEFORE acting
|
|
122
|
+
* on it, so a buggy or adversarial scanner fails loud instead of silently
|
|
123
|
+
* corrupting (or silently escaping the fold gate in) a path/command.
|
|
124
|
+
* @param {string} text
|
|
125
|
+
* @param {{ index: number, char: string, latinEquivalent: string }} finding
|
|
126
|
+
* @returns {void}
|
|
127
|
+
*/
|
|
128
|
+
function assertFinding(text, finding) {
|
|
129
|
+
// Fail loud on a finding that does not match the actual bytes at its offset:
|
|
130
|
+
// a buggy/adversarial scanner reporting a wrong char/index would otherwise
|
|
131
|
+
// silently corrupt the path/command, defeating the deny-rule protection.
|
|
132
|
+
// A negative index is the gap the startsWith guard alone misses: when `char`
|
|
133
|
+
// is a prefix of the text, `startsWith(char, -1)` is true (the offset is
|
|
134
|
+
// clamped to 0), and the slice math below then mangles the string instead of
|
|
135
|
+
// throwing — so range-check the index explicitly first.
|
|
136
|
+
if (!Number.isInteger(finding.index) || finding.index < 0)
|
|
137
|
+
throw new Error(
|
|
138
|
+
`Confusable finding has an out-of-range index ${finding.index}`,
|
|
139
|
+
);
|
|
140
|
+
// An empty `char` makes startsWith("", i) vacuously true for ANY index, so
|
|
141
|
+
// the splice inserts latinEquivalent without consuming a code point — silent
|
|
142
|
+
// insertion-corruption — and describeFolds then crashes on "".codePointAt(0).
|
|
143
|
+
// A finding must name a real matched glyph, so reject it loudly rather than
|
|
144
|
+
// let a buggy/adversarial scanner corrupt the input.
|
|
145
|
+
if (finding.char === "")
|
|
146
|
+
throw new Error(
|
|
147
|
+
`Confusable finding at index ${finding.index} has an empty char`,
|
|
148
|
+
);
|
|
149
|
+
// An ASCII `char` is not a confusable — it is already its own canon. Folding
|
|
150
|
+
// it is a no-op at best, but the fold gate reads a non-ASCII code point's
|
|
151
|
+
// flagged/unflagged status to decide a token's fate, so a scanner claiming an
|
|
152
|
+
// ASCII character is confusable is reporting something the contract says
|
|
153
|
+
// cannot happen. Fail loud rather than act on a finding we cannot interpret.
|
|
154
|
+
if (!hasNonAscii(finding.char))
|
|
155
|
+
throw new Error(
|
|
156
|
+
`Confusable finding at index ${finding.index} names an ASCII char ${JSON.stringify(finding.char)}`,
|
|
157
|
+
);
|
|
158
|
+
if (!text.startsWith(finding.char, finding.index))
|
|
159
|
+
throw new Error(
|
|
160
|
+
`Confusable finding does not match input at index ${finding.index}: expected ${JSON.stringify(finding.char)}`,
|
|
161
|
+
);
|
|
162
|
+
// An empty `latinEquivalent` slips past the ASCII loop below (it never
|
|
163
|
+
// iterates) and would splice the glyph to nothing — silently DELETING a
|
|
164
|
+
// character from a path/command. That is the same class of silent corruption
|
|
165
|
+
// the non-ASCII guard rejects, so fail loud here too rather than let a
|
|
166
|
+
// buggy/adversarial scanner erase input.
|
|
167
|
+
if (finding.latinEquivalent === "")
|
|
168
|
+
throw new Error(
|
|
169
|
+
`Confusable finding for ${JSON.stringify(
|
|
170
|
+
finding.char,
|
|
171
|
+
)} at index ${finding.index} has an empty latinEquivalent`,
|
|
172
|
+
);
|
|
173
|
+
// The replacement must be the ASCII canon the contract promises. A non-ASCII
|
|
174
|
+
// `latinEquivalent` would fold one confusable into ANOTHER look-alike (e.g.
|
|
175
|
+
// Cyrillic а → Cyrillic е), defeating the whole point — the cross-script
|
|
176
|
+
// deny-rule bypass would survive — and silently break the fold-to-ASCII
|
|
177
|
+
// invariant callers rely on, so reject it loudly.
|
|
178
|
+
for (const ch of finding.latinEquivalent)
|
|
179
|
+
if (/** @type {number} */ (ch.codePointAt(0)) > 0x7f)
|
|
180
|
+
throw new Error(
|
|
181
|
+
`Confusable latinEquivalent ${JSON.stringify(
|
|
182
|
+
finding.latinEquivalent,
|
|
183
|
+
)} is not ASCII`,
|
|
184
|
+
);
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
/**
|
|
188
|
+
* True for a code point that ends a token: any ASCII character that is not a
|
|
189
|
+
* letter or digit (space, `/`, `.`, `-`, `_`, quotes, shell metacharacters). A
|
|
190
|
+
* token is therefore a maximal run of ASCII alphanumerics and non-ASCII glyphs.
|
|
191
|
+
* That is not a shell word or a path segment — it is deliberately the SMALLEST
|
|
192
|
+
* unit that still keeps a word of foreign prose whole, so one such word can
|
|
193
|
+
* never switch folding off for the tokens around it.
|
|
194
|
+
* @param {string} ch a single code point
|
|
195
|
+
* @returns {boolean}
|
|
196
|
+
*/
|
|
197
|
+
function isTokenBoundary(ch) {
|
|
198
|
+
if (hasNonAscii(ch)) return false;
|
|
199
|
+
const code = ch.charCodeAt(0);
|
|
200
|
+
const isDigit = code >= 0x30 && code <= 0x39;
|
|
201
|
+
const isUpper = code >= 0x41 && code <= 0x5a;
|
|
202
|
+
const isLower = code >= 0x61 && code <= 0x7a;
|
|
203
|
+
return !isDigit && !isUpper && !isLower;
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
/**
|
|
207
|
+
* Keep only the findings whose TOKEN is foldable — see the module header for the
|
|
208
|
+
* two conditions (folds to pure ASCII; is not a lone glyph) and THREAT-MODEL.md
|
|
209
|
+
* for why declining the rest forfeits no enforcement.
|
|
210
|
+
*
|
|
211
|
+
* Every finding is validated before it is judged, so an adversarial scanner's
|
|
212
|
+
* bogus finding throws rather than being quietly dropped by the gate.
|
|
213
|
+
* @param {string} text
|
|
214
|
+
* @param {Array<{ index: number, char: string, latinEquivalent: string }>} findings
|
|
215
|
+
* @returns {Array<{ index: number, char: string, latinEquivalent: string }>}
|
|
216
|
+
*/
|
|
217
|
+
export function selectFoldableFindings(text, findings) {
|
|
218
|
+
for (const finding of findings) assertFinding(text, finding);
|
|
219
|
+
|
|
220
|
+
// Every UTF-16 offset a finding covers, not just its start: a scanner is free
|
|
221
|
+
// to report a multi-code-point match, and treating only the first offset as
|
|
222
|
+
// flagged would read the rest as unmapped and reject an otherwise ASCII token.
|
|
223
|
+
const flagged = new Set();
|
|
224
|
+
for (const finding of findings)
|
|
225
|
+
for (let i = 0; i < finding.char.length; i++)
|
|
226
|
+
flagged.add(finding.index + i);
|
|
227
|
+
|
|
228
|
+
// Mark the offsets belonging to a foldable token, so the per-finding lookup
|
|
229
|
+
// below is O(1). An interval scan per finding would be quadratic on a
|
|
230
|
+
// glyph-stuffed command — this runs on every tool call.
|
|
231
|
+
const foldableAt = new Uint8Array(text.length);
|
|
232
|
+
let start = 0;
|
|
233
|
+
let foldable = true;
|
|
234
|
+
let glyphs = 0;
|
|
235
|
+
let anchored = false;
|
|
236
|
+
let index = 0;
|
|
237
|
+
/**
|
|
238
|
+
* Close the token ending at `end`, marking it foldable when it holds no
|
|
239
|
+
* unmapped glyph and is more than a lone glyph, then reset for the next one.
|
|
240
|
+
* @param {number} end
|
|
241
|
+
*/
|
|
242
|
+
const flushToken = (end) => {
|
|
243
|
+
if (foldable && (glyphs > 1 || anchored)) foldableAt.fill(1, start, end);
|
|
244
|
+
foldable = true;
|
|
245
|
+
glyphs = 0;
|
|
246
|
+
anchored = false;
|
|
247
|
+
};
|
|
248
|
+
for (const ch of text) {
|
|
249
|
+
if (isTokenBoundary(ch)) {
|
|
250
|
+
flushToken(index);
|
|
251
|
+
start = index + ch.length;
|
|
252
|
+
} else {
|
|
253
|
+
glyphs++;
|
|
254
|
+
if (!hasNonAscii(ch)) anchored = true;
|
|
255
|
+
// An unmapped non-ASCII glyph: this token cannot fold to ASCII.
|
|
256
|
+
else if (!flagged.has(index)) foldable = false;
|
|
257
|
+
}
|
|
258
|
+
index += ch.length;
|
|
259
|
+
}
|
|
260
|
+
flushToken(index);
|
|
261
|
+
|
|
262
|
+
// Every offset the finding covers, not just its first: `char` may span a
|
|
263
|
+
// boundary into a token the gate rejected, and folding it would rewrite that
|
|
264
|
+
// token — the exact mangling this gate exists to prevent.
|
|
265
|
+
return findings.filter((finding) =>
|
|
266
|
+
[...finding.char].every((_, i) => foldableAt[finding.index + i] === 1),
|
|
267
|
+
);
|
|
268
|
+
}
|
|
269
|
+
|
|
90
270
|
/**
|
|
91
271
|
* Replace every scan-flagged confusable with its ASCII (latin) equivalent.
|
|
92
272
|
* `index` is a UTF-16 offset into `text` and `char` is the matched glyph (which
|
|
@@ -101,53 +281,10 @@ export function foldConfusables(text, findings) {
|
|
|
101
281
|
for (const finding of [...findings].sort(
|
|
102
282
|
(lhs, rhs) => rhs.index - lhs.index,
|
|
103
283
|
)) {
|
|
104
|
-
//
|
|
105
|
-
//
|
|
106
|
-
//
|
|
107
|
-
|
|
108
|
-
// is a prefix of the text, `startsWith(char, -1)` is true (the offset is
|
|
109
|
-
// clamped to 0), and the slice math below then mangles the string instead of
|
|
110
|
-
// throwing — so range-check the index explicitly first.
|
|
111
|
-
if (!Number.isInteger(finding.index) || finding.index < 0)
|
|
112
|
-
throw new Error(
|
|
113
|
-
`Confusable finding has an out-of-range index ${finding.index}`,
|
|
114
|
-
);
|
|
115
|
-
// An empty `char` makes startsWith("", i) vacuously true for ANY index, so
|
|
116
|
-
// the slice below inserts latinEquivalent without consuming a code point —
|
|
117
|
-
// silent insertion-corruption — and describeFolds then crashes on
|
|
118
|
-
// "".codePointAt(0). A finding must name a real matched glyph, so reject it
|
|
119
|
-
// loudly rather than let a buggy/adversarial scanner corrupt the input.
|
|
120
|
-
if (finding.char === "")
|
|
121
|
-
throw new Error(
|
|
122
|
-
`Confusable finding at index ${finding.index} has an empty char`,
|
|
123
|
-
);
|
|
124
|
-
if (!folded.startsWith(finding.char, finding.index))
|
|
125
|
-
throw new Error(
|
|
126
|
-
`Confusable finding does not match input at index ${finding.index}: expected ${JSON.stringify(finding.char)}`,
|
|
127
|
-
);
|
|
128
|
-
// An empty `latinEquivalent` slips past the ASCII loop below (it never
|
|
129
|
-
// iterates) and would splice the glyph to nothing — silently DELETING a
|
|
130
|
-
// character from a path/command. That is the same class of silent corruption
|
|
131
|
-
// the non-ASCII guard rejects, so fail loud here too rather than let a
|
|
132
|
-
// buggy/adversarial scanner erase input.
|
|
133
|
-
if (finding.latinEquivalent === "")
|
|
134
|
-
throw new Error(
|
|
135
|
-
`Confusable finding for ${JSON.stringify(
|
|
136
|
-
finding.char,
|
|
137
|
-
)} at index ${finding.index} has an empty latinEquivalent`,
|
|
138
|
-
);
|
|
139
|
-
// The replacement must be the ASCII canon the contract promises. A non-ASCII
|
|
140
|
-
// `latinEquivalent` would fold one confusable into ANOTHER look-alike (e.g.
|
|
141
|
-
// Cyrillic а → Cyrillic е), defeating the whole point — the cross-script
|
|
142
|
-
// deny-rule bypass would survive — and silently break the fold-to-ASCII
|
|
143
|
-
// invariant callers rely on, so reject it loudly.
|
|
144
|
-
for (const ch of finding.latinEquivalent)
|
|
145
|
-
if (/** @type {number} */ (ch.codePointAt(0)) > 0x7f)
|
|
146
|
-
throw new Error(
|
|
147
|
-
`Confusable latinEquivalent ${JSON.stringify(
|
|
148
|
-
finding.latinEquivalent,
|
|
149
|
-
)} is not ASCII`,
|
|
150
|
-
);
|
|
284
|
+
// Validate against the partially-folded text: the highest-index-first order
|
|
285
|
+
// leaves every not-yet-spliced offset byte-identical to `text`, so the
|
|
286
|
+
// startsWith check still sees the original glyph at the reported index.
|
|
287
|
+
assertFinding(folded, finding);
|
|
151
288
|
folded =
|
|
152
289
|
folded.slice(0, finding.index) +
|
|
153
290
|
finding.latinEquivalent +
|
|
@@ -190,8 +327,13 @@ export function normalizeConfusables(
|
|
|
190
327
|
for (const k of candidates) {
|
|
191
328
|
const { findings } = scan(toolInput[k]);
|
|
192
329
|
if (findings.length === 0) continue;
|
|
193
|
-
|
|
194
|
-
|
|
330
|
+
// Precision gate: fold only what a deny rule could actually match (see
|
|
331
|
+
// selectFoldableFindings). Report the folds APPLIED, not the ones scanned —
|
|
332
|
+
// naming a glyph the model can still see in the field would be a lie.
|
|
333
|
+
const foldable = selectFoldableFindings(toolInput[k], findings);
|
|
334
|
+
if (foldable.length === 0) continue;
|
|
335
|
+
updatedInput[k] = foldConfusables(toolInput[k], foldable);
|
|
336
|
+
normalized.push(`${k} (${describeFolds(foldable)})`);
|
|
195
337
|
}
|
|
196
338
|
|
|
197
339
|
if (normalized.length === 0) return null;
|
package/types/confusables.d.mts
CHANGED
|
@@ -12,6 +12,26 @@ export function hasNonAscii(value: string): boolean;
|
|
|
12
12
|
* @returns {string}
|
|
13
13
|
*/
|
|
14
14
|
export function normalizeContext(normalized: string[]): string;
|
|
15
|
+
/**
|
|
16
|
+
* Keep only the findings whose TOKEN is foldable — see the module header for the
|
|
17
|
+
* two conditions (folds to pure ASCII; is not a lone glyph) and THREAT-MODEL.md
|
|
18
|
+
* for why declining the rest forfeits no enforcement.
|
|
19
|
+
*
|
|
20
|
+
* Every finding is validated before it is judged, so an adversarial scanner's
|
|
21
|
+
* bogus finding throws rather than being quietly dropped by the gate.
|
|
22
|
+
* @param {string} text
|
|
23
|
+
* @param {Array<{ index: number, char: string, latinEquivalent: string }>} findings
|
|
24
|
+
* @returns {Array<{ index: number, char: string, latinEquivalent: string }>}
|
|
25
|
+
*/
|
|
26
|
+
export function selectFoldableFindings(text: string, findings: Array<{
|
|
27
|
+
index: number;
|
|
28
|
+
char: string;
|
|
29
|
+
latinEquivalent: string;
|
|
30
|
+
}>): Array<{
|
|
31
|
+
index: number;
|
|
32
|
+
char: string;
|
|
33
|
+
latinEquivalent: string;
|
|
34
|
+
}>;
|
|
15
35
|
/**
|
|
16
36
|
* Replace every scan-flagged confusable with its ASCII (latin) equivalent.
|
|
17
37
|
* `index` is a UTF-16 offset into `text` and `char` is the matched glyph (which
|
|
@@ -62,13 +82,42 @@ export function normalizeConfusables(tool: string, toolInput: any, { scan, field
|
|
|
62
82
|
* an ASCII deny rule, so an attacker could slip a denied path/command past a
|
|
63
83
|
* filter by spelling it in look-alike code points.
|
|
64
84
|
*
|
|
65
|
-
* Folding is per
|
|
66
|
-
*
|
|
67
|
-
*
|
|
68
|
-
*
|
|
69
|
-
*
|
|
70
|
-
*
|
|
71
|
-
*
|
|
85
|
+
* Folding is gated per TOKEN (a maximal run of ASCII alphanumerics and
|
|
86
|
+
* non-ASCII glyphs — see isTokenBoundary). A token folds only when both hold:
|
|
87
|
+
*
|
|
88
|
+
* 1. Folding makes it pure ASCII — every non-ASCII code point in it is
|
|
89
|
+
* flagged. The bypass requires the folded token to come out byte-equal to
|
|
90
|
+
* an ASCII deny-rule target, so a token left holding an unmapped glyph
|
|
91
|
+
* could never match one anyway; skipping it costs no enforcement.
|
|
92
|
+
* 2. It is more than a lone non-ASCII glyph standing between two boundaries.
|
|
93
|
+
* A one-code-point token is a one-letter foreign word — Russian "с", "о",
|
|
94
|
+
* "у", "а" are among the most frequent words in the language — as readily
|
|
95
|
+
* as it is a disguised argument, and no deny rule targets a single
|
|
96
|
+
* character, so the evidence does not support rewriting it.
|
|
97
|
+
*
|
|
98
|
+
* That still catches the all-confusable disguise ("раѕѕwd" → "passwd") and the
|
|
99
|
+
* anchored one ("pаsswd" → "passwd"), which is exactly the bypass to
|
|
100
|
+
* close, while leaving genuine non-Latin prose intact: "Привет" keeps unmapped
|
|
101
|
+
* П/и/в/т, so its mapped р/е are not folded and the word survives byte-for-byte.
|
|
102
|
+
*
|
|
103
|
+
* WHY THE GATE: without it, folding mangled any Cyrillic/Greek text passing
|
|
104
|
+
* through a command or path — a Russian commit message, issue body, or filename
|
|
105
|
+
* came out transliterated into garbage. A false positive here rewrites content
|
|
106
|
+
* the operator wrote, which this repo weighs as the worse failure.
|
|
107
|
+
*
|
|
108
|
+
* RESIDUAL: a multi-letter foreign word composed ENTIRELY of mapped confusables
|
|
109
|
+
* (the Russian "сор" → "cop") is indistinguishable from a disguised ASCII token
|
|
110
|
+
* by construction, and still folds. Documented in THREAT-MODEL.md, not worked
|
|
111
|
+
* around — the alternatives (a field-level gate, a "looks like a path" shape
|
|
112
|
+
* heuristic) either open a bypass or trade one guess for another.
|
|
113
|
+
*
|
|
114
|
+
* ORDERING: the soundness argument assumes no later layer erases code points
|
|
115
|
+
* from the same field, which would let an unmapped glyph the gate relied on
|
|
116
|
+
* disappear after the decision. Layer 4 runs before sanitizeAuthoredContent on
|
|
117
|
+
* Bash.command; keep it there.
|
|
118
|
+
*
|
|
119
|
+
* Genuine non-confusable non-ASCII (accented Latin, CJK, emoji) is untouched
|
|
120
|
+
* regardless, since a faithful scanner does not flag it.
|
|
72
121
|
*
|
|
73
122
|
* The confusable scanner is INJECTED, never imported: the canonical engine
|
|
74
123
|
* (namespace-guard's vision-weighted map) is a heavy, separately-owned peer.
|