@sema-agent/primitives 7.67.3 → 7.68.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -96,8 +96,9 @@ form and token head an environment value is read by), the credential words a nam
96
96
  credentials but are none, and `names`: the names that mark a credential in an assignment (`places: ["assignment"]`) or as a
97
97
  URL query or fragment parameter (`"url-param"`).
98
98
  Every layer that looks for a secret reads this table, so a shape added to it is found everywhere at once. The table holds
99
- no layer's policy: which form is read, how a text is read first, how a URL's username is treated and which mark replaces
100
- a hit are the caller's, passed in whole on every call. Nothing here has a default.
99
+ no layer's policy: which form is read, how a text is read first and how a URL's username is treated are the caller's,
100
+ passed in whole on every call. Nothing here has a default. Each value family row names the mark its hit is written as
101
+ (`mark`, below).
101
102
 
102
103
  ```ts
103
104
  interface SecretScanPolicy {
@@ -107,32 +108,49 @@ interface SecretScanPolicy {
107
108
  urlUserinfo: "keep-username" | "whole"; // a URL userinfo: keep the user and withhold the password, or withhold all of it
108
109
  onUnresolvedReading: "secret" | "ignore"; // what a reading that cannot read a text to its end answers
109
110
  }
110
- interface SecretNotation {
111
- value: string; // the mark a hit is written as
112
- byKind?: Record<string, string>; // ...or, for a row of this kind, this mark
113
- byFamily?: Record<string, string>; // ...or, for this row, this mark
114
- overStrip: string; // the whole text, when no reading can afford it
115
- }
116
111
  ```
117
112
 
118
113
  | Function | Answers |
119
114
  | --- | --- |
120
115
  | `scanSecrets(text, policy)` | `{ outcome: "spans", spans }` — every hit as `{ start, end, family, kind }` in `text`'s own coordinates, sorted (empty = none found) — or `{ outcome: "whole", reason }` with `reason` one of `scan-budget` (more than 64 places where format characters stood), `reading-unresolved` (the percent layers ran out, under `onUnresolvedReading: "secret"`), `mapping-failed`; a hit in a derived text is reported over the written units it was read from |
121
116
  | `isSecretShaped(text, policy)` | `scanSecrets` found a span or answered the whole text |
122
- | `userinfoHolds(userinfo)` | a URL userinfo (no trailing `@`) holds a user or a password — an empty one and the bare separator `:` hold neither (the reading the `keep-username` rendering withholds by) |
123
- | `renderSecrets(text, policy, notation, { passes?, report? })` | `text` with every hit replaced by its mark; `passes` are the caller's own rules, run after the table's in the same pipeline; `report.findings` receives one finding per hit (`report.preexistingMarkers` is the caller's to set) |
117
+ | `userinfoHolds(userinfo)` | a URL userinfo (no trailing `@`) holds a user or a password — an empty one and the bare separator `:` hold neither (the reading the `keep-username` rendering withholds by; a mark is one token, its `:` splits nothing) |
118
+ | `renderSecrets(text, policy, { passes?, report? }?)` | `text` with every hit replaced by its row's mark (the fragment grade's by that mark's `:partial` member; a text no reading can afford, whole, by `«redacted:oversized»`); `passes` are the caller's own rules, run after the table's in the same pipeline, each `marker` a member of `REDACTION_MARKS`; `report.findings` receives one finding per hit (`report.preexistingMarkers` is the caller's to set) |
119
+ | `renderWithheld(text, spans)` | `text` with each `{ start, end, mark }` span withheld by the same law `renderSecrets` writes its own hits by — for a layer that finds its regions itself |
124
120
  | `isSensitiveRecordKey(key)` / `isSensitiveHeaderName(name)` | a record member / an HTTP header name names a credential |
125
121
  | `envNameView()` | the readings of environment variable names (exact names, credential words, a namespace's names `anySegmentUnder(namespace)`) |
126
122
  | `envValueView()` | the readings of environment values, off the rows: `forms` (each row's env value form, in table order), `tokenHead` (each row's token head, one anchored alternation), `flagWords` |
127
123
  | `secretReadingViews(text, readings)` | the texts the readings derive — the written text first, each once — each with `toOriginalSpan(start, end)` (the whole written units a span was read from), and `unresolved` (the percent layers ran out while one more would still decode) |
128
124
 
129
- A policy or a notation that is not whole (a missing or unknown member, an empty mark, a mark holding `$` before a digit,
130
- a `byKind` / `byFamily` key no row has, two rows of one pass spelled differently) throws a `TypeError`. Both are read on
131
- every call; what they read as picks a compilation kept across calls. A mark is written as spelled. `scanSecrets`
132
- answers where a credential is; `renderSecrets` also leaves a hit that already reads as its own mark, which only a
133
- notation can tell. `URL_OF_ANY_SCHEME` is the URL grammar (a global regex: read it with `replace` / `matchAll`), for a
134
- layer that withholds whole URLs. `CREDENTIAL_MIN_CHARS` is the fewest characters a string can have and still be read as
135
- a credential.
125
+ A policy that is not whole (a missing or unknown member), a caller's pass whose `marker` is no member of the marks table,
126
+ or whose string `replace` writes anything but that member and `$n` references, throws a `TypeError`; so does a
127
+ `renderWithheld` span that is not a range of the text or whose mark is no member. The policy is read on every call; what
128
+ it reads as picks a compilation kept across calls. `scanSecrets` answers where its forms match, a mark the text
129
+ holds included (`password=«redacted:key»` is a span); `renderSecrets` leaves a hit whose withheld part is exactly a mark
130
+ the text already holds (below). `URL_OF_ANY_SCHEME` is the URL grammar (a global regex: read it with
131
+ `replace` / `matchAll`), for a layer that withholds whole URLs. `CREDENTIAL_MIN_CHARS` is the fewest characters a string
132
+ can have and still be read as a credential.
133
+
134
+ ### Redaction marks
135
+
136
+ `REDACTION_MARKS` is the one table of what stands in a text where a layer withheld part of it — fifteen members, frozen:
137
+
138
+ | Name | Member | Written for |
139
+ | --- | --- | --- |
140
+ | `bare` | `«redacted»` | a value whose place in the text already says what it was: a credential name's value, an assignment, a header value, a scheme word's credential (`Authorization: «redacted»`), the password after a kept username (`https://alice:«redacted»@host`), a URL query or fragment value |
141
+ | `key` | `«redacted:key»` | a credential read by its shape alone (`sk-…`, `AIza…`, a JWT, …) |
142
+ | `gh-token` / `aws-key` / `slack-token` / `private-key` | `«redacted:gh-token»` … | a credential read by its shape alone, of that vendor or kind |
143
+ | `<kind>:partial` | `«redacted:key:partial»` … | the same five kinds, found by the `fragment` grade (a cut residue) |
144
+ | `userinfo` | `«redacted:userinfo»` | a URL userinfo withheld whole |
145
+ | `url` / `path` | `«redacted:url»` / `«redacted:path»` | a whole URL / an absolute path, withheld by a layer that hides where things are |
146
+ | `oversized` | `«redacted:oversized»` | a stretch withheld because it could not be read |
147
+
148
+ `isRedactionMark(s)`, `redactionMarkerSpans(s)` and `eachRedactionMarkerSpan(s)` (lazy) are the one reader: they
149
+ recognize the members exactly — `«redacted:evil»` and square-bracket spellings (`[redacted]`, `[redacted-url]`) are
150
+ ordinary text. A mark is one token to the renderer: a hit whose withheld part is a mark the text already holds is no hit
151
+ (`password=«redacted:key»` stays as written), a hit reaching into or inside a mark takes the whole mark, and hits that
152
+ overlap or touch write one mark — the widest hit's (of two over one region, the later pass's). Rendering a rendered text
153
+ again changes nothing.
136
154
 
137
155
  Every count in the table follows one character or one character class and is closed — "`n` or more" is written as `n`
138
156
  of them and then a plain loop, and a group's count is written out — so a reading keeps no regex backtrack state per
@@ -148,8 +166,7 @@ exported table changes only that caller's reading.
148
166
  import { renderSecrets, scanSecrets } from "@sema-agent/primitives";
149
167
 
150
168
  const policy = Object.freeze({ grade: "wide", readings: Object.freeze([Object.freeze({ kind: "raw" })]), urlUserinfo: "keep-username", onUnresolvedReading: "secret" });
151
- const notation = Object.freeze({ value: "[hidden]", overStrip: "[hidden]" });
152
- renderSecrets("clone https://me:pa55word@git.example/r", policy, notation); // "clone https://me:[hidden]@git.example/r"
169
+ renderSecrets("clone https://me:pa55word@git.example/r", policy); // "clone https://me:«redacted»@git.example/r"
153
170
  scanSecrets("clone https://me:pa55word@git.example/r", policy); // { outcome: "spans", spans: [{ start: 17, end: 25, family: "url-userinfo", kind: "url-credentials" }] }
154
171
  ```
155
172
 
@@ -164,5 +181,6 @@ removed or re-kinded export is named as breaking in the `@sema-agent/core` relea
164
181
  | --- | --- |
165
182
  | Credential reference | `readCredentialRef` `CREDENTIAL_REF_KINDS` `CREDENTIAL_REF_TRUST` `CREDENTIAL_REF_TEXT_MAX` `CREDENTIAL_REF_MAP_MAX` `CREDENTIAL_REF_PROBLEM_REASONS` |
166
183
  | Types | `CredentialRef` `CredentialRefKind` `CredentialRefRequestAdmission` `CredentialRefProblem` `CredentialRefProblemReason` `CredentialRefReading` |
167
- | Secret-shape detection | `CREDENTIAL_VOCABULARY` `CREDENTIAL_MIN_CHARS` `URL_OF_ANY_SCHEME` `SECRET_READING_KINDS` `scanSecrets` `isSecretShaped` `renderSecrets` `isSensitiveRecordKey` `isSensitiveHeaderName` `envNameView` `envValueView` `secretReadingViews` `splitCamelCase` |
168
- | Types | `CredentialVocabulary` `ValueFamilyRow` `CredentialWordRow` `CredentialWordPosition` `CredentialNameRow` `CredentialNamePlace` `EnvNameView` `EnvValueView` `RecordKeyView` `RedactionConfidence` `RedactionSource` `SecretGrade` `SecretReading` `SecretReadingKind` `SecretReadingView` `SecretReadingViews` `SecretScanPolicy` `SecretNotation` `SecretScan` `SecretSpan` `RenderOptions` `RedactionPass` `RedactionFinding` `RedactionReport` `SummarizableFinding` |
184
+ | Secret-shape detection | `CREDENTIAL_VOCABULARY` `CREDENTIAL_MIN_CHARS` `URL_OF_ANY_SCHEME` `SECRET_READING_KINDS` `scanSecrets` `isSecretShaped` `renderSecrets` `renderWithheld` `isSensitiveRecordKey` `isSensitiveHeaderName` `envNameView` `envValueView` `secretReadingViews` `splitCamelCase` |
185
+ | Redaction marks | `REDACTION_MARKS` `isRedactionMark` `redactionMarkerSpans` `eachRedactionMarkerSpan` |
186
+ | Types | `CredentialVocabulary` `ValueFamilyRow` `CredentialWordRow` `CredentialWordPosition` `CredentialNameRow` `CredentialNamePlace` `EnvNameView` `EnvValueView` `RecordKeyView` `RedactionConfidence` `RedactionSource` `SecretGrade` `SecretReading` `SecretReadingKind` `SecretReadingView` `SecretReadingViews` `SecretScanPolicy` `SecretScan` `SecretSpan` `RenderOptions` `RedactionPass` `RedactionFinding` `RedactionReport` `SummarizableFinding` `WithheldSpan` `RedactionMark` `RedactionMarkName` |
package/dist/index.d.ts CHANGED
@@ -2,4 +2,5 @@
2
2
  export { CREDENTIAL_REF_KINDS, CREDENTIAL_REF_TRUST, CREDENTIAL_REF_TEXT_MAX, CREDENTIAL_REF_MAP_MAX, CREDENTIAL_REF_PROBLEM_REASONS, readCredentialRef, type CredentialRef, type CredentialRefKind, type CredentialRefRequestAdmission, type CredentialRefProblem, type CredentialRefProblemReason, type CredentialRefReading, } from "./credential-ref.js";
3
3
  export { CREDENTIAL_VOCABULARY, CREDENTIAL_MIN_CHARS, URL_OF_ANY_SCHEME, SCHEME_FOLLOWER_WORDS, envNameView, envValueView, splitCamelCase, isSensitiveRecordKey, isSensitiveHeaderName, type CredentialVocabulary, type ValueFamilyRow, type CredentialFamilyId, type CredentialWordRow, type CredentialWordPosition, type CredentialNameRow, type CredentialNamePlace, type CredentialNameStrength, type EnvNameView, type EnvValueView, type RecordKeyView, type RedactionConfidence, type RedactionSource, } from "./secret-vocabulary.js";
4
4
  export { SECRET_READING_KINDS, secretReadingViews, type SecretReading, type SecretReadingKind, type SecretReadingView, type SecretReadingViews, } from "./secret-readings.js";
5
- export { scanSecrets, isSecretShaped, renderSecrets, userinfoHolds, type SecretGrade, type SecretScanPolicy, type SecretNotation, type SecretScan, type SecretSpan, type RenderOptions, type RedactionPass, type RedactionFinding, type RedactionReport, type SummarizableFinding, } from "./secret-scan.js";
5
+ export { REDACTION_MARKS, isRedactionMark, eachRedactionMarkerSpan, redactionMarkerSpans, type RedactionMark, type RedactionMarkName, } from "./redaction-marks.js";
6
+ export { scanSecrets, isSecretShaped, renderSecrets, renderWithheld, userinfoHolds, type SecretGrade, type SecretScanPolicy, type SecretScan, type SecretSpan, type RenderOptions, type RedactionPass, type RedactionFinding, type RedactionReport, type SummarizableFinding, type WithheldSpan, } from "./secret-scan.js";
package/dist/index.js CHANGED
@@ -1,4 +1,5 @@
1
1
  export { CREDENTIAL_REF_KINDS, CREDENTIAL_REF_TRUST, CREDENTIAL_REF_TEXT_MAX, CREDENTIAL_REF_MAP_MAX, CREDENTIAL_REF_PROBLEM_REASONS, readCredentialRef, } from "./credential-ref.js";
2
2
  export { CREDENTIAL_VOCABULARY, CREDENTIAL_MIN_CHARS, URL_OF_ANY_SCHEME, SCHEME_FOLLOWER_WORDS, envNameView, envValueView, splitCamelCase, isSensitiveRecordKey, isSensitiveHeaderName, } from "./secret-vocabulary.js";
3
3
  export { SECRET_READING_KINDS, secretReadingViews, } from "./secret-readings.js";
4
- export { scanSecrets, isSecretShaped, renderSecrets, userinfoHolds, } from "./secret-scan.js";
4
+ export { REDACTION_MARKS, isRedactionMark, eachRedactionMarkerSpan, redactionMarkerSpans, } from "./redaction-marks.js";
5
+ export { scanSecrets, isSecretShaped, renderSecrets, renderWithheld, userinfoHolds, } from "./secret-scan.js";
@@ -0,0 +1,49 @@
1
+ /**
2
+ * The redaction marks — the ONE table of what stands in a text where a layer withheld part of it, and the ONE reader of that
3
+ * table. Every layer that writes a mark (the engine's renderer, a host's own redactor, a display) takes a member of
4
+ * {@link REDACTION_MARKS}; every layer that must tell "a layer withheld this" from the text's own words reads
5
+ * {@link eachRedactionMarkerSpan} / {@link redactionMarkerSpans} / {@link isRedactionMark}. A mark spelled anywhere else is a
6
+ * second source that drifts the first time this table changes.
7
+ *
8
+ * One rule picks the member a layer writes: where the text around the mark already says what was withheld (a credential
9
+ * name's value, an assignment, a header value, a scheme word's credential, the password after a kept username, a URL query or
10
+ * fragment value, a sent credential a remote echoed), the mark is the bare `«redacted»`; where it does not, the mark names the
11
+ * kind — a credential read by its shape (`key`, or the vendor's own word), a URL userinfo withheld whole, a whole URL, an
12
+ * absolute path, or a stretch withheld because it could not be read (`oversized`). A credential residue found by the fragment
13
+ * grade adds `:partial`. Who withheld the text is not in the mark: a host that needs it reads the report's findings.
14
+ *
15
+ * The reader recognizes the members, exactly — not the shape they share: `«redacted:evil»` and the retired square-bracket
16
+ * spellings (`[redacted]`, `[redacted-url]` …) are ordinary text. Zero imports: every layer and package above reads it.
17
+ */
18
+ /** The marks, by name: `bare`, a kind, or a kind and `:partial`. The value is the member as written; the name is the
19
+ * member's words after `redacted` (the bare mark has none). Frozen: a write by a reader would change what every writer
20
+ * writes. */
21
+ export declare const REDACTION_MARKS: Readonly<{
22
+ readonly bare: "«redacted»";
23
+ readonly key: "«redacted:key»";
24
+ readonly "key:partial": "«redacted:key:partial»";
25
+ readonly "gh-token": "«redacted:gh-token»";
26
+ readonly "gh-token:partial": "«redacted:gh-token:partial»";
27
+ readonly "aws-key": "«redacted:aws-key»";
28
+ readonly "aws-key:partial": "«redacted:aws-key:partial»";
29
+ readonly "slack-token": "«redacted:slack-token»";
30
+ readonly "slack-token:partial": "«redacted:slack-token:partial»";
31
+ readonly "private-key": "«redacted:private-key»";
32
+ readonly "private-key:partial": "«redacted:private-key:partial»";
33
+ readonly userinfo: "«redacted:userinfo»";
34
+ readonly url: "«redacted:url»";
35
+ readonly path: "«redacted:path»";
36
+ readonly oversized: "«redacted:oversized»";
37
+ }>;
38
+ /** A mark's name in {@link REDACTION_MARKS}. */
39
+ export type RedactionMarkName = keyof typeof REDACTION_MARKS;
40
+ /** A member of {@link REDACTION_MARKS}: the only strings a layer writes where it withheld text. */
41
+ export type RedactionMark = (typeof REDACTION_MARKS)[RedactionMarkName];
42
+ /** Is `s`, whole, a member of {@link REDACTION_MARKS}? */
43
+ export declare function isRedactionMark(s: string): s is RedactionMark;
44
+ /** Where the marks sit in `s`, one [start, end) span at a time, leftmost first. Lazy: a reader that stops early (a display cut
45
+ * stops at its bound) pays for the marks it looked at, not for every mark an input of a million of them holds. Members never
46
+ * overlap: each opens with the only `«` and closes with the only `»` it holds. */
47
+ export declare function eachRedactionMarkerSpan(s: string): Generator<[number, number], void, undefined>;
48
+ /** Where the marks sit in `s`, all of them, as [start, end) spans, leftmost first. */
49
+ export declare function redactionMarkerSpans(s: string): Array<[number, number]>;
@@ -0,0 +1,40 @@
1
+ export const REDACTION_MARKS = Object.freeze({
2
+ bare: "«redacted»",
3
+ key: "«redacted:key»",
4
+ "key:partial": "«redacted:key:partial»",
5
+ "gh-token": "«redacted:gh-token»",
6
+ "gh-token:partial": "«redacted:gh-token:partial»",
7
+ "aws-key": "«redacted:aws-key»",
8
+ "aws-key:partial": "«redacted:aws-key:partial»",
9
+ "slack-token": "«redacted:slack-token»",
10
+ "slack-token:partial": "«redacted:slack-token:partial»",
11
+ "private-key": "«redacted:private-key»",
12
+ "private-key:partial": "«redacted:private-key:partial»",
13
+ userinfo: "«redacted:userinfo»",
14
+ url: "«redacted:url»",
15
+ path: "«redacted:path»",
16
+ oversized: "«redacted:oversized»",
17
+ });
18
+ const MEMBER_SHAPE = /^«redacted(?::[a-z-]{1,16})?(?::[a-z-]{1,16})?»$/;
19
+ const MEMBERS = new Set(Object.values(REDACTION_MARKS));
20
+ for (const member of MEMBERS) {
21
+ if (!MEMBER_SHAPE.test(member))
22
+ throw new Error(`redaction marks: ${member} is not of the marks' shape`);
23
+ for (const other of MEMBERS)
24
+ if (other !== member && other.startsWith(member))
25
+ throw new Error(`redaction marks: ${member} is the head of ${other}`);
26
+ }
27
+ const escapeLiteral = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
28
+ const MEMBER_RE = new RegExp([...MEMBERS].map(escapeLiteral).join("|"), "g");
29
+ export function isRedactionMark(s) {
30
+ return MEMBERS.has(s);
31
+ }
32
+ export function* eachRedactionMarkerSpan(s) {
33
+ if (!s.includes("«redacted"))
34
+ return;
35
+ for (const m of s.matchAll(MEMBER_RE))
36
+ yield [m.index, m.index + m[0].length];
37
+ }
38
+ export function redactionMarkerSpans(s) {
39
+ return [...eachRedactionMarkerSpan(s)];
40
+ }
@@ -1,32 +1,35 @@
1
1
  /**
2
2
  * The matcher and the renderer of the credential vocabulary (secret-vocabulary.ts), each reading the caller's policy
3
- * whole — the grade of form, the readings a text is read through, the URL userinfo rendering — and the renderer the
4
- * caller's notation (which marker a hit is written as):
3
+ * whole — the grade of form, the readings a text is read through, the URL userinfo rendering:
5
4
  * - {@link scanSecrets} answers WHERE the vocabulary sees a credential (original-text spans, or "the whole text" when a
6
5
  * reading cannot be afforded), and {@link isSecretShaped} whether it sees one at all;
7
- * - {@link renderSecrets} answers what the text reads as with every credential withheld, in the caller's notation. A caller
8
- * that also withholds what is not a credential (a layer's own URL or path rule) appends its passes, which run in the
9
- * same pipeline, through the same readings.
10
- * No layer's policy or marker lives here: every one is an argument, every argument is required.
6
+ * - {@link renderSecrets} answers what the text reads as with every credential withheld, each hit written as its row's
7
+ * member of the marks table (redaction-marks.ts). A caller that also withholds what is not a credential (a layer's own
8
+ * URL or path rule) appends its passes, which run in the same pipeline, through the same readings.
9
+ * No layer's policy lives here: every one is an argument, every argument is required. No mark is spelled here: every one
10
+ * is a member of the marks table.
11
11
  *
12
12
  * The renderer is one SEQUENTIAL multi-pass engine: each pass scans the text the earlier passes left, so a hit that runs
13
- * into an earlier marker reads the marker the caller chose. A row whose wide form captures a label keeps it and replaces
14
- * only the value (`GITHUB_TOKEN=ghp_xxx` keeps `GITHUB_TOKEN=`), and a second run over already-rendered text is a no-op.
15
- * Hits that touch write one mark, whichever pass made each.
16
- * Every pass declares what it detects (kind), how sure it is (confidence) and the marker it writes; a report collector,
17
- * when given, receives one finding per hit in the ORIGINAL input's coordinates, mapped back through the earlier passes'
18
- * edits (best-effort where redactions overlap — a span landing inside an earlier replacement anchors to that edit's start).
13
+ * into an earlier marker reads that marker. A row whose wide form captures a label keeps it and replaces only the value
14
+ * (`GITHUB_TOKEN=ghp_xxx` keeps `GITHUB_TOKEN=`), and a second run over already-rendered text is a no-op. A mark is
15
+ * atomic: a hit whose withheld part IS a mark is no hit (an earlier layer's mark stands as that layer wrote it), and a hit
16
+ * that reaches into a mark takes the whole mark with it. Hits that touch or overlap write one mark, whichever pass made each
17
+ * ({@link renderWithheld} is that one law, exported). Every pass declares what it detects (kind), how sure it is
18
+ * (confidence) and the marker it writes; a report collector, when given, receives one finding per hit in the ORIGINAL
19
+ * input's coordinates, mapped back through the earlier passes' edits (best-effort where redactions overlap — a span landing
20
+ * inside an earlier replacement anchors to that edit's start).
19
21
  */
20
- import { type RedactionConfidence, type RedactionSource, type ValueFamilyRow, type CredentialFamilyId } from "./secret-vocabulary.js";
22
+ import { type CredentialFamilyId, type RedactionConfidence, type RedactionSource, type ValueFamilyRow } from "./secret-vocabulary.js";
21
23
  import { type ReadingSet, type SecretReading } from "./secret-readings.js";
22
- /** One redaction hit: what was detected, where (ORIGINAL-input coordinates), how confidently, the marker
23
- * literal written in its place, which rule table minted it, and — present when a vocabulary row minted it — the
24
- * value family (the row's stable id; a host-leak hit such as a URL or a path names none). */
24
+ import { type RedactionMark } from "./redaction-marks.js";
25
+ /** One redaction hit: what was detected, where (ORIGINAL-input coordinates), how confidently, the mark written in its
26
+ * place, which rule table minted it, and — present when a vocabulary row minted it — the value family (the row's stable
27
+ * id; a host-leak hit such as a URL or a path names none). */
25
28
  export interface RedactionFinding {
26
29
  kind: string;
27
30
  span: readonly [number, number];
28
31
  confidence: RedactionConfidence;
29
- marker: string;
32
+ marker: RedactionMark;
30
33
  source: RedactionSource;
31
34
  family?: string;
32
35
  }
@@ -37,10 +40,10 @@ export interface SummarizableFinding {
37
40
  confidence: RedactionConfidence;
38
41
  }
39
42
  /** The optional collector the renderer accepts. `findings` receives one entry per hit. `preexistingMarkers` counts the
40
- * marks ALREADY present in the input before any pass ran — a nonzero value means the text arrived pre-redacted or carries
41
- * caller-declared marks, either way "a mark is present" is not proof the pipeline ran. Which strings are marks is the
42
- * caller's notation, so the caller sets it; the renderer never writes it. Caller-owned: pass `{ findings: [] }` and read
43
- * it back after the call. */
43
+ * marks ALREADY present in the input before any pass ran (the marks table's reader) — a nonzero value means the text
44
+ * arrived pre-redacted or carries marks another layer wrote, either way "a mark is present" is not proof the pipeline
45
+ * ran. The caller sets it; the renderer never writes it. Caller-owned: pass `{ findings: [] }` and read it back after
46
+ * the call. */
44
47
  export interface RedactionReport {
45
48
  findings: RedactionFinding[];
46
49
  preexistingMarkers?: number;
@@ -50,17 +53,37 @@ export interface RedactionReport {
50
53
  export interface RedactionPass {
51
54
  kind: string;
52
55
  confidence: RedactionConfidence;
53
- marker: string;
56
+ /** The member of the marks table this pass's replacement writes. A replacer function may write another member per hit
57
+ * (the vocabulary's userinfo row writes the bare mark where it keeps a username; its vendor group writes each hit's row
58
+ * mark); a finding records the member its hit wrote. */
59
+ marker: RedactionMark;
54
60
  /** The rule table that owns this pass's rule — copied verbatim onto every finding it mints (ANG-11), whoever
55
61
  * runs the pass. */
56
62
  source: RedactionSource;
57
63
  re: RegExp;
58
- /** A literal replacement (only `$1` substitution is used by these passes) or a replacer fn. */
64
+ /** A literal replacement (only `$1` substitution is used by these passes) or a replacer fn; either writes its member of
65
+ * the marks table in place of what it withholds. A caller's literal replacement is checked when the renderer is called:
66
+ * its text outside the group references is `marker`, or the call is refused (a `TypeError`). */
59
67
  replace: string | ((match: string, ...groups: string[]) => string);
60
68
  /** The value family a hit belongs to: one id, or — for a pass joining several rows — the id of the row that
61
69
  * matches at the hit's offset in the pass input. Absent = the pass is no vocabulary row. */
62
70
  family?: string | ((input: string, at: number) => string);
63
71
  }
72
+ /** One region of a text a layer withholds, and the member of the marks table written in its place. */
73
+ export interface WithheldSpan {
74
+ readonly start: number;
75
+ readonly end: number;
76
+ readonly mark: RedactionMark;
77
+ }
78
+ /**
79
+ * `text` with every span withheld, by the one law the renderer writes its own hits by — for a layer that finds its regions
80
+ * itself (the engine's eraser of the credentials a request carried): a span that is, exactly, a mark `text` already holds
81
+ * is no span (the mark stands as written); one that reaches into a mark takes the whole mark; spans that overlap or touch
82
+ * are one region, written as the mark of its widest span (of two over the same region, the later one in `spans`). `text`
83
+ * itself when nothing is withheld. Throws a `TypeError` on a span that is not a range of `text` or whose mark is no member
84
+ * of the marks table.
85
+ */
86
+ export declare function renderWithheld(text: string, spans: readonly WithheldSpan[]): string;
64
87
  /** The `format-free` reading of a text that holds a format character: the text with every `\p{Cf}` removed (the view),
65
88
  * where the format characters stood (`cuts`: the view offset of every view character that has format characters right
66
89
  * before it, the first character excepted), and the way back to the text's own coordinates. */
@@ -78,11 +101,10 @@ export interface FormatFreeReading {
78
101
  }
79
102
  /** The reading of `input` (a text that holds a format character). Not on the package barrel; the module's own pins read it. */
80
103
  export declare function readFormatFree(input: string): FormatFreeReading;
81
- /** One run of the engine: the marker a text no reading can afford is replaced with whole (`overStrip`), the readings
82
- * (the policy's, as a set), what a text the percent layers cannot read to its end answers (the policy's
83
- * `onUnresolvedReading`; read only when the readings hold `percent-layers`), and the optional report. */
104
+ /** One run of the engine: the readings (the policy's, as a set), what a text the percent layers cannot read to its end
105
+ * answers (the policy's `onUnresolvedReading`; read only when the readings hold `percent-layers`), and the optional report.
106
+ * A text no reading can afford is replaced whole by the `oversized` mark. */
84
107
  export interface EngineRun {
85
- readonly overStrip: string;
86
108
  readonly readings: ReadingSet;
87
109
  readonly onUnresolvedReading?: SecretScanPolicy["onUnresolvedReading"] | undefined;
88
110
  readonly report?: RedactionReport | undefined;
@@ -118,16 +140,6 @@ export interface SecretScanPolicy {
118
140
  readonly urlUserinfo: "keep-username" | "whole";
119
141
  readonly onUnresolvedReading: "secret" | "ignore";
120
142
  }
121
- /** The marks a hit is written as — the caller's own spellings, none of them a default here. A row's hit is written as
122
- * `byFamily[row id]`, else `byKind[row kind]`, else `value`; a text no reading can afford is replaced whole by
123
- * `overStrip`. Rows that run as one pass (they share a `pass`) are one alternation and write one mark: a `byFamily` that
124
- * spells two of them differently is refused. A mark is a non-empty string. */
125
- export interface SecretNotation {
126
- readonly value: string;
127
- readonly byKind?: Readonly<Record<string, string>>;
128
- readonly byFamily?: Readonly<Partial<Record<CredentialFamilyId, string>>>;
129
- readonly overStrip: string;
130
- }
131
143
  /** Where the vocabulary sees a credential: the withheld part of one hit (a kept label or username outside it), in the
132
144
  * text's own coordinates, with the row that found it. */
133
145
  export interface SecretSpan {
@@ -149,37 +161,42 @@ export type SecretScan = {
149
161
  readonly outcome: "whole";
150
162
  readonly reason: "reading-unresolved" | "scan-budget" | "mapping-failed";
151
163
  };
152
- /** What {@link renderSecrets} takes beside the policy and the notation: the caller's own passes, run after the vocabulary's
153
- * in the same pipeline and through the same readings (each writes its own `marker`), and a report collector. */
164
+ /** What {@link renderSecrets} takes beside the policy: the caller's own passes, run after the vocabulary's in the same
165
+ * pipeline and through the same readings (each writes its own `marker`, a member of the marks table), and a report
166
+ * collector. */
154
167
  export interface RenderOptions {
155
168
  readonly passes?: readonly RedactionPass[] | undefined;
156
169
  readonly report?: RedactionReport | undefined;
157
170
  }
158
- /** Does a URL userinfo (no trailing `@`) hold a user (before its first `:`) or a password (after it)? An empty interval holds
159
- * nothing — `scheme://@host` and `scheme://:@host` hold neither — as {@link userinfoWithheld} reads under `keep-username`. */
171
+ /** Does a URL userinfo (no trailing `@`) hold a user (before its first `:` outside its marks) or a password (after it)? An empty
172
+ * interval holds nothing — `scheme://@host` and `scheme://:@host` hold neither — as {@link userinfoWithheld} reads under
173
+ * `keep-username`. */
160
174
  export declare function userinfoHolds(userinfo: string): boolean;
161
175
  /**
162
176
  * The wide forms rendered as passes, in the vocabulary's order: one pass per group (a group of several rows is ONE
163
- * leftmost alternation, its hits attributed to the row that matches at the hit's offset). A row whose wide form
164
- * captures a label keeps it ahead of the marker; the URL userinfo is split per `urlUserinfo`.
177
+ * leftmost alternation, its hits attributed to the row that matches at the hit's offset, each written as that row's
178
+ * mark). A row whose wide form captures a label keeps it ahead of the marker; the URL userinfo is split per `urlUserinfo`
179
+ * — a userinfo withheld whole writes the row's `userinfo` mark, a password withheld behind a kept username the bare mark
180
+ * (the username says what follows it).
165
181
  */
166
- export declare function egressPasses(groups: ReadonlyArray<readonly ValueFamilyRow[]>, markOf: (row: ValueFamilyRow) => string, urlUserinfo: SecretScanPolicy["urlUserinfo"]): RedactionPass[];
182
+ export declare function egressPasses(groups: ReadonlyArray<readonly ValueFamilyRow[]>, urlUserinfo: SecretScanPolicy["urlUserinfo"]): RedactionPass[];
167
183
  /**
168
- * `text` with every credential the vocabulary finds under `policy` withheld, written in `notation`, and the caller's own
169
- * passes ({@link RenderOptions.passes}) run after the vocabulary's in the same pipeline. One finding per hit goes to
170
- * `options.report` when given (`report.preexistingMarkers` is the caller's to set: which strings are marks is its
171
- * notation). The policy and the notation are read and checked on every call; what they read as picks a compilation kept
172
- * across calls. A mark is written as spelled. Throws a `TypeError` on a policy or a notation that is not whole.
184
+ * `text` with every credential the vocabulary finds under `policy` withheld, each hit written as its row's member of the
185
+ * marks table (the fragment grade's as the `:partial` member), and the caller's own passes ({@link RenderOptions.passes})
186
+ * run after the vocabulary's in the same pipeline. One finding per hit goes to `options.report` when given
187
+ * (`report.preexistingMarkers` is the caller's to set). The policy and the caller's passes are read and checked on every
188
+ * call; what the policy reads as picks a compilation kept across calls. Throws a `TypeError` on a policy that is not whole
189
+ * or a caller's pass whose marker is no member of the marks table.
173
190
  */
174
- export declare function renderSecrets(text: string, policy: SecretScanPolicy, notation: SecretNotation, options?: RenderOptions): string;
191
+ export declare function renderSecrets(text: string, policy: SecretScanPolicy, options?: RenderOptions): string;
175
192
  /**
176
193
  * Where the vocabulary sees a credential in `text` under `policy`: every hit of the grade's form of every row, on every
177
194
  * reading of the policy — each text the text readings derive, scanned by the character readings — as the hit's withheld
178
195
  * part in `text`'s own coordinates (the whole written units a derived hit was read from; deduplicated, sorted by start) —
179
196
  * or the whole text, when a reading cannot be afforded (more than 64 places where format characters stood: `scan-budget`)
180
197
  * or the percent layers ran out before the text did under `onUnresolvedReading: "secret"` (`reading-unresolved`). The
181
- * matcher answers where a credential is; the renderer additionally leaves a hit that already reads as its own rendering
182
- * (a value that is already a mark), which only a notation can tell. Throws a `TypeError` on a policy that is not whole.
198
+ * matcher answers where its forms match, a mark the text holds included (`password=«redacted:key»` is a span here); the renderer
199
+ * leaves a hit whose withheld part is exactly a mark the text holds. Throws a `TypeError` on a policy that is not whole.
183
200
  */
184
201
  export declare function scanSecrets(text: string, policy: SecretScanPolicy): SecretScan;
185
202
  /** Does the vocabulary see a credential in `text` under `policy`? {@link scanSecrets} answers the whole text or a span. */