@doccloak/core 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +84 -0
- package/dist/dom/docx.d.ts +29 -2
- package/dist/dom/docx.js +67 -27
- package/dist/dom/index.d.ts +2 -0
- package/dist/dom/index.js +3 -0
- package/dist/dom/office.d.ts +75 -0
- package/dist/dom/office.js +116 -0
- package/dist/dom/opc.d.ts +48 -0
- package/dist/dom/opc.js +159 -0
- package/dist/dom/xlsx.d.ts +64 -0
- package/dist/dom/xlsx.js +280 -0
- package/dist/engine.d.ts +8 -5
- package/dist/engine.js +21 -7
- package/dist/index.d.ts +2 -0
- package/dist/index.js +3 -0
- package/dist/providers/bardsai.js +16 -16
- package/dist/providers/gliner-base.d.ts +92 -0
- package/dist/providers/gliner-base.js +357 -0
- package/dist/providers/gliner.d.ts +22 -6
- package/dist/providers/gliner.js +43 -20
- package/dist/session.d.ts +36 -1
- package/dist/session.js +205 -30
- package/package.json +3 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,90 @@ All notable changes to `@doccloak/core` are documented here. The format is
|
|
|
4
4
|
based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the
|
|
5
5
|
project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [0.11.0] - 2026-09-23
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
|
|
11
|
+
- **Variant-faithful restore (T171):** person variants still share one
|
|
12
|
+
identity, but every distinct variant now gets its own token, so
|
|
13
|
+
`deanonymize` writes back exactly what was there instead of the longest
|
|
14
|
+
known form. In labeled mode the token keeps the group's number and adds
|
|
15
|
+
a positional suffix: `[PERSON_2]` for "John Smith", `[PERSON_2_LAST]`
|
|
16
|
+
for "Smith", `[PERSON_2_FIRST]` for "John", `_MIDDLE` / `_SHORT` /
|
|
17
|
+
`_FULL` / `_ALT` for the other shapes, with a counter on collisions
|
|
18
|
+
(`[PERSON_2_LAST_2]` for "smith"). In surrogate mode a variant that is a
|
|
19
|
+
strict part of the canonical name takes the matching words of the
|
|
20
|
+
surrogate with the variant's casing ("Smith" -> "Nowak", "smith" ->
|
|
21
|
+
"nowak"); fuller forms and reshaped surrogates fall back to the group
|
|
22
|
+
surrogate as before. Blanked mode is unchanged. The tolerant restore
|
|
23
|
+
pass resolves mangled variant tokens like any other.
|
|
24
|
+
- **PERSON pre-pass in `anonymizeText`:** people are mapped before other
|
|
25
|
+
entities, fullest name first and then in reading order, so a variant
|
|
26
|
+
group always forms around the full name and several people are numbered
|
|
27
|
+
in the order they appear (`[PERSON_1]` is now the first person in the
|
|
28
|
+
text; other entity types keep their historical numbering order).
|
|
29
|
+
- **`renameLabel` returns the applied `[oldLabel, newLabel]` pairs** (was
|
|
30
|
+
`void`). Renaming a group's base token re-derives its variant tokens
|
|
31
|
+
(`[PERSON_1]` -> `[CLIENT]` takes `[PERSON_1_LAST]` to `[CLIENT_LAST]`);
|
|
32
|
+
renaming a variant token renames only that token and detaches it from
|
|
33
|
+
the group.
|
|
34
|
+
- `deserialize` / `importEntries` re-link variant tokens to their base and
|
|
35
|
+
keep legacy many-to-one maps restoring the longest variant.
|
|
36
|
+
|
|
37
|
+
### Fixed
|
|
38
|
+
|
|
39
|
+
- **Office value scrubbing applies longer values first and sees URL-encoded
|
|
40
|
+
forms:** hyperlink targets, field instructions and formula literals in
|
|
41
|
+
.docx / .xlsx exports are scrubbed with the longest sensitive value first,
|
|
42
|
+
so a name variant listed before the full name ("Kowalski" before
|
|
43
|
+
"Jan Kowalski") no longer leaves "Jan" behind. External relationship
|
|
44
|
+
targets also match the percent-encoded form of each value, so
|
|
45
|
+
"Jan%20Kowalski" in a link is removed as a whole. `.docx` and `.xlsx`
|
|
46
|
+
writers now share one `replaceSensitiveValues` implementation.
|
|
47
|
+
|
|
48
|
+
## [0.10.0] - 2026-09-23
|
|
49
|
+
|
|
50
|
+
### Added
|
|
51
|
+
|
|
52
|
+
- **GLiNER PII Base provider (T122):** new `gliner-base` provider for
|
|
53
|
+
`knowledgator/gliner-pii-base-v1.0` (~197 MB quint8, Apache-2.0,
|
|
54
|
+
pinned revision + SHA-256). Span-level (`markerV0`) decoding with
|
|
55
|
+
zero-shot custom labels; strongest option for English and dates.
|
|
56
|
+
Exported as `GlinerBaseProvider` plus `GLINER_BASE_MODEL_URL` /
|
|
57
|
+
`_REVISION` / `_SHA256`, `buildSpanIndices` and `decodeSpanLogits`.
|
|
58
|
+
`ProviderId` now includes `'gliner-base'` and `createEngine` registers it.
|
|
59
|
+
|
|
60
|
+
- **Office file redaction (T110):** `@doccloak/core/dom` gains
|
|
61
|
+
`readXlsx` / `writeAnonymizedXlsx` / `isExcelFile` and a
|
|
62
|
+
detection-driven `redactOfficeFile` / `analyzeOfficeFile` flow for
|
|
63
|
+
.docx and .xlsx that keeps placeholders consistent with a live
|
|
64
|
+
`AnonymizationSession`. `writeAnonymizedDocx` accepts an option to
|
|
65
|
+
accept tracked changes (deletions dropped, insertions unwrapped,
|
|
66
|
+
author/date change records removed). Defaults preserve the previous
|
|
67
|
+
output byte-for-byte.
|
|
68
|
+
|
|
69
|
+
### Changed
|
|
70
|
+
|
|
71
|
+
- **Light model swapped (T121):** the `gliner` provider now loads
|
|
72
|
+
`knowledgator/gliner-pii-small-v1.0` (~83 MB, same tokenizer family)
|
|
73
|
+
instead of `gliner-pii-edge-v1.0`. Cached copies of the old edge model
|
|
74
|
+
are evicted best-effort on load. Labels read "GLiNER PII Small".
|
|
75
|
+
|
|
76
|
+
- **Provider descriptions (T127):** registry copy reflects the 2026-08
|
|
77
|
+
14-language benchmark. `pickDefaultProvider` is unchanged: `bardsai` on
|
|
78
|
+
capable devices, `gliner` on mobile / low-memory devices.
|
|
79
|
+
|
|
80
|
+
- **ONNX Runtime 1.29.0:** providers import `onnxruntime-web/webgpu`
|
|
81
|
+
(native WebGPU EP with wasm fallback). Hosts that self-host the ORT
|
|
82
|
+
runtime files must now serve `ort-wasm-simd-threaded.asyncify.wasm`
|
|
83
|
+
and `.asyncify.mjs` instead of the plain / `.jsep` pair.
|
|
84
|
+
|
|
85
|
+
### Fixed
|
|
86
|
+
|
|
87
|
+
- **bardsai with transformers.js 4.x:** subtoken alignment no longer calls
|
|
88
|
+
the removed `tokenizer.model.convert_ids_to_tokens` (which made
|
|
89
|
+
detection throw on transformers.js 4); it uses `tokenizer.tokenize()`.
|
|
90
|
+
|
|
7
91
|
## [0.9.0] - 2026-08-21
|
|
8
92
|
|
|
9
93
|
### Added
|
package/dist/dom/docx.d.ts
CHANGED
|
@@ -13,8 +13,10 @@ export { normalizeReplacements };
|
|
|
13
13
|
export type { ValueReplacement };
|
|
14
14
|
/**
|
|
15
15
|
* Represents a text node in the docx XML with its position in the flat text.
|
|
16
|
+
* Also reused by the xlsx module (T110): the same flat-text replacement
|
|
17
|
+
* machinery applies to any XML part whose text lives in leaf elements.
|
|
16
18
|
*/
|
|
17
|
-
interface TextNodeMapping {
|
|
19
|
+
export interface TextNodeMapping {
|
|
18
20
|
/** The <w:t> element */
|
|
19
21
|
element: Element;
|
|
20
22
|
/** Start index in the flat text */
|
|
@@ -59,6 +61,31 @@ interface ContentPartExtraction {
|
|
|
59
61
|
* Read a .docx file and extract its text content with position mapping.
|
|
60
62
|
*/
|
|
61
63
|
export declare function readDocx(file: File): Promise<DocxExtraction>;
|
|
64
|
+
/**
|
|
65
|
+
* Options for writeAnonymizedDocx (T110). All default to off, preserving the
|
|
66
|
+
* historical output byte-for-byte for existing callers.
|
|
67
|
+
*/
|
|
68
|
+
export interface DocxWriteOptions {
|
|
69
|
+
/**
|
|
70
|
+
* Accept and remove tracked changes: deletions (w:del and friends) are
|
|
71
|
+
* dropped, insertions (w:ins, w:moveTo) are unwrapped into plain runs, and
|
|
72
|
+
* property-change records (w:rPrChange etc., which carry author and date
|
|
73
|
+
* fingerprints) are removed. Oregon SB 2025-205 flags revision metadata as
|
|
74
|
+
* identifying, so the file-redaction flow turns this on.
|
|
75
|
+
*/
|
|
76
|
+
acceptTrackedChanges?: boolean;
|
|
77
|
+
}
|
|
78
|
+
/**
|
|
79
|
+
* Apply a list of flat-text replacements to mapped text nodes, handling
|
|
80
|
+
* replacements that span multiple elements. Ranges are normalized (sorted,
|
|
81
|
+
* overlaps clamped) and applied end-to-start so earlier positions stay valid.
|
|
82
|
+
* Shared by the docx and xlsx writers (T110).
|
|
83
|
+
*/
|
|
84
|
+
export declare function applyTextReplacements(textNodes: TextNodeMapping[], replacements: Array<{
|
|
85
|
+
start: number;
|
|
86
|
+
end: number;
|
|
87
|
+
replacement: string;
|
|
88
|
+
}>): void;
|
|
62
89
|
/**
|
|
63
90
|
* Apply text replacements to the docx XML, preserving all formatting.
|
|
64
91
|
* Takes the original extraction and a list of replacements (sorted by position),
|
|
@@ -71,7 +98,7 @@ export declare function writeAnonymizedDocx(extraction: DocxExtraction, replacem
|
|
|
71
98
|
start: number;
|
|
72
99
|
end: number;
|
|
73
100
|
replacement: string;
|
|
74
|
-
}>, valueReplacements?: ValueReplacement[]): Promise<Blob>;
|
|
101
|
+
}>, valueReplacements?: ValueReplacement[], options?: DocxWriteOptions): Promise<Blob>;
|
|
75
102
|
/**
|
|
76
103
|
* Get the file extension from a filename.
|
|
77
104
|
*/
|
package/dist/dom/docx.js
CHANGED
|
@@ -8,6 +8,7 @@
|
|
|
8
8
|
*/
|
|
9
9
|
import JSZip from 'jszip';
|
|
10
10
|
import { normalizeReplacements } from "../docx.js";
|
|
11
|
+
import { replaceSensitiveValues } from "./opc.js";
|
|
11
12
|
export { normalizeReplacements };
|
|
12
13
|
const W_NS = 'http://schemas.openxmlformats.org/wordprocessingml/2006/main';
|
|
13
14
|
/**
|
|
@@ -153,6 +154,63 @@ export async function readDocx(file) {
|
|
|
153
154
|
contentParts,
|
|
154
155
|
};
|
|
155
156
|
}
|
|
157
|
+
/**
|
|
158
|
+
* Apply a list of flat-text replacements to mapped text nodes, handling
|
|
159
|
+
* replacements that span multiple elements. Ranges are normalized (sorted,
|
|
160
|
+
* overlaps clamped) and applied end-to-start so earlier positions stay valid.
|
|
161
|
+
* Shared by the docx and xlsx writers (T110).
|
|
162
|
+
*/
|
|
163
|
+
export function applyTextReplacements(textNodes, replacements) {
|
|
164
|
+
const sorted = normalizeReplacements(replacements);
|
|
165
|
+
for (const repl of [...sorted].reverse()) {
|
|
166
|
+
applyReplacement(textNodes, repl.start, repl.end, repl.replacement);
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
/**
|
|
170
|
+
* Tracked-change elements that are removed outright when accepting changes:
|
|
171
|
+
* deleted content, move sources, and every *PrChange record (each carries a
|
|
172
|
+
* w:author/w:date pair identifying the reviser).
|
|
173
|
+
*/
|
|
174
|
+
const TRACKED_REMOVE = new Set([
|
|
175
|
+
'del', 'moveFrom', 'delText', 'delInstrText',
|
|
176
|
+
'rPrChange', 'pPrChange', 'sectPrChange', 'tblPrChange', 'tblGridChange',
|
|
177
|
+
'tcPrChange', 'trPrChange', 'numberingChange',
|
|
178
|
+
'cellDel', 'cellIns', 'cellMerge',
|
|
179
|
+
'moveFromRangeStart', 'moveFromRangeEnd', 'moveToRangeStart', 'moveToRangeEnd',
|
|
180
|
+
'customXmlDelRangeStart', 'customXmlDelRangeEnd',
|
|
181
|
+
'customXmlInsRangeStart', 'customXmlInsRangeEnd',
|
|
182
|
+
'customXmlMoveFromRangeStart', 'customXmlMoveFromRangeEnd',
|
|
183
|
+
'customXmlMoveToRangeStart', 'customXmlMoveToRangeEnd',
|
|
184
|
+
]);
|
|
185
|
+
/** Tracked-change wrappers whose content is kept: the wrapper is unwrapped. */
|
|
186
|
+
const TRACKED_UNWRAP = new Set(['ins', 'moveTo']);
|
|
187
|
+
/**
|
|
188
|
+
* Accept all tracked changes in one content part: remove deletions and
|
|
189
|
+
* change records, unwrap insertions. Operates on the live DOM after text
|
|
190
|
+
* replacements have been applied, so redaction offsets are unaffected.
|
|
191
|
+
*/
|
|
192
|
+
function acceptTrackedChangesInPart(xmlDoc) {
|
|
193
|
+
// Snapshot first: live HTMLCollections reorder while elements are removed.
|
|
194
|
+
const all = xmlDoc.getElementsByTagName('*');
|
|
195
|
+
const snapshot = [];
|
|
196
|
+
for (let i = 0; i < all.length; i++)
|
|
197
|
+
snapshot.push(all[i]);
|
|
198
|
+
for (const el of snapshot) {
|
|
199
|
+
if (el.namespaceURI !== W_NS)
|
|
200
|
+
continue;
|
|
201
|
+
if (TRACKED_REMOVE.has(el.localName)) {
|
|
202
|
+
el.parentNode?.removeChild(el);
|
|
203
|
+
}
|
|
204
|
+
else if (TRACKED_UNWRAP.has(el.localName)) {
|
|
205
|
+
const parent = el.parentNode;
|
|
206
|
+
if (!parent)
|
|
207
|
+
continue;
|
|
208
|
+
while (el.firstChild)
|
|
209
|
+
parent.insertBefore(el.firstChild, el);
|
|
210
|
+
parent.removeChild(el);
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
}
|
|
156
214
|
/**
|
|
157
215
|
* Apply text replacements to the docx XML, preserving all formatting.
|
|
158
216
|
* Takes the original extraction and a list of replacements (sorted by position),
|
|
@@ -161,20 +219,17 @@ export async function readDocx(file) {
|
|
|
161
219
|
*
|
|
162
220
|
* Returns a new .docx file as a Blob.
|
|
163
221
|
*/
|
|
164
|
-
export async function writeAnonymizedDocx(extraction, replacements, valueReplacements = []) {
|
|
165
|
-
|
|
166
|
-
//
|
|
167
|
-
|
|
168
|
-
// Build a map of flat-text ranges that need replacement
|
|
169
|
-
// Process from end to start to preserve positions
|
|
170
|
-
const reverseSorted = [...sorted].reverse();
|
|
171
|
-
for (const repl of reverseSorted) {
|
|
172
|
-
applyReplacement(extraction.textNodes, repl.start, repl.end, repl.replacement);
|
|
173
|
-
}
|
|
222
|
+
export async function writeAnonymizedDocx(extraction, replacements, valueReplacements = [], options = {}) {
|
|
223
|
+
// Apply flat-text replacements to the mapped text nodes (spanning runs when
|
|
224
|
+
// needed), from end to start so positions stay valid.
|
|
225
|
+
applyTextReplacements(extraction.textNodes, replacements);
|
|
174
226
|
// Serialize all modified XML documents back to the zip
|
|
175
227
|
const serializer = new XMLSerializer();
|
|
176
228
|
for (const part of extraction.contentParts) {
|
|
177
229
|
sanitizeContentPartAttributes(part.xmlDoc, valueReplacements);
|
|
230
|
+
if (options.acceptTrackedChanges) {
|
|
231
|
+
acceptTrackedChangesInPart(part.xmlDoc);
|
|
232
|
+
}
|
|
178
233
|
const xmlStr = serializer.serializeToString(part.xmlDoc);
|
|
179
234
|
extraction.zip.file(part.path, xmlStr);
|
|
180
235
|
}
|
|
@@ -201,7 +256,7 @@ function sanitizeContentPartAttributes(xmlDoc, valueReplacements) {
|
|
|
201
256
|
attr.value = 'R';
|
|
202
257
|
}
|
|
203
258
|
else if (attr.localName === 'instr') {
|
|
204
|
-
attr.value =
|
|
259
|
+
attr.value = replaceSensitiveValues(attr.value, valueReplacements, false);
|
|
205
260
|
}
|
|
206
261
|
}
|
|
207
262
|
if (el.localName === 'docPr' || el.localName === 'cNvPr') {
|
|
@@ -214,21 +269,6 @@ function sanitizeContentPartAttributes(xmlDoc, valueReplacements) {
|
|
|
214
269
|
}
|
|
215
270
|
}
|
|
216
271
|
}
|
|
217
|
-
/**
|
|
218
|
-
* Replace every occurrence of each sensitive value (case-insensitive) in a string.
|
|
219
|
-
* Uses a replacer function so '$' sequences in values or placeholders are inert.
|
|
220
|
-
*/
|
|
221
|
-
function replaceValues(input, valueReplacements, urlEncode) {
|
|
222
|
-
let result = input;
|
|
223
|
-
for (const { value, replacement } of valueReplacements) {
|
|
224
|
-
if (!value)
|
|
225
|
-
continue;
|
|
226
|
-
const escaped = value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
227
|
-
const substitute = urlEncode ? encodeURIComponent(replacement) : replacement;
|
|
228
|
-
result = result.replace(new RegExp(escaped, 'gi'), () => substitute);
|
|
229
|
-
}
|
|
230
|
-
return result;
|
|
231
|
-
}
|
|
232
272
|
/**
|
|
233
273
|
* Scrub metadata parts of the package that survive text replacement untouched:
|
|
234
274
|
* document properties, comment author registry, revision fingerprints,
|
|
@@ -329,7 +369,7 @@ async function sanitizeDocxMetadata(zip, valueReplacements) {
|
|
|
329
369
|
}
|
|
330
370
|
if (rel.getAttribute('TargetMode') === 'External') {
|
|
331
371
|
const target = rel.getAttribute('Target') ?? '';
|
|
332
|
-
const scrubbed =
|
|
372
|
+
const scrubbed = replaceSensitiveValues(target, valueReplacements, true);
|
|
333
373
|
if (scrubbed !== target)
|
|
334
374
|
rel.setAttribute('Target', scrubbed);
|
|
335
375
|
}
|
package/dist/dom/index.d.ts
CHANGED
package/dist/dom/index.js
CHANGED
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @doccloak/core/dom - detection-driven office file redaction (T110).
|
|
3
|
+
*
|
|
4
|
+
* Ties the format modules (docx.ts, xlsx.ts) to the detection pipeline and an
|
|
5
|
+
* AnonymizationSession, so a host can hand over a File plus its live session
|
|
6
|
+
* and get back a redacted copy whose placeholders are consistent with every
|
|
7
|
+
* prompt redacted in the same session: an AI reply that mentions the file's
|
|
8
|
+
* placeholders restores through the same session map.
|
|
9
|
+
*
|
|
10
|
+
* PDF is intentionally NOT handled here: layout-preserving PDF redaction is
|
|
11
|
+
* out of scope for this module.
|
|
12
|
+
*/
|
|
13
|
+
import type { DetectedEntity } from '../types.js';
|
|
14
|
+
import type { AnonymizationSession } from '../session.js';
|
|
15
|
+
/** File kinds the office redaction flow can regenerate. */
|
|
16
|
+
export type OfficeFileKind = 'docx' | 'xlsx';
|
|
17
|
+
/**
|
|
18
|
+
* Classify a filename into a supported office kind, or null when the format
|
|
19
|
+
* cannot be regenerated by this module (.doc has its own binary writer in
|
|
20
|
+
* src/doc.ts; PDF output is permanently out of scope for browser hosts).
|
|
21
|
+
*/
|
|
22
|
+
export declare function officeFileKind(filename: string): OfficeFileKind | null;
|
|
23
|
+
/** A detect function, typically DocCloakEngine.detect or detectEntities. */
|
|
24
|
+
export type DetectFn = (text: string) => Promise<DetectedEntity[]>;
|
|
25
|
+
export interface OfficeAnalysis {
|
|
26
|
+
kind: OfficeFileKind;
|
|
27
|
+
/** Flat text extracted from every text-bearing part of the package */
|
|
28
|
+
plainText: string;
|
|
29
|
+
/** Entities detected in the flat text (offsets refer to plainText) */
|
|
30
|
+
entities: DetectedEntity[];
|
|
31
|
+
}
|
|
32
|
+
/**
|
|
33
|
+
* Extract a file's text and run detection WITHOUT touching the session. Hosts
|
|
34
|
+
* use this to show the user what was found before asking for consent to
|
|
35
|
+
* regenerate; a cancelled upload then leaves no trace in the session map.
|
|
36
|
+
* Extraction is deterministic, so entities returned here can be passed to
|
|
37
|
+
* redactOfficeFile later and land on identical offsets.
|
|
38
|
+
*/
|
|
39
|
+
export declare function analyzeOfficeFile(file: File, detect: DetectFn): Promise<OfficeAnalysis>;
|
|
40
|
+
export interface RedactOfficeFileOptions {
|
|
41
|
+
/** The live session; file placeholders join the same map as prompt ones. */
|
|
42
|
+
session: AnonymizationSession;
|
|
43
|
+
/**
|
|
44
|
+
* Entities to redact, from a prior analyzeOfficeFile call (offsets refer to
|
|
45
|
+
* that call's plainText). When omitted, detect is required and is run here.
|
|
46
|
+
*/
|
|
47
|
+
entities?: DetectedEntity[];
|
|
48
|
+
/** Detect function; used only when entities were not supplied. */
|
|
49
|
+
detect?: DetectFn;
|
|
50
|
+
/** Accept and strip tracked changes in docx output. Default: true. */
|
|
51
|
+
acceptTrackedChanges?: boolean;
|
|
52
|
+
}
|
|
53
|
+
export interface OfficeRedactionResult {
|
|
54
|
+
kind: OfficeFileKind;
|
|
55
|
+
/** The redacted file; the input file/buffer is never modified. */
|
|
56
|
+
blob: Blob;
|
|
57
|
+
/** Suggested output name: "report.docx" becomes "report.redacted.docx". */
|
|
58
|
+
suggestedName: string;
|
|
59
|
+
/** The entities that were redacted (offsets refer to plainText) */
|
|
60
|
+
entities: DetectedEntity[];
|
|
61
|
+
/** Flat text the detection ran on */
|
|
62
|
+
plainText: string;
|
|
63
|
+
/** The redacted flat text (placeholders applied), for preview UIs */
|
|
64
|
+
redactedText: string;
|
|
65
|
+
}
|
|
66
|
+
/** "report.docx" -> "report.redacted.docx"; extensionless names get a suffix. */
|
|
67
|
+
export declare function redactedFileName(filename: string): string;
|
|
68
|
+
/**
|
|
69
|
+
* Redact a DOCX or XLSX file: extract text, detect (or reuse supplied
|
|
70
|
+
* entities), map every entity through the session, and regenerate the file
|
|
71
|
+
* with placeholders spliced into the XML, formatting preserved and metadata
|
|
72
|
+
* stripped. Placeholder issuing goes through session.anonymize, so a value
|
|
73
|
+
* already seen in a prompt reuses its existing placeholder and vice versa.
|
|
74
|
+
*/
|
|
75
|
+
export declare function redactOfficeFile(file: File, options: RedactOfficeFileOptions): Promise<OfficeRedactionResult>;
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @doccloak/core/dom - detection-driven office file redaction (T110).
|
|
3
|
+
*
|
|
4
|
+
* Ties the format modules (docx.ts, xlsx.ts) to the detection pipeline and an
|
|
5
|
+
* AnonymizationSession, so a host can hand over a File plus its live session
|
|
6
|
+
* and get back a redacted copy whose placeholders are consistent with every
|
|
7
|
+
* prompt redacted in the same session: an AI reply that mentions the file's
|
|
8
|
+
* placeholders restores through the same session map.
|
|
9
|
+
*
|
|
10
|
+
* PDF is intentionally NOT handled here: layout-preserving PDF redaction is
|
|
11
|
+
* out of scope for this module.
|
|
12
|
+
*/
|
|
13
|
+
import { getFileExtension, readDocx, writeAnonymizedDocx } from "./docx.js";
|
|
14
|
+
import { readXlsx, writeAnonymizedXlsx } from "./xlsx.js";
|
|
15
|
+
/**
|
|
16
|
+
* Classify a filename into a supported office kind, or null when the format
|
|
17
|
+
* cannot be regenerated by this module (.doc has its own binary writer in
|
|
18
|
+
* src/doc.ts; PDF output is permanently out of scope for browser hosts).
|
|
19
|
+
*/
|
|
20
|
+
export function officeFileKind(filename) {
|
|
21
|
+
const ext = getFileExtension(filename);
|
|
22
|
+
if (ext === 'docx')
|
|
23
|
+
return 'docx';
|
|
24
|
+
if (ext === 'xlsx')
|
|
25
|
+
return 'xlsx';
|
|
26
|
+
return null;
|
|
27
|
+
}
|
|
28
|
+
/**
|
|
29
|
+
* Extract a file's text and run detection WITHOUT touching the session. Hosts
|
|
30
|
+
* use this to show the user what was found before asking for consent to
|
|
31
|
+
* regenerate; a cancelled upload then leaves no trace in the session map.
|
|
32
|
+
* Extraction is deterministic, so entities returned here can be passed to
|
|
33
|
+
* redactOfficeFile later and land on identical offsets.
|
|
34
|
+
*/
|
|
35
|
+
export async function analyzeOfficeFile(file, detect) {
|
|
36
|
+
const kind = officeFileKind(file.name);
|
|
37
|
+
if (!kind)
|
|
38
|
+
throw new Error(`Unsupported office file: ${file.name}`);
|
|
39
|
+
const plainText = kind === 'docx'
|
|
40
|
+
? (await readDocx(file)).plainText
|
|
41
|
+
: (await readXlsx(file)).plainText;
|
|
42
|
+
const entities = await detect(plainText);
|
|
43
|
+
return { kind, plainText, entities };
|
|
44
|
+
}
|
|
45
|
+
/** "report.docx" -> "report.redacted.docx"; extensionless names get a suffix. */
|
|
46
|
+
export function redactedFileName(filename) {
|
|
47
|
+
const dot = filename.lastIndexOf('.');
|
|
48
|
+
if (dot <= 0)
|
|
49
|
+
return `${filename}.redacted`;
|
|
50
|
+
return `${filename.slice(0, dot)}.redacted${filename.slice(dot)}`;
|
|
51
|
+
}
|
|
52
|
+
/**
|
|
53
|
+
* Redact a DOCX or XLSX file: extract text, detect (or reuse supplied
|
|
54
|
+
* entities), map every entity through the session, and regenerate the file
|
|
55
|
+
* with placeholders spliced into the XML, formatting preserved and metadata
|
|
56
|
+
* stripped. Placeholder issuing goes through session.anonymize, so a value
|
|
57
|
+
* already seen in a prompt reuses its existing placeholder and vice versa.
|
|
58
|
+
*/
|
|
59
|
+
export async function redactOfficeFile(file, options) {
|
|
60
|
+
const kind = officeFileKind(file.name);
|
|
61
|
+
if (!kind)
|
|
62
|
+
throw new Error(`Unsupported office file: ${file.name}`);
|
|
63
|
+
const extraction = kind === 'docx' ? await readDocx(file) : await readXlsx(file);
|
|
64
|
+
const plainText = extraction.plainText;
|
|
65
|
+
let entities = options.entities;
|
|
66
|
+
if (!entities) {
|
|
67
|
+
if (!options.detect) {
|
|
68
|
+
throw new Error('redactOfficeFile requires either entities or a detect function');
|
|
69
|
+
}
|
|
70
|
+
entities = await options.detect(plainText);
|
|
71
|
+
}
|
|
72
|
+
const { session } = options;
|
|
73
|
+
// Drop entities whose offsets do not match this extraction (defensive: a
|
|
74
|
+
// stale analysis of a different file must not splice garbage), then map
|
|
75
|
+
// through the session in reading order. Surrogate mode maps PERSON values
|
|
76
|
+
// first, mirroring AnonymizationSession.anonymizeText, so derived
|
|
77
|
+
// surrogates (emails from owner names) stay consistent with prompts.
|
|
78
|
+
const valid = entities
|
|
79
|
+
.filter((e) => e.start >= 0 && e.end <= plainText.length && e.start < e.end
|
|
80
|
+
&& plainText.slice(e.start, e.end) === e.value)
|
|
81
|
+
.sort((a, b) => a.start - b.start || a.end - b.end);
|
|
82
|
+
if (session.getMode() === 'surrogate') {
|
|
83
|
+
for (const entity of valid) {
|
|
84
|
+
if (entity.type === 'PERSON')
|
|
85
|
+
session.anonymize(entity.value, entity.type);
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
const replacements = valid.map((entity) => ({
|
|
89
|
+
start: entity.start,
|
|
90
|
+
end: entity.end,
|
|
91
|
+
replacement: session.anonymize(entity.value, entity.type),
|
|
92
|
+
}));
|
|
93
|
+
// Value-level scrubbing (relationship targets, formulas, field instructions)
|
|
94
|
+
// uses the whole session map, so a value redacted in an earlier prompt is
|
|
95
|
+
// also caught when it hides in a hyperlink target of this file.
|
|
96
|
+
const valueReplacements = session.getEntries()
|
|
97
|
+
.filter((entry) => entry.replacement !== '________')
|
|
98
|
+
.map((entry) => ({ value: entry.original, replacement: entry.replacement }));
|
|
99
|
+
let redactedText = plainText;
|
|
100
|
+
for (const repl of [...replacements].sort((a, b) => b.start - a.start)) {
|
|
101
|
+
redactedText = redactedText.slice(0, repl.start) + repl.replacement + redactedText.slice(repl.end);
|
|
102
|
+
}
|
|
103
|
+
const blob = kind === 'docx'
|
|
104
|
+
? await writeAnonymizedDocx(extraction, replacements, valueReplacements, {
|
|
105
|
+
acceptTrackedChanges: options.acceptTrackedChanges ?? true,
|
|
106
|
+
})
|
|
107
|
+
: await writeAnonymizedXlsx(extraction, replacements, valueReplacements);
|
|
108
|
+
return {
|
|
109
|
+
kind,
|
|
110
|
+
blob,
|
|
111
|
+
suggestedName: redactedFileName(file.name),
|
|
112
|
+
entities: valid,
|
|
113
|
+
plainText,
|
|
114
|
+
redactedText,
|
|
115
|
+
};
|
|
116
|
+
}
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @doccloak/core/dom - shared OPC (Open Packaging Conventions) helpers (T110).
|
|
3
|
+
*
|
|
4
|
+
* Both .docx and .xlsx are OPC packages: a zip with XML parts, docProps
|
|
5
|
+
* metadata, relationship files and an optional thumbnail. The metadata
|
|
6
|
+
* scrubbing that is format-independent lives here so the xlsx module does not
|
|
7
|
+
* duplicate the docx module's logic. The docx module keeps its historical
|
|
8
|
+
* implementation untouched (behavior-preservation for existing callers); new
|
|
9
|
+
* code paths use these helpers.
|
|
10
|
+
*/
|
|
11
|
+
import type JSZip from 'jszip';
|
|
12
|
+
import type { ValueReplacement } from '../docx.js';
|
|
13
|
+
/** Parse one XML part from the zip; null when absent or malformed. */
|
|
14
|
+
export declare function readXmlPart(zip: JSZip, path: string): Promise<Document | null>;
|
|
15
|
+
/** Serialize an XML document back into the zip at the given path. */
|
|
16
|
+
export declare function writeXmlPart(zip: JSZip, path: string, doc: Document): void;
|
|
17
|
+
/** Blank the text content of every element whose local name is in the set. */
|
|
18
|
+
export declare function blankElementsByLocalName(doc: Document, names: Set<string>): void;
|
|
19
|
+
/**
|
|
20
|
+
* Replace every occurrence of each sensitive value (case-insensitive) in a
|
|
21
|
+
* string. A replacer function keeps '$' sequences in values/placeholders inert.
|
|
22
|
+
*
|
|
23
|
+
* Longer values are applied first so a name variant ("Kowalski") can never
|
|
24
|
+
* pre-empt the fuller form it is part of ("Jan Kowalski") and leave the rest
|
|
25
|
+
* of it behind. When the input is a URL (urlEncode), the percent-encoded form
|
|
26
|
+
* of each value is matched too, so "Jan%20Kowalski" in a hyperlink target is
|
|
27
|
+
* scrubbed as a whole.
|
|
28
|
+
*/
|
|
29
|
+
export declare function replaceSensitiveValues(input: string, valueReplacements: ValueReplacement[], urlEncode: boolean): string;
|
|
30
|
+
/**
|
|
31
|
+
* Scrub the shared docProps parts:
|
|
32
|
+
* - core.xml: author identities and content-describing fields
|
|
33
|
+
* - app.xml: organization and template/path hints
|
|
34
|
+
* - custom.xml: every leaf value (property names/structure kept)
|
|
35
|
+
*/
|
|
36
|
+
export declare function scrubDocPropsParts(zip: JSZip): Promise<void>;
|
|
37
|
+
/**
|
|
38
|
+
* Remove the package thumbnail (a rendered image of page/sheet one leaks
|
|
39
|
+
* content wholesale) together with its package relationship and content-type
|
|
40
|
+
* override.
|
|
41
|
+
*/
|
|
42
|
+
export declare function removePackageThumbnail(zip: JSZip): Promise<void>;
|
|
43
|
+
/**
|
|
44
|
+
* Scrub external relationship targets (mailto:/https: hyperlinks keep the raw
|
|
45
|
+
* sensitive value even when the display text was redacted) in every .rels file
|
|
46
|
+
* matched by the given pattern.
|
|
47
|
+
*/
|
|
48
|
+
export declare function scrubExternalRelTargets(zip: JSZip, relsPattern: RegExp, valueReplacements: ValueReplacement[]): Promise<void>;
|