text-sanctifier 1.3.10 → 1.3.11

package/README.md

@@ -106,20 +106,22 @@ Used by `summonSanctifier(options)`. All boolean options default to `false`. Pro
 | `collapseSpaces` | Collapse runs of ordinary spaces to one; keep tabs and newlines. |
 | `finalTrim` | Apply JavaScript `.trim()` to the result. |
 | `maxLength` | Input cap in UTF-16 code units, checked before processing. Default `Infinity`; negative, non-finite or non-number values mean no cap. |
-| `onMaxLength` | Default `'throw'`: throw `RangeError` above the cap. `'truncate'`: slice input to the cap before processing (may split a surrogate pair). `'noop'`: return original input unchanged. Invalid values use `'throw'`. |
+| `onMaxLength` | Default `'throw'`: throw `RangeError` above the cap. `'truncate'`: slice input to at most the cap before processing, never through a surrogate pair. `'noop'`: return original input unchanged. Invalid values use `'throw'`. |
 
 ```js
 import { summonSanctifier } from 'text-sanctifier';
 const clean = summonSanctifier({ purgeInvisibleChars: true, maxLength: 10000, onMaxLength: 'throw' });
 ```
 
+Every sanitizer replaces unpaired surrogates with U+FFFD before anything else, so removing a character between two halves cannot join them into a new one.
+
 ### Errors
 
 All text-processing exports throw `TypeError` on non-string input, including the preset functions, the sanitizer returned by `summonSanctifier`, `findHiddenRuns` and `decodeTagText`. Inputs are not coerced to strings. Sanitizers also throw `RangeError` when input exceeds `maxLength` with `onMaxLength: 'throw'`.
 
 ### Presets as standalone imports
 
-Each preset is also its own export: `sanctifyStrict`, `sanctifyLoose`, `sanctifyKeyboardOnly`, `sanctifyKeyboardOnlyEmoji` and `sanctifyPasted`. They are the same functions as `summonSanctifier.strict` and the rest. Import one directly and a bundler leaves out `summonSanctifier` and the other presets (about <!-- gen:size:sanctifyPasted -->1.1 KiB<!-- /gen --> gzipped instead of <!-- gen:size:summonSanctifier.pasted -->1.4 KiB<!-- /gen -->).
+Each preset is also its own export: `sanctifyStrict`, `sanctifyLoose`, `sanctifyKeyboardOnly`, `sanctifyKeyboardOnlyEmoji` and `sanctifyPasted`. They are the same functions as `summonSanctifier.strict` and the rest. Import one directly and a bundler leaves out `summonSanctifier` and the other presets (about <!-- gen:size:sanctifyPasted -->1.1 KiB<!-- /gen --> gzipped instead of <!-- gen:size:summonSanctifier.pasted -->1.5 KiB<!-- /gen -->).
 
 ```js
 import { sanctifyPasted } from 'text-sanctifier';
@@ -187,7 +189,7 @@ const report = inspectText(input);
 */
 ```
 
-Use `inspectText` to preflight text content before rendering, storing, or linting. It's a diagnostic tool to help inform sanitization needs. Pass the report to `getRecommendedSanctifierOptions(report)` to auto-generate config flags for `summonSanctifier()`.
+Use `inspectText` to preflight text content before rendering, storing, or linting. It's a diagnostic tool to help inform sanitization needs. Pass the report to `getRecommendedSanctifierOptions(report)` to auto-generate config flags for `summonSanctifier()`. It turns on what removes trash without losing meaning; `purgeEmojis` and `keyboardOnlyFilter` delete content, so you opt in to those yourself.
 
 ### `inspectTextDetailed(text: string, options?: { maxFindings?: number }): DetailedTrashReport`
 
@@ -299,6 +301,7 @@ The package ships a `text-sanctifier` command for CI and coding agents, which ca
 ```sh
 npx text-sanctifier --help
 npx text-sanctifier --version
+npx text-sanctifier check
 npx text-sanctifier check src docs "**/*.md"
 npx text-sanctifier fix --dry-run src
 npx text-sanctifier reveal notes.md
@@ -361,13 +364,13 @@ The repository publishes a [pre-commit](https://pre-commit.com) hook manifest, s
 # .pre-commit-config.yaml
 repos:
   - repo: https://github.com/iWhatty/text-sanctifier
-    rev: v1.3.10  # a release tag
+    rev: v1.3.11  # a release tag
     hooks:
       - id: text-sanctifier            # every staged text file
       - id: text-sanctifier-ai-config  # agent instruction files, with --ai-config
 ```
 
-- `text-sanctifier` runs `text-sanctifier check` on staged text files. Use `exclude:` for files that legitimately contain these characters, or `args: [--strip-bom]` and the other `check` options.
+- `text-sanctifier` runs `text-sanctifier check` on staged text files, as pre-commit defines them: it can drop a file with control bytes near its start before the hook sees it, so scan the repository with `check` as well if that matters to you. Use `exclude:` for files that legitimately contain these characters, or `args: [--strip-bom]` and the other `check` options.
 - `text-sanctifier-ai-config` runs `text-sanctifier check --ai-config` on staged files that match the `--ai-config` file list.
 - A hook fails the commit when it finds something (exit code 1). Neither hook changes files; run `text-sanctifier fix` yourself.
 
@@ -400,9 +403,11 @@ git diff | npx text-sanctifier reveal
 
 - Paths can be files, directories or globs (`*`, `**`, `?`, `[...]`, `{a,b}`). Quote globs so the shell passes them through.
 - Inside a git work tree, directories and globs list files with `git ls-files`, so anything `.gitignore` excludes is skipped. Outside git, `.git` and `node_modules` are skipped. A file named explicitly is always read.
-- Binary files (a NUL byte in the first 8000 bytes) are skipped and listed under `skipped` in JSON. Stdin and AI configuration files are never treated as binary: a NUL in them is reported as `nul`.
+- Binary files are skipped, counted in the summary and listed under `skipped` in JSON. A file counts as binary when it is not valid UTF-8 and its first 8000 bytes hold a NUL and more than 5% control bytes, so a stray NUL or bad byte does not hide a text file: the NUL is reported as `nul`. Stdin and AI configuration files are never treated as binary. `fix` is more careful, because it rewrites files: in a directory or glob it also skips any file with a NUL in those bytes, and it cleans such a file only if you name it.
+- UTF-16 files with a BOM are decoded and checked. `fix` never rewrites them and lists them under `skipped`.
 - Files that are not valid UTF-8 are still checked, with each invalid byte read as U+FFFD, so one bad byte cannot hide the rest of the file. `check` notes them on stderr and lists them under `notUtf8` in JSON; `fix` never writes them and lists them under `skipped`.
-- `-`, or no paths with piped input, reads stdin (with `--ai-config`, only `-` does). `fix` writes the cleaned text to stdout and the report to stderr.
+- `-` reads stdin. With no paths, `check` reads piped input, and scans the current directory when stdin is a terminal, the null device or empty. `check --ai-config` always searches the current directory with no paths. `fix` and `reveal` read piped input. `fix` writes the cleaned text to stdout and the report to stderr.
+- A path or glob that matches no files, or a run that checks nothing, exits 2 instead of reporting a clean result. So does `fix` when it refuses stdin or a file you named. Files it skips in a directory or glob don't fail the run.
 
 ### Exit codes
 
@@ -410,7 +415,7 @@ git diff | npx text-sanctifier reveal
 | --- | --- | --- | --- |
 | 0 | nothing found | done (or nothing to do) | done |
 | 1 | findings | `--dry-run` with changes pending | not used |
-| 2 | usage error, missing path or unreadable file | same | same |
+| 2 | usage error, missing or unreadable input, a path matching no files, or nothing checked | same, or an input you named that it refused to rewrite | same as `check` |
 
 ---
 

package/bin/text-sanctifier.js

@@ -10,7 +10,7 @@
 // Plain Node (>=18), no dependencies. Exit codes: 0 clean, 1 findings (check,
 // or fix --dry-run with changes pending), 2 usage or I/O error.
 
-import { readFileSync, writeFileSync, statSync, readdirSync, existsSync } from 'node:fs';
+import { readFileSync, writeFileSync, statSync, fstatSync, readdirSync } from 'node:fs';
 import { resolve, join, sep, relative } from 'node:path';
 import { spawnSync } from 'node:child_process';
 import { pathToFileURL } from 'node:url';
@@ -32,8 +32,12 @@ Usage:
   text-sanctifier reveal [paths...]
 
 Paths may be files, directories or globs (quote globs so the shell leaves them
-alone). "-", or no paths with piped input, reads stdin. Directories and globs
-skip files ignored by git (inside a git work tree) and skip binary files.
+alone). "-" reads stdin. With no paths, check reads piped input, or scans the current
+directory when stdin is a terminal, the null device or empty. Fix and reveal read
+piped input. Directories and globs skip files ignored by git (inside
+a git work tree). Binary files are skipped; files that are text apart from a stray
+NUL or bad byte are not, and UTF-16 files with a BOM are decoded. A path or glob that
+matches no files, or a run that checks nothing, exits 2.
 
 check options:
   --format F           text (default), json or sarif (SARIF 2.1.0)
@@ -77,6 +81,7 @@ const BOM = 0xFEFF;
 const STDIN = '-';
 const STDIN_LABEL = '<stdin>';
 const BINARY_SNIFF_BYTES = 8000;
+const BINARY_CONTROL_SHARE = 0.05;
 const WALK_SKIP_DIRS = new Set(['.git', 'node_modules']);
 const CASE_INSENSITIVE_FS = process.platform === 'win32' || process.platform === 'darwin';
 
@@ -206,21 +211,36 @@ function isGitWorkTree(cwd) {
 }
 
 /**
- * Lists tracked and untracked-but-not-ignored files under one pathspec.
+ * Lists tracked and untracked-but-not-ignored files under the given pathspecs.
  * Returns null when git cannot answer (for example a path outside the work tree).
  * @param {string} cwd
- * @param {string} pathspec
+ * @param {string[]} pathspecs
  * @returns {string[]|null}
  */
-function gitListFiles(cwd, pathspec) {
+function gitListFiles(cwd, pathspecs) {
     const r = spawnSync(
         'git',
-        ['ls-files', '-z', '--cached', '--others', '--exclude-standard', '--', pathspec],
+        ['ls-files', '-z', '--cached', '--others', '--exclude-standard', '--', ...pathspecs],
         { cwd, encoding: 'utf8', maxBuffer: 1 << 28 }
     );
     if (r.error || r.status !== 0) return null;
-    // --cached lists tracked files deleted from the work tree; drop them
-    return r.stdout.split('\0').filter((f) => f && existsSync(resolve(cwd, f)));
+    // --cached also lists tracked files deleted from the work tree, submodules and
+    // symlinks to directories; keep only what can be read as a file
+    return r.stdout.split('\0').filter((f) => {
+        if (!f) return false;
+        try { return statSync(resolve(cwd, f)).isFile(); } catch { return false; }
+    });
+}
+
+/**
+ * Expands `{a,b}` into one glob per alternative: git pathspecs have no brace syntax.
+ * @param {string} glob
+ * @returns {string[]}
+ */
+function expandBraces(glob) {
+    const m = /\{([^{}]*)\}/.exec(glob);
+    if (!m) return [glob];
+    return m[1].split(',').flatMap((alt) => expandBraces(glob.slice(0, m.index) + alt + glob.slice(m.index + m[0].length)));
 }
 
 /**
@@ -366,7 +386,7 @@ function collectInputs(args, cwd, warn, aiConfig = false) {
 
         if (inGit === undefined) inGit = isGitWorkTree(cwd);
         let listed =
-            (inGit && gitListFiles(cwd, isGlob ? `:(glob)${toSlash(arg)}` : toSlash(arg))) ||
+            (inGit && gitListFiles(cwd, isGlob ? expandBraces(toSlash(arg)).map((g) => `:(glob)${g}`) : [toSlash(arg)])) ||
             (isGlob ? walkGlob(cwd, arg) : walk(cwd, arg));
 
         if (aiConfig && !isGlob) {
@@ -374,6 +394,7 @@ function collectInputs(args, cwd, warn, aiConfig = false) {
             listed = listed.filter((f) => isAiConfigPath(toSlash(relative(base, resolve(cwd, f)))));
         } else if (listed.length === 0) {
             warn(`${arg}: no files matched`);
+            errors++;
         }
         for (const f of listed) add(f, false);
     }
@@ -385,24 +406,57 @@ function collectInputs(args, cwd, warn, aiConfig = false) {
 
 const utf8 = new TextDecoder('utf-8', { fatal: true, ignoreBOM: true });
 const utf8Lenient = new TextDecoder('utf-8', { ignoreBOM: true });
+const utf16 = new TextDecoder('utf-16le', { ignoreBOM: true });
 
 /**
- * Decodes UTF-8, keeping a leading BOM. With `sniff`, data with a NUL in its first
- * BINARY_SNIFF_BYTES is binary and not decoded. Bytes that are not UTF-8 become U+FFFD
- * and set `lossy`, so one bad byte cannot hide the rest of a file from `check`.
+ * A NUL plus a share of control bytes that text does not have, in the first
+ * BINARY_SNIFF_BYTES. One stray NUL is not enough, or it would hide a file from `check`.
  * @param {Uint8Array} bytes
+ */
+function looksBinary(bytes) {
+    const sample = bytes.subarray(0, BINARY_SNIFF_BYTES);
+    if (!sample.includes(0)) return false;
+    let controls = 0;
+    for (const b of sample) if (b < 9 || (b > 13 && b < 32)) controls++;
+    return controls / sample.length > BINARY_CONTROL_SHARE;
+}
+
+/**
+ * Decodes UTF-8, or UTF-16 with a BOM (setting `wide`), keeping a leading BOM. With
+ * `sniff`, data that is not UTF-8 and looks binary is not decoded. Other bytes that are
+ * not UTF-8 become U+FFFD and set `lossy`, so one bad byte cannot hide the rest of a file.
+ * @param {Buffer} bytes
  * @param {boolean} sniff
- * @returns {{ text: string|null, reason?: string, lossy?: boolean }}
+ * @returns {{ text: string|null, reason?: string, lossy?: boolean, wide?: boolean }}
  */
 function decode(bytes, sniff) {
-    if (sniff && bytes.subarray(0, BINARY_SNIFF_BYTES).includes(0)) return { text: null, reason: 'binary' };
+    if (bytes[0] === 0xFF && bytes[1] === 0xFE) return { text: utf16.decode(bytes), wide: true };
+    if (bytes[0] === 0xFE && bytes[1] === 0xFF) {
+        return { text: utf16.decode(Buffer.from(bytes.subarray(0, bytes.length & ~1)).swap16()), wide: true };
+    }
     try {
         return { text: utf8.decode(bytes) };
-    } catch {
+    } catch (e) {
+        if (!(e instanceof TypeError)) throw e; // too large for a string, not bad UTF-8
+        if (sniff && looksBinary(bytes)) return { text: null, reason: 'binary' };
         return { text: utf8Lenient.decode(bytes), lossy: true };
     }
 }
 
+/**
+ * True when stdin may hold input for check with no paths: anything but a terminal,
+ * the null device or an empty file. Pipes must be read to detect empty input;
+ * Windows pipes do not report isFIFO().
+ */
+function stdinHasInput() {
+    try {
+        const stat = fstatSync(0);
+        return !stat.isCharacterDevice() && !(stat.isFile() && stat.size === 0);
+    } catch {
+        return false;
+    }
+}
+
 /** @returns {Promise<Buffer>} */
 function readStdin() {
     return new Promise((res, rej) => {
@@ -656,11 +710,23 @@ async function main(argv) {
     if (opts.version) { out(VERSION); return 0; }
 
     const cwd = process.cwd();
+    const warn = (msg) => err(`text-sanctifier: ${msg}`);
     let paths = opts.paths;
-    // --ai-config with no paths searches the current directory even when stdin is not a
-    // terminal, so a CI job with no stdin still scans the repository; `-` reads stdin.
-    if (paths.length === 0 && opts.aiConfig) {
+    /** @type {Buffer | undefined} */
+    let stdinBytes;
+    // check with no paths reads nonempty stdin once, otherwise scans the current
+    // directory. AI configuration discovery always defaults to the directory.
+    if (paths.length === 0 && opts.command === 'check') {
         paths = ['.'];
+        if (!opts.aiConfig && stdinHasInput()) {
+            try {
+                stdinBytes = await readStdin();
+            } catch (e) {
+                warn(`${STDIN_LABEL}: ${e.code || e.message}`);
+                return 2;
+            }
+            if (stdinBytes.length > 0) paths = [STDIN];
+        }
     } else if (paths.length === 0) {
         if (process.stdin.isTTY) {
             err('text-sanctifier: no paths given and nothing piped on stdin.\nRun "text-sanctifier --help" for usage.');
@@ -669,7 +735,6 @@ async function main(argv) {
         paths = [STDIN];
     }
 
-    const warn = (msg) => err(`text-sanctifier: ${msg}`);
     const { inputs, errors: pathErrors } = collectInputs(paths, cwd, warn, opts.aiConfig);
     let errors = pathErrors;
     /** @type {Array<{ path: string, reason: string }>} */
@@ -682,11 +747,12 @@ async function main(argv) {
      * @param {(label: string, text: string, input: { path: string, explicit: boolean }) => void} visit
      */
     const forEachText = async (visit) => {
+        let visited = 0;
         for (const input of inputs) {
             const label = input.path === STDIN ? STDIN_LABEL : input.path;
             let bytes;
             try {
-                bytes = input.path === STDIN ? await readStdin() : readFileSync(resolve(cwd, input.path));
+                bytes = input.path === STDIN ? (stdinBytes ?? await readStdin()) : readFileSync(resolve(cwd, input.path));
             } catch (e) {
                 warn(`${label}: ${e.code || e.message}`);
                 errors++;
@@ -694,26 +760,45 @@ async function main(argv) {
             }
             // Files that look binary are skipped. Stdin and AI configuration files are text
             // by intent, so a NUL in one is a finding, not a reason to skip it.
-            const { text, reason, lossy } = decode(bytes, input.path !== STDIN && !opts.aiConfig);
-            if (text === null) {
+            let decoded;
+            try {
+                decoded = decode(bytes, input.path !== STDIN && !opts.aiConfig);
+            } catch (e) {
+                warn(`${label}: ${e.message}`);
+                errors++;
+                continue;
+            }
+            const { text, lossy, wide } = decoded;
+            let { reason } = decoded;
+            // fix writes UTF-8 in place, so it never rewrites what it could not decode or
+            // what was not UTF-8. A directory scan also leaves any file with a NUL alone;
+            // a file you name is cleaned if it is otherwise UTF-8.
+            if (opts.command === 'fix' && !reason) {
+                if (lossy) reason = 'not UTF-8';
+                else if (wide) reason = 'UTF-16';
+                else if (!input.explicit && bytes.subarray(0, BINARY_SNIFF_BYTES).includes(0)) reason = 'binary';
+            }
+            if (reason) {
                 skipped.push({ path: label, reason });
-                if (input.explicit) warn(`${label}: skipped (${reason})`);
+                if (input.explicit || reason !== 'binary') warn(`${label}: skipped (${reason})`);
+                // Asked to clean this input and did not: that must not look like success
+                if (opts.command === 'fix' && input.explicit) errors++;
                 continue;
             }
             if (lossy) {
-                // fix never rewrites bytes it could not decode
-                if (opts.command === 'fix') {
-                    skipped.push({ path: label, reason: 'not UTF-8' });
-                    warn(`${label}: skipped (not UTF-8)`);
-                    continue;
-                }
                 notUtf8.push(label);
                 warn(`${label}: not valid UTF-8; checked with invalid bytes read as U+FFFD`);
             }
+            visited++;
             visit(label, text, input);
         }
+        // A clean result must mean something was read. Finding no AI configuration files is
+        // an answer; --ai-config reports that itself.
+        if (visited === 0 && errors === 0 && !opts.aiConfig) errors++;
     };
 
+    const skippedNote = () => (skipped.length > 0 ? `; ${plural(skipped.length, 'file')} skipped` : '');
+
     if (opts.command === 'check') {
         const files = [];
         let scanned = 0;
@@ -765,8 +850,9 @@ async function main(argv) {
                 out('No AI configuration files found.');
                 if (opts.paths.length === 0 && !process.stdin.isTTY) out('To check piped text, pass "-": text-sanctifier check --ai-config -');
             }
-            else if (total === 0) out(`No ${opts.aiConfig ? 'hidden text, ' : ''}invisible or control characters in ${plural(scanned, kind)}.`);
-            else out(`${plural(total, 'finding')}${opts.aiConfig ? ` (${plural(hiddenRunCount, 'hidden run')})` : ''} in ${files.length} of ${plural(scanned, kind)}.`);
+            else if (scanned === 0) out(`Nothing checked${skippedNote()}.`);
+            else if (total === 0) out(`No ${opts.aiConfig ? 'hidden text, ' : ''}invisible or control characters in ${plural(scanned, kind)}${skippedNote()}.`);
+            else out(`${plural(total, 'finding')}${opts.aiConfig ? ` (${plural(hiddenRunCount, 'hidden run')})` : ''} in ${files.length} of ${plural(scanned, kind)}${skippedNote()}.`);
         }
         return errors > 0 ? 2 : total > 0 ? 1 : 0;
     }
@@ -858,9 +944,11 @@ async function main(argv) {
             if (f.newlinesNormalized) parts.push('normalized newlines');
             report(`${verb} ${f.path}: ${parts.join('; ')}`);
         }
-        report(files.length === 0
-            ? `Nothing to fix in ${scanned} file${scanned === 1 ? '' : 's'}.`
-            : `${opts.dryRun ? 'Would change' : 'Changed'} ${files.length} of ${scanned} file${scanned === 1 ? '' : 's'}.`);
+        report(scanned === 0
+            ? `Nothing checked${skippedNote()}.`
+            : files.length === 0
+                ? `Nothing to fix in ${plural(scanned, 'file')}${skippedNote()}.`
+                : `${opts.dryRun ? 'Would change' : 'Changed'} ${files.length} of ${plural(scanned, 'file')}${skippedNote()}.`);
     }
     return errors > 0 ? 2 : opts.dryRun && files.length > 0 ? 1 : 0;
 }

package/dist/text-sanctifier.min.js

@@ -1,6 +1,6 @@
-var y=/[\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F-\u009F\u061C\u200E\u200F\u202A-\u202E\u2066-\u2069]+/g,d=/[\u00A0\u00AD\u034F\u061C\u115F\u1160\u17B4\u17B5\u180B-\u180F\u2000-\u200F\u202A-\u202F\u205F-\u206F\u3000\u3164\uFE00-\uFE0F\uFEFF\uFFA0\uFFF0-\uFFF8\u{1BCA0}-\u{1BCA3}\u{1D173}-\u{1D17A}\u{E0000}-\u{E0FFF}]+/gu;function A(e){return e<8192?D(e):v(e)}function D(e){return e<174?e<32?e<11?e<1?"nul":e<9?"c0_control":null:e<13?"c0_control":e<14?null:"c0_control":e<160?e<127?null:e<128?"c0_control":"c1_control":e<161?"nonbreaking_space":e<173?null:"invisible_format":e<4449?e<1564?e<847?null:e<848?"invisible_format":null:e<1565?"bidi_control":e<4447?null:"hangul_filler":e<6155?e<6068?null:e<6070?"invisible_format":null:e<6159?e<6158?"variation_selector":"invisible_format":e<6160?"variation_selector":null}function v(e){return e<8304?e<8240?e<8208?e<8203?"nonbreaking_space":e<8206?"zero_width":"bidi_control":e<8234?null:e<8239?"bidi_control":"nonbreaking_space":e<8289?e<8287?null:e<8288?"nonbreaking_space":"zero_width":e<8294?"invisible_format":e<8298?"bidi_control":"invisible_format":e<65040?e<12644?e<12288?null:e<12289?"nonbreaking_space":null:e<12645?"hangul_filler":e<65024?null:"variation_selector":e<65440?e<65279?null:e<65280?"bom":null:e<65520?e<65441?"hangul_filler":null:e<65529?"invisible_format":null}function I(e){return e<917504?e<113828?e<113824?null:"invisible_format":e<119155?null:e<119163?"invisible_format":null:e<917760?e<917632?"tag":"invisible_format":e<918e3?"variation_selector":e<921600?"invisible_format":null}var g=/[^\x20-\x7E\r\n]+/gu,B=/[\u2018\u2019\u201A\u201B\u2032\u2035]/g,G=/[\u201C\u201D\u201E\u201F\u2033\u2036\u00AB\u00BB]/g,X=/[\u2012\u2013\u2014\u2015\u2212]/g,k=/\u2026/g,O=/[\u2022\u00B7]/g,P=/[\uFF01-\uFF5E]/g;function S(e){return e.replace(B,"'").replace(G,'"').replace(X,"-").replace(k,"...").replace(O,"*").replace(P,r=>String.fromCharCode(r.charCodeAt(0)-65248))}function M(){try{return new RegExp("(?:\\p{Extended_Pictographic}(?:\\uFE0F|\\uFE0E)?(?:\\u200D(?:\\p{Extended_Pictographic}|\\w)+)*)","gu")}catch{return/[\u{2700}-\u{27BF}\u{1F300}-\u{1FAFF}\u{1F3FB}-\u{1F3FF}\u200D\uFE0F\u{1F1E6}-\u{1F1FF}]/gu}}var x=M();function m(e,r){return e.lastIndex=0,e.test(r)}function U(e){let r=S(e);if(!m(g,r))return!1;for(let n of r)if(!(n===`
-`||n==="\r"||n>=" "&&n<="~")&&!m(x,n))return!0;return!1}function j(e){if(typeof e!="string")throw new TypeError("inspectText expects a string input.");let r=[],n={hasControlChars:!1,hasInvisibleChars:!1,hasMixedNewlines:!1,newlineStyle:null,hasEmojis:!1,hasNonKeyboardChars:!1,summary:r},u=(s,c,f)=>{s&&(n[c]=!0,r.push(f))};u(m(y,e),"hasControlChars","Control characters detected."),u(m(d,e),"hasInvisibleChars","Invisible Unicode characters detected."),u(m(x,e),"hasEmojis","Emojis detected.");let{mixed:o,types:i}=b(e);return n.hasMixedNewlines=o,n.newlineStyle=o?"Mixed":i[0]||null,n.newlineStyle&&r.push(o?"Mixed newline styles detected.":`Consistent newline style: ${n.newlineStyle}`),u(U(e),"hasNonKeyboardChars","Non-keyboard characters detected."),n}var z=100;function C(e,r={}){if(typeof e!="string")throw new TypeError("inspectTextDetailed expects a string input.");let n=r&&r.maxFindings,u=n===1/0||Number.isInteger(n)&&n>=0?n:z,o={bidi_control:0,zero_width:0,nonbreaking_space:0,bom:0,nul:0,c0_control:0,c1_control:0,invisible_format:0,hangul_filler:0,variation_selector:0,tag:0},i=[],s=0,c=1,f=0;for(let a=0,h=e.length;a<h;a++){let l=e.charCodeAt(a);if(l>=32&&l<127)continue;if(l===10){c++,f=a+1;continue}if(l===13){e.charCodeAt(a+1)!==10&&(c++,f=a+1);continue}let E=a,F=l,p;if(l>=55296&&l<=56319){let t=e.charCodeAt(a+1);if(!(t>=56320&&t<=57343))continue;F=65536+(l-55296<<10)+(t-56320),p=I(F),a++}else p=A(l);if(p!==null&&(o[p]++,s++,i.length<u)){let t={offset:E,line:c,column:E-f+1,codePoint:F,category:p};p==="bom"&&(t.atStart=E===0),i.push(t)}}return{findings:i,counts:o,total:s,truncated:s>i.length}}function b(e){if(typeof e!="string")throw new TypeError("getNewlineStats expects a string input.");let r=0,n=0,u=0;for(let s=0,c=e.length;s<c;s++){let f=e.charCodeAt(s);f===13?e.charCodeAt(s+1)===10?(r++,s++):n++:f===10&&u++}let o={crlf:r,cr:n,lf:u},i=[];return o.crlf>0&&i.push("CRLF"),o.cr>0&&i.push("CR"),o.lf>0&&i.push("LF"),{...o,types:i,mixed:i.length>1}}function K(e){return{purgeInvisibleChars:e.hasInvisibleChars,purgeEmojis:e.hasEmojis,nukeControls:e.hasControlChars,keyboardOnlyFilter:e.hasNonKeyboardChars,normalizeNewlines:e.hasMixedNewlines||e.newlineStyle==="CRLF"||e.newlineStyle==="CR"}}function H(e={}){let r=!!e.normalizeSpaces,n=!!e.purgeZeroWidthSpaces,u=!!e.purgeInvisibleChars,o=!!e.purgeEmojis,i=!!e.nukeControls,s=!!e.keyboardOnlyFilter,c=!!e.normalizeNewlines,f=!!e.trimSpacesAroundNewlines,a=!!e.collapseNewLines,h=!!e.preserveParagraphs,l=!!e.collapseSpaces,E=!!e.finalTrim,F=Number.isFinite(e.maxLength)&&e.maxLength>=0?e.maxLength:1/0,p=e.onMaxLength==="truncate"||e.onMaxLength==="noop"||e.onMaxLength==="throw"?e.onMaxLength:"throw";return t=>_(t,u,o,i,s,c,f,a,h,l,E,F,p,r,n)}function R(e){return _(e,!0,!0,!0,!1,!0,!0,!0,!1,!0,!0)}function w(e){return _(e,!1,!1,!1,!1,!0,!0,!0,!0,!0,!0)}function N(e){return _(e,!1,!1,!1,!0,!0,!0,!1,!1,!1,!0)}function T(e){return _(e,!0,!0,!0,!0,!0,!0,!0,!1,!0,!0)}function L(e){return _(e,!1,!1,!1,!1,!0,!1,!1,!1,!1,!1,1/0,"throw",!0,!0)}var W=Object.assign(H,{strict:R,loose:w,keyboardOnlyEmoji:N,keyboardOnly:T,pasted:L});function _(e,r=!1,n=!1,u=!1,o=!1,i=!1,s=!1,c=!1,f=!1,a=!1,h=!1,l=1/0,E="throw",F=!1,p=!1){if(typeof e!="string")throw new TypeError("sanctifyText expects a string input.");if(E!=="truncate"&&E!=="noop"&&E!=="throw"&&(E="throw"),e.length>l)switch(E){case"truncate":e=e.slice(0,l);break;case"noop":return e;case"throw":default:throw new RangeError(`sanctifyText input length ${e.length} exceeds maxLength ${l}.`)}let t=e;return F&&(t=t.replace(J," ")),p&&(t=t.replace(V,"")),r&&(t=Z(t)),n&&(t=q(t)),u&&(t=le(t)),o&&(t=Y(t,n)),i&&(t=ee(t)),s&&(t=ne(t)),c&&(t=oe(t,f)),a&&(t=se(t)),h?t.trim():t}var J=/[\u00A0\u2000-\u200A\u202F\u205F]/g,V=/[\u200B\u2060\uFEFF]/g;function Z(e){return e.replace(d,"")}function $(e){if(x.lastIndex=0,!x.test(e))return"";x.lastIndex=0;let r="";for(let n of e.matchAll(x))r+=n[0];return r}function Y(e,r=!1){let n=S(e);return r?n.replace(g,""):n.replace(g,u=>$(u))}function q(e){return e.replace(x,"")}var Q=/\r\n|\r|\n/g;function ee(e,r=`
-`){return e.replace(Q,r)}var re=/[ \t]*(\n+)[ \t]*/g;function ne(e){return e.replace(re,"$1")}var te=/\n{2,}/g,ie=/\n{3,}/g;function oe(e,r){return r?e.replace(ie,`
+var y=/[\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F-\u009F\u061C\u200E\u200F\u202A-\u202E\u2066-\u2069]+/g,S=/[\u00A0\u00AD\u034F\u061C\u115F\u1160\u17B4\u17B5\u180B-\u180F\u2000-\u200F\u202A-\u202F\u205F-\u206F\u3000\u3164\uFE00-\uFE0F\uFEFF\uFFA0\uFFF0-\uFFF8\u{1BCA0}-\u{1BCA3}\u{1D173}-\u{1D17A}\u{E0000}-\u{E0FFF}]+/gu;function I(e){return e<8192?v(e):B(e)}function v(e){return e<174?e<32?e<11?e<1?"nul":e<9?"c0_control":null:e<13?"c0_control":e<14?null:"c0_control":e<160?e<127?null:e<128?"c0_control":"c1_control":e<161?"nonbreaking_space":e<173?null:"invisible_format":e<4449?e<1564?e<847?null:e<848?"invisible_format":null:e<1565?"bidi_control":e<4447?null:"hangul_filler":e<6155?e<6068?null:e<6070?"invisible_format":null:e<6159?e<6158?"variation_selector":"invisible_format":e<6160?"variation_selector":null}function B(e){return e<8304?e<8240?e<8208?e<8203?"nonbreaking_space":e<8206?"zero_width":"bidi_control":e<8234?null:e<8239?"bidi_control":"nonbreaking_space":e<8289?e<8287?null:e<8288?"nonbreaking_space":"zero_width":e<8294?"invisible_format":e<8298?"bidi_control":"invisible_format":e<65040?e<12644?e<12288?null:e<12289?"nonbreaking_space":null:e<12645?"hangul_filler":e<65024?null:"variation_selector":e<65440?e<65279?null:e<65280?"bom":null:e<65520?e<65441?"hangul_filler":null:e<65529?"invisible_format":null}function b(e){return e<917504?e<113828?e<113824?null:"invisible_format":e<119155?null:e<119163?"invisible_format":null:e<917760?e<917632?"tag":"invisible_format":e<918e3?"variation_selector":e<921600?"invisible_format":null}var g=/[^\x20-\x7E\r\n]+/gu,G=/[\u2018\u2019\u201A\u201B\u2032\u2035]/g,X=/[\u201C\u201D\u201E\u201F\u2033\u2036\u00AB\u00BB]/g,M=/[\u2012\u2013\u2014\u2015\u2212]/g,O=/\u2026/g,k=/[\u2022\u00B7]/g,P=/[\uFF01-\uFF5E]/g;function d(e){return e.replace(G,"'").replace(X,'"').replace(M,"-").replace(O,"...").replace(k,"*").replace(P,r=>String.fromCharCode(r.charCodeAt(0)-65248))}function U(){try{return new RegExp("(?:\\p{Extended_Pictographic}(?:\\uFE0F|\\uFE0E)?(?:\\u200D(?:\\p{Extended_Pictographic}|\\w)+)*)","gu")}catch{return/[\u{2700}-\u{27BF}\u{1F300}-\u{1FAFF}\u{1F3FB}-\u{1F3FF}\u200D\uFE0F\u{1F1E6}-\u{1F1FF}]/gu}}var p=U();function m(e,r){return e.lastIndex=0,e.test(r)}function z(e){let r=d(e);if(!m(g,r))return!1;for(let n of r)if(!(n===`
+`||n==="\r"||n>=" "&&n<="~")&&!m(p,n))return!0;return!1}function j(e){if(typeof e!="string")throw new TypeError("inspectText expects a string input.");let r=[],n={hasControlChars:!1,hasInvisibleChars:!1,hasMixedNewlines:!1,newlineStyle:null,hasEmojis:!1,hasNonKeyboardChars:!1,summary:r},u=(l,c,f)=>{l&&(n[c]=!0,r.push(f))};u(m(y,e),"hasControlChars","Control characters detected."),u(m(S,e),"hasInvisibleChars","Invisible Unicode characters detected."),u(m(p,e),"hasEmojis","Emojis detected.");let{mixed:o,types:i}=R(e);return n.hasMixedNewlines=o,n.newlineStyle=o?"Mixed":i[0]||null,n.newlineStyle&&r.push(o?"Mixed newline styles detected.":`Consistent newline style: ${n.newlineStyle}`),u(z(e),"hasNonKeyboardChars","Non-keyboard characters detected."),n}var K=100;function A(e,r={}){if(typeof e!="string")throw new TypeError("inspectTextDetailed expects a string input.");let n=r&&r.maxFindings,u=n===1/0||Number.isInteger(n)&&n>=0?n:K,o={bidi_control:0,zero_width:0,nonbreaking_space:0,bom:0,nul:0,c0_control:0,c1_control:0,invisible_format:0,hangul_filler:0,variation_selector:0,tag:0},i=[],l=0,c=1,f=0;for(let a=0,h=e.length;a<h;a++){let s=e.charCodeAt(a);if(s>=32&&s<127)continue;if(s===10){c++,f=a+1;continue}if(s===13){e.charCodeAt(a+1)!==10&&(c++,f=a+1);continue}let x=a,F=s,E;if(s>=55296&&s<=56319){let t=e.charCodeAt(a+1);if(!(t>=56320&&t<=57343))continue;F=65536+(s-55296<<10)+(t-56320),E=b(F),a++}else E=I(s);if(E!==null&&(o[E]++,l++,i.length<u)){let t={offset:x,line:c,column:x-f+1,codePoint:F,category:E};E==="bom"&&(t.atStart=x===0),i.push(t)}}return{findings:i,counts:o,total:l,truncated:l>i.length}}function R(e){if(typeof e!="string")throw new TypeError("getNewlineStats expects a string input.");let r=0,n=0,u=0;for(let l=0,c=e.length;l<c;l++){let f=e.charCodeAt(l);f===13?e.charCodeAt(l+1)===10?(r++,l++):n++:f===10&&u++}let o={crlf:r,cr:n,lf:u},i=[];return o.crlf>0&&i.push("CRLF"),o.cr>0&&i.push("CR"),o.lf>0&&i.push("LF"),{...o,types:i,mixed:i.length>1}}function H(e){return{normalizeSpaces:e.hasInvisibleChars,purgeInvisibleChars:e.hasInvisibleChars,purgeEmojis:!1,nukeControls:e.hasControlChars,keyboardOnlyFilter:!1,normalizeNewlines:e.hasMixedNewlines||e.newlineStyle==="CRLF"||e.newlineStyle==="CR"}}function W(e={}){let r=!!e.normalizeSpaces,n=!!e.purgeZeroWidthSpaces,u=!!e.purgeInvisibleChars,o=!!e.purgeEmojis,i=!!e.nukeControls,l=!!e.keyboardOnlyFilter,c=!!e.normalizeNewlines,f=!!e.trimSpacesAroundNewlines,a=!!e.collapseNewLines,h=!!e.preserveParagraphs,s=!!e.collapseSpaces,x=!!e.finalTrim,F=Number.isFinite(e.maxLength)&&e.maxLength>=0?e.maxLength:1/0,E=e.onMaxLength==="truncate"||e.onMaxLength==="noop"||e.onMaxLength==="throw"?e.onMaxLength:"throw";return t=>_(t,u,o,i,l,c,f,a,h,s,x,F,E,r,n)}function w(e){return _(e,!0,!0,!0,!1,!0,!0,!0,!1,!0,!0)}function N(e){return _(e,!1,!1,!1,!1,!0,!0,!0,!0,!0,!0)}function T(e){return _(e,!1,!1,!1,!0,!0,!0,!1,!1,!1,!0)}function L(e){return _(e,!0,!0,!0,!0,!0,!0,!0,!1,!0,!0)}function D(e){return _(e,!1,!1,!1,!1,!0,!1,!1,!1,!1,!1,1/0,"throw",!0,!0)}var J=Object.assign(W,{strict:w,loose:N,keyboardOnlyEmoji:T,keyboardOnly:L,pasted:D});function _(e,r=!1,n=!1,u=!1,o=!1,i=!1,l=!1,c=!1,f=!1,a=!1,h=!1,s=1/0,x="throw",F=!1,E=!1){if(typeof e!="string")throw new TypeError("sanctifyText expects a string input.");if(e.length>s)switch(x){case"truncate":{let C=e.charCodeAt(s-1);e=e.slice(0,C>=55296&&C<=56319?s-1:s)}break;case"noop":return e;default:throw new RangeError(`sanctifyText input length ${e.length} exceeds maxLength ${s}.`)}let t=e.replace($,"\uFFFD");return F&&(t=t.replace(V," ")),E&&(t=t.replace(Z,"")),r&&(t=Y(t)),n&&(t=ee(t)),u&&(t=fe(t)),o&&(t=Q(t,n)),i&&(t=ne(t)),l&&(t=ie(t)),c&&(t=se(t,f)),a&&(t=ae(t)),h?t.trim():t}var V=/[\u00A0\u2000-\u200A\u202F\u205F]/g,$=/[\uD800-\uDFFF]/gu,Z=/[\u200B\u2060\uFEFF]/g;function Y(e){return e.replace(S,"")}function q(e){if(p.lastIndex=0,!p.test(e))return"";p.lastIndex=0;let r="";for(let n of e.matchAll(p))r+=n[0];return r}function Q(e,r=!1){let n=d(e);return r?n.replace(g,""):n.replace(g,u=>q(u))}function ee(e){return e.replace(p,"")}var re=/\r\n|\r|\n/g;function ne(e,r=`
+`){return e.replace(re,r)}var te=/[ \t]*(\n+)[ \t]*|([ \t]+)/g;function ie(e){return e.replace(te,"$1$2")}var oe=/\n{2,}/g,ue=/\n{3,}/g;function se(e,r){return r?e.replace(ue,`
 
-`):e.replace(te,`
-`)}var ue=/ {2,}/g;function se(e){return e.replace(ue," ")}function le(e){return e.replace(y,"")}function ae(e,r={}){if(typeof e!="string")throw new TypeError("revealInvisible expects a string input.");let n=r&&typeof r.format=="function"?r.format:i=>"<U+"+i.toString(16).toUpperCase().padStart(4,"0")+">",u="",o=0;for(let i of C(e,{maxFindings:1/0}).findings)u+=e.slice(o,i.offset)+n(i.codePoint,i.category),o=i.offset+(i.codePoint>65535?2:1);return u+e.slice(o)}var fe="1.3.10";export{fe as VERSION,b as getNewlineStats,K as getRecommendedSanctifierOptions,j as inspectText,C as inspectTextDetailed,ae as revealInvisible,T as sanctifyKeyboardOnly,N as sanctifyKeyboardOnlyEmoji,w as sanctifyLoose,L as sanctifyPasted,R as sanctifyStrict,W as summonSanctifier};
+`):e.replace(oe,`
+`)}var le=/ {2,}/g;function ae(e){return e.replace(le," ")}function fe(e){return e.replace(y,"")}function ce(e,r={}){if(typeof e!="string")throw new TypeError("revealInvisible expects a string input.");let n=r&&typeof r.format=="function"?r.format:i=>"<U+"+i.toString(16).toUpperCase().padStart(4,"0")+">",u="",o=0;for(let i of A(e,{maxFindings:1/0}).findings)u+=e.slice(o,i.offset)+n(i.codePoint,i.category),o=i.offset+(i.codePoint>65535?2:1);return u+e.slice(o)}var Ee="1.3.11";export{Ee as VERSION,R as getNewlineStats,H as getRecommendedSanctifierOptions,j as inspectText,A as inspectTextDetailed,ce as revealInvisible,L as sanctifyKeyboardOnly,T as sanctifyKeyboardOnlyEmoji,N as sanctifyLoose,D as sanctifyPasted,w as sanctifyStrict,J as summonSanctifier};

package/dist/types/inspectText.d.ts

@@ -43,7 +43,10 @@ export function inspectTextDetailed(text: string, options?: InspectDetailedOptio
  */
 export function getNewlineStats(text: string): NewlineStats;
 /**
- * Creates defaultOptions for summonSanctifier based on inspectText result
+ * Creates defaultOptions for summonSanctifier based on inspectText result.
+ * Recommends only what removes trash without losing meaning: non-ASCII text and emoji are
+ * content, so `purgeEmojis` and `keyboardOnlyFilter` stay opt-in, and look-alike spaces
+ * become ordinary spaces so that words do not run together.
  * @param {!UnicodeTrashReport} report
  * @return {!SanctifyOptions}
  */

package/dist/types/sanctifyText.d.ts

@@ -68,7 +68,7 @@ export function sanctifyPasted(text: string): string;
  * @param {number} [maxLength=Infinity] - Hard input length cap (UTF-16 code units via `text.length`).
  * @param {'throw'|'truncate'|'noop'} [onMaxLength='throw'] - Behavior when `text.length` exceeds `maxLength`:
  *   - `'throw'`: throw a `RangeError`
- *   - `'truncate'`: slice to `maxLength` before processing
+ *   - `'truncate'`: slice to at most `maxLength` before processing, never through a surrogate pair
  *   - `'noop'`: return the original input unchanged (skips processing)
  * @param {boolean} [normalizeSpaces=false] - Replace look-alike spaces with an ordinary space. Runs first,
  *   so `purgeInvisibleChars` no longer joins the words around them.

package/dist/types/version.generated.d.ts

@@ -1,2 +1,2 @@
 /** The installed version of text-sanctifier, as in its package.json. */
-export const VERSION: "1.3.10";
+export const VERSION: "1.3.11";

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "text-sanctifier",
-  "version": "1.3.10",
+  "version": "1.3.11",
   "type": "module",
   "description": "Find and remove invisible Unicode in web input, model-bound text and repositories.",
   "engines": {
@@ -48,15 +48,17 @@
     "build": "node scripts/build.js",
     "build:types": "tsc -p tsconfig.types.json",
     "test:types": "tsc -p tests/types/tsconfig.json",
-    "test": "node tests/sanctifyText.test.js && node tests/inspectTextDetailed.test.js && node --expose-gc tests/hiddenText.test.js && node --expose-gc tests/context.test.js && node tests/cli.test.js && node tests/release.test.js",
+    "test": "node tests/sanctifyText.test.js && node tests/inspectTextDetailed.test.js && node --expose-gc tests/hiddenText.test.js && node --expose-gc tests/context.test.js && node tests/cli.test.js && node tests/release.test.js && node tests/names.test.js && node tests/docs.test.js && node tests/repo.test.js",
     "test-min": "node tests/sanctifyText.test.min.js",
     "check:self": "node bin/text-sanctifier.js check .",
     "check:names": "node scripts/check-names.js",
-    "ci": "npm run gen:check && npm run check:names && npm run check:self && npm run test && npm run build && npm run size && npm run build:types && npm run test:types && npm run test-min",
+    "check:repo": "node scripts/check-repo.js",
+    "ci": "npm run gen:check && npm run check:names && npm run check:repo && npm run check:self && npm run test && npm run build && npm run size && npm run build:types && npm run test:types && npm run test-min",
     "size": "node scripts/check-size.js",
     "bench": "node --expose-gc scripts/bench.js",
     "release:prep": "node scripts/release.js prep",
     "release:verify": "node scripts/release.js verify",
+    "prepublishOnly": "node scripts/release.js check-publish",
     "hooks:install": "git config core.hooksPath scripts/hooks"
   },
   "keywords": [
@@ -83,7 +85,7 @@
     "test": "tests"
   },
   "devDependencies": {
-    "esbuild": "^0.25.3",
-    "typescript": "^5.9.3"
+    "esbuild": "0.25.12",
+    "typescript": "5.9.3"
   }
 }

package/src/inspectText.js

@@ -322,16 +322,20 @@ export function getNewlineStats(text) {
 }
 
 /**
- * Creates defaultOptions for summonSanctifier based on inspectText result
+ * Creates defaultOptions for summonSanctifier based on inspectText result.
+ * Recommends only what removes trash without losing meaning: non-ASCII text and emoji are
+ * content, so `purgeEmojis` and `keyboardOnlyFilter` stay opt-in, and look-alike spaces
+ * become ordinary spaces so that words do not run together.
  * @param {!UnicodeTrashReport} report
  * @return {!SanctifyOptions}
  */
 export function getRecommendedSanctifierOptions(report) {
     return {
+        normalizeSpaces: report.hasInvisibleChars,
         purgeInvisibleChars: report.hasInvisibleChars,
-        purgeEmojis: report.hasEmojis,
+        purgeEmojis: false,
         nukeControls: report.hasControlChars,
-        keyboardOnlyFilter: report.hasNonKeyboardChars,
+        keyboardOnlyFilter: false,
         normalizeNewlines:
             report.hasMixedNewlines || report.newlineStyle === 'CRLF' || report.newlineStyle === 'CR',
     };

package/src/sanctifyText.js

@@ -294,7 +294,7 @@ export const summonSanctifier = /* @__PURE__ */ Object.assign(createSanctifier,
  * @param {number} [maxLength=Infinity] - Hard input length cap (UTF-16 code units via `text.length`).
  * @param {'throw'|'truncate'|'noop'} [onMaxLength='throw'] - Behavior when `text.length` exceeds `maxLength`:
  *   - `'throw'`: throw a `RangeError`
- *   - `'truncate'`: slice to `maxLength` before processing
+ *   - `'truncate'`: slice to at most `maxLength` before processing, never through a surrogate pair
  *   - `'noop'`: return the original input unchanged (skips processing)
  * @param {boolean} [normalizeSpaces=false] - Replace look-alike spaces with an ordinary space. Runs first,
  *   so `purgeInvisibleChars` no longer joins the words around them.
@@ -324,26 +324,27 @@ export function sanctifyText(
         throw new TypeError('sanctifyText expects a string input.');
     }
 
-    if (onMaxLength !== 'truncate' && onMaxLength !== 'noop' && onMaxLength !== 'throw') {
-        onMaxLength = 'throw';
-    }
-
     if (text.length > maxLength) {
         switch (onMaxLength) {
             case 'truncate':
-                text = text.slice(0, maxLength);
+                {
+                    // Cut before a surrogate pair, not through it
+                    const last = text.charCodeAt(maxLength - 1);
+                    text = text.slice(0, last >= 0xD800 && last <= 0xDBFF ? maxLength - 1 : maxLength);
+                }
                 break;
             case 'noop':
                 return text;
-            case 'throw':
-            default:
+            default: // 'throw', and any invalid value
                 throw new RangeError(
                     `sanctifyText input length ${text.length} exceeds maxLength ${maxLength}.`
                 );
         }
     }
 
-    let cleaned = text;
+    // Unpaired surrogates become U+FFFD first, so removing a character between two
+    // halves cannot join them into a new character, such as a hidden tag.
+    let cleaned = text.replace(LONE_SURROGATES_REGEX, '\uFFFD');
 
     // Look-alike spaces become ordinary spaces, before anything can delete them
     if (normalizeSpaces) cleaned = cleaned.replace(SPACE_LOOKALIKES_REGEX, ' ');
@@ -389,6 +390,9 @@ export function sanctifyText(
  */
 export const SPACE_LOOKALIKES_REGEX = /[\u00A0\u2000-\u200A\u202F\u205F]/g;
 
+/** With the `u` flag a surrogate pair is one character, so this matches only unpaired halves. */
+const LONE_SURROGATES_REGEX = /[\uD800-\uDFFF]/gu;
+
 /** Zero-width space, word joiner and BOM: invisible and never needed in plain text. */
 export const ZERO_WIDTH_SPACES_REGEX = /[\u200B\u2060\uFEFF]/g;
 
@@ -497,9 +501,11 @@ function normalizeNewlineChars(text, normalized = '\n') {
  * @param {string} text
  * @returns {string}
  */
-const TRIM_SPACES_AROUND_NEWLINES_REGEX = /[ \t]*(\n+)[ \t]*/g;
+// The second branch consumes a run with no newline after it, so matching resumes past the
+// run instead of retrying from each of its characters, which took quadratic time.
+const TRIM_SPACES_AROUND_NEWLINES_REGEX = /[ \t]*(\n+)[ \t]*|([ \t]+)/g;
 function trimSpacesAroundNewlineChars(text) {
-    return text.replace(TRIM_SPACES_AROUND_NEWLINES_REGEX, '$1');
+    return text.replace(TRIM_SPACES_AROUND_NEWLINES_REGEX, '$1$2');
 }
 
 

package/src/version.generated.js

@@ -2,4 +2,4 @@
 // Generated from package.json by scripts/gen-version.js (`npm run gen`); do not edit.
 
 /** The installed version of text-sanctifier, as in its package.json. */
-export const VERSION = '1.3.10';
+export const VERSION = '1.3.11';

package/src/index.d.ts

@@ -1,22 +0,0 @@
-import { inspectText } from './inspectText.js';
-import { inspectTextDetailed } from './inspectText.js';
-import { getNewlineStats } from './inspectText.js';
-import { getRecommendedSanctifierOptions } from './inspectText.js';
-import { summonSanctifier } from './sanctifyText.js';
-import { sanctifyStrict } from './sanctifyText.js';
-import { sanctifyLoose } from './sanctifyText.js';
-import { sanctifyKeyboardOnlyEmoji } from './sanctifyText.js';
-import { sanctifyKeyboardOnly } from './sanctifyText.js';
-import { sanctifyPasted } from './sanctifyText.js';
-import { revealInvisible } from './reveal.js';
-import { VERSION } from './version.generated.js';
-export { inspectText, inspectTextDetailed, getNewlineStats, getRecommendedSanctifierOptions, summonSanctifier, sanctifyStrict, sanctifyLoose, sanctifyKeyboardOnlyEmoji, sanctifyKeyboardOnly, sanctifyPasted, revealInvisible, VERSION };
-export type UnicodeTrashReport = import('./inspectText.js').UnicodeTrashReport;
-export type TrashCategory = import('./inspectText.js').TrashCategory;
-export type TrashFinding = import('./inspectText.js').TrashFinding;
-export type TrashCounts = import('./inspectText.js').TrashCounts;
-export type DetailedTrashReport = import('./inspectText.js').DetailedTrashReport;
-export type InspectDetailedOptions = import('./inspectText.js').InspectDetailedOptions;
-export type NewlineStats = import('./inspectText.js').NewlineStats;
-export type SanctifyOptions = import('./sanctifyText.js').SanctifyOptions;
-export type RevealOptions = import('./reveal.js').RevealOptions;