purifai 3.0.0 → 3.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +191 -154
  2. package/benchmark/results/v3.json +1152 -1152
  3. package/docs/benchmarks/v3.md +44 -44
  4. package/package.json +2 -2
package/README.md CHANGED
@@ -3,15 +3,16 @@
3
3
 
4
4
  # Purifai
5
5
 
6
- [npm](https://www.npmjs.com/package/purifai) ·
7
- [Project notes](https://worksonmy.dev/projects/purifai) ·
8
- [Runnable examples](https://github.com/moji2002/purifai/tree/main/examples) ·
9
- [Issues](https://github.com/moji2002/purifai/issues)
6
+ [![npm version](https://img.shields.io/npm/v/purifai.svg)](https://www.npmjs.com/package/purifai)
7
+ [![CI](https://github.com/moji2002/purifai/actions/workflows/ci.yml/badge.svg)](https://github.com/moji2002/purifai/actions/workflows/ci.yml)
8
+ [![gzip: 23.7 KiB](https://img.shields.io/badge/gzip-23.7_KiB-2f855a)](docs/benchmarks/v3.md)
9
+ [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
10
10
 
11
- Purifai is a fixed-policy HTML-to-readable-text converter for servers, browsers,
12
- and edge runtimes. It incrementally removes non-reader bodies, decodes the full
13
- WHATWG character-reference set, and preserves useful structure such as headings,
14
- paragraphs, lists, links, image alternatives, code, and simple tables.
11
+ **Readable text from hostile HTML—without a DOM.**
12
+
13
+ Purifai is a fixed-policy HTML-to-text converter for servers, browsers, and
14
+ edge runtimes. It keeps useful document structure, drops non-reader bodies, and
15
+ enforces input, output, nesting, and retained-token limits while scanning.
15
16
 
16
17
  ```ts
17
18
  import { toText } from 'purifai';
@@ -19,6 +20,8 @@ import { toText } from 'purifai';
19
20
  const text = toText(
20
21
  '<script>alert(1)</script><h2>Release</h2><ul><li>Fast</li></ul>',
21
22
  );
23
+
24
+ console.log(text);
22
25
  // Release
23
26
  //
24
27
  // - Fast
@@ -27,32 +30,16 @@ const text = toText(
27
30
  A flat tag remover can leak `alert(1)` from the script body and collapse the
28
31
  remaining text. Purifai drops that body and formats the reader content.
29
32
 
30
- The output is a JavaScript string, not safe HTML. Use one of these supported
31
- sinks:
32
-
33
- ```ts
34
- import { escapeHtmlText, toText } from 'purifai';
35
-
36
- element.textContent = toText(untrustedHtml);
37
- element.innerHTML = escapeHtmlText(toText(untrustedHtml));
38
- ```
39
-
40
- Prefer `textContent`. `escapeHtmlText` exists for an HTML text context only; it
41
- does not make a value safe for an attribute, URL, JavaScript, CSS, or template
42
- source.
43
-
44
- ## Why it exists
33
+ ## Choose Purifai when
45
34
 
46
- Purifai targets one narrow intersection:
35
+ - HTML may be large, malformed, or hostile.
36
+ - You want readable plain text—not preserved markup or a browser DOM.
37
+ - Conversion must have deterministic resource limits.
38
+ - The same implementation must run in Node, Bun, Deno, Workers, and browsers.
39
+ - Streaming should produce the same result regardless of chunk boundaries.
47
40
 
48
- - readable extraction instead of flat deletion;
49
- - deterministic input, output, nesting, and retained-token limits;
50
- - chunk-invariant Web `TransformStream` conversion;
51
- - no DOM, tree, Node built-in, or runtime dependency; and
52
- - one side-effect-free artifact across server, browser, and edge runtimes.
53
-
54
- It does not preserve markup and does not classify a user's intent. If either is
55
- your requirement, use a tool designed for that different job.
41
+ If you need selector-driven formatting, complex table layout, or allow-listed
42
+ safe HTML, jump to [Which tool should you choose?](#which-tool-should-you-choose).
56
43
 
57
44
  ## Install
58
45
 
@@ -60,57 +47,70 @@ your requirement, use a tool designed for that different job.
60
47
  npm install purifai
61
48
  ```
62
49
 
63
- Purifai v3 requires Node.js 22 or newer when used in Node.
64
-
65
- ## API
66
-
67
- ### `toText(html, options?)`
50
+ Purifai v3 requires Node.js 22 or newer when used in Node. It ships ESM and
51
+ CommonJS exports and has zero runtime dependencies.
68
52
 
69
- Converts one string and returns readable text. A breached limit throws a
70
- `PurifaiLimitError`.
53
+ ## Quick start
71
54
 
72
55
  ```ts
73
56
  import { toText } from 'purifai';
74
57
 
75
- const text = toText('<h1>Guide</h1><p>Start here.</p>', {
58
+ const text = toText('<h1>Guide</h1><p>Start <strong>here</strong>.</p>', {
76
59
  layout: 'readable',
77
- links: 'label-and-url',
60
+ links: 'label',
78
61
  images: 'alt',
79
- baseUrl: 'https://docs.example/',
80
- limits: { input: 1_000_000, output: 250_000, depth: 64, token: 65_536 },
81
62
  });
63
+
64
+ // Guide
65
+ //
66
+ // Start here.
82
67
  ```
83
68
 
84
- ### `convert(html, options?)`
69
+ `toText` returns a JavaScript string. It does not return safe HTML.
85
70
 
86
- Returns the text plus a frozen conversion report. It is the only entry point
87
- that can deliberately return a bounded prefix instead of throwing.
71
+ ## Safe output
72
+
73
+ Prefer a text sink:
88
74
 
89
75
  ```ts
90
- import { convert } from 'purifai';
76
+ element.textContent = toText(untrustedHtml);
77
+ ```
91
78
 
92
- const result = convert(largeHtml, {
93
- limits: { output: 20_000 },
94
- overflow: 'truncate',
95
- });
79
+ If the only available sink is an HTML text node, escape the text explicitly:
96
80
 
97
- result.text;
98
- result.truncatedBy; // 'output' or null
99
- result.scanComplete; // false after truncation
100
- result.consumedInputCodeUnits;
101
- result.outputCodeUnits;
102
- result.droppedContainers; // e.g. { script: 2, style: 1 }
81
+ ```ts
82
+ import { escapeHtmlText, toText } from 'purifai';
83
+
84
+ element.innerHTML = escapeHtmlText(toText(untrustedHtml));
103
85
  ```
104
86
 
105
- Truncation is explicit, deterministic, and never emits half of a UTF-16
106
- surrogate pair. When multiple limits meet at the same point, the first observed
107
- limit is reported.
87
+ `escapeHtmlText` is only for an HTML text context. It does not make a value safe
88
+ for an attribute, URL, JavaScript, CSS, or template source. A displayed URL is
89
+ also still text; moving it into `href` requires a separate URL-policy decision.
90
+
91
+ ## Why Purifai
92
+
93
+ Most HTML-to-text tools optimize for either minimal tag removal or broad
94
+ formatting control. Purifai targets a narrower intersection:
108
95
 
109
- ### `createTextTransform(options?)`
96
+ | Requirement | Purifai behavior |
97
+ | --- | --- |
98
+ | Reader-friendly output | Preserves headings, paragraphs, lists, quotes, code, simple tables, links, and image alternatives |
99
+ | Non-reader content | Drops bodies such as `script`, `style`, `template`, `iframe`, `svg`, and `math` |
100
+ | Hostile-input bounds | Enforces input, output, depth, and aggregate retained-token limits during scanning |
101
+ | Streaming | Uses a native Web `TransformStream` with chunk-invariant output |
102
+ | Portability | Uses no DOM, document tree, Node built-in, or runtime dependency |
103
+ | Predictability | Fixed policy, validated options, explicit overflow behavior, and frozen reports |
104
+
105
+ That fixed scope is the reason to choose Purifai. It deliberately does not
106
+ preserve markup, reconstruct CSS layout, expose custom formatters, or classify a
107
+ user's intent.
110
108
 
111
- Returns a native `TransformStream<string, string>` with a `result` promise. The
112
- stream uses the same state machine and produces exactly the same joined text as
113
- `toText`, regardless of chunk boundaries.
109
+ ## Streaming
110
+
111
+ `createTextTransform` converts incrementally using the same state machine as
112
+ `toText`. Joining its output produces exactly the same text for every possible
113
+ input chunking.
114
114
 
115
115
  ```ts
116
116
  import { createTextTransform } from 'purifai';
@@ -122,147 +122,184 @@ const transform = createTextTransform({ links: 'label-and-url' });
122
122
  const readable = response.body
123
123
  .pipeThrough(new TextDecoderStream())
124
124
  .pipeThrough(transform);
125
- const reader = readable.getReader();
126
125
 
127
- for (;;) {
128
- const { done, value } = await reader.read();
129
- if (done) break;
130
- consumeText(value);
126
+ for await (const chunk of readable) {
127
+ consumeText(chunk);
131
128
  }
132
129
 
133
130
  const report = await transform.result;
134
131
  ```
135
132
 
136
- Stream conversion always throws on a breached limit. Output may already have
137
- been enqueued when `readable` and `transform.result` reject, so discard partial
138
- output unless your application has deliberately defined it as useful. Purifai
139
- does not buffer the whole result to make an error transactional.
140
-
141
- ### `escapeHtmlText(text)`
133
+ Stream conversion always throws when a limit is breached. Some output may
134
+ already have been enqueued when `readable` and `transform.result` reject, so
135
+ discard partial output unless your application explicitly accepts it.
142
136
 
143
- Encodes `&`, `<`, `>`, `"`, and `'` for an HTML text node. It is lossless and is
144
- for plain text—including `toText` output—when the only available sink is
145
- `innerHTML`.
137
+ ## Bounded conversion
146
138
 
147
- ### `PurifaiLimitError`
148
-
149
- Extends `RangeError` and exposes `kind`, `limit`, and `observed`.
139
+ `toText` throws a `PurifaiLimitError` when any configured limit is exceeded.
140
+ Use `convert` only when a bounded prefix is an acceptable result:
150
141
 
151
142
  ```ts
152
- import { PurifaiLimitError, toText } from 'purifai';
143
+ import { convert } from 'purifai';
153
144
 
154
- try {
155
- toText(html, { limits: { input: 10_000 } });
156
- } catch (error) {
157
- if (error instanceof PurifaiLimitError) {
158
- console.error(error.kind, error.limit, error.observed);
159
- }
160
- }
145
+ const result = convert(largeHtml, {
146
+ limits: { input: 1_000_000, output: 20_000, depth: 64, token: 65_536 },
147
+ overflow: 'truncate',
148
+ });
149
+
150
+ result.text;
151
+ result.truncatedBy; // 'input', 'output', 'depth', 'token', or null
152
+ result.scanComplete; // false after truncation
153
+ result.consumedInputCodeUnits;
154
+ result.outputCodeUnits;
155
+ result.droppedContainers; // e.g. { script: 2, style: 1 }
161
156
  ```
162
157
 
163
- ## Options and defaults
158
+ Truncation is explicit and deterministic, and never emits half of a UTF-16
159
+ surrogate pair. `toText` and `createTextTransform` never truncate silently.
160
+
161
+ ## Options
164
162
 
165
- Unknown keys and invalid values throw `TypeError`; Purifai does not silently
166
- guess around configuration mistakes.
163
+ Unknown keys and invalid values throw `TypeError`; Purifai does not guess around
164
+ configuration mistakes.
167
165
 
168
166
  | Option | Type | Default | Meaning |
169
167
  | --- | --- | --- | --- |
170
- | `layout` | `'readable' \| 'compact'` | `'readable'` | Structural newlines/lists/tables, or normalized single-space text |
171
- | `links` | `'label' \| 'label-and-url' \| 'drop'` | `'label'` | Keep label, append an accepted display URL, or drop the link body |
168
+ | `layout` | `'readable' \| 'compact'` | `'readable'` | Structural boundaries, or normalized single-space text |
169
+ | `links` | `'label' \| 'label-and-url' \| 'drop'` | `'label'` | Keep the label, append an accepted display URL, or drop the link body |
172
170
  | `images` | `'alt' \| 'drop'` | `'alt'` | Emit decoded non-empty `alt` text, or omit images |
173
171
  | `baseUrl` | `string \| URL` | none | Resolve relative display URLs against a credential-free HTTP(S) base |
174
172
  | `limits.input` | non-negative safe integer | `1_000_000` | Maximum input UTF-16 code units consumed |
175
173
  | `limits.output` | non-negative safe integer | `250_000` | Maximum output UTF-16 code units emitted |
176
174
  | `limits.depth` | non-negative safe integer | `64` | Maximum live structural nesting |
177
- | `limits.token` | non-negative safe integer | `65_536` | Maximum aggregate retained token/attribute code units |
175
+ | `limits.token` | non-negative safe integer | `65_536` | Maximum aggregate retained token and attribute code units |
178
176
  | `overflow` | `'throw' \| 'truncate'` | `'throw'` | `convert` only; other APIs always throw |
179
177
 
180
- All four limits are enforced while scanning, before unbounded caller-controlled
181
- state can accumulate. Values measure JavaScript UTF-16 code units, not encoded
182
- bytes.
178
+ All four limits are enforced before unbounded caller-controlled state can
179
+ accumulate. Values measure JavaScript UTF-16 code units, not encoded bytes.
183
180
 
184
- ## Link policy
181
+ ### Display URL policy
185
182
 
186
- `label-and-url` emits a destination as display text, never as an active link. It
183
+ `label-and-url` emits destinations as display text, never as active links. It
187
184
  accepts absolute `http:`, `https:`, and `mailto:` URLs. Relative URLs require a
188
- validated HTTP(S) `baseUrl`. Control characters, whitespace-split schemes,
189
- protocol-relative inputs, leading backslashes, credentials, unsupported schemes,
190
- and invalid URLs are omitted while their visible label remains.
191
-
192
- The returned URL string is still only text. Do not move it into `href` without a
193
- separate URL-policy decision at that sink.
185
+ validated HTTP(S) `baseUrl`. Credentials, controls, ambiguous schemes,
186
+ protocol-relative inputs, leading backslashes, unsupported schemes, and invalid
187
+ URLs are omitted while their visible label remains.
194
188
 
195
- ## Extraction fidelity
189
+ ## Extraction policy
196
190
 
197
- Purifai intentionally removes source and non-reader bodies including `script`,
198
- `style`, `template`, `iframe`, `noscript`, `noembed`, `noframes`, `svg`, and
199
- `math`. It preserves selected fallback/form text, decodes `textarea`, preserves
200
- literal `xmp`, and treats `plaintext` as text through end of input.
191
+ Purifai removes source and non-reader bodies including `script`, `style`,
192
+ `template`, `iframe`, `noscript`, `noembed`, `noframes`, `svg`, and `math`. It
193
+ preserves selected fallback and form text, decodes the complete pinned WHATWG
194
+ character-reference set, preserves literal `xmp`, and treats `plaintext` as text
195
+ through end of input.
201
196
 
202
197
  This is a bounded extraction grammar, not browser tree construction. It does not
203
198
  recreate CSS layout, browser `innerText`, complex `rowspan`/`colspan` tables,
204
199
  SVG/MathML semantics, selector rules, custom formatters, or browser-equivalent
205
- malformed-markup recovery. Simple rows and cells are represented with tabs and
206
- line boundaries.
207
-
208
- ## Which tool should you choose?
209
-
210
- | Need | Choice |
211
- | --- | --- |
212
- | Fixed-policy readable text, hostile-input bounds, and portable Web streaming | Choose Purifai |
213
- | Selectors, custom formatters, advanced tables, wrapping, and broader formatting control | Choose `html-to-text` |
214
- | The smallest flat tag-removal operation | Choose stable `striptags` |
215
- | Preserve an allow-listed safe HTML fragment | Choose DOMPurify or `sanitize-html` |
216
-
217
- These tools are not interchangeable. In particular, DOMPurify and
218
- `sanitize-html` are the right category when safe markup must survive.
200
+ malformed-markup recovery.
219
201
 
220
202
  ## Benchmarks
221
203
 
222
204
  The checked category benchmark pins `striptags@3.2.0` and
223
- `html-to-text@10.0.0`. It measures exact readability/body-removal fixtures,
224
- isolated warm median and p95 one-shot latency, and fresh-process peak RSS. The
225
- throughput path gives every package the same materialized string; the memory path
226
- lets Purifai consume lazy 16,384-code-unit chunks because streaming ingestion is
227
- the product claim.
205
+ `html-to-text@10.0.0`. It measures reviewed readability and body-removal
206
+ fixtures, isolated warm median and p95 latency, and fresh-process peak RSS.
207
+
208
+ On the recorded Apple M1 / Node 24 run, Purifai passed all 11 category gates:
209
+
210
+ - 8/8 readability fixtures and all 5 non-reader-body fixtures;
211
+ - lower hostile-input p95 than `html-to-text` on four hostile corpora; and
212
+ - lower streaming peak RSS than `html-to-text` on all five memory corpora.
228
213
 
229
- On the recorded Apple M1 / Node 24 run, Purifai passed all 11 category gates: all
230
- readability and body-removal fixtures, lower hostile-input p95 than
231
- `html-to-text` on four hostile corpora, and lower streaming peak RSS on all five
232
- memory corpora. `striptags` remains faster on some flat-strip cases, which is not
233
- Purifai's claim.
214
+ `striptags` remains faster on some flat-strip cases. That is not Purifai's
215
+ claim. Results are machine-, runtime-, and corpus-specific.
234
216
 
235
- See the [complete methodology, raw-result link, and tables](docs/benchmarks/v3.md).
236
- Reproduce it with `pnpm run bench`; re-check the saved gates with
237
- `pnpm run bench:check`.
217
+ See the [complete methodology, raw results, and tables](docs/benchmarks/v3.md).
218
+ Reproduce measurements with `pnpm run bench`; check the recorded release gates
219
+ with `pnpm run bench:check`.
238
220
 
239
- ## Size and runtime matrix
221
+ ## Size, portability, and release proof
240
222
 
241
- The complete minified ESM runtime—including the full 2,231-name WHATWG entity
242
- data—is gated at 25 KiB using deterministic `gzip -9`. The recorded artifact is
243
- 23,689 bytes. `pnpm run test:size` also checks the packed exports, zero runtime
244
- dependencies, cold import time, and retained import heap.
223
+ The complete minified ESM runtime—including all 2,231 pinned WHATWG entity
224
+ names—is 23,689 bytes with deterministic `gzip -9`. The release gate also checks
225
+ packed exports, zero runtime dependencies, cold import time, and retained import
226
+ heap.
245
227
 
246
- The release matrix exercises the same packed ESM artifact:
228
+ The same packed artifact is tested in:
247
229
 
248
- | Runtime | Required release coverage |
230
+ | Runtime | Release coverage |
249
231
  | --- | --- |
250
232
  | Node.js | 22, 24, and 26; ESM and CommonJS |
251
- | Bun | ESM and CommonJS consumers |
252
- | Deno | ESM consumer |
253
- | Cloudflare Workers | real `workerd`, without Node compatibility |
233
+ | Bun | ESM and CommonJS |
234
+ | Deno | ESM |
235
+ | Cloudflare Workers | Real `workerd`, without Node compatibility |
254
236
  | Browsers | Chromium, Firefox, and WebKit |
255
237
 
256
- Browser qualification also reparses escaped output in a real DOM with a working
257
- positive control. This validates the documented sinks; it is not a universal
258
- claim about every output context.
238
+ Release qualification also includes 10,000 seeded malformed-input cases,
239
+ adversarial scaling checks, safe-sink tests with a positive control, package
240
+ smoke tests, and npm OIDC provenance bound to the tagged GitHub source commit.
241
+
242
+ ## API reference
243
+
244
+ ### `toText(html, options?) → string`
245
+
246
+ Converts one HTML string into readable text. Throws `TypeError` for invalid
247
+ input or options and `PurifaiLimitError` for a breached limit.
248
+
249
+ ### `convert(html, options?) → ConversionResult`
250
+
251
+ Returns text plus a frozen report containing completion, truncation, consumed
252
+ input, output length, and dropped-container counts. It is the only API that can
253
+ return a deliberately truncated prefix.
254
+
255
+ ### `createTextTransform(options?) → TextTransform`
256
+
257
+ Returns a native `TransformStream<string, string>` with a `result` promise for
258
+ the frozen conversion report. Limit failures reject both the stream and the
259
+ promise with the same error object.
260
+
261
+ ### `escapeHtmlText(text) → string`
262
+
263
+ Losslessly encodes `&`, `<`, `>`, `"`, and `'` for an HTML text-node context.
264
+
265
+ ### `PurifaiLimitError`
266
+
267
+ Extends `RangeError` and exposes `kind`, `limit`, and `observed`.
268
+
269
+ ```ts
270
+ import { PurifaiLimitError, toText } from 'purifai';
271
+
272
+ try {
273
+ toText(html, { limits: { input: 10_000 } });
274
+ } catch (error) {
275
+ if (error instanceof PurifaiLimitError) {
276
+ console.error(error.kind, error.limit, error.observed);
277
+ }
278
+ }
279
+ ```
280
+
281
+ ## Which tool should you choose?
282
+
283
+ | Need | Choice |
284
+ | --- | --- |
285
+ | Fixed-policy readable text, hostile-input bounds, and portable Web streaming | Choose Purifai |
286
+ | Selectors, custom formatters, advanced tables, wrapping, and broad formatting control | Choose `html-to-text` |
287
+ | The smallest flat tag-removal operation | Choose stable `striptags` |
288
+ | Preserve an allow-listed safe HTML fragment | Choose DOMPurify or `sanitize-html` |
289
+
290
+ These categories are not interchangeable. DOMPurify and `sanitize-html` are
291
+ the correct category when safe markup must survive.
259
292
 
260
293
  ## Migration and development
261
294
 
262
295
  V3 is a clean break. See the [v3 migration guide](docs/migration-v3.md) for every
263
- removed export and option. Contributor setup and the full verification commands
264
- are in [CONTRIBUTING.md](CONTRIBUTING.md).
296
+ removed export and option.
297
+
298
+ - [Runnable examples](examples)
299
+ - [Contributor guide](CONTRIBUTING.md)
300
+ - [Project notes](https://worksonmy.dev/projects/purifai)
301
+ - [Issues](https://github.com/moji2002/purifai/issues)
265
302
 
266
303
  ## License
267
304
 
268
- MIT
305
+ [MIT](LICENSE)