@nodable/entities 1.0.0

package/README.md

@@ -0,0 +1,476 @@
+# `@nodable/entities`
+
+Standalone, zero-dependency XML/HTML entity replacement with:
+
+- **5 entity categories** processed in a fixed, predictable order
+- **Persistent vs. input entity separation** — no state leaks between documents
+- **`getInstance()`** — clean per-document reset without cloning
+- **Composable named entity groups** (HTML, currency, math, arrows, numeric refs)
+- **Security limits** — cap total expansions and expanded length per document
+- **Granular limit targeting** — apply limits to any subset of categories
+- **`postCheck` hook** — inspect or sanitize the fully resolved string
+
+---
+
+## Installation
+
+```sh
+npm install @nodable/entities
+```
+
+---
+
+## Quick Start
+
+```js
+import EntityReplacer from '@nodable/entities';
+
+const replacer = new EntityReplacer({ default: true });
+
+replacer.replace('5 < 10 && x > 0');
+// → '5 < 10 && x > 0'
+```
+
+With named entity groups:
+
+```js
+import EntityReplacer, { COMMON_HTML, CURRENCY_ENTITIES } from '@nodable/entities';
+
+const replacer = new EntityReplacer({
+  default: true,
+  system: { ...COMMON_HTML, ...CURRENCY_ENTITIES },
+});
+
+replacer.replace('&copy; 2024 &mdash; Price: &pound;9.99');
+// → '© 2024 — Price: £9.99'
+```
+
+---
+
+## Entity Categories
+
+Entities are processed in this fixed order — not configurable:
+
+```
+persistent external → input/runtime → system → default → amp
+```
+
+### `persistent external` — Caller-supplied configuration entities
+
+Entities set at configuration time that survive across all documents. Never wiped by `getInstance()`. Set via `setExternalEntities()` or `addExternalEntity()` / `addEntity()`.
+
+```js
+const replacer = new EntityReplacer({ default: true });
+replacer.setExternalEntities({ brand: 'Acme Corp', product: 'Widget Pro' });
+replacer.replace('&brand; makes &product;');
+// → 'Acme Corp makes Widget Pro'
+```
+
+### `input / runtime` — Per-document DOCTYPE entities
+
+Entities injected by the parser from the document's DOCTYPE block. Stored separately from persistent entities and **wiped on every `getInstance()` call** so they cannot leak between documents.
+
+Set via `addInputEntities()`. Never call this manually — `BaseOutputBuilder` calls it automatically.
+
+### `system` — Named entity groups
+
+Opt-in. Trusted programmer-supplied groups. Compose freely:
+
+```js
+import {
+  COMMON_HTML,
+  CURRENCY_ENTITIES,
+  MATH_ENTITIES,
+  ARROW_ENTITIES,
+  NUMERIC_ENTITIES,
+} from '@nodable/entities';
+
+const replacer = new EntityReplacer({
+  system: { ...COMMON_HTML, ...MATH_ENTITIES },
+});
+```
+
+| Group               | Contents |
+|---------------------|----------|
+| `COMMON_HTML`       | `&nbsp;` `&copy;` `&reg;` `&trade;` `&mdash;` `&ndash;` `&hellip;` `&laquo;` `&raquo;` `&lsquo;` `&rsquo;` `&ldquo;` `&rdquo;` `&bull;` `&para;` `&sect;` `&deg;` `&frac12;` `&frac14;` `&frac34;` |
+| `CURRENCY_ENTITIES` | `&cent;` `&pound;` `&yen;` `&euro;` `&inr;` `&curren;` `&fnof;` |
+| `MATH_ENTITIES`     | `&times;` `&divide;` `&plusmn;` `&minus;` `&sup2;` `&sup3;` `&permil;` `&infin;` `&sum;` `&prod;` `&radic;` `&ne;` `&le;` `&ge;` |
+| `ARROW_ENTITIES`    | `&larr;` `&uarr;` `&rarr;` `&darr;` `&harr;` `&lArr;` `&uArr;` `&rArr;` `&dArr;` `&hArr;` |
+| `NUMERIC_ENTITIES`  | `&#NNN;` decimal and `&#xHH;` hex refs — any valid Unicode code point |
+
+### `default` — Built-in XML entities
+
+Always on unless explicitly disabled.
+
+| Entity   | Output |
+|----------|--------|
+| `&lt;`   | `<`    |
+| `&gt;`   | `>`    |
+| `&quot;` | `"`    |
+| `&apos;` | `'`    |
+
+### `amp` — Final pass
+
+`&amp;` → `&`
+
+Processed **after all other categories** to prevent double-expansion:
+- `&amp;lt;` → `&lt;` ✓ (not `<`)
+- `&amp;amp;` → `&amp;` ✓ (not `&`)
+
+---
+
+## Constructor API
+
+```js
+const replacer = new EntityReplacer({
+  // Category toggles
+  default: true,     // true (default) | false | custom EntityTable object
+  amp:     true,     // true (default) | false | null
+  system:  false,    // false (default) | true for COMMON_HTML | EntityTable object
+
+  // Security limits — 0 = unlimited
+  maxTotalExpansions: 0,
+  maxExpandedLength:  0,
+
+  // Which categories count against the limits
+  applyLimitsTo: 'external',   // 'external' (default) | 'all' | ['external', 'system'] | ...
+
+  // Post-processing hook — fires once on the fully resolved string
+  postCheck: null,   // (resolved: string, original: string) => string
+});
+```
+
+---
+
+## EntityReplacer Instance Methods
+
+### `replace(str)`
+
+Replace all entity references in `str`. Returns `str` unchanged (same reference) if no `&` is present — fast path.
+
+```js
+replacer.replace('Tom &amp; Jerry &lt;cartoons&gt;');
+// → 'Tom & Jerry <cartoons>'
+```
+
+### `setExternalEntities(map)`
+
+Replace the full set of **persistent** external entities. These survive across all documents and are not cleared by `getInstance()`.
+
+```js
+replacer.setExternalEntities({ brand: 'Acme', year: '2025' });
+```
+
+Calling this a second time replaces the entire persistent map. Values containing `&` are silently skipped.
+
+### `addExternalEntity(key, value)`
+
+Append a single persistent external entity without disturbing the rest.
+
+```js
+replacer.addExternalEntity('brand', 'Acme');
+replacer.addExternalEntity('year', '2025');
+```
+
+### `addInputEntities(map)`
+
+Inject **input/runtime** (DOCTYPE) entities for the current document. These are stored separately from persistent entities and wiped on the next `getInstance()` call. Also resets per-document expansion counters.
+
+```js
+// Called automatically by BaseOutputBuilder — no manual wiring needed.
+replacer.addInputEntities(doctypeEntityMap);
+```
+
+Values containing `&` are silently skipped. Accepts pre-built `{ regex, val }` or `{ regx, val }` objects as produced by `DocTypeReader`.
+
+### `getInstance()`
+
+Reset all per-document state and return `this`.
+
+**Clears:**
+- input/runtime entities (DOCTYPE)
+- `_totalExpansions` counter
+- `_expandedLength` counter
+
+**Preserves:**
+- persistent external entities set via `setExternalEntities()` / `addExternalEntity()`
+- all constructor config
+
+The builder factory calls this when creating a new builder instance, ensuring each document starts clean whether or not it has a DOCTYPE.
+
+```js
+// In a builder factory:
+getInstance() {
+  const builder = new MyBuilder(this.config);
+  builder.entityParser = this.entityVP.getInstance();
+  return builder;
+}
+```
+
+---
+
+## Document-to-Document Safety
+
+A key design goal is that entities from one document never bleed into the next. Here's how the two categories work together:
+
+```
+Document 1 parse:
+  factory.getInstance()           → evp.getInstance()  [clears input, resets counters]
+  builder sees DOCTYPE            → evp.addInputEntities({ version: '1.0' })
+  builder processes values        → evp.parse('&brand; v&version;') → 'Acme v1.0'
+
+Document 2 parse (no DOCTYPE):
+  factory.getInstance()           → evp.getInstance()  [clears &version;, resets counters]
+  no DOCTYPE                      → addInputEntities() not called
+  builder processes values        → evp.parse('&brand; v&version;') → 'Acme v&version;'
+                                    ↑ persistent &brand; works
+                                                 ↑ &version; is gone — correct
+```
+
+---
+
+## Security Controls
+
+### Expansion count limit
+
+Caps the number of entity references that may be expanded per document.
+
+```js
+const replacer = new EntityReplacer({ maxTotalExpansions: 1000 });
+```
+
+Throws `Error` if exceeded:
+> `[EntityReplacer] Entity expansion count limit exceeded: 1001 > 1000`
+
+### Expanded length limit
+
+Caps the total number of characters *added* by entity expansion per document.
+
+```js
+const replacer = new EntityReplacer({ maxExpandedLength: 65536 });
+```
+
+Throws `Error` if exceeded:
+> `[EntityReplacer] Expanded content length limit exceeded: 65537 > 65536`
+
+### `applyLimitsTo`
+
+Controls which categories count against the limits.
+
+```js
+// Default — only untrusted injected entities (safest)
+applyLimitsTo: 'external'
+
+// All categories
+applyLimitsTo: 'all'
+
+// Specific combination
+applyLimitsTo: ['external', 'system']
+applyLimitsTo: ['external', 'default']
+```
+
+---
+
+## `postCheck` Hook
+
+Fires **once** on the fully resolved string, after all categories have been processed. Not called if the string is unchanged (no `&` present or no matches found).
+
+```js
+// Signature
+postCheck: (resolved: string, original: string) => string
+```
+
+- `resolved` — string after all entity replacements
+- `original` — the original input string before any replacement
+- Must **return a string**
+- To reject expansion: `return original`
+- To sanitize: return a modified version of `resolved`
+
+Examples:
+
+```js
+// Reject if expansion produces any HTML tags
+postCheck: (resolved, original) =>
+  /<[a-z]/i.test(resolved) ? original : resolved
+
+// Strip all tag-like content from the result
+postCheck: (resolved) =>
+  resolved.replace(/<[^>]*>/g, '')
+```
+
+---
+
+## `EntitiesValueParser` — flex-xml-parser adapter
+
+`EntitiesValueParser` wraps `EntityReplacer` and implements the `ValueParser` interface used by `@nodable/flexible-xml-parser`.
+
+### Setup
+
+```js
+import { EntitiesValueParser, COMMON_HTML } from '@nodable/entities';
+
+const evp = new EntitiesValueParser({
+  system: COMMON_HTML,
+  maxTotalExpansions: 500,
+});
+
+// Persistent entities — survive across all documents:
+evp.setExternalEntities({ brand: 'Acme', product: 'Widget' });
+
+// Register with the builder factory:
+myBuilder.registerValueParser('entity', evp);
+
+const parser = new XMLParser({ OutputBuilder: myBuilder });
+parser.parse(xml);
+```
+
+### Constructor options
+
+All `EntityReplacerOptions` are accepted, plus one extra:
+
+```js
+new EntitiesValueParser({
+  // All EntityReplacer options...
+  default: true,
+  system: COMMON_HTML,
+  maxTotalExpansions: 1000,
+  postCheck: (resolved, original) => resolved,
+
+  // Extra: initial persistent entity map (same as calling setExternalEntities after construction)
+  entities: { copy: '©', trade: '™', brand: 'Acme Corp' },
+})
+```
+
+### `setExternalEntities(map)`
+
+Replace the full persistent entity map. These entities survive across all documents.
+
+```js
+evp.setExternalEntities({ brand: 'Acme', copy: '©' });
+```
+
+### `addEntity(key, value)`
+
+Append a single persistent external entity. Previously registered entities are preserved.
+
+```js
+evp.addEntity('copy',  '©');
+evp.addEntity('trade', '™');
+evp.addEntity('year',  '2024');
+```
+
+Throws if `key` contains `&` or `;`, or if `value` contains `&`.
+
+### `getInstance()` — called by builder factory
+
+Reset per-document state (input entities + counters) and return `this`. The builder factory calls this each time it creates a new builder instance.
+
+```js
+// In your CompactObjBuilderFactory.getInstance():
+getInstance() {
+  const builder = new CompactObjBuilder(this._config);
+  // Reset EVP for the new document:
+  builder.entityParser = this._entityVP.getInstance();
+  return builder;
+}
+```
+
+### `addInputEntities(entities)` — called automatically
+
+Receives the DOCTYPE entity map from `BaseOutputBuilder` once per parse. Resets per-document expansion counters. Accepts both plain string values and `{ regx, val }` objects from `DocTypeReader`.
+
+### `parse(val, context?)`
+
+Implements the `ValueParser` interface. `context` is accepted but ignored. Returns non-string input unchanged.
+
+---
+
+## Custom Entity Tables
+
+Pass any plain object as `default` or `system` to replace the built-in set:
+
+```js
+const myEntities = {
+  br:  { regex: /&br;/g,  val: '\n' },
+  tab: { regex: /&tab;/g, val: '\t' },
+};
+
+const replacer = new EntityReplacer({ default: myEntities });
+replacer.replace('line1&br;line2&tab;indented');
+// → 'line1\nline2\tindented'
+```
+
+Extend the built-in tables via spreading:
+
+```js
+import { DEFAULT_XML_ENTITIES } from '@nodable/entities';
+
+const replacer = new EntityReplacer({
+  default: { ...DEFAULT_XML_ENTITIES, br: { regex: /&br;/g, val: '\n' } },
+});
+```
+
+---
+
+## Comparison with `entities` npm package
+
+| Feature                                        | `entities` pkg    | `@nodable/entities` |
+|------------------------------------------------|-------------------|---------------------|
+| XML entity decoding                            | ✅                | ✅                  |
+| HTML entity decoding                           | ✅ full ~2000      | ✅ grouped, composable |
+| Numeric refs with leading zeros                | ✅                | ✅                  |
+| DOCTYPE / external entity injection            | ❌                | ✅                  |
+| Persistent vs. input entity separation         | ❌                | ✅                  |
+| Per-document reset via `getInstance()`         | ❌                | ✅                  |
+| Expansion count limit                          | ❌                | ✅                  |
+| Expanded length limit                          | ❌                | ✅                  |
+| `applyLimitsTo` granularity                    | ❌                | ✅                  |
+| `postCheck` hook                               | ❌                | ✅                  |
+| Encoding / HTML escaping                       | ✅                | ❌ out of scope     |
+| Zero dependencies                              | ✅                | ✅                  |
+
+---
+
+## TypeScript
+
+Full TypeScript declarations are included via `index.d.ts`. No `@types/` package needed.
+
+```ts
+import EntityReplacer, {
+  EntitiesValueParser,
+  COMMON_HTML,
+  EntityTable,
+  EntityReplacerOptions,
+  EntitiesValueParserOptions,
+} from '@nodable/entities';
+
+// EntityReplacer
+const opts: EntityReplacerOptions = {
+  default: true,
+  system: COMMON_HTML,
+  maxTotalExpansions: 500,
+  postCheck: (resolved, original) =>
+    /<script/i.test(resolved) ? original : resolved,
+};
+const replacer = new EntityReplacer(opts);
+replacer.setExternalEntities({ brand: 'Acme' });
+replacer.getInstance(); // reset for new document
+replacer.addInputEntities({ version: '1.0' }); // from DOCTYPE
+
+// EntitiesValueParser
+const evpOpts: EntitiesValueParserOptions = {
+  system: COMMON_HTML,
+  entities: { brand: 'Acme' },
+};
+const evp = new EntitiesValueParser(evpOpts);
+evp.addEntity('copy', '©');
+evp.getInstance();                            // called by builder factory
+evp.addInputEntities({ company: 'Nodable' }); // called by BaseOutputBuilder
+const result: string = evp.parse('&lt;&copy;&brand;');
+```
+
+
+
+## License
+
+MIT

package/package.json

@@ -0,0 +1,49 @@
+{
+  "name": "@nodable/entities",
+  "version": "1.0.0",
+  "description": "Replace XML, HTML, External entites with security controls",
+  "main": "./src/index.js",
+  "type": "module",
+  "sideEffects": false,
+  "types": "./src/index.d.ts",
+  "scripts": {
+    "test": "node --experimental-vm-modules node_modules/.bin/jest",
+    "test:watch": "node --experimental-vm-modules node_modules/.bin/jest --watch",
+    "test:coverage": "node --experimental-vm-modules node_modules/.bin/jest --coverage",
+    "lint": "eslint src/ test/"
+  },
+  "files": [
+    "src",
+    "README.md"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/nodable/val-parsers.git"
+  },
+  "keywords": [
+    "fast",
+    "xml",
+    "html",
+    "entity"
+  ],
+  "author": "Amit Gupta (https://solothought.com)",
+  "license": "MIT",
+  "publishConfig": {
+    "access": "public"
+  },
+  "devDependencies": {
+    "jest": "^29.7.0"
+  },
+  "funding": [
+    {
+      "type": "github",
+      "url": "https://github.com/sponsors/nodable"
+    }
+  ],
+  "jest": {
+    "testMatch": [
+      "**/?(*.)+(spec|test).[jt]s?(x)",
+      "**/*_spec.[jt]s?(x)"
+    ]
+  }
+}

package/src/EntitiesValueParser.js

@@ -0,0 +1,152 @@
+import EntityReplacer from './EntityReplacer.js';
+
+/**
+ * EntitiesValueParser — value-parser adapter that wraps `EntityReplacer`.
+ *
+ * Register an instance under the key `'entity'` on a `@nodable/flexible-xml-parser`
+ * output builder factory to enable entity expansion for all parsed text values.
+ *
+ * ## Lifecycle
+ *
+ * 1. **Construction** — supply configuration and optional persistent entities.
+ * 2. **`setExternalEntities(map)`** — (re)set the full persistent entity map.
+ *    Or use `addEntity(key, value)` to add one at a time.
+ * 3. **`getInstance()`** — builder factory calls this when creating a new builder
+ *    instance. Resets input entities and per-document counters. Returns `this`.
+ * 4. **`addInputEntities(map)`** — builder calls this if the document has a
+ *    DOCTYPE block. Stores entities for *this document only*.
+ * 5. **`parse(val)`** — called by the builder for each text value.
+ *
+ * ```js
+ * const evp = new EntitiesValueParser({ system: COMMON_HTML });
+ * evp.setExternalEntities({ brand: 'Acme' });
+ * builder.registerValueParser('entity', evp);
+ * ```
+ *
+ * -------------------------------------------------------------------------
+ * Constructor options (all optional)
+ * -------------------------------------------------------------------------
+ *
+ * `default`             — `true` (default) | `false`/`null` | custom EntityTable
+ * `system`              — `false` (default) | `true` for COMMON_HTML | EntityTable
+ * `amp`                 — `true` (default) | `false`/`null`
+ * `maxTotalExpansions`  — max entity refs expanded per document (0 = unlimited)
+ * `maxExpandedLength`   — max characters added by expansion per document (0 = unlimited)
+ * `applyLimitsTo`       — which categories count toward limits (default: `'external'`)
+ * `postCheck`           — `(resolved, original) => string` hook
+ * `entities`            — initial persistent entity map, e.g. `{ copy: '©' }`
+ */
+export default class EntitiesValueParser {
+  constructor(options = {}) {
+    this._replacer = new EntityReplacer(options);
+
+    // Load any entities provided inline at construction time as persistent entities
+    if (options.entities && typeof options.entities === 'object') {
+      const init = {};
+      for (const [key, val] of Object.entries(options.entities)) {
+        this._validateEntityArgs(key, val);
+        init[key] = val;
+      }
+      this._replacer.setExternalEntities(init);
+    }
+  }
+
+  // -------------------------------------------------------------------------
+  // Persistent external entity registration
+  // -------------------------------------------------------------------------
+
+  /**
+   * Replace the full set of persistent external entities.
+   * These survive across documents and are never wiped by `getInstance()`.
+   *
+   * @param {Record<string, string>} map  — e.g. `{ copy: '©', brand: 'Acme' }`
+   */
+  setExternalEntities(map) {
+    for (const [key, val] of Object.entries(map)) {
+      this._validateEntityArgs(key, val);
+    }
+    this._replacer.setExternalEntities(map);
+  }
+
+  /**
+   * Add (or replace) a single persistent external entity.
+   * Existing persistent entities are preserved.
+   *
+   * @param {string} key   — bare name without `&` / `;`, e.g. `'copy'`
+   * @param {string} value — replacement string, e.g. `'©'`
+   */
+  addEntity(key, value) {
+    this._validateEntityArgs(key, value);
+    this._replacer.addExternalEntity(key, value);
+  }
+
+  // -------------------------------------------------------------------------
+  // Builder factory integration
+  // -------------------------------------------------------------------------
+
+  /**
+   * Reset per-document state (input entities + expansion counters) and return `this`.
+   *
+   * The builder factory calls this when creating a new builder instance so that
+   * DOCTYPE entities from a previous document are never carried over.
+   *
+   * @returns {EntitiesValueParser} `this`
+   */
+  getInstance() {
+    this._replacer.getInstance();
+    return this;
+  }
+
+  // -------------------------------------------------------------------------
+  // DOCTYPE integration — called by BaseOutputBuilder
+  // -------------------------------------------------------------------------
+
+  /**
+   * Receive DOCTYPE entities from the output builder.
+   *
+   * These are stored separately from persistent entities and wiped on the next
+   * `getInstance()` call. Resets per-document expansion counters.
+   *
+   * @param {Record<string, string | { regx: RegExp, val: string | Function }>} entities
+   *   Raw entity map from `DocTypeReader` — values may be plain strings or
+   *   `{ regx, val }` objects (note: `regx`, not `regex`, matching the reader's output).
+   */
+  addInputEntities(entities) {
+    this._replacer.addInputEntities(entities);
+  }
+
+  // -------------------------------------------------------------------------
+  // ValueParser interface
+  // -------------------------------------------------------------------------
+
+  /**
+   * Replace entity references in `val`.
+   *
+   * @param {string} val
+   * @param {object} [_context]
+   * @returns {string}
+   */
+  parse(val, _context) {
+    if (typeof val !== 'string') return val;
+    return this._replacer.replace(val);
+  }
+
+  // -------------------------------------------------------------------------
+  // Private helpers
+  // -------------------------------------------------------------------------
+
+  _validateEntityArgs(key, value) {
+    if (typeof key !== 'string' || key.includes('&') || key.includes(';')) {
+      throw new Error(
+        `[EntitiesValueParser] Entity key must not contain '&' or ';'. ` +
+        `Use 'copy' for '&copy;', got: ${JSON.stringify(key)}`
+      );
+    }
+    if (typeof value !== 'string' || value.includes('&')) {
+      throw new Error(
+        `[EntitiesValueParser] Entity value must be a plain string that does not ` +
+        `contain '&', got: ${JSON.stringify(value)}`
+      );
+    }
+  }
+}