@structured-world/structured-public-domains 0.0.6 → 0.0.8

package/README.md

@@ -0,0 +1,146 @@
+# structured-public-domains
+
+Compact Public Suffix List (PSL) for Rust.
+
+[![CI](https://github.com/structured-world/structured-public-domains/actions/workflows/ci.yml/badge.svg)](https://github.com/structured-world/structured-public-domains/actions/workflows/ci.yml)
+[![Crates.io](https://img.shields.io/crates/v/structured-public-domains.svg)](https://crates.io/crates/structured-public-domains)
+[![npm](https://img.shields.io/npm/v/@structured-world/structured-public-domains.svg)](https://www.npmjs.com/package/@structured-world/structured-public-domains)
+[![docs.rs](https://docs.rs/structured-public-domains/badge.svg)](https://docs.rs/structured-public-domains)
+[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
+
+- **Zero** runtime dependencies
+- **~108KB** embedded data (compact binary trie)
+- **~2.4M lookups/sec** on a single core (~420 ns per lookup)
+- **O(depth * log k)** trie traversal with per-node binary search (typically 2-3 steps)
+- Wildcard (`*.jp`) and exception (`!metro.tokyo.jp`) rules
+- Based on the official Public Suffix List (ICANN and private sections)
+- Checked daily against [publicsuffix.org](https://publicsuffix.org/)
+
+**Terminology:** A *public suffix* (e.g., `com`, `co.uk`) is the part of a domain under which users can register names. The *registrable domain* (eTLD+1) is one label above the suffix (e.g., `example.co.uk`).
+
+## Usage (Rust)
+
+```rust
+use structured_public_domains::{lookup, registrable_domain, is_known_suffix};
+
+let info = lookup("www.example.co.uk").unwrap();
+assert_eq!(info.suffix(), "co.uk");
+assert_eq!(info.registrable_domain(), Some("example.co.uk"));
+assert!(info.is_known());
+
+// Helpers
+assert_eq!(registrable_domain("sub.example.com"), Some("example.com".to_string()));
+assert!(is_known_suffix("example.com"));
+```
+
+## Usage (JavaScript / TypeScript)
+
+The same PSL trie ships as a native TypeScript npm package — no WebAssembly, no
+runtime dependencies. The ~108KB binary trie is embedded and decoded lazily on
+first call, so every function is **synchronous** with no `init()`: it drops
+straight into Node, browsers, bundlers, and downstream libraries. Ships both
+ESM and CommonJS with type declarations for each.
+
+```sh
+npm install @structured-world/structured-public-domains
+```
+
+```typescript
+// ESM
+import { lookup, registrableDomain, isKnownSuffix } from '@structured-world/structured-public-domains';
+
+const info = lookup('www.example.co.uk');
+// info.suffix            → "co.uk"
+// info.registrableDomain → "example.co.uk"
+// info.known             → true
+
+registrableDomain('sub.example.com');  // "example.com"
+isKnownSuffix('example.com');          // true
+```
+
+```javascript
+// CommonJS (e.g. default NestJS)
+const { lookup, registrableDomain, isKnownSuffix } = require('@structured-world/structured-public-domains');
+```
+
+### Raw trie data
+
+The embedded binary trie is exposed for consumers that want to walk it
+themselves (the format matches the Rust crate's `src/psl.bin`):
+
+```typescript
+import { pslData } from '@structured-world/structured-public-domains';
+
+const bytes = pslData();   // Uint8Array — a defensive copy of the trie blob
+```
+
+The JS lookup is verified byte-for-byte against the Rust implementation over the
+entire PSL on every CI run, so both languages return identical results.
+
+### Tiny build (runtime-fetched, no embedded data)
+
+For consumers who want always-fresh PSL **without bumping the package version**,
+the `/tiny` entry ships *without* the embedded blob. It fetches the prebuilt
+binary trie at runtime and caches it locally (Node: temp file with a TTL;
+browser: CacheStorage). After the first `await load()`, the lookup API is
+identical and synchronous.
+
+```typescript
+import { load, registrableDomain } from '@structured-world/structured-public-domains/tiny';
+
+await load();                              // fetch + cache once (default: jsDelivr CDN)
+registrableDomain('sub.example.co.uk');    // "example.co.uk"
+
+// Options: custom source, TTL, cache dir, or force refresh.
+await load({ url: 'https://psl.example.com/psl.bin', cacheTtlMs: 3_600_000, force: true });
+```
+
+The default source is the same `psl.bin` served from this package's jsDelivr CDN,
+pinned to the installed `major.minor` range — so it always tracks the latest
+PSL-data patch release (same trie format) but never a future format-breaking
+version the bundled parser can't read. Results are identical to the embedded
+build. Use the full `.` entry when you want zero network and instant startup; use
+`/tiny` when install size and
+always-current data matter more.
+
+## Performance
+
+Benchmarks on Apple M-series (criterion, `cargo bench`):
+
+| Benchmark | Time | Throughput |
+|-----------|------|-----------|
+| Simple (`example.com`) | ~420 ns | ~2.4M/s |
+| Nested (`www.example.co.uk`) | ~425 ns | ~2.4M/s |
+| Deep subdomain (`a.b.c.d.example.com`) | ~500 ns | ~2.0M/s |
+| Bare TLD (`com`) | ~195 ns | ~5.1M/s |
+| Private domain (`mysite.github.io`) | ~450 ns | ~2.2M/s |
+| Long chain (`very.deep...co.uk`) | ~500 ns | ~2.0M/s |
+
+**Runtime memory:** The PSL trie is parsed lazily on first call (`OnceLock`), then cached for the lifetime of the process. Runtime footprint is ~530 KB (sorted `Vec` children with binary search lookup). The ~108KB binary blob is embedded in the binary at compile time.
+
+## Why not `psl`?
+
+| | `psl` | `structured-public-domains` |
+|---|---|---|
+| Embedded data | ~876KB (codegen match tree) | **108KB** (compact binary trie) |
+| Source size | 2.4MB codegen | 300 lines + 108KB blob |
+| Runtime deps | None | **None** |
+| Runtime memory | N/A (static) | **~530KB** |
+| Lookup | O(depth) match tree | O(depth * log k) trie walk |
+| Auto-update | New crate version | Daily GitHub Actions check |
+
+Both crates have comparable lookup speed and zero runtime dependencies. `structured-public-domains` has ~8x smaller embedded data and auto-updates daily via GitHub Actions with domain-level changelogs.
+
+## Support the Project
+
+<div align="center">
+
+![USDT TRC-20 Donation QR Code](assets/usdt-qr.svg)
+
+USDT (TRC-20): `TFDsezHa1cBkoeZT5q2T49Wp66K8t2DmdA`
+
+</div>
+
+## License
+
+Apache License 2.0

package/dist/chunk-EON4VJGA.js

@@ -0,0 +1,121 @@
+// src/trie.ts
+var utf8 = new TextDecoder("utf-8", { fatal: true });
+function parseTrie(data) {
+  const cursor = { pos: 0 };
+  const root = parseNode(data, cursor);
+  if (cursor.pos !== data.length) {
+    throw new Error("PSL data: trailing bytes after root node");
+  }
+  return root;
+}
+function parseNode(data, cursor) {
+  const flags = byteAt(data, cursor.pos++);
+  if ((flags & -2) !== 0) {
+    throw new Error("PSL data: reserved flag bits set");
+  }
+  const lo = byteAt(data, cursor.pos++);
+  const hi = byteAt(data, cursor.pos++);
+  let numChildren = lo | hi << 8;
+  const MIN_CHILD_ENCODED_LEN = 5;
+  const remaining = data.length - cursor.pos;
+  if (remaining < 0 || numChildren > Math.floor(remaining / MIN_CHILD_ENCODED_LEN)) {
+    throw new Error("PSL data: num_children exceeds remaining bytes");
+  }
+  const labels = [];
+  const children = [];
+  let prev;
+  for (; numChildren > 0; numChildren--) {
+    const labelLen = byteAt(data, cursor.pos++);
+    if (labelLen === 0) {
+      throw new Error("PSL data: empty label");
+    }
+    const labelEnd = cursor.pos + labelLen;
+    if (labelEnd > data.length) {
+      throw new Error("PSL data: label runs past end of data");
+    }
+    const label = decodeLabel(data, cursor.pos, labelEnd);
+    cursor.pos = labelEnd;
+    if (prev !== void 0 && !(label > prev)) {
+      throw new Error("PSL data: children not strictly sorted");
+    }
+    prev = label;
+    labels.push(label);
+    children.push(parseNode(data, cursor));
+  }
+  return { suffixBoundary: (flags & 1) !== 0, labels, children };
+}
+function byteAt(data, i) {
+  if (i >= data.length) {
+    throw new Error("PSL data: unexpected end of data");
+  }
+  return data[i];
+}
+function decodeLabel(data, start, end) {
+  for (let i = start; i < end; i++) {
+    if (data[i] >= 128) return utf8.decode(data.subarray(start, end));
+  }
+  let s = "";
+  for (let i = start; i < end; i++) s += String.fromCharCode(data[i]);
+  return s;
+}
+function indexOfChild(node, label) {
+  let lo = 0;
+  let hi = node.labels.length - 1;
+  while (lo <= hi) {
+    const mid = lo + hi >>> 1;
+    const cur = node.labels[mid];
+    if (cur === label) return mid;
+    if (cur < label) lo = mid + 1;
+    else hi = mid - 1;
+  }
+  return -1;
+}
+function childOf(node, label) {
+  const i = indexOfChild(node, label);
+  return i < 0 ? void 0 : node.children[i];
+}
+function hasChild(node, label) {
+  return indexOfChild(node, label) >= 0;
+}
+function lookupTrie(root, domain) {
+  const trimmed = domain.trim();
+  const stripped = trimmed.endsWith(".") ? trimmed.slice(0, -1) : trimmed;
+  if (stripped === "") return void 0;
+  const labels = stripped.split(".").reverse();
+  for (const label of labels) {
+    if (label === "" || label === "*" || label.startsWith("!")) return void 0;
+  }
+  let node = root;
+  let suffixDepth = 0;
+  let known = false;
+  for (let depth = 0; depth < labels.length; depth++) {
+    const label = labels[depth].toLowerCase();
+    if (hasChild(node, "*")) {
+      if (hasChild(node, "!" + label)) {
+        suffixDepth = depth;
+      } else {
+        suffixDepth = depth + 1;
+      }
+      known = true;
+    }
+    const child = childOf(node, label);
+    if (child !== void 0) {
+      if (child.suffixBoundary) {
+        suffixDepth = depth + 1;
+        known = true;
+      }
+      node = child;
+      continue;
+    }
+    break;
+  }
+  if (suffixDepth === 0) {
+    suffixDepth = 1;
+    known = false;
+  }
+  const suffix = labels.slice(0, suffixDepth).reverse().map((l) => l.toLowerCase()).join(".");
+  const registrableDomain = labels.length > suffixDepth ? `${labels[suffixDepth].toLowerCase()}.${suffix}` : void 0;
+  return { suffix, registrableDomain, known };
+}
+
+export { lookupTrie, parseTrie };

package/dist/index.cjs

@@ -0,0 +1,161 @@
+'use strict';
+
+var pslData_cjs = require('./psl-data.cjs');
+
+// src/index.ts
+
+// src/trie.ts
+var utf8 = new TextDecoder("utf-8", { fatal: true });
+function parseTrie(data) {
+  const cursor = { pos: 0 };
+  const root = parseNode(data, cursor);
+  if (cursor.pos !== data.length) {
+    throw new Error("PSL data: trailing bytes after root node");
+  }
+  return root;
+}
+function parseNode(data, cursor) {
+  const flags = byteAt(data, cursor.pos++);
+  if ((flags & -2) !== 0) {
+    throw new Error("PSL data: reserved flag bits set");
+  }
+  const lo = byteAt(data, cursor.pos++);
+  const hi = byteAt(data, cursor.pos++);
+  let numChildren = lo | hi << 8;
+  const MIN_CHILD_ENCODED_LEN = 5;
+  const remaining = data.length - cursor.pos;
+  if (remaining < 0 || numChildren > Math.floor(remaining / MIN_CHILD_ENCODED_LEN)) {
+    throw new Error("PSL data: num_children exceeds remaining bytes");
+  }
+  const labels = [];
+  const children = [];
+  let prev;
+  for (; numChildren > 0; numChildren--) {
+    const labelLen = byteAt(data, cursor.pos++);
+    if (labelLen === 0) {
+      throw new Error("PSL data: empty label");
+    }
+    const labelEnd = cursor.pos + labelLen;
+    if (labelEnd > data.length) {
+      throw new Error("PSL data: label runs past end of data");
+    }
+    const label = decodeLabel(data, cursor.pos, labelEnd);
+    cursor.pos = labelEnd;
+    if (prev !== void 0 && !(label > prev)) {
+      throw new Error("PSL data: children not strictly sorted");
+    }
+    prev = label;
+    labels.push(label);
+    children.push(parseNode(data, cursor));
+  }
+  return { suffixBoundary: (flags & 1) !== 0, labels, children };
+}
+function byteAt(data, i) {
+  if (i >= data.length) {
+    throw new Error("PSL data: unexpected end of data");
+  }
+  return data[i];
+}
+function decodeLabel(data, start, end) {
+  for (let i = start; i < end; i++) {
+    if (data[i] >= 128) return utf8.decode(data.subarray(start, end));
+  }
+  let s = "";
+  for (let i = start; i < end; i++) s += String.fromCharCode(data[i]);
+  return s;
+}
+function indexOfChild(node, label) {
+  let lo = 0;
+  let hi = node.labels.length - 1;
+  while (lo <= hi) {
+    const mid = lo + hi >>> 1;
+    const cur = node.labels[mid];
+    if (cur === label) return mid;
+    if (cur < label) lo = mid + 1;
+    else hi = mid - 1;
+  }
+  return -1;
+}
+function childOf(node, label) {
+  const i = indexOfChild(node, label);
+  return i < 0 ? void 0 : node.children[i];
+}
+function hasChild(node, label) {
+  return indexOfChild(node, label) >= 0;
+}
+function lookupTrie(root, domain) {
+  const trimmed = domain.trim();
+  const stripped = trimmed.endsWith(".") ? trimmed.slice(0, -1) : trimmed;
+  if (stripped === "") return void 0;
+  const labels = stripped.split(".").reverse();
+  for (const label of labels) {
+    if (label === "" || label === "*" || label.startsWith("!")) return void 0;
+  }
+  let node = root;
+  let suffixDepth = 0;
+  let known = false;
+  for (let depth = 0; depth < labels.length; depth++) {
+    const label = labels[depth].toLowerCase();
+    if (hasChild(node, "*")) {
+      if (hasChild(node, "!" + label)) {
+        suffixDepth = depth;
+      } else {
+        suffixDepth = depth + 1;
+      }
+      known = true;
+    }
+    const child = childOf(node, label);
+    if (child !== void 0) {
+      if (child.suffixBoundary) {
+        suffixDepth = depth + 1;
+        known = true;
+      }
+      node = child;
+      continue;
+    }
+    break;
+  }
+  if (suffixDepth === 0) {
+    suffixDepth = 1;
+    known = false;
+  }
+  const suffix = labels.slice(0, suffixDepth).reverse().map((l) => l.toLowerCase()).join(".");
+  const registrableDomain2 = labels.length > suffixDepth ? `${labels[suffixDepth].toLowerCase()}.${suffix}` : void 0;
+  return { suffix, registrableDomain: registrableDomain2, known };
+}
+
+// src/index.ts
+var cachedBytes;
+var cachedTrie;
+function decodeBase64(b64) {
+  if (typeof Buffer !== "undefined") {
+    return new Uint8Array(Buffer.from(b64, "base64"));
+  }
+  const binary = atob(b64);
+  const out = new Uint8Array(binary.length);
+  for (let i = 0; i < binary.length; i++) out[i] = binary.charCodeAt(i);
+  return out;
+}
+function bytes() {
+  return cachedBytes ??= decodeBase64(pslData_cjs.PSL_BASE64);
+}
+function trie() {
+  return cachedTrie ??= parseTrie(bytes());
+}
+function lookup(domain) {
+  return lookupTrie(trie(), domain);
+}
+function registrableDomain(domain) {
+  return lookup(domain)?.registrableDomain;
+}
+function isKnownSuffix(domain) {
+  return lookup(domain)?.known ?? false;
+}
+function pslData() {
+  return bytes().slice();
+}
+
+exports.isKnownSuffix = isKnownSuffix;
+exports.lookup = lookup;
+exports.pslData = pslData;
+exports.registrableDomain = registrableDomain;

package/dist/index.d.cts

@@ -0,0 +1,36 @@
+import { D as DomainInfo } from './trie-jGeN4GI6.cjs';
+
+/**
+ * Look up a domain in the Public Suffix List.
+ *
+ * Returns `undefined` for empty or invalid input (empty labels, or the PSL
+ * sentinel labels `*` / `!prefix`).
+ *
+ * @example
+ * ```ts
+ * const info = lookup("www.example.co.uk");
+ * // info.suffix            → "co.uk"
+ * // info.registrableDomain → "example.co.uk"
+ * // info.known             → true
+ * ```
+ */
+declare function lookup(domain: string): DomainInfo | undefined;
+/**
+ * Extract the registrable domain (eTLD+1).
+ *
+ * Returns `undefined` if the domain is itself a public suffix, or the input is
+ * invalid.
+ */
+declare function registrableDomain(domain: string): string | undefined;
+/** Check whether a domain's suffix is a known (explicit) entry in the PSL. */
+declare function isKnownSuffix(domain: string): boolean;
+/**
+ * The raw compact binary PSL trie (DFS preorder) embedded in this package.
+ *
+ * Returns a defensive copy so callers can hand the blob to their own parser
+ * without risking the cached singleton. The format matches the Rust crate's
+ * `src/psl.bin`; see `scripts/build-psl.py` for the layout.
+ */
+declare function pslData(): Uint8Array;
+
+export { DomainInfo, isKnownSuffix, lookup, pslData, registrableDomain };

package/dist/index.d.ts

@@ -0,0 +1,36 @@
+import { D as DomainInfo } from './trie-jGeN4GI6.js';
+
+/**
+ * Look up a domain in the Public Suffix List.
+ *
+ * Returns `undefined` for empty or invalid input (empty labels, or the PSL
+ * sentinel labels `*` / `!prefix`).
+ *
+ * @example
+ * ```ts
+ * const info = lookup("www.example.co.uk");
+ * // info.suffix            → "co.uk"
+ * // info.registrableDomain → "example.co.uk"
+ * // info.known             → true
+ * ```
+ */
+declare function lookup(domain: string): DomainInfo | undefined;
+/**
+ * Extract the registrable domain (eTLD+1).
+ *
+ * Returns `undefined` if the domain is itself a public suffix, or the input is
+ * invalid.
+ */
+declare function registrableDomain(domain: string): string | undefined;
+/** Check whether a domain's suffix is a known (explicit) entry in the PSL. */
+declare function isKnownSuffix(domain: string): boolean;
+/**
+ * The raw compact binary PSL trie (DFS preorder) embedded in this package.
+ *
+ * Returns a defensive copy so callers can hand the blob to their own parser
+ * without risking the cached singleton. The format matches the Rust crate's
+ * `src/psl.bin`; see `scripts/build-psl.py` for the layout.
+ */
+declare function pslData(): Uint8Array;
+
+export { DomainInfo, isKnownSuffix, lookup, pslData, registrableDomain };

package/dist/index.js

@@ -0,0 +1,34 @@
+import { lookupTrie, parseTrie } from './chunk-EON4VJGA.js';
+import { PSL_BASE64 } from './psl-data.cjs';
+
+var cachedBytes;
+var cachedTrie;
+function decodeBase64(b64) {
+  if (typeof Buffer !== "undefined") {
+    return new Uint8Array(Buffer.from(b64, "base64"));
+  }
+  const binary = atob(b64);
+  const out = new Uint8Array(binary.length);
+  for (let i = 0; i < binary.length; i++) out[i] = binary.charCodeAt(i);
+  return out;
+}
+function bytes() {
+  return cachedBytes ??= decodeBase64(PSL_BASE64);
+}
+function trie() {
+  return cachedTrie ??= parseTrie(bytes());
+}
+function lookup(domain) {
+  return lookupTrie(trie(), domain);
+}
+function registrableDomain(domain) {
+  return lookup(domain)?.registrableDomain;
+}
+function isKnownSuffix(domain) {
+  return lookup(domain)?.known ?? false;
+}
+function pslData() {
+  return bytes().slice();
+}
+
+export { isKnownSuffix, lookup, pslData, registrableDomain };