grapheme-conformance 0.1.2 → 0.1.3

package/README.md

@@ -2,6 +2,11 @@
 
 **Your string splitter is probably wrong about Hindi.**
 
+[![build](https://github.com/tamerkalla/grapheme-conformance/actions/workflows/release.yml/badge.svg)](https://github.com/tamerkalla/grapheme-conformance/actions/workflows/release.yml)
+[![npm](https://img.shields.io/npm/v/grapheme-conformance.svg)](https://www.npmjs.com/package/grapheme-conformance)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![provenance](https://img.shields.io/badge/provenance-attested-brightgreen.svg)](https://www.npmjs.com/package/grapheme-conformance)
+
 Unicode publishes `GraphemeBreakTest.txt`: a machine-readable answer key that
 says exactly where a string may be split into user-perceived characters. This
 package scores any JavaScript grapheme segmenter against it, tells you which

package/VERIFY.md

@@ -1,29 +1,50 @@
 # VERIFY
 
-Run one command:
+This reproduces the scoreboard's two Unicode-16.0.0-and-15.1.0 columns from the
+published package, in a clean directory. It does not require this repository
+to be checked out.
 
-```
-npm run scoreboard
-```
+```bash
+mkdir -p grapheme-conformance-verify && cd grapheme-conformance-verify
+npm init -y >/dev/null 2>&1
+npm install grapheme-conformance@latest grapheme-splitter@1.0.4 graphemer@1.4.0 runes2@1.1.4 unicode-segmenter@0.17.3 >/dev/null 2>&1
+cat > verify.mjs <<'JS'
+import { score, vectors } from 'grapheme-conformance';
+import { splitGraphemes } from 'unicode-segmenter/grapheme';
+import graphemerModule from 'graphemer';
+import GraphemeSplitter from 'grapheme-splitter';
+import { runes } from 'runes2';
 
-It builds the package, scores every library against every vendored copy of
-Unicode's `GraphemeBreakTest.txt`, writes `SCOREBOARD.md`, and prints the table
-below to stdout.
+const Graphemer = graphemerModule.default ?? graphemerModule;
+const intl = new Intl.Segmenter('en', { granularity: 'grapheme' });
+const graphemer = new Graphemer();
+const splitter = new GraphemeSplitter();
 
-## Expected output
+const impls = {
+  'Intl.Segmenter': (s) => [...intl.segment(s)].map((p) => p.segment),
+  'unicode-segmenter': (s) => [...splitGraphemes(s)],
+  graphemer: (s) => graphemer.splitGraphemes(s),
+  'grapheme-splitter': (s) => splitter.splitGraphemes(s),
+  runes2: (s) => runes(s),
+};
 
-Compare ten integers. These two columns are the specified baseline.
+for (const version of ['15.1.0', '16.0.0']) {
+  const vecs = vectors[version];
+  const row = Object.entries(impls)
+    .map(([name, fn]) => `${name}=${score(fn, vecs).passed}`)
+    .join(' ');
+  console.log(`${version}: ${row}`);
+}
+JS
+node verify.mjs
+```
 
-| implementation | U15.1.0 (1187) | U16.0.0 (1093) |
-|---|---|---|
-| `Intl.Segmenter` | 1186 | 1092 |
-| `unicode-segmenter` | 1186 | 1092 |
-| `graphemer` | 1180 | 1086 |
-| `grapheme-splitter` | 1175 | 1081 |
-| `runes2` | 730 | 695 |
+Expected output:
 
-The printed table shows each cell as `passed/total (rate%)`; the integer to
-check is the one before the slash. If all ten match, the build is correct.
+```text
+15.1.0: Intl.Segmenter=1186 unicode-segmenter=1186 graphemer=1180 grapheme-splitter=1175 runes2=730
+16.0.0: Intl.Segmenter=1092 unicode-segmenter=1092 graphemer=1086 grapheme-splitter=1081 runes2=695
+```
 
 ## One caveat, and it is the only one
 
@@ -37,22 +58,29 @@ matrix:
 | Node 20.x | 1186 | 1092 | split |
 | Node 18.20.8 | 1187 | 1093 | not split |
 
-Run `npm run scoreboard` on Node 22 or 20 to get the two baseline numbers. On
-Node 18 the `Intl.Segmenter` row reads `1187` and `1093` instead: that ICU
-predates the deviation and passes every case. The other four rows are pinned to
-exact versions and hold identically on every Node.
+Run the command above on Node 22 or 20 to get the expected output above. On
+Node 18 the `Intl.Segmenter` value reads `1187` and `1093` instead: that ICU
+predates the deviation and passes every case. The other four values are
+pinned to exact library versions and hold identically on every Node.
 
 The baseline was verified on Node 22.22.2 (ICU 78.2, Unicode 17.0).
 
-## Everything else
+## Reproducing SCOREBOARD.md and the rest of the gates
 
-```
-npm test        # 104 assertions, offline, including all ten integers above
+The committed `SCOREBOARD.md` (all four vendored Unicode versions, not just
+the two above) is generated from this repository's own scripts, which are not
+part of the published package, so reproducing it requires a checkout:
+
+```bash
+git clone https://github.com/tamerkalla/grapheme-conformance.git && cd grapheme-conformance
+npm ci
+npm run scoreboard          # regenerates SCOREBOARD.md and prints it
+npm run scoreboard:check    # asserts it is byte-identical to a fresh run
 npm run typecheck
+npm test                    # includes all ten integers above
 npm run build
-npm run smoke   # loads the ESM and CJS entry points and scores Intl.Segmenter
-npm run scoreboard:check   # asserts SCOREBOARD.md is byte-identical to a fresh run
+node scripts/smoke.mjs      # loads the ESM and CJS entry points
 ```
 
-No network access is required by any of these. The vectors are committed under
-`vectors/` and are never fetched.
+No network access is required by any of these. The vectors are committed
+under `vectors/` and are never fetched.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "grapheme-conformance",
-  "version": "0.1.2",
+  "version": "0.1.3",
   "description": "Score any JavaScript grapheme segmenter against Unicode's official GraphemeBreakTest.txt. Find out which string splitters are wrong, and about what.",
   "keywords": [
     "unicode",
@@ -52,6 +52,7 @@
   "scripts": {
     "build": "tsup",
     "typecheck": "tsc --noEmit",
+    "pretest": "npm run build",
     "test": "vitest run",
     "prescoreboard": "npm run build",
     "scoreboard": "node scripts/scoreboard.mjs",
@@ -67,6 +68,7 @@
     "tsup": "8.3.5",
     "typescript": "5.7.2",
     "unicode-segmenter": "0.17.3",
-    "vitest": "2.1.8"
+    "vitest": "2.1.8",
+    "yaml": "2.9.0"
   }
 }