grapheme-conformance 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/VERIFY.md +14 -7
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -16,7 +16,7 @@ It does not ship another segmenter.
|
|
|
16
16
|
|
|
17
17
|
## The finding
|
|
18
18
|
|
|
19
|
-
`क्षि`
|
|
19
|
+
`क्षि` is Devanagari: four code points, one character a reader sees. Split it
|
|
20
20
|
with a popular library and you get two clusters. `grapheme-splitter` and
|
|
21
21
|
`graphemer` both break it; `runes2` breaks it into three.
|
|
22
22
|
|
|
@@ -171,7 +171,7 @@ See [VERIFY.md](./VERIFY.md): one command, ten integers to compare.
|
|
|
171
171
|
|
|
172
172
|
## Scope
|
|
173
173
|
|
|
174
|
-
Grapheme clusters only. Word and sentence breaking are out of scope
|
|
174
|
+
Grapheme clusters only. Word and sentence breaking are out of scope,
|
|
175
175
|
`unicode-segmenter` does not implement them, which leaves too few
|
|
176
176
|
implementations to be worth a scoreboard.
|
|
177
177
|
|
package/VERIFY.md
CHANGED
|
@@ -50,18 +50,25 @@ Expected output:
|
|
|
50
50
|
|
|
51
51
|
`Intl.Segmenter` is not a fixed library. It is scored against whatever ICU the
|
|
52
52
|
host Node ships, so its row moves with the runtime. Measured across the CI
|
|
53
|
-
matrix:
|
|
53
|
+
matrix, against these two older keys:
|
|
54
54
|
|
|
55
55
|
| runtime | 15.1.0 | 16.0.0 | `2701 200D 2701` |
|
|
56
56
|
|---|---|---|---|
|
|
57
|
-
| Node 22.22.2 (ICU 78.2) | 1186 | 1092 |
|
|
58
|
-
| Node 20.x | 1186 | 1092 |
|
|
59
|
-
| Node 18.20.8 | 1187 | 1093 |
|
|
57
|
+
| Node 22.22.2 (ICU 78.2) | 1186 | 1092 | two clusters |
|
|
58
|
+
| Node 20.x | 1186 | 1092 | two clusters |
|
|
59
|
+
| Node 18.20.8 | 1187 | 1093 | one cluster |
|
|
60
60
|
|
|
61
61
|
Run the command above on Node 22 or 20 to get the expected output above. On
|
|
62
|
-
Node 18 the `Intl.Segmenter` value reads `1187` and `1093` instead
|
|
63
|
-
|
|
64
|
-
|
|
62
|
+
Node 18 the `Intl.Segmenter` value reads `1187` and `1093` instead. The other
|
|
63
|
+
four values are pinned to exact library versions and hold identically on
|
|
64
|
+
every Node.
|
|
65
|
+
|
|
66
|
+
That single differing case is not an ICU bug. Unicode 17.0 removed U+2701
|
|
67
|
+
from `Extended_Pictographic`, so GB11 stopped applying and two clusters became
|
|
68
|
+
the correct answer; newer ICU implements that change and these two older keys
|
|
69
|
+
predate it. Scored against `17.0.0`, which is what the CLI uses by default,
|
|
70
|
+
ICU on Node 22 passes all 766 cases. Score an implementation against the key
|
|
71
|
+
it targets.
|
|
65
72
|
|
|
66
73
|
The baseline was verified on Node 22.22.2 (ICU 78.2, Unicode 17.0).
|
|
67
74
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "grapheme-conformance",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.6",
|
|
4
4
|
"description": "Score any JavaScript grapheme segmenter against Unicode's official GraphemeBreakTest.txt. Find out which string splitters are wrong, and about what.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"unicode",
|