indickit 0.10.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,389 @@
1
+ # Changelog
2
+
3
+ Each version is one Git tag `v<version>`, one npm version, and one Go
4
+ module version. The **Rules** line of an entry names the rules version of
5
+ each utility. When a utility's rules version changed, some inputs give a
6
+ different output than before: recompute what you stored from the old
7
+ version, and compare `RULES_VERSION` (or `SEARCH_VERSION`) to know which
8
+ rows. A version whose rules line equals the one before it gives every
9
+ input the same output.
10
+
11
+ ## 0.12.0 (2026-10-12)
12
+
13
+ Rules: phonetic 2026-10-11.8, normalize 2026-10-12, segment 2026-10-11, stem 2026-10-12, romanize 2026-10-12, deromanize 2026-10-09.2, phonetic-search 2026-10-11.8
14
+
15
+ - Faster, same output: `deromanize` 2.0x in Go and 1.35x in TypeScript (a
16
+ Hindi word in Go: 11.0 -> 4.4 ms); `romanize` 1.5-2.0x in Go and 1.1x in
17
+ TypeScript (a Hindi sentence in Go: 31 -> 15 ms). The n-gram tables are
18
+ sorted arrays, not one map entry per pair, and the last letter builds only
19
+ the spellings that can be returned, best first.
20
+ - Faster, same output: `Search` 2.0x in Go and 2.4x in TypeScript, `NewIndex`
21
+ 1.6x and 1.8x. A word's keys are computed once per call, and a pair of
22
+ words is scored once.
23
+ - Less memory, faster loads: a loaded Hindi deromanizer keeps 18 MB of heap
24
+ in Go (was 35 MB) and loads in 36 ms (was 111 ms); a romanizer keeps 15 MB
25
+ (was 29 MB) and loads in 40 ms (was 96 ms). The Go word lists are kept as
26
+ bytes. (0.11.0 already made TypeScript's tables flat: a Hindi deromanizer
27
+ 130 -> 35 MB, a romanizer 194 -> 42 MB.) A test in each port fails if a
28
+ loaded language grows back past its bound.
29
+ - Changed output: phonetic 2026-10-11.7 → 2026-10-11.8, phonetic-search
30
+ follows. Recompute stored keys. The Perso-Arabic letter ھ (U+06BE) is read
31
+ as h: Sindhi writes its h with it (سنھا = Sinha, جوھي = Juhi), and the
32
+ reader skipped it. On Wikidata people (DEV) Sindhi rises from 78% to 83%,
33
+ Urdu, Kashmiri and Hindi by under a point, the wrong-match rate unchanged.
34
+ 2,251 of the 199,992 test words (those with ھ) gain a key; every key of
35
+ 0.11.0 is kept but for 20 of them, with ھ at the start or after a vowel,
36
+ where the h-less key goes (شاھ = Shah: c → ch); the one that matters is a
37
+ variant spelling of Chaudhry (چوھدری; the usual چودھری keeps its key).
38
+ - Changed output: stem rules 2026-10-11 → 2026-10-12, Kannada and Marathi
39
+ only. Kannada gains 6 endings (ದನ್ನು ಂದು ಿವೆ ಂತೆ ಾಗ ತ್ತ) and Marathi 10
40
+ (postpositions written solid: ंमध्ये बद्दल मुळे ंनी ंना द्वारे कडे ंचा िक
41
+ ान), chosen on PIB queries inside the wrong-hit budget those languages
42
+ had not spent; 6,001 of the 729,361 conformance rows change (kn 3,130,
43
+ mr 2,871). On PIB sentences of May 2021, which no rule was chosen on and
44
+ which were read once, Kannada finds 0.5 points more of the right
45
+ sentences (95%: +0.2 to +1.0) and Marathi 0.8 (+0.5 to +1.4), at 0.4
46
+ and 0.2 points of precision. Recompute stored stems in kn and mr.
47
+ - Changed output: normalize rules 2026-10-11 → 2026-10-12. Recompute stored
48
+ normalized text. "Looks the same" is now decided by every font we hold that
49
+ draws what `normalize` writes, 2 to 15 families a script compared by glyph
50
+ outline, not by Noto and Anek alone, and the joiner table and tree were
51
+ rebuilt with them. On the 12.9M words of the measurement sets, drawn in
52
+ those fonts, the words whose look changes fall from 54 to 17 (all malformed
53
+ input or a font's quirk); the held-out number (88.1%) is unchanged, and the
54
+ groups made equal move by under two points in Malayalam and Bengali (down)
55
+ and three in Perso-Arabic (up). Three faults fixed: khanda ta (ত্ + ZWJ → ৎ)
56
+ is not written where the ZWJ joins ত্ to a cluster with a left-side vowel
57
+ sign or a reph, or under a reph (উত্‍ক্ষেপন keeps its ZWJ); an old Malayalam
58
+ chillu before a vowel sign stays as typed; a ZWJ between a Malayalam
59
+ consonant and its vowel sign (ന + ZWJ + ു) is kept, which fonts with
60
+ traditional ligatures draw. Two more, found on web text no rule was chosen
61
+ on (IndicCorp v2, read once): the chillu k (ൿ) is no longer written (not in
62
+ Unicode's DoNotEmit list; Noto draws it apart), and no chillu is written
63
+ before a second virama; Malayalam words whose look changes there fall from
64
+ 16,327 to 47 of 83M, with 91.8% of joiner groups made equal (Odia 80.5%).
65
+ Odia still loses a ZWNJ in some acronyms and a ZWJ after ଡ଼ + virama (310
66
+ of 922,000 words): docs/normalize.md. The browser file grows from 16 KB to
67
+ 22 KB gzipped: the tree holds more contexts; `deromanize` (22 → 28 KB)
68
+ and `search-key` (19 → 26 KB) embed the same tree and grow with it.
69
+ - Changed output: romanize rules 2026-10-11.2 → 2026-10-12, the names mode
70
+ only (words and places are unchanged). The ten best spellings by the
71
+ alignment score are scored again with a letter 5-gram over the training
72
+ English spellings, and the best four of that order are returned: the rank
73
+ had English-side habits wrong (the schwa, ee against i, a lost h), and the
74
+ accepted spelling was usually already in the list. On development names
75
+ that neither tool's training holds, the first spelling is right 51.7% of
76
+ the time, was 47.0% (IndicXlit 39.7%); any of four 72.8%, was 70.4%. On
77
+ names held out to the end (Wikidata people of Bangladesh, Pakistan, Nepal
78
+ and Sri Lanka, read once, 10,773 unseen tokens) the first spelling goes from
79
+ 37.9% to 41.0%, no language falling. Of the 1,008 names rows of the
80
+ conformance file, 83 change their first spelling and 794 their order.
81
+ `rules.json` gains `rescore`; the model file `lang/shared/names-lm.bin`
82
+ (148 KB brotli) is loaded with every names page, in Go through the
83
+ language package. The names lookup now holds the names the shipped beam
84
+ spells otherwise (13,495 entries, was 16,632 at the reference's beam).
85
+ TypeScript `fromBytes` takes the letter model as a fifth argument for the
86
+ names mode.
87
+
88
+ ## 0.11.0 (2026-10-12)
89
+
90
+ Rules: phonetic 2026-10-11.7, normalize 2026-10-11, segment 2026-10-11, stem 2026-10-11, romanize 2026-10-11.2, deromanize 2026-10-09.2, phonetic-search 2026-10-11.7
91
+
92
+ - Changed output: normalize rules 2026-10-07 → 2026-10-11, Tamil only. A
93
+ ZWNJ or ZWJ between consonant + pulli and a consonant is deleted, unless
94
+ the pair is one of the three Tamil conjuncts (க்ஷ, ஶ்ர, ஸ்ர): ப் + ZWNJ + ப →
95
+ ப்ப. Before, a class-based rule kept most of these and deleted the ZWNJ of
96
+ க்ஷ in some words, a change a reader could see. On IndicCorp Tamil text,
97
+ look-alike groups with a joiner made equal: 89.6% → 98.8%. `fold` follows.
98
+ `rules.json` gains `pairs`; the ports read it.
99
+ - Changed output: segment rules 2026-10-06 → 2026-10-11, Odia only. ଓ +
100
+ virama + ବ is one letter (ଓ୍ବି in ଓ୍ବିକେଟ୍ "wicket"); Odia news words split
101
+ wrongly: 0.13% → 0.04%. ଓ୍ୱ stays two letters (two fonts draw it apart).
102
+ - Docs: normalize's Malayalam limits (the most possible is 71.0% in 15
103
+ fonts; a ZWJ before a vowel sign; Suruma lacks the atomic chillus).
104
+ - Added: deromanize `textMixed` / Go `TextMixed`, and `isEnglish` / Go
105
+ `English`, for Hindi typed with English words in it (Hinglish): a word on
106
+ the Hindi English list (10,000 English Wikipedia words, less those a Hindi
107
+ typist writes often) stays as typed. TypeScript fetches the list (22 KB
108
+ gzipped) only with `load("hi", "words", { english: true })`. On
109
+ L3Cube-HingLID, read once, 87.4% of words are called right (IndicLID
110
+ roman: 72.1%); 2.4% of Hindi words stay in Latin. Every other output is
111
+ unchanged, so the deromanize rules version is unchanged.
112
+ - Added: romanize mode `"places"` (Go `romanize.Places`), rules
113
+ 2026-10-11 → 2026-10-11.2, for place names. Own tables for bn hi ml mr or
114
+ sat ta te ur, learned from Wikidata place names and, for hi mr or, village
115
+ names of the Local Government Directory; the words tables in the other
116
+ languages. Held out, read once, the training's own names left out: village
117
+ names 57.6% first spelling right (words mode 48.0%, IndicXlit 52.8%);
118
+ OpenStreetMap place names 44.4% (40.6%, IndicXlit 49.1%). The words and
119
+ names modes are unchanged since 2026-10-11.
120
+ - Changed output: romanize rules 2026-10-09 → 2026-10-11, words mode.
121
+ Bodo writes ख थ फ as kh th ph, as Bodo speakers type them (थाखाय →
122
+ thakhai, was takai), not as Aksharantar's k t f. On typed Bodo sentences
123
+ (Bhasha-Abhijnaanam, read once) the first spelling is right 71.9% of the
124
+ time, was 50.3% (IndicXlit 49.3%); on Aksharantar's word list it falls
125
+ from 75.6% to 45.6%. The shared Brahmic table was retrained, so a few
126
+ lower-ranked spellings in other languages change too (61 of 1,018 words
127
+ conformance rows; no language's first spelling moved by more than 0.2
128
+ points). The names mode is unchanged.
129
+
130
+ - Changed output: stem rules 2026-10-09 → 2026-10-11, Sanskrit only. 17
131
+ endings added (the anusvara spellings ानां ां ायां, the passive यते यन्ते,
132
+ the infinitive तुं, the absolutive त्वा, and others); 1,301 of the 418,730
133
+ conformance rows change, all Sanskrit. On lemmas of the Digital Corpus of
134
+ Sanskrit, read once, recall rises from 34.6% to 40.5% at 2.7% wrong hits.
135
+ - Docs: `stem` damages some names (README Limits, `docs/stem.md#names`).
136
+ - Changed output: phonetic 2026-10-07 → 2026-10-11.7. Recompute stored keys.
137
+ `Keys` gives more keys to some words; every key a word had in 0.10.0 it
138
+ still has (checked on every test word, but two long Urdu words whose
139
+ readings exceed the cap of 32 keys a word, was 16), so every pair 0.10.0
140
+ matched still matches. On held-out OpenStreetMap place names the key
141
+ finds 89% of name pairs (romanize + Soundex: 71%).
142
+ - Bengali-script ং and Devanagari ं also read as ŋ: সিং = Singh (Bengali
143
+ and Assamese), and Bodo spelling (दं = dong; Bodo running text 74% →
144
+ 86% of words).
145
+ - Tamil ட்ச also reads as க்ஷ: லட்சுமி = Lakshmi. A Tamil name ending in
146
+ ன் or ர் is also keyed without it; a Tamil word starting with a vowel
147
+ also keys with h (உசைன் = Hussain). Latin zh is ழ (Kanimozhi);
148
+ Malayalam റ്റ is also tt and ന്റ also nt (ആന്റണി = Antony). Tamil +1.6 and
149
+ Malayalam +1.9 points on Wikidata (DEV); on BPCC name pairs, read once,
150
+ Tamil +0.9 and Malayalam +1.5.
151
+ - Assamese: ৱ after a vowel (not before i) is also a glide (গোৱেল =
152
+ Goyal), and the case endings ৰ, ৱে, লৈ, ৰপৰা are also read off.
153
+ - A one-letter Latin word also keys as its spoken name: M = em = एम =
154
+ எம். M and N, and H and S, now share a key as initials.
155
+ - Santali: an initial ᱚ is also a (Ol Chiki writes the Indic short a
156
+ with it: Arjun = ᱚᱨᱡᱩᱱ), and ᱣ ᱶ after a vowel is also a glide;
157
+ Santali 74.1% → 84.7% on Wikidata (DEV).
158
+ - Sindhi in Devanagari: the implosives ॻ ॼ ॾ ॿ are read (they had no
159
+ key); व्ह is also v (चव्हाण = Chavan); the long Marathi / Konkani case
160
+ endings (च्या ांच्या मध्ये ांना), the Konkani -ान -ीन -ाईन -ाक -ाचो -ाची
161
+ -ाचें, the Sanskrit -स्य and the Sindhi / Bodo -नि are also read off. On
162
+ BPCC name pairs, read once, +0.49 points (Bodo +3.5).
163
+ - Urdu, Kashmiri and Sindhi script: an initial alif or ع is also i, e or u
164
+ (اندر = Indra, عمران = Imran, which did not match before), and ہ ح
165
+ after a consonant is also silent (جہانگیانی = Jhangiani). Urdu +4.8,
166
+ Kashmiri +4.3 points on Wikidata (DEV); on BPCC name pairs, read once,
167
+ Urdu +3.2. Latin zh is ழ's l or z + h, so both Kanimozhi and Mazhar
168
+ (مظہر) match.
169
+ - Tamil's இ or உ before an initial r or l is also dropped (இராஜ்பால் =
170
+ Rajpal); व after a consonant and before u is also the au glide (கவுர் =
171
+ Kaur); ड ढ after the first letter are also r, the flap written without its nukta
172
+ (राबडी = Rabri). Tamil +3.8 points on Wikidata (DEV).
173
+ - Changed output: `NameKeys`, `JoinedKeys` and `Match` (`nameKeys`,
174
+ `joinedKeys`, `match`) drop the leading titles (Shri, Smt., Dr, श्रीमती,
175
+ திரு, …; `titles` in rules.json) of each comma-separated part of a name:
176
+ "Bajwa, Sardar Partap Singh" reads as Bajwa Partap Singh. A name of titles
177
+ only keeps its last word. `Words` still splits only. A second roster as
178
+ written, the Rajya Sabha: the key finds 54.8% → 87.7%.
179
+ - Changed output: `JoinedKeys` (`joinedKeys`) returns keys of 6 classes
180
+ or more (`JoinedMinClasses`, was 8): a name written as one word in one
181
+ script and as two in another meets more often (Amartya Sen =
182
+ अमर्त्यसेन). +1.6 points on Wikidata (DEV), +2.7 on OpenStreetMap places
183
+ (read once).
184
+ - Changed output: phonetic search 2026-10-09 → 2026-10-11.7. It keys with
185
+ phonetic rules 2026-10-11.7, drops titles as `NameKeys` does, and its
186
+ thresholds were chosen again on the same data: names 74 → 76, villages
187
+ 72 → 73, text 80 → 81 and 70 → 71 (`Thresholds` / `THRESHOLDS`). On held-out
188
+ OpenStreetMap places it finds 93% of places in five languages (ICU +
189
+ fuzzy match: 74%), at about 3 wrong places per search per 10,000 names.
190
+ - Size: the title list, the letter names and the new rules add about
191
+ 1.3 KB gzipped to `phonetic`: 6.9 → 8.2 KB. `phonetic-search` falls
192
+ 17.0 → 14.5 KB: its 2^x tables are stored as second differences, and its
193
+ two fine rules files as patches over `rules.json` (every score unchanged).
194
+ - Changed output: stem, in the same rules 2026-10-11. Gujarati, Marathi,
195
+ Maithili and Santali run their endings twice, as Kannada does; Maithili
196
+ and Santali gain 7 endings each. On text no rule saw, read once:
197
+ Gujarati +1.7 and Marathi +1.6 points of recall (PIB), Maithili +0.95
198
+ (BPCC), Santali +0.56 (FLORES+); precision falls by at most 1.3 points
199
+ against the old output. About 12,900 conformance rows change, in these
200
+ four languages only.
201
+ - `stem`: Sindhi in Perso-Arabic and Manipuri in Bengali script get their
202
+ own endings in the same tables, chosen on FLORES+ and checked once on its
203
+ unseen half (Manipuri: 5.6 points more of the right sentences than exact
204
+ search). Sindhi in Perso-Arabic then gained more endings, checked once on
205
+ the unseen test part of the Sindhi UD treebank: the share of inflected
206
+ forms that reach their word's stem rose from 20.2% to 33.0%, level with
207
+ Snowball's Arabic stemmer, with fewer wrong merges. Words in those
208
+ scripts were returned unchanged before; no Devanagari or Meetei Mayek
209
+ output changes.
210
+ - New: `indickit/search-key` (Go `searchkey.Key`), one search key per word:
211
+ normalize, stem, then fold, the order measured best. Fold after stem
212
+ finds 2.4 points more of the right passages in Hindi (MIRACL) and changes
213
+ nothing in Bengali or Telugu. The README's search recipe now uses it, and
214
+ `docs/stem.md` gives an OpenSearch/Elasticsearch setup with a trigram
215
+ field beside the keys.
216
+ - New in `stem`: two options. `minStem` (Go `StemWith`, `Options.MinStem`)
217
+ keeps stems longer, for fields of names; 4 halves the names a search
218
+ loses. `indickit/stem-compounds` (Go `CompoundKey`), a separate 1.6 KB
219
+ file, keys a Malayalam word by its first compound member; on held-out
220
+ news it finds 5.5 points more than `stem` alone. The default output does
221
+ not change.
222
+ - New in `stem`: Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri
223
+ (Meetei Mayek), Santali and Sindhi (Devanagari), in the same rules
224
+ 2026-10-11: `stem` now covers all 22 languages. Endings chosen on BPCC's
225
+ human translations; on a part no rule saw, 3.7–11.4 points more of the
226
+ right sentences than exact search, at most 4.4 points less precise.
227
+ About 254,000 conformance rows are new; none changed. The browser file
228
+ grows to 4 KB gzipped.
229
+ - Language tags: `ory` (Odia) and `npi` (Nepali), as FLORES+ and NLLB
230
+ write them, now name the `or` and `ne` tables.
231
+ - New in `stem`: Odia (`or`), in the same rules 2026-10-11. 11 endings
232
+ chosen on PIB search; on FLORES+, read once, 6.9 points more of the right
233
+ sentences than exact search, 3.4 points less precise. Odia's 24,578
234
+ conformance rows are new.
235
+
236
+ ## 0.10.0 (2026-10-09)
237
+
238
+ Rules: phonetic 2026-10-07, normalize 2026-10-07, segment 2026-10-06, stem 2026-10-09, romanize 2026-10-09, deromanize 2026-10-09.2, phonetic-search 2026-10-09
239
+
240
+ - Changed output: phonetic search 2026-10-08 → 2026-10-09, for language
241
+ tags only. `NewIndex` and `Score` (Go), `loadSearch` (TypeScript) took
242
+ a bare code (`hi`) where every other utility takes a tag: `hi-IN`,
243
+ `HI` and `hin` found no Latin table and scored by the fine keys alone.
244
+ They now read the tag as the other utilities do. The tables are
245
+ unchanged: every score for a bare code is the same.
246
+ - Changed output: `kok` (the Konkani macrolanguage) names the `gom`
247
+ tables in every utility's tag reading. romanize and deromanize did
248
+ this already; stem and normalize hold no Konkani table, so nothing
249
+ changes there.
250
+ - Removed exports: TypeScript `_internal` (phonetic) and `unify`
251
+ (romanize), which no documented use needed. The phonetic engine now
252
+ lives in `js/phonetic-engine.ts`, shared by `phonetic` and
253
+ `phonetic-search`, not an entry of the package.
254
+ - Fixed: `phonetic.NewIndex` and `Score` (Go), `index` and `score`
255
+ (TypeScript) given a profile name other than `names` or `text` crashed
256
+ later with no cause named (Go: divide by zero in the first `Search`;
257
+ TypeScript: a `TypeError`). They now fail at once, naming the profile.
258
+ - Fixed: the search index built every combination of a name's word keys
259
+ before cutting to 256 (a 26-word name of two-key words: 2^26 strings,
260
+ 8 s in Go, 4.9 s in TypeScript). It now builds the first 256 only, as
261
+ `NameKeys` does; the keys are the same.
262
+ - Go `stem` checks at load that `max_end + min_stem` in rules.json is
263
+ below its fixed walk (16): a larger value would have cut differently
264
+ from the reference, silently.
265
+
266
+ ## 0.9.0 (2026-10-09)
267
+
268
+ Rules: phonetic 2026-10-07, normalize 2026-10-07, segment 2026-10-06, stem 2026-10-09, romanize 2026-10-09, deromanize 2026-10-09.2, phonetic-search 2026-10-08
269
+
270
+ - New: `wellformed`, a check that a word is drawn as written
271
+ (`indickit/wellformed`).
272
+ - Changed output: stem rules 2026-10-06 → 2026-10-09. Kannada applies its
273
+ stem twice (ಪ್ರಧಾನಮಂತ್ರಿಯವರು → ಪ್ರಧಾನಮಂತ್ರಿ): on held-out PIB text,
274
+ recall +0.86 points, precision −0.75. No other language changes.
275
+ - Changed output: romanize rules 2026-10-07 → 2026-10-08. A ZWJ or ZWNJ
276
+ inside a word no longer cuts the word (प्राप्\u200dत gave "the"). Manipuri
277
+ in Bengali script is romanized from the Brahmic tables (রামেন gave "en");
278
+ the `meetei` table group is gone, and `words-brahmic.bin` grows from
279
+ 2,543 to 2,679 KB.
280
+ - Changed output: deromanize rules 2026-10-08 → 2026-10-09. Names mode no
281
+ longer ranks first a spelling that writes nothing for a consonant
282
+ (ml "murmu" gave മു, ks "rajnath" gave راج). Still open: sd "Rafi" in
283
+ words mode keeps a comma chunk, ranked second.
284
+ - Changed output, Go only: NFC now equals Python and ICU on long runs of
285
+ combining marks (Go inserted U+034F after 30 marks). Names split on the
286
+ same spaces in Go and TypeScript (Go missed 24, among them the no-break
287
+ space).
288
+ - Language tags: `hi-IN`, `HI` and `hin` now mean Hindi in Go and
289
+ TypeScript; before, they silently meant no language.
290
+ - Fixed: TypeScript `normalize` and `fold` threw `RangeError` above about
291
+ 125,000 code points; Go `NameKeys` took quadratic time (70.6 s → 0.09 s
292
+ on 1 MB of Tamil).
293
+ - Go and TypeScript read one pinned Unicode 15.0 table for the blocks
294
+ indickit has rules for. Cost: about 2.9 KB gzipped in each bundle that
295
+ reads it; the phonetic, normalize and romanize bundles are now over their
296
+ README budgets.
297
+ - Fixed: Go `phonetic.Score` and `Index.Search` panicked (integer divide
298
+ by zero) on a query word of about 10,000 letters; they now give the
299
+ reference's scores. TypeScript `score` threw `RangeError` from about
300
+ 125,000 candidates in Node.
301
+ - Faster on long input, same output: `wellformed` (262,144 code points:
302
+ 2.3 s -> 0.03 s in Go, 7.2 s -> 0.04 s in TypeScript) and the `romanize`
303
+ and `deromanize` decoders (a 4,096-letter word ran out of 6 GB in Go). `phonetic-search`
304
+ still grows with (query words) x (candidate words): cap user input
305
+ (README, Limits).
306
+ - Changed output, TypeScript only: `romanize` text() keeps a ZWJ or ZWNJ
307
+ that no run of the script goes on from (at the start, after a space,
308
+ after an unassigned code point), as Go always did; it deleted it.
309
+ - Changed output: romanize rules 2026-10-08 -> 2026-10-09 and deromanize
310
+ 2026-10-09 -> 2026-10-09.1. A word of more than 256 letters (`max_letters`)
311
+ gets one spelling: the first spelling of each 256-letter piece, joined.
312
+ Words that long are not words; the beam was quadratic on a long run of
313
+ characters it has no table for (32,000 random code points: 49 s).
314
+ - Faster, same output: `deromanize` 1.5-1.6x (a Hindi word in the browser:
315
+ 19 -> 12 ms) and `romanize` 1.2-1.4x, in Go and TypeScript. The
316
+ `romanize` browser file is now 9 KB.
317
+ - Changed output: deromanize rules 2026-10-09.1 -> 2026-10-09.2. No
318
+ spelling is empty: "q" in Bengali gave ক, ক্ and an empty spelling; the
319
+ empty one is dropped, as `romanize` always did (11 of 3,033 conformance
320
+ rows change, each by that drop).
321
+ - The browser files are minified: each is 0.3 to 3.5 KB smaller gzipped
322
+ (`phonetic` 7 KB, `normalize` 16 KB, `romanize` 7 KB, `deromanize`
323
+ 21 KB, `phonetic-search` 17 KB). Same code, same output (`check-dist`).
324
+
325
+ ## 0.8.0 (2026-10-08)
326
+
327
+ Rules: phonetic 2026-10-07, normalize 2026-10-07, segment 2026-10-06, stem 2026-10-06, romanize 2026-10-07, deromanize 2026-10-08, phonetic-search 2026-10-08
328
+
329
+ - New: `deromanize`, Latin typing to Indian script (`indickit/deromanize`).
330
+ - New: `phonetic-search`, a scored search over phonetic keys
331
+ (`indickit/phonetic-search`).
332
+ - Changed output: phonetic rules 2026-10-06.1 → 2026-10-07, which added the
333
+ folds `brahmic-av-u`, `flap-r` and `latin-ngh`.
334
+ - README: a table of the 22 languages and what each utility covers.
335
+ - 0.7.0 was committed but never tagged or published; its work is here.
336
+
337
+ ## 0.6.0 (2026-10-07)
338
+
339
+ Rules: phonetic 2026-10-06.1, normalize 2026-10-07, segment 2026-10-06, stem 2026-10-06, romanize 2026-10-07
340
+
341
+ - New: `romanize`, Indian text in Latin letters as people spell it
342
+ (`indickit/romanize`). Its tables load at run time, beside the module or
343
+ from jsDelivr at this version's tag.
344
+
345
+ ## 0.5.0 (2026-10-07)
346
+
347
+ Rules: phonetic 2026-10-06.1, normalize 2026-10-07, segment 2026-10-06, stem 2026-10-06
348
+
349
+ - Changed output: `fold` rules 2026-10-07.
350
+ - Changed output: phonetic 2026-10-06.1, a fix for digits.
351
+ - Language tags on `normalize` and `fold`.
352
+
353
+ ## 0.4.3 (2026-10-06)
354
+
355
+ Rules: phonetic 2026-10-06, normalize 2026-10-05, segment 2026-10-06, stem 2026-10-06
356
+
357
+ - First version on npm. README names the floors: Go 1.22, Node 20,
358
+ TypeScript `moduleResolution` nodenext or bundler.
359
+
360
+ ## 0.4.2, 0.4.1 (2026-10-06)
361
+
362
+ Rules: as 0.4.0.
363
+
364
+ - Go floor 1.22. 0.4.1 failed its README test and was not published.
365
+
366
+ ## 0.4.0 (2026-10-06)
367
+
368
+ Rules: phonetic 2026-10-06, normalize 2026-10-05, segment 2026-10-06, stem 2026-10-06
369
+
370
+ - New: `stem`, one search key for the forms of a word, 13 languages.
371
+
372
+ ## 0.3.0 (2026-10-06)
373
+
374
+ Rules: phonetic 2026-10-06, normalize 2026-10-05, segment 2026-10-06
375
+
376
+ - New: `segment`, the letters a reader sees.
377
+
378
+ ## 0.2.0 (2026-10-06)
379
+
380
+ Rules: phonetic 2026-10-06, normalize 2026-10-05
381
+
382
+ - New: `normalize`, one encoding for text that looks the same.
383
+ - Changed output: phonetic rules 2026-10-05 → 2026-10-06.
384
+
385
+ ## 0.1.0 (2026-10-05)
386
+
387
+ Rules: phonetic 2026-10-05
388
+
389
+ - New: `phonetic`, a key that matches one name across scripts.