which-dialect 0.5.6 → 0.5.9

package/README.md

@@ -2,7 +2,7 @@
 
 Translate words **between languages and between dialects**, meaning by meaning: Northern Vietnamese *ngô* into Southern *bắp*, English *cool* (as in "awesome") into Southern Vietnamese *ngầu*, English "I" into the right Vietnamese pronoun for who you're talking to, US *truck* into UK *lorry*. Dictionary data comes from [Wiktionary](https://en.wiktionary.org/) (via [Kaikki.org](https://kaikki.org/)) and loads only when you need it.
 
-**Status:** which-dialect is starting with **Vietnamese**, with English as the bridge language, and the current focus is cleaning up Vietnamese translations. **More languages are planned.** The translator and data format are language-general, so adding one is a config file plus a data package (see [Adding a language](#adding-a-language)). Spanish was supported early on and has been removed for now so the work can focus on Vietnamese.
+**Status:** which-dialect is starting with **Vietnamese**, with English as the bridge language, and the current focus is cleaning up Vietnamese translations. **More languages are planned.** The translator and data format are language-general, so adding one is a config file plus a data package (see [Adding a language](#adding-a-language)). **Spanish** is supported too, by country and region group (Spain, Mexico, Latin America, Río de la Plata, …).
 
 ```ts
 import { createTranslator } from 'which-dialect'
@@ -18,7 +18,7 @@ await tr.translate('cool', { from: 'en', to: 'vi', toRegion: 'Southern', meaning
 
 Each result is a list of **groups, one per meaning** of the source word, each with its part of speech, its definitions and its translations, so a word's meanings never get mixed up.
 
-**Data:** Wiktionary data of 2026-09-25 for both languages. See [`DATA_UPDATES.md`](DATA_UPDATES.md) for each language's dates and update history.
+**Data:** Wiktionary data of 2026-09-25 (Vietnamese: 2026-09-28). See [`DATA_UPDATES.md`](DATA_UPDATES.md) for each language's dates and update history.
 
 You install one small package. Dictionary data is **not bundled**: each lookup fetches one small file (usually 5–100 kB compressed) from the language's data package on the jsDelivr CDN, and caches it.
 
@@ -27,9 +27,10 @@ You install one small package. Dictionary data is **not bundled**: each lookup f
 | Language | Code | Data package | Regions | Entries | Region-tagged senses |
 |---|---|---|---|---|---|
 | Vietnamese | `vi` | [`which-dialect-vi`](packages/vi) | Northern, Central, Southern | 35,394 | 1,330 of 42,419 (3.1%) |
+| Spanish | `es` | [`which-dialect-es`](packages/es) | 23 countries (Spain, Mexico, Argentina, Cuba, …), plus groups: Latin America, Central America, Caribbean, South America, Río de la Plata | 123,995 | 11,940 of 153,963 (7.8%) |
 | English | `en` | [`which-dialect-en`](packages/en) | 15 countries/areas (US, UK, Australia, India, …), plus groups: North America, British Isles, Commonwealth, … | 594,544 | 37,339 of 776,085 (4.8%) |
 
-Vietnamese and English can be translated in both directions, and between any of their regions, including between two dialects of one language (Northern → Southern Vietnamese, US → UK English). English is also the bridge every language's definitions are written in.
+Any of these languages can be translated into any other, in both directions and between any of their regions, including between two dialects of one language (Northern → Southern Vietnamese, US → UK English, Spain → Mexico Spanish) and dialect to dialect across languages (Mexican Spanish → Southern Vietnamese). English is also the bridge every language's definitions are written in.
 
 More languages are planned. Each one is a config file in [`languages/`](languages) plus a data package in `packages/<code>`; see [Adding a language](#adding-a-language).
 
@@ -92,20 +93,20 @@ Meanings are ordered by `meaning` (if given), then by being tagged for `fromRegi
 
 ### Hand-picked words
 
-For common words, a speaker of the target language knows the natural word even where the data ranks another first: *get* "To fetch, bring, take" ranks Vietnamese *đưa* ("to hand, to bring") above *lấy*, because *đưa*'s definition matches more of the English words and no *lấy* definition says "fetch". Each language can list **hand-picked words for English meanings** (`picks` in `languages/<code>.ts`, built into `picks.json`). In a meaning that has picks, the picked words come first (`bridge: 'picked'`), then the ranked words; other meanings are unchanged. Picks can carry a region tag, like translation tables: with `toRegion`, picks tagged for it come first and picks tagged only for other regions are left to the ranking. A pick can also mark its meaning as the one to put first when no `pos` or `meaning` is given.
+For common words, a speaker of the target language knows the natural word even where the data ranks another first: *get* "To fetch, bring, take" ranks Vietnamese *đưa* ("to hand, to bring") above *lấy*, because *đưa*'s definition matches more of the English words and no *lấy* definition says "fetch". Each language can list **hand-picked words for English meanings** (`picks` in `languages/<code>.ts`, built into `picks.json`). In a meaning that has picks, the picked words come first (`bridge: 'picked'`), then the ranked words; other meanings are unchanged. Picks can carry a region tag, like translation tables: with `toRegion`, picks tagged for it come first and picks tagged only for other regions are left to the ranking. A pick can also mark its meaning as the one to put first when no `pos` or `meaning` is given. A pick can be a phrase the dictionary doesn't list (*chưa bao giờ*, "never"; marked `phrase: true`), and a row can `exclude` words that should never be given for its meaning (*mát mặt* for *well* "in good health").
 
-Picks are keyed by English meaning, so each language needs one list however many languages there are: translating from any other language reaches them through the English meaning (as with translation tables). They're optional: a language without picks uses the ranking alone. `picks: false` shows the ranking alone, and `npm run evaluate` reports both. Vietnamese picks so far: *get* "fetch" → *lấy*, *mang*; *get* "obtain" → *lấy*.
+Picks are keyed by English meaning, so each language needs one list however many languages there are: translating from any other language reaches them through the English meaning (as with translation tables). They're optional: a language without picks uses the ranking alone. `picks: false` shows the ranking alone, and `npm run evaluate` reports both. Vietnamese picks so far: *get* "fetch" → *lấy*, *mang*; *get* "obtain" → *lấy*; a few grammar words; and 54 meanings from a native speaker's review of the 300 most common English words (*can* "know how to" → *biết*; *how* → *như thế nào*, Southern *làm sao*; *life* "a living being" → *mạng*).
 
 ### How accurate is it?
 
-`npm run evaluate` runs known-correct translations across language pairs and dialect pairs (English → Vietnamese by region, including pronouns, Northern → Southern Vietnamese, US ↔ UK, Vietnamese → English). A case passes when a correct word is in the top 3 of the first meaning:
+`npm run evaluate` runs known-correct translations across language pairs and dialect pairs (English → Vietnamese by region, including pronouns, English → Spanish by country, Spanish ↔ Vietnamese, Northern → Southern Vietnamese, Spain ↔ Mexico, US ↔ UK, Vietnamese → English). A case passes when a correct word is in the top 3 of the first meaning:
 
 | Set | Correct word in top 3 | Correct word first |
 |---|---|---|
-| Tuning set (used while developing) | 49 / 49 (100%) | 49 / 49 (100%); 48 / 49 without picks |
-| Held-out set (written afterwards, not tuned against) | 17 / 17 (100%) | 16 / 17 (94%) |
+| Tuning set (used while developing) | 88 / 91 (97%) | 82 / 91 (90%); 81 / 91 without picks |
+| Held-out set (written afterwards, not tuned against) | 29 / 30 (97%) | 28 / 30 (93%) |
 
-The cases involving Spanish were removed with it. The held-out set has since been looked at, so new held-out cases are needed before relying on it (see [`FUTURE_IMPROVEMENTS.md`](FUTURE_IMPROVEMENTS.md)).
+With Spanish back (2026-10-29), its cases are back too; the misses are Spanish (*cerdo* without a part of speech, and Mexican "popcorn" and "cat"; see [`FUTURE_IMPROVEMENTS.md`](FUTURE_IMPROVEMENTS.md)). The held-out set has since been looked at, so new held-out cases are needed before relying on it (see [`FUTURE_IMPROVEMENTS.md`](FUTURE_IMPROVEMENTS.md)).
 
 ## Checking text
 
@@ -208,7 +209,7 @@ Each sentence has `original`, `parts`, `corrected`, `changes` (`{ from, to, why,
 
 ### `createDictionary({ lang, baseUrl?, load? })`
 
-One language's dictionary. `baseUrl` defaults to `https://cdn.jsdelivr.net/npm/which-dialect-<lang>@0.1/data`.
+One language's dictionary. `baseUrl` defaults to `https://cdn.jsdelivr.net/npm/which-dialect-<lang>@0.2/data`.
 
 - `dictionary.lookup(word)`: all entries for a word, with every sense's definitions, regions, labels, synonyms and variant links (`altOf`). The word as written, else lowercase, else capitalized, so names written in lowercase are found (*nhật* → *Nhật*, Japan; *nhật bản* → *Nhật Bản*).
 - `dictionary.searchEnglish(term, { region?, pos?, exclude?, limit?, allSenses? })`: words for an English term in this language, one per word, best first (no meaning handling: use the translator for that). `allSenses: true` returns every matching sense of those words instead.
@@ -216,7 +217,7 @@ One language's dictionary. `baseUrl` defaults to `https://cdn.jsdelivr.net/npm/w
 - `dictionary.segment(text)`: the text split into the language's words, longest dictionary match first (up to the checker's `maxWordUnits`, 3 for Vietnamese): `Hôm nay tôi ăn cơm.` → *Hôm nay* · *tôi* · *ăn cơm*, each `{ text, start, end, entries }` (offsets into the NFC text; `entries` empty for words the dictionary doesn't have). Use it to treat a word of several syllables as one, e.g. to show its meaning.
 - `dictionary.variants(word, { limit? })`: other words with the same letters but other accents that the dictionary has, most common first, each `{ word, entries, frequency? }`: *muộn* → *muốn*, *mượn*, *muôn*…; for words of several syllables every syllable varies (*hom nay* → *hôm nay*). Capitalized headwords count too, unless they're only a person's name: *nhat* → *nhất*, *Nhật* (Japan), *nhạt*…; *nhat ban* → *Nhật Bản*. For apps that let learners pick the word they meant: a real word with the wrong tone passes the spellchecker. Languages written in syllables only (`[]` otherwise).
 - `dictionary.checker()` / `dictionary.syllables()`: the language's checker settings and syllable list (with frequencies), used by `createChecker` (`{}` for languages without them).
-- `dictionary.picks()`: the language's [hand-picked words](#hand-picked-words), `{ word, pos, gloss, picks: [{ word, tags? }], first? }` per English meaning (empty for languages without any).
+- `dictionary.picks()`: the language's [hand-picked words](#hand-picked-words), `{ word, pos, gloss, picks: [{ word, tags? }], exclude?, first? }` per English meaning (`gloss` is the sense's own definition: the last one, for a sense nested under a heading) (empty for languages without any).
 - `dictionary.meta()`: the language's name, regions, region groups, source, license, build date and counts.
 - `dictionary.pronouns({ listener?, region?, speaker?, exclude? })`: how to say "I", "you", "he/she", "we", plural "you" and "they" depending on who you're talking to (or about), one row per relationship, for languages whose pronouns depend on it (so far Vietnamese; empty for others). Each row has the columns `self`, `addressee`, `third`, `selfPlural`, `addresseePlural` and `thirdPlural`. Each word says where it comes from: a dictionary definition (`source: 'gloss'`, with the `gloss`), a grammar rule for a regular compound the dictionary doesn't list (`source: 'rule'`, e.g. *các anh*, with a `note`), or a hand-written override (`source: 'override'`, with a `note`); and, when it has one, the `gender` of the person it refers to and whether a "we" is `inclusive`.
 
@@ -311,7 +312,7 @@ Generated data isn't committed (it would bloat git history); it's built before p
 
 ### Adding a language
 
-1. Add `languages/<code>.ts`: the Kaikki language name, the language's regions (and groups, if any), and how to read regions from Wiktionary's tags. See [`languages/vi.ts`](languages/vi.ts) (dialect areas and a pronoun table) and [`languages/en.ts`](languages/en.ts) (countries with region groups, and trimming a very large language). Spanish was built this way too (countries and region groups; `languages/es.ts` in the git history). Set `placeNames: {}` so its place names translate ("Japan" ↔ its name for Japan). Set `skipFormOf` for heavily inflected languages; for very large ones, see the trimming options in [`scripts/language-config.ts`](scripts/language-config.ts) (`dropTechnical`, `formOfPos`, `keepFormOf`, `maxSensesPerEntry`, `shardLength`, …).
+1. Add `languages/<code>.ts`: the Kaikki language name, the language's regions (and groups, if any), and how to read regions from Wiktionary's tags. See [`languages/vi.ts`](languages/vi.ts) (dialect areas and a pronoun table) and [`languages/en.ts`](languages/en.ts) (countries with region groups, and trimming a very large language). [`languages/es.ts`](languages/es.ts) shows countries with region groups (Latin America, Río de la Plata) and `skipFormOf`. Set `placeNames: {}` so its place names translate ("Japan" ↔ its name for Japan). Set `skipFormOf` for heavily inflected languages; for very large ones, see the trimming options in [`scripts/language-config.ts`](scripts/language-config.ts) (`dropTechnical`, `formOfPos`, `keepFormOf`, `maxSensesPerEntry`, `shardLength`, …).
 2. Copy a data package (`packages/vi`) to `packages/<code>` and update its `package.json` and `README.md`.
 3. Add the language's code to `translationLangs` in `languages/en.ts` (so English keeps its translation tables for it) and rebuild English.
 4. `npm run build:data -- <code>`, check the results, add a few real-data tests and evaluation cases, and publish.
@@ -330,4 +331,4 @@ npm publish -w which-dialect-vi
 npm publish -w which-dialect
 ```
 
-The API loads data versions matching `DATA_VERSION` in `packages/core/src/index.ts` (currently `0.1`). Data-only updates can publish new 0.1.x data versions without touching the API; bump `DATA_VERSION` when the data format changes.
+The API loads data versions matching `DATA_VERSION` in `packages/core/src/index.ts` (currently `0.2`; `0.1` up to which-dialect 0.5.6). Data-only updates can publish new 0.2.x data versions without touching the API; bump `DATA_VERSION` when the data format changes.

package/dist/check.js

@@ -82,6 +82,8 @@ const PLAIN_TYPO_MIN_ZIPF = 4;
 // Candidates kept per misspelled syllable.
 const SPELLING_CANDIDATES = 3;
 async function spelling(ctx) {
+    if (ctx.config.units === 'words')
+        return wordSpelling(ctx);
     if (ctx.config.units !== 'syllables' || !ctx.meta.syllables)
         return [];
     const syllables = await ctx.dict.syllables();
@@ -222,6 +224,60 @@ async function spelling(ctx) {
     }
     return issues;
 }
+// Wiktionary's own misspelling entries ("misspelling of también", "common misspelling of …"). A word counts
+// as misspelt when it's a misspelling and otherwise only an out-of-use spelling ("tambien": misspelling, and
+// obsolete spelling, of también).
+const MISSPELLING = /\bmisspelling of ([\p{L}\p{M}'-]+)/iu;
+const NOT_IN_USE_SPELLING = /\b(misspelling|obsolete spelling|archaic spelling|dated spelling|obsolete form|archaic form) of\b/i;
+// Zipf points below the most common look-alike beyond which another isn't suggested.
+const RARE_VARIANT_GAP = 2;
+// Spelling and accents for languages written in words (Spanish), from the dictionary alone: a word it doesn't
+// know, as a headword or an inflected form (the forms index), that matches known words once accents are
+// dropped gets the most common of them ("cancion" → "canción", "tambien" → "también"). Known words are left
+// alone even when an accented one is more
+// common ("esta" is a word, as "está" is), and so are unknown words with no such match (names, other
+// languages, typos this can't fix), and words capitalized mid-sentence (names).
+async function wordSpelling(ctx) {
+    const issues = [];
+    for (const sentence of ctx.sentences) {
+        for (const w of sentence.words) {
+            if (w.units.length !== 1)
+                continue;
+            // Words the dictionary lists only as misspellings ("tambien": "misspelling of también") count as unknown,
+            // with what they're a misspelling of.
+            const senses = w.entries.flatMap((e) => e.senses);
+            const misspelt = senses.some((s) => MISSPELLING.test(s.glosses[0] ?? '')) && senses.every((s) => NOT_IN_USE_SPELLING.test(s.glosses[0] ?? ''));
+            if (w.entries.length && !misspelt)
+                continue;
+            const u = w.units[0];
+            if (ctx.inBase(u) || !/\p{L}/u.test(u.text))
+                continue;
+            if (u !== sentence.units[0] && /\p{Lu}/u.test(u.text.charAt(0)))
+                continue;
+            const variants = await ctx.dict.variants(u.text, { limit: 3 });
+            // Rare look-alikes don't crowd out the word meant ("nino": niño, not also ñiño).
+            const best = variants[0]?.frequency;
+            const found = unique([
+                ...(misspelt ? senses.map((s) => s.altOf ?? MISSPELLING.exec(s.glosses[0] ?? '')?.[1]) : []).filter((x) => Boolean(x)),
+                ...variants.filter((v) => best === undefined || (v.frequency ?? -Infinity) >= best - RARE_VARIANT_GAP).map((v) => v.word),
+            ]).slice(0, 3);
+            if (!found.length)
+                continue;
+            // Missing accents is the common learner slip; a wrong accent is more surely wrong.
+            const accented = plain(u.text) !== u.text;
+            issues.push({
+                rule: 'spelling',
+                severity: accented ? 'error' : 'warning',
+                start: u.start,
+                end: u.end,
+                text: u.text,
+                message: `${quote(u.text)} isn't a ${ctx.meta.name} word. Did you mean ${listOf(found.map(quote), 'or')}?`,
+                suggestions: found.map((c) => matchCase(u.text, c)),
+            });
+        }
+    }
+    return issues;
+}
 // Words in the learner's own language (English by default), with the target-language word to use: "market"
 // → "chợ". Runs of base units are split into the longest phrases the base dictionary knows ("ice cream"),
 // each translated in the region and register. Words the language usually has no word for

package/dist/detect.js

@@ -12,7 +12,8 @@ import { plain } from "./text.js";
  * way, using its accented form's frequency: "rat" is "rất" (far more common than English "rat"); close
  * calls ("met": mệt or English met) go to the target only when their neighbors are the target's ("Tôi met
  * quá"), so English stays English among English ("kem but it was": but, not bút). A word only the target
- * knows without its accents ("khong") is still the target's. The base is known through the translator, so
+ * knows without its accents ("khong") is still the target's. Languages written in words do the same with
+ * whole words and their inflected forms (Spanish "cancion" → canción, "tambien" → también). The base is known through the translator, so
  * inflected forms count ("went", "cooked").
  */
 // Zipf points by which a word both languages know must be more common in one to count as that language's
@@ -29,8 +30,12 @@ export async function markLanguages(list, o) {
     // 'exact': a target word as written; 'accents': one only with accents added ("khong" → "không").
     const targetKnows = async (u) => {
         const lower = u.text.toLowerCase();
-        if (!syllableLanguage)
-            return (await o.target.lookup(lower)).length > 0 ? 'exact' : false;
+        // Languages written in words compare whole words: "cancion" is Spanish "canción" without its accent.
+        if (!syllableLanguage) {
+            if ((await o.target.lookup(lower)).length > 0)
+                return 'exact';
+            return (await o.target.variants(lower, { limit: 1 })).length > 0 ? 'accents' : false;
+        }
         if (lower in syllables || lower !== plain(lower))
             return 'exact';
         return plainSyllables.has(lower) ? 'accents' : false;
@@ -47,7 +52,9 @@ export async function markLanguages(list, o) {
         return t === 'accents' ? o.lang : '?';
     }));
     // The most common form a plain-letter unit takes once accents are added ("rat" → rất, 6.4).
-    const accentedFrequency = (u) => {
+    const accentedFrequency = async (u) => {
+        if (!syllableLanguage)
+            return (await o.target.variants(u.text.toLowerCase(), { limit: 1 }))[0]?.frequency ?? 0;
         const bare = plain(u.text.toLowerCase());
         let best = 0;
         for (const [s, f] of Object.entries(syllables))

package/dist/index.d.ts

@@ -5,7 +5,7 @@ export * from './pos.ts';
  * Data packages this version of the API reads by default, as a jsDelivr version range.
  * Bump it when the data format changes.
  */
-export declare const DATA_VERSION = "0.1";
+export declare const DATA_VERSION = "0.2";
 /** Labels left out of search results unless `exclude` is passed. */
 export declare const DEFAULT_EXCLUDED_LABELS: string[];
 /** Loads one data file by its path inside the data folder, e.g. "meta.json" or "en/no.json". */
@@ -99,8 +99,9 @@ export type Dictionary = {
     /**
      * Other words written with the same letters but other accents, that the dictionary has, most common
      * first: "muộn" → muốn, mượn, …; for words of several syllables, every syllable varies ("hom nay" →
-     * "hôm nay"). For languages written in syllables (with a syllable list); `[]` otherwise. The word
-     * itself isn't included.
+     * "hôm nay"). Languages written in words (with no syllable list) compare whole words, headwords and
+     * inflected forms alike: Spanish "cancion" → canción, "esta" → está, ésta. The word itself isn't
+     * included.
      */
     variants(word: string, options?: {
         limit?: number;

package/dist/index.js

@@ -1,12 +1,12 @@
 import { PERSONAL_NAME, PRONOUN_PERSONS, fromStored, normalizeEnglish, shardKey, } from "./types.js";
-import { accentCombinations, segment as segmentUnits, syllablesByPlain, units } from "./text.js";
+import { accentCombinations, plain, segment as segmentUnits, syllablesByPlain, units } from "./text.js";
 export * from "./types.js";
 export * from "./pos.js";
 /**
  * Data packages this version of the API reads by default, as a jsDelivr version range.
  * Bump it when the data format changes.
  */
-export const DATA_VERSION = '0.1';
+export const DATA_VERSION = '0.2';
 /** Labels left out of search results unless `exclude` is passed. */
 export const DEFAULT_EXCLUDED_LABELS = [
     'vulgar', 'offensive', 'derogatory', 'archaic', 'obsolete', 'dated', 'historical', 'rare', 'abbreviation',
@@ -67,16 +67,40 @@ export function createDictionary(options) {
     async function shard(kind, term) {
         const m = await meta();
         const key = shardKey(term, m.shardLength);
-        return m.shards[kind].includes(key) ? loadOnce(`${kind}/${key}.json`) : null;
+        return m.shards[kind]?.includes(key) ? loadOnce(`${kind}/${key}.json`) : null;
+    }
+    // An inflected form from the forms index as an entry: one sense, "first-person singular present
+    // indicative of hablar", pointing at its lemma (altOf), used in every region like the lemma.
+    function formEntries(m, word, forms) {
+        const descriptions = m.forms?.descriptions ?? [];
+        return forms.map(([lemma, pos, description, frequency]) => ({
+            word,
+            pos,
+            senses: [
+                {
+                    glosses: [(descriptions[description] ?? 'form of {}').replace('{}', lemma)],
+                    regions: m.regions,
+                    regionTagged: false,
+                    labels: [],
+                    altOf: lemma,
+                },
+            ],
+            ...(frequency !== undefined ? { frequency } : {}),
+        }));
     }
     async function lookup(word) {
         const w = word.trim();
-        const [m, data] = await Promise.all([meta(), shard('words', w)]);
+        const [m, data, forms] = await Promise.all([meta(), shard('words', w), shard('forms', w)]);
         // The word as written, else lowercase ("Mẹ" → "mẹ"), else capitalized for names written in lowercase
         // ("nhật" → "Nhật", Japan; "nhật bản" → "Nhật Bản"), which spellchecker suggestions and learners do.
         const titled = w.toLowerCase().replace(/(^|[\s-])(\p{Ll})/gu, (_, sep, c) => sep + c.toUpperCase());
-        const stored = data?.[w] ?? data?.[w.toLowerCase()] ?? data?.[titled] ?? [];
-        return stored.map((e) => ({ ...e, senses: e.senses.map((s) => fromStored(s, m.regions)) }));
+        const key = [w, w.toLowerCase(), titled].find((k) => data?.[k] || forms?.[k]);
+        if (!key)
+            return [];
+        // Headword entries first, then the word as an inflected form of others ("música": the noun, then the
+        // feminine of "músico").
+        const entries = (data?.[key] ?? []).map((e) => ({ ...e, senses: e.senses.map((s) => fromStored(s, m.regions)) }));
+        return [...entries, ...formEntries(m, key, forms?.[key] ?? [])];
     }
     const checker = async () => {
         const m = await meta();
@@ -86,6 +110,31 @@ export function createDictionary(options) {
         const m = await meta();
         return m.syllables ? loadOnce('syllables.json') : {};
     };
+    // Variants for languages written in words: other headwords and inflected forms with the same letters
+    // once accents are dropped. Shards are keyed without accents, so they're all in the word's own shards.
+    async function wordVariants(word, limit) {
+        const written = word.trim().normalize('NFC').toLowerCase();
+        const target = plain(written);
+        if (!target)
+            return [];
+        const [data, forms] = await Promise.all([shard('words', written), shard('forms', written)]);
+        const candidates = new Set();
+        for (const key of [...Object.keys(data ?? {}), ...Object.keys(forms ?? {})]) {
+            const lower = key.toLowerCase();
+            if (lower !== written && plain(lower) === target)
+                candidates.add(lower);
+        }
+        const found = await Promise.all([...candidates].map(async (c) => ({ word: c, entries: await lookup(c) })));
+        const frequency = (entries) => {
+            const known = entries.map((e) => e.frequency).filter((f) => f !== undefined);
+            return known.length ? Math.max(...known) : undefined;
+        };
+        return found
+            .filter((v) => v.entries.length)
+            .map((v) => ({ ...v, frequency: frequency(v.entries) }))
+            .sort((a, b) => (b.frequency ?? -1) - (a.frequency ?? -1))
+            .slice(0, limit);
+    }
     return {
         meta,
         lookup,
@@ -100,7 +149,7 @@ export function createDictionary(options) {
         async variants(word, { limit = 5 } = {}) {
             const list = await syllables();
             if (!Object.keys(list).length)
-                return [];
+                return wordVariants(word, limit);
             const written = word.trim().normalize('NFC').toLowerCase();
             const parts = written.split(/\s+/);
             const combos = accentCombinations(parts, syllablesByPlain(list), {

package/dist/translate.js

@@ -445,8 +445,9 @@ export function createTranslator(options = {}) {
         return hits;
     }
     async function englishMeanings(sense, from, bridge) {
+        // A meaning's gloss is its own (last) definition: a nested sense's first is the heading it shares.
         if (from === 'en')
-            return [{ word: sense.lemma, pos: sense.pos, gloss: sense.glosses[0] ?? '', translations: sense.translations }];
+            return [{ word: sense.lemma, pos: sense.pos, gloss: sense.glosses.at(-1) ?? '', translations: sense.translations }];
         const found = [];
         for (const term of bridge.slice(0, 3)) {
             for (const entry of await dict('en').lookup(term)) {
@@ -454,8 +455,9 @@ export function createTranslator(options = {}) {
                     const own = s.translations?.[from];
                     if (!own?.some((t) => t.word === sense.lemma || t.word === sense.word))
                         continue;
-                    if (!found.some((f) => f.word === entry.word && f.gloss === s.glosses[0])) {
-                        found.push({ word: entry.word, pos: entry.pos, gloss: s.glosses[0], translations: s.translations });
+                    const gloss = s.glosses.at(-1) ?? '';
+                    if (!found.some((f) => f.word === entry.word && f.gloss === gloss)) {
+                        found.push({ word: entry.word, pos: entry.pos, gloss, translations: s.translations });
                     }
                 }
             }
@@ -522,6 +524,13 @@ export function createTranslator(options = {}) {
             ...(entry.frequency !== undefined ? { frequency: entry.frequency } : {}),
         };
     }
+    // A hand-picked phrase the dictionary doesn't list: used everywhere, or where its tags say.
+    function phraseHit(word, sense, places, meta) {
+        return {
+            word, pos: sense.pos, gloss: sense.glosses.at(-1) ?? '', regions: places ? [...places] : meta.regions,
+            regionTagged: Boolean(places), labels: [], senseIndex: 0, senses: 1, primary: true, phrase: true,
+        };
+    }
     // The source word's uses in a pronoun table, with the definition each came from.
     function pronounUses(rows, word) {
         const w = word.trim();
@@ -722,7 +731,8 @@ export function createTranslator(options = {}) {
                 // Hand-picked words for this meaning go first, in the listed order: a speaker's judgment of the
                 // natural word where the ranking puts another first. With a target region, picks tagged for it
                 // lead, and picks tagged only for other regions (or words not used there) are left to the
-                // ranking. The ranked words follow. (A pronoun-table row, below, still comes before them.)
+                // ranking. The ranked words follow. (A pronoun-table row, below, still comes before them.) A pick
+                // that isn't a headword is a phrase ("bữa tối"), given as is. Words a row excludes are dropped.
                 const rows = pickRows.length
                     ? await englishMeanings(sense, from, bridge).then((ms) => pickRows.filter((r) => ms.some((m) => m.word === r.word && m.pos === r.pos && m.gloss === r.gloss)))
                     : [];
@@ -737,13 +747,17 @@ export function createTranslator(options = {}) {
                         continue;
                     if (to === from && t.word === sense.lemma && !places?.size)
                         continue;
-                    const hit = scored.get(t.word) ?? (await tableHit(t.word, to, posList, toWanted, exclude, bridge));
+                    const hit = scored.get(t.word) ??
+                        (await tableHit(t.word, to, posList, toWanted, exclude, bridge)) ??
+                        (t.word.includes(' ') ? phraseHit(t.word, sense, places, toMeta) : null);
                     if (!hit || hit.labels.some((l) => exclude.includes(l)))
                         continue;
                     if (!places && !intersects(hit.regions, toWanted))
                         continue;
                     picked.push({ hit, tagged: Boolean(toWanted && places) });
                 }
+                for (const w of rows.flatMap((r) => r.exclude ?? []))
+                    scored.delete(w);
                 if (picked.length) {
                     const ordered = [...picked.filter((p) => p.tagged), ...picked.filter((p) => !p.tagged)];
                     const top = Math.max(0, ...[...scored.values()].map((t) => t.score));

package/dist/types.d.ts

@@ -63,6 +63,8 @@ export type Hit = {
     frequency?: number;
     /** true when the English term is the first meaning listed in the gloss (e.g. "now" in "now, today"). */
     primary: boolean;
+    /** true for a hand-picked phrase that isn't a dictionary headword ("bữa tối"); its details are the pick's. */
+    phrase?: boolean;
 };
 export type LanguageMeta = {
     /** ISO 639 code, e.g. "vi". */
@@ -99,10 +101,11 @@ export type LanguageMeta = {
         regionTaggedSenses: number;
         englishTerms: number;
     };
-    /** Shard file names (without .json) in words/ and en/. */
+    /** Shard file names (without .json) in words/ and en/, and forms/ when the language has a forms index. */
     shards: {
         words: string[];
         en: string[];
+        forms?: string[];
     };
     /** Letters per shard key; see shardKey. Omitted means 2. */
     shardLength?: number;
@@ -116,6 +119,16 @@ export type LanguageMeta = {
     syllables?: boolean;
     /** true when the language has sentence frames (frames.json, see FramesData). */
     frames?: boolean;
+    /**
+     * The language's inflected forms are in a compact index (forms/<shard>.json, see FormShard) rather than
+     * stored as entries: Spanish's conjugations and plurals are most of its source and would triple its
+     * data. `descriptions` are what each form is, with "{}" where the lemma goes ("first-person singular
+     * present indicative of {}"); a stored form points into it.
+     */
+    forms?: {
+        descriptions: string[];
+        count: number;
+    };
 };
 /**
  * Pronoun slots in sentence frames ({I}, {YOU}, …) and the pronoun-table column each one fills from, for
@@ -213,12 +226,18 @@ export type CheckerConfig = {
  * translating between two other languages reaches it through the English meaning.
  */
 export type PickRow = {
-    /** The English headword, part of speech and first definition of the meaning, as in the English data. */
+    /**
+     * The English headword, part of speech and definition of the meaning, as in the English data. For a
+     * nested sense (a heading plus its own definition: "old", "Of a living being…") it's the sense's own,
+     * last definition, so senses under one heading are told apart.
+     */
     word: string;
     pos: string;
     gloss: string;
-    /** Best first. `tags` name regions like translation-table tags ("Southern"). */
+    /** Best first. `tags` name regions like translation-table tags ("Southern"). A pick needn't be a headword ("bữa tối"). */
     picks: TableTranslation[];
+    /** Words never given for this meaning (a speaker removed them: wrong or unnatural here). */
+    exclude?: string[];
     /** Put this meaning first when the caller gives no `pos` or `meaning` ("got" → "Have/has."). */
     first?: boolean;
 };
@@ -300,6 +319,14 @@ export type StoredHit = Omit<Hit, 'regions' | 'regionTagged' | 'labels'> & {
 };
 /** words/<shard>.json: entries keyed by headword. */
 export type WordShard = Record<string, StoredEntry[]>;
+/**
+ * One inflected form in forms/<shard>.json: [lemma, part of speech, index into meta.forms.descriptions,
+ * Zipf frequency (omitted when unknown)]. lookup() turns it into an entry with one sense whose gloss is
+ * the description and whose `altOf` is the lemma, so the translator follows it to the lemma's meanings.
+ */
+export type StoredForm = [lemma: string, pos: string, description: number, frequency?: number];
+/** forms/<shard>.json: a language's inflected forms keyed by form ("hablo" → hablar). */
+export type FormShard = Record<string, StoredForm[]>;
 /** en/<shard>.json: hits keyed by lowercase English term. */
 export type EnglishShard = Record<string, StoredHit[]>;
 /** Converts a full sense or hit to its stored form. */

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "which-dialect",
-  "version": "0.5.6",
-  "description": "Translate words between languages and dialects, meaning by meaning, starting with Vietnamese (Northern, Central, Southern) and English, with Wiktionary data loaded on demand. More languages are planned.",
+  "version": "0.5.9",
+  "description": "Translate words between languages and dialects, meaning by meaning, starting with Vietnamese (Northern, Central, Southern), Spanish (by country) and English, with Wiktionary data loaded on demand. More languages are planned.",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -26,6 +26,7 @@
     "translation",
     "language-learning",
     "wiktionary",
+    "spanish",
     "vietnamese"
   ],
   "license": "MIT",