which-dialect 0.1.0 → 0.2.0

package/README.md

@@ -18,6 +18,8 @@ await tr.translate('cool', { from: 'en', to: 'vi', toRegion: 'Southern', meaning
 
 Each result is a list of **groups, one per meaning** of the source word, each with its part of speech, its definitions and its translations, so a word's meanings never get mixed up.
 
+**Data:** Wiktionary data of 2026-09-25 for both languages. See [`DATA_UPDATES.md`](DATA_UPDATES.md) for each language's dates and update history.
+
 You install one small package. Dictionary data is **not bundled**: each lookup fetches one small file (usually 5–100 kB compressed) from the language's data package on the jsDelivr CDN, and caches it.
 
 ## Languages
@@ -60,11 +62,12 @@ Loads each language's data on demand. The options are only needed to self-host t
 | `exclude` | Labels to leave out (default: vulgar, offensive, derogatory, archaic, obsolete, dated, historical, rare, abbreviation). |
 | `limit` | Translations per meaning (default 5). |
 | `allSenses` | Return every meaning, including ones with no translation, instead of only useful ones. For letting users pick a meaning. |
+| `picks` | Use the target language's [hand-picked words](#hand-picked-words) (default `true`). `false` returns the ranking alone. |
 
 It returns `TranslationGroup[]`, most relevant meaning first. Each group has:
 
 - `source`: the meaning being translated: `lemma` (the base or standard word, e.g. *say* for *said*), `pos`, `glosses` (its English definitions), `regions`, `labels`, and `via` when the word led there ("simple past of say", "Southern Vietnam form of không").
-- `translations`: target words, best first, each with its own `pos`, `gloss`, `regions`, `labels`, a `score`, and `bridge` (how it was found; `'pronouns'` for the listener's words, which also carry `relationship`, e.g. "Your parents").
+- `translations`: target words, best first, each with its own `pos`, `gloss`, `regions`, `labels`, a `score`, and `bridge` (how it was found; `'picked'` for [hand-picked words](#hand-picked-words); `'pronouns'` for the listener's words, which also carry `relationship`, e.g. "Your parents").
 - `pronounUses` (source words in the source language's pronoun table): the relationships the word is used in with this meaning, `{ id, label, person: 'self' | 'addressee', speaker?, regions, note?, warning? }`. Uses no definition covers get a group of their own, translated as "I" or "you": Vietnamese *em* → "I/me, when talking to: someone a bit older (man); …; your teacher" and "you, when talking to: someone younger; …"; *ba* → "you, when talking to: your parents". Pass `pos: 'pron'` to get only these.
 - `relationships` (singular "I"/"you" meanings into a language with a pronoun table): the word for each relationship, `{ id, label, words, warning? }`, filtered by `toRegion`, `speaker` and `exclude`, for showing "it depends on who you're talking to".
 
@@ -83,19 +86,69 @@ Every language's data defines its words in English, so English is the bridge bet
 5. **Within one language, synonyms are direct equivalents.** Wiktionary lists dialect words as synonyms (*ngô* → *bắp*, *lợn* → *heo*, *lift* → *elevator*), in either direction (*dạ* lists *vâng*).
 6. **Ranking** favors **common words** ([word frequencies](#word-frequencies): *anh* over the niche *cô nương* for "you"), target words whose definition shares the source definition's details, whose *main* meaning is the match, that are tagged for the target region, and that keep the **register**: a polite word translates to a polite word (Northern *vâng* → Southern *dạ*), slang to slang.
 
+**Hand-picked words** come before all of these for the meanings that have them (see below).
+
 Meanings are ordered by `meaning` (if given), then by being tagged for `fromRegion`, then the dictionary's own order, which lists main meanings first. That order is a guess when a word has several parts of speech: English *just* is listed as an adjective ("fair") before the adverb. Pass `pos` or `meaning` when you know which one you want.
 
+### Hand-picked words
+
+For common words, a speaker of the target language knows the natural word even where the data ranks another first: *get* "To fetch, bring, take" ranks Vietnamese *đưa* ("to hand, to bring") above *lấy*, because *đưa*'s definition matches more of the English words and no *lấy* definition says "fetch". Each language can list **hand-picked words for English meanings** (`picks` in `languages/<code>.ts`, built into `picks.json`). In a meaning that has picks, the picked words come first (`bridge: 'picked'`), then the ranked words; other meanings are unchanged. Picks can carry a region tag, like translation tables: with `toRegion`, picks tagged for it come first and picks tagged only for other regions are left to the ranking. A pick can also mark its meaning as the one to put first when no `pos` or `meaning` is given.
+
+Picks are keyed by English meaning, so each language needs one list however many languages there are: translating from any other language reaches them through the English meaning (as with translation tables). They're optional: a language without picks uses the ranking alone. `picks: false` shows the ranking alone, and `npm run evaluate` reports both. Vietnamese picks so far: *get* "fetch" → *lấy*, *mang*; *get* "obtain" → *lấy*.
+
 ### How accurate is it?
 
 `npm run evaluate` runs known-correct translations across language pairs and dialect pairs (English → Vietnamese by region, including pronouns, Northern → Southern Vietnamese, US ↔ UK, Vietnamese → English). A case passes when a correct word is in the top 3 of the first meaning:
 
 | Set | Correct word in top 3 | Correct word first |
 |---|---|---|
-| Tuning set (used while developing) | 42 / 42 (100%) | 42 / 42 (100%) |
+| Tuning set (used while developing) | 49 / 49 (100%) | 49 / 49 (100%); 48 / 49 without picks |
 | Held-out set (written afterwards, not tuned against) | 17 / 17 (100%) | 16 / 17 (94%) |
 
 The cases involving Spanish were removed with it. The held-out set has since been looked at, so new held-out cases are needed before relying on it (see [`FUTURE_IMPROVEMENTS.md`](FUTURE_IMPROVEMENTS.md)).
 
+## Checking text
+
+### `createChecker({ baseUrl?, load?, dictionary? })` → `checker.check(text, options)`
+
+Checks a learner's sentence in one language and returns issues with the exact place in the text, a short explanation and suggested fixes. It runs in the browser like the rest of the library: no API calls, no cost.
+
+```ts
+import { createChecker } from 'which-dialect'
+
+const checker = createChecker()
+const { text, issues } = await checker.check('Tôi muốn ăn thịt lợn và bắp khong?', {
+  lang: 'vi', region: 'Southern', listener: 'parent',
+})
+// “Tôi”: Talking to your parents, say “con” for “I”, not “Tôi”.               → Con
+// “lợn”: “lợn” is Northern; in Southern Vietnamese, say “heo”.                → heo, cúi, ỉn
+// “khong”: “khong” isn't a Vietnamese syllable. Did you mean “không”, “khổng” or “khống”?
+// “khong”: Talking to your parents, end the sentence with “ạ” to sound polite. (suggestion)
+```
+
+| Option | Meaning |
+|---|---|
+| `lang` | The text's language. |
+| `region` | The region the text should be in. Without it, words that don't fit the rest of the text's region are pointed out (*lợn*, Northern, next to *nha*, Southern). |
+| `listener` | Who you're talking to, as a [pronoun-table](#looking-words-up) row id (`'parent'`, `'teacher'`, `'friend'`, …): pronouns and polite endings are checked against it. |
+| `about` | Who you're talking about, for "he/she" and "they" pronouns. |
+| `speaker` | `'male'` or `'female'`, for pronouns that depend on it. |
+| `rules` | Turn checks off, e.g. `{ 'polite-ending': false }`. |
+
+Each issue has `rule`, `severity` (`'error'`: not a word of the language; `'warning'`: wrong for the chosen region or relationship; `'suggestion'`: often better), `start`/`end` (offsets into the returned `text`, the input in Unicode NFC form), `text`, `message` and `suggestions` (replacements for `text`, best first).
+
+The checks:
+
+- **`spelling`**: syllables that aren't in the language, with the accented forms they could be (*khong* → *không*). Plain-letter words with no accented form are left alone (names, foreign words). Languages written in syllables only (Vietnamese).
+- **`dialect`**: words whose main meaning is tagged for another region (*lợn* in Southern text → *heo*; *muỗng* in Northern → *thìa*; in English, US *apartment* in UK text → *flat*). Words with an untagged main meaning are never flagged (*má* is "cheek" everywhere).
+- **`pronoun-relationship`**: pronouns that don't fit the listener (*tôi* to a parent → *con*; to a teacher → *em*).
+- **`pronoun-pair`**: "I" and "you" pronouns that never go together (*tao* goes with *mày*, not *bạn*), when no listener is given.
+- **`polite-ending`**: a suggestion to end sentences with *ạ* when talking to parents, elders and teachers (not for exclamations).
+
+The checker only speaks up when it's confident, so **no issues means none of the checks found anything, not that the text is correct**. It doesn't check word order, classifiers or whether a sentence sounds natural yet. Only reliable pronouns are checked: kinship words like *con* (also "child", and a classifier in *con chó*) aren't, and *bạn* (also "friend") only gets suggestions.
+
+**Every language uses the same checks.** They read only the language's data and its `checker` settings in `languages/<code>.ts` (how text splits into words, which pronouns are reliable, polite endings), so a check whose data a language doesn't have is skipped. A language without checker settings still gets the `dialect` check.
+
 ## Looking words up
 
 ### `createDictionary({ lang, baseUrl?, load? })`
@@ -104,6 +157,8 @@ One language's dictionary. `baseUrl` defaults to `https://cdn.jsdelivr.net/npm/w
 
 - `dictionary.lookup(word)`: all entries for a word, with every sense's definitions, regions, labels, synonyms and variant links (`altOf`).
 - `dictionary.searchEnglish(term, { region?, pos?, exclude?, limit?, allSenses? })`: words for an English term in this language, one per word, best first (no meaning handling: use the translator for that). `allSenses: true` returns every matching sense of those words instead.
+- `dictionary.checker()` / `dictionary.syllables()`: the language's checker settings and syllable list (with frequencies), used by `createChecker` (`{}` for languages without them).
+- `dictionary.picks()`: the language's [hand-picked words](#hand-picked-words), `{ word, pos, gloss, picks: [{ word, tags? }], first? }` per English meaning (empty for languages without any).
 - `dictionary.meta()`: the language's name, regions, region groups, source, license, build date and counts.
 - `dictionary.pronouns({ listener?, region?, speaker?, exclude? })`: how to say "I", "you", "he/she", "we", plural "you" and "they" depending on who you're talking to (or about), one row per relationship, for languages whose pronouns depend on it (so far Vietnamese; empty for others). Each row has the columns `self`, `addressee`, `third`, `selfPlural`, `addresseePlural` and `thirdPlural`. Each word says where it comes from: a dictionary definition (`source: 'gloss'`, with the `gloss`), a grammar rule for a regular compound the dictionary doesn't list (`source: 'rule'`, e.g. *các anh*, with a `note`), or a hand-written override (`source: 'override'`, with a `note`); and, when it has one, the `gender` of the person it refers to and whether a "we" is `inclusive`.
 
@@ -172,17 +227,26 @@ If you **redistribute modified data**, it has to stay under CC BY-SA 4.0. Using
 
 Requires Node 22.6+ (TypeScript scripts run with `--experimental-strip-types`).
 
+Generated data (`packages/<code>/data`) isn't committed, so a fresh clone needs it first. Two ways:
+
+- **Quick start: `npm run fetch:data`.** Downloads the published data packages from npm (a few seconds), then rebuilds hand-picked words from the repo's configs. You get exactly the published data, so tests and `npm run evaluate` match the numbers in this README. Use this unless you're changing how data is built.
+- **Build from Wiktionary: `npm run build:data:all`.** Downloads Kaikki's files (English 3.3 GB, Vietnamese 79 MB; cached in `.cache/`) and builds every language, **English first**: English is the bridge between languages, and other languages' hand-picked words are checked against it. Kaikki always serves its latest dump, so this can be newer than the published data; the build records the dates in [`DATA_UPDATES.md`](DATA_UPDATES.md). Building one language alone (`npm run build:data -- vi`) also works once English is built.
+
 ```sh
 npm install
-npm run build:data -- vi             # download Kaikki's file for a language (cached in .cache/) and build packages/<code>/data
-npm run build:data -- vi --refresh   # re-download first
+npm run fetch:data                   # quick start: the published data (-- --force replaces data you already have)
+npm run build:data:all               # or build every language from Kaikki, English first (-- --refresh re-downloads)
+npm run build:data -- vi             # build one language (after English); -- vi --refresh re-downloads first
 npm test                             # unit tests, plus checks against whichever languages are built
-npm run evaluate                     # translation accuracy on known-correct cases (-- --verbose to see every case)
+npm run evaluate                     # translation accuracy on known-correct cases, with and without picks (-- --verbose to see every case)
+npm run build:picks -- vi            # rebuild only a language's hand-picked words (after editing `picks` in languages/vi.ts)
+npm run picks-sheet -- vi            # review sheet (.cache/picks-vi.csv) of the top English words' meanings; -- --words file.txt, -- --top 500
+npm run picks-sheet -- vi --apply sheet.csv   # config rows for the rows whose your_pick is filled in
 npm run typecheck
 npm run build                        # compile the API to packages/core/dist
 ```
 
-Generated data isn't committed (it would bloat git history); it's built before publishing. Downloads: Vietnamese 79 MB, English 3.3 GB. Builds take seconds (English: about 25 s and 1.4 GB of memory).
+Generated data isn't committed (it would bloat git history); it's built before publishing. Builds take seconds (English: about 25 s and 1.4 GB of memory).
 
 ### Adding a language
 
@@ -190,13 +254,15 @@ Generated data isn't committed (it would bloat git history); it's built before p
 2. Copy a data package (`packages/vi`) to `packages/<code>` and update its `package.json` and `README.md`.
 3. Add the language's code to `translationLangs` in `languages/en.ts` (so English keeps its translation tables for it) and rebuild English.
 4. `npm run build:data -- <code>`, check the results, add a few real-data tests and evaluation cases, and publish.
+5. Optional: [checker](#checking-text) settings (`checker` in `languages/<code>.ts`): `units: 'syllables'` for languages written in syllables (also builds the syllable list for accent suggestions), `maxWordUnits`, the reliable `pronouns` per pronoun-table column, `ambiguousPronouns`, and `politeEndings`. Without them, the language is read as space-separated words and gets the region check.
+6. Optional, later: [hand-picked words](#hand-picked-words). A speaker runs `npm run picks-sheet -- <code>`, fills in `your_pick` only where the first choice is wrong, and pastes the `--apply` output into `picks`.
 
 Languages in non-Latin scripts (Chinese, Arabic, Russian, …) will need a script-aware version of `shardKey` first: today files are split by the first two Latin letters.
 
 ### Publishing
 
 ```sh
-npm run build:data -- en --refresh && npm run build:data -- vi --refresh && npm test
+npm run build:data:all -- --refresh && npm test    # records the new dates in DATA_UPDATES.md: commit it
 npm publish -w which-dialect-en
 npm publish -w which-dialect-vi
 npm publish -w which-dialect

package/dist/check.d.ts

@@ -0,0 +1,45 @@
+import { type TranslatorOptions } from './translate.ts';
+export type CheckRule = 'spelling' | 'dialect' | 'pronoun-relationship' | 'pronoun-pair' | 'polite-ending';
+export type CheckIssue = {
+    rule: CheckRule;
+    /**
+     * 'error': almost certainly wrong (not a word of the language). 'warning': wrong for the chosen region
+     * or relationship. 'suggestion': often better; worth a look.
+     */
+    severity: 'error' | 'warning' | 'suggestion';
+    /** Character offsets of `text` in the checked text (`CheckResult.text`, the input in Unicode NFC form). */
+    start: number;
+    end: number;
+    /** The text the issue is about. */
+    text: string;
+    /** One sentence for the learner, e.g. “khong” isn't a Vietnamese syllable. Did you mean “không”? */
+    message: string;
+    /** Replacements for `text`, best first; empty when there's nothing specific to suggest. */
+    suggestions: string[];
+};
+export type CheckOptions = {
+    /** Language code of the text, e.g. "vi". */
+    lang: string;
+    /** Region or region group the text should be in, e.g. "Southern". Without it, mixed regions are flagged. */
+    region?: string;
+    /** Who you're talking to, as a pronoun-table row id (see Dictionary.pronouns), e.g. "parent". */
+    listener?: string;
+    /** Who you're talking about, as a pronoun-table row id, for "he/she" and "they" pronouns. */
+    about?: string;
+    /** The speaker's gender, for pronouns that depend on it. */
+    speaker?: 'male' | 'female';
+    /** Turn rules off, e.g. { 'polite-ending': false }. All are on by default. */
+    rules?: Partial<Record<CheckRule, boolean>>;
+};
+export type CheckResult = {
+    /** The checked text: the input in Unicode NFC form, which the issues' offsets refer to. */
+    text: string;
+    /** Sorted by position. */
+    issues: CheckIssue[];
+};
+export type Checker = {
+    check(text: string, options: CheckOptions): Promise<CheckResult>;
+};
+/** Same as the translator's: where each language's data comes from. */
+export type CheckerOptions = TranslatorOptions;
+export declare function createChecker(options?: CheckerOptions): Checker;

package/dist/check.js

@@ -0,0 +1,369 @@
+// Checking text in one language: spelling and accents, words from another region, and pronouns and
+// polite endings that fit who you're talking to.
+//
+// Every rule reads only the language's data: its dictionary, pronoun table, checker.json and
+// syllables.json (see CheckerConfig). Adding a language means adding data and settings in
+// languages/<lang>.ts, not code here. A rule whose data a language doesn't have is skipped, so a language
+// without checker settings is still checked for words from another region.
+//
+// The checker only speaks up when it's confident and stays silent otherwise, so no issues means none of
+// the checks found anything, not that the text is correct.
+import { createDictionary, resolveRegion } from "./index.js";
+import { createTranslator } from "./translate.js";
+import { PRONOUN_PERSONS, } from "./types.js";
+// How each pronoun-table column is named in messages.
+const PERSON_NAMES = {
+    self: 'I', addressee: 'you', third: 'he/she', selfPlural: 'we', addresseePlural: 'you (plural)', thirdPlural: 'they',
+};
+// Columns chosen by `about` (who you're talking about); the rest by `listener`.
+const ABOUT_PERSONS = new Set(['third', 'thirdPlural']);
+const SELF_PERSONS = new Set(['self', 'selfPlural']);
+const ADDRESSEE_PERSONS = new Set(['addressee', 'addresseePlural']);
+// A run of letters (with their accents), digits and apostrophes: a word, or a syllable in syllable languages.
+const UNIT = /[\p{L}\p{M}\p{N}'’]+/gu;
+const SENTENCE_END = /[.!?…]+|\n+/g;
+const quote = (s) => `“${s}”`;
+const lowerFirst = (s) => s.charAt(0).toLowerCase() + s.slice(1);
+const listOf = (items, conjunction = 'and') => items.length <= 1 ? items.join('') : `${items.slice(0, -1).join(', ')} ${conjunction} ${items[items.length - 1]}`;
+// Keeps a capital first letter when replacing a capitalized word ("Tôi" → "Con").
+const matchCase = (original, replacement) => original.charAt(0) !== original.charAt(0).toLowerCase() ? replacement.charAt(0).toUpperCase() + replacement.slice(1) : replacement;
+// Without accents, for finding the accented forms of a syllable: "không" → "khong", "đi" → "di".
+const plain = (s) => s.normalize('NFD').replace(/\p{M}/gu, '').replace(/đ/g, 'd').replace(/Đ/g, 'D').normalize('NFC');
+const unique = (items) => [...new Set(items)];
+function units(text) {
+    return [...text.matchAll(UNIT)].map((m) => ({ text: m[0], start: m.index, end: m.index + m[0].length }));
+}
+function sentencesOf(text, all) {
+    const ends = [...text.matchAll(SENTENCE_END)].map((m) => m.index);
+    const out = [[]];
+    let next = 0;
+    for (const u of all) {
+        while (next < ends.length && ends[next] < u.start) {
+            if (out[out.length - 1].length)
+                out.push([]);
+            next++;
+        }
+        out[out.length - 1].push(u);
+    }
+    return out.filter((s) => s.length);
+}
+// Units join into one word only when just spaces separate them ("thịt heo", not "thịt, heo").
+const joined = (text, a, b) => /^[^\S\n]+$/.test(text.slice(a.end, b.start));
+// Longest match against the dictionary: at each position, the longest run of units that's a headword
+// ("kết quả" is one word, so its "quả" isn't checked on its own). Unknown single units become words with
+// no entries.
+async function segment(text, list, maxUnits, lookup) {
+    const words = [];
+    for (let i = 0; i < list.length;) {
+        let taken = null;
+        for (let k = Math.min(maxUnits, list.length - i); k >= 1; k--) {
+            const span = list.slice(i, i + k);
+            if (span.some((u, j) => j > 0 && !joined(text, span[j - 1], u)))
+                continue;
+            const phrase = span.map((u) => u.text).join(' ');
+            const entries = /\d/.test(phrase) ? [] : await lookup(phrase);
+            if (entries.length || k === 1) {
+                const start = span[0].start;
+                const end = span[k - 1].end;
+                taken = { text: text.slice(start, end), lower: phrase.toLowerCase(), start, end, units: span, entries };
+                break;
+            }
+        }
+        words.push(taken);
+        i += taken.units.length;
+    }
+    return words;
+}
+// --- Rules -----------------------------------------------------------------------------------------------
+// Spelling and accents, for languages written in syllables with a syllable list: a syllable that isn't in
+// the language is flagged, with the accented forms it could be ("khong" → "không"). Plain-letter
+// syllables with no accented form are left alone: usually a name or a foreign word ("email").
+async function spelling(ctx) {
+    if (ctx.config.units !== 'syllables' || !ctx.meta.syllables)
+        return [];
+    const syllables = await ctx.dict.syllables();
+    const byPlain = new Map();
+    for (const s of Object.keys(syllables))
+        byPlain.set(plain(s), [...(byPlain.get(plain(s)) ?? []), s]);
+    for (const list of byPlain.values())
+        list.sort((a, b) => syllables[b] - syllables[a]);
+    const issues = [];
+    for (const word of ctx.sentences.flatMap((s) => s.words)) {
+        if (word.entries.length)
+            continue;
+        for (const u of word.units) {
+            const lower = u.text.toLowerCase().replace(/[’']/g, '');
+            if (/\d/.test(lower) || lower in syllables)
+                continue;
+            const candidates = (byPlain.get(plain(lower)) ?? []).slice(0, 3);
+            const accented = lower !== plain(lower);
+            if (!candidates.length && !accented)
+                continue;
+            issues.push({
+                rule: 'spelling',
+                // Missing accents on plain letters is the common learner slip; a wrong accent is more surely wrong.
+                severity: accented ? 'error' : 'warning',
+                start: u.start,
+                end: u.end,
+                text: u.text,
+                message: candidates.length
+                    ? `${quote(u.text)} isn't a ${ctx.meta.name} syllable. Did you mean ${listOf(candidates.map((c) => quote(matchCase(u.text, c))), 'or')}?`
+                    : `${quote(u.text)} isn't a ${ctx.meta.name} syllable. Check the spelling and accents.`,
+                suggestions: candidates.map((c) => matchCase(u.text, c)),
+            });
+        }
+    }
+    return issues;
+}
+// Words from another region: a word whose main meaning (its first sense) is tagged for regions that don't
+// include the text's region. With `region`, against that; without, against the regions most of the text's
+// regional words share (so "lợn" stands out in otherwise Southern text). Words with an untagged main
+// meaning are never flagged (Vietnamese "má" is "cheek" everywhere, "mother" only in the South).
+async function dialect(ctx) {
+    const all = new Set(ctx.meta.regions);
+    const regional = ctx.sentences
+        .flatMap((s) => s.words)
+        .map((word) => ({ word, sense: word.entries[0]?.senses[0] }))
+        .filter((x) => Boolean(x.sense?.regionTagged && x.sense.regions.length < all.size));
+    if (!regional.length)
+        return [];
+    const issue = (word, regions, message, suggestions, severity) => ({
+        rule: 'dialect',
+        severity,
+        start: word.start,
+        end: word.end,
+        text: word.text,
+        message: `${quote(word.text)} is ${listOf(regions)}${message}` +
+            // The best word only; the others (often niche: Southern "cúi", "ỉn" after "heo") are in suggestions.
+            (suggestions.length ? `, say ${quote(suggestions[0])}.` : '.'),
+        suggestions,
+    });
+    if (ctx.options.region) {
+        const target = resolveRegion(ctx.meta, ctx.options.region);
+        const where = listOf(ctx.meta.regions.filter((r) => target.has(r)));
+        const issues = [];
+        for (const { word, sense } of regional) {
+            if (sense.regions.some((r) => target.has(r)))
+                continue;
+            const suggestions = await ctx.suggestRegional(word, ctx.options.region);
+            issues.push(issue(word, sense.regions, `; in ${where} ${ctx.meta.name}`, suggestions, 'warning'));
+        }
+        return issues;
+    }
+    // No region given: the regions most regional words share are the text's region, and words that fit none
+    // of them stand out. When no region has a majority but some words can't go together ("lợn", Northern,
+    // with "nha", Central and Southern), every regional word is pointed out, without a guess.
+    const counts = new Map();
+    for (const { sense } of regional)
+        for (const r of sense.regions)
+            counts.set(r, (counts.get(r) ?? 0) + 1);
+    const most = Math.max(...counts.values());
+    const majority = new Set([...counts].filter(([, n]) => n === most).map(([r]) => r));
+    const outliers = regional.filter(({ sense }) => !sense.regions.some((r) => majority.has(r)));
+    if (outliers.length) {
+        const where = ctx.meta.regions.filter((r) => majority.has(r));
+        return Promise.all(outliers.map(async ({ word, sense }) => issue(word, sense.regions, `, but the rest of the text is ${listOf(where)}; there`, await ctx.suggestRegional(word, where[0]), 'suggestion')));
+    }
+    const clash = regional.some((a) => regional.some((b) => !a.sense.regions.some((r) => b.sense.regions.includes(r))));
+    if (!clash)
+        return [];
+    const mixed = listOf(regional.map(({ word, sense }) => `${quote(word.text)} (${listOf(sense.regions)})`));
+    return regional.map(({ word, sense }) => issue(word, sense.regions, `. This text mixes regions: ${mixed}. Pick one region`, [], 'suggestion'));
+}
+// Pronouns that don't fit who you're talking to (`listener`) or about (`about`): "tôi" to a parent, where
+// it's "con". Only the language's reliable pronouns are checked (CheckerConfig.pronouns); ones that are
+// also common nouns ("bạn": friend) only get a hedged suggestion.
+async function pronounRelationship(ctx) {
+    if (!ctx.options.listener && !ctx.options.about)
+        return [];
+    const rows = await ctx.rows();
+    const issues = [];
+    for (const m of ctx.pronounMatches()) {
+        const about = ABOUT_PERSONS.has(m.person);
+        const row = rows.find((r) => r.id === (about ? ctx.options.about : ctx.options.listener));
+        const fit = unique((row?.[m.person] ?? []).map((c) => c.word));
+        if (!row || !fit.length || fit.includes(m.lower))
+            continue;
+        const ambiguous = ctx.config.ambiguousPronouns?.includes(m.lower);
+        const who = `${about ? 'Talking about' : 'Talking to'} ${lowerFirst(row.label)}`;
+        issues.push({
+            rule: 'pronoun-relationship',
+            severity: ambiguous ? 'suggestion' : 'warning',
+            start: m.start,
+            end: m.end,
+            text: m.text,
+            message: (ambiguous ? `If ${quote(m.text)} means “${PERSON_NAMES[m.person]}” here: ${lowerFirst(who)}` : who) +
+                `, say ${listOf(fit.map(quote), 'or')} for “${PERSON_NAMES[m.person]}”, not ${quote(m.text)}.`,
+            suggestions: fit.map((w) => matchCase(m.text, w)),
+        });
+    }
+    return issues;
+}
+// "I" and "you" pronouns that never go together (Vietnamese "tao" goes with "mày", not "bạn"): no row of the
+// pronoun table has both. Only when no listener is given; with one, pronoun-relationship covers it.
+async function pronounPair(ctx) {
+    if (ctx.options.listener)
+        return [];
+    const matches = ctx.pronounMatches();
+    const selves = matches.filter((m) => SELF_PERSONS.has(m.person));
+    const yous = matches.filter((m) => ADDRESSEE_PERSONS.has(m.person));
+    if (!selves.length || !yous.length)
+        return [];
+    const rows = await ctx.rows();
+    const words = (r, persons) => [...persons].flatMap((p) => r[p].map((c) => c.word));
+    const issues = [];
+    for (const you of yous) {
+        for (const me of selves) {
+            const withMe = rows.filter((r) => words(r, SELF_PERSONS).includes(me.lower));
+            if (!withMe.length || withMe.some((r) => words(r, ADDRESSEE_PERSONS).includes(you.lower)))
+                continue;
+            const partners = unique(withMe.flatMap((r) => r[you.person].map((c) => c.word)));
+            issues.push({
+                rule: 'pronoun-pair',
+                severity: ctx.config.ambiguousPronouns?.includes(you.lower) ? 'suggestion' : 'warning',
+                start: you.start,
+                end: you.end,
+                text: you.text,
+                message: `${quote(me.text)} and ${quote(you.text)} don't go together` +
+                    (partners.length ? `: with ${quote(me.text)}, say ${listOf(partners.map(quote), 'or')}.` : '.'),
+                suggestions: partners.map((w) => matchCase(you.text, w)),
+            });
+            break;
+        }
+    }
+    return issues;
+}
+// Polite endings when speaking up (the listener's row is marked `respect`): Vietnamese sentences to parents,
+// elders and teachers usually end in "ạ". A suggestion, not an error: it isn't always needed.
+async function politeEnding(ctx) {
+    const endings = ctx.config.politeEndings ?? [];
+    if (!ctx.options.listener || !endings.length)
+        return [];
+    const row = (await ctx.rows()).find((r) => r.id === ctx.options.listener);
+    if (!row?.respect)
+        return [];
+    const issues = [];
+    for (const s of ctx.sentences) {
+        const last = s.units[s.units.length - 1];
+        if (endings.includes(last.text.toLowerCase()))
+            continue;
+        // Exclamations ("Con nhớ má quá!") are often fine without it.
+        if (/^[^\p{L}\p{N}]*!/u.test(ctx.text.slice(last.end)))
+            continue;
+        issues.push({
+            rule: 'polite-ending',
+            severity: 'suggestion',
+            start: last.start,
+            end: last.end,
+            text: last.text,
+            message: `Talking to ${lowerFirst(row.label)}, end the sentence with ${listOf(endings.map(quote), 'or')} to sound polite.`,
+            suggestions: endings.map((e) => `${last.text} ${e}`),
+        });
+    }
+    return issues;
+}
+const RULES = {
+    spelling,
+    dialect,
+    'pronoun-relationship': pronounRelationship,
+    'pronoun-pair': pronounPair,
+    'polite-ending': politeEnding,
+};
+const SEVERITY_ORDER = { error: 0, warning: 1, suggestion: 2 };
+export function createChecker(options = {}) {
+    const cache = new Map();
+    const dict = (lang) => {
+        let d = cache.get(lang);
+        if (!d) {
+            d = options.dictionary?.(lang) ?? createDictionary({ lang, baseUrl: options.baseUrl?.(lang), load: options.load?.(lang) });
+            cache.set(lang, d);
+        }
+        return d;
+    };
+    const translator = createTranslator({ ...options, dictionary: dict });
+    const lookups = new Map();
+    return {
+        async check(input, opts) {
+            const text = input.normalize('NFC');
+            const d = dict(opts.lang);
+            const [meta, config] = await Promise.all([d.meta(), d.checker()]);
+            // Unknown listener/about ids throw, listing the valid ones (as the translator does).
+            if (meta.pronouns) {
+                if (opts.listener)
+                    await d.pronouns({ listener: opts.listener });
+                if (opts.about)
+                    await d.pronouns({ listener: opts.about });
+            }
+            const lookup = (phrase) => {
+                const key = `${opts.lang}\u0000${phrase}`;
+                let pending = lookups.get(key);
+                if (!pending) {
+                    pending = d.lookup(phrase);
+                    lookups.set(key, pending);
+                }
+                return pending;
+            };
+            const sentences = [];
+            for (const list of sentencesOf(text, units(text))) {
+                sentences.push({ units: list, words: await segment(text, list, config.maxWordUnits ?? 3, lookup) });
+            }
+            let rows;
+            let matches;
+            const ctx = {
+                text,
+                options: opts,
+                meta,
+                config,
+                sentences,
+                dict: d,
+                rows: () => (rows ??= meta.pronouns ? d.pronouns({ region: opts.region, speaker: opts.speaker }) : Promise.resolve([])),
+                // The language's reliable pronouns in the text, longest first ("chúng tôi" before "tôi"), matched on
+                // units directly so pronouns the dictionary doesn't list as one word ("tụi mày") are found too.
+                pronounMatches: () => (matches ??= findPronouns(text, sentences, config)),
+                suggestRegional: async (word, region) => {
+                    try {
+                        const [group] = await translator.translate(word.lower, {
+                            from: opts.lang, to: opts.lang, toRegion: region, pos: word.entries[0]?.pos, limit: 3,
+                        });
+                        return (group?.translations ?? []).map((t) => t.word).filter((w) => w !== word.lower);
+                    }
+                    catch {
+                        return [];
+                    }
+                },
+            };
+            const results = await Promise.all(Object.keys(RULES).filter((r) => opts.rules?.[r] !== false).map((r) => RULES[r](ctx)));
+            const issues = results.flat().sort((a, b) => a.start - b.start || SEVERITY_ORDER[a.severity] - SEVERITY_ORDER[b.severity]);
+            return { text, issues };
+        },
+    };
+}
+function findPronouns(text, sentences, config) {
+    const byWord = new Map();
+    for (const person of PRONOUN_PERSONS)
+        for (const w of config.pronouns?.[person] ?? [])
+            byWord.set(w.toLowerCase(), person);
+    if (!byWord.size)
+        return [];
+    const longest = Math.max(...[...byWord.keys()].map((w) => w.split(' ').length));
+    const out = [];
+    sentences.forEach((s, sentence) => {
+        for (let i = 0; i < s.units.length;) {
+            let step = 1;
+            for (let k = Math.min(longest, s.units.length - i); k >= 1; k--) {
+                const span = s.units.slice(i, i + k);
+                if (span.some((u, j) => j > 0 && !joined(text, span[j - 1], u)))
+                    continue;
+                const lower = span.map((u) => u.text.toLowerCase()).join(' ');
+                const person = byWord.get(lower);
+                if (person) {
+                    out.push({ text: text.slice(span[0].start, span[k - 1].end), lower, start: span[0].start, end: span[k - 1].end, person, sentence });
+                    step = k;
+                    break;
+                }
+            }
+            i += step;
+        }
+    });
+    return out;
+}

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { type PronounRow, type Entry, type Hit, type LanguageMeta } from './types.ts';
+import { type CheckerConfig, type PronounRow, type Entry, type Hit, type LanguageMeta, type PickRow } from './types.ts';
 export * from './types.ts';
 export * from './pos.ts';
 /**
@@ -65,6 +65,12 @@ export type Dictionary = {
      * for languages without a pronoun table.
      */
     pronouns(options?: PronounOptions): Promise<PronounRow[]>;
+    /** Hand-picked first choices for English meanings (see PickRow). Empty for languages without any. */
+    picks(): Promise<PickRow[]>;
+    /** How the checker reads this language (see CheckerConfig); `{}` for languages without settings. */
+    checker(): Promise<CheckerConfig>;
+    /** Syllable → Zipf frequency (0 when unknown), for languages written in syllables; `{}` otherwise. */
+    syllables(): Promise<Record<string, number>>;
 };
 /**
  * The regions a region or region-group name stands for (e.g. English "North America" -> US, Canada),
@@ -74,3 +80,4 @@ export declare function resolveRegion(meta: LanguageMeta, region: string | undef
 export declare function dataUrl(lang: string): string;
 export declare function createDictionary(options: DictionaryOptions): Dictionary;
 export * from './translate.ts';
+export * from './check.ts';

package/dist/index.js

@@ -103,6 +103,18 @@ export function createDictionary(options) {
             const kept = new Set(words.map((h) => h.word));
             return ranked.filter((h) => kept.has(h.word));
         },
+        async picks() {
+            const m = await meta();
+            return m.picks ? loadOnce('picks.json') : [];
+        },
+        async checker() {
+            const m = await meta();
+            return m.checker ? loadOnce('checker.json') : {};
+        },
+        async syllables() {
+            const m = await meta();
+            return m.syllables ? loadOnce('syllables.json') : {};
+        },
         async pronouns({ listener, region, speaker, exclude = DEFAULT_EXCLUDED_LABELS } = {}) {
             const m = await meta();
             if (!m.pronouns)
@@ -128,3 +140,4 @@ export function createDictionary(options) {
     };
 }
 export * from "./translate.js";
+export * from "./check.js";

package/dist/translate.d.ts

@@ -27,8 +27,9 @@ export type Translation = Hit & {
     score: number;
     /**
      * How this word was found: the English term that linked it, "synonym" for a same-language synonym,
-     * "table" for Wiktionary's translation table, or "pronouns" for a pronoun-table row's words (the
-     * `listener`'s or `about`'s row, or the default row).
+     * "table" for Wiktionary's translation table, "picked" for the target language's hand-picked words
+     * for this meaning (see PickRow), or "pronouns" for a pronoun-table row's words (the `listener`'s or
+     * `about`'s row, or the default row).
      */
     bridge: string;
     /** Example sentences for this word in the matched sense, when the dictionary has them. */
@@ -124,6 +125,11 @@ export type TranslateOptions = {
      * translations repeat another sense's. Default false. Useful for letting users pick a meaning.
      */
     allSenses?: boolean;
+    /**
+     * Use the target language's hand-picked words (default true). false returns the ranking alone, e.g. to
+     * see what picks change.
+     */
+    picks?: boolean;
 };
 export type Translator = {
     /** The senses of a word, following forms and variant spellings to the words they belong to. */

package/dist/translate.js

@@ -5,7 +5,7 @@
 // for those terms with a compatible part of speech, in the target region. Each source sense is translated
 // on its own, so a word's meanings don't get mixed up ("cool" the temperature vs. "cool" the compliment).
 import { compatiblePos } from "./pos.js";
-import { displayGloss, glossTerms, normalizeEnglish, PRONOUN_PERSONS, } from "./types.js";
+import { displayGloss, glossTerms, normalizeEnglish, PRONOUN_PERSONS, TRAILING_PARTICLE, } from "./types.js";
 import { createDictionary, DEFAULT_EXCLUDED_LABELS, resolveRegion, } from "./index.js";
 /**
  * Possible base words for a regularly inflected English word, most likely first: "walked" -> walk,
@@ -118,9 +118,61 @@ const MIN_SCORE = 1;
 // A variant that's only another spelling of a different word ("yeah": pronunciation spelling of "year")
 // ranks below the word's own meanings.
 const SPELLING_VIA = /pronunciation spelling|misspelling|eye dialect|nonstandard spelling|dated spelling|informal spelling/i;
+// Among a form's own meanings, one that needs a following construction ("Must; have/has (to).", "used
+// with have", "followed by …") is narrower than a standalone one ("got": "Have/has." comes first). Only
+// there: function words' main meanings are constructions ("would": "Past tense of will; usually
+// followed by a bare infinitive"), and demoting them everywhere put "would" → "hoàn cảnh" first.
+const CONSTRUCTION = /\((?:to|with [^)]*|followed by [^)]*)\)|\bused with\b|\bfollowed by\b/i;
 // Senses whose labels are all in this set rank after mainstream senses.
 const MARGINAL_LABELS = new Set(['informal', 'slang', 'colloquial', 'dialectal', 'rare', 'nonstandard', 'Internet', 'humorous']);
+// How a form reached its base word, when it's an inflection ("simple past of get", "form of run").
+const INFLECTION_VIA = /\b(?:past|participle|plural|singular|present|third-person|comparative|superlative|gerund)\b|^form of /i;
+// Everyday speech, not slang: a form's own colloquial meaning ("got": "Have/has.") isn't marginal.
+const EVERYDAY_LABELS = new Set(['informal', 'colloquial']);
 const LABEL_PENALTY = new Set(['slang', 'Internet', 'humorous', 'literary', 'dialectal', 'uncommon', 'rare', 'euphemistic']);
+// A target gloss that lists the bridge term itself as an equivalent ("muốn": "to want"; "má": "mother;
+// mom") is a plain translation, but a short gloss can't earn definition overlap, since the term's own
+// words don't count. Wordier glosses did ("mắc": "to want to (go to the bathroom, laugh, etc.); to have
+// the desire to" overlapped "desire"), so a narrow use beat the everyday word. An exact equivalent
+// scores like one overlapping word, for main terms only: a side term's exact match ("corn" → "cereal" →
+// "ngũ cốc") is a looser equivalent.
+const EXACT_GLOSS_BONUS = 3;
+// The term narrowed by a parenthetical right after it ("to want to (go to the bathroom …)") is a
+// restricted use of the word, not its general meaning (see narrows()).
+const NARROWED_PENALTY = 1.5;
+// A parenthetical narrows a term when it names something the source meaning doesn't: "to want to (go to
+// the bathroom, laugh, etc.)", "in/at (an outdoor or external location)". Restatements ("ngầu": "cool
+// (awesome; fantastic; great)" for cool = awesome), grammar ("to wait (for)") and usage notes, which say
+// how the word is used rather than what it means ("dạ": "yes (answers a question)"), don't narrow.
+const USAGE_NOTE = /^\s*(?:used|said|\w+s\s+(?:a|an|the|that|for|to|with|about)\b)/i;
+function narrows(parenthetical, context) {
+    return contentWords(parenthetical).size > 0 && !USAGE_NOTE.test(parenthetical) && overlap(context, parenthetical) === 0;
+}
+// A target sense marked passive ("được": "to obtain, to get (passively)") is a different voice from an
+// active source meaning ("get": "To obtain; to acquire", where "lấy" is the everyday word). Sized to
+// offset a very common word's frequency: "được" is common mostly as a helper verb ("được đi", "làm
+// được"), and frequency is per word, not per meaning.
+const PASSIVE = /\bpassive(?:ly)?\b/i;
+const VOICE_MISMATCH_PENALTY = 3;
+function exactGloss(gloss, term, context) {
+    // "to wait for" is plain "wait", but a part with a narrowing parenthetical isn't ("to want to (…)").
+    // Like glossTerms, only the text after a colon is the meaning ("Used as an answer …: no"). A question
+    // ("dạ": "yes? (asks for more information)") isn't the plain word.
+    let cleaned = gloss.replace(/\(([^)]*)\)/g, (_, inner) => (narrows(inner, context) ? '(' : ' '));
+    if (cleaned.includes(':'))
+        cleaned = cleaned.slice(cleaned.lastIndexOf(':') + 1);
+    return cleaned.split(/[;,/]/).some((part) => {
+        if (part.includes('('))
+            return false;
+        const t = normalizeEnglish(part.replace(/[.!]+\s*$/, ''));
+        return t === term || t.replace(TRAILING_PARTICLE, '') === term;
+    });
+}
+function narrowed(gloss, term, context) {
+    const escaped = term.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+    const m = new RegExp(`\\b${escaped}\\b(?:\\s+to)?\\s*\\(([^)]*)`, 'i').exec(gloss);
+    return m !== null && narrows(m[1], context);
+}
 // Register: a polite source word should translate to a polite word, a casual one to a casual one.
 const POLITE = new Set(['polite', 'formal', 'honorific', 'literary']);
 const CASUAL = new Set(['informal', 'colloquial', 'slang', 'familiar', 'vulgar', 'Internet']);
@@ -147,16 +199,25 @@ function registerFit(source, target, wanted) {
 // what the word means. Without the jargon, Vietnamese "con" ("used when talking to someone older than
 // the speaker") matched English "I" ("the speaker or writer") on "speaker".
 const STOPWORDS = new Set(('a an the to of or and in on for with by as at from that this be is are was it its used use any some one ' +
-    'something someone very more most usually especially often also who which what when being having into ' +
+    'something someone very more most usually especially often also who which what when being have has having into ' +
     'speaker speakers writer referred refer refers referring grammatical subject object sentence clause ' +
     'word words form forms term expression phrase talking speaking addressed addressing person people ' +
     'particular certain various generally sense usage meaning').split(' '));
-function contentWords(text) {
+function contentWords(text, keep) {
     return new Set(text
         .toLowerCase()
         .replace(/[^a-z' ]+/g, ' ')
         .split(' ')
-        .filter((w) => w.length > 2 && !STOPWORDS.has(w)));
+        .filter((w) => w.length > 2 && (!STOPWORDS.has(w) || keep?.has(w))));
+}
+// The caller's `meaning` words all count, even ones too common to count in definitions ("have" picks
+// "got" = "To have", not "To obtain").
+function meaningOverlap(words, text) {
+    let n = 0;
+    for (const w of contentWords(text, words))
+        if (words.has(w))
+            n++;
+    return n;
 }
 function overlap(a, text) {
     let n = 0;
@@ -187,10 +248,15 @@ function bridgeTerms(sense, from) {
     }
     // Headings ("As a copulative verb:") carry no meaning, so the specific definitions are used.
     const glosses = sense.glosses.filter((g) => !g.trim().endsWith(':'));
+    // A first definition that is only a short list of equivalents ("To fetch, bring, take.") names the
+    // meaning with every item equally, so all of them are main terms ("take" → "lấy" scored no position
+    // bonus, and words found only through the ambiguous "get" itself came first: hóng, bắt, ra khỏi).
     glosses.slice(0, 2).forEach((gloss, gi) => {
-        for (const [term, position] of glossTerms(gloss)) {
+        const found = glossTerms(gloss);
+        const list = gi === 0 && isTermList(gloss, found.length);
+        for (const [term, position] of found) {
             terms.push(term);
-            if (gi === 0 && position === 0)
+            if (gi === 0 && (position === 0 || list))
                 main.add(term);
         }
     });
@@ -206,6 +272,11 @@ function bridgeTerms(sense, from) {
     }
     return { terms: unique.slice(0, 8), synonymsFrom, main };
 }
+// Every comma/semicolon part of the definition became a term, and each is at most three words.
+function isTermList(gloss, termCount) {
+    const parts = gloss.replace(/\.\s*$/, '').split(/[;,]/);
+    return termCount >= 2 && parts.length === termCount && parts.every((p) => p.trim().split(/\s+/).length <= 3);
+}
 export function createTranslator(options = {}) {
     const cache = new Map();
     const dict = (lang) => {
@@ -351,34 +422,37 @@ export function createTranslator(options = {}) {
         }
         return hits;
     }
-    // The translation-table words for a source sense in the target language. English senses carry their
-    // table; for other languages, the English sense whose table lists the source word (in the source
-    // language) is the same meaning, and its target-language list applies.
-    async function tableFor(sense, from, to, bridge) {
+    async function englishMeanings(sense, from, bridge) {
         if (from === 'en')
-            return to === 'en' ? [] : (sense.translations?.[to] ?? []);
+            return [{ word: sense.lemma, pos: sense.pos, gloss: sense.glosses[0] ?? '', translations: sense.translations }];
         const found = [];
-        const add = (list) => {
-            for (const t of list ?? [])
-                if (!found.some((f) => f.word === t.word))
-                    found.push(t);
-        };
         for (const term of bridge.slice(0, 3)) {
             for (const entry of await dict('en').lookup(term)) {
                 for (const s of entry.senses) {
                     const own = s.translations?.[from];
                     if (!own?.some((t) => t.word === sense.lemma || t.word === sense.word))
                         continue;
-                    // Into English, the English word whose table lists the source word is itself the translation.
-                    if (to === 'en')
-                        add([{ word: entry.word }]);
-                    else
-                        add(s.translations?.[to]);
+                    if (!found.some((f) => f.word === entry.word && f.gloss === s.glosses[0])) {
+                        found.push({ word: entry.word, pos: entry.pos, gloss: s.glosses[0], translations: s.translations });
+                    }
                 }
             }
         }
         return found;
     }
+    async function tableFor(sense, from, to, bridge) {
+        if (from === 'en')
+            return to === 'en' ? [] : (sense.translations?.[to] ?? []);
+        const found = [];
+        for (const m of await englishMeanings(sense, from, bridge)) {
+            // Into English, the English word whose table lists the source word is itself the translation.
+            const list = to === 'en' ? [{ word: m.word }] : (m.translations?.[to] ?? []);
+            for (const t of list)
+                if (!found.some((f) => f.word === t.word))
+                    found.push(t);
+        }
+        return found;
+    }
     // Which of the target's regions a table tag names ("Latin-America" -> its countries), if any.
     function tagRegions(tags, meta) {
         const out = new Set();
@@ -480,8 +554,12 @@ export function createTranslator(options = {}) {
     }
     const api = {
         senses,
-        async translate(word, { from, to, fromRegion, toRegion, pos, meaning, register: wantedRegister, listener, about, speaker, exclude = DEFAULT_EXCLUDED_LABELS, limit = 5, allSenses = false }) {
+        async translate(word, { from, to, fromRegion, toRegion, pos, meaning, register: wantedRegister, listener, about, speaker, exclude = DEFAULT_EXCLUDED_LABELS, limit = 5, allSenses = false, picks: usePicks = true }) {
             const [sourceSenses, toMeta, fromMeta] = await Promise.all([senses(word, { from, fromRegion, pos }), dict(to).meta(), dict(from).meta()]);
+            // The target language's hand-picked words for English meanings (picks.json), when it has any.
+            const pickRows = usePicks && to !== 'en' && toMeta.picks ? await dict(to).picks() : [];
+            // Senses whose pick row asks to come first when the caller gives no pos or meaning.
+            const pickedFirst = new Set();
             // The target's pronoun table, for personal-pronoun senses (throws for an unknown listener/about).
             const pronounRows = toMeta.pronouns && sourceSenses.some((s) => personOf(s, from))
                 ? await dict(to).pronouns({ region: toRegion, speaker, exclude })
@@ -492,7 +570,7 @@ export function createTranslator(options = {}) {
                 await dict(to).pronouns({ listener: about });
             const toWanted = resolveRegion(toMeta, toRegion);
             const hasFrequencies = Boolean(toMeta.frequencySource);
-            const meaningWords = meaning ? contentWords(meaning) : null;
+            const meaningWords = meaning ? contentWords(meaning, new Set(meaning.toLowerCase().split(/[^a-z']+/))) : null;
             const groups = [];
             for (const sense of sourceSenses) {
                 const { terms: bridge, synonymsFrom, main } = bridgeTerms(sense, from);
@@ -523,6 +601,7 @@ export function createTranslator(options = {}) {
                     // The term's own words say nothing about which meaning matched, so they don't count as overlap.
                     const termWords = contentWords(term);
                     const ctx = new Set([...context].filter((w) => !termWords.has(w)));
+                    const isMain = main.has(term) || termIndex === 0;
                     const hits = to === 'en'
                         ? await englishCandidates(term, posList, toMeta, toRegion, exclude)
                         // A generous limit: the dictionary's quick ranking puts region-tagged words first, and the
@@ -546,7 +625,10 @@ export function createTranslator(options = {}) {
                         // less (common words win anyway), and kinship words list "you" after "I/me" (Vietnamese "anh"),
                         // so the penalty is small then.
                         const score = 3 * overlap(ctx, hit.gloss) +
-                            (main.has(term) || termIndex === 0 ? 2 : termIndex === 1 ? 1 : 0) +
+                            (isMain && exactGloss(hit.gloss, term, ctx) ? EXACT_GLOSS_BONUS : 0) -
+                            (narrowed(hit.gloss, term, ctx) ? NARROWED_PENALTY : 0) -
+                            (PASSIVE.test(hit.gloss) && !PASSIVE.test(sense.glosses.join(' ')) ? VOICE_MISMATCH_PENALTY : 0) +
+                            (isMain ? 2 : termIndex === 1 ? 1 : 0) +
                             (hit.primary ? 1 : 0) +
                             // Tagged for the target region: a full bonus when the match is the word's main meaning, a
                             // small one otherwise ("bá cháy" is Southern for "awesome", not for "cool" the temperature).
@@ -608,6 +690,36 @@ export function createTranslator(options = {}) {
                             scored.set(t.word, { ...t, score: t.score + 4, bridge: 'synonym' });
                     }
                 }
+                // Hand-picked words for this meaning go first, in the listed order: a speaker's judgment of the
+                // natural word where the ranking puts another first. With a target region, picks tagged for it
+                // lead, and picks tagged only for other regions (or words not used there) are left to the
+                // ranking. The ranked words follow. (A pronoun-table row, below, still comes before them.)
+                const rows = pickRows.length
+                    ? await englishMeanings(sense, from, bridge).then((ms) => pickRows.filter((r) => ms.some((m) => m.word === r.word && m.pos === r.pos && m.gloss === r.gloss)))
+                    : [];
+                if (rows.some((r) => r.first))
+                    pickedFirst.add(sense);
+                const picked = [];
+                for (const t of rows.flatMap((r) => r.picks)) {
+                    if (picked.some((p) => p.hit.word === t.word))
+                        continue;
+                    const places = tagRegions(t.tags, toMeta);
+                    if (toWanted && places && ![...places].some((r) => toWanted.has(r)))
+                        continue;
+                    if (to === from && t.word === sense.lemma && !places?.size)
+                        continue;
+                    const hit = scored.get(t.word) ?? (await tableHit(t.word, to, posList, toWanted, exclude, bridge));
+                    if (!hit || hit.labels.some((l) => exclude.includes(l)))
+                        continue;
+                    if (!places && !intersects(hit.regions, toWanted))
+                        continue;
+                    picked.push({ hit, tagged: Boolean(toWanted && places) });
+                }
+                if (picked.length) {
+                    const ordered = [...picked.filter((p) => p.tagged), ...picked.filter((p) => !p.tagged)];
+                    const top = Math.max(0, ...[...scored.values()].map((t) => t.score));
+                    ordered.forEach(({ hit }, i) => scored.set(hit.word, { ...hit, score: top + 1 + (ordered.length - i) * 0.1, bridge: 'picked' }));
+                }
                 // Personal pronouns: the chosen row's words go first (the `listener`'s for I/you/we, the `about`'s
                 // for he/she/they, else the default row's), fitted to the pronoun's gender (he/she) and
                 // inclusiveness (we). Words tagged for the target region, then those fitting the register, lead.
@@ -706,15 +818,34 @@ export function createTranslator(options = {}) {
                 const chosen = ABOUT_PERSONS.has(p.person) ? about : listener;
                 return (chosen ? 10 : 0) - (hasSingular && !SINGULAR_PERSONS.has(p.person) ? 3 : 0);
             };
+            // A form with its own entries ("got": "Have/has.", colloquial) as well as its base word's ("get"):
+            // the form's own meanings are why it has entries at all, so they stay ahead of the base word's, and
+            // a colloquial/informal label doesn't push them behind (slang and Internet labels still do: "felt"
+            // "To thoroughly defeat", Internet). A construction ("Must; have/has (to).") then only reorders
+            // them among themselves.
+            // Only inflections ("got": simple past of get), not contractions ("kinda") or spelling variants.
+            const formWithBase = ownSenses && sourceSenses.some((s) => s.via && INFLECTION_VIA.test(s.via));
+            const labelRelevance = (g) => {
+                const { labels } = g.source;
+                const marginal = labels.length > 0 && labels.every((l) => MARGINAL_LABELS.has(l));
+                const everyday = labels.length > 0 && labels.every((l) => EVERYDAY_LABELS.has(l));
+                if (formWithBase && !g.source.via && (!marginal || everyday)) {
+                    return 1 - (CONSTRUCTION.test(g.source.glosses.join(' ')) ? 0.5 : 0);
+                }
+                return marginal ? -1 : 0;
+            };
             const relevance = (g) => (meaningWords
-                ? 10 * overlap(meaningWords, g.source.glosses.join(' ')) + 5 * overlap(meaningWords, g.translations[0]?.gloss ?? '')
+                ? 10 * meaningOverlap(meaningWords, g.source.glosses.join(' ')) + 5 * meaningOverlap(meaningWords, g.translations[0]?.gloss ?? '')
                 : 0) -
                 (ownSenses && g.source.via && SPELLING_VIA.test(g.source.via) ? 2 : 0) +
                 (fromRegion && g.source.regionTagged ? 3 : 0) +
-                pronounRelevance(g) -
-                (g.source.labels.length && g.source.labels.every((l) => MARGINAL_LABELS.has(l)) ? 1 : 0);
+                pronounRelevance(g) +
+                labelRelevance(g) +
+                (!meaning && pickedFirst.has(g.source) ? 5 : 0);
+            // A sense with no translations says nothing, so it goes after the ones that do ("got": "Expressing
+            // obligation; used with have." finds nothing; "Must; have/has (to)." gives "phải").
             const ordered = groups
-                .map((g, i) => ({ g, i, r: relevance(g) }))
+                .map((g, i) => ({ g, i, r: relevance(g) - (g.translations.length ? 0 : 100) }))
                 .sort((a, b) => b.r - a.r || a.i - b.i)
                 .map((x) => x.g);
             if (allSenses)

package/dist/types.d.ts

@@ -108,6 +108,51 @@ export type LanguageMeta = {
     shardLength?: number;
     /** true when the language has a pronoun table (pronouns.json, see PronounRow). */
     pronouns?: boolean;
+    /** true when the language has hand-picked translations (picks.json, see PickRow). */
+    picks?: boolean;
+    /** true when the language has checker settings (checker.json, see CheckerConfig). */
+    checker?: boolean;
+    /** true when the language has a syllable list (syllables.json: syllable → Zipf frequency, 0 when unknown). */
+    syllables?: boolean;
+};
+/**
+ * How the checker reads one language (languages/<lang>.ts `checker`, stored as data/checker.json). Every
+ * field is optional: a language without it is read as space-separated words, and gets the checks that
+ * only need its dictionary (dialect mixing).
+ */
+export type CheckerConfig = {
+    /**
+     * What spaces separate: 'words' (English), or 'syllables' that group into dictionary words (Vietnamese
+     * "thịt heo", "kết quả"). With 'syllables' the build also writes syllables.json for accent suggestions.
+     */
+    units?: 'words' | 'syllables';
+    /** The longest dictionary word to look for, in units (default 3). */
+    maxWordUnits?: number;
+    /**
+     * Pronouns that are reliably pronouns, by pronoun-table column ("tôi", "tao", "mày"). The pronoun checks
+     * look only at these: kinship words like Vietnamese "con" (also "child", a classifier) aren't listed.
+     */
+    pronouns?: Partial<Record<PronounPerson, string[]>>;
+    /** Listed pronouns that are also common nouns ("bạn": you, or friend); checked, but only as suggestions. */
+    ambiguousPronouns?: string[];
+    /** Sentence endings expected when speaking up (pronoun-table rows marked `respect`), e.g. Vietnamese "ạ". */
+    politeEndings?: string[];
+};
+/**
+ * Hand-picked first choices for one English meaning, in one target language (picks.json). A speaker's
+ * judgment of the natural word where the data ranks another first ("get", "To fetch, bring, take" →
+ * Vietnamese "lấy", not "đưa"). Keyed by English meaning, so each language needs one list, and
+ * translating between two other languages reaches it through the English meaning.
+ */
+export type PickRow = {
+    /** The English headword, part of speech and first definition of the meaning, as in the English data. */
+    word: string;
+    pos: string;
+    gloss: string;
+    /** Best first. `tags` name regions like translation-table tags ("Southern"). */
+    picks: TableTranslation[];
+    /** Put this meaning first when the caller gives no `pos` or `meaning` ("got" → "Have/has."). */
+    first?: boolean;
 };
 /**
  * One word in the pronoun table: how to say "I", "you", "he/she", "we", plural "you" or "they" in one
@@ -163,6 +208,8 @@ export type PronounRow = {
     thirdPlural: PronounChoice[];
     /** The row to use when the relationship isn't known (neutral words: Vietnamese "tôi", "bạn"). */
     default?: boolean;
+    /** Speaking up to this person (parents, elders, teachers): polite sentence endings are expected. */
+    respect?: boolean;
     /** A caution for the whole row, e.g. that "tao"/"mày" are rude outside close friendships. */
     warning?: string;
 };
@@ -218,6 +265,7 @@ export declare function normalizeEnglish(term: string): string;
  * headings ending in ":" are skipped in favor of the specific definition.
  */
 export declare function displayGloss(glosses: string[]): string;
+export declare const TRAILING_PARTICLE: RegExp;
 /**
  * Splits an English gloss into search terms, main meaning first: "now, today, this time" ->
  * [["now", 0], ["today", 1], ["this time", 2]]. Parenthesized notes and final punctuation are dropped.

package/dist/types.js

@@ -55,7 +55,7 @@ export function displayGloss(glosses) {
     return glosses.find((g) => !g.trim().endsWith(':')) ?? glosses[0] ?? '';
 }
 // Trailing words dropped to also match the bare verb: "wait for" is found by "wait" too.
-const TRAILING_PARTICLE = / (for|to|at|on|with|about|of|in|into|up|out|off|over)$/;
+export const TRAILING_PARTICLE = / (for|to|at|on|with|about|of|in|into|up|out|off|over)$/;
 /**
  * Splits an English gloss into search terms, main meaning first: "now, today, this time" ->
  * [["now", 0], ["today", 1], ["this time", 2]]. Parenthesized notes and final punctuation are dropped.
@@ -72,8 +72,11 @@ export function glossTerms(gloss) {
         cleaned = cleaned.slice(cleaned.lastIndexOf(':') + 1);
     const terms = [];
     let position = 0;
-    // "I/me" lists two meanings, like "I; me".
-    for (const part of cleaned.split(/[;,/]/)) {
+    // "I/me" lists two meanings, like "I; me". Sentences too: "To have. See usage notes." means "have",
+    // and cross-references ("See usage notes", "Compare shall", "Synonym: …") aren't meanings.
+    for (const part of cleaned.split(/[;,/]|\.\s+/)) {
+        if (/^\s*(?:see|compare|cf|synonyms?|antonyms?)\b/i.test(part))
+            continue;
         // "etc." is not a meaning: "walking etc" -> "walking", and a lone "etc" is skipped.
         const t = normalizeEnglish(part.replace(/[.!?]+\s*$/, '').replace(/\betc\.?$/i, '').trim());
         if (!t || t.split(' ').length > 4 || !/^[a-z][a-z' -]*$/.test(t))

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "which-dialect",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Translate words between languages and dialects, meaning by meaning, starting with Vietnamese (Northern, Central, Southern) and English, with Wiktionary data loaded on demand. More languages are planned.",
   "type": "module",
   "main": "./dist/index.js",