which-dialect 0.2.0 → 0.3.0

package/README.md

@@ -143,12 +143,60 @@ The checks:
 - **`dialect`**: words whose main meaning is tagged for another region (*lợn* in Southern text → *heo*; *muỗng* in Northern → *thìa*; in English, US *apartment* in UK text → *flat*). Words with an untagged main meaning are never flagged (*má* is "cheek" everywhere).
 - **`pronoun-relationship`**: pronouns that don't fit the listener (*tôi* to a parent → *con*; to a teacher → *em*).
 - **`pronoun-pair`**: "I" and "you" pronouns that never go together (*tao* goes with *mày*, not *bạn*), when no listener is given.
+- **`pronoun-consistency`**: switching words for the same person within one text (*tôi* in one sentence, *tui* in the next): keep the first.
 - **`polite-ending`**: a suggestion to end sentences with *ạ* when talking to parents, elders and teachers (not for exclamations).
 
 The checker only speaks up when it's confident, so **no issues means none of the checks found anything, not that the text is correct**. It doesn't check word order, classifiers or whether a sentence sounds natural yet. Only reliable pronouns are checked: kinship words like *con* (also "child", and a classifier in *con chó*) aren't, and *bạn* (also "friend") only gets suggestions.
 
 **Every language uses the same checks.** They read only the language's data and its `checker` settings in `languages/<code>.ts` (how text splits into words, which pronouns are reliable, polite endings), so a check whose data a language doesn't have is skipped. A language without checker settings still gets the `dialect` check.
 
+## Sentence frames
+
+Common sentences with slots, filled for a region, a listener and a register: *Where is {place}?* is *{place} ở **đâu**?*, and in Central Vietnamese *{place} ở **mô**?*; *Have you eaten yet?* to a parent in the South is *Ba ăn cơm chưa ạ?*, to a friend *Bạn ăn cơm chưa?*.
+
+```ts
+import { createPhrasebook } from 'which-dialect'
+
+const pb = createPhrasebook()
+await pb.frames({ lang: 'vi', region: 'Southern', topic: 'questions' })       // every frame, slots open
+await pb.render('ask-where', { lang: 'vi', region: 'Central', slots: { place: 'bathroom' } })
+// → { text: 'Phòng tắm ở mô?', parts: [{ text: 'Phòng tắm', kind: 'content', from: 'bathroom' }, …,
+//      { text: 'mô', kind: 'word', why: 'where (Central)' }, …], complete: true }
+await pb.match('Do you want coffee?', { lang: 'vi' })                         // → 'Bạn có muốn cà phê không?'
+await pb.matchFrames('Bạn có đi?', { lang: 'vi' })                            // → 'Bạn có đi không?' (missing "không")
+```
+
+Options: `lang`, `region`, `listener` (a pronoun-table row id; without it, the neutral default row), `speaker`, `register` (`'casual'` allows colloquial words like Southern *hông*, *tui*). Content slots take English (translated for you, in the region) or `{ text }` in the language itself. `parts` says why each word was chosen, for highlighting. `matchFrames` recognizes a frame in a sentence in the language itself and returns the changes the frame suggests: a missing required word (*không*), a word from another region (*mô* in Southern text → *đâu*), missing accents in a frame position (*Toi* → *Tôi*), and, only when a listener is given, a pronoun that doesn't fit them.
+
+The frames themselves are written once, in English, in [`languages/frames.ts`](languages/frames.ts) (34 so far: journal sentences, feelings, questions, requests, greetings). Each language says how it expresses them (`frames` in `languages/<code>.ts`), with pronoun slots (`{I}`, `{YOU}`, `{WE_INCL}`, …, filled from the pronoun table) and **frame words** for what changes by region (`frameWords`: Vietnamese `{WHERE}` *đâu* / Central *mô*, `{HOW}` Northern *thế nào* / *sao* / Central *răng*, `{SOFT}` Northern *nhé* / Central and Southern *nha*, `{Q_END}` *không* / Southern casual *hông*, `{POLITE}` *ạ* when speaking up). Each frame word comes from a dictionary definition, keeping its region and labels, or from an override with a note, like the pronoun table. A language without frames returns none.
+
+## Reviewing journal entries
+
+`createReviewer().review(entry, { lang, base?, region?, listener?, speaker?, register? })` reviews an entry written in the target language, the learner's own language (`base`, default `'en'`), or a mix of both, and returns the corrected entry sentence by sentence:
+
+```ts
+import { createReviewer } from 'which-dialect'
+
+const review = await createReviewer().review(
+  'Hôm nay tôi đi market với má. Tôi muốn ăn thịt lợn but it was expensive. Ngày mai tui đi lại khong?',
+  { lang: 'vi', region: 'Southern' },
+)
+review.corrected
+// 'Hôm nay tôi đi chợ với má. Tôi muốn ăn thịt heo nhưng mắc quá. Ngày mai tôi đi lại không?'
+review.sentences[1].changes
+// [{ from: 'but it was expensive', to: 'nhưng mắc quá', kind: 'frame', why: 'Sentence frame “but it was {quality}”' },
+//  { from: 'lợn', to: 'heo', kind: 'dialect', why: '“lợn” is Northern; in Southern Vietnamese, say “heo”.' }]
+```
+
+For each sentence:
+
+1. **Each word is marked as the target or the base language** (`parts`), from the two dictionaries alone. Words that could be either (*an*, *to*) take their neighbors' language; an English word that's Vietnamese only with accents added (*but*: *bút*) stays English, while Vietnamese typed without accents (*khong*) stays Vietnamese.
+2. **Base-language parts are replaced:** a whole sentence or clause that follows a sentence frame is filled in (*but it was expensive* → *nhưng mắc quá*; *Where is the bathroom?* → *Phòng tắm ở đâu?*), and words or short phrases (up to 3 words) are translated in the region (*market* → *chợ*). Anything longer is left as written and listed in `unchecked`: the review doesn't guess.
+3. **The grammar checker runs on the whole entry**, so consistency checks see every sentence (*tôi* earlier, *tui* later). Its errors and warnings are applied (`changes`); its suggestions, like ending with *ạ*, are returned as `hints`.
+4. **The corrected sentence is checked against sentence frames** (*Bạn có đi chợ?* → *Bạn có đi chợ không?*), and the frames it follows are listed (`frames`), for suggesting patterns to reuse.
+
+Each sentence has `original`, `parts`, `corrected`, `changes` (`{ from, to, why, kind }`), `hints`, `frames` (`{ id, en, text }`, the frame's English and its pattern) and `unchecked`. For a journal, leave `listener` out: people written about (*má*, *thầy*) aren't treated as the listener. Rules can't fix everything: scrambled word order, missing verbs, or English sentences no frame covers come back as written, and plain-letter words that are also Vietnamese words (*toi*, *di*) are only fixed when a sentence frame shows what they should be.
+
 ## Looking words up
 
 ### `createDictionary({ lang, baseUrl?, load? })`
@@ -157,6 +205,7 @@ One language's dictionary. `baseUrl` defaults to `https://cdn.jsdelivr.net/npm/w
 
 - `dictionary.lookup(word)`: all entries for a word, with every sense's definitions, regions, labels, synonyms and variant links (`altOf`).
 - `dictionary.searchEnglish(term, { region?, pos?, exclude?, limit?, allSenses? })`: words for an English term in this language, one per word, best first (no meaning handling: use the translator for that). `allSenses: true` returns every matching sense of those words instead.
+- `dictionary.frames()`: the language's sentence frames and frame words, unfilled (`{ frames, words }`; empty without frames).
 - `dictionary.checker()` / `dictionary.syllables()`: the language's checker settings and syllable list (with frequencies), used by `createChecker` (`{}` for languages without them).
 - `dictionary.picks()`: the language's [hand-picked words](#hand-picked-words), `{ word, pos, gloss, picks: [{ word, tags? }], first? }` per English meaning (empty for languages without any).
 - `dictionary.meta()`: the language's name, regions, region groups, source, license, build date and counts.
@@ -255,7 +304,8 @@ Generated data isn't committed (it would bloat git history); it's built before p
 3. Add the language's code to `translationLangs` in `languages/en.ts` (so English keeps its translation tables for it) and rebuild English.
 4. `npm run build:data -- <code>`, check the results, add a few real-data tests and evaluation cases, and publish.
 5. Optional: [checker](#checking-text) settings (`checker` in `languages/<code>.ts`): `units: 'syllables'` for languages written in syllables (also builds the syllable list for accent suggestions), `maxWordUnits`, the reliable `pronouns` per pronoun-table column, `ambiguousPronouns`, and `politeEndings`. Without them, the language is read as space-separated words and gets the region check.
-6. Optional, later: [hand-picked words](#hand-picked-words). A speaker runs `npm run picks-sheet -- <code>`, fills in `your_pick` only where the first choice is wrong, and pastes the `--apply` output into `picks`.
+6. Optional: [sentence frames](#sentence-frames): `frames` (how the language says each frame in `languages/frames.ts`) and `frameWords` (the words that change by region, each from a definition or an override with a note). Frames reach other languages through the shared catalog; a new frame goes in the catalog once.
+7. Optional, later: [hand-picked words](#hand-picked-words). A speaker runs `npm run picks-sheet -- <code>`, fills in `your_pick` only where the first choice is wrong, and pastes the `--apply` output into `picks`.
 
 Languages in non-Latin scripts (Chinese, Arabic, Russian, …) will need a script-aware version of `shardKey` first: today files are split by the first two Latin letters.
 

package/dist/check.d.ts

@@ -1,5 +1,5 @@
 import { type TranslatorOptions } from './translate.ts';
-export type CheckRule = 'spelling' | 'dialect' | 'pronoun-relationship' | 'pronoun-pair' | 'polite-ending';
+export type CheckRule = 'spelling' | 'dialect' | 'pronoun-relationship' | 'pronoun-pair' | 'pronoun-consistency' | 'polite-ending';
 export type CheckIssue = {
     rule: CheckRule;
     /**

package/dist/check.js

@@ -9,6 +9,7 @@
 // The checker only speaks up when it's confident and stays silent otherwise, so no issues means none of
 // the checks found anything, not that the text is correct.
 import { createDictionary, resolveRegion } from "./index.js";
+import { joined, matchCase, plain, segment, sentencesOf, units } from "./text.js";
 import { createTranslator } from "./translate.js";
 import { PRONOUN_PERSONS, } from "./types.js";
 // How each pronoun-table column is named in messages.
@@ -19,61 +20,10 @@ const PERSON_NAMES = {
 const ABOUT_PERSONS = new Set(['third', 'thirdPlural']);
 const SELF_PERSONS = new Set(['self', 'selfPlural']);
 const ADDRESSEE_PERSONS = new Set(['addressee', 'addresseePlural']);
-// A run of letters (with their accents), digits and apostrophes: a word, or a syllable in syllable languages.
-const UNIT = /[\p{L}\p{M}\p{N}'’]+/gu;
-const SENTENCE_END = /[.!?…]+|\n+/g;
 const quote = (s) => `“${s}”`;
 const lowerFirst = (s) => s.charAt(0).toLowerCase() + s.slice(1);
 const listOf = (items, conjunction = 'and') => items.length <= 1 ? items.join('') : `${items.slice(0, -1).join(', ')} ${conjunction} ${items[items.length - 1]}`;
-// Keeps a capital first letter when replacing a capitalized word ("Tôi" → "Con").
-const matchCase = (original, replacement) => original.charAt(0) !== original.charAt(0).toLowerCase() ? replacement.charAt(0).toUpperCase() + replacement.slice(1) : replacement;
-// Without accents, for finding the accented forms of a syllable: "không" → "khong", "đi" → "di".
-const plain = (s) => s.normalize('NFD').replace(/\p{M}/gu, '').replace(/đ/g, 'd').replace(/Đ/g, 'D').normalize('NFC');
 const unique = (items) => [...new Set(items)];
-function units(text) {
-    return [...text.matchAll(UNIT)].map((m) => ({ text: m[0], start: m.index, end: m.index + m[0].length }));
-}
-function sentencesOf(text, all) {
-    const ends = [...text.matchAll(SENTENCE_END)].map((m) => m.index);
-    const out = [[]];
-    let next = 0;
-    for (const u of all) {
-        while (next < ends.length && ends[next] < u.start) {
-            if (out[out.length - 1].length)
-                out.push([]);
-            next++;
-        }
-        out[out.length - 1].push(u);
-    }
-    return out.filter((s) => s.length);
-}
-// Units join into one word only when just spaces separate them ("thịt heo", not "thịt, heo").
-const joined = (text, a, b) => /^[^\S\n]+$/.test(text.slice(a.end, b.start));
-// Longest match against the dictionary: at each position, the longest run of units that's a headword
-// ("kết quả" is one word, so its "quả" isn't checked on its own). Unknown single units become words with
-// no entries.
-async function segment(text, list, maxUnits, lookup) {
-    const words = [];
-    for (let i = 0; i < list.length;) {
-        let taken = null;
-        for (let k = Math.min(maxUnits, list.length - i); k >= 1; k--) {
-            const span = list.slice(i, i + k);
-            if (span.some((u, j) => j > 0 && !joined(text, span[j - 1], u)))
-                continue;
-            const phrase = span.map((u) => u.text).join(' ');
-            const entries = /\d/.test(phrase) ? [] : await lookup(phrase);
-            if (entries.length || k === 1) {
-                const start = span[0].start;
-                const end = span[k - 1].end;
-                taken = { text: text.slice(start, end), lower: phrase.toLowerCase(), start, end, units: span, entries };
-                break;
-            }
-        }
-        words.push(taken);
-        i += taken.units.length;
-    }
-    return words;
-}
 // --- Rules -----------------------------------------------------------------------------------------------
 // Spelling and accents, for languages written in syllables with a syllable list: a syllable that isn't in
 // the language is flagged, with the accented forms it could be ("khong" → "không"). Plain-letter
@@ -233,6 +183,33 @@ async function pronounPair(ctx) {
     }
     return issues;
 }
+// Switching words for the same person within one text ("tôi" in one sentence, "tui" in the next): the first
+// one used is kept. Useful for journals and longer messages. Ambiguous pronouns ("bạn": also "friend") are
+// never the ones flagged as a warning.
+async function pronounConsistency(ctx) {
+    const first = new Map();
+    const issues = [];
+    for (const m of ctx.pronounMatches()) {
+        const earlier = first.get(m.person);
+        if (!earlier) {
+            first.set(m.person, m);
+            continue;
+        }
+        if (earlier.lower === m.lower)
+            continue;
+        const ambiguous = ctx.config.ambiguousPronouns?.includes(m.lower) || ctx.config.ambiguousPronouns?.includes(earlier.lower);
+        issues.push({
+            rule: 'pronoun-consistency',
+            severity: ambiguous ? 'suggestion' : 'warning',
+            start: m.start,
+            end: m.end,
+            text: m.text,
+            message: `You used ${quote(earlier.text)} for “${PERSON_NAMES[m.person]}” earlier; keep one word: ${quote(earlier.lower)}.`,
+            suggestions: [matchCase(m.text, earlier.lower)],
+        });
+    }
+    return issues;
+}
 // Polite endings when speaking up (the listener's row is marked `respect`): Vietnamese sentences to parents,
 // elders and teachers usually end in "ạ". A suggestion, not an error: it isn't always needed.
 async function politeEnding(ctx) {
@@ -267,6 +244,7 @@ const RULES = {
     dialect,
     'pronoun-relationship': pronounRelationship,
     'pronoun-pair': pronounPair,
+    'pronoun-consistency': pronounConsistency,
     'polite-ending': politeEnding,
 };
 const SEVERITY_ORDER = { error: 0, warning: 1, suggestion: 2 };

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { type CheckerConfig, type PronounRow, type Entry, type Hit, type LanguageMeta, type PickRow } from './types.ts';
+import { type CheckerConfig, type FramesData, type PronounRow, type Entry, type Hit, type LanguageMeta, type PickRow } from './types.ts';
 export * from './types.ts';
 export * from './pos.ts';
 /**
@@ -71,6 +71,8 @@ export type Dictionary = {
     checker(): Promise<CheckerConfig>;
     /** Syllable → Zipf frequency (0 when unknown), for languages written in syllables; `{}` otherwise. */
     syllables(): Promise<Record<string, number>>;
+    /** The language's sentence frames and frame words, unfilled (see FramesData); empty without frames. */
+    frames(): Promise<FramesData>;
 };
 /**
  * The regions a region or region-group name stands for (e.g. English "North America" -> US, Canada),
@@ -81,3 +83,6 @@ export declare function dataUrl(lang: string): string;
 export declare function createDictionary(options: DictionaryOptions): Dictionary;
 export * from './translate.ts';
 export * from './check.ts';
+export * from './text.ts';
+export * from './phrasebook.ts';
+export * from './review.ts';

package/dist/index.js

@@ -115,6 +115,16 @@ export function createDictionary(options) {
             const m = await meta();
             return m.syllables ? loadOnce('syllables.json') : {};
         },
+        async frames() {
+            const m = await meta();
+            if (!m.frames)
+                return { frames: [], words: {} };
+            const stored = await loadOnce('frames.json');
+            return {
+                frames: stored.frames,
+                words: Object.fromEntries(Object.entries(stored.words).map(([k, w]) => [k, { ...w, choices: w.choices.map((c) => fromStored(c, m.regions)) }])),
+            };
+        },
         async pronouns({ listener, region, speaker, exclude = DEFAULT_EXCLUDED_LABELS } = {}) {
             const m = await meta();
             if (!m.pronouns)
@@ -141,3 +151,6 @@ export function createDictionary(options) {
 }
 export * from "./translate.js";
 export * from "./check.js";
+export * from "./text.js";
+export * from "./phrasebook.js";
+export * from "./review.js";

package/dist/phrasebook.d.ts

@@ -0,0 +1,80 @@
+import { type Translator, type TranslatorOptions } from './translate.ts';
+export type FrameOptions = {
+    /** The language to say it in, e.g. "vi". */
+    lang: string;
+    /** Region or region group; its words come first, and words only used elsewhere are left out. */
+    region?: string;
+    /** Who you're talking to, as a pronoun-table row id; without it, the table's default row. */
+    listener?: string;
+    /** The speaker's gender, for pronouns that depend on it. */
+    speaker?: 'male' | 'female';
+    /** 'casual' allows colloquial words (Southern "hông", "tui"); otherwise they're left out. */
+    register?: 'casual' | 'neutral' | 'polite';
+};
+/** One piece of a filled frame, with why it was chosen, for highlighting and explanations. */
+export type FramePart = {
+    text: string;
+    /** Fixed wording, a frame word ({WHERE}), a pronoun ({I}) or a content slot ({place}). */
+    kind: 'fixed' | 'word' | 'pronoun' | 'content';
+    slot?: string;
+    /** e.g. "where (Central)", "“I”, talking to your parents". */
+    why?: string;
+    regions?: string[];
+    /** For content slots given in English: the English, and whether no translation was found. */
+    from?: string;
+    untranslated?: boolean;
+};
+export type FilledFrame = {
+    id: string;
+    topic: string;
+    /** The English way to say it, with slots filled when given. */
+    en: string;
+    /** The sentence; open content slots stay as {name}. */
+    text: string;
+    parts: FramePart[];
+    /** The content slots and their parts of speech. */
+    slots: {
+        name: string;
+        pos: 'noun' | 'verb' | 'adj';
+    }[];
+    /** true when every content slot is filled (and translated). */
+    complete: boolean;
+};
+/** A content slot value: English (translated for you) or `{ text }` in the target language. */
+export type SlotValue = string | {
+    text: string;
+};
+/** A frame recognized in a sentence in the language itself, and what the frame says should change. */
+export type FrameMatch = {
+    frame: {
+        id: string;
+        en: string;
+    };
+    /** The sentence as the frame would have it, for the given region, listener and register. */
+    text: string;
+    changes: {
+        slot: string;
+        from: string;
+        to: string;
+        why: string;
+    }[];
+};
+export type Phrasebook = {
+    /** The language's frames, filled for the region, listener and register, with content slots open. */
+    frames(options: FrameOptions & {
+        topic?: string;
+    }): Promise<FilledFrame[]>;
+    /** One frame with its content slots filled. Unknown ids throw, listing the valid ones. */
+    render(id: string, options: FrameOptions & {
+        slots?: Record<string, SlotValue>;
+    }): Promise<FilledFrame>;
+    /** The frame an English sentence (or clause) follows, filled with its slot values; null when none. */
+    match(english: string, options: FrameOptions): Promise<FilledFrame | null>;
+    /** Frames a sentence in the language itself follows, with the changes the frame suggests. */
+    matchFrames(sentence: string, options: FrameOptions): Promise<FrameMatch[]>;
+};
+export type PhrasebookOptions = TranslatorOptions & {
+    /** Share a translator (and its dictionaries) instead of creating one. */
+    translator?: Translator;
+};
+export declare function createPhrasebook(options?: PhrasebookOptions): Phrasebook;

package/dist/phrasebook.js

@@ -0,0 +1,254 @@
+// Sentence frames: common sentences with slots ("Where is {place}?" → "{place} ở {WHERE}?"), filled for a
+// region, a listener and a register. The frames, their words and the pronoun table are the language's data
+// (frames.json, pronouns.json), so this file has no language-specific rules: it picks words by region and
+// labels, fills pronouns from the pronoun table, and translates content slots with the translator.
+//
+// Three directions: list and fill frames (`frames`, `render`), recognize an English sentence as a frame
+// (`match`), and recognize a frame in a sentence in the language itself (`matchFrames`), which is how the
+// journal review checks a sentence against frames ("Bạn có đi?" is the yes/no frame missing "không").
+import { createDictionary, resolveRegion } from "./index.js";
+import { matchCase, plain } from "./text.js";
+import { createTranslator } from "./translate.js";
+import { FRAME_PRONOUNS, } from "./types.js";
+const SLOT = /\{([A-Za-z_]+)\}/g;
+// Words for a casual register only.
+const CASUAL = new Set(['colloquial', 'informal', 'familiar', 'slang', 'Internet']);
+const PERSON_NAMES = {
+    self: 'I', addressee: 'you', third: 'he/she', selfPlural: 'we', addresseePlural: 'you (plural)', thirdPlural: 'they',
+};
+const lowerFirst = (s) => s.charAt(0).toLowerCase() + s.slice(1);
+const quote = (s) => `“${s}”`;
+// Words and their accent-less forms ("tôi", "toi"), longest first so "chúng tôi" wins over "tôi".
+const withPlain = (words) => [...new Set(words.flatMap((w) => [w, plain(w)]))].sort((a, b) => b.length - a.length);
+const escape = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+// English sentences compare without case, curly quotes, extra spaces or the final punctuation.
+const normalizeEn = (s) => s.toLowerCase().replace(/[’‘]/g, "'").replace(/\s+/g, ' ').trim().replace(/[.!?…]+$/, '').trim();
+// Spaces after filling: one between words, none before punctuation or after an empty slot.
+const tidy = (s) => s.replace(/\s+/g, ' ').replace(/\s+([.,!?…])/g, '$1').trim();
+function choose(choices, s) {
+    return choices
+        .filter((c) => !s.wanted || c.regions.some((r) => s.wanted.has(r)))
+        .filter((c) => c.when !== 'respect' || s.row?.respect)
+        .filter((c) => s.casual || !c.labels.some((l) => CASUAL.has(l)))
+        .map((c, i) => ({ c, i, regional: s.wanted && c.regionTagged ? 1 : 0, casual: s.casual && c.labels.some((l) => CASUAL.has(l)) ? 1 : 0 }))
+        .sort((a, b) => b.casual - a.casual || b.regional - a.regional || a.i - b.i)[0]?.c;
+}
+function pronounChoices(row, slot) {
+    const p = FRAME_PRONOUNS[slot];
+    if (!row || !p)
+        return [];
+    return row[p.person].filter((c) => p.inclusive === undefined || c.inclusive === undefined || c.inclusive === p.inclusive);
+}
+export function createPhrasebook(options = {}) {
+    const cache = new Map();
+    const dict = (lang) => {
+        let d = cache.get(lang);
+        if (!d) {
+            d = options.dictionary?.(lang) ?? createDictionary({ lang, baseUrl: options.baseUrl?.(lang), load: options.load?.(lang) });
+            cache.set(lang, d);
+        }
+        return d;
+    };
+    const translator = options.translator ?? createTranslator({ ...options, dictionary: dict });
+    async function setting(o) {
+        const d = dict(o.lang);
+        const [meta, data] = await Promise.all([d.meta(), d.frames()]);
+        // An unknown listener throws, listing the valid ones (as the translator does).
+        if (meta.pronouns && o.listener)
+            await d.pronouns({ listener: o.listener });
+        const rows = meta.pronouns ? await d.pronouns({ region: o.region, speaker: o.speaker }) : [];
+        const row = o.listener ? rows.find((r) => r.id === o.listener) : rows.find((r) => r.default);
+        return { meta, data, rows, row, wanted: resolveRegion(meta, o.region), casual: o.register === 'casual' };
+    }
+    // The word for a frame word or pronoun slot, with why; empty when the slot has no word here (no "ạ"
+    // unless the listener's row is marked respect).
+    function slotWord(name, s) {
+        const words = s.data.words[name];
+        if (words) {
+            const c = choose(words.choices, s);
+            if (!c)
+                return { text: '', kind: 'word', slot: name };
+            return {
+                text: c.word, kind: 'word', slot: name, regions: c.regionTagged ? c.regions : undefined,
+                why: `${words.meaning}${c.regionTagged ? ` (${c.regions.join(', ')})` : ''}${c.note ? `: ${c.note}` : ''}`,
+            };
+        }
+        const c = choose(pronounChoices(s.row, name), s);
+        const person = PERSON_NAMES[FRAME_PRONOUNS[name]?.person ?? ''];
+        if (!c)
+            return { text: '', kind: 'pronoun', slot: name };
+        return {
+            text: c.word, kind: 'pronoun', slot: name, regions: c.regionTagged ? c.regions : undefined,
+            why: `“${person}”${s.row && !s.row.default ? `, talking to ${lowerFirst(s.row.label)}` : ''}`,
+        };
+    }
+    async function content(frame, name, value, o) {
+        if (value === undefined)
+            return { text: `{${name}}`, kind: 'content', slot: name };
+        if (typeof value !== 'string')
+            return { text: value.text, kind: 'content', slot: name };
+        const english = value.trim().replace(/^(the|a|an|some)\s+/i, '');
+        const [group] = await translator.translate(english, {
+            from: 'en', to: o.lang, toRegion: o.region, pos: frame.slots[name], register: o.register, limit: 1,
+        }).catch(() => []);
+        const word = group?.translations[0]?.word;
+        return word
+            ? { text: word, kind: 'content', slot: name, from: value, why: `“${english}”`, regions: group.translations[0].regionTagged ? group.translations[0].regions : undefined }
+            : { text: value, kind: 'content', slot: name, from: value, untranslated: true };
+    }
+    async function fill(frame, o, s, values = {}) {
+        const parts = [];
+        let last = 0;
+        for (const m of frame.text.matchAll(SLOT)) {
+            if (m.index > last)
+                parts.push({ text: frame.text.slice(last, m.index), kind: 'fixed' });
+            const name = m[1];
+            parts.push(frame.slots[name] ? await content(frame, name, values[name], o) : slotWord(name, s));
+            last = m.index + m[0].length;
+        }
+        if (last < frame.text.length)
+            parts.push({ text: frame.text.slice(last), kind: 'fixed' });
+        let text = tidy(parts.map((p) => p.text).join(''));
+        if (!frame.clause)
+            text = text.charAt(0).toUpperCase() + text.slice(1);
+        const slots = Object.entries(frame.slots).map(([name, pos]) => ({ name, pos }));
+        const en = Object.keys(frame.slots).reduce((s, name) => (typeof values[name] === 'string' ? s.replace(`{${name}}`, values[name]) : s), frame.en[0]);
+        return {
+            id: frame.id, topic: frame.topic, en, text,
+            parts: parts.filter((p) => p.text),
+            slots,
+            complete: slots.every(({ name }) => values[name] !== undefined) && !parts.some((p) => p.untranslated),
+        };
+    }
+    async function frameById(id, o) {
+        const s = await setting(o);
+        const frame = s.data.frames.find((f) => f.id === id);
+        if (!frame) {
+            throw new Error(`which-dialect: unknown frame "${id}" for ${s.meta.name}. Use one of: ${s.data.frames.map((f) => f.id).join(', ') || '(none)'}`);
+        }
+        return { frame, s };
+    }
+    return {
+        async frames(o) {
+            const s = await setting(o);
+            return Promise.all(s.data.frames.filter((f) => !o.topic || f.topic === o.topic).map((f) => fill(f, o, s)));
+        },
+        async render(id, o) {
+            const { frame, s } = await frameById(id, o);
+            return fill(frame, o, s, o.slots);
+        },
+        async match(english, o) {
+            const s = await setting(o);
+            const sentence = normalizeEn(english);
+            for (const frame of s.data.frames) {
+                for (const template of frame.en) {
+                    const names = [];
+                    const pattern = escape(normalizeEn(template)).replace(/\\\{([A-Za-z_]+)\\\}/g, (_, name) => {
+                        names.push(name);
+                        return '(.+?)';
+                    });
+                    const m = new RegExp(`^${pattern}$`, 'u').exec(sentence);
+                    if (!m)
+                        continue;
+                    const values = Object.fromEntries(names.map((n, i) => [n, m[i + 1]]));
+                    const filled = await fill(frame, o, s, values);
+                    return { ...filled, en: english.trim() };
+                }
+            }
+            return null;
+        },
+        async matchFrames(sentence, o) {
+            const s = await setting(o);
+            const d = dict(o.lang);
+            // Every pronoun of a column, in every row and region: a frame position decides it's a pronoun.
+            const allRows = s.meta.pronouns ? await d.pronouns({ exclude: [] }) : [];
+            const terminal = /[?]\s*$/.test(sentence) ? '?' : '';
+            const core = sentence.trim().replace(/[.!?…]+$/, '').trim();
+            const out = [];
+            for (const frame of s.data.frames) {
+                if (frame.text.trim().endsWith('?') !== (terminal === '?'))
+                    continue;
+                const tokens = [...frame.text.replace(/[.!?…]+\s*$/, '').matchAll(/\{([A-Za-z_]+)\}|[^{]+/g)].map((m) => m[1] ? { slot: m[1] } : { fixed: m[0] });
+                // Frames without fixed words ("{thing} {HOW}?") match only with every word present; frames with
+                // fixed words can spot a missing required word ("Bạn có đi?" → "… không?"). Frames with neither
+                // fixed words nor frame words ("{I} {quality}.") would match almost anything, so they're skipped.
+                const anchored = tokens.some((t) => t.fixed && /\p{L}/u.test(t.fixed));
+                if (!anchored && !tokens.some((t) => t.slot && s.data.words[t.slot]))
+                    continue;
+                const slots = [];
+                let pattern = '';
+                for (const t of tokens) {
+                    if (t.fixed !== undefined) {
+                        pattern += t.fixed.trim() ? `\\s*${escape(t.fixed.trim()).replace(/\s+/g, '\\s+')}\\s*` : '\\s*';
+                        continue;
+                    }
+                    const name = t.slot;
+                    if (frame.slots[name]) {
+                        slots.push({ name, kind: 'content', optional: false });
+                        pattern += '(.+?)';
+                    }
+                    else if (s.data.words[name]) {
+                        const alts = withPlain(s.data.words[name].choices.map((c) => c.word)).map(escape);
+                        const optional = Boolean(s.data.words[name].optional) || anchored;
+                        slots.push({ name, kind: 'word', optional });
+                        pattern += `(${alts.join('|')})${optional ? '?' : ''}`;
+                    }
+                    else {
+                        const alts = withPlain(allRows.flatMap((r) => pronounChoices(r, name)).map((c) => c.word));
+                        if (!alts.length)
+                            continue;
+                        slots.push({ name, kind: 'pronoun', optional: false });
+                        pattern += `(${alts.map(escape).join('|')})`;
+                    }
+                }
+                const m = new RegExp(`^${pattern}$`, 'iu').exec(core);
+                if (!m)
+                    continue;
+                const changes = [];
+                const filled = {};
+                slots.forEach((slot, i) => {
+                    let captured = m[i + 1] ?? '';
+                    filled[slot.name] = captured;
+                    if (slot.kind === 'content')
+                        return;
+                    // A word missing its accents in a frame position ("Toi" where the frame has a pronoun) is that word.
+                    const known = slot.kind === 'word'
+                        ? s.data.words[slot.name].choices.map((c) => c.word)
+                        : allRows.flatMap((r) => pronounChoices(r, slot.name)).map((c) => c.word);
+                    if (captured && !known.includes(captured.toLowerCase())) {
+                        const accented = known.find((w) => plain(w) === captured.toLowerCase());
+                        if (accented) {
+                            const fixed = matchCase(captured, accented);
+                            changes.push({ slot: slot.name, from: captured, to: fixed, why: `${quote(fixed)} needs its accents` });
+                            filled[slot.name] = captured = fixed;
+                        }
+                    }
+                    const expected = slotWord(slot.name, s);
+                    if (slot.kind === 'pronoun') {
+                        // Pronouns change only for a chosen listener: in a journal, "Má" may be who you write about.
+                        const fits = pronounChoices(s.row, slot.name).some((c) => c.word === captured.toLowerCase());
+                        if (o.listener && expected.text && !fits) {
+                            filled[slot.name] = matchCase(captured, expected.text);
+                            changes.push({ slot: slot.name, from: captured, to: filled[slot.name], why: expected.why ?? '' });
+                        }
+                        return;
+                    }
+                    const words = s.data.words[slot.name];
+                    const allowed = words.choices.filter((c) => choose([c], s));
+                    if (captured && !allowed.some((c) => c.word === captured.toLowerCase()) && expected.text) {
+                        filled[slot.name] = matchCase(captured, expected.text);
+                        changes.push({ slot: slot.name, from: captured, to: filled[slot.name], why: expected.why ?? '' });
+                    }
+                    else if (!captured && !words.optional && expected.text) {
+                        filled[slot.name] = expected.text;
+                        changes.push({ slot: slot.name, from: '', to: expected.text, why: `${words.meaning}: this frame needs it` });
+                    }
+                });
+                let text = tidy(frame.text.replace(/[.!?…]+\s*$/, '').replace(SLOT, (_, name) => filled[name] ?? ''));
+                text = matchCase(core, text) + (terminal || sentence.trim().match(/[.!?…]+$/)?.[0] || '');
+                out.push({ frame: { id: frame.id, en: frame.en[0] }, text, changes });
+            }
+            return out;
+        },
+    };
+}

package/dist/review.d.ts

@@ -0,0 +1,56 @@
+import { type CheckRule } from './check.ts';
+import { type FrameOptions } from './phrasebook.ts';
+import { type TranslatorOptions } from './translate.ts';
+export type ReviewOptions = FrameOptions & {
+    /** The learner's own language, for the parts written in it (default "en"). */
+    base?: string;
+};
+export type ReviewChange = {
+    /** What was written (empty when a missing word was added). */
+    from: string;
+    to: string;
+    /** One sentence for the learner. */
+    why: string;
+    /** 'translated' (a base-language word), 'frame' (a sentence frame), or the checker rule. */
+    kind: 'translated' | 'frame' | CheckRule;
+};
+/** A checker suggestion that wasn't applied ("end with ạ"), for the learner to consider. */
+export type ReviewHint = {
+    rule: CheckRule;
+    text: string;
+    message: string;
+    suggestions: string[];
+};
+export type ReviewSentence = {
+    /** Offsets of the sentence in Review.text, including its final punctuation. */
+    start: number;
+    end: number;
+    original: string;
+    /** The sentence split by language, e.g. [{ text: 'Hôm nay tôi đi', lang: 'vi' }, { text: 'market', lang: 'en' }, …]. */
+    parts: {
+        text: string;
+        lang: string;
+    }[];
+    corrected: string;
+    changes: ReviewChange[];
+    hints: ReviewHint[];
+    /** Sentence frames the sentence follows or was built from: the frame's English and its pattern. */
+    frames: {
+        id: string;
+        en: string;
+        text: string;
+    }[];
+    /** Parts the review couldn't handle (longer base-language text that follows no frame), left as written. */
+    unchecked: string[];
+};
+export type Review = {
+    /** The entry, in Unicode NFC form; sentence offsets refer to it. */
+    text: string;
+    /** The whole entry with every sentence corrected. */
+    corrected: string;
+    sentences: ReviewSentence[];
+};
+export type Reviewer = {
+    review(entry: string, options: ReviewOptions): Promise<Review>;
+};
+export declare function createReviewer(options?: TranslatorOptions): Reviewer;

package/dist/review.js

@@ -0,0 +1,201 @@
+// Reviewing a journal entry written in the target language, the learner's own language, or a mix of both
+// ("Hôm nay tôi đi market với má"). For each sentence:
+//
+//   1. Each word is marked as the target or the base language, from the two dictionaries alone (accents and
+//      syllables, headwords, inflected forms); words that could be either take their neighbors' language.
+//   2. Base-language parts are replaced: a whole sentence or clause that follows a sentence frame is filled
+//      in ("but it was expensive" → "nhưng mắc quá"), short phrases are translated word by word ("market" →
+//      "chợ"), and anything longer is listed as unchecked rather than guessed.
+//   3. The grammar checker runs on the whole entry, so consistency checks see every sentence ("tôi" here,
+//      "tui" there); its errors and warnings are applied, its suggestions returned as hints.
+//   4. The corrected sentence is checked against sentence frames ("Bạn có đi?" → "Bạn có đi không?").
+//
+// Everything is rule-based and runs on the language's data: no API, and nothing language-specific here.
+import { createChecker } from "./check.js";
+import { createDictionary } from "./index.js";
+import { createPhrasebook } from "./phrasebook.js";
+import { plain, sentencesOf, units } from "./text.js";
+import { createTranslator } from "./translate.js";
+// The longest base-language phrase translated word by word; longer ones need a frame.
+const MAX_TRANSLATED_UNITS = 3;
+const TERMINAL = /^[.!?…]+/;
+export function createReviewer(options = {}) {
+    const cache = new Map();
+    const dict = (lang) => {
+        let d = cache.get(lang);
+        if (!d) {
+            d = options.dictionary?.(lang) ?? createDictionary({ lang, baseUrl: options.baseUrl?.(lang), load: options.load?.(lang) });
+            cache.set(lang, d);
+        }
+        return d;
+    };
+    const translator = createTranslator({ ...options, dictionary: dict });
+    const phrasebook = createPhrasebook({ ...options, dictionary: dict, translator });
+    const checker = createChecker({ ...options, dictionary: dict });
+    // Which language each unit is in. A unit the target knows and the base doesn't is the target's, and the
+    // other way round; units both or neither know ("an", "to", names) take the nearest decided neighbor's.
+    // In syllable languages, a base-language word that's a target syllable only once accents are added
+    // ("but": bút) is the base language's; a word only the target knows without its accents ("khong") is
+    // still the target's.
+    async function languages(list, lang, base) {
+        const target = dict(lang);
+        const syllables = await target.syllables();
+        const syllableLanguage = Object.keys(syllables).length > 0;
+        const plainSyllables = syllableLanguage ? new Set(Object.keys(syllables).map(plain)) : new Set();
+        // 'exact': a target word as written; 'accents': one only with accents added ("khong" → "không").
+        const targetKnows = async (u) => {
+            const lower = u.text.toLowerCase();
+            if (!syllableLanguage)
+                return (await target.lookup(lower)).length > 0 ? 'exact' : false;
+            if (lower in syllables || lower !== plain(lower))
+                return 'exact';
+            return plainSyllables.has(lower) ? 'accents' : false;
+        };
+        const baseKnows = async (u) => (await translator.senses(u.text, { from: base }).catch(() => [])).length > 0;
+        const marks = await Promise.all(list.map(async (u) => {
+            if (/\d/.test(u.text))
+                return '?';
+            const [t, b] = await Promise.all([targetKnows(u), baseKnows(u)]);
+            if (t === 'exact')
+                return b ? '?' : lang;
+            if (b)
+                return base;
+            return t === 'accents' ? lang : '?';
+        }));
+        return marks.map((m, i) => {
+            if (m !== '?')
+                return m;
+            for (let d = 1; d < marks.length; d++) {
+                if (i - d >= 0 && marks[i - d] !== '?')
+                    return marks[i - d];
+                if (i + d < marks.length && marks[i + d] !== '?')
+                    return marks[i + d];
+            }
+            return lang;
+        });
+    }
+    return {
+        async review(entry, o) {
+            const text = entry.normalize('NFC');
+            const base = o.base ?? 'en';
+            const meta = await dict(o.lang).meta();
+            // A frame as shown to the learner: its English and its pattern for these settings, slots open.
+            const frameInfo = new Map();
+            const frameOf = async (id) => {
+                if (!frameInfo.has(id)) {
+                    const f = await phrasebook.render(id, o);
+                    frameInfo.set(id, { id, en: f.en, text: f.text });
+                }
+                return frameInfo.get(id);
+            };
+            const drafts = [];
+            for (const list of sentencesOf(text, units(text))) {
+                const start = list[0].start;
+                const coreEnd = list[list.length - 1].end;
+                const end = coreEnd + (TERMINAL.exec(text.slice(coreEnd))?.[0].length ?? 0);
+                const langs = await languages(list, o.lang, base);
+                const groups = [];
+                list.forEach((u, i) => {
+                    if (groups.length && groups[groups.length - 1].lang === langs[i])
+                        groups[groups.length - 1].units.push(u);
+                    else
+                        groups.push({ lang: langs[i], units: [u] });
+                });
+                const draft = {
+                    start, coreEnd, end, replacements: [], frames: [], unchecked: [],
+                    parts: groups.map((g) => ({ text: text.slice(g.units[0].start, g.units[g.units.length - 1].end), lang: g.lang })),
+                };
+                const whole = groups.length === 1 && groups[0].lang === base;
+                for (const g of groups.filter((g) => g.lang === base)) {
+                    const from = text.slice(g.units[0].start, g.units[g.units.length - 1].end);
+                    const at = { start: g.units[0].start, end: g.units[g.units.length - 1].end };
+                    // A whole sentence is matched with its own punctuation ("Where is the bathroom?").
+                    const frame = await phrasebook.match(whole ? text.slice(start, end) : from, o);
+                    if (frame && frame.complete) {
+                        let to = frame.text.replace(/[.!?…]+$/, '');
+                        if (!whole && g.units[0].start !== start)
+                            to = to.charAt(0).toLowerCase() + to.slice(1);
+                        const info = await frameOf(frame.id);
+                        draft.replacements.push({ ...at, to, change: { kind: 'frame', from, to, why: `Sentence frame “${info.en}”` } });
+                        draft.frames.push(info);
+                        continue;
+                    }
+                    if (!whole && g.units.length <= MAX_TRANSLATED_UNITS) {
+                        const english = from.replace(/^(the|a|an)\s+/i, '');
+                        const [group] = await translator.translate(english, { from: base, to: o.lang, toRegion: o.region, register: o.register, limit: 1 }).catch(() => []);
+                        const word = group?.translations[0]?.word;
+                        if (word) {
+                            draft.replacements.push({ ...at, to: word, change: { kind: 'translated', from, to: word, why: `“${english}” in ${meta.name}` } });
+                            continue;
+                        }
+                    }
+                    draft.unchecked.push(from);
+                }
+                drafts.push(draft);
+            }
+            // The entry with base-language parts replaced, keeping where each sentence now is.
+            let draftText = '';
+            let cursor = 0;
+            const placed = [];
+            for (const d of drafts) {
+                draftText += text.slice(cursor, d.start);
+                const s = draftText.length;
+                let core = text.slice(d.start, d.coreEnd);
+                for (const r of [...d.replacements].sort((a, b) => b.start - a.start)) {
+                    core = core.slice(0, r.start - d.start) + r.to + core.slice(r.end - d.start);
+                }
+                if (/\p{Lu}/u.test(text.charAt(d.start)))
+                    core = core.charAt(0).toUpperCase() + core.slice(1);
+                draftText += core + text.slice(d.coreEnd, d.end);
+                placed.push({ start: s, coreEnd: s + core.length, end: draftText.length });
+                cursor = d.end;
+            }
+            draftText += text.slice(cursor);
+            // 3. The checker on the whole entry; 4. frames on each corrected sentence.
+            const { issues } = await checker.check(draftText, { lang: o.lang, region: o.region, listener: o.listener, about: undefined, speaker: o.speaker });
+            const sentences = [];
+            for (const [i, d] of drafts.entries()) {
+                const p = placed[i];
+                const mine = issues.filter((x) => x.start >= p.start && x.end <= p.coreEnd);
+                const changes = d.replacements.sort((a, b) => a.start - b.start).map((r) => r.change);
+                const hints = [];
+                let core = draftText.slice(p.start, p.coreEnd);
+                let lastStart = Infinity;
+                for (const x of [...mine].sort((a, b) => b.start - a.start)) {
+                    const applies = (x.severity === 'error' || x.severity === 'warning') && x.suggestions.length && x.end <= lastStart;
+                    if (!applies) {
+                        hints.push({ rule: x.rule, text: x.text, message: x.message, suggestions: x.suggestions });
+                        continue;
+                    }
+                    core = core.slice(0, x.start - p.start) + x.suggestions[0] + core.slice(x.end - p.start);
+                    changes.push({ kind: x.rule, from: x.text, to: x.suggestions[0], why: x.message });
+                    lastStart = x.start;
+                }
+                const punctuation = draftText.slice(p.coreEnd, p.end);
+                let corrected = core + punctuation;
+                const frames = [...d.frames];
+                for (const m of await phrasebook.matchFrames(corrected, o)) {
+                    if (!frames.some((f) => f.id === m.frame.id))
+                        frames.push(await frameOf(m.frame.id));
+                    if (m.changes.length && corrected === core + punctuation) {
+                        corrected = m.text;
+                        for (const c of m.changes)
+                            changes.push({ kind: 'frame', from: c.from, to: c.to, why: `${c.why} (sentence frame “${m.frame.en}”)` });
+                    }
+                }
+                sentences.push({
+                    start: d.start, end: d.end, original: text.slice(d.start, d.end), parts: d.parts,
+                    corrected, changes, hints: hints.reverse(), frames, unchecked: d.unchecked,
+                });
+            }
+            let corrected = '';
+            cursor = 0;
+            for (const s of sentences) {
+                corrected += text.slice(cursor, s.start) + s.corrected;
+                cursor = s.end;
+            }
+            corrected += text.slice(cursor);
+            return { text, corrected, sentences };
+        },
+    };
+}

package/dist/text.d.ts

@@ -0,0 +1,31 @@
+import type { Entry } from './types.ts';
+export declare const UNIT: RegExp;
+export declare const SENTENCE_END: RegExp;
+export type Unit = {
+    text: string;
+    start: number;
+    end: number;
+};
+export type Word = {
+    text: string;
+    lower: string;
+    start: number;
+    end: number;
+    units: Unit[];
+    entries: Entry[];
+};
+export declare function units(text: string): Unit[];
+/** Units grouped into sentences, which end at . ! ? … or a line break. */
+export declare function sentencesOf(text: string, all: Unit[]): Unit[][];
+/** Units join into one word only when just spaces separate them ("thịt heo", not "thịt, heo"). */
+export declare const joined: (text: string, a: Unit, b: Unit) => boolean;
+/**
+ * Longest match against the dictionary: at each position, the longest run of units that's a headword
+ * ("kết quả" is one word, so its "quả" isn't checked on its own). Unknown single units become words with
+ * no entries.
+ */
+export declare function segment(text: string, list: Unit[], maxUnits: number, lookup: (phrase: string) => Promise<Entry[]>): Promise<Word[]>;
+/** Without accents: "không" → "khong", "đi" → "di". */
+export declare const plain: (s: string) => string;
+/** Keeps a capital first letter when replacing a capitalized word ("Tôi" → "Con"). */
+export declare const matchCase: (original: string, replacement: string) => string;

package/dist/text.js

@@ -0,0 +1,57 @@
+// Splitting text into units, sentences and dictionary words: shared by the checker, sentence frames and
+// the journal review. Language-general: units are runs of letters and digits; a language written in
+// syllables groups them into dictionary words by longest match (CheckerConfig.units / maxWordUnits).
+// A run of letters (with their accents), digits and apostrophes: a word, or a syllable in syllable languages.
+export const UNIT = /[\p{L}\p{M}\p{N}'’]+/gu;
+export const SENTENCE_END = /[.!?…]+|\n+/g;
+export function units(text) {
+    return [...text.matchAll(UNIT)].map((m) => ({ text: m[0], start: m.index, end: m.index + m[0].length }));
+}
+/** Units grouped into sentences, which end at . ! ? … or a line break. */
+export function sentencesOf(text, all) {
+    const ends = [...text.matchAll(SENTENCE_END)].map((m) => m.index);
+    const out = [[]];
+    let next = 0;
+    for (const u of all) {
+        while (next < ends.length && ends[next] < u.start) {
+            if (out[out.length - 1].length)
+                out.push([]);
+            next++;
+        }
+        out[out.length - 1].push(u);
+    }
+    return out.filter((s) => s.length);
+}
+/** Units join into one word only when just spaces separate them ("thịt heo", not "thịt, heo"). */
+export const joined = (text, a, b) => /^[^\S\n]+$/.test(text.slice(a.end, b.start));
+/**
+ * Longest match against the dictionary: at each position, the longest run of units that's a headword
+ * ("kết quả" is one word, so its "quả" isn't checked on its own). Unknown single units become words with
+ * no entries.
+ */
+export async function segment(text, list, maxUnits, lookup) {
+    const words = [];
+    for (let i = 0; i < list.length;) {
+        let taken = null;
+        for (let k = Math.min(maxUnits, list.length - i); k >= 1; k--) {
+            const span = list.slice(i, i + k);
+            if (span.some((u, j) => j > 0 && !joined(text, span[j - 1], u)))
+                continue;
+            const phrase = span.map((u) => u.text).join(' ');
+            const entries = /\d/.test(phrase) ? [] : await lookup(phrase);
+            if (entries.length || k === 1) {
+                const start = span[0].start;
+                const end = span[k - 1].end;
+                taken = { text: text.slice(start, end), lower: phrase.toLowerCase(), start, end, units: span, entries };
+                break;
+            }
+        }
+        words.push(taken);
+        i += taken.units.length;
+    }
+    return words;
+}
+/** Without accents: "không" → "khong", "đi" → "di". */
+export const plain = (s) => s.normalize('NFD').replace(/\p{M}/gu, '').replace(/đ/g, 'd').replace(/Đ/g, 'D').normalize('NFC');
+/** Keeps a capital first letter when replacing a capitalized word ("Tôi" → "Con"). */
+export const matchCase = (original, replacement) => original.charAt(0) !== original.charAt(0).toLowerCase() ? replacement.charAt(0).toUpperCase() + replacement.slice(1) : replacement;

package/dist/types.d.ts

@@ -114,6 +114,64 @@ export type LanguageMeta = {
     checker?: boolean;
     /** true when the language has a syllable list (syllables.json: syllable → Zipf frequency, 0 when unknown). */
     syllables?: boolean;
+    /** true when the language has sentence frames (frames.json, see FramesData). */
+    frames?: boolean;
+};
+/**
+ * Pronoun slots in sentence frames ({I}, {YOU}, …) and the pronoun-table column each one fills from, for
+ * the listener (or the table's default row).
+ */
+export declare const FRAME_PRONOUNS: Record<string, {
+    person: PronounPerson;
+    inclusive?: boolean;
+}>;
+/**
+ * One choice for a frame word slot ({WHERE} → "đâu", "mô"): from a dictionary definition (region and
+ * labels from that sense) or a hand-written override with a note. `when: 'respect'` choices are used only
+ * when the listener's pronoun-table row is marked respect (Vietnamese "ạ").
+ */
+export type FrameWordChoice = {
+    word: string;
+    gloss?: string;
+    source: 'gloss' | 'override';
+    regions: string[];
+    regionTagged: boolean;
+    labels: string[];
+    note?: string;
+    when?: 'respect';
+};
+/** A frame word slot: what it means, whether a sentence is fine without it, and its choices in order. */
+export type FrameWords = {
+    meaning: string;
+    optional?: boolean;
+    choices: FrameWordChoice[];
+};
+/**
+ * A sentence frame in one language: the catalog's id, topic, English ways to say it and slots, and this
+ * language's `text` with {slot} placeholders: content slots (lowercase, from `slots`), pronoun slots
+ * (FRAME_PRONOUNS) and frame word slots (uppercase, from FramesData.words).
+ */
+export type FrameData = {
+    id: string;
+    topic: string;
+    en: string[];
+    slots: Record<string, 'noun' | 'verb' | 'adj'>;
+    clause?: boolean;
+    text: string;
+};
+export type FramesData = {
+    frames: FrameData[];
+    words: Record<string, FrameWords>;
+};
+export type StoredFrameWordChoice = Omit<FrameWordChoice, 'regions' | 'regionTagged' | 'labels'> & {
+    regions?: string[];
+    labels?: string[];
+};
+export type StoredFramesData = {
+    frames: FrameData[];
+    words: Record<string, Omit<FrameWords, 'choices'> & {
+        choices: StoredFrameWordChoice[];
+    }>;
 };
 /**
  * How the checker reads one language (languages/<lang>.ts `checker`, stored as data/checker.json). Every

package/dist/types.js

@@ -1,5 +1,18 @@
 // Shapes of the data. The build script (scripts/build-language.ts) writes the Stored* types to the
 // data files; the API in index.ts reads them and returns the full Sense/Entry/Hit types.
+/**
+ * Pronoun slots in sentence frames ({I}, {YOU}, …) and the pronoun-table column each one fills from, for
+ * the listener (or the table's default row).
+ */
+export const FRAME_PRONOUNS = {
+    I: { person: 'self' },
+    YOU: { person: 'addressee' },
+    WE: { person: 'selfPlural', inclusive: false },
+    WE_INCL: { person: 'selfPlural', inclusive: true },
+    YOU_PL: { person: 'addresseePlural' },
+    HE_SHE: { person: 'third' },
+    THEY: { person: 'thirdPlural' },
+};
 /** The pronoun table's columns: I, you, he/she, we, you (plural), they. */
 export const PRONOUN_PERSONS = ['self', 'addressee', 'third', 'selfPlural', 'addresseePlural', 'thirdPlural'];
 /** Converts a full sense or hit to its stored form. */

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "which-dialect",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Translate words between languages and dialects, meaning by meaning, starting with Vietnamese (Northern, Central, Southern) and English, with Wiktionary data loaded on demand. More languages are planned.",
   "type": "module",
   "main": "./dist/index.js",