which-dialect 0.3.0 → 0.4.0

package/README.md

@@ -111,44 +111,47 @@ The cases involving Spanish were removed with it. The held-out set has since bee
 
 ### `createChecker({ baseUrl?, load?, dictionary? })` → `checker.check(text, options)`
 
-Checks a learner's sentence in one language and returns issues with the exact place in the text, a short explanation and suggested fixes. It runs in the browser like the rest of the library: no API calls, no cost.
+A spellchecker for learners: it gives a correction for every word it can, with the exact place in the text and a short reason. Words written in the learner's own language (English by default) get the target-language word. Further checks (regions, pronouns, politeness) can be turned on. It runs in the browser like the rest of the library: no API calls, no cost.
 
 ```ts
 import { createChecker } from 'which-dialect'
 
 const checker = createChecker()
-const { text, issues } = await checker.check('Tôi muốn ăn thịt lợn và bắp khong?', {
-  lang: 'vi', region: 'Southern', listener: 'parent',
+const { text, issues, parts } = await checker.check('Toi muon an the ice cream but it was very expensive.', {
+  lang: 'vi', region: 'Southern',
 })
-// “Tôi”: Talking to your parents, say “con” for “I”, not “Tôi”.               → Con
-// “lợn”: “lợn” is Northern; in Southern Vietnamese, say “heo”.                → heo, cúi, ỉn
-// “khong”: “khong” isn't a Vietnamese syllable. Did you mean “không”, “khổng” or “khống”?
-// “khong”: Talking to your parents, end the sentence with “ạ” to sound polite. (suggestion)
+// Toi → Tôi · muon → muốn · an → ăn          (accents)
+// the → (left out: Vietnamese has no "the") · ice cream → kem · but → nhưng · it → nó
+// was → là · very → rất · expensive → mắc    (English words, in Southern Vietnamese)
 ```
 
 | Option | Meaning |
 |---|---|
-| `lang` | The text's language. |
-| `region` | The region the text should be in. Without it, words that don't fit the rest of the text's region are pointed out (*lợn*, Northern, next to *nha*, Southern). |
-| `listener` | Who you're talking to, as a [pronoun-table](#looking-words-up) row id (`'parent'`, `'teacher'`, `'friend'`, …): pronouns and polite endings are checked against it. |
-| `about` | Who you're talking about, for "he/she" and "they" pronouns. |
-| `speaker` | `'male'` or `'female'`, for pronouns that depend on it. |
-| `rules` | Turn checks off, e.g. `{ 'polite-ending': false }`. |
+| `lang` | The text's language (the target). |
+| `base` | The learner's own language, whose words get translated (default `'en'`). |
+| `region` | The region's words are used for translations (*expensive* → Southern *mắc*, Northern *đắt*), and the `dialect` check, when on, flags other regions' words. |
+| `register` | `'casual'` allows colloquial translations; otherwise plain ones come first. |
+| `rules` | Turn checks on or off. By default only `spelling` and `foreign-word` run; e.g. `{ dialect: true, 'pronoun-relationship': true }`. |
+| `listener`, `about`, `speaker` | For the pronoun and polite-ending checks: who you're talking to and about (pronoun-table row ids), and the speaker's gender. |
 
-Each issue has `rule`, `severity` (`'error'`: not a word of the language; `'warning'`: wrong for the chosen region or relationship; `'suggestion'`: often better), `start`/`end` (offsets into the returned `text`, the input in Unicode NFC form), `text`, `message` and `suggestions` (replacements for `text`, best first).
+The result has `text` (the input in Unicode NFC form), `issues` and `parts` (the text split by language). Each issue has `rule`, `severity` (`'error'`: not a word of the language; `'warning'`: probably wrong, or wrong for the region or relationship; `'suggestion'`: often better), `start`/`end` (offsets into `text`), `text`, `message` and `suggestions` (replacements for `text`, best first; `''` means leave it out).
 
-The checks:
+**On by default (the spellchecker):**
 
-- **`spelling`**: syllables that aren't in the language, with the accented forms they could be (*khong* → *không*). Plain-letter words with no accented form are left alone (names, foreign words). Languages written in syllables only (Vietnamese).
-- **`dialect`**: words whose main meaning is tagged for another region (*lợn* in Southern text → *heo*; *muỗng* in Northern → *thìa*; in English, US *apartment* in UK text → *flat*). Words with an untagged main meaning are never flagged (*má* is "cheek" everywhere).
+- **`spelling`**: syllables that aren't in the language get the accented forms they could be (*khong* → *không*); rare plain-letter words next to a far more common accented form too (*toi* → *tôi*, *hom* → *hôm*), and, in a sentence typed without accents, any more common accented form (*an* → *ăn*, *di* → *đi*; not *con* → *còn*, about as common). Names (capitalized mid-sentence), the language's listed pronouns (*tui*) and plain-letter words with no accented form (*email*) are left alone. Languages written in syllables only (Vietnamese).
+- **`foreign-word`**: words in the learner's own language get the target-language word, in the region and register (*market* → *chợ*, *ice cream* → *kem*). Phrases the base dictionary knows are translated as one, falling back to single words. Words the language has no word for are suggested to be left out (`leaveOut` in its checker settings: Vietnamese has no *the*, *a*, *an*). Which language each word is in comes from the two dictionaries: an English word that's Vietnamese only with accents added (*but*: *bút*) is English; Vietnamese typed without accents (*khong*) is Vietnamese; words that could be either (*an*, *tui*) follow their neighbors, leaning to the target language.
+
+**Opt-in (`rules`):**
+
+- **`dialect`**: words whose main meaning is tagged for another region (*lợn* in Southern text → *heo*; *muỗng* in Northern → *thìa*; in English, US *apartment* in UK text → *flat*). Without `region`, words that don't fit the rest of the text's region are pointed out. Words with an untagged main meaning are never flagged (*má* is "cheek" everywhere).
 - **`pronoun-relationship`**: pronouns that don't fit the listener (*tôi* to a parent → *con*; to a teacher → *em*).
 - **`pronoun-pair`**: "I" and "you" pronouns that never go together (*tao* goes with *mày*, not *bạn*), when no listener is given.
 - **`pronoun-consistency`**: switching words for the same person within one text (*tôi* in one sentence, *tui* in the next): keep the first.
 - **`polite-ending`**: a suggestion to end sentences with *ạ* when talking to parents, elders and teachers (not for exclamations).
 
-The checker only speaks up when it's confident, so **no issues means none of the checks found anything, not that the text is correct**. It doesn't check word order, classifiers or whether a sentence sounds natural yet. Only reliable pronouns are checked: kinship words like *con* (also "child", and a classifier in *con chó*) aren't, and *bạn* (also "friend") only gets suggestions.
+Corrections are word by word: they don't fix word order or grammar, so English translated word by word reads like English in Vietnamese words (*it was very expensive* → *nó là rất mắc*). Whole English sentences or clauses that follow a [sentence frame](#sentence-frames) get the frame's natural wording instead in the [journal review](#reviewing-journal-entries). The checker only speaks up when it's confident, so **no issues means none of the checks found anything, not that the text is correct**.
 
-**Every language uses the same checks.** They read only the language's data and its `checker` settings in `languages/<code>.ts` (how text splits into words, which pronouns are reliable, polite endings), so a check whose data a language doesn't have is skipped. A language without checker settings still gets the `dialect` check.
+**Every language uses the same checks.** They read only the language's data and its `checker` settings in `languages/<code>.ts` (how text splits into words, which pronouns are reliable, polite endings, words with no equivalent), so a check whose data a language doesn't have is skipped. A language without checker settings is read as space-separated words: English words in it are still translated, and the `dialect` check works.
 
 ## Sentence frames
 
@@ -166,13 +169,13 @@ await pb.match('Do you want coffee?', { lang: 'vi' })                         //
 await pb.matchFrames('Bạn có đi?', { lang: 'vi' })                            // → 'Bạn có đi không?' (missing "không")
 ```
 
-Options: `lang`, `region`, `listener` (a pronoun-table row id; without it, the neutral default row), `speaker`, `register` (`'casual'` allows colloquial words like Southern *hông*, *tui*). Content slots take English (translated for you, in the region) or `{ text }` in the language itself. `parts` says why each word was chosen, for highlighting. `matchFrames` recognizes a frame in a sentence in the language itself and returns the changes the frame suggests: a missing required word (*không*), a word from another region (*mô* in Southern text → *đâu*), missing accents in a frame position (*Toi* → *Tôi*), and, only when a listener is given, a pronoun that doesn't fit them.
+Options: `lang`, `region`, `listener` (a pronoun-table row id; without it, the neutral default row), `speaker`, `register` (`'casual'` allows colloquial words like Southern *hông*, *tui*). Content slots take English (translated for you, in the region) or `{ text }` in the language itself. `parts` says why each word was chosen, for highlighting. `matchFrames` recognizes a frame in a sentence in the language itself (the journal review uses it to list the frames a sentence follows) and returns the changes the frame would suggest: a missing required word (*không*), a word from another region (*mô* in Southern text → *đâu*), missing accents in a frame position (*Toi* → *Tôi*), and, only when a listener is given, a pronoun that doesn't fit them.
 
 The frames themselves are written once, in English, in [`languages/frames.ts`](languages/frames.ts) (34 so far: journal sentences, feelings, questions, requests, greetings). Each language says how it expresses them (`frames` in `languages/<code>.ts`), with pronoun slots (`{I}`, `{YOU}`, `{WE_INCL}`, …, filled from the pronoun table) and **frame words** for what changes by region (`frameWords`: Vietnamese `{WHERE}` *đâu* / Central *mô*, `{HOW}` Northern *thế nào* / *sao* / Central *răng*, `{SOFT}` Northern *nhé* / Central and Southern *nha*, `{Q_END}` *không* / Southern casual *hông*, `{POLITE}` *ạ* when speaking up). Each frame word comes from a dictionary definition, keeping its region and labels, or from an override with a note, like the pronoun table. A language without frames returns none.
 
 ## Reviewing journal entries
 
-`createReviewer().review(entry, { lang, base?, region?, listener?, speaker?, register? })` reviews an entry written in the target language, the learner's own language (`base`, default `'en'`), or a mix of both, and returns the corrected entry sentence by sentence:
+`createReviewer().review(entry, { lang, base?, region?, register?, checks?, listener?, speaker? })` reviews an entry written in the target language, the learner's own language (`base`, default `'en'`), or a mix of both, and returns the corrected entry, in the target language, sentence by sentence:
 
 ```ts
 import { createReviewer } from 'which-dialect'
@@ -182,20 +185,23 @@ const review = await createReviewer().review(
   { lang: 'vi', region: 'Southern' },
 )
 review.corrected
-// 'Hôm nay tôi đi chợ với má. Tôi muốn ăn thịt heo nhưng mắc quá. Ngày mai tôi đi lại không?'
+// 'Hôm nay tôi đi chợ với má. Tôi muốn ăn thịt lợn nhưng mắc quá. Ngày mai tui đi lại không?'
 review.sentences[1].changes
-// [{ from: 'but it was expensive', to: 'nhưng mắc quá', kind: 'frame', why: 'Sentence frame “but it was {quality}”' },
-//  { from: 'lợn', to: 'heo', kind: 'dialect', why: '“lợn” is Northern; in Southern Vietnamese, say “heo”.' }]
+// [{ from: 'but it was expensive', to: 'nhưng mắc quá', kind: 'frame', why: 'Sentence frame “but it was {quality}”' }]
+
+// With the regional and consistency checks on:
+await createReviewer().review(entry, { lang: 'vi', region: 'Southern', checks: { dialect: true, 'pronoun-consistency': true } })
+// 'Hôm nay tôi đi chợ với má. Tôi muốn ăn thịt heo nhưng mắc quá. Ngày mai tôi đi lại không?'
 ```
 
 For each sentence:
 
-1. **Each word is marked as the target or the base language** (`parts`), from the two dictionaries alone. Words that could be either (*an*, *to*) take their neighbors' language; an English word that's Vietnamese only with accents added (*but*: *bút*) stays English, while Vietnamese typed without accents (*khong*) stays Vietnamese.
-2. **Base-language parts are replaced:** a whole sentence or clause that follows a sentence frame is filled in (*but it was expensive* → *nhưng mắc quá*; *Where is the bathroom?* → *Phòng tắm ở đâu?*), and words or short phrases (up to 3 words) are translated in the region (*market* → *chợ*). Anything longer is left as written and listed in `unchecked`: the review doesn't guess.
-3. **The grammar checker runs on the whole entry**, so consistency checks see every sentence (*tôi* earlier, *tui* later). Its errors and warnings are applied (`changes`); its suggestions, like ending with *ạ*, are returned as `hints`.
-4. **The corrected sentence is checked against sentence frames** (*Bạn có đi chợ?* → *Bạn có đi chợ không?*), and the frames it follows are listed (`frames`), for suggesting patterns to reuse.
+1. **Each word is marked as the target or the base language** (`parts`), as in the checker's `foreign-word` check.
+2. **A whole base-language sentence or clause that follows a sentence frame is filled in** with the frame's wording (*but it was expensive* → *nhưng mắc quá*; *Where is the bathroom?* → *Phòng tắm ở đâu?*).
+3. **The checker runs on the whole entry**: by default the spellchecker, so every remaining English word gets its translation and every misspelled word its accents; `checks` turns on more (regions, pronouns, consistency across sentences, polite endings). Its errors and warnings are applied (`changes`); its suggestions, like ending with *ạ*, are returned as `hints`.
+4. **The frames the corrected sentence follows are listed** (`frames`), for suggesting patterns to reuse. Frames don't change the sentence.
 
-Each sentence has `original`, `parts`, `corrected`, `changes` (`{ from, to, why, kind }`), `hints`, `frames` (`{ id, en, text }`, the frame's English and its pattern) and `unchecked`. For a journal, leave `listener` out: people written about (*má*, *thầy*) aren't treated as the listener. Rules can't fix everything: scrambled word order, missing verbs, or English sentences no frame covers come back as written, and plain-letter words that are also Vietnamese words (*toi*, *di*) are only fixed when a sentence frame shows what they should be.
+Each sentence has `original`, `parts`, `corrected`, `changes` (`{ from, to, why, kind }`; `to` is `''` when a word is left out), `hints`, `frames` (`{ id, en, text }`, the frame's English and its pattern) and `unchecked` (English words with no translation, left as written). For a journal, leave `listener` out: people written about (*má*, *thầy*) aren't treated as the listener. Corrections are word by word except for frames: word order and grammar aren't fixed (*I am very happy* → *Tôi là rất mừng*).
 
 ## Looking words up
 

package/dist/check.d.ts

@@ -1,5 +1,10 @@
 import { type TranslatorOptions } from './translate.ts';
-export type CheckRule = 'spelling' | 'dialect' | 'pronoun-relationship' | 'pronoun-pair' | 'pronoun-consistency' | 'polite-ending';
+export type CheckRule = 'spelling' | 'foreign-word' | 'dialect' | 'pronoun-relationship' | 'pronoun-pair' | 'pronoun-consistency' | 'polite-ending';
+/**
+ * Which checks run unless `rules` says otherwise: the spellchecker (spelling, and words in the learner's
+ * own language, with their translation). The rest are opt-in, e.g. `rules: { dialect: true }`.
+ */
+export declare const DEFAULT_CHECK_RULES: Record<CheckRule, boolean>;
 export type CheckIssue = {
     rule: CheckRule;
     /**
@@ -20,15 +25,28 @@ export type CheckIssue = {
 export type CheckOptions = {
     /** Language code of the text, e.g. "vi". */
     lang: string;
-    /** Region or region group the text should be in, e.g. "Southern". Without it, mixed regions are flagged. */
+    /**
+     * The learner's own language, whose words in the text are translated (`foreign-word`). Default "en";
+     * none when it's the text's language.
+     */
+    base?: string;
+    /**
+     * Region or region group the text should be in, e.g. "Southern": translations use its words, and the
+     * `dialect` check flags other regions' words (without it, mixed regions).
+     */
     region?: string;
+    /** 'casual' allows colloquial translations (Southern "tui"); otherwise plain ones come first. */
+    register?: 'casual' | 'neutral' | 'polite';
     /** Who you're talking to, as a pronoun-table row id (see Dictionary.pronouns), e.g. "parent". */
     listener?: string;
     /** Who you're talking about, as a pronoun-table row id, for "he/she" and "they" pronouns. */
     about?: string;
     /** The speaker's gender, for pronouns that depend on it. */
     speaker?: 'male' | 'female';
-    /** Turn rules off, e.g. { 'polite-ending': false }. All are on by default. */
+    /**
+     * Turn checks on or off. By default only the spellchecker runs (DEFAULT_CHECK_RULES: `spelling` and
+     * `foreign-word`); opt in to others, e.g. { dialect: true, 'pronoun-relationship': true }.
+     */
     rules?: Partial<Record<CheckRule, boolean>>;
 };
 export type CheckResult = {
@@ -36,6 +54,13 @@ export type CheckResult = {
     text: string;
     /** Sorted by position. */
     issues: CheckIssue[];
+    /** The text split by language (the target and the base), in order; one part when no base is given. */
+    parts: {
+        text: string;
+        lang: string;
+        start: number;
+        end: number;
+    }[];
 };
 export type Checker = {
     check(text: string, options: CheckOptions): Promise<CheckResult>;

package/dist/check.js

@@ -1,17 +1,33 @@
-// Checking text in one language: spelling and accents, words from another region, and pronouns and
-// polite endings that fit who you're talking to.
+// Checking text in one language. By default it's a spellchecker that gives a correction for every word it
+// can: syllables that aren't in the language get their accented forms ("khong" → "không"), and words in the
+// learner's own language (English by default) get the target-language word ("market" → "chợ"). Further
+// checks are opt-in (`rules`): words from another region, pronouns and polite endings that fit who you're
+// talking to, and consistency within the text.
 //
 // Every rule reads only the language's data: its dictionary, pronoun table, checker.json and
 // syllables.json (see CheckerConfig). Adding a language means adding data and settings in
-// languages/<lang>.ts, not code here. A rule whose data a language doesn't have is skipped, so a language
-// without checker settings is still checked for words from another region.
+// languages/<lang>.ts, not code here. A rule whose data a language doesn't have is skipped.
 //
 // The checker only speaks up when it's confident and stays silent otherwise, so no issues means none of
 // the checks found anything, not that the text is correct.
 import { createDictionary, resolveRegion } from "./index.js";
 import { joined, matchCase, plain, segment, sentencesOf, units } from "./text.js";
+import { markLanguages } from "./detect.js";
 import { createTranslator } from "./translate.js";
 import { PRONOUN_PERSONS, } from "./types.js";
+/**
+ * Which checks run unless `rules` says otherwise: the spellchecker (spelling, and words in the learner's
+ * own language, with their translation). The rest are opt-in, e.g. `rules: { dialect: true }`.
+ */
+export const DEFAULT_CHECK_RULES = {
+    spelling: true,
+    'foreign-word': true,
+    dialect: false,
+    'pronoun-relationship': false,
+    'pronoun-pair': false,
+    'pronoun-consistency': false,
+    'polite-ending': false,
+};
 // How each pronoun-table column is named in messages.
 const PERSON_NAMES = {
     self: 'I', addressee: 'you', third: 'he/she', selfPlural: 'we', addresseePlural: 'you (plural)', thirdPlural: 'they',
@@ -25,9 +41,19 @@ const lowerFirst = (s) => s.charAt(0).toLowerCase() + s.slice(1);
 const listOf = (items, conjunction = 'and') => items.length <= 1 ? items.join('') : `${items.slice(0, -1).join(', ')} ${conjunction} ${items[items.length - 1]}`;
 const unique = (items) => [...new Set(items)];
 // --- Rules -----------------------------------------------------------------------------------------------
-// Spelling and accents, for languages written in syllables with a syllable list: a syllable that isn't in
-// the language is flagged, with the accented forms it could be ("khong" → "không"). Plain-letter
-// syllables with no accented form are left alone: usually a name or a foreign word ("email").
+// Spelling and accents, for languages written in syllables with a syllable list (with frequencies):
+// - a syllable that isn't in the language is flagged, with the accented forms it could be ("khong" →
+//   "không"); plain-letter syllables with no accented form are left alone (names, foreign words: "email");
+// - a plain-letter syllable that is a word but a rare one, next to a much more common accented form, is
+//   probably missing its accents ("toi" → "tôi", "hom" → "hôm"). In a sentence typed without accents (at
+//   least half plain letters and a missing-accents error already, or 3+ words with no accent at all; half,
+//   because filled-in sentence frames add accented words), any accented form that's more common
+//   counts ("an" → "ăn"; but not "con" → "còn", about as common). Left alone: capitalized words
+//   mid-sentence (names) and the language's listed pronouns, some written without accents ("tui").
+// Units inside a known word, or in the learner's own language, aren't checked.
+// Zipf points by which an accented form must be more common than a plain-letter word to be suggested.
+const ACCENTS_GAP = 1.5;
+const ACCENTS_GAP_UNACCENTED_TEXT = 0.2;
 async function spelling(ctx) {
     if (ctx.config.units !== 'syllables' || !ctx.meta.syllables)
         return [];
@@ -38,29 +64,111 @@ async function spelling(ctx) {
     for (const list of byPlain.values())
         list.sort((a, b) => syllables[b] - syllables[a]);
     const issues = [];
-    for (const word of ctx.sentences.flatMap((s) => s.words)) {
-        if (word.entries.length)
-            continue;
-        for (const u of word.units) {
+    const suggest = (u, candidates, severity, message) => issues.push({ rule: 'spelling', severity, start: u.start, end: u.end, text: u.text, message, suggestions: candidates.map((c) => matchCase(u.text, c)) });
+    const options = (u, candidates) => listOf(candidates.map((c) => quote(matchCase(u.text, c))), 'or');
+    const pronouns = new Set(Object.values(ctx.config.pronouns ?? {}).flat().map((w) => w.toLowerCase()));
+    for (const sentence of ctx.sentences) {
+        const lone = sentence.words.filter((w) => !w.entries.length || w.units.length === 1).flatMap((w) => w.units).filter((u) => !ctx.inBase(u));
+        const target = sentence.units.filter((u) => !ctx.inBase(u) && /\p{L}/u.test(u.text));
+        const mostlyPlain = target.filter((u) => u.text.toLowerCase() === plain(u.text.toLowerCase())).length >= target.length * 0.5;
+        let missingAccents = false;
+        const rare = [];
+        for (const u of lone) {
             const lower = u.text.toLowerCase().replace(/[’']/g, '');
-            if (/\d/.test(lower) || lower in syllables)
+            if (/\d/.test(lower))
                 continue;
-            const candidates = (byPlain.get(plain(lower)) ?? []).slice(0, 3);
-            const accented = lower !== plain(lower);
-            if (!candidates.length && !accented)
+            const inWord = sentence.words.some((w) => w.entries.length && w.units.length > 1 && w.units.includes(u));
+            if (inWord)
                 continue;
-            issues.push({
-                rule: 'spelling',
+            if (!(lower in syllables)) {
+                const candidates = (byPlain.get(plain(lower)) ?? []).slice(0, 3);
+                const accented = lower !== plain(lower);
+                if (!candidates.length && !accented)
+                    continue;
+                if (!accented)
+                    missingAccents = true;
                 // Missing accents on plain letters is the common learner slip; a wrong accent is more surely wrong.
-                severity: accented ? 'error' : 'warning',
-                start: u.start,
-                end: u.end,
-                text: u.text,
-                message: candidates.length
-                    ? `${quote(u.text)} isn't a ${ctx.meta.name} syllable. Did you mean ${listOf(candidates.map((c) => quote(matchCase(u.text, c))), 'or')}?`
-                    : `${quote(u.text)} isn't a ${ctx.meta.name} syllable. Check the spelling and accents.`,
-                suggestions: candidates.map((c) => matchCase(u.text, c)),
-            });
+                suggest(u, candidates, accented ? 'error' : 'warning', candidates.length
+                    ? `${quote(u.text)} isn't a ${ctx.meta.name} syllable. Did you mean ${options(u, candidates)}?`
+                    : `${quote(u.text)} isn't a ${ctx.meta.name} syllable. Check the spelling and accents.`);
+            }
+            else if (lower === plain(lower) && !pronouns.has(lower) && (u === sentence.units[0] || !/\p{Lu}/u.test(u.text.charAt(0)))) {
+                rare.push(u);
+            }
+        }
+        // Typed without accents: mostly plain letters, and an accent error already, or 3+ words with no accent at all.
+        const unaccented = mostlyPlain && (missingAccents || (target.length >= 3 && target.every((u) => u.text.toLowerCase() === plain(u.text.toLowerCase()))));
+        for (const u of rare) {
+            const lower = u.text.toLowerCase();
+            const gap = unaccented ? ACCENTS_GAP_UNACCENTED_TEXT : ACCENTS_GAP;
+            const better = (byPlain.get(lower) ?? []).filter((c) => c !== lower && syllables[c] >= syllables[lower] + gap - 1e-9).slice(0, 3);
+            if (!better.length)
+                continue;
+            suggest(u, better, 'warning', unaccented
+                ? `${quote(u.text)} looks typed without accents. Did you mean ${options(u, better)}?`
+                : `${quote(u.text)} is a rare word; did you mean ${options(u, better)}?`);
+        }
+    }
+    return issues;
+}
+// Words in the learner's own language (English by default), with the target-language word to use: "market"
+// → "chợ". Runs of base units are split into the longest phrases the base dictionary knows ("ice cream"),
+// each translated in the region and register. Words the language usually has no word for
+// (CheckerConfig.leaveOut: Vietnamese has no articles) are suggested to be left out.
+async function foreignWord(ctx) {
+    const base = ctx.base;
+    if (!base)
+        return [];
+    const leaveOut = new Set((ctx.config.leaveOut ?? []).map((w) => w.toLowerCase()));
+    const issues = [];
+    for (const s of ctx.sentences) {
+        const runs = [];
+        s.units.forEach((u, i) => {
+            if (!ctx.inBase(u))
+                return;
+            const last = runs[runs.length - 1];
+            if (last && last[last.length - 1] === s.units[i - 1] && joined(ctx.text, s.units[i - 1], u))
+                last.push(u);
+            else
+                runs.push([u]);
+        });
+        for (const run of runs) {
+            for (let i = 0; i < run.length;) {
+                // The longest phrase the base dictionary knows and the language has a word for; single words
+                // always give an issue, with or without a translation.
+                let found;
+                for (let k = Math.min(MAX_FOREIGN_UNITS, run.length - i); k >= 1 && !found; k--) {
+                    const phrase = ctx.text.slice(run[i].start, run[i + k - 1].end);
+                    if (k === 1 && leaveOut.has(phrase.toLowerCase())) {
+                        found = { k, words: [], leave: true };
+                        break;
+                    }
+                    if (k > 1 && !(await ctx.translator.senses(phrase, { from: base }).catch(() => [])).length)
+                        continue;
+                    const [group] = await ctx.translator.translate(phrase, {
+                        from: base, to: ctx.options.lang, toRegion: ctx.options.region, register: ctx.options.register, listener: ctx.options.listener, limit: 3,
+                    }).catch(() => []);
+                    const words = unique((group?.translations ?? []).map((t) => t.word));
+                    if (words.length || k === 1)
+                        found = { k, words, leave: false };
+                }
+                const { k, words, leave } = found;
+                const span = run.slice(i, i + k);
+                const text = ctx.text.slice(span[0].start, span[k - 1].end);
+                i += k;
+                const at = { rule: 'foreign-word', severity: 'error', start: span[0].start, end: span[k - 1].end, text };
+                if (leave) {
+                    issues.push({ ...at, message: `${quote(text)} isn't ${ctx.meta.name}; ${ctx.meta.name} usually has no word for it, so leave it out.`, suggestions: [''] });
+                    continue;
+                }
+                issues.push({
+                    ...at,
+                    message: words.length
+                        ? `${quote(text)} isn't ${ctx.meta.name}: say ${listOf(words.map(quote), 'or')}.`
+                        : `${quote(text)} isn't ${ctx.meta.name}, and no translation was found.`,
+                    suggestions: words.map((w) => (span[0].start === s.units[0].start ? matchCase(text, w) : w)),
+                });
+            }
         }
     }
     return issues;
@@ -73,6 +181,7 @@ async function dialect(ctx) {
     const all = new Set(ctx.meta.regions);
     const regional = ctx.sentences
         .flatMap((s) => s.words)
+        .filter((word) => !word.units.some((u) => ctx.inBase(u)))
         .map((word) => ({ word, sense: word.entries[0]?.senses[0] }))
         .filter((x) => Boolean(x.sense?.regionTagged && x.sense.regions.length < all.size));
     if (!regional.length)
@@ -239,8 +348,11 @@ async function politeEnding(ctx) {
     }
     return issues;
 }
+// The longest base-language phrase looked up as one ("ice cream", "take care of").
+const MAX_FOREIGN_UNITS = 3;
 const RULES = {
     spelling,
+    'foreign-word': foreignWord,
     dialect,
     'pronoun-relationship': pronounRelationship,
     'pronoun-pair': pronounPair,
@@ -281,10 +393,13 @@ export function createChecker(options = {}) {
                 }
                 return pending;
             };
+            const base = (opts.base ?? 'en') === opts.lang ? undefined : (opts.base ?? 'en');
             const sentences = [];
             for (const list of sentencesOf(text, units(text))) {
-                sentences.push({ units: list, words: await segment(text, list, config.maxWordUnits ?? 3, lookup) });
+                const langs = base ? await markLanguages(list, { lang: opts.lang, base, target: d, baseDictionary: dict(base), translator }) : list.map(() => opts.lang);
+                sentences.push({ units: list, langs, words: await segment(text, list, config.maxWordUnits ?? 3, lookup) });
             }
+            const baseUnits = new Set(sentences.flatMap((s) => s.units.filter((_, i) => s.langs[i] !== opts.lang).map((u) => u.start)));
             let rows;
             let matches;
             const ctx = {
@@ -298,6 +413,9 @@ export function createChecker(options = {}) {
                 // The language's reliable pronouns in the text, longest first ("chúng tôi" before "tôi"), matched on
                 // units directly so pronouns the dictionary doesn't list as one word ("tụi mày") are found too.
                 pronounMatches: () => (matches ??= findPronouns(text, sentences, config)),
+                base,
+                inBase: (u) => baseUnits.has(u.start),
+                translator,
                 suggestRegional: async (word, region) => {
                     try {
                         const [group] = await translator.translate(word.lower, {
@@ -310,9 +428,22 @@ export function createChecker(options = {}) {
                     }
                 },
             };
-            const results = await Promise.all(Object.keys(RULES).filter((r) => opts.rules?.[r] !== false).map((r) => RULES[r](ctx)));
+            const results = await Promise.all(Object.keys(RULES).filter((r) => opts.rules?.[r] ?? DEFAULT_CHECK_RULES[r]).map((r) => RULES[r](ctx)));
             const issues = results.flat().sort((a, b) => a.start - b.start || SEVERITY_ORDER[a.severity] - SEVERITY_ORDER[b.severity]);
-            return { text, issues };
+            // The text split by language: consecutive units in one language, with the text between them.
+            const parts = [];
+            for (const s of sentences) {
+                s.units.forEach((u, i) => {
+                    const last = parts[parts.length - 1];
+                    if (last && last.lang === s.langs[i])
+                        last.end = u.end;
+                    else
+                        parts.push({ text: '', lang: s.langs[i], start: u.start, end: u.end });
+                });
+            }
+            for (const p of parts)
+                p.text = text.slice(p.start, p.end);
+            return { text, issues, parts };
         },
     };
 }

package/dist/detect.d.ts

@@ -0,0 +1,10 @@
+import type { Dictionary } from './index.ts';
+import { type Unit } from './text.ts';
+import type { Translator } from './translate.ts';
+export declare function markLanguages(list: Unit[], o: {
+    lang: string;
+    base: string;
+    target: Dictionary;
+    baseDictionary: Dictionary;
+    translator: Translator;
+}): Promise<string[]>;

package/dist/detect.js

@@ -0,0 +1,58 @@
+// Which language each word of a mixed text is in: the target language or the learner's own (the base,
+// English by default), from the two languages' data alone. Shared by the checker and the journal review.
+import { plain } from "./text.js";
+/**
+ * Marks each unit (of one sentence) with `lang` or `base`. A unit the target knows and the base doesn't
+ * is the target's, and the other way round; units both or neither know ("an", "to", names) take the
+ * nearest decided neighbors' language when they agree. When they don't, the target's, unless the word is
+ * far more common in the base (the text is mainly the target language: Vietnamese "an" stays Vietnamese
+ * next to English "the", and "tui" between "but" and "không"). In languages written in syllables, a base
+ * word that's a target syllable only once accents are added ("but": bút) is the base's; a word only the
+ * target knows without its accents ("khong") is still the target's. The base is known through the translator, so
+ * inflected forms count ("went", "cooked").
+ */
+// Zipf points by which a word between the two languages must be more common in the base to count as base.
+const BASE_MARGIN = 2;
+export async function markLanguages(list, o) {
+    const syllables = await o.target.syllables();
+    const syllableLanguage = Object.keys(syllables).length > 0;
+    const plainSyllables = syllableLanguage ? new Set(Object.keys(syllables).map(plain)) : new Set();
+    // 'exact': a target word as written; 'accents': one only with accents added ("khong" → "không").
+    const targetKnows = async (u) => {
+        const lower = u.text.toLowerCase();
+        if (!syllableLanguage)
+            return (await o.target.lookup(lower)).length > 0 ? 'exact' : false;
+        if (lower in syllables || lower !== plain(lower))
+            return 'exact';
+        return plainSyllables.has(lower) ? 'accents' : false;
+    };
+    const baseKnows = async (u) => (await o.translator.senses(u.text, { from: o.base }).catch(() => [])).length > 0;
+    const marks = await Promise.all(list.map(async (u) => {
+        if (/\d/.test(u.text))
+            return '?';
+        const [t, b] = await Promise.all([targetKnows(u), baseKnows(u)]);
+        if (t === 'exact')
+            return b ? '?' : o.lang;
+        if (b)
+            return o.base;
+        return t === 'accents' ? o.lang : '?';
+    }));
+    // How common a word is in each language (Zipf; 0 when unknown).
+    const targetFrequency = async (u) => {
+        const lower = u.text.toLowerCase();
+        if (syllableLanguage)
+            return syllables[lower] ?? 0;
+        return (await o.target.lookup(lower))[0]?.frequency ?? 0;
+    };
+    const baseFrequency = async (u) => (await o.baseDictionary.lookup(u.text.toLowerCase()))[0]?.frequency ?? 0;
+    return Promise.all(marks.map(async (m, i) => {
+        if (m !== '?')
+            return m;
+        const left = marks.slice(0, i).reverse().find((x) => x !== '?');
+        const right = marks.slice(i + 1).find((x) => x !== '?');
+        if (left && right && left !== right) {
+            return (await baseFrequency(list[i])) - (await targetFrequency(list[i])) >= BASE_MARGIN ? o.base : o.lang;
+        }
+        return left ?? right ?? o.lang;
+    }));
+}

package/dist/index.d.ts

@@ -86,3 +86,4 @@ export * from './check.ts';
 export * from './text.ts';
 export * from './phrasebook.ts';
 export * from './review.ts';
+export * from './detect.ts';

package/dist/index.js

@@ -154,3 +154,4 @@ export * from "./check.js";
 export * from "./text.js";
 export * from "./phrasebook.js";
 export * from "./review.js";
+export * from "./detect.js";

package/dist/phrasebook.d.ts

@@ -39,6 +39,8 @@ export type FilledFrame = {
     }[];
     /** true when every content slot is filled (and translated). */
     complete: boolean;
+    /** true for frames that can be part of a sentence ("but it was {quality}"). */
+    clause?: boolean;
 };
 /** A content slot value: English (translated for you) or `{ text }` in the target language. */
 export type SlotValue = string | {

package/dist/phrasebook.js

@@ -114,7 +114,7 @@ export function createPhrasebook(options = {}) {
         const slots = Object.entries(frame.slots).map(([name, pos]) => ({ name, pos }));
         const en = Object.keys(frame.slots).reduce((s, name) => (typeof values[name] === 'string' ? s.replace(`{${name}}`, values[name]) : s), frame.en[0]);
         return {
-            id: frame.id, topic: frame.topic, en, text,
+            id: frame.id, topic: frame.topic, en, text, ...(frame.clause ? { clause: true } : {}),
             parts: parts.filter((p) => p.text),
             slots,
             complete: slots.every(({ name }) => values[name] !== undefined) && !parts.some((p) => p.untranslated),

package/dist/review.d.ts

@@ -2,17 +2,20 @@ import { type CheckRule } from './check.ts';
 import { type FrameOptions } from './phrasebook.ts';
 import { type TranslatorOptions } from './translate.ts';
 export type ReviewOptions = FrameOptions & {
-    /** The learner's own language, for the parts written in it (default "en"). */
+    /** The learner's own language, whose words are translated into the target language (default "en"). */
     base?: string;
+    /** Checks to add to the spellchecker, e.g. { dialect: true, 'pronoun-consistency': true } (see CheckRule). */
+    checks?: Partial<Record<CheckRule, boolean>>;
 };
 export type ReviewChange = {
-    /** What was written (empty when a missing word was added). */
+    /** What was written. */
     from: string;
+    /** The correction; empty when the word is left out (Vietnamese has no "the"). */
     to: string;
     /** One sentence for the learner. */
     why: string;
-    /** 'translated' (a base-language word), 'frame' (a sentence frame), or the checker rule. */
-    kind: 'translated' | 'frame' | CheckRule;
+    /** 'frame' (a base-language sentence or clause filled into a sentence frame), or the checker rule. */
+    kind: 'frame' | CheckRule;
 };
 /** A checker suggestion that wasn't applied ("end with ạ"), for the learner to consider. */
 export type ReviewHint = {
@@ -40,7 +43,7 @@ export type ReviewSentence = {
         en: string;
         text: string;
     }[];
-    /** Parts the review couldn't handle (longer base-language text that follows no frame), left as written. */
+    /** Words the review couldn't correct (a base-language word with no translation), left as written. */
     unchecked: string[];
 };
 export type Review = {

package/dist/review.js

@@ -1,24 +1,26 @@
 // Reviewing a journal entry written in the target language, the learner's own language, or a mix of both
-// ("Hôm nay tôi đi market với má"). For each sentence:
+// ("Hôm nay tôi đi market với má"), and giving the corrected entry in the target language. For each sentence:
 //
-//   1. Each word is marked as the target or the base language, from the two dictionaries alone (accents and
-//      syllables, headwords, inflected forms); words that could be either take their neighbors' language.
-//   2. Base-language parts are replaced: a whole sentence or clause that follows a sentence frame is filled
-//      in ("but it was expensive" → "nhưng mắc quá"), short phrases are translated word by word ("market" →
-//      "chợ"), and anything longer is listed as unchecked rather than guessed.
-//   3. The grammar checker runs on the whole entry, so consistency checks see every sentence ("tôi" here,
-//      "tui" there); its errors and warnings are applied, its suggestions returned as hints.
-//   4. The corrected sentence is checked against sentence frames ("Bạn có đi?" → "Bạn có đi không?").
+//   1. Each word is marked as the target or the base language (detect.ts), from the two dictionaries alone.
+//   2. A whole base-language sentence or clause that follows a sentence frame is filled in ("but it was
+//      expensive" → "nhưng mắc quá"; "Where is the bathroom?" → "Phòng tắm ở đâu?").
+//   3. The checker runs on the whole entry. By default that's the spellchecker: every misspelled word gets
+//      its accents ("khong" → "không") and every remaining base-language word its translation ("market" →
+//      "chợ"); opt-in checks (`checks`: regions, pronouns, consistency, polite endings) join it. Errors and
+//      warnings are applied; suggestions are returned as hints.
+//   4. The sentence frames the corrected sentence follows are listed, for the learner to reuse. Frames don't
+//      change the sentence.
 //
 // Everything is rule-based and runs on the language's data: no API, and nothing language-specific here.
 import { createChecker } from "./check.js";
+import { markLanguages } from "./detect.js";
 import { createDictionary } from "./index.js";
 import { createPhrasebook } from "./phrasebook.js";
-import { plain, sentencesOf, units } from "./text.js";
+import { sentencesOf, units } from "./text.js";
 import { createTranslator } from "./translate.js";
-// The longest base-language phrase translated word by word; longer ones need a frame.
-const MAX_TRANSLATED_UNITS = 3;
 const TERMINAL = /^[.!?…]+/;
+// Spaces after a word was left out: one between words, none before punctuation.
+const tidy = (s) => s.replace(/[^\S\n]+/g, ' ').replace(/ ([.,!?…])/g, '$1').trim();
 export function createReviewer(options = {}) {
     const cache = new Map();
     const dict = (lang) => {
@@ -32,53 +34,10 @@ export function createReviewer(options = {}) {
     const translator = createTranslator({ ...options, dictionary: dict });
     const phrasebook = createPhrasebook({ ...options, dictionary: dict, translator });
     const checker = createChecker({ ...options, dictionary: dict });
-    // Which language each unit is in. A unit the target knows and the base doesn't is the target's, and the
-    // other way round; units both or neither know ("an", "to", names) take the nearest decided neighbor's.
-    // In syllable languages, a base-language word that's a target syllable only once accents are added
-    // ("but": bút) is the base language's; a word only the target knows without its accents ("khong") is
-    // still the target's.
-    async function languages(list, lang, base) {
-        const target = dict(lang);
-        const syllables = await target.syllables();
-        const syllableLanguage = Object.keys(syllables).length > 0;
-        const plainSyllables = syllableLanguage ? new Set(Object.keys(syllables).map(plain)) : new Set();
-        // 'exact': a target word as written; 'accents': one only with accents added ("khong" → "không").
-        const targetKnows = async (u) => {
-            const lower = u.text.toLowerCase();
-            if (!syllableLanguage)
-                return (await target.lookup(lower)).length > 0 ? 'exact' : false;
-            if (lower in syllables || lower !== plain(lower))
-                return 'exact';
-            return plainSyllables.has(lower) ? 'accents' : false;
-        };
-        const baseKnows = async (u) => (await translator.senses(u.text, { from: base }).catch(() => [])).length > 0;
-        const marks = await Promise.all(list.map(async (u) => {
-            if (/\d/.test(u.text))
-                return '?';
-            const [t, b] = await Promise.all([targetKnows(u), baseKnows(u)]);
-            if (t === 'exact')
-                return b ? '?' : lang;
-            if (b)
-                return base;
-            return t === 'accents' ? lang : '?';
-        }));
-        return marks.map((m, i) => {
-            if (m !== '?')
-                return m;
-            for (let d = 1; d < marks.length; d++) {
-                if (i - d >= 0 && marks[i - d] !== '?')
-                    return marks[i - d];
-                if (i + d < marks.length && marks[i + d] !== '?')
-                    return marks[i + d];
-            }
-            return lang;
-        });
-    }
     return {
         async review(entry, o) {
             const text = entry.normalize('NFC');
             const base = o.base ?? 'en';
-            const meta = await dict(o.lang).meta();
             // A frame as shown to the learner: its English and its pattern for these settings, slots open.
             const frameInfo = new Map();
             const frameOf = async (id) => {
@@ -93,7 +52,7 @@ export function createReviewer(options = {}) {
                 const start = list[0].start;
                 const coreEnd = list[list.length - 1].end;
                 const end = coreEnd + (TERMINAL.exec(text.slice(coreEnd))?.[0].length ?? 0);
-                const langs = await languages(list, o.lang, base);
+                const langs = base === o.lang ? list.map(() => o.lang) : await markLanguages(list, { lang: o.lang, base, target: dict(o.lang), baseDictionary: dict(base), translator });
                 const groups = [];
                 list.forEach((u, i) => {
                     if (groups.length && groups[groups.length - 1].lang === langs[i])
@@ -102,38 +61,45 @@ export function createReviewer(options = {}) {
                         groups.push({ lang: langs[i], units: [u] });
                 });
                 const draft = {
-                    start, coreEnd, end, replacements: [], frames: [], unchecked: [],
+                    start, coreEnd, end, replacements: [], frames: [],
                     parts: groups.map((g) => ({ text: text.slice(g.units[0].start, g.units[g.units.length - 1].end), lang: g.lang })),
                 };
                 const whole = groups.length === 1 && groups[0].lang === base;
+                const fill = async (span, frame) => {
+                    const from = text.slice(span[0].start, span[span.length - 1].end);
+                    const info = await frameOf(frame.id);
+                    let to = frame.text.replace(/[.!?…]+$/, '');
+                    if (span[0].start !== start)
+                        to = to.charAt(0).toLowerCase() + to.slice(1);
+                    draft.replacements.push({
+                        start: span[0].start, end: span[span.length - 1].end, to,
+                        change: { kind: 'frame', from, to, why: `Sentence frame “${info.en}”` },
+                    });
+                    draft.frames.push(info);
+                };
                 for (const g of groups.filter((g) => g.lang === base)) {
-                    const from = text.slice(g.units[0].start, g.units[g.units.length - 1].end);
-                    const at = { start: g.units[0].start, end: g.units[g.units.length - 1].end };
                     // A whole sentence is matched with its own punctuation ("Where is the bathroom?").
-                    const frame = await phrasebook.match(whole ? text.slice(start, end) : from, o);
-                    if (frame && frame.complete) {
-                        let to = frame.text.replace(/[.!?…]+$/, '');
-                        if (!whole && g.units[0].start !== start)
-                            to = to.charAt(0).toLowerCase() + to.slice(1);
-                        const info = await frameOf(frame.id);
-                        draft.replacements.push({ ...at, to, change: { kind: 'frame', from, to, why: `Sentence frame “${info.en}”` } });
-                        draft.frames.push(info);
+                    const whole_ = whole ? text.slice(start, end) : text.slice(g.units[0].start, g.units[g.units.length - 1].end);
+                    const frame = await phrasebook.match(whole_, o);
+                    if (frame?.complete) {
+                        await fill(g.units, frame);
                         continue;
                     }
-                    if (!whole && g.units.length <= MAX_TRANSLATED_UNITS) {
-                        const english = from.replace(/^(the|a|an)\s+/i, '');
-                        const [group] = await translator.translate(english, { from: base, to: o.lang, toRegion: o.region, register: o.register, limit: 1 }).catch(() => []);
-                        const word = group?.translations[0]?.word;
-                        if (word) {
-                            draft.replacements.push({ ...at, to: word, change: { kind: 'translated', from, to: word, why: `“${english}” in ${meta.name}` } });
+                    // Otherwise a clause frame ending the part, longest first ("the ice cream but it was expensive":
+                    // "but it was expensive"); the rest is translated word by word by the checker. Only spans ending
+                    // with the part, so a clause isn't cut short ("but it was very" with "expensive" left over).
+                    for (let len = g.units.length - 1; len >= 2; len--) {
+                        const span = g.units.slice(g.units.length - len);
+                        const m = await phrasebook.match(text.slice(span[0].start, span[len - 1].end), o);
+                        if (!m?.complete || !m.clause)
                             continue;
-                        }
+                        await fill(span, m);
+                        break;
                     }
-                    draft.unchecked.push(from);
                 }
                 drafts.push(draft);
             }
-            // The entry with base-language parts replaced, keeping where each sentence now is.
+            // The entry with those parts filled in, keeping where each sentence now is.
             let draftText = '';
             let cursor = 0;
             const placed = [];
@@ -151,41 +117,49 @@ export function createReviewer(options = {}) {
                 cursor = d.end;
             }
             draftText += text.slice(cursor);
-            // 3. The checker on the whole entry; 4. frames on each corrected sentence.
-            const { issues } = await checker.check(draftText, { lang: o.lang, region: o.region, listener: o.listener, about: undefined, speaker: o.speaker });
+            // 3. The checker on the whole entry; 4. the frames each corrected sentence follows.
+            const { issues } = await checker.check(draftText, {
+                lang: o.lang, base, region: o.region, listener: o.listener, speaker: o.speaker, register: o.register, rules: o.checks,
+            });
             const sentences = [];
             for (const [i, d] of drafts.entries()) {
                 const p = placed[i];
                 const mine = issues.filter((x) => x.start >= p.start && x.end <= p.coreEnd);
                 const changes = d.replacements.sort((a, b) => a.start - b.start).map((r) => r.change);
                 const hints = [];
+                const unchecked = [];
+                const applied = [];
                 let core = draftText.slice(p.start, p.coreEnd);
                 let lastStart = Infinity;
                 for (const x of [...mine].sort((a, b) => b.start - a.start)) {
-                    const applies = (x.severity === 'error' || x.severity === 'warning') && x.suggestions.length && x.end <= lastStart;
-                    if (!applies) {
-                        hints.push({ rule: x.rule, text: x.text, message: x.message, suggestions: x.suggestions });
+                    if (!x.suggestions.length) {
+                        if (x.rule === 'foreign-word')
+                            unchecked.unshift(x.text);
+                        else
+                            hints.unshift({ rule: x.rule, text: x.text, message: x.message, suggestions: x.suggestions });
+                        continue;
+                    }
+                    if (x.severity === 'suggestion' || x.end > lastStart) {
+                        hints.unshift({ rule: x.rule, text: x.text, message: x.message, suggestions: x.suggestions });
                         continue;
                     }
                     core = core.slice(0, x.start - p.start) + x.suggestions[0] + core.slice(x.end - p.start);
-                    changes.push({ kind: x.rule, from: x.text, to: x.suggestions[0], why: x.message });
+                    applied.unshift({ kind: x.rule, from: x.text, to: x.suggestions[0], why: x.message });
                     lastStart = x.start;
                 }
-                const punctuation = draftText.slice(p.coreEnd, p.end);
-                let corrected = core + punctuation;
+                changes.push(...applied);
+                core = tidy(core);
+                if (/\p{Lu}/u.test(draftText.charAt(p.start)))
+                    core = core.charAt(0).toUpperCase() + core.slice(1);
+                const corrected = core + draftText.slice(p.coreEnd, p.end);
                 const frames = [...d.frames];
                 for (const m of await phrasebook.matchFrames(corrected, o)) {
                     if (!frames.some((f) => f.id === m.frame.id))
                         frames.push(await frameOf(m.frame.id));
-                    if (m.changes.length && corrected === core + punctuation) {
-                        corrected = m.text;
-                        for (const c of m.changes)
-                            changes.push({ kind: 'frame', from: c.from, to: c.to, why: `${c.why} (sentence frame “${m.frame.en}”)` });
-                    }
                 }
                 sentences.push({
                     start: d.start, end: d.end, original: text.slice(d.start, d.end), parts: d.parts,
-                    corrected, changes, hints: hints.reverse(), frames, unchecked: d.unchecked,
+                    corrected, changes, hints, frames, unchecked,
                 });
             }
             let corrected = '';

package/dist/types.d.ts

@@ -195,6 +195,11 @@ export type CheckerConfig = {
     ambiguousPronouns?: string[];
     /** Sentence endings expected when speaking up (pronoun-table rows marked `respect`), e.g. Vietnamese "ạ". */
     politeEndings?: string[];
+    /**
+     * English words (English is the bridge) this language usually has no word for, so the checker suggests
+     * leaving them out rather than translating them: Vietnamese has no articles ("the", "a", "an").
+     */
+    leaveOut?: string[];
 };
 /**
  * Hand-picked first choices for one English meaning, in one target language (picks.json). A speaker's

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "which-dialect",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Translate words between languages and dialects, meaning by meaning, starting with Vietnamese (Northern, Central, Southern) and English, with Wiktionary data loaded on demand. More languages are planned.",
   "type": "module",
   "main": "./dist/index.js",