@capsiynau/intelligence 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,104 @@ All notable changes to `@capsiynau/intelligence` go here. Format follows
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.5.0] - 2026-07-25
10
+
11
+ Additive. No existing import path changes shape, so 0.4.x consumers can
12
+ upgrade without touching their code.
13
+
14
+ ### Added
15
+
16
+ - **`dialect` subpath.** The Welsh regional lexicon as data rather than as a
17
+ prompt. `applyDialect(text, region)` rewrites off-region word choices
18
+ deterministically and case-preservingly ("Rŵan" to "Nawr", "mas" to
19
+ "allan"); `checkDialect(text, region)` reports the same findings WITHOUT
20
+ changing the text, so a reviewer gets evidence rather than a silent
21
+ rewrite; `listDialectTerms(region)` exposes the table for a management
22
+ screen or a sign-off export. `region` accepts the same values as
23
+ `getDialectSystemPrompt`, with `broadcast` and `formal` both mapping onto
24
+ the neutral lexicon.
25
+
26
+ This does not replace `getDialectSystemPrompt`. Register ("avoid
27
+ colloquialisms") has no deterministic transform and stays a prompt
28
+ concern; word choice does have one and should not be left to a model that
29
+ applies it unevenly and unverifiably. The two compose.
30
+
31
+ Deliberate limits, all pinned by tests: mutated surface forms are not
32
+ matched (needs syntactic context, as in `welsh/mutations.js`), pronoun and
33
+ copula forms are excluded because "mae o'r ardal" would be corrupted, and
34
+ where both regional forms are marked with no neutral standard
35
+ (taid/tad-cu) a neutral target leaves the text alone rather than picking a
36
+ side. Word boundaries use lookarounds over a Welsh letter class, not `\b`,
37
+ which treats the diacritic in "rŵan" as a word boundary and matches inside
38
+ the word.
39
+
40
+ **The seed table is not natively reviewed.** `DIALECT_TERMS_STATUS`
41
+ reports `reviewed: false` in code, and consumers should treat output as a
42
+ draft until that changes.
43
+
44
+ ## [0.4.0] - 2026-07-22
45
+
46
+ First release to ship TypeScript declarations. Everything here is
47
+ additive: no existing import path changes shape, so 0.3.x consumers can
48
+ upgrade without touching their code.
49
+
50
+ ### Added
51
+
52
+ - **`speakers` subpath.** `assessDiarisation(utterances, segments, opts)`
53
+ is a diarisation QUALITY GATE: it computes clipped-union coverage so
54
+ speech outside the transcript window cannot inflate the number,
55
+ suppresses labels below `minCoverage` (0.6), and suppresses a sole
56
+ speaker on audio longer than 90s (the classic diarisation false
57
+ negative, where it failed to separate voices rather than there being
58
+ one voice). `mapSpeakersToNames(acoustic, labelled, opts)` maps
59
+ anonymous "Speaker 1" labels onto real names by greatest time overlap,
60
+ requiring the winner to cover >= 50% of that speaker's talk time AND
61
+ beat the runner-up by 1.5x; ambiguous speakers stay unmapped so the
62
+ caller keeps the anonymous label. `clampedUnionMs` is exported for
63
+ callers that want the coverage primitive on its own. Pure and
64
+ dependency-free.
65
+ - **`corrections/pure` subpath.** `extractProperNounCorrections(before,
66
+ after)` without any of Capsiynau's schema. See "Changed" below.
67
+ - **TypeScript declarations for every subpath.** Generated from the
68
+ existing JSDoc into `types/` and shipped in the tarball, with per-subpath
69
+ `types` conditions in the `exports` map plus a top-level `types` field.
70
+ Consumers no longer need a hand-written ambient shim; Nodiadau's
71
+ `src/types/capsiynau-intelligence.d.ts` is redundant from this release
72
+ and can be deleted in the notes-app repo.
73
+ Regenerate with `npm run build:types` in `packages/intelligence`. The
74
+ output is committed so publishing needs no build step.
75
+ - First unit tests for the corrections module
76
+ (`__tests__/corrections-pure.test.js`, 14 cases). The heuristic had
77
+ shipped since 0.1.0 covered only by a single smoke call in the publish
78
+ workflow.
79
+
80
+ ### Changed
81
+
82
+ - **`corrections` is split into a portable half and a Capsiynau half.**
83
+ `extractProperNounCorrections` (a whitespace-token diff plus the
84
+ Title-Case proper-noun heuristic) moved to `src/corrections/pure.js`
85
+ and is importable as `@capsiynau/intelligence/corrections/pure`.
86
+ `captureCorrections` stays in `src/corrections/tracker.js`: it reads
87
+ `live_segments` and `live_sessions` and writes `word_boost_approved`
88
+ via the `bump_word_boost_approved` RPC, so it only works against
89
+ Capsiynau's database and is not a shared capability.
90
+
91
+ **Backwards-compatible.** `@capsiynau/intelligence/corrections` still
92
+ exports BOTH halves, so existing imports keep working unchanged. Only
93
+ a consumer importing `src/corrections/tracker.js` by deep path would
94
+ notice, and no published entry point allows that.
95
+
96
+ Why: an app on a different database (Postia is on a separate Supabase
97
+ project entirely) could not import the heuristic without inheriting
98
+ tables it does not have. This is the lib-versus-API rule applied:
99
+ pure functions of their inputs ship as library modules, anything
100
+ needing a database ships as an API endpoint.
101
+
102
+ - JSDoc on `spellcheck/checkSentence` and `spellcheck/suggest` now
103
+ documents the option-bag members that were already accepted but
104
+ undocumented (`nspell`, `accept`, `welsh`). No behaviour change; it
105
+ makes the generated declarations accurate rather than `object`.
106
+
9
107
  ## [0.3.0] - 2026-05-19
10
108
 
11
109
  ### Fixed
package/README.md CHANGED
@@ -67,14 +67,27 @@ const errors = checkSentence('Mae cymareg yn iath pwysig', {
67
67
  const fixes = suggest('cymareg', { nspell: cy, max: 6 })
68
68
  // → ['Cymraeg']
69
69
 
70
- // Proper-noun extraction (correction-learning heuristic)
71
- import { extractProperNounCorrections } from '@capsiynau/intelligence/corrections'
70
+ // Proper-noun extraction (correction-learning heuristic). Import from
71
+ // /corrections/pure if you want the heuristic with no Capsiynau schema
72
+ // attached; /corrections re-exports it too.
73
+ import { extractProperNounCorrections } from '@capsiynau/intelligence/corrections/pure'
72
74
 
73
75
  const corrections = extractProperNounCorrections(
74
76
  'Allard Perry spoke at the event',
75
77
  'Aled Parry spoke at the event',
76
78
  )
77
- // → ['Aled Parry']
79
+ // → ['Aled', 'Parry'] (corrected TOKENS, not the phrase)
80
+
81
+ // Diarisation quality gate: decide whether speaker labels are trustworthy
82
+ // enough to show at all.
83
+ import { assessDiarisation } from '@capsiynau/intelligence/speakers'
84
+
85
+ const verdict = assessDiarisation(utterances, segments)
86
+ // e.g. for a 10-minute recording that diarisation collapsed to one speaker:
87
+ // → { reliable: false, speakerCount: 1, coverage: 0.933…, durationMs: 600000,
88
+ // reason: 'sole_speaker_long_audio' }
89
+ // Suppress the labels when reliable === false. "Speaker 1" on everything
90
+ // when four people spoke is worse than showing nothing.
78
91
 
79
92
  // And to persist into word_boost_approved (caller provides the Supabase
80
93
  // client; the package itself has zero Supabase coupling):
@@ -91,9 +104,27 @@ const persisted = await captureCorrections(before, after, { sessionId, supabase
91
104
  |--------|----------|
92
105
  | `@capsiynau/intelligence/welsh` | `normaliseWelsh`, `possibleRoots` + mutation helpers, `splitIntoLetters` + digraph utilities |
93
106
  | `@capsiynau/intelligence/spellcheck` | `checkWord`, `checkSentence`, `tokenise`, `suggest` — mutation-aware, accepts caller's nspell instance |
94
- | `@capsiynau/intelligence/corrections` | `extractProperNounCorrections` (pure), `captureCorrections` (accepts caller-provided Supabase client) |
107
+ | `@capsiynau/intelligence/corrections` | Both halves below, for backwards compatibility |
108
+ | `@capsiynau/intelligence/corrections/pure` | `extractProperNounCorrections` - portable, no database |
109
+ | `@capsiynau/intelligence/speakers` | `assessDiarisation` (diarisation quality gate), `mapSpeakersToNames`, `clampedUnionMs` |
95
110
  | `@capsiynau/intelligence/glossary` | Glossary Builder pipeline helpers: `parse`, `prompts`, `tools`, `url`, `import`, `persist` |
96
111
 
112
+ Every subpath ships TypeScript declarations (`types/`, generated from the
113
+ JSDoc), so no ambient shim is needed in a TS consumer.
114
+
115
+ ### Portability
116
+
117
+ Two exports are **Capsiynau-specific** and will not work against another
118
+ database. They stay in the package but are not shared capabilities:
119
+
120
+ - `captureCorrections` (`/corrections`) reads `live_segments` and
121
+ `live_sessions`, writes `word_boost_approved` via the
122
+ `bump_word_boost_approved` RPC. Import `/corrections/pure` instead if you
123
+ only want the heuristic.
124
+ - `glossary/persist` builds rows shaped to Capsiynau's `glossaries` table.
125
+
126
+ Everything else is a pure function of its inputs.
127
+
97
128
  ## Design
98
129
 
99
130
  - **Caller-provided dictionaries.** The package doesn't bundle nspell or
package/package.json CHANGED
@@ -1,23 +1,24 @@
1
1
  {
2
2
  "name": "@capsiynau/intelligence",
3
- "version": "0.3.0",
3
+ "version": "0.5.0",
4
4
  "description": "Bilingual (Welsh + English) intelligence layer: text normalisation, mutation handling, digraph splitting, spellcheck primitives, and glossary/correction utilities. Pure functions, runs in Node or browser.",
5
5
  "type": "module",
6
6
  "license": "MIT",
7
7
  "author": "Aled Parry <aled@aledparry.com>",
8
- "homepage": "https://github.com/aledprysparry/Capsiynau_V2_ClaudeAI/tree/main/packages/intelligence",
8
+ "homepage": "https://github.com/aledprysparry/Capsiynau.com/tree/main/packages/intelligence",
9
9
  "repository": {
10
10
  "type": "git",
11
- "url": "git+https://github.com/aledprysparry/Capsiynau_V2_ClaudeAI.git",
11
+ "url": "git+https://github.com/aledprysparry/Capsiynau.com.git",
12
12
  "directory": "packages/intelligence"
13
13
  },
14
14
  "bugs": {
15
- "url": "https://github.com/aledprysparry/Capsiynau_V2_ClaudeAI/issues"
15
+ "url": "https://github.com/aledprysparry/Capsiynau.com/issues"
16
16
  },
17
17
  "keywords": [
18
18
  "welsh",
19
19
  "cymraeg",
20
20
  "bilingual",
21
+ "dialect",
21
22
  "spellcheck",
22
23
  "mutation",
23
24
  "digraph",
@@ -27,15 +28,44 @@
27
28
  "capsiynau"
28
29
  ],
29
30
  "main": "./src/index.js",
31
+ "types": "./types/index.d.ts",
30
32
  "exports": {
31
- ".": "./src/index.js",
32
- "./welsh": "./src/welsh/index.js",
33
- "./corrections": "./src/corrections/index.js",
34
- "./glossary": "./src/glossary/index.js",
35
- "./spellcheck": "./src/spellcheck/index.js"
33
+ ".": {
34
+ "types": "./types/index.d.ts",
35
+ "default": "./src/index.js"
36
+ },
37
+ "./welsh": {
38
+ "types": "./types/welsh/index.d.ts",
39
+ "default": "./src/welsh/index.js"
40
+ },
41
+ "./corrections": {
42
+ "types": "./types/corrections/index.d.ts",
43
+ "default": "./src/corrections/index.js"
44
+ },
45
+ "./corrections/pure": {
46
+ "types": "./types/corrections/pure.d.ts",
47
+ "default": "./src/corrections/pure.js"
48
+ },
49
+ "./glossary": {
50
+ "types": "./types/glossary/index.d.ts",
51
+ "default": "./src/glossary/index.js"
52
+ },
53
+ "./spellcheck": {
54
+ "types": "./types/spellcheck/index.d.ts",
55
+ "default": "./src/spellcheck/index.js"
56
+ },
57
+ "./speakers": {
58
+ "types": "./types/speakers/index.d.ts",
59
+ "default": "./src/speakers/index.js"
60
+ },
61
+ "./dialect": {
62
+ "types": "./types/dialect/index.d.ts",
63
+ "default": "./src/dialect/index.js"
64
+ }
36
65
  },
37
66
  "files": [
38
67
  "src",
68
+ "types",
39
69
  "README.md",
40
70
  "SCHEMA.md",
41
71
  "CHANGELOG.md",
@@ -58,7 +88,8 @@
58
88
  "access": "public"
59
89
  },
60
90
  "scripts": {
61
- "test": "vitest run __tests__"
91
+ "test": "vitest run __tests__",
92
+ "build:types": "tsc -p tsconfig.json"
62
93
  },
63
94
  "sideEffects": false
64
95
  }
@@ -1,6 +1,16 @@
1
1
  /**
2
2
  * @capsiynau/intelligence/corrections
3
3
  *
4
- * Public surface for the correction-learning loop.
4
+ * Public surface for the correction-learning loop. Re-exports BOTH
5
+ * halves so this import path keeps its full 0.3.0 surface:
6
+ *
7
+ * - `./pure.js` portable. Diff + proper-noun heuristic. No database.
8
+ * - `./tracker.js` Capsiynau-specific. Reads live_sessions /
9
+ * live_segments, writes word_boost_approved.
10
+ *
11
+ * A consumer on a different schema should import
12
+ * `@capsiynau/intelligence/corrections/pure` instead, and get the
13
+ * heuristic without the schema coupling.
5
14
  */
15
+ export * from './pure.js'
6
16
  export * from './tracker.js'
@@ -0,0 +1,133 @@
1
+ /**
2
+ * @capsiynau/intelligence/corrections/pure
3
+ *
4
+ * The PORTABLE half of the correction-learning loop: a whitespace-token
5
+ * diff plus a Title-Case proper-noun heuristic. Pure functions of their
6
+ * inputs - no database, no API key, no spend, no per-tenant state - so
7
+ * per the lib-versus-API rule this half is a library capability that any
8
+ * consuming app can import.
9
+ *
10
+ * Split out of `./tracker.js` in 0.4.0. The other half,
11
+ * `captureCorrections()`, reads `live_segments` / `live_sessions` and
12
+ * writes `word_boost_approved` via the `bump_word_boost_approved` RPC:
13
+ * that is Capsiynau's schema, not a shared capability, and an app on a
14
+ * different database cannot use it. Importing from here gets you the
15
+ * heuristic without inheriting any of that.
16
+ *
17
+ * `./corrections` still re-exports both halves, so existing consumers
18
+ * (Nodiadau's src/lib/correctionTracker.ts) keep working unchanged.
19
+ *
20
+ * Ported verbatim (TS -> JS) from Nodiadau commit 5d75dde
21
+ * (aledprysparry/notes-app#239). Keep parity with the Nodiadau
22
+ * implementation - any heuristic change here must mirror in
23
+ * notes-app/src/lib/correctionTracker.ts (or vice versa) or the two
24
+ * apps' learning behaviour silently diverges.
25
+ */
26
+
27
+ /**
28
+ * Words that are *not* worth promoting even if Title-Case. Keep narrow -
29
+ * this is a deny-list, not a dictionary.
30
+ */
31
+ const PROPER_NOUN_STOPWORDS = new Set([
32
+ // English filler
33
+ 'The', 'This', 'That', 'These', 'Those', 'There', 'Their', 'They',
34
+ 'Then', 'When', 'Where', 'What', 'Which', 'While', 'Whose', 'Whom',
35
+ 'With', 'Without', 'About', 'Above', 'After', 'Again', 'Against',
36
+ 'Also', 'Some', 'Such', 'Same', 'Should', 'Would', 'Could', 'Might',
37
+ 'Have', 'Been', 'Were', 'From', 'Into', 'Onto',
38
+ // Welsh filler (Title-Case start-of-sentence is common in Welsh too)
39
+ 'Mae', "Mae'r", "Mae'n", 'Dyma', 'Dyna', 'Roedd', "Roedd'r",
40
+ 'Rwy', 'Rwyt', 'Bydd', "Bydd'r", 'Bod', 'Cael',
41
+ 'Wedi', 'Wedyn', 'Felly', 'Ond', 'Achos', 'Ar', "Ar y",
42
+ 'Yn', 'Mewn', 'Heb', 'Drwy', 'Drwyddo', 'Drwyddi', 'Drwyddyn',
43
+ ])
44
+
45
+ const MIN_PROPER_NOUN_LENGTH = 4
46
+ const ALPHA_ONLY = /^[\p{L}\p{M}'-]+$/u
47
+
48
+ /**
49
+ * Tokenise on whitespace AND strip trailing punctuation per word. Keeps
50
+ * in-word punctuation (apostrophes, hyphens) which Welsh names need
51
+ * ("ap Iolo", "Tŷ'r-pol", etc.).
52
+ *
53
+ * @param {string} text
54
+ * @returns {string[]}
55
+ */
56
+ function tokenise(text) {
57
+ return text
58
+ .trim()
59
+ .split(/\s+/)
60
+ .map((tok) => tok.replace(/^[\p{P}\p{S}]+|[\p{P}\p{S}]+$/gu, ''))
61
+ .filter(Boolean)
62
+ }
63
+
64
+ /**
65
+ * Returns true if the token looks like a name/place/brand we want to bias
66
+ * toward. Heuristics, not a dictionary.
67
+ *
68
+ * --- PART OF THE LANGUAGE-AGNOSTIC CONTRACT --------------------------
69
+ * The relay's `word_boost_approved` loader (worker/live-relay.js) reads
70
+ * rows from this table language-agnostically, trusting that this
71
+ * heuristic only matches semantically language-invariant terms (Title-
72
+ * Case proper nouns: personal names, brand names, place names).
73
+ *
74
+ * If you ever loosen this heuristic to also match per-language vocabulary
75
+ * (hyphenated compounds, lowercase acronyms, dialect markers, domain
76
+ * lexicon, etc.), the loader has to follow. Either:
77
+ * (a) re-introduce a per-language filter on the read side, OR
78
+ * (b) add a row-level `word_boost_approved.cross_language` discriminator
79
+ * so the loader can decide per row (Phase 1 of notes-app#262 queues
80
+ * this micro-add).
81
+ *
82
+ * Change the heuristic, change the loader.
83
+ * ---------------------------------------------------------------------
84
+ *
85
+ * @param {string} token
86
+ * @returns {boolean}
87
+ */
88
+ function looksLikeProperNoun(token) {
89
+ if (!token || token.length < MIN_PROPER_NOUN_LENGTH) return false
90
+ if (!ALPHA_ONLY.test(token)) return false
91
+ const first = token[0]
92
+ // Must start with an upper-case letter
93
+ if (first.toUpperCase() !== first || first.toLowerCase() === first) return false
94
+ if (PROPER_NOUN_STOPWORDS.has(token)) return false
95
+ // Avoid promoting ALL-CAPS strings (likely acronyms or shouting)
96
+ if (token.toUpperCase() === token && token.length > 4) return false
97
+ return true
98
+ }
99
+
100
+ /**
101
+ * Compute the set of proper-noun corrections worth promoting.
102
+ *
103
+ * Diffs `before` against `after` on whitespace tokens and returns the
104
+ * corrected tokens that look like proper nouns. Structural edits (the
105
+ * token count changed) return `[]`: word-for-word swaps are the
106
+ * high-signal class and everything else is too noisy to learn from.
107
+ *
108
+ * In Capsiynau these terms become rows in `word_boost_approved` (per-org,
109
+ * tagged with the session's language) and the relay's loader treats those
110
+ * rows as language-agnostic on read - see the contract note on
111
+ * `looksLikeProperNoun` above for the full invariant and what must change
112
+ * if the heuristic ever expands beyond proper nouns. The function itself
113
+ * knows nothing about that table; a consumer on a different schema can
114
+ * do whatever it likes with the returned terms.
115
+ *
116
+ * @param {string} before Original text before the edit.
117
+ * @param {string} after Text after the edit.
118
+ * @returns {string[]} Deduplicated corrected tokens worth biasing toward.
119
+ */
120
+ export function extractProperNounCorrections(before, after) {
121
+ const a = tokenise(before)
122
+ const b = tokenise(after)
123
+ // Skip structural changes (token count differs) - same heuristic as the
124
+ // 2am vocabulary-learning cron. Word-for-word swaps are the high-signal class.
125
+ if (a.length !== b.length || a.length === 0) return []
126
+ const out = new Set()
127
+ for (let i = 0; i < a.length; i++) {
128
+ if (a[i] === b[i]) continue
129
+ const corrected = b[i]
130
+ if (looksLikeProperNoun(corrected)) out.add(corrected)
131
+ }
132
+ return Array.from(out)
133
+ }
@@ -1,74 +1,59 @@
1
1
  /**
2
2
  * @capsiynau/intelligence/corrections/tracker
3
3
  *
4
+ * The CAPSIYNAU-PERSISTENT half of the correction-learning loop. This
5
+ * module is NOT portable: it reads `live_segments` and `live_sessions`
6
+ * and writes `word_boost_approved` via the `bump_word_boost_approved`
7
+ * RPC. An app on a different database cannot use it. The Supabase client
8
+ * is dependency-injected (0.2.0), which removed the module-alias coupling
9
+ * but not the schema coupling.
10
+ *
11
+ * The portable half - the diff plus the proper-noun heuristic - lives in
12
+ * `./pure.js` and was split out here in 0.4.0 so a consumer can import
13
+ * `@capsiynau/intelligence/corrections/pure` without inheriting any of
14
+ * Capsiynau's schema. `./index.js` re-exports both halves, so importing
15
+ * `@capsiynau/intelligence/corrections` still gets you everything.
16
+ *
4
17
  * Moved here in Phase A (see docs/intelligence-layer-phase-a.md +
5
18
  * PR #448) so Nodiadau can git-subtree this package rather than
6
19
  * maintain a hand-mirrored copy. Original path: src/lib/correctionTracker.js.
7
20
  *
8
- * Ported verbatim (TS → JS) from Nodiadau commit 5d75dde
9
- * (aledprysparry/notes-app#239) so Capsiynau Live edits feed the same
10
- * closed-loop vocabulary learning channel.
11
- *
12
21
  * When a moderator fixes a Whisper mis-recognition in the live transcript
13
- * (e.g. "Allard Perry" "Aled Parry"), we want the model to recognise the
22
+ * (e.g. "Allard Perry" -> "Aled Parry"), we want the model to recognise the
14
23
  * corrected term next time. The Capsiynau live-relay reads
15
24
  * `word_boost_approved` for the session's org + language and injects
16
25
  * matching terms into the Whisper `prompt` parameter (PR #421).
17
26
  *
18
27
  * Flow:
19
- * 1. Diff the segment text before/after the edit (whitespace-tokenised).
20
- * 2. Find equal-length word swaps if the lengths differ, skip; structural
21
- * changes are too noisy to feed back into vocabulary biasing.
22
- * 3. For each swap, decide whether the corrected token looks like a proper
23
- * noun worth biasing (Title-Case, ≥4 chars, not a Welsh/English
24
- * start-of-sentence stopword, alphabetic-only).
25
- * 4. Upsert into word_boost_approved with the session's organisation_id +
26
- * source_language. RLS scopes by org via get_user_org_ids() so a term
27
- * never leaks into another tenant's bias.
28
- *
29
- * Keep parity with the Nodiadau implementation — any heuristic change here
30
- * must mirror in notes-app/src/lib/correctionTracker.ts (or vice versa) or
31
- * the two apps' learning behaviour silently diverges.
28
+ * 1. Diff the segment text before/after the edit and pick out the
29
+ * proper-noun swaps worth learning (`./pure.js`).
30
+ * 2. Upsert them into word_boost_approved with the session's
31
+ * organisation_id + source_language. RLS scopes by org via
32
+ * get_user_org_ids() so a term never leaks into another tenant's bias.
32
33
  */
33
34
 
34
35
  // Phase B (2026-05-17, PR #495): the Supabase client is passed in by
35
36
  // the caller rather than imported from a Capsiynau-specific path. This
36
37
  // removes the package's only coupling to the host app's module alias
37
38
  // (`@/api/supabaseClient`) and lets Nodiadau consume captureCorrections
38
- // without a re-export shim. Breaking API change vs 0.1.0 bumped to
39
+ // without a re-export shim. Breaking API change vs 0.1.0 - bumped to
39
40
  // 0.2.0. Callers now pass `supabase` in the options bag alongside
40
41
  // `sessionId`.
41
42
 
42
- /**
43
- * Words that are *not* worth promoting even if Title-Case. Keep narrow —
44
- * this is a deny-list, not a dictionary.
45
- */
46
- const PROPER_NOUN_STOPWORDS = new Set([
47
- // English filler
48
- 'The', 'This', 'That', 'These', 'Those', 'There', 'Their', 'They',
49
- 'Then', 'When', 'Where', 'What', 'Which', 'While', 'Whose', 'Whom',
50
- 'With', 'Without', 'About', 'Above', 'After', 'Again', 'Against',
51
- 'Also', 'Some', 'Such', 'Same', 'Should', 'Would', 'Could', 'Might',
52
- 'Have', 'Been', 'Were', 'From', 'Into', 'Onto',
53
- // Welsh filler (Title-Case start-of-sentence is common in Welsh too)
54
- 'Mae', "Mae'r", "Mae'n", 'Dyma', 'Dyna', 'Roedd', "Roedd'r",
55
- 'Rwy', 'Rwyt', 'Bydd', "Bydd'r", 'Bod', 'Cael',
56
- 'Wedi', 'Wedyn', 'Felly', 'Ond', 'Achos', 'Ar', "Ar y",
57
- 'Yn', 'Mewn', 'Heb', 'Drwy', 'Drwyddo', 'Drwyddi', 'Drwyddyn',
58
- ])
59
-
60
- const MIN_PROPER_NOUN_LENGTH = 4
61
- const ALPHA_ONLY = /^[\p{L}\p{M}'-]+$/u
43
+ import { extractProperNounCorrections } from './pure.js'
62
44
 
63
45
  /**
64
46
  * Collapse the session's source_language to the two-bucket convention used
65
47
  * everywhere else in this repo for STT bias loading: cy-GB (any Welsh
66
- * variant broadcast, north, south, formal) or en-GB (everything else,
48
+ * variant - broadcast, north, south, formal) or en-GB (everything else,
67
49
  * including NULL).
68
50
  *
69
51
  * Pre-this fix, this module defaulted unknown / NULL to 'cy-GB', which
70
52
  * silently routed English users' legacy-session learned vocab into the
71
53
  * Welsh bias bucket. Mirrors notes-app/src/lib/correctionTracker.ts.
54
+ *
55
+ * @param {string|null|undefined} source
56
+ * @returns {'cy-GB'|'en-GB'}
72
57
  */
73
58
  function normaliseLanguageBucket(source) {
74
59
  // Case-insensitive: legacy / externally-written source_language values
@@ -79,85 +64,19 @@ function normaliseLanguageBucket(source) {
79
64
  return 'en-GB'
80
65
  }
81
66
 
82
- /**
83
- * Tokenise on whitespace AND strip trailing punctuation per word. Keeps
84
- * in-word punctuation (apostrophes, hyphens) which Welsh names need
85
- * ("ap Iolo", "Tŷ'r-pol", etc.).
86
- */
87
- function tokenise(text) {
88
- return text
89
- .trim()
90
- .split(/\s+/)
91
- .map((tok) => tok.replace(/^[\p{P}\p{S}]+|[\p{P}\p{S}]+$/gu, ''))
92
- .filter(Boolean)
93
- }
94
-
95
- /**
96
- * Returns true if the token looks like a name/place/brand we want to bias
97
- * toward. Heuristics, not a dictionary.
98
- *
99
- * ─── PART OF THE LANGUAGE-AGNOSTIC CONTRACT ──────────────────────────
100
- * The relay's `word_boost_approved` loader (worker/live-relay.js) reads
101
- * rows from this table language-agnostically, trusting that this
102
- * heuristic only matches semantically language-invariant terms (Title-
103
- * Case proper nouns: personal names, brand names, place names).
104
- *
105
- * If you ever loosen this heuristic to also match per-language vocabulary
106
- * (hyphenated compounds, lowercase acronyms, dialect markers, domain
107
- * lexicon, etc.), the loader has to follow. Either:
108
- * (a) re-introduce a per-language filter on the read side, OR
109
- * (b) add a row-level `word_boost_approved.cross_language` discriminator
110
- * so the loader can decide per row (Phase 1 of notes-app#262 queues
111
- * this micro-add).
112
- *
113
- * Change the heuristic, change the loader.
114
- * ─────────────────────────────────────────────────────────────────────
115
- */
116
- function looksLikeProperNoun(token) {
117
- if (!token || token.length < MIN_PROPER_NOUN_LENGTH) return false
118
- if (!ALPHA_ONLY.test(token)) return false
119
- const first = token[0]
120
- // Must start with an upper-case letter
121
- if (first.toUpperCase() !== first || first.toLowerCase() === first) return false
122
- if (PROPER_NOUN_STOPWORDS.has(token)) return false
123
- // Avoid promoting ALL-CAPS strings (likely acronyms or shouting)
124
- if (token.toUpperCase() === token && token.length > 4) return false
125
- return true
126
- }
127
-
128
- /**
129
- * Compute the set of proper-noun corrections worth promoting. Exported
130
- * for unit tests.
131
- *
132
- * Returned terms become rows in `word_boost_approved` (per-org, with the
133
- * session's source_language tag). The relay's loader treats those rows
134
- * as language-agnostic on read - see the contract note on
135
- * `looksLikeProperNoun` above for the full invariant and what must
136
- * change if the heuristic ever expands beyond proper nouns.
137
- */
138
- export function extractProperNounCorrections(before, after) {
139
- const a = tokenise(before)
140
- const b = tokenise(after)
141
- // Skip structural changes (token count differs) — same heuristic as the
142
- // 2am vocabulary-learning cron. Word-for-word swaps are the high-signal class.
143
- if (a.length !== b.length || a.length === 0) return []
144
- const out = new Set()
145
- for (let i = 0; i < a.length; i++) {
146
- if (a[i] === b[i]) continue
147
- const corrected = b[i]
148
- if (looksLikeProperNoun(corrected)) out.add(corrected)
149
- }
150
- return Array.from(out)
151
- }
152
-
153
67
  /**
154
68
  * Upsert proper-noun corrections into word_boost_approved for the session's
155
69
  * org + source language. Returns the list of terms persisted (excludes
156
- * duplicates already in the table upsert is silent on conflict).
70
+ * duplicates already in the table - upsert is silent on conflict).
157
71
  *
158
- * Never throws failures are logged and swallowed so a segment-edit save
72
+ * Never throws - failures are logged and swallowed so a segment-edit save
159
73
  * is never blocked by a vocabulary-learning hiccup.
160
74
  *
75
+ * CAPSIYNAU-SPECIFIC. Requires `live_sessions`, `live_segments` and
76
+ * `word_boost_approved`. If you only want the heuristic, import
77
+ * `extractProperNounCorrections` from
78
+ * `@capsiynau/intelligence/corrections/pure`.
79
+ *
161
80
  * @param {string} before Original segment text before the edit.
162
81
  * @param {string} after Text after the edit.
163
82
  * @param {object} opts
@@ -174,6 +93,7 @@ export function extractProperNounCorrections(before, after) {
174
93
  * @param {object} opts.supabase Caller-provided Supabase client (DI).
175
94
  * Must have `.from()` and `.rpc()` methods.
176
95
  * RLS scopes writes by org.
96
+ * @returns {Promise<string[]>} Terms persisted (empty on any failure).
177
97
  */
178
98
  export async function captureCorrections(before, after, { sessionId, segmentId, supabase }) {
179
99
  if (!supabase) {
@@ -225,7 +145,7 @@ export async function captureCorrections(before, after, { sessionId, segmentId,
225
145
 
226
146
  const language = normaliseLanguageBucket(segmentLanguage || sess.source_language)
227
147
 
228
- // Phase 4.2 call the SECURITY DEFINER RPC for an atomic
148
+ // Phase 4.2 - call the SECURITY DEFINER RPC for an atomic
229
149
  // INSERT ... ON CONFLICT DO UPDATE SET confirmed_by_count =
230
150
  // confirmed_by_count + 1, last_confirmed_at = now(). This is what makes
231
151
  // repeat confirmations of the same proper noun count toward bias
@@ -246,12 +166,12 @@ export async function captureCorrections(before, after, { sessionId, segmentId,
246
166
  p_auto_approved: true,
247
167
  })
248
168
  if (rpcErr) {
249
- // RPC not deployed on this env bail out and fall back to upsert.
169
+ // RPC not deployed on this env -> bail out and fall back to upsert.
250
170
  if (rpcErr.code === 'PGRST202' || /Could not find the function/i.test(rpcErr.message || '')) {
251
171
  rpcMissing = true
252
172
  break
253
173
  }
254
- // Other errors (forbidden, network) log and skip this term.
174
+ // Other errors (forbidden, network) - log and skip this term.
255
175
  console.warn('[correctionTracker] bump_word_boost_approved error:', rpcErr.message)
256
176
  continue
257
177
  }