@capsiynau/intelligence 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,112 @@
+# Changelog
+
+All notable changes to `@capsiynau/intelligence` go here. Format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versions follow
+[SemVer](https://semver.org/).
+
+## [Unreleased]
+
+### Fixed
+
+- `corrections/captureCorrections` now binds the `live_segments` lookup
+  to `(id, session_id)` together when `segmentId` is provided, not
+  `id` alone. A stale or mismatched `segmentId` from a client race or
+  tampered payload could otherwise reference a row in a different
+  session within the same org and contaminate the `word_boost_approved`
+  bucket with that other segment's language. RLS prevents cross-org
+  leakage but not cross-session-within-org. Codex follow-up on PR #511.
+
+### Added
+
+- `corrections/captureCorrections` now accepts an optional `segmentId`
+  in its options. When provided, the function looks up the segment's
+  per-row `language` column (from `live_segments.language`, shipped in
+  Capsiynau PR #509) and uses it as the bucket for the new
+  `word_boost_approved` row. Falls back to `session.source_language`
+  if `segmentId` is absent or the segment's language is null.
+- Why: in a mixed-language session (Phase 3 of notes-app#262 ships
+  per-chunk Whisper auto-detect), a correction in the non-primary
+  language would otherwise land in the wrong bucket. Phase 2 of the
+  sprint plumbs the segment-aware path now so corrections start
+  landing correctly the moment Phase 3 ships.
+- Backwards-compatible: existing callers that pass only `sessionId`
+  keep working exactly as before. The new parameter is opt-in.
+
+### Documented
+
+- Contract docstring on `corrections/tracker.looksLikeProperNoun` and
+  `corrections/tracker.extractProperNounCorrections` formalises the
+  language-agnostic invariant the relay's `word_boost_approved` loader
+  depends on. Proper nouns persisted by this heuristic are read across
+  languages so a name corrected in a Welsh session biases the next
+  English session in the same org. Loosening the heuristic to match
+  per-language vocabulary requires a coordinated change on the read
+  side - see the docstring for the two options (per-language filter
+  or row-level `cross_language` discriminator per Phase 1 of
+  notes-app#262).
+- No behaviour change. Companion to Capsiynau PR #504 on
+  `feature/phase-15-live-relay` which ships the read-side fix.
+
+## [0.2.0] - 2026-05-17
+
+### Changed (BREAKING)
+
+- `corrections/captureCorrections(before, after, opts)` now requires
+  `opts.supabase` (caller-provided Supabase client). Previously the
+  function imported `@/api/supabaseClient` directly, which coupled the
+  package to a host-app-specific module alias. Removing that import
+  unblocks the first npm publish and lets Nodiadau consume
+  `captureCorrections` without a `@/api/supabaseClient` shim.
+
+### Fixed
+
+- `corrections/normaliseLanguageBucket` now matches `cy` prefixes
+  case-insensitively. Legacy / externally-written `source_language`
+  values like `'CY-GB'` or `'Cy-GB-north'` were previously classified
+  as `en-GB`, sending Welsh learned terms into the wrong bias bucket
+  (Codex catch on notes-app#261). One-line `.toLowerCase()` fix; no
+  behaviour change for already-lowercase values.
+
+  **Migration**:
+
+  ```js
+  // Before (0.1.0):
+  await captureCorrections(before, after, { sessionId })
+
+  // After (0.2.0):
+  import { supabase } from '@/api/supabaseClient'  // or your own client
+  await captureCorrections(before, after, { sessionId, supabase })
+  ```
+
+  The function now calls `console.warn` and returns `[]` when
+  `opts.supabase` is missing — never throws. All other behaviour
+  (proper-noun heuristic, RPC fallback, NULL-language en-GB default)
+  is unchanged.
+
+- README "Known issues" block on `captureCorrections` removed — the
+  Supabase coupling is gone in this release.
+
+## [0.1.0] - 2026-05-15
+
+Initial extraction from `src/lib/` and `api/_lib/`. No behaviour change
+versus the original in-repo paths — old import paths still resolve via
+re-export shims during Phase A.
+
+### Added
+
+- `welsh/normaliser` — Welsh text normalisation (diacritics, basic forms).
+  Moved from `src/lib/welshNormaliser.js` + `api/_lib/welshNormaliser.js`.
+- `welsh/mutations` — soft / nasal / aspirate mutation helpers.
+  Extracted from the normaliser.
+- `welsh/digraphs` — handling for `ch`, `dd`, `ff`, `ng`, `ll`, `ph`, `rh`, `th`.
+- `corrections/extractProperNounCorrections` — Title-Case proper-noun
+  heuristic. Pure function.
+- `corrections/captureCorrections` — writes promoted corrections to
+  `word_boost_approved`. **Currently couples to Supabase via direct
+  import** — see Unreleased / Known issues above.
+- `glossary/*` — pure utilities for the Glossary Builder pipeline:
+  `parse`, `prompts`, `tools`, `url`, `import`, `persist`.
+- `spellcheck/core` — `checkWord`, `checkSentence`, `tokenise`, `suggest`.
+  Mutation-aware. Accepts a caller-provided nspell instance (the package
+  itself does not bundle `nspell` or dictionary data — consumers bring
+  their own via the optional peer deps).

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Aled Parry / Capsiynau
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,134 @@
+# @capsiynau/intelligence
+
+Bilingual (Welsh + English) intelligence layer: text normalisation, mutation
+handling, digraph splitting, spellcheck primitives, and glossary / correction
+utilities.
+
+**Pure functions, runs in Node or browser.** No DOM dependencies, no
+network calls, no Supabase coupling — the one export that writes to
+Supabase (`captureCorrections`) takes the client as an argument so the
+package itself stays universal.
+
+## Install
+
+```bash
+npm install @capsiynau/intelligence
+
+# Spellcheck consumers also need the optional peer deps:
+npm install nspell dictionary-cy dictionary-en-gb
+```
+
+The dictionary peer-deps are optional — only required if you use the
+`/spellcheck` subpath. Welsh normalisation, mutations, digraphs, and the
+glossary helpers all work without them.
+
+## Quick start
+
+```js
+// Welsh normalisation (whitespace, casing, common typos for downstream
+// comparison — see SCHEMA.md for the exact rules)
+import { normaliseWelsh } from '@capsiynau/intelligence/welsh'
+
+normaliseWelsh('  Gymraeg ')  // → 'Gymraeg'
+
+// Mutation helpers — possibleRoots returns every plausible root form
+// for a possibly-mutated surface word, tagged with the mutation kind
+import { possibleRoots } from '@capsiynau/intelligence/welsh'
+
+possibleRoots('Gymraeg')
+// → [{ root: 'Gymraeg', kind: null }, { root: 'Cymraeg', kind: 'soft' }]
+
+// Digraph-aware letter splitting (Welsh treats `ch`/`dd`/`ll`/... as single letters)
+import { splitIntoLetters, welshLength } from '@capsiynau/intelligence/welsh'
+
+splitIntoLetters('llyfr')  // → ['ll', 'y', 'f', 'r']
+welshLength('llyfr')       // → 4 (not 5)
+
+// Spellcheck (caller provides the nspell instance — dictionary-cy v2+ is
+// already a parsed `{aff, dic}` object, so the construction is synchronous)
+import nspell from 'nspell'
+import cyDict from 'dictionary-cy'
+import { checkSentence, suggest } from '@capsiynau/intelligence/spellcheck'
+
+const cy = nspell(cyDict)
+
+// `accept` is a Set of proper nouns / glossary terms / learned-vocab
+// (lowercased) that must never be flagged. `welsh: true` enables
+// mutation-aware fallback so `gath` resolves to `cath` via soft-mutation
+// reversal before the word is declared misspelled.
+const errors = checkSentence('Mae cymareg yn iath pwysig', {
+  nspell: cy,
+  accept: new Set(),
+  welsh: true,
+})
+// → [{ word: 'cymareg', start: 4, end: 11 }, { word: 'iath', start: 15, end: 19 }]
+
+// Suggestions per misspelled token
+const fixes = suggest('cymareg', { nspell: cy, max: 6 })
+// → ['Cymraeg']
+
+// Proper-noun extraction (correction-learning heuristic)
+import { extractProperNounCorrections } from '@capsiynau/intelligence/corrections'
+
+const corrections = extractProperNounCorrections(
+  'Allard Perry spoke at the event',
+  'Aled Parry spoke at the event',
+)
+// → ['Aled Parry']
+
+// And to persist into word_boost_approved (caller provides the Supabase
+// client; the package itself has zero Supabase coupling):
+import { captureCorrections } from '@capsiynau/intelligence/corrections'
+import { supabase } from './your-supabase-client'  // your own client
+
+const persisted = await captureCorrections(before, after, { sessionId, supabase })
+// → array of terms written (silent on duplicates; never throws)
+```
+
+## Subpaths
+
+| Import | Provides |
+|--------|----------|
+| `@capsiynau/intelligence/welsh` | `normaliseWelsh`, `possibleRoots` + mutation helpers, `splitIntoLetters` + digraph utilities |
+| `@capsiynau/intelligence/spellcheck` | `checkWord`, `checkSentence`, `tokenise`, `suggest` — mutation-aware, accepts caller's nspell instance |
+| `@capsiynau/intelligence/corrections` | `extractProperNounCorrections` (pure), `captureCorrections` (accepts caller-provided Supabase client) |
+| `@capsiynau/intelligence/glossary` | Glossary Builder pipeline helpers: `parse`, `prompts`, `tools`, `url`, `import`, `persist` |
+
+## Design
+
+- **Caller-provided dictionaries.** The package doesn't bundle nspell or
+  hunspell data — those are ~700 KB Welsh + ~250 KB English. Consumers
+  bring their own so they can lazy-load, cache in IndexedDB, or use
+  WebAssembly variants without the package dictating choice.
+- **Pure where possible.** Welsh / spellcheck / glossary modules are
+  side-effect-free. Easy to test, easy to compose, runs in any JS runtime.
+- **Mutation-aware spellcheck.** A Welsh spellchecker that doesn't
+  understand initial consonant mutations is useless on real Welsh text.
+  `checkWord` reverses soft / nasal / aspirate mutations before looking
+  up the root, so `Gymraeg` and `Nghymraeg` both resolve to `Cymraeg`.
+
+## Versioning
+
+- **Patch** (`0.1.x`) — bug fixes, no API change
+- **Minor** (`0.x.0`) — additive API, no consumer changes required
+- **Major** (`x.0.0`) — breaking changes
+
+Pre-1.0: the API is firming up. Pin a minor version in production
+consumers (`"@capsiynau/intelligence": "~0.1.0"`) and watch the
+CHANGELOG for breaking moves.
+
+## Supabase schema contract
+
+See `SCHEMA.md` for the public contract on the tables the glossary +
+corrections modules interact with (when wired up by the caller).
+
+## Cross-repo sync (Capsiynau ecosystem)
+
+Capsiynau consumes this package directly via the monorepo path.
+Nodiadau consumes via either `npm install @capsiynau/intelligence`
+(once published) or `git subtree pull` (canonical pattern documented
+at `docs/publish-intelligence-package.md`).
+
+## License
+
+MIT. See [LICENSE](./LICENSE).

package/SCHEMA.md

@@ -0,0 +1,75 @@
+# @capsiynau/intelligence — Supabase schema contract
+
+This document records the Supabase tables the intelligence layer reads from and writes to. **The package itself is stateless** — it does not own these tables. Both Capsiynau and Nodiadau provision them via their own migrations against the shared Supabase project.
+
+Treat this file as the **public contract**: bumping a column type, dropping a column, or changing RLS semantics requires a major-version bump on `@capsiynau/intelligence` and a coordinated Nodiadau sync.
+
+## Tables
+
+| Table | Purpose | Consumer | Authoritative migration |
+|-------|---------|----------|--------------------------|
+| `word_boost_approved` | Approved learned vocabulary, ranked by `confirmed_by_count`. Read at session start by live-relay to bias Whisper. | live-relay, suggestion UI | `20260515_word_boost_approved.sql` + `..._confidence.sql` + `..._realtime.sql` |
+| `word_boost_candidates` | Pending vocab from correctionTracker, awaiting moderator approval. | LiveSessionMonitor, AdminMonitor | `20260515_word_boost_candidates_org_scope.sql` |
+| `correction_logs` | Full edit history per segment. Source signal for learning loops. | LiveSessionMonitor, paraphrase pipeline | `schema.sql` (since launch) |
+| `paraphrase_corrections` | Refinement memory: which paraphrase patterns to favour / suppress. | live-polish pipeline (PR #436) | `20260515_paraphrase_corrections.sql` (#435) |
+| `series_memory` | Long-form context across recurring shows/sessions. | refinement worker | `20260515_series_memory.sql` |
+| `glossaries` + `terms` + `term_votes` | Manually curated org/global glossaries. Higher priority than learned vocab. | editor, glossary UI | `schema.sql` |
+| `metadata_reviews` | QA approvals on AI-generated metadata. | metadata pipeline | `schema.sql` |
+
+## Isolation invariants
+
+Every table above is **org-scoped via `get_user_org_ids()`** in its RLS policy. Cross-org leakage is impossible from the client. Server writes use the admin client + explicit `.eq('organisation_id', ...)` to enforce the boundary.
+
+**Never** widen an RLS policy to query `memberships` directly — it causes 42P17 infinite recursion. See root `CLAUDE.md` for the rule.
+
+## Reading vs writing — which layer
+
+The intelligence library only contains **client-side helpers** (heuristic logic for what to learn from a correction, normalisation, mutation). The actual database writes happen in **app code**:
+
+| Operation | Where it lives | Why |
+|-----------|----------------|-----|
+| `extractProperNounCandidates(diff)` (pure logic) | `@capsiynau/intelligence/corrections` | Heuristic — shared |
+| Inserting into `word_boost_candidates` | `src/pages/LiveSessionMonitor.jsx` (Capsiynau) / Nodiadau equivalent | Needs admin Supabase client; org_id from auth context |
+| Promoting candidate → approved | Server endpoint (admin moderation UI) | Privileged, audited |
+| Reading approved vocab for STT bias | `worker/live-relay.js` (Capsiynau-specific) | Pipeline integration |
+
+This split is deliberate. The library stays universal (no Supabase dependency in the universal entry point) and the apps own their I/O.
+
+## Versioning policy
+
+- **Patch** — no schema impact (rename a helper, fix a bug)
+- **Minor** — additive only (new helper, new exported constant). No DB change required.
+- **Major** — semantic change to any contract above. Triggers:
+  - Dropping a column the library references
+  - Changing the meaning of a column (e.g. `confirmed_by_count` becomes a sum vs a count)
+  - Tightening or loosening RLS in a way the library assumes
+  - Renaming a table
+
+Major bumps require: PR description listing the breaking change, Nodiadau-side companion PR ready before merge, paired migration with explicit migration notes for operators.
+
+## Schema drift detection
+
+Until automated, the manual check before any major release:
+
+```sql
+-- Snapshot the relevant tables' columns
+SELECT table_name, column_name, data_type, is_nullable
+FROM information_schema.columns
+WHERE table_name IN (
+  'word_boost_approved', 'word_boost_candidates', 'correction_logs',
+  'paraphrase_corrections', 'series_memory',
+  'glossaries', 'terms', 'term_votes', 'metadata_reviews'
+)
+ORDER BY table_name, ordinal_position;
+
+-- And the policies
+SELECT tablename, policyname, cmd, qual
+FROM pg_policies
+WHERE tablename IN (
+  'word_boost_approved', 'word_boost_candidates', 'correction_logs',
+  'paraphrase_corrections', 'series_memory',
+  'glossaries', 'terms', 'term_votes', 'metadata_reviews'
+);
+```
+
+Diff against the previous release's snapshot. Any breaking change → major version bump.

package/package.json

@@ -0,0 +1,64 @@
+{
+  "name": "@capsiynau/intelligence",
+  "version": "0.2.0",
+  "description": "Bilingual (Welsh + English) intelligence layer: text normalisation, mutation handling, digraph splitting, spellcheck primitives, and glossary/correction utilities. Pure functions, runs in Node or browser.",
+  "type": "module",
+  "license": "MIT",
+  "author": "Aled Parry <aled@aledparry.com>",
+  "homepage": "https://github.com/aledprysparry/Capsiynau_V2_ClaudeAI/tree/main/packages/intelligence",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/aledprysparry/Capsiynau_V2_ClaudeAI.git",
+    "directory": "packages/intelligence"
+  },
+  "bugs": {
+    "url": "https://github.com/aledprysparry/Capsiynau_V2_ClaudeAI/issues"
+  },
+  "keywords": [
+    "welsh",
+    "cymraeg",
+    "bilingual",
+    "spellcheck",
+    "mutation",
+    "digraph",
+    "normaliser",
+    "glossary",
+    "nlp",
+    "capsiynau"
+  ],
+  "main": "./src/index.js",
+  "exports": {
+    ".": "./src/index.js",
+    "./welsh": "./src/welsh/index.js",
+    "./corrections": "./src/corrections/index.js",
+    "./glossary": "./src/glossary/index.js",
+    "./spellcheck": "./src/spellcheck/index.js"
+  },
+  "files": [
+    "src",
+    "README.md",
+    "SCHEMA.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=18"
+  },
+  "peerDependencies": {
+    "nspell": "^2.1.0",
+    "dictionary-cy": "^2.0.0",
+    "dictionary-en-gb": "^3.0.0"
+  },
+  "peerDependenciesMeta": {
+    "nspell": { "optional": true },
+    "dictionary-cy": { "optional": true },
+    "dictionary-en-gb": { "optional": true }
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "test": "vitest run __tests__"
+  },
+  "sideEffects": false
+}

package/src/corrections/index.js

@@ -0,0 +1,6 @@
+/**
+ * @capsiynau/intelligence/corrections
+ *
+ * Public surface for the correction-learning loop.
+ */
+export * from './tracker.js'