@jcode.labs/mimir 0.3.0 → 0.4.0

package/CHANGELOG.md

@@ -1,10 +1,29 @@
 # Changelog
 
+## 0.4.0 - 2026-06-28
+
+- Reposition Mimir as sovereign local RAG for confidential datasets and AI agents.
+- Expand default ingestion to common text, Office/OpenDocument, data, config, log, and source-code
+  file types.
+- Add `includeExtensions` / `KB_INCLUDE_EXTENSIONS` for custom UTF-8 text file extensions.
+- Add the optional `mimir-audio-summary` bundled skill for confidential audio summaries.
+- Install both the main Mimir skill and optional audio-summary skill with `kb install-skill`.
+- Improve agent guidance for deep multi-query retrieval before synthesis.
+- Make Mimir core retrieval-only: `kb ask` now returns cited context for external agents or LLMs
+  instead of generating answers internally.
+- Add optional Transformers.js semantic embeddings through `embeddingProvider: "transformers"`.
+- Remove Ollama providers and keep `embeddingProvider: "local-hash"` as the no-model default.
+- Move the repository to a simple pnpm workspace monorepo without adding Turbo.
+- Move the core `@jcode.labs/mimir` package into `packages/mimir`.
+- Add `@jcode.labs/mimir-tts` for plug-and-play JS/ONNX WAV rendering without Python or ffmpeg.
+- Add `kb audio` and update the audio-summary skill to use Mimir TTS before advanced fallback
+  engines.
+
 ## 0.3.0 - 2026-06-28
 
-- Add confidentiality hardening defaults: local-only Ollama network policy, built-in
-  redaction before indexing, metadata-only access logs, and bounded MCP retrieval.
-- Add `kb security-audit` for zero-telemetry, network, redaction, gitignore, storage, and
+- Add confidentiality hardening defaults: built-in redaction before indexing, metadata-only access
+  logs, and bounded MCP retrieval.
+- Add `kb security-audit` for zero-telemetry, provider, redaction, gitignore, storage, and
   MCP posture checks.
 - Add `kb destroy-index --yes` to remove generated vector indexes.
 - Add release verification artifacts: npm tarball, SHA256 checksums, SBOM, and manifest.

package/CONTRIBUTING.md

@@ -0,0 +1,28 @@
+# Contributing
+
+Mimir is an open-source project under the MIT License. Issues and pull requests are welcome.
+
+## Development
+
+Use Node.js 20+ and pnpm:
+
+```bash
+pnpm install
+pnpm validate
+```
+
+`pnpm validate` runs Biome, TypeScript, Vitest, the production CLI/MCP smoke test, and npm
+package metadata checks.
+
+## Pull Requests
+
+- Open pull requests against `main`.
+- Keep changes focused and include tests or smoke coverage for behavior changes.
+- Do not commit private documents, generated vector stores, environment files, tokens, or
+  credentials.
+- Use conventional commit messages such as `feat: add source parser` or
+  `fix: handle empty index`.
+
+## Security
+
+Do not report vulnerabilities through public issues. Follow [`SECURITY.md`](./SECURITY.md).

package/README.md

@@ -5,11 +5,18 @@
 [![npm](https://img.shields.io/npm/v/@jcode.labs/mimir)](https://www.npmjs.com/package/@jcode.labs/mimir)
 [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
 
-Open-source, local-first memory and retrieval for private project knowledge.
+Open-source, sovereign local RAG for confidential datasets and AI agents.
 
-Mimir provides a TypeScript CLI and library that can be installed in any Node.js
-repository. It indexes files from the target repository, stores vectors locally with LanceDB,
-and uses Ollama for local embeddings and answers.
+Mimir provides a TypeScript CLI, library, MCP server, and portable agent skills that can be
+installed in any Node.js repository. It indexes local files from the target repository, stores
+vectors locally with LanceDB, and can use either built-in local-hash retrieval or optional
+Transformers.js semantic embeddings. Mimir core returns cited retrieval context; answer synthesis
+belongs to the AI agent, LLM, or local model runtime you choose around it.
+
+The intended use case is simple: put confidential company, institutional, legal, operational, or
+research documents in a private local folder, index them locally, then let any compatible AI agent or
+LLM workflow retrieve grounded context for summaries, briefs, audits, and decision support without
+shipping the dataset to a hosted RAG service.
 
 Created by Jean-Baptiste Thery and published under the JCode Labs npm scope.
 
@@ -41,22 +48,50 @@ Suggested GitHub Sponsors tiers:
 
 Early public package. APIs may evolve before `1.0.0`.
 
+## What Mimir Is For
+
+- Build a local RAG knowledge base inside any repository.
+- Analyze confidential datasets while keeping raw files and generated indexes local.
+- Give Claude, Codex, Cursor, internal assistants, or other MCP-compatible tools the same private
+  retrieval layer.
+- Retrieve grounded local evidence through CLI, library calls, MCP tools, or the bundled agent
+  skills so your chosen AI agent can produce cited summaries.
+- Optionally create listenable WAV summaries with `kb audio`, `@jcode.labs/mimir-tts`, and the
+  bundled `mimir-audio-summary` skill.
+
+Mimir is not a hosted SaaS, not a remote vector database, and not a certified high-assurance system.
+For regulated or state-grade environments, pair it with encrypted disks, controlled machines, release
+verification, and an external security review.
+
+## Use Cases
+
+Mimir is useful whenever the source material should stay local but an AI agent still needs grounded
+context.
+
+| Use case | Example questions |
+| --- | --- |
+| Understand a code repository | "Where is authentication implemented?", "What depends on this module?", "Summarize the payment flow." |
+| Understand architecture | "What services exist?", "What are the data boundaries?", "Which components are risky to change?" |
+| Analyze specifications | "What does the technical spec require?", "Which requirements are still unclear?", "Generate an implementation checklist." |
+| Work through a request for proposal or tender | "What are the mandatory constraints?", "Which documents prove compliance?", "What risks should be clarified?" |
+| Study courses and training material | "Summarize chapter three.", "Create revision questions.", "Compare these two concepts." |
+| Analyze a book or long report | "Extract the main thesis.", "Find recurring arguments.", "Create a chapter-by-chapter brief." |
+| Build an internal knowledge base | "What is the policy for incident review?", "Who owns this process?", "Which source says that?" |
+| Prepare meetings or decisions | "Give me a one-page briefing.", "What is missing before deciding?", "List action items and evidence." |
+| Ask questions over offline documents | "Which files mention local-only operation?", "What evidence supports this claim?" |
+| Generate audio briefings | "Create a listenable summary of the current dossier using offline TTS." |
+
 ## Requirements
 
 - Node.js 20+
 - pnpm, npm, yarn or bun
-- Ollama running locally
-- Embedding model installed once:
-
-```bash
-ollama pull nomic-embed-text
-```
-
-Optional answer model:
-
-```bash
-ollama pull gemma4
-```
+- No model runtime is required for the default `embeddingProvider: "local-hash"` mode.
+- Optional semantic embeddings use Transformers.js with local model files under `.mimir/models` by
+  default.
+- Generated answers are intentionally outside Mimir core. Use Claude, Codex, OpenAI, a local model
+  MCP server, or another trusted model runtime to synthesize from Mimir's cited context.
+- Optional audio summaries use the separate `@jcode.labs/mimir-tts` workspace package. It renders
+  WAV files with Transformers.js and does not require Python, ffmpeg, Piper, XTTS, or a local server.
 
 ## Install From npm
 
@@ -76,23 +111,26 @@ npm install --save-dev @jcode.labs/mimir
 
 Maintainer tokens are only needed to publish new versions.
 
-## Install From Git
+## Install From Source Checkout
 
 ```bash
-pnpm add -D git+ssh://git@github.com/jcode-works/jcode-mimir.git
+git clone git@github.com:jcode-works/jcode-mimir.git
+cd jcode-mimir
+pnpm install
+pnpm build
 ```
 
 For local development:
 
 ```bash
-pnpm add -D file:../jcode-mimir
+pnpm add -D file:../jcode-mimir/packages/mimir
 ```
 
 Before creating an npm tarball later, run:
 
 ```bash
 pnpm build
-pnpm pack
+pnpm --dir packages/mimir pack
 ```
 
 ## Use In Any Repository
@@ -126,9 +164,158 @@ npx kb security-audit
 npx kb status
 ```
 
-## Agent Skill And MCP
+## Choose A Retrieval Mode
+
+Mimir has two embedding modes.
+
+### Default Local Hash Retrieval
+
+Use this when you want a fully local, no-model smoke test or a dependency-light setup. Retrieval is
+lexical/hash-based, not semantic.
+
+`.kb/config.json`:
+
+```json
+{
+  "embeddingProvider": "local-hash"
+}
+```
+
+Commands:
+
+```bash
+pnpm exec kb ingest
+pnpm exec kb search "offline retrieval approval"
+pnpm exec kb ask "What evidence supports offline operation?"
+```
+
+`kb ask` always returns cited retrieved passages instead of a generated synthesis. You can pass those
+passages to any LLM or agent you trust.
+
+### Optional Semantic Embeddings With Transformers.js
 
-Mimir ships with a portable agent skill and a standard MCP server.
+Use this when you want better semantic retrieval while keeping Mimir core free of an LLM server.
+
+`.kb/config.json`:
+
+```json
+{
+  "embeddingProvider": "transformers",
+  "embeddingModel": "mixedbread-ai/mxbai-embed-xsmall-v1",
+  "embeddingModelPath": ".mimir/models",
+  "transformersAllowRemoteModels": false
+}
+```
+
+Commands:
+
+```bash
+pnpm exec kb ingest
+pnpm exec kb ask "Which passages support offline operation?"
+```
+
+Keep `transformersAllowRemoteModels` false for confidential or air-gapped work and preload model
+files into `embeddingModelPath`. Set it to true only when you explicitly allow Transformers.js to
+download model files from Hugging Face.
+
+## Dependency Footprint
+
+Mimir can run retrieval without a model runtime. Some runtime dependencies remain because they own
+core features:
+
+| Dependency | Why it remains |
+| --- | --- |
+| @huggingface/transformers | optional local semantic embeddings |
+| LanceDB | local vector storage and nearest-neighbor retrieval |
+| MCP SDK | MCP server for compatible agents |
+| fast-glob | safe source-file discovery |
+| unpdf, html-to-text, yaml, fflate | document parsing for PDF, HTML, YAML, Office/OpenDocument ZIP files |
+| commander, zod, picocolors | CLI, config validation, readable terminal output |
+
+Removing more dependencies is possible only by dropping features or replacing them with smaller
+internal implementations. The current low-friction path is dependency-light at runtime for users who
+choose `local-hash`, while preserving richer parsing, MCP support, and optional semantic embeddings.
+
+## Example Test Workspace
+
+This repository includes a synthetic example under
+[`examples/sovereign-rag-demo`](./examples/sovereign-rag-demo). It can be used to test ingestion,
+retrieval, `security-audit`, and custom text extensions without using private documents.
+
+From a local checkout:
+
+```bash
+pnpm build
+cd examples/sovereign-rag-demo
+node ../../dist/cli.js security-audit
+node ../../dist/cli.js ingest
+node ../../dist/cli.js search "offline retrieval approval"
+node ../../dist/cli.js audit
+```
+
+The example uses the default local-hash retrieval mode, so it can run without downloading an
+embedding or chat model.
+
+## Typical Workflows
+
+### Understand A Codebase
+
+```bash
+pnpm exec kb init
+printf "src\nREADME.md\ndocs\n" >> .kb/sources.txt
+pnpm exec kb ingest
+pnpm exec kb search "authentication flow"
+pnpm exec kb ask "Explain the architecture and cite the relevant files."
+```
+
+### Analyze Specifications Or A Course
+
+```bash
+pnpm exec kb ingest
+pnpm exec kb ask "Summarize the requirements and list open questions."
+pnpm exec kb ask "Create revision questions from the indexed course material."
+```
+
+### Work Offline
+
+```bash
+pnpm exec kb security-audit --strict
+pnpm exec kb ingest
+pnpm exec kb search "incident review policy"
+pnpm exec kb ask "What does the local evidence prove?"
+```
+
+Use `embeddingProvider: "local-hash"` for a no-model offline workflow. Use
+`embeddingProvider: "transformers"` with preloaded model files for semantic offline retrieval.
+Generated answers should come from a trusted external agent or model runtime.
+
+### Generate A Local Audio Briefing
+
+Mimir includes a plug-and-play JS text-to-speech path for listenable summaries:
+
+```bash
+pnpm exec kb audio --doctor
+pnpm exec kb audio /tmp/MIMIR-SUMMARY-project.txt --out .mimir/audio/project-summary.wav
+```
+
+The command writes WAV output locally and does not require Python or ffmpeg. The first render can
+download a public Transformers.js-compatible model into `.mimir/models/tts`; the narration text is
+processed locally. For confidential air-gapped work, preload model files and run:
+
+```bash
+pnpm exec kb audio /tmp/MIMIR-SUMMARY-project.txt --out .mimir/audio/project-summary.wav --offline
+```
+
+The standalone package can also be installed directly:
+
+```bash
+pnpm add -D @jcode.labs/mimir-tts
+pnpm exec mimir-tts render /tmp/MIMIR-SUMMARY-project.txt --out .mimir/audio/project-summary.wav
+```
+
+## Agent Skills And MCP
+
+Mimir ships with portable agent skills and a standard MCP server.
 
 Install the agent kit into a repository:
 
@@ -140,12 +327,14 @@ This creates:
 
 ```plain text
 .mimir/skills/mimir/SKILL.md
+.mimir/skills/mimir-audio-summary/SKILL.md
 .mimir/mcp.json
 .mimir/README.md
 ```
 
-Agents that support skill folders can load `.mimir/skills/mimir/`. Other agents can read the
-generated `.mimir/README.md` and use the MCP config snippet.
+Agents that support skill folders can load `.mimir/skills/mimir/` for deep local RAG usage.
+Load `.mimir/skills/mimir-audio-summary/` only when an optional spoken summary is needed.
+Other agents can read the generated `.mimir/README.md` and use the MCP config snippet.
 
 Start the MCP server from the repository root:
 
@@ -161,6 +350,10 @@ MCP tools exposed:
 - `mimir_audit`
 - `mimir_security_audit`
 
+This MCP layer is the recommended way to let any compatible LLM or agent query the same local
+knowledge base. The LLM does not need to know about LanceDB or the raw file layout; it asks Mimir for
+ranked passages or cited context and uses the returned citations.
+
 Print the bundled skill path from the installed package:
 
 ```bash
@@ -190,7 +383,9 @@ state.
 Mimir is designed for private repositories and sensitive local evidence.
 
 - Zero telemetry: no analytics or document content is sent to JCode Labs.
-- Local-only network policy: Ollama must be on loopback by default.
+- No LLM generation in core: Mimir returns cited context for the agent/runtime you choose.
+- Local-hash by default: no model runtime is required for the default retrieval path.
+- Transformers.js remote model loading is disabled by default.
 - Redaction before indexing: common secrets and identifiers are redacted before chunks are
   embedded and stored.
 - Metadata-only access logs: query hashes and action metadata are logged, not raw queries.
@@ -214,6 +409,8 @@ read [`SECURITY-HARDENING.md`](./SECURITY-HARDENING.md).
 
 ## Supported Files
 
+Mimir supports common text, document, data, config, log, and source-code files out of the box:
+
 - Markdown: `.md`, `.mdx`
 - Text: `.txt`, `.text`
 - JSON: `.json`
@@ -221,6 +418,32 @@ read [`SECURITY-HARDENING.md`](./SECURITY-HARDENING.md).
 - CSV/TSV: `.csv`, `.tsv`
 - HTML: `.html`, `.htm`
 - PDF: `.pdf`
+- Office/OpenDocument: `.docx`, `.pptx`, `.xlsx`, `.odt`, `.ods`, `.odp`
+- Rich text: `.rtf`
+- Line data and logs: `.jsonl`, `.ndjson`, `.log`
+- XML feeds and documents: `.xml`, `.rss`, `.atom`
+- Config and data files: `.toml`, `.ini`, `.conf`, `.cfg`, `.properties`, `.sql`
+- Source code: `.ts`, `.tsx`, `.js`, `.jsx`, `.py`, `.go`, `.rs`, `.java`, `.rb`, `.php`,
+  `.cs`, `.c`, `.cpp`, `.h`, `.css`
+
+Custom UTF-8 text extensions can be enabled without changing code:
+
+```json
+{
+  "includeExtensions": [".transcript", ".evidence"]
+}
+```
+
+Or through:
+
+```bash
+KB_INCLUDE_EXTENSIONS=".transcript,.evidence" pnpm exec kb ingest
+```
+
+Images, scans, audio/video files, old proprietary Office binaries such as `.doc`, and other formats
+that are not listed should be OCRed, transcribed, converted, or exported to text/PDF/HTML first.
+Mimir intentionally avoids pretending that every binary format can be indexed safely without
+extraction logic.
 
 ## Config
 
@@ -232,11 +455,11 @@ read [`SECURITY-HARDENING.md`](./SECURITY-HARDENING.md).
   "storageDir": ".kb/storage",
   "sourcesFile": ".kb/sources.txt",
   "accessLogPath": ".kb/access.log",
+  "embeddingModelPath": ".mimir/models",
   "tableName": "chunks",
-  "ollamaHost": "http://localhost:11434",
-  "networkPolicy": "local-only",
-  "embedModel": "nomic-embed-text",
-  "llmModel": "gemma4:latest",
+  "embeddingProvider": "local-hash",
+  "embeddingModel": "mixedbread-ai/mxbai-embed-xsmall-v1",
+  "transformersAllowRemoteModels": false,
   "redaction": {
     "enabled": true,
     "builtIn": true,
@@ -246,7 +469,8 @@ read [`SECURITY-HARDENING.md`](./SECURITY-HARDENING.md).
   "mcpMaxTopK": 10,
   "topK": 5,
   "chunkSize": 1200,
-  "chunkOverlap": 150
+  "chunkOverlap": 150,
+  "includeExtensions": []
 }
 ```
 
@@ -256,10 +480,10 @@ Environment overrides:
 - `KB_STORAGE_DIR`
 - `KB_SOURCES_FILE`
 - `KB_ACCESS_LOG_PATH`
-- `KB_OLLAMA_HOST`
-- `KB_NETWORK_POLICY`
-- `KB_EMBED_MODEL`
-- `KB_LLM_MODEL`
+- `KB_EMBEDDING_PROVIDER`
+- `KB_EMBEDDING_MODEL`
+- `KB_EMBEDDING_MODEL_PATH`
+- `KB_TRANSFORMERS_ALLOW_REMOTE_MODELS`
 - `KB_REDACTION_ENABLED`
 - `KB_REDACTION_BUILT_IN`
 - `KB_ACCESS_LOG`
@@ -267,6 +491,7 @@ Environment overrides:
 - `KB_TOP_K`
 - `KB_CHUNK_SIZE`
 - `KB_CHUNK_OVERLAP`
+- `KB_INCLUDE_EXTENSIONS`
 
 ## Library API
 
@@ -280,8 +505,9 @@ const answer = await ask("What documents support the project timeline?")
 
 ## Privacy
 
-- Embeddings and answers use local Ollama by default.
-- Remote Ollama hosts are blocked unless `networkPolicy` explicitly allows them.
+- Mimir core does not generate answers or call a chat model.
+- `local-hash` can run ingestion, search, and cited retrieval without a model runtime.
+- Transformers.js remote model loading is disabled by default.
 - Built-in redaction runs before indexing by default.
 - Access logs store query hashes, not raw queries.
 - The vector index is stored locally.

package/SECURITY-HARDENING.md

@@ -1,13 +1,16 @@
 # Mimir Security Hardening
 
-Mimir is a local-first knowledge base for private project documents. It is built to minimize
-data movement, but it is not a certified high-assurance system.
+Mimir is a sovereign local RAG knowledge base for confidential project documents and datasets. It is
+built to minimize data movement, but it is not a certified high-assurance system.
 
 ## Current Guarantees
 
 - Zero telemetry: Mimir does not send usage analytics or document content to JCode Labs.
-- Local-only network policy by default: document text can only be sent to loopback Ollama hosts
-  unless the repository explicitly opts in to broader network access.
+- Retrieval-only core: Mimir does not call a chat model or generate LLM answers.
+- No-model retrieval mode: `embeddingProvider: "local-hash"` can ingest, search, and return cited
+  passages without a model server.
+- Optional semantic embeddings: `embeddingProvider: "transformers"` uses Transformers.js, with
+  remote model loading disabled by default through `transformersAllowRemoteModels: false`.
 - Redaction before indexing: built-in DLP patterns redact common secrets and identifiers before
   chunks are embedded and stored.
 - Metadata-only access logs: access logs contain action metadata and query hashes, not raw
@@ -16,13 +19,15 @@ data movement, but it is not a certified high-assurance system.
   default.
 - MCP is read-focused: destructive tools are not exposed over MCP, and MCP retrieval is capped by
   `mcpMaxTopK`.
+- Optional audio summaries use `kb audio` / `@jcode.labs/mimir-tts` for local WAV rendering with
+  Transformers.js. They do not require Python, ffmpeg, Piper, XTTS, or a local TTS server.
 - npm releases are published with provenance from the protected GitHub Actions workflow.
 - Release artifacts include a package tarball, SHA256 checksums, SBOM, and manifest.
 
 ## Threat Model
 
-Mimir protects against accidental repository leaks, accidental remote LLM usage, accidental secret
-indexing, and weak release traceability.
+Mimir protects against accidental repository leaks, accidental built-in LLM usage, accidental online
+TTS usage for generated summaries, accidental secret indexing, and weak release traceability.
 
 Mimir does not protect against a compromised local machine, malicious dependencies already present
 in the runtime, a user with filesystem access to the same checkout, or forensic recovery from an
@@ -50,32 +55,45 @@ pnpm build
 pnpm release:artifacts
 ```
 
-Move the generated tarball from `release-artifacts/` into the offline environment and install it:
+Move the generated tarballs from `release-artifacts/` into the offline environment and install them:
 
 ```bash
-pnpm add -D ./jcode.labs-mimir-<version>.tgz
+pnpm add -D ./jcode.labs-mimir-tts-<version>.tgz ./jcode.labs-mimir-<version>.tgz
 pnpm exec kb init
 pnpm exec kb ingest
 ```
 
-Ollama and the required models must also be preloaded inside the offline environment.
+For semantic embeddings, preload the Transformers.js-compatible embedding model files inside the
+offline environment under the configured `embeddingModelPath`. For audio, preload the TTS model
+files under `.mimir/models/tts` and render with `pnpm exec kb audio <text-file> --offline`.
 
 ## Zero Network Posture
 
-Default config:
+Default no-model config:
 
 ```json
 {
-  "ollamaHost": "http://localhost:11434",
-  "networkPolicy": "local-only"
+  "embeddingProvider": "local-hash"
 }
 ```
 
-Allowed policies:
+Optional semantic config:
 
-- `local-only`: only loopback hosts such as `localhost` and `127.0.0.1`.
-- `allow-private`: loopback and private LAN hosts.
-- `allow-any`: any host. Use only when the remote endpoint is explicitly trusted.
+```json
+{
+  "embeddingProvider": "transformers",
+  "embeddingModel": "mixedbread-ai/mxbai-embed-xsmall-v1",
+  "embeddingModelPath": ".mimir/models",
+  "transformersAllowRemoteModels": false
+}
+```
+
+The local-hash mode performs lexical/hash retrieval only. It is useful for smoke tests,
+dependency-light offline workflows, and handing cited passages to another trusted LLM. It is not
+equivalent to model semantic retrieval.
+
+Keep `transformersAllowRemoteModels` false for confidential or air-gapped work. If it is true,
+Transformers.js may download model files from Hugging Face during model loading.
 
 Run:
 
@@ -108,6 +126,25 @@ Custom patterns can be added in `.kb/config.json`:
 
 Redaction changes the indexed text, not the raw files under `private/`.
 
+## Optional Audio Summaries
+
+`kb install-skill` installs an optional `mimir-audio-summary` skill. It is designed for listenable
+briefings from a local Mimir index. The default renderer is `kb audio`, backed by
+`@jcode.labs/mimir-tts` and Transformers.js.
+
+Confidentiality defaults:
+
+- narration text is written to a temp file outside the repository;
+- generated WAV audio should be written under `.mimir/audio/`;
+- `.mimir/` is ignored by Git;
+- Python, ffmpeg, Piper, XTTS, and local TTS servers are not required for the default path;
+- the first online-enabled render may download public model weights into `.mimir/models/tts`, but
+  the narration text is processed locally;
+- `--offline` disables remote model loading and requires preloaded model files.
+
+Generated audio can still contain sensitive information. Treat it like a derived confidential
+document.
+
 ## MCP Hardening
 
 MCP gives an agent access to retrieved private context. Use it only for agents running under the
@@ -124,16 +161,17 @@ For team use, prefer one checkout per user or per role. Mimir does not implement
 
 ## Release Verification
 
-The protected npm workflow runs validation, generates release artifacts, and publishes with
-provenance:
+The protected npm workflow runs validation, generates release artifacts, and publishes both
+workspace packages with provenance:
 
 ```bash
-npm publish --access public --provenance
+pnpm --dir packages/mimir-tts publish --access public --provenance --no-git-checks
+pnpm --dir packages/mimir publish --access public --provenance --no-git-checks
 ```
 
 Release artifacts include:
 
-- npm tarball;
+- npm tarballs for `@jcode.labs/mimir-tts` and `@jcode.labs/mimir`;
 - `SHA256SUMS`;
 - CycloneDX SBOM;
 - `release-manifest.json`.

package/dist/chunking.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"chunking.d.ts","sourceRoot":"","sources":["../src/chunking.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,cAAc,EAAE,SAAS,EAAE,MAAM,YAAY,CAAA;AAE3D,wBAAgB,aAAa,CAC3B,QAAQ,EAAE,cAAc,EACxB,SAAS,EAAE,MAAM,EACjB,YAAY,EAAE,MAAM,GACnB,SAAS,EAAE,CAqCb"}
+{"version":3,"file":"chunking.d.ts","sourceRoot":"","sources":["../src/chunking.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,cAAc,EAAE,SAAS,EAAE,MAAM,YAAY,CAAA;AAM3D,wBAAgB,aAAa,CAC3B,QAAQ,EAAE,cAAc,EACxB,SAAS,EAAE,MAAM,EACjB,YAAY,EAAE,MAAM,GACnB,SAAS,EAAE,CAqCb"}

package/dist/chunking.js

@@ -1,4 +1,7 @@
 import { createHash } from "node:crypto";
+const PARAGRAPH_BREAK_MIN_RATIO = 0.45;
+const SENTENCE_BREAK_MIN_RATIO = 0.55;
+const WHITESPACE_BREAK_MIN_RATIO = 0.75;
 export function chunkDocument(document, chunkSize, chunkOverlap) {
     if (!document.text) {
         return [];
@@ -39,15 +42,15 @@ function chooseChunkEnd(text, cursor, chunkSize) {
     }
     const window = text.slice(cursor, hardEnd);
     const paragraphBreak = window.lastIndexOf("\n\n");
-    if (paragraphBreak > chunkSize * 0.45) {
+    if (paragraphBreak > chunkSize * PARAGRAPH_BREAK_MIN_RATIO) {
         return cursor + paragraphBreak;
     }
     const sentenceBreak = Math.max(window.lastIndexOf(". "), window.lastIndexOf("? "), window.lastIndexOf("! "));
-    if (sentenceBreak > chunkSize * 0.55) {
+    if (sentenceBreak > chunkSize * SENTENCE_BREAK_MIN_RATIO) {
         return cursor + sentenceBreak + 1;
     }
     const whitespace = window.lastIndexOf(" ");
-    if (whitespace > chunkSize * 0.75) {
+    if (whitespace > chunkSize * WHITESPACE_BREAK_MIN_RATIO) {
         return cursor + whitespace;
     }
     return hardEnd;

package/dist/chunking.js.map

@@ -1 +1 @@
-{"version":3,"file":"chunking.js","sourceRoot":"","sources":["../src/chunking.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AAGxC,MAAM,UAAU,aAAa,CAC3B,QAAwB,EACxB,SAAiB,EACjB,YAAoB;IAEpB,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnB,OAAO,EAAE,CAAA;IACX,CAAC;IAED,MAAM,MAAM,GAAgB,EAAE,CAAA;IAC9B,IAAI,MAAM,GAAG,CAAC,CAAA;IACd,IAAI,UAAU,GAAG,CAAC,CAAA;IAElB,OAAO,MAAM,GAAG,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,cAAc,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC,CAAA;QAC5D,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAA;QAEpD,IAAI,IAAI,EAAE,CAAC;YACT,MAAM,EAAE,GAAG,UAAU,CAAC,QAAQ,CAAC;iBAC5B,MAAM,CAAC,GAAG,QAAQ,CAAC,IAAI,CAAC,YAAY,IAAI,UAAU,IAAI,IAAI,EAAE,CAAC;iBAC7D,MAAM,CAAC,KAAK,CAAC,CAAA;YAChB,MAAM,CAAC,IAAI,CAAC;gBACV,EAAE;gBACF,MAAM,EAAE,QAAQ,CAAC,IAAI,CAAC,MAAM;gBAC5B,YAAY,EAAE,QAAQ,CAAC,IAAI,CAAC,YAAY;gBACxC,UAAU;gBACV,IAAI;gBACJ,QAAQ,EAAE,QAAQ,CAAC,IAAI,CAAC,QAAQ;gBAChC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,KAAK;gBAC1B,OAAO,EAAE,QAAQ,CAAC,IAAI,CAAC,OAAO;aAC/B,CAAC,CAAA;YACF,UAAU,IAAI,CAAC,CAAA;QACjB,CAAC;QAED,IAAI,GAAG,IAAI,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YAChC,MAAK;QACP,CAAC;QACD,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,GAAG,YAAY,EAAE,MAAM,GAAG,CAAC,CAAC,CAAA;IACnD,CAAC;IAED,OAAO,MAAM,CAAA;AACf,CAAC;AAED,SAAS,cAAc,CAAC,IAAY,EAAE,MAAc,EAAE,SAAiB;IACrE,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,MAAM,GAAG,SAAS,EAAE,IAAI,CAAC,MAAM,CAAC,CAAA;IACzD,IAAI,OAAO,KAAK,IAAI,CAAC,MAAM,EAAE,CAAC;QAC5B,OAAO,OAAO,CAAA;IAChB,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,OAAO,CAAC,CAAA;IAC1C,MAAM,cAAc,GAAG,MAAM,CAAC,WAAW,CAAC,MAAM,CAAC,CAAA;IACjD,IAAI,cAAc,GAAG,SAAS,GAAG,IAAI,EAAE,CAAC;QACtC,OAAO,MAAM,GAAG,cAAc,CAAA;IAChC,CAAC;IAED,MAAM,aAAa,GAAG,IAAI,CAAC,GAAG,CAC5B,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,EACxB,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,EACxB,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,CACzB,CAAA;IACD,IAAI,aAAa,GAAG,SAAS,GAAG,IAAI,EAAE,CAAC;QACrC,OAAO,MAAM,GAAG,aAAa,GAAG,CAAC,CAAA;IACnC,CAAC;IAED,MAAM,UAAU,GAAG,MAAM,CAAC,WAAW,CAAC,GAAG,CAAC,CAAA;IAC1C,IAAI,UAAU,GAAG,SAAS,GAAG,IAAI,EAAE,CAAC;QAClC,OAAO,MAAM,GAAG,UAAU,CAAA;IAC5B,CAAC;IAED,OAAO,OAAO,CAAA;AAChB,CAAC"}
+{"version":3,"file":"chunking.js","sourceRoot":"","sources":["../src/chunking.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AAGxC,MAAM,yBAAyB,GAAG,IAAI,CAAA;AACtC,MAAM,wBAAwB,GAAG,IAAI,CAAA;AACrC,MAAM,0BAA0B,GAAG,IAAI,CAAA;AAEvC,MAAM,UAAU,aAAa,CAC3B,QAAwB,EACxB,SAAiB,EACjB,YAAoB;IAEpB,IAAI,CAAC,QAAQ,CAAC,IAAI,EAAE,CAAC;QACnB,OAAO,EAAE,CAAA;IACX,CAAC;IAED,MAAM,MAAM,GAAgB,EAAE,CAAA;IAC9B,IAAI,MAAM,GAAG,CAAC,CAAA;IACd,IAAI,UAAU,GAAG,CAAC,CAAA;IAElB,OAAO,MAAM,GAAG,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;QACrC,MAAM,GAAG,GAAG,cAAc,CAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC,CAAA;QAC5D,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAA;QAEpD,IAAI,IAAI,EAAE,CAAC;YACT,MAAM,EAAE,GAAG,UAAU,CAAC,QAAQ,CAAC;iBAC5B,MAAM,CAAC,GAAG,QAAQ,CAAC,IAAI,CAAC,YAAY,IAAI,UAAU,IAAI,IAAI,EAAE,CAAC;iBAC7D,MAAM,CAAC,KAAK,CAAC,CAAA;YAChB,MAAM,CAAC,IAAI,CAAC;gBACV,EAAE;gBACF,MAAM,EAAE,QAAQ,CAAC,IAAI,CAAC,MAAM;gBAC5B,YAAY,EAAE,QAAQ,CAAC,IAAI,CAAC,YAAY;gBACxC,UAAU;gBACV,IAAI;gBACJ,QAAQ,EAAE,QAAQ,CAAC,IAAI,CAAC,QAAQ;gBAChC,KAAK,EAAE,QAAQ,CAAC,IAAI,CAAC,KAAK;gBAC1B,OAAO,EAAE,QAAQ,CAAC,IAAI,CAAC,OAAO;aAC/B,CAAC,CAAA;YACF,UAAU,IAAI,CAAC,CAAA;QACjB,CAAC;QAED,IAAI,GAAG,IAAI,QAAQ,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YAChC,MAAK;QACP,CAAC;QACD,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,GAAG,YAAY,EAAE,MAAM,GAAG,CAAC,CAAC,CAAA;IACnD,CAAC;IAED,OAAO,MAAM,CAAA;AACf,CAAC;AAED,SAAS,cAAc,CAAC,IAAY,EAAE,MAAc,EAAE,SAAiB;IACrE,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,MAAM,GAAG,SAAS,EAAE,IAAI,CAAC,MAAM,CAAC,CAAA;IACzD,IAAI,OAAO,KAAK,IAAI,CAAC,MAAM,EAAE,CAAC;QAC5B,OAAO,OAAO,CAAA;IAChB,CAAC;IAED,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,OAAO,CAAC,CAAA;IAC1C,MAAM,cAAc,GAAG,MAAM,CAAC,WAAW,CAAC,MAAM,CAAC,CAAA;IACjD,IAAI,cAAc,GAAG,SAAS,GAAG,yBAAyB,EAAE,CAAC;QAC3D,OAAO,MAAM,GAAG,cAAc,CAAA;IAChC,CAAC;IAED,MAAM,aAAa,GAAG,IAAI,CAAC,GAAG,CAC5B,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,EACxB,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,EACxB,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,CACzB,CAAA;IACD,IAAI,aAAa,GAAG,SAAS,GAAG,wBAAwB,EAAE,CAAC;QACzD,OAAO,MAAM,GAAG,aAAa,GAAG,CAAC,CAAA;IACnC,CAAC;IAED,MAAM,UAAU,GAAG,MAAM,CAAC,WAAW,CAAC,GAAG,CAAC,CAAA;IAC1C,IAAI,UAAU,GAAG,SAAS,GAAG,0BAA0B,EAAE,CAAC;QACxD,OAAO,MAAM,GAAG,UAAU,CAAA;IAC5B,CAAC;IAED,OAAO,OAAO,CAAA;AAChB,CAAC"}