@jcode.labs/ragmir 2.9.2 → 2.9.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,50 +1,70 @@
1
1
  # Ragmir Core
2
2
 
3
- `@jcode.labs/ragmir` is the main Ragmir package. It provides the `rgr` command-line interface, a
4
- TypeScript retrieval API, a local MCP server, and portable agent helpers.
3
+ [![npm version](https://img.shields.io/npm/v/@jcode.labs/ragmir)](https://www.npmjs.com/package/@jcode.labs/ragmir)
4
+ [![npm downloads](https://img.shields.io/npm/dm/@jcode.labs/ragmir)](https://www.npmjs.com/package/@jcode.labs/ragmir)
5
+ [![Node.js](https://img.shields.io/node/v/@jcode.labs/ragmir)](https://www.npmjs.com/package/@jcode.labs/ragmir)
6
+ [![MIT](https://img.shields.io/npm/l/@jcode.labs/ragmir)](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE)
5
7
 
6
- Use it when a repository needs searchable, cited context without uploading its documents to a hosted
7
- knowledge base. It is retrieval-first: the default `local-hash` mode works offline, without an
8
- account or model download.
8
+ **The CLI, TypeScript API, and local MCP server for cited project retrieval.**
9
9
 
10
- ## Choose this package when you need
10
+ `@jcode.labs/ragmir` indexes the files a project selects and returns source-backed passages without
11
+ a hosted document store. The default `local-hash` retrieval path works offline, without an account
12
+ or model download.
11
13
 
12
- | Need | What Core provides |
14
+ [Project overview](https://github.com/jcode-works/jcode-ragmir#readme) ·
15
+ [CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md) ·
16
+ [API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
17
+
18
+ ## Install Core when you need
19
+
20
+ | Goal | Core interface |
13
21
  | --- | --- |
14
- | Search project documents from a terminal | `rgr search` with source citations. |
15
- | Give an AI agent bounded access to repository context | A local stdio MCP server and generated helpers. |
16
- | Build retrieval into a Node.js tool | A typed TypeScript API for ingesting and searching. |
17
- | Index PDFs, Markdown, source files, Office files, HTML, or CSV | Local extraction, redaction, chunking, and storage. |
18
- | Work with a confidential repository offline | Ignored local `.ragmir/` state and offline-first retrieval. |
22
+ | Search repository documents with citations | `rgr search` |
23
+ | Audit a local knowledge base | `rgr audit`, `rgr doctor`, and `rgr security-audit` |
24
+ | Add retrieval to a Node.js application | Typed `ingest`, `search`, `ask`, and `research` exports |
25
+ | Give an AI agent bounded project context | Local stdio MCP server and generated helpers |
26
+ | Index scanned PDFs | Embedded text first, with optional page-aware local OCR |
19
27
 
20
- For answer generation, install the optional [Ragmir Chat](https://www.npmjs.com/package/@jcode.labs/ragmir-chat)
21
- package. For audio summaries, install [Ragmir TTS](https://www.npmjs.com/package/@jcode.labs/ragmir-tts).
28
+ Core retrieves evidence. It does not require a generative model and does not produce an ungrounded
29
+ answer. Add [Ragmir Chat](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) only when local
30
+ GGUF synthesis is useful, or [Ragmir TTS](https://www.npmjs.com/package/@jcode.labs/ragmir-tts)
31
+ when you need audio output.
22
32
 
23
- ## Quick start
33
+ ## First cited search
24
34
 
25
- Install Core in the repository that owns the documents you want to search:
35
+ Requires Node.js 20 or later.
26
36
 
27
37
  ```bash
28
38
  npm install --save-dev @jcode.labs/ragmir
29
39
  npx rgr setup
30
- npx rgr sources add "docs/**/*.md"
40
+ npx rgr sources add "README.md" "docs/**/*.md"
31
41
  npx rgr ingest
32
- npx rgr search "Which decision changed the deployment?"
42
+ npx rgr search "Which decision changed the rollout?"
33
43
  ```
34
44
 
35
- `rgr setup` creates ignored local state. `rgr ingest` incrementally extracts text, applies configured
36
- redaction, chunks the content, and writes the local index. Every search result identifies its source
37
- file, excerpt, chunk, and PDF page when available.
45
+ The project now owns an ignored `.ragmir/` directory containing configuration and generated local
46
+ state. Ingestion is incremental, and every result identifies the source path, chunk, line range, and
47
+ PDF page when one is available.
38
48
 
39
- ## How it works
49
+ ## CLI essentials
40
50
 
41
- 1. Select files with `rgr sources add` or `.ragmir/sources.txt`.
42
- 2. Ragmir extracts and redacts text locally, then writes a local index under `.ragmir/`.
43
- 3. `rgr search`, the TypeScript API, and MCP return ranked passages with citations.
44
- 4. Semantic embeddings, OCR, local chat, and audio are explicit opt-ins.
51
+ ```bash
52
+ # Readiness and source coverage
53
+ npx rgr doctor
54
+ npx rgr audit --unsupported
55
+ npx rgr security-audit
56
+
57
+ # Retrieval
58
+ npx rgr search "deployment decision"
59
+ npx rgr ask "What evidence supports the deployment decision?"
60
+ npx rgr research "deployment obligations" --compact
61
+
62
+ # Machine-readable output
63
+ npx rgr search "deployment decision" --json
64
+ ```
45
65
 
46
- For a scanned PDF, OCR runs only on pages that have no embedded text, through a command configured on
47
- your machine. Ragmir never sends PDF pages to a cloud OCR service.
66
+ `ask` returns cited retrieval context, not LLM synthesis. `research` performs an audit-backed,
67
+ multi-query retrieval pass and reports missing or weak evidence.
48
68
 
49
69
  ## TypeScript API
50
70
 
@@ -53,39 +73,100 @@ import { ingest, search } from "@jcode.labs/ragmir"
53
73
 
54
74
  await ingest({ cwd: process.cwd() })
55
75
 
56
- const results = await search("Which decision changed the deployment?", { topK: 5 })
76
+ const results = await search("Which decision changed the rollout?", {
77
+ cwd: process.cwd(),
78
+ topK: 5,
79
+ })
57
80
 
58
81
  for (const result of results) {
59
- console.log(result.relativePath, result.citation, result.text)
82
+ console.log(result.citation, result.text)
60
83
  }
61
84
  ```
62
85
 
63
- Use `ask()` for cited retrieval context without an LLM, `research()` for audit-backed multi-query
64
- retrieval, and `serveMcp()` to start the local MCP server from your own integration.
86
+ Frequently used exports:
87
+
88
+ | Export | Purpose |
89
+ | --- | --- |
90
+ | `setupProject`, `addSourceEntries` | Initialize project state and select files |
91
+ | `ingest`, `audit` | Build the index and compare it with files on disk |
92
+ | `search`, `ask`, `research` | Retrieve cited passages at different levels of breadth |
93
+ | `doctor`, `securityAudit` | Inspect readiness and local privacy posture |
94
+ | `serveMcp` | Start the read-focused local MCP server |
95
+ | `configurePdfOcr`, `inspectPdfOcr` | Configure and inspect local PDF OCR |
65
96
 
66
- ## Agent setup
97
+ See the [complete API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
98
+ for options and result shapes.
67
99
 
68
- Generate local MCP helpers for the agents used by the repository:
100
+ ## Connect an AI agent
69
101
 
70
102
  ```bash
71
103
  npx rgr setup --agents claude,codex,kimi,opencode,cline
104
+ npx rgr doctor
105
+ ```
106
+
107
+ Ragmir writes helper files for the selected clients and points them at the current project. MCP
108
+ exposes status, search, ask, research, audit, evaluation, usage, and security tools. It does not
109
+ expose index deletion.
110
+
111
+ ## Retrieval modes
112
+
113
+ | Mode | Network requirement | Best for |
114
+ | --- | --- | --- |
115
+ | `local-hash` | None | Default offline retrieval, setup-free projects, and deterministic evaluation |
116
+ | `transformers` | Explicit model preload or download | Higher-quality semantic matching with local inference |
117
+
118
+ Enable semantic retrieval explicitly:
119
+
120
+ ```bash
121
+ npx rgr setup --semantic
122
+ npx rgr ingest --rebuild
123
+ ```
124
+
125
+ `local-hash` is lexical/hash retrieval. It should not be presented as equivalent to semantic
126
+ embeddings.
127
+
128
+ ## Documents and OCR
129
+
130
+ Core parses common source, text, structured-data, Office, OpenDocument, EPUB, HTML, email, notebook,
131
+ and PDF formats. Projects can add custom text extensions. Run `rgr audit --unsupported` to see every
132
+ skipped file and recommendation.
133
+
134
+ For scanned PDFs:
135
+
136
+ ```bash
137
+ npx rgr ocr setup --engine auto
138
+ npx rgr ingest --rebuild
72
139
  ```
73
140
 
74
- The generated helper points the agent at the current project, never at Ragmir's package installation
75
- directory. Retrieval stays bounded and read-focused.
141
+ Ragmir prefers embedded PDF text and invokes OCR only for blank pages. OCR runs through a configured
142
+ local executable, never a shell string or cloud OCR service.
143
+
144
+ ## Local data and privacy
76
145
 
77
- ## Technology and local data
146
+ - Project paths resolve from `cwd` or explicit configuration, not the npm installation directory.
147
+ - Generated indexes, models, reports, audio, and access logs belong under ignored `.ragmir/` state.
148
+ - Redaction runs before indexing when configured, but it is not a compliance certification.
149
+ - External extractors and model downloads are opt-in system boundaries.
150
+ - MCP retrieval is bounded and access logs contain metadata, not query text or retrieved passages.
78
151
 
79
- Core is TypeScript for Node.js 20 or later. It uses LanceDB for local storage, the Model Context
80
- Protocol TypeScript SDK for agent integration, and Transformers.js only when semantic retrieval is
81
- explicitly enabled. Its generated `.ragmir/` state is local and should stay ignored by Git.
152
+ Read [configuration and privacy](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
153
+ and [security hardening](https://github.com/jcode-works/jcode-ragmir/blob/main/SECURITY-HARDENING.md)
154
+ before indexing sensitive material.
155
+
156
+ ## Runnable examples
157
+
158
+ | Example | Demonstrates |
159
+ | --- | --- |
160
+ | [Sovereign RAG demo](https://github.com/jcode-works/jcode-ragmir/tree/main/packages/ragmir-core/examples/sovereign-rag-demo) | CLI ingestion, search, audit, redaction, and evaluation |
161
+ | [Library API demo](https://github.com/jcode-works/jcode-ragmir/tree/main/packages/ragmir-core/examples/library-api-demo) | The public library surface against fictional files |
162
+ | [Document evidence benchmark](https://github.com/jcode-works/jcode-ragmir/tree/main/packages/ragmir-core/examples/document-evidence-benchmark) | Recall and exact citation evaluation |
82
163
 
83
- ## Further reading
164
+ ## Documentation
84
165
 
85
- - [Ragmir overview and package comparison](https://github.com/jcode-works/jcode-ragmir#readme)
86
166
  - [CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md)
87
167
  - [TypeScript API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
88
- - [Configuration and privacy](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
168
+ - [Configuration](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
89
169
  - [Agent integration](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/agent-integration.md)
170
+ - [Troubleshooting](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/troubleshooting.md)
90
171
 
91
- Ragmir is open source under the [MIT License](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE).
172
+ Ragmir Core is open source under the [MIT License](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE).
package/dist/version.d.ts CHANGED
@@ -1,2 +1,2 @@
1
- export declare const VERSION = "2.9.2";
1
+ export declare const VERSION = "2.9.3";
2
2
  //# sourceMappingURL=version.d.ts.map
package/dist/version.js CHANGED
@@ -1,2 +1,2 @@
1
- export const VERSION = "2.9.2";
1
+ export const VERSION = "2.9.3";
2
2
  //# sourceMappingURL=version.js.map
@@ -1,36 +1,90 @@
1
- # Document Evidence Benchmark
1
+ # Document evidence benchmark
2
2
 
3
- Synthetic document benchmark for Ragmir retrieval. It is intentionally safe to commit: the contract,
4
- RFP, runbook, spec, and legal/tax notes are fictional and contain no customer data.
3
+ A deterministic, public-safe benchmark for Ragmir retrieval and citation quality.
5
4
 
6
- This benchmark checks three things:
5
+ Use this example when you want to prove more than "a relevant file appeared." Its six golden queries
6
+ check that Ragmir retrieves the expected path and the exact file, line, chunk, or PDF-page citation.
7
+ Every document is fictional and safe to commit.
7
8
 
8
- - `Recall@K`: each query retrieves the expected source file or passage.
9
- - exact citations: each query retrieves the expected `relative/path:Lx-Ly#chunkIndex` citation.
10
- - embedded PDF text: the synthetic PDF is extracted and retrieved with an exact
11
- `relative/path:p1:Lx-Ly#chunkIndex` page citation.
9
+ ## What it measures
12
10
 
13
- The corpus is small on purpose so every source file fits in a single chunk. That keeps the expected
14
- citations deterministic while exercising the same CLI path used for private dogfooding.
11
+ | Signal | What a passing result means |
12
+ | --- | --- |
13
+ | `Recall@K` | An expected source path appears within the configured top K results |
14
+ | Exact text citation | The expected `relative/path:Lx-Ly#chunkIndex` is returned |
15
+ | Exact PDF citation | The expected `relative/path:pN:Lx-Ly#chunkIndex` is returned |
15
16
 
16
- ## Run From This Repository Checkout
17
+ The corpus is intentionally small enough for each source to fit in one chunk. That makes citation
18
+ expectations stable while exercising the same ingestion and evaluation code used for private
19
+ knowledge bases.
17
20
 
18
- Build Ragmir once from the repository root:
21
+ ## Corpus
22
+
23
+ | Fictional source | Evidence type |
24
+ | --- | --- |
25
+ | `raw/contracts/master-services-agreement.md` | Contract and residency obligations |
26
+ | `raw/contracts/pdf-control-evidence.pdf` | Embedded PDF text with a page citation |
27
+ | `raw/rfp/security-questionnaire.md` | Security and hosting requirements |
28
+ | `raw/runbooks/incident-response-runbook.md` | Operational evidence collection |
29
+ | `raw/specs/agent-integration-spec.md` | Agent and retrieval boundaries |
30
+ | `raw/legal-tax/residency-review-note.md` | Professional-review disclaimer |
31
+
32
+ [`golden-queries.json`](./golden-queries.json) maps each test query to its expected path and exact
33
+ citation.
34
+
35
+ ## Run the benchmark
36
+
37
+ From the repository root, install dependencies and build the packages:
19
38
 
20
39
  ```bash
40
+ pnpm install --frozen-lockfile
21
41
  pnpm build
22
42
  ```
23
43
 
24
- Then run the benchmark from this folder:
44
+ Then run the benchmark from this directory:
25
45
 
26
46
  ```bash
27
47
  cd packages/ragmir-core/examples/document-evidence-benchmark
28
48
  node ../../dist/cli.js init
29
- node ../../dist/cli.js ingest
49
+ node ../../dist/cli.js ingest --rebuild
30
50
  node ../../dist/cli.js evaluate --golden golden-queries.json --json
31
51
  node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 1
32
52
  ```
33
53
 
34
- Use this example as a public-safe template for private document evaluations. Keep real contracts,
35
- RFPs, runbooks, tax notes, golden queries, and generated JSON reports outside Git or under ignored
36
- local Ragmir state.
54
+ The last command exits non-zero unless every golden query succeeds at the configured threshold.
55
+ Keep the JSON output when you need machine-readable evidence for CI or a retrieval-quality review.
56
+
57
+ ## Read a failure
58
+
59
+ 1. Check `rgr audit` for missing, stale, or skipped source files.
60
+ 2. Inspect the failed query's returned paths and citations in JSON output.
61
+ 3. Decide whether the corpus, expected citation, chunking, or retrieval mode changed.
62
+ 4. Update a golden expectation only when the intended product behavior changed.
63
+
64
+ Do not lower `--fail-under` merely to make a regression pass.
65
+
66
+ ## Adapt it to a private corpus
67
+
68
+ Copy the structure, not the fictional documents:
69
+
70
+ ```text
71
+ private-evaluation/
72
+ ├── .ragmir/config.json
73
+ ├── golden-queries.json
74
+ └── raw/
75
+ ```
76
+
77
+ For each important question, record the smallest set of expected source paths. Add exact citations
78
+ when the source is stable enough to make line, page, and chunk boundaries meaningful. Keep private
79
+ contracts, RFPs, runbooks, tax notes, scans, golden queries, and evaluation reports outside Git or
80
+ under ignored local state.
81
+
82
+ ## Safety
83
+
84
+ - The committed corpus contains no customer or production data.
85
+ - The benchmark uses the offline `local-hash` provider and needs no model download.
86
+ - Generated `.ragmir/storage` and evaluation output stay local and ignored.
87
+ - A perfect synthetic score proves this fixture, not universal retrieval quality.
88
+
89
+ Return to the [Ragmir Core README](../../README.md) or try the
90
+ [sovereign RAG demo](../sovereign-rag-demo/README.md) for a broader CLI walkthrough.
@@ -1,43 +1,88 @@
1
- # Library API Demo
1
+ # Library API demo
2
2
 
3
- Runnable smoke for the `@jcode.labs/ragmir` **library** API, meant for local development of Ragmir
4
- Core itself. Where [`sovereign-rag-demo`](../sovereign-rag-demo) drives the **CLI**
5
- (`node ../../dist/cli.js ...`), this demo `import`s the public TypeScript surface the same way an
6
- external consumer would:
3
+ A runnable smoke test for the public `@jcode.labs/ragmir` TypeScript API.
7
4
 
8
- ```js
9
- import { ask, audit, ingest, search } from "@jcode.labs/ragmir"
10
- ```
5
+ Use this example when you are changing Ragmir Core or evaluating it as a library rather than as a
6
+ CLI. It imports the same package name an external consumer uses, runs the full retrieval loop, and
7
+ prints cited results from a fictional local corpus.
8
+
9
+ ## What it proves
10
+
11
+ The demo exercises four public operations in order:
12
+
13
+ 1. `ingest({ cwd, rebuild: true })` parses, redacts, chunks, embeds, and stores the corpus.
14
+ 2. `search(query, { cwd, topK })` returns ranked source passages.
15
+ 3. `ask(query, { cwd, topK })` returns retrieval-only cited context without LLM synthesis.
16
+ 4. `audit(cwd)` compares supported files with the current index.
11
17
 
12
- Node self-referencing resolves `@jcode.labs/ragmir` to this repository's local build
13
- (`packages/ragmir-core/dist`), **never** the npm-published package. So you can validate your local
14
- changes to the library end to end, without `npx` silently running a released version.
18
+ Node's package self-reference resolves `@jcode.labs/ragmir` to this checkout's local
19
+ `packages/ragmir-core/dist` build. It never falls back to the npm-published version, so the result
20
+ reflects the code you are currently reviewing.
15
21
 
16
- ## Run
22
+ ## Run it
17
23
 
18
24
  From the repository root:
19
25
 
20
26
  ```bash
27
+ pnpm install --frozen-lockfile
21
28
  pnpm example
22
29
  ```
23
30
 
24
- That builds Ragmir Core, then runs the demo. It reuses the committed synthetic corpus from
25
- `sovereign-rag-demo`, so it needs no private documents and writes only to that example's gitignored
26
- `.ragmir/storage`.
31
+ The root command builds the published packages required by Core and runs [`run.mjs`](./run.mjs). The
32
+ output is organized into `ingest`, `search`, `ask`, and `audit` sections so a reviewer can see each
33
+ public step succeed.
27
34
 
28
- To run it directly against the already-built `dist/` without rebuilding (run `pnpm build` first;
29
- `dist/` is gitignored, so a clean clone has none):
35
+ To rerun only the script after an existing build:
30
36
 
31
37
  ```bash
32
38
  node packages/ragmir-core/examples/library-api-demo/run.mjs
33
39
  ```
34
40
 
35
- ## What it exercises
41
+ `dist/` is ignored, so a clean clone must be built at least once.
42
+
43
+ ## The public API pattern
44
+
45
+ The essential integration is intentionally small:
46
+
47
+ ```js
48
+ import { ask, audit, ingest, search } from "@jcode.labs/ragmir"
49
+
50
+ await ingest({ cwd: projectRoot, rebuild: true })
51
+
52
+ const results = await search("offline retrieval approval", {
53
+ cwd: projectRoot,
54
+ topK: 3,
55
+ })
56
+
57
+ const context = await ask("What evidence supports offline operation?", {
58
+ cwd: projectRoot,
59
+ topK: 3,
60
+ })
61
+
62
+ const report = await audit(projectRoot)
63
+ ```
64
+
65
+ All project-relative state is resolved from `cwd`. This is the key boundary for applications that
66
+ use Ragmir across more than one repository.
67
+
68
+ ## Data used by the demo
69
+
70
+ The script reuses the sibling [sovereign RAG demo](../sovereign-rag-demo/README.md):
71
+
72
+ - committed synthetic Markdown, CSV, JSONL, YAML, and custom text files;
73
+ - an offline `local-hash` configuration;
74
+ - gitignored `.ragmir/storage` output;
75
+ - no private documents and no model download.
76
+
77
+ Change the queries in `run.mjs` when testing a retrieval behavior, but keep the committed demo
78
+ deterministic and public-safe. Use a separate ignored corpus for private evaluations.
79
+
80
+ ## When to use another example
36
81
 
37
- - `ingest({ cwd, rebuild: true })` — parse, redact, chunk, and embed the synthetic corpus.
38
- - `search(query, { cwd, topK })` — ranked cited passages.
39
- - `ask(query, { cwd, topK })` — retrieval-only cited context (no LLM synthesis in core).
40
- - `audit(cwd)` — indexed, supported, and skipped file counts.
82
+ | Need | Example |
83
+ | --- | --- |
84
+ | Learn the complete CLI workflow | [Sovereign RAG demo](../sovereign-rag-demo/README.md) |
85
+ | Measure exact paths and citations | [Document evidence benchmark](../document-evidence-benchmark/README.md) |
86
+ | Explore every exported function and option | [Core API reference](../../README.md#typescript-api) |
41
87
 
42
- It uses `embeddingProvider: "local-hash"` (from the reused corpus config), so it runs fully offline
43
- with no model download.
88
+ Return to the [Ragmir Core README](../../README.md) for installation, MCP, OCR, and privacy guidance.
@@ -1,76 +1,109 @@
1
- # Sovereign RAG Demo
1
+ # Sovereign RAG demo
2
2
 
3
- Synthetic test workspace for Ragmir. It is intentionally safe to commit: every document is fictional,
4
- generic, and designed only to exercise local ingestion, retrieval, redaction, custom extensions, and
5
- security-audit flows.
3
+ A complete, fictional workspace for learning Ragmir's local CLI workflow.
6
4
 
7
- This folder must never contain real-world sensitive, regulated, or production documents.
5
+ Use this example to ingest several file types, search with citations, inspect privacy posture, find an
6
+ unsupported file, evaluate golden queries, and prepare agent access. It runs with offline
7
+ `local-hash` retrieval and contains no customer or production data.
8
8
 
9
- ## What It Covers
9
+ ## Scenario
10
10
 
11
- - Markdown operational briefs.
12
- - CSV dataset inventories.
13
- - JSONL incident timelines.
14
- - YAML policy metadata.
15
- - A custom `.evidence` text extension enabled through `.ragmir/config.json`.
16
- - Unsupported/skipped-file reporting with recommendations through `audit --unsupported`.
11
+ The corpus models an operations team reviewing whether an evidence workflow can stay local:
17
12
 
18
- ## Run From This Repository Checkout
13
+ | File | Role in the scenario |
14
+ | --- | --- |
15
+ | `raw/operations-brief.md` | Approval, ownership, and operating requirements |
16
+ | `raw/dataset-inventory.csv` | Accepted and rejected datasets |
17
+ | `raw/incident-timeline.jsonl` | Time-ordered operational evidence |
18
+ | `raw/security-policy.yaml` | Model-loading and security rules |
19
+ | `raw/review-notes.evidence` | Custom text extension configured for this project |
20
+ | `raw/facility-scan.heic` | Deliberately unsupported file used by the audit example |
19
21
 
20
- Build Ragmir once from the repository root:
22
+ ## Run the full workflow
23
+
24
+ From the repository root:
21
25
 
22
26
  ```bash
27
+ pnpm install --frozen-lockfile
23
28
  pnpm build
24
29
  ```
25
30
 
26
- Then run the CLI from this folder:
31
+ Then enter this example and run the local CLI build:
27
32
 
28
33
  ```bash
29
34
  cd packages/ragmir-core/examples/sovereign-rag-demo
30
35
  node ../../dist/cli.js init
36
+
31
37
  # POSIX only: this example deliberately uses a custom tracked rawDir.
32
38
  chmod 700 raw
39
+
33
40
  node ../../dist/cli.js security-audit
34
- node ../../dist/cli.js ingest
41
+ node ../../dist/cli.js ingest --rebuild
35
42
  node ../../dist/cli.js search "offline retrieval approval"
36
- node ../../dist/cli.js search "dataset residency"
37
43
  node ../../dist/cli.js ask "What evidence supports offline operation?"
38
- node ../../dist/cli.js evaluate --golden golden-queries.json
44
+ node ../../dist/cli.js research "sovereign deployment evidence" --compact
39
45
  node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 1
40
- node ../../dist/cli.js audit
41
46
  node ../../dist/cli.js audit --unsupported
42
47
  node ../../dist/cli.js status
43
48
  ```
44
49
 
45
- `init` is idempotent here and applies owner-only permissions to the copied `.ragmir` config. The
46
- example deliberately keeps documents under the custom tracked `raw/` path, so secure that directory
47
- explicitly on POSIX before a strict audit. Git and package archives cannot encode its `0700` mode.
50
+ `init` is idempotent. Git and npm archives cannot preserve the example's intended `0700` raw-folder
51
+ mode, so the explicit `chmod` keeps the raw corpus owner-only on POSIX systems.
48
52
 
49
- This example uses `embeddingProvider: "local-hash"`, so it does not require a model runtime.
50
- Retrieval is lexical/hash-based rather than model-semantic.
53
+ The security audit still reports that `.ragmir/` is not fully ignored. That warning is expected only
54
+ for this public fixture because it deliberately commits `.ragmir/config.json` and
55
+ `.ragmir/sources.txt`. Generated storage, models, reports, audio, logs, and salts remain ignored.
56
+ Normal private projects should ignore the complete `.ragmir/` directory.
51
57
 
52
- ## Useful Test Queries
58
+ ## What to look for
53
59
 
54
- - `offline retrieval approval`
55
- - `dataset residency`
56
- - `incident containment evidence`
57
- - `who owns the usage review`
58
- - `what documents support sovereign deployment`
60
+ - Search and ask output identify the source path and chunk.
61
+ - `ask` returns cited passages, not a generated answer from a hosted LLM.
62
+ - `research` combines several retrieval queries and reports source diagnostics.
63
+ - The audit surfaces the fixture-specific tracked-config warning described above.
64
+ - The golden evaluation finds the expected sources for all four questions.
65
+ - `audit --unsupported` explains why the HEIC fixture was skipped and recommends local OCR.
66
+ - `status` confirms the active provider and local index state.
67
+
68
+ ## Useful queries
69
+
70
+ ```text
71
+ offline retrieval approval
72
+ dataset residency
73
+ incident containment evidence
74
+ who owns the usage review
75
+ what documents support sovereign deployment
76
+ ```
59
77
 
60
- ## Agent Report Prompt
78
+ Try a precise query first, then compare it with `research --compact` when the question spans several
79
+ files.
61
80
 
62
- After running `rgr setup` in a real project, an MCP-compatible agent can use the generated
63
- `.ragmir/mcp.json`. For this synthetic demo, ask an agent:
81
+ ## Give the demo to an agent
64
82
 
65
- ```plain text
83
+ In a real project, `rgr setup` generates an MCP helper under `.ragmir/`. After setup, an
84
+ MCP-compatible agent can follow a prompt like:
85
+
86
+ ```text
66
87
  Use Ragmir to inspect the local knowledge base, search for "offline retrieval approval", and write a
67
88
  cited Markdown report under .ragmir/reports/demo-sovereign-rag.md. Mention any unsupported or stale
68
89
  files reported by the audit.
69
90
  ```
70
91
 
71
- ## Switch To Transformers Semantic Mode
92
+ The report path stays under ignored local state. Do not commit generated reports that may contain
93
+ retrieved project evidence.
94
+
95
+ ## Compare semantic retrieval
96
+
97
+ The committed configuration uses:
98
+
99
+ ```json
100
+ {
101
+ "embeddingProvider": "local-hash"
102
+ }
103
+ ```
72
104
 
73
- To compare no-model retrieval with semantic local retrieval, change `.ragmir/config.json`:
105
+ This is offline lexical/hash retrieval, not model-semantic search. To run a deliberate semantic
106
+ comparison, configure a Transformers.js model, preload it under `.ragmir/models`, and rebuild:
74
107
 
75
108
  ```json
76
109
  {
@@ -82,22 +115,33 @@ To compare no-model retrieval with semantic local retrieval, change `.ragmir/con
82
115
  }
83
116
  ```
84
117
 
85
- Preload the model files under `.ragmir/models` for offline work, then rebuild the index:
86
-
87
118
  ```bash
88
- node ../../dist/cli.js ingest
89
- node ../../dist/cli.js ask "What documents support sovereign deployment?"
119
+ node ../../dist/cli.js ingest --rebuild
90
120
  node ../../dist/cli.js evaluate --golden golden-queries.json --json
91
- node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 0.8 --json
92
121
  ```
93
122
 
94
- ## Generated State
123
+ Keep remote model loading disabled for an offline comparison. A lower semantic score may indicate
124
+ that queries or thresholds need tuning, not that a gate should be weakened.
125
+
126
+ ## Generated state and cleanup
95
127
 
96
- Generated state stays local and ignored:
128
+ All runtime state stays under the ignored directory:
97
129
 
98
- ```plain text
130
+ ```text
99
131
  .ragmir/
100
132
  ```
101
133
 
102
- Do not replace these synthetic documents with real confidential files inside the Ragmir package
103
- repository.
134
+ Remove only the generated index when you want a clean rerun:
135
+
136
+ ```bash
137
+ node ../../dist/cli.js destroy-index --yes
138
+ ```
139
+
140
+ Never replace the fictional fixtures in this package with real confidential documents. Use a
141
+ separate ignored workspace for private dogfooding.
142
+
143
+ ## Continue exploring
144
+
145
+ - [Library API demo](../library-api-demo/README.md): call the public TypeScript API.
146
+ - [Document evidence benchmark](../document-evidence-benchmark/README.md): require exact paths and citations.
147
+ - [Ragmir Core README](../../README.md): installation, MCP, OCR, and privacy reference.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@jcode.labs/ragmir",
3
- "version": "2.9.2",
3
+ "version": "2.9.3",
4
4
  "description": "Ragmir Core: open-source sovereign local RAG for confidential datasets and AI agents.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -69,8 +69,8 @@
69
69
  "unpdf": "^1.4.0",
70
70
  "yaml": "^2.9.0",
71
71
  "zod": "^4.1.13",
72
- "@jcode.labs/ragmir-chat": "2.9.2",
73
- "@jcode.labs/ragmir-tts": "2.9.2"
72
+ "@jcode.labs/ragmir-chat": "2.9.3",
73
+ "@jcode.labs/ragmir-tts": "2.9.3"
74
74
  },
75
75
  "devDependencies": {
76
76
  "@types/html-to-text": "^9.0.4",