@jcode.labs/ragmir 2.18.0 → 2.18.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (7) hide show
  1. package/README.md +54 -91
  2. package/dist/version.d.ts +1 -1
  3. package/dist/version.js +1 -1
  4. package/examples/document-evidence-benchmark/README.md +25 -64
  5. package/examples/library-api-demo/README.md +17 -57
  6. package/examples/sovereign-rag-demo/README.md +37 -97
  7. package/package.json +5 -5
package/README.md CHANGED
@@ -5,21 +5,18 @@
5
5
  [![Node.js](https://img.shields.io/node/v/@jcode.labs/ragmir)](https://www.npmjs.com/package/@jcode.labs/ragmir)
6
6
  [![MIT](https://img.shields.io/npm/l/@jcode.labs/ragmir)](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE)
7
7
 
8
- Confidential local RAG for coding agents and Node.js applications.
9
-
10
- *Stop sending confidential documents directly to the cloud.*
11
-
12
- Core indexes the project files you select and retrieves bounded, cited evidence offline by default.
13
- It does not upload the corpus, call an LLM, or open an HTTP port.
8
+ Confidential local RAG for coding agents and Node.js applications. Core indexes the project files
9
+ you choose and retrieves bounded, cited evidence offline by default. It uploads no corpus, calls no
10
+ LLM, and opens no HTTP port.
14
11
 
15
12
  [Project overview](https://github.com/jcode-works/jcode-ragmir#readme) ·
16
- [CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md) ·
17
- [API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md) ·
13
+ [CLI](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md) ·
14
+ [API](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md) ·
18
15
  [Agent integration](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/agent-integration.md)
19
16
 
20
17
  ## Install and retrieve
21
18
 
22
- Requires Node.js 22 or later. Releases are gated on Linux x64 and macOS ARM64 with Node.js 22.
19
+ Requires Node.js 22 or later.
23
20
 
24
21
  ```bash
25
22
  npm install --save-dev @jcode.labs/ragmir
@@ -29,55 +26,22 @@ npx rgr ingest
29
26
  npx rgr search "Which decision changed the rollout?"
30
27
  ```
31
28
 
32
- Setup keeps configuration, the index, helpers, reports, and metadata-only access logs under ignored
33
- `.ragmir/` state. Ingestion is incremental, resumable, and serialized across local writer
34
- processes. Source, chunk, vector, concurrency, and batch windows are bounded, with durable progress
35
- committed per file. Completed ingestion refreshes incomplete full-text coverage and runs bounded
36
- LanceDB maintenance when mutation or fragment thresholds are reached. Sidecar replacement flushes
37
- before rename, synchronizes the storage directory where supported, and retains one validated
38
- activation manifest for explicit recovery diagnostics. Inspect maintenance with
39
- `npx rgr storage optimize --dry-run --json`. Exact vector search is retained below 100,000 rows;
40
- larger tables use a benchmarked IVF-PQ policy only with complete coverage. Use
41
- `npx rgr search "query" --exact-vector-search` to bypass ANN for diagnostics. Rebuild generations
42
- use private reader leases and
43
- bounded retention; inspect them with `npx rgr storage generations --json` before running
44
- `npx rgr storage gc --dry-run --json`. Failed changed files keep explicitly stale
45
- last-known-good rows by default; repair or source deletion reconciles them deterministically.
46
- Returned citations expose only verifiable coordinates: source lines for line-preserving text, PDF
47
- pages, PPTX slides, XLSX sheets and cells, and EPUB spine positions.
48
- Run `npx rgr audit --unsupported` to compare the selected files with the index and see what was
49
- skipped.
50
- Custom redaction expressions are screened for catastrophic backtracking. `npx rgr security-audit`
51
- also reports permissions, ignored/tracked state for every private path, and local extractor
52
- authority.
53
- `npx rgr status` and normal `npx rgr doctor` consume compact manifest health without opening chunk
54
- storage. Use `npx rgr doctor --deep` or `npx rgr audit` for explicit O(corpus) live diagnostics.
29
+ Generated configuration, indexes, helpers, reports, and metadata-only logs stay under ignored
30
+ `.ragmir/` state. Ingestion is incremental, resumable, bounded by source, chunk, vector, file, batch,
31
+ and concurrency windows, and serialized across local writer processes.
55
32
 
56
33
  ## Choose an interface
57
34
 
58
- | Interface | Use |
35
+ | Interface | Use it for |
59
36
  | --- | --- |
60
- | `rgr` CLI | Setup, ingest, search, audit, and JSON automation |
61
- | TypeScript API | Typed retrieval in a script or stateful Node.js worker |
62
- | Local stdio MCP | Bounded context for compatible coding agents |
63
-
64
- The default `local-hash` provider needs no model download. It is lexical/hash retrieval, not
65
- semantic embeddings. Enable local Transformers.js embeddings explicitly with `rgr setup --semantic`
66
- and rebuild the index. Bundled profiles use pinned model commits, and setup stores the resolved
67
- artifact digest so persisted compatibility identifies the exact weights. `local-hash` does not
68
- resolve Transformers.js, ONNX Runtime, or Sharp.
69
-
70
- Hybrid retrieval has stable source-and-chunk tie-breaks, two-pass recall-safe diversification, and
71
- provider-aware abstention. Pass `explain: true` to inspect vector and lexical contributions, FTS or
72
- fallback activation and reason, candidate and coverage budgets, plus the active ranking-policy
73
- fingerprint. Search, embedding, and ingestion use independent bounded queues per project root;
74
- overload and queue deadlines return stable retryable errors, and explained searches include queue
75
- wait.
76
- An empty result means every candidate failed the evidence threshold.
77
-
78
- `research` uses language-aware query expansion with deterministic cross-query ranking. It reads
79
- manifest health by default and exposes explicit timeout, code-file, code-byte, concurrency, and
80
- result budgets. Request `fullAudit: true` only when the report also needs a fresh source inventory.
37
+ | `rgr` CLI | Setup, ingest, search, audit, maintenance, and JSON automation |
38
+ | TypeScript API | Typed retrieval in scripts and long-running Node.js workers |
39
+ | Local stdio MCP | Bounded, read-focused context for compatible agents |
40
+
41
+ The default `local-hash` provider works offline with no model download. Enable semantic
42
+ Transformers.js embeddings explicitly with `rgr setup --semantic`, then rebuild. Core remains
43
+ retrieval-only in both modes. Use `explain: true` or `--explain` to inspect lexical and vector
44
+ contributions, fallback decisions, budgets, queue wait, and the active ranking policy.
81
45
 
82
46
  ## TypeScript API
83
47
 
@@ -92,50 +56,49 @@ try {
92
56
  timeoutMs: 10_000,
93
57
  })
94
58
 
95
- for (const result of results) {
96
- console.log(result.citation, result.text)
97
- }
59
+ for (const result of results) console.log(result.citation, result.text)
98
60
  } finally {
99
61
  await ragmir.close()
100
62
  }
101
63
  ```
102
64
 
103
- Reuse one client per project root in long-running processes. It caches one immutable read snapshot
104
- until atomic generation replacement, closes retired table handles after their last active reader,
105
- and `close()` flushes metadata-only access logs before releasing model ownership. The final owner
106
- disposes its Transformers pipeline only after active inference finishes. Top-level
107
- `ingest`, `search`, `ask`, and `research` functions remain available for one-shot scripts. `ask`
108
- returns cited retrieval context, not a generated answer.
109
-
110
- Core also exports setup, preview, audit, evaluation, source-routing, privacy, OCR, MCP, and agent
111
- integration helpers. See the [complete API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
112
- for public options and result types.
113
-
114
- Optional PDF OCR processes only blank pages. The generated setup batches bounded page groups and
115
- stores private content-addressed page results, so interrupted scans resume only missing pages and
116
- warm output remains identical without launching OCR subprocesses.
117
-
118
- ## Optional packages and boundaries
119
-
120
- - [`@jcode.labs/ragmir-chat`](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) adds local GGUF
121
- answer generation from retrieved passages.
65
+ Reuse one client per project root in a long-running process. It owns the local connection, read
66
+ snapshot, active operation lifecycle, metadata-only log flush, and optional embedding-model lease.
67
+ Top-level `ingest`, `search`, `ask`, and `research` functions remain available for one-shot scripts.
68
+ `ask` returns cited context, not generated prose.
69
+
70
+ ## Guarantees and boundaries
71
+
72
+ - Citations use source lines only for line-preserving text, plus PDF pages, PPTX slides, XLSX
73
+ sheets and cells, and EPUB spine positions.
74
+ - Rebuilds activate only after row and manifest validation. Interrupted rebuilds leave the previous
75
+ searchable generation active.
76
+ - Exact vector search remains the policy below 100,000 rows. Larger tables use quality-gated IVF-PQ
77
+ with complete coverage and an exact diagnostic mode.
78
+ - Search, embedding, and ingestion use independent bounded queues. Overload and queue deadlines are
79
+ stable retryable errors.
80
+ - Optional OCR processes only blank PDF pages through a configured local executable and private
81
+ resumable cache.
82
+ - `rgr status` and normal `rgr doctor` read compact manifest health. Use `rgr doctor --deep` or
83
+ `rgr audit` for a live source inventory.
84
+ - `rgr security-audit` checks permissions, Git ignore coverage, tracked private paths, redaction,
85
+ and local extractor authority.
86
+
87
+ The [CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md),
88
+ [API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md), and
89
+ [configuration guide](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
90
+ hold the complete options and operational detail.
91
+
92
+ ## Optional packages
93
+
94
+ - [`@jcode.labs/ragmir-chat`](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) adds cited
95
+ answer generation with a verified local GGUF profile.
122
96
  - [`@jcode.labs/ragmir-tts`](https://www.npmjs.com/package/@jcode.labs/ragmir-tts) renders reviewed
123
97
  text as local audio or explicit online speech.
124
98
 
125
- Core installs and starts without either add-on. A hosted agent receives only the passages your
126
- integration gives it, under that provider's data policy. Use a local consumer when no passage may
127
- leave the workstation. Ragmir has no hosted storage or cloud-sync layer; teams synchronize source
128
- files separately and build one local index per developer. Keep shared directory or glob contracts
129
- stable, align the Ragmir version, configuration, embedding provider, and model, then compare
130
- `corpusFingerprint` from `rgr status --json` after both indexes are ready with no missing or stale
131
- files. Matching values identify the same indexed relative paths and source bytes. Use
132
- `sourceFingerprintMode: "strict"` when a synchronization tool can preserve metadata while replacing
133
- content, and never share an actively written `.ragmir/storage/` directory. Older manifests expose a
134
- `null` fingerprint until the next successful ingestion.
135
-
136
- Read the [configuration guide](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md),
137
- [security hardening guide](https://github.com/jcode-works/jcode-ragmir/blob/main/SECURITY-HARDENING.md),
138
- and [troubleshooting guide](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/troubleshooting.md)
139
- before indexing sensitive material.
99
+ Core installs and starts without either add-on. A hosted agent receives only passages your
100
+ integration sends under that provider's data policy; use a local consumer when no passage may leave
101
+ the workstation. Teams synchronize their source folder and tracked configuration, then build one
102
+ local index per developer. See the [team workflow](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/agent-integration.md#team-knowledge-bases).
140
103
 
141
104
  Ragmir Core is open source under the [MIT License](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE).
package/dist/version.d.ts CHANGED
@@ -1,2 +1,2 @@
1
- export declare const VERSION = "2.18.0";
1
+ export declare const VERSION = "2.18.1";
2
2
  //# sourceMappingURL=version.d.ts.map
package/dist/version.js CHANGED
@@ -1,2 +1,2 @@
1
- export const VERSION = "2.18.0";
1
+ export const VERSION = "2.18.1";
2
2
  //# sourceMappingURL=version.js.map
package/examples/document-evidence-benchmark/README.md CHANGED
@@ -1,52 +1,36 @@
1
1
  # Document evidence benchmark
2
2
 
3
- A deterministic, public-safe benchmark for the retrieval and citation quality coding agents receive
4
- from Ragmir's confidential local RAG pipeline.
3
+ A deterministic, public-safe benchmark for the retrieval and citation quality a coding agent
4
+ receives from Ragmir. Six golden queries require the expected path and an exact file, line, chunk,
5
+ or PDF-page citation. The fictional corpus runs offline with `local-hash`.
5
6
 
6
- Use this example when you want to prove more than "a relevant file appeared." Its six golden queries
7
- check that Ragmir retrieves the expected path and an exact verifiable file, line, chunk, or PDF-page
8
- citation.
9
- Every document is fictional and safe to commit. The benchmark keeps its corpus and generated index
10
- on the machine and runs the default `local-hash` retrieval offline.
7
+ ## What passes
11
8
 
12
- ## What it measures
13
-
14
- | Signal | What a passing result means |
9
+ | Signal | Requirement |
15
10
  | --- | --- |
16
- | `Recall@K` | An expected source path appears within the configured top K results |
17
- | Exact text citation | The expected `relative/path:Lx-Ly#chunkIndex` is returned |
18
- | Exact PDF citation | The expected `relative/path:pN#chunkIndex` is returned without invented source lines |
19
-
20
- The corpus is intentionally small enough for each source to fit in one chunk. That makes citation
21
- expectations stable while exercising the same ingestion and evaluation code used for private
22
- knowledge bases.
11
+ | Recall@K | An expected source appears in the configured top K |
12
+ | Text citation | Exact `relative/path:Lx-Ly#chunkIndex` |
13
+ | PDF citation | Exact `relative/path:pN#chunkIndex`, with no invented source lines |
23
14
 
24
- ## Corpus
15
+ Each source fits in one chunk so citation expectations stay stable while exercising the production
16
+ ingestion and evaluation pipeline.
25
17
 
26
- | Fictional source | Evidence type |
18
+ | Fictional source | Evidence |
27
19
  | --- | --- |
28
20
  | `raw/contracts/master-services-agreement.md` | Contract and residency obligations |
29
- | `raw/contracts/pdf-control-evidence.pdf` | Embedded PDF text with a page citation |
21
+ | `raw/contracts/pdf-control-evidence.pdf` | Embedded PDF text and page citation |
30
22
  | `raw/rfp/security-questionnaire.md` | Security and hosting requirements |
31
23
  | `raw/runbooks/incident-response-runbook.md` | Operational evidence collection |
32
24
  | `raw/specs/agent-integration-spec.md` | Agent and retrieval boundaries |
33
25
  | `raw/legal-tax/residency-review-note.md` | Professional-review disclaimer |
34
26
 
35
- [`golden-queries.json`](./golden-queries.json) maps each test query to its expected path and exact
36
- citation.
27
+ [`golden-queries.json`](./golden-queries.json) maps questions to expected paths and citations.
37
28
 
38
- ## Run the benchmark
39
-
40
- From the repository root, install dependencies and build the packages:
29
+ ## Run it
41
30
 
42
31
  ```bash
43
32
  pnpm install --frozen-lockfile
44
33
  pnpm build
45
- ```
46
-
47
- Then run the benchmark from this directory:
48
-
49
- ```bash
50
34
  cd packages/ragmir-core/examples/document-evidence-benchmark
51
35
  node ../../dist/cli.js init
52
36
  node ../../dist/cli.js ingest --rebuild
@@ -54,40 +38,17 @@ node ../../dist/cli.js evaluate --golden golden-queries.json --json
54
38
  node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 1
55
39
  ```
56
40
 
57
- The last command exits non-zero unless every golden query succeeds at the configured threshold.
58
- Keep the JSON output when you need machine-readable evidence for CI or a retrieval-quality review.
59
-
60
- ## Read a failure
61
-
62
- 1. Check `rgr audit` for missing, stale, or skipped source files.
63
- 2. Inspect the failed query's returned paths and citations in JSON output.
64
- 3. Decide whether the corpus, expected citation, chunking, or retrieval mode changed.
65
- 4. Update a golden expectation only when the intended product behavior changed.
66
-
67
- Do not lower `--fail-under` merely to make a regression pass.
68
-
69
- ## Adapt it to a private corpus
70
-
71
- Copy the structure, not the fictional documents:
72
-
73
- ```text
74
- private-evaluation/
75
- ├── .ragmir/config.json
76
- ├── golden-queries.json
77
- └── raw/
78
- ```
79
-
80
- For each important question, record the smallest set of expected source paths. Add exact citations
81
- when the source is stable enough to make line, page, and chunk boundaries meaningful. Keep private
82
- contracts, RFPs, runbooks, tax notes, scans, golden queries, and evaluation reports outside Git or
83
- under ignored local state.
41
+ The final command fails unless every query meets the threshold. Keep JSON output when CI or a
42
+ retrieval review needs machine-readable evidence.
84
43
 
85
- ## Safety
44
+ On failure, run `rgr audit`, inspect returned paths and citations, then decide whether retrieval,
45
+ chunking, the corpus, or the intended expectation changed. Update a golden query only for an
46
+ intentional behavior change; never lower `--fail-under` to hide a regression.
86
47
 
87
- - The committed corpus contains no customer or production data.
88
- - The benchmark uses the offline `local-hash` provider and needs no model download.
89
- - Generated `.ragmir/storage` and evaluation output stay local and ignored.
90
- - A perfect synthetic score proves this fixture, not universal retrieval quality.
48
+ For private evaluation, copy the `.ragmir/config.json`, `golden-queries.json`, and `raw/` structure
49
+ into an ignored workspace. Record the smallest expected source set and exact coordinates only when
50
+ the source is stable. Never commit private documents, queries, or reports.
91
51
 
92
- Return to the [Ragmir Core README](../../README.md) or try the
93
- [confidential local RAG demo](../sovereign-rag-demo/README.md) for a broader CLI walkthrough.
52
+ A perfect score proves this fixture, not universal retrieval quality. Continue with the
53
+ [confidential local RAG demo](../sovereign-rag-demo/README.md) or
54
+ [Core README](../../README.md).
package/examples/library-api-demo/README.md CHANGED
@@ -1,50 +1,24 @@
1
1
  # Library API demo
2
2
 
3
- A runnable smoke test for the public `@jcode.labs/ragmir` TypeScript API behind confidential local
4
- RAG workflows for coding agents and scripts.
5
-
6
- Use this example when you are changing Ragmir Core or evaluating it as a library rather than as a
7
- CLI. It imports the same package name an external consumer uses, runs the full retrieval loop, and
8
- prints cited results from a fictional local corpus. The default `local-hash` path keeps the corpus
9
- and generated index on the machine and performs retrieval offline.
10
-
11
- ## What it proves
12
-
13
- The demo opens one persistent client and exercises four public operations in order:
14
-
15
- 1. `ragmir.ingest({ rebuild: true })` parses, redacts, chunks, embeds, and stores the corpus.
16
- 2. `ragmir.search(query, { topK })` returns ranked source passages.
17
- 3. `ragmir.ask(query, { topK })` returns retrieval-only cited context without LLM synthesis.
18
- 4. `ragmir.status()` reports the active knowledge base and indexed coverage.
19
-
20
- Node's package self-reference resolves `@jcode.labs/ragmir` to this checkout's local
21
- `packages/ragmir-core/dist` build. It never falls back to the npm-published version, so the result
22
- reflects the code you are currently reviewing.
3
+ A runnable smoke test for the public `@jcode.labs/ragmir` TypeScript API. It opens one persistent
4
+ client, ingests a fictional local corpus, retrieves cited passages, returns retrieval-only context,
5
+ and reports status through the same package name an external consumer imports.
23
6
 
24
7
  ## Run it
25
8
 
26
- From the repository root:
27
-
28
9
  ```bash
29
10
  pnpm install --frozen-lockfile
30
11
  pnpm example
31
12
  ```
32
13
 
33
- The root command builds the published packages required by Core and runs [`run.mjs`](./run.mjs). The
34
- output is organized into `ingest`, `search`, `ask`, and `status` sections so a reviewer can see each
35
- public step succeed.
36
-
37
- To rerun only the script after an existing build:
14
+ The root command builds the packages and runs [`run.mjs`](./run.mjs). Package self-reference points
15
+ to this checkout's `packages/ragmir-core/dist`, never the npm release. To rerun after a build:
38
16
 
39
17
  ```bash
40
18
  node packages/ragmir-core/examples/library-api-demo/run.mjs
41
19
  ```
42
20
 
43
- `dist/` is ignored, so a clean clone must be built at least once.
44
-
45
- ## The public API pattern
46
-
47
- The essential integration is intentionally small:
21
+ ## Integration pattern
48
22
 
49
23
  ```js
50
24
  import { createRagmirClient } from "@jcode.labs/ragmir"
@@ -52,39 +26,25 @@ import { createRagmirClient } from "@jcode.labs/ragmir"
52
26
  const ragmir = await createRagmirClient({ cwd: projectRoot })
53
27
  try {
54
28
  await ragmir.ingest({ rebuild: true, timeoutMs: 30_000 })
55
-
56
29
  const results = await ragmir.search("offline retrieval approval", { topK: 3 })
57
-
58
30
  const context = await ragmir.ask("What evidence supports offline operation?", { topK: 3 })
59
-
60
31
  const status = await ragmir.status()
61
32
  } finally {
62
33
  await ragmir.close()
63
34
  }
64
35
  ```
65
36
 
66
- All project-relative state is resolved once from `cwd`. Reuse one client per project root in a
67
- long-running Node.js process and close it during shutdown. For one-shot scripts, the top-level
68
- `ingest`, `search`, `ask`, and `audit` functions remain available.
69
-
70
- ## Data used by the demo
37
+ The demo proves `ingest`, `search`, `ask`, and `status` in order. `ask` returns cited context without
38
+ LLM synthesis. Reuse one client per project root in a long-running process; one-shot top-level
39
+ functions remain available.
71
40
 
72
- The script reuses the sibling [confidential local RAG demo](../sovereign-rag-demo/README.md):
41
+ The script shares the sibling demo's synthetic Markdown, CSV, JSONL, YAML, and custom-text corpus.
42
+ It uses offline `local-hash`, writes only ignored `.ragmir/storage`, downloads no model, and contains
43
+ no private documents. Keep committed changes deterministic and use a separate ignored corpus for
44
+ private tests.
73
45
 
74
- - committed synthetic Markdown, CSV, JSONL, YAML, and custom text files;
75
- - an offline `local-hash` configuration;
76
- - gitignored `.ragmir/storage` output;
77
- - no private documents and no model download.
78
-
79
- Change the queries in `run.mjs` when testing a retrieval behavior, but keep the committed demo
80
- deterministic and public-safe. Use a separate ignored corpus for private evaluations.
81
-
82
- ## When to use another example
83
-
84
- | Need | Example |
46
+ | Need | Continue with |
85
47
  | --- | --- |
86
- | Learn the complete CLI workflow | [Confidential local RAG demo](../sovereign-rag-demo/README.md) |
87
- | Measure exact paths and citations | [Document evidence benchmark](../document-evidence-benchmark/README.md) |
88
- | Explore every exported function and option | [Complete TypeScript API reference](../../../../docs/api-reference.md) |
89
-
90
- Return to the [Ragmir Core README](../../README.md) for installation, MCP, OCR, and privacy guidance.
48
+ | Complete CLI workflow | [Confidential local RAG demo](../sovereign-rag-demo/README.md) |
49
+ | Exact path and citation gates | [Document evidence benchmark](../document-evidence-benchmark/README.md) |
50
+ | Every exported option | [TypeScript API reference](../../../../docs/api-reference.md) |
package/examples/sovereign-rag-demo/README.md CHANGED
@@ -1,42 +1,29 @@
1
1
  # Confidential local RAG demo
2
2
 
3
- A complete, fictional workspace for learning how agents and scripts retrieve cited local project
4
- context with Ragmir.
5
-
6
- Use this example to ingest several file types, search with citations, inspect privacy posture, find
7
- an unsupported file, evaluate golden queries, and prepare agent access. It keeps the fictional
8
- corpus and generated index on the machine, runs with offline `local-hash` retrieval, and contains no
9
- customer or production data.
3
+ A complete fictional workspace for learning how agents and scripts retrieve cited project evidence
4
+ with Ragmir. It covers multiple formats, privacy checks, unsupported files, golden queries, and
5
+ agent access while keeping corpus and index local with offline `local-hash` retrieval.
10
6
 
11
7
  ## Scenario
12
8
 
13
- The corpus models an operations team reviewing whether an evidence workflow can stay local:
14
-
15
- | File | Role in the scenario |
9
+ | File | Role |
16
10
  | --- | --- |
17
11
  | `raw/operations-brief.md` | Approval, ownership, and operating requirements |
18
12
  | `raw/dataset-inventory.csv` | Accepted and rejected datasets |
19
13
  | `raw/incident-timeline.jsonl` | Time-ordered operational evidence |
20
14
  | `raw/security-policy.yaml` | Model-loading and security rules |
21
- | `raw/review-notes.evidence` | Custom text extension configured for this project |
22
- | `raw/facility-scan.heic` | Deliberately unsupported file used by the audit example |
15
+ | `raw/review-notes.evidence` | Project-configured text extension |
16
+ | `raw/facility-scan.heic` | Deliberately unsupported audit fixture |
23
17
 
24
- ## Run the full workflow
25
-
26
- From the repository root:
18
+ ## Run the workflow
27
19
 
28
20
  ```bash
29
21
  pnpm install --frozen-lockfile
30
22
  pnpm build
31
- ```
32
-
33
- Then enter this example and run the local CLI build:
34
-
35
- ```bash
36
23
  cd packages/ragmir-core/examples/sovereign-rag-demo
37
24
  node ../../dist/cli.js init
38
25
 
39
- # POSIX only: this example deliberately uses a custom tracked rawDir.
26
+ # POSIX only: Git and npm archives do not preserve this fixture's intended mode.
40
27
  chmod 700 raw
41
28
 
42
29
  node ../../dist/cli.js security-audit
@@ -49,101 +36,54 @@ node ../../dist/cli.js audit --unsupported
49
36
  node ../../dist/cli.js status
50
37
  ```
51
38
 
52
- `init` is idempotent. Git and npm archives cannot preserve the example's intended `0700` raw-folder
53
- mode, so the explicit `chmod` keeps the raw corpus owner-only on POSIX systems.
39
+ `init` is idempotent. The security audit intentionally warns that `.ragmir/` is not fully ignored
40
+ because this public fixture commits its configuration and source list. Generated storage, models,
41
+ reports, audio, logs, and salts remain ignored. Private projects should ignore the complete
42
+ `.ragmir/` directory.
54
43
 
55
- The security audit still reports that `.ragmir/` is not fully ignored. That warning is expected only
56
- for this public fixture because it deliberately commits `.ragmir/config.json` and
57
- `.ragmir/sources.txt`. Generated storage, models, reports, audio, logs, and salts remain ignored.
58
- Normal private projects should ignore the complete `.ragmir/` directory.
44
+ Expected results:
59
45
 
60
- ## What to look for
46
+ - search and ask identify source paths and chunks;
47
+ - `ask` returns evidence, not a generated answer;
48
+ - `research` combines bounded queries and source diagnostics;
49
+ - every golden query finds its expected source;
50
+ - the audit explains why HEIC was skipped and suggests local OCR;
51
+ - status confirms the active provider and local index.
61
52
 
62
- - Search and ask output identify the source path and chunk.
63
- - `ask` returns cited passages, not a generated answer from a hosted LLM.
64
- - `research` combines several retrieval queries and reports source diagnostics.
65
- - The audit surfaces the fixture-specific tracked-config warning described above.
66
- - The golden evaluation finds the expected sources for all four questions.
67
- - `audit --unsupported` explains why the HEIC fixture was skipped and recommends local OCR.
68
- - `status` confirms the active provider and local index state.
53
+ Useful queries include `dataset residency`, `incident containment evidence`,
54
+ `who owns the usage review`, and `what documents support sovereign deployment`.
69
55
 
70
- ## Useful queries
56
+ ## Connect an agent
71
57
 
72
- ```text
73
- offline retrieval approval
74
- dataset residency
75
- incident containment evidence
76
- who owns the usage review
77
- what documents support sovereign deployment
78
- ```
79
-
80
- Try a precise query first, then compare it with `research --compact` when the question spans several
81
- files.
82
-
83
- ## Give the demo to an agent
84
-
85
- In a real project, `rgr setup` generates an MCP helper under `.ragmir/`. After setup, an
86
- MCP-compatible agent can follow a prompt like:
58
+ In a real project, `rgr setup` writes an ignored MCP helper. A compatible agent can then follow:
87
59
 
88
60
  ```text
89
- Use Ragmir to inspect the local knowledge base, search for "offline retrieval approval", and write a
90
- cited Markdown report under .ragmir/reports/demo-sovereign-rag.md. Mention any unsupported or stale
91
- files reported by the audit.
61
+ Use Ragmir to search for "offline retrieval approval" and write a cited report under
62
+ .ragmir/reports/demo-sovereign-rag.md. Mention unsupported or stale files reported by the audit.
92
63
  ```
93
64
 
94
- The report path stays under ignored local state. Do not commit generated reports that may contain
95
- retrieved project evidence.
65
+ Keep retrieved reports under ignored local state.
96
66
 
97
67
  ## Compare semantic retrieval
98
68
 
99
- The committed configuration uses:
69
+ The committed configuration uses offline lexical/hash retrieval:
100
70
 
101
71
  ```json
102
- {
103
- "embeddingProvider": "local-hash"
104
- }
72
+ { "embeddingProvider": "local-hash" }
105
73
  ```
106
74
 
107
- This is offline lexical/hash retrieval, not model-semantic search. To run a deliberate semantic
108
- comparison, configure a Transformers.js model, preload it under `.ragmir/models`, and rebuild:
75
+ For a deliberate semantic comparison, prepare a Transformers.js model locally, set
76
+ `embeddingProvider`, `embeddingModel`, `embeddingModelRevision`, `embeddingModelPath`, and
77
+ `transformersAllowRemoteModels: false`, then rebuild and rerun evaluation. Keep remote loading off.
78
+ A lower semantic score is a tuning signal, never a reason to weaken a quality gate.
109
79
 
110
- ```json
111
- {
112
- "embeddingProvider": "transformers",
113
- "embeddingModel": "intfloat/multilingual-e5-small",
114
- "embeddingModelRevision": "main",
115
- "embeddingModelPath": ".ragmir/models",
116
- "transformersAllowRemoteModels": false
117
- }
118
- ```
119
-
120
- ```bash
121
- node ../../dist/cli.js ingest --rebuild
122
- node ../../dist/cli.js evaluate --golden golden-queries.json --json
123
- ```
124
-
125
- Keep remote model loading disabled for an offline comparison. A lower semantic score may indicate
126
- that queries or thresholds need tuning, not that a gate should be weakened.
127
-
128
- ## Generated state and cleanup
129
-
130
- All runtime state stays under the ignored directory:
131
-
132
- ```text
133
- .ragmir/
134
- ```
135
-
136
- Remove only the generated index when you want a clean rerun:
80
+ All generated state stays under `.ragmir/`. Reset only the index with:
137
81
 
138
82
  ```bash
139
83
  node ../../dist/cli.js destroy-index --yes
140
84
  ```
141
85
 
142
- Never replace the fictional fixtures in this package with real confidential documents. Use a
143
- separate ignored workspace for private dogfooding.
144
-
145
- ## Continue exploring
146
-
147
- - [Library API demo](../library-api-demo/README.md): call the public TypeScript API.
148
- - [Document evidence benchmark](../document-evidence-benchmark/README.md): require exact paths and citations.
149
- - [Ragmir Core README](../../README.md): installation, MCP, OCR, and privacy reference.
86
+ Never replace the fictional fixtures with confidential documents. Continue with the
87
+ [Library API demo](../library-api-demo/README.md),
88
+ [Document evidence benchmark](../document-evidence-benchmark/README.md), or
89
+ [Core README](../../README.md).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@jcode.labs/ragmir",
3
- "version": "2.18.0",
3
+ "version": "2.18.1",
4
4
  "description": "Confidential local RAG for coding agents, with offline cited retrieval through CLI, MCP, and TypeScript.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -78,8 +78,8 @@
78
78
  "zod": "^4.1.13"
79
79
  },
80
80
  "peerDependencies": {
81
- "@jcode.labs/ragmir-chat": "2.18.0",
82
- "@jcode.labs/ragmir-tts": "2.18.0"
81
+ "@jcode.labs/ragmir-chat": "2.18.1",
82
+ "@jcode.labs/ragmir-tts": "2.18.1"
83
83
  },
84
84
  "peerDependenciesMeta": {
85
85
  "@jcode.labs/ragmir-chat": {
@@ -96,8 +96,8 @@
96
96
  "publint": "^0.3.21",
97
97
  "typescript": "^5.9.3",
98
98
  "vitest": "^4.0.15",
99
- "@jcode.labs/ragmir-chat": "2.18.0",
100
- "@jcode.labs/ragmir-tts": "2.18.0"
99
+ "@jcode.labs/ragmir-chat": "2.18.1",
100
+ "@jcode.labs/ragmir-tts": "2.18.1"
101
101
  },
102
102
  "scripts": {
103
103
  "bench:compare": "node benchmarks/compare.mjs",