@jcode.labs/ragmir 2.9.1 → 2.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -1,16 +1,172 @@
|
|
|
1
1
|
# Ragmir Core
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
[](https://www.npmjs.com/package/@jcode.labs/ragmir)
|
|
4
|
+
[](https://www.npmjs.com/package/@jcode.labs/ragmir)
|
|
5
|
+
[](https://www.npmjs.com/package/@jcode.labs/ragmir)
|
|
6
|
+
[](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE)
|
|
7
|
+
|
|
8
|
+
**The CLI, TypeScript API, and local MCP server for cited project retrieval.**
|
|
9
|
+
|
|
10
|
+
`@jcode.labs/ragmir` indexes the files a project selects and returns source-backed passages without
|
|
11
|
+
a hosted document store. The default `local-hash` retrieval path works offline, without an account
|
|
12
|
+
or model download.
|
|
13
|
+
|
|
14
|
+
[Project overview](https://github.com/jcode-works/jcode-ragmir#readme) ·
|
|
15
|
+
[CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md) ·
|
|
16
|
+
[API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
|
|
17
|
+
|
|
18
|
+
## Install Core when you need
|
|
19
|
+
|
|
20
|
+
| Goal | Core interface |
|
|
21
|
+
| --- | --- |
|
|
22
|
+
| Search repository documents with citations | `rgr search` |
|
|
23
|
+
| Audit a local knowledge base | `rgr audit`, `rgr doctor`, and `rgr security-audit` |
|
|
24
|
+
| Add retrieval to a Node.js application | Typed `ingest`, `search`, `ask`, and `research` exports |
|
|
25
|
+
| Give an AI agent bounded project context | Local stdio MCP server and generated helpers |
|
|
26
|
+
| Index scanned PDFs | Embedded text first, with optional page-aware local OCR |
|
|
27
|
+
|
|
28
|
+
Core retrieves evidence. It does not require a generative model and does not produce an ungrounded
|
|
29
|
+
answer. Add [Ragmir Chat](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) only when local
|
|
30
|
+
GGUF synthesis is useful, or [Ragmir TTS](https://www.npmjs.com/package/@jcode.labs/ragmir-tts)
|
|
31
|
+
when you need audio output.
|
|
32
|
+
|
|
33
|
+
## First cited search
|
|
34
|
+
|
|
35
|
+
Requires Node.js 20 or later.
|
|
5
36
|
|
|
6
37
|
```bash
|
|
7
38
|
npm install --save-dev @jcode.labs/ragmir
|
|
8
39
|
npx rgr setup
|
|
9
|
-
npx rgr sources add "docs/**/*.md"
|
|
40
|
+
npx rgr sources add "README.md" "docs/**/*.md"
|
|
10
41
|
npx rgr ingest
|
|
11
|
-
npx rgr search "
|
|
42
|
+
npx rgr search "Which decision changed the rollout?"
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
The project now owns an ignored `.ragmir/` directory containing configuration and generated local
|
|
46
|
+
state. Ingestion is incremental, and every result identifies the source path, chunk, line range, and
|
|
47
|
+
PDF page when one is available.
|
|
48
|
+
|
|
49
|
+
## CLI essentials
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
# Readiness and source coverage
|
|
53
|
+
npx rgr doctor
|
|
54
|
+
npx rgr audit --unsupported
|
|
55
|
+
npx rgr security-audit
|
|
56
|
+
|
|
57
|
+
# Retrieval
|
|
58
|
+
npx rgr search "deployment decision"
|
|
59
|
+
npx rgr ask "What evidence supports the deployment decision?"
|
|
60
|
+
npx rgr research "deployment obligations" --compact
|
|
61
|
+
|
|
62
|
+
# Machine-readable output
|
|
63
|
+
npx rgr search "deployment decision" --json
|
|
12
64
|
```
|
|
13
65
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
66
|
+
`ask` returns cited retrieval context, not LLM synthesis. `research` performs an audit-backed,
|
|
67
|
+
multi-query retrieval pass and reports missing or weak evidence.
|
|
68
|
+
|
|
69
|
+
## TypeScript API
|
|
70
|
+
|
|
71
|
+
```ts
|
|
72
|
+
import { ingest, search } from "@jcode.labs/ragmir"
|
|
73
|
+
|
|
74
|
+
await ingest({ cwd: process.cwd() })
|
|
75
|
+
|
|
76
|
+
const results = await search("Which decision changed the rollout?", {
|
|
77
|
+
cwd: process.cwd(),
|
|
78
|
+
topK: 5,
|
|
79
|
+
})
|
|
80
|
+
|
|
81
|
+
for (const result of results) {
|
|
82
|
+
console.log(result.citation, result.text)
|
|
83
|
+
}
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Frequently used exports:
|
|
87
|
+
|
|
88
|
+
| Export | Purpose |
|
|
89
|
+
| --- | --- |
|
|
90
|
+
| `setupProject`, `addSourceEntries` | Initialize project state and select files |
|
|
91
|
+
| `ingest`, `audit` | Build the index and compare it with files on disk |
|
|
92
|
+
| `search`, `ask`, `research` | Retrieve cited passages at different levels of breadth |
|
|
93
|
+
| `doctor`, `securityAudit` | Inspect readiness and local privacy posture |
|
|
94
|
+
| `serveMcp` | Start the read-focused local MCP server |
|
|
95
|
+
| `configurePdfOcr`, `inspectPdfOcr` | Configure and inspect local PDF OCR |
|
|
96
|
+
|
|
97
|
+
See the [complete API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
|
|
98
|
+
for options and result shapes.
|
|
99
|
+
|
|
100
|
+
## Connect an AI agent
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
npx rgr setup --agents claude,codex,kimi,opencode,cline
|
|
104
|
+
npx rgr doctor
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Ragmir writes helper files for the selected clients and points them at the current project. MCP
|
|
108
|
+
exposes status, search, ask, research, audit, evaluation, usage, and security tools. It does not
|
|
109
|
+
expose index deletion.
|
|
110
|
+
|
|
111
|
+
## Retrieval modes
|
|
112
|
+
|
|
113
|
+
| Mode | Network requirement | Best for |
|
|
114
|
+
| --- | --- | --- |
|
|
115
|
+
| `local-hash` | None | Default offline retrieval, setup-free projects, and deterministic evaluation |
|
|
116
|
+
| `transformers` | Explicit model preload or download | Higher-quality semantic matching with local inference |
|
|
117
|
+
|
|
118
|
+
Enable semantic retrieval explicitly:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
npx rgr setup --semantic
|
|
122
|
+
npx rgr ingest --rebuild
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
`local-hash` is lexical/hash retrieval. It should not be presented as equivalent to semantic
|
|
126
|
+
embeddings.
|
|
127
|
+
|
|
128
|
+
## Documents and OCR
|
|
129
|
+
|
|
130
|
+
Core parses common source, text, structured-data, Office, OpenDocument, EPUB, HTML, email, notebook,
|
|
131
|
+
and PDF formats. Projects can add custom text extensions. Run `rgr audit --unsupported` to see every
|
|
132
|
+
skipped file and recommendation.
|
|
133
|
+
|
|
134
|
+
For scanned PDFs:
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
npx rgr ocr setup --engine auto
|
|
138
|
+
npx rgr ingest --rebuild
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Ragmir prefers embedded PDF text and invokes OCR only for blank pages. OCR runs through a configured
|
|
142
|
+
local executable, never a shell string or cloud OCR service.
|
|
143
|
+
|
|
144
|
+
## Local data and privacy
|
|
145
|
+
|
|
146
|
+
- Project paths resolve from `cwd` or explicit configuration, not the npm installation directory.
|
|
147
|
+
- Generated indexes, models, reports, audio, and access logs belong under ignored `.ragmir/` state.
|
|
148
|
+
- Redaction runs before indexing when configured, but it is not a compliance certification.
|
|
149
|
+
- External extractors and model downloads are opt-in system boundaries.
|
|
150
|
+
- MCP retrieval is bounded and access logs contain metadata, not query text or retrieved passages.
|
|
151
|
+
|
|
152
|
+
Read [configuration and privacy](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
|
|
153
|
+
and [security hardening](https://github.com/jcode-works/jcode-ragmir/blob/main/SECURITY-HARDENING.md)
|
|
154
|
+
before indexing sensitive material.
|
|
155
|
+
|
|
156
|
+
## Runnable examples
|
|
157
|
+
|
|
158
|
+
| Example | Demonstrates |
|
|
159
|
+
| --- | --- |
|
|
160
|
+
| [Sovereign RAG demo](https://github.com/jcode-works/jcode-ragmir/tree/main/packages/ragmir-core/examples/sovereign-rag-demo) | CLI ingestion, search, audit, redaction, and evaluation |
|
|
161
|
+
| [Library API demo](https://github.com/jcode-works/jcode-ragmir/tree/main/packages/ragmir-core/examples/library-api-demo) | The public library surface against fictional files |
|
|
162
|
+
| [Document evidence benchmark](https://github.com/jcode-works/jcode-ragmir/tree/main/packages/ragmir-core/examples/document-evidence-benchmark) | Recall and exact citation evaluation |
|
|
163
|
+
|
|
164
|
+
## Documentation
|
|
165
|
+
|
|
166
|
+
- [CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md)
|
|
167
|
+
- [TypeScript API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
|
|
168
|
+
- [Configuration](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
|
|
169
|
+
- [Agent integration](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/agent-integration.md)
|
|
170
|
+
- [Troubleshooting](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/troubleshooting.md)
|
|
171
|
+
|
|
172
|
+
Ragmir Core is open source under the [MIT License](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE).
|
package/dist/version.d.ts
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export declare const VERSION = "2.9.
|
|
1
|
+
export declare const VERSION = "2.9.3";
|
|
2
2
|
//# sourceMappingURL=version.d.ts.map
|
package/dist/version.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export const VERSION = "2.9.
|
|
1
|
+
export const VERSION = "2.9.3";
|
|
2
2
|
//# sourceMappingURL=version.js.map
|
|
@@ -1,36 +1,90 @@
|
|
|
1
|
-
# Document
|
|
1
|
+
# Document evidence benchmark
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
RFP, runbook, spec, and legal/tax notes are fictional and contain no customer data.
|
|
3
|
+
A deterministic, public-safe benchmark for Ragmir retrieval and citation quality.
|
|
5
4
|
|
|
6
|
-
|
|
5
|
+
Use this example when you want to prove more than "a relevant file appeared." Its six golden queries
|
|
6
|
+
check that Ragmir retrieves the expected path and the exact file, line, chunk, or PDF-page citation.
|
|
7
|
+
Every document is fictional and safe to commit.
|
|
7
8
|
|
|
8
|
-
|
|
9
|
-
- exact citations: each query retrieves the expected `relative/path:Lx-Ly#chunkIndex` citation.
|
|
10
|
-
- embedded PDF text: the synthetic PDF is extracted and retrieved with an exact
|
|
11
|
-
`relative/path:p1:Lx-Ly#chunkIndex` page citation.
|
|
9
|
+
## What it measures
|
|
12
10
|
|
|
13
|
-
|
|
14
|
-
|
|
11
|
+
| Signal | What a passing result means |
|
|
12
|
+
| --- | --- |
|
|
13
|
+
| `Recall@K` | An expected source path appears within the configured top K results |
|
|
14
|
+
| Exact text citation | The expected `relative/path:Lx-Ly#chunkIndex` is returned |
|
|
15
|
+
| Exact PDF citation | The expected `relative/path:pN:Lx-Ly#chunkIndex` is returned |
|
|
15
16
|
|
|
16
|
-
|
|
17
|
+
The corpus is intentionally small enough for each source to fit in one chunk. That makes citation
|
|
18
|
+
expectations stable while exercising the same ingestion and evaluation code used for private
|
|
19
|
+
knowledge bases.
|
|
17
20
|
|
|
18
|
-
|
|
21
|
+
## Corpus
|
|
22
|
+
|
|
23
|
+
| Fictional source | Evidence type |
|
|
24
|
+
| --- | --- |
|
|
25
|
+
| `raw/contracts/master-services-agreement.md` | Contract and residency obligations |
|
|
26
|
+
| `raw/contracts/pdf-control-evidence.pdf` | Embedded PDF text with a page citation |
|
|
27
|
+
| `raw/rfp/security-questionnaire.md` | Security and hosting requirements |
|
|
28
|
+
| `raw/runbooks/incident-response-runbook.md` | Operational evidence collection |
|
|
29
|
+
| `raw/specs/agent-integration-spec.md` | Agent and retrieval boundaries |
|
|
30
|
+
| `raw/legal-tax/residency-review-note.md` | Professional-review disclaimer |
|
|
31
|
+
|
|
32
|
+
[`golden-queries.json`](./golden-queries.json) maps each test query to its expected path and exact
|
|
33
|
+
citation.
|
|
34
|
+
|
|
35
|
+
## Run the benchmark
|
|
36
|
+
|
|
37
|
+
From the repository root, install dependencies and build the packages:
|
|
19
38
|
|
|
20
39
|
```bash
|
|
40
|
+
pnpm install --frozen-lockfile
|
|
21
41
|
pnpm build
|
|
22
42
|
```
|
|
23
43
|
|
|
24
|
-
Then run the benchmark from this
|
|
44
|
+
Then run the benchmark from this directory:
|
|
25
45
|
|
|
26
46
|
```bash
|
|
27
47
|
cd packages/ragmir-core/examples/document-evidence-benchmark
|
|
28
48
|
node ../../dist/cli.js init
|
|
29
|
-
node ../../dist/cli.js ingest
|
|
49
|
+
node ../../dist/cli.js ingest --rebuild
|
|
30
50
|
node ../../dist/cli.js evaluate --golden golden-queries.json --json
|
|
31
51
|
node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 1
|
|
32
52
|
```
|
|
33
53
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
54
|
+
The last command exits non-zero unless every golden query succeeds at the configured threshold.
|
|
55
|
+
Keep the JSON output when you need machine-readable evidence for CI or a retrieval-quality review.
|
|
56
|
+
|
|
57
|
+
## Read a failure
|
|
58
|
+
|
|
59
|
+
1. Check `rgr audit` for missing, stale, or skipped source files.
|
|
60
|
+
2. Inspect the failed query's returned paths and citations in JSON output.
|
|
61
|
+
3. Decide whether the corpus, expected citation, chunking, or retrieval mode changed.
|
|
62
|
+
4. Update a golden expectation only when the intended product behavior changed.
|
|
63
|
+
|
|
64
|
+
Do not lower `--fail-under` merely to make a regression pass.
|
|
65
|
+
|
|
66
|
+
## Adapt it to a private corpus
|
|
67
|
+
|
|
68
|
+
Copy the structure, not the fictional documents:
|
|
69
|
+
|
|
70
|
+
```text
|
|
71
|
+
private-evaluation/
|
|
72
|
+
├── .ragmir/config.json
|
|
73
|
+
├── golden-queries.json
|
|
74
|
+
└── raw/
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
For each important question, record the smallest set of expected source paths. Add exact citations
|
|
78
|
+
when the source is stable enough to make line, page, and chunk boundaries meaningful. Keep private
|
|
79
|
+
contracts, RFPs, runbooks, tax notes, scans, golden queries, and evaluation reports outside Git or
|
|
80
|
+
under ignored local state.
|
|
81
|
+
|
|
82
|
+
## Safety
|
|
83
|
+
|
|
84
|
+
- The committed corpus contains no customer or production data.
|
|
85
|
+
- The benchmark uses the offline `local-hash` provider and needs no model download.
|
|
86
|
+
- Generated `.ragmir/storage` and evaluation output stay local and ignored.
|
|
87
|
+
- A perfect synthetic score proves this fixture, not universal retrieval quality.
|
|
88
|
+
|
|
89
|
+
Return to the [Ragmir Core README](../../README.md) or try the
|
|
90
|
+
[sovereign RAG demo](../sovereign-rag-demo/README.md) for a broader CLI walkthrough.
|
|
@@ -1,43 +1,88 @@
|
|
|
1
|
-
# Library API
|
|
1
|
+
# Library API demo
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
Core itself. Where [`sovereign-rag-demo`](../sovereign-rag-demo) drives the **CLI**
|
|
5
|
-
(`node ../../dist/cli.js ...`), this demo `import`s the public TypeScript surface the same way an
|
|
6
|
-
external consumer would:
|
|
3
|
+
A runnable smoke test for the public `@jcode.labs/ragmir` TypeScript API.
|
|
7
4
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
5
|
+
Use this example when you are changing Ragmir Core or evaluating it as a library rather than as a
|
|
6
|
+
CLI. It imports the same package name an external consumer uses, runs the full retrieval loop, and
|
|
7
|
+
prints cited results from a fictional local corpus.
|
|
8
|
+
|
|
9
|
+
## What it proves
|
|
10
|
+
|
|
11
|
+
The demo exercises four public operations in order:
|
|
12
|
+
|
|
13
|
+
1. `ingest({ cwd, rebuild: true })` parses, redacts, chunks, embeds, and stores the corpus.
|
|
14
|
+
2. `search(query, { cwd, topK })` returns ranked source passages.
|
|
15
|
+
3. `ask(query, { cwd, topK })` returns retrieval-only cited context without LLM synthesis.
|
|
16
|
+
4. `audit(cwd)` compares supported files with the current index.
|
|
11
17
|
|
|
12
|
-
Node self-
|
|
13
|
-
|
|
14
|
-
|
|
18
|
+
Node's package self-reference resolves `@jcode.labs/ragmir` to this checkout's local
|
|
19
|
+
`packages/ragmir-core/dist` build. It never falls back to the npm-published version, so the result
|
|
20
|
+
reflects the code you are currently reviewing.
|
|
15
21
|
|
|
16
|
-
## Run
|
|
22
|
+
## Run it
|
|
17
23
|
|
|
18
24
|
From the repository root:
|
|
19
25
|
|
|
20
26
|
```bash
|
|
27
|
+
pnpm install --frozen-lockfile
|
|
21
28
|
pnpm example
|
|
22
29
|
```
|
|
23
30
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
31
|
+
The root command builds the published packages required by Core and runs [`run.mjs`](./run.mjs). The
|
|
32
|
+
output is organized into `ingest`, `search`, `ask`, and `audit` sections so a reviewer can see each
|
|
33
|
+
public step succeed.
|
|
27
34
|
|
|
28
|
-
To
|
|
29
|
-
`dist/` is gitignored, so a clean clone has none):
|
|
35
|
+
To rerun only the script after an existing build:
|
|
30
36
|
|
|
31
37
|
```bash
|
|
32
38
|
node packages/ragmir-core/examples/library-api-demo/run.mjs
|
|
33
39
|
```
|
|
34
40
|
|
|
35
|
-
|
|
41
|
+
`dist/` is ignored, so a clean clone must be built at least once.
|
|
42
|
+
|
|
43
|
+
## The public API pattern
|
|
44
|
+
|
|
45
|
+
The essential integration is intentionally small:
|
|
46
|
+
|
|
47
|
+
```js
|
|
48
|
+
import { ask, audit, ingest, search } from "@jcode.labs/ragmir"
|
|
49
|
+
|
|
50
|
+
await ingest({ cwd: projectRoot, rebuild: true })
|
|
51
|
+
|
|
52
|
+
const results = await search("offline retrieval approval", {
|
|
53
|
+
cwd: projectRoot,
|
|
54
|
+
topK: 3,
|
|
55
|
+
})
|
|
56
|
+
|
|
57
|
+
const context = await ask("What evidence supports offline operation?", {
|
|
58
|
+
cwd: projectRoot,
|
|
59
|
+
topK: 3,
|
|
60
|
+
})
|
|
61
|
+
|
|
62
|
+
const report = await audit(projectRoot)
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
All project-relative state is resolved from `cwd`. This is the key boundary for applications that
|
|
66
|
+
use Ragmir across more than one repository.
|
|
67
|
+
|
|
68
|
+
## Data used by the demo
|
|
69
|
+
|
|
70
|
+
The script reuses the sibling [sovereign RAG demo](../sovereign-rag-demo/README.md):
|
|
71
|
+
|
|
72
|
+
- committed synthetic Markdown, CSV, JSONL, YAML, and custom text files;
|
|
73
|
+
- an offline `local-hash` configuration;
|
|
74
|
+
- gitignored `.ragmir/storage` output;
|
|
75
|
+
- no private documents and no model download.
|
|
76
|
+
|
|
77
|
+
Change the queries in `run.mjs` when testing a retrieval behavior, but keep the committed demo
|
|
78
|
+
deterministic and public-safe. Use a separate ignored corpus for private evaluations.
|
|
79
|
+
|
|
80
|
+
## When to use another example
|
|
36
81
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
82
|
+
| Need | Example |
|
|
83
|
+
| --- | --- |
|
|
84
|
+
| Learn the complete CLI workflow | [Sovereign RAG demo](../sovereign-rag-demo/README.md) |
|
|
85
|
+
| Measure exact paths and citations | [Document evidence benchmark](../document-evidence-benchmark/README.md) |
|
|
86
|
+
| Explore every exported function and option | [Core API reference](../../README.md#typescript-api) |
|
|
41
87
|
|
|
42
|
-
|
|
43
|
-
with no model download.
|
|
88
|
+
Return to the [Ragmir Core README](../../README.md) for installation, MCP, OCR, and privacy guidance.
|
|
@@ -1,76 +1,109 @@
|
|
|
1
|
-
# Sovereign RAG
|
|
1
|
+
# Sovereign RAG demo
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
generic, and designed only to exercise local ingestion, retrieval, redaction, custom extensions, and
|
|
5
|
-
security-audit flows.
|
|
3
|
+
A complete, fictional workspace for learning Ragmir's local CLI workflow.
|
|
6
4
|
|
|
7
|
-
|
|
5
|
+
Use this example to ingest several file types, search with citations, inspect privacy posture, find an
|
|
6
|
+
unsupported file, evaluate golden queries, and prepare agent access. It runs with offline
|
|
7
|
+
`local-hash` retrieval and contains no customer or production data.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## Scenario
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
- CSV dataset inventories.
|
|
13
|
-
- JSONL incident timelines.
|
|
14
|
-
- YAML policy metadata.
|
|
15
|
-
- A custom `.evidence` text extension enabled through `.ragmir/config.json`.
|
|
16
|
-
- Unsupported/skipped-file reporting with recommendations through `audit --unsupported`.
|
|
11
|
+
The corpus models an operations team reviewing whether an evidence workflow can stay local:
|
|
17
12
|
|
|
18
|
-
|
|
13
|
+
| File | Role in the scenario |
|
|
14
|
+
| --- | --- |
|
|
15
|
+
| `raw/operations-brief.md` | Approval, ownership, and operating requirements |
|
|
16
|
+
| `raw/dataset-inventory.csv` | Accepted and rejected datasets |
|
|
17
|
+
| `raw/incident-timeline.jsonl` | Time-ordered operational evidence |
|
|
18
|
+
| `raw/security-policy.yaml` | Model-loading and security rules |
|
|
19
|
+
| `raw/review-notes.evidence` | Custom text extension configured for this project |
|
|
20
|
+
| `raw/facility-scan.heic` | Deliberately unsupported file used by the audit example |
|
|
19
21
|
|
|
20
|
-
|
|
22
|
+
## Run the full workflow
|
|
23
|
+
|
|
24
|
+
From the repository root:
|
|
21
25
|
|
|
22
26
|
```bash
|
|
27
|
+
pnpm install --frozen-lockfile
|
|
23
28
|
pnpm build
|
|
24
29
|
```
|
|
25
30
|
|
|
26
|
-
Then run the CLI
|
|
31
|
+
Then enter this example and run the local CLI build:
|
|
27
32
|
|
|
28
33
|
```bash
|
|
29
34
|
cd packages/ragmir-core/examples/sovereign-rag-demo
|
|
30
35
|
node ../../dist/cli.js init
|
|
36
|
+
|
|
31
37
|
# POSIX only: this example deliberately uses a custom tracked rawDir.
|
|
32
38
|
chmod 700 raw
|
|
39
|
+
|
|
33
40
|
node ../../dist/cli.js security-audit
|
|
34
|
-
node ../../dist/cli.js ingest
|
|
41
|
+
node ../../dist/cli.js ingest --rebuild
|
|
35
42
|
node ../../dist/cli.js search "offline retrieval approval"
|
|
36
|
-
node ../../dist/cli.js search "dataset residency"
|
|
37
43
|
node ../../dist/cli.js ask "What evidence supports offline operation?"
|
|
38
|
-
node ../../dist/cli.js
|
|
44
|
+
node ../../dist/cli.js research "sovereign deployment evidence" --compact
|
|
39
45
|
node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 1
|
|
40
|
-
node ../../dist/cli.js audit
|
|
41
46
|
node ../../dist/cli.js audit --unsupported
|
|
42
47
|
node ../../dist/cli.js status
|
|
43
48
|
```
|
|
44
49
|
|
|
45
|
-
`init` is idempotent
|
|
46
|
-
|
|
47
|
-
explicitly on POSIX before a strict audit. Git and package archives cannot encode its `0700` mode.
|
|
50
|
+
`init` is idempotent. Git and npm archives cannot preserve the example's intended `0700` raw-folder
|
|
51
|
+
mode, so the explicit `chmod` keeps the raw corpus owner-only on POSIX systems.
|
|
48
52
|
|
|
49
|
-
|
|
50
|
-
|
|
53
|
+
The security audit still reports that `.ragmir/` is not fully ignored. That warning is expected only
|
|
54
|
+
for this public fixture because it deliberately commits `.ragmir/config.json` and
|
|
55
|
+
`.ragmir/sources.txt`. Generated storage, models, reports, audio, logs, and salts remain ignored.
|
|
56
|
+
Normal private projects should ignore the complete `.ragmir/` directory.
|
|
51
57
|
|
|
52
|
-
##
|
|
58
|
+
## What to look for
|
|
53
59
|
|
|
54
|
-
-
|
|
55
|
-
- `
|
|
56
|
-
- `
|
|
57
|
-
-
|
|
58
|
-
-
|
|
60
|
+
- Search and ask output identify the source path and chunk.
|
|
61
|
+
- `ask` returns cited passages, not a generated answer from a hosted LLM.
|
|
62
|
+
- `research` combines several retrieval queries and reports source diagnostics.
|
|
63
|
+
- The audit surfaces the fixture-specific tracked-config warning described above.
|
|
64
|
+
- The golden evaluation finds the expected sources for all four questions.
|
|
65
|
+
- `audit --unsupported` explains why the HEIC fixture was skipped and recommends local OCR.
|
|
66
|
+
- `status` confirms the active provider and local index state.
|
|
67
|
+
|
|
68
|
+
## Useful queries
|
|
69
|
+
|
|
70
|
+
```text
|
|
71
|
+
offline retrieval approval
|
|
72
|
+
dataset residency
|
|
73
|
+
incident containment evidence
|
|
74
|
+
who owns the usage review
|
|
75
|
+
what documents support sovereign deployment
|
|
76
|
+
```
|
|
59
77
|
|
|
60
|
-
|
|
78
|
+
Try a precise query first, then compare it with `research --compact` when the question spans several
|
|
79
|
+
files.
|
|
61
80
|
|
|
62
|
-
|
|
63
|
-
`.ragmir/mcp.json`. For this synthetic demo, ask an agent:
|
|
81
|
+
## Give the demo to an agent
|
|
64
82
|
|
|
65
|
-
|
|
83
|
+
In a real project, `rgr setup` generates an MCP helper under `.ragmir/`. After setup, an
|
|
84
|
+
MCP-compatible agent can follow a prompt like:
|
|
85
|
+
|
|
86
|
+
```text
|
|
66
87
|
Use Ragmir to inspect the local knowledge base, search for "offline retrieval approval", and write a
|
|
67
88
|
cited Markdown report under .ragmir/reports/demo-sovereign-rag.md. Mention any unsupported or stale
|
|
68
89
|
files reported by the audit.
|
|
69
90
|
```
|
|
70
91
|
|
|
71
|
-
|
|
92
|
+
The report path stays under ignored local state. Do not commit generated reports that may contain
|
|
93
|
+
retrieved project evidence.
|
|
94
|
+
|
|
95
|
+
## Compare semantic retrieval
|
|
96
|
+
|
|
97
|
+
The committed configuration uses:
|
|
98
|
+
|
|
99
|
+
```json
|
|
100
|
+
{
|
|
101
|
+
"embeddingProvider": "local-hash"
|
|
102
|
+
}
|
|
103
|
+
```
|
|
72
104
|
|
|
73
|
-
|
|
105
|
+
This is offline lexical/hash retrieval, not model-semantic search. To run a deliberate semantic
|
|
106
|
+
comparison, configure a Transformers.js model, preload it under `.ragmir/models`, and rebuild:
|
|
74
107
|
|
|
75
108
|
```json
|
|
76
109
|
{
|
|
@@ -82,22 +115,33 @@ To compare no-model retrieval with semantic local retrieval, change `.ragmir/con
|
|
|
82
115
|
}
|
|
83
116
|
```
|
|
84
117
|
|
|
85
|
-
Preload the model files under `.ragmir/models` for offline work, then rebuild the index:
|
|
86
|
-
|
|
87
118
|
```bash
|
|
88
|
-
node ../../dist/cli.js ingest
|
|
89
|
-
node ../../dist/cli.js ask "What documents support sovereign deployment?"
|
|
119
|
+
node ../../dist/cli.js ingest --rebuild
|
|
90
120
|
node ../../dist/cli.js evaluate --golden golden-queries.json --json
|
|
91
|
-
node ../../dist/cli.js evaluate --golden golden-queries.json --fail-under 0.8 --json
|
|
92
121
|
```
|
|
93
122
|
|
|
94
|
-
|
|
123
|
+
Keep remote model loading disabled for an offline comparison. A lower semantic score may indicate
|
|
124
|
+
that queries or thresholds need tuning, not that a gate should be weakened.
|
|
125
|
+
|
|
126
|
+
## Generated state and cleanup
|
|
95
127
|
|
|
96
|
-
|
|
128
|
+
All runtime state stays under the ignored directory:
|
|
97
129
|
|
|
98
|
-
```
|
|
130
|
+
```text
|
|
99
131
|
.ragmir/
|
|
100
132
|
```
|
|
101
133
|
|
|
102
|
-
|
|
103
|
-
|
|
134
|
+
Remove only the generated index when you want a clean rerun:
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
node ../../dist/cli.js destroy-index --yes
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Never replace the fictional fixtures in this package with real confidential documents. Use a
|
|
141
|
+
separate ignored workspace for private dogfooding.
|
|
142
|
+
|
|
143
|
+
## Continue exploring
|
|
144
|
+
|
|
145
|
+
- [Library API demo](../library-api-demo/README.md): call the public TypeScript API.
|
|
146
|
+
- [Document evidence benchmark](../document-evidence-benchmark/README.md): require exact paths and citations.
|
|
147
|
+
- [Ragmir Core README](../../README.md): installation, MCP, OCR, and privacy reference.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@jcode.labs/ragmir",
|
|
3
|
-
"version": "2.9.
|
|
3
|
+
"version": "2.9.3",
|
|
4
4
|
"description": "Ragmir Core: open-source sovereign local RAG for confidential datasets and AI agents.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -69,8 +69,8 @@
|
|
|
69
69
|
"unpdf": "^1.4.0",
|
|
70
70
|
"yaml": "^2.9.0",
|
|
71
71
|
"zod": "^4.1.13",
|
|
72
|
-
"@jcode.labs/ragmir-chat": "2.9.
|
|
73
|
-
"@jcode.labs/ragmir-tts": "2.9.
|
|
72
|
+
"@jcode.labs/ragmir-chat": "2.9.3",
|
|
73
|
+
"@jcode.labs/ragmir-tts": "2.9.3"
|
|
74
74
|
},
|
|
75
75
|
"devDependencies": {
|
|
76
76
|
"@types/html-to-text": "^9.0.4",
|