@jcode.labs/ragmir 2.18.0 → 2.19.0

package/README.md

@@ -5,21 +5,18 @@
 [![Node.js](https://img.shields.io/node/v/@jcode.labs/ragmir)](https://www.npmjs.com/package/@jcode.labs/ragmir)
 [![MIT](https://img.shields.io/npm/l/@jcode.labs/ragmir)](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE)
 
-Confidential local RAG for coding agents and Node.js applications.
-
-*Stop sending confidential documents directly to the cloud.*
-
-Core indexes the project files you select and retrieves bounded, cited evidence offline by default.
-It does not upload the corpus, call an LLM, or open an HTTP port.
+Confidential local RAG for coding agents and Node.js applications. Core indexes the project files
+you choose and retrieves bounded, cited evidence offline by default. It uploads no corpus, calls no
+LLM, and opens no HTTP port.
 
 [Project overview](https://github.com/jcode-works/jcode-ragmir#readme) ·
-[CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md) ·
-[API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md) ·
+[CLI](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md) ·
+[API](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md) ·
 [Agent integration](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/agent-integration.md)
 
 ## Install and retrieve
 
-Requires Node.js 22 or later. Releases are gated on Linux x64 and macOS ARM64 with Node.js 22.
+Requires Node.js 22 or later.
 
 ```bash
 npm install --save-dev @jcode.labs/ragmir
@@ -29,55 +26,22 @@ npx rgr ingest
 npx rgr search "Which decision changed the rollout?"
 ```
 
-Setup keeps configuration, the index, helpers, reports, and metadata-only access logs under ignored
-`.ragmir/` state. Ingestion is incremental, resumable, and serialized across local writer
-processes. Source, chunk, vector, concurrency, and batch windows are bounded, with durable progress
-committed per file. Completed ingestion refreshes incomplete full-text coverage and runs bounded
-LanceDB maintenance when mutation or fragment thresholds are reached. Sidecar replacement flushes
-before rename, synchronizes the storage directory where supported, and retains one validated
-activation manifest for explicit recovery diagnostics. Inspect maintenance with
-`npx rgr storage optimize --dry-run --json`. Exact vector search is retained below 100,000 rows;
-larger tables use a benchmarked IVF-PQ policy only with complete coverage. Use
-`npx rgr search "query" --exact-vector-search` to bypass ANN for diagnostics. Rebuild generations
-use private reader leases and
-bounded retention; inspect them with `npx rgr storage generations --json` before running
-`npx rgr storage gc --dry-run --json`. Failed changed files keep explicitly stale
-last-known-good rows by default; repair or source deletion reconciles them deterministically.
-Returned citations expose only verifiable coordinates: source lines for line-preserving text, PDF
-pages, PPTX slides, XLSX sheets and cells, and EPUB spine positions.
-Run `npx rgr audit --unsupported` to compare the selected files with the index and see what was
-skipped.
-Custom redaction expressions are screened for catastrophic backtracking. `npx rgr security-audit`
-also reports permissions, ignored/tracked state for every private path, and local extractor
-authority.
-`npx rgr status` and normal `npx rgr doctor` consume compact manifest health without opening chunk
-storage. Use `npx rgr doctor --deep` or `npx rgr audit` for explicit O(corpus) live diagnostics.
+Generated configuration, indexes, helpers, reports, and metadata-only logs stay under ignored
+`.ragmir/` state. Ingestion is incremental, resumable, bounded by source, chunk, vector, file, batch,
+and concurrency windows, and serialized across local writer processes.
 
 ## Choose an interface
 
-| Interface | Use |
+| Interface | Use it for |
 | --- | --- |
-| `rgr` CLI | Setup, ingest, search, audit, and JSON automation |
-| TypeScript API | Typed retrieval in a script or stateful Node.js worker |
-| Local stdio MCP | Bounded context for compatible coding agents |
-
-The default `local-hash` provider needs no model download. It is lexical/hash retrieval, not
-semantic embeddings. Enable local Transformers.js embeddings explicitly with `rgr setup --semantic`
-and rebuild the index. Bundled profiles use pinned model commits, and setup stores the resolved
-artifact digest so persisted compatibility identifies the exact weights. `local-hash` does not
-resolve Transformers.js, ONNX Runtime, or Sharp.
-
-Hybrid retrieval has stable source-and-chunk tie-breaks, two-pass recall-safe diversification, and
-provider-aware abstention. Pass `explain: true` to inspect vector and lexical contributions, FTS or
-fallback activation and reason, candidate and coverage budgets, plus the active ranking-policy
-fingerprint. Search, embedding, and ingestion use independent bounded queues per project root;
-overload and queue deadlines return stable retryable errors, and explained searches include queue
-wait.
-An empty result means every candidate failed the evidence threshold.
-
-`research` uses language-aware query expansion with deterministic cross-query ranking. It reads
-manifest health by default and exposes explicit timeout, code-file, code-byte, concurrency, and
-result budgets. Request `fullAudit: true` only when the report also needs a fresh source inventory.
+| `rgr` CLI | Setup, ingest, search, audit, maintenance, and JSON automation |
+| TypeScript API | Typed retrieval in scripts and long-running Node.js workers |
+| Local stdio MCP | Bounded, read-focused context for compatible agents |
+
+The default `local-hash` provider works offline with no model download. Enable semantic
+Transformers.js embeddings explicitly with `rgr setup --semantic`, then rebuild. Core remains
+retrieval-only in both modes. Use `explain: true` or `--explain` to inspect lexical and vector
+contributions, fallback decisions, budgets, queue wait, and the active ranking policy.
 
 ## TypeScript API
 
@@ -92,50 +56,54 @@ try {
     timeoutMs: 10_000,
   })
 
-  for (const result of results) {
-    console.log(result.citation, result.text)
-  }
+  for (const result of results) console.log(result.citation, result.text)
 } finally {
   await ragmir.close()
 }
 ```
 
-Reuse one client per project root in long-running processes. It caches one immutable read snapshot
-until atomic generation replacement, closes retired table handles after their last active reader,
-and `close()` flushes metadata-only access logs before releasing model ownership. The final owner
-disposes its Transformers pipeline only after active inference finishes. Top-level
-`ingest`, `search`, `ask`, and `research` functions remain available for one-shot scripts. `ask`
-returns cited retrieval context, not a generated answer.
-
-Core also exports setup, preview, audit, evaluation, source-routing, privacy, OCR, MCP, and agent
-integration helpers. See the [complete API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md)
-for public options and result types.
-
-Optional PDF OCR processes only blank pages. The generated setup batches bounded page groups and
-stores private content-addressed page results, so interrupted scans resume only missing pages and
-warm output remains identical without launching OCR subprocesses.
-
-## Optional packages and boundaries
-
-- [`@jcode.labs/ragmir-chat`](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) adds local GGUF
-  answer generation from retrieved passages.
+Reuse one client per project root in a long-running process. It owns the local connection, read
+snapshot, active operation lifecycle, metadata-only log flush, and optional embedding-model lease.
+Top-level `ingest`, `search`, `ask`, and `research` functions remain available for one-shot scripts.
+`ask` returns cited context, not generated prose.
+
+## Guarantees and boundaries
+
+- Citations use source lines only for line-preserving text, plus PDF pages, PPTX slides, XLSX
+  sheets and cells, and EPUB spine positions.
+- Rebuilds activate only after row and manifest validation. Interrupted rebuilds leave the previous
+  searchable generation active.
+- Exact vector search remains the policy below 100,000 rows. Larger tables use quality-gated IVF-PQ
+  with complete coverage and an exact diagnostic mode.
+- Search, embedding, and ingestion use independent bounded queues. Overload and queue deadlines are
+  stable retryable errors.
+- Optional OCR processes only blank PDF pages through a configured local executable and private
+  resumable cache.
+- `rgr status` and normal `rgr doctor` read compact manifest health. Use `rgr doctor --deep` or
+  `rgr audit` for a live source inventory.
+- `rgr security-audit` checks permissions, Git ignore coverage, tracked private paths, redaction,
+  and local extractor authority.
+- `rgr team snapshot` and `rgr team compare` explain configuration and per-file drift without
+  sharing source text or guessing which copy is authoritative.
+- After a package update, `rgr upgrade --check` previews compatibility; `rgr upgrade` safely stages
+  any required rebuild without deleting the active index first.
+
+The [CLI reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/cli-reference.md),
+[API reference](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/api-reference.md), and
+[configuration guide](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md)
+hold the complete options and operational detail.
+
+## Optional packages
+
+- [`@jcode.labs/ragmir-chat`](https://www.npmjs.com/package/@jcode.labs/ragmir-chat) adds cited
+  answer generation with a verified local GGUF profile.
 - [`@jcode.labs/ragmir-tts`](https://www.npmjs.com/package/@jcode.labs/ragmir-tts) renders reviewed
   text as local audio or explicit online speech.
 
-Core installs and starts without either add-on. A hosted agent receives only the passages your
-integration gives it, under that provider's data policy. Use a local consumer when no passage may
-leave the workstation. Ragmir has no hosted storage or cloud-sync layer; teams synchronize source
-files separately and build one local index per developer. Keep shared directory or glob contracts
-stable, align the Ragmir version, configuration, embedding provider, and model, then compare
-`corpusFingerprint` from `rgr status --json` after both indexes are ready with no missing or stale
-files. Matching values identify the same indexed relative paths and source bytes. Use
-`sourceFingerprintMode: "strict"` when a synchronization tool can preserve metadata while replacing
-content, and never share an actively written `.ragmir/storage/` directory. Older manifests expose a
-`null` fingerprint until the next successful ingestion.
-
-Read the [configuration guide](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/configuration.md),
-[security hardening guide](https://github.com/jcode-works/jcode-ragmir/blob/main/SECURITY-HARDENING.md),
-and [troubleshooting guide](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/troubleshooting.md)
-before indexing sensitive material.
+Core installs and starts without either add-on. A hosted agent receives only passages your
+integration sends under that provider's data policy; use a local consumer when no passage may leave
+the workstation. Teams synchronize their source folder and tracked configuration, then build one
+local index per developer. Exchange an authorized metadata-only snapshot and run `rgr team compare`
+to resolve exact drift. See the [team workflow](https://github.com/jcode-works/jcode-ragmir/blob/main/docs/agent-integration.md#team-knowledge-bases).
 
 Ragmir Core is open source under the [MIT License](https://github.com/jcode-works/jcode-ragmir/blob/main/LICENSE).

package/dist/cli.js

@@ -33,12 +33,15 @@ import { bundledSkillPath, installAgentSkills, installSkill, parseAgentTargets,
 import { addSourceEntries, listSourceEntries } from "./sources.js";
 import { optimizeStorage } from "./storage-maintenance.js";
 import { readIndexManifestHeader } from "./store.js";
+import { compareTeamSnapshots, createTeamSnapshot, readTeamSnapshot, writeTeamSnapshot, } from "./team-diagnostics.js";
+import { inspectUpgrade, upgradeProject } from "./upgrade.js";
 import { VERSION } from "./version.js";
 const SEARCH_TEXT_PREVIEW_LENGTH = 900;
 const CHAT_PACKAGE_NAME = "@jcode.labs/ragmir-chat";
 const TTS_PACKAGE_NAME = "@jcode.labs/ragmir-tts";
 const DEPRECATED_CLI_NAMES = new Set(["ragmir", "kb"]);
 const PUBLIC_CLI_NAME = "rgr";
+const TEAM_DIFF_PREVIEW_LIMIT = 20;
 const program = new Command();
 const deprecatedCliName = deprecatedCliInvocation();
 if (deprecatedCliName !== null) {
@@ -192,6 +195,53 @@ program
     }
     printDoctor(report);
 });
+program
+    .command("upgrade")
+    .description("Check upgrade compatibility, refresh agent helpers, and safely rebuild an incompatible index.")
+    .option("--check", "Inspect the upgrade without changing local state.")
+    .option("--force-agent-skills", "Replace same-name native skills after reviewing that they can be overwritten.")
+    .option("--json", "Print machine-readable JSON.")
+    .action(async (options, command) => {
+    const cwd = projectRoot(command);
+    if (options.check) {
+        const inspection = await inspectUpgrade(cwd);
+        if (options.json) {
+            console.log(JSON.stringify(inspection, null, 2));
+            return;
+        }
+        console.log(`status=${inspection.status}`);
+        console.log(`runtimeRagmirVersion=${inspection.runtimeRagmirVersion}`);
+        console.log(`indexedWithRagmirVersion=${inspection.indexedWithRagmirVersion ?? "unavailable"}`);
+        console.log(`ready=${inspection.ready}`);
+        console.log(`safeActivation=${inspection.safeActivation}`);
+        if (inspection.reason) {
+            console.log(`reason=${inspection.reason}`);
+        }
+        return;
+    }
+    const result = await upgradeProject({
+        cwd,
+        forceAgentSkills: options.forceAgentSkills === true,
+    });
+    if (options.json) {
+        console.log(JSON.stringify(result, null, 2));
+        return;
+    }
+    console.log(`status=${result.status}`);
+    console.log(`runtimeRagmirVersion=${result.runtimeRagmirVersion}`);
+    console.log(`indexedWithRagmirVersion=${result.indexedWithRagmirVersion ?? "unavailable"}`);
+    console.log(`ready=${result.ready}`);
+    console.log(`safeActivation=${result.safeActivation}`);
+    console.log(`action=${result.action}`);
+    console.log(`previousIndexedWithRagmirVersion=${result.previousIndexedWithRagmirVersion ?? "unavailable"}`);
+    console.log(`previousIndexKeptUntilActivation=${result.previousIndexKeptUntilActivation}`);
+    if (result.reason) {
+        console.log(`reason=${result.reason}`);
+    }
+    if (!result.ready) {
+        process.exitCode = 1;
+    }
+});
 program
     .command("setup")
     .description("Initialize Ragmir, expose native agent skills, generate MCP helpers, and ingest when safe.")
@@ -740,6 +790,55 @@ program
         console.log(`${marker} ${base.id} format=${format} root=${base.projectRoot}`);
     }
 });
+const teamCommand = program
+    .command("team")
+    .description("Export and compare privacy-bounded team index snapshots.");
+teamCommand
+    .command("snapshot")
+    .description("Export relative paths, checksums, configuration, readiness, and corpus identity.")
+    .requiredOption("--output <path>", "Snapshot file to create or replace.")
+    .option("--label <label>", "Human-readable environment label.", "local")
+    .option("--json", "Print machine-readable JSON.")
+    .action(async (options, command) => {
+    const cwd = projectRoot(command);
+    const snapshot = await createTeamSnapshot({ cwd, label: options.label });
+    const outputPath = await writeTeamSnapshot(snapshot, path.resolve(cwd, options.output));
+    if (options.json) {
+        console.log(JSON.stringify({ outputPath, snapshot }, null, 2));
+        return;
+    }
+    console.log(pc.green("Team snapshot ready."));
+    console.log(`outputPath=${outputPath}`);
+    console.log(`label=${snapshot.label}`);
+    console.log(`ready=${snapshot.ready}`);
+    console.log(`corpusFingerprint=${snapshot.corpus.fingerprint ?? "unavailable"}`);
+    console.log(`indexedFiles=${snapshot.corpus.indexedFiles}`);
+    console.log(`notice=${snapshot.notice}`);
+});
+teamCommand
+    .command("compare")
+    .description("Compare the current local index with a peer snapshot and print actionable drift.")
+    .argument("<snapshot>", "Peer snapshot created by `rgr team snapshot`.")
+    .option("--local-label <label>", "Human-readable label for this environment.", "local")
+    .option("--json", "Print machine-readable JSON.")
+    .option("--strict", "Exit with code 1 unless both indexes are synchronized.")
+    .action(async (snapshotPath, options, command) => {
+    const cwd = projectRoot(command);
+    const [local, peer] = await Promise.all([
+        createTeamSnapshot({ cwd, label: options.localLabel }),
+        readTeamSnapshot(path.resolve(cwd, snapshotPath)),
+    ]);
+    const comparison = compareTeamSnapshots(local, peer);
+    if (options.json) {
+        console.log(JSON.stringify(comparison, null, 2));
+    }
+    else {
+        printTeamComparison(comparison);
+    }
+    if (options.strict && !comparison.synchronized) {
+        process.exitCode = 1;
+    }
+});
 program
     .command("status")
     .description("Show active configuration, readiness, corpus fingerprint, and index counts.")
@@ -1484,6 +1583,60 @@ function researchEvidencePreview(evidence) {
     }
     return evidence.text.replace(/\s+/gu, " ").trim().slice(0, SEARCH_TEXT_PREVIEW_LENGTH);
 }
+function printTeamComparison(comparison) {
+    const statusColor = comparison.synchronized ? pc.green : pc.yellow;
+    console.log(statusColor(`Team status: ${comparison.status}`));
+    console.log(`summary=${comparison.summary}`);
+    console.log(`local=${comparison.localLabel}`);
+    console.log(`peer=${comparison.peerLabel}`);
+    console.log(`sameConfiguration=${comparison.sameConfiguration}`);
+    console.log(`sameCorpus=${comparison.sameCorpus}`);
+    console.log(`authorityDecisionRequired=${comparison.authorityDecisionRequired}`);
+    if (comparison.configurationDifferences.length > 0) {
+        console.log("");
+        console.log(pc.cyan("Configuration differences:"));
+        for (const difference of comparison.configurationDifferences) {
+            console.log(`  - ${difference.field} (${difference.scope}, rebuild=${difference.requiresRebuild}): local=${formatTeamValue(difference.local)} peer=${formatTeamValue(difference.peer)}`);
+        }
+    }
+    printTeamPaths("Local-only files", comparison.files.localOnly);
+    printTeamPaths("Peer-only files", comparison.files.peerOnly);
+    if (comparison.files.changed.length > 0) {
+        console.log("");
+        console.log(pc.cyan(`Changed files (${comparison.files.changed.length}):`));
+        for (const file of comparison.files.changed.slice(0, TEAM_DIFF_PREVIEW_LIMIT)) {
+            console.log(`  - ${file.relativePath} local=${file.localChecksum.slice(0, 12)} peer=${file.peerChecksum.slice(0, 12)}`);
+        }
+        printTeamOmitted(comparison.files.changed.length);
+    }
+    if (comparison.recommendedActions.length > 0) {
+        console.log("");
+        console.log(pc.cyan("Recommended actions:"));
+        for (const [index, action] of comparison.recommendedActions.entries()) {
+            console.log(`  ${index + 1}. ${action}`);
+        }
+    }
+}
+function printTeamPaths(title, files) {
+    if (files.length === 0) {
+        return;
+    }
+    console.log("");
+    console.log(pc.cyan(`${title} (${files.length}):`));
+    for (const file of files.slice(0, TEAM_DIFF_PREVIEW_LIMIT)) {
+        console.log(`  - ${file}`);
+    }
+    printTeamOmitted(files.length);
+}
+function printTeamOmitted(total) {
+    const omitted = total - TEAM_DIFF_PREVIEW_LIMIT;
+    if (omitted > 0) {
+        console.log(`  ... ${omitted} more; use --json for the complete diff.`);
+    }
+}
+function formatTeamValue(value) {
+    return Array.isArray(value) ? JSON.stringify(value) : String(value);
+}
 function printPdfOcrStatus(status) {
     console.log(`privacyProfile=${status.privacyProfile}`);
     console.log(`configured=${status.configured}`);