llmnav 0.5.1

package/docs/benchmarking.md

@@ -0,0 +1,164 @@
+# Benchmarking navigation impact
+
+## Two benchmark layers
+
+LLMNav measures two different things.
+
+The repository retrieval layer asks whether a task query reaches the correct semantic card quickly and without excessive memory.
+
+The end-to-end agent layer asks whether the coding agent opens less irrelevant source, uses fewer uncached tokens, and still completes the task correctly.
+
+Do not substitute one layer for the other.
+
+## Built-in v0.2 regression suite
+
+Run:
+
+```sh
+npm test
+```
+
+The large synthetic search test creates 5,000 indexed cards and 50 deterministic queries. It compares the v0.2 prepared inverted-index path with the v0.1-compatible legacy path.
+
+It requires:
+
+* expected first result for every query
+* identical ranked-result checksum
+* lower elapsed time for the isolated v0.2 query loop
+* bounded generated search-index bytes
+* bounded RSS and heap use
+
+The incremental source fixture requires a no-op run to parse zero files, a one-file edit to parse one file, and a one-card semantic edit to retokenize one card. It then forces a full rebuild and compares deterministic output bytes.
+
+These tests are regression gates. They do not claim a universal speedup on every repository or machine.
+
+## Reproducible v0.2 measurement
+
+Run:
+
+```sh
+npm run benchmark:v0.2
+```
+
+The command creates a temporary synthetic repository with 1,000 source files and 5,000 cards by default. It records:
+
+* cold initial generation
+* one-file incremental regeneration
+* the same change through forced full regeneration
+* no-op incremental regeneration
+* byte equality between incremental and full cache trees
+* fresh-process v0.1-compatible query timing
+* fresh-process v0.2 inverted-index query timing
+* median and p95 query latency
+* RSS, heap, index bytes, and search-index bytes
+* exact result checksum equality
+
+The benchmark writes raw JSON under `benchmarks/results/` and regenerates `docs/performance-v0.2.md`.
+
+Environment variables can reduce or expand the fixture during development:
+
+```sh
+LLMNAV_BENCH_FILES=500 LLMNAV_BENCH_QUERY_RUNS=5 npm run benchmark:v0.2
+```
+
+Published repository results use the defaults. The report states that each query starts a fresh Node.js process and that the operating-system filesystem cache is not flushed. That distinction prevents a warm page cache from being mislabeled as cold disk I/O.
+
+## Measurement integrity
+
+A benchmark result is accepted only after these checks pass:
+
+```text
+legacy expected result count equals query count
+inverted expected result count equals query count
+legacy and inverted result checksums match
+incremental and full generated cache trees are byte-identical
+reported fixture cardinality matches generated files and cards
+```
+
+Wall-clock speed is reported exactly as measured. It is not used to invent a speedup estimate for other hardware.
+
+## Repository search evaluation
+
+Add real task descriptions to `.llmnav/eval/queries.jsonl`.
+
+```jsonl
+{"query":"duplicate webhook applies payment twice","expected":["billing.payment.apply-provider-event"]}
+{"query":"same collapse seed produces a different arena","expected":["game.arena.collapse-sequence"]}
+```
+
+Run:
+
+```sh
+npx llmnav eval --json
+```
+
+Track at least Recall@1, Recall@5, and mean reciprocal rank. Record every acceptable target ID when a task has multiple valid entry points.
+
+## End-to-end agent metrics
+
+A successful deployment should reduce exploration cost without reducing task correctness.
+
+| Metric | Meaning |
+| --- | --- |
+| first correct source time | elapsed time until the agent opens the right declaration |
+| files opened | breadth of repository exploration |
+| search tool calls | grep, glob, tree, language-server, and LLMNav calls |
+| source lines read | code inserted into model context |
+| uncached input tokens | new model input processed for the task |
+| token cache ratio | cached input divided by total input |
+| task success | tests and acceptance criteria pass |
+| wrong-edit rate | edits made in an irrelevant boundary |
+
+A request-level cache hit is not enough. Track cached tokens and uncached tokens separately.
+
+## Experimental groups
+
+Compare three configurations.
+
+| Group | Configuration |
+| --- | --- |
+| `baseline` | current agent instructions and search tools |
+| `comments-only` | source cards without generated query or catalog tools |
+| `full` | cards, generated catalogs, inverted query, bounded context, and stable prompt placement |
+
+The comments-only group exposes designs that merely add prompt tokens without improving navigation.
+
+## Task set and repetition
+
+Use 30 to 50 historical tasks when possible. Mix bug fixes, feature additions, API changes, refactors, performance work, test failures, migrations, and cross-module workflows.
+
+Agent execution is variable. Repeat tasks with the same model, reasoning configuration, tools, repository state, and timeout policy. Report medians and failure ranges rather than one favorable run.
+
+## Initial deployment gates
+
+These are targets to validate per repository, not measured package guarantees.
+
+| Metric | Initial target |
+| --- | ---: |
+| Recall@1 | at least 0.75 |
+| Recall@5 | at least 0.90 |
+| median first correct symbol calls | at most 2 navigation calls |
+| median files opened | at most 4 |
+| uncached input tokens | at least 40% below baseline |
+| first correct symbol time | at least 35% below baseline |
+| task success | no decrease from baseline |
+| stale semantic metadata failures | zero |
+
+## Failure analysis
+
+Classify every miss before changing ranking weights.
+
+```text
+missing card
+weak or generic role
+missing task-language alias
+overloaded search phrase
+wrong module boundary
+missing semantic relation
+declaration attachment failure
+generated cache stale
+lexical ranker limitation
+task legitimately requires broader structural analysis
+```
+
+Only measured lexical failures justify adding a more complex retrieval layer.

package/docs/ci.md

@@ -0,0 +1,196 @@
+# CI and enforcement
+
+## Consumer repository gates
+
+A practical LLMNav gate has four stages.
+
+```sh
+npx llmnav format --check
+npx llmnav check --format github
+npx llmnav generate --full --check
+npx llmnav eval
+```
+
+`format --check` rejects non-canonical source-card serialization.
+
+`check` rejects invalid meaning, unresolved or retired semantic relations, malformed registry state, missing configured coverage, path escape, and declaration attachment failures.
+
+Use `npx llmnav check --format sarif > llmnav.sarif` when a code-scanning system accepts SARIF 2.1.0 rather than GitHub workflow annotations. SARIF output is deterministic for the same ordered diagnostics and uses repository-relative paths.
+
+`generate --full --check` performs a read-only reconstruction from canonical source and rejects stale or poisoned primary indexes, inverted indexes, file state, graph state, manifests, and catalogs. Use plain incremental generation during local iteration; release and CI gates should not trust disposable accelerators.
+
+`eval` catches ranking regressions that remain syntactically valid.
+
+## Machine-readable impact gate
+
+CI or an agent orchestrator can inspect one generation result without parsing terminal text.
+
+```sh
+npx llmnav generate --json > llmnav-generation.json
+```
+
+Useful fields are:
+
+```text
+changedFiles
+changedCards[].id
+changedCards[].change
+changedCards[].dimensions
+affectedCatalogs[].kind
+affectedCatalogs[].id
+incremental.files.parsedFiles
+incremental.cards.indexedCards
+incremental.graph.rebuiltPartitions
+transaction.recovered
+transaction.recoveryAction
+```
+
+Array order is deterministic. A policy can require review when `semantic` changes occur in `auth`, `money`, or `privacy` cards while allowing body-only changes to proceed normally.
+
+## Included GitHub Actions workflow
+
+This repository's `.github/workflows/ci.yml` executes:
+
+| Operating system | Node.js |
+| --- | --- |
+| Ubuntu | 22, 24 |
+| Windows | 22, 24 |
+
+Every matrix entry runs linting, all tests, source-card validation, and generated-cache verification. Node.js 22 entries also install the actual npm tarball into a clean project and run initialization, generation, query, validation, verification, and doctor commands.
+
+The Windows matrix is required because directory rename, file locking, process interruption, and executable shims differ materially from Linux.
+
+A smaller consumer workflow can use:
+
+```yaml
+name: LLMNav
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  llmnav:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-node@v7
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - run: npx llmnav format --check
+      - run: npx llmnav check --format github
+      - run: npx llmnav generate --full --check
+      - run: npx llmnav eval
+```
+
+Repositories that depend on Windows development should add a `windows-latest` job rather than assuming cache replacement behaves identically.
+
+## Transaction failure regression
+
+The repository test suite injects failures at several transaction phases:
+
+```text
+after writing a staged artifact, before a journal exists
+after moving the previous cache to backup
+after installing a new cache, before commit
+```
+
+The assertions compare the restored `index.json`, ID registry, and stable order with their previous bytes, prove concurrent writers serialize, and prove a query waits instead of rolling back an active writer. Transient Windows error codes `EACCES`, `EBUSY`, `EEXIST`, `ENOTEMPTY`, and `EPERM` are injected into rename operations to verify bounded retries.
+
+These tests are not a substitute for the Windows CI runner. They make exact phases reproducible on every platform, while the matrix executes the same filesystem workflow on Windows.
+
+## Performance regression gates
+
+`tests/performance.test.js` uses a 5,000-card synthetic index and compares the v0.2 prepared search path with the v0.1-compatible legacy path.
+
+The test requires:
+
+* every query to return the expected first card
+* complete result checksum equality between search paths
+* v0.2 elapsed time below the legacy elapsed time in the isolated query loop
+* generated search index below the configured size limit
+* RSS and heap use below configured limits
+
+`tests/incremental.test.js` uses a multi-file source fixture and requires:
+
+* no-op generation to parse zero files when stat hints match
+* a one-file edit to parse exactly one file
+* one-card semantic change to retokenize exactly one card
+* full and incremental output to be byte-identical
+* graph and graph-state output to be byte-identical after partition reuse
+
+The standalone `npm run benchmark:v0.2` command records timings but is not a hard CI gate because shared runner variance is too high. Its raw JSON and generated Markdown report preserve the exact environment and fixture.
+
+## npm package gate
+
+```sh
+npm run smoke:pack
+```
+
+The smoke test creates the actual npm tarball, installs it in a clean temporary project, confirms zero runtime dependencies, and executes the installed `bin/llmnav.js` rather than importing source from the working tree.
+
+This catches missing files in the package allowlist, broken executable paths, stale type or schema payloads, and package-only initialization failures.
+
+## Coverage rules
+
+LLMNav does not force every source file to contain a card. Repositories define boundaries that must be covered.
+
+```json
+{
+  "coverageRules": [
+    {
+      "name": "HTTP entry points",
+      "match": ["src/routes/**/*.ts", "src/api/**/*.ts"],
+      "scope": "file",
+      "requiredFields": ["effect", "stability"]
+    },
+    {
+      "name": "database migrations",
+      "match": ["migrations/**/*.sql"],
+      "scope": "file",
+      "requiredFields": ["invariant", "risk"]
+    }
+  ]
+}
+```
+
+Do not add a broad `src/**/*.ts` rule. It converts a selective navigation protocol into mandatory comment noise.
+
+## Diagnostic policy
+
+Errors block generation. Warnings do not block `check` by default.
+
+| Code | Meaning |
+| --- | --- |
+| `LNV001` | required card or field missing |
+| `LNV002` | duplicate, invalid, retired, or conflicting ID |
+| `LNV003` | volatile data stored in a source card |
+| `LNV004` | generated structural relation maintained by hand |
+| `LNV005` | invalid key or controlled value |
+| `LNV006` | weak or oversized role |
+| `LNV007` | search phrase quality, count, or saturation failure |
+| `LNV008` | unresolved semantic relation |
+| `LNV009` | exported API or effective configuration fingerprint drift |
+| `LNV010` | non-canonical or stale generated representation |
+| `LNV011` | parser or declaration attachment failure |
+| `LNV012` | deleted or renamed ID lacks lifecycle handling |
+| `LNV013` | card or repository comment budget exceeded |
+| `LNV014` | generated graph input is missing, malformed, unsafe, or incompatible |
+| `LNV014` | search regression gate failure |
+
+## Pull-request review
+
+Review semantic card changes as contract changes, not harmless comments. A useful PR report is the JSON output from generation plus evaluation results.
+
+Body-only changes should not force semantic text edits. Conversely, a changed invariant or externally visible role should not be hidden inside a body-only diff.
+
+## Generated files
+
+Commit `.llmnav/cache`. It is consumed by agents, diffable during review, and verified deterministically.
+
+Do not commit `.llmnav/state`, `.llmnav/.transactions`, `.llmnav/generation.lock`, or `.llmnav/generation-transaction.json`. They contain volatile acceleration, ownership, or interrupted-operation state.
+
+Do not manually edit generated cache files. Manual edits fail manifest or generation verification and are replaced on the next successful transaction.

package/docs/cli.md

@@ -0,0 +1,233 @@
+# CLI reference
+
+## Global behavior
+
+Repository commands accept `--root <path>`. Without it, the CLI searches ancestors for `.llmnav`; when none exists, it falls back to the nearest `package.json` or `.git` boundary. The repository-independent `tools` command does not accept a root.
+
+Commands with structured results accept `--json`.
+
+| Status | Meaning |
+| ---: | --- |
+| 0 | success |
+| 1 | validation, generation, evaluation, integrity, or lookup failure |
+| 2 | invalid command usage |
+| 86 | test-only injected process interruption during a cache transaction |
+
+Unknown options, missing option values, and invalid integer values fail instead of being ignored.
+
+## `llmnav init`
+
+```sh
+llmnav init [--agents <adapters>] [--package-scripts] [--force]
+```
+
+Creates project configuration, schema, registry, lexicon, evaluation file, agent protocol, deterministic cache, volatile state ignore rules, and transaction ignore rules.
+
+`--agents` accepts `agents`, `claude`, `copilot`, `cursor`, a comma-separated combination, `all`, or `none`. The default is `agents`.
+
+`--package-scripts` adds `llmnav:check`, `llmnav:format`, `llmnav:generate`, and `llmnav:eval` to an existing package.json.
+
+`--force` refreshes configuration templates, schema, managed agent blocks, and `.llmnav/.gitignore`. It never resets semantic ID history, catalog order, aliases, evaluation cases, or source cards.
+
+## `llmnav check`
+
+```sh
+llmnav check [paths...] [--format text|json|github|sarif|editor]
+```
+
+Validates syntax, canonical key order, required fields, IDs, role quality, search phrase limits, controlled effects and risks, relation targets, registry consistency, configured coverage, block size, comment ratio, and symbol attachment.
+
+Passing paths restricts source parsing, but project-level relation and registry checks are most reliable on a full scan.
+
+`--format github` emits workflow commands suitable for GitHub Actions annotations.
+
+`--format sarif` emits a deterministic SARIF 2.1.0 log with one rule per LLMNav diagnostic code and repository-relative artifact locations. The command exit status still depends on LLMNav errors, not on the selected serialization.
+
+`--format editor` emits schemaVersion 1 documents with repository-relative paths, zero-based ranges, numeric and textual severity, code, source, and message. Editors bind paths to workspace URIs themselves.
+
+## `llmnav format`
+
+```sh
+llmnav format [paths...] [--check]
+```
+
+Canonicalizes key order, list serialization, spacing, and terminators. The formatter refuses to rewrite malformed cards, unknown fields, overlapping blocks, or duplicate scalar fields.
+
+`--check` reports files that would change and exits with status 1 without writing.
+
+## `llmnav generate`
+
+```sh
+llmnav generate [--check] [--full] [--json]
+llmnav index [--check] [--full] [--json]
+```
+
+`index` remains an alias for `generate`.
+
+Generation performs these operations:
+
+1. Recover an interrupted cache transaction when a journal exists.
+2. Reuse unchanged parsed files from `file-state.json`.
+3. Parse changed files and validate the complete project.
+4. Reuse unchanged card search documents from `search-index.json`.
+5. Build every deterministic artifact in memory.
+6. Write and verify a complete staging cache.
+7. Replace the live cache through a recoverable directory transaction.
+8. Persist volatile stat hints after the deterministic cache commits.
+
+`--check`, also accepted as `--verify`, performs no writes and fails when generated artifacts differ. It still validates whether a pending transaction must be recovered before reading the cache.
+
+`--full` bypasses `file-state.json`, `search-index.json`, and `graph-state.json` accelerators and rebuilds every derived artifact from canonical source. CI, release, and trust-boundary verification should use `--full --check`; incremental mode remains the default for local iteration.
+
+Generation stops before cache mutation when semantic validation contains errors.
+
+### JSON output
+
+```json
+{
+  "ok": true,
+  "changedFiles": [],
+  "changedCards": [],
+  "affectedCatalogs": [],
+  "incremental": {
+    "enabled": true,
+    "files": {
+      "totalFiles": 120,
+      "parsedFiles": 1,
+      "reusedFiles": 119,
+      "reusedFilesByStat": 119,
+      "reusedFilesByHash": 0,
+      "deletedFiles": 0,
+      "bytesRead": 824,
+      "cardsParsed": 2,
+      "cardsReused": 418
+    },
+    "cards": {
+      "totalCards": 420,
+      "reusedCards": 419,
+      "indexedCards": 1,
+      "removedCards": 0,
+      "changedIds": ["billing.credit.reserve"],
+      "removedIds": []
+    },
+    "graph": {
+      "totalPartitions": 420,
+      "reusedPartitions": 419,
+      "rebuiltPartitions": 1,
+      "removedPartitions": 0
+    }
+  },
+  "transaction": {
+    "committed": true,
+    "skipped": false,
+    "recovered": false,
+    "recoveryAction": "none"
+  },
+  "diagnostics": []
+}
+```
+
+`changedCards` is sorted by semantic ID. Each record identifies `added`, `modified`, or `removed` and lists changed `semantic`, `structure`, and `body` hash dimensions.
+
+`affectedCatalogs` contains only repository, module, agent-context, or prompt-prefix catalogs whose generated bytes changed. Generic cache files remain visible through `changedFiles`.
+
+A no-op generation returns zero parsed files, zero indexed cards, and `transaction.skipped=true` when volatile stat hints are available and current.
+
+## `llmnav query`
+
+```sh
+llmnav query "<task language>" [--top <n>] [--json]
+```
+
+Ranks cards using exact ID matching, multilingual aliases, pre-tokenized deterministic posting lists, inverse document frequency, exact phrase bonuses, and one-hop semantic relation expansion.
+
+Only query text is tokenized per invocation. Card fields are read from `search-index.json`. A missing or incompatible search index is rebuilt in memory from the v0.1-compatible `index.json`.
+
+The ranker uses no network calls, embeddings, model API, MCP server, or hosted service. `--top` is bounded from 1 to 100.
+
+Before reading the cache, `query` waits for an active generation lock and then recovers an abandoned transaction if necessary. It never rolls back a live writer or observes an uncommitted cache as authoritative.
+
+## `llmnav show`
+
+```sh
+llmnav show <semantic-id> [--json]
+```
+
+Prints one compact card. Redirected registry IDs resolve to their active target. Qualified and unique external workspace IDs resolve to generated graph definitions when no local card exists. Ambiguous unqualified workspace IDs fail and report the qualified candidates.
+
+## `llmnav context`
+
+```sh
+llmnav context <semantic-id> [--depth <n>] [--budget <tokens>] [--max-edges <n>] [--json]
+```
+
+Resolves redirected IDs, traverses confidence-ordered outgoing and incoming graph edges breadth-first, and emits cards until the approximate token budget is reached. `--max-edges` defaults to 24 and can be set to zero to disable graph edge traversal. When no compatible graph is available, source-card semantic relations remain the fallback.
+
+Depth defaults to 1 and is bounded from 0 to 8. Budget defaults to 2,500 approximate tokens and is bounded from 128 to 100,000. The root card is retained even when it must be truncated.
+
+## `llmnav eval`
+
+```sh
+llmnav eval [--file <queries.jsonl>] [--top <n>] [--json]
+```
+
+Runs records in this format:
+
+```json
+{"query":"task language","expected":["one.id","another.id"]}
+```
+
+Reports Recall@1, Recall@5, and mean reciprocal rank. Gates come from `.llmnav/config.json`. The selected query file must remain inside the repository and cannot traverse a symbolic link.
+
+An empty query file succeeds with zero metrics. It is not evidence of search quality.
+
+## `llmnav doctor`
+
+```sh
+llmnav doctor [--json]
+```
+
+Checks:
+
+* configuration and Node.js version
+* interrupted transaction recovery
+* required control and generated files
+* primary index and inverted-index consistency
+* file-state schema compatibility
+* manifest hashes
+* generated-file drift against a full source rebuild
+* unreplaced release metadata in the LLMNav repository itself
+
+`doctor` may perform transaction recovery, but it does not regenerate stale cache content.
+
+## `llmnav spec`
+
+```sh
+llmnav spec [--json]
+```
+
+Prints the active source specification version, canonical key order, stability values, effect vocabulary, risk vocabulary, and semantic relation types. The package version is separate from the `llmnav/1` source grammar version.
+
+## `llmnav tools`
+
+```sh
+llmnav tools [--json]
+```
+
+Prints the fixed provider-neutral tool definitions for `query`, `show`, `context`, and `check`. JSON output wraps the ordered definitions in a schemaVersion 1 object. Tool inputs reject unknown fields and never accept a repository root; a trusted host binds repository scope when it calls the library dispatcher.
+
+## `llmnav bundle`
+
+```sh
+llmnav bundle [--json]
+```
+
+Loads `.llmnav/cache/prompt-prefix.json`, rejects incompatible or manifest-mismatched content, and prints its package, repository, and module cache partitions. Text output is a compact ID, scope, estimated-token, and hash summary. JSON output returns the complete deterministic bundle including partition content.
+
+## `llmnav editor`
+
+```sh
+llmnav editor vscode
+```
+
+Prints a schemaVersion 1 editor integration envelope containing a VS Code `tasks.json` configuration. The command is repository-independent and does not write `.vscode` files. Merge the returned task into an existing configuration when necessary.