openlore 2.0.7 → 2.0.8

package/README.md

@@ -33,7 +33,7 @@ Three layers, each usable independently:
 |-------|-------------|----------|
 | **1. Static Analysis** | Call graph, clusters, McCabe CC, external deps → `CODEBASE.md` digest | No |
 | **2. Spec Layer** | LLM-generated living specs, ADRs, drift detection, decision gates | For generation |
-| **3. Agent Runtime** | 45 MCP tools — `orient()`, semantic search, graph expansion | No |
+| **3. Agent Runtime** | 50 MCP tools — `orient()`, semantic search, graph expansion | No |
 
 You can use layer 1 alone to give agents structural context. Add layer 2 for semantic intent and architectural governance through OpenSpec-compatible living specifications. Layer 3 keeps that context continuously accessible through graph-native MCP tools once `openlore mcp` is running.
 
@@ -173,6 +173,32 @@ One graph query replaces most exploratory file reads. The agent knows exactly wh
 
 ---
 
+## Agent Cheat Sheet
+
+The full surface is 50 tools, but day-to-day work needs a handful. Reach for the right one by situation:
+
+| Situation | Tool |
+|-----------|------|
+| Starting any task | `orient(task)` — functions, callers, specs, insertion points in one call |
+| "Which file/function handles X?" | `search_code` |
+| Call topology across many files | `get_subgraph` / `analyze_impact` |
+| "What's the blast radius if I change this?" | `analyze_impact` — risk score + up/downstream chain + **governing decisions** |
+| "What decisions constrain this code?" | `analyze_impact` / `get_subgraph` → `governingDecisions` (Spec 16) |
+| Planning where to add a feature | `suggest_insertion_points` |
+| "How does request X reach function Y?" | `trace_execution_path` |
+| "I changed X — which tests should I run?" | `select_tests` — backward reachability to the reaching tests + paths (Spec 19) |
+| "What's dead / what dies if I delete X?" | `find_dead_code` — cross-language reachability, confidence-tagged candidates (Spec 20) |
+| "What changed structurally / whose callers are now stale?" | `structural_diff` — graph diff, stale callers, rename flags (Spec 21) |
+| "What changes together with this / what's volatile?" | `get_change_coupling` — co-change + churn from git history (Spec 22) |
+| "May I add this import here / what breaks the architecture?" | `check_architecture` — pre-edit verdict against declared rules (Spec 23) |
+| Recording an architectural choice | `record_decision` **before** writing the code |
+| Reading / checking a spec | `get_spec` · `search_specs` · `check_spec_drift` |
+| Ranking what changed by risk | `detect_changes` |
+
+Everything else (read a file, grep, list files) uses your native tools. Full reference: [docs/mcp-tools.md](docs/mcp-tools.md).
+
+---
+
 ## Use OpenLore as a Claude Code Skill
 
 OpenLore ships a canonical [Claude Code Skill](https://docs.claude.com/en/docs/claude-code/skills) at [`skills/openlore-orient/`](skills/openlore-orient/). Install it once and Claude Code will automatically call `orient()` at the start of every task — no `CLAUDE.md` editing required.
@@ -215,11 +241,31 @@ Compares git changes against spec mappings in milliseconds. Detects: Gap (code c
 
 **MCP** (no API key)
 
-45 graph-native tools exposed over stdio. Together they act as a persistent architectural runtime for coding agents: orientation, graph traversal, semantic retrieval, drift awareness, decision context, and structural risk analysis.
-`orient()` is the main entry point — one call replaces 10+ file reads. `detect_changes` risk-scores changed functions using call graph centrality × change type multiplier. See [docs/mcp-tools.md](docs/mcp-tools.md).
+50 graph-native tools exposed over stdio. Together they act as a persistent architectural runtime for coding agents: orientation, graph traversal, semantic retrieval, drift awareness, decision context, and structural risk analysis.
+`orient()` is the main entry point — one call replaces 10+ file reads. `detect_changes` risk-scores changed functions using call graph centrality × change type multiplier. Every tool call runs the same guards — input validation against its schema (bad args → JSON-RPC `-32602`), a per-tool timeout, a deterministic output-size cap, and normalized error codes — and the surface carries complete MCP `annotations`. See [docs/mcp-tools.md](docs/mcp-tools.md).
 
 `orient()` runs in **~430µs p50** against a 15k-node codebase (TypeScript compiler, ~79k edges). Full benchmark results: [scripts/BENCHMARKS.md](scripts/BENCHMARKS.md).
 
+**Test impact selection** (no API key, Spec 19)
+
+`select_tests` answers "I changed `parseConfig()` — which tests should I run?" by walking the call graph **backward** from the change to every test that transitively reaches it (via `calls` + `tested_by` + inheritance edges), returning each test with its reaching path. This is static, call-graph-based regression test selection (RTS) — established CS — served to the agent at edit time instead of to CI after the fact. grep can't do it (the reach is through indirect calls); the model is slow and guesses; a deterministic graph does it instantly. It is an honest **over-approximate prioritizer** ("run these first"), not a sound replacement for the full suite — the response states its posture, coverage, and caveats (dynamic dispatch / DI can under-select). Inputs: a symbol set or a git diff. Deterministic and offline. See [docs/test-impact-selection.md](docs/test-impact-selection.md).
+
+**Reachability & dead-code** (no API key, Spec 20)
+
+`find_dead_code` runs cross-language mark-and-sweep over the call graph: reachability from roots (tests, imported symbols, route handlers, `main`), candidate-dead = the unreached remainder, and "what becomes dead if I delete X?" = the set reachable only through X. Prior art (knip, ts-prune) is TS/JS-only; this rides the unified tree-sitter graph across 15+ languages. Results are **confidence-tagged candidates, never deletion authority** — dynamic dispatch, DI, framework routing, and externally-consumed exports cause false positives, stated in the response. A conservative module-level liveness signal keeps high-confidence candidates trustworthy (it cut them from ~470 to ~35 on a real repo). See [docs/reachability-dead-code.md](docs/reachability-dead-code.md).
+
+**Structural change analysis** (no API key, Spec 21)
+
+`structural_diff` is a graph diff — the structural complement to `git diff`. Between two states (working tree vs a ref, or two refs) it reports functions and edges added/removed, signature changes, and the existing callers in *other* files now **stale** because a callee's signature moved under them. A review/refactor agent gets "this removed `gamma`, changed `alpha`'s signature, and 5 of its callers are now stale" instead of "these 40 lines changed". Only the changed files are re-parsed (old via `git show`, new via the working tree), so it is cheap and never mutates the canonical graph; rename/move ambiguity is flagged, not guessed. See [docs/structural-diff.md](docs/structural-diff.md).
+
+**Change-coupling & volatility** (no API key, Spec 22)
+
+`get_change_coupling` mines two facts from local git history that the call graph structurally cannot see: **co-change coupling** ("these files almost always change together" — the *invisible* coupling with no import or call edge) and **volatility/churn** ("this file changed 23 times" — a risk flag). Prior art (CodeScene) puts it well: change coupling "isn’t possible to calculate from code alone — it is mined from git." Surfaced additively in `orient` as caution signals. Support/confidence thresholds and a bulk-commit filter keep it honest; it is an **advisory signal, correlation not causation**. Local, deterministic, no network (reuses the Spec 18 git ingestion). See [docs/change-coupling.md](docs/change-coupling.md).
+
+**Architecture invariant guardrails** (no API key, Spec 23)
+
+`check_architecture` turns an architectural rule from a post-hoc CI failure into a **pre-write** guardrail. A repo declares constraints — `layers`, `forbidden`, `allowedOnly` — in `.openlore/architecture.json` (or via an `Invariant:` marker on a synced ADR, so a recorded decision *carries* its invariant), and the tool answers, before the agent writes the import, *"may a file under A import B?"* with a deterministic verdict + the governing rule + why, plus a full violation scan. Prior art (ArchUnit, dependency-cruiser, import-linter) enforces architecture in CI *after* the code is written and per-language; OpenLore's contributions are **cross-language** rules over the unified dependency graph and **agent-facing, pre-edit** evaluation (reusing the same `classifyLayerEdge` primitive that powers `CODEBASE.md`'s layer report). Opt-in and fully inert until rules are declared; never LLM-inferred; complements, not replaces, CI linters. See [docs/architecture-invariants.md](docs/architecture-invariants.md).
+
 **Epistemic Lease** (no API key)
 
 > **Core principle**: EpistemicLease models architectural drift as a behavioral navigation phenomenon rather than a semantic understanding problem. Context decay is driven by where the agent goes (cross-module trajectory), not what it knows.
@@ -247,6 +293,12 @@ When fresh, injection is zero-overhead. Calling `orient()` resets the tracker. U
 
 Agents call `record_decision` before writing code. Consolidation runs immediately in the background. At commit time, a pre-commit hook gates the commit until all verified decisions are reviewed and written back as requirements in `spec.md` files. Decisions are classified by scope (`local / component / cross-domain / system`); only `cross-domain` and `system` decisions produce ADR files, keeping the decision log signal-dense.
 
+Decisions are also **first-class graph nodes**. At analyze time the active decision store is projected — the same parser→projector split that puts Infrastructure-as-Code on the graph — into `decision::<id>` nodes joined to the files they govern by `affects` edges. The relationship is stored, not recomputed: `analyze_impact` and `get_subgraph` return the governing decisions of a symbol and its blast radius as typed neighbors (`nodeType: "decision"`), and `orient` reports which relevant files each decision governs. This turns "what architectural decisions constrain this code, and what does changing it implicate?" into a deterministic graph query — the join no code-navigation competitor offers. The JSON store stays authoritative; the projection is derived and rebuilt on every analyze. See [docs/specs/openlore-spec-16-decisions-as-graph-nodes.md](docs/specs/openlore-spec-16-decisions-as-graph-nodes.md).
+
+**Provenance** (no API key, local-only)
+
+Reads the local `.git` history (and local `gh` if present) to project `authored_by` (file → person) and `changed_in_pr` (file → PR) edges onto the graph, so `orient` answers "last changed by X in PR #N" — provenance grep cannot surface. **No OAuth, no cloud connector, nothing is ever uploaded**: the git-only path needs no network, and `gh` is an optional enrichment that degrades gracefully when absent or unauthenticated. Bounded (last-touch + top-N recent authors + recent PRs per file) so the graph never bloats; deterministic for a fixed git state. The same local history feeds the change-coupling instrument (Spec 22). See [docs/provenance.md](docs/provenance.md).
+
 **Telemetry** (opt-in, no API key)
 
 Cognitive telemetry for empirical measurement of EpistemicLease behavior. Gated by `OPENLORE_TELEMETRY=1` — disabled by default. Writes append-only JSONL to `.openlore/telemetry/` per domain. Agent identity is captured from the MCP `initialize` handshake, enabling per-agent behavioral comparison.
@@ -274,25 +326,66 @@ Key metrics: **obstinacy index** (tool calls after stale before orient — measu
 
 OpenSpec provides semantic intent and workflow structure. openlore maintains the evolving implementation as a continuously queryable architectural graph for agents.
 
+```mermaid
+flowchart TD
+    Code[Codebase] --> Analyze[openlore analyze<br/>tree-sitter · pure static analysis]
+    Analyze --> DB[(SQLite graph store<br/>.openlore/analysis/call-graph.db)]
+    Analyze --> Digest[CODEBASE.md<br/>~600-token structural digest]
+
+    subgraph shared["Projected onto shared node + edge primitives"]
+      direction LR
+      CodeNodes[functions + call edges]
+      Iac[IaC resources + references]
+      Dec[decisions + affects edges]
+    end
+    Analyze --> CodeNodes
+    Analyze --> Iac
+    Analyze -. active decision store .-> Dec
+    CodeNodes --> DB
+    Iac --> DB
+    Dec --> DB
+
+    DB --> MCP[50 MCP tools<br/>orient · BFS · search · analyze_impact]
+    MCP --> Agent((Coding Agent))
+
+    Code -. optional, API key .-> Gen[openlore generate]
+    Gen --> Specs[openspec/specs/*.md<br/>RFC 2119 living specs]
+    Code --> Drift[openlore drift<br/>spec/code drift, ms, no API]
+    Agent -. record_decision .-> Gate[decisions pre-commit gate]
+    Gate --> Specs
 ```
-Codebase
-   │
-   ▼
-openlore analyze ──► SQLite graph store (.openlore/analysis/call-graph.db)
-                          │                      │
-                          │              MCP tools (orient, BFS, search…)
-                          │                      │
-                     Artifact Generator        Agent
-                          │
-                    ┌─────┴──────┐
-                    ▼            ▼
-              CODEBASE.md   (optional)
-                         openlore generate ──► openspec/specs/*.md
-                         openlore drift   ──► drift report
-                         openlore decisions ► ADR gates
+
+The graph and the OpenSpec spec layer are co-equal: the graph makes orientation fast, the specs make it semantically grounded. Drift detection and decision gates connect both. Crucially, application code, Infrastructure-as-Code, and architectural **decisions** all project onto one shared set of node/edge primitives — so a single traversal answers questions that span all three. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full pipeline diagram.
+
+**Decisions on the graph** (Spec 16) — a decision becomes a node joined to the files it governs by `affects` edges, so impact analysis returns governance as a neighbor:
+
+```mermaid
+flowchart LR
+    D["decision::c6d1ad07<br/>North-star substrate"]:::dec
+    D -- affects --> F1[src/cli/export/scip.ts]
+    D -- affects --> F2[src/core/analyzer/iac/project.ts]
+    F1 -- calls --> G[exportScip]
+    classDef dec fill:#6f42c1,stroke:#4b2e83,color:#fff
 ```
 
-The graph and the OpenSpec spec layer are co-equal: the graph makes orientation fast, the specs make it semantically grounded. Drift detection and decision gates connect both. See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full pipeline diagram.
+---
+
+## Design Decisions
+
+OpenLore dogfoods its own decision system. These ADRs were recorded with `record_decision`, gated at commit, and synced into `openspec/specs/` — and (per Spec 16) are now projected onto the graph itself. They are the load-bearing constraints behind the architecture above:
+
+| Decision | Rationale | Where |
+|----------|-----------|-------|
+| **North star is a deterministic structural context substrate** | Local-first plumbing (like tree-sitter/SCIP/LSP) that agents build on; every feature must make the coding-agent case more useful and stay grounded in static analysis, not LLM guessing | [ADR-0001](openspec/decisions/adr-0001-north-star-is-a-deterministic-structural-context-s.md) |
+| **IaC resources project onto the existing graph primitives** | One projector maps infrastructure onto `FunctionNode`/`CallEdge` so every MCP tool works on IaC with zero new tooling | `analyzer` spec · `src/core/analyzer/iac/project.ts` |
+| **Decisions project onto the graph the same way** | A parser→projector split turns the decision store into `decision::` nodes + `affects` edges — governance becomes a deterministic graph join | `analyzer` spec · `src/core/decisions/project.ts` (Spec 16) |
+| **EdgeStore uses SCHEMA_VERSION rebuild-on-bump, not migrations** | The graph is fully derivable from source, so a schema change drops and rebuilds — no migration code, no drift | `analyzer` spec · `src/core/services/edge-store.ts` |
+| **BM25 keyword retrieval is the zero-network floor** | `orient`/`search_code` work with no API key or embedding server; dense embeddings are an optional upgrade, never a requirement | `analyzer` spec · Spec 06 |
+| **SCIP is a one-way export, not a round-trip format** | The SQLite graph stays canonical; SCIP exports only the subset it can model, avoiding a lossy bidirectional contract | `cli` spec · `src/cli/export/scip.ts` |
+| **MCP exposes a curated `navigation` preset, not all 50 tools** | A lean graph-traversal surface is what wins the Spec 14 agent benchmark; the full set stays available opt-in | `cli` spec · Spec 14 |
+| **Decision consolidation is serialized with a cross-process file lock** | Concurrent `record_decision` calls were losing drafts; a lock makes consolidation safe and every commit instant | `cli` spec · Spec 15 |
+
+This table is not aspirational documentation — it is the live decision log the pre-commit gate enforces and Spec 16 makes queryable. See [docs/governance-dogfooding.md](docs/governance-dogfooding.md).
 
 ---
 
@@ -326,7 +419,7 @@ The manifest captures the public API surface, HTTP routes, stats, dependencies,
 
 | Topic | Doc |
 |-------|-----|
-| MCP tools reference (45 tools + parameters) | [docs/mcp-tools.md](docs/mcp-tools.md) |
+| MCP tools reference (50 tools + parameters) | [docs/mcp-tools.md](docs/mcp-tools.md) |
 | Agent setup (Claude Code, Cline, OpenCode, Vibe…) | [docs/agent-setup.md](docs/agent-setup.md) |
 | `openlore install` — auto-configure agent surfaces | [docs/install.md](docs/install.md) |
 | LLM providers + embedding config | [docs/providers.md](docs/providers.md) |
@@ -335,6 +428,13 @@ The manifest captures the public API surface, HTTP routes, stats, dependencies,
 | CI/CD integration | [docs/ci-cd.md](docs/ci-cd.md) |
 | Preflight CI staleness gate | [docs/preflight.md](docs/preflight.md) |
 | SCIP export (Sourcegraph/Glean interop) | [docs/scip-export.md](docs/scip-export.md) |
+| Cross-domain impact (code ↔ infrastructure) | [docs/cross-domain-impact.md](docs/cross-domain-impact.md) |
+| Local provenance (git/PR, no OAuth) | [docs/provenance.md](docs/provenance.md) |
+| Test impact selection (which tests to run) | [docs/test-impact-selection.md](docs/test-impact-selection.md) |
+| Reachability & dead-code analysis | [docs/reachability-dead-code.md](docs/reachability-dead-code.md) |
+| Structural change analysis (graph diff) | [docs/structural-diff.md](docs/structural-diff.md) |
+| Change-coupling & volatility (git-mined) | [docs/change-coupling.md](docs/change-coupling.md) |
+| Architecture invariant guardrails (pre-edit) | [docs/architecture-invariants.md](docs/architecture-invariants.md) |
 | Federation manifest (cross-repo) | [docs/federation.md](docs/federation.md) |
 | CLI command reference | [docs/cli-reference.md](docs/cli-reference.md) |
 | Interactive graph viewer | [docs/viewer.md](docs/viewer.md) |
@@ -378,6 +478,8 @@ The manifest captures the public API surface, HTTP routes, stats, dependencies,
 
 **Infrastructure-as-Code**: Terraform/HCL · Kubernetes · Helm · CloudFormation · Ansible · Pulumi · AWS CDK · CDKTF — IaC resources and their references are projected onto the same graph as application code, so `orient`, `search_code`, `get_subgraph`, and `analyze_impact` answer "what is the blast radius of changing this security group / ConfigMap / IAM role?" with zero new tooling. See [docs/iac.md](docs/iac.md).
 
+**Cross-domain impact** (Spec 17): for embedded IaC (Pulumi/CDK/CDKTF), the code that provisions a resource is linked to it by a `references` edge, so `analyze_impact` traverses the code↔infra boundary **end-to-end** — "what infrastructure does this handler reach?" and the reverse, "what code breaks if I change this resource?". Infra neighbors are surfaced as a typed, ecosystem-tagged `crossDomain` block, distinct from the code blast radius. A code-only navigator structurally cannot answer this. Reproducible example: [docs/cross-domain-impact.md](docs/cross-domain-impact.md).
+
 ---
 
 ## Development