mapterrain 0.1.1 → 0.3.0

package/README.md

@@ -1,480 +1,203 @@
 # Terrain
 
-**Map your test terrain.** Understand your test system in 30 seconds.
+**Open-source CI pre-flight layer for AI/ML systems and the tests around them. Runs locally. No API key required.**
 
-```bash
-# Homebrew
-brew install pmclSF/terrain/mapterrain
-
-# npm
-npm install -g mapterrain
-
-cd your-repo
-terrain analyze
-```
-
-That's it. No config, no setup, no test execution required.
-
-> **New here?** Read the [Quickstart Guide](docs/quickstart.md) to understand your first report in 5 minutes.
-
----
-
-Terrain is a test system intelligence platform. It reads your repository — test code, source structure, coverage data, runtime artifacts, ownership files, and local policy — and builds a structural model of how your tests relate to your code. From that model it surfaces risk, quality gaps, redundancy, fragile dependencies, and migration readiness, all without running a single test.
-
-The core idea: every codebase has a *test terrain* — the shape of its testing infrastructure, the density of coverage across areas, the hidden fault lines where a fixture change cascades into thousands of tests. Terrain makes that shape visible and navigable so you can make informed decisions about what to test, what to fix, and where to invest.
-
-## What "Test Terrain" Means
-
-Most teams know what tests they have. Few teams understand the *terrain* underneath:
-
-- Which source files have no structural test coverage?
-- Which shared fixtures fan out to thousands of tests, making any change to them a blast-radius problem?
-- Which test clusters are near-duplicates burning CI time?
-- Which areas have tests but weak assertions that wouldn't catch a real regression?
-- When you change `auth/session.ts`, which 41 of your 18,000 tests actually matter?
-
-Test terrain is the structural topology of your test system — the dependency graph, the coverage landscape, the duplication clusters, the fanout hotspots, the skip debt. Terrain maps it.
-
-## Problems Terrain Solves
-
-**"We don't know the state of our test system."** Teams inherit test suites they didn't write. Terrain gives a baseline in seconds: framework mix, coverage confidence, duplication, risk posture.
-
-**"CI takes too long and we don't know what to cut."** Terrain identifies redundant tests, high-fanout fixtures, and confidence-based test selection — showing where CI time is wasted and what can be safely reduced.
-
-**"We changed auth code — what tests should we worry about?"** Terrain traces your change through the import graph and structural dependencies, returning the impacted tests ranked by confidence, with reason chains explaining each selection.
-
-**"A tool flagged something but won't explain why."** Every Terrain finding carries an evidence chain. `terrain explain` shows exactly what signals, dependency paths, and scoring rules produced each decision.
-
-**"We're migrating frameworks and need to know what's blocking us."** Migration readiness, blockers by type, and preview-scoped difficulty assessment — all derived from static analysis.
-
-## The Four Canonical Workflows
-
-Terrain is organized around four questions. Everything else is a supporting view.
-
-```
-terrain analyze     "What is the state of our test system?"
-terrain insights    "What should we fix in our test system?"
-terrain impact      "What validations matter for this change?"
-terrain explain     "Why did Terrain make this decision?"
-```
-
-### 1. Analyze — understand the test system
-
-```bash
-terrain analyze
-```
-
-```
-Terrain — Test Suite Analysis
-============================================================
-
-  conftest.py fixture fans out to 3,100 tests — any change retriggers the frame/ suite.
-
-Key Findings
-------------------------------------------------------------
-  1. [HIGH] 23 source files (18%) have low structural coverage
-  2. [HIGH] 8 duplicate test clusters with 0.91+ similarity
-  3. [MEDIUM] 34 xfail markers older than 180 days
-
-Repository Profile
-------------------------------------------------------------
-  Test volume:          very large
-  CI pressure:          high
-  Coverage confidence:  medium
-  Redundancy level:     medium
-  Fanout burden:        high
-
-Validation Inventory
-------------------------------------------------------------
-  Test files:     1,047
-  Test cases:     52,341
-  Frameworks:
-    pytest                1,047 files [unit]
-
-Risk Posture
-------------------------------------------------------------
-  health:              MODERATE
-  coverage_depth:      ELEVATED
-  coverage_diversity:  STRONG
-  structural_risk:     STRONG
-  operational_risk:    STRONG
-
-Next steps:
-  terrain insights            prioritized actions and recommendations
-  terrain impact              what tests matter for this change?
-```
-
-### 2. Insights — find what to improve
-
-```bash
-terrain insights
-```
-
-```
-Terrain — Test System Health Report
-============================================================
-
-Health Grade: C
-
-Reliability Problems (2)
-  [HIGH] 12 flaky tests with >10% failure rate
-  [MEDIUM] 34 skipped tests consuming CI resources
-
-Coverage Debt (2)
-  [HIGH] 23 source files (18%) have low structural coverage
-  [MEDIUM] conftest.py fixture fans out to 3,100 tests
-
-Recommended Actions
-  1. [reliability] Quarantine 12 flaky tests
-     why: Flaky tests block unrelated PRs and erode CI trust.
-  2. [coverage] Add tests for 23 uncovered source files
-     why: Changes in uncovered files cannot trigger test selection.
-  3. [optimization] Consolidate 8 duplicate test clusters
-     why: 0.91+ similarity — redundant CI cost with no coverage benefit.
-```
-
-### 3. Impact — understand what a change affects
-
-```bash
-terrain impact --base main
-```
-
-```
-Terrain Impact Analysis
-============================================================
-
-Summary: 3 file(s) changed, 41 test(s) relevant. Posture: needs_attention.
-
-Impacted tests:          127 of 52,341 total
-Coverage confidence:     High
-
-Recommended Tests (41)
-------------------------------------------------------------
-  tests/groupby/test_groupby.py [exact]
-    Covers: groupby.py:GroupBy.aggregate
-  tests/groupby/test_apply.py [exact]
-    Covers: groupby.py:GroupBy.apply
-  tests/resample/test_base.py [inferred]
-    Reached via shared fixture path
-  ...and 38 more
-
-Protection Gaps
-------------------------------------------------------------
-  [medium] pandas/core/groupby/ops.py — no covering tests found
-
-Next steps:
-  terrain impact --show tests      full test list
-  terrain impact --show gaps       all protection gaps
-```
-
-### 4. Explain — understand why
-
-```bash
-terrain explain tests/io/json/test_pandas.py
-```
-
-```
-Test File: tests/io/json/test_pandas.py
-Framework: pytest
-Tests: 84    Assertions: 312
-
-Signals (3):
-  [high]   networkDependency: 12 tests use @pytest.mark.network — flaky in CI
-  [medium] weakAssertion: 8 bare assert statements without descriptive messages
-  [low]    xfailAccumulation: 3 xfail markers older than 180 days
-
-Next steps:
-  terrain explain selection       explain overall test selection strategy
-  terrain impact --show tests     see all impacted tests
-```
-
-See [Canonical User Journeys](docs/product/canonical-user-journeys.md) for the full workflow specification and [example outputs](docs/examples/) for detailed report samples.
-
-## Product Philosophy
-
-**Inference first.** Terrain reads code. It parses imports, detects frameworks, resolves coverage relationships, and builds dependency graphs from what already exists in the repository. No annotations, no test tagging, no SDK integration required.
-
-**Zero-config by default.** `terrain analyze` works on any repository with test files. Coverage data, runtime artifacts, ownership files, and policy rules are optional inputs that enrich the model — but the core analysis requires nothing beyond the code itself.
-
-**Explainability over magic.** Every finding carries evidence: which signal type, what confidence level, what dependency path, what scoring rule. `terrain explain` exposes the full reasoning chain behind any decision. Teams should never wonder *why* Terrain said something.
-
-**Conservative under uncertainty.** When Terrain encounters ambiguity — a dependency path with low confidence, a file that might or might not be a test — it flags the uncertainty rather than guessing. Impact analysis uses fallback policies with explicit confidence penalties rather than silently expanding scope.
-
-**System health, not individual productivity.** Terrain measures the test system. It never attributes quality to individual developers. Ownership information is used for routing and triage, not scoring.
-
-## Who Uses Terrain
-
-Terrain is framework-agnostic and language-aware. The same analysis model applies across:
+Rename `customer_name` to `full_name`, and a prompt in another part of the repo may still ask the model for `customer_name`. Change a retriever, and the eval that would catch the quality drop may never run. Terrain catches those gaps in the PR, before the change merges.
 
-- **Frontend teams** — React/Vue component tests, Playwright/Cypress E2E suites, Vitest/Jest unit tests
-- **Backend teams** — Go test suites, pytest collections, JUnit hierarchies, integration test infrastructure
-- **Mobile teams** — cross-platform test suites with standard test frameworks
-- **QA / SDET** — test portfolio management, coverage gap analysis, migration planning across frameworks
-- **SRE / Platform** — CI optimization, test selection for pipelines, policy enforcement
-- **AI / ML evaluation** — evaluation suite structure, benchmark test management, coverage across model behaviors
+Terrain does that by connecting source code, tests, prompts, schemas, eval outputs, runtime artifacts, and coverage reports into one CI-ready protection graph. It turns that graph into ranked findings, explainable impact, JUnit test cases, GitHub annotations, SARIF for security findings, MCP context for AI assistants, and portfolio rollups across repos.
 
-The structural model is the same. The signals and recommendations adapt to the framework and test patterns detected.
+## Why install Terrain?
 
-## AI Testing in CI
+Most teams already have tests, coverage, and AI evals. The hard part is knowing whether those signals still protect the change in front of you. A developer can see that unit tests passed or that an eval failed, but still not know whether a PR touched an AI/ML surface, which validations should run, what coverage is missing, or why a downstream regression is connected to their diff.
 
-Terrain gives AI components the same CI safety net as regular tests:
+Terrain gives every PR a pre-merge protection map. In one local command it answers:
 
-- **Surface discovery** — automatically detects prompts, contexts, datasets, tool definitions, RAG pipelines, and eval scenarios in your code
-- **Impact-scoped selection** — `terrain ai run --base main` runs only the eval scenarios affected by your change
-- **Protection gaps** — `terrain pr` flags changed AI surfaces that have no eval scenario covering them
-- **Policy enforcement** — block PRs that modify uncovered AI surfaces, regress accuracy, or trigger safety failures
-- **GitHub Action** — drop-in `terrain-ai.yml` workflow template for AI CI gates
+- What is weak or missing in the current test system?
+- What does this PR put at risk?
+- Which tests or evals matter for this change?
+- Why did the gate fail, and what should be fixed first?
 
-The same structural graph that powers test selection for regular code also traces AI surface dependencies, so a change to a prompt template triggers the right eval scenarios automatically.
+That makes day-to-day work less manual:
 
-## How CI Optimization Emerges
+- Instead of tracing code -> API -> prompt -> eval by hand, run `terrain report pr` and see what the change affects.
+- Instead of rerunning every eval because no one knows which one matters, run the validations Terrain selects for the diff.
+- Instead of reading a failed eval in isolation, use the cause path to see which code, schema, prompt, or retrieval change led there.
+- Instead of letting weak assertions, skipped tests, and uncovered exports accumulate quietly, surface them as concrete findings before release.
 
-Terrain does not start with CI optimization. It starts with understanding.
+The result is faster review, fewer broad reruns, clearer fixes, and fewer regressions that only make sense after someone reconstructs the system by hand.
 
-When you run `terrain analyze`, Terrain builds a structural model: which tests exist, which source files they cover, how they depend on shared fixtures, where duplication lives. From that model, CI optimization *emerges*:
+Bring your own stack. Terrain reads what teams already use: `pytest`, `jest`, `go test`, Playwright, Promptfoo, DeepEval, Ragas, Great Expectations, Gauntlet-style eval-result JSON, JUnit, LCOV, Istanbul, and repository metadata. It does not replace those tools; it unifies their evidence into one local, deterministic CI gate.
 
-- **Test selection** — `terrain impact` traces a diff through the dependency graph and returns only the tests that structurally matter, ranked by confidence. This is not a heuristic skip list — it is a graph traversal with evidence.
-- **Redundancy reduction** — `terrain insights` surfaces duplicate test clusters. Removing or consolidating them directly reduces CI time without reducing coverage.
-- **Fanout control** — High-fanout fixtures that trigger thousands of tests on any change are identified and prioritized for splitting.
-- **Confidence-based runs** — Impact analysis assigns confidence scores. CI pipelines can run high-confidence tests immediately and defer low-confidence tests to nightly runs.
-- **What to test next** — `terrain insights` ranks untested source files by dependency count, telling you which test to write first for maximum impact.
+Project status: Terrain is pre-1.0 and actively developed. In 0.3.0, the stable path is the CLI, local/CI artifact contract, GitHub Actions flow, AI/eval artifact ingestion, and portfolio aggregation. The VS Code extension is alpha; marketplace listings, full LSP integration, the plugin runtime, and some preview rules are future work.
 
-The result is faster CI that comes from *understanding the test system*, not from skipping tests and hoping for the best.
-
-## Installation
-
-### Homebrew (macOS and Linux)
+## Install
 
 ```bash
+# macOS / Linux
 brew install pmclSF/terrain/mapterrain
-```
-
-After the first install, you can also tap once and use the short formula name:
-
-```bash
-brew tap pmclSF/terrain
-brew install mapterrain
-```
 
-### npm
-
-```bash
+# npm (Node 22+ required; macOS/Linux amd64+arm64, Windows amd64)
 npm install -g mapterrain
-```
-
-### Go install
 
-```bash
+# Go
 go install github.com/pmclSF/terrain/cmd/terrain@latest
 ```
 
-Requires Go 1.23 or later.
+Pre-built archives are available for macOS and Linux on amd64/arm64, plus Windows on amd64, from the [releases page](https://github.com/pmclSF/terrain/releases). Each release is signed with Sigstore + cosign; see [SECURITY-DATA-HANDLING.md](SECURITY-DATA-HANDLING.md) for verification details.
 
-### Pre-built binaries
+Package names vary by distribution, but the installed CLI is always `terrain`. The npm package is `mapterrain`; the Homebrew formula is `pmclSF/terrain/mapterrain`.
 
-Download the appropriate binary for your platform from [GitHub Releases](https://github.com/pmclSF/terrain/releases), then:
+## Get started
 
 ```bash
-chmod +x terrain
-sudo mv terrain /usr/local/bin/
+cd your-repo
+terrain analyze         # What's the state of our AI + test system?
+terrain report pr       # What does this change put at risk?
 ```
 
-Binaries are available for macOS, Linux, and Windows (amd64 and arm64).
-
-### Build from source
+Source analysis covers Python, TypeScript/JavaScript, Go, and Java in 0.3.0. Ruby source is not analyzed in 0.3.0, but Ruby/RSpec and other ecosystems can still contribute dependency, runtime, coverage, and eval artifacts, so mixed-language repos get useful signal. No config required; optional artifacts sharpen findings when present.
 
-```bash
-git clone https://github.com/pmclSF/terrain.git
-cd terrain
-go build -o terrain ./cmd/terrain
-```
+## What it catches
 
-### Verify installation
+Terrain models the AI surface alongside the test surface and looks for drift across them:
 
-```bash
-terrain --version
-```
+- **Prompt-schema drift** — prompts that reference fields renamed in a schema living in a different language
+- **Hardcoded API keys** — provider-shaped secrets in source (OpenAI, Anthropic, AWS, GCP, etc.)
+- **Eval coverage gaps** — AI surfaces (prompts, agents, RAG pipelines) with no scenario covering them
+- **Model deprecations** — deprecated model IDs lingering in production paths
+- **Cross-language edges** — TS/JS ↔ Python/Go/Java via OpenAPI, tRPC, gRPC, GraphQL, HTTP routes
+- **Framework-migration blockers** — Jest ↔ Vitest, JUnit 4 ↔ 5, Cypress ↔ Playwright
+- **Portfolio drift across repos** — manifest-backed `terrain portfolio --from` rollups show framework-of-record drift across a polyrepo
+- **Untested exports + weak assertions** — public API surfaces with no covering test; assertions that pass on too much
+- **Fixture fanout + duplicate clusters + skip debt** — structural problems that erode CI signal
 
-## Quick Start
+Each finding carries a stable rule ID, severity, confidence, evidence, and documented remediation. Run `terrain explain <rule-id>` for the long form.
 
-```bash
-# Detect coverage/runtime data paths (recommended first step)
-terrain init
+## What it looks like
 
-# Analyze the current repository
-terrain analyze
+```
+Terrain · Test Suite Analysis
+────────────────────────────────────────────────────────────
 
-# JSON output for any command
-terrain analyze --json
+  conftest.py fixture fans out to 3,100 tests — any change retriggers the frame/ suite.
 
-# See what a change affects
-terrain impact --base main
+Key Findings
+────────────────────────────────────────────────────────────
+  1. [HIGH] 23 exported code units have no linked tests
+  2. [MED]  12 test files have weak assertion density
+  3. [LOW]  7 skipped-test patterns need review
 
-# Get prioritized recommendations
-terrain insights
+Risk Posture
+────────────────────────────────────────────────────────────
+  health:                  Moderate
+  coverage_depth:          Elevated
+  coverage_diversity:      Strong
+  structural_risk:         Strong
+  Signals:                 65 (8 high, 34 medium, 23 low)
 ```
 
-## Commands
+> Representative output from a large pandas-style repository. Format and labels are stable across runs; the specific numbers vary by repo. Full sample reports for `analyze`, `insights`, `impact`, and `explain` are in [docs/examples/](docs/examples/).
 
-### Primary commands
+## Workflow
 
 | Command | Question |
 |---------|----------|
 | `terrain analyze` | What is the state of our test system? |
-| `terrain insights` | What should we fix in our test system? |
-| `terrain impact` | What validations matter for this change? |
-| `terrain explain <target>` | Why did Terrain make this decision? |
-
-### Supporting commands
-
-| Command | Purpose |
-|---------|---------|
-| `terrain init` | Detect data files and print recommended analyze command |
-| `terrain summary` | Executive summary with risk, trends, benchmark readiness |
-| `terrain focus` | Prioritized next actions |
-| `terrain posture` | Detailed posture breakdown with measurement evidence |
-| `terrain portfolio` | Portfolio intelligence: cost, breadth, leverage, redundancy |
-| `terrain metrics` | Aggregate metrics scorecard |
-| `terrain compare` | Compare two snapshots for trend tracking |
-| `terrain select-tests` | Recommend protective test set for a change |
-| `terrain pr` | PR/change-scoped analysis |
-| `terrain show <type> <id>` | Drill into test, unit, owner, or finding |
-| `terrain migration <sub>` | Migration readiness, blockers, or preview |
-| `terrain policy check` | Evaluate local policy rules |
-| `terrain export benchmark` | Privacy-safe JSON export for benchmarking |
-| `terrain serve` | Local HTTP server with HTML report and JSON API |
-
-### AI / eval
-
-| Command | Purpose |
-|---------|---------|
-| `terrain ai list` | List detected scenarios, prompts, datasets, eval files |
-| `terrain ai doctor` | Validate AI/eval setup and surface configuration issues |
-| `terrain ai run` | Execute eval scenarios and collect results |
-| `terrain ai replay` | Replay and verify a previous eval run artifact |
-| `terrain ai record` | Record eval results as a baseline snapshot |
-| `terrain ai baseline` | Manage eval baselines (show, compare) |
-
-### Conversion / migration
-
-| Command | Purpose |
-|---------|---------|
-| `terrain convert <source>` | Go-native test conversion (25 directions) |
-| `terrain convert-config <source>` | Convert framework config files |
-| `terrain migrate <dir>` | Project-wide migration with state tracking |
-| `terrain estimate <dir>` | Estimate migration complexity |
-| `terrain status` | Show migration progress |
-| `terrain checklist` | Generate migration checklist |
-| `terrain doctor [path]` | Run migration diagnostics |
-| `terrain reset` | Clear migration state |
-| `terrain list-conversions` | List supported conversion directions |
-| `terrain shorthands` | List shorthand aliases (e.g., `cy2pw`, `jest2vt`) |
-| `terrain detect <file-or-dir>` | Detect dominant framework |
-
-### Advanced / debug
-
-| Command | Purpose |
-|---------|---------|
-| `terrain debug graph` | Dependency graph statistics |
-| `terrain debug coverage` | Structural coverage analysis |
-| `terrain debug fanout` | High-fanout node analysis |
-| `terrain debug duplicates` | Duplicate test cluster analysis |
-| `terrain debug depgraph` | Full dependency graph analysis (all engines) |
-
-Repository-scoped commands support `--root PATH`, and machine-readable commands support `--json`. Most analysis commands support `--verbose` for additional detail. Run `terrain <command> --help` for full flag documentation.
-
-## Architecture Overview
-
-Terrain is built around a signal-first architecture:
+| `terrain report pr` | What does this change put at risk? |
+| `terrain report insights` | What should we fix? |
+| `terrain report impact` | What validations matter for this change? |
+| `terrain report explain <target>` | Why did Terrain make this decision? |
 
-```
-Repository scan  →  Signal detection  →  Risk modeling  →  Reporting
-     │                    │                    │               │
-  test files         framework-specific    explainable     human-readable
-  source files       pattern detectors     risk scoring    + JSON output
-  coverage data      quality signals       with evidence
-  runtime artifacts  health signals        chains
-  ownership files    migration signals
-  policy rules       governance signals
-```
+The bare forms (`terrain insights`, `terrain impact`, `terrain explain`) work as aliases. AI / eval verbs, framework conversion, debug drill-downs, and the slash-receiver round out the surface — full reference in [docs/cli-spec.md](docs/cli-spec.md).
 
-- **Signals** are the core abstraction — every finding is a structured signal with type, severity, confidence, evidence, and location
-- **Snapshots** (`TestSuiteSnapshot`) are the canonical serialized artifact — the complete structural model of a test system at a point in time
-- **Risk surfaces** are derived from signals with explainable scoring across dimensions (quality, reliability, speed, governance)
-- **Dependency graphs** model import relationships, fixture fanout, and structural coverage
-- **Reports** synthesize signals, risk, trends, and benchmark readiness into actionable output
+## CI integration
 
-```
-cmd/terrain/     CLI (30+ commands)
-internal/        47 Go packages covering analysis, signals, risk,
-                 impact, depgraph, measurement, reporting, and more
+```yaml
+# .github/workflows/terrain.yml
+name: terrain
+on: pull_request
+
+jobs:
+  analyze:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+        with: { fetch-depth: 0 }
+      - uses: actions/setup-node@v6
+        with: { node-version: '22.x' }
+      - run: npm install -g mapterrain
+      - run: |
+          terrain test \
+            --junit terrain-results.xml \
+            --summary "$GITHUB_STEP_SUMMARY"
 ```
 
-See [DESIGN.md](DESIGN.md) for the full architecture overview and package map, [docs/architecture/](docs/architecture/) for detailed design documents, and [docs/json-schema.md](docs/json-schema.md) for JSON output structure.
+`terrain test` is the CI-mode wrapper. The JUnit XML lets your CI render Terrain findings as test cases; setting `--summary "$GITHUB_STEP_SUMMARY"` makes them appear on the workflow run page automatically.
 
-## Snapshot Workflow
+For a blocking gate, add `--fail-on=high` (or `--fail-on=critical` for the strictest threshold). For onboarding a repo with existing debt, pair with `--new-findings-only --baseline <path>` so only regressions block the build. To restrict the gate to AI-related changes, add a `paths:` filter on prompt / schema / Python / TS file globs (see [docs/examples/gate/](docs/examples/gate/) for the full templates).
 
-Terrain supports local snapshot history for trend tracking:
+## Boundaries
 
-```bash
-# Save a snapshot
-terrain analyze --write-snapshot
+Terrain is intentionally a CI/local pre-flight layer. It composes with the tools teams already trust instead of replacing them:
 
-# Later, save another snapshot
-terrain analyze --write-snapshot
+- It reads what `pytest`, `jest`, `go test`, Playwright, Promptfoo, DeepEval, Ragas, Great Expectations, and Gauntlet-style eval-result artifacts produce. The AI namespace can execute supported eval-framework commands for `terrain ai run`, but Terrain does not replace those frameworks.
+- It ingests coverage reports when you have them; it does not instrument code.
+- It complements Semgrep / CodeQL / Sonar rather than replacing application-code bug finding.
+- It routes ownership data to findings; it never produces leaderboards or per-developer scores.
+- It runs locally with zero outbound network calls during analysis — verifiable with `terrain --print-network`. The install paths download signed binaries from GitHub Releases; Terrain itself does not upload analysis results.
+- CI artifacts and MCP responses can include repo paths, line numbers, diagnostic text, and sometimes source excerpts. Treat the CI platform or AI assistant you connect as part of the trust boundary.
 
-# Compare the two most recent snapshots
-terrain compare
+Under the hood, Terrain combines source analysis, dependency-graph construction, artifact ingestion, and deterministic rule evaluation. The product is the CI gate and protection graph; static analysis is one mechanism inside it.
 
-# Executive summary automatically includes trend highlights
-terrain summary
-```
+## AI-aware testing
 
-Snapshots are stored in `.terrain/snapshots/` as timestamped JSON files.
+The same dependency graph that powers test selection for application code also traces AI surface edges. When Terrain can map a prompt-template, schema, or eval-artifact change to declared or inferred scenarios, it selects the impacted evals automatically.
 
-## Policy
+- `terrain ai run --base main` — run only the evals your change affects
+- `terrain report pr` — flag changed AI surfaces with no covering eval
+- `terrain ai findings` — AI eval-gap findings with evidence chains
 
-Define local policy rules in `.terrain/policy.yaml`:
+Two generators help adopters harden a prompt before deploying it:
 
-```yaml
-rules:
-  disallow_skipped_tests: true
-  max_weak_assertions: 10
-  max_mock_heavy_tests: 5
+```bash
+terrain inject --prompt prompts/main.md       # generate jailbreak-shaped test inputs
+terrain scaffold --schema schemas/input.json  # generate boundary-case mutation tests
 ```
 
-Then check compliance:
+Both emit runnable pytest / vitest scaffolds you drop into your test tree. Terrain never calls the model; the assertion is yours.
 
-```bash
-terrain policy check        # human-readable output
-terrain policy check --json # JSON output for CI
-```
+## MCP integration
+
+`terrain mcp` exposes the last analyze run to AI coding assistants (Claude Code, Cursor, others) over the [Model Context Protocol](https://modelcontextprotocol.io). The assistant can query findings, drill into surfaces, and read baselines without you copy-pasting JSON — so "why is this PR failing Terrain?" gets a useful answer in the IDE instead of a context-switch to the terminal.
 
-Exit code 0 = pass, 2 = violations found, 1 = error.
+The MCP server is local and read-only. The assistant client decides what context, if any, is sent to its model provider.
+
+## Plugins
+
+Third-party detectors ship as YAML manifests; `terrain plugins manifest <path>` validates one against the stable schema. The plugin runtime — the loader that executes registered detectors — is reserved for a future release. Adopters can author and publish manifests now and they'll be loadable when the runtime ships. See [`examples/plugins/example-manifest.yaml`](examples/plugins/example-manifest.yaml).
 
 ## Documentation
 
-- [Quickstart Guide](docs/quickstart.md) — understand your first report in 5 minutes
-- [CLI Specification](docs/cli-spec.md) — full command and flag reference
-- [Example Reports](docs/examples/) — analyze, impact, insights, explain output samples
-- [Canonical User Journeys](docs/product/canonical-user-journeys.md) — primary workflows and expected outcomes
-- [Signal Model](docs/signal-model.md) — the core signal abstraction
-- [Architecture](docs/architecture/) — design documents and technical specifications
-- [Contributing](CONTRIBUTING.md) — how to build, test, and extend Terrain
+**Get started**
 
-## Development
+- [Quickstart](docs/quickstart.md) — first report in 5 minutes
+- [CLI specification](docs/cli-spec.md) — full command + flag reference
+- [Example reports](docs/examples/) — analyze / impact / insights / explain samples
 
-```bash
-# Build
-go build -o terrain ./cmd/terrain
+**Reference**
 
-# Test
-go test ./cmd/... ./internal/...
+- [Design](DESIGN.md) — architecture, package map, signal pipeline
+- [Severity rubric](docs/severity-rubric.md) — severity labels and configuration
+- [Compatibility](docs/compatibility.md) — supported OSes, Go versions, frameworks, schemas
+- [Glossary](docs/glossary.md) — Terrain-specific vocabulary
+- [Versioning](docs/versioning.md) — what counts as a breaking change
 
-# Full release verification
-make release-verify
-```
+**Project**
+
+- [CHANGELOG](CHANGELOG.md) — release history
+- [Security](SECURITY.md) — supported versions + vulnerability disclosure
+- [Contributing](CONTRIBUTING.md) — how to build, test, and extend Terrain
 
 ## License
 
-Apache License 2.0 — see [LICENSE](LICENSE) for details.
+Apache License 2.0 — see [LICENSE](LICENSE).

package/SECURITY.md

@@ -2,7 +2,7 @@
 
 ## Supported Versions
 
-Security updates are provided for the latest published major version of Terrain, covering both the Go CLI (`terrain`) and the JavaScript converter package.
+Security updates are provided for the latest published major version of Terrain (the `terrain` Go CLI, distributed via Homebrew, npm, and direct binary).
 
 ## Reporting a Vulnerability
 

package/bin/postinstall.js

@@ -1,12 +1,42 @@
 #!/usr/bin/env node
 
-import { ensureTerrainBinary } from './terrain-installer.js';
+import {
+  ensureTerrainBinary,
+  writeInstallFailureMarker,
+  clearInstallFailureMarker,
+} from './terrain-installer.js';
 
+// We intentionally don't fail npm install when the binary fetch fails —
+// CI pipelines that run `npm install` as part of a larger flow can
+// recover from a transient download issue, and forcing every cosign-
+// missing host to fail the install would be more disruptive than the
+// failure mode itself. But a silent warning is also wrong: a missing
+// binary should not be discovered five minutes later when the user
+// runs `terrain analyze` and gets a confusing retry.
+//
+// Compromise: write a marker file describing the failure. The CLI
+// trampoline reads it on first run and prints a clear, framed error
+// pointing at the remediation (install cosign, or set the documented
+// opt-out env var) instead of attempting a silent retry.
 try {
   await ensureTerrainBinary({ quiet: false });
+  // Clean up any stale marker from a previous failed install.
+  await clearInstallFailureMarker();
 } catch (error) {
+  await writeInstallFailureMarker(error);
   process.stderr.write(
-    `[mapterrain] Warning: ${error.message}\n` +
-      '[mapterrain] The `terrain` command will try again on first run.\n'
+    '\n' +
+      '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n' +
+      '! mapterrain: binary install FAILED                              !\n' +
+      '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n' +
+      '\n' +
+      `${error.message}\n` +
+      '\n' +
+      'npm install reports success, but the `terrain` binary is NOT\n' +
+      'installed. Running `terrain` will fail with the same error\n' +
+      'until the underlying issue is resolved.\n' +
+      '\n' +
+      'Marker written to ~/.terrain/install-failure.log\n' +
+      '\n'
   );
 }

package/bin/terrain-installer.js

@@ -1,6 +1,7 @@
 #!/usr/bin/env node
 
 import { execFileSync, spawn } from 'child_process';
+import { createHash } from 'crypto';
 import { createWriteStream, existsSync } from 'fs';
 import fs from 'fs/promises';
 import https from 'https';
@@ -17,8 +18,85 @@ const packageJson = JSON.parse(
 
 const GITHUB_OWNER = 'pmclSF';
 const GITHUB_REPO = 'terrain';
+const DOWNLOAD_TIMEOUT_MS = 120000;
+
+// installFailureMarkerPath returns the path where a failed install
+// records its error. The CLI trampoline checks this before retrying
+// so users see a clear remediation message instead of a confusing
+// retry of the same failure. ~/.terrain is also where local snapshots
+// live, so the location is already a Terrain working directory.
+function installFailureMarkerPath() {
+  return path.join(os.homedir(), '.terrain', 'install-failure.log');
+}
 
-function currentTarget() {
+// writeInstallFailureMarker is called from postinstall.js when
+// `npm install` fails to fetch / verify the binary. It captures the
+// error so the next `terrain` invocation can print it verbatim
+// without attempting another silent retry.
+export async function writeInstallFailureMarker(error) {
+  try {
+    const markerPath = installFailureMarkerPath();
+    await fs.mkdir(path.dirname(markerPath), { recursive: true });
+    const body = JSON.stringify(
+      {
+        timestamp: new Date().toISOString(),
+        message: error?.message ?? String(error),
+        stack: error?.stack ?? null,
+        platform: `${process.platform}/${process.arch}`,
+        version: packageJson.version,
+      },
+      null,
+      2
+    );
+    await fs.writeFile(markerPath, body, 'utf8');
+  } catch (writeErr) {
+    // Failing to write the marker is itself non-fatal; the postinstall
+    // warning has already been printed.
+    process.stderr.write(
+      `[mapterrain] (could not record install-failure marker: ${writeErr.message})\n`
+    );
+  }
+}
+
+// clearInstallFailureMarker removes the marker on a successful
+// install or successful first run. Idempotent.
+export async function clearInstallFailureMarker() {
+  try {
+    await fs.unlink(installFailureMarkerPath());
+  } catch (err) {
+    if (err.code !== 'ENOENT') {
+      // ENOENT is the happy path (no marker existed). Anything else
+      // is unexpected; surface it but don't fail.
+      process.stderr.write(
+        `[mapterrain] (could not clear install-failure marker: ${err.message})\n`
+      );
+    }
+  }
+}
+
+// readInstallFailureMarker returns the recorded error message, or
+// null if no marker exists.
+async function readInstallFailureMarker() {
+  try {
+    const body = await fs.readFile(installFailureMarkerPath(), 'utf8');
+    return JSON.parse(body);
+  } catch (err) {
+    return null;
+  }
+}
+
+const SUPPORTED_TARGETS = new Set([
+  'darwin/amd64',
+  'darwin/arm64',
+  'linux/amd64',
+  'linux/arm64',
+  'windows/amd64',
+]);
+
+export function targetForPlatform(
+  platform = process.platform,
+  arch = process.arch
+) {
   const goosMap = {
     darwin: 'darwin',
     linux: 'linux',
@@ -29,12 +107,20 @@ function currentTarget() {
     arm64: 'arm64',
   };
 
-  const goos = goosMap[process.platform];
-  const goarch = goarchMap[process.arch];
+  const goos = goosMap[platform];
+  const goarch = goarchMap[arch];
   if (!goos || !goarch) {
     throw new Error(
-      `Unsupported platform ${process.platform}/${process.arch}. ` +
-        'Install Terrain manually from GitHub Releases or via Homebrew.'
+      `Unsupported platform ${platform}/${arch}. ` +
+        'Install Terrain manually from GitHub Releases, Homebrew on macOS/Linux, or source.'
+    );
+  }
+  if (!SUPPORTED_TARGETS.has(`${goos}/${goarch}`)) {
+    throw new Error(
+      `Unsupported prebuilt Terrain target ${goos}/${goarch}. ` +
+        'Published npm-install binaries are available for ' +
+        `${Array.from(SUPPORTED_TARGETS).sort().join(', ')}. ` +
+        'Install Terrain manually from source with `go install github.com/pmclSF/terrain/cmd/terrain@latest`.'
     );
   }
 
@@ -46,6 +132,10 @@ function currentTarget() {
   };
 }
 
+function currentTarget() {
+  return targetForPlatform();
+}
+
 function isDevelopmentCheckout(rootDir = packageRoot) {
   return existsSync(path.join(rootDir, 'cmd', 'terrain'));
 }
@@ -71,11 +161,223 @@ function archiveFileName(version) {
   return `terrain_${version}_${target.goos}_${target.goarch}.${target.archiveExt}`;
 }
 
-function archiveDownloadUrl(version) {
+function releaseDownloadBaseUrl(version) {
   const baseUrl =
     process.env.TERRAIN_INSTALLER_BASE_URL ||
     `https://github.com/${GITHUB_OWNER}/${GITHUB_REPO}/releases/download`;
-  return `${baseUrl}/v${version}/${archiveFileName(version)}`;
+  return `${baseUrl}/v${version}`;
+}
+
+function archiveDownloadUrl(version) {
+  return `${releaseDownloadBaseUrl(version)}/${archiveFileName(version)}`;
+}
+
+function signatureDownloadUrl(version) {
+  return `${archiveDownloadUrl(version)}.sig`;
+}
+
+function certificateDownloadUrl(version) {
+  return `${archiveDownloadUrl(version)}.pem`;
+}
+
+function checksumDownloadUrl(version) {
+  return `${releaseDownloadBaseUrl(version)}/checksums.txt`;
+}
+
+function expectedSignerIdentity(version) {
+  // The keyless Sigstore signature is anchored to the GitHub Actions workflow
+  // that ran goreleaser at release time. The workflow runs on the v<version>
+  // tag, so the OIDC subject identity is deterministic.
+  return (
+    `https://github.com/${GITHUB_OWNER}/${GITHUB_REPO}` +
+    `/.github/workflows/release.yml@refs/tags/v${version}`
+  );
+}
+
+const SIGSTORE_OIDC_ISSUER = 'https://token.actions.githubusercontent.com';
+
+function isCosignAvailable() {
+  try {
+    execFileSync('cosign', ['version'], { stdio: 'pipe' });
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+export function checksumFromManifest(manifestText, archiveName) {
+  for (const rawLine of manifestText.split(/\r?\n/)) {
+    const line = rawLine.trim();
+    if (!line) {
+      continue;
+    }
+    const match = line.match(/^([a-fA-F0-9]{64})\s+(.+)$/);
+    if (!match) {
+      continue;
+    }
+    const candidate = path.basename(match[2].replace(/^\*/, '').trim());
+    if (candidate === archiveName) {
+      return match[1].toLowerCase();
+    }
+  }
+  return null;
+}
+
+export function verifyChecksumDigest(manifestText, archiveName, actualDigest) {
+  const expected = checksumFromManifest(manifestText, archiveName);
+  if (!expected) {
+    throw new Error(
+      `checksums.txt did not contain an entry for ${archiveName}`
+    );
+  }
+  const actual = actualDigest.toLowerCase();
+  if (actual !== expected) {
+    throw new Error(
+      `checksum mismatch for ${archiveName}: expected ${expected}, got ${actual}`
+    );
+  }
+  return expected;
+}
+
+async function sha256File(filePath) {
+  const hash = createHash('sha256');
+  hash.update(await fs.readFile(filePath));
+  return hash.digest('hex');
+}
+
+async function verifyChecksum({ archivePath, version, tempDir, quiet }) {
+  const checksumsPath = path.join(tempDir, 'checksums.txt');
+  await downloadFile(checksumDownloadUrl(version), checksumsPath);
+
+  const archiveName = path.basename(archivePath);
+  const manifest = await fs.readFile(checksumsPath, 'utf8');
+  const actual = await sha256File(archivePath);
+  verifyChecksumDigest(manifest, archiveName, actual);
+
+  log(`Checksum verified for ${archiveName}`, quiet);
+  return { verified: true, reason: 'checksum' };
+}
+
+// Sigstore signature verification.
+//
+// 0.3.x policy: Sigstore verification is MANDATORY by default. If
+// `cosign` is not available on the host, the install fails with a
+// clear remediation pointer. The escape for trusted/CI/air-gapped
+// environments is the documented opt-out
+// `TERRAIN_INSTALLER_SKIP_VERIFY=1`.
+//
+// Earlier revisions silently degraded to "checksum-only" when cosign
+// was missing, which meant a typical npm-install on a host without
+// cosign (most macOS / Linux dev machines) skipped Sigstore entirely
+// without any signal in the install log beyond a one-line "falling
+// back" message: the strong-integrity guarantee we advertise degraded
+// silently to weak by default. Mandatory verification closes the gap;
+// the env-var escape keeps adoption viable.
+//
+// Escape hatches:
+//
+//   - TERRAIN_INSTALLER_SKIP_VERIFY=1 — fully opt out (CI / air-gapped).
+//     Prints a WARNING so the bypass is auditable.
+//   - TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 — opt-in degrade-to-
+//     checksum behavior for hosts that genuinely cannot install
+//     cosign.
+//
+// Once cosign is on the host, every verify failure is a hard error.
+async function verifySignatureBestEffort({
+  archivePath,
+  version,
+  tempDir,
+  quiet,
+  env,
+}) {
+  if (env.TERRAIN_INSTALLER_SKIP_VERIFY === '1') {
+    log(
+      'WARNING: signature verification skipped (TERRAIN_INSTALLER_SKIP_VERIFY=1). ' +
+        'Set this only in trusted CI / air-gapped environments where ' +
+        'integrity is established by another channel.',
+      quiet
+    );
+    return { verified: false, reason: 'skipped-by-env' };
+  }
+
+  if (!isCosignAvailable()) {
+    if (env.TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN === '1') {
+      log(
+        'cosign not found on PATH. Continuing with checksum-only verification ' +
+          'because TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 is set. ' +
+          'For stronger integrity guarantees install cosign ' +
+          '(https://github.com/sigstore/cosign) and reinstall.',
+        quiet
+      );
+      return await verifyChecksum({ archivePath, version, tempDir, quiet });
+    }
+    throw new Error(
+      'cosign is required to verify the Sigstore signature on the Terrain ' +
+        'release archive, but was not found on PATH.\n\n' +
+        'Resolve by one of:\n' +
+        '  1. Install cosign: https://github.com/sigstore/cosign#installation\n' +
+        '     (Homebrew: `brew install cosign`. Linux: see release notes.)\n' +
+        '  2. If this host genuinely cannot install cosign and you trust the ' +
+        'GitHub-provided checksum file, set ' +
+        'TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 to fall back to ' +
+        'checksum-only verification.\n' +
+        '  3. To skip integrity verification entirely (NOT recommended), ' +
+        'set TERRAIN_INSTALLER_SKIP_VERIFY=1.'
+    );
+  }
+
+  const sigPath = path.join(tempDir, `${path.basename(archivePath)}.sig`);
+  const certPath = path.join(tempDir, `${path.basename(archivePath)}.pem`);
+
+  try {
+    await downloadFile(signatureDownloadUrl(version), sigPath);
+    await downloadFile(certificateDownloadUrl(version), certPath);
+  } catch (error) {
+    // Hard error in 0.3: if cosign is present, the signature download
+    // is required. The release pipeline produces signatures for every
+    // archive; their absence is a real failure mode worth surfacing.
+    throw new Error(
+      `cosign is installed but the Sigstore signature artifacts for ` +
+        `terrain ${version} could not be downloaded: ${error.message}. ` +
+        `Set TERRAIN_INSTALLER_SKIP_VERIFY=1 to bypass at your own risk.`
+    );
+  }
+
+  try {
+    execFileSync(
+      'cosign',
+      [
+        'verify-blob',
+        '--certificate',
+        certPath,
+        '--signature',
+        sigPath,
+        '--certificate-identity',
+        expectedSignerIdentity(version),
+        '--certificate-oidc-issuer',
+        SIGSTORE_OIDC_ISSUER,
+        archivePath,
+      ],
+      { stdio: 'pipe' }
+    );
+    log(
+      `Verified Sigstore signature for ${path.basename(archivePath)}.`,
+      quiet
+    );
+    return { verified: true, reason: 'ok' };
+  } catch (error) {
+    // Hard error in 0.3: a verify-blob failure means the archive on disk
+    // does NOT match the signed certificate. Aborting the install is
+    // strictly safer than silently continuing.
+    const detail = error.stderr
+      ? error.stderr.toString().trim()
+      : error.message;
+    throw new Error(
+      `cosign verify-blob FAILED for ${path.basename(archivePath)}: ${detail}. ` +
+        `The downloaded archive does not match its Sigstore signature; ` +
+        `the binary may have been tampered with. Install aborted.`
+    );
+  }
 }
 
 async function ensureDirectory(dir) {
@@ -96,7 +398,18 @@ function log(message, quiet = false) {
   }
 }
 
-async function downloadFile(url, destinationPath) {
+// MAX_REDIRECTS caps redirect chains to defend against misconfigured
+// proxies that loop. 5 covers every normal redirect chain (GitHub
+// release → CDN → storage backend) with margin to spare. Without this
+// cap the recursion is unbounded — a redirect loop hangs the installer
+// until the OS kills it.
+const MAX_REDIRECTS = 5;
+
+async function downloadFile(
+  url,
+  destinationPath,
+  redirectsRemaining = MAX_REDIRECTS
+) {
   await new Promise((resolve, reject) => {
     const request = https.get(
       url,
@@ -113,8 +426,22 @@ async function downloadFile(url, destinationPath) {
           response.headers.location
         ) {
           response.resume();
+          if (redirectsRemaining <= 0) {
+            reject(
+              new Error(
+                `download exceeded ${MAX_REDIRECTS} redirects for ${url}; ` +
+                  'check for proxy redirect loops or set ' +
+                  'TERRAIN_INSTALLER_BASE_URL to a direct download host.'
+              )
+            );
+            return;
+          }
           try {
-            await downloadFile(response.headers.location, destinationPath);
+            await downloadFile(
+              response.headers.location,
+              destinationPath,
+              redirectsRemaining - 1
+            );
             resolve();
           } catch (error) {
             reject(error);
@@ -141,6 +468,13 @@ async function downloadFile(url, destinationPath) {
       }
     );
 
+    request.setTimeout(DOWNLOAD_TIMEOUT_MS, () => {
+      request.destroy(
+        new Error(
+          `download timed out after ${DOWNLOAD_TIMEOUT_MS / 1000}s for ${url}`
+        )
+      );
+    });
     request.on('error', reject);
   });
 }
@@ -233,6 +567,13 @@ export async function ensureTerrainBinary({
       quiet
     );
     await downloadFile(archiveDownloadUrl(version), archivePath);
+    await verifySignatureBestEffort({
+      archivePath,
+      version,
+      tempDir,
+      quiet,
+      env,
+    });
     await ensureDirectory(extractDir);
     extractArchive(archivePath, extractDir);
 
@@ -268,6 +609,25 @@ export async function runTerrainCli(argv = process.argv.slice(2)) {
     return;
   }
 
+  // Check for a recorded install failure before attempting a silent
+  // retry. If `npm install` failed to fetch/verify the binary, the
+  // marker file records the original error; surface it verbatim
+  // instead of pretending nothing happened.
+  const marker = await readInstallFailureMarker();
+  if (marker && !existsSync(installedBinaryPath(rootDir))) {
+    throw new Error(
+      'Terrain binary is not installed.\n\n' +
+        `Recorded install failure (${marker.timestamp}, ${marker.platform}, v${marker.version}):\n` +
+        `  ${marker.message}\n\n` +
+        'Resolve the underlying issue, then either:\n' +
+        '  - Re-run `npm install -g mapterrain` after installing cosign\n' +
+        '  - Set TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 to fall back to\n' +
+        '    checksum-only verification, or\n' +
+        '  - Set TERRAIN_INSTALLER_SKIP_VERIFY=1 to skip verification entirely.\n\n' +
+        'Marker file: ~/.terrain/install-failure.log'
+    );
+  }
+
   let binaryPath;
   try {
     binaryPath = await ensureTerrainBinary({ rootDir });
@@ -287,6 +647,9 @@ export async function runTerrainCli(argv = process.argv.slice(2)) {
     );
   }
 
+  // First successful run after a failed install: clear the marker.
+  await clearInstallFailureMarker();
+
   await runBinary(binaryPath, argv);
 }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mapterrain",
-  "version": "0.1.1",
+  "version": "0.3.0",
   "description": "Terrain test intelligence CLI.",
   "type": "module",
   "bin": {
@@ -23,10 +23,11 @@
   },
   "scripts": {
     "postinstall": "node bin/postinstall.js",
-    "test": "node scripts/verify-pack.js",
-    "lint": "eslint \"bin/*.js\" \"scripts/*.js\"",
-    "format": "prettier --write \"bin/*.js\" \"scripts/*.js\"",
-    "format:check": "prettier --check \"bin/*.js\" \"scripts/*.js\"",
+    "test": "npm run test:unit && node scripts/verify-pack.js",
+    "test:unit": "node --test scripts/test-installer-marker.mjs scripts/test-release-sanity.mjs",
+    "lint": "eslint \"bin/*.js\" \"scripts/*.js\" \"scripts/*.mjs\"",
+    "format": "prettier --write \"bin/*.js\" \"scripts/*.js\" \"scripts/*.mjs\"",
+    "format:check": "prettier --check \"bin/*.js\" \"scripts/*.js\" \"scripts/*.mjs\"",
     "start": "node bin/terrain-cli.js",
     "convert": "node bin/terrain-cli.js convert",
     "lint-staged": "lint-staged",
@@ -81,4 +82,4 @@
     "node": ">=22.0.0"
   },
   "sideEffects": false
-}
+}