mapterrain 0.1.2 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,482 +1,203 @@
1
1
  # Terrain
2
2
 
3
- **Map your test terrain.** Understand your test system in 30 seconds.
3
+ **Open-source CI pre-flight layer for AI/ML systems and the tests around them. Runs locally. No API key required.**
4
4
 
5
- ```bash
6
- # Homebrew
7
- brew install pmclSF/terrain/mapterrain
8
-
9
- # npm
10
- npm install -g mapterrain
11
-
12
- cd your-repo
13
- terrain analyze
14
- ```
15
-
16
- That's it. No config, no setup, no test execution required.
5
+ Rename `customer_name` to `full_name`, and a prompt in another part of the repo may still ask the model for `customer_name`. Change a retriever, and the eval that would catch the quality drop may never run. Terrain catches those gaps in the PR, before the change merges.
17
6
 
18
- > **New here?** Read the [Quickstart Guide](docs/quickstart.md) to understand your first report in 5 minutes.
7
+ Terrain does that by connecting source code, tests, prompts, schemas, eval outputs, runtime artifacts, and coverage reports into one CI-ready protection graph. It turns that graph into ranked findings, explainable impact, JUnit test cases, GitHub annotations, SARIF for security findings, MCP context for AI assistants, and portfolio rollups across repos.
19
8
 
20
- ---
9
+ ## Why install Terrain?
21
10
 
22
- Terrain is a test system intelligence platform. It reads your repository test code, source structure, coverage data, runtime artifacts, ownership files, and local policy and builds a structural model of how your tests relate to your code. From that model it surfaces risk, quality gaps, redundancy, fragile dependencies, and migration readiness, all without running a single test.
11
+ Most teams already have tests, coverage, and AI evals. The hard part is knowing whether those signals still protect the change in front of you. A developer can see that unit tests passed or that an eval failed, but still not know whether a PR touched an AI/ML surface, which validations should run, what coverage is missing, or why a downstream regression is connected to their diff.
23
12
 
24
- The core idea: every codebase has a *test terrain* — the shape of its testing infrastructure, the density of coverage across areas, the hidden fault lines where a fixture change cascades into thousands of tests. Terrain makes that shape visible and navigable so you can make informed decisions about what to test, what to fix, and where to invest.
13
+ Terrain gives every PR a pre-merge protection map. In one local command it answers:
25
14
 
26
- ## What "Test Terrain" Means
15
+ - What is weak or missing in the current test system?
16
+ - What does this PR put at risk?
17
+ - Which tests or evals matter for this change?
18
+ - Why did the gate fail, and what should be fixed first?
27
19
 
28
- Most teams know what tests they have. Few teams understand the *terrain* underneath:
20
+ That makes day-to-day work less manual:
29
21
 
30
- - Which source files have no structural test coverage?
31
- - Which shared fixtures fan out to thousands of tests, making any change to them a blast-radius problem?
32
- - Which test clusters are near-duplicates burning CI time?
33
- - Which areas have tests but weak assertions that wouldn't catch a real regression?
34
- - When you change `auth/session.ts`, which 41 of your 18,000 tests actually matter?
22
+ - Instead of tracing code -> API -> prompt -> eval by hand, run `terrain report pr` and see what the change affects.
23
+ - Instead of rerunning every eval because no one knows which one matters, run the validations Terrain selects for the diff.
24
+ - Instead of reading a failed eval in isolation, use the cause path to see which code, schema, prompt, or retrieval change led there.
25
+ - Instead of letting weak assertions, skipped tests, and uncovered exports accumulate quietly, surface them as concrete findings before release.
35
26
 
36
- Test terrain is the structural topology of your test system the dependency graph, the coverage landscape, the duplication clusters, the fanout hotspots, the skip debt. Terrain maps it.
27
+ The result is faster review, fewer broad reruns, clearer fixes, and fewer regressions that only make sense after someone reconstructs the system by hand.
37
28
 
38
- ## Problems Terrain Solves
29
+ Bring your own stack. Terrain reads what teams already use: `pytest`, `jest`, `go test`, Playwright, Promptfoo, DeepEval, Ragas, Great Expectations, Gauntlet-style eval-result JSON, JUnit, LCOV, Istanbul, and repository metadata. It does not replace those tools; it unifies their evidence into one local, deterministic CI gate.
39
30
 
40
- **"We don't know the state of our test system."** Teams inherit test suites they didn't write. Terrain gives a baseline in seconds: framework mix, coverage confidence, duplication, risk posture.
31
+ Project status: Terrain is pre-1.0 and actively developed. In 0.3.0, the stable path is the CLI, local/CI artifact contract, GitHub Actions flow, AI/eval artifact ingestion, and portfolio aggregation. The VS Code extension is alpha; marketplace listings, full LSP integration, the plugin runtime, and some preview rules are future work.
41
32
 
42
- **"CI takes too long and we don't know what to cut."** Terrain identifies redundant tests, high-fanout fixtures, and confidence-based test selection — showing where CI time is wasted and what can be safely reduced.
43
-
44
- **"We changed auth code — what tests should we worry about?"** Terrain traces your change through the import graph and structural dependencies, returning the impacted tests ranked by confidence, with reason chains explaining each selection.
45
-
46
- **"A tool flagged something but won't explain why."** Every Terrain finding carries an evidence chain. `terrain explain` shows exactly what signals, dependency paths, and scoring rules produced each decision.
47
-
48
- **"We're migrating frameworks and need to know what's blocking us."** Migration readiness, blockers by type, and preview-scoped difficulty assessment — all derived from static analysis.
49
-
50
- ## The Four Canonical Workflows
51
-
52
- Terrain is organized around four questions. Everything else is a supporting view.
53
-
54
- ```
55
- terrain analyze "What is the state of our test system?"
56
- terrain insights "What should we fix in our test system?"
57
- terrain impact "What validations matter for this change?"
58
- terrain explain "Why did Terrain make this decision?"
59
- ```
60
-
61
- > **About the example outputs below.** The CLI dumps in this section illustrate the *shape* of Terrain's reports on a large pandas-style repository — they are not literal output from a single live run. A few specific signals shown (`xfailAccumulation` age, statistical flaky-test failure rates, the `0.91+` duplicate similarity threshold) are marked `[experimental]` or `[planned]` in 0.1.2; see [docs/release/feature-status.md](docs/release/feature-status.md) for what's stable, what's experimental, and what's planned. The headline "30 seconds" promise refers to small-to-medium repos (≤ 1,000 test files) on commodity hardware; expect 5–15 seconds on a typical service repo and longer on monorepos.
62
-
63
- ### 1. Analyze — understand the test system
64
-
65
- ```bash
66
- terrain analyze
67
- ```
68
-
69
- ```
70
- Terrain — Test Suite Analysis
71
- ============================================================
72
-
73
- conftest.py fixture fans out to 3,100 tests — any change retriggers the frame/ suite.
74
-
75
- Key Findings
76
- ------------------------------------------------------------
77
- 1. [HIGH] 23 source files (18%) have low structural coverage
78
- 2. [HIGH] 8 duplicate test clusters with 0.91+ similarity
79
- 3. [MEDIUM] 34 xfail markers older than 180 days
80
-
81
- Repository Profile
82
- ------------------------------------------------------------
83
- Test volume: very large
84
- CI pressure: high
85
- Coverage confidence: medium
86
- Redundancy level: medium
87
- Fanout burden: high
88
-
89
- Validation Inventory
90
- ------------------------------------------------------------
91
- Test files: 1,047
92
- Test cases: 52,341
93
- Frameworks:
94
- pytest 1,047 files [unit]
95
-
96
- Risk Posture
97
- ------------------------------------------------------------
98
- health: MODERATE
99
- coverage_depth: ELEVATED
100
- coverage_diversity: STRONG
101
- structural_risk: STRONG
102
- operational_risk: STRONG
103
-
104
- Next steps:
105
- terrain insights prioritized actions and recommendations
106
- terrain impact what tests matter for this change?
107
- ```
108
-
109
- ### 2. Insights — find what to improve
110
-
111
- ```bash
112
- terrain insights
113
- ```
114
-
115
- ```
116
- Terrain — Test System Health Report
117
- ============================================================
118
-
119
- Health Grade: C
120
-
121
- Reliability Problems (2)
122
- [HIGH] 12 flaky tests with >10% failure rate
123
- [MEDIUM] 34 skipped tests consuming CI resources
124
-
125
- Coverage Debt (2)
126
- [HIGH] 23 source files (18%) have low structural coverage
127
- [MEDIUM] conftest.py fixture fans out to 3,100 tests
128
-
129
- Recommended Actions
130
- 1. [reliability] Quarantine 12 flaky tests
131
- why: Flaky tests block unrelated PRs and erode CI trust.
132
- 2. [coverage] Add tests for 23 uncovered source files
133
- why: Changes in uncovered files cannot trigger test selection.
134
- 3. [optimization] Consolidate 8 duplicate test clusters
135
- why: 0.91+ similarity — redundant CI cost with no coverage benefit.
136
- ```
137
-
138
- ### 3. Impact — understand what a change affects
139
-
140
- ```bash
141
- terrain impact --base main
142
- ```
143
-
144
- ```
145
- Terrain Impact Analysis
146
- ============================================================
147
-
148
- Summary: 3 file(s) changed, 41 test(s) relevant. Posture: needs_attention.
149
-
150
- Impacted tests: 127 of 52,341 total
151
- Coverage confidence: High
152
-
153
- Recommended Tests (41)
154
- ------------------------------------------------------------
155
- tests/groupby/test_groupby.py [exact]
156
- Covers: groupby.py:GroupBy.aggregate
157
- tests/groupby/test_apply.py [exact]
158
- Covers: groupby.py:GroupBy.apply
159
- tests/resample/test_base.py [inferred]
160
- Reached via shared fixture path
161
- ...and 38 more
162
-
163
- Protection Gaps
164
- ------------------------------------------------------------
165
- [medium] pandas/core/groupby/ops.py — no covering tests found
166
-
167
- Next steps:
168
- terrain impact --show tests full test list
169
- terrain impact --show gaps all protection gaps
170
- ```
171
-
172
- ### 4. Explain — understand why
173
-
174
- ```bash
175
- terrain explain tests/io/json/test_pandas.py
176
- ```
177
-
178
- ```
179
- Test File: tests/io/json/test_pandas.py
180
- Framework: pytest
181
- Tests: 84 Assertions: 312
182
-
183
- Signals (3):
184
- [high] networkDependency: 12 tests use @pytest.mark.network — flaky in CI
185
- [medium] weakAssertion: 8 bare assert statements without descriptive messages
186
- [low] xfailAccumulation: 3 xfail markers older than 180 days
187
-
188
- Next steps:
189
- terrain explain selection explain overall test selection strategy
190
- terrain impact --show tests see all impacted tests
191
- ```
192
-
193
- See [Canonical User Journeys](docs/product/canonical-user-journeys.md) for the full workflow specification and [example outputs](docs/examples/) for detailed report samples.
194
-
195
- ## Product Philosophy
196
-
197
- **Inference first.** Terrain reads code. It parses imports, detects frameworks, resolves coverage relationships, and builds dependency graphs from what already exists in the repository. No annotations, no test tagging, no SDK integration required.
198
-
199
- **Zero-config by default.** `terrain analyze` works on any repository with test files. Coverage data, runtime artifacts, ownership files, and policy rules are optional inputs that enrich the model — but the core analysis requires nothing beyond the code itself.
200
-
201
- **Explainability over magic.** Every finding carries evidence: which signal type, what confidence level, what dependency path, what scoring rule. `terrain explain` exposes the full reasoning chain behind any decision. Teams should never wonder *why* Terrain said something.
202
-
203
- **Conservative under uncertainty.** When Terrain encounters ambiguity — a dependency path with low confidence, a file that might or might not be a test — it flags the uncertainty rather than guessing. Impact analysis uses fallback policies with explicit confidence penalties rather than silently expanding scope.
204
-
205
- **System health, not individual productivity.** Terrain measures the test system. It never attributes quality to individual developers. Ownership information is used for routing and triage, not scoring.
206
-
207
- ## Who Uses Terrain
208
-
209
- Terrain is framework-agnostic and language-aware. The same analysis model applies across:
210
-
211
- - **Frontend teams** — React/Vue component tests, Playwright/Cypress E2E suites, Vitest/Jest unit tests
212
- - **Backend teams** — Go test suites, pytest collections, JUnit hierarchies, integration test infrastructure
213
- - **Mobile teams** — cross-platform test suites with standard test frameworks
214
- - **QA / SDET** — test portfolio management, coverage gap analysis, migration planning across frameworks
215
- - **SRE / Platform** — CI optimization, test selection for pipelines, policy enforcement
216
- - **AI / ML evaluation** — evaluation suite structure, benchmark test management, coverage across model behaviors
217
-
218
- The structural model is the same. The signals and recommendations adapt to the framework and test patterns detected.
219
-
220
- ## AI Testing in CI
221
-
222
- Terrain gives AI components the same CI safety net as regular tests:
223
-
224
- - **Surface discovery** — automatically detects prompts, contexts, datasets, tool definitions, RAG pipelines, and eval scenarios in your code
225
- - **Impact-scoped selection** — `terrain ai run --base main` runs only the eval scenarios affected by your change
226
- - **Protection gaps** — `terrain pr` flags changed AI surfaces that have no eval scenario covering them
227
- - **Policy enforcement** — block PRs that modify uncovered AI surfaces, regress accuracy, or trigger safety failures
228
- - **GitHub Action** — drop-in `terrain-ai.yml` workflow template for AI CI gates
229
-
230
- The same structural graph that powers test selection for regular code also traces AI surface dependencies, so a change to a prompt template triggers the right eval scenarios automatically.
231
-
232
- ## How CI Optimization Emerges
233
-
234
- Terrain does not start with CI optimization. It starts with understanding.
235
-
236
- When you run `terrain analyze`, Terrain builds a structural model: which tests exist, which source files they cover, how they depend on shared fixtures, where duplication lives. From that model, CI optimization *emerges*:
237
-
238
- - **Test selection** — `terrain impact` traces a diff through the dependency graph and returns only the tests that structurally matter, ranked by confidence. This is not a heuristic skip list — it is a graph traversal with evidence.
239
- - **Redundancy reduction** — `terrain insights` surfaces duplicate test clusters. Removing or consolidating them directly reduces CI time without reducing coverage.
240
- - **Fanout control** — High-fanout fixtures that trigger thousands of tests on any change are identified and prioritized for splitting.
241
- - **Confidence-based runs** — Impact analysis assigns confidence scores. CI pipelines can run high-confidence tests immediately and defer low-confidence tests to nightly runs.
242
- - **What to test next** — `terrain insights` ranks untested source files by dependency count, telling you which test to write first for maximum impact.
243
-
244
- The result is faster CI that comes from *understanding the test system*, not from skipping tests and hoping for the best.
245
-
246
- ## Installation
247
-
248
- ### Homebrew (macOS and Linux)
33
+ ## Install
249
34
 
250
35
  ```bash
36
+ # macOS / Linux
251
37
  brew install pmclSF/terrain/mapterrain
252
- ```
253
-
254
- After the first install, you can also tap once and use the short formula name:
255
-
256
- ```bash
257
- brew tap pmclSF/terrain
258
- brew install mapterrain
259
- ```
260
38
 
261
- ### npm
262
-
263
- ```bash
39
+ # npm (Node 22+ required; macOS/Linux amd64+arm64, Windows amd64)
264
40
  npm install -g mapterrain
265
- ```
266
-
267
- ### Go install
268
41
 
269
- ```bash
42
+ # Go
270
43
  go install github.com/pmclSF/terrain/cmd/terrain@latest
271
44
  ```
272
45
 
273
- Requires Go 1.23 or later.
46
+ Pre-built archives are available for macOS and Linux on amd64/arm64, plus Windows on amd64, from the [releases page](https://github.com/pmclSF/terrain/releases). Each release is signed with Sigstore + cosign; see [SECURITY-DATA-HANDLING.md](SECURITY-DATA-HANDLING.md) for verification details.
274
47
 
275
- ### Pre-built binaries
48
+ Package names vary by distribution, but the installed CLI is always `terrain`. The npm package is `mapterrain`; the Homebrew formula is `pmclSF/terrain/mapterrain`.
276
49
 
277
- Download the appropriate binary for your platform from [GitHub Releases](https://github.com/pmclSF/terrain/releases), then:
50
+ ## Get started
278
51
 
279
52
  ```bash
280
- chmod +x terrain
281
- sudo mv terrain /usr/local/bin/
53
+ cd your-repo
54
+ terrain analyze # What's the state of our AI + test system?
55
+ terrain report pr # What does this change put at risk?
282
56
  ```
283
57
 
284
- Binaries are available for macOS, Linux, and Windows (amd64 and arm64).
285
-
286
- ### Build from source
58
+ Source analysis covers Python, TypeScript/JavaScript, Go, and Java in 0.3.0. Ruby source is not analyzed in 0.3.0, but Ruby/RSpec and other ecosystems can still contribute dependency, runtime, coverage, and eval artifacts, so mixed-language repos get useful signal. No config required; optional artifacts sharpen findings when present.
287
59
 
288
- ```bash
289
- git clone https://github.com/pmclSF/terrain.git
290
- cd terrain
291
- go build -o terrain ./cmd/terrain
292
- ```
60
+ ## What it catches
293
61
 
294
- ### Verify installation
62
+ Terrain models the AI surface alongside the test surface and looks for drift across them:
295
63
 
296
- ```bash
297
- terrain --version
298
- ```
64
+ - **Prompt-schema drift** — prompts that reference fields renamed in a schema living in a different language
65
+ - **Hardcoded API keys** — provider-shaped secrets in source (OpenAI, Anthropic, AWS, GCP, etc.)
66
+ - **Eval coverage gaps** — AI surfaces (prompts, agents, RAG pipelines) with no scenario covering them
67
+ - **Model deprecations** — deprecated model IDs lingering in production paths
68
+ - **Cross-language edges** — TS/JS ↔ Python/Go/Java via OpenAPI, tRPC, gRPC, GraphQL, HTTP routes
69
+ - **Framework-migration blockers** — Jest ↔ Vitest, JUnit 4 ↔ 5, Cypress ↔ Playwright
70
+ - **Portfolio drift across repos** — manifest-backed `terrain portfolio --from` rollups show framework-of-record drift across a polyrepo
71
+ - **Untested exports + weak assertions** — public API surfaces with no covering test; assertions that pass on too much
72
+ - **Fixture fanout + duplicate clusters + skip debt** — structural problems that erode CI signal
299
73
 
300
- ## Quick Start
74
+ Each finding carries a stable rule ID, severity, confidence, evidence, and documented remediation. Run `terrain explain <rule-id>` for the long form.
301
75
 
302
- ```bash
303
- # Detect coverage/runtime data paths (recommended first step)
304
- terrain init
76
+ ## What it looks like
305
77
 
306
- # Analyze the current repository
307
- terrain analyze
78
+ ```
79
+ Terrain · Test Suite Analysis
80
+ ────────────────────────────────────────────────────────────
308
81
 
309
- # JSON output for any command
310
- terrain analyze --json
82
+ conftest.py fixture fans out to 3,100 tests — any change retriggers the frame/ suite.
311
83
 
312
- # See what a change affects
313
- terrain impact --base main
84
+ Key Findings
85
+ ────────────────────────────────────────────────────────────
86
+ 1. [HIGH] 23 exported code units have no linked tests
87
+ 2. [MED] 12 test files have weak assertion density
88
+ 3. [LOW] 7 skipped-test patterns need review
314
89
 
315
- # Get prioritized recommendations
316
- terrain insights
90
+ Risk Posture
91
+ ────────────────────────────────────────────────────────────
92
+ health: Moderate
93
+ coverage_depth: Elevated
94
+ coverage_diversity: Strong
95
+ structural_risk: Strong
96
+ Signals: 65 (8 high, 34 medium, 23 low)
317
97
  ```
318
98
 
319
- ## Commands
99
+ > Representative output from a large pandas-style repository. Format and labels are stable across runs; the specific numbers vary by repo. Full sample reports for `analyze`, `insights`, `impact`, and `explain` are in [docs/examples/](docs/examples/).
320
100
 
321
- ### Primary commands
101
+ ## Workflow
322
102
 
323
103
  | Command | Question |
324
104
  |---------|----------|
325
105
  | `terrain analyze` | What is the state of our test system? |
326
- | `terrain insights` | What should we fix in our test system? |
327
- | `terrain impact` | What validations matter for this change? |
328
- | `terrain explain <target>` | Why did Terrain make this decision? |
329
-
330
- ### Supporting commands
331
-
332
- | Command | Purpose |
333
- |---------|---------|
334
- | `terrain init` | Detect data files and print recommended analyze command |
335
- | `terrain summary` | Executive summary with risk, trends, benchmark readiness |
336
- | `terrain focus` | Prioritized next actions |
337
- | `terrain posture` | Detailed posture breakdown with measurement evidence |
338
- | `terrain portfolio` | Portfolio intelligence: cost, breadth, leverage, redundancy |
339
- | `terrain metrics` | Aggregate metrics scorecard |
340
- | `terrain compare` | Compare two snapshots for trend tracking |
341
- | `terrain select-tests` | Recommend protective test set for a change |
342
- | `terrain pr` | PR/change-scoped analysis |
343
- | `terrain show <type> <id>` | Drill into test, unit, owner, or finding |
344
- | `terrain migration <sub>` | Migration readiness, blockers, or preview |
345
- | `terrain policy check` | Evaluate local policy rules |
346
- | `terrain export benchmark` | Privacy-safe JSON export for benchmarking |
347
- | `terrain serve` | Local HTTP server with HTML report and JSON API |
348
-
349
- ### AI / eval
350
-
351
- | Command | Purpose |
352
- |---------|---------|
353
- | `terrain ai list` | List detected scenarios, prompts, datasets, eval files |
354
- | `terrain ai doctor` | Validate AI/eval setup and surface configuration issues |
355
- | `terrain ai run` | Execute eval scenarios and collect results |
356
- | `terrain ai replay` | Replay and verify a previous eval run artifact |
357
- | `terrain ai record` | Record eval results as a baseline snapshot |
358
- | `terrain ai baseline` | Manage eval baselines (show, compare) |
359
-
360
- ### Conversion / migration
361
-
362
- | Command | Purpose |
363
- |---------|---------|
364
- | `terrain convert <source>` | Go-native test conversion (25 directions) |
365
- | `terrain convert-config <source>` | Convert framework config files |
366
- | `terrain migrate <dir>` | Project-wide migration with state tracking |
367
- | `terrain estimate <dir>` | Estimate migration complexity |
368
- | `terrain status` | Show migration progress |
369
- | `terrain checklist` | Generate migration checklist |
370
- | `terrain doctor [path]` | Run migration diagnostics |
371
- | `terrain reset` | Clear migration state |
372
- | `terrain list-conversions` | List supported conversion directions |
373
- | `terrain shorthands` | List shorthand aliases (e.g., `cy2pw`, `jest2vt`) |
374
- | `terrain detect <file-or-dir>` | Detect dominant framework |
375
-
376
- ### Advanced / debug
377
-
378
- | Command | Purpose |
379
- |---------|---------|
380
- | `terrain debug graph` | Dependency graph statistics |
381
- | `terrain debug coverage` | Structural coverage analysis |
382
- | `terrain debug fanout` | High-fanout node analysis |
383
- | `terrain debug duplicates` | Duplicate test cluster analysis |
384
- | `terrain debug depgraph` | Full dependency graph analysis (all engines) |
385
-
386
- Repository-scoped commands support `--root PATH`, and machine-readable commands support `--json`. Most analysis commands support `--verbose` for additional detail. Run `terrain <command> --help` for full flag documentation.
387
-
388
- ## Architecture Overview
389
-
390
- Terrain is built around a signal-first architecture:
106
+ | `terrain report pr` | What does this change put at risk? |
107
+ | `terrain report insights` | What should we fix? |
108
+ | `terrain report impact` | What validations matter for this change? |
109
+ | `terrain report explain <target>` | Why did Terrain make this decision? |
391
110
 
392
- ```
393
- Repository scan → Signal detection → Risk modeling → Reporting
394
- │ │ │ │
395
- test files framework-specific explainable human-readable
396
- source files pattern detectors risk scoring + JSON output
397
- coverage data quality signals with evidence
398
- runtime artifacts health signals chains
399
- ownership files migration signals
400
- policy rules governance signals
401
- ```
111
+ The bare forms (`terrain insights`, `terrain impact`, `terrain explain`) work as aliases. AI / eval verbs, framework conversion, debug drill-downs, and the slash-receiver round out the surface — full reference in [docs/cli-spec.md](docs/cli-spec.md).
402
112
 
403
- - **Signals** are the core abstraction — every finding is a structured signal with type, severity, confidence, evidence, and location
404
- - **Snapshots** (`TestSuiteSnapshot`) are the canonical serialized artifact — the complete structural model of a test system at a point in time
405
- - **Risk surfaces** are derived from signals with explainable scoring across dimensions (quality, reliability, speed, governance)
406
- - **Dependency graphs** model import relationships, fixture fanout, and structural coverage
407
- - **Reports** synthesize signals, risk, trends, and benchmark readiness into actionable output
113
+ ## CI integration
408
114
 
409
- ```
410
- cmd/terrain/ CLI (30+ commands)
411
- internal/ 47 Go packages covering analysis, signals, risk,
412
- impact, depgraph, measurement, reporting, and more
115
+ ```yaml
116
+ # .github/workflows/terrain.yml
117
+ name: terrain
118
+ on: pull_request
119
+
120
+ jobs:
121
+ analyze:
122
+ runs-on: ubuntu-latest
123
+ steps:
124
+ - uses: actions/checkout@v6
125
+ with: { fetch-depth: 0 }
126
+ - uses: actions/setup-node@v6
127
+ with: { node-version: '22.x' }
128
+ - run: npm install -g mapterrain
129
+ - run: |
130
+ terrain test \
131
+ --junit terrain-results.xml \
132
+ --summary "$GITHUB_STEP_SUMMARY"
413
133
  ```
414
134
 
415
- See [DESIGN.md](DESIGN.md) for the full architecture overview and package map, [docs/architecture/](docs/architecture/) for detailed design documents, and [docs/json-schema.md](docs/json-schema.md) for JSON output structure.
135
+ `terrain test` is the CI-mode wrapper. The JUnit XML lets your CI render Terrain findings as test cases; setting `--summary "$GITHUB_STEP_SUMMARY"` makes them appear on the workflow run page automatically.
416
136
 
417
- ## Snapshot Workflow
137
+ For a blocking gate, add `--fail-on=high` (or `--fail-on=critical` for the strictest threshold). For onboarding a repo with existing debt, pair with `--new-findings-only --baseline <path>` so only regressions block the build. To restrict the gate to AI-related changes, add a `paths:` filter on prompt / schema / Python / TS file globs (see [docs/examples/gate/](docs/examples/gate/) for the full templates).
418
138
 
419
- Terrain supports local snapshot history for trend tracking:
139
+ ## Boundaries
420
140
 
421
- ```bash
422
- # Save a snapshot
423
- terrain analyze --write-snapshot
141
+ Terrain is intentionally a CI/local pre-flight layer. It composes with the tools teams already trust instead of replacing them:
424
142
 
425
- # Later, save another snapshot
426
- terrain analyze --write-snapshot
143
+ - It reads what `pytest`, `jest`, `go test`, Playwright, Promptfoo, DeepEval, Ragas, Great Expectations, and Gauntlet-style eval-result artifacts produce. The AI namespace can execute supported eval-framework commands for `terrain ai run`, but Terrain does not replace those frameworks.
144
+ - It ingests coverage reports when you have them; it does not instrument code.
145
+ - It complements Semgrep / CodeQL / Sonar rather than replacing application-code bug finding.
146
+ - It routes ownership data to findings; it never produces leaderboards or per-developer scores.
147
+ - It runs locally with zero outbound network calls during analysis — verifiable with `terrain --print-network`. The install paths download signed binaries from GitHub Releases; Terrain itself does not upload analysis results.
148
+ - CI artifacts and MCP responses can include repo paths, line numbers, diagnostic text, and sometimes source excerpts. Treat the CI platform or AI assistant you connect as part of the trust boundary.
427
149
 
428
- # Compare the two most recent snapshots
429
- terrain compare
150
+ Under the hood, Terrain combines source analysis, dependency-graph construction, artifact ingestion, and deterministic rule evaluation. The product is the CI gate and protection graph; static analysis is one mechanism inside it.
430
151
 
431
- # Executive summary automatically includes trend highlights
432
- terrain summary
433
- ```
152
+ ## AI-aware testing
434
153
 
435
- Snapshots are stored in `.terrain/snapshots/` as timestamped JSON files.
154
+ The same dependency graph that powers test selection for application code also traces AI surface edges. When Terrain can map a prompt-template, schema, or eval-artifact change to declared or inferred scenarios, it selects the impacted evals automatically.
436
155
 
437
- ## Policy
156
+ - `terrain ai run --base main` — run only the evals your change affects
157
+ - `terrain report pr` — flag changed AI surfaces with no covering eval
158
+ - `terrain ai findings` — AI eval-gap findings with evidence chains
438
159
 
439
- Define local policy rules in `.terrain/policy.yaml`:
160
+ Two generators help adopters harden a prompt before deploying it:
440
161
 
441
- ```yaml
442
- rules:
443
- disallow_skipped_tests: true
444
- max_weak_assertions: 10
445
- max_mock_heavy_tests: 5
162
+ ```bash
163
+ terrain inject --prompt prompts/main.md # generate jailbreak-shaped test inputs
164
+ terrain scaffold --schema schemas/input.json # generate boundary-case mutation tests
446
165
  ```
447
166
 
448
- Then check compliance:
167
+ Both emit runnable pytest / vitest scaffolds you drop into your test tree. Terrain never calls the model; the assertion is yours.
449
168
 
450
- ```bash
451
- terrain policy check # human-readable output
452
- terrain policy check --json # JSON output for CI
453
- ```
169
+ ## MCP integration
170
+
171
+ `terrain mcp` exposes the last analyze run to AI coding assistants (Claude Code, Cursor, others) over the [Model Context Protocol](https://modelcontextprotocol.io). The assistant can query findings, drill into surfaces, and read baselines without you copy-pasting JSON so "why is this PR failing Terrain?" gets a useful answer in the IDE instead of a context-switch to the terminal.
454
172
 
455
- Exit code 0 = pass, 2 = violations found, 1 = error.
173
+ The MCP server is local and read-only. The assistant client decides what context, if any, is sent to its model provider.
174
+
175
+ ## Plugins
176
+
177
+ Third-party detectors ship as YAML manifests; `terrain plugins manifest <path>` validates one against the stable schema. The plugin runtime — the loader that executes registered detectors — is reserved for a future release. Adopters can author and publish manifests now and they'll be loadable when the runtime ships. See [`examples/plugins/example-manifest.yaml`](examples/plugins/example-manifest.yaml).
456
178
 
457
179
  ## Documentation
458
180
 
459
- - [Quickstart Guide](docs/quickstart.md) — understand your first report in 5 minutes
460
- - [CLI Specification](docs/cli-spec.md) — full command and flag reference
461
- - [Example Reports](docs/examples/) — analyze, impact, insights, explain output samples
462
- - [Canonical User Journeys](docs/product/canonical-user-journeys.md) — primary workflows and expected outcomes
463
- - [Signal Model](docs/signal-model.md) — the core signal abstraction
464
- - [Architecture](docs/architecture/) — design documents and technical specifications
465
- - [Contributing](CONTRIBUTING.md) — how to build, test, and extend Terrain
181
+ **Get started**
466
182
 
467
- ## Development
183
+ - [Quickstart](docs/quickstart.md) — first report in 5 minutes
184
+ - [CLI specification](docs/cli-spec.md) — full command + flag reference
185
+ - [Example reports](docs/examples/) — analyze / impact / insights / explain samples
468
186
 
469
- ```bash
470
- # Build
471
- go build -o terrain ./cmd/terrain
187
+ **Reference**
472
188
 
473
- # Test
474
- go test ./cmd/... ./internal/...
189
+ - [Design](DESIGN.md) — architecture, package map, signal pipeline
190
+ - [Severity rubric](docs/severity-rubric.md) — severity labels and configuration
191
+ - [Compatibility](docs/compatibility.md) — supported OSes, Go versions, frameworks, schemas
192
+ - [Glossary](docs/glossary.md) — Terrain-specific vocabulary
193
+ - [Versioning](docs/versioning.md) — what counts as a breaking change
475
194
 
476
- # Full release verification
477
- make release-verify
478
- ```
195
+ **Project**
196
+
197
+ - [CHANGELOG](CHANGELOG.md) — release history
198
+ - [Security](SECURITY.md) — supported versions + vulnerability disclosure
199
+ - [Contributing](CONTRIBUTING.md) — how to build, test, and extend Terrain
479
200
 
480
201
  ## License
481
202
 
482
- Apache License 2.0 — see [LICENSE](LICENSE) for details.
203
+ Apache License 2.0 — see [LICENSE](LICENSE).
package/SECURITY.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## Supported Versions
4
4
 
5
- Security updates are provided for the latest published major version of Terrain, covering both the Go CLI (`terrain`) and the JavaScript converter package.
5
+ Security updates are provided for the latest published major version of Terrain (the `terrain` Go CLI, distributed via Homebrew, npm, and direct binary).
6
6
 
7
7
  ## Reporting a Vulnerability
8
8
 
@@ -1,12 +1,42 @@
1
1
  #!/usr/bin/env node
2
2
 
3
- import { ensureTerrainBinary } from './terrain-installer.js';
3
+ import {
4
+ ensureTerrainBinary,
5
+ writeInstallFailureMarker,
6
+ clearInstallFailureMarker,
7
+ } from './terrain-installer.js';
4
8
 
9
+ // We intentionally don't fail npm install when the binary fetch fails —
10
+ // CI pipelines that run `npm install` as part of a larger flow can
11
+ // recover from a transient download issue, and forcing every cosign-
12
+ // missing host to fail the install would be more disruptive than the
13
+ // failure mode itself. But a silent warning is also wrong: a missing
14
+ // binary should not be discovered five minutes later when the user
15
+ // runs `terrain analyze` and gets a confusing retry.
16
+ //
17
+ // Compromise: write a marker file describing the failure. The CLI
18
+ // trampoline reads it on first run and prints a clear, framed error
19
+ // pointing at the remediation (install cosign, or set the documented
20
+ // opt-out env var) instead of attempting a silent retry.
5
21
  try {
6
22
  await ensureTerrainBinary({ quiet: false });
23
+ // Clean up any stale marker from a previous failed install.
24
+ await clearInstallFailureMarker();
7
25
  } catch (error) {
26
+ await writeInstallFailureMarker(error);
8
27
  process.stderr.write(
9
- `[mapterrain] Warning: ${error.message}\n` +
10
- '[mapterrain] The `terrain` command will try again on first run.\n'
28
+ '\n' +
29
+ '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n' +
30
+ '! mapterrain: binary install FAILED !\n' +
31
+ '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n' +
32
+ '\n' +
33
+ `${error.message}\n` +
34
+ '\n' +
35
+ 'npm install reports success, but the `terrain` binary is NOT\n' +
36
+ 'installed. Running `terrain` will fail with the same error\n' +
37
+ 'until the underlying issue is resolved.\n' +
38
+ '\n' +
39
+ 'Marker written to ~/.terrain/install-failure.log\n' +
40
+ '\n'
11
41
  );
12
42
  }
@@ -1,6 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
 
3
3
  import { execFileSync, spawn } from 'child_process';
4
+ import { createHash } from 'crypto';
4
5
  import { createWriteStream, existsSync } from 'fs';
5
6
  import fs from 'fs/promises';
6
7
  import https from 'https';
@@ -17,8 +18,85 @@ const packageJson = JSON.parse(
17
18
 
18
19
  const GITHUB_OWNER = 'pmclSF';
19
20
  const GITHUB_REPO = 'terrain';
21
+ const DOWNLOAD_TIMEOUT_MS = 120000;
22
+
23
+ // installFailureMarkerPath returns the path where a failed install
24
+ // records its error. The CLI trampoline checks this before retrying
25
+ // so users see a clear remediation message instead of a confusing
26
+ // retry of the same failure. ~/.terrain is also where local snapshots
27
+ // live, so the location is already a Terrain working directory.
28
+ function installFailureMarkerPath() {
29
+ return path.join(os.homedir(), '.terrain', 'install-failure.log');
30
+ }
20
31
 
21
- function currentTarget() {
32
+ // writeInstallFailureMarker is called from postinstall.js when
33
+ // `npm install` fails to fetch / verify the binary. It captures the
34
+ // error so the next `terrain` invocation can print it verbatim
35
+ // without attempting another silent retry.
36
+ export async function writeInstallFailureMarker(error) {
37
+ try {
38
+ const markerPath = installFailureMarkerPath();
39
+ await fs.mkdir(path.dirname(markerPath), { recursive: true });
40
+ const body = JSON.stringify(
41
+ {
42
+ timestamp: new Date().toISOString(),
43
+ message: error?.message ?? String(error),
44
+ stack: error?.stack ?? null,
45
+ platform: `${process.platform}/${process.arch}`,
46
+ version: packageJson.version,
47
+ },
48
+ null,
49
+ 2
50
+ );
51
+ await fs.writeFile(markerPath, body, 'utf8');
52
+ } catch (writeErr) {
53
+ // Failing to write the marker is itself non-fatal; the postinstall
54
+ // warning has already been printed.
55
+ process.stderr.write(
56
+ `[mapterrain] (could not record install-failure marker: ${writeErr.message})\n`
57
+ );
58
+ }
59
+ }
60
+
61
+ // clearInstallFailureMarker removes the marker on a successful
62
+ // install or successful first run. Idempotent.
63
+ export async function clearInstallFailureMarker() {
64
+ try {
65
+ await fs.unlink(installFailureMarkerPath());
66
+ } catch (err) {
67
+ if (err.code !== 'ENOENT') {
68
+ // ENOENT is the happy path (no marker existed). Anything else
69
+ // is unexpected; surface it but don't fail.
70
+ process.stderr.write(
71
+ `[mapterrain] (could not clear install-failure marker: ${err.message})\n`
72
+ );
73
+ }
74
+ }
75
+ }
76
+
77
+ // readInstallFailureMarker returns the recorded error message, or
78
+ // null if no marker exists.
79
+ async function readInstallFailureMarker() {
80
+ try {
81
+ const body = await fs.readFile(installFailureMarkerPath(), 'utf8');
82
+ return JSON.parse(body);
83
+ } catch (err) {
84
+ return null;
85
+ }
86
+ }
87
+
88
+ const SUPPORTED_TARGETS = new Set([
89
+ 'darwin/amd64',
90
+ 'darwin/arm64',
91
+ 'linux/amd64',
92
+ 'linux/arm64',
93
+ 'windows/amd64',
94
+ ]);
95
+
96
+ export function targetForPlatform(
97
+ platform = process.platform,
98
+ arch = process.arch
99
+ ) {
22
100
  const goosMap = {
23
101
  darwin: 'darwin',
24
102
  linux: 'linux',
@@ -29,12 +107,20 @@ function currentTarget() {
29
107
  arm64: 'arm64',
30
108
  };
31
109
 
32
- const goos = goosMap[process.platform];
33
- const goarch = goarchMap[process.arch];
110
+ const goos = goosMap[platform];
111
+ const goarch = goarchMap[arch];
34
112
  if (!goos || !goarch) {
35
113
  throw new Error(
36
- `Unsupported platform ${process.platform}/${process.arch}. ` +
37
- 'Install Terrain manually from GitHub Releases or via Homebrew.'
114
+ `Unsupported platform ${platform}/${arch}. ` +
115
+ 'Install Terrain manually from GitHub Releases, Homebrew on macOS/Linux, or source.'
116
+ );
117
+ }
118
+ if (!SUPPORTED_TARGETS.has(`${goos}/${goarch}`)) {
119
+ throw new Error(
120
+ `Unsupported prebuilt Terrain target ${goos}/${goarch}. ` +
121
+ 'Published npm-install binaries are available for ' +
122
+ `${Array.from(SUPPORTED_TARGETS).sort().join(', ')}. ` +
123
+ 'Install Terrain manually from source with `go install github.com/pmclSF/terrain/cmd/terrain@latest`.'
38
124
  );
39
125
  }
40
126
 
@@ -46,6 +132,10 @@ function currentTarget() {
46
132
  };
47
133
  }
48
134
 
135
+ function currentTarget() {
136
+ return targetForPlatform();
137
+ }
138
+
49
139
  function isDevelopmentCheckout(rootDir = packageRoot) {
50
140
  return existsSync(path.join(rootDir, 'cmd', 'terrain'));
51
141
  }
@@ -71,11 +161,15 @@ function archiveFileName(version) {
71
161
  return `terrain_${version}_${target.goos}_${target.goarch}.${target.archiveExt}`;
72
162
  }
73
163
 
74
- function archiveDownloadUrl(version) {
164
+ function releaseDownloadBaseUrl(version) {
75
165
  const baseUrl =
76
166
  process.env.TERRAIN_INSTALLER_BASE_URL ||
77
167
  `https://github.com/${GITHUB_OWNER}/${GITHUB_REPO}/releases/download`;
78
- return `${baseUrl}/v${version}/${archiveFileName(version)}`;
168
+ return `${baseUrl}/v${version}`;
169
+ }
170
+
171
+ function archiveDownloadUrl(version) {
172
+ return `${releaseDownloadBaseUrl(version)}/${archiveFileName(version)}`;
79
173
  }
80
174
 
81
175
  function signatureDownloadUrl(version) {
@@ -86,6 +180,10 @@ function certificateDownloadUrl(version) {
86
180
  return `${archiveDownloadUrl(version)}.pem`;
87
181
  }
88
182
 
183
+ function checksumDownloadUrl(version) {
184
+ return `${releaseDownloadBaseUrl(version)}/checksums.txt`;
185
+ }
186
+
89
187
  function expectedSignerIdentity(version) {
90
188
  // The keyless Sigstore signature is anchored to the GitHub Actions workflow
91
189
  // that ran goreleaser at release time. The workflow runs on the v<version>
@@ -107,13 +205,84 @@ function isCosignAvailable() {
107
205
  }
108
206
  }
109
207
 
110
- // Best-effort signature verification. In 0.1.2 this is warn-only: a missing
111
- // cosign, missing signature artifact, or verification failure logs to stderr
112
- // and does NOT block install. The signing pipeline is still maturing and we
113
- // don't want to break npm installs while it stabilises.
208
+ export function checksumFromManifest(manifestText, archiveName) {
209
+ for (const rawLine of manifestText.split(/\r?\n/)) {
210
+ const line = rawLine.trim();
211
+ if (!line) {
212
+ continue;
213
+ }
214
+ const match = line.match(/^([a-fA-F0-9]{64})\s+(.+)$/);
215
+ if (!match) {
216
+ continue;
217
+ }
218
+ const candidate = path.basename(match[2].replace(/^\*/, '').trim());
219
+ if (candidate === archiveName) {
220
+ return match[1].toLowerCase();
221
+ }
222
+ }
223
+ return null;
224
+ }
225
+
226
+ export function verifyChecksumDigest(manifestText, archiveName, actualDigest) {
227
+ const expected = checksumFromManifest(manifestText, archiveName);
228
+ if (!expected) {
229
+ throw new Error(
230
+ `checksums.txt did not contain an entry for ${archiveName}`
231
+ );
232
+ }
233
+ const actual = actualDigest.toLowerCase();
234
+ if (actual !== expected) {
235
+ throw new Error(
236
+ `checksum mismatch for ${archiveName}: expected ${expected}, got ${actual}`
237
+ );
238
+ }
239
+ return expected;
240
+ }
241
+
242
+ async function sha256File(filePath) {
243
+ const hash = createHash('sha256');
244
+ hash.update(await fs.readFile(filePath));
245
+ return hash.digest('hex');
246
+ }
247
+
248
+ async function verifyChecksum({ archivePath, version, tempDir, quiet }) {
249
+ const checksumsPath = path.join(tempDir, 'checksums.txt');
250
+ await downloadFile(checksumDownloadUrl(version), checksumsPath);
251
+
252
+ const archiveName = path.basename(archivePath);
253
+ const manifest = await fs.readFile(checksumsPath, 'utf8');
254
+ const actual = await sha256File(archivePath);
255
+ verifyChecksumDigest(manifest, archiveName, actual);
256
+
257
+ log(`Checksum verified for ${archiveName}`, quiet);
258
+ return { verified: true, reason: 'checksum' };
259
+ }
260
+
261
+ // Sigstore signature verification.
262
+ //
263
+ // 0.3.x policy: Sigstore verification is MANDATORY by default. If
264
+ // `cosign` is not available on the host, the install fails with a
265
+ // clear remediation pointer. The escape for trusted/CI/air-gapped
266
+ // environments is the documented opt-out
267
+ // `TERRAIN_INSTALLER_SKIP_VERIFY=1`.
268
+ //
269
+ // Earlier revisions silently degraded to "checksum-only" when cosign
270
+ // was missing, which meant a typical npm-install on a host without
271
+ // cosign (most macOS / Linux dev machines) skipped Sigstore entirely
272
+ // without any signal in the install log beyond a one-line "falling
273
+ // back" message: the strong-integrity guarantee we advertise degraded
274
+ // silently to weak by default. Mandatory verification closes the gap;
275
+ // the env-var escape keeps adoption viable.
276
+ //
277
+ // Escape hatches:
278
+ //
279
+ // - TERRAIN_INSTALLER_SKIP_VERIFY=1 — fully opt out (CI / air-gapped).
280
+ // Prints a WARNING so the bypass is auditable.
281
+ // - TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 — opt-in degrade-to-
282
+ // checksum behavior for hosts that genuinely cannot install
283
+ // cosign.
114
284
  //
115
- // In 0.2 this becomes hard-fail unless TERRAIN_INSTALLER_SKIP_VERIFY=1 is set,
116
- // at which point the warning escalates to an error.
285
+ // Once cosign is on the host, every verify failure is a hard error.
117
286
  async function verifySignatureBestEffort({
118
287
  archivePath,
119
288
  version,
@@ -123,20 +292,38 @@ async function verifySignatureBestEffort({
123
292
  }) {
124
293
  if (env.TERRAIN_INSTALLER_SKIP_VERIFY === '1') {
125
294
  log(
126
- 'Skipping signature verification (TERRAIN_INSTALLER_SKIP_VERIFY=1).',
295
+ 'WARNING: signature verification skipped (TERRAIN_INSTALLER_SKIP_VERIFY=1). ' +
296
+ 'Set this only in trusted CI / air-gapped environments where ' +
297
+ 'integrity is established by another channel.',
127
298
  quiet
128
299
  );
129
300
  return { verified: false, reason: 'skipped-by-env' };
130
301
  }
131
302
 
132
303
  if (!isCosignAvailable()) {
133
- log(
134
- 'cosign not found on PATH; skipping signature verification. ' +
135
- 'Install cosign (https://github.com/sigstore/cosign) for stronger ' +
136
- 'integrity guarantees in future releases.',
137
- quiet
304
+ if (env.TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN === '1') {
305
+ log(
306
+ 'cosign not found on PATH. Continuing with checksum-only verification ' +
307
+ 'because TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 is set. ' +
308
+ 'For stronger integrity guarantees install cosign ' +
309
+ '(https://github.com/sigstore/cosign) and reinstall.',
310
+ quiet
311
+ );
312
+ return await verifyChecksum({ archivePath, version, tempDir, quiet });
313
+ }
314
+ throw new Error(
315
+ 'cosign is required to verify the Sigstore signature on the Terrain ' +
316
+ 'release archive, but was not found on PATH.\n\n' +
317
+ 'Resolve by one of:\n' +
318
+ ' 1. Install cosign: https://github.com/sigstore/cosign#installation\n' +
319
+ ' (Homebrew: `brew install cosign`. Linux: see release notes.)\n' +
320
+ ' 2. If this host genuinely cannot install cosign and you trust the ' +
321
+ 'GitHub-provided checksum file, set ' +
322
+ 'TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 to fall back to ' +
323
+ 'checksum-only verification.\n' +
324
+ ' 3. To skip integrity verification entirely (NOT recommended), ' +
325
+ 'set TERRAIN_INSTALLER_SKIP_VERIFY=1.'
138
326
  );
139
- return { verified: false, reason: 'cosign-missing' };
140
327
  }
141
328
 
142
329
  const sigPath = path.join(tempDir, `${path.basename(archivePath)}.sig`);
@@ -146,12 +333,14 @@ async function verifySignatureBestEffort({
146
333
  await downloadFile(signatureDownloadUrl(version), sigPath);
147
334
  await downloadFile(certificateDownloadUrl(version), certPath);
148
335
  } catch (error) {
149
- log(
150
- `Could not fetch signature artifacts (${error.message}); ` +
151
- 'skipping verification.',
152
- quiet
336
+ // Hard error in 0.3: if cosign is present, the signature download
337
+ // is required. The release pipeline produces signatures for every
338
+ // archive; their absence is a real failure mode worth surfacing.
339
+ throw new Error(
340
+ `cosign is installed but the Sigstore signature artifacts for ` +
341
+ `terrain ${version} could not be downloaded: ${error.message}. ` +
342
+ `Set TERRAIN_INSTALLER_SKIP_VERIFY=1 to bypass at your own risk.`
153
343
  );
154
- return { verified: false, reason: 'sig-download-failed' };
155
344
  }
156
345
 
157
346
  try {
@@ -177,14 +366,17 @@ async function verifySignatureBestEffort({
177
366
  );
178
367
  return { verified: true, reason: 'ok' };
179
368
  } catch (error) {
180
- log(
181
- `WARNING: cosign verify-blob failed for ${path.basename(archivePath)}. ` +
182
- 'The downloaded archive may be tampered with. Continuing install ' +
183
- '(verification will become mandatory in 0.2). Error: ' +
184
- (error.stderr ? error.stderr.toString().trim() : error.message),
185
- quiet
369
+ // Hard error in 0.3: a verify-blob failure means the archive on disk
370
+ // does NOT match the signed certificate. Aborting the install is
371
+ // strictly safer than silently continuing.
372
+ const detail = error.stderr
373
+ ? error.stderr.toString().trim()
374
+ : error.message;
375
+ throw new Error(
376
+ `cosign verify-blob FAILED for ${path.basename(archivePath)}: ${detail}. ` +
377
+ `The downloaded archive does not match its Sigstore signature; ` +
378
+ `the binary may have been tampered with. Install aborted.`
186
379
  );
187
- return { verified: false, reason: 'verify-failed' };
188
380
  }
189
381
  }
190
382
 
@@ -206,7 +398,18 @@ function log(message, quiet = false) {
206
398
  }
207
399
  }
208
400
 
209
- async function downloadFile(url, destinationPath) {
401
+ // MAX_REDIRECTS caps redirect chains to defend against misconfigured
402
+ // proxies that loop. 5 covers every normal redirect chain (GitHub
403
+ // release → CDN → storage backend) with margin to spare. Without this
404
+ // cap the recursion is unbounded — a redirect loop hangs the installer
405
+ // until the OS kills it.
406
+ const MAX_REDIRECTS = 5;
407
+
408
+ async function downloadFile(
409
+ url,
410
+ destinationPath,
411
+ redirectsRemaining = MAX_REDIRECTS
412
+ ) {
210
413
  await new Promise((resolve, reject) => {
211
414
  const request = https.get(
212
415
  url,
@@ -223,8 +426,22 @@ async function downloadFile(url, destinationPath) {
223
426
  response.headers.location
224
427
  ) {
225
428
  response.resume();
429
+ if (redirectsRemaining <= 0) {
430
+ reject(
431
+ new Error(
432
+ `download exceeded ${MAX_REDIRECTS} redirects for ${url}; ` +
433
+ 'check for proxy redirect loops or set ' +
434
+ 'TERRAIN_INSTALLER_BASE_URL to a direct download host.'
435
+ )
436
+ );
437
+ return;
438
+ }
226
439
  try {
227
- await downloadFile(response.headers.location, destinationPath);
440
+ await downloadFile(
441
+ response.headers.location,
442
+ destinationPath,
443
+ redirectsRemaining - 1
444
+ );
228
445
  resolve();
229
446
  } catch (error) {
230
447
  reject(error);
@@ -251,6 +468,13 @@ async function downloadFile(url, destinationPath) {
251
468
  }
252
469
  );
253
470
 
471
+ request.setTimeout(DOWNLOAD_TIMEOUT_MS, () => {
472
+ request.destroy(
473
+ new Error(
474
+ `download timed out after ${DOWNLOAD_TIMEOUT_MS / 1000}s for ${url}`
475
+ )
476
+ );
477
+ });
254
478
  request.on('error', reject);
255
479
  });
256
480
  }
@@ -385,6 +609,25 @@ export async function runTerrainCli(argv = process.argv.slice(2)) {
385
609
  return;
386
610
  }
387
611
 
612
+ // Check for a recorded install failure before attempting a silent
613
+ // retry. If `npm install` failed to fetch/verify the binary, the
614
+ // marker file records the original error; surface it verbatim
615
+ // instead of pretending nothing happened.
616
+ const marker = await readInstallFailureMarker();
617
+ if (marker && !existsSync(installedBinaryPath(rootDir))) {
618
+ throw new Error(
619
+ 'Terrain binary is not installed.\n\n' +
620
+ `Recorded install failure (${marker.timestamp}, ${marker.platform}, v${marker.version}):\n` +
621
+ ` ${marker.message}\n\n` +
622
+ 'Resolve the underlying issue, then either:\n' +
623
+ ' - Re-run `npm install -g mapterrain` after installing cosign\n' +
624
+ ' - Set TERRAIN_INSTALLER_ALLOW_MISSING_COSIGN=1 to fall back to\n' +
625
+ ' checksum-only verification, or\n' +
626
+ ' - Set TERRAIN_INSTALLER_SKIP_VERIFY=1 to skip verification entirely.\n\n' +
627
+ 'Marker file: ~/.terrain/install-failure.log'
628
+ );
629
+ }
630
+
388
631
  let binaryPath;
389
632
  try {
390
633
  binaryPath = await ensureTerrainBinary({ rootDir });
@@ -404,6 +647,9 @@ export async function runTerrainCli(argv = process.argv.slice(2)) {
404
647
  );
405
648
  }
406
649
 
650
+ // First successful run after a failed install: clear the marker.
651
+ await clearInstallFailureMarker();
652
+
407
653
  await runBinary(binaryPath, argv);
408
654
  }
409
655
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "mapterrain",
3
- "version": "0.1.2",
3
+ "version": "0.3.1",
4
4
  "description": "Terrain test intelligence CLI.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -23,10 +23,11 @@
23
23
  },
24
24
  "scripts": {
25
25
  "postinstall": "node bin/postinstall.js",
26
- "test": "node scripts/verify-pack.js",
27
- "lint": "eslint \"bin/*.js\" \"scripts/*.js\"",
28
- "format": "prettier --write \"bin/*.js\" \"scripts/*.js\"",
29
- "format:check": "prettier --check \"bin/*.js\" \"scripts/*.js\"",
26
+ "test": "npm run test:unit && node scripts/verify-pack.js",
27
+ "test:unit": "node --test scripts/test-installer-marker.mjs scripts/test-release-sanity.mjs",
28
+ "lint": "eslint \"bin/*.js\" \"scripts/*.js\" \"scripts/*.mjs\"",
29
+ "format": "prettier --write \"bin/*.js\" \"scripts/*.js\" \"scripts/*.mjs\"",
30
+ "format:check": "prettier --check \"bin/*.js\" \"scripts/*.js\" \"scripts/*.mjs\"",
30
31
  "start": "node bin/terrain-cli.js",
31
32
  "convert": "node bin/terrain-cli.js convert",
32
33
  "lint-staged": "lint-staged",
@@ -81,4 +82,4 @@
81
82
  "node": ">=22.0.0"
82
83
  },
83
84
  "sideEffects": false
84
- }
85
+ }