@nurix/etna 0.20.36 → 0.20.38

package/data/harnesses/hr/skills/evaluator/criteria/testing.md

@@ -1,24 +1,24 @@
 # Testing & Resilience — Extraction Plan
 
-- **Sub-criterion:** 1.2 (AI-only)
-- **Weight:** 12.5 pts (4 Code AI × 12.5 = 50 pts half)
+- **Sub-criterion:** 1.2 (Production-Grade Code)
+- **Weight:** 12.5 pts
 - **Contenders:** Semgrep + filesystem + GHA + LLM (joint load-bearing) + CodeRabbit resilience reuse (**calibrating modifier**) + **multi-protocol route-auth scan** (HTTP + WebSocket + gRPC + tRPC — feeds the `AUTH` gate).
 - **Source quote:** _"Unit/integration tests, error handling, input validation. It should survive more than the happy path."_
 
-Bands are **absolute** — thresholds in the _Banding_ section below are fixed numeric cutoffs. Gates this recipe owns: **`UNTESTED`** (hard, `testing ≤ 1` when `test_file_count == 0`), **`TESTS_TRIVIAL`** (soft, `testing ≤ 2` when `assertion_density < 1.0` OR `trivial_body_ratio > 0.8`), and **`AUTH`** (fires `testing ≤ 2` when network-routed handlers exist and ≥ 1 mutating endpoint isn't behind an auth middleware/dependency/decorator, detected across HTTP [FastAPI/Express/Flask/Gin/Django POST/PUT/PATCH/DELETE], WebSocket [ws/Socket.io/SignalR], gRPC [grpc/grpcio unary+streaming], tRPC [@trpc/server mutations]). Raw dumps: `raw/testing-auth-scan-{http,ws,grpc,trpc}.json`. Data fields: `routed_mutating_endpoints_total`, `routed_mutating_endpoints_unguarded`, `auth_by_protocol: {http: N, ws: N, grpc: N, trpc: N}`. AUTH gate is skipped if no routed handlers are detected.
+Bands are **absolute** — the thresholds in _Aggregation_ are fixed numeric cutoffs. This recipe owns three gates — `UNTESTED`, `TESTS_TRIVIAL` and `AUTH` — defined in § Gates.
 
-> Four load-bearing signals + one calibrating modifier (CR), integrated by the scorer. No weighted-math internal split — the LLM emits a single 0–5 band and cites the most load-bearing signal.
+Four load-bearing signals (1, 2, 4, 5) and one calibrating modifier (3, CodeRabbit) feed the band; Signal 6 feeds only the `AUTH` gate. There is no weighted internal split — the LLM emits a single 0–5 band and cites the most load-bearing signal.
 
 ---
 
 ## Preflight
 
-Before running this recipe, execute [`../../preflight.md`](../../preflight.md). Required:
+Before running this recipe, execute [`../preflight.md`](../preflight.md). Required:
 
 - `semgrep` (polyglot static analysis — resilience rule-set)
 - `gh`, `jq` (CI workflow introspection + CR evidence scrape)
 
-If `semgrep` cannot be installed, emit `semgrep_resilience_findings: null, semgrep_NOT_RUN: true, reason: "..."` and flag loudly in `§6.1`. Do not substitute grep for it — the patterns Semgrep catches (`bare-except`, `empty-catch`, `unhandled-promise`) are structural, not lexical.
+If `semgrep` cannot be installed, emit `semgrep_resilience_findings: null, semgrep_NOT_RUN: true, reason: "..."` and flag it in the workspace README's raw artifact index. **Never** substitute grep — the patterns Semgrep catches (`bare-except`, `empty-catch`, `unhandled-promise`) are structural, not lexical.
 
 ---
 
@@ -36,7 +36,7 @@ find <repo> -type f \
   -not -path '*/node_modules/*' -not -path '*/.venv/*' \
   | wc -l
 
-# Source file count (for ratio) — language-aware; see "Per-language thresholds" below
+# Source file count (for ratio) — language-aware; see "Per-language test ratio thresholds" below
 find <repo> -type f \
   \( -name '*.py' -o -name '*.ts' -o -name '*.tsx' \
      -o -name '*.js' -o -name '*.jsx' -o -name '*.go' \) \
@@ -68,7 +68,7 @@ echo "$TEST_FILES" | xargs grep -cE \
 
 Capture: `test_file_count`, `src_file_count`, `test_ratio`, `framework_configured`, `assertion_density` (assertions per test file), `trivial_body_ratio` (trivial bodies ÷ total test bodies).
 
-**Primary language detection** (for thresholds — see Aggregation):
+**Primary language detection** (selects the test-ratio threshold in _Aggregation_):
 
 ```bash
 find <repo> -type f \
@@ -98,7 +98,7 @@ Capture: `ci_has_test_job` (bool), `latest_test_conclusion` (SUCCESS/FAILURE/NON
 
 ### Signal 3 — CodeRabbit reuse (**calibrating modifier**)
 
-Reuse the CR scrape from 1.1 (see [code-quality.md](code-quality.md#signal-6--coderabbit-scrape-calibrating-modifier) Signal 6). Filter to resilience-relevant categories `[bug, warning]` — these are CR's signal for missing error handling, swallowed exceptions, or validation gaps on the PRs it reviewed.
+Reuse the CR scrape from 1.1 ([code-quality.md Signal 6](code-quality.md#signal-6--coderabbit-scrape-calibrating-modifier)) — never re-scrape. Filter to the resilience categories `[bug, warning]`, CR's signal for missing error handling, swallowed exceptions and validation gaps.
 
 ```yaml
 categories_in_scope: [bug, warning]
@@ -108,21 +108,20 @@ Capture:
 
 - `cr_resilience_flags = by_category.bug + by_category.warning`
 - `cr_resilience_per_kloc = cr_resilience_flags ÷ (scc_code_loc ÷ 1000)` (lower-is-better; `null` when `cr_pr_count == 0` or `cr_resilience_flags == 0` on a zero-actionable walkthrough)
-- `cr_semgrep_overlap` — count of `file:line` flagged by both CR and Semgrep Signal 4. Prefer CR's context in evidence when they overlap; surface the overlap count for transparency.
+- `cr_semgrep_overlap` — count of `file:line` flagged by both CR and Semgrep Signal 4. Prefer CR's context in evidence where they overlap.
 
-**Banding rule:** CR is a calibrating modifier — it pulls the band down when `cr_resilience_per_kloc` is high and (when non-null) participates in the aggregation table below alongside `semgrep_resilience_per_kloc`. When CR data is empty, `cr_resilience_per_kloc` is `null` and the modifier is a no-op. Full rationale + plan-upgrade escape hatch in [`SKILL.md → CodeRabbit — calibrating modifier with null-data semantics`](../SKILL.md#coderabbit--calibrating-modifier-with-null-data-semantics).
-
-**Semgrep remains the broadest resilience signal** (full-tree scan, not just reviewed diffs). Signal 4 below.
+**Banding rule:** when non-null, `cr_resilience_per_kloc` sits in the _Aggregation_ table beside `semgrep_resilience_per_kloc` and pulls the band down when high; when CR data is empty it is `null` and a no-op ([`SKILL.md → CodeRabbit — calibrating modifier with null-data semantics`](../SKILL.md#coderabbit--calibrating-modifier-with-null-data-semantics)). Semgrep (Signal 4) is the broader resilience signal — it scans the full tree, not just reviewed diffs.
 
 ### Signal 4 — Semgrep scan
 
 ```bash
-semgrep --config p/python --config p/javascript --config p/typescript \
+timeout 180 semgrep --config p/python --config p/javascript --config p/typescript \
         --config p/go \
-        --json <repo> > /tmp/semgrep.json
+        --json --timeout 30 --disable-version-check --metrics=off \
+        <repo> > /tmp/semgrep.json
 ```
 
-**Filter to resilience-relevant rules only.** Tight pattern (catches the actual phenomena, drops noise):
+**Filter to resilience-relevant rules only:**
 
 ```bash
 jq '[.results[] | select(
@@ -132,13 +131,13 @@ jq '[.results[] | select(
 )]' /tmp/semgrep.json
 ```
 
-**What the pattern deliberately excludes, and why:**
+**Never widen the pattern with these** — each looks like a natural addition:
 
-- `error|exception` — matched every rule name containing those words (most security rules do), inflating findings ~10×.
-- `injection|deserial|unsafe` — security territory; belongs in 1.4 Maintainability / SECURITY gate to avoid double-counting.
-- `await|promise` (bare) — scoped to `unchecked-await` / `unhandled-promise` so async-correctness rules aren't indiscriminately matched.
+- `error|exception` — matches every rule name containing those words (most security rules do), inflating findings ~10×.
+- `injection|deserial|unsafe` — security territory, owned by 1.4 Maintainability; matching it here double-counts.
+- bare `await|promise` — matches async-correctness rules indiscriminately; the pattern scopes them to `unchecked-await` / `unhandled-promise`.
 
-`p/security-audit` config is excluded — it produces security-skewed noise. Language packs alone carry the resilience rules we want.
+**Never add `p/security-audit`** — it produces security-skewed noise; the language packs carry the resilience rules.
 
 Capture:
 
@@ -147,7 +146,7 @@ Capture:
 - `semgrep_resilience_by_severity` (ERROR / WARNING / INFO)
 - `semgrep_resilience_per_kloc`
 
-**Note:** Semgrep ERROR severity does **not** act as a hard score cap — signal only.
+Semgrep ERROR severity is a signal only — **never** a hard score cap.
 
 ### Signal 5 — LLM sample (3 handler-or-resilience-dense files)
 
@@ -158,7 +157,7 @@ Tiered file selection — walk tiers until 3 files are picked:
 - Matches `*route*`, `*handler*`, `*controller*`, `*api*`, `*server*`, `*endpoint*`, `*view*`
 - Under `src/`, `app/`, `internal/`, `pkg/` or root
 
-**Tier 2 — resilience-dense files (for non-API repos: CLIs, agents, batch jobs):** Rank source files by `(try|catch|raise|throw|except|panic|recover) count ÷ LOC`; pick top 3. These are where resilience _matters_, regardless of repo shape.
+**Tier 2 — resilience-dense files (non-API repos: CLIs, agents, batch jobs):** rank source files by `(try|catch|raise|throw|except|panic|recover) count ÷ LOC`; pick the top 3.
 
 ```bash
 for f in $(find <repo>/src <repo>/app <repo>/internal \
@@ -181,7 +180,7 @@ Capture: `llm_resilience_score` (0–5), `llm_notes` (one sentence), `llm_files_
 
 ### Signal 6 — Multi-protocol AUTH route-scan (AUTH gate trigger)
 
-Enterprise backends ship mutating endpoints. A POST/PUT/PATCH/DELETE (HTTP) — or a WebSocket message handler, gRPC unary/streaming method, or tRPC mutation — that doesn't run behind an auth middleware / dependency / decorator is an enterprise-grade resilience failure: anyone can mutate server state. This signal scans for routed mutating handlers across four protocols and checks each one for an auth guard. **Skipped (gate does not fire) if no routed handlers are detected** — a CLI or batch job isn't expected to carry auth.
+A mutating handler — HTTP POST/PUT/PATCH/DELETE, a WebSocket message handler, a gRPC unary/streaming method, a tRPC mutation — not behind an auth middleware / dependency / decorator lets anyone mutate server state. Scan for routed mutating handlers across the four protocols and check each for an auth guard. **Skipped (the gate does not fire) when no routed handlers are detected** — a CLI or batch job carries no auth.
 
 ```bash
 # HTTP — FastAPI / Flask / Django / Express / Gin / Echo
@@ -232,7 +231,7 @@ semgrep --config=auto --timeout 30 --metrics=off --disable-version-check --json
         <repo> > /tmp/auth-trpc.json 2>/dev/null || true
 ```
 
-Parse each tool's output for the number of mutating handlers and, critically, whether each one is adjacent to or wrapped by an auth guard. A handler is "guarded" when any of these are present in the same scope:
+Parse each output for the mutating handlers and whether each is adjacent to or wrapped by an auth guard. A handler is "guarded" when any of these is present in the same scope:
 
 - **Python/FastAPI**: `Depends(...)` call on the function signature that resolves to an auth dep, `@require_auth`/`@login_required` decorator, or a router prefix/dependency declaring auth
 - **Python/Django**: `@login_required` / `@permission_required` / DRF `permission_classes`
@@ -243,7 +242,7 @@ Parse each tool's output for the number of mutating handlers and, critically, wh
 - **WebSocket**: handshake-time auth in `connect` handler OR per-message token validation
 - **gRPC**: `ServerInterceptor` registered at `grpc.Server(interceptors=[...])` OR explicit `context.authenticate(...)` in the method
 
-The scan runs with a 30 s per-rule cap + 120 s wall-clock ceiling per protocol; tools that time out emit `<protocol>_NOT_RUN: true, reason: "semgrep timeout"` with no effect on the gate.
+Run each protocol's scan under a 30 s per-rule cap and a 120 s wall-clock ceiling; a scan that times out emits `<protocol>_NOT_RUN: true, reason: "semgrep timeout"` and has no effect on the gate.
 
 Aggregate per protocol, then across all protocols:
 
@@ -268,7 +267,7 @@ Capture:
 
 ## Raw dumps (flat — files under `raw/` named `testing-*`)
 
-Each signal writes its raw output to the flat `raw/` directory at the workspace root. Absent files = not extracted. Aggregates go to `scorecard.yaml` under `data.`.
+Each signal writes its raw output to the flat `raw/` directory; an absent file means not extracted. Aggregates go to `scorecard.yaml` under `data.`.
 
 | File | Source signal | Shape / note |
 | --- | --- | --- |
@@ -287,7 +286,7 @@ CodeRabbit comments are dumped once by 1.1 under `raw/code-quality-cr-comments.j
 
 ## Aggregation → 0–5 band
 
-Scorer integrates the 5 signals. Rough directionality:
+Integrate Signals 1–5. Rough directionality:
 
 | Signal | Pulls score UP | Pulls score DOWN |
 | --- | --- | --- |
@@ -300,7 +299,7 @@ Scorer integrates the 5 signals. Rough directionality:
 
 ### Per-language test ratio thresholds
 
-Go colocates `foo_test.go` with `foo.go`; Python typically uses a separate `tests/` tree; TS varies. Fixed thresholds distort across stacks — use language-aware floors:
+A single threshold distorts across stacks — Go colocates `foo_test.go` with `foo.go`, Python keeps a separate `tests/` tree, TS varies — so the floors are per language, keyed on `primary_language`:
 
 | Primary language        | Top-band ratio | Floor (pulls band down) |
 | ----------------------- | -------------- | ----------------------- |
@@ -328,7 +327,7 @@ Go colocates `foo_test.go` with `foo.go`; Python typically uses a separate `test
 | `TESTS_TRIVIAL` (soft) | `test_file_count > 0` AND (`assertion_density < 1.0` per test file OR `trivial_body_ratio > 0.8`) | `testing` ≤ 2 |
 | `AUTH` | `routed_mutating_endpoints_total ≥ 1` AND `routed_mutating_endpoints_unguarded ≥ 1`. Covers HTTP (POST/PUT/PATCH/DELETE), WebSocket event handlers, gRPC unary+streaming, tRPC mutations. Skipped when `auth_NOT_RUN == true` (no routed handlers detected — CLI/batch/library). | `testing` ≤ 2 |
 
-All three can fire independently and stack. `AUTH` is the "enterprise-grade mutation without authentication" case — a POST endpoint that lets any unauthenticated caller mutate server state is a testing failure regardless of how clean the happy-path tests are. `TESTS_TRIVIAL` is the "tests exist but are meaningless" case (a single `def test(): pass` file passes `UNTESTED` but should not reach band 3+).
+All three fire independently and stack. `AUTH` caps the band however clean the happy-path tests are — an endpoint any unauthenticated caller can use to mutate state is a resilience failure. `TESTS_TRIVIAL` catches meaningless tests: a single `def test(): pass` file passes `UNTESTED` but must not reach band 3+.
 
 ---
 
@@ -338,10 +337,10 @@ All three can fire independently and stack. `AUTH` is the "enterprise-grade muta
 - No CI workflows → drop signal 2, rely on 1, 4, 5.
 - `cr_pr_count == 0` OR CR empty (Free-plan walkthrough-only) → `cr_resilience_per_kloc: null`, Signal 3 modifier is a no-op; rely on 1, 2, 4, 5 for the band.
 - Semgrep doesn't support a repo's primary language → drop signal 4, rely on 1, 2, 5.
-- Semgrep not installed (preflight failed) → emit `semgrep_NOT_RUN: true`, rely on 1, 2, 5, flag in §6.1.
+- Semgrep not installed (preflight failed) → emit `semgrep_NOT_RUN: true`, rely on 1, 2, 5, flag it in the README's raw artifact index.
 - LLM tier 1 & 2 yield nothing → use tier 3, annotate `llm_sample_tier: 3` (confidence is lower).
 
-Annotate evidence with `"Signals used: [...]"` so the audit trail is explicit.
+Annotate evidence with `"Signals used: [...]"`.
 
 ---
 
@@ -384,16 +383,12 @@ cr_semgrep_overlap: 0
 signals_used: [filesystem, gha, semgrep, llm, coderabbit_empty]
 ```
 
-`null` signals the scorer to skip the CR modifier for this project.
-
 ---
 
 ## Caveats
 
-- Semgrep pre-built packs skew toward security. A tight regex keeps resilience focus; worth 3–5 custom rules if scoring reveals gaps.
-- CodeRabbit reuse is a calibrating modifier, not a primary axis; `null` on Free-plan walkthrough-only repos means no-op. Full rationale + plan-upgrade escape hatch in [`SKILL.md → CodeRabbit — calibrating modifier with null-data semantics`](../SKILL.md#coderabbit--calibrating-modifier-with-null-data-semantics).
-- `AI_AUTHORED_SUSPECTED` flag (from 1.1) remains informational — LLM reviewer should note AI-generated resilience patterns (e.g. copy-pasted try/except blocks with identical logging).
-- LLM sample size is 3. Diverse codebases may score unevenly. Evidence must name the files.
-- `trivial_body_ratio` heuristic may false-positive on pytest `parametrize` stubs where the real body lives in fixtures. Acceptable — over-triggering a soft gate is safer than under-triggering.
-- Per-language `test_ratio` thresholds are **absolute**. Each language's floor/top band is fixed in the table above.
-- AUTH scan's guard-detection is LLM-adjudicated — Semgrep surfaces the mutating handler; the scorer reads surrounding context to decide whether a guard is present. This is a deliberate trade-off: pattern-matching every auth idiom across 4+ protocols and 10+ frameworks is combinatorial, and an LLM with the source in front of it is more accurate than a brittle megapattern. When the LLM's confidence on guard-detection is low (unusual framework, ambiguous middleware chain), emit `auth_reason: "guard detection uncertain"` and let the gate fire conservatively — false-positive AUTH is cheaper than false-negative.
+- Semgrep's pre-built packs skew toward security; the tight regex keeps the resilience focus. Prefer adding 3–5 custom rules when scoring reveals gaps.
+- AI-authored code (`AI_AUTHORED_SUSPECTED`, defined in 1.1's Caveats) is informational — note AI-generated resilience patterns such as copy-pasted try/except blocks with identical logging.
+- LLM sample size is 3, so diverse codebases may score unevenly. Evidence **must** name the files.
+- `trivial_body_ratio` may false-positive on pytest `parametrize` stubs whose real body lives in fixtures. Accepted — over-triggering a soft gate is safer than under-triggering.
+- AUTH guard detection is LLM-adjudicated: Semgrep surfaces the mutating handler and the scorer reads the surrounding source, because a pattern for every auth idiom across 4 protocols and 10+ frameworks is combinatorial and brittle. When confidence is low (unusual framework, ambiguous middleware chain), emit `auth_reason: "guard detection uncertain"` and let the gate fire — a false-positive AUTH is cheaper than a false negative.

package/data/harnesses/hr/skills/evaluator/criteria/ux-mechanics.md

@@ -5,7 +5,7 @@
 - **Contenders:** UX heuristic grep bundle (web) + clig.dev greps (CLI) + **enterprise API quality signals** (API-only path) + LLM read (layered, path-dependent)
 - **Source quote:** _"Clear user flows, thoughtful UX (even with minimal UI). Usability, not just features."_
 
-This recipe scores usability and user-flow quality across three surfaces (web UI / CLI / API-only). Bands are **absolute**. The **API-only path is not capped** — enterprise backends can score up to 5. API-only signal bundle: (a) OpenAPI/Swagger spec presence + completeness (all routes documented, examples present, error shapes declared), (b) route naming consistency (kebab / snake / camel singular-vs-plural coherence), (c) response envelope + error-handling consistency (uniform `{data, error}` or RFC 7807), (d) versioning strategy (`/v1/` prefix, `Accept-Version` header, or clearly no-version-needed), (e) idempotency on mutations (Idempotency-Key header support on POST/PUT/PATCH), (f) rate-limiting presence (middleware + headers), (g) input validation at boundaries (schema validation on incoming requests), (h) API documentation quality (written docs beyond autogen). Three-path detection (web / CLI / API-only) writes `raw/ux-mechanics-path-detection.json`. Gates: none owned by this recipe.
+This recipe scores usability and user-flow quality on three surfaces — web UI (Path A), CLI (Path B), API-only (Path C). Bands are **absolute**, and the **API-only path is uncapped**: a polished enterprise backend scores up to 5, like an equally polished web UI. This recipe owns no gates.
 
 ---
 
@@ -34,7 +34,7 @@ grep -rlE 'Flask|FastAPI|express\(\)|gin\.Default|fiber\.New|@nestjs/core' \
   <local-repo-path>/src <local-repo-path>/app <local-repo-path>/internal 2>/dev/null
 ```
 
-Capture: `path` ∈ `web-ui` / `cli` / `backend-only` / `mixed` (run all relevant bundles).
+Capture: `path` ∈ `web-ui` / `cli` / `backend-only` / `mixed` in `raw/ux-mechanics-path-detection.json`; in `scorecard.yaml` and evidence, record the path as its letter — `A` (web-ui), `B` (cli), `C` (backend-only). For `mixed`, run every relevant bundle.
 
 ---
 
@@ -129,7 +129,7 @@ grep -rlE 'logging\.|logger\.|log\.|pino|winston|zerolog|zap' \
 grep -cE '^\s*\$ |^\s*```(sh|bash|console|shell)' <local-repo-path>/README.md 2>/dev/null
 ````
 
-Capture 6 booleans + counts:
+Capture the six conformance signals:
 
 - `cli_help_wired` (bool)
 - `cli_stderr_discipline` (bool)
@@ -137,12 +137,23 @@ Capture 6 booleans + counts:
 - `cli_common_flags_count`
 - `cli_logging_present` (bool)
 - `cli_readme_examples` (count of fenced cmd blocks)
-- `cli_clig_conformance_score` = count of the 6 booleans satisfied (0-6)
+- `cli_clig_conformance_score` = count of the six signals satisfied (0-6)
 
 ---
 
 ## Path C — Backend-only (API-UX surrogate)
 
+API quality is judged on eight axes; C1–C3 extract (a), (c) and (h) mechanically, and the Signal X read judges the rest:
+
+- **(a) OpenAPI/Swagger** — spec present and complete (all routes documented, examples present, error shapes declared)
+- **(b) Route naming** — consistent kebab / snake / camel, coherent singular-vs-plural
+- **(c) Response envelope + errors** — uniform shape (`{data, error}` or RFC 7807)
+- **(d) Versioning** — `/v1/` prefix, `Accept-Version` header, or clearly no version needed
+- **(e) Idempotency on mutations** — `Idempotency-Key` header support on POST/PUT/PATCH
+- **(f) Rate limiting** — middleware + headers
+- **(g) Input validation** — schema validation on incoming requests at the boundary
+- **(h) Documentation** — written API docs beyond autogen
+
 ### Signal C1 — OpenAPI / docs presence
 
 ```bash
@@ -188,7 +199,11 @@ Capture:
 
 ## Signal X (shared) — LLM read
 
-**Web-UI path:** read top 2 route files + main layout + 1 form component → score "thoughtful UX" **CLI path:** read main command file + 1 error-handling path → score "thoughtful CLI" **Backend-only path:** read 2 route handlers + error-response module → score "thoughtful API"
+| Path | Read | Score |
+| --- | --- | --- |
+| Web-UI | top 2 route files + main layout + 1 form component | thoughtful UX |
+| CLI | main command file + 1 error-handling path | thoughtful CLI |
+| Backend-only | 2 route handlers + the error-response module | thoughtful API, across the Path C axes |
 
 Capture:
 
@@ -201,7 +216,7 @@ Capture:
 
 ## Raw dumps (flat — files under `raw/` named `ux-mechanics-*`)
 
-Each signal writes its raw output to the flat `raw/` directory at the workspace root. Absent files = not extracted or path not applicable. Aggregates go to `scorecard.yaml` under `data.`.
+Each signal writes its raw output to the flat `raw/` directory; an absent file means not extracted or path not applicable. Aggregates go to `scorecard.yaml` under `data.`.
 
 | File | Source | Shape / note |
 | --- | --- | --- |
@@ -217,7 +232,7 @@ Each signal writes its raw output to the flat `raw/` directory at the workspace
 
 ## Aggregation → 0–5 band
 
-Scorer integrates the path-specific signals + LLM read.
+Integrate the path-specific signals and the Signal X read.
 
 **Banding (absolute):**
 
@@ -276,8 +291,7 @@ data:
 
 ## Caveats
 
-- **UX heuristic grep is noisy.** Presence of `isLoading` ≠ loading UI is actually shown. LLM adjudicates when score conflicts with heuristic count.
+- **The UX heuristic grep is noisy** — `isLoading` present ≠ a loading UI shown. The LLM adjudicates when its score conflicts with the heuristic count.
 - **Route-orphan detection is stack-specific** — reliable in Next.js file-based routing, weaker in hand-rolled React Router.
-- **Backend-only / API-only is uncapped.** Enterprise API design (OpenAPI quality + versioning + idempotency + rate-limiting + input validation) can score up to 5 on the strength of those signals, matching an equally polished web UI.
-- **CLI assessment via clig.dev** assumes conformance signals quality — some CLIs ship deliberately minimal with well-chosen defaults; LLM should adjudicate.
-- **Accessibility deliberately out of scope.** A11y lint and the `aria/role/htmlFor` heuristic are not part of this recipe because tool coverage is uneven and a11y is out of scope for this rubric. UX heuristics, clig.dev conformance, API-UX surrogates, and the LLM read remain.
+- **clig.dev conformance is a proxy for quality** — some CLIs ship deliberately minimal with well-chosen defaults; the LLM adjudicates.
+- **Accessibility is out of scope.** **Never** add a11y lint or an `aria/role/htmlFor` heuristic — tool coverage is uneven and a11y is outside this rubric.

package/data/harnesses/hr/skills/evaluator/criteria/value-articulation.md

@@ -4,23 +4,23 @@
 - **Weight:** 12.5 pts
 - **Source quote:** _"Does the README make a case for impact? Are claims quantified (time saved, cost, quality)?"_
 
-This recipe scores how well the README articulates value — whether it makes a clear case for impact with a named audience and quantified outcomes (time / cost / quality).
+This recipe scores how well the README articulates value — a clear case for impact with a named audience and quantified outcomes (time / cost / quality).
 
 ---
 
 ## Extraction recipe
 
-README-centric. The evaluator parses the README for impact claims, audience, scope coverage, and quantified outcomes.
+README-centric: parse the README for impact claims, audience, scope coverage, and quantified outcomes.
 
 ### Signal 1 — README impact read (LLM interpretive)
 
-Feed the LLM the full README (truncated to the first ~1000 lines) plus a list of shipped capabilities (extracted from the `mechanical_ambition` recipe's `shipped_*` fields in its `raw/mechanical-ambition-llm-read.md`, if that recipe has already run for this project):
+Feed the LLM the README (first ~1000 lines) plus the shipped capabilities — the `shipped` list in `raw/mechanical-ambition-llm-read.md`, when that recipe has already run for this project:
 
 ```bash
 head -1000 "$REPO/README.md" > /tmp/val-art-readme.md
 ```
 
-Write the LLM read to `raw/value-articulation-llm-read.md`. The LLM emits:
+Write the LLM read to `raw/value-articulation-llm-read.md`. It emits:
 
 ```yaml
 scope_breadth: N                              # number of distinct capabilities the README claims
@@ -59,7 +59,7 @@ Aggregate into `raw/value-articulation-proxies.json`:
 | **1** | No value claim, or README is an unmodified starter/template README. `value_articulation_score ≤ 1`. |
 | **0** | Empty README. |
 
-**Over-promise penalty:** if `completeness_ratio < 0.5` AND the README claims shipped functionality (not "WIP"), cap the band at **2** — the README is writing checks the code doesn't cash.
+**Over-promise penalty:** if `completeness_ratio < 0.5` AND the README claims shipped functionality (not "WIP"), cap the band at **2** — the README promises what the code does not deliver.
 
 ---
 
@@ -72,5 +72,5 @@ Aggregate into `raw/value-articulation-proxies.json`:
 
 ## Not in scope
 
-- Judging whether the claimed impact is realistic needs real operational/runtime data — out of scope; this recipe scores how well the README *articulates* value, not whether the claimed magnitude holds up.
-- Measuring actual runtime behavior against claims — that's a separate runtime-verification pass, out of scope here.
+- **Whether the claimed impact is realistic** — that needs operational data; this recipe scores how well the README *articulates* value, not whether the magnitude holds up.
+- **Runtime behavior against claims** — a separate runtime-verification pass.

package/data/harnesses/hr/skills/evaluator/preflight.md

@@ -1,74 +1,55 @@
 # Evaluator — Preflight
 
-> **Run before scoring a project.** This skill depends on external tools that must exist on the host. If a tool is missing, **install it** — do not silently fall back to grep / LLM-only. A silent fallback poisons the gate checks (VULN / SECRETS / AUTH / OBSERVABILITY) and makes the scorecard dishonest.
+> **Run before scoring a project.** If a tool is missing, **install it** — never fall back to grep or an LLM-only read. A silent fallback corrupts the scorecard twice: the absolute banding tables are calibrated against real tool output (`lizard` CCN, `semgrep` finding density), so an estimate lands in the wrong band; and `gitleaks` catches entropic tokens `grep` cannot while `pip-audit` / `trivy` catch CVEs no heuristic will, so a missed `SECRETS` or `VULN` hit becomes a dishonest pass.
 
-The evaluator is allowed to install its own tools via `brew` / `pipx` / `npm -g`. Installing is cheap (~1 minute total) and idempotent. Ignoring a missing tool is not.
+The evaluator installs its own tools via `brew` / `pipx` / `npm -g` / `go install`. Installing is cheap (~1 minute total) and idempotent.
 
----
+## Invocation
+
+Run `bash .claude/skills/evaluator/preflight.sh` — idempotent, so re-running no-ops when every tool is present. `preflight.sh` is the executable contract and holds the canonical install commands; edit there, then bring the table below to match.
 
 ## Required tools
 
 | Tool | Purpose | Recipes that use it | Install (macOS, primary) |
 | --- | --- | --- | --- |
 | `gh` | GitHub API (CR scrape, GHA runs, PR enumeration) | 1.1, 1.2, 2.4 | `brew install gh` |
-| `jq` | JSON parsing in shell pipelines | all | `brew install jq` |
 | `gitleaks` | committed-secret scan — fires `SECRETS` gate | 1.4 | `brew install gitleaks` |
-| `lizard` | polyglot complexity / god-function detection (per-language CCN: Py 10 / JS-TS 10 / Java 12 / Go 15) | 1.1, 1.4 | `brew install lizard` |
-| `scc` | LoC + cocomo estimate (replaces tokei/cloc) | 1.4, 2.4a | `brew install scc` |
-| `semgrep` | polyglot static analysis (quality + resilience + multi-protocol auth) | 1.1, 1.2, 2.4a | `brew install semgrep` |
+| `lizard` | polyglot complexity / god-function detection (per-language CCN: Py 10 / JS-TS 10 / Java 12 / Go 15) | 1.1, 1.4 | `pipx install lizard` — **never** `brew install lizard`, which is an unrelated LZ5 compressor of the same name |
+| `scc` | LoC + COCOMO estimate | 1.4, 2.4 | `brew install scc` |
+| `semgrep` | polyglot static analysis (quality + resilience + multi-protocol auth) | 1.1, 1.2, 2.4 | `brew install semgrep` |
 | `hadolint` | Dockerfile lint | 1.3 | `brew install hadolint` |
-| `ruff` | Python lint + formatter | 1.1 (Python repos), 2.4a (tier-A) | `brew install ruff` |
-| `shellcheck` | shell-script lint | 2.2a (CLI path) | `brew install shellcheck` |
-| `tree` | folder-tree read for LLM context | 1.4, 2.2a, 2.4a | `brew install tree` |
+| `ruff` | Python lint + formatter | 1.1 (Python repos), 2.4 (Tier-A) | `brew install ruff` |
+| `shellcheck` | shell-script lint | 2.2 (CLI path) | `brew install shellcheck` |
+| `tree` | folder-tree read for LLM context | 1.4, 2.1, 2.2, 2.4 | `brew install tree` |
 | `golangci-lint` | Go aggregate linter | 1.1 (Go repos) | `brew install golangci-lint` |
-| `biome` | TS/JS lint + format (preferred over eslint for speed) | 1.1 (TS/JS repos) | `npm i -g @biomejs/biome` |
+| `pnpm` | installs and audits pnpm-locked JS repos | 1.4 | `brew install pnpm` |
+| `yarn` | installs and audits yarn-locked JS repos | 1.4 | `brew install yarn` |
+| `trivy` | filesystem dep-vuln scan (OS + language packages) — feeds `VULN` gate | 1.4 | `brew install trivy` |
+| `biome` | TS/JS lint (preferred over eslint for speed) | 1.1 (TS/JS repos) | `npm i -g @biomejs/biome` |
 | `knip` | TS/JS dead-code + unused-deps | 1.4 | `npm i -g knip` |
-| `pipx` | isolated install for Python CLIs below | — | `brew install pipx && pipx ensurepath` |
+| `pipx` | isolated install for the Python CLIs below | — | `brew install pipx && pipx ensurepath` |
 | `deptry` | Python unused-deps | 1.4 | `pipx install deptry` |
 | `pip-audit` | Python dep-vuln scan — feeds `VULN` gate | 1.4 (Python repos) | `pipx install pip-audit` |
-| `govulncheck` | Go dep-vuln scan — feeds `VULN` gate | 1.4 (Go repos) | `go install golang.org/x/vuln/cmd/govulncheck@latest` (conditional — skipped if go not on PATH) |
-| `trivy` | Container + JS dep-vuln scan — feeds `VULN` gate | 1.4 | `brew install trivy` |
+| `govulncheck` | Go dep-vuln scan — feeds `VULN` gate | 1.4 (Go repos) | `go install golang.org/x/vuln/cmd/govulncheck@latest` — only when `go` is on PATH; otherwise skipped and Go vuln scans record `_NOT_RUN: true` with reason `"govulncheck unavailable (go not installed)"` |
 
-Host prerequisites that must already exist:
+Host prerequisites — the script never installs these and exits 2 if any is missing:
 
-- `brew` (macOS) or `apt` (Linux)
+- `brew`
+- `jq`
 - `node` + `npm` (≥ 18)
 - `python3` (≥ 3.10) — `pipx` layers on top
 
----
-
-## Invocation
-
-Run once per host: `bash .claude/skills/evaluator/preflight.sh`. Idempotent — re-running no-ops when every tool is already present. Canonical install commands live in `preflight.sh` (the executable contract); edit there, not here. Host prerequisites (`brew`, `node`/`npm`, `python3`) must already exist — the script halts loud if any are missing.
-
----
+Each install runs under a 180 s cap from `timeout` or `gtimeout`; on a host with neither, the script warns and installs uncapped (`brew install coreutils` restores the cap). Preflight ends by checking that the `lizard` on PATH is the cyclomatic-complexity analyzer, and fails if brew's compressor shadows it.
 
 ## Required behaviour for scoring agents
 
-1. **Run preflight first.** Before extracting any signal, execute the preflight script.
-2. **If preflight exits non-zero, STOP.** Do not proceed to scoring with missing tools. Return a clear failure message to the caller naming which tools failed to install.
-3. **No silent fallback.** A signal that requires a tool must either run the tool or emit `{signal}_NOT_RUN: true` with `reason: "preflight failed"` in the matching sub-criterion's `data` block in `scorecard.yaml`. This is a loud failure mode, not a graceful one.
-4. **Re-running preflight is free.** Every invocation should run it — the first run installs; subsequent runs no-op.
-
----
+1. **Run preflight first** — before extracting any signal, on every invocation; the first run installs, later runs no-op.
+2. **If preflight exits non-zero, stop.** Never score with missing tools; return a failure message to the caller naming the tools that failed to install.
+3. **No silent fallback.** A signal that needs a tool either runs it or emits `{signal}_NOT_RUN: true` with a `reason` naming the cause in that sub-criterion's `data` block in `scorecard.yaml` — a loud failure, never a graceful substitute.
 
 ## When a tool does not apply
 
-Some tools apply only when a repo contains the target language (e.g. `ruff` on Python, `golangci-lint` on Go, `knip` on TS/JS). In that case:
-
-- The tool is **not** required to be installed for repos where the language is absent.
-- The signal is emitted as `{signal}: null, reason: "language_not_present"` — NOT as `_NOT_RUN`.
-- Preflight still installs all tools idempotently; language filtering happens at the recipe level, not preflight.
-
-This separates "tool missing from host" (a failure) from "tool not applicable here" (expected).
-
----
-
-## Rationale
-
-Silent fallback to grep / LLM equivalents corrupts the scorecard in two ways:
-
-1. **Wrong bands.** The absolute banding tables in each recipe are calibrated against real tool output (e.g. `lizard` CCN, `semgrep` finding density). A grep-estimated substitute lands the project in the wrong band.
-2. **Under-reporting bias.** `gitleaks` catches entropic tokens `grep` cannot; `pip-audit` / `trivy` catch CVEs no heuristic will. Missing them turns a SECRETS- or VULN-gate hit into silence — a dishonest pass.
+A language-specific tool (`ruff` on Python, `golangci-lint` on Go, `knip` on TS/JS) does not apply to a repo without that language:
 
-Installing tools is cheap and one-time. Skipping them silently produces a scorecard that looks complete but isn't.
+- Emit the signal as `{signal}: null, reason: "language_not_present"` — **not** as `_NOT_RUN`, which means the tool was missing from the host (a failure), where `null` means not applicable here (expected).
+- Preflight still installs every tool; language filtering happens in the recipes, not in preflight.