@windyroad/risk-scorer 0.10.3 → 0.11.0-preview.387
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +2 -20
- package/hooks/external-comms-gate.sh +20 -3
- package/hooks/lib/external-comms-key.sh +78 -21
- package/hooks/lib/risk-gate.sh +43 -6
- package/hooks/test/external-comms-gate.bats +26 -0
- package/hooks/test/risk-gate.bats +91 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -65,7 +65,7 @@ The plugin includes six specialised agents:
|
|
|
65
65
|
| `wr-risk-scorer:plan` | Reviews implementation plans for risk |
|
|
66
66
|
| `wr-risk-scorer:policy` | Validates `RISK-POLICY.md` for ISO 31000 compliance |
|
|
67
67
|
| `wr-risk-scorer:external-comms` | Reviews drafts of outbound prose (gh issues/PRs, advisories, npm publish, changeset bodies) for confidential-information leaks per `RISK-POLICY.md` |
|
|
68
|
-
| `wr-risk-scorer:inbound-report` | Reviews inbound third-party reports (problem-report issues, Q&A discussions, security-advisory submissions) for Request-risk + Fix-risk per `RISK-POLICY.md` § Inbound Report Risk Classes — sibling of `:external-comms` (NOT extension). Consumed by the assessment-pipeline (P079 / ADR-062). Serves
|
|
68
|
+
| `wr-risk-scorer:inbound-report` | Reviews inbound third-party reports (problem-report issues, Q&A discussions, security-advisory submissions) for Request-risk + Fix-risk per `RISK-POLICY.md` § Inbound Report Risk Classes — sibling of `:external-comms` (NOT extension). Consumed by the assessment-pipeline (P079 / ADR-062). Serves the report-without-pre-classifying acknowledgement (verdict-on-close) and the mechanical-stage carve-out. |
|
|
69
69
|
|
|
70
70
|
## On-demand assessment skills
|
|
71
71
|
|
|
@@ -74,7 +74,7 @@ The plugin includes six specialised agents:
|
|
|
74
74
|
| `/wr-risk-scorer:assess-wip` | WIP risk nudge for the current uncommitted diff |
|
|
75
75
|
| `/wr-risk-scorer:assess-release` | Pipeline risk assessment for the unpushed queue (pre-satisfies the commit gate) |
|
|
76
76
|
| `/wr-risk-scorer:assess-external-comms` | External-comms leak review for a draft outbound body (pre-satisfies the external-comms gate) |
|
|
77
|
-
| `/wr-risk-scorer:assess-inbound-report` | Inbound-report risk review for a third-party submission — two-axis (Request-risk + Fix-risk) classification per `RISK-POLICY.md` (P079 / ADR-062). Serves
|
|
77
|
+
| `/wr-risk-scorer:assess-inbound-report` | Inbound-report risk review for a third-party submission — two-axis (Request-risk + Fix-risk) classification per `RISK-POLICY.md` (P079 / ADR-062). Serves on-demand assessment and pre-flight governance checks. |
|
|
78
78
|
| `/wr-risk-scorer:create-risk` | Create a standing-risk register entry (interactive authoring; orchestrator-driven prefilled invocation via `--slug` / `--prefill` flags per ADR-059) |
|
|
79
79
|
| `/wr-risk-scorer:bootstrap-catalog` | Bootstrap `docs/risks/` register from existing `.risk-reports/` corpus per ADR-059 — walks reports, dedupes by ADR-056 slug, emits one `R<NNN>-<slug>.active.md` per unique slug. Idempotent. Auto-triggers from `/install-updates` Step 6.5.1 when register is empty + `RISK-POLICY.md` present + `.risk-reports/` non-empty |
|
|
80
80
|
| `/wr-risk-scorer:update-policy` | Generate or update `RISK-POLICY.md` |
|
|
@@ -110,24 +110,6 @@ The canonical hook lives at `packages/shared/hooks/external-comms-gate.sh` and
|
|
|
110
110
|
is synced into each consumer plugin via `scripts/sync-external-comms-gate.sh`
|
|
111
111
|
per ADR-017 (CI runs `npm run check:external-comms-gate` to detect drift).
|
|
112
112
|
|
|
113
|
-
## Jobs to be Done
|
|
114
|
-
|
|
115
|
-
This plugin serves the [Jobs to be Done](../../docs/jtbd/) below. Per [ADR-051](../../docs/decisions/051-jtbd-anchored-readme-with-drift-advisory.proposed.md), the persona-grouped JTBD anchor is the canonical source of truth for the README's value framing.
|
|
116
|
-
|
|
117
|
-
### Tech lead / consultant
|
|
118
|
-
|
|
119
|
-
- **[JTBD-202 Run Pre-Flight Governance Checks Before Release or Handover](../../docs/jtbd/tech-lead/JTBD-202-pre-flight-governance-check.proposed.md)** — `/wr-risk-scorer:assess-release` produces a structured release-readiness score (commit, push, release layers) that is attachable to a release note or handover doc.
|
|
120
|
-
|
|
121
|
-
### Solo developer
|
|
122
|
-
|
|
123
|
-
- **[JTBD-001 Enforce Governance Without Slowing Down](../../docs/jtbd/solo-developer/JTBD-001-enforce-governance.proposed.md)** — pipeline risk is scored on every edit, commit, and push without manual invocation; secret-leak detection runs in the same gate.
|
|
124
|
-
- **[JTBD-002 Ship AI-Assisted Code with Confidence](../../docs/jtbd/solo-developer/JTBD-002-ship-with-confidence.proposed.md)** — every release passes through ISO 31000-aligned criteria defined in the project's own `RISK-POLICY.md` so the safety bar is the team's, not the agent's.
|
|
125
|
-
- **[JTBD-005 Invoke Governance Assessments On Demand](../../docs/jtbd/solo-developer/JTBD-005-assess-on-demand.proposed.md)** — `/wr-risk-scorer:assess-wip`, `assess-release`, and `assess-external-comms` give an on-demand assessment surface outside the hook gate cycle.
|
|
126
|
-
|
|
127
|
-
### Plugin user
|
|
128
|
-
|
|
129
|
-
- **[JTBD-302 Trust That the README Describes the Plugin I Just Installed](../../docs/jtbd/plugin-user/JTBD-302-trust-readme-describes-installed-behaviour.proposed.md)** — this README is anchored on current JTBD job IDs; drift between prose and shipped behaviour is detectable at retro time per ADR-051.
|
|
130
|
-
|
|
131
113
|
## Updating and Uninstalling
|
|
132
114
|
|
|
133
115
|
```bash
|
|
@@ -24,7 +24,13 @@
|
|
|
24
24
|
# (Voice-tone evaluator: skips leak pre-filter — leak detection is the
|
|
25
25
|
# risk evaluator's concern; voice-tone reviews tone/voice only.)
|
|
26
26
|
# 4. Otherwise: check for THIS evaluator's per-evaluator marker keyed on
|
|
27
|
-
#
|
|
27
|
+
# compute_external_comms_key(draft, surface) =
|
|
28
|
+
# sha256(normalize(draft, surface) + '\n' + surface) — the SINGLE
|
|
29
|
+
# canonical key shared with the mark hook (lib/external-comms-key.sh).
|
|
30
|
+
# For the changeset-author surface normalize() strips the leading YAML
|
|
31
|
+
# frontmatter block so the gate (which sees the FULL Write content) and
|
|
32
|
+
# the mark hook (which sees only the <draft> body) hash identical input
|
|
33
|
+
# (P010 / ADR-028 amended 2026-05-25). Marker present → permit.
|
|
28
34
|
# Marker absent → deny with directive to delegate to this plugin's
|
|
29
35
|
# subagent (configured via external-comms-evaluator.conf).
|
|
30
36
|
#
|
|
@@ -48,6 +54,12 @@ set -euo pipefail
|
|
|
48
54
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
49
55
|
# shellcheck source=lib/leak-detect.sh
|
|
50
56
|
source "$SCRIPT_DIR/lib/leak-detect.sh"
|
|
57
|
+
# shellcheck source=lib/external-comms-key.sh
|
|
58
|
+
# Provides compute_external_comms_key — the SINGLE canonical marker-key
|
|
59
|
+
# normalization shared with the PostToolUse mark hook (ADR-028 amended
|
|
60
|
+
# 2026-05-25 / P010). Sourced via the same $SCRIPT_DIR/lib convention as
|
|
61
|
+
# leak-detect.sh so byte-identity holds across the synced per-package copies.
|
|
62
|
+
source "$SCRIPT_DIR/lib/external-comms-key.sh"
|
|
51
63
|
|
|
52
64
|
# ---------- Per-package evaluator config (ADR-028 amended 2026-05-14) ----------
|
|
53
65
|
# Each consumer plugin ships its own external-comms-evaluator.conf alongside this
|
|
@@ -231,7 +243,12 @@ fi
|
|
|
231
243
|
# ---------- Marker-based gate (per-evaluator marker per ADR-028 amended 2026-05-14) ----------
|
|
232
244
|
SESSION_DIR="${TMPDIR:-/tmp}/claude-risk-${SESSION_ID}"
|
|
233
245
|
mkdir -p "$SESSION_DIR"
|
|
234
|
-
|
|
246
|
+
# Canonical marker key — normalize() strips changeset frontmatter + trailing
|
|
247
|
+
# whitespace so this PreToolUse key matches the PostToolUse mark-hook key
|
|
248
|
+
# (compute_external_comms_key in lib/external-comms-key.sh; P010 / ADR-028
|
|
249
|
+
# amended 2026-05-25). For changeset-author $DRAFT is the FULL Write content
|
|
250
|
+
# (frontmatter + body); compute_external_comms_key strips the frontmatter.
|
|
251
|
+
KEY=$(compute_external_comms_key "$DRAFT" "$SURFACE")
|
|
235
252
|
MARKER="${SESSION_DIR}/external-comms-${EXTERNAL_COMMS_EVALUATOR_ID}-reviewed-${KEY}"
|
|
236
253
|
|
|
237
254
|
if [ -f "$MARKER" ]; then
|
|
@@ -244,7 +261,7 @@ fi
|
|
|
244
261
|
# PostToolUse mark hook can derive the canonical marker key locally
|
|
245
262
|
# (sha256(DRAFT + '\n' + SURFACE)). Single fire per gate cycle.
|
|
246
263
|
VERDICT_PREFIX="${EXTERNAL_COMMS_VERDICT_PREFIX:-EXTERNAL_COMMS_${EXTERNAL_COMMS_EVALUATOR_ID^^}}"
|
|
247
|
-
REASON=$(printf 'BLOCKED (external-comms gate / %s evaluator): %s draft has not been reviewed by %s. Delegate to %s (subagent_type: '"'"'%s'"'"') with a prompt that starts with the line `SURFACE: %s` and wraps the draft body verbatim inside `<draft>...</draft>` markers. The PostToolUse hook derives the marker key from that structure and marks the draft reviewed when the subagent emits %s_VERDICT: PASS — single fire suffices. Use %s for an interactive walkthrough. Override only when intentional: BYPASS_RISK_GATE=1.' \
|
|
264
|
+
REASON=$(printf 'BLOCKED (external-comms gate / %s evaluator): %s draft has not been reviewed by %s. Delegate to %s (subagent_type: '"'"'%s'"'"') with a prompt that starts with the line `SURFACE: %s` and wraps the draft body verbatim inside `<draft>...</draft>` markers (for the changeset-author surface the body is the changeset summary WITHOUT the leading `---` frontmatter block — the gate strips frontmatter before hashing the marker key). The PostToolUse hook derives the marker key from that structure and marks the draft reviewed when the subagent emits %s_VERDICT: PASS — single fire suffices. Use %s for an interactive walkthrough. Override only when intentional: BYPASS_RISK_GATE=1.' \
|
|
248
265
|
"$EXTERNAL_COMMS_EVALUATOR_ID" "$SURFACE" "$EXTERNAL_COMMS_SUBAGENT_TYPE" "$EXTERNAL_COMMS_SUBAGENT_TYPE" "$EXTERNAL_COMMS_SUBAGENT_TYPE" "$SURFACE" "$VERDICT_PREFIX" "$EXTERNAL_COMMS_ASSESS_SKILL")
|
|
249
266
|
deny_with_reason "$REASON"
|
|
250
267
|
exit 0
|
|
@@ -1,44 +1,101 @@
|
|
|
1
1
|
#!/bin/bash
|
|
2
2
|
# Shared helper: derive the external-comms marker key from an agent's
|
|
3
3
|
# tool_input.prompt by extracting the structured `SURFACE: <name>` line
|
|
4
|
-
# and `<draft>...</draft>` block, then computing
|
|
5
|
-
#
|
|
6
|
-
#
|
|
4
|
+
# and `<draft>...</draft>` block, then computing the canonical key via
|
|
5
|
+
# compute_external_comms_key — the same key shape the gate computes at
|
|
6
|
+
# PreToolUse time (external-comms-gate.sh).
|
|
7
7
|
#
|
|
8
8
|
# P166 + ADR-028 amended 2026-05-16: the PostToolUse:Agent mark hook
|
|
9
9
|
# derives the marker key from observed runtime state instead of trusting
|
|
10
10
|
# an agent-emitted EXTERNAL_COMMS_<EVAL>_KEY line. Removes the
|
|
11
11
|
# double-invocation cost class — single fire per gate cycle suffices.
|
|
12
12
|
#
|
|
13
|
+
# P010 + ADR-028 amended 2026-05-25: the marker key is computed via the
|
|
14
|
+
# SINGLE canonical normalization in compute_external_comms_key, shared
|
|
15
|
+
# byte-for-byte between the gate (PreToolUse) and this mark-hook helper
|
|
16
|
+
# (PostToolUse). The normalization strips the changeset YAML frontmatter
|
|
17
|
+
# block before hashing (the gate sees the FULL Write content incl.
|
|
18
|
+
# frontmatter; the agent wraps only the body in <draft>) and applies a
|
|
19
|
+
# single canonical trailing-whitespace strip so the two sides cannot
|
|
20
|
+
# diverge. Fixes the deny-after-PASS marker-key mismatch (P198 / #149).
|
|
21
|
+
#
|
|
13
22
|
# Canonical source: packages/shared/hooks/lib/external-comms-key.sh
|
|
14
23
|
# Synced byte-identically into each consumer plugin's hooks/lib/ via
|
|
15
24
|
# scripts/sync-external-comms-gate.sh (ADR-017 duplicate-script pattern).
|
|
25
|
+
|
|
26
|
+
# ---------------------------------------------------------------------------
|
|
27
|
+
# compute_external_comms_key <draft> <surface>
|
|
16
28
|
#
|
|
17
|
-
#
|
|
18
|
-
#
|
|
19
|
-
#
|
|
20
|
-
#
|
|
29
|
+
# THE single source of truth for the external-comms marker key. Both the
|
|
30
|
+
# PreToolUse gate and the PostToolUse mark hook compute the key through
|
|
31
|
+
# this function so they hash byte-identical input (ADR-028 amended
|
|
32
|
+
# 2026-05-25). Echoes the 64-char lowercase sha256 hex on stdout.
|
|
33
|
+
#
|
|
34
|
+
# normalize(draft, surface):
|
|
35
|
+
# - changeset-author surface: strip the leading YAML frontmatter block
|
|
36
|
+
# (`---\n...\n---\n` plus the blank line after it). Changeset files
|
|
37
|
+
# carry `---\n"@windyroad/x": minor\n---\n\n<body>`; the gate sees the
|
|
38
|
+
# whole thing via tool_input.content while the agent wraps only the
|
|
39
|
+
# <body> in <draft>. Stripping frontmatter makes the two inputs equal.
|
|
40
|
+
# All other surfaces (gh-*, npm-publish) are already body-only via the
|
|
41
|
+
# gate's --body / --field extraction, so they are left unchanged.
|
|
42
|
+
# - all surfaces: rstrip ALL trailing whitespace. This single canonical
|
|
43
|
+
# newline normalization subsumes both the gate's `$()` trailing-newline
|
|
44
|
+
# strip and this helper's `<draft>` regex single-newline strip, so the
|
|
45
|
+
# two sides are provably symmetric on trailing whitespace.
|
|
46
|
+
# key = sha256(normalize(draft, surface) + '\n' + surface)
|
|
47
|
+
# ---------------------------------------------------------------------------
|
|
48
|
+
compute_external_comms_key() {
|
|
49
|
+
local draft="$1" surface="$2"
|
|
50
|
+
EXTCOMMS_DRAFT="$draft" EXTCOMMS_SURFACE="$surface" python3 -c "
|
|
51
|
+
import os, re, hashlib
|
|
52
|
+
draft = os.environ.get('EXTCOMMS_DRAFT', '')
|
|
53
|
+
surface = os.environ.get('EXTCOMMS_SURFACE', '')
|
|
54
|
+
# changeset-author: strip the leading YAML frontmatter block + blank line.
|
|
55
|
+
if surface == 'changeset-author':
|
|
56
|
+
draft = re.sub(r'^---\n.*?\n---\n\n?', '', draft, count=1, flags=re.DOTALL)
|
|
57
|
+
# Single canonical newline normalization: strip all trailing whitespace.
|
|
58
|
+
draft = draft.rstrip()
|
|
59
|
+
print(hashlib.sha256((draft + '\n' + surface).encode('utf-8')).hexdigest())
|
|
60
|
+
" 2>/dev/null
|
|
61
|
+
}
|
|
21
62
|
|
|
63
|
+
# ---------------------------------------------------------------------------
|
|
64
|
+
# derive_external_comms_key_from_prompt <prompt>
|
|
65
|
+
#
|
|
66
|
+
# Extracts the (SURFACE, draft-body) pair from an agent prompt's structured
|
|
67
|
+
# `SURFACE: <name>` line + `<draft>...</draft>` block, then delegates to
|
|
68
|
+
# compute_external_comms_key so the normalization lives in exactly one place.
|
|
69
|
+
#
|
|
70
|
+
# Returns the 64-char hex sha256 on stdout when both markers are present in
|
|
71
|
+
# the prompt. Returns empty string when either marker is absent — the caller
|
|
72
|
+
# falls back to the agent-emitted KEY for backward compatibility with cached
|
|
73
|
+
# old SKILL.md / agent prompts.
|
|
74
|
+
# ---------------------------------------------------------------------------
|
|
22
75
|
derive_external_comms_key_from_prompt() {
|
|
23
76
|
local prompt="$1"
|
|
24
77
|
[ -n "$prompt" ] || { echo ""; return 0; }
|
|
25
|
-
|
|
26
|
-
|
|
78
|
+
# Extract SURFACE + <draft> body in one pass. The two fields are emitted
|
|
79
|
+
# \x1f-separated (ASCII unit separator) so a body containing newlines — or
|
|
80
|
+
# an empty body — round-trips through command substitution intact (only
|
|
81
|
+
# trailing newlines are dropped, which compute_external_comms_key rstrips
|
|
82
|
+
# anyway). Empty output when either marker is absent → empty key.
|
|
83
|
+
local extracted
|
|
84
|
+
extracted=$(printf '%s' "$prompt" | python3 -c "
|
|
85
|
+
import sys, re
|
|
27
86
|
text = sys.stdin.read()
|
|
28
|
-
# DRAFT
|
|
29
|
-
#
|
|
30
|
-
# so the body content does not capture wrapping newlines.
|
|
87
|
+
# DRAFT: non-greedy match between <draft>...</draft>, tolerating an optional
|
|
88
|
+
# newline immediately after <draft> and before </draft>.
|
|
31
89
|
draft_match = re.search(r'<draft>\n?(.*?)\n?</draft>', text, re.DOTALL)
|
|
32
|
-
# SURFACE
|
|
33
|
-
#
|
|
34
|
-
# token: letter + word/hyphen chars.
|
|
90
|
+
# SURFACE: anchored to line start (MULTILINE) so prose like 'context says
|
|
91
|
+
# SURFACE: x' does not match. Surface name is a single letter+word/hyphen token.
|
|
35
92
|
surface_match = re.search(r'^SURFACE:\s*([A-Za-z][\w-]*)', text, re.MULTILINE)
|
|
36
93
|
if not draft_match or not surface_match:
|
|
37
|
-
print('')
|
|
38
94
|
sys.exit(0)
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
"
|
|
95
|
+
sys.stdout.write(surface_match.group(1) + '\x1f' + draft_match.group(1))
|
|
96
|
+
" 2>/dev/null) || extracted=""
|
|
97
|
+
[ -n "$extracted" ] || { echo ""; return 0; }
|
|
98
|
+
local surface="${extracted%%$'\x1f'*}"
|
|
99
|
+
local body="${extracted#*$'\x1f'}"
|
|
100
|
+
compute_external_comms_key "$body" "$surface"
|
|
44
101
|
}
|
package/hooks/lib/risk-gate.sh
CHANGED
|
@@ -100,16 +100,53 @@ check_risk_gate() {
|
|
|
100
100
|
return 1
|
|
101
101
|
fi
|
|
102
102
|
|
|
103
|
-
# 5. Threshold check
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
103
|
+
# 5. Threshold check — block when the score EXCEEDS the project's
|
|
104
|
+
# RISK-POLICY.md risk appetite (P007 / ADR-065). The threshold is the
|
|
105
|
+
# adopter's documented appetite, not a code constant: a project whose
|
|
106
|
+
# policy sets a higher appetite (e.g. "exceeds 9") must not have its
|
|
107
|
+
# within-appetite changes gate-rejected.
|
|
108
|
+
# Precedence: RISK_APPETITE env override > RISK-POLICY.md § Risk Appetite
|
|
109
|
+
# parse > default 4. Default 4 reproduces the prior hardcoded `score >= 5`
|
|
110
|
+
# behaviour exactly for integer scores (5 blocks, 4 passes) when the
|
|
111
|
+
# policy is absent or unparseable. The parse is tolerant of the phrasings
|
|
112
|
+
# "Threshold: N", "exceeds N", and "N/Low appetite", scoped to the
|
|
113
|
+
# "## Risk Appetite" section. Cost ~3-8ms/invocation (ADR-065 § Consequences).
|
|
114
|
+
local DECISION
|
|
115
|
+
DECISION=$(RISK_SCORE_VAL="$SCORE" RISK_APPETITE_ENV="${RISK_APPETITE:-}" python3 -c "
|
|
116
|
+
import os, re, sys
|
|
117
|
+
try:
|
|
118
|
+
score = float(os.environ['RISK_SCORE_VAL'])
|
|
119
|
+
except Exception:
|
|
120
|
+
print('no 4'); sys.exit(0)
|
|
121
|
+
N = None
|
|
122
|
+
override = os.environ.get('RISK_APPETITE_ENV', '').strip()
|
|
123
|
+
if override.isdigit():
|
|
124
|
+
N = int(override)
|
|
125
|
+
else:
|
|
126
|
+
try:
|
|
127
|
+
text = open('RISK-POLICY.md', encoding='utf-8').read()
|
|
128
|
+
except Exception:
|
|
129
|
+
text = ''
|
|
130
|
+
if text:
|
|
131
|
+
# Scope to the '## Risk Appetite' section so unrelated numbers
|
|
132
|
+
# elsewhere in the policy cannot match.
|
|
133
|
+
sec = re.search(r'##\s*Risk Appetite\s*(.*?)(?=\n##\s|\Z)', text, re.DOTALL | re.IGNORECASE)
|
|
134
|
+
scope = sec.group(1) if sec else text
|
|
135
|
+
for pat in (r'Threshold:\s*(\d+)', r'exceeds\s+(\d+)', r'(\d+)\s*/\s*Low appetite'):
|
|
136
|
+
m = re.search(pat, scope, re.IGNORECASE)
|
|
137
|
+
if m:
|
|
138
|
+
N = int(m.group(1)); break
|
|
139
|
+
if N is None:
|
|
140
|
+
N = 4
|
|
141
|
+
print(('yes' if score > N else 'no') + ' ' + str(N))
|
|
142
|
+
" 2>/dev/null || echo "no 4")
|
|
143
|
+
local DENIED="${DECISION%% *}"
|
|
144
|
+
local APPETITE="${DECISION##* }"
|
|
108
145
|
|
|
109
146
|
if [ "$DENIED" = "yes" ]; then
|
|
110
147
|
RISK_GATE_CATEGORY="threshold"
|
|
111
148
|
RISK_GATE_SCORE="$SCORE"
|
|
112
|
-
RISK_GATE_REASON="${ACTION} risk score ${SCORE}/25
|
|
149
|
+
RISK_GATE_REASON="${ACTION} risk score ${SCORE}/25 exceeds the project appetite of ${APPETITE}/25 (RISK-POLICY.md). To proceed: (1) split the ${ACTION}, (2) add risk-reducing measures, or (3) for a LIVE INCIDENT, delegate to wr-risk-scorer:pipeline (subagent_type: 'wr-risk-scorer:pipeline') with incident context for an incident bypass."
|
|
113
150
|
return 1
|
|
114
151
|
fi
|
|
115
152
|
|
|
@@ -186,3 +186,29 @@ run_hook() {
|
|
|
186
186
|
[[ "$output" == *"deny"* ]]
|
|
187
187
|
[[ "$output" == *"wr-risk-scorer:external-comms"* ]]
|
|
188
188
|
}
|
|
189
|
+
|
|
190
|
+
# ---------------------------------------------------------------------------
|
|
191
|
+
# P010 / ADR-028 amended 2026-05-25 — deny-after-PASS regression.
|
|
192
|
+
# The gate sees the FULL Write content (YAML frontmatter + body) on the
|
|
193
|
+
# changeset-author surface, but the mark hook keys the marker on the body
|
|
194
|
+
# the agent wrapped in <draft>. Before the fix the gate hashed the full
|
|
195
|
+
# content (incl. frontmatter), so the body-keyed PASS marker landed at a
|
|
196
|
+
# key the gate never re-read → permanent deny-after-PASS. After the fix the
|
|
197
|
+
# gate strips frontmatter before hashing, so a body-keyed marker permits.
|
|
198
|
+
# ---------------------------------------------------------------------------
|
|
199
|
+
|
|
200
|
+
@test "P010: changeset Write permits when the PASS marker is keyed on the <draft> body (frontmatter stripped before hash)" {
|
|
201
|
+
BODY="external-comms gate strips changeset frontmatter before key hash"
|
|
202
|
+
SURFACE="changeset-author"
|
|
203
|
+
# Marker keyed on the body the mark-hook helper derives from <draft> — for a
|
|
204
|
+
# frontmatter-free body the canonical key equals the raw printf-of-body key.
|
|
205
|
+
KEY=$(printf '%s\n%s' "$BODY" "$SURFACE" | shasum -a 256 | cut -d' ' -f1)
|
|
206
|
+
touch "${RDIR}/external-comms-risk-reviewed-${KEY}"
|
|
207
|
+
|
|
208
|
+
# The gate sees the full changeset file: frontmatter + blank line + body.
|
|
209
|
+
CONTENT=$'---\n"@windyroad/risk-scorer": patch\n---\n\n'"$BODY"
|
|
210
|
+
INPUT=$(build_write_input ".changeset/p010-fix.md" "$CONTENT")
|
|
211
|
+
run_hook "$INPUT"
|
|
212
|
+
[ "$status" -eq 0 ]
|
|
213
|
+
[ -z "$output" ]
|
|
214
|
+
}
|
|
@@ -17,6 +17,21 @@ setup() {
|
|
|
17
17
|
|
|
18
18
|
teardown() {
|
|
19
19
|
rm -rf "${TMPDIR:-/tmp}/claude-risk-${TEST_SESSION}"
|
|
20
|
+
[ -n "${POLICY_DIR:-}" ] && rm -rf "$POLICY_DIR"
|
|
21
|
+
unset RISK_APPETITE 2>/dev/null || true
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
# Write a RISK-POLICY.md whose § Risk Appetite carries $1 into a fresh temp
|
|
25
|
+
# dir and cd into it, so check_risk_gate reads the appetite from cwd. Pass an
|
|
26
|
+
# empty string to OMIT the file entirely (absent-policy default path).
|
|
27
|
+
_use_policy() {
|
|
28
|
+
local appetite_section="$1"
|
|
29
|
+
POLICY_DIR=$(mktemp -d)
|
|
30
|
+
if [ -n "$appetite_section" ]; then
|
|
31
|
+
printf '# Risk Policy\n\n## Risk Appetite\n\n%s\n\n## Impact Levels\n| L | x |\n' \
|
|
32
|
+
"$appetite_section" > "$POLICY_DIR/RISK-POLICY.md"
|
|
33
|
+
fi
|
|
34
|
+
cd "$POLICY_DIR"
|
|
20
35
|
}
|
|
21
36
|
|
|
22
37
|
# Helper: call check_risk_gate directly (not via run) so RISK_GATE_REASON is visible
|
|
@@ -219,3 +234,79 @@ _write_matching_hash() {
|
|
|
219
234
|
[ "$RISK_GATE_CATEGORY" = "threshold" ]
|
|
220
235
|
[ "$RISK_GATE_SCORE" = "7" ]
|
|
221
236
|
}
|
|
237
|
+
|
|
238
|
+
# ---------------------------------------------------------------------------
|
|
239
|
+
# P007 / ADR-065 — the block threshold is the project's RISK-POLICY.md risk
|
|
240
|
+
# appetite (block when score > N), not a hardcoded 5. Default N=4 when the
|
|
241
|
+
# policy is absent or unparseable, which reproduces the prior `score >= 5`
|
|
242
|
+
# behaviour exactly for integer scores. Precedence: RISK_APPETITE env >
|
|
243
|
+
# RISK-POLICY.md parse > default 4. (Behavioural fixtures per ADR-052.)
|
|
244
|
+
# ---------------------------------------------------------------------------
|
|
245
|
+
|
|
246
|
+
@test "appetite 9 (exceeds 9): score 7 within the 5-9 band PASSES" {
|
|
247
|
+
_use_policy 'Pipeline gates block when cumulative residual risk exceeds 9.'
|
|
248
|
+
printf '7' > "$SCORE_FILE"
|
|
249
|
+
rm -f "$HASH_FILE"
|
|
250
|
+
assert_gate_allows "$TEST_SESSION" "commit"
|
|
251
|
+
}
|
|
252
|
+
|
|
253
|
+
@test "appetite 9 (exceeds 9): score 10 above the band FAILS and deny renders the parsed appetite 9/25" {
|
|
254
|
+
_use_policy 'Pipeline gates block when cumulative residual risk exceeds 9.'
|
|
255
|
+
printf '10' > "$SCORE_FILE"
|
|
256
|
+
rm -f "$HASH_FILE"
|
|
257
|
+
assert_gate_denies "$TEST_SESSION" "commit" "appetite of 9/25"
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
@test "appetite via 'Threshold: 9' phrasing: score 9 PASSES, score 10 FAILS" {
|
|
261
|
+
_use_policy '**Threshold: 9 (Medium)**'
|
|
262
|
+
rm -f "$HASH_FILE"
|
|
263
|
+
printf '9' > "$SCORE_FILE"
|
|
264
|
+
assert_gate_allows "$TEST_SESSION" "commit"
|
|
265
|
+
printf '10' > "$SCORE_FILE"
|
|
266
|
+
assert_gate_denies "$TEST_SESSION" "commit" "10/25"
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
@test "appetite 4 (exceeds 4): score 4 PASSES, score 5 FAILS" {
|
|
270
|
+
_use_policy 'Pipeline gates block when cumulative residual risk exceeds 4.'
|
|
271
|
+
rm -f "$HASH_FILE"
|
|
272
|
+
printf '4' > "$SCORE_FILE"
|
|
273
|
+
assert_gate_allows "$TEST_SESSION" "commit"
|
|
274
|
+
printf '5' > "$SCORE_FILE"
|
|
275
|
+
assert_gate_denies "$TEST_SESSION" "commit" "appetite of 4/25"
|
|
276
|
+
}
|
|
277
|
+
|
|
278
|
+
@test "absent RISK-POLICY.md: defaults to appetite 4 (4 PASSES, 5 FAILS)" {
|
|
279
|
+
_use_policy '' # no RISK-POLICY.md in the temp dir
|
|
280
|
+
rm -f "$HASH_FILE"
|
|
281
|
+
printf '4' > "$SCORE_FILE"
|
|
282
|
+
assert_gate_allows "$TEST_SESSION" "commit"
|
|
283
|
+
printf '5' > "$SCORE_FILE"
|
|
284
|
+
assert_gate_denies "$TEST_SESSION" "commit" "appetite of 4/25"
|
|
285
|
+
}
|
|
286
|
+
|
|
287
|
+
@test "unparseable RISK-POLICY.md (no appetite integer): defaults to appetite 4" {
|
|
288
|
+
_use_policy 'We are conservative about risk but state no number here.'
|
|
289
|
+
rm -f "$HASH_FILE"
|
|
290
|
+
printf '4' > "$SCORE_FILE"
|
|
291
|
+
assert_gate_allows "$TEST_SESSION" "commit"
|
|
292
|
+
printf '5' > "$SCORE_FILE"
|
|
293
|
+
assert_gate_denies "$TEST_SESSION" "commit" "appetite of 4/25"
|
|
294
|
+
}
|
|
295
|
+
|
|
296
|
+
@test "fractional score 4.5 FAILS under default appetite 4 (4.5 > 4)" {
|
|
297
|
+
_use_policy ''
|
|
298
|
+
rm -f "$HASH_FILE"
|
|
299
|
+
printf '4.5' > "$SCORE_FILE"
|
|
300
|
+
assert_gate_denies "$TEST_SESSION" "commit" "4.5/25"
|
|
301
|
+
}
|
|
302
|
+
|
|
303
|
+
@test "RISK_APPETITE env override takes precedence over RISK-POLICY.md parse" {
|
|
304
|
+
_use_policy 'Pipeline gates block when cumulative residual risk exceeds 4.'
|
|
305
|
+
rm -f "$HASH_FILE"
|
|
306
|
+
RISK_APPETITE=9
|
|
307
|
+
printf '7' > "$SCORE_FILE"
|
|
308
|
+
assert_gate_allows "$TEST_SESSION" "commit"
|
|
309
|
+
printf '10' > "$SCORE_FILE"
|
|
310
|
+
assert_gate_denies "$TEST_SESSION" "commit" "appetite of 9/25"
|
|
311
|
+
unset RISK_APPETITE
|
|
312
|
+
}
|
package/package.json
CHANGED