@windyroad/risk-scorer 0.2.0-preview.62 → 0.2.1-preview.68
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/pipeline.md +24 -7
- package/agents/plan.md +10 -2
- package/agents/wip.md +17 -7
- package/hooks/test/risk-score-mark.bats +84 -11
- package/package.json +1 -1
package/agents/pipeline.md
CHANGED
|
@@ -98,18 +98,35 @@ Assess whether each action is risk-reducing, risk-neutral, or risk-increasing. I
|
|
|
98
98
|
|
|
99
99
|
For live incidents (outage, security, information disclosure), include `RISK_BYPASS: incident`.
|
|
100
100
|
|
|
101
|
-
##
|
|
101
|
+
## Below-Appetite Output Rule (ADR-013 Rule 5)
|
|
102
102
|
|
|
103
|
-
|
|
104
|
-
|
|
103
|
+
When ALL cumulative scores are within appetite (≤ 4 per RISK-POLICY.md), your output MUST contain ONLY:
|
|
104
|
+
1. The Pipeline Risk Report structure (layers, risk items, summary table)
|
|
105
|
+
2. `RISK_SCORES: commit=N push=N release=N`
|
|
106
|
+
3. `RISK_BYPASS: reducing` (if applicable)
|
|
105
107
|
|
|
106
|
-
|
|
108
|
+
Do NOT emit: "Suggested Actions", "Your call:", advisory warnings, back-pressure notes, or any prose that implies the user needs to make a decision. Policy-authorised releases proceed silently.
|
|
107
109
|
|
|
108
|
-
|
|
110
|
+
## Above-Appetite Remediations
|
|
111
|
+
|
|
112
|
+
When ANY cumulative score exceeds appetite (> 4), emit a structured `RISK_REMEDIATIONS:` block after the `RISK_SCORES:` line. This gives the calling skill machine-readable input for structured decision prompts.
|
|
113
|
+
|
|
114
|
+
Format:
|
|
115
|
+
```
|
|
116
|
+
RISK_REMEDIATIONS:
|
|
117
|
+
- R1 | <description of remediation> | <files affected>
|
|
118
|
+
- R2 | <description of remediation> | <files affected>
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Include downstream back-pressure in the remediation list:
|
|
122
|
+
- **Commit**: If adding this commit would push the push queue risk >= 5, include a remediation to split the commit.
|
|
123
|
+
- **Push**: If pushing would push the release queue risk >= 5, include a remediation to release first.
|
|
109
124
|
|
|
110
|
-
|
|
125
|
+
Do NOT emit free-text "Your call:" or "consider splitting" prose. The structured `RISK_REMEDIATIONS:` block is the only output for above-appetite guidance.
|
|
111
126
|
|
|
112
|
-
|
|
127
|
+
## Confidential Information Disclosure
|
|
128
|
+
|
|
129
|
+
Check diffs for business metrics (revenue, user counts, pricing, traffic volumes). Flag as a standalone risk if found.
|
|
113
130
|
|
|
114
131
|
## Report History
|
|
115
132
|
|
package/agents/plan.md
CHANGED
|
@@ -22,8 +22,8 @@ You are the Risk Scorer in plan review mode. Assess both the plan's own risk AND
|
|
|
22
22
|
|
|
23
23
|
## Verdict Logic
|
|
24
24
|
|
|
25
|
-
- **PASS** if both the plan's own residual risk AND projected release risk are within appetite
|
|
26
|
-
- **FAIL** if either exceeds appetite —
|
|
25
|
+
- **PASS** if both the plan's own residual risk AND projected release risk are within appetite. Do NOT emit advisory prose, suggestions, or "consider" recommendations on PASS — the plan is policy-authorised (ADR-013 Rule 5).
|
|
26
|
+
- **FAIL** if either exceeds appetite — emit a structured `RISK_REMEDIATIONS:` block (see below) explaining which dimension failed and what the plan should include.
|
|
27
27
|
|
|
28
28
|
## Output Format
|
|
29
29
|
|
|
@@ -49,6 +49,14 @@ You are the Risk Scorer in plan review mode. Assess both the plan's own risk AND
|
|
|
49
49
|
|
|
50
50
|
End your report with `RISK_VERDICT: PASS` or `RISK_VERDICT: FAIL` on its own line. A PostToolUse hook reads this and writes the marker files — do NOT write files yourself.
|
|
51
51
|
|
|
52
|
+
On FAIL, emit a structured `RISK_REMEDIATIONS:` block after the verdict:
|
|
53
|
+
```
|
|
54
|
+
RISK_REMEDIATIONS:
|
|
55
|
+
- R1 | <description of what the plan must add/change> | <affected area>
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Do NOT emit free-text "consider" or "you should" prose. The structured block is the only output for above-appetite guidance.
|
|
59
|
+
|
|
52
60
|
## Control Discovery
|
|
53
61
|
|
|
54
62
|
For each control claimed to reduce risk:
|
package/agents/wip.md
CHANGED
|
@@ -45,15 +45,25 @@ Always provide the cumulative risk picture:
|
|
|
45
45
|
- [specific guidance based on current pipeline state]
|
|
46
46
|
```
|
|
47
47
|
|
|
48
|
-
|
|
48
|
+
### Below-Appetite Rule (ADR-013 Rule 5)
|
|
49
49
|
|
|
50
|
-
If cumulative risk **
|
|
51
|
-
- "Commit your current changes to move WIP forward"
|
|
52
|
-
- "Write tests for [risk item from report]" — name the specific risk and test file
|
|
53
|
-
- "The release report flags [X] — address it before adding more changes"
|
|
54
|
-
- "Push your commits to get CI feedback"
|
|
50
|
+
If cumulative risk is **within appetite** (< 5): provide the assessment table and verdict only. Do NOT emit advisory prose, recommendations, or suggestions. The verdict is `RISK_VERDICT: CONTINUE`.
|
|
55
51
|
|
|
56
|
-
|
|
52
|
+
### Above-Appetite Remediations
|
|
53
|
+
|
|
54
|
+
If cumulative risk **exceeds appetite** (>= 5): provide the assessment table, then emit a structured `RISK_REMEDIATIONS:` block with specific risk-reducing actions:
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
RISK_REMEDIATIONS:
|
|
58
|
+
- R1 | Commit current changes to move WIP forward | <uncommitted files>
|
|
59
|
+
- R2 | Write tests for <risk item from report> | <test file to create/extend>
|
|
60
|
+
- R3 | Address release report risk <X> before adding more changes | <affected files>
|
|
61
|
+
- R4 | Push commits to get CI feedback | N/A
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Do NOT emit free-text suggestions as prose. The structured block is the only output for above-appetite guidance.
|
|
65
|
+
|
|
66
|
+
The verdict is `RISK_VERDICT: PAUSE`. This blocks the next edit until the risk is addressed.
|
|
57
67
|
|
|
58
68
|
## Control Discovery
|
|
59
69
|
|
|
@@ -1,28 +1,101 @@
|
|
|
1
1
|
#!/usr/bin/env bats
|
|
2
2
|
|
|
3
|
-
# Tests for risk-score-mark.sh
|
|
3
|
+
# Tests for risk-score-mark.sh — verifies the PostToolUse:Agent hook
|
|
4
|
+
# parses risk-scorer agent output and writes the right files into
|
|
5
|
+
# the session-scoped risk dir.
|
|
6
|
+
#
|
|
7
|
+
# Per ADR-005 (P011): behavioural assertions are functional — they
|
|
8
|
+
# pipe mock hook input to the script and assert on side-effects, not
|
|
9
|
+
# on what the source happens to contain. The four "echo X | grep X"
|
|
10
|
+
# tautologies that previously lived here have been removed (they
|
|
11
|
+
# always passed regardless of hook behaviour).
|
|
4
12
|
|
|
5
13
|
setup() {
|
|
6
14
|
SCRIPT_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)"
|
|
15
|
+
HOOK="$SCRIPT_DIR/risk-score-mark.sh"
|
|
16
|
+
ORIG_DIR="$PWD"
|
|
17
|
+
TEST_DIR=$(mktemp -d)
|
|
18
|
+
cd "$TEST_DIR"
|
|
19
|
+
TMPDIR="$TEST_DIR/tmp"
|
|
20
|
+
export TMPDIR
|
|
21
|
+
mkdir -p "$TMPDIR"
|
|
22
|
+
SESSION_ID="test-session-$$"
|
|
23
|
+
RDIR="$TMPDIR/claude-risk-${SESSION_ID}"
|
|
7
24
|
}
|
|
8
25
|
|
|
9
|
-
|
|
10
|
-
|
|
26
|
+
teardown() {
|
|
27
|
+
cd "$ORIG_DIR"
|
|
28
|
+
rm -rf "$TEST_DIR"
|
|
11
29
|
}
|
|
12
30
|
|
|
13
|
-
|
|
14
|
-
|
|
31
|
+
# Helper: build the PostToolUse:Agent JSON envelope and pipe it to the hook.
|
|
32
|
+
# AGENT_OUTPUT is wrapped in tool_response.content[0].text to match the
|
|
33
|
+
# real Claude Code PostToolUse hook payload shape.
|
|
34
|
+
run_hook() {
|
|
35
|
+
local subagent="$1"
|
|
36
|
+
local agent_output="$2"
|
|
37
|
+
python3 -c "
|
|
38
|
+
import json, sys
|
|
39
|
+
print(json.dumps({
|
|
40
|
+
'tool_name': 'Agent',
|
|
41
|
+
'session_id': '${SESSION_ID}',
|
|
42
|
+
'tool_input': {'subagent_type': '${subagent}'},
|
|
43
|
+
'tool_response': {'content': [{'type': 'text', 'text': sys.stdin.read()}]}
|
|
44
|
+
}))" <<<"$agent_output" | bash "$HOOK"
|
|
15
45
|
}
|
|
16
46
|
|
|
17
|
-
|
|
18
|
-
|
|
47
|
+
# --- Pipeline scorer: writes commit/push/release score files ---
|
|
48
|
+
|
|
49
|
+
@test "pipeline: writes commit/push/release scores from RISK_SCORES line" {
|
|
50
|
+
run_hook "wr-risk-scorer:pipeline" "Header text
|
|
51
|
+
RISK_SCORES: commit=2 push=3 release=1
|
|
52
|
+
Trailing text"
|
|
53
|
+
[ "$(cat "$RDIR/commit")" = "2" ]
|
|
54
|
+
[ "$(cat "$RDIR/push")" = "3" ]
|
|
55
|
+
[ "$(cat "$RDIR/release")" = "1" ]
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
@test "pipeline: writes reducing bypass markers when RISK_BYPASS: reducing" {
|
|
59
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=2 push=2 release=0
|
|
60
|
+
RISK_BYPASS: reducing"
|
|
61
|
+
[ -f "$RDIR/reducing-commit" ]
|
|
62
|
+
[ -f "$RDIR/reducing-push" ]
|
|
63
|
+
[ -f "$RDIR/reducing-release" ]
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
@test "pipeline: writes incident bypass marker when RISK_BYPASS: incident" {
|
|
67
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=10 release=10
|
|
68
|
+
RISK_BYPASS: incident"
|
|
69
|
+
[ -f "$RDIR/incident-release" ]
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
@test "pipeline: writes nothing when output has no RISK_SCORES line" {
|
|
73
|
+
run_hook "wr-risk-scorer:pipeline" "No score line in this output"
|
|
74
|
+
[ ! -f "$RDIR/commit" ]
|
|
75
|
+
[ ! -f "$RDIR/push" ]
|
|
76
|
+
[ ! -f "$RDIR/release" ]
|
|
19
77
|
}
|
|
20
78
|
|
|
21
|
-
|
|
22
|
-
|
|
79
|
+
# --- Plan scorer: writes plan-reviewed marker on PASS only ---
|
|
80
|
+
|
|
81
|
+
@test "plan: writes plan-reviewed marker on RISK_VERDICT: PASS" {
|
|
82
|
+
run_hook "wr-risk-scorer:plan" "RISK_VERDICT: PASS"
|
|
83
|
+
[ -f "$RDIR/plan-reviewed" ]
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
@test "plan: does NOT write plan-reviewed marker on RISK_VERDICT: FAIL" {
|
|
87
|
+
run_hook "wr-risk-scorer:plan" "RISK_VERDICT: FAIL"
|
|
88
|
+
[ ! -f "$RDIR/plan-reviewed" ]
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
# --- Subagent routing: case guard ignores non-risk-scorer agents ---
|
|
92
|
+
|
|
93
|
+
@test "case guard: skips unrelated agent without writing files" {
|
|
94
|
+
run_hook "wr-architect:agent" "RISK_SCORES: commit=99 push=99 release=99"
|
|
95
|
+
[ ! -f "$RDIR/commit" ]
|
|
23
96
|
}
|
|
24
97
|
|
|
25
|
-
@test "case guard matches wr-risk-scorer:pipeline" {
|
|
98
|
+
@test "case guard: matches wr-risk-scorer:pipeline subagent" {
|
|
26
99
|
SUBAGENT="wr-risk-scorer:pipeline"
|
|
27
100
|
case "$SUBAGENT" in
|
|
28
101
|
*risk-scorer*) true ;;
|
|
@@ -30,7 +103,7 @@ setup() {
|
|
|
30
103
|
esac
|
|
31
104
|
}
|
|
32
105
|
|
|
33
|
-
@test "case guard does NOT match
|
|
106
|
+
@test "case guard: does NOT match wr-architect:agent" {
|
|
34
107
|
SUBAGENT="wr-architect:agent"
|
|
35
108
|
case "$SUBAGENT" in
|
|
36
109
|
*risk-scorer*) false ;;
|
package/package.json
CHANGED