@windyroad/risk-scorer 0.3.1 → 0.3.2-preview.77
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/agents/pipeline.md
CHANGED
|
@@ -111,13 +111,17 @@ Do NOT emit: "Suggested Actions", "Your call:", advisory warnings, back-pressure
|
|
|
111
111
|
|
|
112
112
|
When ANY cumulative score exceeds appetite (> 4), emit a structured `RISK_REMEDIATIONS:` block after the `RISK_SCORES:` line. This gives the calling skill machine-readable input for structured decision prompts.
|
|
113
113
|
|
|
114
|
-
Format:
|
|
114
|
+
Format (5 columns — machine-readable for structured AskUserQuestion prompts in calling skills):
|
|
115
115
|
```
|
|
116
116
|
RISK_REMEDIATIONS:
|
|
117
|
-
- R1 | <description of remediation> | <files affected>
|
|
118
|
-
- R2 | <description of remediation> | <files affected>
|
|
117
|
+
- R1 | <description of remediation> | <effort S/M/L> | <risk_delta -N> | <files affected>
|
|
118
|
+
- R2 | <description of remediation> | <effort S/M/L> | <risk_delta -N> | <files affected>
|
|
119
119
|
```
|
|
120
120
|
|
|
121
|
+
Column definitions:
|
|
122
|
+
- **effort**: estimated size of the remediation — S (< 1h, single file), M (1-4h, few files), L (> 4h, multiple files)
|
|
123
|
+
- **risk_delta**: estimated reduction in residual risk if this remediation is applied (e.g., `-3` means risk drops by 3 points)
|
|
124
|
+
|
|
121
125
|
Include downstream back-pressure in the remediation list:
|
|
122
126
|
- **Commit**: If adding this commit would push the push queue risk >= 5, include a remediation to split the commit.
|
|
123
127
|
- **Push**: If pushing would push the release queue risk >= 5, include a remediation to release first.
|
package/agents/plan.md
CHANGED
|
@@ -49,12 +49,16 @@ You are the Risk Scorer in plan review mode. Assess both the plan's own risk AND
|
|
|
49
49
|
|
|
50
50
|
End your report with `RISK_VERDICT: PASS` or `RISK_VERDICT: FAIL` on its own line. A PostToolUse hook reads this and writes the marker files — do NOT write files yourself.
|
|
51
51
|
|
|
52
|
-
On FAIL, emit a structured `RISK_REMEDIATIONS:` block after the verdict:
|
|
52
|
+
On FAIL, emit a structured `RISK_REMEDIATIONS:` block after the verdict (5 columns — machine-readable for structured AskUserQuestion prompts in calling skills):
|
|
53
53
|
```
|
|
54
54
|
RISK_REMEDIATIONS:
|
|
55
|
-
- R1 | <description of what the plan must add/change> | <affected area>
|
|
55
|
+
- R1 | <description of what the plan must add/change> | <effort S/M/L> | <risk_delta -N> | <affected area>
|
|
56
56
|
```
|
|
57
57
|
|
|
58
|
+
Column definitions:
|
|
59
|
+
- **effort**: estimated size of the remediation — S (< 1h, single file), M (1-4h, few files), L (> 4h, multiple files)
|
|
60
|
+
- **risk_delta**: estimated reduction in residual risk if this remediation is applied
|
|
61
|
+
|
|
58
62
|
Do NOT emit free-text "consider" or "you should" prose. The structured block is the only output for above-appetite guidance.
|
|
59
63
|
|
|
60
64
|
## Control Discovery
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
#!/usr/bin/env bats
|
|
2
|
+
# Doc-lint guard: risk-scorer agent prompts must define a structured
|
|
3
|
+
# machine-readable RISK_REMEDIATIONS block with the full 5-column format.
|
|
4
|
+
#
|
|
5
|
+
# Structural assertions — Permitted Exception to the source-grep ban (ADR-005 / P011).
|
|
6
|
+
# These tests assert that agent specification documents conform to the
|
|
7
|
+
# structured-interaction contract (ADR-013) and the machine-readable format
|
|
8
|
+
# required by P021.
|
|
9
|
+
#
|
|
10
|
+
# Background: P021 identified that above-appetite risk-scorer output used
|
|
11
|
+
# free-text "Your call:" prose. The fix defined a structured RISK_REMEDIATIONS:
|
|
12
|
+
# block. This test guards that all three scoring modes (pipeline, wip, plan)
|
|
13
|
+
# define the block AND include the full 5-column format so calling skills
|
|
14
|
+
# can render structured option prompts with effort and risk-delta context.
|
|
15
|
+
#
|
|
16
|
+
# Cross-reference:
|
|
17
|
+
# ADR-013: docs/decisions/013-structured-user-interaction-for-governance-decisions.proposed.md
|
|
18
|
+
# P021: docs/problems/021-governance-skill-structured-prompts.known-error.md
|
|
19
|
+
# @jtbd JTBD-001 (enforce governance without slowing down)
|
|
20
|
+
# @jtbd JTBD-002 (ship with confidence — structured remediations are auditable)
|
|
21
|
+
|
|
22
|
+
setup() {
|
|
23
|
+
AGENTS_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)"
|
|
24
|
+
PIPELINE="${AGENTS_DIR}/pipeline.md"
|
|
25
|
+
WIP="${AGENTS_DIR}/wip.md"
|
|
26
|
+
PLAN="${AGENTS_DIR}/plan.md"
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
30
|
+
# pipeline.md: above-appetite structured output
|
|
31
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
32
|
+
|
|
33
|
+
@test "pipeline.md defines RISK_REMEDIATIONS block" {
|
|
34
|
+
# Must emit a structured block, not free-text, above appetite (ADR-013 Rule 1).
|
|
35
|
+
run grep -q "RISK_REMEDIATIONS:" "$PIPELINE"
|
|
36
|
+
[ "$status" -eq 0 ]
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
@test "pipeline.md RISK_REMEDIATIONS format includes effort column" {
|
|
40
|
+
# 5-column format: id | description | effort (S/M/L) | risk_delta (-N) | files_touched
|
|
41
|
+
# This column allows calling skills to size each remediation and present
|
|
42
|
+
# a structured AskUserQuestion with effort context.
|
|
43
|
+
run grep -q "effort" "$PIPELINE"
|
|
44
|
+
[ "$status" -eq 0 ]
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
@test "pipeline.md RISK_REMEDIATIONS format includes risk_delta column" {
|
|
48
|
+
# risk_delta lets calling skills show how much each remediation reduces score.
|
|
49
|
+
run grep -q "risk_delta" "$PIPELINE"
|
|
50
|
+
[ "$status" -eq 0 ]
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
@test "pipeline.md defines Below-Appetite Output Rule" {
|
|
54
|
+
# Below appetite: silent pass, no advisory prose (ADR-013 Rule 5).
|
|
55
|
+
run grep -q "Below-Appetite" "$PIPELINE"
|
|
56
|
+
[ "$status" -eq 0 ]
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
60
|
+
# wip.md: above-appetite structured output
|
|
61
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
62
|
+
|
|
63
|
+
@test "wip.md defines RISK_REMEDIATIONS block" {
|
|
64
|
+
run grep -q "RISK_REMEDIATIONS:" "$WIP"
|
|
65
|
+
[ "$status" -eq 0 ]
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
@test "wip.md RISK_REMEDIATIONS format includes effort column" {
|
|
69
|
+
run grep -q "effort" "$WIP"
|
|
70
|
+
[ "$status" -eq 0 ]
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
@test "wip.md RISK_REMEDIATIONS format includes risk_delta column" {
|
|
74
|
+
run grep -q "risk_delta" "$WIP"
|
|
75
|
+
[ "$status" -eq 0 ]
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
@test "wip.md defines Below-Appetite Rule" {
|
|
79
|
+
run grep -q "Below-Appetite" "$WIP"
|
|
80
|
+
[ "$status" -eq 0 ]
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
84
|
+
# plan.md: FAIL-verdict structured output
|
|
85
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
86
|
+
|
|
87
|
+
@test "plan.md defines RISK_REMEDIATIONS block" {
|
|
88
|
+
run grep -q "RISK_REMEDIATIONS:" "$PLAN"
|
|
89
|
+
[ "$status" -eq 0 ]
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
@test "plan.md RISK_REMEDIATIONS format includes effort column" {
|
|
93
|
+
run grep -q "effort" "$PLAN"
|
|
94
|
+
[ "$status" -eq 0 ]
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
@test "plan.md RISK_REMEDIATIONS format includes risk_delta column" {
|
|
98
|
+
run grep -q "risk_delta" "$PLAN"
|
|
99
|
+
[ "$status" -eq 0 ]
|
|
100
|
+
}
|
package/agents/wip.md
CHANGED
|
@@ -53,14 +53,19 @@ If cumulative risk is **within appetite** (< 5): provide the assessment table an
|
|
|
53
53
|
|
|
54
54
|
If cumulative risk **exceeds appetite** (>= 5): provide the assessment table, then emit a structured `RISK_REMEDIATIONS:` block with specific risk-reducing actions:
|
|
55
55
|
|
|
56
|
+
Format (5 columns — machine-readable for structured AskUserQuestion prompts in calling skills):
|
|
56
57
|
```
|
|
57
58
|
RISK_REMEDIATIONS:
|
|
58
|
-
- R1 | Commit current changes to move WIP forward | <uncommitted files>
|
|
59
|
-
- R2 | Write tests for <risk item from report> | <test file to create/extend>
|
|
60
|
-
- R3 | Address release report risk <X> before adding more changes | <affected files>
|
|
61
|
-
- R4 | Push commits to get CI feedback | N/A
|
|
59
|
+
- R1 | Commit current changes to move WIP forward | S | -2 | <uncommitted files>
|
|
60
|
+
- R2 | Write tests for <risk item from report> | M | -3 | <test file to create/extend>
|
|
61
|
+
- R3 | Address release report risk <X> before adding more changes | M | -4 | <affected files>
|
|
62
|
+
- R4 | Push commits to get CI feedback | S | -1 | N/A
|
|
62
63
|
```
|
|
63
64
|
|
|
65
|
+
Column definitions:
|
|
66
|
+
- **effort**: estimated size of the remediation — S (< 1h, single file), M (1-4h, few files), L (> 4h, multiple files)
|
|
67
|
+
- **risk_delta**: estimated reduction in residual risk if this remediation is applied (e.g., `-3` means risk drops by 3 points)
|
|
68
|
+
|
|
64
69
|
Do NOT emit free-text suggestions as prose. The structured block is the only output for above-appetite guidance.
|
|
65
70
|
|
|
66
71
|
The verdict is `RISK_VERDICT: PAUSE`. This blocks the next edit until the risk is addressed.
|