@windyroad/risk-scorer 0.4.2 → 0.5.0-preview.270
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/agents/pipeline.md +27 -6
- package/bin/wr-risk-scorer-drain-register-queue +2 -0
- package/hooks/risk-score-mark.sh +83 -1
- package/hooks/test/risk-score-mark-register-queue.bats +253 -0
- package/package.json +3 -2
- package/scripts/drain-register-queue.sh +287 -0
- package/scripts/test/drain-register-queue.bats +282 -0
package/agents/pipeline.md
CHANGED
|
@@ -179,17 +179,19 @@ When a pipeline run identifies a **register-worthy risk shape**, emit a structur
|
|
|
179
179
|
2. **Confidentiality disclosure** — the Confidential Information Disclosure check (below) flagged business metrics (revenue, user counts, pricing, client names, traffic volumes) in the diff. Confidentiality leaks are standing-risk-shaped even after the immediate remediation.
|
|
180
180
|
3. **User-stated precondition** — the User-Stated Preconditions Check (below) flagged an unmet paired capability as a standalone Risk item. Unmet preconditions are standing-risk-shaped because the dependency gap persists until the paired capability ships.
|
|
181
181
|
|
|
182
|
-
### Format (bulleted-list shape, multi-hint capable)
|
|
182
|
+
### Format (3-column bulleted-list shape, multi-hint capable) — ADR-056
|
|
183
183
|
|
|
184
|
-
A single pipeline run MAY surface more than one register-worthy shape (e.g. both an above-appetite residual AND a confidentiality leak). Emit one bullet per triggered condition:
|
|
184
|
+
A single pipeline run MAY surface more than one register-worthy shape (e.g. both an above-appetite residual AND a confidentiality leak). Emit one bullet per triggered condition. The PREFERRED format is 3-column with an explicit risk-slug:
|
|
185
185
|
|
|
186
186
|
```
|
|
187
187
|
RISK_REGISTER_HINT:
|
|
188
|
-
- above-appetite-residual | <one-line prefill describing the risk>
|
|
189
|
-
- confidentiality-disclosure | <one-line prefill citing what was flagged>
|
|
190
|
-
- user-stated-precondition | <one-line prefill citing the unmet precondition>
|
|
188
|
+
- above-appetite-residual | <risk-slug> | <one-line prefill describing the risk>
|
|
189
|
+
- confidentiality-disclosure | <risk-slug> | <one-line prefill citing what was flagged>
|
|
190
|
+
- user-stated-precondition | <risk-slug> | <one-line prefill citing the unmet precondition>
|
|
191
191
|
```
|
|
192
192
|
|
|
193
|
+
The hook accepts BOTH the 3-column shape (preferred) and the legacy 2-column shape (`<reason-tag> | <prose>`) for backward compatibility per ADR-056's dual-parse contract. When emitting the legacy shape, the hook derives the slug from the reason-tag plus the prose prefix. Always prefer the 3-column shape so the slug is agent-computed and stable across runs.
|
|
194
|
+
|
|
193
195
|
### Reason-tag vocabulary (enumerated — reserved)
|
|
194
196
|
|
|
195
197
|
The first column is one of exactly three reserved tags. Do NOT invent new tags; open new tickets to extend the vocabulary.
|
|
@@ -200,7 +202,26 @@ The first column is one of exactly three reserved tags. Do NOT invent new tags;
|
|
|
200
202
|
| `confidentiality-disclosure` | Business metric or client detail flagged in diff | Confidential Information Disclosure |
|
|
201
203
|
| `user-stated-precondition` | Paired capability unmet; standalone Risk item | User-Stated Preconditions Check |
|
|
202
204
|
|
|
203
|
-
|
|
205
|
+
### Risk-slug column (NEW — ADR-056)
|
|
206
|
+
|
|
207
|
+
The second column is a filename-safe kebab-case identifier the agent computes from the risk's canonical shape. The slug is the dedupe key — N reports producing the same slug collapse to ONE register entry (per the user direction *"for each risk in .risk-reports there should be something in the risk register"*).
|
|
208
|
+
|
|
209
|
+
Slug computation rules:
|
|
210
|
+
|
|
211
|
+
1. Lowercase, hyphen-separated.
|
|
212
|
+
2. Drop articles (the, a, an), prepositions in long phrases, and trailing date markers.
|
|
213
|
+
3. Stable across pipeline runs: identical risk shape → identical slug. Do NOT include timestamps, session IDs, or commit SHAs in the slug.
|
|
214
|
+
4. Maximum 60 characters; truncate at word boundary if longer.
|
|
215
|
+
5. If slug computation is genuinely ambiguous (rare), fall back to `<reason-tag>-<noun-phrase>` form.
|
|
216
|
+
|
|
217
|
+
Examples:
|
|
218
|
+
- `cumulative-residual-commit-layer-above-appetite` (above-appetite-residual)
|
|
219
|
+
- `revenue-figures-leaked-in-changeset` (confidentiality-disclosure)
|
|
220
|
+
- `cross-plugin-version-mismatch-precondition-unmet` (user-stated-precondition)
|
|
221
|
+
|
|
222
|
+
### Prefill column
|
|
223
|
+
|
|
224
|
+
The third column is free-form prose — a one-line prefill carried into the eventual register entry's Description field. Keep it concise (≤ 1 line).
|
|
204
225
|
|
|
205
226
|
### Consumption semantics (post-loop)
|
|
206
227
|
|
package/hooks/risk-score-mark.sh
CHANGED
|
@@ -77,7 +77,89 @@ if echo "$SUBAGENT" | grep -qE 'risk-scorer.pipeline'; then
|
|
|
77
77
|
REPORT_DIR=".risk-reports"
|
|
78
78
|
mkdir -p "$REPORT_DIR"
|
|
79
79
|
TIMESTAMP=$(date -u +%Y-%m-%dT%H-%M-%S)
|
|
80
|
-
|
|
80
|
+
REPORT_PATH="${REPORT_DIR}/${TIMESTAMP}-commit.md"
|
|
81
|
+
echo "$AGENT_OUTPUT" > "$REPORT_PATH"
|
|
82
|
+
|
|
83
|
+
# ---------------------------------------------------------------------------
|
|
84
|
+
# Risk register queue (ADR-056 Phase 2a)
|
|
85
|
+
# Parse RISK_REGISTER_HINT: bullets and append one JSONL line each to
|
|
86
|
+
# .afk-run-state/risk-register-queue.jsonl. Consumer skills (work-problems,
|
|
87
|
+
# manage-problem, install-updates, assess-release) drain the queue in
|
|
88
|
+
# subsequent iters per ADR-014 commit-grain discipline.
|
|
89
|
+
#
|
|
90
|
+
# Dual-parse contract: accept BOTH 3-col (preferred, agent-emitted slug) and
|
|
91
|
+
# 2-col legacy shapes for backward compatibility while in-flight prompt
|
|
92
|
+
# caches transition.
|
|
93
|
+
#
|
|
94
|
+
# Best-effort: errors are swallowed (queue persistence is recoverable via
|
|
95
|
+
# Phase 3 backfill from .risk-reports/). ADR-045 Pattern 2: silent on stdout.
|
|
96
|
+
# ---------------------------------------------------------------------------
|
|
97
|
+
{
|
|
98
|
+
QUEUE_DIR=".afk-run-state"
|
|
99
|
+
QUEUE_FILE="${QUEUE_DIR}/risk-register-queue.jsonl"
|
|
100
|
+
HINT_BLOCK=$(echo "$AGENT_OUTPUT" | awk '
|
|
101
|
+
/^RISK_REGISTER_HINT:[[:space:]]*$/ { in_block=1; next }
|
|
102
|
+
in_block && /^[[:space:]]*$/ { in_block=0; next }
|
|
103
|
+
in_block && /^[A-Z_]+:/ { in_block=0; next }
|
|
104
|
+
in_block && /^- / { print }
|
|
105
|
+
')
|
|
106
|
+
if [ -n "$HINT_BLOCK" ]; then
|
|
107
|
+
mkdir -p "$QUEUE_DIR"
|
|
108
|
+
QUEUE_TS=$(date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
109
|
+
while IFS= read -r BULLET; do
|
|
110
|
+
[ -n "$BULLET" ] || continue
|
|
111
|
+
# Strip leading "- " marker
|
|
112
|
+
PAYLOAD="${BULLET#- }"
|
|
113
|
+
PAYLOAD="${PAYLOAD#-}"
|
|
114
|
+
PAYLOAD="${PAYLOAD# }"
|
|
115
|
+
# Count pipe-separated columns (handle 2-col vs 3-col)
|
|
116
|
+
N_PIPES=$(echo "$PAYLOAD" | awk -F'|' '{print NF-1}')
|
|
117
|
+
case "$N_PIPES" in
|
|
118
|
+
1)
|
|
119
|
+
# 2-col legacy: <reason-tag> | <prose>
|
|
120
|
+
REASON=$(echo "$PAYLOAD" | awk -F'|' '{gsub(/^[ \t]+|[ \t]+$/, "", $1); print $1}')
|
|
121
|
+
SLUG_FROM=$(echo "$PAYLOAD" | awk -F'|' '{gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}')
|
|
122
|
+
PREFILL="$SLUG_FROM"
|
|
123
|
+
SLUG_SOURCE="derived"
|
|
124
|
+
# Derive slug: reason-tag + first 5 word-stems of prose, kebab, ≤60 chars
|
|
125
|
+
SLUG_BODY=$(echo "$SLUG_FROM" | tr '[:upper:]' '[:lower:]' \
|
|
126
|
+
| sed -E 's/[^a-z0-9 ]+/ /g; s/\b(the|a|an|is|of|to|in|for|on|at|by|and|or)\b//g; s/[[:space:]]+/ /g; s/^ //; s/ $//' \
|
|
127
|
+
| awk '{out=""; for(i=1;i<=NF && i<=5;i++){out = out (i==1?"":"-") $i} print out}')
|
|
128
|
+
SLUG="${REASON}-${SLUG_BODY}"
|
|
129
|
+
SLUG=$(echo "$SLUG" | cut -c1-60)
|
|
130
|
+
;;
|
|
131
|
+
2|*)
|
|
132
|
+
# 3-col preferred: <reason-tag> | <slug> | <prose>
|
|
133
|
+
REASON=$(echo "$PAYLOAD" | awk -F'|' '{gsub(/^[ \t]+|[ \t]+$/, "", $1); print $1}')
|
|
134
|
+
SLUG=$(echo "$PAYLOAD" | awk -F'|' '{gsub(/^[ \t]+|[ \t]+$/, "", $2); print $2}')
|
|
135
|
+
PREFILL=$(echo "$PAYLOAD" | awk -F'|' '{ for(i=3;i<=NF;i++){printf "%s%s", (i==3?"":"|"), $i} print "" }' \
|
|
136
|
+
| sed -E 's/^[ \t]+//; s/[ \t]+$//')
|
|
137
|
+
SLUG_SOURCE="agent"
|
|
138
|
+
;;
|
|
139
|
+
esac
|
|
140
|
+
# Validate reason-tag is one of three reserved values; skip otherwise
|
|
141
|
+
case "$REASON" in
|
|
142
|
+
above-appetite-residual|confidentiality-disclosure|user-stated-precondition) ;;
|
|
143
|
+
*) continue ;;
|
|
144
|
+
esac
|
|
145
|
+
# Skip if slug or prefill is empty (malformed bullet)
|
|
146
|
+
[ -n "$SLUG" ] && [ -n "$PREFILL" ] || continue
|
|
147
|
+
# Append JSONL line via python3 to ensure proper escaping
|
|
148
|
+
python3 -c "
|
|
149
|
+
import json, sys
|
|
150
|
+
print(json.dumps({
|
|
151
|
+
'ts': '$QUEUE_TS',
|
|
152
|
+
'session_id': '$SESSION_ID',
|
|
153
|
+
'report_path': '$REPORT_PATH',
|
|
154
|
+
'reason_tag': '$REASON',
|
|
155
|
+
'risk_slug': '$SLUG',
|
|
156
|
+
'slug_source': '$SLUG_SOURCE',
|
|
157
|
+
'prefill': sys.argv[1],
|
|
158
|
+
}))
|
|
159
|
+
" "$PREFILL" >> "$QUEUE_FILE" 2>/dev/null || true
|
|
160
|
+
done <<< "$HINT_BLOCK"
|
|
161
|
+
fi
|
|
162
|
+
} 2>/dev/null || true
|
|
81
163
|
fi
|
|
82
164
|
|
|
83
165
|
# ---------------------------------------------------------------------------
|
|
@@ -0,0 +1,253 @@
|
|
|
1
|
+
#!/usr/bin/env bats
|
|
2
|
+
|
|
3
|
+
# Tests for the RISK_REGISTER_HINT queue-write extension to risk-score-mark.sh
|
|
4
|
+
# (ADR-056 Phase 2a). Verifies the PostToolUse:Agent hook parses the
|
|
5
|
+
# RISK_REGISTER_HINT block from pipeline-agent output and appends one JSONL
|
|
6
|
+
# line per valid bullet to .afk-run-state/risk-register-queue.jsonl.
|
|
7
|
+
#
|
|
8
|
+
# Behavioural fixtures per ADR-052: each test pipes a mock agent output to
|
|
9
|
+
# the hook and asserts on side-effects (queue file content / shape / silence).
|
|
10
|
+
# No structural grep against source.
|
|
11
|
+
#
|
|
12
|
+
# Cross-references:
|
|
13
|
+
# ADR-056: docs/decisions/056-risk-register-back-channel-write-contract.proposed.md
|
|
14
|
+
# ADR-045: hook injection budget Pattern 2 (silent on stdout)
|
|
15
|
+
# P033: docs/problems/033-no-persistent-risk-register.known-error.md (driver)
|
|
16
|
+
# P110: pipeline back-channel hint (consumer of this contract)
|
|
17
|
+
|
|
18
|
+
setup() {
|
|
19
|
+
SCRIPT_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)"
|
|
20
|
+
HOOK="$SCRIPT_DIR/risk-score-mark.sh"
|
|
21
|
+
ORIG_DIR="$PWD"
|
|
22
|
+
TEST_DIR=$(mktemp -d)
|
|
23
|
+
cd "$TEST_DIR"
|
|
24
|
+
TMPDIR="$TEST_DIR/tmp"
|
|
25
|
+
export TMPDIR
|
|
26
|
+
mkdir -p "$TMPDIR"
|
|
27
|
+
SESSION_ID="test-session-$$"
|
|
28
|
+
RDIR="$TMPDIR/claude-risk-${SESSION_ID}"
|
|
29
|
+
QUEUE_FILE="$TEST_DIR/.afk-run-state/risk-register-queue.jsonl"
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
teardown() {
|
|
33
|
+
cd "$ORIG_DIR"
|
|
34
|
+
rm -rf "$TEST_DIR"
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
# Build mock PostToolUse:Agent JSON envelope and pipe to the hook.
|
|
38
|
+
run_hook() {
|
|
39
|
+
local subagent="$1"
|
|
40
|
+
local agent_output="$2"
|
|
41
|
+
python3 -c "
|
|
42
|
+
import json, sys
|
|
43
|
+
print(json.dumps({
|
|
44
|
+
'tool_name': 'Agent',
|
|
45
|
+
'session_id': '${SESSION_ID}',
|
|
46
|
+
'tool_input': {'subagent_type': '${subagent}'},
|
|
47
|
+
'tool_response': {'content': [{'type': 'text', 'text': sys.stdin.read()}]}
|
|
48
|
+
}))" <<<"$agent_output" | bash "$HOOK"
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
# Capture hook stdout (separate from filesystem side-effects).
|
|
52
|
+
run_hook_capture_stdout() {
|
|
53
|
+
local subagent="$1"
|
|
54
|
+
local agent_output="$2"
|
|
55
|
+
python3 -c "
|
|
56
|
+
import json, sys
|
|
57
|
+
print(json.dumps({
|
|
58
|
+
'tool_name': 'Agent',
|
|
59
|
+
'session_id': '${SESSION_ID}',
|
|
60
|
+
'tool_input': {'subagent_type': '${subagent}'},
|
|
61
|
+
'tool_response': {'content': [{'type': 'text', 'text': sys.stdin.read()}]}
|
|
62
|
+
}))" <<<"$agent_output" | bash "$HOOK"
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
# ---------------------------------------------------------------------------
|
|
66
|
+
# 3-column (preferred) parse path
|
|
67
|
+
# ---------------------------------------------------------------------------
|
|
68
|
+
|
|
69
|
+
@test "3-col hint with one above-appetite bullet → one JSONL line, slug_source=agent" {
|
|
70
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=12 push=8 release=4
|
|
71
|
+
|
|
72
|
+
RISK_REGISTER_HINT:
|
|
73
|
+
- above-appetite-residual | cumulative-residual-commit-layer-above-appetite | Cumulative residual reached 12/25 due to mass-edit across 17 files."
|
|
74
|
+
[ -f "$QUEUE_FILE" ]
|
|
75
|
+
LINE_COUNT=$(wc -l < "$QUEUE_FILE")
|
|
76
|
+
[ "$LINE_COUNT" -eq 1 ]
|
|
77
|
+
REASON=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['reason_tag'])" < "$QUEUE_FILE")
|
|
78
|
+
SLUG=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['risk_slug'])" < "$QUEUE_FILE")
|
|
79
|
+
SOURCE=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['slug_source'])" < "$QUEUE_FILE")
|
|
80
|
+
PREFILL=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['prefill'])" < "$QUEUE_FILE")
|
|
81
|
+
[ "$REASON" = "above-appetite-residual" ]
|
|
82
|
+
[ "$SLUG" = "cumulative-residual-commit-layer-above-appetite" ]
|
|
83
|
+
[ "$SOURCE" = "agent" ]
|
|
84
|
+
[[ "$PREFILL" == *"mass-edit across 17 files"* ]]
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
@test "3-col hint with three bullets → three JSONL lines in order, all slug_source=agent" {
|
|
88
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=8 release=5
|
|
89
|
+
|
|
90
|
+
RISK_REGISTER_HINT:
|
|
91
|
+
- above-appetite-residual | cumulative-residual-above-appetite | Above-appetite residual.
|
|
92
|
+
- confidentiality-disclosure | revenue-figures-leaked | Revenue figures in changeset.
|
|
93
|
+
- user-stated-precondition | paired-capability-unmet | Pair B not yet shipped."
|
|
94
|
+
LINE_COUNT=$(wc -l < "$QUEUE_FILE")
|
|
95
|
+
[ "$LINE_COUNT" -eq 3 ]
|
|
96
|
+
TAGS=$(python3 -c "
|
|
97
|
+
import json
|
|
98
|
+
with open('$QUEUE_FILE') as f:
|
|
99
|
+
for line in f:
|
|
100
|
+
print(json.loads(line)['reason_tag'])
|
|
101
|
+
")
|
|
102
|
+
EXPECTED="above-appetite-residual
|
|
103
|
+
confidentiality-disclosure
|
|
104
|
+
user-stated-precondition"
|
|
105
|
+
[ "$TAGS" = "$EXPECTED" ]
|
|
106
|
+
ALL_AGENT=$(python3 -c "
|
|
107
|
+
import json
|
|
108
|
+
with open('$QUEUE_FILE') as f:
|
|
109
|
+
src = [json.loads(line)['slug_source'] for line in f]
|
|
110
|
+
print(all(s == 'agent' for s in src))
|
|
111
|
+
")
|
|
112
|
+
[ "$ALL_AGENT" = "True" ]
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
@test "3-col hint includes report_path matching the just-written .risk-reports file" {
|
|
116
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=5 release=2
|
|
117
|
+
|
|
118
|
+
RISK_REGISTER_HINT:
|
|
119
|
+
- above-appetite-residual | example-slug | Example."
|
|
120
|
+
REPORT_PATH=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['report_path'])" < "$QUEUE_FILE")
|
|
121
|
+
[[ "$REPORT_PATH" == .risk-reports/*-commit.md ]]
|
|
122
|
+
[ -f "$REPORT_PATH" ]
|
|
123
|
+
}
|
|
124
|
+
|
|
125
|
+
# ---------------------------------------------------------------------------
|
|
126
|
+
# 2-column legacy parse path (backward compatibility)
|
|
127
|
+
# ---------------------------------------------------------------------------
|
|
128
|
+
|
|
129
|
+
@test "2-col legacy hint → JSONL line with derived slug, slug_source=derived" {
|
|
130
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=12 push=8 release=4
|
|
131
|
+
|
|
132
|
+
RISK_REGISTER_HINT:
|
|
133
|
+
- above-appetite-residual | Cumulative residual risk for commit layer."
|
|
134
|
+
[ -f "$QUEUE_FILE" ]
|
|
135
|
+
LINE_COUNT=$(wc -l < "$QUEUE_FILE")
|
|
136
|
+
[ "$LINE_COUNT" -eq 1 ]
|
|
137
|
+
SOURCE=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['slug_source'])" < "$QUEUE_FILE")
|
|
138
|
+
SLUG=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['risk_slug'])" < "$QUEUE_FILE")
|
|
139
|
+
[ "$SOURCE" = "derived" ]
|
|
140
|
+
# Derived slug starts with reason-tag prefix
|
|
141
|
+
[[ "$SLUG" == above-appetite-residual-* ]]
|
|
142
|
+
# Derived slug is filename-safe (lowercase, kebab, no spaces)
|
|
143
|
+
[[ "$SLUG" =~ ^[a-z0-9-]+$ ]]
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
@test "2-col legacy hint: same prefill produces same derived slug across runs" {
|
|
147
|
+
PREFILL_TEXT="Cumulative residual risk for commit layer."
|
|
148
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=12 push=8 release=4
|
|
149
|
+
|
|
150
|
+
RISK_REGISTER_HINT:
|
|
151
|
+
- above-appetite-residual | $PREFILL_TEXT"
|
|
152
|
+
SLUG_1=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['risk_slug'])" < "$QUEUE_FILE")
|
|
153
|
+
rm -f "$QUEUE_FILE"
|
|
154
|
+
sleep 1 # ensure different timestamp on second .risk-reports write
|
|
155
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=12 push=8 release=4
|
|
156
|
+
|
|
157
|
+
RISK_REGISTER_HINT:
|
|
158
|
+
- above-appetite-residual | $PREFILL_TEXT"
|
|
159
|
+
SLUG_2=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['risk_slug'])" < "$QUEUE_FILE")
|
|
160
|
+
[ "$SLUG_1" = "$SLUG_2" ]
|
|
161
|
+
}
|
|
162
|
+
|
|
163
|
+
@test "Mixed 3-col and 2-col bullets in same block → both shapes appended with correct slug_source" {
|
|
164
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=5 release=2
|
|
165
|
+
|
|
166
|
+
RISK_REGISTER_HINT:
|
|
167
|
+
- above-appetite-residual | preferred-slug | First risk.
|
|
168
|
+
- confidentiality-disclosure | Second risk in legacy shape."
|
|
169
|
+
LINE_COUNT=$(wc -l < "$QUEUE_FILE")
|
|
170
|
+
[ "$LINE_COUNT" -eq 2 ]
|
|
171
|
+
SOURCES=$(python3 -c "
|
|
172
|
+
import json
|
|
173
|
+
with open('$QUEUE_FILE') as f:
|
|
174
|
+
for line in f:
|
|
175
|
+
print(json.loads(line)['slug_source'])
|
|
176
|
+
")
|
|
177
|
+
EXPECTED="agent
|
|
178
|
+
derived"
|
|
179
|
+
[ "$SOURCES" = "$EXPECTED" ]
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
# ---------------------------------------------------------------------------
|
|
183
|
+
# Silence + no-op paths
|
|
184
|
+
# ---------------------------------------------------------------------------
|
|
185
|
+
|
|
186
|
+
@test "no hint emitted (silent-pass) → queue file is not created" {
|
|
187
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=2 push=2 release=1
|
|
188
|
+
RISK_BYPASS: reducing"
|
|
189
|
+
[ ! -f "$QUEUE_FILE" ]
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
@test "empty agent output → no queue file, no crash" {
|
|
193
|
+
run_hook "wr-risk-scorer:pipeline" ""
|
|
194
|
+
[ ! -f "$QUEUE_FILE" ]
|
|
195
|
+
}
|
|
196
|
+
|
|
197
|
+
@test "malformed hint bullet (invalid reason-tag) is skipped; valid bullet appended" {
|
|
198
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=5 release=2
|
|
199
|
+
|
|
200
|
+
RISK_REGISTER_HINT:
|
|
201
|
+
- not-a-real-tag | bogus-slug | Should be skipped.
|
|
202
|
+
- above-appetite-residual | valid-slug | Should be kept."
|
|
203
|
+
LINE_COUNT=$(wc -l < "$QUEUE_FILE")
|
|
204
|
+
[ "$LINE_COUNT" -eq 1 ]
|
|
205
|
+
SLUG=$(python3 -c "import json,sys; print(json.loads(sys.stdin.readline())['risk_slug'])" < "$QUEUE_FILE")
|
|
206
|
+
[ "$SLUG" = "valid-slug" ]
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
# ---------------------------------------------------------------------------
|
|
210
|
+
# Append semantics (queue is append-only; dedupe is drain-step concern)
|
|
211
|
+
# ---------------------------------------------------------------------------
|
|
212
|
+
|
|
213
|
+
@test "two consecutive hook runs with same hint → six JSONL lines (queue is append-only)" {
|
|
214
|
+
HINT="RISK_SCORES: commit=10 push=5 release=2
|
|
215
|
+
|
|
216
|
+
RISK_REGISTER_HINT:
|
|
217
|
+
- above-appetite-residual | slug-a | First.
|
|
218
|
+
- confidentiality-disclosure | slug-b | Second.
|
|
219
|
+
- user-stated-precondition | slug-c | Third."
|
|
220
|
+
run_hook "wr-risk-scorer:pipeline" "$HINT"
|
|
221
|
+
sleep 1
|
|
222
|
+
run_hook "wr-risk-scorer:pipeline" "$HINT"
|
|
223
|
+
LINE_COUNT=$(wc -l < "$QUEUE_FILE")
|
|
224
|
+
[ "$LINE_COUNT" -eq 6 ]
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
# ---------------------------------------------------------------------------
|
|
228
|
+
# Directory creation
|
|
229
|
+
# ---------------------------------------------------------------------------
|
|
230
|
+
|
|
231
|
+
@test ".afk-run-state/ absent → hook creates it; queue file written" {
|
|
232
|
+
[ ! -d ".afk-run-state" ]
|
|
233
|
+
run_hook "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=5 release=2
|
|
234
|
+
|
|
235
|
+
RISK_REGISTER_HINT:
|
|
236
|
+
- above-appetite-residual | example | Example."
|
|
237
|
+
[ -d ".afk-run-state" ]
|
|
238
|
+
[ -f "$QUEUE_FILE" ]
|
|
239
|
+
}
|
|
240
|
+
|
|
241
|
+
# ---------------------------------------------------------------------------
|
|
242
|
+
# ADR-045 Pattern 2: silent on stdout
|
|
243
|
+
# ---------------------------------------------------------------------------
|
|
244
|
+
|
|
245
|
+
@test "hook stdout is empty on queue-write success (ADR-045 Pattern 2)" {
|
|
246
|
+
STDOUT=$(run_hook_capture_stdout "wr-risk-scorer:pipeline" "RISK_SCORES: commit=10 push=5 release=2
|
|
247
|
+
|
|
248
|
+
RISK_REGISTER_HINT:
|
|
249
|
+
- above-appetite-residual | quiet-slug | Quiet please.")
|
|
250
|
+
[ -z "$STDOUT" ]
|
|
251
|
+
# Verify the side-effect did happen
|
|
252
|
+
[ -f "$QUEUE_FILE" ]
|
|
253
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@windyroad/risk-scorer",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.5.0-preview.270",
|
|
4
4
|
"description": "Pipeline risk scoring, commit/push gates, and secret leak detection",
|
|
5
5
|
"bin": {
|
|
6
6
|
"windyroad-risk-scorer": "./bin/install.mjs"
|
|
@@ -24,6 +24,7 @@
|
|
|
24
24
|
"hooks/",
|
|
25
25
|
"skills/",
|
|
26
26
|
".claude-plugin/",
|
|
27
|
-
"lib/"
|
|
27
|
+
"lib/",
|
|
28
|
+
"scripts/"
|
|
28
29
|
]
|
|
29
30
|
}
|
|
@@ -0,0 +1,287 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# packages/risk-scorer/scripts/drain-register-queue.sh
|
|
3
|
+
#
|
|
4
|
+
# Drains .afk-run-state/risk-register-queue.jsonl into docs/risks/
|
|
5
|
+
# register entries per ADR-056 (Phase 2b consumer-skill drain contract).
|
|
6
|
+
#
|
|
7
|
+
# The Phase 2a hook (risk-score-mark.sh) enqueues one JSONL line per
|
|
8
|
+
# RISK_REGISTER_HINT bullet emitted by wr-risk-scorer:pipeline. This script
|
|
9
|
+
# is invoked by consumer skills (this iter: /wr-itil:work-problems Step 6.4)
|
|
10
|
+
# to materialise queued hints into docs/risks/R<NNN>-<slug>.active.md files.
|
|
11
|
+
#
|
|
12
|
+
# Usage:
|
|
13
|
+
# drain-register-queue.sh [<project-root>]
|
|
14
|
+
#
|
|
15
|
+
# Default <project-root> is $(pwd).
|
|
16
|
+
#
|
|
17
|
+
# Behaviour:
|
|
18
|
+
# - Idempotent: empty queue OR missing docs/risks/ → no-op exit 0.
|
|
19
|
+
# - Dedupe by risk_slug — N hints for same slug → 1 register file with
|
|
20
|
+
# N Evidence Log entries (per user direction "for each risk in
|
|
21
|
+
# .risk-reports there should be something in the register").
|
|
22
|
+
# - New risks: minted as R<NNN>-<slug>.active.md with auto-scaffolded
|
|
23
|
+
# status, ADR-026 sentinels for ungrounded scoring fields.
|
|
24
|
+
# - Existing risks (slug match): Evidence Log appended; scoring untouched.
|
|
25
|
+
# - README Register table updated with one row per new risk (ADR-056 §3d).
|
|
26
|
+
# - Files staged via `git add` — caller commits per ADR-014.
|
|
27
|
+
# - Queue truncated only on successful drain. No-op cases preserve queue.
|
|
28
|
+
#
|
|
29
|
+
# Stdout (key=value, caller-parseable):
|
|
30
|
+
# entries_drained=N — total JSONL lines processed
|
|
31
|
+
# new_risks_created=N — new R<NNN> files written
|
|
32
|
+
# evidence_appended=N — slug-matched existing files updated
|
|
33
|
+
# next_action=commit-staged|none — caller's commit decision
|
|
34
|
+
#
|
|
35
|
+
# Exit codes:
|
|
36
|
+
# 0 — success or no-op
|
|
37
|
+
# non-zero — hard failure (template missing, write error, git failure)
|
|
38
|
+
#
|
|
39
|
+
# @adr ADR-056 (queue-and-drain contract; Phase 2b consumer drain)
|
|
40
|
+
# @adr ADR-026 (not estimated — no prior data sentinel for ungrounded scoring)
|
|
41
|
+
# @adr ADR-019 (ticket-creator dual-source ID via local-max + origin-max)
|
|
42
|
+
# @adr ADR-049 (resolved via bin/wr-risk-scorer-drain-register-queue shim)
|
|
43
|
+
# @adr ADR-052 (behavioural-fixture coverage at scripts/test/drain-register-queue.bats)
|
|
44
|
+
# @problem P033 (Phase 2b)
|
|
45
|
+
|
|
46
|
+
set -uo pipefail
|
|
47
|
+
|
|
48
|
+
PROJECT_ROOT="${1:-$(pwd)}"
|
|
49
|
+
QUEUE_FILE="${PROJECT_ROOT}/.afk-run-state/risk-register-queue.jsonl"
|
|
50
|
+
RISKS_DIR="${PROJECT_ROOT}/docs/risks"
|
|
51
|
+
TEMPLATE_FILE="${RISKS_DIR}/TEMPLATE.md"
|
|
52
|
+
README_FILE="${RISKS_DIR}/README.md"
|
|
53
|
+
|
|
54
|
+
emit_no_op() {
|
|
55
|
+
echo "entries_drained=0"
|
|
56
|
+
echo "new_risks_created=0"
|
|
57
|
+
echo "evidence_appended=0"
|
|
58
|
+
echo "next_action=none"
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
if [ ! -f "$QUEUE_FILE" ] || [ ! -s "$QUEUE_FILE" ]; then
|
|
62
|
+
emit_no_op
|
|
63
|
+
exit 0
|
|
64
|
+
fi
|
|
65
|
+
|
|
66
|
+
if [ ! -d "$RISKS_DIR" ] || [ ! -f "$TEMPLATE_FILE" ] || [ ! -f "$README_FILE" ]; then
|
|
67
|
+
emit_no_op
|
|
68
|
+
exit 0
|
|
69
|
+
fi
|
|
70
|
+
|
|
71
|
+
LOCAL_MAX=$(ls "$RISKS_DIR"/R*.md 2>/dev/null \
|
|
72
|
+
| sed 's|.*/||' | grep -oE '^R[0-9]+' | sed 's/^R//' | sort -n | tail -1 || true)
|
|
73
|
+
LOCAL_MAX="${LOCAL_MAX:-0}"
|
|
74
|
+
|
|
75
|
+
ORIGIN_MAX=$(cd "$PROJECT_ROOT" && git ls-tree --name-only origin/main docs/risks/ 2>/dev/null \
|
|
76
|
+
| sed 's|^docs/risks/||' | grep -oE '^R[0-9]+' | sed 's/^R//' | sort -n | tail -1 || true)
|
|
77
|
+
ORIGIN_MAX="${ORIGIN_MAX:-0}"
|
|
78
|
+
|
|
79
|
+
# Force base-10 — bash arithmetic treats leading-zero values as octal,
|
|
80
|
+
# so R099 → 099 → "value too great for base" without the 10# prefix.
|
|
81
|
+
NEXT_ID=$(( (10#$LOCAL_MAX > 10#$ORIGIN_MAX ? 10#$LOCAL_MAX : 10#$ORIGIN_MAX) + 1 ))
|
|
82
|
+
|
|
83
|
+
DRAIN_RESULT=$(python3 - "$QUEUE_FILE" "$RISKS_DIR" "$TEMPLATE_FILE" "$README_FILE" "$NEXT_ID" "$PROJECT_ROOT" <<'PYEOF'
|
|
84
|
+
import json
|
|
85
|
+
import os
|
|
86
|
+
import re
|
|
87
|
+
import sys
|
|
88
|
+
from collections import OrderedDict
|
|
89
|
+
from datetime import datetime
|
|
90
|
+
|
|
91
|
+
queue_file, risks_dir, template_file, readme_file, next_id_str, project_root = sys.argv[1:7]
|
|
92
|
+
next_id = int(next_id_str)
|
|
93
|
+
|
|
94
|
+
hints = []
|
|
95
|
+
with open(queue_file, 'r', encoding='utf-8') as f:
|
|
96
|
+
for line in f:
|
|
97
|
+
line = line.strip()
|
|
98
|
+
if not line:
|
|
99
|
+
continue
|
|
100
|
+
try:
|
|
101
|
+
entry = json.loads(line)
|
|
102
|
+
except json.JSONDecodeError:
|
|
103
|
+
continue
|
|
104
|
+
if not all(k in entry for k in ('risk_slug', 'reason_tag', 'prefill', 'report_path')):
|
|
105
|
+
continue
|
|
106
|
+
hints.append(entry)
|
|
107
|
+
|
|
108
|
+
if not hints:
|
|
109
|
+
print("entries_drained=0")
|
|
110
|
+
print("new_risks_created=0")
|
|
111
|
+
print("evidence_appended=0")
|
|
112
|
+
print("next_action=none")
|
|
113
|
+
sys.exit(0)
|
|
114
|
+
|
|
115
|
+
groups = OrderedDict()
|
|
116
|
+
for h in hints:
|
|
117
|
+
slug = h['risk_slug']
|
|
118
|
+
if slug not in groups:
|
|
119
|
+
groups[slug] = []
|
|
120
|
+
groups[slug].append(h)
|
|
121
|
+
|
|
122
|
+
existing = {}
|
|
123
|
+
for fn in os.listdir(risks_dir):
|
|
124
|
+
if fn in ('README.md', 'TEMPLATE.md'):
|
|
125
|
+
continue
|
|
126
|
+
m = re.match(r'^R(\d+)-(.+)\.active\.md$', fn)
|
|
127
|
+
if m:
|
|
128
|
+
existing[m.group(2)] = fn
|
|
129
|
+
|
|
130
|
+
today = datetime.utcnow().strftime('%Y-%m-%d')
|
|
131
|
+
|
|
132
|
+
new_risks = []
|
|
133
|
+
appended_evidence = []
|
|
134
|
+
|
|
135
|
+
for slug, group in groups.items():
|
|
136
|
+
evidence_lines = [
|
|
137
|
+
f"- {h['ts']}: fired in `{h['report_path']}` (reason: {h['reason_tag']})"
|
|
138
|
+
for h in group
|
|
139
|
+
]
|
|
140
|
+
evidence_block = "\n".join(evidence_lines)
|
|
141
|
+
|
|
142
|
+
if slug in existing:
|
|
143
|
+
fn = existing[slug]
|
|
144
|
+
path = os.path.join(risks_dir, fn)
|
|
145
|
+
with open(path, 'r', encoding='utf-8') as f:
|
|
146
|
+
content = f.read()
|
|
147
|
+
if "## Evidence Log" in content:
|
|
148
|
+
content = re.sub(
|
|
149
|
+
r'(## Evidence Log\n(?:.*?\n)*?)(\n## |\Z)',
|
|
150
|
+
lambda m: m.group(1).rstrip() + "\n" + evidence_block + "\n" + m.group(2),
|
|
151
|
+
content,
|
|
152
|
+
count=1,
|
|
153
|
+
flags=re.DOTALL,
|
|
154
|
+
)
|
|
155
|
+
else:
|
|
156
|
+
new_section = f"\n## Evidence Log\n\nAuto-populated from `.risk-reports/` via Phase 2b drain.\n\n{evidence_block}\n"
|
|
157
|
+
if "## Change Log" in content:
|
|
158
|
+
content = content.replace("## Change Log", new_section + "\n## Change Log", 1)
|
|
159
|
+
else:
|
|
160
|
+
content = content.rstrip() + "\n" + new_section
|
|
161
|
+
with open(path, 'w', encoding='utf-8') as f:
|
|
162
|
+
f.write(content)
|
|
163
|
+
appended_evidence.append((fn, [h['report_path'] for h in group]))
|
|
164
|
+
else:
|
|
165
|
+
rid = f"R{next_id:03d}"
|
|
166
|
+
next_id += 1
|
|
167
|
+
fn = f"{rid}-{slug}.active.md"
|
|
168
|
+
path = os.path.join(risks_dir, fn)
|
|
169
|
+
prefill = next((h['prefill'] for h in group if h.get('prefill')), '(no description provided)')
|
|
170
|
+
sentinel = "not estimated — no prior data"
|
|
171
|
+
title = slug.replace('-', ' ').title()
|
|
172
|
+
body = f"""# Risk {rid}: {title}
|
|
173
|
+
|
|
174
|
+
**Status**: Active (auto-scaffolded — pending review)
|
|
175
|
+
**Category**: <!-- pending review — auto-scaffolded from pipeline hint -->
|
|
176
|
+
**Identified**: {today}
|
|
177
|
+
**Owner**: pending review
|
|
178
|
+
**Last reviewed**: {today}
|
|
179
|
+
**Next review**: {today}
|
|
180
|
+
**Curation**: pending review (auto-scaffolded {today})
|
|
181
|
+
|
|
182
|
+
## Description
|
|
183
|
+
|
|
184
|
+
{prefill}
|
|
185
|
+
|
|
186
|
+
> Auto-scaffolded by the Phase 2b drain (ADR-056) from a `wr-risk-scorer:pipeline`
|
|
187
|
+
> RISK_REGISTER_HINT bullet. The description is the agent's prefill; scoring
|
|
188
|
+
> fields below carry the ADR-026 ungrounded-output sentinel until human curation.
|
|
189
|
+
|
|
190
|
+
## Inherent Risk
|
|
191
|
+
|
|
192
|
+
Impact × Likelihood *before* controls.
|
|
193
|
+
|
|
194
|
+
- **Impact**: {sentinel}
|
|
195
|
+
- **Likelihood**: {sentinel}
|
|
196
|
+
- **Inherent Score**: {sentinel}
|
|
197
|
+
- **Inherent Band**: {sentinel}
|
|
198
|
+
|
|
199
|
+
## Controls
|
|
200
|
+
|
|
201
|
+
- pending review — controls to be enumerated during curation.
|
|
202
|
+
|
|
203
|
+
## Residual Risk
|
|
204
|
+
|
|
205
|
+
Impact × Likelihood *after* controls.
|
|
206
|
+
|
|
207
|
+
- **Impact**: {sentinel}
|
|
208
|
+
- **Likelihood**: {sentinel}
|
|
209
|
+
- **Residual Score**: {sentinel}
|
|
210
|
+
- **Residual Band**: {sentinel}
|
|
211
|
+
- **Within appetite?**: pending — scoring not estimated
|
|
212
|
+
|
|
213
|
+
## Treatment
|
|
214
|
+
|
|
215
|
+
pending review — treatment decision deferred until scoring is curated.
|
|
216
|
+
|
|
217
|
+
## Monitoring
|
|
218
|
+
|
|
219
|
+
- **Trigger to re-assess**: any new pipeline hint with this risk_slug
|
|
220
|
+
- **Metrics**: count of `.risk-reports/` entries citing this slug
|
|
221
|
+
|
|
222
|
+
## Related
|
|
223
|
+
|
|
224
|
+
- Criteria: `RISK-POLICY.md`
|
|
225
|
+
- Realised-as: <!-- link to docs/problems/P<NNN> when known -->
|
|
226
|
+
- Treatment ADRs: <!-- link to docs/decisions/ADR-<NNN> when treatment lands -->
|
|
227
|
+
|
|
228
|
+
## Evidence Log
|
|
229
|
+
|
|
230
|
+
Auto-populated from `.risk-reports/` via Phase 2b drain.
|
|
231
|
+
|
|
232
|
+
{evidence_block}
|
|
233
|
+
|
|
234
|
+
## Change Log
|
|
235
|
+
|
|
236
|
+
- {today}: Auto-scaffolded by Phase 2b drain (ADR-056). Pending human curation.
|
|
237
|
+
"""
|
|
238
|
+
with open(path, 'w', encoding='utf-8') as f:
|
|
239
|
+
f.write(body)
|
|
240
|
+
new_risks.append((rid, slug, fn, prefill))
|
|
241
|
+
|
|
242
|
+
if new_risks:
|
|
243
|
+
with open(readme_file, 'r', encoding='utf-8') as f:
|
|
244
|
+
readme = f.read()
|
|
245
|
+
rows = []
|
|
246
|
+
for rid, slug, fn, prefill in new_risks:
|
|
247
|
+
title = slug.replace('-', ' ').title()
|
|
248
|
+
rows.append(f"| [{rid}]({fn}) | {title} | pending | — | — | pending | pending | {today} |")
|
|
249
|
+
new_rows_block = "\n".join(rows) + "\n"
|
|
250
|
+
if "## Retired" in readme:
|
|
251
|
+
readme = readme.replace("## Retired", new_rows_block + "\n## Retired", 1)
|
|
252
|
+
else:
|
|
253
|
+
readme = readme.rstrip() + "\n" + new_rows_block
|
|
254
|
+
with open(readme_file, 'w', encoding='utf-8') as f:
|
|
255
|
+
f.write(readme)
|
|
256
|
+
|
|
257
|
+
print(f"entries_drained={len(hints)}")
|
|
258
|
+
print(f"new_risks_created={len(new_risks)}")
|
|
259
|
+
print(f"evidence_appended={len(appended_evidence)}")
|
|
260
|
+
if new_risks or appended_evidence:
|
|
261
|
+
print("next_action=commit-staged")
|
|
262
|
+
else:
|
|
263
|
+
print("next_action=none")
|
|
264
|
+
PYEOF
|
|
265
|
+
)
|
|
266
|
+
PY_STATUS=$?
|
|
267
|
+
|
|
268
|
+
if [ "$PY_STATUS" -ne 0 ]; then
|
|
269
|
+
echo "$DRAIN_RESULT"
|
|
270
|
+
exit "$PY_STATUS"
|
|
271
|
+
fi
|
|
272
|
+
|
|
273
|
+
echo "$DRAIN_RESULT"
|
|
274
|
+
|
|
275
|
+
ENTRIES_DRAINED=$(echo "$DRAIN_RESULT" | grep -E '^entries_drained=' | cut -d= -f2)
|
|
276
|
+
NEW_RISKS=$(echo "$DRAIN_RESULT" | grep -E '^new_risks_created=' | cut -d= -f2)
|
|
277
|
+
EVIDENCE_APPENDED=$(echo "$DRAIN_RESULT" | grep -E '^evidence_appended=' | cut -d= -f2)
|
|
278
|
+
|
|
279
|
+
if [ "${NEW_RISKS:-0}" != "0" ] || [ "${EVIDENCE_APPENDED:-0}" != "0" ]; then
|
|
280
|
+
(cd "$PROJECT_ROOT" && git add docs/risks 2>/dev/null) || true
|
|
281
|
+
fi
|
|
282
|
+
|
|
283
|
+
if [ "${ENTRIES_DRAINED:-0}" != "0" ]; then
|
|
284
|
+
: > "$QUEUE_FILE"
|
|
285
|
+
fi
|
|
286
|
+
|
|
287
|
+
exit 0
|
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
#!/usr/bin/env bats
|
|
2
|
+
# Behavioural-fixture coverage for packages/risk-scorer/scripts/drain-register-queue.sh
|
|
3
|
+
# per ADR-052 (behavioural tests default) and ADR-056 (Phase 2b drain contract).
|
|
4
|
+
#
|
|
5
|
+
# The drain script consumes .afk-run-state/risk-register-queue.jsonl (produced
|
|
6
|
+
# by risk-score-mark.sh per ADR-056 Phase 2a) and materialises register entries
|
|
7
|
+
# in docs/risks/. The script is invoked by consumer skills (this iter:
|
|
8
|
+
# /wr-itil:work-problems Step 6.4); subsequent iters wire additional consumers.
|
|
9
|
+
|
|
10
|
+
setup() {
|
|
11
|
+
REPO_ROOT="$(cd "$(dirname "$BATS_TEST_FILENAME")/../../../.." && pwd)"
|
|
12
|
+
SCRIPT="$REPO_ROOT/packages/risk-scorer/scripts/drain-register-queue.sh"
|
|
13
|
+
SHIM="$REPO_ROOT/packages/risk-scorer/bin/wr-risk-scorer-drain-register-queue"
|
|
14
|
+
WORK_DIR="$(mktemp -d)"
|
|
15
|
+
cd "$WORK_DIR"
|
|
16
|
+
# Minimal git setup — drain script does origin-max lookup via git ls-tree
|
|
17
|
+
git init --quiet
|
|
18
|
+
git config user.email "drain-test@example.com"
|
|
19
|
+
git config user.name "Drain Test"
|
|
20
|
+
git commit --quiet --allow-empty -m "init"
|
|
21
|
+
# Mock template + README
|
|
22
|
+
mkdir -p docs/risks .afk-run-state
|
|
23
|
+
cp "$REPO_ROOT/docs/risks/TEMPLATE.md" docs/risks/TEMPLATE.md
|
|
24
|
+
cp "$REPO_ROOT/docs/risks/README.md" docs/risks/README.md
|
|
25
|
+
cp "$REPO_ROOT/docs/risks/R001-confidential-info-leak-via-public-repo-push.active.md" docs/risks/
|
|
26
|
+
git add docs/risks
|
|
27
|
+
git commit --quiet -m "seed risks"
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
teardown() {
|
|
31
|
+
cd /
|
|
32
|
+
rm -rf "$WORK_DIR"
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
@test "shim wrapper exists and is executable" {
|
|
36
|
+
[ -x "$SHIM" ]
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
@test "shim resolves canonical script (not exit 127)" {
|
|
40
|
+
run "$SHIM" "$WORK_DIR"
|
|
41
|
+
[ "$status" -ne 127 ]
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
@test "empty queue → no-op, exit 0, no writes (ADR-056 idempotent)" {
|
|
45
|
+
: > .afk-run-state/risk-register-queue.jsonl
|
|
46
|
+
before_count=$(find docs/risks -name 'R*.active.md' 2>/dev/null | wc -l | tr -d ' ')
|
|
47
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
48
|
+
[ "$status" -eq 0 ]
|
|
49
|
+
echo "$output" | grep -q '^entries_drained=0$'
|
|
50
|
+
echo "$output" | grep -q '^next_action=none$'
|
|
51
|
+
after_count=$(find docs/risks -name 'R*.active.md' 2>/dev/null | wc -l | tr -d ' ')
|
|
52
|
+
[ "$before_count" = "$after_count" ]
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
@test "missing queue file → no-op, exit 0" {
|
|
56
|
+
rm -f .afk-run-state/risk-register-queue.jsonl
|
|
57
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
58
|
+
[ "$status" -eq 0 ]
|
|
59
|
+
echo "$output" | grep -q '^entries_drained=0$'
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
@test "missing docs/risks/ → no-op, exit 0 (Phase 1 scaffold not yet fired)" {
|
|
63
|
+
rm -rf docs/risks
|
|
64
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
65
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/x.md","reason_tag":"above-appetite-residual","risk_slug":"foo","slug_source":"agent","prefill":"prose"}
|
|
66
|
+
EOF
|
|
67
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
68
|
+
[ "$status" -eq 0 ]
|
|
69
|
+
echo "$output" | grep -q '^entries_drained=0$'
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
@test "single hint, no existing match → creates R<NNN>-<slug>.active.md" {
|
|
73
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
74
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/2026-05-03T14-00-00-commit.md","reason_tag":"above-appetite-residual","risk_slug":"cumulative-residual-commit","slug_source":"agent","prefill":"Cumulative residual at commit hit High band."}
|
|
75
|
+
EOF
|
|
76
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
77
|
+
[ "$status" -eq 0 ]
|
|
78
|
+
echo "$output" | grep -q '^entries_drained=1$'
|
|
79
|
+
echo "$output" | grep -q '^new_risks_created=1$'
|
|
80
|
+
echo "$output" | grep -q '^next_action=commit-staged$'
|
|
81
|
+
# R002 because R001 already exists in the seeded README
|
|
82
|
+
[ -f docs/risks/R002-cumulative-residual-commit.active.md ]
|
|
83
|
+
grep -q 'Status.*Active.*auto-scaffolded.*pending review' docs/risks/R002-cumulative-residual-commit.active.md
|
|
84
|
+
grep -q 'Curation.*pending review' docs/risks/R002-cumulative-residual-commit.active.md
|
|
85
|
+
grep -q 'not estimated.*no prior data' docs/risks/R002-cumulative-residual-commit.active.md
|
|
86
|
+
grep -q 'Cumulative residual at commit hit High band' docs/risks/R002-cumulative-residual-commit.active.md
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
@test "single hint creates README Register table row (ADR-056 step 3d)" {
|
|
90
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
91
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/x.md","reason_tag":"above-appetite-residual","risk_slug":"my-test-risk","slug_source":"agent","prefill":"Test risk prose."}
|
|
92
|
+
EOF
|
|
93
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
94
|
+
[ "$status" -eq 0 ]
|
|
95
|
+
# README must contain a row for the new risk in the Register table
|
|
96
|
+
grep -qE '\| \[R002\]\(R002-my-test-risk\.active\.md\) \|' docs/risks/README.md
|
|
97
|
+
# Stub scoring renders as em-dash columns
|
|
98
|
+
grep -qE 'R002.*my-test-risk.*\|.*—.*\|.*—.*\|.*pending' docs/risks/README.md
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
@test "multiple hints with same slug → one register file, multiple Evidence Log lines" {
|
|
102
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
103
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r1.md","reason_tag":"above-appetite-residual","risk_slug":"shared-slug","slug_source":"agent","prefill":"First mention."}
|
|
104
|
+
{"ts":"2026-05-03T14:01:00Z","session_id":"s1","report_path":".risk-reports/r2.md","reason_tag":"above-appetite-residual","risk_slug":"shared-slug","slug_source":"agent","prefill":"Second mention."}
|
|
105
|
+
{"ts":"2026-05-03T14:02:00Z","session_id":"s2","report_path":".risk-reports/r3.md","reason_tag":"above-appetite-residual","risk_slug":"shared-slug","slug_source":"agent","prefill":"Third mention."}
|
|
106
|
+
EOF
|
|
107
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
108
|
+
[ "$status" -eq 0 ]
|
|
109
|
+
echo "$output" | grep -q '^entries_drained=3$'
|
|
110
|
+
echo "$output" | grep -q '^new_risks_created=1$'
|
|
111
|
+
# Exactly one register file
|
|
112
|
+
[ "$(find docs/risks -name 'R*-shared-slug.active.md' | wc -l | tr -d ' ')" = "1" ]
|
|
113
|
+
# Evidence Log section cites all three reports
|
|
114
|
+
grep -q '.risk-reports/r1.md' docs/risks/R*-shared-slug.active.md
|
|
115
|
+
grep -q '.risk-reports/r2.md' docs/risks/R*-shared-slug.active.md
|
|
116
|
+
grep -q '.risk-reports/r3.md' docs/risks/R*-shared-slug.active.md
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
@test "two distinct slugs in same queue → two register files with sequential IDs" {
|
|
120
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
121
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r1.md","reason_tag":"above-appetite-residual","risk_slug":"first-slug","slug_source":"agent","prefill":"First risk."}
|
|
122
|
+
{"ts":"2026-05-03T14:01:00Z","session_id":"s1","report_path":".risk-reports/r2.md","reason_tag":"confidentiality-disclosure","risk_slug":"second-slug","slug_source":"agent","prefill":"Second risk."}
|
|
123
|
+
EOF
|
|
124
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
125
|
+
[ "$status" -eq 0 ]
|
|
126
|
+
echo "$output" | grep -q '^entries_drained=2$'
|
|
127
|
+
echo "$output" | grep -q '^new_risks_created=2$'
|
|
128
|
+
[ -f docs/risks/R002-first-slug.active.md ]
|
|
129
|
+
[ -f docs/risks/R003-second-slug.active.md ]
|
|
130
|
+
}
|
|
131
|
+
|
|
132
|
+
@test "existing match → appends Evidence Log only, no new file, no scoring change (ADR-056 step 3b)" {
|
|
133
|
+
# Pre-seed an existing risk file with this slug
|
|
134
|
+
cat > docs/risks/R042-known-risk.active.md <<'EOF'
|
|
135
|
+
# Risk R042: Known Risk
|
|
136
|
+
|
|
137
|
+
**Status**: Active
|
|
138
|
+
**Category**: operational
|
|
139
|
+
**Identified**: 2026-04-01
|
|
140
|
+
**Owner**: solo-developer
|
|
141
|
+
**Last reviewed**: 2026-04-01
|
|
142
|
+
**Next review**: 2026-10-01
|
|
143
|
+
|
|
144
|
+
## Description
|
|
145
|
+
|
|
146
|
+
Pre-existing curated risk.
|
|
147
|
+
|
|
148
|
+
## Inherent Risk
|
|
149
|
+
|
|
150
|
+
- **Impact**: 3 (Moderate)
|
|
151
|
+
- **Likelihood**: 2 (Unlikely)
|
|
152
|
+
- **Inherent Score**: 6
|
|
153
|
+
- **Inherent Band**: Medium
|
|
154
|
+
|
|
155
|
+
## Controls
|
|
156
|
+
|
|
157
|
+
- **control-x** — does the thing. Implemented in path/x.
|
|
158
|
+
|
|
159
|
+
## Residual Risk
|
|
160
|
+
|
|
161
|
+
- **Impact**: 2 (Minor)
|
|
162
|
+
- **Likelihood**: 2 (Unlikely)
|
|
163
|
+
- **Residual Score**: 4
|
|
164
|
+
- **Residual Band**: Low
|
|
165
|
+
- **Within appetite?**: Yes
|
|
166
|
+
|
|
167
|
+
## Treatment
|
|
168
|
+
|
|
169
|
+
Mitigate. Justified.
|
|
170
|
+
|
|
171
|
+
## Monitoring
|
|
172
|
+
|
|
173
|
+
- **Trigger to re-assess**: never
|
|
174
|
+
- **Metrics**: none
|
|
175
|
+
|
|
176
|
+
## Related
|
|
177
|
+
|
|
178
|
+
- Criteria: `RISK-POLICY.md`
|
|
179
|
+
|
|
180
|
+
## Change Log
|
|
181
|
+
|
|
182
|
+
- 2026-04-01: Initial identification.
|
|
183
|
+
EOF
|
|
184
|
+
|
|
185
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
186
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/new-fire.md","reason_tag":"above-appetite-residual","risk_slug":"known-risk","slug_source":"agent","prefill":"Fired again."}
|
|
187
|
+
EOF
|
|
188
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
189
|
+
[ "$status" -eq 0 ]
|
|
190
|
+
echo "$output" | grep -q '^entries_drained=1$'
|
|
191
|
+
echo "$output" | grep -q '^new_risks_created=0$'
|
|
192
|
+
echo "$output" | grep -q '^evidence_appended=1$'
|
|
193
|
+
# Existing file untouched on scoring lines
|
|
194
|
+
grep -q 'Inherent Score.*: 6$' docs/risks/R042-known-risk.active.md
|
|
195
|
+
grep -q 'Residual Score.*: 4$' docs/risks/R042-known-risk.active.md
|
|
196
|
+
# Evidence Log section now exists
|
|
197
|
+
grep -q '.risk-reports/new-fire.md' docs/risks/R042-known-risk.active.md
|
|
198
|
+
# No R<NNN+1> file created
|
|
199
|
+
[ ! -f docs/risks/R002-known-risk.active.md ]
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
@test "queue truncated on success" {
|
|
203
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
204
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r1.md","reason_tag":"above-appetite-residual","risk_slug":"truncate-test","slug_source":"agent","prefill":"prose."}
|
|
205
|
+
EOF
|
|
206
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
207
|
+
[ "$status" -eq 0 ]
|
|
208
|
+
# Queue file is empty after success
|
|
209
|
+
[ ! -s .afk-run-state/risk-register-queue.jsonl ]
|
|
210
|
+
}
|
|
211
|
+
|
|
212
|
+
@test "queue NOT truncated on no-op (no docs/risks/ dir)" {
|
|
213
|
+
rm -rf docs/risks
|
|
214
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
215
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r1.md","reason_tag":"above-appetite-residual","risk_slug":"preserve-on-skip","slug_source":"agent","prefill":"prose."}
|
|
216
|
+
EOF
|
|
217
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
218
|
+
[ "$status" -eq 0 ]
|
|
219
|
+
# Queue preserved when drain skips — Phase 1 scaffolding may land later
|
|
220
|
+
[ -s .afk-run-state/risk-register-queue.jsonl ]
|
|
221
|
+
}
|
|
222
|
+
|
|
223
|
+
@test "stdout key=value shape (caller-parseable)" {
|
|
224
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
225
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r.md","reason_tag":"above-appetite-residual","risk_slug":"shape-test","slug_source":"agent","prefill":"prose."}
|
|
226
|
+
EOF
|
|
227
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
228
|
+
[ "$status" -eq 0 ]
|
|
229
|
+
# All four required keys present
|
|
230
|
+
echo "$output" | grep -qE '^entries_drained=[0-9]+$'
|
|
231
|
+
echo "$output" | grep -qE '^new_risks_created=[0-9]+$'
|
|
232
|
+
echo "$output" | grep -qE '^evidence_appended=[0-9]+$'
|
|
233
|
+
echo "$output" | grep -qE '^next_action=(commit-staged|none)$'
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
@test "files staged after successful drain (ready for caller commit)" {
|
|
237
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
238
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r.md","reason_tag":"above-appetite-residual","risk_slug":"stage-test","slug_source":"agent","prefill":"prose."}
|
|
239
|
+
EOF
|
|
240
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
241
|
+
[ "$status" -eq 0 ]
|
|
242
|
+
# Caller should be able to git commit immediately
|
|
243
|
+
staged=$(git diff --cached --name-only)
|
|
244
|
+
echo "$staged" | grep -q 'docs/risks/R002-stage-test.active.md'
|
|
245
|
+
echo "$staged" | grep -q 'docs/risks/README.md'
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
@test "origin-max collision avoidance (ADR-019 ticket-creator dual-source ID)" {
|
|
249
|
+
# Simulate origin/main having higher R-numbers than local. The drain script
|
|
250
|
+
# MUST consult origin-max so parallel adopter sessions don't mint duplicate IDs.
|
|
251
|
+
# We mock by creating a branch with R099 file then resetting local but keeping
|
|
252
|
+
# the ref reachable as origin/main.
|
|
253
|
+
cat > docs/risks/R099-future-risk.active.md <<'EOF'
|
|
254
|
+
# Risk R099: Future risk
|
|
255
|
+
EOF
|
|
256
|
+
git add docs/risks/R099-future-risk.active.md
|
|
257
|
+
git commit --quiet -m "high-id"
|
|
258
|
+
git update-ref refs/remotes/origin/main HEAD
|
|
259
|
+
git rm --quiet docs/risks/R099-future-risk.active.md
|
|
260
|
+
git commit --quiet -m "remove from local"
|
|
261
|
+
# Now local-max sees only R001 (from seeded README) but origin-max should see R099
|
|
262
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
263
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r.md","reason_tag":"above-appetite-residual","risk_slug":"collision-guard","slug_source":"agent","prefill":"prose."}
|
|
264
|
+
EOF
|
|
265
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
266
|
+
[ "$status" -eq 0 ]
|
|
267
|
+
# Next ID must be R100, not R002
|
|
268
|
+
[ -f docs/risks/R100-collision-guard.active.md ]
|
|
269
|
+
[ ! -f docs/risks/R002-collision-guard.active.md ]
|
|
270
|
+
}
|
|
271
|
+
|
|
272
|
+
@test "malformed JSONL line skipped, valid lines processed" {
|
|
273
|
+
cat > .afk-run-state/risk-register-queue.jsonl <<EOF
|
|
274
|
+
not-json-at-all
|
|
275
|
+
{"ts":"2026-05-03T14:00:00Z","session_id":"s1","report_path":".risk-reports/r.md","reason_tag":"above-appetite-residual","risk_slug":"good-line","slug_source":"agent","prefill":"valid prose."}
|
|
276
|
+
{"ts":"bad","incomplete":true}
|
|
277
|
+
EOF
|
|
278
|
+
run bash "$SCRIPT" "$WORK_DIR"
|
|
279
|
+
[ "$status" -eq 0 ]
|
|
280
|
+
echo "$output" | grep -q '^new_risks_created=1$'
|
|
281
|
+
[ -f docs/risks/R002-good-line.active.md ]
|
|
282
|
+
}
|