@windyroad/risk-scorer 0.5.0-preview.279 → 0.6.0-preview.282
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +2 -1
- package/agents/pipeline.md +30 -0
- package/agents/test/risk-scorer-catalog-consumption.bats +138 -0
- package/bin/wr-risk-scorer-extract-risks-from-reports +3 -0
- package/package.json +1 -1
- package/scripts/extract-risks-from-reports.sh +445 -0
- package/scripts/test/extract-risks-from-reports.bats +267 -0
- package/skills/bootstrap-catalog/SKILL.md +218 -0
- package/skills/bootstrap-catalog/test/bootstrap-catalog.bats +162 -0
- package/skills/create-risk/SKILL.md +47 -4
- package/skills/create-risk/test/create-risk-flag-driven.bats +136 -0
package/README.md
CHANGED
|
@@ -73,7 +73,8 @@ The plugin includes six specialised agents:
|
|
|
73
73
|
| `/wr-risk-scorer:assess-wip` | WIP risk nudge for the current uncommitted diff |
|
|
74
74
|
| `/wr-risk-scorer:assess-release` | Pipeline risk assessment for the unpushed queue (pre-satisfies the commit gate) |
|
|
75
75
|
| `/wr-risk-scorer:assess-external-comms` | External-comms leak review for a draft outbound body (pre-satisfies the external-comms gate) |
|
|
76
|
-
| `/wr-risk-scorer:create-risk` | Create a standing-risk register entry |
|
|
76
|
+
| `/wr-risk-scorer:create-risk` | Create a standing-risk register entry (interactive authoring; orchestrator-driven prefilled invocation via `--slug` / `--prefill` flags per ADR-059) |
|
|
77
|
+
| `/wr-risk-scorer:bootstrap-catalog` | Bootstrap `docs/risks/` register from existing `.risk-reports/` corpus per ADR-059 — walks reports, dedupes by ADR-056 slug, emits one `R<NNN>-<slug>.active.md` per unique slug. Idempotent. Auto-triggers from `/install-updates` Step 6.5.1 when register is empty + `RISK-POLICY.md` present + `.risk-reports/` non-empty |
|
|
77
78
|
| `/wr-risk-scorer:update-policy` | Generate or update `RISK-POLICY.md` |
|
|
78
79
|
|
|
79
80
|
## External-comms gate
|
package/agents/pipeline.md
CHANGED
|
@@ -42,6 +42,32 @@ You receive structured pipeline state context with these sections:
|
|
|
42
42
|
- **UNRELEASED CHANGES**: Changeset count and cumulative diff
|
|
43
43
|
- **STALE FILES**: Modified files uncommitted for over 24h
|
|
44
44
|
|
|
45
|
+
## Catalog Consumption Protocol (ADR-059)
|
|
46
|
+
|
|
47
|
+
Before scoring, READ the standing-risk catalog at `docs/risks/` and filter to risks applicable to THIS action. The catalog is the persistent record of risk classes the project has surfaced; consuming it eliminates the wasted-effort cost of re-deriving risk classes on every assessment AND closes the missed-risk-class hazard (forgetting a class the agent surfaced before because it didn't think of it this time). Per `RISK-POLICY.md` `## Risk Catalog` section.
|
|
48
|
+
|
|
49
|
+
**Filter mechanism — hybrid (slug-token-match primary, judgement fallback):**
|
|
50
|
+
|
|
51
|
+
1. **Slug-token-match (primary, deterministic)** — for each `R<NNN>-<slug>.active.md` entry in `docs/risks/`, extract the slug from the filename. Tokenise the slug (split on hyphens). If any token appears in the diff content, commit message, or recent prompt context, the entry is **slug-matched** for this action.
|
|
52
|
+
2. **Free-form judgement (fallback)** — for entries the slug-match path missed, READ the entry's `## Description` section and judge applicability against the diff/commit/prompt context. If the description names a risk shape that THIS action plausibly triggers, the entry is **judgement-matched**.
|
|
53
|
+
3. **Logging** — record the match path on each matched risk-item so the next agent can carry it forward (see Risk Item Format below).
|
|
54
|
+
|
|
55
|
+
**Residual reconciliation:**
|
|
56
|
+
|
|
57
|
+
- The catalog entry's residual is the **lifetime baseline** under documented controls (the controls present in the project as a whole).
|
|
58
|
+
- THIS action's residual is the baseline modulated by the controls present (or absent) in this specific change.
|
|
59
|
+
- The pipeline's `RISK_SCORES:` output MUST carry the per-action residual, NOT the catalog's lifetime baseline. Gates fire on per-action thresholds.
|
|
60
|
+
- The catalog's residual is meaningful CONTEXT: log it as `Catalog baseline:` in the risk-item block so reviewers can compare the lifetime baseline against this-action's residual.
|
|
61
|
+
|
|
62
|
+
**Empty catalog handling:**
|
|
63
|
+
|
|
64
|
+
- If `docs/risks/` is empty (no `R*-*.active.md` files) BUT `RISK-POLICY.md` is present AND `.risk-reports/` is non-empty, emit a one-line nudge in the report body (NOT the `RISK_SCORES:` line): `"Risk register is empty; run /install-updates or /wr-risk-scorer:bootstrap-catalog to bootstrap from .risk-reports/ corpus."` Do NOT halt; do NOT block; do NOT inflate the per-action residual to compensate.
|
|
65
|
+
- If `docs/risks/` is empty AND `RISK-POLICY.md` is absent, the project hasn't opted into the catalog framing. Silent skip the catalog protocol; proceed with regeneration-from-scratch as before this protocol landed.
|
|
66
|
+
|
|
67
|
+
**Per-run hit-rate observability:**
|
|
68
|
+
|
|
69
|
+
After scoring, emit a `CATALOG_HIT_RATE: matched=N missed=M` line to the report (where `matched` counts catalog-matched risks AND `missed` counts risks the agent surfaced this run that weren't in the catalog — those become `RISK_REGISTER_HINT:` candidates per ADR-056). Below ~30% sustained hit rate is a Reassessment signal per ADR-059.
|
|
70
|
+
|
|
45
71
|
## Cumulative Risk Report
|
|
46
72
|
|
|
47
73
|
The report MUST assess risk cumulatively, building up from the release queue:
|
|
@@ -81,11 +107,15 @@ Commit score >= push score >= release score (risk accumulates upward).
|
|
|
81
107
|
- Inherent impact: N/5 (Label) - [why]
|
|
82
108
|
- Inherent likelihood: N/5 (Label) - [why]
|
|
83
109
|
- Inherent risk: N/25 (Label)
|
|
110
|
+
- Catalog match: [slug-token | judgement | none]
|
|
111
|
+
- Catalog baseline: R<NNN> residual=N/25 (Label) — [if matched, cite the catalog entry's lifetime residual; omit line entirely when match=none]
|
|
84
112
|
- Controls:
|
|
85
113
|
- [Specific test file/scenario or hook name] - reduces [dimension] from N to N because [rationale]
|
|
86
114
|
- **Residual risk: N/25 (Label)**
|
|
87
115
|
```
|
|
88
116
|
|
|
117
|
+
The `Catalog match:` and `Catalog baseline:` lines (ADR-059) make the catalog consumption auditable per risk-item. `slug-token` indicates the primary deterministic match; `judgement` indicates the fallback applicability judgement; `none` indicates the risk wasn't in the catalog (and the agent should consider whether to emit a `RISK_REGISTER_HINT:` for it per ADR-056).
|
|
118
|
+
|
|
89
119
|
### Score File Values
|
|
90
120
|
|
|
91
121
|
- Commit score: Layer 3 cumulative (highest)
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
#!/usr/bin/env bats
|
|
2
|
+
# Doc-lint guard: pipeline scorer MUST define the catalog consumption protocol
|
|
3
|
+
# per ADR-059 — read docs/risks/ first, hybrid filter (slug-token primary,
|
|
4
|
+
# judgement fallback), residual reconciliation (per-action residual in
|
|
5
|
+
# RISK_SCORES, catalog lifetime baseline in risk-item block), per-run
|
|
6
|
+
# CATALOG_HIT_RATE observability line.
|
|
7
|
+
#
|
|
8
|
+
# Structural assertions — Permitted Exception to the source-grep ban (ADR-005 / P011).
|
|
9
|
+
# Agent prompts are specification documents; behavioural verification of an LLM's
|
|
10
|
+
# output is out of scope for bats — the contract document is what consuming
|
|
11
|
+
# orchestrators and reviewers rely on. This pattern matches existing tests in
|
|
12
|
+
# this directory (see risk-scorer-register-hint.bats).
|
|
13
|
+
#
|
|
14
|
+
# Cross-reference:
|
|
15
|
+
# ADR-059: docs/decisions/059-pipeline-consume-catalog-and-bootstrap-from-reports.proposed.md
|
|
16
|
+
# ADR-056: docs/decisions/056-risk-register-back-channel-write-contract.proposed.md (slug primitive consumed)
|
|
17
|
+
# ADR-015: docs/decisions/015-on-demand-assessment-skills.proposed.md (pure-scorer contract preserved)
|
|
18
|
+
# ADR-026: docs/decisions/026-agent-output-grounding.proposed.md
|
|
19
|
+
# P168: docs/problems/168-risk-scorer-doesnt-consume-catalog-or-bootstrap.known-error.md
|
|
20
|
+
# P167: docs/problems/167-risk-register-aggregate-reads-as-dont-ship.known-error.md
|
|
21
|
+
# @jtbd JTBD-001 (enforce governance without slowing down — closes missed-risk-class hazard)
|
|
22
|
+
# @jtbd JTBD-202 (pre-flight governance — catalog as ISO 31000/27001 audit-trail artefact)
|
|
23
|
+
|
|
24
|
+
setup() {
|
|
25
|
+
AGENTS_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)"
|
|
26
|
+
PIPELINE="${AGENTS_DIR}/pipeline.md"
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
30
|
+
# Contract surface: Catalog Consumption Protocol section exists
|
|
31
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
32
|
+
|
|
33
|
+
@test "pipeline.md defines Catalog Consumption Protocol section" {
|
|
34
|
+
run grep -q "## Catalog Consumption Protocol" "$PIPELINE"
|
|
35
|
+
[ "$status" -eq 0 ]
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
@test "pipeline.md cites ADR-059 in the catalog protocol section" {
|
|
39
|
+
run grep -q "ADR-059" "$PIPELINE"
|
|
40
|
+
[ "$status" -eq 0 ]
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
@test "pipeline.md names docs/risks/ as the catalog read source" {
|
|
44
|
+
run grep -qE "READ.*docs/risks/|read.*standing-risk catalog at .docs/risks/" "$PIPELINE"
|
|
45
|
+
[ "$status" -eq 0 ]
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
49
|
+
# Hybrid filter: slug-token-match primary, judgement fallback
|
|
50
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
51
|
+
|
|
52
|
+
@test "pipeline.md describes slug-token-match as primary filter path" {
|
|
53
|
+
run grep -qE "[Ss]lug-token-match.*primary|[Ss]lug-token-match \(primary" "$PIPELINE"
|
|
54
|
+
[ "$status" -eq 0 ]
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
@test "pipeline.md describes judgement as fallback filter path" {
|
|
58
|
+
run grep -qE "[Jj]udgement.*fallback|[Ff]ree-form judgement.*fallback" "$PIPELINE"
|
|
59
|
+
[ "$status" -eq 0 ]
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
63
|
+
# Risk Item Format: Catalog match + Catalog baseline lines
|
|
64
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
65
|
+
|
|
66
|
+
@test "pipeline.md Risk Item Format includes Catalog match line" {
|
|
67
|
+
run grep -q "Catalog match:" "$PIPELINE"
|
|
68
|
+
[ "$status" -eq 0 ]
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
@test "pipeline.md Risk Item Format includes Catalog baseline line" {
|
|
72
|
+
run grep -q "Catalog baseline:" "$PIPELINE"
|
|
73
|
+
[ "$status" -eq 0 ]
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
@test "pipeline.md names the three Catalog match values" {
|
|
77
|
+
# slug-token | judgement | none — matches the ADR-059 verdict E3 contract.
|
|
78
|
+
run grep -qE "slug-token.*judgement.*none|slug-token \| judgement \| none" "$PIPELINE"
|
|
79
|
+
[ "$status" -eq 0 ]
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
83
|
+
# Residual reconciliation: per-action residual in RISK_SCORES, baseline contextual
|
|
84
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
85
|
+
|
|
86
|
+
@test "pipeline.md names per-action residual as RISK_SCORES output" {
|
|
87
|
+
run grep -qE "RISK_SCORES.*per-action residual|per-action residual.*RISK_SCORES" "$PIPELINE"
|
|
88
|
+
[ "$status" -eq 0 ]
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
@test "pipeline.md describes catalog lifetime baseline as context not RISK_SCORES" {
|
|
92
|
+
run grep -qE "lifetime baseline|Catalog baseline:" "$PIPELINE"
|
|
93
|
+
[ "$status" -eq 0 ]
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
97
|
+
# Hit-rate observability: CATALOG_HIT_RATE line emitted per run
|
|
98
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
99
|
+
|
|
100
|
+
@test "pipeline.md defines CATALOG_HIT_RATE observability line" {
|
|
101
|
+
run grep -q "CATALOG_HIT_RATE:" "$PIPELINE"
|
|
102
|
+
[ "$status" -eq 0 ]
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
@test "pipeline.md names the CATALOG_HIT_RATE matched + missed columns" {
|
|
106
|
+
run grep -qE "matched=N missed=M|CATALOG_HIT_RATE: matched" "$PIPELINE"
|
|
107
|
+
[ "$status" -eq 0 ]
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
111
|
+
# Empty catalog handling: nudge but do NOT halt or inflate residual
|
|
112
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
113
|
+
|
|
114
|
+
@test "pipeline.md handles empty catalog with nudge not halt" {
|
|
115
|
+
run grep -qE "[Ee]mpty catalog|catalog is empty.*nudge|do NOT halt" "$PIPELINE"
|
|
116
|
+
[ "$status" -eq 0 ]
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
@test "pipeline.md cites bootstrap-catalog skill in empty catalog nudge" {
|
|
120
|
+
run grep -qE "bootstrap-catalog|/install-updates.*bootstrap" "$PIPELINE"
|
|
121
|
+
[ "$status" -eq 0 ]
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
125
|
+
# Pure-scorer contract preserved: no Write tool grant added
|
|
126
|
+
# ──────────────────────────────────────────────────────────────────────────────
|
|
127
|
+
|
|
128
|
+
@test "pipeline.md preserves pure-scorer contract (Read + Glob only)" {
|
|
129
|
+
# The agent's tool grant must remain Read + Glob per ADR-015.
|
|
130
|
+
# Adding Write would break the architectural boundary ADR-059 verdict F2 preserves.
|
|
131
|
+
run grep -qE "^ - Read$" "$PIPELINE"
|
|
132
|
+
[ "$status" -eq 0 ]
|
|
133
|
+
run grep -qE "^ - Glob$" "$PIPELINE"
|
|
134
|
+
[ "$status" -eq 0 ]
|
|
135
|
+
# Negative: Write tool MUST NOT appear in tool grant
|
|
136
|
+
run grep -qE "^ - Write$" "$PIPELINE"
|
|
137
|
+
[ "$status" -ne 0 ]
|
|
138
|
+
}
|
package/package.json
CHANGED
|
@@ -0,0 +1,445 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# extract-risks-from-reports.sh — extract standing-risk catalog entries from
|
|
3
|
+
# the .risk-reports/ corpus per ADR-059. Two-phase contract:
|
|
4
|
+
#
|
|
5
|
+
# PHASE 1 (deterministic — this script's responsibility):
|
|
6
|
+
# - Walk .risk-reports/*.md
|
|
7
|
+
# - Parse RISK_REGISTER_HINT: bullets per ADR-056 3-column shape
|
|
8
|
+
# - Group by slug (dedupe)
|
|
9
|
+
# - Write one docs/risks/R<NNN>-<slug>.active.md per unique slug
|
|
10
|
+
# - Generate docs/risks/README.md Register table
|
|
11
|
+
# - List unhinted reports for Phase 2 LLM-walk
|
|
12
|
+
#
|
|
13
|
+
# PHASE 2 (LLM-driven — bootstrap-catalog SKILL.md responsibility):
|
|
14
|
+
# - Read each unhinted report listed in Phase 1 output
|
|
15
|
+
# - Apply ADR-056 slug-computation rules to derive slug + prefill
|
|
16
|
+
# - Re-invoke this script with --derived-slugs <file> to add them
|
|
17
|
+
#
|
|
18
|
+
# The entry shape is inlined here (the docs/risks/TEMPLATE.md was wiped
|
|
19
|
+
# 2026-05-04 per P167; per user direction "There shouldn't be a template
|
|
20
|
+
# in the directory, because that should be part of the risk creation/
|
|
21
|
+
# capture skill that the extractor uses" — the create-risk skill and
|
|
22
|
+
# this extractor own the entry shape).
|
|
23
|
+
#
|
|
24
|
+
# Usage:
|
|
25
|
+
# extract-risks-from-reports.sh # walk + write entries + README
|
|
26
|
+
# extract-risks-from-reports.sh --dry-run # walk + report; no writes
|
|
27
|
+
# extract-risks-from-reports.sh --derived-slugs F # add slugs from F (Phase 2 input)
|
|
28
|
+
# extract-risks-from-reports.sh --reports DIR # use DIR instead of .risk-reports/
|
|
29
|
+
# extract-risks-from-reports.sh --target DIR # write to DIR instead of docs/risks/
|
|
30
|
+
#
|
|
31
|
+
# Exit codes:
|
|
32
|
+
# 0 — success (any number of entries written, including 0)
|
|
33
|
+
# 1 — pre-condition failure (RISK-POLICY.md absent, no reports, etc.)
|
|
34
|
+
# 2 — usage error
|
|
35
|
+
|
|
36
|
+
set -euo pipefail
|
|
37
|
+
|
|
38
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
39
|
+
# Argument parsing
|
|
40
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
41
|
+
|
|
42
|
+
REPORTS_DIR=".risk-reports"
|
|
43
|
+
TARGET_DIR="docs/risks"
|
|
44
|
+
DRY_RUN=0
|
|
45
|
+
DERIVED_SLUGS_FILE=""
|
|
46
|
+
|
|
47
|
+
while [ "$#" -gt 0 ]; do
|
|
48
|
+
case "$1" in
|
|
49
|
+
--dry-run) DRY_RUN=1; shift ;;
|
|
50
|
+
--derived-slugs) DERIVED_SLUGS_FILE="$2"; shift 2 ;;
|
|
51
|
+
--reports) REPORTS_DIR="$2"; shift 2 ;;
|
|
52
|
+
--target) TARGET_DIR="$2"; shift 2 ;;
|
|
53
|
+
-h|--help)
|
|
54
|
+
sed -n '2,/^set -/p' "$0" | grep '^#' | sed 's/^# \?//'
|
|
55
|
+
exit 0 ;;
|
|
56
|
+
*) echo "Unknown argument: $1" >&2; exit 2 ;;
|
|
57
|
+
esac
|
|
58
|
+
done
|
|
59
|
+
|
|
60
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
61
|
+
# Pre-conditions
|
|
62
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
63
|
+
|
|
64
|
+
if [ ! -f "RISK-POLICY.md" ]; then
|
|
65
|
+
echo "PRE-CONDITION FAILED: RISK-POLICY.md not found in cwd. Project hasn't opted into the catalog framing." >&2
|
|
66
|
+
echo " Recovery: run /wr-risk-scorer:update-policy first." >&2
|
|
67
|
+
exit 1
|
|
68
|
+
fi
|
|
69
|
+
|
|
70
|
+
if [ ! -d "$REPORTS_DIR" ]; then
|
|
71
|
+
echo "PRE-CONDITION FAILED: $REPORTS_DIR/ directory not found." >&2
|
|
72
|
+
echo " Recovery: nothing to extract; the corpus is empty." >&2
|
|
73
|
+
exit 1
|
|
74
|
+
fi
|
|
75
|
+
|
|
76
|
+
REPORT_COUNT=$(find "$REPORTS_DIR" -maxdepth 1 -name '*.md' -type f 2>/dev/null | wc -l | tr -d ' ')
|
|
77
|
+
if [ "$REPORT_COUNT" = "0" ]; then
|
|
78
|
+
echo "PRE-CONDITION FAILED: $REPORTS_DIR/ has zero *.md files." >&2
|
|
79
|
+
echo " Recovery: nothing to extract; the corpus is empty." >&2
|
|
80
|
+
exit 1
|
|
81
|
+
fi
|
|
82
|
+
|
|
83
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
84
|
+
# Phase 1 — deterministic extraction from RISK_REGISTER_HINT bullets
|
|
85
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
86
|
+
|
|
87
|
+
# Use Python for the parse + dedupe. ADR-056 3-column shape:
|
|
88
|
+
# - <reason-tag> | <risk-slug> | <prefill prose>
|
|
89
|
+
# Legacy 2-column shape (ADR-056 dual-parse fallback):
|
|
90
|
+
# - <reason-tag> | <prefill prose>
|
|
91
|
+
# When 2-column, derive slug from reason-tag + first 5 word-stems of prefill
|
|
92
|
+
# (lowercase, kebab, drop articles), capped at 60 chars per ADR-055.
|
|
93
|
+
|
|
94
|
+
WORK_DIR=$(mktemp -d)
|
|
95
|
+
trap 'rm -rf "$WORK_DIR"' EXIT
|
|
96
|
+
|
|
97
|
+
python3 <<PYEOF > "${WORK_DIR}/extracted.tsv"
|
|
98
|
+
import os, re, sys, glob, datetime
|
|
99
|
+
from collections import defaultdict
|
|
100
|
+
|
|
101
|
+
REPORTS_DIR = "$REPORTS_DIR"
|
|
102
|
+
ARTICLES = {"the", "a", "an", "of", "to", "in", "on", "at", "for", "with", "by", "from"}
|
|
103
|
+
RESERVED_TAGS = {"above-appetite-residual", "confidentiality-disclosure", "user-stated-precondition"}
|
|
104
|
+
|
|
105
|
+
def slug_from_prefill(reason_tag, prefill, cap=60):
|
|
106
|
+
text = re.sub(r'[^a-zA-Z0-9\s-]', ' ', prefill.lower())
|
|
107
|
+
words = [w for w in text.split() if w and w not in ARTICLES][:5]
|
|
108
|
+
base = '-'.join(words) if words else reason_tag
|
|
109
|
+
return base[:cap].rstrip('-') or reason_tag
|
|
110
|
+
|
|
111
|
+
# Group by slug → list of (source_file, reason_tag, prefill, slug_source)
|
|
112
|
+
by_slug = defaultdict(list)
|
|
113
|
+
|
|
114
|
+
for path in sorted(glob.glob(f"{REPORTS_DIR}/*.md")):
|
|
115
|
+
try:
|
|
116
|
+
with open(path, 'r') as f:
|
|
117
|
+
content = f.read()
|
|
118
|
+
except Exception:
|
|
119
|
+
continue
|
|
120
|
+
# Find RISK_REGISTER_HINT block; extract bullets until next blank-line-then-non-list
|
|
121
|
+
m = re.search(r'^RISK_REGISTER_HINT:\s*\n((?:^- .*\n?)+)', content, re.MULTILINE)
|
|
122
|
+
if not m:
|
|
123
|
+
continue
|
|
124
|
+
block = m.group(1)
|
|
125
|
+
for line in block.split('\n'):
|
|
126
|
+
line = line.strip()
|
|
127
|
+
if not line.startswith('- '):
|
|
128
|
+
continue
|
|
129
|
+
body = line[2:].strip()
|
|
130
|
+
parts = [p.strip() for p in body.split('|')]
|
|
131
|
+
if len(parts) >= 3:
|
|
132
|
+
reason_tag, slug, prefill = parts[0], parts[1], '|'.join(parts[2:]).strip()
|
|
133
|
+
slug_source = "agent"
|
|
134
|
+
elif len(parts) == 2:
|
|
135
|
+
reason_tag, prefill = parts[0], parts[1]
|
|
136
|
+
slug = slug_from_prefill(reason_tag, prefill)
|
|
137
|
+
slug_source = "derived"
|
|
138
|
+
else:
|
|
139
|
+
continue
|
|
140
|
+
if reason_tag not in RESERVED_TAGS:
|
|
141
|
+
continue # invalid tag — skip silently
|
|
142
|
+
by_slug[slug].append((path, reason_tag, prefill, slug_source))
|
|
143
|
+
|
|
144
|
+
# Emit deterministic-extracted slugs as TSV: slug \t source_count \t first_reason_tag \t first_prefill \t source_files (comma-sep)
|
|
145
|
+
for slug in sorted(by_slug.keys()):
|
|
146
|
+
entries = by_slug[slug]
|
|
147
|
+
source_files = ",".join(sorted(set(e[0] for e in entries)))
|
|
148
|
+
print(f"{slug}\t{len(entries)}\t{entries[0][1]}\t{entries[0][2]}\t{source_files}")
|
|
149
|
+
PYEOF
|
|
150
|
+
|
|
151
|
+
EXTRACTED_COUNT=$(wc -l < "${WORK_DIR}/extracted.tsv" | tr -d ' ')
|
|
152
|
+
|
|
153
|
+
# Find unhinted reports (no RISK_REGISTER_HINT block) — Phase 2 candidates
|
|
154
|
+
python3 <<PYEOF > "${WORK_DIR}/unhinted.txt"
|
|
155
|
+
import os, re, glob
|
|
156
|
+
REPORTS_DIR = "$REPORTS_DIR"
|
|
157
|
+
unhinted = []
|
|
158
|
+
for path in sorted(glob.glob(f"{REPORTS_DIR}/*.md")):
|
|
159
|
+
try:
|
|
160
|
+
with open(path, 'r') as f:
|
|
161
|
+
content = f.read()
|
|
162
|
+
except Exception:
|
|
163
|
+
continue
|
|
164
|
+
if not re.search(r'^RISK_REGISTER_HINT:\s*\n- ', content, re.MULTILINE):
|
|
165
|
+
unhinted.append(path)
|
|
166
|
+
for p in unhinted:
|
|
167
|
+
print(p)
|
|
168
|
+
PYEOF
|
|
169
|
+
|
|
170
|
+
UNHINTED_COUNT=$(wc -l < "${WORK_DIR}/unhinted.txt" | tr -d ' ')
|
|
171
|
+
|
|
172
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
173
|
+
# Phase 2 — derived slugs from caller (e.g. SKILL.md LLM-walk output)
|
|
174
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
175
|
+
|
|
176
|
+
if [ -n "$DERIVED_SLUGS_FILE" ] && [ -f "$DERIVED_SLUGS_FILE" ]; then
|
|
177
|
+
# Append derived slugs to extracted.tsv (same TSV format)
|
|
178
|
+
cat "$DERIVED_SLUGS_FILE" >> "${WORK_DIR}/extracted.tsv"
|
|
179
|
+
EXTRACTED_COUNT=$(wc -l < "${WORK_DIR}/extracted.tsv" | tr -d ' ')
|
|
180
|
+
fi
|
|
181
|
+
|
|
182
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
183
|
+
# Dry-run early exit
|
|
184
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
185
|
+
|
|
186
|
+
if [ "$DRY_RUN" = "1" ]; then
|
|
187
|
+
echo "DRY-RUN summary:"
|
|
188
|
+
echo " reports walked: $REPORT_COUNT"
|
|
189
|
+
echo " hinted (deterministic): $EXTRACTED_COUNT entries"
|
|
190
|
+
echo " unhinted (Phase 2 todo): $UNHINTED_COUNT reports"
|
|
191
|
+
echo
|
|
192
|
+
echo "Extracted slugs:"
|
|
193
|
+
awk -F'\t' '{print " - "$1" (sources: "$2")"}' "${WORK_DIR}/extracted.tsv"
|
|
194
|
+
echo
|
|
195
|
+
echo "Unhinted reports (sample):"
|
|
196
|
+
head -5 "${WORK_DIR}/unhinted.txt" | sed 's|^| - |'
|
|
197
|
+
if [ "$UNHINTED_COUNT" -gt 5 ]; then
|
|
198
|
+
echo " ... and $((UNHINTED_COUNT - 5)) more"
|
|
199
|
+
fi
|
|
200
|
+
exit 0
|
|
201
|
+
fi
|
|
202
|
+
|
|
203
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
204
|
+
# Write entries
|
|
205
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
206
|
+
|
|
207
|
+
mkdir -p "$TARGET_DIR"
|
|
208
|
+
TODAY=$(date -u '+%Y-%m-%d')
|
|
209
|
+
|
|
210
|
+
# Compute starting R<NNN> ID — live-filesystem-max + 1 (defaults to R001 when fresh).
|
|
211
|
+
# Note: deviates from ADR-019 dual-source convention because this script bootstraps
|
|
212
|
+
# the catalog as a clean slate. After wipe, origin still carries the wiped R001-R006
|
|
213
|
+
# until push, so origin-max would force next=7+ when the user wants R001 from clean
|
|
214
|
+
# slate. Filesystem-only is correct for the bootstrap case; ADR-019 still applies to
|
|
215
|
+
# /wr-risk-scorer:create-risk for incremental adds post-bootstrap.
|
|
216
|
+
LOCAL_MAX=$(ls "$TARGET_DIR/"R*.active.md "$TARGET_DIR/"R*.retired.md 2>/dev/null | sed 's|.*/R||' | grep -oE '^[0-9]+' | sort -n | tail -1 || true)
|
|
217
|
+
NEXT_ID=$(( ${LOCAL_MAX:-0} + 1 ))
|
|
218
|
+
|
|
219
|
+
CREATED=0
|
|
220
|
+
APPENDED=0
|
|
221
|
+
|
|
222
|
+
while IFS=$'\t' read -r slug count reason_tag prefill source_files; do
|
|
223
|
+
[ -z "$slug" ] && continue
|
|
224
|
+
# Idempotency: glob for existing R*-<slug>.active.md
|
|
225
|
+
existing=$(ls "$TARGET_DIR/"R*-"${slug}.active.md" 2>/dev/null | head -1 || true)
|
|
226
|
+
if [ -n "$existing" ]; then
|
|
227
|
+
# Append source_files to existing entry's Source Evidence block
|
|
228
|
+
{
|
|
229
|
+
echo ""
|
|
230
|
+
echo "<!-- Source-Evidence append (extract-risks-from-reports.sh, $TODAY) -->"
|
|
231
|
+
echo "Additional sources for slug \`$slug\`:"
|
|
232
|
+
for s in $(echo "$source_files" | tr ',' '\n'); do
|
|
233
|
+
echo "- \`$s\`"
|
|
234
|
+
done
|
|
235
|
+
} >> "$existing"
|
|
236
|
+
APPENDED=$((APPENDED + 1))
|
|
237
|
+
continue
|
|
238
|
+
fi
|
|
239
|
+
|
|
240
|
+
# Heuristic category from reason_tag
|
|
241
|
+
case "$reason_tag" in
|
|
242
|
+
confidentiality-disclosure) category="infosec" ;;
|
|
243
|
+
*) category="operational" ;;
|
|
244
|
+
esac
|
|
245
|
+
|
|
246
|
+
# Derive a Title Case title from the slug
|
|
247
|
+
title=$(echo "$slug" | tr '-' ' ' | awk '{for(i=1;i<=NF;i++) $i=toupper(substr($i,1,1)) tolower(substr($i,2));}1')
|
|
248
|
+
|
|
249
|
+
R_ID=$(printf '%03d' "$NEXT_ID")
|
|
250
|
+
filename="$TARGET_DIR/R${R_ID}-${slug}.active.md"
|
|
251
|
+
|
|
252
|
+
cat > "$filename" <<ENTRY
|
|
253
|
+
# Risk R${R_ID}: ${title}
|
|
254
|
+
|
|
255
|
+
**Status**: Active (auto-scaffolded — pending review)
|
|
256
|
+
**Category**: ${category}
|
|
257
|
+
**Identified**: ${TODAY}
|
|
258
|
+
**Owner**: pending review
|
|
259
|
+
**Last reviewed**: ${TODAY}
|
|
260
|
+
**Next review**: pending review
|
|
261
|
+
**Curation**: pending review (auto-scaffolded ${TODAY})
|
|
262
|
+
|
|
263
|
+
## Description
|
|
264
|
+
|
|
265
|
+
${prefill}
|
|
266
|
+
|
|
267
|
+
## Inherent Risk
|
|
268
|
+
|
|
269
|
+
- **Impact**: not estimated — no prior data
|
|
270
|
+
- **Likelihood**: not estimated — no prior data
|
|
271
|
+
- **Inherent Score**: not estimated — no prior data
|
|
272
|
+
- **Inherent Band**: not estimated — no prior data
|
|
273
|
+
|
|
274
|
+
## Controls
|
|
275
|
+
|
|
276
|
+
(pending review — controls are project-specific and pending human curation)
|
|
277
|
+
|
|
278
|
+
## Residual Risk
|
|
279
|
+
|
|
280
|
+
- **Impact**: not estimated — no prior data
|
|
281
|
+
- **Likelihood**: not estimated — no prior data
|
|
282
|
+
- **Residual Score**: not estimated — no prior data
|
|
283
|
+
- **Residual Band**: not estimated — no prior data
|
|
284
|
+
- **Within appetite?**: pending — scoring not estimated
|
|
285
|
+
|
|
286
|
+
## Treatment
|
|
287
|
+
|
|
288
|
+
pending review
|
|
289
|
+
|
|
290
|
+
## Monitoring
|
|
291
|
+
|
|
292
|
+
- **Trigger to re-assess**: when human curation lands or when controls change
|
|
293
|
+
- **Metrics**: extracted-from \`.risk-reports/\` count for slug \`${slug}\`: ${count}
|
|
294
|
+
|
|
295
|
+
## Related
|
|
296
|
+
|
|
297
|
+
- Criteria: \`RISK-POLICY.md\`
|
|
298
|
+
- Auto-scaffolded by: \`extract-risks-from-reports.sh\` (per ADR-059)
|
|
299
|
+
|
|
300
|
+
## Source Evidence (auto-scaffolded ${TODAY})
|
|
301
|
+
|
|
302
|
+
Aggregated from ${count} \`.risk-reports/\` entries (slug: \`${slug}\`, reason-tag: \`${reason_tag}\`):
|
|
303
|
+
ENTRY
|
|
304
|
+
|
|
305
|
+
for s in $(echo "$source_files" | tr ',' '\n'); do
|
|
306
|
+
echo "- \`$s\`" >> "$filename"
|
|
307
|
+
done
|
|
308
|
+
|
|
309
|
+
cat >> "$filename" <<ENTRY
|
|
310
|
+
|
|
311
|
+
Re-rate when new reports surface against this slug or when controls change.
|
|
312
|
+
|
|
313
|
+
## Change Log
|
|
314
|
+
|
|
315
|
+
- ${TODAY}: Auto-scaffolded by \`extract-risks-from-reports.sh\` from \`.risk-reports/\` corpus per ADR-059.
|
|
316
|
+
ENTRY
|
|
317
|
+
|
|
318
|
+
NEXT_ID=$((NEXT_ID + 1))
|
|
319
|
+
CREATED=$((CREATED + 1))
|
|
320
|
+
done < "${WORK_DIR}/extracted.tsv"
|
|
321
|
+
|
|
322
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
323
|
+
# Generate README.md
|
|
324
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
325
|
+
|
|
326
|
+
cat > "$TARGET_DIR/README.md" <<README_HEADER
|
|
327
|
+
# Risk Register
|
|
328
|
+
|
|
329
|
+
> ISO 31000 / ISO 27001 standing-risk inventory. Per-risk files live alongside this index.
|
|
330
|
+
> Last reviewed: ${TODAY} (auto-generated by \`extract-risks-from-reports.sh\` per ADR-059)
|
|
331
|
+
|
|
332
|
+
## Purpose
|
|
333
|
+
|
|
334
|
+
This directory is the **persistent risk register** for the Windy Road Agent Plugins suite. It is distinct from:
|
|
335
|
+
|
|
336
|
+
- \`RISK-POLICY.md\` — defines the *criteria* (impact/likelihood scales, appetite, treatment principles).
|
|
337
|
+
- \`.risk-reports/\` — ephemeral **per-change** pipeline risk reports produced by the risk-scorer on each commit/push/release. Auto-deleted after 7 days.
|
|
338
|
+
- \`docs/problems/\` — ITIL problem management (concrete defects and their fixes).
|
|
339
|
+
|
|
340
|
+
The risk register captures **standing risks** — risks that persist across changes and require ongoing treatment. Each risk has an owner, treatment plan, inherent and residual scores, and review date.
|
|
341
|
+
|
|
342
|
+
Per ADR-059, the register is populated by \`/wr-risk-scorer:bootstrap-catalog\` (one-shot) and \`/wr-risk-scorer:create-risk\` (on-demand or orchestrator-prefilled). The entry shape is owned by the create-risk skill — there is intentionally no \`TEMPLATE.md\` in this directory (per user direction 2026-05-04).
|
|
343
|
+
|
|
344
|
+
## ISO Mapping
|
|
345
|
+
|
|
346
|
+
| ISO Clause | Artefact in this repo |
|
|
347
|
+
|------------|-----------------------|
|
|
348
|
+
| ISO 31000 § 6.4.2 — Risk treatment | Each risk file's \`Treatment\` section |
|
|
349
|
+
| ISO 31000 § 6.4.3 — Residual risk | Each risk file's \`Residual Score\` section |
|
|
350
|
+
| ISO 31000 § 6.5 — Monitoring and review | \`Review date\` field + periodic review pass |
|
|
351
|
+
| ISO 27001 § 6.1.2 — Risk assessment | Risks tagged \`category: infosec\` |
|
|
352
|
+
| ISO 27001 § 6.1.3 — Risk treatment / SoA | \`Treatment\` + \`Controls\` sections |
|
|
353
|
+
|
|
354
|
+
## Structure
|
|
355
|
+
|
|
356
|
+
- One file per risk: \`R<NNN>-<kebab-case-slug>.<status>.md\`
|
|
357
|
+
- Status suffixes: \`.active.md\`, \`.accepted.md\` (consciously tolerated), \`.retired.md\` (no longer relevant)
|
|
358
|
+
- Risks retired, not deleted — historical record is preserved
|
|
359
|
+
- Cross-references to \`docs/problems/P<NNN>\` and \`docs/decisions/ADR-<NNN>\` welcome
|
|
360
|
+
- Auto-scaffolded entries carry \`Status: Active (auto-scaffolded — pending review)\` + \`Curation: pending review\` markers; human curation upgrades them to \`Status: Active\`.
|
|
361
|
+
|
|
362
|
+
## Register
|
|
363
|
+
|
|
364
|
+
| ID | Title | Category | Status | Curation |
|
|
365
|
+
|----|-------|----------|--------|----------|
|
|
366
|
+
README_HEADER
|
|
367
|
+
|
|
368
|
+
# Append register rows
|
|
369
|
+
if ls "$TARGET_DIR/"R*.active.md >/dev/null 2>&1; then
|
|
370
|
+
for entry in "$TARGET_DIR/"R*.active.md; do
|
|
371
|
+
[ -f "$entry" ] || continue
|
|
372
|
+
fn=$(basename "$entry")
|
|
373
|
+
# Extract title (line 1 after "# Risk R<NNN>: ")
|
|
374
|
+
title=$(head -1 "$entry" | sed -E 's/^# Risk R[0-9]+: //')
|
|
375
|
+
# Extract category
|
|
376
|
+
category=$(grep -m1 '^\*\*Category\*\*:' "$entry" | sed -E 's/^\*\*Category\*\*:\s*//' | tr -d '\n')
|
|
377
|
+
# Extract Status
|
|
378
|
+
status=$(grep -m1 '^\*\*Status\*\*:' "$entry" | sed -E 's/^\*\*Status\*\*:\s*//' | tr -d '\n')
|
|
379
|
+
# Extract Curation if present
|
|
380
|
+
curation=$(grep -m1 '^\*\*Curation\*\*:' "$entry" | sed -E 's/^\*\*Curation\*\*:\s*//' | tr -d '\n')
|
|
381
|
+
[ -z "$curation" ] && curation="(human-curated)"
|
|
382
|
+
rid=$(echo "$fn" | grep -oE '^R[0-9]+')
|
|
383
|
+
echo "| [$rid]($fn) | $title | $category | $status | $curation |" >> "$TARGET_DIR/README.md"
|
|
384
|
+
done
|
|
385
|
+
fi
|
|
386
|
+
|
|
387
|
+
cat >> "$TARGET_DIR/README.md" <<README_FOOTER
|
|
388
|
+
|
|
389
|
+
## Retired
|
|
390
|
+
|
|
391
|
+
| ID | Title | Retired date | Reason |
|
|
392
|
+
|----|-------|--------------|--------|
|
|
393
|
+
README_FOOTER
|
|
394
|
+
|
|
395
|
+
# Append retired rows
|
|
396
|
+
if ls "$TARGET_DIR/"R*.retired.md >/dev/null 2>&1; then
|
|
397
|
+
for entry in "$TARGET_DIR/"R*.retired.md; do
|
|
398
|
+
[ -f "$entry" ] || continue
|
|
399
|
+
fn=$(basename "$entry")
|
|
400
|
+
title=$(head -1 "$entry" | sed -E 's/^# Risk R[0-9]+: //')
|
|
401
|
+
rid=$(echo "$fn" | grep -oE '^R[0-9]+')
|
|
402
|
+
echo "| [$rid]($fn) | $title | (see file) | (see file) |" >> "$TARGET_DIR/README.md"
|
|
403
|
+
done
|
|
404
|
+
fi
|
|
405
|
+
|
|
406
|
+
cat >> "$TARGET_DIR/README.md" <<README_FOOTER2
|
|
407
|
+
|
|
408
|
+
## How to Add a Risk
|
|
409
|
+
|
|
410
|
+
- **On demand**: invoke \`/wr-risk-scorer:create-risk\` (interactive authoring) or \`/wr-risk-scorer:create-risk --slug <slug> --prefill <prose>\` (orchestrator-driven prefilled invocation per ADR-059).
|
|
411
|
+
- **From the report corpus**: invoke \`/wr-risk-scorer:bootstrap-catalog\` (one-shot bootstrap that walks \`.risk-reports/\` and writes one entry per unique slug per ADR-056 / ADR-059).
|
|
412
|
+
|
|
413
|
+
## How to Review
|
|
414
|
+
|
|
415
|
+
On review date, re-assess likelihood and residual score. Update controls as systems evolve. Retire risks that no longer apply (rename to \`.retired.md\`).
|
|
416
|
+
|
|
417
|
+
Auto-scaffolded entries (Status: \`Active (auto-scaffolded — pending review)\`) await human curation: assign scoring, document controls, set Treatment, then flip Status to \`Active\`.
|
|
418
|
+
README_FOOTER2
|
|
419
|
+
|
|
420
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
421
|
+
# Report
|
|
422
|
+
# ─────────────────────────────────────────────────────────────────────────────
|
|
423
|
+
|
|
424
|
+
echo "Risk register extraction complete."
|
|
425
|
+
echo
|
|
426
|
+
echo " Reports walked: $REPORT_COUNT"
|
|
427
|
+
echo " Hinted (deterministic): $EXTRACTED_COUNT slugs"
|
|
428
|
+
echo " New entries created: $CREATED"
|
|
429
|
+
echo " Existing entries updated: $APPENDED"
|
|
430
|
+
echo " Unhinted (Phase 2 todo): $UNHINTED_COUNT reports"
|
|
431
|
+
echo
|
|
432
|
+
echo "Output:"
|
|
433
|
+
echo " - $TARGET_DIR/R<NNN>-<slug>.active.md (one per unique slug)"
|
|
434
|
+
echo " - $TARGET_DIR/README.md (regenerated)"
|
|
435
|
+
echo
|
|
436
|
+
if [ "$UNHINTED_COUNT" -gt 0 ]; then
|
|
437
|
+
echo "Phase 2 LLM-walk required for $UNHINTED_COUNT unhinted reports."
|
|
438
|
+
echo " Sample (first 5):"
|
|
439
|
+
head -5 "${WORK_DIR}/unhinted.txt" | sed 's|^| - |'
|
|
440
|
+
if [ "$UNHINTED_COUNT" -gt 5 ]; then
|
|
441
|
+
echo " ... and $((UNHINTED_COUNT - 5)) more"
|
|
442
|
+
fi
|
|
443
|
+
echo " Driver: bootstrap-catalog SKILL.md Step 2 — agent walks each, derives slug + prefill,"
|
|
444
|
+
echo " re-invokes this script with --derived-slugs <file> to add them."
|
|
445
|
+
fi
|