@windyroad/risk-scorer 0.17.2 → 0.17.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,7 +1,11 @@
1
+ <!-- Generated from packages/risk-scorer/skills/*/SKILL.md by packages/risk-scorer/scripts/sync-codex-skills.mjs during npm pack. Do not edit packaged output directly. -->
2
+
3
+ > Codex runtime note: use `request_user_input` where this skill needs structured user input. If a step refers to Claude-style agent dispatch or `subagent_type`, invoke the matching installed Codex agent when available; in `codex exec` or any runtime without custom-agent dispatch, perform the same review inline from the plugin's sibling `agents/*.md` instructions and preserve the structured verdict blocks.
4
+
1
5
  ---
2
6
  name: wr-risk-scorer:update-policy
3
7
  description: Create or update the project's RISK-POLICY.md per ISO 31000 and the risk-scorer agent. Examines the project to derive business-specific impact levels.
4
- allowed-tools: Read, Write, Edit, Bash, Glob, Grep, AskUserQuestion, Agent
8
+ allowed-tools: Read, Write, Edit, Bash, Glob, Grep, request_user_input, Agent
5
9
  ---
6
10
 
7
11
  # Risk Policy Generator
@@ -26,7 +30,7 @@ The risk-scorer agent, problem management skill, and any other process that need
26
30
 
27
31
  ### 1. Read the risk-scorer agent contract
28
32
 
29
- Read `.claude/agents/risk-scorer-pipeline.md` to understand what the scorer expects from `RISK-POLICY.md`. Extract:
33
+ Read `agents/pipeline.md` to understand what the scorer expects from `RISK-POLICY.md`. Extract:
30
34
 
31
35
  - What fields the agent reads from the policy (look for "Read `RISK-POLICY.md`" in "Your Role")
32
36
  - The impact level labels used in the agent's risk matrix (look for the "Product Reference Table")
@@ -124,9 +128,9 @@ The risk matrix is used by both the **risk-scorer agent** (pipeline risk assessm
124
128
 
125
129
  ### 6. Confirm with the user
126
130
 
127
- You MUST use the AskUserQuestion tool (not plain text output) to collect user confirmation. Do not proceed to step 6a or step 7 until you have received answers via AskUserQuestion.
131
+ You MUST use the request_user_input tool (not plain text output) to collect user confirmation. Do not proceed to step 6a or step 7 until you have received answers via request_user_input.
128
132
 
129
- Call AskUserQuestion with a single message that presents:
133
+ Call request_user_input with a single message that presents:
130
134
 
131
135
  1. The drafted impact levels (as a table) and asks whether they accurately reflect what matters most
132
136
  2. The risk appetite threshold -- present the label bands from the agent contract (step 1) and recommend a threshold based on project maturity. Ask the user to confirm or adjust. A prototype with no real users may tolerate higher risk than a production system with paying users or compliance requirements
@@ -134,7 +138,7 @@ Call AskUserQuestion with a single message that presents:
134
138
 
135
139
  ### 6a. Tight-appetite warning when threshold < 5 (ADR-086)
136
140
 
137
- If the user picked an appetite threshold below 5 in step 6, fire a second `AskUserQuestion` confirm-with-warning before proceeding. The Low band's ceiling under ADR-086 is 5; an appetite below 5 means a class of risks (those with Impact=5/Severe and no impact-reducing control available) can never be within appetite — the policy is mathematically infeasible for that class. The user can still set the tighter threshold (some domains genuinely want to prohibit severe-impact activities), but the consequence must be a conscious choice, not a quiet trap.
141
+ If the user picked an appetite threshold below 5 in step 6, fire a second `request_user_input` confirm-with-warning before proceeding. The Low band's ceiling under ADR-086 is 5; an appetite below 5 means a class of risks (those with Impact=5/Severe and no impact-reducing control available) can never be within appetite — the policy is mathematically infeasible for that class. The user can still set the tighter threshold (some domains genuinely want to prohibit severe-impact activities), but the consequence must be a conscious choice, not a quiet trap.
138
142
 
139
143
  **Build the warning's example list** (cite concrete activity-classes the user is about to prohibit):
140
144
 
@@ -152,7 +156,7 @@ If the user picked an appetite threshold below 5 in step 6, fire a second `AskUs
152
156
  - **Confirm: appetite N, with severe-impact activities prohibited** — the user explicitly accepts the trade-off. Proceed to step 7.
153
157
  - **Revise the appetite** — return to step 6 with the appetite question only.
154
158
 
155
- **Non-interactive (AFK) fallback** per ADR-013 Rule 6: if `AskUserQuestion` is unavailable, do NOT proceed — the warning is load-bearing (prohibits an entire risk class) and silently consuming it would normalise the prohibition. Halt with a clear "appetite < 5 selected interactively required" message for the orchestrator to drain later.
159
+ **Non-interactive (AFK) fallback** per ADR-013 Rule 6: if `request_user_input` is unavailable, do NOT proceed — the warning is load-bearing (prohibits an entire risk class) and silently consuming it would normalise the prohibition. Halt with a clear "appetite < 5 selected interactively required" message for the orchestrator to drain later.
156
160
 
157
161
  ### 7. Validate draft with risk-scorer agent
158
162
 
@@ -1,3 +1,7 @@
1
+ <!-- Generated from packages/risk-scorer/skills/*/SKILL.md by packages/risk-scorer/scripts/sync-codex-skills.mjs during npm pack. Do not edit packaged output directly. -->
2
+
3
+ > Codex runtime note: use `request_user_input` where this skill needs structured user input. If a step refers to Claude-style agent dispatch or `subagent_type`, invoke the matching installed Codex agent when available; in `codex exec` or any runtime without custom-agent dispatch, perform the same review inline from the plugin's sibling `agents/*.md` instructions and preserve the structured verdict blocks.
4
+
1
5
  ---
2
6
  name: wr-risk-scorer:wip
3
7
  description: Invokable SKILL wrapper around the wr-risk-scorer:wip nudge agent. Delegates to the agent via the Agent tool and returns the agent's structured WIP risk verdict. Internal-use plumbing used by `/wr-risk-scorer:assess-wip` per ADR-015's Confirmation literal phrasing. End users should invoke `/wr-risk-scorer:assess-wip` instead.
@@ -1,132 +0,0 @@
1
- #!/usr/bin/env bats
2
- # Contract assertions for /wr-risk-scorer:assess-inbound-report skill
3
- # (RFC-004 Slice B — on-demand wrapper per ADR-015). Peer of
4
- # /wr-risk-scorer:assess-external-comms.
5
- #
6
- # Structural assertions — Permitted Exception to the source-grep ban
7
- # per ADR-005 / P011 / ADR-037 / ADR-052 § Surface 2. SKILL.md prose
8
- # governs LLM-driven runtime behaviour; behavioural-replay testing
9
- # requires a synthetic agent harness (P012 / P176). Until that harness
10
- # lands, contract bats assert the load-bearing contract elements are
11
- # present so future edits don't silently strip them.
12
- #
13
- # @problem P079
14
- # @rfc RFC-004 (Slice B)
15
- # @adr ADR-062 (sibling subagent + on-demand wrapper)
16
- # @adr ADR-015 (on-demand assessment skills — § Scope table extended)
17
- # @adr ADR-044 (decision-delegation — taste / silent-mechanical authority)
18
- # @jtbd JTBD-005 (invoke governance assessments on demand)
19
- # @jtbd JTBD-202 (pre-flight governance checks before release/handover)
20
- # @jtbd JTBD-001 (mechanical-stage carve-out on pipeline pre-satisfier path)
21
-
22
- setup() {
23
- SKILL_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)"
24
- SKILL_FILE="${SKILL_DIR}/SKILL.md"
25
- ADR_015="$(cd "${SKILL_DIR}/../../../.." && pwd)/docs/decisions/015-on-demand-assessment-skills.proposed.md"
26
- }
27
-
28
- @test "SKILL.md exists and has frontmatter" {
29
- [ -f "$SKILL_FILE" ]
30
- run head -1 "$SKILL_FILE"
31
- [ "$status" -eq 0 ]
32
- [ "$output" = "---" ]
33
- }
34
-
35
- @test "frontmatter name is wr-risk-scorer:assess-inbound-report" {
36
- run grep -nE '^name: wr-risk-scorer:assess-inbound-report$' "$SKILL_FILE"
37
- [ "$status" -eq 0 ]
38
- }
39
-
40
- @test "frontmatter allowed-tools includes Skill (delegates to subagent)" {
41
- # ADR-015 § Gate Marker Interaction: on-demand skills MUST delegate
42
- # via Skill tool; never write markers directly.
43
- run grep -nE '^allowed-tools:.*Skill' "$SKILL_FILE"
44
- [ "$status" -eq 0 ]
45
- }
46
-
47
- @test "frontmatter allowed-tools includes AskUserQuestion (manual-mode step 6)" {
48
- # Step 6 (manual invocation only — silent on pipeline pre-satisfier
49
- # invocations per P132) uses AskUserQuestion to surface next-step
50
- # options.
51
- run grep -nE '^allowed-tools:.*AskUserQuestion' "$SKILL_FILE"
52
- [ "$status" -eq 0 ]
53
- }
54
-
55
- @test "frontmatter allowed-tools includes Bash (gh issue fetch in step 2)" {
56
- # Step 2 can call `gh issue view --json body,author,labels` to fetch
57
- # the report body when only a URL/ref is supplied.
58
- run grep -nE '^allowed-tools:.*Bash' "$SKILL_FILE"
59
- [ "$status" -eq 0 ]
60
- }
61
-
62
- # ──────────────────────────────────────────────────────────────────────────────
63
- # Delegation to the sibling subagent (NOT marker self-writes)
64
- # ──────────────────────────────────────────────────────────────────────────────
65
-
66
- @test "skill delegates to wr-risk-scorer:inbound-report subagent" {
67
- run grep -nE 'wr-risk-scorer:inbound-report' "$SKILL_FILE"
68
- [ "$status" -eq 0 ]
69
- }
70
-
71
- @test "skill MUST NOT write to /tmp/ markers directly (ADR-009 + ADR-015 boundary)" {
72
- # PostToolUse:Agent hook (risk-score-mark.sh) owns marker writes per
73
- # ADR-009 + ADR-015 § Gate Marker Interaction.
74
- run grep -inE 'NOT write.*/tmp|PostToolUse hook' "$SKILL_FILE"
75
- [ "$status" -eq 0 ]
76
- }
77
-
78
- # ──────────────────────────────────────────────────────────────────────────────
79
- # Mechanical-stage carve-out: pipeline pre-satisfier path is silent
80
- # ──────────────────────────────────────────────────────────────────────────────
81
-
82
- @test "skill names the mechanical-stage carve-out (P132) for pipeline pre-satisfier path" {
83
- run grep -inE 'P132|mechanical-stage carve-out' "$SKILL_FILE"
84
- [ "$status" -eq 0 ]
85
- }
86
-
87
- @test "step 6 AskUserQuestion fires ONLY on manual invocation (not pipeline pre-satisfier)" {
88
- # The carve-out is the load-bearing protection for JTBD-001 + JTBD-006
89
- # against inverse-P078 drift. The pipeline pre-satisfier path MUST be
90
- # silent on this step. Match the contract in either direction:
91
- # manual-only firing OR pipeline-pre-satisfier silent-on-step.
92
- run grep -inE 'invoked manually.*pre-flight|manual only|silent on this step|silent on.*pipeline pre-satisfier' "$SKILL_FILE"
93
- [ "$status" -eq 0 ]
94
- }
95
-
96
- # ──────────────────────────────────────────────────────────────────────────────
97
- # Persona anchors (JTBD-005 + JTBD-202)
98
- # ──────────────────────────────────────────────────────────────────────────────
99
-
100
- @test "skill cites JTBD-005 (invoke on demand) as primary persona driver" {
101
- run grep -nE 'JTBD-005' "$SKILL_FILE"
102
- [ "$status" -eq 0 ]
103
- }
104
-
105
- @test "skill cites JTBD-202 (pre-flight governance checks) as secondary persona driver" {
106
- run grep -nE 'JTBD-202' "$SKILL_FILE"
107
- [ "$status" -eq 0 ]
108
- }
109
-
110
- # ──────────────────────────────────────────────────────────────────────────────
111
- # ADR-015 Scope table row exists for assess-inbound-report
112
- # ──────────────────────────────────────────────────────────────────────────────
113
-
114
- @test "ADR-015 Scope table includes the assess-inbound-report row" {
115
- [ -f "$ADR_015" ]
116
- run grep -nE '`assess-inbound-report`' "$ADR_015"
117
- [ "$status" -eq 0 ]
118
- run grep -nE '`wr-risk-scorer:inbound-report`' "$ADR_015"
119
- [ "$status" -eq 0 ]
120
- }
121
-
122
- @test "ADR-015 Confirmation checkbox covers assess-inbound-report skill" {
123
- run grep -nE '\[x\] `packages/risk-scorer/skills/assess-inbound-report/SKILL\.md` created' "$ADR_015"
124
- [ "$status" -eq 0 ]
125
- }
126
-
127
- @test "ADR-015 Related section names ADR-062 + P079 (driver references)" {
128
- run grep -nE 'ADR-062.*inbound|inbound.*ADR-062' "$ADR_015"
129
- [ "$status" -eq 0 ]
130
- run grep -nE 'P079' "$ADR_015"
131
- [ "$status" -eq 0 ]
132
- }
@@ -1,162 +0,0 @@
1
- #!/usr/bin/env bats
2
- # Contract guard: the on-demand assessment SKILLs (assess-release, assess-wip,
3
- # assess-external-comms) MUST delegate to their scoring agent via the Skill
4
- # tool — not via the Agent tool — matching ADR-015's Confirmation literal
5
- # phrasing ("the skill delegates to wr-risk-scorer:<agent> via the Skill
6
- # tool"). Closes the P205 contradiction surfaced by ADR-015 Confirmation
7
- # vs. SKILL.md prose mismatch.
8
- #
9
- # Structural assertions — Permitted Exception to the source-grep ban
10
- # (ADR-005 / P011), same framing as risk-scorer-register-hint.bats. SKILL.md
11
- # prose IS the contract document the orchestrator (Claude) consumes when
12
- # executing the SKILL; an LLM-output behavioural check is out of scope for
13
- # bats and is the responsibility of the promptfoo harness (ADR-075).
14
- #
15
- # What is asserted (contract, not implementation):
16
- # 1. Each assess-* SKILL's step 5 (release) / step 3 (wip) / step 4
17
- # (external-comms) names `skill:` as the delegation tool parameter
18
- # with the correct wrapper SKILL name as the target.
19
- # 2. None of the assess-* SKILLs name `subagent_type:` as the delegation
20
- # tool parameter (the P205 contradiction class).
21
- # 3. Each wrapper SKILL (`pipeline`, `wip`, `external-comms`) exists at
22
- # its expected path, is namespaced `wr-risk-scorer:<name>`, and
23
- # delegates to its sibling agent via `subagent_type:`.
24
- #
25
- # Cross-reference:
26
- # P205: docs/problems/known-error/205-wr-risk-scorer-assess-release-skill-md-step-5-prose-says-skill-tool-but-provides-subagent-type.md
27
- # ADR-015: docs/decisions/015-on-demand-assessment-skills.proposed.md (Confirmation criteria 189-193)
28
- # ADR-052: docs/decisions/052-behavioural-tests-default.proposed.md (Permitted Exception)
29
- # @jtbd JTBD-005 (invoke governance assessments on demand)
30
- # @jtbd JTBD-101 (extend the suite — plugins expose corresponding skills)
31
-
32
- setup() {
33
- SKILLS_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/../.." && pwd)"
34
-
35
- ASSESS_RELEASE="${SKILLS_DIR}/assess-release/SKILL.md"
36
- ASSESS_WIP="${SKILLS_DIR}/assess-wip/SKILL.md"
37
- ASSESS_EXTERNAL_COMMS="${SKILLS_DIR}/assess-external-comms/SKILL.md"
38
-
39
- WRAPPER_PIPELINE="${SKILLS_DIR}/pipeline/SKILL.md"
40
- WRAPPER_WIP="${SKILLS_DIR}/wip/SKILL.md"
41
- WRAPPER_EXTERNAL_COMMS="${SKILLS_DIR}/external-comms/SKILL.md"
42
- }
43
-
44
- # ──────────────────────────────────────────────────────────────────────────────
45
- # Consumer SKILLs delegate via Skill tool (skill: parameter)
46
- # ──────────────────────────────────────────────────────────────────────────────
47
-
48
- @test "assess-release delegates via skill: wr-risk-scorer:pipeline" {
49
- [ -f "$ASSESS_RELEASE" ]
50
- run grep -E "^skill: wr-risk-scorer:pipeline$" "$ASSESS_RELEASE"
51
- [ "$status" -eq 0 ]
52
- }
53
-
54
- @test "assess-release does NOT use subagent_type: in its delegation block" {
55
- [ -f "$ASSESS_RELEASE" ]
56
- # The P205 contradiction was: prose says "Skill tool" but provides
57
- # subagent_type: wr-risk-scorer:pipeline. After the fix, no
58
- # `subagent_type:` line may appear in the delegation block.
59
- run grep -E "^subagent_type: wr-risk-scorer:pipeline$" "$ASSESS_RELEASE"
60
- [ "$status" -ne 0 ]
61
- }
62
-
63
- @test "assess-wip delegates via skill: wr-risk-scorer:wip" {
64
- [ -f "$ASSESS_WIP" ]
65
- run grep -E "^skill: wr-risk-scorer:wip$" "$ASSESS_WIP"
66
- [ "$status" -eq 0 ]
67
- }
68
-
69
- @test "assess-wip does NOT use subagent_type: in its delegation block" {
70
- [ -f "$ASSESS_WIP" ]
71
- run grep -E "^subagent_type: wr-risk-scorer:wip$" "$ASSESS_WIP"
72
- [ "$status" -ne 0 ]
73
- }
74
-
75
- @test "assess-external-comms delegates via skill: wr-risk-scorer:external-comms" {
76
- [ -f "$ASSESS_EXTERNAL_COMMS" ]
77
- run grep -E "^skill: wr-risk-scorer:external-comms$" "$ASSESS_EXTERNAL_COMMS"
78
- [ "$status" -eq 0 ]
79
- }
80
-
81
- @test "assess-external-comms does NOT use subagent_type: in its delegation block" {
82
- [ -f "$ASSESS_EXTERNAL_COMMS" ]
83
- run grep -E "^subagent_type: wr-risk-scorer:external-comms$" "$ASSESS_EXTERNAL_COMMS"
84
- [ "$status" -ne 0 ]
85
- }
86
-
87
- # ──────────────────────────────────────────────────────────────────────────────
88
- # Wrapper SKILLs exist with correct names and delegate to the agent
89
- # ──────────────────────────────────────────────────────────────────────────────
90
-
91
- @test "wrapper SKILL packages/risk-scorer/skills/pipeline/SKILL.md exists" {
92
- [ -f "$WRAPPER_PIPELINE" ]
93
- }
94
-
95
- @test "wrapper SKILL pipeline declares name: wr-risk-scorer:pipeline" {
96
- [ -f "$WRAPPER_PIPELINE" ]
97
- run grep -E "^name: wr-risk-scorer:pipeline$" "$WRAPPER_PIPELINE"
98
- [ "$status" -eq 0 ]
99
- }
100
-
101
- @test "wrapper SKILL pipeline delegates to the pipeline agent via subagent_type:" {
102
- [ -f "$WRAPPER_PIPELINE" ]
103
- run grep -E "^subagent_type: wr-risk-scorer:pipeline$" "$WRAPPER_PIPELINE"
104
- [ "$status" -eq 0 ]
105
- }
106
-
107
- @test "wrapper SKILL packages/risk-scorer/skills/wip/SKILL.md exists" {
108
- [ -f "$WRAPPER_WIP" ]
109
- }
110
-
111
- @test "wrapper SKILL wip declares name: wr-risk-scorer:wip" {
112
- [ -f "$WRAPPER_WIP" ]
113
- run grep -E "^name: wr-risk-scorer:wip$" "$WRAPPER_WIP"
114
- [ "$status" -eq 0 ]
115
- }
116
-
117
- @test "wrapper SKILL wip delegates to the wip agent via subagent_type:" {
118
- [ -f "$WRAPPER_WIP" ]
119
- run grep -E "^subagent_type: wr-risk-scorer:wip$" "$WRAPPER_WIP"
120
- [ "$status" -eq 0 ]
121
- }
122
-
123
- @test "wrapper SKILL packages/risk-scorer/skills/external-comms/SKILL.md exists" {
124
- [ -f "$WRAPPER_EXTERNAL_COMMS" ]
125
- }
126
-
127
- @test "wrapper SKILL external-comms declares name: wr-risk-scorer:external-comms" {
128
- [ -f "$WRAPPER_EXTERNAL_COMMS" ]
129
- run grep -E "^name: wr-risk-scorer:external-comms$" "$WRAPPER_EXTERNAL_COMMS"
130
- [ "$status" -eq 0 ]
131
- }
132
-
133
- @test "wrapper SKILL external-comms delegates to the external-comms agent via subagent_type:" {
134
- [ -f "$WRAPPER_EXTERNAL_COMMS" ]
135
- run grep -E "^subagent_type: wr-risk-scorer:external-comms$" "$WRAPPER_EXTERNAL_COMMS"
136
- [ "$status" -eq 0 ]
137
- }
138
-
139
- # ──────────────────────────────────────────────────────────────────────────────
140
- # Wrapper SKILLs disambiguate from end-user assess-* surfaces
141
- # ──────────────────────────────────────────────────────────────────────────────
142
-
143
- @test "wrapper pipeline description names assess-release as the end-user surface" {
144
- [ -f "$WRAPPER_PIPELINE" ]
145
- # JTBD-005 persona-fit: solo developer must not land on the raw wrapper
146
- # and miss the assess-* gate-satisfaction wrap-up. Description must
147
- # disambiguate.
148
- run grep -E "assess-release" "$WRAPPER_PIPELINE"
149
- [ "$status" -eq 0 ]
150
- }
151
-
152
- @test "wrapper wip description names assess-wip as the end-user surface" {
153
- [ -f "$WRAPPER_WIP" ]
154
- run grep -E "assess-wip" "$WRAPPER_WIP"
155
- [ "$status" -eq 0 ]
156
- }
157
-
158
- @test "wrapper external-comms description names assess-external-comms as the end-user surface" {
159
- [ -f "$WRAPPER_EXTERNAL_COMMS" ]
160
- run grep -E "assess-external-comms" "$WRAPPER_EXTERNAL_COMMS"
161
- [ "$status" -eq 0 ]
162
- }
@@ -1,168 +0,0 @@
1
- #!/usr/bin/env bats
2
- # Doc-lint guard: /wr-risk-scorer:bootstrap-catalog SKILL.md MUST define
3
- # the runtime contract per ADR-059 verdicts A4 (on-demand surface) +
4
- # B1 (slug dedupe) + C1 (no threshold) + D1 (Source Evidence required).
5
- #
6
- # Structural assertions — Permitted Exception to the source-grep ban (ADR-005 / P011).
7
- # SKILL.md is a specification document; behavioural verification of LLM-driven
8
- # skills is out of scope for bats. The contract document is what consuming
9
- # orchestrators (install-updates Step 6.5.1 auto-trigger) rely on.
10
- #
11
- # Cross-reference:
12
- # ADR-059: docs/decisions/059-pipeline-consume-catalog-and-bootstrap-from-reports.proposed.md
13
- # ADR-056: docs/decisions/056-risk-register-back-channel-write-contract.proposed.md (slug primitive)
14
- # ADR-026: docs/decisions/026-agent-output-grounding.proposed.md (sentinel)
15
- # ADR-047: docs/decisions/047-install-updates-scaffolds-governance-artefacts.proposed.md (parent)
16
- # P168: docs/problems/168-risk-scorer-doesnt-consume-catalog-or-bootstrap.known-error.md
17
- # @jtbd JTBD-001 (enforce governance without slowing down — bootstrap eliminates miss-rate gap)
18
- # @jtbd JTBD-006 (AFK-safety — bootstrap is idempotent + non-interactive)
19
- # @jtbd JTBD-202 (pre-flight governance — catalog as ISO 31000/27001 audit-trail)
20
-
21
- setup() {
22
- SKILL_DIR="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)"
23
- SKILL="${SKILL_DIR}/SKILL.md"
24
- }
25
-
26
- # ──────────────────────────────────────────────────────────────────────────────
27
- # Frontmatter: name, description, allowed-tools, maturity
28
- # ──────────────────────────────────────────────────────────────────────────────
29
-
30
- @test "bootstrap-catalog SKILL.md has frontmatter with correct namespaced name" {
31
- run grep -qE "^name: wr-risk-scorer:bootstrap-catalog" "$SKILL"
32
- [ "$status" -eq 0 ]
33
- }
34
-
35
- @test "bootstrap-catalog SKILL.md frontmatter declares Write tool" {
36
- run grep -qE "^allowed-tools:.*Write" "$SKILL"
37
- [ "$status" -eq 0 ]
38
- }
39
-
40
- @test "bootstrap-catalog SKILL.md frontmatter declares maturity per ADR-053" {
41
- run grep -qE "^maturity: (proposed|accepted|recommended)" "$SKILL"
42
- [ "$status" -eq 0 ]
43
- }
44
-
45
- # ──────────────────────────────────────────────────────────────────────────────
46
- # Pre-conditions (Step 0): RISK-POLICY.md present, docs/risks/ scaffolded, .risk-reports/ non-empty
47
- # ──────────────────────────────────────────────────────────────────────────────
48
-
49
- @test "bootstrap-catalog SKILL.md requires RISK-POLICY.md presence" {
50
- run grep -qE "RISK-POLICY.md.*present|requires.*RISK-POLICY" "$SKILL"
51
- [ "$status" -eq 0 ]
52
- }
53
-
54
- @test "bootstrap-catalog SKILL.md owns docs/risks/ directory lifecycle (no separate scaffold required)" {
55
- # Updated 2026-05-05: the previous "requires scaffold" assertion is obsolete.
56
- # Per user direction 2026-05-04 (commit 8edaf7b), the Phase 1 scaffold step
57
- # + TEMPLATE.md were wiped because the scaffolded entries were wrong content;
58
- # bootstrap-catalog now owns the directory's full lifecycle (mkdir -p on demand,
59
- # writes README + per-slug entries, no separate scaffold step required).
60
- # The SKILL.md was rewritten accordingly; this test now asserts the new contract.
61
- run grep -qE "may or may not exist|creates it on demand|owns the directory's full lifecycle" "$SKILL"
62
- [ "$status" -eq 0 ]
63
- }
64
-
65
- @test "bootstrap-catalog SKILL.md requires .risk-reports/ corpus non-empty" {
66
- run grep -qE "[.]risk-reports/.*non-empty|[.]risk-reports/.*at least one" "$SKILL"
67
- [ "$status" -eq 0 ]
68
- }
69
-
70
- # ──────────────────────────────────────────────────────────────────────────────
71
- # Slug dedupe per ADR-056
72
- # ──────────────────────────────────────────────────────────────────────────────
73
-
74
- @test "bootstrap-catalog SKILL.md cites ADR-056 for slug primitive" {
75
- run grep -q "ADR-056" "$SKILL"
76
- [ "$status" -eq 0 ]
77
- }
78
-
79
- @test "bootstrap-catalog SKILL.md describes dedupe-by-slug per ADR-056" {
80
- run grep -qE "[Dd]edupe by slug|dedupe.*slug|slug is the dedupe key|N reports.*same slug.*ONE" "$SKILL"
81
- [ "$status" -eq 0 ]
82
- }
83
-
84
- # ──────────────────────────────────────────────────────────────────────────────
85
- # No threshold (Verdict C1)
86
- # ──────────────────────────────────────────────────────────────────────────────
87
-
88
- @test "bootstrap-catalog SKILL.md emits one entry per unique slug (no frequency floor)" {
89
- run grep -qE "one R<NNN>-<slug>[.]active[.]md per unique slug|one .*per unique slug" "$SKILL"
90
- [ "$status" -eq 0 ]
91
- }
92
-
93
- # ──────────────────────────────────────────────────────────────────────────────
94
- # Source Evidence block required (Verdict D1)
95
- # ──────────────────────────────────────────────────────────────────────────────
96
-
97
- @test "bootstrap-catalog SKILL.md requires Source Evidence block on new entries" {
98
- run grep -q "## Source Evidence" "$SKILL"
99
- [ "$status" -eq 0 ]
100
- }
101
-
102
- @test "bootstrap-catalog SKILL.md cites originating .risk-reports/ files in Source Evidence" {
103
- run grep -qE "originating reports|originating.*[.]risk-reports/|cite.*[.]risk-reports/" "$SKILL"
104
- [ "$status" -eq 0 ]
105
- }
106
-
107
- @test "bootstrap-catalog SKILL.md cites ADR-026 grounding for Source Evidence pattern" {
108
- run grep -qE "ADR-026.*grounding|grounding.*ADR-026" "$SKILL"
109
- [ "$status" -eq 0 ]
110
- }
111
-
112
- # ──────────────────────────────────────────────────────────────────────────────
113
- # ADR-026 sentinel for ungrounded scoring fields
114
- # ──────────────────────────────────────────────────────────────────────────────
115
-
116
- @test "bootstrap-catalog SKILL.md uses ADR-026 sentinel for ungrounded scoring" {
117
- run grep -q "not estimated — no prior data" "$SKILL"
118
- [ "$status" -eq 0 ]
119
- }
120
-
121
- @test "bootstrap-catalog SKILL.md sets pending-review Status on auto-scaffolded entries" {
122
- run grep -q "Active (auto-scaffolded — pending review)" "$SKILL"
123
- [ "$status" -eq 0 ]
124
- }
125
-
126
- # ──────────────────────────────────────────────────────────────────────────────
127
- # Idempotency contract
128
- # ──────────────────────────────────────────────────────────────────────────────
129
-
130
- @test "bootstrap-catalog SKILL.md describes idempotency via file-existence per slug" {
131
- run grep -qE "[Ii]dempotent|file-existence|file existence test|re-run produces zero diff|safe to invoke at any time" "$SKILL"
132
- [ "$status" -eq 0 ]
133
- }
134
-
135
- @test "bootstrap-catalog SKILL.md handles slug collision via Source Evidence append" {
136
- run grep -qE "[Mm]atch exists.*append|slug collision|append.*Source Evidence" "$SKILL"
137
- [ "$status" -eq 0 ]
138
- }
139
-
140
- # ──────────────────────────────────────────────────────────────────────────────
141
- # Empty corpus / no reports handling
142
- # ──────────────────────────────────────────────────────────────────────────────
143
-
144
- @test "bootstrap-catalog SKILL.md exits cleanly when .risk-reports/ is empty" {
145
- run grep -qE "no-op|exit cleanly.*no.*reports|nothing to walk|empty.*[Bb]ootstrap.*no-op" "$SKILL"
146
- [ "$status" -eq 0 ]
147
- }
148
-
149
- # ──────────────────────────────────────────────────────────────────────────────
150
- # Commit per ADR-014 single-commit grain
151
- # ──────────────────────────────────────────────────────────────────────────────
152
-
153
- @test "bootstrap-catalog SKILL.md cites ADR-014 single-commit grain" {
154
- run grep -q "ADR-014" "$SKILL"
155
- [ "$status" -eq 0 ]
156
- }
157
-
158
- # ──────────────────────────────────────────────────────────────────────────────
159
- # Pure-scorer carve-out: this skill needs Write tool (it's the writer)
160
- # ──────────────────────────────────────────────────────────────────────────────
161
-
162
- @test "bootstrap-catalog SKILL.md is the orchestrator-side write surface (per Verdict G)" {
163
- # ADR-059 verdict G: bootstrap-catalog skill needs Write tool; pipeline agent
164
- # stays Read+Glob only (pure-scorer contract preserved). This skill's Write
165
- # grant is the legitimate orchestrator-side write surface.
166
- run grep -qE "needs Write tool|orchestrator-side|on-demand surface|Verdict G" "$SKILL"
167
- [ "$status" -eq 0 ]
168
- }