bigpowers 2.62.0 → 2.64.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/.pi/package.json +1 -1
  2. package/.pi/prompts/build-epic.md +2 -1
  3. package/.pi/prompts/develop-tdd.md +4 -0
  4. package/.pi/prompts/investigate-bug.md +5 -0
  5. package/.pi/prompts/plan-refactor.md +5 -0
  6. package/.pi/prompts/run-benchmark.md +67 -48
  7. package/.pi/prompts/security-review.md +4 -0
  8. package/.pi/prompts/seed-conventions.md +33 -0
  9. package/.pi/prompts/setup-environment.md +15 -0
  10. package/.pi/prompts/wire-observability.md +4 -0
  11. package/.pi/skills/build-epic/SKILL.md +2 -1
  12. package/.pi/skills/develop-tdd/SKILL.md +4 -0
  13. package/.pi/skills/investigate-bug/SKILL.md +5 -0
  14. package/.pi/skills/plan-refactor/SKILL.md +5 -0
  15. package/.pi/skills/run-benchmark/SKILL.md +67 -48
  16. package/.pi/skills/security-review/SKILL.md +4 -0
  17. package/.pi/skills/seed-conventions/SKILL.md +33 -0
  18. package/.pi/skills/setup-environment/SKILL.md +15 -0
  19. package/.pi/skills/wire-observability/SKILL.md +4 -0
  20. package/CHANGELOG.md +24 -0
  21. package/CLAUDE.md +4 -0
  22. package/CONVENTIONS.md +10 -0
  23. package/README.md +6 -0
  24. package/SKILL-INDEX.md +1 -1
  25. package/package.json +5 -2
  26. package/scripts/build-receipts.sh +107 -0
  27. package/scripts/build-skill-graph.sh +226 -0
  28. package/scripts/build-skill-index.sh +10 -20
  29. package/scripts/generate-skill-index.sh +2 -4
  30. package/scripts/record-cycle-time.sh +6 -4
  31. package/scripts/regenerate-lockfile.sh +10 -38
  32. package/scripts/sync-skills.sh +106 -79
  33. package/scripts/validate-okf.sh +28 -0
  34. package/scripts/verify-install.sh +60 -0
  35. package/skills/build-epic/SKILL.md +2 -1
  36. package/skills/develop-tdd/SKILL.md +4 -0
  37. package/skills/investigate-bug/SKILL.md +5 -0
  38. package/skills/plan-refactor/SKILL.md +5 -0
  39. package/skills/run-benchmark/SKILL.md +67 -49
  40. package/skills/security-review/SKILL.md +4 -0
  41. package/skills/seed-conventions/REFERENCE.md +31 -0
  42. package/skills/seed-conventions/SKILL.md +2 -0
  43. package/skills/setup-environment/SKILL.md +15 -0
  44. package/skills/wire-observability/SKILL.md +4 -0
  45. package/skills-lock.json +84 -84
package/.pi/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "bigpowers",
3
- "version": "2.62.0",
3
+ "version": "2.64.0",
4
4
  "description": "74 skills — 73 agent skills synthesizing 17 years of software engineering discipline into a prescriptive methodology for solo developers",
5
5
  "keywords": [
6
6
  "pi-package"
@@ -33,7 +33,8 @@ Orchestrates the **build** flow for a single epic: survey → plan tasks → kic
33
33
  2. **Step 0 — Threat Model:** Run `security-review` against the epic's scope (read from the epic capsule). Output `specs/security/epics/<epic-id>/THREAT_MODEL.md` with surface area, vulnerability categories, risk level, and mitigation guidance.
34
34
  3. **Assess Impact (Step 2):** Before writing tasks, run `assess-impact --lightweight` on the proposed change. If the risk score exceeds 7, gate — require a `grill-me` session. Write the impact report to `specs/IMPACT-<epic>-<story>.md`. For net-new code with no existing dependents, skip.
35
35
  4. **BCP Tracking (Step 2):** After `plan-work` completes, read the `bcps:` count (Business Complexity Points story size) from the epic capsule and carry it into `state.yaml` as `epic_cycle.story_bcps = N`.
36
- 5. If `epic_cycle.step` missing, set to `1`.
36
+ 5. **BCP Plus (Step 2, optional):** When the epic capsule carries a `bcp_plus_breakdown` (13-dimension sizing), also carry it into `state.yaml` as `epic_cycle.bcp_plus`. The breakdown maps each of the 13 dimensions to an integer count. NFR dimensions (11–13) that are N/A per the NFR Gate may be set to 0.
37
+ 6. If `epic_cycle.step` missing, set to `1`.
37
38
  6. Run **only the current step** (resume mode) unless user asked for full auto-run.
38
39
  7. After step verify passes, increment `epic_cycle.step` in `state.yaml` (or `bash scripts/bp-yaml-set.sh` if available).
39
40
  8. On story complete, set `execution-status.yaml` story key to `done`; run `bash scripts/sync-status-from-epics.sh`.
@@ -101,6 +101,10 @@ If this cycle modified files in `.github/workflows/`, run the CI dry-run procedu
101
101
  Gate: READY -> next: verify-work
102
102
  Writes: state.yaml handoff.next_skill = verify-work
103
103
 
104
+ ## BCP Plus Integration
105
+
106
+ At story completion, if the story was sized with BCP Plus (13-dimension breakdown), log the `bcp_plus.total` alongside the standard `bcps:` count in the story's tasks.yaml. The breakdown is available from the epic capsule's `bcp_plus_breakdown` field. See `docs/references/bcp-plus.md` for the full methodology and NFR Gate pattern.
107
+
104
108
  ---
105
109
 
106
110
  # Develop TDD — Reference
@@ -117,3 +117,8 @@ A numbered list of RED-GREEN cycles:
117
117
  </diagnosis-template>
118
118
 
119
119
  After writing the bug file, print a one-line summary of the root cause and suggest running `kickoff-branch` next to create a fix branch.
120
+
121
+ ## References
122
+
123
+ - [Feathers' Seams & Characterization Tests](../../docs/references/feathers.md) — seam types, characterization tests, and the legacy code change algorithm
124
+ - [Fowler's Code Smells](../../docs/references/fowler.md) — identifying structural problems by smell before investigating
@@ -76,4 +76,9 @@ Any further notes about the refactor.
76
76
 
77
77
  After writing `specs/REFACTOR_LATEST.md`, suggest running `kickoff-branch` next to create a refactor branch.
78
78
 
79
+ ## References
80
+
81
+ - [Fowler's Refactoring Catalog](../../docs/references/fowler.md) — canonical refactoring vocabulary and code-smell taxonomy
82
+ - [Beck's Tidy First?](../../docs/references/kent-beck.md) — structural change before behavioral change
83
+
79
84
  ## Verify
@@ -1,5 +1,5 @@
1
1
  ---
2
- description: Run skill quality benchmarks from specs/benchmarks/ definitions and write pass@k reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions.
2
+ description: Run skill quality benchmarks from specs/benchmarks/ definitions — N-run with/without-skill delta grading, train/validation split, pass@k + benchmark.json reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions.
3
3
  ---
4
4
 
5
5
 
@@ -7,59 +7,78 @@ description: Run skill quality benchmarks from specs/benchmarks/ definitions and
7
7
 
8
8
  > **HARD GATE** — Do NOT use benchmark scores to declare a skill "good" or "bad" in isolation. Benchmarks measure relative quality vs. a baseline — they catch regressions, they do not certify correctness.
9
9
 
10
- Reads benchmark definitions from `specs/benchmarks/`, executes each scenario's grader, and writes a structured `pass@k` report that `evolve-skill` consumes.
10
+ Reads benchmark definitions from `specs/benchmarks/`, executes each scenario's grader with and without the skill loaded, and writes a structured `pass@k` report with delta grading that `evolve-skill` consumes.
11
11
 
12
- ## Usage
12
+ ## With/Without-Skill Delta Grading
13
13
 
14
- ```bash
15
- # Benchmark a single skill
16
- run-benchmark <skill-name>
14
+ Every scenario runs N times (default 3) in two modes: **with** the skill loaded and **without** (bare agent with only CLAUDE.md). The delta `Δ = pass@k_with − pass@k_without` isolates the skill's causal contribution. A negative delta is a regression flag.
17
15
 
18
- # Benchmark all skills with definitions
19
- run-benchmark --all
16
+ ## Train/Validation Split
20
17
 
21
- # Pin current results as baseline
22
- run-benchmark <skill-name> --baseline
23
- ```
18
+ Benchmark definitions partition scenarios into two sets:
24
19
 
25
- ## Process
20
+ | Set | Tag | Purpose |
21
+ |-----|-----|---------|
22
+ | **Train** | `split: train` | Development scenarios — used while iterating. Hitting 100% on train is expected. |
23
+ | **Validation** | `split: validation` | Held-out scenarios — the real quality signal. Overfitting train while validation stagnates is a design smell. |
24
+
25
+ `pass@k` is reported separately for train and validation. Validation score is authoritative; train score is iteration guidance only.
26
26
 
27
- 1. **Locate definition** — Read `specs/benchmarks/<skill>.yaml`. If absent, report: `"No benchmark definition found for <skill>. Create specs/benchmarks/<skill>.yaml first."` and stop.
28
-
29
- 2. **Run each scenario** — For each scenario in `scenarios[]`:
30
- - **Code grader:** Run `grader.command` in repo root via `bash -c`. Exit 0 → PASS. Non-zero → FAIL. Timeout: 15 seconds.
31
- - **Rubric grader:** Present each criterion to the agent as a yes/no question about the scenario output. ≥ 80% yes → PASS, else FAIL.
32
-
33
- 3. **Calculate pass@k** — `pass@k = sum(weight of PASS scenarios) / sum(all weights)`. Round to 2 decimal places.
34
-
35
- 4. **Write report** to `specs/benchmarks/reports/BENCHMARK-<skill>-<YYYY-MM-DD>.yaml`:
36
-
37
- ```yaml
38
- skill: survey-context
39
- run_date: "2026-06-22"
40
- pass_at_k: 0.83
41
- total_scenarios: 3
42
- passed: 2
43
- failed: 1
44
- scenarios:
45
- - id: s01
46
- name: "detects active epic from state.yaml"
47
- result: PASS
48
- weight: 1.0
49
- - id: s02
50
- name: "reads release-plan.yaml and reports next epic"
51
- result: PASS
52
- weight: 1.0
53
- - id: s03
54
- name: "handles missing state.yaml gracefully"
55
- result: FAIL
56
- weight: 0.5
57
- failure_note: "crashed instead of suggesting state.yaml creation"
27
+ ## Usage
28
+
29
+ ```bash
30
+ run-benchmark <skill-name> # benchmark single skill
31
+ run-benchmark --all # benchmark all with definitions
32
+ run-benchmark <skill-name> --baseline # pin results as baseline
58
33
  ```
59
34
 
60
- 5. **Baseline mode** (`--baseline`) — Copy the report to `specs/benchmarks/reports/BASELINE-<skill>.yaml`. This is the reference point for regression checks in `evolve-skill`.
35
+ ## Process
61
36
 
62
- 6. **Compare to baseline** — If a `BASELINE-<skill>.yaml` exists, compare `pass_at_k`. Report:
63
- - `IMPROVED: 0.67 → 0.83`
64
- - `REGRESSION: 0.83 → 0.67 — do NOT ship this change`
65
- - `STABLE: 0.83 = 0.83`
37
+ 1. **Locate definition** — Read `specs/benchmarks/<skill>.yaml`. If absent, stop with message.
38
+
39
+ 2. **Partition scenarios** — Split by `split` field (`train` → iteration, `validation` → authoritative, default: `validation`).
40
+
41
+ 3. **Run each scenario (N-run delta)** — For each scenario, run grader N times (default 3, configurable via `runs:`):
42
+ - **Without skill:** Agent with only CLAUDE.md/CONVENTIONS.md
43
+ - **With skill:** Agent with the skill under test active
44
+ - Code grader: `bash -c <command>`, exit 0 → PASS. Timeout: 15s.
45
+ - Rubric grader: yes/no per criterion, ≥ 80% yes → PASS.
46
+ - Record: `{scenario_id: {with: [P/F,...], without: [P/F,...]}}`
47
+
48
+ 4. **Calculate scores** — Per split (train, validation) and mode (with, without):
49
+ - `pass@k = sum(weight × pass_rate) / sum(weights)` where `pass_rate = passes/runs`
50
+ - `Δ = pass@k_with − pass@k_without` — causal contribution
51
+ - Round to 2 decimal places
52
+
53
+ 5. **Write benchmark.json** to `specs/benchmarks/reports/benchmark-<skill>.json`:
54
+ ```json
55
+ {"skill":"survey-context","run_date":"2026-06-22","runs_per_scenario":3,"train":{"with_skill":0.92,"without_skill":0.67,"delta":0.25,"scenarios":["s01","s02"]},"validation":{"with_skill":0.83,"without_skill":0.60,"delta":0.23,"scenarios":["s03","s04","s05"]}}
56
+ ```
57
+
58
+ 6. **Write YAML report** to `specs/benchmarks/reports/BENCHMARK-<skill>-<YYYY-MM-DD>.yaml`:
59
+ ```yaml
60
+ skill: survey-context
61
+ run_date: "2026-06-22"
62
+ runs_per_scenario: 3
63
+ train:
64
+ pass_at_k_with: 0.92
65
+ pass_at_k_without: 0.67
66
+ delta: 0.25
67
+ validation:
68
+ pass_at_k_with: 0.83
69
+ pass_at_k_without: 0.60
70
+ delta: 0.23
71
+ scenarios:
72
+ - id: s01
73
+ split: train
74
+ with_pass_rate: 1.0
75
+ without_pass_rate: 0.67
76
+ delta: 0.33
77
+ weight: 1.0
78
+ ```
79
+
80
+ 7. **Baseline** (`--baseline`) — Copy to `BASELINE-<skill>.yaml` + `baseline-<skill>.json`.
81
+
82
+ 8. **Compare to baseline** — `IMPROVED: Δ 0.17 → 0.25` / `REGRESSION: Δ 0.25 → 0.17 — do NOT ship` / `STABLE`.
83
+
84
+ 9. **Delta threshold gate** — Validation Δ < 0.0 blocks release. Δ < 0.05 warns (marginal). Min meaningful threshold: 0.05.
@@ -23,6 +23,10 @@ description: > AI-powered security analysis of code changes — traces data flow
23
23
 
24
24
  Covered: SQLi, XSS, SSRF, command injection, auth bypass, unsafe deserialization, path traversal, IDOR, crypto flaws, secrets exposure, template injection, NoSQLi
25
25
 
26
+ ## BCP Plus Integration
27
+
28
+ This skill maps to **BCP Plus dimension 12 (Security & Compliance)**. When BCP Plus sizing is active, the threat model categories above correspond to sub-elements within dimension 12. The NFR Gate rule applies: standard-expectation items (e.g., "use HTTPS", "hash passwords") score 0 with a one-line rationale; only above-standard security requirements contribute to the dimension 12 count. See `docs/references/bcp-plus.md` for the full 13-dimension framework and NFR Gate pattern.
29
+
26
30
  ## Integration points
27
31
 
28
32
  | Skill | Touchpoint |
@@ -28,6 +28,8 @@ Ask the user these questions (one at a time, wait for each answer):
28
28
  5. **Conventions** — "Any naming, file organization, or patterns all agents must follow?"
29
29
  6. **Never-do list** — "What are the hard stops? Things an agent must never touch?"
30
30
  7. **Defensive code categories** — "Which apply? (Rate limit / Retry / Circuit breaker / Timeout / Graceful degradation)"
31
+ 8. **Local tool wiring (optional)** — "Wire bigpowers skills for project-local tools that global install can't reach? (Cursor + OpenCode only)" If yes, generate the local wiring artifacts described in [REFERENCE.md](REFERENCE.md) §Local tool wiring. If no, skip — the standard seed output is unchanged.
32
+ 8. **Local tool wiring (optional)** — "Wire bigpowers skills for project-local tools that global install can't reach? (Cursor + OpenCode only)" If yes, generate the local wiring artifacts described in [REFERENCE.md](REFERENCE.md) §Local tool wiring. If no, skip — the standard seed output is unchanged.
31
33
 
32
34
  ## Generate files
33
35
 
@@ -130,3 +132,34 @@ Use the standard bigpowers CONVENTIONS.md as the base. Fill in the project-speci
130
132
  ## Stack profile fragments
131
133
 
132
134
  If the user selected a stack profile, merge the matching `profiles/<name>.md` fragment into the generated `CONVENTIONS.md` under a `## Stack Conventions` section. Profiles supply language-specific commands, architecture patterns, and never-do additions.
135
+
136
+ ## Local tool wiring (optional interview step 8)
137
+
138
+ Offered after the standard interview. Covers the two tools that global install (`scripts/install.sh`) structurally cannot reach because they read project-root config, not global paths.
139
+
140
+ ### Cursor — project-local `.cursor/rules` symlink
141
+
142
+ ```bash
143
+ # From the project root:
144
+ ln -sfn <bigpowers-install-path>/.cursor/rules .cursor/rules
145
+ ```
146
+
147
+ Cursor reads `.cursor/rules/` from the project root. This symlink gives every project access to bigpowers skills as Cursor rules without duplicating the files. Run once per project.
148
+
149
+ ### OpenCode — project-local `opencode.json` + `AGENTS.md`
150
+
151
+ `opencode.json` (project root):
152
+ ```json
153
+ {
154
+ "$schema": "https://opencode.ai/config.json",
155
+ "instructions": [".cursor/rules/*.mdc", "AGENTS.md"]
156
+ }
157
+ ```
158
+
159
+ OpenCode reads `opencode.json` from the project root, NOT from a global path. The `instructions` array points to the local `.cursor/rules` symlink (from the Cursor step above) and the project's `AGENTS.md`. Both must exist in the project for OpenCode to see bigpowers skills.
160
+
161
+ `AGENTS.md` is already generated by the standard interview (step 2 of Generate Files). When local tool wiring is opted in, ensure `AGENTS.md` includes the standard agent-config template header `# [Project Name] — OpenCode`.
162
+
163
+ ### When to offer
164
+
165
+ Only offer local tool wiring when the user's project will be opened in Cursor or OpenCode. These tools are project-root scoped by design — no global installer can solve them. Global install (`install.sh`) already handles Claude Code, Gemini CLI, and pi globally. Do not offer for tools that read global config.
@@ -18,6 +18,21 @@ Idempotent prep so BUILD phase commands succeed on first run.
18
18
  5. Run smoke: lint + one fast test or `--version` on key tools.
19
19
  6. Record versions in `specs/state.yaml` under Environment.
20
20
 
21
+ ## BCP Plus Counter (optional)
22
+
23
+ The `big-counter` tool is an optional dependency for BCP Plus 13-dimension story sizing:
24
+
25
+ ```bash
26
+ # Install from PyPI (recommended)
27
+ pip install big-counter
28
+
29
+ # Or from npm
30
+ npm install -g big-counter
31
+ ```
32
+
33
+ Verify the install: `big-counter --version`
34
+ Skip if BCP Plus sizing is not needed for this project.
35
+
21
36
  ## Verify
22
37
 
23
38
  → verify: `test -f CLAUDE.md && grep -q 'Test' CLAUDE.md && echo "OK: CLAUDE.md has test table" || echo "MISSING: test commands"`
@@ -90,3 +90,7 @@ Place setup scripts in `scripts/setup.sh` (or language-appropriate equivalent).
90
90
  - [ ] Run `scripts/setup.sh` twice — second run should produce no errors
91
91
  - [ ] Health check command returns success
92
92
  - [ ] No sensitive data in log output
93
+
94
+ ## BCP Plus Integration
95
+
96
+ This skill maps to **BCP Plus dimension 11 (Quality Attributes)** — specifically performance, reliability, and availability instrumentation. Structured logging, health checks, and idempotent setup scripts are observable manifestations of quality-attribute requirements. The NFR Gate rule applies: standard-expectation items (e.g., "the app should log errors") score 0 with a one-line rationale; only above-standard quality requirements (e.g., p99 latency tracking, SLO-based alerting) contribute to the dimension 11 count. See `docs/references/bcp-plus.md` for the full 13-dimension framework and NFR Gate pattern.
@@ -35,7 +35,8 @@ Orchestrates the **build** flow for a single epic: survey → plan tasks → kic
35
35
  2. **Step 0 — Threat Model:** Run `security-review` against the epic's scope (read from the epic capsule). Output `specs/security/epics/<epic-id>/THREAT_MODEL.md` with surface area, vulnerability categories, risk level, and mitigation guidance.
36
36
  3. **Assess Impact (Step 2):** Before writing tasks, run `assess-impact --lightweight` on the proposed change. If the risk score exceeds 7, gate — require a `grill-me` session. Write the impact report to `specs/IMPACT-<epic>-<story>.md`. For net-new code with no existing dependents, skip.
37
37
  4. **BCP Tracking (Step 2):** After `plan-work` completes, read the `bcps:` count (Business Complexity Points story size) from the epic capsule and carry it into `state.yaml` as `epic_cycle.story_bcps = N`.
38
- 5. If `epic_cycle.step` missing, set to `1`.
38
+ 5. **BCP Plus (Step 2, optional):** When the epic capsule carries a `bcp_plus_breakdown` (13-dimension sizing), also carry it into `state.yaml` as `epic_cycle.bcp_plus`. The breakdown maps each of the 13 dimensions to an integer count. NFR dimensions (11–13) that are N/A per the NFR Gate may be set to 0.
39
+ 6. If `epic_cycle.step` missing, set to `1`.
39
40
  6. Run **only the current step** (resume mode) unless user asked for full auto-run.
40
41
  7. After step verify passes, increment `epic_cycle.step` in `state.yaml` (or `bash scripts/bp-yaml-set.sh` if available).
41
42
  8. On story complete, set `execution-status.yaml` story key to `done`; run `bash scripts/sync-status-from-epics.sh`.
@@ -103,6 +103,10 @@ If this cycle modified files in `.github/workflows/`, run the CI dry-run procedu
103
103
  Gate: READY -> next: verify-work
104
104
  Writes: state.yaml handoff.next_skill = verify-work
105
105
 
106
+ ## BCP Plus Integration
107
+
108
+ At story completion, if the story was sized with BCP Plus (13-dimension breakdown), log the `bcp_plus.total` alongside the standard `bcps:` count in the story's tasks.yaml. The breakdown is available from the epic capsule's `bcp_plus_breakdown` field. See `docs/references/bcp-plus.md` for the full methodology and NFR Gate pattern.
109
+
106
110
  ---
107
111
 
108
112
  # Develop TDD — Reference
@@ -119,3 +119,8 @@ A numbered list of RED-GREEN cycles:
119
119
  </diagnosis-template>
120
120
 
121
121
  After writing the bug file, print a one-line summary of the root cause and suggest running `kickoff-branch` next to create a fix branch.
122
+
123
+ ## References
124
+
125
+ - [Feathers' Seams & Characterization Tests](../../docs/references/feathers.md) — seam types, characterization tests, and the legacy code change algorithm
126
+ - [Fowler's Code Smells](../../docs/references/fowler.md) — identifying structural problems by smell before investigating
@@ -78,4 +78,9 @@ Any further notes about the refactor.
78
78
 
79
79
  After writing `specs/REFACTOR_LATEST.md`, suggest running `kickoff-branch` next to create a refactor branch.
80
80
 
81
+ ## References
82
+
83
+ - [Fowler's Refactoring Catalog](../../docs/references/fowler.md) — canonical refactoring vocabulary and code-smell taxonomy
84
+ - [Beck's Tidy First?](../../docs/references/kent-beck.md) — structural change before behavioral change
85
+
81
86
  ## Verify
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: run-benchmark
3
- description: "Run skill quality benchmarks from specs/benchmarks/ definitions and write pass@k reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions."
3
+ description: "Run skill quality benchmarks from specs/benchmarks/ definitions — N-run with/without-skill delta grading, train/validation split, pass@k + benchmark.json reports. Use before and after evolve-skill to prove quality changes are improvements, not regressions."
4
4
  model: haiku
5
5
  ---
6
6
 
@@ -9,59 +9,78 @@ model: haiku
9
9
 
10
10
  > **HARD GATE** — Do NOT use benchmark scores to declare a skill "good" or "bad" in isolation. Benchmarks measure relative quality vs. a baseline — they catch regressions, they do not certify correctness.
11
11
 
12
- Reads benchmark definitions from `specs/benchmarks/`, executes each scenario's grader, and writes a structured `pass@k` report that `evolve-skill` consumes.
12
+ Reads benchmark definitions from `specs/benchmarks/`, executes each scenario's grader with and without the skill loaded, and writes a structured `pass@k` report with delta grading that `evolve-skill` consumes.
13
13
 
14
- ## Usage
14
+ ## With/Without-Skill Delta Grading
15
15
 
16
- ```bash
17
- # Benchmark a single skill
18
- run-benchmark <skill-name>
16
+ Every scenario runs N times (default 3) in two modes: **with** the skill loaded and **without** (bare agent with only CLAUDE.md). The delta `Δ = pass@k_with − pass@k_without` isolates the skill's causal contribution. A negative delta is a regression flag.
19
17
 
20
- # Benchmark all skills with definitions
21
- run-benchmark --all
18
+ ## Train/Validation Split
22
19
 
23
- # Pin current results as baseline
24
- run-benchmark <skill-name> --baseline
25
- ```
20
+ Benchmark definitions partition scenarios into two sets:
26
21
 
27
- ## Process
22
+ | Set | Tag | Purpose |
23
+ |-----|-----|---------|
24
+ | **Train** | `split: train` | Development scenarios — used while iterating. Hitting 100% on train is expected. |
25
+ | **Validation** | `split: validation` | Held-out scenarios — the real quality signal. Overfitting train while validation stagnates is a design smell. |
26
+
27
+ `pass@k` is reported separately for train and validation. Validation score is authoritative; train score is iteration guidance only.
28
28
 
29
- 1. **Locate definition** — Read `specs/benchmarks/<skill>.yaml`. If absent, report: `"No benchmark definition found for <skill>. Create specs/benchmarks/<skill>.yaml first."` and stop.
30
-
31
- 2. **Run each scenario** — For each scenario in `scenarios[]`:
32
- - **Code grader:** Run `grader.command` in repo root via `bash -c`. Exit 0 → PASS. Non-zero → FAIL. Timeout: 15 seconds.
33
- - **Rubric grader:** Present each criterion to the agent as a yes/no question about the scenario output. ≥ 80% yes → PASS, else FAIL.
34
-
35
- 3. **Calculate pass@k** — `pass@k = sum(weight of PASS scenarios) / sum(all weights)`. Round to 2 decimal places.
36
-
37
- 4. **Write report** to `specs/benchmarks/reports/BENCHMARK-<skill>-<YYYY-MM-DD>.yaml`:
38
-
39
- ```yaml
40
- skill: survey-context
41
- run_date: "2026-06-22"
42
- pass_at_k: 0.83
43
- total_scenarios: 3
44
- passed: 2
45
- failed: 1
46
- scenarios:
47
- - id: s01
48
- name: "detects active epic from state.yaml"
49
- result: PASS
50
- weight: 1.0
51
- - id: s02
52
- name: "reads release-plan.yaml and reports next epic"
53
- result: PASS
54
- weight: 1.0
55
- - id: s03
56
- name: "handles missing state.yaml gracefully"
57
- result: FAIL
58
- weight: 0.5
59
- failure_note: "crashed instead of suggesting state.yaml creation"
29
+ ## Usage
30
+
31
+ ```bash
32
+ run-benchmark <skill-name> # benchmark single skill
33
+ run-benchmark --all # benchmark all with definitions
34
+ run-benchmark <skill-name> --baseline # pin results as baseline
60
35
  ```
61
36
 
62
- 5. **Baseline mode** (`--baseline`) — Copy the report to `specs/benchmarks/reports/BASELINE-<skill>.yaml`. This is the reference point for regression checks in `evolve-skill`.
37
+ ## Process
63
38
 
64
- 6. **Compare to baseline** — If a `BASELINE-<skill>.yaml` exists, compare `pass_at_k`. Report:
65
- - `IMPROVED: 0.67 → 0.83`
66
- - `REGRESSION: 0.83 → 0.67 — do NOT ship this change`
67
- - `STABLE: 0.83 = 0.83`
39
+ 1. **Locate definition** — Read `specs/benchmarks/<skill>.yaml`. If absent, stop with message.
40
+
41
+ 2. **Partition scenarios** — Split by `split` field (`train` → iteration, `validation` → authoritative, default: `validation`).
42
+
43
+ 3. **Run each scenario (N-run delta)** — For each scenario, run grader N times (default 3, configurable via `runs:`):
44
+ - **Without skill:** Agent with only CLAUDE.md/CONVENTIONS.md
45
+ - **With skill:** Agent with the skill under test active
46
+ - Code grader: `bash -c <command>`, exit 0 → PASS. Timeout: 15s.
47
+ - Rubric grader: yes/no per criterion, ≥ 80% yes → PASS.
48
+ - Record: `{scenario_id: {with: [P/F,...], without: [P/F,...]}}`
49
+
50
+ 4. **Calculate scores** — Per split (train, validation) and mode (with, without):
51
+ - `pass@k = sum(weight × pass_rate) / sum(weights)` where `pass_rate = passes/runs`
52
+ - `Δ = pass@k_with − pass@k_without` — causal contribution
53
+ - Round to 2 decimal places
54
+
55
+ 5. **Write benchmark.json** to `specs/benchmarks/reports/benchmark-<skill>.json`:
56
+ ```json
57
+ {"skill":"survey-context","run_date":"2026-06-22","runs_per_scenario":3,"train":{"with_skill":0.92,"without_skill":0.67,"delta":0.25,"scenarios":["s01","s02"]},"validation":{"with_skill":0.83,"without_skill":0.60,"delta":0.23,"scenarios":["s03","s04","s05"]}}
58
+ ```
59
+
60
+ 6. **Write YAML report** to `specs/benchmarks/reports/BENCHMARK-<skill>-<YYYY-MM-DD>.yaml`:
61
+ ```yaml
62
+ skill: survey-context
63
+ run_date: "2026-06-22"
64
+ runs_per_scenario: 3
65
+ train:
66
+ pass_at_k_with: 0.92
67
+ pass_at_k_without: 0.67
68
+ delta: 0.25
69
+ validation:
70
+ pass_at_k_with: 0.83
71
+ pass_at_k_without: 0.60
72
+ delta: 0.23
73
+ scenarios:
74
+ - id: s01
75
+ split: train
76
+ with_pass_rate: 1.0
77
+ without_pass_rate: 0.67
78
+ delta: 0.33
79
+ weight: 1.0
80
+ ```
81
+
82
+ 7. **Baseline** (`--baseline`) — Copy to `BASELINE-<skill>.yaml` + `baseline-<skill>.json`.
83
+
84
+ 8. **Compare to baseline** — `IMPROVED: Δ 0.17 → 0.25` / `REGRESSION: Δ 0.25 → 0.17 — do NOT ship` / `STABLE`.
85
+
86
+ 9. **Delta threshold gate** — Validation Δ < 0.0 blocks release. Δ < 0.05 warns (marginal). Min meaningful threshold: 0.05.
@@ -25,6 +25,10 @@ model: sonnet
25
25
 
26
26
  Covered: SQLi, XSS, SSRF, command injection, auth bypass, unsafe deserialization, path traversal, IDOR, crypto flaws, secrets exposure, template injection, NoSQLi
27
27
 
28
+ ## BCP Plus Integration
29
+
30
+ This skill maps to **BCP Plus dimension 12 (Security & Compliance)**. When BCP Plus sizing is active, the threat model categories above correspond to sub-elements within dimension 12. The NFR Gate rule applies: standard-expectation items (e.g., "use HTTPS", "hash passwords") score 0 with a one-line rationale; only above-standard security requirements contribute to the dimension 12 count. See `docs/references/bcp-plus.md` for the full 13-dimension framework and NFR Gate pattern.
31
+
28
32
  ## Integration points
29
33
 
30
34
  | Skill | Touchpoint |
@@ -30,6 +30,8 @@ Ask the user these questions (one at a time, wait for each answer):
30
30
  5. **Conventions** — "Any naming, file organization, or patterns all agents must follow?"
31
31
  6. **Never-do list** — "What are the hard stops? Things an agent must never touch?"
32
32
  7. **Defensive code categories** — "Which apply? (Rate limit / Retry / Circuit breaker / Timeout / Graceful degradation)"
33
+ 8. **Local tool wiring (optional)** — "Wire bigpowers skills for project-local tools that global install can't reach? (Cursor + OpenCode only)" If yes, generate the local wiring artifacts described in [REFERENCE.md](REFERENCE.md) §Local tool wiring. If no, skip — the standard seed output is unchanged.
34
+ 8. **Local tool wiring (optional)** — "Wire bigpowers skills for project-local tools that global install can't reach? (Cursor + OpenCode only)" If yes, generate the local wiring artifacts described in [REFERENCE.md](REFERENCE.md) §Local tool wiring. If no, skip — the standard seed output is unchanged.
33
35
 
34
36
  ## Generate files
35
37
 
@@ -132,3 +134,34 @@ Use the standard bigpowers CONVENTIONS.md as the base. Fill in the project-speci
132
134
  ## Stack profile fragments
133
135
 
134
136
  If the user selected a stack profile, merge the matching `profiles/<name>.md` fragment into the generated `CONVENTIONS.md` under a `## Stack Conventions` section. Profiles supply language-specific commands, architecture patterns, and never-do additions.
137
+
138
+ ## Local tool wiring (optional interview step 8)
139
+
140
+ Offered after the standard interview. Covers the two tools that global install (`scripts/install.sh`) structurally cannot reach because they read project-root config, not global paths.
141
+
142
+ ### Cursor — project-local `.cursor/rules` symlink
143
+
144
+ ```bash
145
+ # From the project root:
146
+ ln -sfn <bigpowers-install-path>/.cursor/rules .cursor/rules
147
+ ```
148
+
149
+ Cursor reads `.cursor/rules/` from the project root. This symlink gives every project access to bigpowers skills as Cursor rules without duplicating the files. Run once per project.
150
+
151
+ ### OpenCode — project-local `opencode.json` + `AGENTS.md`
152
+
153
+ `opencode.json` (project root):
154
+ ```json
155
+ {
156
+ "$schema": "https://opencode.ai/config.json",
157
+ "instructions": [".cursor/rules/*.mdc", "AGENTS.md"]
158
+ }
159
+ ```
160
+
161
+ OpenCode reads `opencode.json` from the project root, NOT from a global path. The `instructions` array points to the local `.cursor/rules` symlink (from the Cursor step above) and the project's `AGENTS.md`. Both must exist in the project for OpenCode to see bigpowers skills.
162
+
163
+ `AGENTS.md` is already generated by the standard interview (step 2 of Generate Files). When local tool wiring is opted in, ensure `AGENTS.md` includes the standard agent-config template header `# [Project Name] — OpenCode`.
164
+
165
+ ### When to offer
166
+
167
+ Only offer local tool wiring when the user's project will be opened in Cursor or OpenCode. These tools are project-root scoped by design — no global installer can solve them. Global install (`install.sh`) already handles Claude Code, Gemini CLI, and pi globally. Do not offer for tools that read global config.
@@ -20,6 +20,21 @@ Idempotent prep so BUILD phase commands succeed on first run.
20
20
  5. Run smoke: lint + one fast test or `--version` on key tools.
21
21
  6. Record versions in `specs/state.yaml` under Environment.
22
22
 
23
+ ## BCP Plus Counter (optional)
24
+
25
+ The `big-counter` tool is an optional dependency for BCP Plus 13-dimension story sizing:
26
+
27
+ ```bash
28
+ # Install from PyPI (recommended)
29
+ pip install big-counter
30
+
31
+ # Or from npm
32
+ npm install -g big-counter
33
+ ```
34
+
35
+ Verify the install: `big-counter --version`
36
+ Skip if BCP Plus sizing is not needed for this project.
37
+
23
38
  ## Verify
24
39
 
25
40
  → verify: `test -f CLAUDE.md && grep -q 'Test' CLAUDE.md && echo "OK: CLAUDE.md has test table" || echo "MISSING: test commands"`
@@ -92,3 +92,7 @@ Place setup scripts in `scripts/setup.sh` (or language-appropriate equivalent).
92
92
  - [ ] Run `scripts/setup.sh` twice — second run should produce no errors
93
93
  - [ ] Health check command returns success
94
94
  - [ ] No sensitive data in log output
95
+
96
+ ## BCP Plus Integration
97
+
98
+ This skill maps to **BCP Plus dimension 11 (Quality Attributes)** — specifically performance, reliability, and availability instrumentation. Structured logging, health checks, and idempotent setup scripts are observable manifestations of quality-attribute requirements. The NFR Gate rule applies: standard-expectation items (e.g., "the app should log errors") score 0 with a one-line rationale; only above-standard quality requirements (e.g., p99 latency tracking, SLO-based alerting) contribute to the dimension 11 count. See `docs/references/bcp-plus.md` for the full 13-dimension framework and NFR Gate pattern.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,27 @@
1
+ # [2.64.0](https://github.com/danielvm-git/bigpowers/compare/v2.63.0...v2.64.0) (2026-07-05)
2
+
3
+
4
+ ### Features
5
+
6
+ * **e33:** Docs Website — fourth generated artifact target ([ff7f3ec](https://github.com/danielvm-git/bigpowers/commit/ff7f3ec6eab7c4b30a9676888468af5c9c9dd196))
7
+ * **e35:** missing historical references — 10 stories, 8 new ref docs ([ffba59f](https://github.com/danielvm-git/bigpowers/commit/ffba59f3ebbe18a72385a3f325a5856a8a53da9e))
8
+ * **e37s01:** BCP Plus counter integration — setup-environment + bcp-plus reference ([f2b4bdf](https://github.com/danielvm-git/bigpowers/commit/f2b4bdf86b6256089b593207aec37480e789edb7))
9
+ * **e37s03:** NFR Gate integration — security-review and wire-observability sizing ([de47871](https://github.com/danielvm-git/bigpowers/commit/de478716f22e1b576a4b936103144aac39e921a4))
10
+ * **e37s04:** Build-epic integration — BCP Plus in story sizing workflow ([a961e6b](https://github.com/danielvm-git/bigpowers/commit/a961e6be3c7f6a984654b38c4a867423ac6c11db))
11
+ * **e43:** showcase repo — full worked example + sync-skills regeneration ([a872ca8](https://github.com/danielvm-git/bigpowers/commit/a872ca82745c6e7b983ef45f42f3251d3df0d680))
12
+ * **e43:** showcase repo — full worked example of bigpowers methodology ([e32373b](https://github.com/danielvm-git/bigpowers/commit/e32373bff09cd0bf01a06406a95fccff8ad54f47)), closes [#dependent](https://github.com/danielvm-git/bigpowers/issues/dependent)
13
+ * **e47:** cross-tool skill distribution — pi install, seed wiring, docs, verify harness ([0109c2f](https://github.com/danielvm-git/bigpowers/commit/0109c2fd87e2c2ef783e335a7e3d10e268ba5162))
14
+ * **e48s01:** benchmark schema v2 — split/runs fields, story tags ([47bc1ef](https://github.com/danielvm-git/bigpowers/commit/47bc1ef92e6c34b01dc199f22b6edf357f5f0120))
15
+ * **e48s01:** run-benchmark — delta grading + train/validation split ([d2d1fe8](https://github.com/danielvm-git/bigpowers/commit/d2d1fe84ec923f34454b7bdac40205667f18760f))
16
+ * **e48s01:** run-benchmark — with/without-skill delta grading + train/validation split ([651d0d8](https://github.com/danielvm-git/bigpowers/commit/651d0d8b2c44477fa3be08162a5e0ecde2db2258))
17
+
18
+ # [2.63.0](https://github.com/danielvm-git/bigpowers/compare/v2.62.0...v2.63.0) (2026-07-05)
19
+
20
+
21
+ ### Features
22
+
23
+ * **e47:** cross-tool skill distribution — pi install, seed wiring, docs, verify harness ([27c68b4](https://github.com/danielvm-git/bigpowers/commit/27c68b49b698e994b783abc85c3750637061664e))
24
+
1
25
  # [2.62.0](https://github.com/danielvm-git/bigpowers/compare/v2.61.4...v2.62.0) (2026-07-05)
2
26
 
3
27
 
package/CLAUDE.md CHANGED
@@ -35,6 +35,8 @@ npm run compliance && bash scripts/run-golden-suite.sh
35
35
 
36
36
  If any gate fails, fix before merging. Run `--baseline` after any intentional increase in skill count or structure.
37
37
 
38
+ **BCP Plus:** For stories sized with the 13-dimension BCP Plus methodology, confirm the `bcp_plus_breakdown` is present in the epic YAML and carried into `state.yaml` as `epic_cycle.bcp_plus`. See `docs/references/bcp-plus.md` for the NFR Gate pattern.
39
+
38
40
  ## Architecture
39
41
 
40
42
  Collection of verb-noun skills under `skills/`, each with a SKILL.md source file and supporting documentation. Runtime specs live in `specs/state.yaml`, `specs/release-plan.yaml`, and `specs/execution-status.yaml`; intent in `specs/product/`; epic shards in `specs/epics/`. The sync-skills.sh script auto-generates artifacts for Cursor (.cursor/rules) and Gemini CLI (.gemini/extensions/bigpowers/) from SKILL.md sources. All planning output goes to specs/ at the project root.
@@ -46,10 +48,12 @@ Collection of verb-noun skills under `skills/`, each with a SKILL.md source file
46
48
  - All planning/spec output goes to specs/ at project root
47
49
  - Artifacts in .cursor/rules and .gemini/ are auto-generated; edit SKILL.md, not artifacts
48
50
  - Run sync-skills.sh after any SKILL.md changes to regenerate artifacts
51
+ - Website content in website/src/content/docs/ is auto-generated by prebuild; edit repo sources, not site files
49
52
 
50
53
  ## Never
51
54
 
52
55
  - Never edit .cursor/rules or .gemini/extensions/ directly — these are generated files
56
+ - Never edit website/src/content/docs/ directly — these are generated files; website/ is the fourth generated artifact target (alongside .cursor/, .gemini/, .pi/)
53
57
  - Never create a skill without a SKILL.md file and proper verb-noun naming
54
58
  - Never push changes without running sync-skills.sh first
55
59
 
package/CONVENTIONS.md CHANGED
@@ -140,6 +140,16 @@ When planning closes, copy to `specs/product/snapshots/release-<version>/` (`rel
140
140
 
141
141
  Validate YAML layout: `bash scripts/validate-specs-yaml.sh`. Patch runtime keys: `bash scripts/bp-yaml-set.sh specs/state.yaml git.branch feat/foo`.
142
142
 
143
+ ### Generated artifact targets
144
+
145
+ Skill content is edited in `skills/*/SKILL.md` and auto-propagated to these targets by `bash scripts/sync-skills.sh`:
146
+ - `.cursor/rules/` — Cursor IDE rules
147
+ - `.gemini/extensions/bigpowers/` — Gemini CLI extensions
148
+ - `.pi/skills/` + `.pi/prompts/` — pi agent
149
+ - `website/src/content/docs/` — Docs website (Astro Starlight, deployed to GitHub Pages)
150
+
151
+ **Never edit any of these targets directly.** Edit the SKILL.md sources and run rebuild scripts (`bash scripts/sync-skills.sh`, `npm run site:build`).
152
+
143
153
  ### Legacy paths (migrate away)
144
154
 
145
155
  | Old | New |
package/README.md CHANGED
@@ -12,6 +12,12 @@ It is not a random collection of best practices. It is a chronological layer cak
12
12
 
13
13
  Published on npm: [bigpowers](https://www.npmjs.com/package/bigpowers). The skill count in the badge above is stamped automatically by `sync-skills.sh`; the canonical catalog is [`SKILL-INDEX.md`](SKILL-INDEX.md).
14
14
 
15
+ Docs: [bigpowers docs site](https://danielvm-git.github.io/bigpowers/) — searchable, Google-discoverable reference for all skills, guides, and ADRs.
16
+
17
+ **This methodology publishes its own evidence — see the [live receipts page](https://danielvm-git.github.io/bigpowers/receipts/).**
18
+
19
+ **See it working:** [bigpowers-showcase](https://github.com/danielvm-git/bigpowers-showcase) — a real URL shortener (CLI + SQLite) built from scratch with the full spec trail committed from day one.
20
+
15
21
  ---
16
22
 
17
23
  ## 🗺 How to Read This README