bigpowers 2.49.0 → 2.50.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.pi/package.json +1 -1
- package/.pi/prompts/evolve-skill.md +7 -6
- package/.pi/skills/evolve-skill/SKILL.md +7 -6
- package/CHANGELOG.md +7 -0
- package/CLAUDE.md +11 -0
- package/CONVENTIONS.md +11 -0
- package/package.json +1 -1
- package/scripts/run-golden-suite.sh +59 -1
- package/skills/evolve-skill/SKILL.md +7 -6
- package/skills-lock.json +1 -1
package/.pi/package.json
CHANGED
|
@@ -9,19 +9,20 @@ description: Benchmark-gated skill evolution — consume bigpowers-benchmark rep
|
|
|
9
9
|
|
|
10
10
|
## Loop
|
|
11
11
|
|
|
12
|
-
1. **
|
|
12
|
+
1. **Regression gate** — Run `bash scripts/run-golden-suite.sh` to catch mechanical regressions (compliance, sync pipeline, size budget) before spending time on benchmark evals. If golden suite fails, fix regressions first — they are pre-requisites for any capability improvement.
|
|
13
|
+
2. **Establish baseline** — Run `run-benchmark <skill> --baseline`. If no definition exists at `specs/benchmarks/<skill>.yaml`, create one following `specs/benchmarks/SCHEMA.md` first. Save report path in `state.yaml`. If `specs/benchmarks/reports/BASELINE-<skill>.yaml` already exists, skip this step.
|
|
13
14
|
|
|
14
|
-
|
|
15
|
+
3. **Identify gap** — Read the baseline report (`specs/benchmarks/reports/BASELINE-<skill>.yaml`). Find scenarios with `result: FAIL` or low `pass_at_k`. This is the measurable gap.
|
|
15
16
|
|
|
16
|
-
|
|
17
|
+
4. **`plan-work`** — Write a minimal change proposal targeting the failing scenarios. Include verify commands.
|
|
17
18
|
|
|
18
|
-
|
|
19
|
+
5. **Edit** via `craft-skill` / direct SKILL.md edit; run `bash scripts/sync-skills.sh`.
|
|
19
20
|
|
|
20
|
-
|
|
21
|
+
6. **Re-run benchmark** — `run-benchmark <skill>`. Compare new `pass_at_k` against baseline.
|
|
21
22
|
- **IMPROVED or STABLE** → advance to step 6.
|
|
22
23
|
- **REGRESSION** (`new pass_at_k < baseline`) → revert the change and loop back to step 3.
|
|
23
24
|
|
|
24
|
-
|
|
25
|
+
7. **Record decision** — Write `specs/adr/NNNN-evolve-<skill>.md` with before/after `pass_at_k` scores. Update `session-state`.
|
|
25
26
|
|
|
26
27
|
## Verify
|
|
27
28
|
|
|
@@ -11,19 +11,20 @@ model: opus
|
|
|
11
11
|
|
|
12
12
|
## Loop
|
|
13
13
|
|
|
14
|
-
1. **
|
|
14
|
+
1. **Regression gate** — Run `bash scripts/run-golden-suite.sh` to catch mechanical regressions (compliance, sync pipeline, size budget) before spending time on benchmark evals. If golden suite fails, fix regressions first — they are pre-requisites for any capability improvement.
|
|
15
|
+
2. **Establish baseline** — Run `run-benchmark <skill> --baseline`. If no definition exists at `specs/benchmarks/<skill>.yaml`, create one following `specs/benchmarks/SCHEMA.md` first. Save report path in `state.yaml`. If `specs/benchmarks/reports/BASELINE-<skill>.yaml` already exists, skip this step.
|
|
15
16
|
|
|
16
|
-
|
|
17
|
+
3. **Identify gap** — Read the baseline report (`specs/benchmarks/reports/BASELINE-<skill>.yaml`). Find scenarios with `result: FAIL` or low `pass_at_k`. This is the measurable gap.
|
|
17
18
|
|
|
18
|
-
|
|
19
|
+
4. **`plan-work`** — Write a minimal change proposal targeting the failing scenarios. Include verify commands.
|
|
19
20
|
|
|
20
|
-
|
|
21
|
+
5. **Edit** via `craft-skill` / direct SKILL.md edit; run `bash scripts/sync-skills.sh`.
|
|
21
22
|
|
|
22
|
-
|
|
23
|
+
6. **Re-run benchmark** — `run-benchmark <skill>`. Compare new `pass_at_k` against baseline.
|
|
23
24
|
- **IMPROVED or STABLE** → advance to step 6.
|
|
24
25
|
- **REGRESSION** (`new pass_at_k < baseline`) → revert the change and loop back to step 3.
|
|
25
26
|
|
|
26
|
-
|
|
27
|
+
7. **Record decision** — Write `specs/adr/NNNN-evolve-<skill>.md` with before/after `pass_at_k` scores. Update `session-state`.
|
|
27
28
|
|
|
28
29
|
## Verify
|
|
29
30
|
|
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,10 @@
|
|
|
1
|
+
# [2.50.0](https://github.com/danielvm-git/bigpowers/compare/v2.49.0...v2.50.0) (2026-07-03)
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
### Features
|
|
5
|
+
|
|
6
|
+
* **e31:** size budget, pre-merge mandate, evolve-skill integration ([#46](https://github.com/danielvm-git/bigpowers/issues/46)) ([fa1dd52](https://github.com/danielvm-git/bigpowers/commit/fa1dd52276d0b6476b892f6fd6eed77eac7c5d41))
|
|
7
|
+
|
|
1
8
|
# [2.49.0](https://github.com/danielvm-git/bigpowers/compare/v2.48.0...v2.49.0) (2026-07-03)
|
|
2
9
|
|
|
3
10
|
|
package/CLAUDE.md
CHANGED
|
@@ -20,6 +20,17 @@ Stack: Markdown / Bash (documentation-based; skills integrate with Claude Code,
|
|
|
20
20
|
| Typecheck | N/A (Markdown / Bash project) |
|
|
21
21
|
| CI platform | GitHub Actions (`.github/workflows/publish.yml`, `sync-skills.yml`) |
|
|
22
22
|
| Compliance | `npm run compliance` |
|
|
23
|
+
| Golden Suite | `bash scripts/run-golden-suite.sh` |
|
|
24
|
+
|
|
25
|
+
### Pre-Merge Checklist
|
|
26
|
+
|
|
27
|
+
Before opening a PR or landing a branch, run:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
npm run compliance && bash scripts/run-golden-suite.sh
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
If any gate fails, fix before merging. Run `--baseline` after any intentional increase in skill count or structure.
|
|
23
34
|
|
|
24
35
|
## Architecture
|
|
25
36
|
|
package/CONVENTIONS.md
CHANGED
|
@@ -28,6 +28,17 @@ All changes to this repository MUST follow the [Conventional Commits 1.0.0](http
|
|
|
28
28
|
- Never call GitHub REST API directly (curl, fetch, etc.)
|
|
29
29
|
- Never create GitHub issues from automated workflows — produce local .md files in specs/ instead
|
|
30
30
|
|
|
31
|
+
### Pre-Merge Golden Gate
|
|
32
|
+
|
|
33
|
+
Before merging any branch, run the deterministic golden suite:
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
bash scripts/run-golden-suite.sh
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
This runs compliance → G-04 self-test. If any gate fails, the merge is blocked.
|
|
40
|
+
Pin a fresh baseline with `--baseline` after major structure changes.
|
|
41
|
+
|
|
31
42
|
## Agent Workflow Mandates
|
|
32
43
|
|
|
33
44
|
**AGENTS MUST NEVER BYPASS THE BIGPOWERS WORKFLOW.**
|
package/package.json
CHANGED
|
@@ -15,12 +15,14 @@ REPORT_DIR="specs/benchmarks/reports"
|
|
|
15
15
|
BASELINE_FILE="$REPORT_DIR/BASELINE-GOLDEN.yaml"
|
|
16
16
|
DRY_RUN=false
|
|
17
17
|
BASELINE_MODE=false
|
|
18
|
+
CHECK_SIZE=false
|
|
18
19
|
|
|
19
20
|
for arg in "$@"; do
|
|
20
21
|
case "$arg" in
|
|
21
22
|
--dry-run) DRY_RUN=true ;;
|
|
22
23
|
--baseline) BASELINE_MODE=true ;;
|
|
23
|
-
|
|
24
|
+
--check-size) CHECK_SIZE=true ;;
|
|
25
|
+
*) echo "Unknown flag: $arg"; echo "Usage: run-golden-suite.sh [--dry-run] [--baseline] [--check-size]"; exit 2 ;;
|
|
24
26
|
esac
|
|
25
27
|
done
|
|
26
28
|
|
|
@@ -180,6 +182,62 @@ if $BASELINE_MODE; then
|
|
|
180
182
|
echo "Baseline pinned: $REPORT_FILE"
|
|
181
183
|
fi
|
|
182
184
|
|
|
185
|
+
# ── Size budget check (e31s04) ───────────────────────────────────────
|
|
186
|
+
|
|
187
|
+
if $CHECK_SIZE; then
|
|
188
|
+
WARN_THRESHOLD=20
|
|
189
|
+
FAIL_THRESHOLD=50
|
|
190
|
+
|
|
191
|
+
if [[ ! -f "$BASELINE_FILE" ]]; then
|
|
192
|
+
echo "No baseline found. Run --baseline first."
|
|
193
|
+
exit 1
|
|
194
|
+
fi
|
|
195
|
+
|
|
196
|
+
echo ""
|
|
197
|
+
echo "Size Budget Check (+${WARN_THRESHOLD}% warn, +${FAIL_THRESHOLD}% fail):"
|
|
198
|
+
SIZE_WARNINGS=0
|
|
199
|
+
SIZE_FAILURES=0
|
|
200
|
+
NEW_SKILLS=0
|
|
201
|
+
|
|
202
|
+
for skill_md in skills/*/SKILL.md; do
|
|
203
|
+
skill_name=$(basename "$(dirname "$skill_md")")
|
|
204
|
+
current_bytes=$(wc -c < "$skill_md" | tr -d ' ')
|
|
205
|
+
|
|
206
|
+
# Get baseline size (best-effort grep from YAML)
|
|
207
|
+
baseline_bytes=$(grep "^ $skill_name:" "$BASELINE_FILE" 2>/dev/null | awk '{print $2}')
|
|
208
|
+
|
|
209
|
+
if [[ -z "$baseline_bytes" ]]; then
|
|
210
|
+
echo " ${YELLOW}NEW${NC} $skill_name: ${current_bytes}b (not in baseline)"
|
|
211
|
+
(( NEW_SKILLS += 1 )) || true
|
|
212
|
+
continue
|
|
213
|
+
fi
|
|
214
|
+
|
|
215
|
+
if [[ "$baseline_bytes" -eq 0 ]]; then
|
|
216
|
+
baseline_bytes=1 # avoid division by zero
|
|
217
|
+
fi
|
|
218
|
+
|
|
219
|
+
pct=$(( (current_bytes - baseline_bytes) * 100 / baseline_bytes ))
|
|
220
|
+
|
|
221
|
+
if [[ "$pct" -ge "$FAIL_THRESHOLD" ]]; then
|
|
222
|
+
echo " ${RED}FAIL${NC} $skill_name: ${current_bytes}b (+${pct}%)"
|
|
223
|
+
(( SIZE_FAILURES += 1 )) || true
|
|
224
|
+
elif [[ "$pct" -ge "$WARN_THRESHOLD" ]]; then
|
|
225
|
+
echo " ${YELLOW}WARN${NC} $skill_name: ${current_bytes}b (+${pct}%)"
|
|
226
|
+
(( SIZE_WARNINGS += 1 )) || true
|
|
227
|
+
else
|
|
228
|
+
echo " ${GREEN}OK${NC} $skill_name: ${current_bytes}b"
|
|
229
|
+
fi
|
|
230
|
+
done
|
|
231
|
+
|
|
232
|
+
echo ""
|
|
233
|
+
echo "Size summary: $SIZE_WARNINGS warnings, $SIZE_FAILURES failures, $NEW_SKILLS new"
|
|
234
|
+
|
|
235
|
+
if [[ "$SIZE_FAILURES" -gt 0 ]]; then
|
|
236
|
+
exit 1
|
|
237
|
+
fi
|
|
238
|
+
exit 0
|
|
239
|
+
fi
|
|
240
|
+
|
|
183
241
|
if [[ "$OVERALL" == "fail" ]]; then
|
|
184
242
|
exit 1
|
|
185
243
|
fi
|
|
@@ -10,19 +10,20 @@ model: opus
|
|
|
10
10
|
|
|
11
11
|
## Loop
|
|
12
12
|
|
|
13
|
-
1. **
|
|
13
|
+
1. **Regression gate** — Run `bash scripts/run-golden-suite.sh` to catch mechanical regressions (compliance, sync pipeline, size budget) before spending time on benchmark evals. If golden suite fails, fix regressions first — they are pre-requisites for any capability improvement.
|
|
14
|
+
2. **Establish baseline** — Run `run-benchmark <skill> --baseline`. If no definition exists at `specs/benchmarks/<skill>.yaml`, create one following `specs/benchmarks/SCHEMA.md` first. Save report path in `state.yaml`. If `specs/benchmarks/reports/BASELINE-<skill>.yaml` already exists, skip this step.
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
3. **Identify gap** — Read the baseline report (`specs/benchmarks/reports/BASELINE-<skill>.yaml`). Find scenarios with `result: FAIL` or low `pass_at_k`. This is the measurable gap.
|
|
16
17
|
|
|
17
|
-
|
|
18
|
+
4. **`plan-work`** — Write a minimal change proposal targeting the failing scenarios. Include verify commands.
|
|
18
19
|
|
|
19
|
-
|
|
20
|
+
5. **Edit** via `craft-skill` / direct SKILL.md edit; run `bash scripts/sync-skills.sh`.
|
|
20
21
|
|
|
21
|
-
|
|
22
|
+
6. **Re-run benchmark** — `run-benchmark <skill>`. Compare new `pass_at_k` against baseline.
|
|
22
23
|
- **IMPROVED or STABLE** → advance to step 6.
|
|
23
24
|
- **REGRESSION** (`new pass_at_k < baseline`) → revert the change and loop back to step 3.
|
|
24
25
|
|
|
25
|
-
|
|
26
|
+
7. **Record decision** — Write `specs/adr/NNNN-evolve-<skill>.md` with before/after `pass_at_k` scores. Update `session-state`.
|
|
26
27
|
|
|
27
28
|
## Verify
|
|
28
29
|
|
package/skills-lock.json
CHANGED
|
@@ -108,7 +108,7 @@
|
|
|
108
108
|
},
|
|
109
109
|
"evolve-skill": {
|
|
110
110
|
"description": "Benchmark-gated skill evolution — consume bigpowers-benchmark report, propose plan-work change, edit skill via craft-skill, re-run benchmark, record ADR. Use when a skill underperforms on benchmark or stocktake finds systemic gap.",
|
|
111
|
-
"sha256": "
|
|
111
|
+
"sha256": "fc60347db8884afd",
|
|
112
112
|
"path": "skills/evolve-skill/SKILL.md"
|
|
113
113
|
},
|
|
114
114
|
"execute-plan": {
|