crewforth 0.0.1 → 3.0.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +3805 -0
- package/LICENSE +21 -0
- package/README.md +37 -7
- package/VERSION +1 -0
- package/adopt.sh +1739 -0
- package/bin/cli.js +497 -0
- package/kit/AGENT_TEMPLATE.md +92 -0
- package/kit/CLAUDE.md +228 -0
- package/kit/README.md +107 -0
- package/kit/agents/crew-backend-expert.md +95 -0
- package/kit/agents/crew-commit-agent.md +96 -0
- package/kit/agents/crew-database-expert.md +84 -0
- package/kit/agents/crew-devops-expert.md +82 -0
- package/kit/agents/crew-frontend-expert.md +85 -0
- package/kit/agents/crew-performance-expert.md +78 -0
- package/kit/agents/crew-planner.md +60 -0
- package/kit/agents/crew-privacy-agent.md +71 -0
- package/kit/agents/crew-review-agent.md +95 -0
- package/kit/agents/crew-security-expert.md +73 -0
- package/kit/agents/crew-session-manager.md +77 -0
- package/kit/agents/crew-test-expert.md +64 -0
- package/kit/eval/doctor.sh +639 -0
- package/kit/eval/gate-report.sh +137 -0
- package/kit/eval/golden-holdout.txt +53 -0
- package/kit/eval/golden-routing.txt +199 -0
- package/kit/eval/lib/crew-env.sh +14 -0
- package/kit/eval/lib/settings-json.awk +154 -0
- package/kit/eval/lib/skill-listing.awk +43 -0
- package/kit/eval/lib/star.sh +70 -0
- package/kit/eval/parser-conformance.sh +612 -0
- package/kit/eval/preflight.sh +186 -0
- package/kit/eval/routing-eval.sh +209 -0
- package/kit/eval/scan-skill.sh +162 -0
- package/kit/eval/smoke-test.sh +6736 -0
- package/kit/eval/update-guard.sh +81 -0
- package/kit/eval/utilization.sh +130 -0
- package/kit/hooks/board-sync.sh +106 -0
- package/kit/hooks/board.sh +962 -0
- package/kit/hooks/commit-msg +49 -0
- package/kit/hooks/context-usage.sh +353 -0
- package/kit/hooks/floor-blocklist.txt +124 -0
- package/kit/hooks/guard-bash.sh +1433 -0
- package/kit/hooks/guard-commit-scan.sh +511 -0
- package/kit/hooks/guard-write.sh +513 -0
- package/kit/hooks/pre-commit +577 -0
- package/kit/hooks/route-hint.sh +185 -0
- package/kit/hooks/secret-blocklist.txt +82 -0
- package/kit/hooks/session-guard.sh +149 -0
- package/kit/hooks/session-rehydrate.sh +41 -0
- package/kit/hooks/session-stats.sh +252 -0
- package/kit/hooks/session-update-check.sh +236 -0
- package/kit/hooks/skill-trust.sh +163 -0
- package/kit/hooks/trace-blocklist.txt +53 -0
- package/kit/settings.json +132 -0
- package/kit/skills/a11y/SKILL.md +44 -0
- package/kit/skills/adr/SKILL.md +74 -0
- package/kit/skills/api-design/SKILL.md +50 -0
- package/kit/skills/automode-policy/SKILL.md +91 -0
- package/kit/skills/automode-policy/references/policy.json +13 -0
- package/kit/skills/automode-policy/scripts/apply.sh +84 -0
- package/kit/skills/automode-policy/scripts/check.sh +102 -0
- package/kit/skills/backend-architecture/SKILL.md +103 -0
- package/kit/skills/brainstorm/SKILL.md +47 -0
- package/kit/skills/ci-pipeline/SKILL.md +38 -0
- package/kit/skills/commit-message/SKILL.md +82 -0
- package/kit/skills/commit-message/references/examples.md +8 -0
- package/kit/skills/confidence-check/SKILL.md +86 -0
- package/kit/skills/crew-board/SKILL.md +55 -0
- package/kit/skills/crew-brainstorm/SKILL.md +19 -0
- package/kit/skills/crew-code-review/SKILL.md +151 -0
- package/kit/skills/crew-code-review/references/history-search.md +19 -0
- package/kit/skills/crew-code-review/references/panel-mode.md +32 -0
- package/kit/skills/crew-code-review/references/receiving.md +23 -0
- package/kit/skills/crew-doctor/SKILL.md +46 -0
- package/kit/skills/crew-gates/SKILL.md +25 -0
- package/kit/skills/crew-handoff/SKILL.md +14 -0
- package/kit/skills/crew-plan/SKILL.md +18 -0
- package/kit/skills/crew-review/SKILL.md +26 -0
- package/kit/skills/crew-ship/SKILL.md +19 -0
- package/kit/skills/crew-skill/SKILL.md +62 -0
- package/kit/skills/crew-studio/SKILL.md +129 -0
- package/kit/skills/crew-update/SKILL.md +43 -0
- package/kit/skills/db-migration/SKILL.md +120 -0
- package/kit/skills/db-migration/references/backup-restore.md +33 -0
- package/kit/skills/db-migration/references/raw-sql.md +30 -0
- package/kit/skills/db-migration/references/tool-matrix.md +35 -0
- package/kit/skills/dependency-audit/SKILL.md +61 -0
- package/kit/skills/dependency-upgrade/SKILL.md +82 -0
- package/kit/skills/deploy/SKILL.md +100 -0
- package/kit/skills/deploy/references/proxy-ssl.md +56 -0
- package/kit/skills/deploy/references/rollback.md +18 -0
- package/kit/skills/deploy/references/scripts.md +9 -0
- package/kit/skills/deploy/references/topology-a.md +128 -0
- package/kit/skills/deploy/references/topology-b.md +44 -0
- package/kit/skills/docs-writer/SKILL.md +53 -0
- package/kit/skills/docs-writer/references/coauthoring.md +34 -0
- package/kit/skills/eval-grader/SKILL.md +75 -0
- package/kit/skills/eval-grader/references/method.md +51 -0
- package/kit/skills/frontend/SKILL.md +56 -0
- package/kit/skills/frontend/references/verify-contract.md +41 -0
- package/kit/skills/frontend-design/SKILL.md +71 -0
- package/kit/skills/frontend-design/references/design-review.md +34 -0
- package/kit/skills/frontend-rn-expo/SKILL.md +25 -0
- package/kit/skills/handoff/SKILL.md +71 -0
- package/kit/skills/handoff/references/cold-reader.md +51 -0
- package/kit/skills/i18n-integrity/SKILL.md +31 -0
- package/kit/skills/incident-runbook/SKILL.md +40 -0
- package/kit/skills/incident-runbook/references/postmortem.md +16 -0
- package/kit/skills/iterate/SKILL.md +40 -0
- package/kit/skills/mcp-builder/SKILL.md +54 -0
- package/kit/skills/mcp-builder/references/building.md +71 -0
- package/kit/skills/observability/SKILL.md +50 -0
- package/kit/skills/performance/SKILL.md +38 -0
- package/kit/skills/performance/references/bottlenecks.md +10 -0
- package/kit/skills/privacy-compliance/SKILL.md +75 -0
- package/kit/skills/red-team/SKILL.md +48 -0
- package/kit/skills/reflect/SKILL.md +57 -0
- package/kit/skills/release/SKILL.md +44 -0
- package/kit/skills/security-scan/SKILL.md +99 -0
- package/kit/skills/security-scan/references/ai-agents.md +107 -0
- package/kit/skills/security-scan/references/fronts.md +99 -0
- package/kit/skills/security-scan/references/prompting.md +23 -0
- package/kit/skills/security-scan/references/reporting.md +75 -0
- package/kit/skills/security-scan/references/verify.md +131 -0
- package/kit/skills/sonarqube-check/SKILL.md +108 -0
- package/kit/skills/sonarqube-check/references/local-engines.md +32 -0
- package/kit/skills/sonarqube-check/references/stand-up-server.md +25 -0
- package/kit/skills/spec-planning/SKILL.md +92 -0
- package/kit/skills/systematic-debugging/SKILL.md +57 -0
- package/kit/skills/systematic-debugging/references/techniques.md +44 -0
- package/kit/skills/teamboard/SKILL.md +88 -0
- package/kit/skills/teamboard/references/setup.md +37 -0
- package/kit/skills/testing/SKILL.md +49 -0
- package/kit/skills/testing/references/flaky-triage.md +51 -0
- package/kit/skills/threat-model/SKILL.md +57 -0
- package/kit/skills/threat-model/references/interview.md +51 -0
- package/kit/skills/threat-model/references/schema.md +42 -0
- package/kit/skills/token-budget/SKILL.md +51 -0
- package/kit/skills/trace-scan/SKILL.md +43 -0
- package/kit/skills/worktree/SKILL.md +40 -0
- package/kit/studio/README.md +266 -0
- package/kit/studio/ensure-node.sh +471 -0
- package/kit/studio/package.json +14 -0
- package/kit/studio/server/hooks/studio-gate.sh +169 -0
- package/kit/studio/server/index.js +902 -0
- package/kit/studio/server/lib/crew-env.js +11 -0
- package/kit/studio/server/lib/fleet.js +146 -0
- package/kit/studio/server/lib/graph.js +533 -0
- package/kit/studio/server/lib/instance.js +135 -0
- package/kit/studio/server/lib/kit-telemetry.js +182 -0
- package/kit/studio/server/lib/kit.js +158 -0
- package/kit/studio/server/lib/palette.js +118 -0
- package/kit/studio/server/lib/peers.js +72 -0
- package/kit/studio/server/lib/permissions.js +150 -0
- package/kit/studio/server/lib/projects.js +294 -0
- package/kit/studio/server/lib/roster.js +191 -0
- package/kit/studio/server/lib/session.js +309 -0
- package/kit/studio/server/lib/terminal.js +91 -0
- package/kit/studio/server/lib/transcript.js +100 -0
- package/kit/studio/web/app.js +1098 -0
- package/kit/studio/web/canvas.js +1179 -0
- package/kit/studio/web/chat.js +543 -0
- package/kit/studio/web/index.html +86 -0
- package/kit/studio/web/md.js +110 -0
- package/kit/studio/web/storage-migrate.js +23 -0
- package/kit/studio/web/style.css +1453 -0
- package/package.json +46 -8
- package/start.sh +764 -0
- package/bin/crewforth.js +0 -10
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sonarqube-check
|
|
3
|
+
description: |
|
|
4
|
+
SonarQube quality gate, any language, no company server needed: run SonarQube Community Build locally (Docker),
|
|
5
|
+
read the real gate + issues, fix by rule id, re-scan and diff. A linter's green build is a pre-check, never the gate.
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# SonarQube quality gate — produce the real report, locally
|
|
9
|
+
|
|
10
|
+
<!-- routing-eval reads the next line; why it sits in the body: AGENT_TEMPLATE.md -->
|
|
11
|
+
Trigger phrases: "sonarqube", "quality gate", "code smell", "sonar scan", "sonar rapor", "A rating"
|
|
12
|
+
|
|
13
|
+
**Only a SonarQube analysis can say "rating A, 0 findings".** A clean `build` says the compiler was happy — a
|
|
14
|
+
different claim, about a different rule set. Passing one off as the other is how a team is told the code is clean
|
|
15
|
+
and then opens a report full of findings.
|
|
16
|
+
|
|
17
|
+
You do **not** need a company server or someone else's token to get the real thing: **SonarQube Community Build is
|
|
18
|
+
free, needs no licence key, and runs in one Docker container on this machine.** That is the default path here — the
|
|
19
|
+
gate is produced locally, offline, and it is a genuine SonarQube verdict, not a proxy for one.
|
|
20
|
+
|
|
21
|
+
Language-agnostic on purpose: SonarQube covers 20+ languages plus IaC (Terraform, Kubernetes, Dockerfile). Detect
|
|
22
|
+
what the project actually uses; never assume a stack.
|
|
23
|
+
|
|
24
|
+
## Step 1 — get a real analysis (pick the first that applies)
|
|
25
|
+
|
|
26
|
+
**a) The project already has a SonarQube** → use it, and skip to step 2.
|
|
27
|
+
|
|
28
|
+
**b) It does not → stand one up locally.** Docker one-liner, the Docker-free server zip, the first login and the
|
|
29
|
+
token: **`references/stand-up-server.md`**. No licence, no external account, nothing leaves the machine.
|
|
30
|
+
|
|
31
|
+
> **Restarting is not installing.** Bringing an already-approved service back up on this machine — the same
|
|
32
|
+
> image, the same volumes — is resuming what the user already said yes to, and this skill's own advice to keep
|
|
33
|
+
> a named volume assumes exactly that. A NEW image, a new volume, or a tool this machine does not have is an
|
|
34
|
+
> install, and needs the answer below.
|
|
35
|
+
>
|
|
36
|
+
> **Nothing here is installed without the user saying so.** Docker, Java, a scanner, a linter — each is a proposal
|
|
37
|
+
> with its cost stated, and the user picks. If they decline all of them, that is a valid answer: go to (c) and
|
|
38
|
+
> report "unverified" rather than installing anything to make a number appear.
|
|
39
|
+
|
|
40
|
+
Scan with whatever runner the project's stack uses — the scanner is the same product for every language:
|
|
41
|
+
|
|
42
|
+
| Project | Scan command |
|
|
43
|
+
|---|---|
|
|
44
|
+
| Any stack (generic) | `sonar-scanner -Dsonar.projectKey=app -Dsonar.sources=. -Dsonar.host.url=http://localhost:9000 -Dsonar.token=$T` |
|
|
45
|
+
| Maven / Gradle | `mvn sonar:sonar -Dsonar.host.url=… -Dsonar.token=$T` · `gradle sonarqube …` |
|
|
46
|
+
| .NET | `dotnet sonarscanner begin /k:app /d:sonar.host.url=… /d:sonar.token=$T` → `dotnet build` → `dotnet sonarscanner end /d:sonar.token=$T` |
|
|
47
|
+
| JS/TS, Python, Go, PHP, Ruby | the generic `sonar-scanner` above (add coverage report paths if the project produces them) |
|
|
48
|
+
|
|
49
|
+
**c) No server is possible at all** (no Docker, no Java, no permission to run one) → run the language's Sonar-rule
|
|
50
|
+
engine locally as a pre-check (**`references/local-engines.md`** — which engine per language, and how close it is
|
|
51
|
+
to Sonar's own rules) and **say what that is worth**, in these words rather than a
|
|
52
|
+
rating: *"the local rule engine reports N issues, none open; the SonarQube rating and the gate were not produced —
|
|
53
|
+
no analysis ran."* Then list what stays unverified: taint/injection, hotspot review state, coverage, duplication,
|
|
54
|
+
files outside the compiled language, and the project's own quality profile. Offer branch (b) — a local server, with
|
|
55
|
+
or without Docker — as the way to close it.
|
|
56
|
+
|
|
57
|
+
**Never invent the verdict.** A rating exists only where an analysis produced one; without it the honest output is
|
|
58
|
+
"unverified", and that is the sentence that saves the user from promising a clean report they cannot back.
|
|
59
|
+
|
|
60
|
+
## Step 2 — read the verdict, don't infer it
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
S=http://localhost:9000; K=app # or the project's own host/key
|
|
64
|
+
curl -sfu "$T:" "$S/api/qualitygates/project_status?projectKey=$K" # pass / fail, and which condition failed
|
|
65
|
+
curl -sfu "$T:" "$S/api/issues/search?componentKeys=$K&resolved=false&ps=500" # rule id + file + line per finding
|
|
66
|
+
curl -sfu "$T:" "$S/api/hotspots/search?projectKey=$K&status=TO_REVIEW" # hotspots are a SEPARATE endpoint
|
|
67
|
+
curl -sfu "$T:" "$S/api/measures/component?component=$K&metricKeys=alert_status,bugs,vulnerabilities,security_hotspots,code_smells,coverage,duplicated_lines_density,reliability_rating,security_rating,sqale_rating"
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Also read the gate's definition (`/api/qualitygates/show`): most gates judge **new code**, not the whole project.
|
|
71
|
+
Chasing overall counts while the gate measures new code burns days and still fails; the reverse leaves real debt.
|
|
72
|
+
Know which one applies before planning the work.
|
|
73
|
+
|
|
74
|
+
## Step 3 — fix by rule, never by impression
|
|
75
|
+
|
|
76
|
+
Group findings by `rule` (`csharpsquid:S1481`, `typescript:S3776`, `python:S5852`, …) and take one rule at a time,
|
|
77
|
+
all occurrences together. Read what the rule actually asks, fix the cause, cite the rule id in the change. The
|
|
78
|
+
domain owner does the fixing — a security rule is `crew-security-expert`'s, a query/index rule is
|
|
79
|
+
`crew-database-expert`'s.
|
|
80
|
+
|
|
81
|
+
**Never close a finding by silencing it.** `#pragma warning disable`, `// NOSONAR`, `eslint-disable`, an exclusion in
|
|
82
|
+
`sonar-project.properties` — each one changes the report without changing the code. It needs a written reason and
|
|
83
|
+
the user's approval, every time.
|
|
84
|
+
|
|
85
|
+
**Security Hotspots do not close by editing code.** Each is reviewed in the UI and marked *safe* (with a rationale)
|
|
86
|
+
or *fixed*. Code changes alone leave it `TO_REVIEW` and the gate keeps failing.
|
|
87
|
+
|
|
88
|
+
## Step 4 — re-scan and diff. The loop is the deliverable
|
|
89
|
+
|
|
90
|
+
Re-run step 1's scan, re-read step 2, and report the **difference**: counts before → after, which rule ids
|
|
91
|
+
disappeared, which remain, which are new. "It should be clean now" is not a result — a second analysis is.
|
|
92
|
+
Repeat until the gate says `OK`, or every remaining finding has a user-approved suppression. Nothing here closes on a first pass.
|
|
93
|
+
|
|
94
|
+
## What even a local SonarQube cannot see
|
|
95
|
+
|
|
96
|
+
Community Build has no **taint/injection analysis** (data flow from user input to a dangerous sink — SQLi, XSS,
|
|
97
|
+
command injection); that engine is in the paid editions. It also has no branch/PR analysis. So a clean local gate
|
|
98
|
+
is not proof of injection safety: cover that with the **`security-scan`** and **`threat-model`** skills plus a
|
|
99
|
+
`crew-security-expert` review, and say which of the two verdicts you are reporting.
|
|
100
|
+
|
|
101
|
+
## DoD
|
|
102
|
+
- The verdict is quoted **from an analysis**: gate status, counts by severity, ratings, analysis date, and where it
|
|
103
|
+
ran (project server or local container).
|
|
104
|
+
- Every finding closed by a code change, or by a suppression the user explicitly approved with a written reason.
|
|
105
|
+
- Hotspots reviewed in the UI, not merely edited around.
|
|
106
|
+
- A re-scan ran after the fixes and the before → after diff by rule id is reported.
|
|
107
|
+
- Injection risk stated separately (taint analysis is not in Community Build).
|
|
108
|
+
- Anything not done is said plainly — never implied clean.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Running Sonar rules locally with no server at all
|
|
2
|
+
|
|
3
|
+
Read this only on branch (c) of step 1 — no server is possible (no Docker, no Java, no permission to run one), or
|
|
4
|
+
as a pre-check before anything is scanned.
|
|
5
|
+
|
|
6
|
+
This is worth more than it sounds: several languages have SonarSource's **own** rule engine as a normal dependency,
|
|
7
|
+
so you can find and fix most rule violations offline, before anything is ever scanned. What no local engine can
|
|
8
|
+
produce is the **verdict** — ratings (A/B/C), gate pass/fail, coverage, duplication, hotspot review state and the
|
|
9
|
+
project's quality profile are computed by the server, from an analysis. Findings yes; a rating no.
|
|
10
|
+
|
|
11
|
+
**The rule, for every language:** find whether SonarSource publishes its own rule engine for the project's
|
|
12
|
+
language and wire *that* into the build or the editor. Where only a third-party look-alike exists, use it and say
|
|
13
|
+
that the rule ids will not match the report. Ask which languages the repo actually contains — most repos have
|
|
14
|
+
more than one, and each needs its own engine.
|
|
15
|
+
|
|
16
|
+
| Stack (alphabetical — no language is the default) | Local engine | How close to Sonar's own rules |
|
|
17
|
+
|---|---|---|
|
|
18
|
+
| Go | `golangci-lint` and friends | Third-party; overlaps in intent, not in rule ids |
|
|
19
|
+
| Java / Kotlin | SonarQube for IDE (SonarLint) in the editor; SpotBugs + PMD in the build | Editor-side runs Sonar analyzers; the build-side pair is a different rule set |
|
|
20
|
+
| JS / TS | `eslint-plugin-sonarjs` | SonarSource's own JS/TS rules, delivered through ESLint |
|
|
21
|
+
| .NET (C# · VB.NET) | `SonarAnalyzer.CSharp` / `.VisualBasic` (Roslyn, at build time) | SonarSource's own analyzer, 480+ C# rules; rule-set parity with the server is not documented either way — close, not proven identical |
|
|
22
|
+
| PHP / Ruby / other | that language's linter | Varies; check whether it is SonarSource's or a look-alike |
|
|
23
|
+
| Python | `ruff` / `pylint` + `bandit` | **Not Sonar rules** — an approximation |
|
|
24
|
+
|
|
25
|
+
**Injection / taint rules are not in any of these** — they run only on SonarQube Server/Cloud, in the commercial
|
|
26
|
+
editions. So even a spotless local run says nothing about SQLi/XSS/command injection; that gap is covered by
|
|
27
|
+
`security-scan` + `threat-model` + a `crew-security-expert` review, and is reported as its own line, never folded
|
|
28
|
+
into a "clean" claim.
|
|
29
|
+
|
|
30
|
+
Pin versions so builds stay reproducible (with Central Package Management, declare it there). Turning warnings into
|
|
31
|
+
errors is the project's call — propose it, don't impose it. And whatever the local run says, the honest wording
|
|
32
|
+
stays: *findings fixed; rating and gate not produced — no analysis ran.*
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Standing up SonarQube Community Build locally
|
|
2
|
+
|
|
3
|
+
Read this only on branch (b) of step 1 — the project has no SonarQube and one is being stood up. No licence, no
|
|
4
|
+
external account, nothing leaves the machine.
|
|
5
|
+
|
|
6
|
+
**The consent rule in SKILL.md governs every line below.** Docker, Java, a scanner — each is a proposal with its
|
|
7
|
+
cost stated, and the user picks. Restarting an already-approved service is not installing; a new image, a new
|
|
8
|
+
volume, or a tool this machine does not have is.
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
# b1 — with Docker (one command)
|
|
12
|
+
docker run -d --name sonarqube -p 9000:9000 sonarqube:community # first boot ~1-2 min
|
|
13
|
+
|
|
14
|
+
# b2 — WITHOUT Docker: the plain server zip. Needs Java 17 or 21 on PATH (`java -version`), nothing else.
|
|
15
|
+
# download SonarQube Community Build, unzip (e.g. C:\sonarqube), then start it:
|
|
16
|
+
# Windows C:\sonarqube\bin\windows-x86-64\StartSonar.bat
|
|
17
|
+
# Linux/macOS ./bin/<platform>/sonar.sh console
|
|
18
|
+
# It runs on an embedded H2 database — no database to install for local use.
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Either way: http://localhost:9000 · first login `admin`/`admin` · set a new password · then My Account → Security →
|
|
22
|
+
generate a token. **That token is local and yours** — it is not a company credential and nothing is sent anywhere.
|
|
23
|
+
|
|
24
|
+
Persist the container (`-v sonarqube_data:/opt/sonarqube/data`) so history survives, and put the key/exclusions in
|
|
25
|
+
`sonar-project.properties` so every later run is identical.
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-planning
|
|
3
|
+
description: |
|
|
4
|
+
Spec-first planning: task breakdown, measurable acceptance criteria, dependency order, risk priority.
|
|
5
|
+
crew-planner applies it; the plan goes to docs/PLAN.md.
|
|
6
|
+
Use when the scope is unclear or the work spans more than one change.
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Spec-First Planning
|
|
10
|
+
|
|
11
|
+
<!-- routing-eval reads the next line; why it sits in the body: AGENT_TEMPLATE.md -->
|
|
12
|
+
Trigger phrases: "plan", "spec", "task breakdown", "acceptance criteria", "roadmap", "how do we split this"
|
|
13
|
+
|
|
14
|
+
Before writing code: what will be done, how it counts as "done", and in what order to proceed become clear.
|
|
15
|
+
|
|
16
|
+
## Steps
|
|
17
|
+
1. **Purpose & scope:** the problem being solved in a single sentence; also write the out-of-scope explicitly (prevent scope creep).
|
|
18
|
+
2. **Split into vertical slices:** the smallest end-to-end working pieces (not horizontal layers). Each slice delivers value on its own.
|
|
19
|
+
3. **A contract for every task:** input · output · **measurable acceptance criterion** (testable) · estimated risk.
|
|
20
|
+
4. **Dependency graph:** which task waits on what; no cycles. **Bring the riskiest/most-unknown to the front** (fail-fast).
|
|
21
|
+
5. **Uncertainties:** assumption list + open questions; do not fill ambiguous spots with a guess, ask with explicit options.
|
|
22
|
+
|
|
23
|
+
## Output (docs/PLAN.md)
|
|
24
|
+
```
|
|
25
|
+
# <Feature> — Plan
|
|
26
|
+
## Acceptance criteria
|
|
27
|
+
- [ ] AC-1: <measurable outcome>
|
|
28
|
+
- [ ] AC-2: <measurable outcome>
|
|
29
|
+
## Tasks (order)
|
|
30
|
+
1. <task> — satisfies: AC-1 — dependency: <none/#n> — risk: <low/medium/high>
|
|
31
|
+
## Assumptions / Open questions
|
|
32
|
+
- ...
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
**Every criterion carries a stable id, and every task names the ids it satisfies.** The id is what makes the plan
|
|
36
|
+
checkable later: "AC-2 is only partly built" can be said, argued about and tracked; "the second checkbox" cannot,
|
|
37
|
+
and renumbering a list silently re-points every reference to it. Never reuse or renumber an id — a dropped
|
|
38
|
+
criterion keeps its id with a strike-through and a one-line reason. A criterion no task satisfies, and a task that
|
|
39
|
+
satisfies no criterion, are both findings before a line of code is written.
|
|
40
|
+
|
|
41
|
+
## Mark what you do not know — do not fill it in
|
|
42
|
+
Where a requirement admits more than one reading, write the marker **`[NEEDS CLARIFICATION: <the question>]`** at
|
|
43
|
+
that exact spot in the plan. Do not resolve it with the likeliest interpretation and move on.
|
|
44
|
+
|
|
45
|
+
This is the difference between a discipline and a hope. "Stop and ask when unsure" (§1) depends on noticing the
|
|
46
|
+
uncertainty in the moment; a marker survives into the artifact, where the user, a reviewer and a later session
|
|
47
|
+
can all see it. A plausible assumption silently written into a spec is indistinguishable from a decision, and
|
|
48
|
+
that is exactly how the wrong feature gets built correctly.
|
|
49
|
+
|
|
50
|
+
Rules that keep it honest:
|
|
51
|
+
- The marker carries the **question**, not the label. `[NEEDS CLARIFICATION: does an expired invite count as
|
|
52
|
+
used, or can it be re-sent?]` is actionable; `[NEEDS CLARIFICATION: invites]` is noise.
|
|
53
|
+
- A plan may ship with markers — that is the point. It may **not** ship with a marker inside an acceptance
|
|
54
|
+
criterion: a criterion nobody can evaluate is not a criterion.
|
|
55
|
+
- Resolve by asking, never by choosing. When the user answers, replace the marker with the answer *and* record
|
|
56
|
+
which reading was rejected — the alternative is what a future reader needs.
|
|
57
|
+
- Zero markers on a genuinely ambiguous brief is a smell, not a win.
|
|
58
|
+
|
|
59
|
+
## Converge — before the work is called done, hold the code up against the plan
|
|
60
|
+
|
|
61
|
+
Tests prove the code does what the tests say. Nothing so far proves it does what the PLAN said: a criterion can be
|
|
62
|
+
quietly half-built with every test green, and code nobody asked for can ride along unremarked. So when a plan
|
|
63
|
+
exists, closing the work includes one pass that reads each criterion against the code as it now stands.
|
|
64
|
+
|
|
65
|
+
Bound the pass to what the plan names — the criteria, the tasks, and the files and components they point at. Do not
|
|
66
|
+
widen the audit into the rest of the codebase. For each `AC-n`, look at the code in that scope and record a finding
|
|
67
|
+
**only where there is a gap**, classified as exactly one of:
|
|
68
|
+
|
|
69
|
+
| Gap | Means |
|
|
70
|
+
|---|---|
|
|
71
|
+
| **missing** | nothing in the code implements the criterion |
|
|
72
|
+
| **partial** | something does, but not all of what the criterion states |
|
|
73
|
+
| **contradicts** | the code does something that conflicts with the criterion or a plan decision |
|
|
74
|
+
| **unrequested** | the code contains work no criterion or task called for |
|
|
75
|
+
|
|
76
|
+
`unrequested` is the one that is easy to skip and should not be: it is scope creep made visible (principle 3,
|
|
77
|
+
"surgical change"). It is surfaced, never deleted by this pass — the user decides whether it stays, and if it
|
|
78
|
+
does, it gets a criterion.
|
|
79
|
+
|
|
80
|
+
Report it as a table — `AC-n | gap | evidence (file/area) | remaining work` — then either:
|
|
81
|
+
- **Converged:** zero findings. Say so in one line, with how many criteria were checked.
|
|
82
|
+
- **Not converged:** append each remaining item to the plan as a task that names its `AC-n`, and go back to
|
|
83
|
+
producing. Repeat the pass after that work, until it converges or the user accepts a named gap explicitly.
|
|
84
|
+
|
|
85
|
+
This is model discipline, not a gate: no exit code can judge "partially built". What makes it hold is the table —
|
|
86
|
+
a pass that produced no table did not run.
|
|
87
|
+
|
|
88
|
+
## DoD (this skill's contribution)
|
|
89
|
+
- Every task's "done" is testable; ordering and dependencies are visible; the riskiest work has been brought to the front.
|
|
90
|
+
- Every unresolved ambiguity carries a `[NEEDS CLARIFICATION: …]` marker; no acceptance criterion contains one.
|
|
91
|
+
- Every criterion has a stable `AC-n` id and at least one task that satisfies it; at close, a converge pass
|
|
92
|
+
has run and either reported **Converged** or left its remaining items in the plan as tasks.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: systematic-debugging
|
|
3
|
+
description: |
|
|
4
|
+
Root-cause a bug before touching a fix: reproduce, isolate, form and test a hypothesis, confirm the cause,
|
|
5
|
+
then fix and verify. Stops guess-driven patching. For persistent, intermittent, or "already tried a few things" bugs.
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Systematic Debugging
|
|
9
|
+
|
|
10
|
+
<!-- routing-eval reads the next line; why it sits in the body: AGENT_TEMPLATE.md -->
|
|
11
|
+
Trigger phrases: "debug", "root cause", "why is this failing", "intermittent bug", "can't reproduce", "is broken", "crashes"
|
|
12
|
+
|
|
13
|
+
One rule holds the whole skill together: **no fix without a confirmed root cause.** A patch that makes the symptom
|
|
14
|
+
disappear without a proven cause is not a fix — it's a coin flip that hides the bug until it returns somewhere worse.
|
|
15
|
+
This skill is the discipline that turns "try things until it works" into "understand, then change one thing."
|
|
16
|
+
|
|
17
|
+
> **Crewforth adaptation (local, .claude/):** Distinct from `iterate` (a self-correction loop over a *task*) and `reflect`
|
|
18
|
+
> (meta-review of an approach) — this skill is for a *defect*. §4 Prohibitions apply; a fix still goes through the
|
|
19
|
+
> project's review/test gates. Don't disable a test or a gate to make a symptom pass (that is masking, not fixing).
|
|
20
|
+
|
|
21
|
+
## The loop (do them in order — skipping a step is why bugs come back)
|
|
22
|
+
1. **Reproduce** — a reliable, minimal repro. Can't reproduce → that IS the first problem; see `references/techniques.md` (intermittent/heisenbug).
|
|
23
|
+
2. **Isolate** — shrink the surface until the failure is in the smallest possible slice (bisect commits, halve the input, disable half the system).
|
|
24
|
+
3. **Hypothesize** — state ONE falsifiable cause: "X fails because Y." Write it down. A vague hunch is not a hypothesis.
|
|
25
|
+
4. **Test the hypothesis** — the cheapest observation that would *disprove* it (a log, a breakpoint, a probe). If it survives, you have the cause; if not, back to 3.
|
|
26
|
+
5. **Confirm the root cause** — you can explain the full chain symptom←…←cause, and you can turn the bug on and off by touching the cause.
|
|
27
|
+
6. **Fix** — the smallest change at the cause (not the symptom). Consider what else shares that cause.
|
|
28
|
+
7. **Verify** — the original repro now passes, a regression test locks it, and nothing nearby broke.
|
|
29
|
+
|
|
30
|
+
## Checklist
|
|
31
|
+
- [ ] Reliable repro captured (exact steps/input/env)
|
|
32
|
+
- [ ] Failure isolated to the smallest slice
|
|
33
|
+
- [ ] One falsifiable hypothesis written down
|
|
34
|
+
- [ ] Hypothesis tested by observation (not by applying a fix and seeing)
|
|
35
|
+
- [ ] Root cause confirmed (can toggle the bug via the cause; full chain explained)
|
|
36
|
+
- [ ] Fix at the cause + regression test added
|
|
37
|
+
- [ ] Original repro passes; no new failures
|
|
38
|
+
|
|
39
|
+
## Anti-patterns (each is guess-driven patching wearing a disguise)
|
|
40
|
+
- **Shotgun debugging** — changing several things at once; now you can't tell what mattered. One change at a time.
|
|
41
|
+
- **Symptom patching** — a `try/catch`, a null-guard, a retry that swallows the failure without explaining it.
|
|
42
|
+
- **"It works now"** without knowing *why* it broke — the bug is dormant, not dead.
|
|
43
|
+
- **Blaming the environment/flake** before isolating — sometimes true, but only *after* step 2, never as the first move.
|
|
44
|
+
- **Deleting/skipping the failing test** to go green — that is masking; the gate exists to catch exactly this.
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## Techniques by symptom
|
|
49
|
+
Bisecting (git + input + system), instrumentation vs. debugger, intermittent/heisenbugs (timing, state, ordering,
|
|
50
|
+
resource), the hypothesis log, and when to stop and ask for a second pair of eyes: **`references/techniques.md`**.
|
|
51
|
+
|
|
52
|
+
## Invariant rules
|
|
53
|
+
1. **No fix without a confirmed root cause** — the one rule; everything else serves it.
|
|
54
|
+
2. **One change at a time** — isolate cause and effect; never shotgun.
|
|
55
|
+
3. **Fix the cause, not the symptom** — a guard that hides the failure is not a fix.
|
|
56
|
+
4. **Lock it with a regression test** — the repro becomes a test so it can't silently return.
|
|
57
|
+
5. **Never mask to go green** — don't disable a test/gate/assertion to make the symptom pass.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Debugging techniques by symptom
|
|
2
|
+
|
|
3
|
+
Loaded on demand — the SKILL.md loop is the method; this is the toolbox for a specific kind of bug.
|
|
4
|
+
|
|
5
|
+
## Isolation: how to shrink the surface (step 2)
|
|
6
|
+
|
|
7
|
+
- **git bisect** — a regression that "used to work". `git bisect start; git bisect bad; git bisect good <rev>` then let it binary-search the commit that introduced it. Automate with `git bisect run <cmd>` where `<cmd>` exits 0 on good, non-zero on bad.
|
|
8
|
+
- **Binary-search the input** — halve the failing input until the smallest reproducing case remains. A 10k-row file that fails narrows to the one row; a 500-line config to the one key.
|
|
9
|
+
- **Halve the system** — disable half the middleware / feature flags / plugins. Failure gone → it's in the disabled half. Repeat.
|
|
10
|
+
- **Minimal repro** — rebuild the failure in isolation (a scratch script, a single failing unit test). If it won't reproduce in isolation, the cause is in the *environment/state* you left out — that's a finding, not a dead end.
|
|
11
|
+
|
|
12
|
+
## Instrumentation vs. debugger
|
|
13
|
+
|
|
14
|
+
- **Logging/print** — best for intermittent, timing, production, and "can't attach" cases; leaves a trace across runs. Log the *hypothesis-relevant* value, not everything. Remove or gate the probes before committing.
|
|
15
|
+
- **Debugger/breakpoint** — best for a reliable, local repro where you want to inspect live state and step. Conditional breakpoints (`x == badValue`) skip the noise.
|
|
16
|
+
- **Assertions/invariants** — add a check that *should* always hold; the point it fires is upstream of the symptom.
|
|
17
|
+
- **Bisect on values** — binary-search *when* a value goes wrong: log at start/middle/end of the pipeline, then narrow.
|
|
18
|
+
|
|
19
|
+
## Intermittent / heisenbugs (can't reliably reproduce)
|
|
20
|
+
|
|
21
|
+
The cause is almost always one of:
|
|
22
|
+
- **Timing / races** — order of concurrent operations; add delays/logging to widen the window and expose it. A bug that vanishes when you add a log is a timing bug (the log changed the timing).
|
|
23
|
+
- **Shared/leaked state** — a previous test/run left state behind; run the case in full isolation and in a different order (`--shuffle`).
|
|
24
|
+
- **Ordering dependence** — passes alone, fails in the suite → cross-test coupling. Bisect the test order.
|
|
25
|
+
- **Resource / environment** — memory pressure, disk, clock, locale, network flake, an unpinned dependency version. Pin and compare.
|
|
26
|
+
- **Uninitialized / nondeterministic** — reads of uninitialized memory, map iteration order, unseeded randomness, `Date.now()`.
|
|
27
|
+
|
|
28
|
+
Make it reproducible *first* (loop the case 100×, add stress, shuffle). An intermittent bug you can't trigger on demand can't be confirmed fixed.
|
|
29
|
+
|
|
30
|
+
## The hypothesis log
|
|
31
|
+
|
|
32
|
+
For a bug that resists a few rounds, write it down — it stops you from re-testing the same dead hypothesis and reveals patterns:
|
|
33
|
+
|
|
34
|
+
```
|
|
35
|
+
Symptom: <exact observable failure>
|
|
36
|
+
Repro: <steps / input / env>
|
|
37
|
+
H1: <cause> → test: <observation> → RESULT: disproved (saw X, not Y)
|
|
38
|
+
H2: <cause> → test: <observation> → RESULT: CONFIRMED (toggles bug)
|
|
39
|
+
Root cause: <the confirmed chain> Fix: <change at cause> Test: <regression test>
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## When to stop and get a second pair of eyes
|
|
43
|
+
|
|
44
|
+
After ~3 disproved hypotheses with no new information, or when you've been staring at the same 20 lines — you likely hold a wrong assumption you can't see. Escalate: hand over the hypothesis log (not "it's broken") so the next person starts from what's *already ruled out*, not from zero. In Crewforth, that handover is the `handoff` skill.
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: teamboard
|
|
3
|
+
description: |
|
|
4
|
+
Experimental: a shared team board. Claim a work item, hand it over, finish it; the claim is a git-ref
|
|
5
|
+
lock, so two people cannot take the same item.
|
|
6
|
+
metadata:
|
|
7
|
+
experimental: true
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Team Board (shared claims + shared item memory)
|
|
11
|
+
|
|
12
|
+
<!-- routing-eval reads the next line; why it sits in the body: AGENT_TEMPLATE.md -->
|
|
13
|
+
Trigger phrases: "team board", "who is working on", "claim", "take this item", "pick up a task", "sprint item", "release the item", "hand the item over", "is anyone on", "board", "of us are working", "stepping on each other", "who is doing what", "who else is working"
|
|
14
|
+
|
|
15
|
+
## The problem this closes
|
|
16
|
+
Every teammate runs Crewforth **locally**. `docs/PLAN.md` and `docs/SESSION_STATE.md` are gitignored, so the fact
|
|
17
|
+
that Ali started item #1 two hours ago exists nowhere the other sessions can see. Two people start the same item;
|
|
18
|
+
a blocked item gets picked up before its dependency lands; whoever inherits half-finished work rebuilds the
|
|
19
|
+
context from scratch. The board makes those three facts shared.
|
|
20
|
+
|
|
21
|
+
## The engine (all commands go through it)
|
|
22
|
+
`bash .claude/hooks/board.sh <cmd>` — `status` · `show <id>` · `claim <id>` · `done <id> [note]` ·
|
|
23
|
+
`drop <id> <note>` · `note <id> <text>` · `add <id> <title> [deps] [external]` · `init` · `sync`.
|
|
24
|
+
Never hand-edit the board; every write must go through the engine or it loses its atomicity.
|
|
25
|
+
|
|
26
|
+
## Why a claim is a real lock, not a convention
|
|
27
|
+
The board lives on a git ref (`refs/crew/board`), not in the worktree. A claim is a commit pushed to that ref, and
|
|
28
|
+
`git push` is fast-forward-only: when two people claim the same item from the same base, **one push is rejected**
|
|
29
|
+
and that session re-reads the board and refuses. This is an atomic compare-and-swap with no server, no token and
|
|
30
|
+
no daemon. The engine builds its commits with git plumbing, so claiming **never touches your working tree,
|
|
31
|
+
index, branch or stash** — you can claim mid-feature with dirty files.
|
|
32
|
+
|
|
33
|
+
## Off by default; on when a team asks for it
|
|
34
|
+
A repo that never ran `init` **has no board and no gates** — no claim, no commit gate, no edit gate, nothing at
|
|
35
|
+
session start. Solo work is unchanged, and so is every project that installed Crewforth before this existed. Do not
|
|
36
|
+
create a board because a repo merely has more than one contributor; create one when the user says the team keeps
|
|
37
|
+
colliding. Creating one, choosing between its three levels and the ref-namespace fallback: **`references/setup.md`**
|
|
38
|
+
— one person runs it once, and everybody else configures nothing.
|
|
39
|
+
|
|
40
|
+
Already have a board and want it out of the way? `/crew-board off` (add `--global` for every repo) releases **all
|
|
41
|
+
three** gates and leaves the board itself intact; `/crew-board on` restores it. `CREW_NO_BOARD=1` does the same for
|
|
42
|
+
one session. This half stays here on purpose: someone a gate has just stopped needs the answer without opening a
|
|
43
|
+
second file.
|
|
44
|
+
|
|
45
|
+
## The rules
|
|
46
|
+
1. **Claim before you touch code.** `claim <id>` fails if someone else holds it, if it is done, or if a
|
|
47
|
+
dependency is unfinished — with the reason and the list of what *is* claimable. Do not argue with a refusal;
|
|
48
|
+
take a free item.
|
|
49
|
+
2. **Two gates, not one, and the early one is the point.** The first file edit is refused while you hold no item
|
|
50
|
+
(`guard-write.sh`) — unclaimed work is caught at minute one, not at commit time after an afternoon of it. The
|
|
51
|
+
commit then has to carry `[#<id>]` for an item you hold and that is `in_progress`, or `[chore]` for work
|
|
52
|
+
belonging to no item. In a repo with no board neither gate exists. `CREW_NO_BOARD=1` disables both for a
|
|
53
|
+
session; say so out loud if you set it.
|
|
54
|
+
3. **Never release silently.** `drop` REQUIRES a handover note, because the whole cost of a handover is the
|
|
55
|
+
context the next person does not have: where exactly it stands, which files, and *why* the rejected approach
|
|
56
|
+
was rejected. `done` takes a completion note (what shipped, commit/PR).
|
|
57
|
+
4. **Never steal a stale claim.** A claim with no heartbeat for `stale_hours` (default 8) is reported as STALE.
|
|
58
|
+
Stale means *ask the owner*, not *take it*. Surface it to the user and let them decide.
|
|
59
|
+
5. **Read before you plan.** At session start the board summary is injected automatically. Before proposing what
|
|
60
|
+
to work on, run `status` — recommending an item somebody already holds is the failure this exists to prevent.
|
|
61
|
+
|
|
62
|
+
## Redaction — the board is shared
|
|
63
|
+
Everything written here is pushed to a remote the whole team reads. `handoff`'s `<private>…</private>` rule
|
|
64
|
+
applies unchanged: strip secrets, tokens, credentials and personal notes, leave `[redacted]`, and point at
|
|
65
|
+
*where* a value lives (env var, secret manager, the person to ask) rather than the value.
|
|
66
|
+
|
|
67
|
+
## Filling the board
|
|
68
|
+
Items come from either direction, and both end at `add`:
|
|
69
|
+
- **From a plan:** `crew-planner` + `spec-planning` produce `docs/PLAN.md` with measurable acceptance criteria and
|
|
70
|
+
a dependency order; each task becomes one `add <id> <title> <deps>`. Keep the plan's dependency edges — they
|
|
71
|
+
are what makes `claim` able to refuse blocked work.
|
|
72
|
+
- **By hand:** the user dictates the items.
|
|
73
|
+
|
|
74
|
+
## Tracker link (honest boundary)
|
|
75
|
+
Teams use Jira, Trello, Linear, GitHub Issues or nothing, so Crewforth **binds to none of them**. An item carries an
|
|
76
|
+
optional `external:` field, which is a link and nothing more — no sync, no import, no write-back. A team that
|
|
77
|
+
wants two-way sync writes an executable `.crew/board-adapter.sh` answering two verbs: `import` (emit
|
|
78
|
+
`<id>|<title>|<deps>|<external>` lines to be fed to `add`) and `export <id> <status>` (push the state outward).
|
|
79
|
+
Crewforth ships no adapter; do not claim integration Crewforth does not have.
|
|
80
|
+
|
|
81
|
+
## When the remote is unreachable
|
|
82
|
+
Claims stay local and are labelled `UNSHARED` — meaning **the team cannot see them**, so the lock is not in force.
|
|
83
|
+
Say that plainly rather than reporting success, and re-run `sync` once the network is back.
|
|
84
|
+
|
|
85
|
+
## DoD
|
|
86
|
+
- Work started only on an item held by this user; the board shows it `in_progress`.
|
|
87
|
+
- An item left for any reason carries a handover note a stranger could resume from.
|
|
88
|
+
- `done` recorded with what shipped, and the dependents it unblocked named to the user.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Setting a board up — one person, one command
|
|
2
|
+
|
|
3
|
+
Read this only when a board is being created or its level changed. Claiming, dropping and finishing an item never
|
|
4
|
+
need it: a teammate who clones the repo configures nothing.
|
|
5
|
+
|
|
6
|
+
## Who runs what
|
|
7
|
+
|
|
8
|
+
- **Whoever starts it:** `/crew-board init`, then add the items. It probes what the server accepts, creates the
|
|
9
|
+
board on the code repo's own `origin`, and that is the whole setup — no account, no token, no service.
|
|
10
|
+
- **A separate board repository:** `/crew-board init --remote <url>` (or an existing remote's name). Use it when
|
|
11
|
+
the board is shared across several repos, or when people who must claim work cannot push to the code repo. It
|
|
12
|
+
gets its own `crew-board` remote and never touches `origin`.
|
|
13
|
+
- **Everyone else: nothing.** They clone as usual. Session start fetches the board on its own (detached), the
|
|
14
|
+
ref namespace is auto-detected including the orphan-branch fallback, and `/crew-board` fetches on the spot if
|
|
15
|
+
the background refresh has not landed yet. Never tell a teammate to run `init` — a second `init` is how a team
|
|
16
|
+
ends up with two boards.
|
|
17
|
+
|
|
18
|
+
## The ref namespace, and the fallback
|
|
19
|
+
|
|
20
|
+
`init` first probes whether the server accepts a custom ref namespace (`refs/crew/board`); if it does not, it
|
|
21
|
+
falls back to the orphan branch `refs/heads/crew-board`, which every server accepts and which enforces the same
|
|
22
|
+
fast-forward rule. Do not merge that branch into code.
|
|
23
|
+
|
|
24
|
+
## Which level to create
|
|
25
|
+
|
|
26
|
+
Do not create a board because a repo merely has more than one contributor; create one when the user says the team
|
|
27
|
+
keeps colliding. Three levels, and the user picks:
|
|
28
|
+
|
|
29
|
+
| | Effect |
|
|
30
|
+
|---|---|
|
|
31
|
+
| no board (default) | nothing at all |
|
|
32
|
+
| `require_item: referenced` in the board's `config` | claims and the shared memory, but no gate: a commit is only checked when it names an item |
|
|
33
|
+
| `require_item: all` (what `init` writes) | claim before you edit, and every commit names an item or `[chore]` |
|
|
34
|
+
|
|
35
|
+
A board with no remote is fine too — you get the item list, the dependency order and the gates, just nothing
|
|
36
|
+
shared. Turning the gates off on an existing board is in SKILL.md, not here: someone a gate has just stopped
|
|
37
|
+
needs that answer without opening a second file.
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: testing
|
|
3
|
+
description: |
|
|
4
|
+
The how of testing: pyramid, AAA, isolation, risk coverage, determinism. Guarantees the DoD's "tests are
|
|
5
|
+
green". crew-test-expert applies it.
|
|
6
|
+
Use when writing or changing tests, or when a suite is flaky, slow or green for the wrong reason.
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Testing Discipline
|
|
10
|
+
|
|
11
|
+
<!-- routing-eval reads the next line; why it sits in the body: AGENT_TEMPLATE.md -->
|
|
12
|
+
Trigger phrases: "write a test", "run the tests", "coverage", "are the tests green", "unit test", "integration test", "tests pass locally", "flaky test", "test is flaky"
|
|
13
|
+
|
|
14
|
+
Goal: **behavior correctness** — test real behavior without breaking product code just to make a test pass.
|
|
15
|
+
|
|
16
|
+
## Principles
|
|
17
|
+
- **Pyramid:** many unit, fewer integration, few end-to-end (e2e). Limit e2e to critical flows.
|
|
18
|
+
- **AAA:** Arrange-Act-Assert; **one test = one behavior**.
|
|
19
|
+
- **Isolation & determinism:** external dependencies are mocked/faked; time and randomness are fixed; test order is independent.
|
|
20
|
+
- **Risk-coverage:** risk, not metrics. Critical path + **boundary** + **negative** + **authorization (IDOR/404)** scenarios.
|
|
21
|
+
- **Naming:** `what_it_tests_under_which_condition_what_it_expects` — on failure it is clear what broke.
|
|
22
|
+
- **Red-green:** first a failing test, then the implementation.
|
|
23
|
+
|
|
24
|
+
## Watch out
|
|
25
|
+
- **Flaky = bug:** an occasionally failing test is not tolerated; it is fixed at the root.
|
|
26
|
+
- In snapshot/golden-file tests, avoid needless brittleness (assert only the meaningful output).
|
|
27
|
+
|
|
28
|
+
## Tests that cannot fail
|
|
29
|
+
|
|
30
|
+
Green is evidence only if the test could have gone red. Shapes that check nothing:
|
|
31
|
+
- **Assertion restates the code:** it recomputes the implementation's own expression, so it moves with every change.
|
|
32
|
+
- **Setup guarantees the result:** arrange plants the value the assert reads back; it holds even if the act never ran.
|
|
33
|
+
- **Expected value derived the way the code derives it:** same formula, query or parser — the same wrong assumption on
|
|
34
|
+
both sides still matches.
|
|
35
|
+
- **A double asserted against its own stub:** the fake is told to return X and the test asserts X; only the fake runs.
|
|
36
|
+
- **No assertion:** it passes because nothing threw. If *not throwing* is the behavior under test, assert that.
|
|
37
|
+
|
|
38
|
+
**Prove it can fail:** break what the test covers — invert a condition, return a wrong constant — confirm it goes red,
|
|
39
|
+
then restore. Still green means the test is documentation, not a gate: fix it or delete it, never count it as coverage.
|
|
40
|
+
|
|
41
|
+
## Flaky triage
|
|
42
|
+
An intermittent failure gets exactly one outcome, decided the day it is seen: **fix** it, **quarantine** it (out of
|
|
43
|
+
the gating lane, never out of the suite, with an owner and a date), or **escalate** it as the product defect it
|
|
44
|
+
signals. Infrastructure flakes may get a bounded, logged retry; product flakes never do. Classifying them, the retry
|
|
45
|
+
rule, and what a quarantine entry must carry: **`references/flaky-triage.md`**.
|
|
46
|
+
|
|
47
|
+
## DoD (this skill's contribution)
|
|
48
|
+
- The project's own test command is green — read it off the manifest/CI (`dotnet test` · `npm test` · `pytest` ·
|
|
49
|
+
`go test ./...`), don't assume one; critical paths are covered; no empty/meaningless tests.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Flaky test triage
|
|
2
|
+
|
|
3
|
+
Loaded on demand — SKILL.md holds the rule (`Flaky = bug`); this is the decision procedure for one that has already
|
|
4
|
+
failed intermittently. A test that passes and fails on the same commit gates nothing, and every red run it produces
|
|
5
|
+
teaches the team to ignore red runs. Triage it the day it is seen and record the outcome.
|
|
6
|
+
|
|
7
|
+
## 1 · Classify first: infrastructure or product
|
|
8
|
+
The two kinds get opposite treatment, so this call comes before the outcome.
|
|
9
|
+
|
|
10
|
+
- **Infrastructure flake** — fails the same way regardless of the test's logic, and clusters in time. Signals:
|
|
11
|
+
identical error text across unrelated tests (connection refused, image pull, no disk, name resolution); a whole
|
|
12
|
+
run or one machine fails and the next run passes with no code change; the assertion is never reached.
|
|
13
|
+
- **Product flake** — follows the data or the ordering. Signals: it tracks one case, fixture or seed; it appears
|
|
14
|
+
only under randomized order or parallel execution; it reproduces in a loop on one machine; the assertion *is*
|
|
15
|
+
reached and the value is wrong.
|
|
16
|
+
- Cannot tell yet → treat it as a product flake. Guessing "infrastructure" is how a real race gets a retry put
|
|
17
|
+
around it.
|
|
18
|
+
|
|
19
|
+
## 2 · Retry policy follows that classification
|
|
20
|
+
- **Infrastructure: a bounded retry is allowed** — a fixed cap decided once and written down, applied where the
|
|
21
|
+
resource is owned (readiness wait, fixture setup, the job step), never around an assertion. Log every retry that
|
|
22
|
+
fires: a retry nobody can see is a failure nobody counts, and a rising count means the label was wrong.
|
|
23
|
+
- **Product: never retry.** A retry there turns a real bug into a slower green — the defect ships and the suite
|
|
24
|
+
reports success. Widening a timeout and sleeping until it passes are the same move under another name.
|
|
25
|
+
|
|
26
|
+
## 3 · Three outcomes — pick exactly one, in writing
|
|
27
|
+
| Outcome | Use when | What it requires |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| **Fix** | the test or the code under it is genuinely wrong: unfrozen clock, unseeded randomness, shared state, order dependence, an assertion on something unordered | the change, plus a run that reproduced the old failure (loop, shuffle, parallel) and now holds |
|
|
30
|
+
| **Quarantine** | the failure is real but not yet understood, and blocking every other change on it costs more than the coverage it holds | a named owner, a date, a linked issue — and the test still executing |
|
|
31
|
+
| **Escalate** | the flake is a symptom, not a test defect: a race, a leak, a resource limit, an unsafe ordering in the code under test | hand it over as a product defect and root-cause it (`systematic-debugging`) |
|
|
32
|
+
|
|
33
|
+
Escalate means the change lands in product code. Rewriting the test to step around the race deletes the only
|
|
34
|
+
evidence that the race exists, and it resurfaces in production instead of in CI.
|
|
35
|
+
|
|
36
|
+
## 4 · Quarantine has a shape, or it is deletion
|
|
37
|
+
- **Scoped, not removed.** The test leaves the *gating* lane, not the suite: it keeps running in a non-gating or
|
|
38
|
+
scheduled job, so its failure rate stays visible and the fix can be proven.
|
|
39
|
+
- **Owner and date, always** — both stored next to the test (marker, tag, or an explicit list the gating job filters
|
|
40
|
+
on) and re-read on that date. If nobody will own it and no date fits, the honest outcome was Fix or Escalate.
|
|
41
|
+
- **Re-decide on the date**, choosing again from the three. A quarantined test nobody returns to is worse than a
|
|
42
|
+
deleted one: it reports coverage that is not being checked.
|
|
43
|
+
- Deleting the test deliberately, with the reason recorded, is a legitimate end state. Silent expiry is not.
|
|
44
|
+
- Quarantine is not a way to go green for a release; an escalate-class flake blocks the release it belongs to.
|
|
45
|
+
|
|
46
|
+
## Boundary with the rest of the discipline
|
|
47
|
+
- Disabling or skipping a red test to finish a task is masking, not quarantine: masking removes the signal,
|
|
48
|
+
quarantine relocates it and puts a date on it.
|
|
49
|
+
- Making an intermittent failure reproduce on demand — loop the case, randomize order, add stress, widen the timing
|
|
50
|
+
window — is `systematic-debugging/references/techniques.md`. Triage without a repro is a guess.
|
|
51
|
+
- The quarantine list is a debt list. If it only grows, nothing is being fixed: report the count and the oldest entry.
|