swe-workflow-skills 0.10.1 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -7
- package/VERSION +1 -1
- package/catalog.json +11 -1
- package/package.json +1 -1
- package/roles.json +5 -0
- package/skills/exploratory-data-analysis/SKILL.md +6 -0
- package/skills/feature-planning/SKILL.md +2 -0
- package/skills/feature-planning/templates/plan.md +1 -0
- package/skills/plan-execution/SKILL.md +15 -1
- package/skills/plan-execution/evals/evals.json +2 -1
- package/skills/project-documentation/SKILL.md +2 -1
- package/skills/project-review/SKILL.md +3 -0
- package/skills/research-methodology/SKILL.md +160 -0
- package/skills/research-methodology/evals/evals.json +58 -0
- package/skills/research-methodology/references/source-verification.md +71 -0
- package/skills/research-methodology/templates/claims-register.md +19 -0
- package/skills/research-methodology/templates/method-decision-record.md +41 -0
- package/skills/research-methodology/templates/traceability.md +50 -0
- package/skills/skill-router/SKILL.md +16 -0
- package/skills/working-agreement/SKILL.md +185 -0
- package/skills/working-agreement/evals/evals.json +60 -0
- package/skills/working-agreement/references/options.md +37 -0
- package/skills/working-agreement/references/validation.md +83 -0
- package/skills/working-agreement/templates/agreement.md +87 -0
package/README.md
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/swe-workflow-skills)
|
|
4
4
|
[](https://github.com/SWEStash/swe-workflow-skills/actions/workflows/roles-check.yml)
|
|
5
|
-

|
|
6
6
|
[](LICENSE)
|
|
7
7
|
|
|
8
|
-
A curated library of **
|
|
8
|
+
A curated library of **68 Claude Code Agent Skills** that walk Claude through the
|
|
9
9
|
software lifecycle the way a disciplined senior engineer would: planning, design,
|
|
10
10
|
TDD, review, security, deployment, incidents, and the project-management work around
|
|
11
11
|
them.
|
|
@@ -31,7 +31,7 @@ Code web, claude.ai chat, and Cowork:
|
|
|
31
31
|
is a **two-step** setup:
|
|
32
32
|
|
|
33
33
|
```bash
|
|
34
|
-
# 1. Install all
|
|
34
|
+
# 1. Install all 68 skills + router + /role + the hook script, and apply the baseline
|
|
35
35
|
npx swe-workflow-skills install --global
|
|
36
36
|
```
|
|
37
37
|
|
|
@@ -107,7 +107,7 @@ request can fan out to several skills. See a full session in
|
|
|
107
107
|
|
|
108
108
|
- **An orchestrator, not auto-trigger roulette.** Description-based auto-triggering is
|
|
109
109
|
probabilistic. Here `skill-router` routes intent to skills by name, and a routing
|
|
110
|
-
harness measures it:
|
|
110
|
+
harness measures it: 67/67 top-1 and 66/67 on boundary cases on the committed k=3
|
|
111
111
|
baseline ([routing benchmark](docs/ROUTING-BENCHMARK.md)).
|
|
112
112
|
- **Every skill stays reachable.** Claude Code lists skill descriptions only up to about
|
|
113
113
|
1% of context, so large libraries stop auto-triggering past 20 to 40 skills. This
|
|
@@ -129,17 +129,17 @@ request can fan out to several skills. See a full session in
|
|
|
129
129
|
|
|
130
130
|
## What's included
|
|
131
131
|
|
|
132
|
-
|
|
132
|
+
68 skills, listed in the **[full catalog → SKILLS.md](docs/SKILLS.md)**:
|
|
133
133
|
|
|
134
134
|
| Area | Count | Examples |
|
|
135
135
|
|------|-------|----------|
|
|
136
136
|
| Software Engineering | 30 | feature-planning, plan-execution, architecture-design, threat-modeling, compliance-privacy, code-archaeology, tdd-workflow, security-audit, browser-verification, subagent-orchestration |
|
|
137
|
-
| Project Management |
|
|
137
|
+
| Project Management | 9 | brainstorming, prd-writing, working-agreement, effort-estimation, build-vs-buy, metrics-and-okrs, retrospective, strategic-review |
|
|
138
138
|
| DevOps | 8 | containerization, cicd-pipeline, release-management, gitops-delivery, resilience-engineering, finops-cost-optimization |
|
|
139
139
|
| Design | 3 | ui-ux-design, frontend-architecture, accessibility-design |
|
|
140
140
|
| MLOps | 3 | ml-pipeline-design, ml-experiment-tracking, ml-model-deployment |
|
|
141
141
|
| AI Engineering | 2 | llm-app-engineering, ai-evaluation |
|
|
142
|
-
| Data Science |
|
|
142
|
+
| Data Science | 4 | research-methodology, exploratory-data-analysis, statistical-analysis, notebook-to-production |
|
|
143
143
|
| Data Engineering | 2 | data-pipeline-design, data-quality |
|
|
144
144
|
| Mobile | 2 | mobile-architecture, mobile-release |
|
|
145
145
|
| Evaluation & Monitoring | 2 | observability-design, test-data-strategy |
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.11.0
|
package/catalog.json
CHANGED
|
@@ -228,7 +228,7 @@
|
|
|
228
228
|
},
|
|
229
229
|
{
|
|
230
230
|
"name": "project-documentation",
|
|
231
|
-
"description": "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review. Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating.",
|
|
231
|
+
"description": "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review; how the project is run (process, tracking, session routines) → working-agreement. Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating.",
|
|
232
232
|
"path": "project-documentation/SKILL.md"
|
|
233
233
|
},
|
|
234
234
|
{
|
|
@@ -251,6 +251,11 @@
|
|
|
251
251
|
"description": "Cut and publish software releases right-sized to the project — semver decisions, changelogs, tagging, automated version bumps (release-please, changesets, semantic-release), publishing gates, npm/PyPI/container registry publishing, pre-release channels, monorepo versioning. Triggers: cut a release, release this, publish to npm, publish to PyPI, publish this package, version bump, what version should this be, semver, changelog, tag a release, release notes, GitHub release, prerelease, release automation, release PR. Deploy-time safety → deployment-checklist; commit/PR hygiene → git-workflow.",
|
|
252
252
|
"path": "release-management/SKILL.md"
|
|
253
253
|
},
|
|
254
|
+
{
|
|
255
|
+
"name": "research-methodology",
|
|
256
|
+
"description": "Run DS/ML research with defensible claims — framework and rigor, data understood before claims, method decision records, claims traced to evidence, sources verified. Production serving → ml-model-deployment. Triggers: research project, thesis, plan the research track, is this finding solid, can I claim this, why this method, literature review, verify these sources, deep research citations.",
|
|
257
|
+
"path": "research-methodology/SKILL.md"
|
|
258
|
+
},
|
|
254
259
|
{
|
|
255
260
|
"name": "resilience-engineering",
|
|
256
261
|
"description": "Design systems that survive failure — failure-mode analysis, resilience patterns (timeouts, retries with backoff, circuit breakers, bulkheads, load shedding, graceful degradation), chaos experiments, disaster recovery with RTO/RPO, backup restore testing, game days. Triggers: chaos engineering, disaster recovery, DR plan, what if the database goes down, single point of failure, failover, circuit breaker, RTO, RPO, game day, high availability. An incident happening NOW → incident-response; undoing a bad deploy → rollback-strategy.",
|
|
@@ -326,6 +331,11 @@
|
|
|
326
331
|
"description": "Evidence gate before any claim that work is done, fixed, passing, or deployed. Route here when a message declares the work finished or asks to commit, push, merge, ship, or close it out, including a brief sign-off, and before git-workflow or release-management act on it. Writing the commit message or PR text itself → git-workflow. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
|
|
327
332
|
"path": "verification-before-completion/SKILL.md"
|
|
328
333
|
},
|
|
334
|
+
{
|
|
335
|
+
"name": "working-agreement",
|
|
336
|
+
"description": "Define and validate how a multi-session project is run — where plans, status and decisions live, session routines, new work, milestone close. README/CONTRIBUTING → project-documentation. Triggers: where is progress tracked, how do we track this, is this milestone done, how is this project run, document our process, working agreement, definition of done, plans and status out of sync.",
|
|
337
|
+
"path": "working-agreement/SKILL.md"
|
|
338
|
+
},
|
|
329
339
|
{
|
|
330
340
|
"name": "writing-skills",
|
|
331
341
|
"description": "Use when authoring, editing, or reviewing a skill in this library — a new SKILL.md, one that under-triggers, or hardening an existing one. Covers description and listing-budget rules, frontmatter, progressive disclosure, the 3-eval rule, and pressure-testing against rationalizations. Triggers: write a skill, new skill, edit a skill, skill isn't triggering.",
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "swe-workflow-skills",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/roles.json
CHANGED
|
@@ -97,6 +97,7 @@
|
|
|
97
97
|
"core": "technical",
|
|
98
98
|
"skills": [
|
|
99
99
|
"ml-pipeline-design",
|
|
100
|
+
"research-methodology",
|
|
100
101
|
"ml-experiment-tracking",
|
|
101
102
|
"ml-model-deployment",
|
|
102
103
|
"ai-evaluation",
|
|
@@ -135,6 +136,7 @@
|
|
|
135
136
|
"core": "technical",
|
|
136
137
|
"skills": [
|
|
137
138
|
"exploratory-data-analysis",
|
|
139
|
+
"research-methodology",
|
|
138
140
|
"statistical-analysis",
|
|
139
141
|
"notebook-to-production",
|
|
140
142
|
"ml-experiment-tracking",
|
|
@@ -171,6 +173,7 @@
|
|
|
171
173
|
"dependency-impact-analysis",
|
|
172
174
|
"technical-debt-review",
|
|
173
175
|
"project-review",
|
|
176
|
+
"working-agreement",
|
|
174
177
|
"configuration-strategy",
|
|
175
178
|
"performance-optimization"
|
|
176
179
|
]
|
|
@@ -187,6 +190,7 @@
|
|
|
187
190
|
"project-proposal",
|
|
188
191
|
"prd-writing",
|
|
189
192
|
"project-review",
|
|
193
|
+
"working-agreement",
|
|
190
194
|
"strategic-review",
|
|
191
195
|
"code-reviewing",
|
|
192
196
|
"incident-response",
|
|
@@ -201,6 +205,7 @@
|
|
|
201
205
|
"skills": [
|
|
202
206
|
"project-proposal",
|
|
203
207
|
"prd-writing",
|
|
208
|
+
"working-agreement",
|
|
204
209
|
"effort-estimation",
|
|
205
210
|
"metrics-and-okrs",
|
|
206
211
|
"retrospective",
|
|
@@ -21,6 +21,12 @@ customer? a customer-month?)? Duplicate keys and mixed grains invalidate every
|
|
|
21
21
|
later statistic. If the eventual use is prediction, note *when* the prediction
|
|
22
22
|
would be made — that timestamp drives the leakage checks in step 5.
|
|
23
23
|
|
|
24
|
+
Also establish **which source is authoritative**: the curated table rather than
|
|
25
|
+
the raw inputs it was built from (trace how it's produced — `code-archaeology` —
|
|
26
|
+
when that isn't documented), and which fields are validated measurements versus
|
|
27
|
+
informational ones (estimates, free text, fields only some forms collected).
|
|
28
|
+
Profiling the wrong table produces a confident description of the wrong data.
|
|
29
|
+
|
|
24
30
|
### 2. Profile structurally
|
|
25
31
|
|
|
26
32
|
Column types, ranges, cardinality, and summary statistics for every column —
|
|
@@ -91,6 +91,8 @@ For each risk, suggest a mitigation strategy or a spike task to reduce uncertain
|
|
|
91
91
|
|
|
92
92
|
Output the complete plan using the template at [templates/plan.md](templates/plan.md) — in the reply itself, not as a promise to produce one. Offer to save it as a markdown file in the project (e.g., `docs/plans/feature-name.md`); saving is a convenience, not the deliverable.
|
|
93
93
|
|
|
94
|
+
If the plan spans several sessions or milestones, fill its **Tracking** field from the project's working agreement (where task status will live). When the project has none, say so and hand off to `working-agreement` so status has a home before execution starts.
|
|
95
|
+
|
|
94
96
|
## Principles Applied
|
|
95
97
|
|
|
96
98
|
- **YAGNI**: Only plan what's needed now. Flag nice-to-haves separately.
|
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
**Date**: [YYYY-MM-DD]
|
|
7
7
|
**Author**: [Name]
|
|
8
8
|
**Status**: Draft | In Review | Approved
|
|
9
|
+
**Tracking**: [where task status lives, per the working agreement — e.g. tracker milestone link, the status column below, or none]
|
|
9
10
|
|
|
10
11
|
## Summary
|
|
11
12
|
[2-3 sentences: what this feature does and why it matters]
|
|
@@ -43,6 +43,13 @@ checkpoint, write down **before starting it**:
|
|
|
43
43
|
A checkpoint whose verification you can't name isn't a checkpoint — split or
|
|
44
44
|
rescope it until it has one.
|
|
45
45
|
|
|
46
|
+
Read the project's **working agreement** too (linked from CLAUDE.md or the
|
|
47
|
+
README): it says where task status is recorded, what evidence closes a task, and
|
|
48
|
+
what a session records when it ends. Steps 3 and 6 follow it. When the plan spans
|
|
49
|
+
sessions and no agreement exists, invoke `working-agreement` — it drafts one under
|
|
50
|
+
stated assumptions in the same response, so execution isn't held up — because
|
|
51
|
+
progress recorded only in this session is invisible to the next one.
|
|
52
|
+
|
|
46
53
|
### Step 2: Execute a Batch, Logging Drift As You Go
|
|
47
54
|
|
|
48
55
|
Work the checkpoint's tasks. Any deviation from the plan — a different approach,
|
|
@@ -50,12 +57,16 @@ an extra task, a skipped step, a surprise — goes into a **drift log** the mome
|
|
|
50
57
|
it happens, even when the deviation is an improvement. Undocumented "better
|
|
51
58
|
ideas" are how a plan dies without anyone deciding to kill it.
|
|
52
59
|
|
|
60
|
+
When a task needs a human — a review, an approval, field data — first do every
|
|
61
|
+
part you can do yourself, then hand over a runbook: what is needed, where the
|
|
62
|
+
data is, what to run, what to look for, and what to send back.
|
|
63
|
+
|
|
53
64
|
### Step 3: Verify the Checkpoint
|
|
54
65
|
|
|
55
66
|
Run the verification you declared in Step 1 — fresh, in full, reading the entire
|
|
56
67
|
output (see `verification-before-completion` for the gate function). Only then
|
|
57
68
|
mark the checkpoint done, recording the evidence next to it: the command and its
|
|
58
|
-
observed result. A failed verification means the checkpoint stays open — fix and
|
|
69
|
+
observed result — in the place the working agreement names for status. A failed verification means the checkpoint stays open — fix and
|
|
59
70
|
re-verify; two consecutive failed checkpoints are a re-planning trigger, not a
|
|
60
71
|
push-through-it signal.
|
|
61
72
|
|
|
@@ -147,6 +158,8 @@ in checkpoint 5, and each looked locally complete while the README fell behind
|
|
|
147
158
|
both. Take the identifiers the plan touched end to end, grep the docs for them,
|
|
148
159
|
and reconcile what they name. The plan's own artifacts count: a checkpoint list
|
|
149
160
|
that still describes the pre-drift design is a stale document like any other.
|
|
161
|
+
So does the status record: close the plan where the working agreement says —
|
|
162
|
+
items closed or moved on with a reason, the plan's header marked closed.
|
|
150
163
|
|
|
151
164
|
## Rationalizations to Reject
|
|
152
165
|
|
|
@@ -182,6 +195,7 @@ that still describes the pre-drift design is a stale document like any other.
|
|
|
182
195
|
## Cross-Skill References
|
|
183
196
|
|
|
184
197
|
- `feature-planning` — creates and (on re-plan) revises the plan this skill executes
|
|
198
|
+
- `working-agreement` — where status is recorded and how sessions start, end and close a plan; invoked at Step 1 when the plan spans sessions and none exists
|
|
185
199
|
- `data-modeling` — where a schema question surfaced mid-execution belongs
|
|
186
200
|
- `architecture-design` — where a structural or boundary question belongs
|
|
187
201
|
- `verification-before-completion` — the per-claim evidence gate each checkpoint applies, and the publish gate on every checkpoint commit
|
|
@@ -10,7 +10,8 @@
|
|
|
10
10
|
"Defines checkpoints with a concrete, named verification (command or observation) for each, declared before executing that batch",
|
|
11
11
|
"Marks a checkpoint done only after running its verification fresh and recording the evidence (command and observed result)",
|
|
12
12
|
"Tracks deviations from the plan in a drift log or equivalent explicit record",
|
|
13
|
-
"States the per-checkpoint reporting rule it will follow — each checkpoint reported as verified-done with its recorded evidence, or explicitly not done — and rules out vague 'mostly done' states"
|
|
13
|
+
"States the per-checkpoint reporting rule it will follow — each checkpoint reported as verified-done with its recorded evidence, or explicitly not done — and rules out vague 'mostly done' states",
|
|
14
|
+
"States where checkpoint status and evidence will be recorded so a later session can find them (per the project's working agreement, or by establishing one when none exists), rather than leaving progress only in the conversation"
|
|
14
15
|
]
|
|
15
16
|
},
|
|
16
17
|
{
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: project-documentation
|
|
3
|
-
description: "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review."
|
|
3
|
+
description: "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review; how the project is run (process, tracking, session routines) → working-agreement."
|
|
4
4
|
when_to_use: "Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating."
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -230,3 +230,4 @@ For code-level documentation (JSDoc, Python docstrings, Go doc comments):
|
|
|
230
230
|
- `architecture-documentation` — C4 diagrams and docs-as-code for the architecture docs this skill points at
|
|
231
231
|
- `verification-before-completion` — the doc-drift and publish gates that read the versioned/internal boundary above
|
|
232
232
|
- `technical-debt-review` — a repo-wide doc-rot audit, as opposed to the drift from the change in front of you
|
|
233
|
+
- `working-agreement` — owns the process document (where things live, session routines, tracking) and its validation; this skill writes the README and CONTRIBUTING text that link to it, and a sync sweep includes it when a change alters how the project is run
|
|
@@ -35,6 +35,9 @@ Read the project's own statements of intent before judging anything:
|
|
|
35
35
|
phase does it say it's in, and what does it promise for the milestone in question?
|
|
36
36
|
- **The hard boundaries** — declared scope and explicit non-goals. These are what
|
|
37
37
|
you measure scope drift against.
|
|
38
|
+
- **The working agreement** — where the project says status lives and how
|
|
39
|
+
milestones close. If "what's done" can't be answered from where it says, or
|
|
40
|
+
there is no agreement, that is a finding: re-validate it with `working-agreement`.
|
|
38
41
|
|
|
39
42
|
Capture this as the baseline. Everything downstream is "reality vs this baseline."
|
|
40
43
|
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: research-methodology
|
|
3
|
+
description: "Run DS/ML research with defensible claims — framework and rigor, data understood before claims, method decision records, claims traced to evidence, sources verified. Production serving → ml-model-deployment."
|
|
4
|
+
when_to_use: "Triggers: research project, thesis, plan the research track, is this finding solid, can I claim this, why this method, literature review, verify these sources, deep research citations."
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Research Methodology
|
|
8
|
+
|
|
9
|
+
Research differs from feature work in one way that changes everything: the
|
|
10
|
+
output is a **claim**, and a claim is only as good as the trail behind it. The
|
|
11
|
+
data was understood, the method was chosen for stated reasons, the experiment
|
|
12
|
+
was defined before its result, the number traces to the code and data that
|
|
13
|
+
produced it, and the sources say what they're cited for.
|
|
14
|
+
|
|
15
|
+
This skill does not impose a structure. The user picks the framework and how
|
|
16
|
+
much rigour the work needs. What it holds fixed are the four rules below,
|
|
17
|
+
because each one, skipped, has produced a claim that had to be withdrawn.
|
|
18
|
+
|
|
19
|
+
## ⛔ The Iron Law
|
|
20
|
+
|
|
21
|
+
**No claim leaves the project — into a thesis, a report, a decision — without
|
|
22
|
+
its trail: the data behind it understood, the method behind it decided on the
|
|
23
|
+
record, the number traced to what produced it, the sources resolved.**
|
|
24
|
+
|
|
25
|
+
A striking result is the one that most needs the trail. "We'll add it in the
|
|
26
|
+
next revision" is how an unchecked claim survives into the final text.
|
|
27
|
+
|
|
28
|
+
## Workflow
|
|
29
|
+
|
|
30
|
+
### Step 1: Agree the Framework and the Rigor
|
|
31
|
+
|
|
32
|
+
Check the project's working agreement for a Methodology section. If there is
|
|
33
|
+
none, offer options with a recommendation — don't impose one:
|
|
34
|
+
|
|
35
|
+
- **CRISP-DM** (the default recommendation): business understanding → data
|
|
36
|
+
understanding → data preparation → modeling → evaluation → deployment, iterated.
|
|
37
|
+
Its data-understanding phase before modeling is the step research most often skips.
|
|
38
|
+
- **TDSP** (Microsoft's Team Data Science Process): similar loop, adds roles and
|
|
39
|
+
standard documents; fits a team with a client.
|
|
40
|
+
- **The user's own**, written down in a few lines.
|
|
41
|
+
|
|
42
|
+
And the **rigour level**: thesis or publication grade (every claim registered,
|
|
43
|
+
sources resolved to their identity), or internal and exploratory (claims marked
|
|
44
|
+
provisional, lighter citation). The user sets it; record it.
|
|
45
|
+
|
|
46
|
+
Write the choice into the working agreement's Methodology section (create the
|
|
47
|
+
agreement via `working-agreement` if the project has none), along with where
|
|
48
|
+
experiments, method decisions and claims live — see
|
|
49
|
+
[templates/traceability.md](templates/traceability.md).
|
|
50
|
+
|
|
51
|
+
**This step is not a gate on delivering.** Plan under the recommended framework
|
|
52
|
+
and state it — *"Planned under CRISP-DM at `<rigour>`; say if you'd rather use
|
|
53
|
+
another"* — then carry the plan in the same response.
|
|
54
|
+
|
|
55
|
+
### Step 2: Understand the Data Before Any Claim
|
|
56
|
+
|
|
57
|
+
**Invoke `exploratory-data-analysis`** before any result is interpreted, and
|
|
58
|
+
plan it as the first item of the track, not an afterthought. Running a benchmark
|
|
59
|
+
early to smoke-test the pipeline is fine; reading meaning into its numbers before
|
|
60
|
+
this step is not. For research it must cover:
|
|
61
|
+
|
|
62
|
+
- **Which source is authoritative** — the curated table, not the raw one it was
|
|
63
|
+
built from. Read how it is produced (`code-archaeology` for the pipeline).
|
|
64
|
+
- **How reliable each field is** — validated measurement, or informational
|
|
65
|
+
(an estimate, a free-text note, a field only some forms collected).
|
|
66
|
+
- **The distribution of every variable a claim will be split by.** A gap between
|
|
67
|
+
two groups means little if one group has 30 members.
|
|
68
|
+
- **What the reference data covers and doesn't.** A survey that lists some
|
|
69
|
+
households in some places makes every detection elsewhere look like a false
|
|
70
|
+
positive: precision against it is a lower bound for the surveyed area, not a
|
|
71
|
+
property of the method.
|
|
72
|
+
|
|
73
|
+
### Step 3: Decide Methods on the Record
|
|
74
|
+
|
|
75
|
+
Every choice that shapes a result — algorithm, matching rule, threshold, metric,
|
|
76
|
+
validation design — gets a **method decision record**
|
|
77
|
+
([templates/method-decision-record.md](templates/method-decision-record.md)):
|
|
78
|
+
the candidates, the evidence for each, the criteria, the decision, and the
|
|
79
|
+
alternatives worth trying later. "DBSCAN is standard" is a reason to list it, not
|
|
80
|
+
to choose it.
|
|
81
|
+
|
|
82
|
+
Include **instrument validity**: was the tool, model or product built and
|
|
83
|
+
evaluated for the variable you're about to report on? A building detector says
|
|
84
|
+
nothing reliable about roof material unless someone measured that. If it wasn't,
|
|
85
|
+
a result about that variable describes the instrument, not the world.
|
|
86
|
+
|
|
87
|
+
### Step 4: Define Each Experiment Before Its Result
|
|
88
|
+
|
|
89
|
+
Before running: hypotheses, primary metric, operating point, decision rule, and
|
|
90
|
+
which method decisions it rests on. Log and compare runs per
|
|
91
|
+
`ml-experiment-tracking`; choose tests, intervals and multiple-comparison
|
|
92
|
+
handling per `statistical-analysis`. A change after results exist is an
|
|
93
|
+
**amendment**: label it as one, with the reason.
|
|
94
|
+
|
|
95
|
+
### Step 5: Trace Every Claim
|
|
96
|
+
|
|
97
|
+
Each number that may be quoted goes in a claims register
|
|
98
|
+
([templates/claims-register.md](templates/claims-register.md)): the claim in
|
|
99
|
+
words, its scope (area, population, product, period), the experiment, the
|
|
100
|
+
commit and data version that produced it, and its status — provisional,
|
|
101
|
+
supported or withdrawn.
|
|
102
|
+
|
|
103
|
+
Write findings at the strength the evidence has:
|
|
104
|
+
- **Measured** — "recall was 0.41 (n = …, 95% CI …) for straw-and-mud roofs
|
|
105
|
+
against 0.78 for chapa, in the surveyed area": state it.
|
|
106
|
+
- **Explained or generalized** — "the product under-detects traditional roofs":
|
|
107
|
+
only after Step 2's distributions, Step 3's instrument check and the
|
|
108
|
+
confounders are done. Until then it is a provisional observation, worded as one.
|
|
109
|
+
|
|
110
|
+
Under deadline, write the supported part now and mark the rest provisional. The
|
|
111
|
+
minimal check a striking subgroup claim needs (group counts, per-group metric
|
|
112
|
+
with intervals, one confounder cross-tab) is usually an hour, not a revision.
|
|
113
|
+
|
|
114
|
+
### Step 6: Verify Sources
|
|
115
|
+
|
|
116
|
+
Every source that carries a claim passes the identity rule in
|
|
117
|
+
[references/source-verification.md](references/source-verification.md): it
|
|
118
|
+
resolves to a DOI, arXiv ID, ISBN with chapter, standard, or versioned page, and
|
|
119
|
+
the quoted text is in a captured copy, not a search summary. Run this as an
|
|
120
|
+
audit over any deep-research report before its citations are reused. Scale it
|
|
121
|
+
to the rigour set in Step 1, and apply the same test to tools: a library or
|
|
122
|
+
product the method depends on must be maintained and widely used, checked per
|
|
123
|
+
`dependency-management`.
|
|
124
|
+
|
|
125
|
+
## Rationalizations to Reject
|
|
126
|
+
|
|
127
|
+
| Excuse | Reality |
|
|
128
|
+
|--------|---------|
|
|
129
|
+
| "The result is striking; the EDA can come later" | Striking subgroup results are the likeliest to be small-n noise or skewed labels. The check is small; the retraction isn't. |
|
|
130
|
+
| "We'll add the data description in the next revision" | The claim gets written now and survives into the final text. Write it at the strength the data supports today. |
|
|
131
|
+
| "DBSCAN / U-Net / X is the standard method" | Standard is a reason to list it as a candidate. Record what else was considered and why it lost. |
|
|
132
|
+
| "The product is state of the art" | For what task? A detector evaluated on building presence hasn't been evaluated on roof material. |
|
|
133
|
+
| "Low precision means the method is poor" | Against an incomplete reference, unmatched detections include real objects nobody surveyed. Bound it to the surveyed area. |
|
|
134
|
+
| "The deep-research report already has citations" | Its citations are claims about sources. Resolve each one before reusing it. |
|
|
135
|
+
| "We need the 1977 original for every citation" | Not for established results: a trusted later work that states them is enough. Chase the original when the claim is contested or load-bearing. |
|
|
136
|
+
|
|
137
|
+
## Red Flags — Stop and Check
|
|
138
|
+
|
|
139
|
+
- A results section exists and no EDA of the labels does.
|
|
140
|
+
- A method appears in code with no record of why it was chosen.
|
|
141
|
+
- A claim is stratified by a variable whose group sizes nobody has looked at.
|
|
142
|
+
- An experiment's decision rule was written after its result.
|
|
143
|
+
- A number in the draft can't be traced to a commit and data version.
|
|
144
|
+
- A citation points at a blog, a summary page, or a search snippet.
|
|
145
|
+
|
|
146
|
+
## Cross-Skill References
|
|
147
|
+
|
|
148
|
+
- **`exploratory-data-analysis` — invoke it in Step 2, before the first result
|
|
149
|
+
is interpreted.** This skill still decides which claims wait on it; EDA does
|
|
150
|
+
the profiling.
|
|
151
|
+
- `statistical-analysis` — tests, intervals, power and multiple comparisons for
|
|
152
|
+
each experiment.
|
|
153
|
+
- `ml-experiment-tracking` — logging, comparing and reproducing runs.
|
|
154
|
+
- `working-agreement` — holds the Methodology section and the rest of how the
|
|
155
|
+
project runs (where status lives, session routines).
|
|
156
|
+
- `plan-execution` — works through the research plan checkpoint by checkpoint.
|
|
157
|
+
- `code-archaeology` — traces how a curated dataset is produced from raw inputs.
|
|
158
|
+
- `notebook-to-production` and `ml-pipeline-design` — when exploratory code
|
|
159
|
+
becomes a pipeline others rely on.
|
|
160
|
+
- Out of scope: serving a model in production → `ml-model-deployment`.
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skill_name": "research-methodology",
|
|
3
|
+
"evals": [
|
|
4
|
+
{
|
|
5
|
+
"id": 1,
|
|
6
|
+
"prompt": "Research project for a master's thesis and an NGO: detect rural households in the Argentine Gran Chaco from free satellite imagery and public building-footprint products (Google Open Buildings, Microsoft, Overture), and validate against the NGO's household survey (about 1,800 households with GPS points, roof material and household size). The repo has a methodology doc, a roadmap with phases A-D, a uv/DVC/MLflow setup and the survey snapshot. Plan the next track: our first experiment benchmarks the existing footprint products against the survey points, clustering footprints into homesteads with DBSCAN, and we also want to say which roof materials the products miss.",
|
|
7
|
+
"expected_output": "Plans the track as research, not as a software feature. Puts understanding the survey data first (an EDA step: distribution of roof material and household size, GPS quality, coverage of the surveyed area, which table is curated) before the benchmark is run or any claim about roof materials is made. Records the method choice (DBSCAN for homestead clustering) as a decision with candidates, reasons and alternatives to try, rather than taking it as given. Pre-registers the experiment's hypotheses, metric and decision rule before results exist. Shows how a later reader traces roadmap phase -> task -> experiment -> code -> result -> claim. Offers a process framework (e.g. CRISP-DM) and asks how much rigour the thesis needs, rather than imposing a rigid structure.",
|
|
8
|
+
"assertions": [
|
|
9
|
+
"Puts an exploratory data analysis of the survey labels (e.g. roof material distribution, household size, GPS quality, survey coverage) before the benchmark results are interpreted or any claim about roof materials is made",
|
|
10
|
+
"Treats the choice of DBSCAN as a method decision to record, with at least one alternative and the reason for choosing it, rather than taking it as given",
|
|
11
|
+
"Pre-registers the experiment: hypotheses, metric and decision rule written before results exist",
|
|
12
|
+
"Shows how a reader traces the work from the roadmap phase to the task, the experiment, the code and the resulting claim",
|
|
13
|
+
"Offers or names a process framework (e.g. CRISP-DM or TDSP) or asks which one to follow, rather than imposing a fixed structure",
|
|
14
|
+
"Questions whether footprint-detection products can support a roof-material claim at all, given they are built to detect buildings rather than classify roofs"
|
|
15
|
+
]
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
"id": 2,
|
|
19
|
+
"prompt": "Experiment A01 is done. Google Open Buildings found 62% of surveyed households overall but only 41% of households with straw-and-mud roofs, versus 78% for corrugated-metal (chapa) roofs. Precision was low, 0.38. Write the findings paragraph for the thesis chapter: the products under-detect traditional roofs, and precision is poor.",
|
|
20
|
+
"expected_output": "Does not write the two findings as established claims yet. Asks or checks what the claims rest on: how roof materials are distributed in the survey (if most roofs are chapa, the straw-and-mud stratum may be small, with wide intervals), whether the detector was ever built or evaluated for roof material, and confounders such as campaign, region or GPS error that correlate with roof type. For precision, notes that the survey covers only surveyed households and areas, so detections outside that coverage are not necessarily false positives, which makes 0.38 a lower bound for the surveyed area, not a property of the product. Offers a provisional wording that states what was measured, with its limits, and lists the checks (EDA, stratum sizes and intervals, instrument validity) that would make it a thesis claim.",
|
|
21
|
+
"assertions": [
|
|
22
|
+
"Does not state 'the products under-detect traditional roofs' as an established finding without first checking the roof-material distribution and stratum sizes in the survey data",
|
|
23
|
+
"Raises whether the detector was built or evaluated for roof material, i.e. whether the instrument is valid for this variable",
|
|
24
|
+
"Raises at least one confounder that could explain the roof-type gap (e.g. campaign, region, settlement density, GPS error)",
|
|
25
|
+
"Points out that precision measured against a survey that does not cover every building in the area treats unsurveyed buildings as false positives, so 0.38 is a lower bound tied to the surveyed area",
|
|
26
|
+
"Offers wording that reports what was measured with its limits, or lists the checks needed before the claim can go into the thesis"
|
|
27
|
+
]
|
|
28
|
+
},
|
|
29
|
+
{
|
|
30
|
+
"id": 3,
|
|
31
|
+
"prompt": "Our churn model is trained and validated. Help me deploy it as a real-time scoring endpoint behind our API, with monitoring for drift and a safe rollout.",
|
|
32
|
+
"expected_output": "Recognizes this as production model deployment, which belongs to ml-model-deployment, not research methodology. Hands off to or works under ml-model-deployment. Does not start a research-framework, method-decision-record or claims-register process for a deployment task.",
|
|
33
|
+
"assertions": [
|
|
34
|
+
"Recognizes the request as deploying a model to production (ml-model-deployment's domain), not as research methodology",
|
|
35
|
+
"Refers to or proceeds under ml-model-deployment for serving, drift monitoring and rollout",
|
|
36
|
+
"Does not impose research artefacts such as a method decision record, a claims register or a research framework on this request"
|
|
37
|
+
]
|
|
38
|
+
}
|
|
39
|
+
],
|
|
40
|
+
"pressure_tests": [
|
|
41
|
+
{
|
|
42
|
+
"id": 1,
|
|
43
|
+
"prompt": "The thesis draft is due to my director on Friday. The benchmark results are in and they look clear: recall 0.62 overall, and the roof finding is striking. There's no time for an EDA now. Just write the results section with the claims; we'll add the data description in the next revision.",
|
|
44
|
+
"pressure": [
|
|
45
|
+
"time",
|
|
46
|
+
"sunk_cost",
|
|
47
|
+
"authority"
|
|
48
|
+
],
|
|
49
|
+
"expected_behavior": "Does not write unsupported claims into the thesis. Separates what can be stated now (the measured recall in the surveyed area, with its definition and scope) from what cannot (the roof-material finding) until the label distribution and stratum sizes are known. Points out that the minimal EDA needed for the roof claim (a cross-tab of roof class counts and recall by stratum with intervals) is small and fits before Friday. Offers the results section with the supported claim written and the roof finding marked provisional or moved to an exploratory note.",
|
|
50
|
+
"assertions": [
|
|
51
|
+
"Does not write the roof-material finding as an established claim in the results section",
|
|
52
|
+
"Writes or offers the supported part (the measured recall with its scope and definition) so the deadline is still served",
|
|
53
|
+
"Names the minimal data check the roof claim needs (e.g. roof-class counts and recall per stratum with intervals) and says it fits before the deadline",
|
|
54
|
+
"Rejects 'add the data description in the next revision' because a claim drafted without the data check tends to survive into the final text"
|
|
55
|
+
]
|
|
56
|
+
}
|
|
57
|
+
]
|
|
58
|
+
}
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Source Verification
|
|
2
|
+
|
|
3
|
+
Adapted from article-craft's `references/source-verification.md`
|
|
4
|
+
(swestash/article-craft), cut down for research projects.
|
|
5
|
+
|
|
6
|
+
The rule: **cite only what you opened.**
|
|
7
|
+
|
|
8
|
+
## Trust tiers
|
|
9
|
+
|
|
10
|
+
- **Tier 1 — cite freely:** peer-reviewed papers and proceedings, standards,
|
|
11
|
+
canonical books cited to chapter or section, primary documentation with a
|
|
12
|
+
stated version or date (dataset cards, product technical reports, API docs).
|
|
13
|
+
- **Tier 2 — cite with attribution:** vendor engineering blogs and release notes
|
|
14
|
+
(primary for the vendor's own product, marketing-adjacent for comparisons);
|
|
15
|
+
public repositories, for what the code actually does; preprints, flagged as
|
|
16
|
+
preprints.
|
|
17
|
+
- **Tier 3 — use to find, never to cite:** community blogs, aggregators, SEO
|
|
18
|
+
pages, forum answers, model-generated summaries (including search snippets and
|
|
19
|
+
WebFetch output). A fact found only in Tier 3 is not established.
|
|
20
|
+
|
|
21
|
+
## The identity rule
|
|
22
|
+
|
|
23
|
+
Every claim that carries weight must end at something with an identity: a DOI,
|
|
24
|
+
an arXiv ID, an ISBN with chapter or page, a standard's designation, a versioned
|
|
25
|
+
documentation URL, a dated release note, a commit SHA. Anything that cannot
|
|
26
|
+
resolve to an identity is Tier 3, whatever it looks like.
|
|
27
|
+
|
|
28
|
+
This catches the two common failures without having to judge pages:
|
|
29
|
+
- **Citogenesis** — a page repeats a claim and gets cited as if it made it.
|
|
30
|
+
Follow it to the work that established it.
|
|
31
|
+
- **Fabricated references** — "Smith et al. 2019" that doesn't exist fails the
|
|
32
|
+
moment you try to resolve it.
|
|
33
|
+
|
|
34
|
+
## Capture, then quote
|
|
35
|
+
|
|
36
|
+
A search snippet or WebFetch result is a summary written by a model, not the
|
|
37
|
+
source. Use it to decide what to open. Then capture the page or PDF to a file
|
|
38
|
+
(with its URL, fetch date and any version it states), read it, and quote from
|
|
39
|
+
the file. A quote should be findable in the captured copy.
|
|
40
|
+
|
|
41
|
+
For two-column PDFs, extract in reading order, or quotes will be interleaved
|
|
42
|
+
across columns.
|
|
43
|
+
|
|
44
|
+
## Proportionality
|
|
45
|
+
|
|
46
|
+
Scale the checking to the claim and the rigour level the project agreed:
|
|
47
|
+
|
|
48
|
+
- **Established results** don't need the original: a 1977 textbook result
|
|
49
|
+
restated in a trusted recent work can be cited through that work. Chase the
|
|
50
|
+
original when the claim is contested, load-bearing for your argument, or the
|
|
51
|
+
later works disagree.
|
|
52
|
+
- **Single studies are evidence, not consensus.** When a key claim rests on one
|
|
53
|
+
paper, check whether later work adopted it, and say so in the text.
|
|
54
|
+
- **Independent corroboration only:** sources that trace to one origin count as one.
|
|
55
|
+
- **Paywalled or unreachable sources:** list them as gaps with what each would
|
|
56
|
+
support. Ask the user for a copy, or cite through a reachable work that
|
|
57
|
+
reports it. Don't cite from the abstract alone as if from the paper.
|
|
58
|
+
|
|
59
|
+
## Auditing a deep-research report
|
|
60
|
+
|
|
61
|
+
Before reusing a report's citations:
|
|
62
|
+
1. List each cited claim and the source it points to.
|
|
63
|
+
2. Resolve each source's identity and tier. Tier 3 → find the primary or drop the claim.
|
|
64
|
+
3. Spot-check quotes against captured copies, starting with the claims the
|
|
65
|
+
project's argument depends on.
|
|
66
|
+
4. Record what was verified, what is a preprint, and what remains a gap.
|
|
67
|
+
|
|
68
|
+
## When verification fails
|
|
69
|
+
|
|
70
|
+
Cut the claim, or narrow it to what the evidence supports. A weaker claim that
|
|
71
|
+
holds is worth more than a strong one that doesn't.
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Claims Register Template
|
|
2
|
+
|
|
3
|
+
Every number or finding that may be quoted outside the project (thesis, report,
|
|
4
|
+
funding request) gets a row. Save as `docs/claims.md`.
|
|
5
|
+
|
|
6
|
+
```markdown
|
|
7
|
+
# Claims
|
|
8
|
+
|
|
9
|
+
A claim may be quoted only at its status. Provisional claims are quoted with
|
|
10
|
+
their caveat or not at all.
|
|
11
|
+
|
|
12
|
+
| ID | Claim, in words | Scope (area, population, product, period) | Evidence (experiment, commit, data version) | Checks done | Status |
|
|
13
|
+
|---|---|---|---|---|---|
|
|
14
|
+
| C-001 | [e.g. "Open Buildings matched 62% of surveyed households (95% CI …)"] | [surveyed parajes, department X, survey 2019–2023] | [A01, abc1234, data v3] | [EDA AE01; n per group; CI method] | Supported |
|
|
15
|
+
| C-002 | [e.g. "Detection is lower for straw-and-mud roofs"] | | | [missing: roof-class counts, instrument validity, confounders] | Provisional |
|
|
16
|
+
|
|
17
|
+
Status values: **Provisional** (measured, checks incomplete) · **Supported**
|
|
18
|
+
(checks done) · **Withdrawn** (with the reason and date).
|
|
19
|
+
```
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# Method Decision Record Template
|
|
2
|
+
|
|
3
|
+
One per method choice that shapes a result. Like an ADR, but the evidence is
|
|
4
|
+
data and literature rather than system constraints. Save as
|
|
5
|
+
`docs/methods/MDR-NNNN-<topic>.md`.
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
# MDR-NNNN: [the question, e.g. "How to group building footprints into homesteads"]
|
|
9
|
+
|
|
10
|
+
**Status:** Proposed | Accepted | Superseded by MDR-NNNN
|
|
11
|
+
**Date:** YYYY-MM-DD
|
|
12
|
+
**Used by:** [phase / work package / experiments]
|
|
13
|
+
|
|
14
|
+
## Question
|
|
15
|
+
[What has to be decided, and which results depend on it.]
|
|
16
|
+
|
|
17
|
+
## Candidates
|
|
18
|
+
| Candidate | How it works (one line) | Evidence for | Evidence against | Sources |
|
|
19
|
+
|---|---|---|---|---|
|
|
20
|
+
| [e.g. DBSCAN] | | | | [DOI / arXiv] |
|
|
21
|
+
| [e.g. single-linkage with distance cut] | | | | |
|
|
22
|
+
| [e.g. alpha shapes per cluster] | | | | |
|
|
23
|
+
|
|
24
|
+
## Instrument validity
|
|
25
|
+
[Was each candidate (or the product/model it relies on) built and evaluated for
|
|
26
|
+
this variable and this setting? If not, what does that limit?]
|
|
27
|
+
|
|
28
|
+
## Criteria
|
|
29
|
+
[How candidates are compared: accuracy on what, cost, interpretability,
|
|
30
|
+
parameter sensitivity. Set before comparing.]
|
|
31
|
+
|
|
32
|
+
## Decision
|
|
33
|
+
[The choice and the reason, in a few sentences. Parameters and how they were set.]
|
|
34
|
+
|
|
35
|
+
## Alternatives to try later
|
|
36
|
+
[Candidates not chosen that would be worth testing if the result is weak, with
|
|
37
|
+
what would trigger trying them.]
|
|
38
|
+
|
|
39
|
+
## Consequences
|
|
40
|
+
[What this decision commits the project to; which claims inherit its limits.]
|
|
41
|
+
```
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Traceability Template
|
|
2
|
+
|
|
3
|
+
The chain a reader follows from "why are we doing this" to "which code produced
|
|
4
|
+
this number", and back. Add it to the working agreement's Methodology section, or
|
|
5
|
+
as `docs/research-workflow.md` linked from there. Rename the paths to the project's.
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
## The chain
|
|
9
|
+
|
|
10
|
+
methodology WHY + WHAT: framework, phases, gates between phases
|
|
11
|
+
│
|
|
12
|
+
roadmap WHEN: phases and milestones, the gate each one must pass
|
|
13
|
+
│
|
|
14
|
+
phase plan TASKS: work packages → acceptance, experiment, status
|
|
15
|
+
│ (status lives where the working agreement says)
|
|
16
|
+
method decisions WHY THIS METHOD: candidates, evidence, decision, alternatives
|
|
17
|
+
│
|
|
18
|
+
experiment README ONE QUESTION: hypotheses and decision rule written before
|
|
19
|
+
│ the run; results; decision; claims produced; how to reproduce
|
|
20
|
+
├── notebook where we look and explain: plots, tables, interpretation
|
|
21
|
+
└── src module tested code for anything reused or reported
|
|
22
|
+
│
|
|
23
|
+
pipeline config + lock what exactly ran: data hashes, code, parameters
|
|
24
|
+
│
|
|
25
|
+
metrics / reports results (aggregates only if the data is sensitive)
|
|
26
|
+
│
|
|
27
|
+
claims register numbers we may quote, each linked to its experiment and status
|
|
28
|
+
|
|
29
|
+
## Which file answers what
|
|
30
|
+
|
|
31
|
+
| Question | Open |
|
|
32
|
+
|---|---|
|
|
33
|
+
| Why this approach, what are the phases? | methodology |
|
|
34
|
+
| Which phase are we in, what's next? | roadmap → phase plan |
|
|
35
|
+
| Why this method, what else could we try? | method decisions |
|
|
36
|
+
| What did we look at in the data? | the EDA experiment and its notebook |
|
|
37
|
+
| What was tested, found and decided? | the experiment README |
|
|
38
|
+
| How do I re-run it? | the experiment README's Reproduce section |
|
|
39
|
+
| Can I quote this number? | claims register |
|
|
40
|
+
| Where does this dataset come from, under what licence? | the data inventory / source registry |
|
|
41
|
+
|
|
42
|
+
## Notebooks and code
|
|
43
|
+
|
|
44
|
+
- **Notebooks** are where we look and explain. They read pipeline outputs and
|
|
45
|
+
import from the package. Commit them in a diff-friendly form (e.g. jupytext).
|
|
46
|
+
- **The package** holds anything reused or reported, with tests. When a notebook
|
|
47
|
+
cell becomes something we rely on, it moves into a module.
|
|
48
|
+
- **The pipeline** runs both, so a rendered notebook always matches the data and
|
|
49
|
+
code it was produced from.
|
|
50
|
+
```
|
|
@@ -94,6 +94,7 @@ installed skill by name, regardless of role.
|
|
|
94
94
|
### Plan & Define
|
|
95
95
|
- **brainstorming** — divergent Socratic ideation *before* a spec exists; opens options, then hands off
|
|
96
96
|
- **feature-planning** — break a feature into scoped tasks, acceptance criteria, dependencies
|
|
97
|
+
- **working-agreement** — define and validate how a multi-session project is run: where status lives, session routines, new work, milestone close
|
|
97
98
|
- **build-vs-buy** — build in-house vs vendor/SaaS vs adopt OSS: TCO, lock-in, exit costs
|
|
98
99
|
- **prd-writing** — write a PRD / RFC / tech spec to align on the WHAT and WHY
|
|
99
100
|
- **project-proposal** — business case / budget / go-no-go before a project starts
|
|
@@ -175,6 +176,7 @@ installed skill by name, regardless of role.
|
|
|
175
176
|
- **data-quality** — dbt tests, expectations, data contracts, freshness, lineage / blast radius
|
|
176
177
|
|
|
177
178
|
### Data Science
|
|
179
|
+
- **research-methodology** — research with defensible claims: framework and rigor, data understood first, method decisions, claims traced, sources verified
|
|
178
180
|
- **exploratory-data-analysis** — profile an unfamiliar dataset: missingness structure, distributions, leakage, hypotheses
|
|
179
181
|
- **statistical-analysis** — hypothesis tests, experiment design/power, confidence intervals, multiple-comparison discipline
|
|
180
182
|
- **notebook-to-production** — refactor analysis notebooks to tested, parameterized, scheduled production code
|
|
@@ -200,6 +202,13 @@ leads instead, and `feature-planning` then scopes the work that the chosen optio
|
|
|
200
202
|
implies. Planning tasks against an unmade decision produces a plan you throw away.
|
|
201
203
|
Either way both skills are in the chain; what changes is which one runs first.
|
|
202
204
|
|
|
205
|
+
**New project / multi-session work**
|
|
206
|
+
`prd-writing` or `feature-planning` (the first plan that spans sessions) →
|
|
207
|
+
`working-agreement` (how the project is run, validated by a fresh reader) →
|
|
208
|
+
`plan-execution` (records status where the agreement says). In a project that
|
|
209
|
+
already runs without one — nobody can say what's done — `working-agreement`
|
|
210
|
+
leads, reconstructing the current process first.
|
|
211
|
+
|
|
203
212
|
**Bug / incident**
|
|
204
213
|
`incident-response` (if prod is down) → `bug-investigating` → `tdd-workflow`
|
|
205
214
|
(regression test) → `verification-before-completion` (the repro now passes, and
|
|
@@ -231,6 +240,13 @@ Notebook to production: `exploratory-data-analysis` (understand the data) →
|
|
|
231
240
|
(modules, tests, scheduling); if the notebook trains a model →
|
|
232
241
|
`ml-pipeline-design` instead.
|
|
233
242
|
|
|
243
|
+
**Research / data-science project**
|
|
244
|
+
`research-methodology` leads: it agrees the framework (CRISP-DM by default, or
|
|
245
|
+
the user's own) and the rigor, and the framework's phases set the order — not a
|
|
246
|
+
fixed chain. What holds under any framework: `exploratory-data-analysis` before
|
|
247
|
+
any result is interpreted, `statistical-analysis` and `ml-experiment-tracking`
|
|
248
|
+
per experiment, `plan-execution` for the work itself.
|
|
249
|
+
|
|
234
250
|
**Pre-public / pre-milestone review**
|
|
235
251
|
`strategic-review` (vision, positioning, market) → `project-review` (scope,
|
|
236
252
|
roadmap, implementation, evidence) → synthesis → `artifact-design` (interactive
|
|
@@ -0,0 +1,185 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: working-agreement
|
|
3
|
+
description: "Define and validate how a multi-session project is run — where plans, status and decisions live, session routines, new work, milestone close. README/CONTRIBUTING → project-documentation."
|
|
4
|
+
when_to_use: "Triggers: where is progress tracked, how do we track this, is this milestone done, how is this project run, document our process, working agreement, definition of done, plans and status out of sync."
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Working Agreement
|
|
8
|
+
|
|
9
|
+
A working agreement says how a project runs, so that humans and agents read
|
|
10
|
+
and write the same places: where the plan lives, where status lives, what a
|
|
11
|
+
session does when it starts and ends, how new work gets in, how a milestone
|
|
12
|
+
closes. This skill does not choose the process for the user. It makes sure one
|
|
13
|
+
is chosen, written down where every later session finds it, and **checked by
|
|
14
|
+
someone who did not write it**.
|
|
15
|
+
|
|
16
|
+
Without one, each session improvises: progress lands in a handoff nobody
|
|
17
|
+
commits, a report doubles as a tracker once, and work starts on a milestone
|
|
18
|
+
that has no plan. Nobody decides to drift; there was nothing to drift from.
|
|
19
|
+
|
|
20
|
+
## ⛔ The Iron Law
|
|
21
|
+
|
|
22
|
+
**An agreement is not in force until a fresh reader can answer every core
|
|
23
|
+
scenario from the documents alone.**
|
|
24
|
+
|
|
25
|
+
The author cannot test it: the author fills every gap from memory without
|
|
26
|
+
noticing. The content of the agreement is the user's choice; the validation is
|
|
27
|
+
not optional.
|
|
28
|
+
|
|
29
|
+
## Workflow
|
|
30
|
+
|
|
31
|
+
### Step 1: Find the Agreement, Then Pick the Mode
|
|
32
|
+
|
|
33
|
+
Look for an agreement on record: a pointer in CLAUDE.md, the README or
|
|
34
|
+
CONTRIBUTING; a `docs/how-we-work.md` or similar; tracker conventions.
|
|
35
|
+
|
|
36
|
+
- **Found** → read it and follow it. Re-validate (Step 5) only when a trigger
|
|
37
|
+
fires: a milestone closes, the process changed, or someone asked a question
|
|
38
|
+
the agreement should have answered.
|
|
39
|
+
- **Not found, small work** (one session, one person, nothing to hand over) →
|
|
40
|
+
skip this skill; say so in one line.
|
|
41
|
+
- **Not found, large work** (spans sessions or milestones, or several people or
|
|
42
|
+
agents) → pick the mode:
|
|
43
|
+
- **Define (greenfield)** — nothing practiced yet. Go to Step 2.
|
|
44
|
+
- **Discover (brownfield)** — practice exists: plans, reports, issues, git
|
|
45
|
+
log, handoffs. First write the **as-is** from that evidence: where each kind
|
|
46
|
+
of fact lives today, and where it cannot be found. Name each gap concretely
|
|
47
|
+
("the plan has no per-task status; the report was written once at close;
|
|
48
|
+
progress lives in gitignored handoffs"). That list is what Step 3 fixes.
|
|
49
|
+
Answer the user's immediate question (is M0 done?) from the same evidence,
|
|
50
|
+
or say exactly what would have to be checked.
|
|
51
|
+
|
|
52
|
+
**Exit condition** — one line, then keep going in the same response:
|
|
53
|
+
*"No agreement on record; `<define | discover>`: `<one-line state of practice>`."*
|
|
54
|
+
|
|
55
|
+
**This step is not a gate on delivering.** When you can't read the repo, state
|
|
56
|
+
what you assume exists and continue. Carry the options, the draft agreement and
|
|
57
|
+
the validation in the same response; questions go next to the draft, not in
|
|
58
|
+
place of it.
|
|
59
|
+
|
|
60
|
+
### Step 2: Offer Options and Recommend One
|
|
61
|
+
|
|
62
|
+
Two decisions, each offered as options with a recommendation fitted to the
|
|
63
|
+
project — see [references/options.md](references/options.md):
|
|
64
|
+
|
|
65
|
+
1. **Process shape** — milestone plans, Kanban, Scrum, or for research work a
|
|
66
|
+
methodology framework (hand that section to `research-methodology`).
|
|
67
|
+
2. **Where status lives** — an external tracker (GitHub/GitLab issues, Jira,
|
|
68
|
+
Linear) when one exists or can be connected; a status column in each plan
|
|
69
|
+
when there is no tool; or no tracking for short single-person work. A planned
|
|
70
|
+
switch ("markdown until the GitLab remote exists") is a valid choice: record
|
|
71
|
+
the trigger for the switch.
|
|
72
|
+
|
|
73
|
+
Don't make the draft wait on the answer: draft under the recommendation and say
|
|
74
|
+
*"Drafted under `<choice>` — pick another and I'll revise."*
|
|
75
|
+
|
|
76
|
+
### Step 3: Write the Agreement
|
|
77
|
+
|
|
78
|
+
Use [templates/agreement.md](templates/agreement.md). Every agreement covers:
|
|
79
|
+
|
|
80
|
+
- **Where things live** — one owner per kind of fact (scope, plan, task status,
|
|
81
|
+
decisions, results, session context), with when each changes. Other places
|
|
82
|
+
link to the owner; they never copy it.
|
|
83
|
+
- **Starting a session** — where to read what to do next, and in what order.
|
|
84
|
+
- **During the work** — where decisions, drift and new findings go.
|
|
85
|
+
- **Ending a session** — where progress and evidence are recorded, what a
|
|
86
|
+
handoff may and may not contain.
|
|
87
|
+
- **New work** — how a new milestone, an added task, a bug and an open question
|
|
88
|
+
each enter.
|
|
89
|
+
- **Definition of Done** — what evidence closes a task.
|
|
90
|
+
- **Closing a milestone** — what is checked, what is written, what moves on.
|
|
91
|
+
- **Asking a human** — requests for human work come as runbooks: what is needed,
|
|
92
|
+
where the data is, what to run, what to look for, what to send back.
|
|
93
|
+
|
|
94
|
+
Rules that keep it honest:
|
|
95
|
+
|
|
96
|
+
- **Status lives in one place.** A plan or report that also carries status is a
|
|
97
|
+
second copy, and the second copy goes stale.
|
|
98
|
+
- **Handoffs are context, not status.** Nobody should need a handoff to know
|
|
99
|
+
what is done.
|
|
100
|
+
- **Versioned documents only use terms a reader can resolve** — no labels from a
|
|
101
|
+
gitignored plan or a chat.
|
|
102
|
+
- **Right-size it.** One page for a two-person project is enough.
|
|
103
|
+
|
|
104
|
+
### Step 4: Prove the Tracker Connection
|
|
105
|
+
|
|
106
|
+
If status lives in an external tool, prove the agent can **read and write** it
|
|
107
|
+
before the agreement depends on it: list the open items, and create, update or
|
|
108
|
+
comment on one. A tool that is configured but not loaded in this session is not
|
|
109
|
+
connected. If it can't be proven now, the agreement names the fallback (a CLI,
|
|
110
|
+
or the human updates the tracker from a list the agent writes) and the fallback
|
|
111
|
+
is what's in force.
|
|
112
|
+
|
|
113
|
+
### Step 5: Validate With a Fresh Reader
|
|
114
|
+
|
|
115
|
+
Walk the scenario bank in [references/validation.md](references/validation.md)
|
|
116
|
+
against the written agreement, **in the same response as the draft**. Don't
|
|
117
|
+
promise the check for after the user answers your questions: the gaps it finds
|
|
118
|
+
are part of what they are answering.
|
|
119
|
+
|
|
120
|
+
- **Preferred:** a subagent that sees only the repo's documents, not this
|
|
121
|
+
conversation, answers each scenario twice: as a new agent session and as a new
|
|
122
|
+
human contributor. Each answer must cite one file and section.
|
|
123
|
+
- **Fallback** (no subagent available, or no repo yet): walk the scenarios
|
|
124
|
+
yourself, in the reply, and label it a self-check — weaker, because you wrote
|
|
125
|
+
it.
|
|
126
|
+
- Add the coverage check for the chosen process shape and, when the project
|
|
127
|
+
already has docs, the conformance check: do they follow the agreement's own
|
|
128
|
+
rules (same reference)?
|
|
129
|
+
|
|
130
|
+
Score each scenario: **answered in one place**, **ambiguous** (two places, or
|
|
131
|
+
two readings), or **missing**. Every ambiguous or missing scenario becomes a
|
|
132
|
+
question for the user or a fix to the draft. Repeat until the core scenarios
|
|
133
|
+
are all answered in one place.
|
|
134
|
+
|
|
135
|
+
**Exit condition:** *"Validated: `<n>/<n>` core scenarios answered from the
|
|
136
|
+
documents. Open: `<questions>`."*
|
|
137
|
+
|
|
138
|
+
### Step 6: Wire It In and Keep It Alive
|
|
139
|
+
|
|
140
|
+
- Put a pointer in CLAUDE.md (agents read it every session) and the README
|
|
141
|
+
(humans): *"`docs/how-we-work.md` is the process; follow it."* Both the
|
|
142
|
+
agreement and the pointer must be **tracked** files: if CLAUDE.md is
|
|
143
|
+
gitignored, a collaborator's assistant never sees it — point from the README or
|
|
144
|
+
a committed AGENTS.md, and keep only personal state in untracked files.
|
|
145
|
+
- **If the agreement and practice disagree, fix the agreement first**, in its
|
|
146
|
+
own commit, then act.
|
|
147
|
+
- Re-validate at milestone close, after a retrospective changes the process,
|
|
148
|
+
during `project-review`, and whenever a question the agreement should answer
|
|
149
|
+
had to be asked.
|
|
150
|
+
|
|
151
|
+
## Rationalizations to Reject
|
|
152
|
+
|
|
153
|
+
| Excuse | Reality |
|
|
154
|
+
|--------|---------|
|
|
155
|
+
| "It looks complete; we'll fix gaps as we hit them" | A gap is hit by the person who needed the answer, at that moment, after the drift has started. Five scenarios take minutes now. |
|
|
156
|
+
| "I wrote it, I can tell it's clear" | The author fills gaps from memory. That is why the reader must be fresh. |
|
|
157
|
+
| "The plan already has a Status field" | A plan's lifecycle (Draft, Active, Closed) is not task status. Where does task 14's status live, and who updates it? |
|
|
158
|
+
| "The handoff says what was done" | Handoffs are uncommitted and per-session. Status nobody else can see isn't status. |
|
|
159
|
+
| "We use GitLab, so tracking is covered" | Choosing a tool isn't an agreement. Which facts live there, which in docs, who updates what, when? |
|
|
160
|
+
| "The MCP server is configured" | Configured isn't connected. Prove one read and one write in this session. |
|
|
161
|
+
| "This is project management overhead for a small project" | Then the agreement is one page. Skipping it is how a small project loses track of what's done. |
|
|
162
|
+
|
|
163
|
+
## Red Flags — Stop and Check
|
|
164
|
+
|
|
165
|
+
- A session starts work and can't name which plan item it advances.
|
|
166
|
+
- The same task's status appears in two documents.
|
|
167
|
+
- "Is X done?" can only be answered by reading git log or a handoff.
|
|
168
|
+
- Work began on a milestone that has no plan.
|
|
169
|
+
- The agreement names a tool that no session has written to.
|
|
170
|
+
|
|
171
|
+
## Cross-Skill References
|
|
172
|
+
|
|
173
|
+
- `plan-execution` — follows the agreement: reads it at Step 1, records status
|
|
174
|
+
where it says, closes out by its rules. When a plan spans sessions and no
|
|
175
|
+
agreement exists, it comes here first.
|
|
176
|
+
- `feature-planning` and `prd-writing` — hand off here when the first plan or
|
|
177
|
+
PRD that spans sessions is written.
|
|
178
|
+
- `research-methodology` — writes the agreement's Methodology section for
|
|
179
|
+
research and data-science projects.
|
|
180
|
+
- `project-documentation` — writes the README and CONTRIBUTING text that link to
|
|
181
|
+
the agreement, and includes the agreement in its sync sweep. The agreement's
|
|
182
|
+
content and validation stay here.
|
|
183
|
+
- `retrospective` — process changes agreed in a retro come back here as edits to
|
|
184
|
+
the agreement, followed by re-validation.
|
|
185
|
+
- `project-review` — re-validates the agreement as part of an execution review.
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skill_name": "working-agreement",
|
|
3
|
+
"evals": [
|
|
4
|
+
{
|
|
5
|
+
"id": 1,
|
|
6
|
+
"prompt": "We're starting a new open-source app for an NGO: offline survey collection on Android, a Python sync server and a web admin. The PRD is written (docs/product/prd-mvp.md) and the roadmap splits the work into milestones M0-M3 (docs/plans/roadmap.md). Two volunteers will build it over several months, mostly in sessions with Claude Code, with no fixed deadlines. There's no Git remote yet; we'll probably use GitLab later. Before we write the first milestone plan, set up how this project will be run so every later session, human or agent, knows what to do.",
|
|
7
|
+
"expected_output": "Treats this as defining the project's working agreement (greenfield). Offers recommended options rather than imposing one: a process shape (milestone plans, Kanban, Scrum) and a tracking choice (external tracker such as GitLab issues once the remote exists, a status table in each plan until then, or none), with a recommendation. Drafts the agreement itself in the same response: where each kind of fact lives with one owner per fact, what to do at session start, during and at session end, how new work enters, a Definition of Done, how a milestone closes, how decisions are recorded. Then validates the draft by walking concrete scenarios against it and lists the gaps or ambiguities as questions. Says the agreement gets a pointer from CLAUDE.md or the README so later sessions find it.",
|
|
8
|
+
"assertions": [
|
|
9
|
+
"Offers at least two options for how progress is tracked (e.g. an external tracker such as GitLab issues, a status table in the plan files, or no tracking), with a recommendation, rather than imposing a single scheme without asking",
|
|
10
|
+
"Drafts the agreement itself in the same response rather than only asking questions or promising to write it later",
|
|
11
|
+
"The draft states where each kind of information lives (plans, task status, decisions, results, session context), with one owner for each kind",
|
|
12
|
+
"The draft gives a routine for starting a session and for ending one: where to read what to do next, and where to record progress",
|
|
13
|
+
"The draft says how new or unplanned work is added and how a milestone is closed",
|
|
14
|
+
"Validates the draft by walking concrete scenarios against it (for example a new session starting, a task blocked on a human, a decision changing) and reports the gaps or ambiguities found as questions, rather than declaring the agreement complete",
|
|
15
|
+
"Says the agreement is linked from CLAUDE.md, the README or an equivalent entry point so later sessions find it"
|
|
16
|
+
]
|
|
17
|
+
},
|
|
18
|
+
{
|
|
19
|
+
"id": 2,
|
|
20
|
+
"prompt": "i can't find the place where the project execution is being tracked. i see the plan files in docs/plans/ but can't easily find what has been done of those so far. it seems there's a gap between the plan and the report. or is the report m0-spike-report meant to be a milestone task tracking log too? i assumed that we could monitor project status this way PRD -> roadmap -> Milestone Plan -> <task progress in new file or section in plan file?> -> reports (if needed). am i missing something? is there a better way to track it? cause i can't fully understand if M0 is done or if planning M1 should come after other M0 pending tasks.\n\nWhat's in the repo:\n- docs/plans/README.md: index of plans\n- docs/plans/roadmap.md: Status: Draft; milestones M0-M3\n- docs/plans/m0-spikes.md: the M0 plan, header 'Status: In progress', task cards with acceptance criteria, no per-task status\n- docs/plans/m0-spike-report.md: written when M0 closed; a 'Status by task' table (Done / Not done with evidence)\n- .local/handoffs/ (gitignored): one handoff file per session, listing what was done and candidate next tasks\n- git log: sessions 6 and 7 built the form engine and the web skeleton, which belong to M1; there is no M1 plan yet\n- no issue tracker, no Git remote",
|
|
21
|
+
"expected_output": "Reconstructs the as-is process from the listed artifacts (brownfield discovery): the plan has no live status, the report doubles as a tracker but only once at close, the handoffs hold progress but are gitignored and invisible to humans, and M1 work started with no M1 plan. Confirms the user's chain is missing its progress step. Answers whether M0 is done from the evidence available, saying what is unknown. Proposes the working agreement: options for where progress lives with a recommendation, the routines that keep it current, and that handoffs are context rather than status. Validates the proposal against the user's own questions as scenarios.",
|
|
22
|
+
"assertions": [
|
|
23
|
+
"Identifies from the listed artifacts where progress currently lives and why it cannot be followed: the plan carries no per-task status, the report was written once at close, and the handoffs that record progress are gitignored",
|
|
24
|
+
"Notices that M1 work (form engine, web skeleton) started before any M1 plan existed",
|
|
25
|
+
"Confirms the chain the user describes is missing a live progress step, rather than saying the current setup already covers it",
|
|
26
|
+
"Offers options for where task progress should live (e.g. a status column in each plan, an external tracker, one status file) with a recommendation",
|
|
27
|
+
"States that session handoffs are context, not the record of status",
|
|
28
|
+
"Gives routines that keep status current: what to update at session end or checkpoint, and what to do when a milestone closes",
|
|
29
|
+
"Answers whether M0 is done from the evidence given, or says exactly what would have to be checked to answer it"
|
|
30
|
+
]
|
|
31
|
+
},
|
|
32
|
+
{
|
|
33
|
+
"id": 3,
|
|
34
|
+
"prompt": "Write a CONTRIBUTING.md for our open-source CLI: how to set up the dev environment (Node 22, pnpm), run the tests, our conventional-commit rule, and how to open a PR.",
|
|
35
|
+
"expected_output": "Recognizes this as contributor documentation, which belongs to project-documentation, not as defining or validating how the project is run. Writes or hands off to project-documentation for the guide. Does not start a working-agreement interview, tracking-options menu or scenario validation for a request that already states its content.",
|
|
36
|
+
"assertions": [
|
|
37
|
+
"Treats the request as writing contributor documentation (project-documentation's domain), not as defining the project's working agreement",
|
|
38
|
+
"Produces the CONTRIBUTING content or hands off to project-documentation, without first asking how the project tracks its work",
|
|
39
|
+
"Does not offer a menu of process or tracking options, or run a scenario validation, for this request"
|
|
40
|
+
]
|
|
41
|
+
}
|
|
42
|
+
],
|
|
43
|
+
"pressure_tests": [
|
|
44
|
+
{
|
|
45
|
+
"id": 1,
|
|
46
|
+
"prompt": "The how-we-work doc is written: it says GitLab issues hold task status, plans hold scope, and handoffs in .local/ hold session context. It looks complete to me. Skip the validation walkthrough, we'll fix gaps as we hit them; I want to start M1 today.",
|
|
47
|
+
"pressure": [
|
|
48
|
+
"time",
|
|
49
|
+
"authority"
|
|
50
|
+
],
|
|
51
|
+
"expected_behavior": "Holds the validation as the step that puts the agreement in force, while keeping it short enough not to block M1. Walks the core scenarios against the three rules as written (session start: where do I pick a task and in what order; session end: who closes the issue and with what evidence; new work mid-milestone; a decision that changes; milestone close) and shows the gaps they expose, such as no rule for issue/plan overlap, no Definition of Done, no close-out routine. Explains that gaps found while working are found by the person who needed the answer, which is the failure being prevented. Offers to run it now in minutes and start M1 right after.",
|
|
52
|
+
"assertions": [
|
|
53
|
+
"Does not skip the validation; runs it, or a short version of it, in this response",
|
|
54
|
+
"Walks concrete scenarios against the three stated rules and names at least two specific gaps they expose (e.g. how a session picks the next task, what evidence closes an issue, where new work goes, how a milestone closes)",
|
|
55
|
+
"Explains why fixing gaps as you hit them fails: the gap surfaces at the moment someone needs the answer, after drift has started",
|
|
56
|
+
"Keeps M1 unblocked: proposes finishing the check quickly and starting M1 the same day rather than delaying it"
|
|
57
|
+
]
|
|
58
|
+
}
|
|
59
|
+
]
|
|
60
|
+
}
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Options: process shape and where status lives
|
|
2
|
+
|
|
3
|
+
Offer two or three options with a recommendation fitted to the project. None of
|
|
4
|
+
these is the right answer in general; each has a situation it fits.
|
|
5
|
+
|
|
6
|
+
## Process shape
|
|
7
|
+
|
|
8
|
+
| Shape | Fits when | What the agreement must then define |
|
|
9
|
+
|---|---|---|
|
|
10
|
+
| **Milestone plans** | Small team or volunteers, AI-assisted sessions, no fixed cadence; work comes in large known chunks (a PRD split into milestones) | Plan lifecycle (Draft → Active → Closed), task index per plan, how a task is picked, milestone close-out and report |
|
|
11
|
+
| **Kanban** | Continuous flow of mixed-size work, no natural batches, interruptions are normal (maintenance, support, ops) | Board columns and what moves a card, WIP limit, pull rule, explicit policy per column, how a card is closed |
|
|
12
|
+
| **Scrum** | A team of about 3+ with a regular cadence and a stakeholder who reviews increments | Sprint length, backlog owner, sprint goal, Definition of Done, review and retro, who plays product owner |
|
|
13
|
+
| **Research framework** (CRISP-DM, TDSP, own) | The work answers questions with data or models; the plan changes with what is found | Hand the Methodology section to `research-methodology`; the rest of the agreement (where things live, sessions, status) still applies |
|
|
14
|
+
|
|
15
|
+
Combinations are normal: milestone plans with a Kanban board inside each
|
|
16
|
+
milestone; a research framework with milestone plans per phase. Name the
|
|
17
|
+
combination; don't leave it implied.
|
|
18
|
+
|
|
19
|
+
## Where task status lives
|
|
20
|
+
|
|
21
|
+
| Option | Fits when | Rules it brings |
|
|
22
|
+
|---|---|---|
|
|
23
|
+
| **External tracker** (GitHub/GitLab issues and milestones, Jira, Linear, Asana, Trello) | One exists, or the repo has a remote that offers one; several people; stakeholders who don't read the repo | Status lives only there. Plans keep scope and a task index that links each task to its item. Say which copy owns a task's details once it becomes an item (usually the item). Prove the agent can read and write it (SKILL.md Step 4) |
|
|
24
|
+
| **Status column in each plan** (markdown in the repo) | No tool, no remote yet, or the team prefers everything in git; cheap to share between agents and humans | One status column (and an evidence column) in the plan's task table, updated at each checkpoint in the same commit as the work. Reports record results at close and carry no live status |
|
|
25
|
+
| **One status file** (`docs/plans/status.md`) | Several plans run at once and a single board is easier to scan | Plans stay static; the status file is the only place status changes. Say what happens to it when a tracker arrives |
|
|
26
|
+
| **None** | One person, short-lived work, nothing to hand over | Say so in the agreement, so a later session doesn't invent a tracker |
|
|
27
|
+
|
|
28
|
+
**A planned switch is a valid option** ("status column now, GitLab issues once the
|
|
29
|
+
remote exists"). Record the trigger and what migrates, so the switch doesn't leave
|
|
30
|
+
two live copies.
|
|
31
|
+
|
|
32
|
+
## Recommending
|
|
33
|
+
|
|
34
|
+
Fit the recommendation to the facts given: team size, cadence, whether a remote
|
|
35
|
+
or tracker exists, who needs to read status without opening the repo. Say why in
|
|
36
|
+
one sentence. When the user names a tool they already use, that is the
|
|
37
|
+
recommendation unless something rules it out.
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# Validating a working agreement
|
|
2
|
+
|
|
3
|
+
The check: can someone who did not write the agreement do the project's normal
|
|
4
|
+
work using only the documents? Run it in the same response as the draft. Don't
|
|
5
|
+
wait for the user's answers to the open questions first; the gaps it finds are
|
|
6
|
+
part of what the user is answering.
|
|
7
|
+
|
|
8
|
+
## Core scenarios
|
|
9
|
+
|
|
10
|
+
Each must be answerable from the documents, in **one** place.
|
|
11
|
+
|
|
12
|
+
| # | Scenario | What a full answer names |
|
|
13
|
+
|---|---|---|
|
|
14
|
+
| 1 | A new session starts. What do I work on next? | Where to find the current milestone or board, how to pick an item, what to read before starting |
|
|
15
|
+
| 2 | I finished a task. How do I mark it done? | Where status lives, what evidence closes it (Definition of Done), who updates it |
|
|
16
|
+
| 3 | The session ends with work half done. What do I record, and where? | Status update location, what goes in a handoff and what doesn't |
|
|
17
|
+
| 4 | I found work the plan doesn't contain. | Where a new task, a bug, a follow-up and a question each go |
|
|
18
|
+
| 5 | A decision changes (a library, a scope cut, a method). | Where decisions are recorded, how the plan and docs follow |
|
|
19
|
+
| 6 | A task needs something only a human can do. | How the request is written (runbook: what, where, run what, look for what, return what) and where it waits, visible to everyone (not in a private handoff) |
|
|
20
|
+
| 7 | Is milestone X done? | Where to look, and what "done" means for a milestone |
|
|
21
|
+
| 8 | The milestone closes. | The close-out steps, what is written, what moves on |
|
|
22
|
+
| 9 | A new human contributor joins. | The entry point (README → agreement), what to read in what order |
|
|
23
|
+
| 10 | Two documents disagree. | Which one owns that fact; the rule that the agreement is fixed first |
|
|
24
|
+
|
|
25
|
+
Add project-specific scenarios when the project has them, e.g. "can I quote this
|
|
26
|
+
number in the thesis?" (research) or "a stakeholder asks for a status report".
|
|
27
|
+
|
|
28
|
+
## Scoring
|
|
29
|
+
|
|
30
|
+
- **Answered** — one document and section gives the answer.
|
|
31
|
+
- **Ambiguous** — two places answer it, or the text allows two readings.
|
|
32
|
+
- **Missing** — no document answers it.
|
|
33
|
+
|
|
34
|
+
Every ambiguous or missing scenario becomes either a fix to the draft or a
|
|
35
|
+
question for the user, written next to the draft.
|
|
36
|
+
|
|
37
|
+
## Conformance check (brownfield, and every re-validation)
|
|
38
|
+
|
|
39
|
+
The scenarios test what the agreement says; this tests whether the project does
|
|
40
|
+
it. Both real-project pilots that shaped this skill found the docs breaking the
|
|
41
|
+
agreement's own rules. Check:
|
|
42
|
+
|
|
43
|
+
- **Status in only one place** — reports, plans, roadmaps or findings tables that
|
|
44
|
+
still carry live status the agreement assigns elsewhere.
|
|
45
|
+
- **One definition each** — Definition of Done, status values, lifecycle values:
|
|
46
|
+
an older copy left in another doc, or a value in use that the agreement never
|
|
47
|
+
defines.
|
|
48
|
+
- **Entry points are tracked** — the reading order an agent or human is told to
|
|
49
|
+
follow must not live in a gitignored file.
|
|
50
|
+
- **Names match the rules** — file and ID patterns the agreement prescribes
|
|
51
|
+
versus what exists.
|
|
52
|
+
|
|
53
|
+
Each violation is fixed in the docs or written into the agreement as an
|
|
54
|
+
explicit exception.
|
|
55
|
+
|
|
56
|
+
## Coverage check by process shape
|
|
57
|
+
|
|
58
|
+
Light, not a ceremony audit: does the agreement say how each of these works here?
|
|
59
|
+
|
|
60
|
+
- **Milestone plans:** plan lifecycle, task index, how a task is picked, changes
|
|
61
|
+
during execution, milestone report, close-out.
|
|
62
|
+
- **Kanban:** columns and what moves a card, WIP limit, pull rule, policy per
|
|
63
|
+
column, blocked items.
|
|
64
|
+
- **Scrum:** backlog and its owner, sprint length and goal, Definition of Done,
|
|
65
|
+
review, retro, product owner.
|
|
66
|
+
- **Research framework:** phases and their exit criteria, where method decisions
|
|
67
|
+
and claims live (from `research-methodology`).
|
|
68
|
+
|
|
69
|
+
## Fresh-reader brief (for a subagent)
|
|
70
|
+
|
|
71
|
+
> You are joining this project for the first time. Read only the files in this
|
|
72
|
+
> repository; you have no other context. For each scenario below, answer as (a) a
|
|
73
|
+
> new AI-assistant session and (b) a new human contributor. For each answer, cite
|
|
74
|
+
> the file and section it comes from. If no document answers it, say MISSING. If
|
|
75
|
+
> two places answer it differently, or the text could be read two ways, say
|
|
76
|
+
> AMBIGUOUS and quote both. Do not guess and do not fill gaps from general
|
|
77
|
+
> practice.
|
|
78
|
+
>
|
|
79
|
+
> Scenarios: [paste the core scenarios plus project-specific ones]
|
|
80
|
+
|
|
81
|
+
Give the subagent the repository, not this conversation. When no subagent is
|
|
82
|
+
available, run the same brief yourself, label the result "self-check by the
|
|
83
|
+
author", and treat any answer you could only give from memory as MISSING.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# Working Agreement Template
|
|
2
|
+
|
|
3
|
+
Save as `docs/how-we-work.md` (or wherever the project keeps process docs) and
|
|
4
|
+
link it from CLAUDE.md and the README. Delete sections that don't apply; one
|
|
5
|
+
page is enough for a small project.
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
# How we work
|
|
9
|
+
|
|
10
|
+
[Who builds this, how (sessions with an AI assistant? a team with a cadence?),
|
|
11
|
+
and any constraint that shapes the process: volunteers, no deadlines, a thesis.]
|
|
12
|
+
|
|
13
|
+
If this document and practice disagree, fix this document first, in its own commit.
|
|
14
|
+
|
|
15
|
+
## Process
|
|
16
|
+
|
|
17
|
+
[The chosen shape — milestone plans / Kanban / Scrum / a research framework —
|
|
18
|
+
in two or three sentences, and why it fits.]
|
|
19
|
+
|
|
20
|
+
## Where things live
|
|
21
|
+
|
|
22
|
+
Each fact has one owner. Other places link to it instead of copying it.
|
|
23
|
+
|
|
24
|
+
| Information | Lives in | Changed when |
|
|
25
|
+
|---|---|---|
|
|
26
|
+
| What we build and why | [PRD path] | Product scope changes |
|
|
27
|
+
| How it's built; decisions | [RFC / ADR path] | A decision is made (a changed decision gets a new record) |
|
|
28
|
+
| Order of milestones | [roadmap path] | A milestone starts or closes |
|
|
29
|
+
| One milestone: scope, acceptance criteria, task index | [plan path pattern] | The plan is written; scope changes |
|
|
30
|
+
| One task: details, status, evidence | [tracker issue / plan status column] | Any time; it is the working copy |
|
|
31
|
+
| Results of a milestone | [report path pattern] | Once, when the milestone closes |
|
|
32
|
+
| Session context: what was tried, next step | [handoff location, never committed] | End of each session |
|
|
33
|
+
|
|
34
|
+
Rules:
|
|
35
|
+
- Status is written in one place only: [where].
|
|
36
|
+
- Handoffs are context, not status. Nobody needs a handoff to know what is done.
|
|
37
|
+
- Versioned documents only use terms defined in versioned documents.
|
|
38
|
+
|
|
39
|
+
## Tracker
|
|
40
|
+
|
|
41
|
+
[Only if an external tracker is used: project link, labels, milestones, how an
|
|
42
|
+
item references the plan, how the agent reaches it (MCP / CLI) and the fallback
|
|
43
|
+
when it can't. If the tracker will change later, the trigger for the switch.]
|
|
44
|
+
|
|
45
|
+
## Starting a session
|
|
46
|
+
|
|
47
|
+
1. [Where to find the current milestone and pick the next item, e.g. "open the
|
|
48
|
+
milestone, take an item whose dependencies are closed".]
|
|
49
|
+
2. [What to read before starting: the item, the decisions it cites, the latest handoff.]
|
|
50
|
+
|
|
51
|
+
## During the work
|
|
52
|
+
|
|
53
|
+
- A decision → [where it is recorded].
|
|
54
|
+
- Drift from the plan → [where it is logged; when it triggers re-planning].
|
|
55
|
+
- New work found → see "New work".
|
|
56
|
+
- Work only a human can do → a runbook in [where]: what is needed, where the
|
|
57
|
+
data is, what to run, what to look for, what to send back.
|
|
58
|
+
|
|
59
|
+
## Ending a session
|
|
60
|
+
|
|
61
|
+
1. [Update status where it lives, with the evidence that closes each item.]
|
|
62
|
+
2. [Update docs only when the session changed what they describe, in the same commit.]
|
|
63
|
+
3. [Write the handoff: context only.]
|
|
64
|
+
|
|
65
|
+
## New work
|
|
66
|
+
|
|
67
|
+
| Kind | Goes to |
|
|
68
|
+
|---|---|
|
|
69
|
+
| A new milestone | [plan first, then tracker] |
|
|
70
|
+
| A task added to a milestone | [plan's task index + a "changes during execution" note + tracker item] |
|
|
71
|
+
| A bug or follow-up | [tracker only / plan's follow-up list] |
|
|
72
|
+
| A question for a stakeholder | [where open questions live] |
|
|
73
|
+
|
|
74
|
+
## Definition of Done
|
|
75
|
+
|
|
76
|
+
A task is done when [the evidence: tests pass on the commit, the acceptance
|
|
77
|
+
criteria are checked, docs that describe it are updated]. "Implemented" or
|
|
78
|
+
"verification pending" is not done.
|
|
79
|
+
|
|
80
|
+
## Closing a milestone
|
|
81
|
+
|
|
82
|
+
1. Every item is closed or moved to the next milestone with a reason.
|
|
83
|
+
2. The report is written: results, evidence, changes, open items.
|
|
84
|
+
3. The docs are checked against what was built.
|
|
85
|
+
4. The plan's header reads Closed; the roadmap shows the next milestone.
|
|
86
|
+
5. This agreement is re-validated.
|
|
87
|
+
```
|