swe-workflow-skills 0.10.1 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,10 +2,10 @@
2
2
 
3
3
  [![npm](https://img.shields.io/npm/v/swe-workflow-skills)](https://www.npmjs.com/package/swe-workflow-skills)
4
4
  [![roles-check](https://github.com/SWEStash/swe-workflow-skills/actions/workflows/roles-check.yml/badge.svg)](https://github.com/SWEStash/swe-workflow-skills/actions/workflows/roles-check.yml)
5
- ![skills](https://img.shields.io/badge/skills-66-blue)
5
+ ![skills](https://img.shields.io/badge/skills-68-blue)
6
6
  [![license: MIT](https://img.shields.io/badge/license-MIT-yellow.svg)](LICENSE)
7
7
 
8
- A curated library of **66 Claude Code Agent Skills** that walk Claude through the
8
+ A curated library of **68 Claude Code Agent Skills** that walk Claude through the
9
9
  software lifecycle the way a disciplined senior engineer would: planning, design,
10
10
  TDD, review, security, deployment, incidents, and the project-management work around
11
11
  them.
@@ -31,7 +31,7 @@ Code web, claude.ai chat, and Cowork:
31
31
  is a **two-step** setup:
32
32
 
33
33
  ```bash
34
- # 1. Install all 66 skills + router + /role + the hook script, and apply the baseline
34
+ # 1. Install all 68 skills + router + /role + the hook script, and apply the baseline
35
35
  npx swe-workflow-skills install --global
36
36
  ```
37
37
 
@@ -107,7 +107,7 @@ request can fan out to several skills. See a full session in
107
107
 
108
108
  - **An orchestrator, not auto-trigger roulette.** Description-based auto-triggering is
109
109
  probabilistic. Here `skill-router` routes intent to skills by name, and a routing
110
- harness measures it: 65/65 top-1 and 64/65 on boundary cases on the committed k=3
110
+ harness measures it: 67/67 top-1 and 66/67 on boundary cases on the committed k=3
111
111
  baseline ([routing benchmark](docs/ROUTING-BENCHMARK.md)).
112
112
  - **Every skill stays reachable.** Claude Code lists skill descriptions only up to about
113
113
  1% of context, so large libraries stop auto-triggering past 20 to 40 skills. This
@@ -129,17 +129,17 @@ request can fan out to several skills. See a full session in
129
129
 
130
130
  ## What's included
131
131
 
132
- 66 skills, listed in the **[full catalog → SKILLS.md](docs/SKILLS.md)**:
132
+ 68 skills, listed in the **[full catalog → SKILLS.md](docs/SKILLS.md)**:
133
133
 
134
134
  | Area | Count | Examples |
135
135
  |------|-------|----------|
136
136
  | Software Engineering | 30 | feature-planning, plan-execution, architecture-design, threat-modeling, compliance-privacy, code-archaeology, tdd-workflow, security-audit, browser-verification, subagent-orchestration |
137
- | Project Management | 8 | brainstorming, prd-writing, effort-estimation, build-vs-buy, metrics-and-okrs, retrospective, strategic-review |
137
+ | Project Management | 9 | brainstorming, prd-writing, working-agreement, effort-estimation, build-vs-buy, metrics-and-okrs, retrospective, strategic-review |
138
138
  | DevOps | 8 | containerization, cicd-pipeline, release-management, gitops-delivery, resilience-engineering, finops-cost-optimization |
139
139
  | Design | 3 | ui-ux-design, frontend-architecture, accessibility-design |
140
140
  | MLOps | 3 | ml-pipeline-design, ml-experiment-tracking, ml-model-deployment |
141
141
  | AI Engineering | 2 | llm-app-engineering, ai-evaluation |
142
- | Data Science | 3 | exploratory-data-analysis, statistical-analysis, notebook-to-production |
142
+ | Data Science | 4 | research-methodology, exploratory-data-analysis, statistical-analysis, notebook-to-production |
143
143
  | Data Engineering | 2 | data-pipeline-design, data-quality |
144
144
  | Mobile | 2 | mobile-architecture, mobile-release |
145
145
  | Evaluation & Monitoring | 2 | observability-design, test-data-strategy |
package/VERSION CHANGED
@@ -1 +1 @@
1
- 0.10.1
1
+ 0.11.0
package/catalog.json CHANGED
@@ -228,7 +228,7 @@
228
228
  },
229
229
  {
230
230
  "name": "project-documentation",
231
- "description": "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review. Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating.",
231
+ "description": "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review; how the project is run (process, tracking, session routines) → working-agreement. Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating.",
232
232
  "path": "project-documentation/SKILL.md"
233
233
  },
234
234
  {
@@ -251,6 +251,11 @@
251
251
  "description": "Cut and publish software releases right-sized to the project — semver decisions, changelogs, tagging, automated version bumps (release-please, changesets, semantic-release), publishing gates, npm/PyPI/container registry publishing, pre-release channels, monorepo versioning. Triggers: cut a release, release this, publish to npm, publish to PyPI, publish this package, version bump, what version should this be, semver, changelog, tag a release, release notes, GitHub release, prerelease, release automation, release PR. Deploy-time safety → deployment-checklist; commit/PR hygiene → git-workflow.",
252
252
  "path": "release-management/SKILL.md"
253
253
  },
254
+ {
255
+ "name": "research-methodology",
256
+ "description": "Run DS/ML research with defensible claims — framework and rigor, data understood before claims, method decision records, claims traced to evidence, sources verified. Production serving → ml-model-deployment. Triggers: research project, thesis, plan the research track, is this finding solid, can I claim this, why this method, literature review, verify these sources, deep research citations.",
257
+ "path": "research-methodology/SKILL.md"
258
+ },
254
259
  {
255
260
  "name": "resilience-engineering",
256
261
  "description": "Design systems that survive failure — failure-mode analysis, resilience patterns (timeouts, retries with backoff, circuit breakers, bulkheads, load shedding, graceful degradation), chaos experiments, disaster recovery with RTO/RPO, backup restore testing, game days. Triggers: chaos engineering, disaster recovery, DR plan, what if the database goes down, single point of failure, failover, circuit breaker, RTO, RPO, game day, high availability. An incident happening NOW → incident-response; undoing a bad deploy → rollback-strategy.",
@@ -326,6 +331,11 @@
326
331
  "description": "Evidence gate before any claim that work is done, fixed, passing, or deployed. Route here when a message declares the work finished or asks to commit, push, merge, ship, or close it out, including a brief sign-off, and before git-workflow or release-management act on it. Writing the commit message or PR text itself → git-workflow. Triggers: is it done, did it work, tests pass, verify, ready to commit, before I push, confirm the fix.",
327
332
  "path": "verification-before-completion/SKILL.md"
328
333
  },
334
+ {
335
+ "name": "working-agreement",
336
+ "description": "Define and validate how a multi-session project is run — where plans, status and decisions live, session routines, new work, milestone close. README/CONTRIBUTING → project-documentation. Triggers: where is progress tracked, how do we track this, is this milestone done, how is this project run, document our process, working agreement, definition of done, plans and status out of sync.",
337
+ "path": "working-agreement/SKILL.md"
338
+ },
329
339
  {
330
340
  "name": "writing-skills",
331
341
  "description": "Use when authoring, editing, or reviewing a skill in this library — a new SKILL.md, one that under-triggers, or hardening an existing one. Covers description and listing-budget rules, frontmatter, progressive disclosure, the 3-eval rule, and pressure-testing against rationalizations. Triggers: write a skill, new skill, edit a skill, skill isn't triggering.",
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "swe-workflow-skills",
3
- "version": "0.10.1",
3
+ "version": "0.11.0",
4
4
  "description": "A curated library of Claude Code Agent Skills covering the full software lifecycle, with orchestrator-routed activation that scales past the skill-listing budget.",
5
5
  "type": "module",
6
6
  "bin": {
package/roles.json CHANGED
@@ -97,6 +97,7 @@
97
97
  "core": "technical",
98
98
  "skills": [
99
99
  "ml-pipeline-design",
100
+ "research-methodology",
100
101
  "ml-experiment-tracking",
101
102
  "ml-model-deployment",
102
103
  "ai-evaluation",
@@ -135,6 +136,7 @@
135
136
  "core": "technical",
136
137
  "skills": [
137
138
  "exploratory-data-analysis",
139
+ "research-methodology",
138
140
  "statistical-analysis",
139
141
  "notebook-to-production",
140
142
  "ml-experiment-tracking",
@@ -171,6 +173,7 @@
171
173
  "dependency-impact-analysis",
172
174
  "technical-debt-review",
173
175
  "project-review",
176
+ "working-agreement",
174
177
  "configuration-strategy",
175
178
  "performance-optimization"
176
179
  ]
@@ -187,6 +190,7 @@
187
190
  "project-proposal",
188
191
  "prd-writing",
189
192
  "project-review",
193
+ "working-agreement",
190
194
  "strategic-review",
191
195
  "code-reviewing",
192
196
  "incident-response",
@@ -201,6 +205,7 @@
201
205
  "skills": [
202
206
  "project-proposal",
203
207
  "prd-writing",
208
+ "working-agreement",
204
209
  "effort-estimation",
205
210
  "metrics-and-okrs",
206
211
  "retrospective",
@@ -21,6 +21,12 @@ customer? a customer-month?)? Duplicate keys and mixed grains invalidate every
21
21
  later statistic. If the eventual use is prediction, note *when* the prediction
22
22
  would be made — that timestamp drives the leakage checks in step 5.
23
23
 
24
+ Also establish **which source is authoritative**: the curated table rather than
25
+ the raw inputs it was built from (trace how it's produced — `code-archaeology` —
26
+ when that isn't documented), and which fields are validated measurements versus
27
+ informational ones (estimates, free text, fields only some forms collected).
28
+ Profiling the wrong table produces a confident description of the wrong data.
29
+
24
30
  ### 2. Profile structurally
25
31
 
26
32
  Column types, ranges, cardinality, and summary statistics for every column —
@@ -91,6 +91,8 @@ For each risk, suggest a mitigation strategy or a spike task to reduce uncertain
91
91
 
92
92
  Output the complete plan using the template at [templates/plan.md](templates/plan.md) — in the reply itself, not as a promise to produce one. Offer to save it as a markdown file in the project (e.g., `docs/plans/feature-name.md`); saving is a convenience, not the deliverable.
93
93
 
94
+ If the plan spans several sessions or milestones, fill its **Tracking** field from the project's working agreement (where task status will live). When the project has none, say so and hand off to `working-agreement` so status has a home before execution starts.
95
+
94
96
  ## Principles Applied
95
97
 
96
98
  - **YAGNI**: Only plan what's needed now. Flag nice-to-haves separately.
@@ -6,6 +6,7 @@
6
6
  **Date**: [YYYY-MM-DD]
7
7
  **Author**: [Name]
8
8
  **Status**: Draft | In Review | Approved
9
+ **Tracking**: [where task status lives, per the working agreement — e.g. tracker milestone link, the status column below, or none]
9
10
 
10
11
  ## Summary
11
12
  [2-3 sentences: what this feature does and why it matters]
@@ -43,6 +43,13 @@ checkpoint, write down **before starting it**:
43
43
  A checkpoint whose verification you can't name isn't a checkpoint — split or
44
44
  rescope it until it has one.
45
45
 
46
+ Read the project's **working agreement** too (linked from CLAUDE.md or the
47
+ README): it says where task status is recorded, what evidence closes a task, and
48
+ what a session records when it ends. Steps 3 and 6 follow it. When the plan spans
49
+ sessions and no agreement exists, invoke `working-agreement` — it drafts one under
50
+ stated assumptions in the same response, so execution isn't held up — because
51
+ progress recorded only in this session is invisible to the next one.
52
+
46
53
  ### Step 2: Execute a Batch, Logging Drift As You Go
47
54
 
48
55
  Work the checkpoint's tasks. Any deviation from the plan — a different approach,
@@ -50,12 +57,16 @@ an extra task, a skipped step, a surprise — goes into a **drift log** the mome
50
57
  it happens, even when the deviation is an improvement. Undocumented "better
51
58
  ideas" are how a plan dies without anyone deciding to kill it.
52
59
 
60
+ When a task needs a human — a review, an approval, field data — first do every
61
+ part you can do yourself, then hand over a runbook: what is needed, where the
62
+ data is, what to run, what to look for, and what to send back.
63
+
53
64
  ### Step 3: Verify the Checkpoint
54
65
 
55
66
  Run the verification you declared in Step 1 — fresh, in full, reading the entire
56
67
  output (see `verification-before-completion` for the gate function). Only then
57
68
  mark the checkpoint done, recording the evidence next to it: the command and its
58
- observed result. A failed verification means the checkpoint stays open — fix and
69
+ observed result — in the place the working agreement names for status. A failed verification means the checkpoint stays open — fix and
59
70
  re-verify; two consecutive failed checkpoints are a re-planning trigger, not a
60
71
  push-through-it signal.
61
72
 
@@ -147,6 +158,8 @@ in checkpoint 5, and each looked locally complete while the README fell behind
147
158
  both. Take the identifiers the plan touched end to end, grep the docs for them,
148
159
  and reconcile what they name. The plan's own artifacts count: a checkpoint list
149
160
  that still describes the pre-drift design is a stale document like any other.
161
+ So does the status record: close the plan where the working agreement says —
162
+ items closed or moved on with a reason, the plan's header marked closed.
150
163
 
151
164
  ## Rationalizations to Reject
152
165
 
@@ -182,6 +195,7 @@ that still describes the pre-drift design is a stale document like any other.
182
195
  ## Cross-Skill References
183
196
 
184
197
  - `feature-planning` — creates and (on re-plan) revises the plan this skill executes
198
+ - `working-agreement` — where status is recorded and how sessions start, end and close a plan; invoked at Step 1 when the plan spans sessions and none exists
185
199
  - `data-modeling` — where a schema question surfaced mid-execution belongs
186
200
  - `architecture-design` — where a structural or boundary question belongs
187
201
  - `verification-before-completion` — the per-claim evidence gate each checkpoint applies, and the publish gate on every checkpoint commit
@@ -10,7 +10,8 @@
10
10
  "Defines checkpoints with a concrete, named verification (command or observation) for each, declared before executing that batch",
11
11
  "Marks a checkpoint done only after running its verification fresh and recording the evidence (command and observed result)",
12
12
  "Tracks deviations from the plan in a drift log or equivalent explicit record",
13
- "States the per-checkpoint reporting rule it will follow — each checkpoint reported as verified-done with its recorded evidence, or explicitly not done — and rules out vague 'mostly done' states"
13
+ "States the per-checkpoint reporting rule it will follow — each checkpoint reported as verified-done with its recorded evidence, or explicitly not done — and rules out vague 'mostly done' states",
14
+ "States where checkpoint status and evidence will be recorded so a later session can find them (per the project's working agreement, or by establishing one when none exists), rather than leaving progress only in the conversation"
14
15
  ]
15
16
  },
16
17
  {
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: project-documentation
3
- description: "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review."
3
+ description: "Write project docs — README, contributing guides, API docs, changelogs, inline docs — and reconcile existing docs after a change made them wrong. Owns doc drift from the change in front of you; a repo-wide doc-rot audit belongs to technical-debt-review; how the project is run (process, tracking, session routines) → working-agreement."
4
4
  when_to_use: "Triggers: write a README, document this project, contributing guide, changelog, API docs, this needs docs, docstrings, do the docs still match, are the docs still accurate, doc sweep, stale docs, doc drift, I renamed X — what else needs updating."
5
5
  ---
6
6
 
@@ -230,3 +230,4 @@ For code-level documentation (JSDoc, Python docstrings, Go doc comments):
230
230
  - `architecture-documentation` — C4 diagrams and docs-as-code for the architecture docs this skill points at
231
231
  - `verification-before-completion` — the doc-drift and publish gates that read the versioned/internal boundary above
232
232
  - `technical-debt-review` — a repo-wide doc-rot audit, as opposed to the drift from the change in front of you
233
+ - `working-agreement` — owns the process document (where things live, session routines, tracking) and its validation; this skill writes the README and CONTRIBUTING text that link to it, and a sync sweep includes it when a change alters how the project is run
@@ -35,6 +35,9 @@ Read the project's own statements of intent before judging anything:
35
35
  phase does it say it's in, and what does it promise for the milestone in question?
36
36
  - **The hard boundaries** — declared scope and explicit non-goals. These are what
37
37
  you measure scope drift against.
38
+ - **The working agreement** — where the project says status lives and how
39
+ milestones close. If "what's done" can't be answered from where it says, or
40
+ there is no agreement, that is a finding: re-validate it with `working-agreement`.
38
41
 
39
42
  Capture this as the baseline. Everything downstream is "reality vs this baseline."
40
43
 
@@ -0,0 +1,160 @@
1
+ ---
2
+ name: research-methodology
3
+ description: "Run DS/ML research with defensible claims — framework and rigor, data understood before claims, method decision records, claims traced to evidence, sources verified. Production serving → ml-model-deployment."
4
+ when_to_use: "Triggers: research project, thesis, plan the research track, is this finding solid, can I claim this, why this method, literature review, verify these sources, deep research citations."
5
+ ---
6
+
7
+ # Research Methodology
8
+
9
+ Research differs from feature work in one way that changes everything: the
10
+ output is a **claim**, and a claim is only as good as the trail behind it. The
11
+ data was understood, the method was chosen for stated reasons, the experiment
12
+ was defined before its result, the number traces to the code and data that
13
+ produced it, and the sources say what they're cited for.
14
+
15
+ This skill does not impose a structure. The user picks the framework and how
16
+ much rigour the work needs. What it holds fixed are the four rules below,
17
+ because each one, skipped, has produced a claim that had to be withdrawn.
18
+
19
+ ## ⛔ The Iron Law
20
+
21
+ **No claim leaves the project — into a thesis, a report, a decision — without
22
+ its trail: the data behind it understood, the method behind it decided on the
23
+ record, the number traced to what produced it, the sources resolved.**
24
+
25
+ A striking result is the one that most needs the trail. "We'll add it in the
26
+ next revision" is how an unchecked claim survives into the final text.
27
+
28
+ ## Workflow
29
+
30
+ ### Step 1: Agree the Framework and the Rigor
31
+
32
+ Check the project's working agreement for a Methodology section. If there is
33
+ none, offer options with a recommendation — don't impose one:
34
+
35
+ - **CRISP-DM** (the default recommendation): business understanding → data
36
+ understanding → data preparation → modeling → evaluation → deployment, iterated.
37
+ Its data-understanding phase before modeling is the step research most often skips.
38
+ - **TDSP** (Microsoft's Team Data Science Process): similar loop, adds roles and
39
+ standard documents; fits a team with a client.
40
+ - **The user's own**, written down in a few lines.
41
+
42
+ And the **rigour level**: thesis or publication grade (every claim registered,
43
+ sources resolved to their identity), or internal and exploratory (claims marked
44
+ provisional, lighter citation). The user sets it; record it.
45
+
46
+ Write the choice into the working agreement's Methodology section (create the
47
+ agreement via `working-agreement` if the project has none), along with where
48
+ experiments, method decisions and claims live — see
49
+ [templates/traceability.md](templates/traceability.md).
50
+
51
+ **This step is not a gate on delivering.** Plan under the recommended framework
52
+ and state it — *"Planned under CRISP-DM at `<rigour>`; say if you'd rather use
53
+ another"* — then carry the plan in the same response.
54
+
55
+ ### Step 2: Understand the Data Before Any Claim
56
+
57
+ **Invoke `exploratory-data-analysis`** before any result is interpreted, and
58
+ plan it as the first item of the track, not an afterthought. Running a benchmark
59
+ early to smoke-test the pipeline is fine; reading meaning into its numbers before
60
+ this step is not. For research it must cover:
61
+
62
+ - **Which source is authoritative** — the curated table, not the raw one it was
63
+ built from. Read how it is produced (`code-archaeology` for the pipeline).
64
+ - **How reliable each field is** — validated measurement, or informational
65
+ (an estimate, a free-text note, a field only some forms collected).
66
+ - **The distribution of every variable a claim will be split by.** A gap between
67
+ two groups means little if one group has 30 members.
68
+ - **What the reference data covers and doesn't.** A survey that lists some
69
+ households in some places makes every detection elsewhere look like a false
70
+ positive: precision against it is a lower bound for the surveyed area, not a
71
+ property of the method.
72
+
73
+ ### Step 3: Decide Methods on the Record
74
+
75
+ Every choice that shapes a result — algorithm, matching rule, threshold, metric,
76
+ validation design — gets a **method decision record**
77
+ ([templates/method-decision-record.md](templates/method-decision-record.md)):
78
+ the candidates, the evidence for each, the criteria, the decision, and the
79
+ alternatives worth trying later. "DBSCAN is standard" is a reason to list it, not
80
+ to choose it.
81
+
82
+ Include **instrument validity**: was the tool, model or product built and
83
+ evaluated for the variable you're about to report on? A building detector says
84
+ nothing reliable about roof material unless someone measured that. If it wasn't,
85
+ a result about that variable describes the instrument, not the world.
86
+
87
+ ### Step 4: Define Each Experiment Before Its Result
88
+
89
+ Before running: hypotheses, primary metric, operating point, decision rule, and
90
+ which method decisions it rests on. Log and compare runs per
91
+ `ml-experiment-tracking`; choose tests, intervals and multiple-comparison
92
+ handling per `statistical-analysis`. A change after results exist is an
93
+ **amendment**: label it as one, with the reason.
94
+
95
+ ### Step 5: Trace Every Claim
96
+
97
+ Each number that may be quoted goes in a claims register
98
+ ([templates/claims-register.md](templates/claims-register.md)): the claim in
99
+ words, its scope (area, population, product, period), the experiment, the
100
+ commit and data version that produced it, and its status — provisional,
101
+ supported or withdrawn.
102
+
103
+ Write findings at the strength the evidence has:
104
+ - **Measured** — "recall was 0.41 (n = …, 95% CI …) for straw-and-mud roofs
105
+ against 0.78 for chapa, in the surveyed area": state it.
106
+ - **Explained or generalized** — "the product under-detects traditional roofs":
107
+ only after Step 2's distributions, Step 3's instrument check and the
108
+ confounders are done. Until then it is a provisional observation, worded as one.
109
+
110
+ Under deadline, write the supported part now and mark the rest provisional. The
111
+ minimal check a striking subgroup claim needs (group counts, per-group metric
112
+ with intervals, one confounder cross-tab) is usually an hour, not a revision.
113
+
114
+ ### Step 6: Verify Sources
115
+
116
+ Every source that carries a claim passes the identity rule in
117
+ [references/source-verification.md](references/source-verification.md): it
118
+ resolves to a DOI, arXiv ID, ISBN with chapter, standard, or versioned page, and
119
+ the quoted text is in a captured copy, not a search summary. Run this as an
120
+ audit over any deep-research report before its citations are reused. Scale it
121
+ to the rigour set in Step 1, and apply the same test to tools: a library or
122
+ product the method depends on must be maintained and widely used, checked per
123
+ `dependency-management`.
124
+
125
+ ## Rationalizations to Reject
126
+
127
+ | Excuse | Reality |
128
+ |--------|---------|
129
+ | "The result is striking; the EDA can come later" | Striking subgroup results are the likeliest to be small-n noise or skewed labels. The check is small; the retraction isn't. |
130
+ | "We'll add the data description in the next revision" | The claim gets written now and survives into the final text. Write it at the strength the data supports today. |
131
+ | "DBSCAN / U-Net / X is the standard method" | Standard is a reason to list it as a candidate. Record what else was considered and why it lost. |
132
+ | "The product is state of the art" | For what task? A detector evaluated on building presence hasn't been evaluated on roof material. |
133
+ | "Low precision means the method is poor" | Against an incomplete reference, unmatched detections include real objects nobody surveyed. Bound it to the surveyed area. |
134
+ | "The deep-research report already has citations" | Its citations are claims about sources. Resolve each one before reusing it. |
135
+ | "We need the 1977 original for every citation" | Not for established results: a trusted later work that states them is enough. Chase the original when the claim is contested or load-bearing. |
136
+
137
+ ## Red Flags — Stop and Check
138
+
139
+ - A results section exists and no EDA of the labels does.
140
+ - A method appears in code with no record of why it was chosen.
141
+ - A claim is stratified by a variable whose group sizes nobody has looked at.
142
+ - An experiment's decision rule was written after its result.
143
+ - A number in the draft can't be traced to a commit and data version.
144
+ - A citation points at a blog, a summary page, or a search snippet.
145
+
146
+ ## Cross-Skill References
147
+
148
+ - **`exploratory-data-analysis` — invoke it in Step 2, before the first result
149
+ is interpreted.** This skill still decides which claims wait on it; EDA does
150
+ the profiling.
151
+ - `statistical-analysis` — tests, intervals, power and multiple comparisons for
152
+ each experiment.
153
+ - `ml-experiment-tracking` — logging, comparing and reproducing runs.
154
+ - `working-agreement` — holds the Methodology section and the rest of how the
155
+ project runs (where status lives, session routines).
156
+ - `plan-execution` — works through the research plan checkpoint by checkpoint.
157
+ - `code-archaeology` — traces how a curated dataset is produced from raw inputs.
158
+ - `notebook-to-production` and `ml-pipeline-design` — when exploratory code
159
+ becomes a pipeline others rely on.
160
+ - Out of scope: serving a model in production → `ml-model-deployment`.
@@ -0,0 +1,58 @@
1
+ {
2
+ "skill_name": "research-methodology",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "Research project for a master's thesis and an NGO: detect rural households in the Argentine Gran Chaco from free satellite imagery and public building-footprint products (Google Open Buildings, Microsoft, Overture), and validate against the NGO's household survey (about 1,800 households with GPS points, roof material and household size). The repo has a methodology doc, a roadmap with phases A-D, a uv/DVC/MLflow setup and the survey snapshot. Plan the next track: our first experiment benchmarks the existing footprint products against the survey points, clustering footprints into homesteads with DBSCAN, and we also want to say which roof materials the products miss.",
7
+ "expected_output": "Plans the track as research, not as a software feature. Puts understanding the survey data first (an EDA step: distribution of roof material and household size, GPS quality, coverage of the surveyed area, which table is curated) before the benchmark is run or any claim about roof materials is made. Records the method choice (DBSCAN for homestead clustering) as a decision with candidates, reasons and alternatives to try, rather than taking it as given. Pre-registers the experiment's hypotheses, metric and decision rule before results exist. Shows how a later reader traces roadmap phase -> task -> experiment -> code -> result -> claim. Offers a process framework (e.g. CRISP-DM) and asks how much rigour the thesis needs, rather than imposing a rigid structure.",
8
+ "assertions": [
9
+ "Puts an exploratory data analysis of the survey labels (e.g. roof material distribution, household size, GPS quality, survey coverage) before the benchmark results are interpreted or any claim about roof materials is made",
10
+ "Treats the choice of DBSCAN as a method decision to record, with at least one alternative and the reason for choosing it, rather than taking it as given",
11
+ "Pre-registers the experiment: hypotheses, metric and decision rule written before results exist",
12
+ "Shows how a reader traces the work from the roadmap phase to the task, the experiment, the code and the resulting claim",
13
+ "Offers or names a process framework (e.g. CRISP-DM or TDSP) or asks which one to follow, rather than imposing a fixed structure",
14
+ "Questions whether footprint-detection products can support a roof-material claim at all, given they are built to detect buildings rather than classify roofs"
15
+ ]
16
+ },
17
+ {
18
+ "id": 2,
19
+ "prompt": "Experiment A01 is done. Google Open Buildings found 62% of surveyed households overall but only 41% of households with straw-and-mud roofs, versus 78% for corrugated-metal (chapa) roofs. Precision was low, 0.38. Write the findings paragraph for the thesis chapter: the products under-detect traditional roofs, and precision is poor.",
20
+ "expected_output": "Does not write the two findings as established claims yet. Asks or checks what the claims rest on: how roof materials are distributed in the survey (if most roofs are chapa, the straw-and-mud stratum may be small, with wide intervals), whether the detector was ever built or evaluated for roof material, and confounders such as campaign, region or GPS error that correlate with roof type. For precision, notes that the survey covers only surveyed households and areas, so detections outside that coverage are not necessarily false positives, which makes 0.38 a lower bound for the surveyed area, not a property of the product. Offers a provisional wording that states what was measured, with its limits, and lists the checks (EDA, stratum sizes and intervals, instrument validity) that would make it a thesis claim.",
21
+ "assertions": [
22
+ "Does not state 'the products under-detect traditional roofs' as an established finding without first checking the roof-material distribution and stratum sizes in the survey data",
23
+ "Raises whether the detector was built or evaluated for roof material, i.e. whether the instrument is valid for this variable",
24
+ "Raises at least one confounder that could explain the roof-type gap (e.g. campaign, region, settlement density, GPS error)",
25
+ "Points out that precision measured against a survey that does not cover every building in the area treats unsurveyed buildings as false positives, so 0.38 is a lower bound tied to the surveyed area",
26
+ "Offers wording that reports what was measured with its limits, or lists the checks needed before the claim can go into the thesis"
27
+ ]
28
+ },
29
+ {
30
+ "id": 3,
31
+ "prompt": "Our churn model is trained and validated. Help me deploy it as a real-time scoring endpoint behind our API, with monitoring for drift and a safe rollout.",
32
+ "expected_output": "Recognizes this as production model deployment, which belongs to ml-model-deployment, not research methodology. Hands off to or works under ml-model-deployment. Does not start a research-framework, method-decision-record or claims-register process for a deployment task.",
33
+ "assertions": [
34
+ "Recognizes the request as deploying a model to production (ml-model-deployment's domain), not as research methodology",
35
+ "Refers to or proceeds under ml-model-deployment for serving, drift monitoring and rollout",
36
+ "Does not impose research artefacts such as a method decision record, a claims register or a research framework on this request"
37
+ ]
38
+ }
39
+ ],
40
+ "pressure_tests": [
41
+ {
42
+ "id": 1,
43
+ "prompt": "The thesis draft is due to my director on Friday. The benchmark results are in and they look clear: recall 0.62 overall, and the roof finding is striking. There's no time for an EDA now. Just write the results section with the claims; we'll add the data description in the next revision.",
44
+ "pressure": [
45
+ "time",
46
+ "sunk_cost",
47
+ "authority"
48
+ ],
49
+ "expected_behavior": "Does not write unsupported claims into the thesis. Separates what can be stated now (the measured recall in the surveyed area, with its definition and scope) from what cannot (the roof-material finding) until the label distribution and stratum sizes are known. Points out that the minimal EDA needed for the roof claim (a cross-tab of roof class counts and recall by stratum with intervals) is small and fits before Friday. Offers the results section with the supported claim written and the roof finding marked provisional or moved to an exploratory note.",
50
+ "assertions": [
51
+ "Does not write the roof-material finding as an established claim in the results section",
52
+ "Writes or offers the supported part (the measured recall with its scope and definition) so the deadline is still served",
53
+ "Names the minimal data check the roof claim needs (e.g. roof-class counts and recall per stratum with intervals) and says it fits before the deadline",
54
+ "Rejects 'add the data description in the next revision' because a claim drafted without the data check tends to survive into the final text"
55
+ ]
56
+ }
57
+ ]
58
+ }
@@ -0,0 +1,71 @@
1
+ # Source Verification
2
+
3
+ Adapted from article-craft's `references/source-verification.md`
4
+ (swestash/article-craft), cut down for research projects.
5
+
6
+ The rule: **cite only what you opened.**
7
+
8
+ ## Trust tiers
9
+
10
+ - **Tier 1 — cite freely:** peer-reviewed papers and proceedings, standards,
11
+ canonical books cited to chapter or section, primary documentation with a
12
+ stated version or date (dataset cards, product technical reports, API docs).
13
+ - **Tier 2 — cite with attribution:** vendor engineering blogs and release notes
14
+ (primary for the vendor's own product, marketing-adjacent for comparisons);
15
+ public repositories, for what the code actually does; preprints, flagged as
16
+ preprints.
17
+ - **Tier 3 — use to find, never to cite:** community blogs, aggregators, SEO
18
+ pages, forum answers, model-generated summaries (including search snippets and
19
+ WebFetch output). A fact found only in Tier 3 is not established.
20
+
21
+ ## The identity rule
22
+
23
+ Every claim that carries weight must end at something with an identity: a DOI,
24
+ an arXiv ID, an ISBN with chapter or page, a standard's designation, a versioned
25
+ documentation URL, a dated release note, a commit SHA. Anything that cannot
26
+ resolve to an identity is Tier 3, whatever it looks like.
27
+
28
+ This catches the two common failures without having to judge pages:
29
+ - **Citogenesis** — a page repeats a claim and gets cited as if it made it.
30
+ Follow it to the work that established it.
31
+ - **Fabricated references** — "Smith et al. 2019" that doesn't exist fails the
32
+ moment you try to resolve it.
33
+
34
+ ## Capture, then quote
35
+
36
+ A search snippet or WebFetch result is a summary written by a model, not the
37
+ source. Use it to decide what to open. Then capture the page or PDF to a file
38
+ (with its URL, fetch date and any version it states), read it, and quote from
39
+ the file. A quote should be findable in the captured copy.
40
+
41
+ For two-column PDFs, extract in reading order, or quotes will be interleaved
42
+ across columns.
43
+
44
+ ## Proportionality
45
+
46
+ Scale the checking to the claim and the rigour level the project agreed:
47
+
48
+ - **Established results** don't need the original: a 1977 textbook result
49
+ restated in a trusted recent work can be cited through that work. Chase the
50
+ original when the claim is contested, load-bearing for your argument, or the
51
+ later works disagree.
52
+ - **Single studies are evidence, not consensus.** When a key claim rests on one
53
+ paper, check whether later work adopted it, and say so in the text.
54
+ - **Independent corroboration only:** sources that trace to one origin count as one.
55
+ - **Paywalled or unreachable sources:** list them as gaps with what each would
56
+ support. Ask the user for a copy, or cite through a reachable work that
57
+ reports it. Don't cite from the abstract alone as if from the paper.
58
+
59
+ ## Auditing a deep-research report
60
+
61
+ Before reusing a report's citations:
62
+ 1. List each cited claim and the source it points to.
63
+ 2. Resolve each source's identity and tier. Tier 3 → find the primary or drop the claim.
64
+ 3. Spot-check quotes against captured copies, starting with the claims the
65
+ project's argument depends on.
66
+ 4. Record what was verified, what is a preprint, and what remains a gap.
67
+
68
+ ## When verification fails
69
+
70
+ Cut the claim, or narrow it to what the evidence supports. A weaker claim that
71
+ holds is worth more than a strong one that doesn't.
@@ -0,0 +1,19 @@
1
+ # Claims Register Template
2
+
3
+ Every number or finding that may be quoted outside the project (thesis, report,
4
+ funding request) gets a row. Save as `docs/claims.md`.
5
+
6
+ ```markdown
7
+ # Claims
8
+
9
+ A claim may be quoted only at its status. Provisional claims are quoted with
10
+ their caveat or not at all.
11
+
12
+ | ID | Claim, in words | Scope (area, population, product, period) | Evidence (experiment, commit, data version) | Checks done | Status |
13
+ |---|---|---|---|---|---|
14
+ | C-001 | [e.g. "Open Buildings matched 62% of surveyed households (95% CI …)"] | [surveyed parajes, department X, survey 2019–2023] | [A01, abc1234, data v3] | [EDA AE01; n per group; CI method] | Supported |
15
+ | C-002 | [e.g. "Detection is lower for straw-and-mud roofs"] | | | [missing: roof-class counts, instrument validity, confounders] | Provisional |
16
+
17
+ Status values: **Provisional** (measured, checks incomplete) · **Supported**
18
+ (checks done) · **Withdrawn** (with the reason and date).
19
+ ```
@@ -0,0 +1,41 @@
1
+ # Method Decision Record Template
2
+
3
+ One per method choice that shapes a result. Like an ADR, but the evidence is
4
+ data and literature rather than system constraints. Save as
5
+ `docs/methods/MDR-NNNN-<topic>.md`.
6
+
7
+ ```markdown
8
+ # MDR-NNNN: [the question, e.g. "How to group building footprints into homesteads"]
9
+
10
+ **Status:** Proposed | Accepted | Superseded by MDR-NNNN
11
+ **Date:** YYYY-MM-DD
12
+ **Used by:** [phase / work package / experiments]
13
+
14
+ ## Question
15
+ [What has to be decided, and which results depend on it.]
16
+
17
+ ## Candidates
18
+ | Candidate | How it works (one line) | Evidence for | Evidence against | Sources |
19
+ |---|---|---|---|---|
20
+ | [e.g. DBSCAN] | | | | [DOI / arXiv] |
21
+ | [e.g. single-linkage with distance cut] | | | | |
22
+ | [e.g. alpha shapes per cluster] | | | | |
23
+
24
+ ## Instrument validity
25
+ [Was each candidate (or the product/model it relies on) built and evaluated for
26
+ this variable and this setting? If not, what does that limit?]
27
+
28
+ ## Criteria
29
+ [How candidates are compared: accuracy on what, cost, interpretability,
30
+ parameter sensitivity. Set before comparing.]
31
+
32
+ ## Decision
33
+ [The choice and the reason, in a few sentences. Parameters and how they were set.]
34
+
35
+ ## Alternatives to try later
36
+ [Candidates not chosen that would be worth testing if the result is weak, with
37
+ what would trigger trying them.]
38
+
39
+ ## Consequences
40
+ [What this decision commits the project to; which claims inherit its limits.]
41
+ ```
@@ -0,0 +1,50 @@
1
+ # Traceability Template
2
+
3
+ The chain a reader follows from "why are we doing this" to "which code produced
4
+ this number", and back. Add it to the working agreement's Methodology section, or
5
+ as `docs/research-workflow.md` linked from there. Rename the paths to the project's.
6
+
7
+ ```markdown
8
+ ## The chain
9
+
10
+ methodology WHY + WHAT: framework, phases, gates between phases
11
+ │
12
+ roadmap WHEN: phases and milestones, the gate each one must pass
13
+ │
14
+ phase plan TASKS: work packages → acceptance, experiment, status
15
+ │ (status lives where the working agreement says)
16
+ method decisions WHY THIS METHOD: candidates, evidence, decision, alternatives
17
+ │
18
+ experiment README ONE QUESTION: hypotheses and decision rule written before
19
+ │ the run; results; decision; claims produced; how to reproduce
20
+ ├── notebook where we look and explain: plots, tables, interpretation
21
+ └── src module tested code for anything reused or reported
22
+ │
23
+ pipeline config + lock what exactly ran: data hashes, code, parameters
24
+ │
25
+ metrics / reports results (aggregates only if the data is sensitive)
26
+ │
27
+ claims register numbers we may quote, each linked to its experiment and status
28
+
29
+ ## Which file answers what
30
+
31
+ | Question | Open |
32
+ |---|---|
33
+ | Why this approach, what are the phases? | methodology |
34
+ | Which phase are we in, what's next? | roadmap → phase plan |
35
+ | Why this method, what else could we try? | method decisions |
36
+ | What did we look at in the data? | the EDA experiment and its notebook |
37
+ | What was tested, found and decided? | the experiment README |
38
+ | How do I re-run it? | the experiment README's Reproduce section |
39
+ | Can I quote this number? | claims register |
40
+ | Where does this dataset come from, under what licence? | the data inventory / source registry |
41
+
42
+ ## Notebooks and code
43
+
44
+ - **Notebooks** are where we look and explain. They read pipeline outputs and
45
+ import from the package. Commit them in a diff-friendly form (e.g. jupytext).
46
+ - **The package** holds anything reused or reported, with tests. When a notebook
47
+ cell becomes something we rely on, it moves into a module.
48
+ - **The pipeline** runs both, so a rendered notebook always matches the data and
49
+ code it was produced from.
50
+ ```
@@ -94,6 +94,7 @@ installed skill by name, regardless of role.
94
94
  ### Plan & Define
95
95
  - **brainstorming** — divergent Socratic ideation *before* a spec exists; opens options, then hands off
96
96
  - **feature-planning** — break a feature into scoped tasks, acceptance criteria, dependencies
97
+ - **working-agreement** — define and validate how a multi-session project is run: where status lives, session routines, new work, milestone close
97
98
  - **build-vs-buy** — build in-house vs vendor/SaaS vs adopt OSS: TCO, lock-in, exit costs
98
99
  - **prd-writing** — write a PRD / RFC / tech spec to align on the WHAT and WHY
99
100
  - **project-proposal** — business case / budget / go-no-go before a project starts
@@ -175,6 +176,7 @@ installed skill by name, regardless of role.
175
176
  - **data-quality** — dbt tests, expectations, data contracts, freshness, lineage / blast radius
176
177
 
177
178
  ### Data Science
179
+ - **research-methodology** — research with defensible claims: framework and rigor, data understood first, method decisions, claims traced, sources verified
178
180
  - **exploratory-data-analysis** — profile an unfamiliar dataset: missingness structure, distributions, leakage, hypotheses
179
181
  - **statistical-analysis** — hypothesis tests, experiment design/power, confidence intervals, multiple-comparison discipline
180
182
  - **notebook-to-production** — refactor analysis notebooks to tested, parameterized, scheduled production code
@@ -200,6 +202,13 @@ leads instead, and `feature-planning` then scopes the work that the chosen optio
200
202
  implies. Planning tasks against an unmade decision produces a plan you throw away.
201
203
  Either way both skills are in the chain; what changes is which one runs first.
202
204
 
205
+ **New project / multi-session work**
206
+ `prd-writing` or `feature-planning` (the first plan that spans sessions) →
207
+ `working-agreement` (how the project is run, validated by a fresh reader) →
208
+ `plan-execution` (records status where the agreement says). In a project that
209
+ already runs without one — nobody can say what's done — `working-agreement`
210
+ leads, reconstructing the current process first.
211
+
203
212
  **Bug / incident**
204
213
  `incident-response` (if prod is down) → `bug-investigating` → `tdd-workflow`
205
214
  (regression test) → `verification-before-completion` (the repro now passes, and
@@ -231,6 +240,13 @@ Notebook to production: `exploratory-data-analysis` (understand the data) →
231
240
  (modules, tests, scheduling); if the notebook trains a model →
232
241
  `ml-pipeline-design` instead.
233
242
 
243
+ **Research / data-science project**
244
+ `research-methodology` leads: it agrees the framework (CRISP-DM by default, or
245
+ the user's own) and the rigor, and the framework's phases set the order — not a
246
+ fixed chain. What holds under any framework: `exploratory-data-analysis` before
247
+ any result is interpreted, `statistical-analysis` and `ml-experiment-tracking`
248
+ per experiment, `plan-execution` for the work itself.
249
+
234
250
  **Pre-public / pre-milestone review**
235
251
  `strategic-review` (vision, positioning, market) → `project-review` (scope,
236
252
  roadmap, implementation, evidence) → synthesis → `artifact-design` (interactive
@@ -0,0 +1,185 @@
1
+ ---
2
+ name: working-agreement
3
+ description: "Define and validate how a multi-session project is run — where plans, status and decisions live, session routines, new work, milestone close. README/CONTRIBUTING → project-documentation."
4
+ when_to_use: "Triggers: where is progress tracked, how do we track this, is this milestone done, how is this project run, document our process, working agreement, definition of done, plans and status out of sync."
5
+ ---
6
+
7
+ # Working Agreement
8
+
9
+ A working agreement says how a project runs, so that humans and agents read
10
+ and write the same places: where the plan lives, where status lives, what a
11
+ session does when it starts and ends, how new work gets in, how a milestone
12
+ closes. This skill does not choose the process for the user. It makes sure one
13
+ is chosen, written down where every later session finds it, and **checked by
14
+ someone who did not write it**.
15
+
16
+ Without one, each session improvises: progress lands in a handoff nobody
17
+ commits, a report doubles as a tracker once, and work starts on a milestone
18
+ that has no plan. Nobody decides to drift; there was nothing to drift from.
19
+
20
+ ## ⛔ The Iron Law
21
+
22
+ **An agreement is not in force until a fresh reader can answer every core
23
+ scenario from the documents alone.**
24
+
25
+ The author cannot test it: the author fills every gap from memory without
26
+ noticing. The content of the agreement is the user's choice; the validation is
27
+ not optional.
28
+
29
+ ## Workflow
30
+
31
+ ### Step 1: Find the Agreement, Then Pick the Mode
32
+
33
+ Look for an agreement on record: a pointer in CLAUDE.md, the README or
34
+ CONTRIBUTING; a `docs/how-we-work.md` or similar; tracker conventions.
35
+
36
+ - **Found** → read it and follow it. Re-validate (Step 5) only when a trigger
37
+ fires: a milestone closes, the process changed, or someone asked a question
38
+ the agreement should have answered.
39
+ - **Not found, small work** (one session, one person, nothing to hand over) →
40
+ skip this skill; say so in one line.
41
+ - **Not found, large work** (spans sessions or milestones, or several people or
42
+ agents) → pick the mode:
43
+ - **Define (greenfield)** — nothing practiced yet. Go to Step 2.
44
+ - **Discover (brownfield)** — practice exists: plans, reports, issues, git
45
+ log, handoffs. First write the **as-is** from that evidence: where each kind
46
+ of fact lives today, and where it cannot be found. Name each gap concretely
47
+ ("the plan has no per-task status; the report was written once at close;
48
+ progress lives in gitignored handoffs"). That list is what Step 3 fixes.
49
+ Answer the user's immediate question (is M0 done?) from the same evidence,
50
+ or say exactly what would have to be checked.
51
+
52
+ **Exit condition** — one line, then keep going in the same response:
53
+ *"No agreement on record; `<define | discover>`: `<one-line state of practice>`."*
54
+
55
+ **This step is not a gate on delivering.** When you can't read the repo, state
56
+ what you assume exists and continue. Carry the options, the draft agreement and
57
+ the validation in the same response; questions go next to the draft, not in
58
+ place of it.
59
+
60
+ ### Step 2: Offer Options and Recommend One
61
+
62
+ Two decisions, each offered as options with a recommendation fitted to the
63
+ project — see [references/options.md](references/options.md):
64
+
65
+ 1. **Process shape** — milestone plans, Kanban, Scrum, or for research work a
66
+ methodology framework (hand that section to `research-methodology`).
67
+ 2. **Where status lives** — an external tracker (GitHub/GitLab issues, Jira,
68
+ Linear) when one exists or can be connected; a status column in each plan
69
+ when there is no tool; or no tracking for short single-person work. A planned
70
+ switch ("markdown until the GitLab remote exists") is a valid choice: record
71
+ the trigger for the switch.
72
+
73
+ Don't make the draft wait on the answer: draft under the recommendation and say
74
+ *"Drafted under `<choice>` — pick another and I'll revise."*
75
+
76
+ ### Step 3: Write the Agreement
77
+
78
+ Use [templates/agreement.md](templates/agreement.md). Every agreement covers:
79
+
80
+ - **Where things live** — one owner per kind of fact (scope, plan, task status,
81
+ decisions, results, session context), with when each changes. Other places
82
+ link to the owner; they never copy it.
83
+ - **Starting a session** — where to read what to do next, and in what order.
84
+ - **During the work** — where decisions, drift and new findings go.
85
+ - **Ending a session** — where progress and evidence are recorded, what a
86
+ handoff may and may not contain.
87
+ - **New work** — how a new milestone, an added task, a bug and an open question
88
+ each enter.
89
+ - **Definition of Done** — what evidence closes a task.
90
+ - **Closing a milestone** — what is checked, what is written, what moves on.
91
+ - **Asking a human** — requests for human work come as runbooks: what is needed,
92
+ where the data is, what to run, what to look for, what to send back.
93
+
94
+ Rules that keep it honest:
95
+
96
+ - **Status lives in one place.** A plan or report that also carries status is a
97
+ second copy, and the second copy goes stale.
98
+ - **Handoffs are context, not status.** Nobody should need a handoff to know
99
+ what is done.
100
+ - **Versioned documents only use terms a reader can resolve** — no labels from a
101
+ gitignored plan or a chat.
102
+ - **Right-size it.** One page for a two-person project is enough.
103
+
104
+ ### Step 4: Prove the Tracker Connection
105
+
106
+ If status lives in an external tool, prove the agent can **read and write** it
107
+ before the agreement depends on it: list the open items, and create, update or
108
+ comment on one. A tool that is configured but not loaded in this session is not
109
+ connected. If it can't be proven now, the agreement names the fallback (a CLI,
110
+ or the human updates the tracker from a list the agent writes) and the fallback
111
+ is what's in force.
112
+
113
+ ### Step 5: Validate With a Fresh Reader
114
+
115
+ Walk the scenario bank in [references/validation.md](references/validation.md)
116
+ against the written agreement, **in the same response as the draft**. Don't
117
+ promise the check for after the user answers your questions: the gaps it finds
118
+ are part of what they are answering.
119
+
120
+ - **Preferred:** a subagent that sees only the repo's documents, not this
121
+ conversation, answers each scenario twice: as a new agent session and as a new
122
+ human contributor. Each answer must cite one file and section.
123
+ - **Fallback** (no subagent available, or no repo yet): walk the scenarios
124
+ yourself, in the reply, and label it a self-check — weaker, because you wrote
125
+ it.
126
+ - Add the coverage check for the chosen process shape and, when the project
127
+ already has docs, the conformance check: do they follow the agreement's own
128
+ rules (same reference)?
129
+
130
+ Score each scenario: **answered in one place**, **ambiguous** (two places, or
131
+ two readings), or **missing**. Every ambiguous or missing scenario becomes a
132
+ question for the user or a fix to the draft. Repeat until the core scenarios
133
+ are all answered in one place.
134
+
135
+ **Exit condition:** *"Validated: `<n>/<n>` core scenarios answered from the
136
+ documents. Open: `<questions>`."*
137
+
138
+ ### Step 6: Wire It In and Keep It Alive
139
+
140
+ - Put a pointer in CLAUDE.md (agents read it every session) and the README
141
+ (humans): *"`docs/how-we-work.md` is the process; follow it."* Both the
142
+ agreement and the pointer must be **tracked** files: if CLAUDE.md is
143
+ gitignored, a collaborator's assistant never sees it — point from the README or
144
+ a committed AGENTS.md, and keep only personal state in untracked files.
145
+ - **If the agreement and practice disagree, fix the agreement first**, in its
146
+ own commit, then act.
147
+ - Re-validate at milestone close, after a retrospective changes the process,
148
+ during `project-review`, and whenever a question the agreement should answer
149
+ had to be asked.
150
+
151
+ ## Rationalizations to Reject
152
+
153
+ | Excuse | Reality |
154
+ |--------|---------|
155
+ | "It looks complete; we'll fix gaps as we hit them" | A gap is hit by the person who needed the answer, at that moment, after the drift has started. Five scenarios take minutes now. |
156
+ | "I wrote it, I can tell it's clear" | The author fills gaps from memory. That is why the reader must be fresh. |
157
+ | "The plan already has a Status field" | A plan's lifecycle (Draft, Active, Closed) is not task status. Where does task 14's status live, and who updates it? |
158
+ | "The handoff says what was done" | Handoffs are uncommitted and per-session. Status nobody else can see isn't status. |
159
+ | "We use GitLab, so tracking is covered" | Choosing a tool isn't an agreement. Which facts live there, which in docs, who updates what, when? |
160
+ | "The MCP server is configured" | Configured isn't connected. Prove one read and one write in this session. |
161
+ | "This is project management overhead for a small project" | Then the agreement is one page. Skipping it is how a small project loses track of what's done. |
162
+
163
+ ## Red Flags — Stop and Check
164
+
165
+ - A session starts work and can't name which plan item it advances.
166
+ - The same task's status appears in two documents.
167
+ - "Is X done?" can only be answered by reading git log or a handoff.
168
+ - Work began on a milestone that has no plan.
169
+ - The agreement names a tool that no session has written to.
170
+
171
+ ## Cross-Skill References
172
+
173
+ - `plan-execution` — follows the agreement: reads it at Step 1, records status
174
+ where it says, closes out by its rules. When a plan spans sessions and no
175
+ agreement exists, it comes here first.
176
+ - `feature-planning` and `prd-writing` — hand off here when the first plan or
177
+ PRD that spans sessions is written.
178
+ - `research-methodology` — writes the agreement's Methodology section for
179
+ research and data-science projects.
180
+ - `project-documentation` — writes the README and CONTRIBUTING text that link to
181
+ the agreement, and includes the agreement in its sync sweep. The agreement's
182
+ content and validation stay here.
183
+ - `retrospective` — process changes agreed in a retro come back here as edits to
184
+ the agreement, followed by re-validation.
185
+ - `project-review` — re-validates the agreement as part of an execution review.
@@ -0,0 +1,60 @@
1
+ {
2
+ "skill_name": "working-agreement",
3
+ "evals": [
4
+ {
5
+ "id": 1,
6
+ "prompt": "We're starting a new open-source app for an NGO: offline survey collection on Android, a Python sync server and a web admin. The PRD is written (docs/product/prd-mvp.md) and the roadmap splits the work into milestones M0-M3 (docs/plans/roadmap.md). Two volunteers will build it over several months, mostly in sessions with Claude Code, with no fixed deadlines. There's no Git remote yet; we'll probably use GitLab later. Before we write the first milestone plan, set up how this project will be run so every later session, human or agent, knows what to do.",
7
+ "expected_output": "Treats this as defining the project's working agreement (greenfield). Offers recommended options rather than imposing one: a process shape (milestone plans, Kanban, Scrum) and a tracking choice (external tracker such as GitLab issues once the remote exists, a status table in each plan until then, or none), with a recommendation. Drafts the agreement itself in the same response: where each kind of fact lives with one owner per fact, what to do at session start, during and at session end, how new work enters, a Definition of Done, how a milestone closes, how decisions are recorded. Then validates the draft by walking concrete scenarios against it and lists the gaps or ambiguities as questions. Says the agreement gets a pointer from CLAUDE.md or the README so later sessions find it.",
8
+ "assertions": [
9
+ "Offers at least two options for how progress is tracked (e.g. an external tracker such as GitLab issues, a status table in the plan files, or no tracking), with a recommendation, rather than imposing a single scheme without asking",
10
+ "Drafts the agreement itself in the same response rather than only asking questions or promising to write it later",
11
+ "The draft states where each kind of information lives (plans, task status, decisions, results, session context), with one owner for each kind",
12
+ "The draft gives a routine for starting a session and for ending one: where to read what to do next, and where to record progress",
13
+ "The draft says how new or unplanned work is added and how a milestone is closed",
14
+ "Validates the draft by walking concrete scenarios against it (for example a new session starting, a task blocked on a human, a decision changing) and reports the gaps or ambiguities found as questions, rather than declaring the agreement complete",
15
+ "Says the agreement is linked from CLAUDE.md, the README or an equivalent entry point so later sessions find it"
16
+ ]
17
+ },
18
+ {
19
+ "id": 2,
20
+ "prompt": "i can't find the place where the project execution is being tracked. i see the plan files in docs/plans/ but can't easily find what has been done of those so far. it seems there's a gap between the plan and the report. or is the report m0-spike-report meant to be a milestone task tracking log too? i assumed that we could monitor project status this way PRD -> roadmap -> Milestone Plan -> <task progress in new file or section in plan file?> -> reports (if needed). am i missing something? is there a better way to track it? cause i can't fully understand if M0 is done or if planning M1 should come after other M0 pending tasks.\n\nWhat's in the repo:\n- docs/plans/README.md: index of plans\n- docs/plans/roadmap.md: Status: Draft; milestones M0-M3\n- docs/plans/m0-spikes.md: the M0 plan, header 'Status: In progress', task cards with acceptance criteria, no per-task status\n- docs/plans/m0-spike-report.md: written when M0 closed; a 'Status by task' table (Done / Not done with evidence)\n- .local/handoffs/ (gitignored): one handoff file per session, listing what was done and candidate next tasks\n- git log: sessions 6 and 7 built the form engine and the web skeleton, which belong to M1; there is no M1 plan yet\n- no issue tracker, no Git remote",
21
+ "expected_output": "Reconstructs the as-is process from the listed artifacts (brownfield discovery): the plan has no live status, the report doubles as a tracker but only once at close, the handoffs hold progress but are gitignored and invisible to humans, and M1 work started with no M1 plan. Confirms the user's chain is missing its progress step. Answers whether M0 is done from the evidence available, saying what is unknown. Proposes the working agreement: options for where progress lives with a recommendation, the routines that keep it current, and that handoffs are context rather than status. Validates the proposal against the user's own questions as scenarios.",
22
+ "assertions": [
23
+ "Identifies from the listed artifacts where progress currently lives and why it cannot be followed: the plan carries no per-task status, the report was written once at close, and the handoffs that record progress are gitignored",
24
+ "Notices that M1 work (form engine, web skeleton) started before any M1 plan existed",
25
+ "Confirms the chain the user describes is missing a live progress step, rather than saying the current setup already covers it",
26
+ "Offers options for where task progress should live (e.g. a status column in each plan, an external tracker, one status file) with a recommendation",
27
+ "States that session handoffs are context, not the record of status",
28
+ "Gives routines that keep status current: what to update at session end or checkpoint, and what to do when a milestone closes",
29
+ "Answers whether M0 is done from the evidence given, or says exactly what would have to be checked to answer it"
30
+ ]
31
+ },
32
+ {
33
+ "id": 3,
34
+ "prompt": "Write a CONTRIBUTING.md for our open-source CLI: how to set up the dev environment (Node 22, pnpm), run the tests, our conventional-commit rule, and how to open a PR.",
35
+ "expected_output": "Recognizes this as contributor documentation, which belongs to project-documentation, not as defining or validating how the project is run. Writes or hands off to project-documentation for the guide. Does not start a working-agreement interview, tracking-options menu or scenario validation for a request that already states its content.",
36
+ "assertions": [
37
+ "Treats the request as writing contributor documentation (project-documentation's domain), not as defining the project's working agreement",
38
+ "Produces the CONTRIBUTING content or hands off to project-documentation, without first asking how the project tracks its work",
39
+ "Does not offer a menu of process or tracking options, or run a scenario validation, for this request"
40
+ ]
41
+ }
42
+ ],
43
+ "pressure_tests": [
44
+ {
45
+ "id": 1,
46
+ "prompt": "The how-we-work doc is written: it says GitLab issues hold task status, plans hold scope, and handoffs in .local/ hold session context. It looks complete to me. Skip the validation walkthrough, we'll fix gaps as we hit them; I want to start M1 today.",
47
+ "pressure": [
48
+ "time",
49
+ "authority"
50
+ ],
51
+ "expected_behavior": "Holds the validation as the step that puts the agreement in force, while keeping it short enough not to block M1. Walks the core scenarios against the three rules as written (session start: where do I pick a task and in what order; session end: who closes the issue and with what evidence; new work mid-milestone; a decision that changes; milestone close) and shows the gaps they expose, such as no rule for issue/plan overlap, no Definition of Done, no close-out routine. Explains that gaps found while working are found by the person who needed the answer, which is the failure being prevented. Offers to run it now in minutes and start M1 right after.",
52
+ "assertions": [
53
+ "Does not skip the validation; runs it, or a short version of it, in this response",
54
+ "Walks concrete scenarios against the three stated rules and names at least two specific gaps they expose (e.g. how a session picks the next task, what evidence closes an issue, where new work goes, how a milestone closes)",
55
+ "Explains why fixing gaps as you hit them fails: the gap surfaces at the moment someone needs the answer, after drift has started",
56
+ "Keeps M1 unblocked: proposes finishing the check quickly and starting M1 the same day rather than delaying it"
57
+ ]
58
+ }
59
+ ]
60
+ }
@@ -0,0 +1,37 @@
1
+ # Options: process shape and where status lives
2
+
3
+ Offer two or three options with a recommendation fitted to the project. None of
4
+ these is the right answer in general; each has a situation it fits.
5
+
6
+ ## Process shape
7
+
8
+ | Shape | Fits when | What the agreement must then define |
9
+ |---|---|---|
10
+ | **Milestone plans** | Small team or volunteers, AI-assisted sessions, no fixed cadence; work comes in large known chunks (a PRD split into milestones) | Plan lifecycle (Draft → Active → Closed), task index per plan, how a task is picked, milestone close-out and report |
11
+ | **Kanban** | Continuous flow of mixed-size work, no natural batches, interruptions are normal (maintenance, support, ops) | Board columns and what moves a card, WIP limit, pull rule, explicit policy per column, how a card is closed |
12
+ | **Scrum** | A team of about 3+ with a regular cadence and a stakeholder who reviews increments | Sprint length, backlog owner, sprint goal, Definition of Done, review and retro, who plays product owner |
13
+ | **Research framework** (CRISP-DM, TDSP, own) | The work answers questions with data or models; the plan changes with what is found | Hand the Methodology section to `research-methodology`; the rest of the agreement (where things live, sessions, status) still applies |
14
+
15
+ Combinations are normal: milestone plans with a Kanban board inside each
16
+ milestone; a research framework with milestone plans per phase. Name the
17
+ combination; don't leave it implied.
18
+
19
+ ## Where task status lives
20
+
21
+ | Option | Fits when | Rules it brings |
22
+ |---|---|---|
23
+ | **External tracker** (GitHub/GitLab issues and milestones, Jira, Linear, Asana, Trello) | One exists, or the repo has a remote that offers one; several people; stakeholders who don't read the repo | Status lives only there. Plans keep scope and a task index that links each task to its item. Say which copy owns a task's details once it becomes an item (usually the item). Prove the agent can read and write it (SKILL.md Step 4) |
24
+ | **Status column in each plan** (markdown in the repo) | No tool, no remote yet, or the team prefers everything in git; cheap to share between agents and humans | One status column (and an evidence column) in the plan's task table, updated at each checkpoint in the same commit as the work. Reports record results at close and carry no live status |
25
+ | **One status file** (`docs/plans/status.md`) | Several plans run at once and a single board is easier to scan | Plans stay static; the status file is the only place status changes. Say what happens to it when a tracker arrives |
26
+ | **None** | One person, short-lived work, nothing to hand over | Say so in the agreement, so a later session doesn't invent a tracker |
27
+
28
+ **A planned switch is a valid option** ("status column now, GitLab issues once the
29
+ remote exists"). Record the trigger and what migrates, so the switch doesn't leave
30
+ two live copies.
31
+
32
+ ## Recommending
33
+
34
+ Fit the recommendation to the facts given: team size, cadence, whether a remote
35
+ or tracker exists, who needs to read status without opening the repo. Say why in
36
+ one sentence. When the user names a tool they already use, that is the
37
+ recommendation unless something rules it out.
@@ -0,0 +1,83 @@
1
+ # Validating a working agreement
2
+
3
+ The check: can someone who did not write the agreement do the project's normal
4
+ work using only the documents? Run it in the same response as the draft. Don't
5
+ wait for the user's answers to the open questions first; the gaps it finds are
6
+ part of what the user is answering.
7
+
8
+ ## Core scenarios
9
+
10
+ Each must be answerable from the documents, in **one** place.
11
+
12
+ | # | Scenario | What a full answer names |
13
+ |---|---|---|
14
+ | 1 | A new session starts. What do I work on next? | Where to find the current milestone or board, how to pick an item, what to read before starting |
15
+ | 2 | I finished a task. How do I mark it done? | Where status lives, what evidence closes it (Definition of Done), who updates it |
16
+ | 3 | The session ends with work half done. What do I record, and where? | Status update location, what goes in a handoff and what doesn't |
17
+ | 4 | I found work the plan doesn't contain. | Where a new task, a bug, a follow-up and a question each go |
18
+ | 5 | A decision changes (a library, a scope cut, a method). | Where decisions are recorded, how the plan and docs follow |
19
+ | 6 | A task needs something only a human can do. | How the request is written (runbook: what, where, run what, look for what, return what) and where it waits, visible to everyone (not in a private handoff) |
20
+ | 7 | Is milestone X done? | Where to look, and what "done" means for a milestone |
21
+ | 8 | The milestone closes. | The close-out steps, what is written, what moves on |
22
+ | 9 | A new human contributor joins. | The entry point (README → agreement), what to read in what order |
23
+ | 10 | Two documents disagree. | Which one owns that fact; the rule that the agreement is fixed first |
24
+
25
+ Add project-specific scenarios when the project has them, e.g. "can I quote this
26
+ number in the thesis?" (research) or "a stakeholder asks for a status report".
27
+
28
+ ## Scoring
29
+
30
+ - **Answered** — one document and section gives the answer.
31
+ - **Ambiguous** — two places answer it, or the text allows two readings.
32
+ - **Missing** — no document answers it.
33
+
34
+ Every ambiguous or missing scenario becomes either a fix to the draft or a
35
+ question for the user, written next to the draft.
36
+
37
+ ## Conformance check (brownfield, and every re-validation)
38
+
39
+ The scenarios test what the agreement says; this tests whether the project does
40
+ it. Both real-project pilots that shaped this skill found the docs breaking the
41
+ agreement's own rules. Check:
42
+
43
+ - **Status in only one place** — reports, plans, roadmaps or findings tables that
44
+ still carry live status the agreement assigns elsewhere.
45
+ - **One definition each** — Definition of Done, status values, lifecycle values:
46
+ an older copy left in another doc, or a value in use that the agreement never
47
+ defines.
48
+ - **Entry points are tracked** — the reading order an agent or human is told to
49
+ follow must not live in a gitignored file.
50
+ - **Names match the rules** — file and ID patterns the agreement prescribes
51
+ versus what exists.
52
+
53
+ Each violation is fixed in the docs or written into the agreement as an
54
+ explicit exception.
55
+
56
+ ## Coverage check by process shape
57
+
58
+ Light, not a ceremony audit: does the agreement say how each of these works here?
59
+
60
+ - **Milestone plans:** plan lifecycle, task index, how a task is picked, changes
61
+ during execution, milestone report, close-out.
62
+ - **Kanban:** columns and what moves a card, WIP limit, pull rule, policy per
63
+ column, blocked items.
64
+ - **Scrum:** backlog and its owner, sprint length and goal, Definition of Done,
65
+ review, retro, product owner.
66
+ - **Research framework:** phases and their exit criteria, where method decisions
67
+ and claims live (from `research-methodology`).
68
+
69
+ ## Fresh-reader brief (for a subagent)
70
+
71
+ > You are joining this project for the first time. Read only the files in this
72
+ > repository; you have no other context. For each scenario below, answer as (a) a
73
+ > new AI-assistant session and (b) a new human contributor. For each answer, cite
74
+ > the file and section it comes from. If no document answers it, say MISSING. If
75
+ > two places answer it differently, or the text could be read two ways, say
76
+ > AMBIGUOUS and quote both. Do not guess and do not fill gaps from general
77
+ > practice.
78
+ >
79
+ > Scenarios: [paste the core scenarios plus project-specific ones]
80
+
81
+ Give the subagent the repository, not this conversation. When no subagent is
82
+ available, run the same brief yourself, label the result "self-check by the
83
+ author", and treat any answer you could only give from memory as MISSING.
@@ -0,0 +1,87 @@
1
+ # Working Agreement Template
2
+
3
+ Save as `docs/how-we-work.md` (or wherever the project keeps process docs) and
4
+ link it from CLAUDE.md and the README. Delete sections that don't apply; one
5
+ page is enough for a small project.
6
+
7
+ ```markdown
8
+ # How we work
9
+
10
+ [Who builds this, how (sessions with an AI assistant? a team with a cadence?),
11
+ and any constraint that shapes the process: volunteers, no deadlines, a thesis.]
12
+
13
+ If this document and practice disagree, fix this document first, in its own commit.
14
+
15
+ ## Process
16
+
17
+ [The chosen shape — milestone plans / Kanban / Scrum / a research framework —
18
+ in two or three sentences, and why it fits.]
19
+
20
+ ## Where things live
21
+
22
+ Each fact has one owner. Other places link to it instead of copying it.
23
+
24
+ | Information | Lives in | Changed when |
25
+ |---|---|---|
26
+ | What we build and why | [PRD path] | Product scope changes |
27
+ | How it's built; decisions | [RFC / ADR path] | A decision is made (a changed decision gets a new record) |
28
+ | Order of milestones | [roadmap path] | A milestone starts or closes |
29
+ | One milestone: scope, acceptance criteria, task index | [plan path pattern] | The plan is written; scope changes |
30
+ | One task: details, status, evidence | [tracker issue / plan status column] | Any time; it is the working copy |
31
+ | Results of a milestone | [report path pattern] | Once, when the milestone closes |
32
+ | Session context: what was tried, next step | [handoff location, never committed] | End of each session |
33
+
34
+ Rules:
35
+ - Status is written in one place only: [where].
36
+ - Handoffs are context, not status. Nobody needs a handoff to know what is done.
37
+ - Versioned documents only use terms defined in versioned documents.
38
+
39
+ ## Tracker
40
+
41
+ [Only if an external tracker is used: project link, labels, milestones, how an
42
+ item references the plan, how the agent reaches it (MCP / CLI) and the fallback
43
+ when it can't. If the tracker will change later, the trigger for the switch.]
44
+
45
+ ## Starting a session
46
+
47
+ 1. [Where to find the current milestone and pick the next item, e.g. "open the
48
+ milestone, take an item whose dependencies are closed".]
49
+ 2. [What to read before starting: the item, the decisions it cites, the latest handoff.]
50
+
51
+ ## During the work
52
+
53
+ - A decision → [where it is recorded].
54
+ - Drift from the plan → [where it is logged; when it triggers re-planning].
55
+ - New work found → see "New work".
56
+ - Work only a human can do → a runbook in [where]: what is needed, where the
57
+ data is, what to run, what to look for, what to send back.
58
+
59
+ ## Ending a session
60
+
61
+ 1. [Update status where it lives, with the evidence that closes each item.]
62
+ 2. [Update docs only when the session changed what they describe, in the same commit.]
63
+ 3. [Write the handoff: context only.]
64
+
65
+ ## New work
66
+
67
+ | Kind | Goes to |
68
+ |---|---|
69
+ | A new milestone | [plan first, then tracker] |
70
+ | A task added to a milestone | [plan's task index + a "changes during execution" note + tracker item] |
71
+ | A bug or follow-up | [tracker only / plan's follow-up list] |
72
+ | A question for a stakeholder | [where open questions live] |
73
+
74
+ ## Definition of Done
75
+
76
+ A task is done when [the evidence: tests pass on the commit, the acceptance
77
+ criteria are checked, docs that describe it are updated]. "Implemented" or
78
+ "verification pending" is not done.
79
+
80
+ ## Closing a milestone
81
+
82
+ 1. Every item is closed or moved to the next milestone with a reason.
83
+ 2. The report is written: results, evidence, changes, open items.
84
+ 3. The docs are checked against what was built.
85
+ 4. The plan's header reads Closed; the roadmap shows the next milestone.
86
+ 5. This agreement is re-validated.
87
+ ```