workflow-toolkit 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/ponytail/SKILL.md +111 -0
- package/.agents/skills/ponytail-audit/SKILL.md +37 -0
- package/.agents/skills/ponytail-debt/SKILL.md +41 -0
- package/.agents/skills/ponytail-gain/SKILL.md +47 -0
- package/.agents/skills/ponytail-help/SKILL.md +70 -0
- package/.agents/skills/ponytail-review/SKILL.md +52 -0
- package/.agents/skills/prompt-review/SKILL.md +60 -0
- package/.agents/skills/wtk/SKILL.md +64 -0
- package/.agents/skills/wtk/references/artifacts.md +82 -0
- package/.agents/skills/wtk/references/evidence.md +68 -0
- package/.agents/skills/wtk/references/git.md +75 -0
- package/.agents/skills/wtk/references/test-contract.md +76 -0
- package/.agents/skills/wtk/references/validation.md +127 -0
- package/.agents/skills/wtk-config/SKILL.md +102 -0
- package/.agents/skills/wtk-config/assets/agents/claude/deep-reviewer.md +40 -0
- package/.agents/skills/wtk-config/assets/agents/claude/designer.md +57 -0
- package/.agents/skills/wtk-config/assets/agents/claude/explorer.md +41 -0
- package/.agents/skills/wtk-config/assets/agents/claude/implementer.md +57 -0
- package/.agents/skills/wtk-config/assets/agents/claude/planner.md +68 -0
- package/.agents/skills/wtk-config/assets/agents/claude/verifier.md +80 -0
- package/.agents/skills/wtk-config/assets/agents/codex/deep-reviewer.toml +41 -0
- package/.agents/skills/wtk-config/assets/agents/codex/designer.toml +55 -0
- package/.agents/skills/wtk-config/assets/agents/codex/explorer.toml +35 -0
- package/.agents/skills/wtk-config/assets/agents/codex/implementer.toml +52 -0
- package/.agents/skills/wtk-config/assets/agents/codex/planner.toml +66 -0
- package/.agents/skills/wtk-config/assets/agents/codex/verifier.toml +78 -0
- package/.agents/skills/wtk-config/assets/agents/cursor/deep-reviewer.md +38 -0
- package/.agents/skills/wtk-config/assets/agents/cursor/designer.md +55 -0
- package/.agents/skills/wtk-config/assets/agents/cursor/explorer.md +39 -0
- package/.agents/skills/wtk-config/assets/agents/cursor/implementer.md +55 -0
- package/.agents/skills/wtk-config/assets/agents/cursor/planner.md +66 -0
- package/.agents/skills/wtk-config/assets/agents/cursor/verifier.md +79 -0
- package/.agents/skills/wtk-config/scripts/ad-index.py +123 -0
- package/.agents/skills/wtk-config/scripts/repository_intelligence.py +671 -0
- package/.agents/skills/wtk-config/scripts/workflow_config.py +972 -0
- package/.agents/skills/wtk-deep-review/SKILL.md +164 -0
- package/.agents/skills/wtk-deep-review/assets/PROMPT.md +56 -0
- package/.agents/skills/wtk-deep-review/assets/REVIEW_UI.html +1340 -0
- package/.agents/skills/wtk-deep-review/assets/findings.schema.json +113 -0
- package/.agents/skills/wtk-deep-review/references/context-pack.md +88 -0
- package/.agents/skills/wtk-deep-review/references/orchestration.md +164 -0
- package/.agents/skills/wtk-deep-review/references/output-contracts.md +149 -0
- package/.agents/skills/wtk-deep-review/references/publish-github.md +83 -0
- package/.agents/skills/wtk-deep-review/references/state-and-learnings.md +64 -0
- package/.agents/skills/wtk-deep-review/references/subagent-runtimes.md +33 -0
- package/.agents/skills/wtk-deep-review/references/taxonomy.md +70 -0
- package/.agents/skills/wtk-deep-review/scripts/_common.py +371 -0
- package/.agents/skills/wtk-deep-review/scripts/build_jobs.py +561 -0
- package/.agents/skills/wtk-deep-review/scripts/build_knowledge.py +349 -0
- package/.agents/skills/wtk-deep-review/scripts/build_manifest.py +479 -0
- package/.agents/skills/wtk-deep-review/scripts/graft_context.py +160 -0
- package/.agents/skills/wtk-deep-review/scripts/graphify_context.py +75 -0
- package/.agents/skills/wtk-deep-review/scripts/merge_findings.py +313 -0
- package/.agents/skills/wtk-deep-review/scripts/render_html.py +240 -0
- package/.agents/skills/wtk-deep-review/scripts/render_review.py +282 -0
- package/.agents/skills/wtk-deep-review/scripts/run_jobs.py +436 -0
- package/.agents/skills/wtk-deep-review/scripts/token_metrics.py +369 -0
- package/.agents/skills/wtk-discover/SKILL.md +270 -0
- package/.agents/skills/wtk-discover/references/document-format.md +149 -0
- package/.agents/skills/wtk-implement/SKILL.md +95 -0
- package/.agents/skills/wtk-implement/references/checklist-format.md +116 -0
- package/.agents/skills/wtk-implement/references/screens.md +129 -0
- package/.agents/skills/wtk-implement/references/test-policy.md +152 -0
- package/.agents/skills/wtk-implement/references/verify.md +297 -0
- package/.agents/skills/wtk-knowledge-check/SKILL.md +16 -0
- package/.agents/skills/wtk-knowledge-check/scripts/check.ts +502 -0
- package/.agents/skills/wtk-knowledge-check/scripts/cli.ts +32 -0
- package/.agents/skills/wtk-knowledge-check/scripts/frontmatter.ts +44 -0
- package/.agents/skills/wtk-lean/.skill-meta.json +6 -0
- package/.agents/skills/wtk-lean/NOTICE.md +18 -0
- package/.agents/skills/wtk-lean/SKILL.md +137 -0
- package/.agents/skills/wtk-lean/references/build.md +168 -0
- package/.agents/skills/wtk-lean/references/checks.md +253 -0
- package/.agents/skills/wtk-lean/references/memory.md +156 -0
- package/.agents/skills/wtk-lean/references/plan.md +514 -0
- package/.agents/skills/wtk-lean/references/verify.md +336 -0
- package/.agents/skills/wtk-lean/scripts/check_commit.py +121 -0
- package/.agents/skills/wtk-lean/scripts/fixtures/checks.md +98 -0
- package/.agents/skills/wtk-lean/scripts/fixtures/plan.md +119 -0
- package/.agents/skills/wtk-lean/scripts/fixtures/verification.md +41 -0
- package/.agents/skills/wtk-lean/scripts/lessons.py +412 -0
- package/.agents/skills/wtk-lean/scripts/selftest.py +401 -0
- package/.agents/skills/wtk-lean/scripts/validate_checks.py +417 -0
- package/.agents/skills/wtk-lean/scripts/validate_plan.py +580 -0
- package/.agents/skills/wtk-lean/scripts/validate_verification.py +357 -0
- package/.agents/skills/wtk-plan/SKILL.md +102 -0
- package/.agents/skills/wtk-plan/references/document-format.md +222 -0
- package/.agents/skills/wtk-qa/SKILL.md +14 -0
- package/.agents/skills/wtk-qa-execute/SKILL.md +115 -0
- package/.agents/skills/wtk-qa-execute/references/fix-loop.md +23 -0
- package/.agents/skills/wtk-qa-execute/references/session-protocol.md +25 -0
- package/.agents/skills/wtk-qa-plan/SKILL.md +111 -0
- package/.agents/skills/wtk-qa-plan/references/profile.md +35 -0
- package/.agents/skills/wtk-ship/SKILL.md +49 -0
- package/.agents/skills/wtk-ship/remediation.py +170 -0
- package/.agents/skills/wtk-ship/scripts/close_feature.py +83 -0
- package/.agents/skills/wtk-ship/scripts/review_convergence.py +373 -0
- package/.wtk.toml.example +93 -0
- package/AGENTS.md +129 -0
- package/NOTICE.md +12 -0
- package/README.md +440 -0
- package/bin/wtk.js +25 -0
- package/docs/toolkit/README.md +56 -0
- package/docs/toolkit/decisions.md +61 -0
- package/docs/toolkit/guidelines/CONTEXT-BUDGET.md +64 -0
- package/docs/toolkit/guidelines/DX.md +67 -0
- package/docs/toolkit/guidelines/FRONTEND.md +73 -0
- package/docs/toolkit/guidelines/KNOWLEDGE-WIKI.md +128 -0
- package/docs/toolkit/guidelines/MODELING.md +134 -0
- package/docs/toolkit/guidelines/QA-EXECUTION.md +46 -0
- package/docs/toolkit/guidelines/QA-SCENARIOS.md +165 -0
- package/docs/toolkit/guidelines/REVIEW-ROUNDS.md +160 -0
- package/docs/toolkit/guidelines/SECURITY.md +160 -0
- package/docs/toolkit/guidelines/UI-UX.md +117 -0
- package/docs/toolkit/guidelines/WORKFLOW-MEMORY.md +73 -0
- package/docs/toolkit/guidelines.md +50 -0
- package/docs/toolkit/loop.md +77 -0
- package/docs/toolkit/purpose.md +51 -0
- package/docs/toolkit/repository-intelligence.md +53 -0
- package/docs/toolkit/reviews.md +77 -0
- package/knowledge/AGENTS.md +316 -0
- package/knowledge/raw/README.md +17 -0
- package/package.json +74 -0
- package/scripts/install_security_skills.py +816 -0
- package/scripts/installer/engine.js +254 -0
- package/scripts/installer/knowledge.js +28 -0
- package/scripts/installer/packets.js +77 -0
- package/scripts/installer/terminal.js +98 -0
- package/scripts/installer/transaction.js +75 -0
- package/skills-lock.json +92 -0
- package/templates/adoption/agents/core.md +23 -0
- package/templates/adoption/agents/quality.md +3 -0
- package/templates/adoption/knowledge/wiki/architecture/index.md +3 -0
- package/templates/adoption/knowledge/wiki/decisions/index.md +3 -0
- package/templates/adoption/knowledge/wiki/design/index.md +3 -0
- package/templates/adoption/knowledge/wiki/domain/index.md +3 -0
- package/templates/adoption/knowledge/wiki/index.md +3 -0
- package/templates/adoption/knowledge/wiki/log.md +3 -0
- package/templates/adoption/knowledge/wiki/open-questions/index.md +3 -0
- package/templates/adoption/knowledge/wiki/product/index.md +3 -0
- package/templates/adoption/knowledge/wiki/research/index.md +3 -0
- package/templates/adoption/product/AGENT-CONTEXT.md +25 -0
|
@@ -0,0 +1,336 @@
|
|
|
1
|
+
# Verify
|
|
2
|
+
|
|
3
|
+
**Goal:** an independent answer to "is every check actually proven?", written as evidence rather
|
|
4
|
+
than as an opinion.
|
|
5
|
+
|
|
6
|
+
This is the only mechanism standing between "done" and a self-report, so it is never prompted,
|
|
7
|
+
never optional, and never skipped because the author feels confident.
|
|
8
|
+
|
|
9
|
+
## Author is not verifier
|
|
10
|
+
|
|
11
|
+
The agent that wrote the code is the author. The Verifier is a **fresh sub-agent** with no
|
|
12
|
+
inherited context, mental model or assumptions - an author re-checking their own work reapplies
|
|
13
|
+
the thinking that produced the gap, and a self-check can only test what the author still
|
|
14
|
+
remembers.
|
|
15
|
+
|
|
16
|
+
**Dispatched by whoever holds the whole feature, never by a builder.** A fresh context is not
|
|
17
|
+
independence on its own: the parent writes the brief, so a Verifier spawned by the agent that
|
|
18
|
+
just closed the last batch inherits that agent's *scope*. It gets pointed at the last batch, and
|
|
19
|
+
a pass over four checks reads exactly like a pass over forty. The range is
|
|
20
|
+
`<feature base>..HEAD` and the set is **every** check in `checks.md`, whoever wrote them.
|
|
21
|
+
|
|
22
|
+
The verdict goes back to the orchestrator and the user, never to a builder. A FAIL returned to
|
|
23
|
+
the author is the author deciding what to do about the author's own work.
|
|
24
|
+
|
|
25
|
+
**It receives:** `plan.md`, `checks.md`, every source the plan marks binding, the diff range, and
|
|
26
|
+
this file. It runs read-only over the real tree and fixes nothing. The plan is an input because
|
|
27
|
+
half the enumerations it sweeps for - a route's statuses, an entity's constraints - are named
|
|
28
|
+
there and only *owe* a row in the checks.
|
|
29
|
+
|
|
30
|
+
**No sub-agent mechanism available?** Then run this file as a fresh-eyes pass in a new session -
|
|
31
|
+
re-read the plan, the checks and the diff from scratch - and write `Verifier: self-verified
|
|
32
|
+
(degraded - no sub-agent)` in the report. The script flags it, which is the point: a degraded
|
|
33
|
+
gate that is visible is worth more than one that pretends.
|
|
34
|
+
|
|
35
|
+
## Read the profile first
|
|
36
|
+
|
|
37
|
+
`checks.md` carries a `Profile:` line, and `light` is the default. Step 1 runs under `ui`. The
|
|
38
|
+
`Coverage` recompute, the `Test policy` verdicts and fault injection run under `standard` and
|
|
39
|
+
`ui`. Everything else runs always: the proofs at `HEAD`, each named test shown to exist and have
|
|
40
|
+
run, one located assertion per check, the level and sampling judgment, and the `Swept existing`
|
|
41
|
+
re-read.
|
|
42
|
+
|
|
43
|
+
**The report's profile must be the one `checks.md` was approved under, and the gate compares
|
|
44
|
+
them.** A report declaring a cheaper profile than the feature was approved under makes a whole
|
|
45
|
+
step disappear with nothing to see, which is worse than a FAIL: a mismatch is an error, not a
|
|
46
|
+
preference. Under `standard` the report owes a `## Faults injected` section and a `## Coverage`
|
|
47
|
+
section; under `ui` it also owes `## Binding sources`; where `checks.md` carries `Test policy`
|
|
48
|
+
rows, it owes a verdict on them. A step that produced no section did not run, and requiring the
|
|
49
|
+
section is what makes "skipped" distinguishable from "forgotten".
|
|
50
|
+
|
|
51
|
+
A step whose input is absent is a line, not an investigation: no binding source, no set row, no
|
|
52
|
+
`Test policy` section - say so in the section and move on.
|
|
53
|
+
|
|
54
|
+
## 1. Check the checks against their binding sources (`ui`)
|
|
55
|
+
|
|
56
|
+
Every other step measures the code against the checks, so a check that is wrong validates cleanly
|
|
57
|
+
and each later step confirms it. No rigour further down catches this, because both ends of every
|
|
58
|
+
later comparison were derived from the same mistaken artifact.
|
|
59
|
+
|
|
60
|
+
Open every source the plan marks **binding** and compare it against the checks and against the
|
|
61
|
+
plan's `Surface`, `Relations` and `Landing`. A URL sitting in `Sources` is not the same as having
|
|
62
|
+
opened it. The plan gives this step something concrete to compare - a route with its statuses, an
|
|
63
|
+
entity with its constraints - instead of only the claims derived from it.
|
|
64
|
+
|
|
65
|
+
**This is a narrow comparison, not a review of the product.** You are looking for a check that
|
|
66
|
+
contradicts a binding source on something that source decides concretely: a state the design
|
|
67
|
+
draws as a band on an existing screen that a check turned into a screen of its own, a label the
|
|
68
|
+
design fixes that a check spells differently, a status a contract names that a check maps
|
|
69
|
+
elsewhere. You are never asking whether the work is worth doing. A contradiction is a finding; a
|
|
70
|
+
preference is not.
|
|
71
|
+
|
|
72
|
+
A check that contradicts a binding source is a finding even with a green proof, and it outranks a
|
|
73
|
+
failing one: a red test says the code disagrees with the checks, this says the checks disagree
|
|
74
|
+
with the decision, and the build ships the second faithfully.
|
|
75
|
+
|
|
76
|
+
**A check nobody wrote contradicts nothing, so comparing cannot find it.** Enumerate instead: per
|
|
77
|
+
screen, list what the design decides that a selector reaches - which controls and indicators are
|
|
78
|
+
present, which absent, their order, their count, what they read, and **how they are arranged**:
|
|
79
|
+
how many regions the screen has and what sits inside what. Confirm each has a check. An element
|
|
80
|
+
with no check is a coverage gap; an element the code renders that the design does not draw is the
|
|
81
|
+
same finding facing the other way.
|
|
82
|
+
|
|
83
|
+
**Arrangement is the one this step keeps missing.** A screen can pass every label, count and
|
|
84
|
+
order check and still be a different composition - a ring where the design draws a bar, one column
|
|
85
|
+
where there are two. Those are selector-reachable and not exempt, so a screen whose checks are all
|
|
86
|
+
copy and no structure is a finding even when each is green. Where the repo has a design system,
|
|
87
|
+
hold the design to structure and hierarchy and the system to the values: a colour that differs
|
|
88
|
+
from the mock is correct if it came from a token.
|
|
89
|
+
|
|
90
|
+
**An exemption that does not enumerate is a gap, not a limitation.** The checks may put spacing,
|
|
91
|
+
colour and type weight out of reach, named against the screen they belong to. They may not say
|
|
92
|
+
"visual fidelity is unproven" and stop - that sentence also covers everything a selector could
|
|
93
|
+
have reached. Treat a blanket clause as a finding, then enumerate what it was covering.
|
|
94
|
+
|
|
95
|
+
## 2. Account for every proof
|
|
96
|
+
|
|
97
|
+
Apply `.agents/skills/wtk/references/validation.md`: independently inspect each proof's recorded
|
|
98
|
+
result and input baseline, then run proofs with missing or invalidated evidence. A fresh Verifier
|
|
99
|
+
still covers every check; it does not rerun unaffected tests solely because the actor or HEAD changed.
|
|
100
|
+
For reused evidence, identify the original command/log, baseline and why its relevant inputs remain
|
|
101
|
+
equivalent. Batch the selected fresh proofs below.
|
|
102
|
+
|
|
103
|
+
**Batch selected proofs by runner target, not one invocation per proof.** Runners take many
|
|
104
|
+
files and many name patterns in a single call - `bin/rails test a_test.rb b_test.rb -n
|
|
105
|
+
"/one|two/"`, `pytest f.py g.py -k "one or two"`, `jest --testPathPattern` with one
|
|
106
|
+
`--testNamePattern` alternation. Batching per *file* is the mistake that looks like batching: forty
|
|
107
|
+
checks across twelve files is twelve process starts, and the process start is the cost.
|
|
108
|
+
|
|
109
|
+
The guarantee is unchanged as long as **each named test appears in the output individually** as
|
|
110
|
+
having run and passed. What is forbidden is substituting a verdict for a result: "the suite is
|
|
111
|
+
green" settles no single check.
|
|
112
|
+
|
|
113
|
+
Then confirm each named test **exists and ran**. A filter matching nothing exits zero on several
|
|
114
|
+
runners - `passWithNoTests` and its equivalents - which would be a green check with no test behind
|
|
115
|
+
it. Show the hit; a name that appears nowhere in the tree is a finding.
|
|
116
|
+
|
|
117
|
+
Do that lookup with `rg -n` and enough context to carry the test body rather than by reading the
|
|
118
|
+
file. The same hit that proves the test exists yields the line numbers step 3 has to cite, so it is
|
|
119
|
+
one search instead of two. One pass per file, not per check.
|
|
120
|
+
|
|
121
|
+
Existing tests may prove a corrected behavior without being edited themselves. Confirm their
|
|
122
|
+
assertions cover the affected invariant. An earlier result remains valid only after checking its
|
|
123
|
+
causal inputs; neither an unchanged test file nor a new commit alone settles validity.
|
|
124
|
+
|
|
125
|
+
## 3. Check the assertion, not its presence
|
|
126
|
+
|
|
127
|
+
For each check, confirm the assertion targets the **check-defined** value, not merely that an
|
|
128
|
+
assertion exists. Cite `file:line` and reproduce the assertion expression.
|
|
129
|
+
|
|
130
|
+
Inspect the assertion and the relevant setup, fixtures and dependencies needed to establish its
|
|
131
|
+
expected value and causal coverage. Trace indirect expected values within that scope; missing
|
|
132
|
+
context is an evidence gap, not automatic proof that a test is wrong. Avoid unrelated fixture or
|
|
133
|
+
repository exploration.
|
|
134
|
+
|
|
135
|
+
Cite the one or two assertions that **settle** the claim, not every assertion in the test. Setup
|
|
136
|
+
lines earn a citation only when the claim names the precondition.
|
|
137
|
+
|
|
138
|
+
**Evidence or zero.** A check with no located `file:line` counts as not proven - per check, never
|
|
139
|
+
one citation standing in for twenty. Search before concluding something is absent, and show the
|
|
140
|
+
search.
|
|
141
|
+
|
|
142
|
+
**Recompute the `Coverage` join rather than reading it** (`standard`, `ui`). A join you only read
|
|
143
|
+
is the author's self-report with a table around it, which is precisely what `light` gives up. For each row, take the members from the code itself and confirm
|
|
144
|
+
each has a proof that asserts it; then sweep both artifacts for sets they never gave a row at all -
|
|
145
|
+
every enumeration named in the plan's `Landing`, `Relations` or `Surface`, in the level evidence,
|
|
146
|
+
or inside a claim. A route in `Surface` whose statuses got no row is the common one. A member with no
|
|
147
|
+
proof is a coverage gap, and a member named in the artifact's own prose with no proof anywhere is a
|
|
148
|
+
worse one, because the author saw it and the table hid it.
|
|
149
|
+
|
|
150
|
+
**Take the members from whatever holds authority over that set, which is not always the code.** A
|
|
151
|
+
provider's statuses come from the provider and a framework's routes from the framework - there the
|
|
152
|
+
code is where the set is discovered. But a set the code is meant to *satisfy* has its authority
|
|
153
|
+
outside it: the screens a design draws, the fields a contract declares. Recomputing those from the
|
|
154
|
+
code asks the author's own output whether the author's own output is complete, and it answers yes
|
|
155
|
+
every time.
|
|
156
|
+
|
|
157
|
+
For a startup-configuration row, read each assembly directly - open the file that constructs it and
|
|
158
|
+
show the line. A suite that boots its own assembly structurally cannot fail on another one, so a
|
|
159
|
+
green proof there is evidence about the test's assembly and nothing else.
|
|
160
|
+
|
|
161
|
+
Two more, mechanically: a claim about nine cases proven on two is a coverage gap; and a claim naming
|
|
162
|
+
a status code, route or response shape whose proofs all sit below that boundary is a **level gap**,
|
|
163
|
+
no matter how many assertions it carries.
|
|
164
|
+
|
|
165
|
+
**Judge the level against the `Test policy` rows whenever the artifact carries them** (`standard`,
|
|
166
|
+
`ui`). Those rows are the bar the author built under, and that section exists precisely because the
|
|
167
|
+
repo's conventions were found not to answer - deferring to the conventions instead measures the
|
|
168
|
+
build against the weaker ruler. Give each row a verdict: for every file it classifies, is the
|
|
169
|
+
required proof there, and does it assert what the coverage expectation demands? A row nobody met is
|
|
170
|
+
a finding even when every check is green. Only where the artifact carries no such section does the
|
|
171
|
+
repo's own convention decide.
|
|
172
|
+
|
|
173
|
+
Where the checks left a value imprecise, record a **precision gap** rather than passing a vague
|
|
174
|
+
assertion - that is a finding about the checks, and the most useful thing this step produces.
|
|
175
|
+
|
|
176
|
+
Read the `Swept` rows that resolve to **existing** against the code: is the constraint they cite
|
|
177
|
+
actually there? A cited constraint that is not there is a finding. Rows that say `n/a` are policy
|
|
178
|
+
the user approved; there is nothing in the code for them to be wrong about.
|
|
179
|
+
|
|
180
|
+
## 4. Inject faults (`standard`, `ui`)
|
|
181
|
+
|
|
182
|
+
A green suite proves the tests run. Fault injection proves they can catch a regression - it is
|
|
183
|
+
the only step that produces that information, and under `light` it does not run, which is what
|
|
184
|
+
that row of the profile table means by "a test that would pass under a wrong implementation".
|
|
185
|
+
|
|
186
|
+
1. **Isolate.** `git worktree add <scratch> HEAD`. Never mutate the real tree, and **never use
|
|
187
|
+
`git stash`**: it records state from *before* the mutation, so popping it does not reverse a
|
|
188
|
+
fault applied afterwards, and on a clean tree it creates no entry at all.
|
|
189
|
+
2. **Baseline.** Record `git status --porcelain` of the real tree first.
|
|
190
|
+
3. **Inject a behaviour-level fault** in the new code: flip a condition, change a returned value or
|
|
191
|
+
status, shift a bound by one, remove a required side effect.
|
|
192
|
+
4. **Run the narrowest covering proof** in the scratch and confirm it FAILS - the mutant is killed.
|
|
193
|
+
5. **Discard** the scratch and confirm the real tree's porcelain matches the baseline. If it
|
|
194
|
+
differs, STOP, restore the tree, and treat the run as invalid.
|
|
195
|
+
|
|
196
|
+
**One fault per distinct assertion surface, not per risky line.** Three mutations killed by the same
|
|
197
|
+
two proofs ran the same experiment three times. Choose faults that force *different* proofs to fail,
|
|
198
|
+
stop once every proof carrying a check has been made to fail once, and cap it at five however risky
|
|
199
|
+
the feature looks - a quota that scales with risk costs most exactly where checks cluster in risky
|
|
200
|
+
code. A second covering proof per fault adds a run and no information. Use real mutation tooling
|
|
201
|
+
where the stack has it (Stryker, mutmut, cargo-mutants, pitest): it gets many mutants from one run.
|
|
202
|
+
|
|
203
|
+
**A surviving mutant is a finding, not a footnote.** It means the assertion would pass under a
|
|
204
|
+
plausible wrong implementation.
|
|
205
|
+
|
|
206
|
+
## 5. Walk the flow with the user (user-facing only)
|
|
207
|
+
|
|
208
|
+
Only where human judgment decides the outcome - a UI flow, an interaction pattern. Backend and
|
|
209
|
+
infrastructure work is settled by the checks. Present one test at a time, expected outcome stated,
|
|
210
|
+
and log anything that is not a clear pass verbatim. Infer severity from the words, never ask for it:
|
|
211
|
+
crash / error / exception → blocker; doesn't work / wrong / missing → major; slow / weird / off →
|
|
212
|
+
minor; colour / font / spacing → cosmetic; unclear → major.
|
|
213
|
+
|
|
214
|
+
## 6. Report
|
|
215
|
+
|
|
216
|
+
Write `.specs/features/<feature>/verification.md`. Lead with the verdict.
|
|
217
|
+
|
|
218
|
+
```markdown
|
|
219
|
+
# <Feature> verification
|
|
220
|
+
|
|
221
|
+
**Verdict**: PASS
|
|
222
|
+
**Profile**: standard
|
|
223
|
+
**Diff range**: <base>..<head>
|
|
224
|
+
**Round**: 1 - full
|
|
225
|
+
**Verifier**: independent sub-agent (author != verifier)
|
|
226
|
+
|
|
227
|
+
## Binding sources
|
|
228
|
+
|
|
229
|
+
| Source | Opened | Contradiction | Uncovered |
|
|
230
|
+
| --- | --- | --- | --- |
|
|
231
|
+
| design `03` overview | yes - artifact URL | none | - |
|
|
232
|
+
|
|
233
|
+
## Checks
|
|
234
|
+
|
|
235
|
+
| Check | Claim | Proof run | Evidence | Result |
|
|
236
|
+
| --- | --- | --- | --- | --- |
|
|
237
|
+
| C1 | suspends, never cancels | `bin/rails test ... -n "/failed_charge_suspends/"` exit 0 | `test/billing/dunning_test.rb:118` - `assert_equal "suspended", sub.status` | PASS |
|
|
238
|
+
|
|
239
|
+
## Coverage
|
|
240
|
+
|
|
241
|
+
| Set (size) | Recomputed from | Member -> proof | Unproven |
|
|
242
|
+
| --- | --- | --- | --- |
|
|
243
|
+
| provider status -> local (9) | provider docs | C2, table-driven over all 9 | - |
|
|
244
|
+
|
|
245
|
+
## Test policy rows
|
|
246
|
+
|
|
247
|
+
| Row | Files it classifies | Required proof | Expectation met |
|
|
248
|
+
| --- | --- | --- | --- |
|
|
249
|
+
| Decides, reached across a boundary | `subscription.rb` | boundary C1 · own layer C2 | yes |
|
|
250
|
+
|
|
251
|
+
## Faults injected
|
|
252
|
+
|
|
253
|
+
| Mutation | Location | Killed |
|
|
254
|
+
| --- | --- | --- |
|
|
255
|
+
| returned status `suspended` -> `cancelled` | `subscription.rb:88` | yes |
|
|
256
|
+
|
|
257
|
+
## Gate
|
|
258
|
+
|
|
259
|
+
`<command>` - <N> passed, 0 failed
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
A non-empty `Contradiction` or `Uncovered` cell, a `Killed` cell reading `no`, a non-empty
|
|
263
|
+
`Unproven` cell, a check whose `Result` is not PASS, or an unmet `Test policy` row all mean the
|
|
264
|
+
verdict is FAIL. The gate script reads those columns and refuses a PASS its own rows contradict,
|
|
265
|
+
so do not soften a row to fit a verdict - change the verdict.
|
|
266
|
+
|
|
267
|
+
Then run the completion gate and return a compact verdict in chat:
|
|
268
|
+
|
|
269
|
+
```bash
|
|
270
|
+
python3 <skill-dir>/scripts/validate_verification.py <feature>
|
|
271
|
+
|
|
272
|
+
`0` is a report that was read and held up. `1` is a report its own rows contradict. `2` is
|
|
273
|
+
**gated nothing** - the feature or the report could not be resolved - and it is not a pass:
|
|
274
|
+
name the feature explicitly rather than letting it search.
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
```markdown
|
|
278
|
+
## Verification: <feature> - PASS
|
|
279
|
+
|
|
280
|
+
**Checks**: 12/12 proven with located evidence
|
|
281
|
+
**Coverage**: 5 sets recomputed, 0 members unproven
|
|
282
|
+
**Faults**: 4 injected, 4 killed
|
|
283
|
+
**Gate**: 138 passed, 0 failed
|
|
284
|
+
**Report**: `.specs/features/<feature>/verification.md`
|
|
285
|
+
|
|
286
|
+
**Ranked gaps** (if FAIL):
|
|
287
|
+
1. <gap> - <check id> - <file:line or "no evidence">
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
## What fails the feature
|
|
291
|
+
|
|
292
|
+
A FAIL, a surviving mutant, an unmet `Test policy` row, a check contradicting a binding source, an
|
|
293
|
+
element or an **arrangement** a binding source decides that no check covers, a blanket exemption in
|
|
294
|
+
place of an enumerated one, a binding source recorded as opened that nobody actually saw, an
|
|
295
|
+
unproven coverage member, or any check without a located `file:line`.
|
|
296
|
+
|
|
297
|
+
**A composition finding fails at the same weight as a wrong label**, and that has to be said because
|
|
298
|
+
it does not feel that way: a missing substring in a CTA reads as a defect while "this screen is a
|
|
299
|
+
different composition" reads as feedback. The second is the larger failure and the one that survives
|
|
300
|
+
to production, so it carries a FAIL to the last round rather than softening into a note.
|
|
301
|
+
|
|
302
|
+
Route gaps back to an implementer as fix work and re-verify, bounded to **three** rounds before
|
|
303
|
+
escalating to the user.
|
|
304
|
+
|
|
305
|
+
## Re-verifying after a fix
|
|
306
|
+
|
|
307
|
+
A later round is scoped by two things: **the fix's diff, and every verdict that was not PASS.**
|
|
308
|
+
Anything else carries forward. Re-running the whole review to reconfirm what a fix could not have
|
|
309
|
+
touched is the cost the three-round bound multiplies by three.
|
|
310
|
+
|
|
311
|
+
Rerun only evidence invalidated by the fix's causal delta. Reuse other proofs after independently
|
|
312
|
+
checking their tested code, transitive dependencies, fixtures, configuration and relevant runtime.
|
|
313
|
+
A failed full run is retained as such; focused retests can prove its corrections without claiming
|
|
314
|
+
that a new full run passed. Full-gate escalation follows `.agents/skills/wtk/references/validation.md`.
|
|
315
|
+
|
|
316
|
+
Everything carried forward identifies its source: mark sections `verified at <sha>` or
|
|
317
|
+
`carried from <sha>` with the input-equivalence reason and retain `Round: 2 - scoped` in the header.
|
|
318
|
+
|
|
319
|
+
Then scope by the diff, not by the fix's intent - a fix to a shared helper, a fixture or a config
|
|
320
|
+
has a wider blast radius than its description:
|
|
321
|
+
|
|
322
|
+
- **Faults** (`standard`, `ui`)**:** re-inject on the surfaces the fix touched, and on any it
|
|
323
|
+
created. A fix that *adds* an assertion is the common case, and its new surface has never been
|
|
324
|
+
made to fail once.
|
|
325
|
+
- **Coverage** (`standard`, `ui`)**:** recompute the rows whose authority the fix touched. A fix
|
|
326
|
+
that adds a branch adds a member.
|
|
327
|
+
- **Citations:** refresh the files the fix touched; line numbers move.
|
|
328
|
+
- **`Test policy` rows:** re-judge the unmet ones, plus any row classifying a touched file.
|
|
329
|
+
- **Step 1:** only where the fix touched the interface, and only for those screens.
|
|
330
|
+
|
|
331
|
+
## 7. Distill lessons
|
|
332
|
+
|
|
333
|
+
The closing action, immediately after the report is written. Turn each grounded failure - a
|
|
334
|
+
surviving mutant, a precision gap, a failed check, an unproven member - into one reusable
|
|
335
|
+
project-local lesson via `scripts/lessons.py`. A clean PASS records nothing, and that is correct
|
|
336
|
+
rather than a miss. Commands and phrasing rules: [memory.md](memory.md).
|
|
@@ -0,0 +1,121 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""
|
|
3
|
+
check_commit.py - deterministic Conventional Commits validation.
|
|
4
|
+
|
|
5
|
+
The per-task atomic-commit rule mandates Conventional Commits 1.0.0. This makes
|
|
6
|
+
that rule checkable instead of trusting the model to remember the format. Pure
|
|
7
|
+
standard library, zero dependencies, agent-agnostic.
|
|
8
|
+
|
|
9
|
+
It reads the message from (in priority order): a positional file path, --message,
|
|
10
|
+
or stdin. The file-path form matches how git passes the message file to a
|
|
11
|
+
`commit-msg` hook, so this doubles as an optional git-level guard WITHOUT
|
|
12
|
+
coupling the skill to any AI agent:
|
|
13
|
+
|
|
14
|
+
ln -s <skill-dir>/scripts/check_commit.py .git/hooks/commit-msg && chmod +x .git/hooks/commit-msg
|
|
15
|
+
|
|
16
|
+
What it checks:
|
|
17
|
+
ERROR - header does not match type(scope)!: description
|
|
18
|
+
ERROR - type is not one of the allowed Conventional Commits types
|
|
19
|
+
ERROR - description is empty, starts uppercase, or ends with a period
|
|
20
|
+
ERROR - `!` breaking marker present but no `BREAKING CHANGE:` footer
|
|
21
|
+
WARN - header longer than 72 characters
|
|
22
|
+
|
|
23
|
+
Usage:
|
|
24
|
+
python3 <skill-dir>/scripts/check_commit.py [msgfile]
|
|
25
|
+
python3 <skill-dir>/scripts/check_commit.py --message "feat(auth): add email validation"
|
|
26
|
+
echo "fix(cart): prevent negative quantity" | python3 <skill-dir>/scripts/check_commit.py
|
|
27
|
+
|
|
28
|
+
Exit codes: 0 pass, 1 violation, 2 usage error.
|
|
29
|
+
"""
|
|
30
|
+
|
|
31
|
+
import argparse
|
|
32
|
+
import os
|
|
33
|
+
import re
|
|
34
|
+
import sys
|
|
35
|
+
|
|
36
|
+
TYPES = ["feat", "fix", "refactor", "docs", "test", "style", "perf", "build", "ci", "chore"]
|
|
37
|
+
HEADER_RE = re.compile(r"^(?P<type>\w+)(?:\((?P<scope>[^)]+)\))?(?P<bang>!)?: (?P<desc>.+)$")
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def read_message(args):
|
|
41
|
+
if args.message is not None:
|
|
42
|
+
return args.message
|
|
43
|
+
if args.msgfile:
|
|
44
|
+
# git passes a path that exists. Anything else is the message itself, which is how a
|
|
45
|
+
# caller invokes this by hand - a traceback there is a worse answer than the check.
|
|
46
|
+
if os.path.isfile(args.msgfile):
|
|
47
|
+
with open(args.msgfile, "r", encoding="utf-8") as f:
|
|
48
|
+
return f.read()
|
|
49
|
+
return args.msgfile
|
|
50
|
+
if not sys.stdin.isatty():
|
|
51
|
+
return sys.stdin.read()
|
|
52
|
+
return ""
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
def check(message):
|
|
56
|
+
errors, warnings = [], []
|
|
57
|
+
# Ignore comment lines (git puts '#' comments in the message file).
|
|
58
|
+
lines = [ln for ln in message.splitlines() if not ln.lstrip().startswith("#")]
|
|
59
|
+
# Trim leading blank lines.
|
|
60
|
+
while lines and not lines[0].strip():
|
|
61
|
+
lines.pop(0)
|
|
62
|
+
if not lines:
|
|
63
|
+
return (["empty commit message"], warnings)
|
|
64
|
+
|
|
65
|
+
header = lines[0].rstrip()
|
|
66
|
+
if len(header) > 72:
|
|
67
|
+
warnings.append(f"header is {len(header)} chars (>72): {header[:60]}...")
|
|
68
|
+
|
|
69
|
+
m = HEADER_RE.match(header)
|
|
70
|
+
if not m:
|
|
71
|
+
errors.append(f"header does not match 'type(scope): description': {header!r}")
|
|
72
|
+
return (errors, warnings)
|
|
73
|
+
|
|
74
|
+
ctype = m.group("type")
|
|
75
|
+
desc = m.group("desc")
|
|
76
|
+
bang = m.group("bang")
|
|
77
|
+
|
|
78
|
+
if ctype not in TYPES:
|
|
79
|
+
errors.append(f"type '{ctype}' is not one of: {', '.join(TYPES)}")
|
|
80
|
+
if not desc.strip():
|
|
81
|
+
errors.append("description is empty")
|
|
82
|
+
else:
|
|
83
|
+
if desc[:1].isupper():
|
|
84
|
+
errors.append(f"description should start lowercase: '{desc[:30]}'")
|
|
85
|
+
if desc.rstrip().endswith("."):
|
|
86
|
+
errors.append("description should not end with a period")
|
|
87
|
+
|
|
88
|
+
body = "\n".join(lines[1:])
|
|
89
|
+
breaking_footer = bool(re.search(r"^BREAKING CHANGE:", body, re.MULTILINE))
|
|
90
|
+
if bang and not breaking_footer:
|
|
91
|
+
errors.append("'!' breaking marker present but no 'BREAKING CHANGE:' footer")
|
|
92
|
+
|
|
93
|
+
return (errors, warnings)
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
def main(argv=None):
|
|
97
|
+
p = argparse.ArgumentParser(prog="check_commit.py", description="Validate a Conventional Commits message.")
|
|
98
|
+
p.add_argument("msgfile", nargs="?", default=None,
|
|
99
|
+
help="a commit message file (as git passes to commit-msg), or the message itself")
|
|
100
|
+
p.add_argument("--message", default=None, help="the commit message as a string")
|
|
101
|
+
args = p.parse_args(argv)
|
|
102
|
+
|
|
103
|
+
message = read_message(args)
|
|
104
|
+
if not message.strip():
|
|
105
|
+
print("check_commit: no message provided (pass a file, --message, or pipe via stdin).", file=sys.stderr)
|
|
106
|
+
return 2
|
|
107
|
+
|
|
108
|
+
errors, warnings = check(message)
|
|
109
|
+
for w in warnings:
|
|
110
|
+
print(f" WARN {w}")
|
|
111
|
+
for e in errors:
|
|
112
|
+
print(f" ERROR {e}")
|
|
113
|
+
if errors:
|
|
114
|
+
print("\ncheck_commit: FAIL - see https://www.conventionalcommits.org/en/v1.0.0/")
|
|
115
|
+
return 1
|
|
116
|
+
print("check_commit: OK")
|
|
117
|
+
return 0
|
|
118
|
+
|
|
119
|
+
|
|
120
|
+
if __name__ == "__main__":
|
|
121
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Dunning on failed charge - checks
|
|
2
|
+
|
|
3
|
+
Profile: standard
|
|
4
|
+
Plan: `.specs/features/billing/plan.md`
|
|
5
|
+
|
|
6
|
+
## Intent
|
|
7
|
+
|
|
8
|
+
11 checks in 2 slices · 2 one-way doors · 0 open
|
|
9
|
+
|
|
10
|
+
## Checks
|
|
11
|
+
|
|
12
|
+
### S1 - Suspension on a failed charge · 4 files · 38 KB · ~10k
|
|
13
|
+
|
|
14
|
+
**C1** - A failed charge sets status to `Suspended`, never `Cancelled` (BILL-01, AC 1)
|
|
15
|
+
Proof: `bin/rails test test/billing/dunning_test.rb -n "/failed_charge_suspends/"`
|
|
16
|
+
|
|
17
|
+
**C2** - Every provider status maps to exactly one local status (BILL-01, AC 1)
|
|
18
|
+
Proof: `bin/rails test test/billing/status_map_test.rb -n "/every_provider_status/"`
|
|
19
|
+
|
|
20
|
+
**C3** - An unknown provider status leaves the status unchanged and records `billing.status_unmapped` (BILL-01, AC 2)
|
|
21
|
+
Proof: `bin/rails test test/billing/status_map_test.rb -n "/unknown_status_is_recorded/"`
|
|
22
|
+
|
|
23
|
+
**C4** - A `Suspended` subscription denies access to paid access groups (BILL-01, AC 3)
|
|
24
|
+
Proof: `bin/rails test test/billing/access_test.rb -n "/suspended_denies_paid_groups/"`
|
|
25
|
+
|
|
26
|
+
**C5** - `Suspended` cannot transition directly to `Trialing` (BILL-01, AC 4)
|
|
27
|
+
Proof: `bin/rails test test/billing/transitions_test.rb -n "/suspended_to_trialing_is_rejected/"`
|
|
28
|
+
|
|
29
|
+
**C6** - A `Suspended` subscription returns to `Active` on a successful charge (BILL-01, AC 1)
|
|
30
|
+
Proof: `bin/rails test test/billing/transitions_test.rb -n "/suspended_to_active/"`
|
|
31
|
+
|
|
32
|
+
**C7** - Two concurrent failed-charge deliveries produce one suspension and the second returns `409` (BILL-01, AC 1)
|
|
33
|
+
Proof: `bin/rails test test/webhooks/ingest_test.rb -n "/concurrent_delivery_conflicts/"`
|
|
34
|
+
|
|
35
|
+
### S2 - Webhook ingest · 5 files · 61 KB · ~15k
|
|
36
|
+
|
|
37
|
+
**C8** - Retrying the same delivery id returns `200` and changes no records (BILL-02, AC 5)
|
|
38
|
+
Proof: `bin/rails test test/webhooks/ingest_test.rb -n "/retry_is_idempotent/"`
|
|
39
|
+
|
|
40
|
+
**C9** - A payload missing a subscription id returns `422` and persists nothing (BILL-02, AC 6)
|
|
41
|
+
Proof: `bin/rails test test/webhooks/ingest_test.rb -n "/missing_subscription_id_is_422/"`
|
|
42
|
+
|
|
43
|
+
**C10** - Each of the 5 provider event types dispatches to its own handler (BILL-02, AC 5)
|
|
44
|
+
Proof: `bin/rails test test/webhooks/dispatch_test.rb -n "/dispatches_every_event_type/"`
|
|
45
|
+
|
|
46
|
+
**C11** - The provider timing out does not change local state (BILL-01, AC 1)
|
|
47
|
+
Proof: `bin/rails test test/billing/dunning_test.rb -n "/provider_timeout_is_inert/"`
|
|
48
|
+
|
|
49
|
+
## Coverage
|
|
50
|
+
|
|
51
|
+
| Set (size) | Member -> proof | Unproven |
|
|
52
|
+
| --- | --- | --- |
|
|
53
|
+
| provider status -> local (9) | C2, table-driven over all 9 | - |
|
|
54
|
+
| webhook event types (5) | `charge_failed` C1 · `charge_succeeded` C6 · `updated` C10 · `deleted` C10 · `trial_will_end` C10 | - |
|
|
55
|
+
| `Suspended` transitions (3) | into it C1 · out to `Active` C6 · rejected to `Trialing` C5 | - |
|
|
56
|
+
| `POST /webhooks/provider` statuses (3) | 200 C8 · 409 C7 · 422 C9 | - |
|
|
57
|
+
| startup config: raw request body (2 assemblies) | app entry point C9 · test harness C8 | - |
|
|
58
|
+
|
|
59
|
+
- Claims naming a status code, route or response shape: C7, C8, C9 - each has a proof that crosses the boundary
|
|
60
|
+
- No other check claims more than the single case its proof exercises
|
|
61
|
+
|
|
62
|
+
## Test policy
|
|
63
|
+
|
|
64
|
+
The repo's guidelines say where tests live and how to run them, and nothing about which level
|
|
65
|
+
proves which code, so these rows are the bar this build runs under.
|
|
66
|
+
|
|
67
|
+
| Code | Required proofs | Coverage expectation |
|
|
68
|
+
| --- | --- | --- |
|
|
69
|
+
| Decides, reached across a boundary | one at the boundary **and** one at its own layer | the contract at the boundary; one asserted case per row of the decision table at its own layer |
|
|
70
|
+
| Decides, not reached across a boundary | one at its own layer | one asserted case per row of the decision table |
|
|
71
|
+
| Entry point that decides nothing | one at the boundary | accepted input, each rejected input, each error path |
|
|
72
|
+
| Instrumentation, pass-throughs | none of its own | covered by its consumer's proof |
|
|
73
|
+
|
|
74
|
+
Evidence:
|
|
75
|
+
|
|
76
|
+
- `app/models/billing/status_map.rb`: dispatches over 9 provider statuses, 0 other branch points -> decides
|
|
77
|
+
- `app/webhooks/ingest.rb`: dispatches over 5 event types, 3 branch points -> decides, reached across a boundary
|
|
78
|
+
- `app/webhooks/payload.rb`: forwards a single call, no conditional -> instrumentation
|
|
79
|
+
- closest analogue in the repo: `app/models/access/policy.rb`, same dispatch shape, already proven at its own layer with one case per row
|
|
80
|
+
|
|
81
|
+
Cost: 2 proofs at their own layer across 2 files. Without these rows, the status map and the
|
|
82
|
+
event dispatch are proven only by a path that happens to traverse them.
|
|
83
|
+
|
|
84
|
+
## Swept
|
|
85
|
+
|
|
86
|
+
- validation: C9
|
|
87
|
+
- failure modes: C11
|
|
88
|
+
- idempotency: C8
|
|
89
|
+
- authorization: existing - the webhook route already sits behind `Webhooks::Verifier` signature checking
|
|
90
|
+
- concurrency: C7
|
|
91
|
+
- data lifecycle: n/a - delivery rows are already pruned by the existing 90-day job
|
|
92
|
+
- dependency failure: C11
|
|
93
|
+
- state transitions: C5, C6
|
|
94
|
+
- observability: C3
|
|
95
|
+
|
|
96
|
+
## Handoff
|
|
97
|
+
|
|
98
|
+
- S1 = 10k, all in Billing; S2 enters Webhooks at 25k total, under the 150k budget - one builder
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# Dunning on failed charge
|
|
2
|
+
|
|
3
|
+
Sources:
|
|
4
|
+
|
|
5
|
+
- TLC-412 - the suspension rule and the 11-day evidence
|
|
6
|
+
- provider webhook reference - the 9 statuses and the 5 event types
|
|
7
|
+
|
|
8
|
+
## Problem
|
|
9
|
+
|
|
10
|
+
A member whose card fails keeps full access until someone notices in the provider dashboard,
|
|
11
|
+
which took 11 days on the last three cases. Support absorbs the refund argument and the member
|
|
12
|
+
learns that paying is optional.
|
|
13
|
+
|
|
14
|
+
When this ships, a failed charge suspends access within one billing cycle and the member sees why.
|
|
15
|
+
|
|
16
|
+
## Out of scope
|
|
17
|
+
|
|
18
|
+
| Excluded | Why |
|
|
19
|
+
| --- | --- |
|
|
20
|
+
| Retry schedule configuration | the provider already owns retries; duplicating it needs its own decision |
|
|
21
|
+
| Dunning emails | copy is not written yet, tracked separately |
|
|
22
|
+
|
|
23
|
+
## Assumptions
|
|
24
|
+
|
|
25
|
+
| Assumption | Chosen default | Rationale | Confirmed? |
|
|
26
|
+
| --- | --- | --- | --- |
|
|
27
|
+
| Grace period after the first failure | none - suspend on the first failed charge | the provider already retried 3 times before it reports a failure | y |
|
|
28
|
+
| Status for a member who was never charged | stays `Active` | absence of a charge is not a failure | y |
|
|
29
|
+
|
|
30
|
+
**Open questions:** none - all resolved or logged above.
|
|
31
|
+
|
|
32
|
+
## Criteria
|
|
33
|
+
|
|
34
|
+
### S1: Suspension on a failed charge (P1)
|
|
35
|
+
|
|
36
|
+
**Acceptance Criteria**
|
|
37
|
+
|
|
38
|
+
1. WHEN the provider reports a failed charge THEN the system SHALL set the subscription status to `Suspended`
|
|
39
|
+
2. IF the reported provider status is unknown THEN the system SHALL leave the status unchanged and record a `billing.status_unmapped` event
|
|
40
|
+
3. WHILE a subscription is `Suspended` the system SHALL deny access to paid access groups
|
|
41
|
+
4. The system SHALL never move a subscription directly from `Suspended` to `Trialing`
|
|
42
|
+
|
|
43
|
+
**Independent test:** replay a failed-charge webhook against a paying member and confirm access is denied.
|
|
44
|
+
|
|
45
|
+
### S2: Webhook ingest (P1)
|
|
46
|
+
|
|
47
|
+
**Acceptance Criteria**
|
|
48
|
+
|
|
49
|
+
5. WHEN a webhook delivery id has already been processed THEN the system SHALL return `200` and change no records
|
|
50
|
+
6. IF the provider payload is missing a subscription id THEN the system SHALL return `422` and persist nothing
|
|
51
|
+
|
|
52
|
+
**Independent test:** post the same delivery twice and confirm one row.
|
|
53
|
+
|
|
54
|
+
## Traceability
|
|
55
|
+
|
|
56
|
+
| ID | Slice | Criteria | Status |
|
|
57
|
+
| --- | --- | --- | --- |
|
|
58
|
+
| BILL-01 | S1 | 1, 2, 3, 4 | Pending |
|
|
59
|
+
| BILL-02 | S2 | 5, 6 | Pending |
|
|
60
|
+
|
|
61
|
+
## Observable
|
|
62
|
+
|
|
63
|
+
| Surface | Decision | Landing |
|
|
64
|
+
| --- | --- | --- |
|
|
65
|
+
| API `POST /webhooks/provider` | response shape | AC 5 |
|
|
66
|
+
| API `POST /webhooks/provider` | error shape and codes | AC 6 |
|
|
67
|
+
| API `POST /webhooks/provider` | who may call it | existing - signature verification in `Webhooks::Verifier` |
|
|
68
|
+
| API `POST /webhooks/provider` | versioning | n/a - the provider pins the payload version in the envelope |
|
|
69
|
+
| API `POST /webhooks/provider` | rate limit behaviour | n/a - inbound webhooks are not throttled; the provider owns retries |
|
|
70
|
+
| screen | n/a - this feature exposes no screen; the member sees the existing locked-community view |
|
|
71
|
+
|
|
72
|
+
## Flow
|
|
73
|
+
|
|
74
|
+
A failed charge suspends the subscription instead of cancelling it, and access follows the status.
|
|
75
|
+
Reuses the signature verifier already in `Webhooks::Verifier` rather than adding a second one, and
|
|
76
|
+
the existing 90-day pruning job for delivery rows.
|
|
77
|
+
|
|
78
|
+
1. provider `POST /webhooks/provider` -> `Webhooks::Verifier` (exists) - signature, then the envelope
|
|
79
|
+
2. `Webhooks::Ingest` (exists) - dedups on `provider_delivery_id`, persists `WebhookDelivery` (door 3)
|
|
80
|
+
3. `Billing::StatusMap` (new, no door - placement per conventions) - provider status -> local status
|
|
81
|
+
4. `Billing::Subscription` (exists) - applies the transition, persists `status`
|
|
82
|
+
5. out: `200` `{}`, and `AccessPolicy` (exists) reads `status` on the next request - no call from here
|
|
83
|
+
|
|
84
|
+
## Relations
|
|
85
|
+
|
|
86
|
+
```mermaid
|
|
87
|
+
erDiagram
|
|
88
|
+
Subscription ||--o{ WebhookDelivery : "reported by"
|
|
89
|
+
WebhookDelivery ||--|| ProviderDelivery : "provider_delivery_id - unique, door 2"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
One-way constraints: `provider_delivery_id` unique (door 2), `status` not null with `suspended`
|
|
93
|
+
in the enum (door 1). No columns and no types here - those come from the repo's conventions and
|
|
94
|
+
are settled in the diff.
|
|
95
|
+
|
|
96
|
+
## Surface
|
|
97
|
+
|
|
98
|
+
| Route | In | Out | Status |
|
|
99
|
+
| --- | --- | --- | --- |
|
|
100
|
+
| `POST /webhooks/provider` | the provider envelope as sent | `{}` · `{error}` | `200`, `409`, `422` |
|
|
101
|
+
|
|
102
|
+
## Landing
|
|
103
|
+
|
|
104
|
+
| One-way door | Literal shape | Alternative rejected |
|
|
105
|
+
| --- | --- | --- |
|
|
106
|
+
| `subscriptions.status` gains `suspended` | enum value, not null, existing rows backfilled to `active` | a boolean `is_suspended` - cannot express the next state |
|
|
107
|
+
| delivery id uniqueness | unique index on `webhook_deliveries.provider_delivery_id` | dedup in application code - two workers race past it |
|
|
108
|
+
| new entity `WebhookDelivery` | one row per delivery, keyed on the provider's delivery id | a column on `subscriptions` - one row per delivery, not per subscription |
|
|
109
|
+
| `POST /webhooks/provider` response codes | `200` on accepted or replayed, `409` on a concurrent duplicate, `422` on a missing subscription id | `204` - the provider retries anything without a body |
|
|
110
|
+
|
|
111
|
+
- Nothing else in this change is hard to reverse
|
|
112
|
+
|
|
113
|
+
## Impact
|
|
114
|
+
|
|
115
|
+
| Front | What changes |
|
|
116
|
+
| --- | --- |
|
|
117
|
+
| domain | new term: `Suspended` - a subscription whose last charge failed and whose access is revoked, lives in `Billing` |
|
|
118
|
+
| domain | existing term: `Active` meant "has a subscription row", now means "has a subscription row and the last charge succeeded" - `AccessPolicy#grant?` and the admin index branch on it today |
|
|
119
|
+
| stored data | backfill now: every existing row gets `active`; the unique index runs against 40k delivery rows and fails if a duplicate already exists, so dedup the table first |
|