create-agent-rig 0.4.0 โ 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +316 -11
- package/README.md +58 -19
- package/package.json +2 -1
- package/packages/cli/dist/commands/create.js +8 -3
- package/packages/cli/dist/commands/init.js +85 -34
- package/packages/cli/dist/commands/upgrade.js +112 -30
- package/packages/cli/dist/index.js +81 -19
- package/packages/cli/dist/lib/copy-tree.js +35 -6
- package/packages/cli/dist/lib/init-settings.js +12 -0
- package/packages/cli/dist/lib/install-set.js +6 -8
- package/packages/cli/dist/lib/manifest.js +31 -9
- package/packages/cli/dist/lib/safe-path.js +30 -0
- package/scripts/prepare.mjs +1 -1
- package/templates/agent-os/init/AGENTS.md +199 -0
- package/templates/agent-os/init/CLAUDE.md +69 -9
- package/templates/agent-os/stack/aws-cdk/.agents/skills/post-deploy-verify/SKILL.md +105 -0
- package/templates/agent-os/stack/aws-cdk/.agents/skills/ro-debug/SKILL.md +117 -0
- package/templates/agent-os/stack/aws-cdk/.claude/agents/cdk-diff-reviewer.md +41 -3
- package/templates/agent-os/stack/aws-cdk/.claude/rules/aws-cdk.md +1 -1
- package/templates/agent-os/stack/aws-cdk/.claude/skills/post-deploy-verify/SKILL.md +41 -0
- package/templates/agent-os/stack/aws-cdk/.codex/agents/cdk-diff-reviewer.toml +4 -0
- package/templates/agent-os/stack/node-ts/.claude/rules/node-ts.md +29 -0
- package/templates/agent-os/universal/.agents/skills/check-premises/SKILL.md +236 -0
- package/templates/agent-os/universal/.agents/skills/loop/SKILL.md +958 -0
- package/templates/agent-os/universal/.agents/skills/new-invariant/SKILL.md +102 -0
- package/templates/agent-os/universal/.agents/skills/new-invariant/guard-invariant.example.mjs +78 -0
- package/templates/agent-os/universal/.agents/skills/new-invariant/guard-invariant.example.test.mjs +89 -0
- package/templates/agent-os/universal/.agents/skills/pr-ship/SKILL.md +396 -0
- package/templates/agent-os/universal/.agents/skills/worktree-task/SKILL.md +73 -0
- package/templates/agent-os/universal/.claude/agents/code-reviewer.md +40 -1
- package/templates/agent-os/universal/.claude/agents/prose-reviewer.md +71 -1
- package/templates/agent-os/universal/.claude/agents/security-scanner.md +40 -0
- package/templates/agent-os/universal/.claude/hooks/gate-stop-dod.mjs +301 -26
- package/templates/agent-os/universal/.claude/hooks/guard-bash.mjs +43 -5
- package/templates/agent-os/universal/.claude/hooks/guard-core-purity.mjs +25 -11
- package/templates/agent-os/universal/.claude/hooks/guard-rulebook.mjs +127 -0
- package/templates/agent-os/universal/.claude/hooks/guard-secret-file.mjs +180 -0
- package/templates/agent-os/universal/.claude/hooks/guard-web-boundary.mjs +32 -13
- package/templates/agent-os/universal/.claude/hooks/inject-rules.mjs +175 -7
- package/templates/agent-os/universal/.claude/hooks/lib/edit-input.mjs +503 -0
- package/templates/agent-os/universal/.claude/rules/autonomy.md +69 -1
- package/templates/agent-os/universal/.claude/rules/invariants.md +104 -19
- package/templates/agent-os/universal/.claude/rules/workflow.md +55 -5
- package/templates/agent-os/universal/.claude/scripts/decision-router.mjs +1260 -0
- package/templates/agent-os/universal/.claude/scripts/detect-missed-gate.mjs +65 -2
- package/templates/agent-os/universal/.claude/scripts/doctor.mjs +351 -0
- package/templates/agent-os/universal/.claude/scripts/git-env.mjs +49 -0
- package/templates/agent-os/universal/.claude/scripts/lib/gate-coverage.mjs +306 -0
- package/templates/agent-os/universal/.claude/scripts/lib/revalidation-points.mjs +28 -0
- package/templates/agent-os/universal/.claude/scripts/lib/secrets.mjs +490 -0
- package/templates/agent-os/universal/.claude/scripts/lib/verdict.mjs +462 -0
- package/templates/agent-os/universal/.claude/scripts/preflight.mjs +33 -17
- package/templates/agent-os/universal/.claude/scripts/queue/as-of.mjs +51 -0
- package/templates/agent-os/universal/.claude/scripts/queue/checkout.mjs +149 -0
- package/templates/agent-os/universal/.claude/scripts/queue/core.mjs +809 -28
- package/templates/agent-os/universal/.claude/scripts/queue/gate-rounds.mjs +160 -0
- package/templates/agent-os/universal/.claude/scripts/queue/github-issues.mjs +95 -16
- package/templates/agent-os/universal/.claude/scripts/queue/index.mjs +543 -15
- package/templates/agent-os/universal/.claude/scripts/queue/jira.mjs +457 -47
- package/templates/agent-os/universal/.claude/scripts/queue/plan-md.mjs +289 -25
- package/templates/agent-os/universal/.claude/scripts/queue/state.mjs +197 -0
- package/templates/agent-os/universal/.claude/scripts/revalidate.mjs +316 -0
- package/templates/agent-os/universal/.claude/scripts/revalidation-report.mjs +180 -0
- package/templates/agent-os/universal/.claude/scripts/run-journal.mjs +435 -0
- package/templates/agent-os/universal/.claude/scripts/run-state.mjs +539 -0
- package/templates/agent-os/universal/.claude/scripts/stop-flag.mjs +15 -8
- package/templates/agent-os/universal/.claude/scripts/unattended-flag.mjs +239 -0
- package/templates/agent-os/universal/.claude/scripts/verdict.mjs +198 -0
- package/templates/agent-os/universal/.claude/settings.json +11 -2
- package/templates/agent-os/universal/.claude/skills/check-premises/SKILL.md +118 -7
- package/templates/agent-os/universal/.claude/skills/loop/SKILL.md +667 -38
- package/templates/agent-os/universal/.claude/skills/pr-ship/SKILL.md +359 -26
- package/templates/agent-os/universal/.codex/agents/code-reviewer.toml +4 -0
- package/templates/agent-os/universal/.codex/agents/prose-reviewer.toml +4 -0
- package/templates/agent-os/universal/.codex/agents/security-scanner.toml +4 -0
- package/templates/agent-os/universal/.codex/agents/test-writer.toml +4 -0
- package/templates/agent-os/universal/.codex/hooks.json +70 -0
- package/templates/agent-os/universal/AGENTS.md +166 -0
- package/templates/agent-os/universal/CLAUDE.md +49 -14
- package/templates/agent-os/universal/PLAN.md +7 -40
- package/templates/agent-os/universal/docs/decisions/closing-a-task.md +98 -0
- package/templates/agent-os/universal/docs/decisions/codex-adapter.md +108 -0
- package/templates/agent-os/universal/docs/decisions/fail-open-guards.md +43 -0
- package/templates/agent-os/universal/docs/decisions/gate-coverage.md +83 -0
- package/templates/agent-os/universal/docs/decisions/review-lanes.md +72 -0
- package/templates/agent-os/universal/docs/decisions/run-directory.md +44 -0
- package/templates/agent-os/universal/docs/decisions/spacing-rations-mechanisms.md +125 -0
- package/templates/agent-os/universal/docs/decisions/stop-conditions-in-a-file.md +46 -0
- package/templates/agent-os/universal/docs/decisions/two-empty-endings.md +74 -0
- package/templates/agent-os/universal/journal/README.md +101 -0
- package/templates/agent-os/universal/layers.json +45 -2
- package/templates/hash-history.json +310 -49
- package/templates/release-ledger.json +9 -0
- package/templates/skeleton/aws-serverless/.github/workflows/deploy.yml +34 -1
- package/templates/skeleton/aws-serverless/README.md +91 -9
- package/templates/skeleton/aws-serverless/apps/web/src/lib/api.ts +9 -2
- package/templates/skeleton/aws-serverless/gitignore +37 -0
- package/templates/skeleton/aws-serverless/infra/bin/app.ts +73 -13
- package/templates/skeleton/aws-serverless/infra/lib/app-stack.ts +136 -4
- package/templates/skeleton/aws-serverless/infra/lib/web-stack.ts +14 -1
- package/templates/skeleton/aws-serverless/infra/test/allowed-origins.test.ts +301 -0
- package/templates/skeleton/aws-serverless/infra/test/app-composition.test.ts +137 -0
- package/templates/skeleton/aws-serverless/infra/test/app-stack.test.ts +36 -20
- package/templates/skeleton/aws-serverless/infra/test/web-stack.test.ts +15 -12
- package/templates/skeleton/aws-serverless/services/api/src/handlers/create-note.ts +33 -6
- package/templates/skeleton/aws-serverless/services/api/src/handlers/list-notes.ts +18 -5
- package/templates/skeleton/aws-serverless/services/api/src/usecases/create-note.ts +11 -0
- package/templates/skeleton/aws-serverless/services/api/test/create-note.handler.test.ts +120 -1
- package/templates/skeleton/aws-serverless/services/api/test/list-notes.test.ts +72 -1
- package/templates/skeleton/node-service/README.md +11 -1
- package/templates/skeleton/node-service/gitignore +34 -0
- package/templates/skeleton/node-service/packages/db/src/note-store.ts +47 -10
- package/templates/skeleton/node-service/packages/db/test/note-store.test.ts +20 -0
- package/templates/skeleton/node-service/services/api/src/main.ts +2 -9
- package/templates/skeleton/node-service/services/api/src/server.ts +93 -10
- package/templates/skeleton/node-service/services/api/src/static-dir.ts +20 -0
- package/templates/skeleton/node-service/services/api/test/server.test.ts +98 -13
- package/templates/skeleton/node-service/services/api/test/static-dir.test.ts +28 -0
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ro-debug
|
|
3
|
+
description: Investigate the deployed runtime read-only โ Lambda logs and error rates, DLQ depth and message age, table reads โ with the traps that have produced confidently-wrong diagnoses before. Use when something on the deployed stage misbehaves, and before writing any fix.
|
|
4
|
+
allowed-tools: Bash, Read, Grep
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Read-only runtime investigation
|
|
8
|
+
|
|
9
|
+
A fix is always a code change through a PR. This skill only **looks** โ and it is
|
|
10
|
+
scoped read-only so that a session diagnosing an incident cannot become a session
|
|
11
|
+
mutating production state under pressure, which is when that decision is worst.
|
|
12
|
+
|
|
13
|
+
## The role this assumes, and the honest caveat
|
|
14
|
+
|
|
15
|
+
Every command below wants a **read-only profile**: permission to read logs,
|
|
16
|
+
metrics, queue attributes and table items, and nothing else. No decrypt, no
|
|
17
|
+
secret reads, no mutations โ so a credential cannot leak through this path even
|
|
18
|
+
by accident.
|
|
19
|
+
|
|
20
|
+
โ **The skeleton does not provision that role.** It ships the application, not
|
|
21
|
+
your account's access model, and minting a role is an **owner action** (a Tier-2
|
|
22
|
+
decision โ it is IAM). Until it exists, either create it once with those four
|
|
23
|
+
read scopes, or accept that you are investigating with wider credentials than the
|
|
24
|
+
task needs and say so in the write-up. Do not silently upgrade to an admin
|
|
25
|
+
profile and carry on.
|
|
26
|
+
|
|
27
|
+
```sh
|
|
28
|
+
export AWS_PROFILE=<your read-only profile>
|
|
29
|
+
export AWS_REGION=__REGION__
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
## Before reading ANY code to explain a runtime behaviour
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
git fetch origin && git rev-parse HEAD "origin/$(git symbolic-ref --short HEAD)"
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
**If the local branch differs from its remote, diagnose from the remote**
|
|
39
|
+
(`git show origin/<branch>:<file>`, or a worktree). **A fetch does not move your
|
|
40
|
+
checkout** โ reading stale local code while describing deployed behaviour is
|
|
41
|
+
the single most reliable way to produce a confident, wrong diagnosis, and it has
|
|
42
|
+
happened more than once. See `.claude/rules/autonomy.md`, "Session staleness".
|
|
43
|
+
|
|
44
|
+
## Recipes
|
|
45
|
+
|
|
46
|
+
**Find the function first** โ CDK generates the physical names, so never guess one:
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
aws lambda list-functions \
|
|
50
|
+
--query "Functions[?contains(FunctionName,'Notes')].FunctionName" --output text
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
**Errors in a window:**
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
aws logs filter-log-events --log-group-name "/aws/lambda/<fn>" \
|
|
57
|
+
--start-time <epoch-ms> --filter-pattern "ERROR" \
|
|
58
|
+
--max-items 20 --query 'events[].message' --output text
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
**DLQ depth, then message age.** Depth comes from the queue; **age does not** โ
|
|
62
|
+
`ApproximateAgeOfOldestMessage` is a **CloudWatch metric, not an SQS attribute**,
|
|
63
|
+
and asking for it as an attribute fails with `InvalidAttributeName`:
|
|
64
|
+
|
|
65
|
+
```sh
|
|
66
|
+
aws sqs get-queue-attributes --queue-url <notes-dlq-url> \
|
|
67
|
+
--attribute-names ApproximateNumberOfMessages
|
|
68
|
+
|
|
69
|
+
aws cloudwatch get-metric-statistics --namespace AWS/SQS \
|
|
70
|
+
--metric-name ApproximateAgeOfOldestMessage \
|
|
71
|
+
--dimensions Name=QueueName,Value=<notes-dlq-name> \
|
|
72
|
+
--start-time <iso> --end-time <iso> --period 300 --statistics Maximum
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Age is what tells you whether a non-empty DLQ **predates** the thing you are
|
|
76
|
+
investigating. A days-old backlog is not your regression, and treating it as one
|
|
77
|
+
sends the whole diagnosis in the wrong direction.
|
|
78
|
+
|
|
79
|
+
**Table reads** โ key by whatever `packages/db` composes; nothing else knows the
|
|
80
|
+
key shape:
|
|
81
|
+
|
|
82
|
+
```sh
|
|
83
|
+
aws dynamodb query --table-name <NotesTable output> \
|
|
84
|
+
--key-condition-expression "pk = :pk" \
|
|
85
|
+
--expression-attribute-values '{":pk":{"S":"NOTE#<id>"}}' --max-items 3
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Error rate:** `AWS/Lambda` `Errors`, `Sum`, by `FunctionName`, over the suspect
|
|
89
|
+
window.
|
|
90
|
+
|
|
91
|
+
## Interpretation rules โ where wrong diagnoses actually come from
|
|
92
|
+
|
|
93
|
+
- ๐ด **An empty metric result is "no signal", never "healthy".** Zero datapoints
|
|
94
|
+
means **no invocations in the window** โ the function was not exercised, so the
|
|
95
|
+
metric says nothing about whether it works. Report it as no signal and go find
|
|
96
|
+
a window with traffic. This is the same rule the `post-deploy-verify` skill
|
|
97
|
+
states, and for the same reason: a vacuous pass is worse than a missing one,
|
|
98
|
+
because it gets believed.
|
|
99
|
+
- ๐ด **`StackStatus: UPDATE_COMPLETE` is stale evidence.** It persists from the
|
|
100
|
+
previous deploy, so it is true of a stack whose latest deploy failed. The
|
|
101
|
+
authoritative signals are the **deploy job's conclusion**
|
|
102
|
+
(`gh run list --workflow deploy`) and the stack's `LastUpdatedTime` freshness.
|
|
103
|
+
- **DNS and asset-upload failures are infrastructure flakes, not code
|
|
104
|
+
regressions.** A deploy that died at asset publish or checkout tells you nothing
|
|
105
|
+
about the change. Your own read-only calls can hit the same flake โ a failed
|
|
106
|
+
probe is `unknown`, not a finding.
|
|
107
|
+
- **Never work around the profile's denials โ that is the point of the profile.**
|
|
108
|
+
Needing a decrypt, a secret or a mutation means the investigation has reached
|
|
109
|
+
its boundary: escalate to the human with what you found
|
|
110
|
+
(`.claude/rules/autonomy.md`, "Escalation format").
|
|
111
|
+
|
|
112
|
+
## What to hand back
|
|
113
|
+
|
|
114
|
+
What was observed (verbatim, not summarised), which window, which signals were
|
|
115
|
+
**unavailable** and why, the current hypothesis, and the narrowest reproduction.
|
|
116
|
+
A read-only investigation that ends in a named uncertainty is a good outcome; one
|
|
117
|
+
that ends in a confident story built on a stale read is not.
|
|
@@ -17,8 +17,9 @@ run it yourself); your output is a verdict. You never fix and never deploy.
|
|
|
17
17
|
come as **BLOCKERS first, then nits** โ one list each, with the resource
|
|
18
18
|
and the rule it violates.
|
|
19
19
|
3. Your message IS the review, not a summary of it: every finding carries the
|
|
20
|
-
resource, the change, the rule, and the smallest fix. End with the verdict
|
|
21
|
-
`DEPLOY: OK`
|
|
20
|
+
resource, the change, the rule, and the smallest fix. End with the verdict
|
|
21
|
+
block below โ `SHIP` where `DEPLOY: OK` used to be, `HOLD` where
|
|
22
|
+
`DEPLOY: BLOCKED` did.
|
|
22
23
|
|
|
23
24
|
## Named rules โ blockers
|
|
24
25
|
|
|
@@ -46,4 +47,41 @@ grant helper, duplication between stacks.
|
|
|
46
47
|
- Read-only: you run `cdk diff` and read code; you never run `cdk deploy`,
|
|
47
48
|
never edit files, never mutate AWS state.
|
|
48
49
|
- An empty diff is a real finding too โ say "no infrastructure change" and
|
|
49
|
-
|
|
50
|
+
return `SHIP`, so the gate leaves a trace either way.
|
|
51
|
+
|
|
52
|
+
## The verdict block
|
|
53
|
+
|
|
54
|
+
End your report with **exactly one** fenced `json` block of this shape, and
|
|
55
|
+
nothing after it. It is what the calling gate reads.
|
|
56
|
+
|
|
57
|
+
```json
|
|
58
|
+
{
|
|
59
|
+
"gate": "cdk-diff-reviewer",
|
|
60
|
+
"verdict": "HOLD",
|
|
61
|
+
"blockers": [
|
|
62
|
+
{
|
|
63
|
+
"file": "infra/lib/api-stack.ts",
|
|
64
|
+
"line": 88,
|
|
65
|
+
"rule": "data loss",
|
|
66
|
+
"note": "the table's RemovalPolicy went to DESTROY โ replacement drops it"
|
|
67
|
+
}
|
|
68
|
+
],
|
|
69
|
+
"advisories": [],
|
|
70
|
+
"evidence": ["cdk diff against the deployed stage"],
|
|
71
|
+
"headSha": "9c1f0a7d4b3e2c5a8f6d0b9e7c4a1f2d3e5b6c70"
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
- `verdict` is `SHIP` (nothing blocking, including an empty diff), `HOLD`, or
|
|
76
|
+
`NOT_APPLICABLE` when the change touches no infrastructure at all.
|
|
77
|
+
- Every blocker names the `rule` it violates, with `file` and `line` when it has
|
|
78
|
+
a location and neither when it does not.
|
|
79
|
+
- A `HOLD` naming no blocker is **refused**, and so is a `SHIP` carrying one:
|
|
80
|
+
`node .claude/scripts/verdict.mjs check <report> cdk-diff-reviewer` is what
|
|
81
|
+
refuses them.
|
|
82
|
+
- **`headSha` is the commit you reviewed** โ `git rev-parse HEAD` in the
|
|
83
|
+
checkout you read. It is what lets `node .claude/scripts/verdict.mjs coverage
|
|
84
|
+
<commit>` tell "this gate answered for the commit being merged" from "it
|
|
85
|
+
answered two pushes ago". A verdict naming no commit is counted as neither
|
|
86
|
+
covered nor missing, so `pr-ship` holds on it โ and only `pr-ship`: no hook
|
|
87
|
+
runs that check, so a session that skips the gate skips this with it.
|
|
@@ -20,7 +20,7 @@ infra/
|
|
|
20
20
|
is drift, and drift is a defect โ reproduce it in CDK or revert it.
|
|
21
21
|
- **Every change under `infra/` passes the `cdk-diff-reviewer` agent before it
|
|
22
22
|
is deployed.** The review reads the synthesized diff (what CloudFormation
|
|
23
|
-
will do), not just the source. Deploying around
|
|
23
|
+
will do), not just the source. Deploying around that agent's `HOLD` is a
|
|
24
24
|
Never-tier action.
|
|
25
25
|
- `cdk synth` stays region-agnostic and credential-free: synth must work on any
|
|
26
26
|
machine, in CI, with nothing configured.
|
|
@@ -55,6 +55,47 @@ Report exactly one, with the evidence lines that justify it:
|
|
|
55
55
|
metric or log result means "no invocations", not "no errors"** โ name a
|
|
56
56
|
vacuous result honestly instead of reporting it as a pass.
|
|
57
57
|
|
|
58
|
+
### The verdict block
|
|
59
|
+
|
|
60
|
+
End your report with **exactly one** fenced `json` block of this shape, and
|
|
61
|
+
nothing after it. The prose above it carries the evidence a human reads; this is
|
|
62
|
+
what the caller acts on โ and what it retypes into
|
|
63
|
+
`node .claude/scripts/run-state.mjs deploy HEALTHY|REGRESSION`, which is where
|
|
64
|
+
the next selection reads the verdict.
|
|
65
|
+
|
|
66
|
+
```json
|
|
67
|
+
{
|
|
68
|
+
"gate": "post-deploy-verify",
|
|
69
|
+
"verdict": "REGRESSION",
|
|
70
|
+
"blockers": [
|
|
71
|
+
{
|
|
72
|
+
"rule": "smoke request",
|
|
73
|
+
"note": "POST /notes returned 502 twice; expected 201"
|
|
74
|
+
}
|
|
75
|
+
],
|
|
76
|
+
"advisories": [],
|
|
77
|
+
"evidence": ["stack LastUpdatedTime is this deploy", "DLQ depth 0"],
|
|
78
|
+
"headSha": "9c1f0a7d4b3e2c5a8f6d0b9e7c4a1f2d3e5b6c70"
|
|
79
|
+
}
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
- `verdict` is `HEALTHY` or `REGRESSION` โ this skill has no third answer, and
|
|
83
|
+
"could not verify" is a `REGRESSION`, never a missing verdict.
|
|
84
|
+
- A `REGRESSION` names one blocker per failed or unverifiable step, with the
|
|
85
|
+
observed output in its `note`. A step has no file, so `file` and `line` are
|
|
86
|
+
omitted here.
|
|
87
|
+
- A `REGRESSION` naming no blocker, and a `HEALTHY` carrying one, are answers
|
|
88
|
+
the caller cannot act on. The caller is what checks: the session that ran the
|
|
89
|
+
deploy runs `node .claude/scripts/verdict.mjs check <report>
|
|
90
|
+
post-deploy-verify` before it retypes the word into `run-state.mjs`. This
|
|
91
|
+
skill runs nothing โ it is read-only by construction.
|
|
92
|
+
- **`headSha` is the commit that was deployed** โ `git rev-parse HEAD` in the
|
|
93
|
+
checkout the deploy went out from. โ **Nothing reads it yet**: `run-state.mjs`
|
|
94
|
+
stores the word alone, so the commit lives in this block and nowhere else.
|
|
95
|
+
Write it anyway โ the pairing is what a later reader needs to tell a stale
|
|
96
|
+
`HEALTHY` from a current one, and `HEALTHY` is the only thing that clears the
|
|
97
|
+
`REGRESSION` latch the next selection reads.
|
|
98
|
+
|
|
58
99
|
## Boundaries
|
|
59
100
|
|
|
60
101
|
- Read-only AWS calls (`describe*`, `get*`, `list*`, `filter-log-events`) plus
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
name = "cdk-diff-reviewer"
|
|
2
|
+
description = "Reviews an infrastructure change via `cdk diff` BEFORE any deploy. MUST run on every change under infra/ โ a deploy without this review is a Never-tier action. Read-only; findings gate the deploy."
|
|
3
|
+
sandbox_mode = "read-only"
|
|
4
|
+
developer_instructions = "You review what a deploy would actually do to running infrastructure. Your\ninput is the change under `infra/` **and** the synthesized diff (`cdk diff`,\nrun it yourself); your output is a verdict. You never fix and never deploy.\n\n## How you work\n\n1. Run `cdk diff` (and read the changed `infra/` sources for intent). The diff\n is the truth: review what CloudFormation will do, not what the TypeScript\n looks like it does.\n2. Walk every resource change and flag it **by named rule** (below). Findings\n come as **BLOCKERS first, then nits** โ one list each, with the resource\n and the rule it violates.\n3. Your message IS the review, not a summary of it: every finding carries the\n resource, the change, the rule, and the smallest fix. End with the verdict\n block below โ `SHIP` where `DEPLOY: OK` used to be, `HOLD` where\n `DEPLOY: BLOCKED` did.\n\n## Named rules โ blockers\n\n- **IAM broadening.** Any policy gaining actions, resources widening to `*`,\n or a grant that outruns what a usecase does today. Least privilege is added\n in the same PR as the need, never \"for later\".\n- **Data loss paths.** A stateful resource (table, bucket, queue) being\n replaced, deleted, or flipping its RemovalPolicy toward DESTROY.\n Logical-id renames on stateful resources are replacements in disguise.\n- **Safety-net removal.** A DLQ detached, an alarm deleted or loosened, a\n retry budget widened to infinity, a dead-letter retention shortened.\n- **Blast-radius growth.** New public surface (endpoints, permissions to\n external principals), broadened network access, cross-stack exports that\n make future changes harder to reverse.\n- **Cost-relevant flips.** On-demand โ provisioned capacity, log retention to\n \"forever\", memory/timeout jumps with no stated reason.\n\n## Nits (report, do not block)\n\nNaming drift, missing descriptions, constructs that could use the narrower\ngrant helper, duplication between stacks.\n\n## Boundaries\n\n- Read-only: you run `cdk diff` and read code; you never run `cdk deploy`,\n never edit files, never mutate AWS state.\n- An empty diff is a real finding too โ say \"no infrastructure change\" and\n return `SHIP`, so the gate leaves a trace either way.\n\n## The verdict block\n\nEnd your report with **exactly one** fenced `json` block of this shape, and\nnothing after it. It is what the calling gate reads.\n\n```json\n{\n \"gate\": \"cdk-diff-reviewer\",\n \"verdict\": \"HOLD\",\n \"blockers\": [\n {\n \"file\": \"infra/lib/api-stack.ts\",\n \"line\": 88,\n \"rule\": \"data loss\",\n \"note\": \"the table's RemovalPolicy went to DESTROY โ replacement drops it\"\n }\n ],\n \"advisories\": [],\n \"evidence\": [\"cdk diff against the deployed stage\"],\n \"headSha\": \"9c1f0a7d4b3e2c5a8f6d0b9e7c4a1f2d3e5b6c70\"\n}\n```\n\n- `verdict` is `SHIP` (nothing blocking, including an empty diff), `HOLD`, or\n `NOT_APPLICABLE` when the change touches no infrastructure at all.\n- Every blocker names the `rule` it violates, with `file` and `line` when it has\n a location and neither when it does not.\n- A `HOLD` naming no blocker is **refused**, and so is a `SHIP` carrying one:\n `node .claude/scripts/verdict.mjs check <report> cdk-diff-reviewer` is what\n refuses them.\n- **`headSha` is the commit you reviewed** โ `git rev-parse HEAD` in the\n checkout you read. It is what lets `node .claude/scripts/verdict.mjs coverage\n <commit>` tell \"this gate answered for the commit being merged\" from \"it\n answered two pushes ago\". A verdict naming no commit is counted as neither\n covered nor missing, so `pr-ship` holds on it โ and only `pr-ship`: no hook\n runs that check, so a session that skips the gate skips this with it."
|
|
@@ -55,3 +55,32 @@ gh api "repos/{owner}/{repo}/commits/$SHA/check-runs" \
|
|
|
55
55
|
# must print: success (a result set containing only a scanner is NOT done)
|
|
56
56
|
```
|
|
57
57
|
|
|
58
|
+
**A head that gets no run at all is a third state, not a slow one.** A
|
|
59
|
+
`pull_request` push can register no workflow run and emit no failure signal โ
|
|
60
|
+
the head sits with a scanner only, and the poll above waits forever while the
|
|
61
|
+
previous head's green sits one line up in the same PR (AR-149: `69b5d65` on
|
|
62
|
+
#130 got no `ci` run; `8ca26e7` on #134 got `ci` and no `e2e`). Tell "not
|
|
63
|
+
registered" apart from "pending" by asking for runs by head, not by PR:
|
|
64
|
+
|
|
65
|
+
```sh
|
|
66
|
+
gh api "repos/{owner}/{repo}/actions/runs?head_sha=$SHA" \
|
|
67
|
+
-q '.workflow_runs[] | "\(.name) \(.event) \(.status) \(.conclusion)"'
|
|
68
|
+
# one line per workflow that ran for THIS sha; a required workflow missing
|
|
69
|
+
# here after a bounded wait (a few minutes) is not registered, not pending
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
โ The names differ in case between the two queries: `workflow_runs[].name`
|
|
73
|
+
is the **workflow** name (`CI`, `E2E`), the check-runs filter above reads the
|
|
74
|
+
**job** name (`ci`, `e2e`). Searching the runs list for `ci` finds nothing.
|
|
75
|
+
|
|
76
|
+
Then the rule, **per required check by name**, never per head: a check with
|
|
77
|
+
no run after the wait is **retriggered**, and the PR is **never merged on an
|
|
78
|
+
older head's green**. The retrigger depends on the workflow โ `ci.yml` has no
|
|
79
|
+
`workflow_dispatch`, so an empty commit (`git commit --allow-empty`) is the
|
|
80
|
+
simplest trigger it has; `e2e.yml` has one, so
|
|
81
|
+
`gh workflow run e2e.yml --ref <branch>` re-runs it on the same head. Pinned
|
|
82
|
+
in the generator's `test/template/pr-flow.test.ts` โ absent in a generated
|
|
83
|
+
rig โ โบ "node-ts names the head that gets no run at all, and says it is
|
|
84
|
+
retriggered rather than waited on" and โบ "the no-run branch is stated per
|
|
85
|
+
required check, not per head".
|
|
86
|
+
|
|
@@ -0,0 +1,236 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: check-premises
|
|
3
|
+
description: Check claims about the code before building on them, or before shipping them. Use after taking a queue item and before the failing test โ whenever the item asserts something exists, is missing, is broken, or works a particular way. Use again before the gate, on the run's own prose, where a behaviour claim nothing backs is UNMEASURED.
|
|
4
|
+
context: fork
|
|
5
|
+
allowed-tools: Read, Grep, Glob, Bash
|
|
6
|
+
argument-hint: <the queue item's text, or the diff and prose to check>
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
A queue item is a **claim about the code**, written by someone who was not
|
|
10
|
+
reading the code at the time. "The retry path swallows the error", "there is no
|
|
11
|
+
validation on that field", "the worker never gets the second message" โ each of
|
|
12
|
+
those is a premise, and the work that follows is only worth doing if it is true.
|
|
13
|
+
|
|
14
|
+
This skill checks the premises. It writes nothing, and it has **two entry points**.
|
|
15
|
+
|
|
16
|
+
| entry point | the claims are | the code is | verdicts |
|
|
17
|
+
| --- | --- | --- | --- |
|
|
18
|
+
| after selection, **before the Red step** | the queue item's, about code it did not read | the repository | `PREMISES HOLD` / `PREMISE FALSE` / `UNVERIFIABLE` |
|
|
19
|
+
| after the work, **before the gate** | your own, in the rulebook prose the diff touches โ and in the PR description if one exists yet | your diff | `PREMISES HOLD` / `UNVERIFIABLE` / `UNMEASURED` |
|
|
20
|
+
|
|
21
|
+
๐ด **Why the second one exists.** A claim you wrote about a mechanism you did not run
|
|
22
|
+
is cheap to write and expensive to find: `prose-reviewer` reaches it only after loading
|
|
23
|
+
the whole diff, and the fix is an edit to one sentence. Same machinery, same question โ *is this claim true?* โ pointed
|
|
24
|
+
at the text the run wrote instead of the text it was handed.
|
|
25
|
+
|
|
26
|
+
The rest of this skill is written for the first entry point. The second one runs the
|
|
27
|
+
same four steps with the diff as the code, and ยง4 carries what is different.
|
|
28
|
+
|
|
29
|
+
**What "rulebook prose" means here is not a new list** โ it is the set
|
|
30
|
+
`.claude/rules/workflow.md` already uses for the `prose-reviewer` trigger: a rule
|
|
31
|
+
file, a skill, an agent spec, a decision record, `CLAUDE.md`, the README. Where a
|
|
32
|
+
rulebook file exists twice (a template source and a generated copy), check the
|
|
33
|
+
**source**; the copy is composed from it. A comment in a test or a hook is in scope
|
|
34
|
+
too when it asserts behaviour โ the file it lives in does not change what a claim is.
|
|
35
|
+
|
|
36
|
+
๐ด **`PREMISE FALSE` belongs to the first entry point only.** At the second one the
|
|
37
|
+
claims are your own and the remedy is an edit, so a false one is not an escalation:
|
|
38
|
+
it is `UNMEASURED`'s neighbour โ delete or correct the sentence and carry on. Reading
|
|
39
|
+
it as the escalation `loop` ยง6 defines would send a finished branch back to the queue
|
|
40
|
+
over one sentence.
|
|
41
|
+
|
|
42
|
+
## Why it sits here and not in review
|
|
43
|
+
|
|
44
|
+
A false premise is not caught later. Review reads the diff against the item, and
|
|
45
|
+
both are wrong in the same direction: the item said the validation was missing,
|
|
46
|
+
the diff adds validation, the reviewer sees a diff that does what the item asked.
|
|
47
|
+
Nobody re-reads the file that had the validation all along. The cost lands as a
|
|
48
|
+
duplicate implementation, a "fix" for a bug that was somewhere else entirely, or
|
|
49
|
+
a refactor of a path that no caller reaches โ all of it green, reviewed, merged.
|
|
50
|
+
|
|
51
|
+
The check is cheap because it is narrow, and the next section is that narrowness.
|
|
52
|
+
|
|
53
|
+
## 1. Write out the claims โ as claims
|
|
54
|
+
|
|
55
|
+
List what the item asserts about the code as it exists **now**. Two to five
|
|
56
|
+
lines. Keep them in the item's own terms; do not repair them while transcribing
|
|
57
|
+
โ a claim you have already improved is one you will not test.
|
|
58
|
+
|
|
59
|
+
Separate the claims from the request. "Add a `GET /notes/:id` route" asserts
|
|
60
|
+
nothing; "the route handler bypasses the usecase layer" does.
|
|
61
|
+
|
|
62
|
+
An item that asserts nothing is done here: verdict `PREMISES HOLD`, one line
|
|
63
|
+
saying there were none. That is a common and perfectly good outcome.
|
|
64
|
+
|
|
65
|
+
## 2. Mark the load-bearing ones
|
|
66
|
+
|
|
67
|
+
๐ด **A claim is load-bearing when its falsity changes what gets built.** Only
|
|
68
|
+
those get verified. **This is not an audit** of the item, the file, or the
|
|
69
|
+
codebase โ the moment it becomes one, it stops being cheap, gets skipped under
|
|
70
|
+
time pressure, and the whole step is lost.
|
|
71
|
+
|
|
72
|
+
| Load-bearing | Not |
|
|
73
|
+
| --- | --- |
|
|
74
|
+
| "there is no X" โ if X exists, the task is already done | a stale line number in the item's description |
|
|
75
|
+
| "X is called from Y" โ if it is not, the fix goes in the wrong place | a misspelled symbol you can resolve at a glance |
|
|
76
|
+
| "X handles the empty case by Z" โ the fix is designed against Z | a claim about a file this task will not touch |
|
|
77
|
+
| "nothing enforces X" โ the whole task is the enforcement | a claim the task's own failing test would immediately expose |
|
|
78
|
+
|
|
79
|
+
That last row is the one worth internalising: a premise the Red step would
|
|
80
|
+
falsify in the next five minutes does not need checking here. This step exists
|
|
81
|
+
for the premises a passing test **would not** catch โ the ones about code the
|
|
82
|
+
task never touches.
|
|
83
|
+
|
|
84
|
+
## 3. Verify each, against the code, with a citation
|
|
85
|
+
|
|
86
|
+
Read the code. Not the tests, not the docs, not another queue item โ those are
|
|
87
|
+
claims too. Each verified premise gets a `file:line` citation; a premise you
|
|
88
|
+
believe but cannot cite is not verified, it is remembered.
|
|
89
|
+
|
|
90
|
+
At the **second** entry point this inverts for one case: a test is exactly what backs
|
|
91
|
+
a behaviour claim, so reading it is the point. The rule above is about not letting a
|
|
92
|
+
test's *name* stand in for what the code does; ยง4 says which artifacts count.
|
|
93
|
+
|
|
94
|
+
## 4. The verdict
|
|
95
|
+
|
|
96
|
+
| Verdict | When | What happens next |
|
|
97
|
+
| --- | --- | --- |
|
|
98
|
+
| `PREMISES HOLD` | every load-bearing claim checked out, or there were none | proceed to the Red step |
|
|
99
|
+
| `PREMISE FALSE` | a load-bearing claim is contradicted by the code | **stop and report** |
|
|
100
|
+
| `UNVERIFIABLE` | a load-bearing claim could not be decided from the code | report it as unverifiable, name what would decide it, and proceed only under a **labelled assumption** |
|
|
101
|
+
| `UNMEASURED` | **second entry point only:** a sentence you wrote asserts behaviour, and nothing you can point at backs it | **delete the sentence, or turn it into a pointer to the test that proves it** โ before the gate |
|
|
102
|
+
|
|
103
|
+
๐ด **The edit belongs to the calling session, not to this skill.** It reports; the
|
|
104
|
+
caller performs the exit before the gate. (The rule is the one at the top of this
|
|
105
|
+
file โ it writes nothing โ not a property of its tool grant.)
|
|
106
|
+
|
|
107
|
+
๐ด **`UNMEASURED` has exactly two exits, and "reword it" is not one of them.** A
|
|
108
|
+
behaviour claim is either backed or it is not; softening the wording keeps an
|
|
109
|
+
unbacked claim in a document agents follow literally. So either the sentence goes,
|
|
110
|
+
or it becomes `see guard-invariant.example.test.mjs โบ "blocks the violation, and the
|
|
111
|
+
reason names what to do instead"` โ the test's whole name, in a file this project
|
|
112
|
+
carries, so one grep lands on it. `invariants.md` ("State the limits") states the norm this verdict
|
|
113
|
+
enforces.
|
|
114
|
+
|
|
115
|
+
**Two questions, and they have different answers โ conflating them is how an unbacked
|
|
116
|
+
sentence survives this check.**
|
|
117
|
+
|
|
118
|
+
*Is the claim founded?* A test you can name, a command whose output is in front of
|
|
119
|
+
you, or a citation to code that does the thing. What does not count: the queue item
|
|
120
|
+
said so (the item is a claim too โ that is what the first entry point is for), it was
|
|
121
|
+
true of the previous design, or it is obviously right.
|
|
122
|
+
|
|
123
|
+
*May it stay in the file as written?* Only the two forms `invariants.md` requires โ
|
|
124
|
+
**generated** from what it describes, or a **pointer** to the test. A command's output
|
|
125
|
+
is not one of them: you saw it, the reader cannot, so a sentence resting on it is still
|
|
126
|
+
`UNMEASURED` however sure you are. Point at the test that runs that command, or say
|
|
127
|
+
something the reader can check.
|
|
128
|
+
|
|
129
|
+
โ **A measurement of this project's own history fits none of the three**, and that is
|
|
130
|
+
a real gap rather than an oversight: the run that produced it is not in the repository,
|
|
131
|
+
and a journal entry does not travel with a rulebook that ships. So a figure about past
|
|
132
|
+
runs belongs in the journal and **not** in a file other projects receive โ where it
|
|
133
|
+
would arrive with no backing at all.
|
|
134
|
+
|
|
135
|
+
A sentence is qualitative when it names **no quantity a reader could check** โ not a
|
|
136
|
+
count, not a share, not a duration. "This has happened here before" qualifies; "this
|
|
137
|
+
happened twice last month" does not, and moves to the journal.
|
|
138
|
+
|
|
139
|
+
That is not a third exit from `UNMEASURED`. The two exits apply to **the sentence in
|
|
140
|
+
front of you**: it goes, or it becomes a pointer. Writing a different sentence โ one
|
|
141
|
+
that claims no measurement, and so needs none โ is the first exit followed by an
|
|
142
|
+
unrelated edit, not a softened version of the same claim. `invariants.md` has a
|
|
143
|
+
worked instance: "Both have happened here, in the same file, within one review
|
|
144
|
+
cycle." No figure, no measurement claimed, and it ships.
|
|
145
|
+
|
|
146
|
+
๐ด **On `PREMISE FALSE` the answer is stop and report โ never quietly work around
|
|
147
|
+
the false premise by building something adjacent that seems useful.** Write what
|
|
148
|
+
the item claimed, what the code actually says with its citation, and what the
|
|
149
|
+
task might become instead. Then let a human re-aim it. The item is wrong, and an
|
|
150
|
+
agent that silently repairs a wrong item produces work nobody asked for, in a
|
|
151
|
+
branch named after a task that does not exist.
|
|
152
|
+
|
|
153
|
+
`UNVERIFIABLE` is not a soft pass. A probe that could not run tells you nothing โ
|
|
154
|
+
so the assumption travels in the open, in the item and in the PR description,
|
|
155
|
+
where the next reader can see which part of the work rests on it.
|
|
156
|
+
|
|
157
|
+
### The verdict block
|
|
158
|
+
|
|
159
|
+
Report in prose for the caller, then end with **exactly one** fenced `json` block
|
|
160
|
+
of this shape, and nothing after it. The word is spaced in prose and one token
|
|
161
|
+
inside the block; both forms are the contract.
|
|
162
|
+
|
|
163
|
+
```json
|
|
164
|
+
{
|
|
165
|
+
"gate": "check-premises",
|
|
166
|
+
"verdict": "PREMISE_FALSE",
|
|
167
|
+
"blockers": [
|
|
168
|
+
{
|
|
169
|
+
"file": "packages/core/src/note.ts",
|
|
170
|
+
"line": 17,
|
|
171
|
+
"rule": "the item claims the schema does not reject an empty title",
|
|
172
|
+
"note": "it does, here โ the reported bug is a caller that skips the validator"
|
|
173
|
+
}
|
|
174
|
+
],
|
|
175
|
+
"advisories": [],
|
|
176
|
+
"evidence": ["read the validator and both call sites"],
|
|
177
|
+
"headSha": "9c1f0a7d4b3e2c5a8f6d0b9e7c4a1f2d3e5b6c70"
|
|
178
|
+
}
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
- `verdict` is `PREMISES_HOLD`, `PREMISE_FALSE`, `UNVERIFIABLE` or `UNMEASURED`
|
|
182
|
+
โ this skill returns no other word, and none of the reviewers' words.
|
|
183
|
+
- One blocker per premise that did not hold, per claim that could not be decided,
|
|
184
|
+
or per unbacked sentence โ `rule` is the claim itself, `note` is what the code
|
|
185
|
+
actually says, and `file`/`line` is the citation ยง3 requires.
|
|
186
|
+
- `PREMISES_HOLD` carries an empty `blockers` list; the other three name at
|
|
187
|
+
least one. A stop verdict without one is not an answer the caller can act on,
|
|
188
|
+
and the caller is what checks: the `loop` skill runs
|
|
189
|
+
`node .claude/scripts/verdict.mjs check <report> check-premises` on what you
|
|
190
|
+
return. Nothing in this skill runs it โ this skill writes nothing at all.
|
|
191
|
+
- **`headSha` is the commit you read** โ `git rev-parse HEAD` in the checkout
|
|
192
|
+
you examined. A premise check answers about a tree, and the tree moves.
|
|
193
|
+
|
|
194
|
+
## Examples โ the three shapes this actually catches
|
|
195
|
+
|
|
196
|
+
**The thing already exists.** Item: "the payload schema does not reject an empty
|
|
197
|
+
title". The schema does reject it, three lines into the validator; the reported
|
|
198
|
+
bug came from a caller that never invoked the validator. Building "the missing
|
|
199
|
+
check" would have added a second, divergent rule and left the real defect โ
|
|
200
|
+
the caller โ in place. Verdict `PREMISE FALSE`; the task becomes a caller fix.
|
|
201
|
+
|
|
202
|
+
**The thing is somewhere else.** Item: "the worker retries forever because the
|
|
203
|
+
retry budget is not applied". The budget is applied, and correctly; the message
|
|
204
|
+
returns to the queue from a path above it that never consumed the budget at all.
|
|
205
|
+
The fix designed against the item would have been written in a file that was not
|
|
206
|
+
the problem. Verdict `PREMISE FALSE`.
|
|
207
|
+
|
|
208
|
+
**Nothing enforces it โ except something does.** Item: "nothing stops a handler
|
|
209
|
+
importing the storage layer directly". A hook does exactly that, and has since
|
|
210
|
+
before the item was filed. Two hours of building a second enforcement mechanism,
|
|
211
|
+
which would then have disagreed with the first. Verdict `PREMISE FALSE`.
|
|
212
|
+
|
|
213
|
+
Note what all three have in common: the resulting work would have been correct,
|
|
214
|
+
tested, reviewable, and useless. That is the failure mode this catches, and it
|
|
215
|
+
is invisible to every gate downstream.
|
|
216
|
+
|
|
217
|
+
## Limits โ stated, because a check trusted past its reach is worse than none
|
|
218
|
+
|
|
219
|
+
- **It reads the code, so it only catches what the code can contradict.** A claim
|
|
220
|
+
about runtime behaviour ("this times out in production"), about intent, or
|
|
221
|
+
about a system this repository does not contain is `UNVERIFIABLE` here, not
|
|
222
|
+
false โ say so rather than guessing.
|
|
223
|
+
- **Each entry point is one pass, at its own end of the task.** A premise that goes
|
|
224
|
+
false *between* them โ a merge lands, a dependency moves โ is a staleness stop rule
|
|
225
|
+
(`.claude/rules/autonomy.md`), not this skill. Neither pass watches the other's
|
|
226
|
+
claims.
|
|
227
|
+
- **It has no opinion on whether the task is worth doing.** True premises and a
|
|
228
|
+
pointless task is a perfectly consistent state, and it belongs to whoever fills
|
|
229
|
+
the queue.
|
|
230
|
+
- ๐ด **Nothing makes this run, and the verdict is a self-report.** No hook fires
|
|
231
|
+
when a task starts building on an unchecked claim, and no artifact outlives the
|
|
232
|
+
step โ so a run that skipped it and a run that passed it look identical
|
|
233
|
+
afterwards. That is the honest description of every rule of this shape here
|
|
234
|
+
(the `loop` skill says the same about its own no-hand-feeding rule), and it is
|
|
235
|
+
why the citation matters: a `file:line` in the report is the one part of this a
|
|
236
|
+
later reader can re-check.
|