@caddis/cli 0.2.4 → 0.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -0
- package/bundles/antigravity-plugin/agents/claude-md-curator.md +7 -4
- package/bundles/antigravity-plugin/plugin.json +1 -1
- package/bundles/antigravity-plugin/skills/_registry.md +6 -0
- package/bundles/antigravity-plugin/skills/advisory-hub/SKILL.md +183 -0
- package/bundles/antigravity-plugin/skills/advisory-hub/reference/context-template.md +214 -0
- package/bundles/antigravity-plugin/skills/advisory-hub/reference/hub-handbook.md +233 -0
- package/bundles/antigravity-plugin/skills/advisory-hub/reference/report-contract.md +94 -0
- package/bundles/antigravity-plugin/skills/advisory-hub/reference/succession.md +96 -0
- package/bundles/antigravity-plugin/skills/brainstorming/SKILL.md +1 -1
- package/bundles/antigravity-plugin/skills/code-review/SKILL.md +7 -0
- package/bundles/antigravity-plugin/skills/context-curator/SKILL.md +1 -1
- package/bundles/antigravity-plugin/skills/cross-review/SKILL.md +71 -14
- package/bundles/antigravity-plugin/skills/dashboard-perf-sweep/SKILL.md +18 -18
- package/bundles/antigravity-plugin/skills/feature-plan/SKILL.md +164 -9
- package/bundles/antigravity-plugin/skills/gate-preflight/SKILL.md +34 -0
- package/bundles/antigravity-plugin/skills/gate-review/SKILL.md +36 -0
- package/bundles/antigravity-plugin/skills/golden-plan/SKILL.md +22 -13
- package/bundles/antigravity-plugin/skills/handoff/SKILL.md +78 -0
- package/bundles/antigravity-plugin/skills/implement/SKILL.md +182 -3
- package/bundles/antigravity-plugin/skills/mermaid-diagrams/SKILL.md +57 -1
- package/bundles/antigravity-plugin/skills/park/SKILL.md +126 -0
- package/bundles/antigravity-plugin/skills/prd/SKILL.md +4 -2
- package/bundles/antigravity-plugin/skills/preflight/SKILL.md +5 -0
- package/bundles/antigravity-plugin/skills/spawn-hub/SKILL.md +287 -0
- package/bundles/antigravity-plugin/skills/validate-phase/SKILL.md +112 -0
- package/bundles/antigravity-plugin/skills/writing-plans/SKILL.md +7 -4
- package/bundles/antigravity-plugin/warm_start_agy.py +28 -2
- package/bundles/antigravity-plugin-extras/plugin.json +1 -1
- package/bundles/antigravity-plugin-extras/skills/agent-orchestration/SKILL.md +9 -9
- package/bundles/antigravity-plugin-extras/skills/cross-review/SKILL.md +56 -6
- package/bundles/antigravity-plugin-extras/skills/requesting-code-review/SKILL.md +1 -1
- package/bundles/antigravity-plugin-extras/skills/schema-migration/SKILL.md +2 -2
- package/bundles/antigravity-plugin-extras/skills/state-tracking/SKILL.md +4 -4
- package/bundles/manifest.json +3 -3
- package/dist/cli.js +55 -4
- package/package.json +1 -1
|
@@ -0,0 +1,233 @@
|
|
|
1
|
+
# Advisory Hub — how the Hub actually validates — verdict template, checklist, techniques, plan drift
|
|
2
|
+
|
|
3
|
+
> **Loaded by:** `/caddis:validate-phase` Step 2
|
|
4
|
+
>
|
|
5
|
+
> Reference for `.github/skills/workflow/advisory-hub/SKILL.md`. The core card carries the
|
|
6
|
+
> model; this carries the detail for one moment of use. Every rule here was earned by a
|
|
7
|
+
> specific failure — the failure is stated alongside it, so a later reader can tell a hard-won
|
|
8
|
+
> rule from an opinion.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## §C — the Hub verdict
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
=== HUB VERDICT — PHASE <n> ===
|
|
16
|
+
REPORT: .caddis/advisory-hub-reports/phase-NN.report.md
|
|
17
|
+
VALIDATED BY: <hub session model/lane> on <ISO 8601 UTC>
|
|
18
|
+
VERDICT: ACCEPT | ACCEPT-WITH-CORRECTION | REJECT
|
|
19
|
+
|
|
20
|
+
--- RE-DERIVATION (what the HUB ran itself, not what the report said) ---
|
|
21
|
+
CLAIM: <…>
|
|
22
|
+
HUB RAN: <the literal command the Hub executed>
|
|
23
|
+
HUB GOT: <…>
|
|
24
|
+
AGREES: YES | NO | UNVERIFIABLE — <why> (repeat per claim)
|
|
25
|
+
|
|
26
|
+
--- GATES RE-RUN ---
|
|
27
|
+
$ <command>
|
|
28
|
+
<output>
|
|
29
|
+
RESULT: PASS | FAIL | NOT RE-RUN — <why>
|
|
30
|
+
|
|
31
|
+
--- COMMITS AUDITED ---
|
|
32
|
+
$ git show --stat <sha>
|
|
33
|
+
MATCHES THE PHASE'S CLAIMED TOUCHES: YES | NO — <what was unexpected>
|
|
34
|
+
|
|
35
|
+
--- SAFETY-RULE RE-CHECK ---
|
|
36
|
+
<per rule in advisory-context §5, or N/A — none declared>
|
|
37
|
+
|
|
38
|
+
--- DEVIATIONS ADJUDICATED ---
|
|
39
|
+
<deviation> → touches locked decision <#>? YES | NO
|
|
40
|
+
→ Hub ruling: allowed | reverted | plan amended — <reason>
|
|
41
|
+
|
|
42
|
+
--- PLAN CORRECTIONS LANDED ---
|
|
43
|
+
<what was edited in the plan file itself, section by section — or NONE>
|
|
44
|
+
|
|
45
|
+
--- CORRECTIONS REQUIRED BEFORE THE NEXT PHASE ---
|
|
46
|
+
1. <exact fix, not a direction>
|
|
47
|
+
|
|
48
|
+
--- WHAT IS NOW STRUCTURAL --- (MANDATORY — a verdict without this is incomplete)
|
|
49
|
+
| Correction | Where it now lives (file + section) | The failure that earned it |
|
|
50
|
+
|---|---|---|
|
|
51
|
+
|
|
52
|
+
NEXT PHASE MAY START: YES | NO
|
|
53
|
+
=== END VERDICT ===
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
**Where does a correction live so it cannot recur?**
|
|
57
|
+
- **Telling the implementing session** — worthless. That session ends and the lesson dies with it.
|
|
58
|
+
- **Putting it in the next phase's prompt** — barely better. It works once, and the Hub must remember to
|
|
59
|
+
repeat it every phase. It won't, eventually.
|
|
60
|
+
- **Putting it in the durable artifact** (`advisory-context.md`) — correct. It binds every future phase
|
|
61
|
+
whether or not anyone remembers it, and it carries the failure that earned it so a later reader knows
|
|
62
|
+
why it exists rather than deleting it as noise.
|
|
63
|
+
|
|
64
|
+
**A verdict that does not end in "what is now structural" is half a verdict.** And **the Hub is not
|
|
65
|
+
exempt from its own contract**: if a correction traces back to advice the Hub itself gave earlier, the
|
|
66
|
+
verdict says so plainly rather than attributing it to the implementer.
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## §D — Hub validation checklist
|
|
71
|
+
|
|
72
|
+
1. **Re-derive every VALUE CHECK.** Run the re-derivation yourself. Do not read the reported result.
|
|
73
|
+
2. **Re-run the exit gates**, at minimum the fast ones (unit tests, lint, type-check). Reported output can
|
|
74
|
+
be stale even when it was honestly captured.
|
|
75
|
+
**And re-run them in a DIFFERENT ENVIRONMENT from the implementer's** — see §E's
|
|
76
|
+
"re-running a gate identically is not independent verification". Re-running the same way
|
|
77
|
+
reproduces the blind spot instead of testing it. This is the sharpest known limit on the
|
|
78
|
+
Hub's one rule, and it cost a failed production deploy to learn.
|
|
79
|
+
3. **`git log` / `git show --stat` the commits** — do the changes match the phase's claimed touches?
|
|
80
|
+
Anything unexpected, especially tooling/config/CI files that never appeared in DEVIATIONS?
|
|
81
|
+
4. **Re-check every rule in advisory-context §5** — or record `N/A — none declared`.
|
|
82
|
+
5. **Any deviation** → is it a §4 locked decision? If so it is the Hub's call, not the implementer's.
|
|
83
|
+
6. **Any surprise** → does the plan's `## Current state` need correcting? Correct it **now, in the plan
|
|
84
|
+
file** (§F).
|
|
85
|
+
7. **Deployed-environment smoke test at milestone boundaries** — against the environment the change
|
|
86
|
+
actually ships to, not dev. Skip cleanly when the project has no deployed environment.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## §E — Hub techniques that actually catch things
|
|
91
|
+
|
|
92
|
+
### Re-running a gate *identically* is not independent verification
|
|
93
|
+
|
|
94
|
+
The Hub's rule is "re-derive, don't trust". Its sharpest limit: **re-deriving the same way
|
|
95
|
+
reproduces the implementer's blind spot instead of testing it.**
|
|
96
|
+
|
|
97
|
+
Worked example. A Hub accepted a milestone partly on *"gates genuinely ran — I re-ran the test
|
|
98
|
+
suite myself, 310 passed."* The very next deploy failed on that same suite. A dependency-injected
|
|
99
|
+
service built a database client **before request validation ran**, so an invalid query parameter
|
|
100
|
+
returned 500 instead of 422 — but *only on the CI runner*, because the local env file is
|
|
101
|
+
gitignored and every developer machine supplied it. Implementer green. Hub green. CI red.
|
|
102
|
+
|
|
103
|
+
> **The rule:** for any phase touching config, a data-store seam, or a dependency-injection
|
|
104
|
+
> boundary, re-run the gate under the *deprived* environment, not just again.
|
|
105
|
+
> `mv <env file> aside && <test command> && mv back` is a one-step runner reproduction.
|
|
106
|
+
|
|
107
|
+
Two corollaries, both earned:
|
|
108
|
+
- **A regression test for this class must not depend on ambient environment.** The test that broke
|
|
109
|
+
CI was structurally incapable of catching its own bug locally. Its replacement monkeypatches the
|
|
110
|
+
dependency to raise, so it bites everywhere.
|
|
111
|
+
- **Construct nothing expensive in a dependency provider or a service `__init__`.** A validation
|
|
112
|
+
error must never need a database.
|
|
113
|
+
|
|
114
|
+
### Mutate the phase's riskiest DECISION, not just its named tests
|
|
115
|
+
|
|
116
|
+
A phase ran three mutations; one — swapping two deliberately-different windows for each other —
|
|
117
|
+
**broke nothing**, because every fixture was single-day so the two windows coincided. The suite
|
|
118
|
+
looked thorough and had a hole exactly where the phase's most consequential choice lived.
|
|
119
|
+
|
|
120
|
+
> When a phase's headline claim is *"these two things are deliberately different"*, mutate them
|
|
121
|
+
> into being the same and check that something fails.
|
|
122
|
+
|
|
123
|
+
### Mutate the GUARDRAIL, with evasion variants — not just the obvious violation
|
|
124
|
+
|
|
125
|
+
A security phase shipped a guard asserting a visibility-only permission could never become a real
|
|
126
|
+
gate. The Hub mutated it three ways:
|
|
127
|
+
|
|
128
|
+
| Variant | Caught? |
|
|
129
|
+
|---|---|
|
|
130
|
+
| `require("nav.x")` — the obvious one | yes |
|
|
131
|
+
| `require(SOME_CONST)` — indirection | yes, by a *different* test |
|
|
132
|
+
| `require("""nav.x""")` — **triple-quoted** | **no — full evasion** |
|
|
133
|
+
|
|
134
|
+
The third planted a live gate on an all-roles key and the entire suite plus both lint gates stayed
|
|
135
|
+
green. Root cause: the guard was a regex plus a substring scan, kept "two ways on purpose" — but
|
|
136
|
+
**both read the source as text, so they shared a blind spot rather than covering for each other.**
|
|
137
|
+
|
|
138
|
+
> **Two checks that can be fooled by the same trick are one check.** Parse the language, don't
|
|
139
|
+
> pattern-match its surface syntax — the parser hands back the *resolved* value, so every quoting
|
|
140
|
+
> style, including ones nobody has thought of, collapses into a single comparison.
|
|
141
|
+
|
|
142
|
+
Test a guardrail the way an adversary would, not the way its author imagined. On a phase whose
|
|
143
|
+
whole risk is "a gate that looks real and isn't", the guard against fake gates being itself fake is
|
|
144
|
+
the failure that matters most.
|
|
145
|
+
|
|
146
|
+
### A test asserting the OUTCOME does not pin the INVARIANT
|
|
147
|
+
|
|
148
|
+
Same phase. A documented invariant read *"break-glass admin resolves **before any database
|
|
149
|
+
lookup**"*. Two tests covered it — and both created the tables first, then asserted the returned
|
|
150
|
+
value. Moving the short-circuit *below* the query left every test green while destroying the
|
|
151
|
+
invariant outright.
|
|
152
|
+
|
|
153
|
+
> If the invariant is *"X happens without Y"*, the test must **withhold Y**. Asserting the result
|
|
154
|
+
> in a world where Y is present proves only that the result is reachable, not that it is
|
|
155
|
+
> independent. Here the fix was one test that never creates the tables — the omission *is* the test.
|
|
156
|
+
|
|
157
|
+
### A render/build check proves it mounted, not that it is right
|
|
158
|
+
|
|
159
|
+
A headless render returned `OK, 0 console errors` for a bar chart drawn upside down. Read the
|
|
160
|
+
screenshot, not the exit code.
|
|
161
|
+
|
|
162
|
+
### Mutation testing — the highest-value check the Hub does
|
|
163
|
+
|
|
164
|
+
**A test that passes but would not fail on the bug is worthless.** Reading a test tells you almost
|
|
165
|
+
nothing; breaking the code and watching the test fail tells you everything.
|
|
166
|
+
|
|
167
|
+
```
|
|
168
|
+
1. Back up the file the test claims to prove something about.
|
|
169
|
+
2. Remove exactly the behaviour under test — the smallest possible break
|
|
170
|
+
(delete the branch, drop the argument, invert the condition).
|
|
171
|
+
3. Run the project's test command for that test file. IT MUST FAIL.
|
|
172
|
+
4. Restore the file.
|
|
173
|
+
5. Re-run. IT MUST PASS AGAIN.
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
Do this for **every** phase that adds a test claiming to prove a behaviour. It takes about a minute, and
|
|
177
|
+
it is the difference between "6 passed" and "6 tests that mean something." **Always restore and re-run —
|
|
178
|
+
never leave a mutation in the tree.** The specific trap worth naming: dropping an argument that has a
|
|
179
|
+
default silently changes behaviour *without erroring* — exactly what a mutation test catches and a code
|
|
180
|
+
read does not.
|
|
181
|
+
|
|
182
|
+
### A reproduction harness can DOWNGRADE a finding — build it anyway
|
|
183
|
+
|
|
184
|
+
A filed finding (GLM executing an injected relay's leftover task before its own prompt) had three
|
|
185
|
+
live observations behind it on one docket, and read as confirmed. Building the harness the note itself
|
|
186
|
+
demanded — replay the real hook's captured stdout into a clean arena via a `--settings` SessionStart
|
|
187
|
+
hook, so the installed plugin can't contaminate the arm — ran it **six** times, three as baselines
|
|
188
|
+
matched to the reported conditions. **Not one reproduced.** The finding was downgraded to "not
|
|
189
|
+
established," not closed: the fixes were kept anyway (cheap, and one is structural), but the writeup
|
|
190
|
+
now says so plainly instead of letting three anecdotes read as a measurement.
|
|
191
|
+
|
|
192
|
+
> **Three observations on one project is a pattern, not a measurement.** A harness that fails to
|
|
193
|
+
> reproduce is not a wasted harness — it is the finding doing its job. Build the falsifiable test even
|
|
194
|
+
> when the finding already feels certain; "confirmed by re-reading the report" and "confirmed by trying
|
|
195
|
+
> to break it and failing" are different claims, and only the second is durable.
|
|
196
|
+
|
|
197
|
+
**The reusable A/B method, for testing a hook whose fix is already installed:** capture the real hook's
|
|
198
|
+
actual stdout (pre-fix and post-fix, run for real against a repo holding the test condition), then
|
|
199
|
+
replay each capture into a **clean arena** via a `--settings` SessionStart hook. Keep the arena's own
|
|
200
|
+
plugin relay-free, so the only relay text in play is the one under test — otherwise the installed
|
|
201
|
+
(possibly still-unpatched) plugin injects its own and contaminates the result. Each arena is its own
|
|
202
|
+
git repo, so the evidence is files and commits, not a transcript read.
|
|
203
|
+
|
|
204
|
+
### Re-run the cross-review yourself
|
|
205
|
+
|
|
206
|
+
A report's own internal contradiction (CLEAN in one section, "pending" in another) is settled in one
|
|
207
|
+
command by just running it directly. **If a cross-review provider times out, try the other configured
|
|
208
|
+
provider before recording "inconclusive."** A single provider's outage is not grounds for skipping
|
|
209
|
+
independent review — on the reference run, the second provider found the real bug the first had missed.
|
|
210
|
+
|
|
211
|
+
### Read the copy, don't trust "verbatim copy"
|
|
212
|
+
|
|
213
|
+
When a phase claims to have copied a function/config/pattern verbatim, read it line by line against the
|
|
214
|
+
original. The easiest thing to drop silently is an argument with a default — dropping it changes
|
|
215
|
+
behaviour without erroring.
|
|
216
|
+
|
|
217
|
+
### Check the report against itself
|
|
218
|
+
|
|
219
|
+
Read the report once as a *document* before validating a single claim. Internal contradictions — a
|
|
220
|
+
verdict field disagreeing with a details field, a model name that doesn't match the claimed lane — have
|
|
221
|
+
found real problems as often as checking the code has.
|
|
222
|
+
|
|
223
|
+
---
|
|
224
|
+
|
|
225
|
+
## §F — Plan drift is the Hub's job
|
|
226
|
+
|
|
227
|
+
Nothing else in caddis owns *"the plan turned out to be wrong, correct it mid-flight."* The Hub is the
|
|
228
|
+
natural owner, and it is the step most likely to be skipped. Any SURPRISE that contradicts the plan's own
|
|
229
|
+
`## Current state` gets corrected **in the plan file, now** — not noted in the verdict for later, not
|
|
230
|
+
left for the next reader to reconcile.
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Advisory Hub — the phase report contract — what an implementing session must file, verbatim
|
|
2
|
+
|
|
3
|
+
> **Loaded by:** `/caddis:implement`, in Advisory-Hub mode, at report-filing time
|
|
4
|
+
>
|
|
5
|
+
> Reference for `.github/skills/workflow/advisory-hub/SKILL.md`. The core card carries the
|
|
6
|
+
> model; this carries the detail for one moment of use. Every rule here was earned by a
|
|
7
|
+
> specific failure — the failure is stated alongside it, so a later reader can tell a hard-won
|
|
8
|
+
> rule from an opinion.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## §B — the phase report contract
|
|
13
|
+
|
|
14
|
+
Every implementation phase, in Advisory-Hub mode, ends by writing this block **to a file**
|
|
15
|
+
(`.caddis/advisory-hub-reports/phase-NN.report.md` — §H), never just to chat. **A report without re-derivable
|
|
16
|
+
evidence is not a report.**
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
=== PHASE REPORT FOR ADVISORY HUB ===
|
|
20
|
+
PLAN: <path to the plan file>
|
|
21
|
+
ADVISORY CTX: <path to <slug>-advisory-context.md>
|
|
22
|
+
PHASE: <n> — <title>
|
|
23
|
+
MILESTONE: <M<n>, or N/A>
|
|
24
|
+
MODEL/LANE: <tier> / <lane> (the model you ACTUALLY ran on)
|
|
25
|
+
DATE: <ISO 8601 UTC>
|
|
26
|
+
STATUS: COMPLETE | BLOCKED | PARTIAL
|
|
27
|
+
|
|
28
|
+
--- COMMITS ---
|
|
29
|
+
<sha> <subject> (one line per commit — including tooling,
|
|
30
|
+
config, and non-source files)
|
|
31
|
+
FILES CHANGED: <n> (+<added>/-<removed>)
|
|
32
|
+
|
|
33
|
+
--- EXIT GATE: verbatim command + verbatim output ---
|
|
34
|
+
$ <the literal command the plan names>
|
|
35
|
+
<paste the real terminal output — do not summarise, retype, or tidy it>
|
|
36
|
+
RESULT: PASS | FAIL (repeat this triple per gate)
|
|
37
|
+
|
|
38
|
+
--- VALUE CHECK ---
|
|
39
|
+
CLAIM: <the number or behaviour this phase asserts>
|
|
40
|
+
RE-DERIVED BY: <the exact independent query/command/script — copy-pasteable,
|
|
41
|
+
not a description of one>
|
|
42
|
+
RE-DERIVATION RETURNED: <what it actually returned>
|
|
43
|
+
MATCH: YES | NO
|
|
44
|
+
|
|
45
|
+
--- PROJECT SAFETY RULES ---
|
|
46
|
+
RULES IN SCOPE: <rule ids from advisory-context §5, or NONE DECLARED>
|
|
47
|
+
RULE <id>: APPLIES THIS PHASE: YES | NO — <why>
|
|
48
|
+
EVIDENCE: <the check that was run + its output, or N/A>
|
|
49
|
+
RESULT: PASS | FAIL | N/A (repeat per rule)
|
|
50
|
+
|
|
51
|
+
--- CROSS-REVIEW ---
|
|
52
|
+
PROVIDER: <deepseek | glm | …> VERDICT: CLEAN | ISSUES | NOT RUN (+ why)
|
|
53
|
+
ISSUES ADDRESSED: <list, or NONE>
|
|
54
|
+
|
|
55
|
+
--- DEVIATIONS FROM PLAN ---
|
|
56
|
+
<each deviation + why, or NONE>
|
|
57
|
+
|
|
58
|
+
--- SURPRISES / CONTRADICTIONS ---
|
|
59
|
+
<anything the real data or code contradicted in the plan's `## Current state`, or NONE>
|
|
60
|
+
<<< STOP CONDITION: if this section is non-empty, STOP after this phase.
|
|
61
|
+
The Hub owns the plan correction — you do not. >>>
|
|
62
|
+
|
|
63
|
+
--- NOT DONE ---
|
|
64
|
+
<anything in the phase left undone + why, or NONE>
|
|
65
|
+
|
|
66
|
+
--- NEXT PHASE READY: YES | NO ---
|
|
67
|
+
=== END REPORT ===
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Non-negotiables for the report itself — each earned by a real failure
|
|
71
|
+
|
|
72
|
+
- **Verbatim means verbatim.** A phase once filed a fabricated framework stack-trace line — one that
|
|
73
|
+
wasn't even valid syntax in that language — in the one section whose entire value is being trustworthy.
|
|
74
|
+
Once output is reconstructed, the Hub has to re-run everything, which defeats the section's whole
|
|
75
|
+
point. If output genuinely cannot be captured, **say so** — that is fine. Inventing it is not.
|
|
76
|
+
- **Reconcile the report against itself before filing.** One report's CROSS-REVIEW said `CLEAN` while its
|
|
77
|
+
DEVIATIONS said "verdict pending" — both cannot be true. Read your own report once as a document before
|
|
78
|
+
submitting it.
|
|
79
|
+
- **Report the model/lane you actually ran on.** One report named a first-party model for a session
|
|
80
|
+
actually running on a third-party lane. The Hub uses this field to judge whether a lane is trustworthy
|
|
81
|
+
for later phases; get it wrong here and the whole lane experiment is uninterpretable.
|
|
82
|
+
- **Report every commit**, including tooling, config, CI, and non-source files. A phase once shipped a
|
|
83
|
+
deploy-pipeline change that never appeared in its own DEVIATIONS section.
|
|
84
|
+
- **Never edit the plan to make your own gate pass.** Run it, report the failure and why, and let the Hub
|
|
85
|
+
fix the plan.
|
|
86
|
+
- **A re-derivation crib's numbers must match the final committed diff, not an earlier point in
|
|
87
|
+
implementation.** One crib was true when written against a 6-test file, then stale by filing time
|
|
88
|
+
because a late fix added 2 more tests — the same mutation against the final 8-test file gave a
|
|
89
|
+
different result. The crib wasn't fabricated or self-contradictory (none of the other five rules would
|
|
90
|
+
have caught it) — it was just stale. **If a late change touches what a crib measured, re-run the crib's
|
|
91
|
+
own commands before filing.**
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
# Advisory Hub — handing the role over — the mechanical audit, the self-check trap, the carried-open ledger
|
|
2
|
+
|
|
3
|
+
> **Loaded by:** `/caddis:spawn-hub` · `/caddis:handoff` Step 2b
|
|
4
|
+
>
|
|
5
|
+
> Reference for `.github/skills/workflow/advisory-hub/SKILL.md`. The core card carries the
|
|
6
|
+
> model; this carries the detail for one moment of use. Every rule here was earned by a
|
|
7
|
+
> specific failure — the failure is stated alongside it, so a later reader can tell a hard-won
|
|
8
|
+
> rule from an opinion.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## §G — The mechanical handoff audit
|
|
13
|
+
|
|
14
|
+
**A context doc is written BEFORE the work happens, so it is ALWAYS stale by the time it is needed.** The
|
|
15
|
+
outgoing Hub's honest first instinct, asked "can we hand off," is always *"yes, the context doc is
|
|
16
|
+
designed for exactly this"* — and that instinct has been measured and found wrong: a real audit found
|
|
17
|
+
seven gaps in a context file the outgoing Hub believed was current, including the single highest-value
|
|
18
|
+
technique the Hub had developed by then.
|
|
19
|
+
|
|
20
|
+
**Do not ask "are there gaps?" Assume there are, and find them mechanically:**
|
|
21
|
+
|
|
22
|
+
1. **List every technique, decision, and correction you actually used this session** — from your own
|
|
23
|
+
actions in this transcript, not from re-reading the context doc. Candidates: did you re-run something
|
|
24
|
+
with a second provider after a timeout? mutation-test a seam? apply a fix instead of only flagging it?
|
|
25
|
+
adjudicate a deviation against a locked decision? correct the plan? fix an index/README in passing?
|
|
26
|
+
2. **For each, check whether the context doc's *instruction itself* already covers it** — not whether the
|
|
27
|
+
word merely appears somewhere. Would a fresh Hub, reading only that file, be told to do the same
|
|
28
|
+
thing?
|
|
29
|
+
3. **The caveat that will burn you:** a literal-string grep over prose produces **false MISSINGs**. A
|
|
30
|
+
`0 matches` result means "this exact string isn't here," **not** "this isn't covered" — real content is
|
|
31
|
+
often phrased differently than the search term. **Read the section before trusting a grep's absence.**
|
|
32
|
+
Search by concept, read the surrounding paragraph, don't stop at the first non-match.
|
|
33
|
+
4. **Land every real gap as a durable rule** in the right section of the advisory-context, carrying the
|
|
34
|
+
specific failure that earned it. A note left "for later" is not a fix.
|
|
35
|
+
|
|
36
|
+
`/caddis:handoff` runs this audit as its Step 2b, whenever the active plan has a companion
|
|
37
|
+
`advisory-context.md` — see that command for the wiring.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## §I — Hub succession: the Hub is a ROLE, not a conversation
|
|
42
|
+
|
|
43
|
+
A long plan outlives any one session. The Hub runs as a **numbered series**, each handing to the
|
|
44
|
+
next through *files* — never chat history. **Write this convention down at the start, not when
|
|
45
|
+
someone asks.** On the reference project three Hubs had already run on hand-written prompts before
|
|
46
|
+
anyone noticed there was no convention; it was written only when the user asked "what were the
|
|
47
|
+
rules for spawning a new Hub?" and the answer was *there aren't any*.
|
|
48
|
+
|
|
49
|
+
Keep a table in `advisory-context.md`:
|
|
50
|
+
|
|
51
|
+
| Hub | Covered | Ended because |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| 1 | pre-Phase-0 setup | context |
|
|
54
|
+
| 2 | Phases 0–1 | context |
|
|
55
|
+
| 3 | Phases 2–6, two deploys | context |
|
|
56
|
+
| 4 | Phases 7+ | — |
|
|
57
|
+
|
|
58
|
+
### What a new Hub does before validating anything
|
|
59
|
+
|
|
60
|
+
1. **Read, in order:** `advisory-context.md` (all of it) → the plan's milestones, execution protocol
|
|
61
|
+
and tracker → the project's rate/measurement rules → the most recent 2–3 verdicts.
|
|
62
|
+
2. **Prove the context landed** by answering a self-check *before* touching a phase (§J).
|
|
63
|
+
3. **Verify the tree independently.** `git fetch`, then count divergence yourself — never trust a
|
|
64
|
+
relay file's number. Also **`git status` and diff any dirty file before treating it as
|
|
65
|
+
authority**: a plan's working copy was once found carrying an *uncommitted revert* that deleted
|
|
66
|
+
the rules a Hub was about to read, behind a clean-looking `git log`. A committed rule is only as
|
|
67
|
+
durable as the working copy the next session opens.
|
|
68
|
+
4. **Run the mechanical handoff audit (§G) early**, while context is still thick — not at the end.
|
|
69
|
+
|
|
70
|
+
### A Hub may amend its predecessor's verdict — that is the mechanism working
|
|
71
|
+
|
|
72
|
+
Amend **in place, in the verdict file, with an attributed block**, so the record shows both the
|
|
73
|
+
original judgement and why it moved. On the reference project a fresh Hub — working only from files,
|
|
74
|
+
with no shared history — downgraded its predecessor's ACCEPT to ACCEPT-WITH-CORRECTION *and* caught
|
|
75
|
+
a deploy-blocking bug that the implementer, two cross-review passes and the prior Hub had all
|
|
76
|
+
missed. **That is the single strongest argument for this pattern.**
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## §J — The context self-check, with a deliberate trap
|
|
81
|
+
|
|
82
|
+
The handoff prompt must force an incoming Hub to *prove* the context landed. Ask it to:
|
|
83
|
+
|
|
84
|
+
- **re-derive one anchor number live** (not quote it);
|
|
85
|
+
- name the decisions that are **accepted but reversible**, and what would reverse them;
|
|
86
|
+
- state where the next deploy gate is **and how it determined that**;
|
|
87
|
+
- say what it must do before its own handoff;
|
|
88
|
+
- name one thing a green automated check does **not** prove.
|
|
89
|
+
|
|
90
|
+
**Include at least one question whose naive answer is wrong.** The reference project's is *"what is
|
|
91
|
+
the funnel's `offered` count, and why is it NOT `SUM(OfferedInd)`?"* — the naive read gives 3,814,
|
|
92
|
+
the correct one 4,783, because the raw flags do not nest. A Hub that only skimmed **fails visibly
|
|
93
|
+
instead of silently**, which is the entire point.
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
@@ -37,7 +37,7 @@ Start by understanding the current project context, then ask questions one at a
|
|
|
37
37
|
## After the Design
|
|
38
38
|
|
|
39
39
|
**Documentation:**
|
|
40
|
-
- Write the validated design to
|
|
40
|
+
- Write the validated design to `.caddis/plans/YYYY-MM-DD-<topic>-design.md`
|
|
41
41
|
- Apply `.github/instructions/document-frontmatter.instructions.md` when creating or updating the design doc
|
|
42
42
|
- Use elements-of-style:writing-clearly-and-concisely skill if available
|
|
43
43
|
- Commit the design document to git
|
|
@@ -120,3 +120,10 @@ try {
|
|
|
120
120
|
- [ ] No obvious performance issues
|
|
121
121
|
- [ ] Code is readable and documented
|
|
122
122
|
- [ ] Breaking changes documented
|
|
123
|
+
|
|
124
|
+
## Machine Verdict (automated / headless runs)
|
|
125
|
+
|
|
126
|
+
**Then, as the very last line of your output (nothing after it), emit the machine verdict:**
|
|
127
|
+
`REVIEW: CLEAN` (no blocking issues found across the categories above) or `REVIEW: BLOCKING` (one
|
|
128
|
+
or more blocking issues). Automated runners read only this line; it must exactly match one of
|
|
129
|
+
those two forms.
|
|
@@ -55,7 +55,7 @@ If the context package is written as a standalone Markdown handoff or context do
|
|
|
55
55
|
| Intent Document | `agent-docs/intents/<feature>.md` | Goal, Constraints, Success Criteria |
|
|
56
56
|
| Pipeline state | `.github/pipeline-state.json` | Current stage, approved gates |
|
|
57
57
|
| Project config | `project-config.md` | Tech stack, constraints, brand tokens |
|
|
58
|
-
| Current plan step | `.
|
|
58
|
+
| Current plan step | `.caddis/plans/<feature>.md` | Current phase/step only |
|
|
59
59
|
|
|
60
60
|
### Tier 2 — Include When Relevant (SHOULD READ)
|
|
61
61
|
| Item | Source | What to extract |
|
|
@@ -10,25 +10,78 @@ Run a cross-vendor review of the current changes with a DIFFERENT model family t
|
|
|
10
10
|
is green and before you commit, or for a second opinion on a risky diff. This is the *cross-vendor*
|
|
11
11
|
complement to `/caddis:code-review`, not a replacement.
|
|
12
12
|
|
|
13
|
-
##
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
13
|
+
## Step 0 — check the key BEFORE reading the diff
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
python "$TOOL" --check-config # resolve $TOOL first, see "Run it" below
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Exit 0 means ready; it prints which providers have a key. **Exit 3 means nothing is configured —
|
|
20
|
+
STOP and ask the user now.** Do not proceed to the review, and do not silently skip it.
|
|
21
|
+
|
|
22
|
+
Ask with `AskUserQuestion`: which provider to set up (DeepSeek, GLM, or another OpenAI-compatible
|
|
23
|
+
endpoint), then write the key they give you to `~/.caddis/keys.env` as `DEEPSEEK_API_KEY=<key>` (or
|
|
24
|
+
the matching variable) and re-run the check. **The keys file lives outside every git repo, so it
|
|
25
|
+
cannot be committed by accident. Never put a key in the repo, in a command, or in your reply.**
|
|
26
|
+
|
|
27
|
+
**Why asking is the fix, and why the script does not do it.** This used to fail at the point of use
|
|
28
|
+
— mid-task, with a commit pending, which is the worst moment to go hunting for a credential. A
|
|
29
|
+
command that fails once at an inconvenient moment does not get retried, so the safety check is
|
|
30
|
+
quietly lost and nothing records that it was lost. But `oss_review.py` also runs headless, from CI
|
|
31
|
+
and from `claude -p` sessions, where a prompt on stdin does not ask a question — it hangs forever.
|
|
32
|
+
So the script answers *"is it configured?"* and **you** do the asking.
|
|
33
|
+
|
|
34
|
+
If a key exists for a different provider than the one requested, the error names it and gives you
|
|
35
|
+
the exact flag (`--provider glm`). Read the message before concluding anything is missing.
|
|
36
|
+
|
|
37
|
+
**Key precedence:** `REVIEW_API_KEY` > the provider's own env var (`DEEPSEEK_API_KEY` /
|
|
38
|
+
`GLM_API_KEY`) > `OSS_API_KEY` > the keys file (`~/.caddis/keys.env`, overridable via
|
|
39
|
+
`CADDIS_KEYS_FILE`). Details: the `coding/cross-review` skill and the **Providers & keys** guide.
|
|
18
40
|
|
|
19
41
|
## Run it
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
42
|
+
Resolve the tool path deterministically — check only these two exact locations, in order, and
|
|
43
|
+
**never search the filesystem for it** (see Rules below):
|
|
44
|
+
```bash
|
|
45
|
+
# THE PLUGIN COPY WINS. It is the one `caddis update` refreshes; a vendored copy is a snapshot
|
|
46
|
+
# of whatever caddis shipped the day someone copied it, and it never updates again.
|
|
47
|
+
if [ -f "${CLAUDE_PLUGIN_ROOT}/scripts/oss_review.py" ]; then
|
|
48
|
+
TOOL="${CLAUDE_PLUGIN_ROOT}/scripts/oss_review.py"
|
|
49
|
+
# A vendored copy that DIFFERS is a live hazard, not a curiosity — say so, do not stay silent.
|
|
50
|
+
if [ -f ".github/tools/oss_review.py" ] && ! diff -q ".github/tools/oss_review.py" "$TOOL" >/dev/null 2>&1; then
|
|
51
|
+
echo "[cross-review] WARNING: .github/tools/oss_review.py differs from the caddis copy and is"
|
|
52
|
+
echo " being IGNORED. Delete it, or keep the local change deliberately."
|
|
53
|
+
fi
|
|
54
|
+
elif [ -f ".github/tools/oss_review.py" ]; then
|
|
55
|
+
TOOL=".github/tools/oss_review.py" # no plugin present — the vendored copy is all there is
|
|
56
|
+
else
|
|
57
|
+
echo "oss_review.py not found at either known location — report this and stop. Do NOT search for it."
|
|
58
|
+
exit 3
|
|
59
|
+
fi
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
> **Why the plugin copy is first.** It used to be second. On 2026-08-10 a repo ran a stale vendored
|
|
63
|
+
> `oss_review.py` and got `REVIEW: CLEAN` on a database write path that was never reviewed. The bug
|
|
64
|
+
> had been fixed on 2026-08-01 — the fix just never reached the caller, because a file someone
|
|
65
|
+
> copied weeks earlier silently won. `caddis_gate.py vendor-drift` now fails on the same condition.
|
|
24
66
|
```
|
|
25
|
-
python
|
|
26
|
-
python
|
|
27
|
-
python
|
|
67
|
+
python "$TOOL" # DeepSeek eyes (default)
|
|
68
|
+
python "$TOOL" --provider glm # GLM eyes
|
|
69
|
+
python "$TOOL" --range $ARGUMENTS # review a git range, e.g. origin/main..HEAD
|
|
28
70
|
```
|
|
29
71
|
Optional: `--base-url <url>`, `--model <id>` (env always overrides the preset).
|
|
30
|
-
|
|
31
|
-
|
|
72
|
+
|
|
73
|
+
**A large diff is batched, not refused.** Over `REVIEW_MAX_DIFF_CHARS` (default 60,000) the tool splits
|
|
74
|
+
on whole-file boundaries and reviews each batch, and the verdict is CLEAN only if every batch is. Exit 2
|
|
75
|
+
now means the narrower case — a **single file** too big to split, or more batches than the cap. When you
|
|
76
|
+
see it, the fix is usually to review that one file separately, not to narrow `--range`; don't raise
|
|
77
|
+
`--max-diff-chars` blind.
|
|
78
|
+
|
|
79
|
+
**Do not alternate providers on a schedule.** DeepSeek is the default because it has actually found real
|
|
80
|
+
bugs that same-vendor passes missed; GLM is the documented fallback, used automatically when DeepSeek is
|
|
81
|
+
unavailable and you did not name a provider. The policy lives in `oss_review.py`'s `FALLBACK_PROVIDERS`
|
|
82
|
+
comment — a rota that sends half your phases to a provider observed timing out three phases running buys
|
|
83
|
+
variety it never actually collects, and a review that silently does not happen is the same failure class
|
|
84
|
+
as a diff that silently is not read. Name a provider explicitly only when you have a reason.
|
|
32
85
|
|
|
33
86
|
## Interpret the exit code
|
|
34
87
|
- **0 — REVIEW: CLEAN** → no blocking issues. Proceed.
|
|
@@ -41,3 +94,7 @@ families have different blind spots, and alternation maximizes what the pair cat
|
|
|
41
94
|
- Read-only second opinion — the tool never edits, commits, or pushes. YOU apply fixes in the main thread.
|
|
42
95
|
- Treat exit 2/3 as blocking-unknown, never as approval (the tool is fail-closed by design).
|
|
43
96
|
- A different vendor means a different style; weigh its findings on merit, don't cargo-cult them.
|
|
97
|
+
- **Never use `find`, `Get-ChildItem -Recurse`, or any other filesystem-wide search to locate
|
|
98
|
+
`oss_review.py`.** Check only the two paths above. On Git Bash under Windows, `find /` (or any
|
|
99
|
+
search rooted at `/`) walks every mounted drive, not just the repo — it can hang or take minutes.
|
|
100
|
+
If neither path exists, stop and report it; don't go hunting for the file.
|
|
@@ -7,9 +7,9 @@ description: Re-measure an app's dashboard/API endpoints for load-time regressio
|
|
|
7
7
|
# dashboard-perf-sweep — re-measure, then judge the shape, not just the time
|
|
8
8
|
|
|
9
9
|
Today an app "earns" the rs-kit refresher pattern only when someone happens to be doing unrelated
|
|
10
|
-
work on it, hits a slow page, and measures it properly. That worked for
|
|
11
|
-
but a snapshot measurement goes stale — a page added last month, or a table that crossed a
|
|
12
|
-
threshold since, has no mechanism to surface itself. This skill re-measures on demand.
|
|
10
|
+
work on it, hits a slow page, and measures it properly. That worked for a couple of apps in the
|
|
11
|
+
fleet, but a snapshot measurement goes stale — a page added last month, or a table that crossed a
|
|
12
|
+
growth threshold since, has no mechanism to surface itself. This skill re-measures on demand.
|
|
13
13
|
|
|
14
14
|
**What already exists and must NOT be rebuilt:** every app in the fleet has its own
|
|
15
15
|
`.caddis/kb/dashboard-performance-tracker.md` with real baseline measurements. The gap this closes
|
|
@@ -40,14 +40,14 @@ growing. Time alone is not sufficient evidence — the fleet's own history prove
|
|
|
40
40
|
|
|
41
41
|
| App | Cold time | Root cause | rs-kit? |
|
|
42
42
|
|---|---|---|---|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
46
|
-
|
|
|
47
|
-
|
|
|
43
|
+
| App A | worst >60s, 5 real breaches | recurring full-table aggregation | **YES** |
|
|
44
|
+
| App B (pre-fix) | 14s–60s+ across several endpoints | recurring full-table aggregation | **YES** |
|
|
45
|
+
| App C | 10.3s | a single ~158MB LOB payload on 9.1k rows, not aggregation | **NO** — SQL aggregate + gzip |
|
|
46
|
+
| App D | 1.9s | cold JSON parse, no DB at all in prod | **NO** — warm the provider |
|
|
47
|
+
| App E | 202ms | n/a — never crossed gate 1 | **NO** |
|
|
48
48
|
|
|
49
|
-
A naive "slow ⇒ recommend rs-kit" tool would have wrongly flagged
|
|
50
|
-
|
|
49
|
+
A naive "slow ⇒ recommend rs-kit" tool would have wrongly flagged App C and App D. That already
|
|
50
|
+
happened once before the shape distinction was made explicit — don't repeat it.
|
|
51
51
|
|
|
52
52
|
## Step 1 — build the endpoint list for this app
|
|
53
53
|
|
|
@@ -103,12 +103,12 @@ Force these questions, in order, for each slow endpoint — don't skip to a verd
|
|
|
103
103
|
1. **Does response size explain most of the time** (large payload, not a slow query)? → not rs-kit;
|
|
104
104
|
recommend trimming fields / gzip / virtualization.
|
|
105
105
|
2. **Is the slow part parsing/serialization, not the DB round-trip?** → not rs-kit; recommend
|
|
106
|
-
caching/warming the parsed form (
|
|
106
|
+
caching/warming the parsed form (App D precedent).
|
|
107
107
|
3. **Does the query scale with total stored rows rather than the response size** — read the actual
|
|
108
108
|
SQL/query plan, not just the timing — **and is the same aggregation recomputed per request on data
|
|
109
109
|
that keeps growing?** → candidate for rs-kit.
|
|
110
110
|
4. **Is it slow only at deep pagination (OFFSET)?** → not rs-kit; recommend keyset paging
|
|
111
|
-
(`exception_id`-style cursor,
|
|
111
|
+
(`exception_id`-style cursor, App B precedent).
|
|
112
112
|
5. **Is a specific WHERE clause unseekable** (function-wrapped column, NVARCHAR/VARCHAR param
|
|
113
113
|
mismatch)? → not rs-kit; recommend an index or a param-type fix.
|
|
114
114
|
|
|
@@ -124,9 +124,9 @@ being able to see a before/after. Print a summary alongside the file edit.
|
|
|
124
124
|
|
|
125
125
|
## Validating this skill against known ground truth
|
|
126
126
|
|
|
127
|
-
Before trusting a new app's output, or after changing the checklist, run the sweep against
|
|
128
|
-
already-known
|
|
129
|
-
|
|
130
|
-
conclude "aggregation-shaped, rs-kit candidate" for the pages
|
|
131
|
-
disagrees with the known-good conclusion for either, that's a
|
|
132
|
-
rationalize the disagreement away.
|
|
127
|
+
Before trusting a new app's output, or after changing the checklist, run the sweep against two
|
|
128
|
+
already-known apps in your fleet and confirm it reproduces the **already-established, human-verified**
|
|
129
|
+
conclusion for each: one that's genuinely fast (should conclude "not a candidate") and one with a
|
|
130
|
+
known aggregation breach (should conclude "aggregation-shaped, rs-kit candidate" for the pages
|
|
131
|
+
already identified). If the output disagrees with the known-good conclusion for either, that's a
|
|
132
|
+
checklist bug — fix the checklist, don't rationalize the disagreement away.
|