@clipboard-health/ai-rules 2.41.2 → 2.43.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/skills/cb-babysit/SKILL.md +4 -4
- package/skills/cb-babysit/scripts/_sentinel.sh +1 -1
- package/skills/cb-ship/SKILL.md +7 -7
- package/skills/cb-ship/references/pr-template.md +1 -1
- package/skills/flaky-closeout/SKILL.md +281 -0
- package/skills/flaky-closeout/references/acceptance-report-2026-07-17.md +110 -0
- package/skills/flaky-closeout/references/comment-templates.md +58 -0
- package/skills/flaky-triage/SKILL.md +58 -17
package/package.json
CHANGED
|
@@ -19,9 +19,9 @@ Resolve bundled "./scripts" paths relative to SKILL.md.
|
|
|
19
19
|
|
|
20
20
|
The skill uses two sentinels with visible footer lines.
|
|
21
21
|
|
|
22
|
-
**Addressed sentinel**: `<sub>🤖 <code>cb-babysit:addressed v1 core@3.
|
|
22
|
+
**Addressed sentinel**: `<sub>🤖 <code>cb-babysit:addressed v1 core@3.22.0</code></sub>`. Appended on its own line at the end of every reply the skill posts so re-runs know which threads and review-body comments are already handled. Dedupe also recognizes the legacy `babysit-pr:addressed v1` prefix from before this skill was renamed.
|
|
23
23
|
|
|
24
|
-
**Follow-up sentinel**: `<sub>🤖 <code>cb-babysit:followup v1 core@3.
|
|
24
|
+
**Follow-up sentinel**: `<sub>🤖 <code>cb-babysit:followup v1 core@3.22.0</code></sub>`. Attached to replies that defer an out-of-scope comment as a tracked follow-up. The sentinel is additive: the post-reply scripts still append the `addressed` sentinel at the end.
|
|
25
25
|
|
|
26
26
|
**Sentinel recency rules.** The script emits a per-thread `activityState` with three values. Step 6a owns the handling rules for each state.
|
|
27
27
|
|
|
@@ -247,7 +247,7 @@ Body templates (the script appends the `addressed` sentinel if missing):
|
|
|
247
247
|
- **Agree**: `Addressed in <commit-url>. <one-line what-changed>.`
|
|
248
248
|
- **Disagree**: `Leaving current behavior. <reasoning>.`
|
|
249
249
|
- **Already fixed**: `Already handled by <commit-url-or-file:line>. <brief pointer>.`
|
|
250
|
-
- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>🤖 <code>cb-babysit:followup v1 core@3.
|
|
250
|
+
- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>🤖 <code>cb-babysit:followup v1 core@3.22.0</code></sub>`
|
|
251
251
|
|
|
252
252
|
For Defer replies, include the follow-up sentinel on its own line as shown. The script will append the `addressed` sentinel after it on its own line, so the final body ends with the follow-up sentinel followed by a blank line followed by the `addressed` sentinel. `grep cb-babysit:followup` finds the deferral and `grep cb-babysit:addressed` still marks the thread handled for dedupe.
|
|
253
253
|
|
|
@@ -263,7 +263,7 @@ The PR-level summary should:
|
|
|
263
263
|
|
|
264
264
|
- Group by source. Use `## Review-body findings` for step-7 work and `## Conversation-tab comments` for step-6b work. Omit a section if its list is empty.
|
|
265
265
|
- Inside each section, group verdicts under **Agree / Disagree / Already fixed / Deferred (out of scope)** subheadings. Omit a subheading if its list is empty.
|
|
266
|
-
- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>🤖 <code>cb-babysit:followup v1 core@3.
|
|
266
|
+
- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>🤖 <code>cb-babysit:followup v1 core@3.22.0</code></sub>` so grep catches them individually.
|
|
267
267
|
- Include the commit URL for fixes.
|
|
268
268
|
- End with a fenced fingerprint block listing every current fingerprint (addressed and deferred) one per line. Include both `reviewBodyComments[].fingerprint` (whole-body, one per automated review) and `activeIssueComments[].fingerprint` (per Conversation-tab comment). Future runs dedupe by matching these against `priorBabysitSentinels`.
|
|
269
269
|
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
SENTINEL_PREFIX='cb-babysit:addressed v1 '
|
|
12
12
|
LEGACY_SENTINEL_PREFIX='babysit-pr:addressed v1 '
|
|
13
|
-
SENTINEL='<sub>🤖 <code>cb-babysit:addressed v1 core@3.
|
|
13
|
+
SENTINEL='<sub>🤖 <code>cb-babysit:addressed v1 core@3.22.0</code></sub>'
|
|
14
14
|
|
|
15
15
|
# Bot author allowlist (JSON array literal). Used by unresolvedPrComments.sh
|
|
16
16
|
# as a fallback when GraphQL's `author.__typename == "Bot"` misses a GitHub
|
package/skills/cb-ship/SKILL.md
CHANGED
|
@@ -28,22 +28,22 @@ Resolve bundled "./references" paths relative to SKILL.md.
|
|
|
28
28
|
7. Output:
|
|
29
29
|
|
|
30
30
|
```plaintext
|
|
31
|
-
- PR: [Complete url] (e.g., `https://github.com/clipboardhealth/core-utils/pull/123`)
|
|
32
|
-
|
|
33
31
|
[One concise sentence describing what was implemented and shipped.]
|
|
34
32
|
|
|
35
33
|
[Optional bullets for the most important user-facing outcomes. Omit when the sentence is sufficient.]
|
|
36
34
|
|
|
37
|
-
|
|
35
|
+
Details:
|
|
38
36
|
|
|
39
37
|
- Directory: [Absolute path]
|
|
40
38
|
- Branch: [Branch name]
|
|
41
39
|
- Review: [N findings — M applied, K dismissed with a one-line reason each] (omit if step 4 skipped or found nothing)
|
|
42
40
|
- Session: [Resume command from 6a] (omit if none)
|
|
43
41
|
- Existing changes: [Uncommitted changes that remain outside the shipped scope] (omit if none)
|
|
42
|
+
|
|
43
|
+
🚀 [Complete url] (e.g., `https://github.com/clipboardhealth/core-utils/pull/123`) 🚀
|
|
44
44
|
```
|
|
45
45
|
|
|
46
|
-
Keep the PR as the
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
`
|
|
46
|
+
Keep the PR as the final line and visually separate it from everything else
|
|
47
|
+
with a blank line. Use the middle section only for meaningful behavior or
|
|
48
|
+
outcome details. Put lower-priority handoff metadata and caveats under
|
|
49
|
+
`Details`.
|
|
@@ -17,5 +17,5 @@ Concisely explain the user intent from session history and the meaningful behavi
|
|
|
17
17
|
|
|
18
18
|
Optional, don't fabricate: ticket links, rollout plan, residual risk, or specific areas for reviewers to focus.
|
|
19
19
|
|
|
20
|
-
<sub>🤖 <code>cb-ship:created v1 core@3.
|
|
20
|
+
<sub>🤖 <code>cb-ship:created v1 core@3.22.0</code></sub>
|
|
21
21
|
```
|
|
@@ -0,0 +1,281 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: flaky-closeout
|
|
3
|
+
description: Sweep the flaky-test pipeline after fixes merge by closing out the root-cause KB, closing clearly superseded tickets as Done under D3, and re-dispatching critic-rejected plans for amendment. Use for the recurring flaky close-out task, a post-merge flaky-fix sweep, or a dry-run/backtest of close-out decisions.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
<!-- cspell:ignore backtest backtesting groundcrew redispatch worktree -->
|
|
7
|
+
|
|
8
|
+
# Flaky Closeout
|
|
9
|
+
|
|
10
|
+
Run three post-merge sweeps in one read-first, write-later pass. This stage
|
|
11
|
+
maintains the pipeline's learning and dispatch loops; it does not diagnose new
|
|
12
|
+
root causes or implement product fixes.
|
|
13
|
+
|
|
14
|
+
Read these files before deciding or writing:
|
|
15
|
+
|
|
16
|
+
- `../flaky-debug/references/fix.md` for the KB close-out contract.
|
|
17
|
+
- `../flaky-debug/references/root-cause-kb/README.md` and every plausible KB
|
|
18
|
+
entry.
|
|
19
|
+
- `../flaky-critic/SKILL.md` and its rubric for D3, marker, and second-reject
|
|
20
|
+
semantics.
|
|
21
|
+
- `../create-groundcrew-ticket/SKILL.md` for dispatch eligibility.
|
|
22
|
+
- `../blind-backtest/SKILL.md` when running `--backtest`.
|
|
23
|
+
- [Comment templates](./references/comment-templates.md) before posting.
|
|
24
|
+
|
|
25
|
+
## Invariants
|
|
26
|
+
|
|
27
|
+
- Never cancel a ticket. Cancel authority remains human-only.
|
|
28
|
+
- Close a superseded ticket as **Done**, not Canceled or Duplicate. Done enters
|
|
29
|
+
the 21-day verifying window so the recurrence loop can reopen a false
|
|
30
|
+
supersession.
|
|
31
|
+
- Require a clear mechanism match before D3 closure: the same KB entry or
|
|
32
|
+
causal mechanism **and** the same failure-signature class.
|
|
33
|
+
- Decide the complete action ledger before any external write.
|
|
34
|
+
- Search existing comments for the exact marker before every write. A marker is
|
|
35
|
+
a lease for that action, not a substitute for checking current state.
|
|
36
|
+
- Batch all KB changes from one run into one focused `core-utils` PR. Never
|
|
37
|
+
commit directly to the default branch and never auto-merge.
|
|
38
|
+
- Do not schedule or re-arm this skill. When invoked by a recurring task, follow
|
|
39
|
+
re-arm instructions from the task description only after the digest.
|
|
40
|
+
|
|
41
|
+
## Modes and window
|
|
42
|
+
|
|
43
|
+
Support these modes:
|
|
44
|
+
|
|
45
|
+
- **Production:** perform reads, build the ledger, then write.
|
|
46
|
+
- **`--dry-run`:** perform every read and decision, but make no Linear, GitHub,
|
|
47
|
+
git, or repository write. Print the exact proposed comments, state changes,
|
|
48
|
+
and KB diff plan. Represent the unavailable KB PR URL as
|
|
49
|
+
`<future KB PR for run <run-key>>`.
|
|
50
|
+
- **`--backtest`:** imply `--dry-run` and follow `../blind-backtest/SKILL.md`.
|
|
51
|
+
Freeze the requested historical cutoff and keep judgments isolated from the
|
|
52
|
+
answer key. If separate assembler and judge contexts are unavailable, stop
|
|
53
|
+
and report that the blind protocol cannot be satisfied.
|
|
54
|
+
|
|
55
|
+
Before the first production run for a new rule revision, run a backtest and a
|
|
56
|
+
current dry run as separate invocations. Do not promote from historical results
|
|
57
|
+
alone.
|
|
58
|
+
|
|
59
|
+
Use an explicit `--since` value or the last successful-run timestamp supplied
|
|
60
|
+
by the recurring task, subtracting three days for overlap. If neither exists,
|
|
61
|
+
look back 21 days. Accept `--bounce-age-days`; default to 3. Print the effective
|
|
62
|
+
window, overlap, cutoff, and bounce age in the digest.
|
|
63
|
+
|
|
64
|
+
## Phase 1: Build the source manifest
|
|
65
|
+
|
|
66
|
+
Start these independent discovery reads concurrently:
|
|
67
|
+
|
|
68
|
+
- primary merged-PR search;
|
|
69
|
+
- missing-label Linear fallback;
|
|
70
|
+
- non-terminal flaky-ticket universe;
|
|
71
|
+
- bounced-ticket queue.
|
|
72
|
+
|
|
73
|
+
After discovery, deduplicate ticket IDs and fetch each full issue and comment set
|
|
74
|
+
once. Cache the records for the entire run.
|
|
75
|
+
|
|
76
|
+
### Merged-fix manifest
|
|
77
|
+
|
|
78
|
+
Search merged PRs across the `ClipboardHealth` organization:
|
|
79
|
+
|
|
80
|
+
1. Primary source: every merged PR carrying `flaky-test-fix` in the effective
|
|
81
|
+
window. Paginate; do not trust a single result page.
|
|
82
|
+
2. Label-hygiene fallback: recent `flaky-implementation` and chronic deep-dive
|
|
83
|
+
tickets whose linked PR merged in the window and demonstrably fixed a flaky
|
|
84
|
+
mechanism, even when the PR omitted the label. Require the PR body or Linear
|
|
85
|
+
attachment to identify the source ticket. Record a missing-label digest
|
|
86
|
+
risk; do not mutate PR labels.
|
|
87
|
+
3. Exclude closed-unmerged PRs. Deduplicate by repository and PR number.
|
|
88
|
+
|
|
89
|
+
Build a cheap first-pass PR record:
|
|
90
|
+
|
|
91
|
+
- repository, PR URL, merge timestamp, merge commit, title, and labels;
|
|
92
|
+
- source Linear ticket ID from the PR body or attachment;
|
|
93
|
+
- cached source ticket ID and related-ticket IDs.
|
|
94
|
+
|
|
95
|
+
Hydrate changed files, diff, detailed validation evidence, and full related
|
|
96
|
+
tickets only after a pending KB close-out or plausible D3 join makes the PR
|
|
97
|
+
actionable. From the hydrated records, extract `KB match:`, pending
|
|
98
|
+
`Knowledge-base close-out`, causal mechanism, signature class, validation
|
|
99
|
+
evidence, and every flake sighting timestamp.
|
|
100
|
+
|
|
101
|
+
If a PR lacks a source ticket ID, report it as unresolved and take no close-out
|
|
102
|
+
action for it.
|
|
103
|
+
|
|
104
|
+
### Bounce manifest
|
|
105
|
+
|
|
106
|
+
List `flaky-implementation` tickets in Triage. Fetch each issue and all comments;
|
|
107
|
+
list results truncate descriptions.
|
|
108
|
+
|
|
109
|
+
Keep a ticket only when it has:
|
|
110
|
+
|
|
111
|
+
- a critic marker with `mode=enforce` and `verdict=reject`; current markers also
|
|
112
|
+
carry `disposition=amend-and-resubmit` or
|
|
113
|
+
`disposition=substantive-reject`, while legacy markers may omit disposition;
|
|
114
|
+
- a verdict older than the bounce-age threshold;
|
|
115
|
+
- no later plan amendment that addresses the verdict;
|
|
116
|
+
- no completed close-out re-dispatch for that verdict.
|
|
117
|
+
|
|
118
|
+
Use the verdict comment ID when available; otherwise use its timestamp. A later
|
|
119
|
+
issue edit is only a possible amendment: read the current plan and confirm that
|
|
120
|
+
it addresses each cited rule before skipping re-dispatch.
|
|
121
|
+
|
|
122
|
+
Treat `verdict=needs-human disposition=second-reject` as terminal
|
|
123
|
+
`needs-human`. Also classify two enforce-mode rejects separated by a plan
|
|
124
|
+
amendment as `needs-human` when processing older marker formats. Never
|
|
125
|
+
re-dispatch either case; include it in the digest for Rocky.
|
|
126
|
+
|
|
127
|
+
A re-dispatch marker on a ticket that is still Groundcrew-ineligible means
|
|
128
|
+
`eligibility mutation pending`, not `skip`. Resume only the missing eligibility
|
|
129
|
+
changes; do not post the comment again.
|
|
130
|
+
|
|
131
|
+
Phase 1 is complete only when every manifest item has full ticket descriptions
|
|
132
|
+
and comments. Do not decide against truncated list results.
|
|
133
|
+
|
|
134
|
+
## Phase 2: Sweep KB close-outs
|
|
135
|
+
|
|
136
|
+
Inspect both the source ticket and related or superseded implementation tickets.
|
|
137
|
+
A pending close-out can live on a different fingerprint family than the PR that
|
|
138
|
+
fixed its mechanism.
|
|
139
|
+
|
|
140
|
+
For every pending close-out:
|
|
141
|
+
|
|
142
|
+
1. Confirm the target PR merged. If not, leave it pending and report it.
|
|
143
|
+
2. Resolve the plan's `KB match:` to an existing entry. If the plan says none,
|
|
144
|
+
create a new entry only when the evidenced causal terminus is genuinely new.
|
|
145
|
+
3. Compare the plan, merged diff, validation, and related recurrence. Prepare
|
|
146
|
+
the required dry-run statement from `flaky-debug/references/fix.md`.
|
|
147
|
+
4. Record:
|
|
148
|
+
- the recurrence or variant addressed;
|
|
149
|
+
- the landed fix and PR;
|
|
150
|
+
- any newly established partial or failed-scope lesson;
|
|
151
|
+
- affected repository/surface, evidence, current status, and `Last reviewed`;
|
|
152
|
+
- symptom-index changes only when the new signature is more discriminating.
|
|
153
|
+
5. Deduplicate updates by KB entry and mechanism. One PR can close several
|
|
154
|
+
tickets; one KB entry should receive one coherent edit.
|
|
155
|
+
|
|
156
|
+
Use this completed-action marker only after the focused KB PR exists:
|
|
157
|
+
|
|
158
|
+
```html
|
|
159
|
+
<!-- flaky-closeout: kb-done pr=<full-core-utils-pr-url> -->
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
After the action ledger is complete, sort each KB action as
|
|
163
|
+
`<entry-path>|<source-PR>|<affected-ticket-IDs>` and hash the resulting list.
|
|
164
|
+
Derive the run key from the recurring task's scheduled occurrence ID or exact
|
|
165
|
+
UTC cutoff timestamp plus the first 12 characters of that ledger hash. Use
|
|
166
|
+
deterministic branch `flaky-closeout/<run-key>` and put this marker in the PR
|
|
167
|
+
body:
|
|
168
|
+
|
|
169
|
+
```html
|
|
170
|
+
<!-- flaky-closeout: kb-run=<run-key> ledger=<full-hash> -->
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Before creating a branch or PR, search that exact head branch. Reuse an open or
|
|
174
|
+
merged PR only when its trusted PR-body marker has the same run key and full
|
|
175
|
+
ledger hash. A different ledger always gets a different run key, even on the
|
|
176
|
+
same day. This verified PR lookup is the idempotency key when a prior run
|
|
177
|
+
created the KB PR but failed before commenting on every ticket.
|
|
178
|
+
|
|
179
|
+
In production, create a fresh `core-utils` worktree on that branch, apply every
|
|
180
|
+
recorded KB edit, run `node --run sync-ai-rules` and `node --run verify` with the
|
|
181
|
+
repository Node version, and ship one PR through the repository's
|
|
182
|
+
`cb-work`/`cb-ship` workflow. Link the PR in a close-out comment on every
|
|
183
|
+
affected ticket. If the PR cannot be created, do not post the marker; report
|
|
184
|
+
the failure and leave the close-out retryable. If the PR exists, reuse it and
|
|
185
|
+
retry only missing ticket comments.
|
|
186
|
+
|
|
187
|
+
## Phase 3: Sweep supersessions under D3
|
|
188
|
+
|
|
189
|
+
From the Phase 1 cache, build indexes over every non-terminal ticket labeled
|
|
190
|
+
`flaky-investigation` or `flaky-implementation`: normalized KB entry, causal
|
|
191
|
+
mechanism, signature class, source PR, and fingerprint family. Join every
|
|
192
|
+
merged fix against those indexes. Do not limit the join to the source
|
|
193
|
+
fingerprint family and do not repeat the organization-wide ticket scan per PR.
|
|
194
|
+
|
|
195
|
+
Use the KB mechanism as the cross-family join key:
|
|
196
|
+
|
|
197
|
+
1. Resolve each candidate's `KB match:` line. If it has no line, require
|
|
198
|
+
equivalent explicit causal evidence in its plan; text similarity alone is
|
|
199
|
+
insufficient.
|
|
200
|
+
2. Confirm the merged PR fixes the same causal locus, not merely a nearby
|
|
201
|
+
symptom.
|
|
202
|
+
3. Confirm the signature class matches, such as provider-topology remount,
|
|
203
|
+
query-row overlay teardown, Cognito alias lookup, or CDC readiness.
|
|
204
|
+
4. Parse the candidate's earliest relevant sighting timestamp and prove it is
|
|
205
|
+
earlier than the PR merge timestamp. If no reliable sighting time exists, do
|
|
206
|
+
not close.
|
|
207
|
+
5. Check that the ticket has no post-merge recurrence for that mechanism.
|
|
208
|
+
|
|
209
|
+
Classify the candidate:
|
|
210
|
+
|
|
211
|
+
- **Clear match:** stage the D3 comment from
|
|
212
|
+
[comment templates](./references/comment-templates.md), then set the ticket to
|
|
213
|
+
Done. Include the PR, mechanism, sighting and merge timestamps, 21-day
|
|
214
|
+
verifying window, and recurrence-loop reopen safety net.
|
|
215
|
+
- **Ambiguous:** stage one possible-supersession comment tagging Rocky and leave
|
|
216
|
+
state unchanged.
|
|
217
|
+
- **No match:** take no action.
|
|
218
|
+
|
|
219
|
+
Markers:
|
|
220
|
+
|
|
221
|
+
```html
|
|
222
|
+
<!-- flaky-closeout: superseded-by=<full-source-pr-url> -->
|
|
223
|
+
<!-- flaky-closeout: possible-supersession pr=<full-source-pr-url> -->
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Post the clear-match comment before setting Done. If the state update fails,
|
|
227
|
+
retain the comment marker and retry only the missing state change on the next
|
|
228
|
+
run. A supersession marker on a still-open ticket therefore means
|
|
229
|
+
`state-change pending`, not `skip`. Never change a ticket to Canceled.
|
|
230
|
+
|
|
231
|
+
## Phase 4: Sweep bounced plans
|
|
232
|
+
|
|
233
|
+
For each eligible first-bounce ticket:
|
|
234
|
+
|
|
235
|
+
1. Quote the decisive critic verdict and rule IDs.
|
|
236
|
+
2. State the exact plan amendment required; do not propose the implementation
|
|
237
|
+
fix yourself.
|
|
238
|
+
3. Validate every field against the eligibility contract in
|
|
239
|
+
`../create-groundcrew-ticket/SKILL.md`.
|
|
240
|
+
4. Post the re-plan instruction with the verdict-specific marker, then make only
|
|
241
|
+
the state, assignee, and label changes required to restore eligibility. Do
|
|
242
|
+
not rewrite repository, parent, or blocker semantics.
|
|
243
|
+
|
|
244
|
+
Do not re-dispatch when a later plan already addresses the verdict, the same
|
|
245
|
+
verdict marker exists, or this is the second bounce. Leave second bounces in
|
|
246
|
+
Triage and list them as `needs-human`.
|
|
247
|
+
|
|
248
|
+
## Phase 5: Execute and verify
|
|
249
|
+
|
|
250
|
+
In production, execute the already-decided ledger in this order:
|
|
251
|
+
|
|
252
|
+
1. Create the single KB PR and add KB close-out comments.
|
|
253
|
+
2. Post D3 or possible-supersession comments and apply D3 state changes.
|
|
254
|
+
3. Post bounce instructions and restore Groundcrew eligibility.
|
|
255
|
+
|
|
256
|
+
Execute each ticket's ordered mutation bundle, then refetch it once. Verify only
|
|
257
|
+
the marker, state, assignee, labels, relations, and linked PR fields that bundle
|
|
258
|
+
could change. Preserve individual mutation failures in the ledger so the next
|
|
259
|
+
run can reconcile marker versus state. Continue independent actions after one
|
|
260
|
+
failure, but never claim a failed action succeeded.
|
|
261
|
+
|
|
262
|
+
In dry-run or backtest, print the same ordered ledger with exact comment bodies
|
|
263
|
+
and state mutations, prefixed `WOULD`; write nothing.
|
|
264
|
+
|
|
265
|
+
## Phase 6: Digest
|
|
266
|
+
|
|
267
|
+
End with:
|
|
268
|
+
|
|
269
|
+
- effective window, mode, merged PR count, source-ticket resolution count, and
|
|
270
|
+
missing-label fallbacks;
|
|
271
|
+
- KB entries and tickets updated, pending, skipped by marker, and KB PR URL;
|
|
272
|
+
- D3 closed, possible supersessions, no-match candidates, and state failures;
|
|
273
|
+
- bounce candidates, re-dispatched, already amended, marker-skipped, and
|
|
274
|
+
needs-human;
|
|
275
|
+
- every ambiguity, missing timestamp, missing ticket ID, incomplete evidence,
|
|
276
|
+
or failed write.
|
|
277
|
+
|
|
278
|
+
When backtesting, use the blind-backtest output contract. See the checked-in
|
|
279
|
+
[2026-07-17 acceptance report](./references/acceptance-report-2026-07-17.md).
|
|
280
|
+
It records a targeted historical check, not a reusable backtest fixture or
|
|
281
|
+
approval for future live rule revisions.
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
<!-- cspell:ignore backtest -->
|
|
2
|
+
|
|
3
|
+
# 2026-07-17 Acceptance Report
|
|
4
|
+
|
|
5
|
+
This is a targeted, write-free acceptance check for the handoff examples. It is
|
|
6
|
+
not a promotion-grade blind backtest, a reusable test fixture, or a replacement
|
|
7
|
+
for live shadow agreement.
|
|
8
|
+
|
|
9
|
+
## Cohort and cutoff
|
|
10
|
+
|
|
11
|
+
- Window: `2026-06-26T00:00:00Z` through `2026-07-17T14:30:00Z`.
|
|
12
|
+
- Primary PR scan: 31 merged `flaky-test-fix` PRs: 23 in
|
|
13
|
+
`cbh-admin-frontend`, five in `cbh-mobile-app`, and three in
|
|
14
|
+
`clipboard-health`.
|
|
15
|
+
- Source-ticket resolution: 30 of 31 primary PRs carry a `STAFF-*` ID in the
|
|
16
|
+
title or body. `clipboard-health#26528` is unresolved and therefore produces
|
|
17
|
+
no action.
|
|
18
|
+
- Merged fix: `ClipboardHealth/cbh-admin-frontend#7608`, merged
|
|
19
|
+
`2026-07-17T13:08:25Z`, source ticket `STAFF-1846`.
|
|
20
|
+
- Supersession candidate: `STAFF-1856`, sighting
|
|
21
|
+
`2026-07-16T14:30:36Z`.
|
|
22
|
+
- Bounce cohort: `STAFF-1860`, `STAFF-1862`, `STAFF-1868`, `STAFF-1873`, and
|
|
23
|
+
`STAFF-1874`; critic verdicts were posted around `2026-07-17T12:09Z`.
|
|
24
|
+
- Bounce evaluation time: `2026-07-17T14:30:00Z` with
|
|
25
|
+
`--bounce-age-days=0`. This override tests verdict detection and routing;
|
|
26
|
+
production retains the three-day default so later amendments can suppress
|
|
27
|
+
re-dispatch.
|
|
28
|
+
|
|
29
|
+
The decision pass uses ticket plans, verdict text, PR body and diff metadata,
|
|
30
|
+
merge time, and sighting time. Human close-out outcomes are used only in the
|
|
31
|
+
result comparison.
|
|
32
|
+
|
|
33
|
+
The KB contract landed at the end of this window. A full-description Linear
|
|
34
|
+
search for `KB close-out` found two implementation plans: `STAFF-1856`, whose
|
|
35
|
+
mechanism was fixed by merged #7608, and `STAFF-1874`, whose proposed fix had
|
|
36
|
+
not merged and therefore remains pending. The bounce scan found the five named
|
|
37
|
+
Triage tickets below.
|
|
38
|
+
|
|
39
|
+
## Proposed dry-run actions
|
|
40
|
+
|
|
41
|
+
### KB close-out
|
|
42
|
+
|
|
43
|
+
`STAFF-1856` contains a pending close-out for
|
|
44
|
+
`query-driven-list-dialog-teardown.md`. PR #7608 fixes the provider-topology
|
|
45
|
+
variant that remounted the authenticated route above the open Home Health menu.
|
|
46
|
+
|
|
47
|
+
Expected update:
|
|
48
|
+
|
|
49
|
+
- Add the provider-topology recurrence as another replaceable-ancestor surface.
|
|
50
|
+
- Record #7608 and its delayed or immediate-token regression coverage.
|
|
51
|
+
- Record that the dialog-only hoist in #7574 was correctly scoped but could not
|
|
52
|
+
prevent a remount from a higher provider boundary.
|
|
53
|
+
- Link the resulting focused `core-utils` KB PR from `STAFF-1856` and
|
|
54
|
+
`STAFF-1846`.
|
|
55
|
+
|
|
56
|
+
### D3 supersession
|
|
57
|
+
|
|
58
|
+
Would close `STAFF-1856` as Done under D3:
|
|
59
|
+
|
|
60
|
+
- same terminal cause: token-dependent `KnockProviderWrapper` topology switch
|
|
61
|
+
remounts the authenticated route;
|
|
62
|
+
- same signature class: an already-resolved menu item detaches before its user
|
|
63
|
+
action starts;
|
|
64
|
+
- sighting `2026-07-16T14:30:36Z` predates merge
|
|
65
|
+
`2026-07-17T13:08:25Z`;
|
|
66
|
+
- #7608 changed the same composition files named by the plan and added the
|
|
67
|
+
requested token-resolution regression.
|
|
68
|
+
|
|
69
|
+
The PR lacked the `flaky-test-fix` label, so it is admitted only through the
|
|
70
|
+
linked flaky deep-dive ticket fallback and reported as a label-hygiene risk.
|
|
71
|
+
|
|
72
|
+
### Bounce re-dispatch
|
|
73
|
+
|
|
74
|
+
Would re-dispatch each first bounce to Todo for plan amendment:
|
|
75
|
+
|
|
76
|
+
| Ticket | Verdict gap to carry into the re-plan instruction |
|
|
77
|
+
| ---------- | ----------------------------------------------------------------------------------------- |
|
|
78
|
+
| STAFF-1860 | C1 requires same-identity reconciliation after ambiguous 504, fresh identity on collision |
|
|
79
|
+
| STAFF-1862 | A3 plan must state fake-timer restoration in `afterEach` |
|
|
80
|
+
| STAFF-1868 | B1 forbids token-minting refresh retries without non-amplifying control |
|
|
81
|
+
| STAFF-1873 | B1/B6/B8 require no uncapped 429 retry and a terminated causal chain |
|
|
82
|
+
| STAFF-1874 | B1/B6/B8 require no uncapped 429 retry and a terminated owning-service cause |
|
|
83
|
+
|
|
84
|
+
No ticket had a prior closeout re-dispatch marker or a second critic rejection
|
|
85
|
+
at the frozen cutoff. Their critic comments use the legacy
|
|
86
|
+
`verdict=reject mode=enforce` marker without a disposition field; the bounce
|
|
87
|
+
scan accepts that enforce-only shape while still excluding shadow markers.
|
|
88
|
+
|
|
89
|
+
## Outcome comparison
|
|
90
|
+
|
|
91
|
+
- **D3:** match. Rocky's `STAFF-1856` close-out cited #7608, the exact mechanism,
|
|
92
|
+
the pre-merge sighting, Done, the 21-day verifying window, and recurrence
|
|
93
|
+
reopen behavior.
|
|
94
|
+
- **KB:** match. The human close-out explicitly transferred the
|
|
95
|
+
provider-topology recurrence, #7608, and the dialog-only-hoist scope lesson to
|
|
96
|
+
the KB close-out.
|
|
97
|
+
- **Bounces:** all five verdicts are detected with the required amendment
|
|
98
|
+
content. This report evaluates routing, not whether the later planner
|
|
99
|
+
produced an acceptable amendment.
|
|
100
|
+
|
|
101
|
+
## Limitations
|
|
102
|
+
|
|
103
|
+
- The cohort was selected from named handoff examples and is not balanced.
|
|
104
|
+
- The #7608 missing-label fallback is necessary for this case but needs live
|
|
105
|
+
monitoring for false inclusion.
|
|
106
|
+
- The acceptance check does not grade ambiguous supersession precision,
|
|
107
|
+
second-bounce handling, KB edit quality across multiple entries, or live
|
|
108
|
+
tool/write idempotency.
|
|
109
|
+
- Before production scheduling, run a larger blind backtest and a live
|
|
110
|
+
write-free shadow pass.
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
<!-- cspell:ignore redispatch -->
|
|
2
|
+
|
|
3
|
+
# Flaky Closeout Comment Templates
|
|
4
|
+
|
|
5
|
+
Fill every placeholder from fetched evidence. Do not post a state-changing
|
|
6
|
+
template with an unknown timestamp, mechanism, or PR. A possible-supersession
|
|
7
|
+
comment may use its explicit missing-timestamp statement because it leaves state
|
|
8
|
+
unchanged.
|
|
9
|
+
|
|
10
|
+
## KB close-out
|
|
11
|
+
|
|
12
|
+
```markdown
|
|
13
|
+
<Use the canonical final close-out statement from
|
|
14
|
+
flaky-debug/references/fix.md verbatim.>
|
|
15
|
+
|
|
16
|
+
<!-- flaky-closeout: kb-done pr=<KB PR URL> -->
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Clear D3 supersession
|
|
20
|
+
|
|
21
|
+
```markdown
|
|
22
|
+
Already fixed (D3): covered by merged PR <source PR URL> from <source ticket>.
|
|
23
|
+
It fixes the same mechanism: <mechanism and causal locus>. This ticket's
|
|
24
|
+
<signature class> sighting at <sighting timestamp> predates the merge at
|
|
25
|
+
<merge timestamp>.
|
|
26
|
+
|
|
27
|
+
Closing Done: the family enters the 21-day verifying window. If this mechanism
|
|
28
|
+
flakes again, the recurrence loop reopens the ticket with its history.
|
|
29
|
+
|
|
30
|
+
<!-- flaky-closeout: superseded-by=<source PR URL> -->
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## Possible supersession
|
|
34
|
+
|
|
35
|
+
```markdown
|
|
36
|
+
@Rocky Warren possible supersession: merged PR <source PR URL> may cover this
|
|
37
|
+
ticket through <shared mechanism>, but <specific ambiguity> prevents a D3
|
|
38
|
+
closure. The relevant sighting is <timestamp or missing-timestamp statement>.
|
|
39
|
+
Please confirm the mechanism match; state is unchanged.
|
|
40
|
+
|
|
41
|
+
<!-- flaky-closeout: possible-supersession pr=<source PR URL> -->
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## Bounce re-dispatch
|
|
45
|
+
|
|
46
|
+
```markdown
|
|
47
|
+
Re-dispatching for plan amendment after the critic verdict at <verdict link or
|
|
48
|
+
timestamp>.
|
|
49
|
+
|
|
50
|
+
Required amendment:
|
|
51
|
+
|
|
52
|
+
- <rule ID>: <specific missing or contradicted plan statement>
|
|
53
|
+
|
|
54
|
+
Amend the plan against the verdict, preserve the existing investigation
|
|
55
|
+
evidence, and resubmit it to the critic before implementation.
|
|
56
|
+
|
|
57
|
+
<!-- flaky-closeout: bounce-redispatch verdict=<comment ID or timestamp> -->
|
|
58
|
+
```
|
|
@@ -1,31 +1,35 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: flaky-triage
|
|
3
|
-
description: Scheduled triage stage between flake-intake and agent dispatch. Clusters flaky-investigation tickets by shared root cause, closes duplicates and infra blips, and releases one canonical ticket per cluster for dispatch. Use when running the recurring flaky-triage task, when asked to triage the flaky queue, or to backtest clustering against historical tickets.
|
|
3
|
+
description: Scheduled triage stage between flake-intake and agent dispatch. Clusters flaky-investigation tickets by shared root cause, closes duplicates and infra blips, and releases one repository-owned canonical ticket per cluster for dispatch. Use when running the recurring flaky-triage task, when asked to triage the flaky queue, or to backtest clustering against historical tickets.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
<!-- cspell:words backtest backtesting deflake dispatchable groundcrew -->
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
The pipeline stage between flake-intake (deterministic, string-level collapse: bursts and storms) and per-ticket dispatch. Intake creates investigation tickets in the **Triage** state, where groundcrew ignores them. This skill clusters them semantically — same root cause, even when error text differs — then performs ticket surgery and releases exactly one dispatchable ticket per root cause and implementation repository.
|
|
9
|
+
|
|
10
|
+
This skill routes and classifies known mechanisms from existing ticket evidence; it never performs a new source-level diagnosis. Do not propose fixes, read test source files, or run flaky-debug. Diagnosis happens later, on the released ticket, through the normal dispatch flow.
|
|
9
11
|
|
|
10
12
|
## Rules
|
|
11
13
|
|
|
12
|
-
- One canonical ticket per root cause, never one per sighting (rubric B5).
|
|
14
|
+
- One repository-owned canonical ticket per root cause, never one per sighting within an implementation repository (rubric B5).
|
|
15
|
+
- One implementation ticket per mechanism cluster per repository. A cross-repo mechanism match coordinates repository-owned fixes; it never consolidates them across repositories.
|
|
13
16
|
- Never hold a ticket hostage: anything you cannot confidently cluster is released as a singleton. The worst case must equal the no-triage baseline.
|
|
14
17
|
- Load `../flaky-critic/references/rubric.md` before citing close-out rules. Cite rubric rule IDs (B5, D1, D2, D3) in every close-out comment.
|
|
15
18
|
- Decide the full cluster picture before writing anything to Linear.
|
|
16
19
|
- Keep bulky data in local scratch files; keep only the manifest and decisions in context.
|
|
17
|
-
- All Linear writes are idempotent
|
|
20
|
+
- All Linear writes are idempotent. Use exact markers only to deduplicate comments. Immediately before state, label, relation, or marker-comment writes, re-read the current value and skip only when the desired state, label, relation, or exact marker already exists.
|
|
21
|
+
- Production Linear writes are single-flight. The recurring task or caller must serialize the entire Phase 4 write pass; if exclusive execution cannot be established, remain in dry-run and report the blocker. After an API conflict, re-fetch the affected tickets and retry only actions that are still missing.
|
|
18
22
|
|
|
19
23
|
**Rubric IDs** (cite the matching ID in every close-out comment):
|
|
20
24
|
|
|
21
|
-
- **B5** — one canonical ticket per root cause, never one per sighting.
|
|
25
|
+
- **B5** — one repository-owned canonical ticket per root cause, never one per sighting within an implementation repository.
|
|
22
26
|
- **D1** — infra/environment incident: cancel with incident window + correlation evidence + why no code fix applies + re-arm note.
|
|
23
27
|
- **D2** — duplicate of canonical: matching evidence + no new implementation ticket needed beyond the canonical.
|
|
24
28
|
- **D3** — already-fixed: matches an investigation/implementation ticket closed in the last 14 days.
|
|
25
29
|
|
|
26
30
|
## Phase 1: Queue
|
|
27
31
|
|
|
28
|
-
Fetch candidate tickets from Linear: project `Groundcrew`, label `flaky-investigation`, state `Triage`. (During migration, also include state `Todo` with no assignee.)
|
|
32
|
+
Fetch candidate tickets from Linear: project `Groundcrew`, label `flaky-investigation`, state `Triage`. (During migration, also include state `Todo` with no assignee.) A marker is not enough to skip a ticket: verify that its promised disposition is reflected in the issue state and relations. If a marked ticket remains in the queue, treat it as a partial action, reconstruct the intended disposition, skip completed writes, and resume the rest. If the queue is empty, skip to the digest.
|
|
29
33
|
|
|
30
34
|
Use `list_issues` only to enumerate the queue; fetch each candidate with `get_issue` — list results truncate descriptions and will silently drop the Flake details JSON. Fetch candidate details in parallel when the available tooling supports it.
|
|
31
35
|
|
|
@@ -37,7 +41,7 @@ Phase 1 is done when every non-skipped candidate has a manifest row — don't ca
|
|
|
37
41
|
|
|
38
42
|
Fetch what is already being worked, to catch duplicates before they spawn plans. Scope searches to the repos, files, normalized errors, helper names, and prior tickets from the Phase 1 manifest first; widen only when the scoped search leaves a plausible duplicate unresolved.
|
|
39
43
|
|
|
40
|
-
- Open Linear tickets labeled `flaky-implementation` (any state but terminal).
|
|
44
|
+
- Open Linear tickets labeled `flaky-implementation` (any state but terminal). Collect plausible IDs first, then fetch each with `get_issue` and read its comments in parallel when the available tooling supports it; list results truncate the plan, `KB match:` statement, critic approval, and PR context needed for mechanism coordination.
|
|
41
45
|
- PRs: search `flaky-test-fix` in both `open` and `closed` states; treat closed-unmerged siblings as duplicate signals, not already-fixed proof. Also try title keywords `flaky`/`deflake`.
|
|
42
46
|
- Open `[storm:` / `[burst:` tickets and their contained fingerprints.
|
|
43
47
|
- Investigation/implementation tickets closed in the last 14 days, for already-fixed matches only after confirming the merged fix is on main and the sighting predates it (D3).
|
|
@@ -60,22 +64,59 @@ Cluster by strongest shared evidence first:
|
|
|
60
64
|
|
|
61
65
|
Do not merge clusters merely because failures share a run. Same-run failures can have different root causes.
|
|
62
66
|
|
|
63
|
-
|
|
67
|
+
### Knowledge-base mechanism coordination
|
|
68
|
+
|
|
69
|
+
Finish the root-cause clusters above before this second pass. Same-repo duplicate decisions take precedence; knowledge-base coordination must not rescue a duplicate or split one root cause into several tickets. Before the lookup, use the Phase 4 rules and chronic threshold to plan every disposition without writing to Linear.
|
|
70
|
+
|
|
71
|
+
For every canonical that the plan would release, plus every plausibly related in-flight implementation ticket from Phase 2:
|
|
72
|
+
|
|
73
|
+
1. Open `../flaky-debug/references/root-cause-kb/README.md` once. Compare every member's failure signature, failure surface, and available causal evidence with the symptom index, collect the union of plausible entries, then read each unique entry once in full, including **What failed and why**.
|
|
74
|
+
2. For each matched entry, widen the Phase 2 in-flight search across repositories using the entry link, entry title, and mechanism terms. Fetch every non-terminal `flaky-implementation` match before deciding the mechanism cluster.
|
|
75
|
+
3. Record one of the exact `KB match:` statements defined in `../flaky-debug/SKILL.md` Phase 1b, including the failed fixes to avoid when an entry matches. Perform the lookup at triage time; do not trust a copied or stale plan line.
|
|
76
|
+
4. Group tickets only when the evidence matches the same KB entry and mechanism. A shared symptom without the entry's mechanism is insufficient. Record the entry link, mechanism, ticket IDs, repositories, and whether each member is a provisional canonical or in-flight implementation.
|
|
77
|
+
|
|
78
|
+
A mechanism cluster exists when at least two surviving provisional canonicals in the run, or one surviving provisional canonical plus an in-flight implementation ticket, match the same entry. The members may represent different fingerprint families. Do not merge members from different repositories, close one as a duplicate of the other, or create a cross-repo canonical. Each repository keeps its own implementation ticket.
|
|
79
|
+
|
|
80
|
+
When several members are in the same repository, apply the existing root-cause and D2 rules first so at most one canonical per mechanism and repository remains. A KB match alone is not D2 evidence; if the root-cause evidence does not support deduplication, keep the tickets independent and record the ambiguity as a risk instead of forcing them into one mechanism cluster. `KB match: none` cannot establish a production mechanism cluster. Record a likely shared mechanism with no matching entry as a KB gap in risks and in dry-run/backtest output; it does not block either ticket's normal release.
|
|
81
|
+
|
|
82
|
+
## Phase 4: Plan, coordinate, then act
|
|
83
|
+
|
|
84
|
+
First, choose exactly one intended disposition per cluster using the ordered rules below. Record the actions without writing to Linear.
|
|
64
85
|
|
|
65
|
-
1. **Matches in-flight work** — error family or
|
|
66
|
-
2. **Matches an open storm/burst ticket** — normalized error matches its signature or contained fingerprints: close members as Duplicate of the storm/burst ticket so late stragglers join the existing event.
|
|
67
|
-
3. **Infra blip / environment incident** — an incident cluster (evidence tier 7): use the Phase 3 threshold above. In production runs, corroborate via Phase 2 (Datadog monitor history / error spike through `pup` when available; passed-on-retry signals). Default
|
|
68
|
-
4. **
|
|
69
|
-
5. **Singleton or uncertain** — release as-is with a brief `<!-- flaky-triage: singleton -->` comment.
|
|
86
|
+
1. **Matches in-flight work** — within the same repository, the error family or root cause matches an open implementation ticket or open PR: plan to close every member as Duplicate linked to that canonical and comment citing B5/D2 with the match evidence. Plan no release. A shared KB mechanism alone is not D2 evidence, especially across repositories; handle it through mechanism coordination below.
|
|
87
|
+
2. **Matches an open storm/burst ticket** — normalized error matches its signature or contained fingerprints: plan to close members as Duplicate of the storm/burst ticket so late stragglers join the existing event.
|
|
88
|
+
3. **Infra blip / environment incident** — an incident cluster (evidence tier 7): use the Phase 3 threshold above. In production runs, corroborate via Phase 2 (Datadog monitor history / error spike through `pup` when available; passed-on-retry signals). Default disposition: plan to close all members as Canceled with a D1 comment. If the incident exposes a hardening gap (e.g., seed path lacks diagnostics), plan to release ONE canonical for that hardening and dup the rest to it — never one per helper.
|
|
89
|
+
4. **Same-repository multi-ticket cluster** — within each implementation repository, pick the canonical (richest evidence: most failures, clearest artifacts). Plan to comment the cluster manifest on it: member IDs, shared mechanism, evidence tier from Phase 3, and the marker `<!-- flaky-triage: cluster=<short-slug> -->`. Plan to close only the same-repository members as Duplicate of the canonical (D2 format) and release the canonical. When a non-incident root cause spans repositories, retain one repository-owned canonical per repository and coordinate them as mechanism-cluster peers; never close one repository's ticket as a duplicate of another repository's canonical.
|
|
90
|
+
5. **Singleton or uncertain** — plan to release as-is with a brief `<!-- flaky-triage: singleton -->` comment.
|
|
70
91
|
|
|
71
|
-
|
|
92
|
+
After every cluster has an intended disposition, apply the chronic screen described below and act on each knowledge-base mechanism cluster. Finish both gates for the entire run before executing any planned release, cancellation, or duplicate action:
|
|
93
|
+
|
|
94
|
+
1. Add pairwise Linear `relatedTo` links between all members. Do not add `blockedBy` or `blocks` relations; repository fixes may proceed in parallel.
|
|
95
|
+
2. Post a scope-boundary comment on every member. List all peer tickets and repositories, state which repository this ticket owns, and strike any `sibling-repo` step that would create a mirror ticket in a repository owned by a peer. Name the most advanced member as the reference implementation when one has an in-flight PR or a plan with a flaky-critic approval marker. Rank an in-flight PR ahead of an approved plan; if members are equally advanced, tell each to reference the peer plans for mechanism consistency.
|
|
96
|
+
3. End each comment with the stable marker `<!-- flaky-triage: mechanism-cluster=<entry-slug> members=<sorted-ticket-ids> -->`. Fetch every member's relations and comments in parallel before writing, then skip each link or comment whose relation or exact marker already exists.
|
|
97
|
+
|
|
98
|
+
Use this comment shape, substituting the member-specific scope and reference:
|
|
99
|
+
|
|
100
|
+
```text
|
|
101
|
+
Mechanism coordination: STAFF-X and STAFF-Y match KB mechanism <entry>.
|
|
102
|
+
|
|
103
|
+
Scope: STAFF-X owns <repo-a>. Skip this ticket's sibling-repo step that would create a mirror ticket in <repo-b>; STAFF-Y owns that repository. Reference <ticket/plan/PR> for mechanism consistency. The fixes may proceed in parallel; do not add a blocked-by relation.
|
|
104
|
+
|
|
105
|
+
<!-- flaky-triage: mechanism-cluster=<entry-slug> members=STAFF-X,STAFF-Y -->
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
For every planned canonical release, search Linear for the fingerprint family with the `flaky-implementation` label across all states and without a date-window restriction, then count unique prior implementation ticket IDs. At **≥3 prior implementation tickets**, remove the planned normal release and leave the canonical in `Triage` unassigned. Create the Linear `chronic` label first if it does not exist. Apply the `chronic` label and route the family to the credential-checked, dossier-first `flaky-deep-dive` skill. Add a ticket comment with the marker `<!-- flaky-triage: chronic -->` and dossier pointers listing every prior implementation ticket ID — but first search the ticket's existing comments for that exact marker and skip posting when it is already present; that absence check is what makes the comment idempotent across retries and re-runs. This ≥3-ticket count is a cheap screen staged before any diagnosis exists; the critic separately applies a sharper **≥2 failed merged fixes** test (see the flaky-critic rubric's Prior-attempts amendment) with the same chronic routing. The thresholds differ by design — by the evidence available at each stage — not by accident.
|
|
72
109
|
|
|
73
110
|
**Release** = set state to `Todo` and assign to the Linear API user (the viewer). Before release, ensure the ticket satisfies the Groundcrew eligibility contract from `../create-groundcrew-ticket/SKILL.md`: repository text in the description, exactly one `agent-*` label, leaf issue, and no non-terminal blockers unless intentionally blocked. Assignment + Todo is what makes groundcrew dispatch it; it also serves as the lease — released tickets never re-enter the queue.
|
|
74
111
|
|
|
112
|
+
Only after the chronic and mechanism-coordination gates finish for the entire run, execute the remaining planned dispositions. Preserve the order within each disposition: post its evidence or scope comment and relation first, then change state or release it. Re-read immediately before each write and, after any conflict, resume only the actions still missing.
|
|
113
|
+
|
|
75
114
|
## Phase 5: Digest
|
|
76
115
|
|
|
77
|
-
End with a compact report: queue size, cluster count with members and evidence tiers, released / duplicated / canceled / chronic-held counts, in-flight matches found, chronic detections with each canonical ticket and its prior implementation ticket IDs, and risks (possible false merges, missing evidence). When run as a recurring task, follow any re-arm instructions in the task description after the digest.
|
|
116
|
+
End with a compact report: queue size, cluster count with members and evidence tiers, released / duplicated / canceled / chronic-held counts, in-flight matches found, chronic detections with each canonical ticket and its prior implementation ticket IDs, and risks (possible false merges, missing evidence, KB gaps). Add a dedicated line in this shape: `mechanism clusters: N — <entry>: STAFF-X (<repo-a>), STAFF-Y (<repo-b>); ...`, or `mechanism clusters: 0`. When run as a recurring task, follow any re-arm instructions in the task description after the digest.
|
|
78
117
|
|
|
79
118
|
## Dry-run and backtest
|
|
80
119
|
|
|
81
|
-
When invoked with `--dry-run`, when the task description says dry run, or when backtesting: perform Phases 1–4 fully but **write nothing to Linear
|
|
120
|
+
When invoked with `--dry-run`, when the task description says dry run, or when backtesting: perform Phases 1–4 fully but **write nothing to Linear**. Print each proposed close/release action, chronic-label creation or application, each `relatedTo` link, the exact scope-boundary comment for every mechanism-cluster member, and the mechanism-cluster digest line. Print likely mechanism clusters that lack a shared KB entry as `KB gap` findings, including their members and repositories, but do not count them as production mechanism clusters. For grading, also render the candidate `relatedTo` links and scope-boundary comments that the missing entry prevents, clearly prefixed `not production-eligible: KB gap`.
|
|
121
|
+
|
|
122
|
+
For a backtest, run blind against the historical tickets you are given. Use only content that existed before the coordination decision; do not read later ticket states, PR outcomes, relations, or human coordination comments until after stating the proposed clusters and actions. Join the result to the human action afterward and report whether member selection, repository ownership, mirror-step removal, and reference-implementation choice agree.
|