@clipboard-health/ai-rules 2.42.0 → 2.43.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -19,9 +19,9 @@ Resolve bundled "./scripts" paths relative to SKILL.md.
|
|
|
19
19
|
|
|
20
20
|
The skill uses two sentinels with visible footer lines.
|
|
21
21
|
|
|
22
|
-
**Addressed sentinel**: `<sub>🤖 <code>cb-babysit:addressed v1 core@3.
|
|
22
|
+
**Addressed sentinel**: `<sub>🤖 <code>cb-babysit:addressed v1 core@3.22.0</code></sub>`. Appended on its own line at the end of every reply the skill posts so re-runs know which threads and review-body comments are already handled. Dedupe also recognizes the legacy `babysit-pr:addressed v1` prefix from before this skill was renamed.
|
|
23
23
|
|
|
24
|
-
**Follow-up sentinel**: `<sub>🤖 <code>cb-babysit:followup v1 core@3.
|
|
24
|
+
**Follow-up sentinel**: `<sub>🤖 <code>cb-babysit:followup v1 core@3.22.0</code></sub>`. Attached to replies that defer an out-of-scope comment as a tracked follow-up. The sentinel is additive: the post-reply scripts still append the `addressed` sentinel at the end.
|
|
25
25
|
|
|
26
26
|
**Sentinel recency rules.** The script emits a per-thread `activityState` with three values. Step 6a owns the handling rules for each state.
|
|
27
27
|
|
|
@@ -247,7 +247,7 @@ Body templates (the script appends the `addressed` sentinel if missing):
|
|
|
247
247
|
- **Agree**: `Addressed in <commit-url>. <one-line what-changed>.`
|
|
248
248
|
- **Disagree**: `Leaving current behavior. <reasoning>.`
|
|
249
249
|
- **Already fixed**: `Already handled by <commit-url-or-file:line>. <brief pointer>.`
|
|
250
|
-
- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>🤖 <code>cb-babysit:followup v1 core@3.
|
|
250
|
+
- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>🤖 <code>cb-babysit:followup v1 core@3.22.0</code></sub>`
|
|
251
251
|
|
|
252
252
|
For Defer replies, include the follow-up sentinel on its own line as shown. The script will append the `addressed` sentinel after it on its own line, so the final body ends with the follow-up sentinel followed by a blank line followed by the `addressed` sentinel. `grep cb-babysit:followup` finds the deferral and `grep cb-babysit:addressed` still marks the thread handled for dedupe.
|
|
253
253
|
|
|
@@ -263,7 +263,7 @@ The PR-level summary should:
|
|
|
263
263
|
|
|
264
264
|
- Group by source. Use `## Review-body findings` for step-7 work and `## Conversation-tab comments` for step-6b work. Omit a section if its list is empty.
|
|
265
265
|
- Inside each section, group verdicts under **Agree / Disagree / Already fixed / Deferred (out of scope)** subheadings. Omit a subheading if its list is empty.
|
|
266
|
-
- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>🤖 <code>cb-babysit:followup v1 core@3.
|
|
266
|
+
- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>🤖 <code>cb-babysit:followup v1 core@3.22.0</code></sub>` so grep catches them individually.
|
|
267
267
|
- Include the commit URL for fixes.
|
|
268
268
|
- End with a fenced fingerprint block listing every current fingerprint (addressed and deferred) one per line. Include both `reviewBodyComments[].fingerprint` (whole-body, one per automated review) and `activeIssueComments[].fingerprint` (per Conversation-tab comment). Future runs dedupe by matching these against `priorBabysitSentinels`.
|
|
269
269
|
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
SENTINEL_PREFIX='cb-babysit:addressed v1 '
|
|
12
12
|
LEGACY_SENTINEL_PREFIX='babysit-pr:addressed v1 '
|
|
13
|
-
SENTINEL='<sub>🤖 <code>cb-babysit:addressed v1 core@3.
|
|
13
|
+
SENTINEL='<sub>🤖 <code>cb-babysit:addressed v1 core@3.22.0</code></sub>'
|
|
14
14
|
|
|
15
15
|
# Bot author allowlist (JSON array literal). Used by unresolvedPrComments.sh
|
|
16
16
|
# as a fallback when GraphQL's `author.__typename == "Bot"` misses a GitHub
|
package/skills/cb-ship/SKILL.md
CHANGED
|
@@ -28,22 +28,22 @@ Resolve bundled "./references" paths relative to SKILL.md.
|
|
|
28
28
|
7. Output:
|
|
29
29
|
|
|
30
30
|
```plaintext
|
|
31
|
-
- PR: [Complete url] (e.g., `https://github.com/clipboardhealth/core-utils/pull/123`)
|
|
32
|
-
|
|
33
31
|
[One concise sentence describing what was implemented and shipped.]
|
|
34
32
|
|
|
35
33
|
[Optional bullets for the most important user-facing outcomes. Omit when the sentence is sufficient.]
|
|
36
34
|
|
|
37
|
-
|
|
35
|
+
Details:
|
|
38
36
|
|
|
39
37
|
- Directory: [Absolute path]
|
|
40
38
|
- Branch: [Branch name]
|
|
41
39
|
- Review: [N findings — M applied, K dismissed with a one-line reason each] (omit if step 4 skipped or found nothing)
|
|
42
40
|
- Session: [Resume command from 6a] (omit if none)
|
|
43
41
|
- Existing changes: [Uncommitted changes that remain outside the shipped scope] (omit if none)
|
|
42
|
+
|
|
43
|
+
🚀 [Complete url] (e.g., `https://github.com/clipboardhealth/core-utils/pull/123`) 🚀
|
|
44
44
|
```
|
|
45
45
|
|
|
46
|
-
Keep the PR as the
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
`
|
|
46
|
+
Keep the PR as the final line and visually separate it from everything else
|
|
47
|
+
with a blank line. Use the middle section only for meaningful behavior or
|
|
48
|
+
outcome details. Put lower-priority handoff metadata and caveats under
|
|
49
|
+
`Details`.
|
|
@@ -17,5 +17,5 @@ Concisely explain the user intent from session history and the meaningful behavi
|
|
|
17
17
|
|
|
18
18
|
Optional, don't fabricate: ticket links, rollout plan, residual risk, or specific areas for reviewers to focus.
|
|
19
19
|
|
|
20
|
-
<sub>🤖 <code>cb-ship:created v1 core@3.
|
|
20
|
+
<sub>🤖 <code>cb-ship:created v1 core@3.22.0</code></sub>
|
|
21
21
|
```
|
|
@@ -1,31 +1,35 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: flaky-triage
|
|
3
|
-
description: Scheduled triage stage between flake-intake and agent dispatch. Clusters flaky-investigation tickets by shared root cause, closes duplicates and infra blips, and releases one canonical ticket per cluster for dispatch. Use when running the recurring flaky-triage task, when asked to triage the flaky queue, or to backtest clustering against historical tickets.
|
|
3
|
+
description: Scheduled triage stage between flake-intake and agent dispatch. Clusters flaky-investigation tickets by shared root cause, closes duplicates and infra blips, and releases one repository-owned canonical ticket per cluster for dispatch. Use when running the recurring flaky-triage task, when asked to triage the flaky queue, or to backtest clustering against historical tickets.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
<!-- cspell:words backtest backtesting deflake dispatchable groundcrew -->
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
The pipeline stage between flake-intake (deterministic, string-level collapse: bursts and storms) and per-ticket dispatch. Intake creates investigation tickets in the **Triage** state, where groundcrew ignores them. This skill clusters them semantically — same root cause, even when error text differs — then performs ticket surgery and releases exactly one dispatchable ticket per root cause and implementation repository.
|
|
9
|
+
|
|
10
|
+
This skill routes and classifies known mechanisms from existing ticket evidence; it never performs a new source-level diagnosis. Do not propose fixes, read test source files, or run flaky-debug. Diagnosis happens later, on the released ticket, through the normal dispatch flow.
|
|
9
11
|
|
|
10
12
|
## Rules
|
|
11
13
|
|
|
12
|
-
- One canonical ticket per root cause, never one per sighting (rubric B5).
|
|
14
|
+
- One repository-owned canonical ticket per root cause, never one per sighting within an implementation repository (rubric B5).
|
|
15
|
+
- One implementation ticket per mechanism cluster per repository. A cross-repo mechanism match coordinates repository-owned fixes; it never consolidates them across repositories.
|
|
13
16
|
- Never hold a ticket hostage: anything you cannot confidently cluster is released as a singleton. The worst case must equal the no-triage baseline.
|
|
14
17
|
- Load `../flaky-critic/references/rubric.md` before citing close-out rules. Cite rubric rule IDs (B5, D1, D2, D3) in every close-out comment.
|
|
15
18
|
- Decide the full cluster picture before writing anything to Linear.
|
|
16
19
|
- Keep bulky data in local scratch files; keep only the manifest and decisions in context.
|
|
17
|
-
- All Linear writes are idempotent
|
|
20
|
+
- All Linear writes are idempotent. Use exact markers only to deduplicate comments. Immediately before state, label, relation, or marker-comment writes, re-read the current value and skip only when the desired state, label, relation, or exact marker already exists.
|
|
21
|
+
- Production Linear writes are single-flight. The recurring task or caller must serialize the entire Phase 4 write pass; if exclusive execution cannot be established, remain in dry-run and report the blocker. After an API conflict, re-fetch the affected tickets and retry only actions that are still missing.
|
|
18
22
|
|
|
19
23
|
**Rubric IDs** (cite the matching ID in every close-out comment):
|
|
20
24
|
|
|
21
|
-
- **B5** — one canonical ticket per root cause, never one per sighting.
|
|
25
|
+
- **B5** — one repository-owned canonical ticket per root cause, never one per sighting within an implementation repository.
|
|
22
26
|
- **D1** — infra/environment incident: cancel with incident window + correlation evidence + why no code fix applies + re-arm note.
|
|
23
27
|
- **D2** — duplicate of canonical: matching evidence + no new implementation ticket needed beyond the canonical.
|
|
24
28
|
- **D3** — already-fixed: matches an investigation/implementation ticket closed in the last 14 days.
|
|
25
29
|
|
|
26
30
|
## Phase 1: Queue
|
|
27
31
|
|
|
28
|
-
Fetch candidate tickets from Linear: project `Groundcrew`, label `flaky-investigation`, state `Triage`. (During migration, also include state `Todo` with no assignee.)
|
|
32
|
+
Fetch candidate tickets from Linear: project `Groundcrew`, label `flaky-investigation`, state `Triage`. (During migration, also include state `Todo` with no assignee.) A marker is not enough to skip a ticket: verify that its promised disposition is reflected in the issue state and relations. If a marked ticket remains in the queue, treat it as a partial action, reconstruct the intended disposition, skip completed writes, and resume the rest. If the queue is empty, skip to the digest.
|
|
29
33
|
|
|
30
34
|
Use `list_issues` only to enumerate the queue; fetch each candidate with `get_issue` — list results truncate descriptions and will silently drop the Flake details JSON. Fetch candidate details in parallel when the available tooling supports it.
|
|
31
35
|
|
|
@@ -37,7 +41,7 @@ Phase 1 is done when every non-skipped candidate has a manifest row — don't ca
|
|
|
37
41
|
|
|
38
42
|
Fetch what is already being worked, to catch duplicates before they spawn plans. Scope searches to the repos, files, normalized errors, helper names, and prior tickets from the Phase 1 manifest first; widen only when the scoped search leaves a plausible duplicate unresolved.
|
|
39
43
|
|
|
40
|
-
- Open Linear tickets labeled `flaky-implementation` (any state but terminal).
|
|
44
|
+
- Open Linear tickets labeled `flaky-implementation` (any state but terminal). Collect plausible IDs first, then fetch each with `get_issue` and read its comments in parallel when the available tooling supports it; list results truncate the plan, `KB match:` statement, critic approval, and PR context needed for mechanism coordination.
|
|
41
45
|
- PRs: search `flaky-test-fix` in both `open` and `closed` states; treat closed-unmerged siblings as duplicate signals, not already-fixed proof. Also try title keywords `flaky`/`deflake`.
|
|
42
46
|
- Open `[storm:` / `[burst:` tickets and their contained fingerprints.
|
|
43
47
|
- Investigation/implementation tickets closed in the last 14 days, for already-fixed matches only after confirming the merged fix is on main and the sighting predates it (D3).
|
|
@@ -60,22 +64,59 @@ Cluster by strongest shared evidence first:
|
|
|
60
64
|
|
|
61
65
|
Do not merge clusters merely because failures share a run. Same-run failures can have different root causes.
|
|
62
66
|
|
|
63
|
-
|
|
67
|
+
### Knowledge-base mechanism coordination
|
|
68
|
+
|
|
69
|
+
Finish the root-cause clusters above before this second pass. Same-repo duplicate decisions take precedence; knowledge-base coordination must not rescue a duplicate or split one root cause into several tickets. Before the lookup, use the Phase 4 rules and chronic threshold to plan every disposition without writing to Linear.
|
|
70
|
+
|
|
71
|
+
For every canonical that the plan would release, plus every plausibly related in-flight implementation ticket from Phase 2:
|
|
72
|
+
|
|
73
|
+
1. Open `../flaky-debug/references/root-cause-kb/README.md` once. Compare every member's failure signature, failure surface, and available causal evidence with the symptom index, collect the union of plausible entries, then read each unique entry once in full, including **What failed and why**.
|
|
74
|
+
2. For each matched entry, widen the Phase 2 in-flight search across repositories using the entry link, entry title, and mechanism terms. Fetch every non-terminal `flaky-implementation` match before deciding the mechanism cluster.
|
|
75
|
+
3. Record one of the exact `KB match:` statements defined in `../flaky-debug/SKILL.md` Phase 1b, including the failed fixes to avoid when an entry matches. Perform the lookup at triage time; do not trust a copied or stale plan line.
|
|
76
|
+
4. Group tickets only when the evidence matches the same KB entry and mechanism. A shared symptom without the entry's mechanism is insufficient. Record the entry link, mechanism, ticket IDs, repositories, and whether each member is a provisional canonical or in-flight implementation.
|
|
77
|
+
|
|
78
|
+
A mechanism cluster exists when at least two surviving provisional canonicals in the run, or one surviving provisional canonical plus an in-flight implementation ticket, match the same entry. The members may represent different fingerprint families. Do not merge members from different repositories, close one as a duplicate of the other, or create a cross-repo canonical. Each repository keeps its own implementation ticket.
|
|
79
|
+
|
|
80
|
+
When several members are in the same repository, apply the existing root-cause and D2 rules first so at most one canonical per mechanism and repository remains. A KB match alone is not D2 evidence; if the root-cause evidence does not support deduplication, keep the tickets independent and record the ambiguity as a risk instead of forcing them into one mechanism cluster. `KB match: none` cannot establish a production mechanism cluster. Record a likely shared mechanism with no matching entry as a KB gap in risks and in dry-run/backtest output; it does not block either ticket's normal release.
|
|
81
|
+
|
|
82
|
+
## Phase 4: Plan, coordinate, then act
|
|
83
|
+
|
|
84
|
+
First, choose exactly one intended disposition per cluster using the ordered rules below. Record the actions without writing to Linear.
|
|
64
85
|
|
|
65
|
-
1. **Matches in-flight work** — error family or
|
|
66
|
-
2. **Matches an open storm/burst ticket** — normalized error matches its signature or contained fingerprints: close members as Duplicate of the storm/burst ticket so late stragglers join the existing event.
|
|
67
|
-
3. **Infra blip / environment incident** — an incident cluster (evidence tier 7): use the Phase 3 threshold above. In production runs, corroborate via Phase 2 (Datadog monitor history / error spike through `pup` when available; passed-on-retry signals). Default
|
|
68
|
-
4. **
|
|
69
|
-
5. **Singleton or uncertain** — release as-is with a brief `<!-- flaky-triage: singleton -->` comment.
|
|
86
|
+
1. **Matches in-flight work** — within the same repository, the error family or root cause matches an open implementation ticket or open PR: plan to close every member as Duplicate linked to that canonical and comment citing B5/D2 with the match evidence. Plan no release. A shared KB mechanism alone is not D2 evidence, especially across repositories; handle it through mechanism coordination below.
|
|
87
|
+
2. **Matches an open storm/burst ticket** — normalized error matches its signature or contained fingerprints: plan to close members as Duplicate of the storm/burst ticket so late stragglers join the existing event.
|
|
88
|
+
3. **Infra blip / environment incident** — an incident cluster (evidence tier 7): use the Phase 3 threshold above. In production runs, corroborate via Phase 2 (Datadog monitor history / error spike through `pup` when available; passed-on-retry signals). Default disposition: plan to close all members as Canceled with a D1 comment. If the incident exposes a hardening gap (e.g., seed path lacks diagnostics), plan to release ONE canonical for that hardening and dup the rest to it — never one per helper.
|
|
89
|
+
4. **Same-repository multi-ticket cluster** — within each implementation repository, pick the canonical (richest evidence: most failures, clearest artifacts). Plan to comment the cluster manifest on it: member IDs, shared mechanism, evidence tier from Phase 3, and the marker `<!-- flaky-triage: cluster=<short-slug> -->`. Plan to close only the same-repository members as Duplicate of the canonical (D2 format) and release the canonical. When a non-incident root cause spans repositories, retain one repository-owned canonical per repository and coordinate them as mechanism-cluster peers; never close one repository's ticket as a duplicate of another repository's canonical.
|
|
90
|
+
5. **Singleton or uncertain** — plan to release as-is with a brief `<!-- flaky-triage: singleton -->` comment.
|
|
70
91
|
|
|
71
|
-
|
|
92
|
+
After every cluster has an intended disposition, apply the chronic screen described below and act on each knowledge-base mechanism cluster. Finish both gates for the entire run before executing any planned release, cancellation, or duplicate action:
|
|
93
|
+
|
|
94
|
+
1. Add pairwise Linear `relatedTo` links between all members. Do not add `blockedBy` or `blocks` relations; repository fixes may proceed in parallel.
|
|
95
|
+
2. Post a scope-boundary comment on every member. List all peer tickets and repositories, state which repository this ticket owns, and strike any `sibling-repo` step that would create a mirror ticket in a repository owned by a peer. Name the most advanced member as the reference implementation when one has an in-flight PR or a plan with a flaky-critic approval marker. Rank an in-flight PR ahead of an approved plan; if members are equally advanced, tell each to reference the peer plans for mechanism consistency.
|
|
96
|
+
3. End each comment with the stable marker `<!-- flaky-triage: mechanism-cluster=<entry-slug> members=<sorted-ticket-ids> -->`. Fetch every member's relations and comments in parallel before writing, then skip each link or comment whose relation or exact marker already exists.
|
|
97
|
+
|
|
98
|
+
Use this comment shape, substituting the member-specific scope and reference:
|
|
99
|
+
|
|
100
|
+
```text
|
|
101
|
+
Mechanism coordination: STAFF-X and STAFF-Y match KB mechanism <entry>.
|
|
102
|
+
|
|
103
|
+
Scope: STAFF-X owns <repo-a>. Skip this ticket's sibling-repo step that would create a mirror ticket in <repo-b>; STAFF-Y owns that repository. Reference <ticket/plan/PR> for mechanism consistency. The fixes may proceed in parallel; do not add a blocked-by relation.
|
|
104
|
+
|
|
105
|
+
<!-- flaky-triage: mechanism-cluster=<entry-slug> members=STAFF-X,STAFF-Y -->
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
For every planned canonical release, search Linear for the fingerprint family with the `flaky-implementation` label across all states and without a date-window restriction, then count unique prior implementation ticket IDs. At **≥3 prior implementation tickets**, remove the planned normal release and leave the canonical in `Triage` unassigned. Create the Linear `chronic` label first if it does not exist. Apply the `chronic` label and route the family to the credential-checked, dossier-first `flaky-deep-dive` skill. Add a ticket comment with the marker `<!-- flaky-triage: chronic -->` and dossier pointers listing every prior implementation ticket ID — but first search the ticket's existing comments for that exact marker and skip posting when it is already present; that absence check is what makes the comment idempotent across retries and re-runs. This ≥3-ticket count is a cheap screen staged before any diagnosis exists; the critic separately applies a sharper **≥2 failed merged fixes** test (see the flaky-critic rubric's Prior-attempts amendment) with the same chronic routing. The thresholds differ by design — by the evidence available at each stage — not by accident.
|
|
72
109
|
|
|
73
110
|
**Release** = set state to `Todo` and assign to the Linear API user (the viewer). Before release, ensure the ticket satisfies the Groundcrew eligibility contract from `../create-groundcrew-ticket/SKILL.md`: repository text in the description, exactly one `agent-*` label, leaf issue, and no non-terminal blockers unless intentionally blocked. Assignment + Todo is what makes groundcrew dispatch it; it also serves as the lease — released tickets never re-enter the queue.
|
|
74
111
|
|
|
112
|
+
Only after the chronic and mechanism-coordination gates finish for the entire run, execute the remaining planned dispositions. Preserve the order within each disposition: post its evidence or scope comment and relation first, then change state or release it. Re-read immediately before each write and, after any conflict, resume only the actions still missing.
|
|
113
|
+
|
|
75
114
|
## Phase 5: Digest
|
|
76
115
|
|
|
77
|
-
End with a compact report: queue size, cluster count with members and evidence tiers, released / duplicated / canceled / chronic-held counts, in-flight matches found, chronic detections with each canonical ticket and its prior implementation ticket IDs, and risks (possible false merges, missing evidence). When run as a recurring task, follow any re-arm instructions in the task description after the digest.
|
|
116
|
+
End with a compact report: queue size, cluster count with members and evidence tiers, released / duplicated / canceled / chronic-held counts, in-flight matches found, chronic detections with each canonical ticket and its prior implementation ticket IDs, and risks (possible false merges, missing evidence, KB gaps). Add a dedicated line in this shape: `mechanism clusters: N — <entry>: STAFF-X (<repo-a>), STAFF-Y (<repo-b>); ...`, or `mechanism clusters: 0`. When run as a recurring task, follow any re-arm instructions in the task description after the digest.
|
|
78
117
|
|
|
79
118
|
## Dry-run and backtest
|
|
80
119
|
|
|
81
|
-
When invoked with `--dry-run`, when the task description says dry run, or when backtesting: perform Phases 1–4 fully but **write nothing to Linear
|
|
120
|
+
When invoked with `--dry-run`, when the task description says dry run, or when backtesting: perform Phases 1–4 fully but **write nothing to Linear**. Print each proposed close/release action, chronic-label creation or application, each `relatedTo` link, the exact scope-boundary comment for every mechanism-cluster member, and the mechanism-cluster digest line. Print likely mechanism clusters that lack a shared KB entry as `KB gap` findings, including their members and repositories, but do not count them as production mechanism clusters. For grading, also render the candidate `relatedTo` links and scope-boundary comments that the missing entry prevents, clearly prefixed `not production-eligible: KB gap`.
|
|
121
|
+
|
|
122
|
+
For a backtest, run blind against the historical tickets you are given. Use only content that existed before the coordination decision; do not read later ticket states, PR outcomes, relations, or human coordination comments until after stating the proposed clusters and actions. Join the result to the human action afterward and report whether member selection, repository ownership, mirror-step removal, and reference-implementation choice agree.
|