@clipboard-health/ai-rules 2.39.0 โ 2.40.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/rules/common/coreLibraries.md +1 -1
- package/skills/cb-babysit/SKILL.md +4 -4
- package/skills/cb-babysit/scripts/_sentinel.sh +1 -1
- package/skills/cb-ship/references/pr-template.md +1 -1
- package/skills/flaky-critic/references/rubric.md +7 -1
- package/skills/flaky-debug/SKILL.md +1 -1
- package/skills/flaky-debug/references/fix.md +37 -1
- package/skills/flaky-debug/references/plan.md +1 -0
package/package.json
CHANGED
|
@@ -30,12 +30,12 @@ When a bug traces into a `@clipboard-health/*` library, read the source code in
|
|
|
30
30
|
- **execution-context**: A lightweight Node.js utility for managing execution contexts and metadata aggregation using AsyncLocalStorage.
|
|
31
31
|
- **json-api**: TypeScript-friendly utilities for adhering to the JSON:API specification.
|
|
32
32
|
- **json-api-nestjs**: TypeScript-friendly utilities for adhering to the JSON:API specification with NestJS.
|
|
33
|
-
- **mongo-jobs**: MongoDB-powered background jobs.
|
|
34
33
|
- **notifications**: Send notifications through third-party providers.
|
|
35
34
|
- **nx-plugin**: An Nx plugin with generators to manage libraries and applications.
|
|
36
35
|
- **oxlint-config**: Shared Oxlint configuration for Clipboard Health repositories.
|
|
37
36
|
- **phone-number**: Phone number utility functions.
|
|
38
37
|
- **playwright-reporter-llm**: Playwright reporter that outputs structured JSON for LLM agents. Minimal console output, flat schema, easy to filter to failures.
|
|
38
|
+
- **playwright-toolkit**: Shared anti-flake primitives for Clipboard Health Playwright suites.
|
|
39
39
|
- **rules-engine**: A pure functional rules engine to keep logic-dense code simple, reliable, understandable, and explainable.
|
|
40
40
|
- **testing-core**: TypeScript-friendly testing utilities.
|
|
41
41
|
- **util-ts**: TypeScript utilities.
|
|
@@ -19,9 +19,9 @@ Resolve bundled "./scripts" paths relative to SKILL.md.
|
|
|
19
19
|
|
|
20
20
|
The skill uses two sentinels with visible footer lines.
|
|
21
21
|
|
|
22
|
-
**Addressed sentinel**: `<sub>๐ค <code>cb-babysit:addressed v1 core@3.
|
|
22
|
+
**Addressed sentinel**: `<sub>๐ค <code>cb-babysit:addressed v1 core@3.19.0</code></sub>`. Appended on its own line at the end of every reply the skill posts so re-runs know which threads and review-body comments are already handled. Dedupe also recognizes the legacy `babysit-pr:addressed v1` prefix from before this skill was renamed.
|
|
23
23
|
|
|
24
|
-
**Follow-up sentinel**: `<sub>๐ค <code>cb-babysit:followup v1 core@3.
|
|
24
|
+
**Follow-up sentinel**: `<sub>๐ค <code>cb-babysit:followup v1 core@3.19.0</code></sub>`. Attached to replies that defer an out-of-scope comment as a tracked follow-up. The sentinel is additive: the post-reply scripts still append the `addressed` sentinel at the end.
|
|
25
25
|
|
|
26
26
|
**Sentinel recency rules.** The script emits a per-thread `activityState` with three values. Step 6a owns the handling rules for each state.
|
|
27
27
|
|
|
@@ -247,7 +247,7 @@ Body templates (the script appends the `addressed` sentinel if missing):
|
|
|
247
247
|
- **Agree**: `Addressed in <commit-url>. <one-line what-changed>.`
|
|
248
248
|
- **Disagree**: `Leaving current behavior. <reasoning>.`
|
|
249
249
|
- **Already fixed**: `Already handled by <commit-url-or-file:line>. <brief pointer>.`
|
|
250
|
-
- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>๐ค <code>cb-babysit:followup v1 core@3.
|
|
250
|
+
- **Defer**: `Out of scope for this PR; this looks like follow-up work rather than something introduced or required by this change. <one-line rationale or pointer if useful>.\n\n<sub>๐ค <code>cb-babysit:followup v1 core@3.19.0</code></sub>`
|
|
251
251
|
|
|
252
252
|
For Defer replies, include the follow-up sentinel on its own line as shown. The script will append the `addressed` sentinel after it on its own line, so the final body ends with the follow-up sentinel followed by a blank line followed by the `addressed` sentinel. `grep cb-babysit:followup` finds the deferral and `grep cb-babysit:addressed` still marks the thread handled for dedupe.
|
|
253
253
|
|
|
@@ -263,7 +263,7 @@ The PR-level summary should:
|
|
|
263
263
|
|
|
264
264
|
- Group by source. Use `## Review-body findings` for step-7 work and `## Conversation-tab comments` for step-6b work. Omit a section if its list is empty.
|
|
265
265
|
- Inside each section, group verdicts under **Agree / Disagree / Already fixed / Deferred (out of scope)** subheadings. Omit a subheading if its list is empty.
|
|
266
|
-
- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>๐ค <code>cb-babysit:followup v1 core@3.
|
|
266
|
+
- Under **Deferred (out of scope)**, list each deferred item as a bullet, followed on its own line by `<sub>๐ค <code>cb-babysit:followup v1 core@3.19.0</code></sub>` so grep catches them individually.
|
|
267
267
|
- Include the commit URL for fixes.
|
|
268
268
|
- End with a fenced fingerprint block listing every current fingerprint (addressed and deferred) one per line. Include both `reviewBodyComments[].fingerprint` (whole-body, one per automated review) and `activeIssueComments[].fingerprint` (per Conversation-tab comment). Future runs dedupe by matching these against `priorBabysitSentinels`.
|
|
269
269
|
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
|
|
11
11
|
SENTINEL_PREFIX='cb-babysit:addressed v1 '
|
|
12
12
|
LEGACY_SENTINEL_PREFIX='babysit-pr:addressed v1 '
|
|
13
|
-
SENTINEL='<sub>๐ค <code>cb-babysit:addressed v1 core@3.
|
|
13
|
+
SENTINEL='<sub>๐ค <code>cb-babysit:addressed v1 core@3.19.0</code></sub>'
|
|
14
14
|
|
|
15
15
|
# Bot author allowlist (JSON array literal). Used by unresolvedPrComments.sh
|
|
16
16
|
# as a fallback when GraphQL's `author.__typename == "Bot"` misses a GitHub
|
|
@@ -17,5 +17,5 @@ Concisely explain the user intent from session history and the meaningful behavi
|
|
|
17
17
|
|
|
18
18
|
Optional, don't fabricate: ticket links, rollout plan, residual risk, or specific areas for reviewers to focus.
|
|
19
19
|
|
|
20
|
-
<sub>๐ค <code>cb-ship:created v1 core@3.
|
|
20
|
+
<sub>๐ค <code>cb-ship:created v1 core@3.19.0</code></sub>
|
|
21
21
|
```
|
|
@@ -8,7 +8,7 @@ This rubric turns Rocky's observed review standards into checkable rules for fla
|
|
|
8
8
|
|
|
9
9
|
Findings from automated reviewers (Mendral, CodeRabbit, etc.) are **advisory inputs, not binding rules**: the critic evaluates each on rubric merits and may decline one with stated reasoning, exactly as cb-babysit's Disagree replies do.
|
|
10
10
|
|
|
11
|
-
**Statement-missing vs substance-missing (backtest amendment, 2026-06-11):** B5/B6/B7/B8 each require an explicit statement in the plan (dedup-check results, Prior attempts section when applicable, confidence score plus observability section, current-main status, causal-chain section). When the _statement_ is absent but the violation is not demonstrated โ no actual duplicate found, no actual prior implementation tickets for the fingerprint family, no actual low-confidence code fix, no actual staleness, no demonstrably symptom-terminated diagnosis โ the verdict is reject with disposition **amend-and-resubmit**: one bounce asking for exactly the missing sections, never escalation. A demonstrated violation (a real duplicate, a mechanism already tried by a prior merged fix, a real โค2/5 code fix, a diagnosis that demonstrably stops at a symptom) is a substantive reject. The 2026-06-11 backtest showed 6 of 7 false rejections were statement-missing on plans predating these conventions; live plans carry the sections by template, but the distinction stays load-bearing for the agreement metric.
|
|
11
|
+
**Statement-missing vs substance-missing (backtest amendment, 2026-06-11):** B5/B6/B7/B8 and C6 each require an explicit statement in the plan (dedup-check results, Prior attempts section when applicable, confidence score plus observability section, current-main status, causal-chain section, sibling-repo check when applicable). When the _statement_ is absent but the violation is not demonstrated โ no actual duplicate found, no actual prior implementation tickets for the fingerprint family, no actual low-confidence code fix, no actual staleness, no demonstrably symptom-terminated diagnosis, no confirmed sibling-repo match โ the verdict is reject with disposition **amend-and-resubmit**: one bounce asking for exactly the missing sections, never escalation. A demonstrated violation (a real duplicate, a mechanism already tried by a prior merged fix, a real โค2/5 code fix, a diagnosis that demonstrably stops at a symptom, a confirmed sibling-repo match with no linked mirror implementation ticket deliverable) is a substantive reject. The 2026-06-11 backtest showed 6 of 7 false rejections were statement-missing on plans predating these conventions; live plans carry the sections by template, but the distinction stays load-bearing for the agreement metric.
|
|
12
12
|
|
|
13
13
|
## 2. Banned patterns
|
|
14
14
|
|
|
@@ -167,6 +167,12 @@ Allowed when many tests fail before user-flow code and the evidence is a tool/ve
|
|
|
167
167
|
|
|
168
168
|
When the root cause is in another team's service, the plan may ship the local resilience fix and must route a recommendation to the owning team (STAFF-1413: frontend rethrow fix + "backend follow-up recommendation (โฆ 503 instead of 401) is noted in the ticket for the owning team"). It may not silently absorb a backend bug into test code.
|
|
169
169
|
|
|
170
|
+
### C6 โ Sibling-repo propagation
|
|
171
|
+
|
|
172
|
+
**Sibling-repo propagation (amendment, 2026-07-16):** For a mechanism plausibly shared between `cbh-admin-frontend` and `cbh-mobile-app`, state the sibling repository, current `main` commit searched, exact helper names or grep-able code patterns, commands and scope, and matches found (or `None found`). A mechanism is plausibly shared when the proposed fix touches the test harness, authentication/session/bootstrap, cache, API client, or a copied capability, helper, or convention. If the same mechanism is present, state that the fix deliverable will include a linked mirror implementation ticket following the Groundcrew ticket conventions, linked to the source implementation ticket and referencing the landed source fix as the pattern to adopt. If the mechanism is not plausibly shared, state `N/A` and why.
|
|
173
|
+
|
|
174
|
+
A plan that fixes a mechanism plausibly shared between the frontend repositories, including a shared-harness mechanism, without a **Sibling-repo check** statement is statement-missing and gets amend-and-resubmit per ยง1. A confirmed match with no linked mirror implementation ticket deliverable is a substantive reject.
|
|
175
|
+
|
|
170
176
|
## 5. Dispositions that need no code
|
|
171
177
|
|
|
172
178
|
### D1 โ Infra blip / environment incident (close as Canceled)
|
|
@@ -100,4 +100,4 @@ If you are in plan mode, present the plan and stop here.
|
|
|
100
100
|
|
|
101
101
|
## Phase 3: Apply the plan (fix mode only)
|
|
102
102
|
|
|
103
|
-
Follow [`references/fix.md`](./references/fix.md). It takes the plan from Phase 2, applies the proposed fix, searches for sibling anti-patterns, and verifies. PR creation is out of scope -- if the user later opens one (or invokes a PR-shipping skill), label it `flaky-test-fix`.
|
|
103
|
+
Follow [`references/fix.md`](./references/fix.md). It takes the plan from Phase 2, validates the recorded sibling frontend check and reruns it when stale or missing, applies the proposed fix, searches for same-repo sibling anti-patterns, and verifies. PR creation is out of scope -- if the user later opens one (or invokes a PR-shipping skill), label it `flaky-test-fix`.
|
|
@@ -12,6 +12,41 @@ Before editing, verify the plan is still current:
|
|
|
12
12
|
- The proposed fix targets the diagnosed failure surface, not only the final assertion.
|
|
13
13
|
- Any retry/wait change is safe and idempotent; it must not repeat one-time credentials, duplicate writes, destructive actions, or rate-limited setup calls.
|
|
14
14
|
|
|
15
|
+
## Validate the Sibling Frontend Check
|
|
16
|
+
|
|
17
|
+
During Phase 2, after the proposed fix is designed and before the plan is finalized, use this recipe to complete the plan's **Sibling-repo check** when the current repository is `cbh-admin-frontend` or `cbh-mobile-app`. During the fix preflight, consume that result by confirming that the recorded sibling `main` commit is still current; rerun the search only when it changed or the plan omitted the field.
|
|
18
|
+
|
|
19
|
+
For a mechanism plausibly shared between `cbh-admin-frontend` and `cbh-mobile-app`, state the sibling repository, current `main` commit searched, exact helper names or grep-able code patterns, commands and scope, and matches found (or `None found`). A mechanism is plausibly shared when the proposed fix touches the test harness, authentication/session/bootstrap, cache, API client, or a copied capability, helper, or convention. If the same mechanism is present, state that the fix deliverable will include a linked mirror implementation ticket following the Groundcrew ticket conventions, linked to the source implementation ticket and referencing the landed source fix as the pattern to adopt. If the mechanism is not plausibly shared, state `N/A` and why.
|
|
20
|
+
|
|
21
|
+
Use this search recipe:
|
|
22
|
+
|
|
23
|
+
1. Map `cbh-admin-frontend` to `cbh-mobile-app`, and `cbh-mobile-app` to `cbh-admin-frontend`.
|
|
24
|
+
2. Search a temporary shallow clone pinned to the sibling repository's `main`:
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
sibling_repo="<cbh-admin-frontend-or-cbh-mobile-app>"
|
|
28
|
+
sibling_root="$(mktemp -d)/$sibling_repo"
|
|
29
|
+
gh repo clone "ClipboardHealth/$sibling_repo" "$sibling_root" -- \
|
|
30
|
+
--depth=1 --branch main --single-branch
|
|
31
|
+
git -C "$sibling_root" rev-parse HEAD
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
3. Search for both the exact helper or capability names and the grep-able structural pattern behind the fix. Limit the first pass to likely harness and test paths, then widen to the repository if needed:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
rg -n --glob '*.{ts,tsx}' \
|
|
38
|
+
'<exact-helper-name>|<grep-able-structural-pattern>' \
|
|
39
|
+
"$sibling_root/<likely-path>"
|
|
40
|
+
# Run only when the scoped search cannot rule the mechanism in or out.
|
|
41
|
+
rg -n --glob '*.{ts,tsx}' \
|
|
42
|
+
'<exact-helper-name>|<grep-able-structural-pattern>' \
|
|
43
|
+
"$sibling_root"
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
4. Inspect each match to decide whether it carries the same mechanism, rather than relying on the text match alone. Record each query run, its scope, and the matching files or `None found` in the plan before applying the fix.
|
|
47
|
+
|
|
48
|
+
Do not edit the sibling repository as part of the current fix. When the same mechanism is present, the deliverable includes a linked mirror implementation ticket created after the source fix lands. Follow the `create-groundcrew-ticket` conventions: include the exact `Repository: <sibling-repo>` line in the description, assign the current Linear user, use Todo with exactly one appropriate `agent-*` label, link the source implementation ticket, and cite the landed PR or commit as the pattern to adopt. If the current workflow stops before merge, carry this as an explicit post-merge deliverable and do not report sibling-repo propagation as complete until the ticket exists.
|
|
49
|
+
|
|
15
50
|
## Apply the Proposed Fix
|
|
16
51
|
|
|
17
52
|
Edit the files listed in the plan's **Proposed fix** field. Keep the change minimal -- the plan already chose between test harness, product, and both.
|
|
@@ -59,7 +94,7 @@ Run the plan's **Validation plan** commands โ including the previously-flaky t
|
|
|
59
94
|
|
|
60
95
|
When opening a PR for a flaky test fix, include `--label flaky-test-fix` in the `gh pr create` command so other agents can find it during Phase 1b deduplication.
|
|
61
96
|
|
|
62
|
-
When documenting the fix in a PR or issue, use this structure. Carry **Confidence**, **Symptom**, **Root cause**, **Evidence**, and **Residual risk** straight over from the plan. Three plan fields rename: **Proposed fix** โ **Fix**, **Sibling candidates** โ **Siblings fixed**, **Validation plan** โ **Validation**. Drop **Open questions** (resolved by fix time):
|
|
97
|
+
When documenting the fix in a PR or issue, use this structure. Carry **Confidence**, **Symptom**, **Root cause**, **Evidence**, **Sibling-repo check**, and **Residual risk** straight over from the plan. Three plan fields rename: **Proposed fix** โ **Fix**, **Sibling candidates** โ **Siblings fixed**, **Validation plan** โ **Validation**. Add the linked mirror ticket URL to **Sibling-repo check** once the source fix lands and the ticket exists. Before then, record `Pending: create linked mirror implementation ticket after <source-PR> lands`; replace that status with the ticket URL when created. Drop **Open questions** (resolved by fix time):
|
|
63
98
|
|
|
64
99
|
- **Test ID:** if provided in prompt
|
|
65
100
|
- **Agent session ID:** your running session ID to resume if needed
|
|
@@ -71,5 +106,6 @@ When documenting the fix in a PR or issue, use this structure. Carry **Confidenc
|
|
|
71
106
|
- **Evidence:** artifacts supporting the diagnosis (traces, network, error messages, screenshots as applicable)
|
|
72
107
|
- **Fix:** test-only, product-only, or both
|
|
73
108
|
- **Siblings fixed:** list of other files where the same anti-pattern was or should be corrected (or "N/A -- fix was test-specific")
|
|
109
|
+
- **Sibling-repo check:** sibling repository and current `main` commit searched; exact helper names or grep-able code patterns; commands, scope, and matches; linked mirror implementation ticket URL when created, or the explicit pending post-merge deliverable before then; or `N/A` with the reason the mechanism was not plausibly shared
|
|
74
110
|
- **Validation:** commands and suites run
|
|
75
111
|
- **Residual risk:** what could still be flaky
|
|
@@ -102,6 +102,7 @@ Produce the plan with these fields:
|
|
|
102
102
|
- **Proposed fix:** test harness, product, or both โ with the specific file(s) and the change you would make
|
|
103
103
|
- **Observability to reach 5/5:** required when confidence is less than 5/5. List the frontend and/or backend telemetry, logging, tracing, reporter, or metric changes that would make this flake's root cause directly visible in artifacts next time (reproduction then completes 5/5). Include another repository only when the evidence implicates it. Use "N/A -- confidence is 5/5" only for a 5/5 plan.
|
|
104
104
|
- **Sibling candidates:** files that appear to share the same anti-pattern, for the reviewer (or fix.md) to confirm. Or "N/A -- fix is test-specific" if the issue is one-off (see [`fix.md`](./fix.md) for what counts as a structural anti-pattern worth searching for).
|
|
105
|
+
- **Sibling-repo check:** for a mechanism plausibly shared between `cbh-admin-frontend` and `cbh-mobile-app`, state the sibling repository, current `main` commit searched, exact helper names or grep-able code patterns, commands and scope, and matches found (or `None found`). A mechanism is plausibly shared when the proposed fix touches the test harness, authentication/session/bootstrap, cache, API client, or a copied capability, helper, or convention. If the same mechanism is present, state that the fix deliverable will include a linked mirror implementation ticket following the Groundcrew ticket conventions, linked to the source implementation ticket and referencing the landed source fix as the pattern to adopt. If the mechanism is not plausibly shared, state `N/A` and why. Use the concrete search recipe in [`fix.md`](./fix.md#validate-the-sibling-frontend-check).
|
|
105
106
|
- **Validation plan:** lint/typecheck commands and test commands to run after applying the fix
|
|
106
107
|
- **Open questions:** anything that needs human input before fixing
|
|
107
108
|
- **Residual risk:** what could still be flaky after the fix
|