@clipboard-health/ai-rules 2.41.1 → 2.41.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (28) hide show
  1. package/package.json +1 -1
  2. package/skills/flaky-critic/SKILL.md +41 -2
  3. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/answer-key.json +242 -0
  4. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/cohort.md +37 -0
  5. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/input-hashes.txt +20 -0
  6. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-01.md +141 -0
  7. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-02.md +115 -0
  8. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-03.md +24 -0
  9. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-04.md +85 -0
  10. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-05.md +78 -0
  11. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-06.md +82 -0
  12. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-07.md +128 -0
  13. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-08.md +99 -0
  14. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-09.md +107 -0
  15. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-10.md +141 -0
  16. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-11.md +98 -0
  17. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-12.md +99 -0
  18. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-13.md +25 -0
  19. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-14.md +111 -0
  20. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-15.md +27 -0
  21. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-16.md +46 -0
  22. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-17.md +101 -0
  23. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-18.md +88 -0
  24. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-19.md +98 -0
  25. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/inputs/case-20.md +91 -0
  26. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts/verdicts.jsonl +20 -0
  27. package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts.md +170 -0
  28. package/skills/flaky-critic/references/rubric.md +17 -2
@@ -0,0 +1,46 @@
1
+ # Historical plan snapshot
2
+
3
+ ## Implementation repositories
4
+
5
+ Primary investigation touched:
6
+
7
+ - [mobile frontend]
8
+ - ClipboardHealth/[worker gateway]
9
+ - ClipboardHealth/[home-health service]
10
+
11
+ The final fix may be either test setup hardening in `[mobile frontend]` or staging config/service alignment for the worker-app BFF home-health API target.
12
+
13
+ ## Related investigation
14
+
15
+ - [ticket redacted]: `Worker accepts a visit invite > should accept a visit invite from the invites page`
16
+
17
+ ## Evidence
18
+
19
+ Run [reference redacted]` failed after the invite and booking APIs succeeded:
20
+
21
+ - `GET /[home-health service]/api/v1/visits/invites` returned a valid invite for worker `6a28d0f9eb110465a75bf4a4`, workplace `Playwright Facility MPR sch8dgis`, visit `2c9d0d13-01f8-4bda-bb7a-a1656e7ed4a2`.
22
+ - `PATCH /[home-health service]/api/v1/visits/2c9d0d13-01f8-4bda-bb7a-a1656e7ed4a2` returned 200 with `status=FILLED` and `bookedWorkerId=6a28d0f9eb110465a75bf4a4`.
23
+ - The subsequent worker-app request returned 200 but empty data twice:
24
+ `GET /worker-app/in-home/case/visit/?filter[booked]=true&filter[qualifications]=RN&include[]=case&include[]=workplace` => `{"data":[],"included":[]}`.
25
+
26
+ The same CI run also had an unticketed sibling failure in `Worker visit lifecycle`: the open visits list returned empty repeatedly for `filter[booked]=false` before any booking.
27
+
28
+ Datadog trace evidence:
29
+
30
+ - Worker-app BFF served `GET /worker-app/in-home/case/visit` using worker id `6a28d0f9eb110465a75bf4a4` and commit `531ec7f1f42caa6ef348784194b9acf56be81a51`.
31
+ - BFF called `cbh-[home-health service]-api.staging.service.local:3000/[home-health service]/api/v1/in-home-cases?filter[status]=OPEN&filter[booked]=true&filter[qualifications]=RN`.
32
+ - [home-health service] ran the raw `Cases`/`Visits` query with `c.status = OPEN`, `v.booked_worker_id = workerId`, and `LOWER(v.worker_req) = RN`, then returned no rows before workplace hydration.
33
+ - The public invite/booking requests in the browser are served as `hh-[home-health service]`, while the BFF read goes through internal `cbh-[home-health service]-api.staging.service.local`. They share the same home-health commit in the trace, but the read path did not see the newly created/updated visit.
34
+
35
+ ## Suggested fix
36
+
37
+ - Confirm whether staging public Home Health API writes and worker-app BFF internal reads point at the same service/database and use the same route prefix. If not, align `HOME_HEALTH_API_URL` or the E2E setup base URL so read/write paths are consistent.
38
+ - In `[mobile frontend]` Home Health Playwright setup, add a worker-visible readiness check after creating a visit/invite or after booking:
39
+ - poll `/worker-app/in-home/case/visit` as the worker until the seeded visit appears
40
+ - fail during setup with worker id, visit id, case id, workplace id, BFF response body, and trace URL/details if it never appears
41
+ - Add regression coverage at the service boundary if feasible: create/seed an RN Home Health visit, then assert `/in-home-cases?filter[status]=OPEN&filter[booked]=false&filter[qualifications]=RN` returns it before booking and `filter[booked]=true` returns it after booking.
42
+
43
+ ## Verification
44
+
45
+ - Re-run `playwright/e2e/homeHealth/workerAcceptsVisitInvite.spec.ts` against staging.
46
+ - Re-run `playwright/e2e/homeHealth/workerVisitLifecycle.spec.ts` because the same empty-list symptom appeared in the same run.
@@ -0,0 +1,101 @@
1
+ # Historical plan snapshot
2
+
3
+ ## Groundcrew
4
+
5
+ Repository: [admin frontend]
6
+ Implementation workflow: use the `cb-work` skill. If that skill is unavailable, use the `core:go` skill.
7
+
8
+ ## Task
9
+
10
+ Stabilize the shared Playwright worker setup path used by the license E2E test when a transient license-manager/auth response is surfaced as HTTP 401 during `worker_license_creation`.
11
+
12
+ Do not change the license UI assertions for this failure. The failing attempt never reached the browser flow; it failed in `Agent.createAgentWithRetry()` while provisioning the worker.
13
+
14
+ ## Required change / Commit Notes
15
+
16
+ - Include this implementation ticket ID in the change body.
17
+
18
+ ## Flake Details
19
+
20
+ ```json
21
+ {
22
+ "repo": "[admin frontend]",
23
+ "test": "Licenses should be able to approve and reject license as admin and verify change license logs",
24
+ "file": "playwright/e2e/licenses.spec.ts:15",
25
+ "framework": "playwright",
26
+ "isNewFlaky": false,
27
+ "failures": [
28
+ {
29
+ "error": "Error: Failed to create agent after 2 attempts.\nDiagnostics: {\"historicalCorrelationKey\":\"[correlation redacted]\",\"correlationId\":\"[correlation redacted]\",\"githubRunId\":\"28306756185\",\"githubRunAttempt\":\"1\",\"githubJob\":\"e2e-tests-staging\",\"shardIndex\":\"2\",\"shardTotal\":\"4\",\"workerIndex\":\"1\",\"parallelIndex\":\"1\",\"pid\":57618,\"status\":401,\"responseShape\":{\"message\":\"string\",\"statusCode\":\"number\"},\"apiGatewayRequestId\":\"fpZk4jo8vHcEJOQ=\",\"sanitizedResponseMessage\":\"Unauthorized\",\"errorName\":\"AxiosError\",\"errorMessage\":\"Request failed with status code 401\",\"errorCode\":\"ERR_BAD_REQUEST\"}",
30
+ "stack": "Error: Failed to create agent after 2 attempts.\nDiagnostics: {\"historicalCorrelationKey\":\"[correlation redacted]\",\"correlationId\":\"[correlation redacted]\",\"githubRunId\":\"28306756185\",\"githubRunAttempt\":\"1\",\"githubJob\":\"e2e-tests-staging\",\"shardIndex\":\"2\",\"shardTotal\":\"4\",\"workerIndex\":\"1\",\"parallelIndex\":\"1\",\"pid\":57618,\"status\":401,\"responseShape\":{\"message\":\"string\",\"statusCode\":\"number\"},\"apiGatewayRequestId\":\"fpZk4jo8vHcEJOQ=\",\"sanitizedResponseMessage\":\"Unauthorized\",\"errorName\":\"AxiosError\",\"errorMessage\":\"Request failed with status code 401\",\"errorCode\":\"ERR_BAD_REQUEST\"}\n at /opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/api/createAgent.ts:160:13\n at /opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/e2e/licenses.spec.ts:31:21\n at WorkerMain._runTest (/opt/actions-runner/_work/[admin frontend]/[admin frontend]/node_modules/dd-trace/packages/datadog-instrumentations/src/playwright.js:1692:5)",
31
+ "branch": "main",
32
+ "pipeline": "[external evidence reference]",
33
+ "commit": "94ea9f242c3211186e9613f743b137043eae003d",
34
+ "durationMs": 21949,
35
+ "shard": "2/4",
36
+ "timestamp": "2026-06-28T00:49:44Z"
37
+ }
38
+ ]
39
+ }
40
+ ```
41
+
42
+ ## Plan Output
43
+
44
+ **Test ID:** `60d14b1d432fd603aa6b-59e4c2e48b8e7a355f7f` from the downloaded `playwright-llm-report` for run [reference redacted]`.
45
+
46
+ **Confidence:** 4/5. The CI artifacts directly show the setup step, status code, request ID, and retry behavior. The only inferred link is the exact backend/auth reason why license-manager returned 401 for a token that had just worked elsewhere.
47
+
48
+ **Failure surface:** Test setup/auth/data. The test failed before app navigation or any license UI action.
49
+
50
+ **Current main status:** The failing code path still exists on current `origin/main`. `playwright/api/createAgent.ts` still classifies this 401 as `non_retryable_agent_setup_failure`; no open `flaky-test-fix` change was found for `licenses.spec.ts`/`createAgent.ts`. Recent setup/auth fixes (`[ticket redacted]`, `[ticket redacted]`, `[ticket redacted]`) improve nearby setup paths but do not cover this exact license-manager 401 classification.
51
+
52
+ **Symptom:** `playwright/e2e/licenses.spec.ts:31` failed in `agentInstance.createAgentWithRetry()` with `Failed to create agent after 2 attempts`. The final error was an Axios 401 from `POST [external evidence reference] with API Gateway request id `fpZk4jo8vHcEJOQ=`.
53
+
54
+ **Root cause:** `Agent.createAgentWithRetry()` retries whole worker setup for known transient setup failures, but its local retry classifier treats a 401 from setup provisioning as non-retryable. In this run, the admin token was freshly minted and validated, HCP creation succeeded, worker token acquisition succeeded, and another admin-token setup call (`stripe_setup`) succeeded. The 401 was isolated to `worker_license_creation` on attempt 2 and the Playwright retry passed with the same test scenario, so the frontend setup helper is bailing on a transient backend/auth failure instead of letting the existing fresh-identity retry boundary recover.
55
+
56
+ **Evidence:**
57
+
58
+ - LLM report summary: 53 tests total, 49 passed, 2 flaky, 0 failed. The license test was flaky: attempt 1 failed after 21.9s, attempt 2 passed after 19.4s.
59
+ - Attempt 1 steps show only setup/fixtures and teardown; browser network summary has 0 retained requests, confirming no UI flow ran before failure.
60
+ - Job log for correlation `[correlation redacted]`:
61
+ - `admin_token_acquisition` success at 00:49:04.
62
+ - `admin_token_validation` success at 00:49:04.
63
+ - attempt 1 `hcp_creation` failed with retryable 504 (`apiGatewayRequestId=fpZiYh88vHcEJeg=`), then the helper retried.
64
+ - attempt 2 `hcp_creation` success, `hcp_auth_token_acquisition` success, and `contract_signing` success.
65
+ - attempt 2 `worker_license_creation` failed after 5040ms with status 401, response `{"message":"Unauthorized","statusCode":401}`, `apiGatewayRequestId=fpZk4jo8vHcEJOQ=`.
66
+ - Current code evidence:
67
+ - `playwright/api/createAgent.ts` runs `worker_license_creation` and `contract_signing` inside `Promise.all` and then classifies 401 as non-retryable.
68
+ - `playwright/api/facility.ts` uses the same `worker_license_creation` endpoint but its outer helper delegates to shared `isRetryableE2eSetupError`, which treats non-403 setup errors as retryable. `createAgent.ts` is stricter and inconsistent for this setup-auth failure.
69
+
70
+ **Proposed fix:**
71
+
72
+ 1. Preserve the failing setup step name in the thrown diagnostics, not only in console logs. A pragmatic implementation is to add a small `E2eSetupStepError` wrapper in `playwright/helpers/e2eSetupDiagnostics.ts` (or equivalent structured metadata) that carries `stepName`, `stepStatus`, HTTP diagnostics, and the original cause from `runE2eSetupStep()`.
73
+ 2. Update `playwright/api/createAgent.ts` retry classification so a 401 from `worker_license_creation` is retryable at the existing `createAgentWithRetry()` whole-agent boundary. Keep 403 non-retryable. Avoid making every 401 globally retryable unless the preserved diagnostics prove it is from an E2E setup provisioning step with a valid prior admin token.
74
+ 3. Prefer retrying the whole `createAgentWithRetry()` attempt with a fresh generated worker identity over retrying the individual create-license POST. That keeps the retry idempotent if a downstream service ever writes state before returning an auth-shaped error.
75
+ 4. Add focused tests around the retry classifier / diagnostics helper if the repo has a nearby test harness pattern. At minimum, cover: 401 `worker_license_creation` is retryable, 403 remains non-retryable, missing admin token remains non-retryable, and existing 408/429/5xx behavior is unchanged.
76
+ 5. In `[admin frontend]`, include `stepName`, request method, sanitized URL/service name, HTTP status, `apiGatewayRequestId`, and `traceparent` in the final `Failed to create agent after ...` diagnostics, not just in preceding console logs.
77
+
78
+ **Sibling candidates:**
79
+
80
+ - `playwright/api/facility.ts`: same `worker_license_creation` endpoint. It appears already more permissive via `isRetryableE2eSetupError`, but verify it still behaves correctly after any diagnostics wrapper change.
81
+ - `playwright/api/agent.ts`: legacy worker setup helper creates licenses and retries setup broadly; audit for diagnostic consistency, but avoid unrelated refactors.
82
+ - [ticket redacted] is a duplicate cluster member for the same `createAgent`/token-guard family and should be covered by the same implementation.
83
+
84
+ **Validation plan:**
85
+
86
+ - `npm run lint:fast -- playwright/api/createAgent.ts playwright/helpers/e2eSetupDiagnostics.ts playwright/api/facility.ts playwright/api/agent.ts`
87
+ - `npm run typecheck`
88
+ - If staging credentials are available, run the focused Playwright spec against staging: `npx playwright test playwright/e2e/licenses.spec.ts --grep "should be able to approve and reject license as admin and verify change license logs" --project "Desktop Chrome"`.
89
+ - If the implementation only changes classifier/helper logic and a unit test is added, run the smallest relevant Jest/Vitest command for that test file as well.
90
+
91
+ **Open questions:** None blocking. The implementation should be conservative and retry only the setup-auth case supported by the artifacts.
92
+
93
+ **Residual risk:** A real persistent license-manager authorization regression would still fail after the retry budget. Without backend auth reason telemetry, future 401s may still require API Gateway/Datadog correlation to determine whether the backend or test setup should own the fix.
94
+
95
+ ## Acceptance Criteria
96
+
97
+ - [ ] `Agent.createAgentWithRetry()` retries the observed transient `worker_license_creation` 401 at the whole-agent setup boundary.
98
+ - [ ] Persistent permission failures such as 403 still bail immediately.
99
+ - [ ] Final setup failure diagnostics include the failing setup step name.
100
+ - [ ] [ticket redacted] and duplicate cluster member [ticket redacted] are referenced in the change body or implementation notes.
101
+ - [ ] Validation commands from the plan are run, or blockers are documented in the change.
@@ -0,0 +1,88 @@
1
+ # Historical plan snapshot
2
+
3
+ ## Implementation task
4
+
5
+ Implement the flaky-test-debugger plan from [ticket redacted] for this Playwright E2E flake.
6
+
7
+ ## Required change / commit instructions
8
+
9
+ - Include this implementation ticket ID in the change body.
10
+ - Implement this plan using the `cb-work` skill or, if unavailable, the `core:go` skill.
11
+
12
+ ## Flake details
13
+
14
+ ```json
15
+ {
16
+ "repo": "[admin frontend]",
17
+ "test": "Schedule daily view (redesign) — desktop workplace user posts a single shift from the Add Shift side panel (SCHEDULE_VIEW_POST_LAUNCH on)",
18
+ "file": "playwright/e2e/dailyView.spec.ts:420",
19
+ "framework": "playwright",
20
+ "isNewFlaky": false,
21
+ "failures": [
22
+ {
23
+ "error": "TimeoutError: locator.click: Timeout 30000ms exceeded. Call log: waiting for getByRole('combobox', { name: 'Role' }); locator canonical; element was not stable; element was detached from the DOM, retrying",
24
+ "branch": "main",
25
+ "pipeline": "[external evidence reference]",
26
+ "commit": "b568dcdc304406da26be6128e183cae35390fa64",
27
+ "durationMs": 57388,
28
+ "shard": "2/4",
29
+ "timestamp": "2026-07-02T19:06:56Z"
30
+ }
31
+ ]
32
+ }
33
+ ```
34
+
35
+ ## Flaky-test-debugger plan
36
+
37
+ **Test ID:** `b03b0307e93e39946710-71ce947e36caa7d4ae78`
38
+
39
+ **Confidence:** 4/5. The Playwright LLM report and screenshot show the panel/Role field is transient after opening, but the report does not retain a close-state RUM event or DOM snapshot at the exact detach moment, so the precise close/remount trigger is inferred.
40
+
41
+ **Failure surface:** User action no-op / test harness synchronization.
42
+
43
+ **Current main status:** Still relevant. The failing helper still performs `await page.getByRole("combobox", { name: "Role" }).click()` immediately after visible checks in `playwright/e2e/dailyView.spec.ts`. Recent main contains prior stabilizations from [ticket redacted]/[ticket redacted] and a later roster Add Shift panel change (`aaa7a899d7e`), but no current open `flaky-test-fix` change and no later change that waits for the Role combobox to be actionable after the side-panel transition/remount window.
44
+
45
+ **Symptom:** Failed attempt opened the Add Shift panel, saw the `Add Shift` title and visible Role combobox, then timed out for 30s clicking the Role combobox. Playwright reported the element was not stable and was detached from the DOM. The failure screenshot shows the schedule page without the Add Shift side panel.
46
+
47
+ **Root cause:** The test starts Role selection as soon as the panel title and Role field are visible. Visibility is too early for this surface: the desktop Add Shift side panel is animated and its form can still re-render/reset while route/day data and panel-related queries settle. The first destructive operation is not the role click, so the test can safely make the panel-open/role-selection phase idempotent before posting, instead of relying on one immediate Role click.
48
+
49
+ **Evidence:**
50
+
51
+ - LLM report summary: 52 total tests, 49 passed, 1 flaky, 2 skipped; this test failed attempt 1 and passed attempt 2.
52
+ - Failed attempt timeline: `openAddShiftPanel` clicked `Add` at ~26.25s, `Add Shift` was visible at ~26.37s, Role was visible at ~26.386s, and the Role click started at ~26.391s then timed out after 30s.
53
+ - Error artifact: `locator.click` on `getByRole('combobox', { name: 'Role' })` hit unstable element and DOM detachment.
54
+ - Screenshot artifact: final failure image shows the daily schedule with no Add Shift side panel open.
55
+ - Network/lifecycle: no `POST /v3/shifts` was emitted. The failed attempt had 333 observed network instances with no group/instance-cap drops; background schedule/workplace/default-time requests were still active around the panel-open window. `GET /api/shiftTimes/get/...qualification=CNA` returned 200.
56
+ - RUM artifact: retained custom action `HCF Admin Web App (V2): Schedule add shift panel state changed` with `isOpen: true`; no retained matching close event.
57
+ - Code path: `postSingleShiftViaAddPanel` in `playwright/e2e/dailyView.spec.ts` waits for title/Role visibility, then immediately clicks Role. `openAddShiftPanel` retries the safe Add click but does not wait for the side-panel transition or Role actionability.
58
+
59
+ **Proposed fix:** Test harness fix in `playwright/e2e/dailyView.spec.ts`.
60
+
61
+ 1. Add a small helper for the safe pre-post phase, for example `selectRoleInAddShiftPanel({ page, roleName: "CNA" })`.
62
+ 2. In that helper, use `expect(...).toPass()` around only idempotent actions:
63
+ - ensure the Add Shift panel is open; if it disappeared before any post, call `openAddShiftPanel` again;
64
+ - locate the Role combobox;
65
+ - use `roleCombobox.click({ trial: true, timeout: 5_000 })` to wait for Playwright actionability/stability without changing state;
66
+ - if the combobox is not already the desired role, click it and choose the option;
67
+ - assert the combobox contains the chosen role before leaving the retry block.
68
+ 3. Keep the destructive post flow outside any retry: wait for date/default times/Post enabled, create the `waitForResponse` promise, click Post once, handle optional rush-fee dialog, then assert the panel closes.
69
+ 4. Reuse the same helper for the mobile bottom-sheet test because it calls the same `postSingleShiftViaAddPanel` path. Do not add `waitForTimeout`; use actionability/trial click and assertions as deterministic signals.
70
+
71
+ **Observability to reach 5/5:**
72
+
73
+ - Add a close-reason field to the existing Add Shift panel RUM state event (`closeButton`, `postSuccess`, `inviteSuccess`, `hostOverlayReplacement`, `unmount`, etc.) so the next occurrence proves whether the panel ended, remounted, or only moved during transition.
74
+ - Extend the Playwright reporter or test helper to attach a targeted screenshot/DOM snapshot when a locator actionability failure mentions `detached from the DOM`, especially for panel/dialog surfaces.
75
+ - Consider adding a test-only `data-testid` or stable panel root role/name for Add Shift panel scoping if the helper still has to use global locators.
76
+
77
+ **Sibling candidates:** The mobile Add Shift bottom-sheet E2E in the same file shares `postSingleShiftViaAddPanel`; update it automatically through the shared helper. No other E2E currently performs this exact Add Shift panel Role selection.
78
+
79
+ **Validation plan:**
80
+
81
+ - `npx oxfmt --check playwright/e2e/dailyView.spec.ts`
82
+ - `npm run lint -- playwright/e2e/dailyView.spec.ts`
83
+ - If deployed E2E env credentials are available: `npm run playwright:run -- --grep "posts a single shift from the Add Shift"`
84
+ - If local Playwright cannot start because `TEST_HELPER_API_KEY` / deployed API env is missing, state that in the change and rely on CI for the E2E run.
85
+
86
+ **Open questions:** None.
87
+
88
+ **Residual risk:** If the panel is genuinely closing for real users without any outside click, the test-harness fix may mask a product bug. The close-reason telemetry above would make that distinguishable in the next occurrence.
@@ -0,0 +1,98 @@
1
+ # Historical plan snapshot
2
+
3
+ ## Groundcrew
4
+
5
+ Repository: [admin frontend]
6
+ Implementation workflow: use the `cb-work` skill when available. If that skill is unavailable, use the `core:go`/`go` skill. Include this implementation ticket ID in the change body.
7
+
8
+ ```bash
9
+
10
+ ```
11
+
12
+ ## Task
13
+
14
+ Fix the shared workplace magic-link Playwright sign-in harness so successful but slow auth completion does not cause `retryUntilPassOrTimeout` to retry on the same page and then wait for the login Email field after the session is already authenticated.
15
+
16
+ ## Acceptance Criteria
17
+
18
+ - [ ] `signInAsWorkplaceUser` no longer times out filling `getByLabel("Email")` when a previous magic-link attempt completes auth after the redirect wait budget.
19
+ - [ ] The helper still retries promptly when the magic link is expired/already used or the auth error screen is visible.
20
+ - [ ] The fix is applied to sibling workplace magic-link/OTP retry paths that navigate to `/login` and fill Email inside `retryUntilPassOrTimeout`.
21
+ - [ ] The change body includes this implementation ticket ID.
22
+
23
+ ## Flake Details
24
+
25
+ ```json
26
+ {
27
+ "repo": "[admin frontend]",
28
+ "test": "Reporting (redesign) — desktop page chrome and all four KPI cards render on landing",
29
+ "file": "playwright/e2e/reporting.spec.ts:15",
30
+ "framework": "playwright",
31
+ "isNewFlaky": true,
32
+ "failures": [
33
+ {
34
+ "error": "Error: locator.fill: Timeout 30000ms exceeded.\nCall log:\n - waiting for getByLabel('Email')\n\n\nCall Log:\n- Timeout 90000ms exceeded while waiting on the predicate",
35
+ "stack": "Error: locator.fill: Timeout 30000ms exceeded.\nCall log:\n - waiting for getByLabel('Email')\n\n\nCall Log:\n- Timeout 90000ms exceeded while waiting on the predicate\n at retryUntilPassOrTimeout (/opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/helpers/common.ts:6:26)\n at signInAsWorkplaceUser (/opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/helpers/scheduleSeed.ts:97:32)\n at /opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/e2e/reporting.spec.ts:20:7\n at WorkerMain._runTest (/opt/actions-runner/_work/[admin frontend]/[admin frontend]/node_modules/dd-trace/packages/datadog-instrumentations/src/playwright.js:1692:5)",
36
+ "branch": "main",
37
+ "pipeline": "[external evidence reference]",
38
+ "commit": "4804367a1bf5dc5f55e097868ac06decc8fc175c",
39
+ "durationMs": [attempt reference redacted],
40
+ "shard": "3/4",
41
+ "timestamp": "2026-07-01T13:51:35Z"
42
+ }
43
+ ]
44
+ }
45
+ ```
46
+
47
+ ## Plan Output
48
+
49
+ **Test ID:** c5fb8ef2d524
50
+
51
+ **Confidence:** 5/5. The LLM report timeline, network events, and screenshot directly show the helper timing out a successful auth attempt, retrying, and then ending on the authenticated Schedule page while waiting for the login Email field.
52
+
53
+ **Failure surface:** test setup/auth/data. The reporting page was never reached in the failed attempt.
54
+
55
+ **Current main status:** the failing commit `4804367a1bf5dc5f55e097868ac06decc8fc175c` is still represented by current main/current branch. The shared `signInAsWorkplaceUser` path and `waitForWorkplaceMagicLinkSignIn` helper still exist. There is no open `flaky-test-fix` change for `reporting.spec.ts`, `scheduleSeed.ts`, or `workplaceSignIn.ts`. Prior commit `766c7575fbd` added the 15s bounded redirect wait, but this flake occurred after that change.
56
+
57
+ **Symptom:** `playwright/e2e/reporting.spec.ts:15` failed in `signInAsWorkplaceUser` before `gotoAppRoute({ url: "/facility/invoice/reporting" })`. The outer `retryUntilPassOrTimeout` expired after the retry callback hit `locator.fill` timeout waiting for `getByLabel('Email')`.
58
+
59
+ **Root cause:** `waitForWorkplaceMagicLinkSignIn` treats a workplace magic-link redirect that takes more than 15s as a failed attempt. In this run, the first magic-link Continue click was valid and Cognito returned success, but authenticated app bootstrap did not finish until roughly 20.7s after the click. The helper timed out at 15s, the retry loop navigated back to `/login` on the same page, and the late success from the first attempt established the session and redirected to `/schedule`. The retry then waited for an Email field on an authenticated Schedule page. Later retries repeated the same pattern because the page was already signed in.
60
+
61
+ **Evidence:**
62
+
63
+ - LLM report summary: 52 total tests, 1 flaky, 0 failed after retry. The flaky entry is exactly `Reporting (redesign) — desktop > page chrome and all four KPI cards render on landing`.
64
+ - Failed attempt timeline: first `/login` navigation at 12.106s, Email fill at 13.324s, magic-link page navigation at 17.336s, Continue click at 18.181s, then `Expect "toHaveURL"` failed after 15.003s.
65
+ - The same failed attempt retried at 34.231s with `Navigate to "/login"`, then `Fill "playwright-fmuepswefho@playwright-hcf.com" getByLabel('Email')` started at 38.854s and timed out after 30.012s.
66
+ - Authenticated workplace traffic began immediately after that retry fill started: `/api/facilityUser/findByEmail` returned 200 at 38.859s and `/api/user/get/...` returned 200 at 38.926s.
67
+ - The failure screenshot shows the authenticated Schedule page for `test facility mvsij0epxt`, not the login page.
68
+ - No reporting API/navigation/assertion ran in the failed attempt; the retry passed and reached `/facility/invoice/reporting` normally.
69
+
70
+ **Proposed fix:** test harness fix in `playwright/helpers/workplaceSignIn.ts` and the magic-link callers.
71
+
72
+ 1. Replace the fixed 15s schedule URL wait with an outcome-based wait: allow a successful `/schedule` redirect for up to the same order as other Cognito login flows (the password helper uses 60s), while racing/polling for visible expired/already-used/auth-error text so bad links still fail promptly and request a fresh link.
73
+ 2. Make retry attempts idempotent when a previous auth attempt finishes late. Before filling Email after `page.goto("/login")`, wait for either the login Email field or an authenticated schedule URL. If schedule is reached, return success instead of filling the form.
74
+ 3. Apply the same already-authenticated/form-ready guard to sibling retry helpers that call `/login` and fill Email inside `retryUntilPassOrTimeout`, notably the local `submitMagicLinkForm` in `playwright/e2e/auth.spec.ts` and the email OTP request helper in `playwright/e2e/cognito/cognitoEmailOtpAuth.spec.ts` if it shares the same retry hazard.
75
+ 4. Keep the diagnostics from `workplaceSignIn.ts`: sanitized current URL, visible auth error text, elapsed time since link request, schedule reached boolean, and magic-link marker.
76
+
77
+ **Observability to reach 5/5:** N/A -- confidence is 5/5 for the test-harness root cause. Optional improvement: add a structured stdout event when workplace magic-link sign-in times out or resolves as already authenticated after a retry, including elapsed time and sanitized URL, so future auth setup flakes can be grouped without opening the screenshot.
78
+
79
+ **Sibling candidates:** all specs using `signInAsWorkplaceUser` from `playwright/helpers/scheduleSeed.ts`; direct magic-link auth retries in `playwright/e2e/auth.spec.ts`; OTP retry flow in `playwright/e2e/cognito/cognitoEmailOtpAuth.spec.ts`.
80
+
81
+ **Validation plan:**
82
+
83
+ - `npm run lint:fast -- playwright/helpers/workplaceSignIn.ts playwright/helpers/scheduleSeed.ts playwright/e2e/auth.spec.ts playwright/e2e/cognito/cognitoEmailOtpAuth.spec.ts`
84
+ - `npx oxfmt --check playwright/helpers/workplaceSignIn.ts playwright/helpers/scheduleSeed.ts playwright/e2e/auth.spec.ts playwright/e2e/cognito/cognitoEmailOtpAuth.spec.ts`
85
+ - `npx cspell --no-must-find-files playwright/helpers/workplaceSignIn.ts playwright/helpers/scheduleSeed.ts playwright/e2e/auth.spec.ts playwright/e2e/cognito/cognitoEmailOtpAuth.spec.ts`
86
+ - `npm run typecheck`
87
+ - With staging E2E credentials available: `npx playwright test playwright/e2e/reporting.spec.ts --project="Desktop Chrome" --grep "page chrome and all four KPI cards render on landing"`
88
+ - Prefer also running the desktop auth magic-link test if credentials are available: `npx playwright test playwright/e2e/auth.spec.ts --project="Desktop Chrome" --grep "clicking the emailed verification link signs the user in"`
89
+
90
+ **Open questions:** none blocking. Use the smallest harness change that preserves fast retry for explicit auth errors while preventing a slow success from poisoning same-page retries.
91
+
92
+ **Residual risk:** this fixes the retry/idempotency race. It does not explain why successful Cognito workplace magic-link completion sometimes takes more than 15s in staging CI; if that latency grows past the new success window, the helper should fail with diagnostics rather than a misleading Email locator timeout.
93
+
94
+ ## Investigation Notes
95
+
96
+ - The repository did not contain `scripts/fetch-llm-report.sh`, so the LLM report was downloaded with `gh run download 28522523810 --name playwright-llm-report`.
97
+ - Artifact path used for investigation: `/tmp/playwright-llm-report-28522523810/llm-report.json`.
98
+ - Failure screenshot decoded locally to `/tmp/reporting-flake-attempt0.png`; it showed the authenticated Schedule page.
@@ -0,0 +1,91 @@
1
+ # Historical plan snapshot
2
+
3
+ ## Implementation instructions
4
+
5
+ Implement the plan below in `[admin frontend]` using the `cb-work` skill or, if unavailable, the `core:go` skill.
6
+
7
+ Include this implementation ticket ID in the change body.
8
+
9
+ ```bash
10
+
11
+ ```
12
+
13
+ ## Flake details
14
+
15
+ ```json
16
+ {
17
+ "repo": "[admin frontend]",
18
+ "test": "Schedule rate negotiations (redesign) — desktop workplace user accepts one rate proposal and ends another from the shift details drawer",
19
+ "file": "playwright/e2e/rateNegotiation.spec.ts:40",
20
+ "framework": "playwright",
21
+ "isNewFlaky": false,
22
+ "failures": [
23
+ {
24
+ "error": "Error: Failed to create agent after 1 attempt.\nDiagnostics: {\"historicalCorrelationKey\":\"[correlation redacted]\",\"correlationId\":\"[correlation redacted]\",\"githubRunId\":\"28798037729\",\"githubRunAttempt\":\"1\",\"githubJob\":\"e2e-tests-staging\",\"shardIndex\":\"3\",\"shardTotal\":\"4\",\"workerIndex\":\"1\",\"parallelIndex\":\"1\",\"pid\":[attempt reference redacted],\"stepName\":\"hcp_creation\",\"stepStatus\":\"failure\",\"durationMs\":166,\"requestMethod\":\"POST\",\"requestUrl\":\"[external evidence reference]",\"requestService\":\"api\",\"status\":400,\"responseShape\":{\"error\":\"string\",\"message\":\"string\",\"statusCode\":\"number\"},\"traceparent\":\"00-00000000000000001732fff1044f982e-197732b055021fe4-01\",\"apiGatewayRequestId\":\"AFm5ki1zvHcESJw=\",\"sanitizedResponseMessage\":\"Error while creating user\\nBad Request\",\"errorName\":\"AxiosError\",\"errorMessage\":\"Request failed with status code 400\",\"errorCode\":\"ERR_BAD_REQUEST\"}",
25
+ "stack": "Error: Failed to create agent after 1 attempt.\nDiagnostics: {\"historicalCorrelationKey\":\"[correlation redacted]\",\"correlationId\":\"[correlation redacted]\",\"githubRunId\":\"28798037729\",\"githubRunAttempt\":\"1\",\"githubJob\":\"e2e-tests-staging\",\"shardIndex\":\"3\",\"shardTotal\":\"4\",\"workerIndex\":\"1\",\"parallelIndex\":\"1\",\"pid\":[attempt reference redacted],\"stepName\":\"hcp_creation\",\"stepStatus\":\"failure\",\"durationMs\":166,\"requestMethod\":\"POST\",\"requestUrl\":\"[external evidence reference]",\"requestService\":\"api\",\"status\":400,\"responseShape\":{\"error\":\"string\",\"message\":\"string\",\"statusCode\":\"number\"},\"traceparent\":\"00-00000000000000001732fff1044f982e-197732b055021fe4-01\",\"apiGatewayRequestId\":\"AFm5ki1zvHcESJw=\",\"sanitizedResponseMessage\":\"Error while creating user\\nBad Request\",\"errorName\":\"AxiosError\",\"errorMessage\":\"Request failed with status code 400\",\"errorCode\":\"ERR_BAD_REQUEST\"}\n at /opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/api/createAgent.ts:152:13\n at /opt/actions-runner/_work/[admin frontend]/[admin frontend]/playwright/e2e/rateNegotiation.spec.ts:84:24\n at WorkerMain._runTest (/opt/actions-runner/_work/[admin frontend]/[admin frontend]/node_modules/dd-trace/packages/datadog-instrumentations/src/playwright.js:1692:5)",
26
+ "branch": "main",
27
+ "pipeline": "[external evidence reference]",
28
+ "commit": "1983e526d20ac66dc2c52102caeef8d6bba92cd2",
29
+ "durationMs": 8074,
30
+ "shard": "3/4",
31
+ "timestamp": "2026-07-06T14:14:13Z"
32
+ }
33
+ ]
34
+ }
35
+ ```
36
+
37
+ ## Plan output
38
+
39
+ **Test ID:** `2f8cacf23bff`
40
+
41
+ **Confidence:** 4/5. The failing artifact shows `createAgentWithRetry()` bailed after one `hcp_creation` 400, and Datadog logs for the same trace show a Cognito `ConflictException` during backend user creation. The only inferred link is which identifier collided, because the API response intentionally returned a generic message.
42
+
43
+ **Failure surface:** test setup/auth/data.
44
+
45
+ **Current main status:** The failing code path still exists on current `origin/main`. Prior fixes are already available: `[reference redacted]` / `dcd544114bc` classified visible `Phone already in use` 400s, and `[reference redacted]` / `6d3f71348c7` scoped that retry to worker creation. This occurrence is not covered because the response body was only `Error while creating user\nBad Request`.
46
+
47
+ **Symptom:** In `playwright/e2e/rateNegotiation.spec.ts:84`, the first worker setup call fails before any browser user-flow assertion. `Agent.createAgentWithRetry()` reports `Failed to create agent after 1 attempt` even though its default max is 5, because `getAgentSetupRetryClassification()` returned `non_retryable_agent_setup_failure` and called `bail()`.
48
+
49
+ **Root cause:** The E2E worker setup retry classifier only treats `hcp_creation` HTTP 400 responses as retryable when the sanitized response includes `Phone already in use`. For this run, `POST /api/user/create` returned a generic 400. Datadog APM/logs for trace id `1671679822332401710` show the backend span `cbh-user POST /api/user/create`, HTTP 400, `BadRequestException: Error while creating user`, and a log error from `/app/src/modules/auth/services/cognito.service.ts` with `ConflictException` in `ensureIdentifierAvailableForEmployeeUpdate`. That is an identity collision/setup-data failure, but the frontend harness cannot see the precise backend conflict reason and therefore does not retry with the next generated worker identity.
50
+
51
+ **Evidence:**
52
+
53
+ - LLM report `/tmp/playwright-llm-report-28798037729/llm-report.json`: 52 tests, 1 flaky, failed attempt error `Failed to create agent after 1 attempt`, passed retry duration 28.475s.
54
+ - Failure diagnostics: `stepName=hcp_creation`, `POST [external evidence reference] status 400, `traceparent=00-00000000000000001732fff1044f982e-197732b055021fe4-01`, `apiGatewayRequestId=AFm5ki1zvHcESJw=`, sanitized response `Error while creating user\nBad Request`.
55
+ - Datadog APM: W3C trace low-64 id `1671679822332401710`; span `6264525095033341259`, service `cbh-user`, resource `POST /api/user/create`, HTTP status code 400, duration 113.5ms, `exception.message=Error while creating user`, `exception.type=BadRequestException`.
56
+ - Datadog logs for that trace: `Error while creating user`; error kind `ConflictException`; stack includes `CognitoService.throwRealWorkerConflict`, `classifyOrphanCandidates`, `ensureIdentifierAvailableForEmployeeUpdate`, `UserManipulationService.handleNewAgentInIdentityProviders`, and `UserManipulationService.createAgent`.
57
+ - Code path: `playwright/api/createAgent.ts` generates a fresh phone/email inside the retry callback, but `playwright/helpers/createAgentRetryClassification.ts` bails on this generic 400 shape.
58
+ - Existing regression coverage is in `scripts/createAgentRetryClassification.test.ts` and currently only covers the visible `Phone already in use` 400 case.
59
+
60
+ **Proposed fix:** Test harness fix.
61
+
62
+ - In `playwright/helpers/createAgentRetryClassification.ts`, extend the worker-creation 400 classifier narrowly. Treat this shape as retryable only when all of these hold: status is 400, diagnostics `stepName` is `hcp_creation`, `requestUrl` or `requestService` identifies `/api/user/create`, and `sanitizedResponseMessage` matches the generic user-creation bad request/conflict shape (`Error while creating user\nBad Request`, and optionally `Conflict Exception` if present).
63
+ - Return the existing `retryable_phone_collision` classification to minimize churn, or rename it to `retryable_worker_identity_collision` if the implementer wants the log label to match email/phone/Cognito identifier conflicts more accurately. If renamed, update the union type and tests in the same change.
64
+ - Do not classify all `hcp_creation` 400s as retryable; request-shape/schema failures should still fail fast.
65
+ - Add/extend Jest coverage in `scripts/createAgentRetryClassification.test.ts` for the generic `/api/user/create` 400 response and for a negative 400 case from an unrelated setup step or endpoint.
66
+
67
+ **Observability to reach 5/5:** Add a safe machine-readable reason to setup diagnostics or backend response/log tags, such as `worker_identity_conflict`, without exposing phone/email. Best options: have `cbh-user` tag/log the conflict classification on the `POST /api/user/create` span, and have `runE2eSetupStep` preserve a non-PII error code when the response includes one. That would remove the need to infer from backend stack names.
68
+
69
+ **Sibling candidates:**
70
+
71
+ - `playwright/api/createAgent.ts` is the failing class helper and should get the benefit through the shared classifier.
72
+ - `playwright/api/facility.ts` already documents the same backend generic `Error while creating user / Conflict Exception` collision shape around its HCP setup path; confirm whether its retry wrapper already converges or needs the same classification behavior.
73
+ - `playwright/api/agent.ts` has an older `createAgentWithRetry()` helper that retries all failures without this classifier; no immediate change required, but it is related setup surface.
74
+
75
+ **Validation plan:**
76
+
77
+ ```bash
78
+ npx jest scripts/createAgentRetryClassification.test.ts --runInBand
79
+ npm run lint:fast -- playwright/helpers/createAgentRetryClassification.ts scripts/createAgentRetryClassification.test.ts
80
+ npm run typecheck
81
+ ```
82
+
83
+ If the implementer can run staging E2E safely, also run the target spec shard or the single spec:
84
+
85
+ ```bash
86
+ npx playwright test playwright/e2e/rateNegotiation.spec.ts --project=chromium
87
+ ```
88
+
89
+ **Open questions:** None blocking. The implementation should use the current sanitized response shape and avoid relying on PII from Datadog logs.
90
+
91
+ **Residual risk:** If staging has enough retained test users that the 9M generated phone space produces frequent collisions, retries will reduce flake rate but not eliminate it forever. A later stronger fix could expand the valid generated phone space or add a test-only reserved identity namespace in the backend.
@@ -0,0 +1,20 @@
1
+ {"file":"case-01.md","verdict":"reject","confidence":"high","classification":["A5"],"rules":["B5","B8","C6"],"statementMissing":["B5-dedup","B8","C6"],"substantive":["none"],"newRuleB8":"statement-missing","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan identifies `sendLoginEmail` discarding the canonical username, but contains no explicit \"Causal chain\" section, complete dedup result, or sibling-repo check for the shared auth mechanism."}
2
+ {"file":"case-02.md","verdict":"reject","confidence":"high","classification":["A3"],"rules":["B8"],"statementMissing":["none"],"substantive":["B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan claims \"Confidence: 5/5\" from the observed boundary timing but provides no fault injection or focused reproduction that induces the blamed clock-boundary cause."}
3
+ {"file":"case-03.md","verdict":"reject","confidence":"high","classification":["A5"],"rules":["B5","B6","B7","B8","C6"],"statementMissing":["B5-dedup","B6","B7","B8","C6"],"substantive":["B5-prior-attempts"],"newRuleB8":"statement-missing","newRulePriorAttempts":"substantive","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The supplied plan says \"the 12 prior stabilization commits did that and the flake recurred each time,\" establishing a chronic family with at least two failed merged fixes."}
4
+ {"file":"case-04.md","verdict":"reject","confidence":"high","classification":["C4"],"rules":["B5","B8","C6"],"statementMissing":["C6"],"substantive":["B5-prior-attempts","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"substantive","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"It states two prior fixes \"are already completed and this June 27 recurrence happened after those changes\" and claims 5/5 without inducing the CloudFront-limit mechanism."}
5
+ {"file":"case-05.md","verdict":"reject","confidence":"high","classification":["C1"],"rules":["B8","C6"],"statementMissing":["C6"],"substantive":["B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan claims \"Confidence: 5/5\" for the collision classifier without a focused reproduction that induces the identity collision and proves the fresh-identity retry."}
6
+ {"file":"case-06.md","verdict":"reject","confidence":"high","classification":["A5"],"rules":["B8","C6"],"statementMissing":["B8","C6"],"substantive":["none"],"newRuleB8":"statement-missing","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"Although the invariant is traced to the global `GoogleMapsSdkProvider`, the plan omits the required explicit \"Causal chain\" and sibling-repo check sections for an app-bootstrap mechanism."}
7
+ {"file":"case-07.md","verdict":"reject","confidence":"high","classification":["A1","C4"],"rules":["B8","C6"],"statementMissing":["B8","C6"],"substantive":["none"],"newRuleB8":"statement-missing","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The workflow evidence shows blank `EXPECTED_DEPLOYED_COMMIT_SHA`, but the plan has no explicit \"Causal chain\" section or sibling-repo check for the shared remote-E2E workflow mechanism."}
8
+ {"file":"case-08.md","verdict":"reject","confidence":"high","classification":["C1"],"rules":["B4","B6","B8","C6"],"statementMissing":["C6"],"substantive":["B4","B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"It admits \"The exact reason the dialog disappeared is inferred\" yet assigns 4/5 confidence and retries the test-side click instead of terminating the product-side remount/disappearance chain."}
9
+ {"file":"case-09.md","verdict":"reject","confidence":"moderate","classification":["A3"],"rules":["B8","C6"],"statementMissing":["B8","C6"],"substantive":["none"],"newRuleB8":"statement-missing","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":true,"justification":"The low-minute boundary and future-time validation are evidenced, but the plan lacks an explicit \"Causal chain\" section and sibling-repo check for the shared timekeeping harness mechanism."}
10
+ {"file":"case-10.md","verdict":"reject","confidence":"high","classification":["A1","A7","C1"],"rules":["A7","B6","B8","C6"],"statementMissing":["C6"],"substantive":["A7","B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan lists three unresolved causes—propagation, premature backend success, or throttling—while requiring a code fix at 4/5 confidence and omitting A7 validation with Playwright retries disabled."}
11
+ {"file":"case-11.md","verdict":"reject","confidence":"high","classification":["A5"],"rules":["B5","B6","B8","C6"],"statementMissing":["B5-dedup","C6"],"substantive":["B5-prior-attempts","B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"substantive","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The snapshot names a \"prior implementation for the same family\" plus prior test-only hardenings, while admitting \"exact identification of the competing history writer\" is still missing."}
12
+ {"file":"case-12.md","verdict":"reject","confidence":"high","classification":["A5"],"rules":["B8","C6"],"statementMissing":["B8","C6"],"substantive":["none"],"newRuleB8":"statement-missing","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The stale `offsetX` completion check is tied to specific component lines, but the plan still omits the mandatory explicit \"Causal chain\" and sibling-repo check sections."}
13
+ {"file":"case-13.md","verdict":"reject","confidence":"moderate","classification":["none"],"rules":["B5"],"statementMissing":["B5-prior-attempts"],"substantive":["none"],"newRuleB8":"none","newRulePriorAttempts":"statement-missing","verdictIgnoringNewRuleStatementGaps":"needs-human","borderline":true,"justification":"The snapshot says \"Previous investigations ... canonical earlier occurrences\" and directs comparison with a \"previous fix,\" but supplies neither a Prior attempts section nor an implementation plan classifiable under A/C/D."}
14
+ {"file":"case-14.md","verdict":"reject","confidence":"high","classification":["C1"],"rules":["B4","B6","B8","C6"],"statementMissing":["C6"],"substantive":["B4","B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan says \"The exact React state transition that collapsed/unmounted the visits subtree is inferred\" but proposes repeated test clicks at 4/5 confidence."}
15
+ {"file":"case-15.md","verdict":"reject","confidence":"moderate","classification":["A7"],"rules":["B5","B6","B7","B8","C6"],"statementMissing":["B5-dedup","B5-prior-attempts","B6","B7","B8","C6"],"substantive":["none"],"newRuleB8":"statement-missing","newRulePriorAttempts":"statement-missing","verdictIgnoringNewRuleStatementGaps":"reject","borderline":true,"justification":"The plan cites prior \"consumer-side mitigations 152fe5ea8d7/32c31abeeef\" but has no Prior attempts, confidence, current-main, causal-chain, dedup, or full sibling-repo sections."}
16
+ {"file":"case-16.md","verdict":"reject","confidence":"high","classification":["A1","C5"],"rules":["A1","B4","B5","B6","B7","B8","C6"],"statementMissing":["B5-dedup","B7","C6"],"substantive":["A1","B4","B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan has not resolved whether writes and reads use different services/databases, yet proposes a test-side poll without a stated bound and says the final fix \"may be either\" harness hardening or service alignment."}
17
+ {"file":"case-17.md","verdict":"reject","confidence":"high","classification":["C1"],"rules":["B6","B8","C6"],"statementMissing":["C6"],"substantive":["B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"It admits \"the exact backend/auth reason why license-manager returned 401\" is unknown, so the causal chain terminates at a status code while a 4/5 code fix retries that status."}
18
+ {"file":"case-18.md","verdict":"reject","confidence":"high","classification":["C1"],"rules":["B4","B5","B6","B8","C6"],"statementMissing":["C6"],"substantive":["B4","B5-prior-attempts","B6","B8"],"newRuleB8":"substantive","newRulePriorAttempts":"substantive","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The current branch contains two \"prior stabilizations\" before this recurrence, while the plan admits the precise close/remount trigger is inferred and the harness fix \"may mask a product bug.\""}
19
+ {"file":"case-19.md","verdict":"reject","confidence":"high","classification":["A1","C1"],"rules":["B8","C6"],"statementMissing":["C6"],"substantive":["B8"],"newRuleB8":"substantive","newRulePriorAttempts":"none","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"The plan claims \"Confidence: 5/5\" from the observed late success but provides no fault injection or focused reproduction inducing the delayed-auth retry race."}
20
+ {"file":"case-20.md","verdict":"reject","confidence":"high","classification":["C1"],"rules":["B5","B8","C6"],"statementMissing":["B8","C6"],"substantive":["B5-prior-attempts"],"newRuleB8":"statement-missing","newRulePriorAttempts":"substantive","verdictIgnoringNewRuleStatementGaps":"reject","borderline":false,"justification":"Two prior fixes already classified and scoped the same worker-identity-collision retry mechanism, and this recurrence re-proposes extending that mechanism to the generic conflict response."}