@clipboard-health/ai-rules 2.41.1 → 2.41.2

package/skills/flaky-critic/references/backtests/2026-07-16-b8-prior-attempts.md

@@ -0,0 +1,170 @@
+# B8 and Prior-attempts blind backtest — 2026-07-16
+
+## Cohort
+
+- **Window:** 2026-06-01 through 2026-07-16.
+- **Size and balance:** 20 historical flaky-fix plans: 14 human approves and 6 human rejects.
+- **Source:** each input is the Linear implementation-ticket description captured in the timestamped GitHub `linear-code` linkback comment when the ticket was linked to its change, before later human review or closing context. All 20 snapshots were present and non-truncated.
+- **Coverage:** browser E2E, mobile E2E, service tests, product races, auth/setup, retry classification, third-party bootstrap, clock boundaries, locators, and CI/deployment guards.
+- **Label caveat:** only one reject had a strong contemporaneous requested-change judgment. Five rejects were stale, superseded, or closed without a final public quality rationale. All 14 approves were integrated; 13 also carried the frontend repositories' release marker.
+- **Blinding:** an outcome-aware assembler fixed and sanitized the cohort. A separate judge read only the current flaky-critic skill, rubric, and 20 opaque case files. Required outcome-leak and identifier scans returned zero matches. The judge returned exactly one well-formed verdict per case.
+- **Audit bundle:** the [cohort notes](./2026-07-16-b8-prior-attempts/cohort.md), [answer key and source manifest](./2026-07-16-b8-prior-attempts/answer-key.json), [input hashes](./2026-07-16-b8-prior-attempts/input-hashes.txt), [structured blind verdicts](./2026-07-16-b8-prior-attempts/verdicts.jsonl), and sanitized `inputs/` are frozen beside this report. The manifest records the exact plan-comment permalink, capture time, pre-outcome check, and SHA-256 for every case.
+- **Correlations:** three cases shared a Cognito identity family, two shared worker-creation collision mechanics, and four touched Home Health through distinct immediate mechanisms. Results are therefore not 20 fully independent mechanisms.
+
+## Raw confusion matrix
+
+The current rubric rejected all 20 cases.
+
+|                    | Human approve | Human reject |
+| ------------------ | ------------: | -----------: |
+| Critic approve     |             0 |            0 |
+| Critic reject      |            14 |            6 |
+| Critic needs-human |             0 |            0 |
+
+- **Agreement:** 6/20 (**30%**).
+- **False-reject rate:** 14/14 human-approved plans (**100%**).
+- **Escalation rate:** 0/20.
+- **Strong-label subset:** 1/15 agreement (**6.7%**). The other five raw matches were weak reject labels, so the 30% headline overstates evidence that the rubric reproduced a human quality judgment.
+- The one strong human reject was also an unearned mechanism match: the human rejected an obsolete authentication direction, while the critic rejected format omissions under B5/B6/B7/B8/C6.
+
+## New-rule bounce attribution
+
+One plan may be attributed to both rules, so per-rule counts are not additive.
+
+| New rule                | Any bounce | Statement-missing | Substantive |
+| ----------------------- | ---------: | ----------------: | ----------: |
+| B8 causal chain         |   19 (95%) |           8 (40%) |    11 (55%) |
+| B5 Prior-attempts       |    7 (35%) |           2 (10%) |     5 (25%) |
+| Either new rule, unique |  20 (100%) |           7 (35%) |    13 (65%) |
+
+The unique row uses the final disposition: seven plans had only statement-missing new-rule findings, while 13 had at least one substantive B8 or Prior-attempts finding. Per-rule columns overlap.
+
+The two statement-missing Prior-attempts bounces omitted the required table after referring to prior implementation work. Five plans received substantive Prior-attempts rejects; four were human-approved. That systematic disagreement is addressed by Proposal 3 below.
+
+## Decomposed and adjusted view
+
+### Era artifacts
+
+All cases predated one or both 2026-07-15/16 plan conventions. The dominant artifacts were:
+
+1. **Missing explicit sections:** 8/20 lacked the exact `Causal chain` heading without a demonstrated broken chain, and 2/20 established prior implementation work without a `Prior attempts` table. This is the same statement-vs-substance distinction recorded in the 2026-06-11 backtest.
+2. **Pre-ceiling 5/5 claims:** four plans claimed 5/5 from artifact correlation without inducing the blamed cause. Three were human-approved and one was human-rejected. The current B8 ceiling correctly treats this as substantive, but the historical authors did not have that convention.
+3. **Legacy rubric fields:** even after ignoring statement-missing B8 and Prior-attempts findings, the full rubric still rejected every case because the plans also predated B5 dedup, B6 observability, B7 current-main, or C6 sibling-repo statements.
+4. **Pre-chronic-routing approvals:** five plans carried substantive Prior-attempts findings under the current rule, including families with multiple prior stabilizations. Four were human-approved before the chronic route and mechanism-comparison convention existed.
+
+Two adjusted views make the assumptions explicit:
+
+- **Full rubric, ignoring only new-rule statement omissions:** 5/20 exact agreement (**25%**) plus one safe needs-human non-match. Legacy statement requirements and substantive findings still reject the other cases.
+- **New-rule attribution only, treating statement-only bounces as passes:** 7/20 agreement (**35%**). Thirteen substantive new-rule rejects remain; three match human rejects and ten reject human-approved plans.
+- **Also treating pre-rule 5/5 overclaims as an era artifact:** 10/20 agreement (**50%**). The remaining disagreements concentrate in unresolved harness/backend diagnoses and mechanism-insensitive Prior-attempts counting.
+
+These adjusted scores are diagnostic, not replacements for the 30% raw score.
+
+### Weak ground truth
+
+Five of six human-reject labels were not clear plan-quality judgments. They contributed five of the six raw matches:
+
+- one stale/not-pursued change;
+- two supersession-like closures;
+- two closures without a final public rejection rationale.
+
+They remain in the fixed cohort and raw matrix, but should not be used to claim the gate reproduced reviewer taste.
+
+### Substantive disagreement
+
+Ten human-approved plans received at least one substantive new-rule reject:
+
+- three deterministic threshold/collision/timing diagnoses claimed 5/5 without induced reproduction;
+- four local harness fixes named a remount, actionability, or classified-auth mechanism but left the deeper trigger partly inferred;
+- four plans were rejected under Prior-attempts because the family had multiple earlier stabilizations or the plan extended an existing mechanism;
+- some plans appear in more than one pattern.
+
+That is 10/20 of the corpus and 10/14 human-approved cases, so it exceeds the task's approximate 10% proposal threshold. The evidence is historical and pre-convention. Rocky reviewed the proposals on 2026-07-16: Proposals 1 and 2 were approved and applied; Proposal 3 was rejected and replaced with the deep-dive adjudication rule below.
+
+## Rubric amendment proposals
+
+### Proposal 1 — deterministic-boundary reproduction equivalence — approved
+
+Approved addition under B8's confidence ceiling:
+
+> **Deterministic-boundary equivalence:** for a pure clock, calendar, configuration, or literal-threshold defect, a failing artifact plus an exact evaluation of the named code expression at the captured inputs counts as the focused lower-level reproduction required for 5/5. The plan must show the before/after values and why the result is deterministic. Timestamp correlation or a plausible boundary narrative without that evaluation remains capped at 4/5.
+
+This covers deterministic non-E2E failures without treating production-style fault injection as the only form of reproduction. Rocky approved it because evaluating the named pure expression at captured inputs is the deterministic reproduction; the before/after-values requirement prevents narrative-only claims. Applied to B8's confidence ceiling.
+
+<!-- flaky-critic: amendment-decision status=approved proposal=P1 date=2026-07-16 -->
+
+### Proposal 2 — named harness-contract terminus — approved with live watch
+
+Approved addition under B8's valid terminal states:
+
+> **Harness-contract terminus:** for a C1/C2 test-harness change, the chain may terminate at a specific violated harness contract in named code when artifacts prove the user action or downstream request never began, and the fix is bounded, idempotent, and diagnostic. If a product, service, or identity-provider cause may still exist, cap confidence at 3/5 and require owning-surface observability or a linked handoff. A generic timeout, detach, 401, or retry success without the named contract and evidence remains a symptom, not a terminus.
+
+The audit's 45% frontend-local finding supports the premise that the causal chain can genuinely terminate at the harness boundary. Rocky approved the exception with the guardrails intact and directed the 14-day monitor to watch specifically for agents laundering generic detach, timeout, 401, or retry-success symptoms as a harness contract. Applied as B8's third valid terminal state.
+
+<!-- flaky-critic: amendment-decision status=approved proposal=P2 date=2026-07-16 -->
+
+### Proposal 3 — mechanism-aware Prior-attempts counting — rejected
+
+Rejected replacement for the substantive-reject and chronic-counting sentences in B5's Prior-attempts amendment:
+
+> **Mechanism-aware prior attempts:** count a merged fix as failed for the ≥2 chronic threshold only when recurrence evidence falsifies the same terminal-cause hypothesis or shows that the same mechanism did not hold. A recurrence in the same fingerprint family after a materially different fix remains required context but does not automatically increment the failed-fix count. Reusing or extending an earlier helper is not a substantive reject when the plan identifies a new, previously unhandled failure signature, shows that the earlier signature remains fixed, and states the exact mechanism delta. Repeating the same causal claim or broadening the same mechanism without new falsifying evidence remains a substantive reject.
+
+Rocky rejected this exemption because agent-self-certified mechanism novelty is the historical failure mode that B5 closes: each of the 12 fullLifecycle fixes could name a different step and error shape while remaining one mechanism. The family key is coarse, but the designed relief valve is the dossier-backed deep-dive track, not a counting exemption.
+
+Replacement applied to B5:
+
+> A claimed mechanism delta does not decrement the ≥2 failed-merged-fix count or return the family to the normal path; preserve the claim in the dossier and make mechanism identity the deep dive's first question.
+
+<!-- flaky-critic: amendment-decision status=rejected proposal=P3 replacement=deep-dive date=2026-07-16 -->
+
+## Borderline cases for human review
+
+- **Case 09 — approved clock-in boundary fix:** the boundary evidence was specific, but the plan lacked the newly required explicit causal-chain section.
+- **Case 13 — weak reject broad retry investigation:** Prior-attempts was statement-missing, while the supplied snapshot did not contain a classifiable implementation plan; without the new rule the result was needs-human.
+- **Case 15 — approved Cognito replacement:** the plan clearly superseded an earlier alias-lookup approach, making Prior-attempts applicable, but its replacement nature also makes the missing table an era artifact rather than evidence of repeated diagnosis.
+
+## Monitoring decision
+
+The historical statement-only amend-and-resubmit rate for the new rules was 7/20 (**35%**), above the ratified 25% threshold. It does **not** trigger a historical rubric change: the cases predate the conventions, and flaky-debug's current `references/plan.md` already requires both `Causal chain` and `Prior attempts`.
+
+Live enforce-mode monitoring is implemented in the flaky-critic skill for the 14-day window starting 2026-07-16 and ending before 2026-07-30:
+
+- denominator: unique gated ticket content-states;
+- numerator: amend-and-resubmit markers naming statement-missing B8 or B5 Prior-attempts;
+- headline plans are deduplicated when both rules fire, with per-rule attribution retained;
+- more than 25% at window close invokes `cb-work` for the narrowest flaky-debug plan-producing source, with a linked implementation ticket as fallback, never a weaker gate;
+- any systematic substantive false-reject pattern triggers a Rocky-approved rubric amendment proposal regardless of rate.
+- the named harness-contract terminus is a specific live-watch category: record every invocation, distinguish evidenced contracts from generic detach/timeout/401/retry-success laundering, and compare later human outcomes;
+- claim and action leases plus a pre-creation ownership recheck prevent overlapping finalizers from creating duplicate work;
+- the final digest and action links are persisted on STAFF-1818 with `<!-- flaky-critic: b8-prior-monitoring-finalized -->` so later runs do not repeat finalization.
+
+## What this establishes
+
+The run confirms that both new rules are mechanically checkable and that the statement-vs-substance split is essential. It also shows that the recent historical corpus is a poor raw agreement benchmark for the current full rubric: the plans predate several required fields, and most negative outcomes are weak labels.
+
+It does not establish that B8 or Prior-attempts should be weakened, nor does it validate the live bounce rate. The ratified two-week enforce-mode monitoring window is the relevant evidence for planner-template propagation and live substantive false rejects.
+
+## Case ledger
+
+| Case | Source                                                                           | Human   | Label  | Critic | B8          | Prior-attempts |
+| ---- | -------------------------------------------------------------------------------- | ------- | ------ | ------ | ----------- | -------------- |
+| 01   | [admin #7373](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7373)   | reject  | strong | reject | statement   | none           |
+| 02   | [backend #26392](https://github.com/ClipboardHealth/clipboard-health/pull/26392) | approve | strong | reject | substantive | none           |
+| 03   | [admin #7559](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7559)   | approve | strong | reject | statement   | substantive    |
+| 04   | [mobile #12521](https://github.com/ClipboardHealth/cbh-mobile-app/pull/12521)    | reject  | weak   | reject | substantive | substantive    |
+| 05   | [admin #7282](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7282)   | approve | strong | reject | substantive | none           |
+| 06   | [mobile #12635](https://github.com/ClipboardHealth/cbh-mobile-app/pull/12635)    | approve | strong | reject | statement   | none           |
+| 07   | [admin #6824](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/6824)   | reject  | weak   | reject | statement   | none           |
+| 08   | [admin #7390](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7390)   | approve | strong | reject | substantive | none           |
+| 09   | [mobile #12496](https://github.com/ClipboardHealth/cbh-mobile-app/pull/12496)    | approve | strong | reject | statement   | none           |
+| 10   | [admin #7380](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7380)   | reject  | weak   | reject | substantive | none           |
+| 11   | [admin #7336](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7336)   | approve | strong | reject | substantive | substantive    |
+| 12   | [mobile #12633](https://github.com/ClipboardHealth/cbh-mobile-app/pull/12633)    | approve | strong | reject | statement   | none           |
+| 13   | [admin #6974](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/6974)   | reject  | weak   | reject | none        | statement      |
+| 14   | [admin #7374](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7374)   | approve | strong | reject | substantive | none           |
+| 15   | [admin #7596](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7596)   | approve | strong | reject | statement   | statement      |
+| 16   | [mobile #12270](https://github.com/ClipboardHealth/cbh-mobile-app/pull/12270)    | reject  | weak   | reject | substantive | none           |
+| 17   | [admin #7250](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7250)   | approve | strong | reject | substantive | none           |
+| 18   | [admin #7370](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7370)   | approve | strong | reject | substantive | substantive    |
+| 19   | [admin #7320](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7320)   | approve | strong | reject | substantive | none           |
+| 20   | [admin #7386](https://github.com/ClipboardHealth/cbh-admin-frontend/pull/7386)   | approve | strong | reject | statement   | substantive    |

package/skills/flaky-critic/references/rubric.md

@@ -10,6 +10,8 @@ Findings from automated reviewers (Mendral, CodeRabbit, etc.) are **advisory inp
 
 **Statement-missing vs substance-missing (backtest amendment, 2026-06-11):** B5/B6/B7/B8 and C6 each require an explicit statement in the plan (dedup-check results, Prior attempts section when applicable, confidence score plus observability section, current-main status, causal-chain section, sibling-repo check when applicable). When the _statement_ is absent but the violation is not demonstrated — no actual duplicate found, no actual prior implementation tickets for the fingerprint family, no actual low-confidence code fix, no actual staleness, no demonstrably symptom-terminated diagnosis, no confirmed sibling-repo match — the verdict is reject with disposition **amend-and-resubmit**: one bounce asking for exactly the missing sections, never escalation. A demonstrated violation (a real duplicate, a mechanism already tried by a prior merged fix, a real ≤2/5 code fix, a diagnosis that demonstrably stops at a symptom, a confirmed sibling-repo match with no linked mirror implementation ticket deliverable) is a substantive reject. The 2026-06-11 backtest showed 6 of 7 false rejections were statement-missing on plans predating these conventions; live plans carry the sections by template, but the distinction stays load-bearing for the agreement metric.
 
+The [2026-07-16 B8 and Prior-attempts blind backtest](./backtests/2026-07-16-b8-prior-attempts.md) records the current-rule results, error decomposition, amendment proposals, and ratified live-monitoring handoff.
+
 ## 2. Banned patterns
 
 ### B1 — Naked retry (retry without failure classification)
@@ -56,6 +58,8 @@ Opening one plan/PR per failing test when the failures share one mechanism (same
 
 **Prior attempts (amendment, 2026-07-16):** when a fingerprint family has prior implementation tickets, the plan MUST contain a **Prior attempts** section listing each prior ticket/PR, what it blamed, what it changed, and the recurrence evidence showing its diagnosis was wrong or incomplete. A missing section is statement-missing and gets amend-and-resubmit per §1. A plan that re-proposes a mechanism a prior merged fix already tried is a substantive reject. A family with **≥2 failed merged fixes** — merged fixes followed by recurrence — is disqualified from the normal fix path: verdict = reject and escalate as needs-human. Apply the `chronic` label and route the family to the credential-checked, dossier-first `flaky-deep-dive` skill. The ≥2 threshold is intentionally different from flaky-triage's ≥3-prior-implementation-tickets screen: triage counts tickets cheaply before any diagnosis exists, while the critic can see merge and recurrence outcomes and applies the sharper failed-merged-fix test. Either stage flagging chronic routes the same way; the counts are staged by the evidence available at each stage, not a contradiction.
 
+A claimed mechanism delta does not decrement the ≥2 failed-merged-fix count or return the family to the normal path; preserve the claim in the dossier and make mechanism identity the deep dive's first question.
+
 **Knowledge-base cross-reference (amendment, 2026-07-16):** in shadow/enforce mode, consult the flaky-debug [root-cause knowledge-base index](../../flaky-debug/references/root-cause-kb/README.md) using the plan's symptom signature, then read each plausible entry's mechanism and **What failed and why** section. An entry that links a prior merged fix for the same mechanism to later recurrence is evidence for a substantive B5 reject under the Prior-attempts rule, even when the current plan omitted that history. Cite the entry and its recurrence evidence in the verdict. An index match on symptom text alone is not sufficient: if the causal mechanism differs or the entry has no failed merged attempt, continue evaluating the plan on its own evidence. In backtest mode, do not consult the live KB; use an entry only when the supplied historical snapshot includes the contemporaneous entry, so later recurrence evidence cannot leak into the historical verdict.
 
 ### B6 — Code fix on low-confidence diagnosis
@@ -82,13 +86,16 @@ Proposing a code fix from a diagnosis that stops at the failure symptom — "the
 
 **Why (amendment, 2026-07-07):** post-Done recurrence is the pipeline's dominant failure mode: in the Jun 2–Jul 6 window, 21 fingerprint families re-flaked after a Done closure, and the fullLifecycle family (33deef731a10) was closed Done six times. Each recurrence traces to a diagnosis that dead-ended at the repo boundary or at a status code and patched the symptom locally. Local-symptom plans are the cheapest to produce, so as long as the gate accepts them, agents keep producing them; this rule makes boundary-crossing the only way to pass.
 
-**Required structure:** the plan must contain a **Causal chain** section: failing assertion → app/UI state → network/trace/log evidence (trace ID, or run-window log queries when the causal event is outside the test's request path: seeding, deploys, CDC/async jobs) → owning service and repo, resolved via the groundtruth ownership registry (`ClipboardHealth/groundtruth`, `registry/services.json` → `registry/repos.json`) → terminal cause cited at file/line, config key, or specific log line. The link types adapt to the failure surface: CI/setup, fixture, component, and unit failures substitute build logs, fixture state, or runner artifacts for network evidence and the owning package/repo for a deployed service — the critic checks the terminus, not a fixed link shape. Two valid terminal states:
+**Required structure:** the plan must contain a **Causal chain** section: failing assertion → app/UI state → network/trace/log evidence (trace ID, or run-window log queries when the causal event is outside the test's request path: seeding, deploys, CDC/async jobs) → owning service and repo, resolved via the groundtruth ownership registry (`ClipboardHealth/groundtruth`, `registry/services.json` → `registry/repos.json`) → terminal cause cited at file/line, config key, or specific log line. The link types adapt to the failure surface: CI/setup, fixture, component, and unit failures substitute build logs, fixture state, or runner artifacts for network evidence and the owning package/repo for a deployed service — the critic checks the terminus, not a fixed link shape. Three valid terminal states:
 
 1. **Terminated:** the chain ends at a cause. A status code, timeout, or throttle is a link, never a terminus.
 2. **Explicitly broken:** the chain names the link where evidence ran out and the observability that would extend it; confidence is capped at 2/5 and B6 applies — the deliverable is the instrumentation, not a code fix.
+3. **Harness-contract terminus:** for a C1/C2 test-harness change, the chain may terminate at a specific violated harness contract in named code when artifacts prove the user action or downstream request never began, and the fix is bounded, idempotent, and diagnostic. If a product, service, or identity-provider cause may still exist, cap confidence at 3/5 and require owning-surface observability or a linked handoff. A generic timeout, detach, 401, or retry success without the named contract and evidence remains a symptom, not a terminus.
 
 **Confidence ceiling:** 5/5 additionally requires reproduction by inducing the blamed cause — fault injection in the harness (delay or fail the specific response, disable the seed step) or a focused lower-level test that deterministically reproduces the race. Without reproduction, the ceiling is 4/5. One inferred _intermediate_ link with an evidenced terminus is still a terminated chain (≤4/5); an unevidenced terminus is a broken chain (cap 2/5).
 
+**Deterministic-boundary equivalence (amendment, 2026-07-16):** for a pure clock, calendar, configuration, or literal-threshold defect, a failing artifact plus an exact evaluation of the named code expression at the captured inputs counts as the focused lower-level reproduction required for 5/5. The plan must show the before/after values and why the result is deterministic. Timestamp correlation or a plausible boundary narrative without that evaluation remains capped at 4/5.
+
 **Detection heuristic:** plan lacks a Causal chain section (statement-missing → amend-and-resubmit per §1); the terminal link is a symptom rather than a cause; the chain implicates another service or repo but cites no code/log evidence from it and doesn't route per C5/D5; or the plan claims 5/5 without reproduction evidence.
 
 ## 3. Allowed/safe patterns
@@ -215,7 +222,15 @@ All seven open questions from the mining pass were answered; the substantive one
 6. **Closed PRs #6824/#6572:** confirmed supersession, not approach rejections; C4's reading stands.
 7. **C1 collision retries:** form depends on identity type and error semantics — regenerate on constraint violations, re-attempt only on transient failures (now in C1).
 
-## 7. Appendix: corpus stats
+## 7. Resolved decisions (Rocky, 2026-07-16)
+
+The STAFF-1818 blind-backtest proposals were reviewed per the standing per-PR amendment policy:
+
+1. **Deterministic-boundary reproduction — approved:** exact evaluation of a named pure expression at captured inputs is deterministic reproduction when the plan shows before/after values.
+2. **Harness-contract terminus — approved with live monitoring:** the exception is limited by the artifact, confidence, and observability guardrails in B8. The 14-day monitor specifically watches for generic detach, timeout, 401, or retry-success symptoms being laundered as a harness contract.
+3. **Mechanism-aware Prior-attempts counting — rejected:** agent-claimed novelty does not reduce the failed-fix count. The ≥2 route remains family-level, and the dossier-backed deep dive adjudicates mechanism identity first.
+
+## 8. Appendix: corpus stats
 
 **Read:** 753 Linear issues enumerated (636 flaky-labeled; statuses: Done 431, Duplicate 156, Canceled 37, open 12); comments read on ~35 sampled issues (weighted to Canceled/canonical/impl tickets — most per-test tickets have only bot sighting comments). 88 `flaky-test-fix` PRs enumerated (74 merged, 13 closed, 1 open); bodies/diffs/review threads read on ~25; Rocky-authored review/issue comments harvested across all 88 (79 inline + 37 conversation comments, mostly cb-babysit); 4 skill-update PRs in core-utils; flake-intake pipeline templates in ClipboardHealth/clipboard.