@codyswann/lisa 4.74.9 → 4.75.1

package/plugins/lisa-expo-devin/skills/playwright-ci-debugging/SKILL.md

@@ -0,0 +1,140 @@
+---
+name: playwright-ci-debugging
+description: Debug Playwright E2E tests that pass locally but fail in CI (or vice versa) in Expo web projects. Covers local reproduction, network interception, CI environment discovery, commit SHA verification, and robust interaction patterns that eliminate flake. Use this skill when a Playwright test is failing in CI, a test is flaky, a PR is blocked by E2E checks, or you need to investigate CI-specific test behavior. Trigger on mentions of CI failure, failing Playwright test, flaky E2E test, or debugging E2E in CI.
+---
+
+# Debugging Playwright E2E Failures in CI
+
+The authoring-side rules (selectors, testID forwarding, naming) are in the `playwright-selectors` skill. This skill is for the other half of the job: when a test fails in CI and you need to find out why.
+
+## Debugging Order
+
+When a Playwright E2E test fails in CI, follow this exact order.
+
+### 1. Reproduce locally first — before anything else
+
+Start the project's dev server, then run the failing test in isolation:
+
+```bash
+# Start the dev server (discover the script from package.json — commonly `start`, `dev`, `start:dev`, or `web`)
+<pkg-manager> run <dev-script>
+
+# In another terminal, run the exact failing test with no retries
+BASE_URL=http://localhost:8081/ npx playwright test <file> --grep "<test name>" --retries=0
+```
+
+`--retries=0` is critical — retries mask flake. `--grep` isolates the single failing test so you're not waiting on a full suite.
+
+**Do NOT read source code, CI logs, or theorize until you can reproduce locally.** Most CI failures reproduce locally if you run the same test against the same served build. Guessing from logs is slow and usually wrong.
+
+### 2. Intercept the network request
+
+When a test involves an API call, set up a Playwright response listener and inspect the status code and response body. A 400/500 response tells you everything.
+
+```typescript
+page.on("response", async (response) => {
+  if (response.url().includes("/api/")) {
+    console.log(response.status(), response.url(), await response.text());
+  }
+});
+```
+
+UI failures are often downstream of a failed API call. The network log is cheaper evidence than the DOM.
+
+### 3. If it doesn't reproduce locally, understand the CI environment first
+
+CI runs against a different environment than your laptop. Before changing anything:
+
+- Find the CI job that runs Playwright (usually `.github/workflows/*.yml`).
+- Identify which `.env.*` file it loads — this varies by target branch (e.g., dev → `.env.development`, staging → `.env.staging`).
+- Note that CI typically builds a static web bundle (`expo export --platform web`) then serves it on `localhost:8081` via `serve dist`. It is NOT hitting your dev server. Timing, bundling, and env vars all differ.
+- Reproduce the CI setup locally: build the static bundle, serve it the same way, then run the test. If it now fails locally, you've isolated an env/build difference.
+
+### 4. Verify commit SHA before trusting CI results
+
+After pushing, confirm the CI run's `headSha` matches your latest commit:
+
+```bash
+gh run list --branch "$(git branch --show-current)" --limit 1 --json headSha,status,conclusion
+git rev-parse HEAD
+```
+
+Bots (review-response, auto-update, dependabot) can push commits between your push and the CI run, overwriting your changes. A green check on a stale SHA tells you nothing about your fix.
+
+## Patterns That Eliminate Flake
+
+### Never use fixed waits before interactions
+
+`waitForTimeout()` as the sole wait before a click is a silent failure waiting to happen — animations and rendering take variable time, especially on slower CI runners. Poll for the expected state, then act.
+
+```typescript
+// BAD — fixed wait; click silently fails if element not yet visible
+await clickVisibleText(page, "Translate");
+await page.waitForTimeout(1000);
+await clickVisibleText(page, "Spanish");
+
+// GOOD — poll for visibility, then click
+await clickVisibleText(page, "Translate");
+await expect
+  .poll(() => hasVisibleText(page, "Spanish"), { timeout: TIMEOUT.expect })
+  .toBe(true);
+await clickVisibleText(page, "Spanish");
+```
+
+### Never silently swallow click return values
+
+Any helper that can return `false` on a missed click (e.g., `clickVisibleText`) should either have its return value asserted or be preceded by a visibility poll. A click that silently returns `false` is a hidden test bug — the test proceeds as if the click happened and fails downstream with a confusing error.
+
+```typescript
+// BAD — return value ignored; test continues on failed click
+await clickVisibleText(page, "Submit");
+
+// GOOD — assert the click happened
+expect(await clickVisibleText(page, "Submit")).toBe(true);
+
+// GOOD — precede with visibility poll
+await expect
+  .poll(() => hasVisibleText(page, "Submit"), { timeout: TIMEOUT.expect })
+  .toBe(true);
+await clickVisibleText(page, "Submit");
+```
+
+### E2E tests must not depend on external API success
+
+Any test that calls an external service (AWS Bedrock, third-party APIs, rate-limited providers) must handle the failure case. Tests should verify UI behavior, not external service uptime. If an external call might fail, the test must accept both outcomes or skip the dependent assertion.
+
+```typescript
+// BAD — assumes translation always succeeds
+await expect
+  .poll(() => hasVisibleText(page, "Show Original"), { timeout: TIMEOUT.expect })
+  .toBe(true);
+
+// GOOD — handle both success and failure
+await expect
+  .poll(
+    async () =>
+      (await hasVisibleText(page, "Show Original")) ||
+      (await hasVisibleText(page, "Translate")),
+    { timeout: TIMEOUT.expect }
+  )
+  .toBe(true);
+
+const translated = await hasVisibleText(page, "Show Original");
+if (!translated) return; // External API failed; skip downstream assertions
+```
+
+The alternative — mocking the external call — is a valid approach when the goal is to test the UI's handling of a successful response. Pick one strategy per test and commit to it.
+
+### E2E assertions must not depend on specific test data
+
+Do not assert specific text (e.g., "No lists detected") when the test user's data state varies across environments. If a zero-row state could have different empty-state messages depending on the user's data, either check for multiple possible states or skip the assertion entirely.
+
+See the `playwright-selectors` skill's "Data independence" section for authoring patterns that avoid this class of bug in the first place.
+
+## Escalation
+
+If you've worked through steps 1–4 and still cannot explain the failure:
+
+- Do NOT disable, skip, or `.fixme` the test to unblock the PR.
+- Do NOT use `--admin` or force-merge to bypass the check.
+- Escalate to a human with: the failing test name, your local reproduction attempt, the network trace, and the commit SHA verification. A real flake that cannot be root-caused is a legitimate reason to pause; it is never a reason to merge red.

package/plugins/lisa-expo-devin/skills/playwright-selectors/SKILL.md

@@ -0,0 +1,496 @@
+---
+name: playwright-selectors
+description: Best practices for writing reliable Playwright E2E tests and adding testID/aria-label selectors in Expo web applications using GlueStack UI and NativeWind. Use this skill when creating, debugging, or modifying Playwright tests, adding E2E test coverage, creating components that need test selectors, reviewing code for testability, or troubleshooting testID/data-testid issues. Trigger on any mention of Playwright, E2E tests, end-to-end tests, testID, data-testid, or GlueStack testing in an Expo web context.
+---
+
+# Playwright E2E Testing for Expo + GlueStack UI
+
+## The #1 Rule: Browser First, Code Second
+
+Before writing ANY Playwright test, open the target page in a browser and manually walk through the flow. Never write tests blind from code reading alone.
+
+Expo/GlueStack apps have complex rendering pipelines — what you see in source code is not always what renders in the DOM. Components may have state-dependent behavior (a button that opens an actionsheet OR a confirm dialog depending on data), elements may live on different tabs than you expect, and testIDs may or may not forward to the web DOM depending on the component type.
+
+### Workflow for each new test
+
+1. **Navigate** to the page using Playwright MCP browser tools
+2. **Click through** the exact user flow the test will cover
+3. **Run** `document.querySelectorAll('[data-testid]')` to see which testIDs are actually in the DOM
+4. **Note** any conditional UI states, loading sequences, and tab navigation required
+5. **Then** write the test code matching exactly what you observed
+
+### Verify testIDs before scaling
+
+Before writing a batch of tests that depend on testIDs, verify ONE testID end-to-end:
+
+```javascript
+// Run this in the browser console or via Playwright MCP evaluate
+document.querySelectorAll('[data-testid]').length
+// Should return > 0 after page fully loads
+```
+
+Accessibility snapshots from Playwright MCP may NOT show `data-testid` attributes. Always use `document.querySelectorAll` for ground truth. Pages may show very few testIDs before full render (e.g., 3 elements on initial load vs 80+ after data loads) — wait for the page to settle before checking.
+
+---
+
+## Selector Strategy
+
+### Priority order
+
+1. **`getByTestId`** — most stable, survives copy changes and redesigns
+2. **`getByRole`** — good for interactive elements (`button`, `tab`, `switch`, `heading`)
+3. **`getByLabel`** — form elements with labels
+4. **`getByPlaceholder`** — inputs with placeholder text
+5. **`getByText`** — fragile, use only when no testID or role is available
+
+The generic web testing advice to prefer `getByRole` over `getByTestId` doesn't fully apply to React Native Web apps because ARIA role mapping is inconsistent across GlueStack components. testIDs are more reliable when properly set up.
+
+```typescript
+// 1. Preferred — testID
+await expect(page.getByTestId("settings:dark-mode-toggle")).toBeVisible();
+
+// 2. Good — role + accessible name
+await page.getByRole("button", { name: "Close dialog" }).click();
+
+// 3. Good — placeholder
+await page.getByPlaceholder("Search players...").fill("Messi");
+
+// 4. Fallback — text (fragile)
+await expect(page.getByText("Settings").first()).toBeVisible();
+```
+
+### Fallback pattern for migration periods
+
+When adding testIDs to components that are deployed separately from tests, use `.or()` to fall back gracefully:
+
+```typescript
+// Works before AND after testID is deployed
+const heading = page
+  .getByTestId("feature:heading")
+  .or(page.getByText("Feature Title").first());
+await expect(heading.first()).toBeVisible();
+```
+
+Remove the `.or()` fallback once the testID is confirmed deployed and working.
+
+---
+
+## testID Naming Convention
+
+Use a namespaced pattern with colons as separators: `screen:element`
+
+### Format
+
+```
+{screen}:{element}
+```
+
+- **screen**: lowercase screen/feature name (e.g., `home`, `profile`, `settings`)
+- **element**: lowercase element identifier (e.g., `container`, `title`, `submit-button`)
+
+### Examples
+
+| testID                      | Description                    |
+| --------------------------- | ------------------------------ |
+| `home:container`            | Main container on home screen  |
+| `home:title`                | Title text on home screen      |
+| `profile:avatar`            | User avatar on profile screen  |
+| `settings:dark-mode-toggle` | Dark mode toggle in settings   |
+| `auth:login-button`         | Login button on auth screen    |
+
+### Rules
+
+1. Use lowercase only
+2. Use colons (`:`) to separate screen from element
+3. Use hyphens (`-`) for multi-word elements
+4. Be descriptive but concise
+5. Avoid redundant words (e.g., `home:home-title` should be `home:title`)
+
+---
+
+## testID Forwarding: Which Components Support What
+
+This is the most critical technical knowledge for this stack. GlueStack UI components have **different** testID behavior depending on their render pipeline.
+
+### The render chain
+
+React Native Web converts `testID` → `data-testid` through its `createDOMProps` function. But this only happens for components that go through RN Web's `createElement` path. GlueStack wraps many components with NativeWind utilities (`withStyleContext`, `tva`) that bypass this path.
+
+### Rules by component type
+
+| Component | testID approach | Why |
+|-----------|----------------|-----|
+| `Pressable` (GlueStack) | `testID={value}` | Wraps RN `Pressable` → `View` → `createDOMProps` ✅ |
+| `View` (react-native) | `testID={value}` | Goes through `createDOMProps` ✅ |
+| `Text` (react-native) | `testID={value}` | Goes through `createDOMProps` ✅ |
+| `Text` (GlueStack `@/components/ui/text`) | `{...{"data-testid": value}}` | NativeWind wrapper renders `<span>`, bypasses `createDOMProps` |
+| `HStack` / `VStack` / `Box` (GlueStack) | `{...{"data-testid": value}}` | Same — NativeWind wrapper bypasses pipeline |
+| `Heading` (GlueStack) | `data-testid={value}` | Renders raw `<h1>`–`<h6>` HTML elements |
+| `Button` (GlueStack) | Unreliable — verify first | May or may not forward depending on version |
+| Third-party (e.g., BouncyCheckbox) | Usually not possible | Use text/role selectors instead |
+
+### How to tell which path a component uses
+
+Trace the component's render chain:
+
+```
+GlueStack Pressable → createPressable({ Root: withStyleContext(RNPressable) })
+  → withStyleContext passes {...props} to RNPressable
+  → RN Web Pressable renders <View {...rest}>
+  → View goes through createElement → createDOMProps
+  → createDOMProps converts testID → data-testid ✅
+```
+
+vs:
+
+```
+GlueStack Text → tva-styled component
+  → Renders <span> or <p> directly
+  → Never hits createDOMProps
+  → testID prop is silently ignored ❌
+```
+
+### Adding a testID to a new component
+
+1. Check the component type against the table above
+2. If it's a `Pressable` or RN `View`/`Text`, use `testID={value}` directly
+3. If it's GlueStack `Text`, `HStack`, `VStack`, `Box`, use `{...{"data-testid": value}}`
+4. If it's GlueStack `Heading`, use `data-testid={value}` as a JSX attribute
+5. **Build and verify** in the browser before writing tests against it
+
+```tsx
+// Pressable — testID prop works
+<Pressable testID="feature:action-button" onPress={handlePress}>
+  <Text>Click me</Text>
+</Pressable>
+
+// GlueStack Text — use data-testid spread
+<Text {...{"data-testid": "feature:section-heading"}}>
+  Section Title
+</Text>
+
+// GlueStack HStack — use data-testid spread
+<HStack {...{"data-testid": "feature:row"}} className="items-center">
+  <Icon as={Star} />
+  <Text>Rating</Text>
+</HStack>
+
+// GlueStack Heading — use data-testid attribute
+<Heading data-testid="feature:page-title" size="lg">
+  Page Title
+</Heading>
+```
+
+---
+
+## When to Add testID
+
+### Add testID To
+
+1. **Interactive elements** that E2E tests will click/interact with
+2. **Key structural containers** for page load verification
+3. **Dynamic content areas** that change based on state
+4. **Form elements** that lack semantic labels
+
+### Do Not Add testID To
+
+1. Every element (over-testing creates maintenance burden)
+2. Elements with good semantic selectors (use getByRole instead)
+3. Decorative elements not needed for testing
+4. Elements inside third-party components (may not propagate)
+
+---
+
+## Accessibility Best Practices
+
+Prefer semantic selectors and aria-labels over testID when possible. This benefits both testing and screen reader users.
+
+### aria-label for Testing and Accessibility
+
+```typescript
+// Correct — benefits both testing and accessibility
+<Pressable
+  accessibilityLabel="Close dialog"
+  onPress={handleClose}
+>
+  <XIcon />
+</Pressable>
+
+// E2E test uses accessible name
+await page.getByRole("button", { name: "Close dialog" }).click();
+```
+
+### accessibilityRole for Semantic Elements
+
+```typescript
+// Correct — semantic role for assistive technology
+<Box accessibilityRole="banner" testID="header:container">
+  <Text accessibilityRole="heading">Welcome</Text>
+</Box>
+
+// E2E test can use role
+await expect(page.getByRole("banner")).toBeVisible();
+await expect(page.getByRole("heading", { name: "Welcome" })).toBeVisible();
+```
+
+---
+
+## CI Architecture: Test the PR's Own Code
+
+Playwright must test against the code in the PR, not a remote deployed environment. If CI tests against a deployed app, new testIDs and component changes are invisible until deployed — creating a frustrating push-wait-fail cycle.
+
+### Expo web setup for CI
+
+The CI pipeline should:
+1. Build the web app: `npx expo export --platform web` (creates `dist/`)
+2. Serve it locally: `npx serve dist -l 8081 -s`
+3. Run Playwright against `http://localhost:8081/`
+
+### playwright.config.ts
+
+```typescript
+import { defineConfig } from "@playwright/test";
+
+export default defineConfig({
+  // In CI, serve the static web build locally
+  ...(process.env.CI
+    ? {
+        webServer: {
+          command: "npx serve dist -l 8081 -s",
+          port: 8081,
+          reuseExistingServer: false,
+        },
+      }
+    : {}),
+
+  use: {
+    baseURL: process.env.CI
+      ? "http://localhost:8081/"
+      : "https://dev.example.com/",
+  },
+});
+```
+
+> **Run local verification with `CI=1`.** In a config shaped like the one above,
+> `webServer` exists only when `CI` is set and `baseURL` otherwise points at the
+> **deployed** environment. So a plain `npx playwright test <spec>` does not test your
+> working tree at all — it tests whatever is deployed. A pass proves nothing about your
+> change, and a failure may be the deployed app failing rather than your code. Use
+> `CI=1 npx playwright test <spec>` whenever you are verifying uncommitted work.
+>
+> **Use a control spec before concluding anything from a red run.** If a spec fails
+> locally, re-run a **known-good sibling spec, unmodified, in the same mode**. If the
+> control fails too, that surface is not exercisable in your environment and the run
+> says nothing about your spec — don't "fix" a spec that was never broken. If the
+> control passes, the defect is genuinely yours.
+>
+> Also read reporter totals carefully: `setup` / auth projects count as passing tests.
+> "2 passed" can mean two auth setups and zero assertions.
+
+---
+
+## Writing Robust Tests
+
+### Data independence
+
+Never assert on data-dependent elements as required. The CI test user may have different data than your local environment.
+
+```typescript
+// BAD — fails if test user has no data
+const tableRows = page.locator("table tr");
+await expect(tableRows.first()).toBeVisible();
+expect(await tableRows.count()).toBeGreaterThan(1);
+
+// GOOD — handles empty state gracefully
+const tableRows = page.locator("table tbody tr");
+const rowCount = await tableRows.count();
+if (rowCount === 0) {
+  await expect(page.getByPlaceholder("Search...")).toBeVisible();
+  return;
+}
+await tableRows.first().click();
+```
+
+### Timeouts
+
+Use environment-aware timeouts from a shared constants file. CI runners are slower than local machines.
+
+```typescript
+export const TIMEOUT = {
+  test: isCI ? 90_000 : 60_000,
+  expect: isCI ? 30_000 : 15_000,
+  navigation: isCI ? 45_000 : 30_000,
+};
+
+// In tests — never hardcode
+await expect(element).toBeVisible({ timeout: TIMEOUT.navigation });
+```
+
+### Serial vs parallel mode
+
+Use serial mode for tests that mutate shared backend state. Read-only tests can run in parallel.
+
+```typescript
+test.describe("Feature with mutations", () => {
+  test.describe.configure({ mode: "serial" });
+});
+```
+
+### State-dependent UI
+
+Some UI elements behave differently depending on application state. Discover this during the browser-first step, then handle both cases:
+
+```typescript
+const addButton = page.getByTestId("feature:add-button").first();
+await expect(addButton).toBeVisible();
+
+const modal = page.getByText("Add to List");
+const isModalVisible = await modal.isVisible();
+
+if (isModalVisible) {
+  await page.getByText("Done").click();
+} else {
+  await expect(addButton).toBeVisible();
+}
+```
+
+---
+
+## Common Pitfalls
+
+### SonarCloud security hotspots
+
+These patterns trigger SonarCloud security hotspot warnings that block PR merges:
+
+```typescript
+// BAD — triggers security hotspot
+page.on("dialog", dialog => dialog.dismiss());
+const result = await element.waitFor().catch(() => false);
+
+// GOOD — use explicit checks instead
+const isVisible = await element.isVisible();
+const count = await elements.count();
+```
+
+### Page not fully rendered
+
+Always wait for a content-dependent element before asserting on testIDs:
+
+```typescript
+// BAD — may run before page renders
+const count = await page.evaluate(() =>
+  document.querySelectorAll('[data-testid]').length
+);
+
+// GOOD — wait for known element first
+await page.waitForLoadState("domcontentloaded");
+const item = page.getByTestId("feature:item").first();
+await item.waitFor({ state: "visible", timeout: 15000 });
+```
+
+### Third-party component testIDs
+
+Components from third-party libraries (e.g., `react-native-bouncy-checkbox`, `react-native-gifted-chat`) generally do NOT forward `testID` to the web DOM. Use text, role, or structural selectors for these.
+
+---
+
+## Reveal-on-Focus Assertions
+
+`toBeVisible()` proves the element exists and is not hidden — it does NOT
+prove the element crossed onto the screen. A focused element scrolled off-view
+still reports visible. For any "focus reveals the element" behavior (skip
+links, focus-trapped modals, scroll-into-view containers), assert on layout,
+not just visibility:
+
+```typescript
+// Rejects when the element has no box at all (display: none, detached).
+const box = await page.getByTestId("dialog:panel").boundingBox();
+expect(box).not.toBeNull();
+
+// Then prove it is actually inside the viewport — the part toBeVisible()
+// cannot say.
+await expect(page.getByTestId("dialog:panel")).toBeInViewport();
+```
+
+Pair them: `boundingBox()` answers "is there a box" (null means never laid
+out), `toBeInViewport()` answers "did it cross onto the screen". An assertion
+that skips the null check reads a missing layout as a viewport miss, which is
+a different bug.
+
+---
+
+## Implementation Checklist
+
+When adding E2E test coverage to a component:
+
+- [ ] Open the page in a browser and walk through the flow first
+- [ ] Run `document.querySelectorAll('[data-testid]')` to see existing testIDs
+- [ ] Identify elements that need selectors for testing
+- [ ] Check component type against the forwarding rules table
+- [ ] Use namespaced testID pattern (`screen:element`) for elements without semantics
+- [ ] Add accessibility labels where beneficial
+- [ ] Verify testID propagates to `data-testid` on web before writing tests
+- [ ] Use environment-aware timeouts, not hardcoded values
+- [ ] Handle empty data states gracefully
+- [ ] Document testIDs in component JSDoc preamble
+
+## Example Component
+
+```typescript
+/**
+ * Profile screen component.
+ *
+ * Test IDs for E2E testing:
+ * - `profile:container` - Main container
+ * - `profile:avatar` - User avatar image
+ *
+ * @module features/profile/screens/Main
+ */
+export const ProfileScreen = () => (
+  <Box testID="profile:container" className="flex-1 p-4">
+    <Image
+      testID="profile:avatar"
+      source={{ uri: user.avatarUrl }}
+      accessibilityLabel={`${user.name}'s profile photo`}
+    />
+    <Text accessibilityRole="heading">
+      {user.name}
+    </Text>
+    <Pressable
+      accessibilityLabel="Edit profile"
+      onPress={handleEdit}
+    >
+      <Text>Edit</Text>
+    </Pressable>
+  </Box>
+);
+```
+
+## Corresponding E2E Test
+
+```typescript
+test.describe("Profile Screen", () => {
+  test.use({ viewport: VIEWPORT.desktop });
+
+  test.beforeEach(async ({ auth }) => {
+    await auth.login();
+  });
+
+  test("displays user information", async ({ page }) => {
+    await page.goto("/profile");
+    await page.waitForLoadState("domcontentloaded");
+
+    // Verify structural container
+    await expect(page.getByTestId("profile:container")).toBeVisible();
+
+    // Prefer accessible queries when available
+    await expect(page.getByRole("heading")).toHaveText("John Doe");
+    await expect(
+      page.getByRole("button", { name: "Edit profile" })
+    ).toBeVisible();
+
+    // Use testID for elements without semantic roles
+    await expect(page.getByTestId("profile:avatar")).toBeVisible();
+  });
+});
+```