testeiya 0.3.9 → 0.3.10

package/skills/codeceptjs/codeceptjs-auth/SKILL.md

@@ -0,0 +1,141 @@
+---
+name: codeceptjs-auth
+description: >
+  Use when a CodeceptJS test needs login, user roles, or session reuse — or
+  when another CodeceptJS skill identifies authorization is required. Also on
+  auth-related test failures. Trigger on login, sign-in, sign-up,
+  authentication, sessions, "logged in", admin/editor/user roles.
+---
+
+# CodeceptJS Auth Plugin
+
+The `auth` plugin logs each user in once, captures cookies (or localStorage/token via overrides), and restores the session for subsequent tests. Stale sessions trigger a fresh login automatically.
+
+If the project already has `auth` configured (fundamentals' discovery output), **reuse** its existing inject name and user keys — don't reconfigure.
+
+## Decide first — ask the user
+
+Four answers shape the plugin. Don't guess any.
+
+1. **Is a session needed at all?** Public flows (landing, signup) → skip the plugin.
+2. **One user or many?** Default one; add more only when actually exercised.
+3. **If many — what splits them?** Role, workspace/tenant, plan tier, sign-in provider, per-test fixture — real systems vary. Ask; use the answer to name `users.<key>` entries.
+4. **What's the auth type?**
+   - **Form** — default; canonical shape below
+   - **OAuth / SSO** — click provider button, drive the IdP page (often separate origin)
+   - **Magic link / passwordless** — UI flow rarely worth automating; prefer an API mint or reading the link from a test mailbox
+   - **API token** — skip the form; write the token into `localStorage` via `executeScript`, or `I.setCookie(...)`
+   - **2FA / OTP** — async `login`; fetch the code from a test mailbox / backdoor before submitting
+
+## Rules
+
+1. **Login flow never lives in the conf.** Put it in `steps_file.js` (if included) or a page object; conf only references it: `login: (I) => I.login()`.
+2. **Credentials from env only** — `.env` loaded via `process.loadEnvFile()` (no dotenv dependency). Passwords wrapped with `secret(...)`. No literals anywhere — conf, steps file, test, git history.
+3. **`.env` is gitignored; `.env.example` is committed** with names, no values. Gitignore `output/*_session.json` too.
+
+## Canonical shape
+
+```js
+// codecept.conf.js — first line of the file
+process.loadEnvFile()
+
+export const config = {
+  include: { I: './steps_file.js' },
+  plugins: {
+    auth: {
+      enabled: true,
+      saveToFile: true,
+      users: {
+        admin: {
+          login: (I) => I.login(),
+          check: (I) => I.see('Welcome, User', '.navbar'),
+        },
+      },
+    },
+  },
+}
+```
+
+```js
+// steps_file.js
+import { secret } from 'codeceptjs'
+const { I } = inject()
+
+export default function () {
+  return actor({
+    login() {
+      I.amOnPage('/login')
+      I.fillField('Email', process.env.ADMIN_EMAIL)
+      I.fillField('Password', secret(process.env.ADMIN_PASSWORD))
+      I.click('Sign in')
+    },
+  })
+}
+```
+
+```sh
+# .env (gitignored)        # .env.example (committed)
+USER_EMAIL=...             USER_EMAIL=
+USER_PASSWORD=<secret>     USER_PASSWORD=
+```
+
+## Pre-flight (before writing config)
+
+1. **Read the real login page** — MCP `run_code` to `/login`, inspect the ARIA snapshot (`codeceptjs-exploration`). Field labels / `name` / `id` / submit control from the actual page, not guesses. Unclear authorization mechanism → ask the user.
+2. **Pick a role-specific post-login marker** — something rendered only for *this* user (navbar username, `data-user-role`).
+3. **Confirm session storage** — cookies (default) for server-rendered apps; localStorage/sessionStorage for SPAs. Verify after a manual login with `I.executeScript(() => Object.keys(localStorage))`. Cookie fetch/restore silently no-op against token storage.
+
+## Verify
+
+Run a one-Scenario file that calls `login(<role>)` then asserts on the post-login marker:
+
+```bash
+npx codeceptjs run --grep '<scenario>' --debug    # real run, not dry — dry-run doesn't init plugins
+```
+
+Enable `saveToFile: true` only after this round-trip succeeds — a bad saved session masks a broken `login`.
+
+Then wire into hooks/tests: `Before(({ login }) => login())` for suite-wide, or per-test as needed.
+
+## Multi-role shape
+
+Only after question 3 is answered. Keys named after whatever splits users *in this system*; one matching actor method per key:
+
+```js
+users: {
+  admin:      { login: (I) => I.loginAsAdmin() },
+  workspaceB: { login: (I) => I.loginToWorkspaceB() },
+}
+```
+
+Don't parameterise into a single `login(key)` — the plugin keys sessions by name, explicit methods read better. Switch mid-Scenario: `session('<key>')` opens a parallel browser context (fundamentals § Writing tests).
+
+## Token / localStorage auth
+
+Override `fetch` / `restore` when sessions live outside cookies:
+
+```js
+admin: {
+  login: (I) => I.loginAsAdmin(),
+  check: (I) => I.see('Admin', '.navbar'),
+  fetch: (I) => I.executeScript(() => localStorage.getItem('session_id')),
+  restore: (I, session) => {
+    I.amOnPage('/')
+    I.executeScript((s) => localStorage.setItem('session_id', s), session)
+  },
+}
+```
+
+`check(I, session)` receives whatever `fetch` returned — throw inside `check` to force fresh login (e.g. `/me` endpoint shows wrong user).
+
+## Pitfalls
+
+- Credentials inlined in conf/test — always env-driven + `secret()`.
+- Forgetting to gitignore `.env` and `output/*_session.json` — both leak credentials.
+
+## Related skills
+
+- `codeceptjs-fundamentals` — secrets rule, sessions, config mutation trap
+- `codeceptjs-exploration` — reading the live login page
+- `writing-codeceptjs-tests` / `refactoring-codeceptjs-tests` — invoke this skill when auth is identified
+- `debugging-codeceptjs-tests` — auth-related failure patterns

package/skills/codeceptjs/codeceptjs-exploration/SKILL.md

@@ -0,0 +1,100 @@
+---
+name: codeceptjs-exploration
+description: >
+  Use when an agent needs to learn what's on a page in CodeceptJS — read the
+  ARIA tree, inspect candidate elements, pick or disambiguate a stable locator.
+  Drives the live browser via MCP `run_code` / `snapshot`. Invoked by
+  writing-codeceptjs-tests, debugging-codeceptjs-tests, and
+  refactoring-codeceptjs-tests whenever page inspection is needed.
+---
+
+# CodeceptJS Page Exploration
+
+Authoring a test, debugging a failure, and refactoring a stale locator share one task: open a page, find the right element, pick a stable locator. This is that playbook.
+
+## Tools
+
+- **`run_code`** — runs CodeceptJS code, returns produced values, captures `console.*`, saves a final-state snapshot. For *do something and look at the result*.
+- **`snapshot`** — captures state without acting (URL, cookies, localStorage, HTML, ARIA, screenshot, console). For "what's on the page right now".
+
+Artifact sources, in preference order:
+
+1. **ARIA snapshot** — structured, no styling noise, easy duplicate/accessibility-name scanning
+2. **Screenshot** — visual confirmation; catches layout breaks ARIA can't show
+3. **HTML** — only when ARIA lacks context (custom widgets without accessible names, attribute-driven behaviour)
+
+## Inspect an element
+
+`I.grabWebElement(locator)` → one WebElement; `I.grabWebElements(locator)` → array. Same cross-helper API on Playwright / Puppeteer / WebDriver.
+
+| You want to … | Method |
+|---|---|
+| Confirm rendered / visible / enabled | `exists()`, `isVisible()`, `isEnabled()` |
+| Read text / value / attribute / property | `getText()`, `getValue()`, `getAttribute(n)`, `getProperty(n)` |
+| Position on page | `getBoundingBox()` — flags offscreen / zero-sized |
+| Rendered markup | `toOuterHTML()`, `toSimplifiedHTML(300)` (truncated, MCP-friendly) |
+| Stable selector for a fix | `toAbsoluteXPath()` |
+| Inside an iframe | `inIframe(async (body) => { ... })` |
+| Drill into children | `$(loc)`, `$$(loc)` |
+| Browser-side function | `evaluate(fn, ...args)` |
+
+## Discover candidates when the obvious locator misses
+
+When `Edit` matches nothing, the control may say "Change", carry `aria-label="Edit user"`, or live in `.btn-edit`. Cast a wide net with a permissive XPath via `I.grabWebElements`, then disambiguate.
+
+OR together in the XPath:
+
+- visible text — `text()` (or `.` for descendants)
+- attributes — `@class`, `@aria-label`, `@title`, `@data-action`, `@id`
+- **synonyms** — edit/change/modify; delete/remove/trash; submit/send/save
+
+Case-insensitive via `translate(...)`:
+
+```
+//*[contains(translate(., 'EDIT', 'edit'), 'edit')
+    or contains(translate(@class, 'EDIT', 'edit'), 'edit')
+    or contains(translate(@aria-label, 'EDIT', 'edit'), 'edit')
+    or contains(translate(., 'CHANGE', 'change'), 'change')]
+```
+
+Then iterate `toSimplifiedHTML(150)` over the results, pick the right candidate, commit a stable locator from its discriminating attribute or text.
+
+## Pick a stable locator
+
+Two decisions in order: **which region scopes the lookup** (context), **what identifies the element inside it**. Region first keeps the identifier short and semantic — the discriminator found during disambiguation belongs in the context argument:
+
+```js
+I.click('Edit user', '.user-row')      // ✅ region + what the user sees
+I.click('#user-row-42 button.edit')    // ❌ same element, brittle, unreadable
+```
+
+Stable regions: landmarks (`nav`, `main`, `{ role: 'dialog' }`), app-shell containers (`.sidebar`, `.toolbar`, `.modal`), rows/cards identified by data via `locate(...)`. Identifier priority (full rationale: `codeceptjs-fundamentals` § Locators):
+
+1. Visible label / accessible name — plain string already matches `aria-label`; don't expand to `{ css: '[aria-label="..."]' }`
+2. ARIA role when ambiguous within context or role is part of the check
+3. `$name` via `customLocator` when team test attributes exist
+4. Composed CSS, still scoped: `I.click('button.edit', '#user-row-42')`
+5. `toAbsoluteXPath()` — last resort; flag the team to add a `data-testid`
+
+**Never commit an unverified locator** — confirm via `run_code` (`I.seeElement(loc, context)` or `grabWebElement(loc)`) that it matches exactly one element.
+
+## Common patterns
+
+- Strict mode 2+ matches → `grabWebElements('Save')` + `toSimplifiedHTML(200)` each, find discriminator, pass as context: `I.click('Save', '.modal')`
+- Button rendered but doesn't act → `grabWebElement('Submit')` + `isEnabled()` + `getBoundingBox()` — disabled? offscreen? zero-sized?
+- Wrong row in a list → `grabWebElements('.user-row')`, `getText()` per row to identify, `getAttribute('data-id')` for stable hook
+- Inside iframe → `(await I.grabWebElement('iframe.editor')).inIframe(async (body) => body.$('button'))`
+
+## Things to avoid
+
+- Choosing a locator without seeing candidates first.
+- Committing `toAbsoluteXPath()` when a semantic locator is available.
+- Committing unscoped locators where a context keeps them short.
+- Ignoring the screenshot — "exists in HTML" ≠ "user can see it".
+- `usePlaywrightTo` / `useWebDriverTo` when WebElement methods cover it.
+
+## Related skills
+
+- `codeceptjs-fundamentals` — locator priority, await rule
+- `writing-codeceptjs-tests` — invokes this during Mode B exploration
+- `debugging-codeceptjs-tests` — invokes this for live inspection; offline variant via `codeceptq`

package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md

@@ -0,0 +1,196 @@
+---
+name: codeceptjs-fundamentals
+description: >
+  Run first when working with any CodeceptJS 4 project — before writing,
+  debugging, refactoring, or migrating tests. Teaches the framework's
+  non-obvious rules and runs four-step discovery (`check` → read config →
+  `list` → `dry-run`) reporting which helpers, plugins, page objects, custom
+  actions, and tests are active. Other CodeceptJS skills depend on this output.
+---
+
+# CodeceptJS Fundamentals
+
+Two jobs, in order: learn the rules below, then discover what *this* project has configured.
+
+## Gate
+
+- CodeceptJS 4 is **ESM/TypeScript only**; tests, configs, page objects, helpers use `import`/`export`.
+- No `"type": "module"` in package.json → add it before anything else.
+- TypeScript: config `codecept.conf.ts`, TS loader entry in `require: [...]`.
+- Project on 3.x or CommonJS (`require()`, removed plugins like `autoLogin`, helper `Nightmare`) → stop, recommend `migrate-codeceptjs-4`. Don't patch files piecemeal — migration is whole-project.
+
+## Main rule
+
+- Tests are written from the user's perspective: a linear scenario of actions, readable as prose.
+  - Good: `I.click('Login')`, `I.fillField('Email', ...)`, `I.see('Welcome')`
+- **Tests are declarative, helpers imperative — recommended layering:**
+  - Scenario shows *what* the user does via `I.*`; implementation details live below
+  - Low-level access (`this.helpers['Playwright'].page`, fetch, filesystem) works fine inside a Scenario, but it's recommended to push it into a helper and expose one `I.*` action instead
+- Keep tests short: repeated sequences → actor method, page object, or step object.
+- Prefer semantic locators over selectors so tests survive markup churn.
+
+## Where things go (recommended placement)
+
+- Site-wide actions (`login`, dropdowns, rich text editors) → **actor file** (custom steps)
+- Page/screen actions + locators → **page object**; SPA screen = one page object
+- Site-wide widgets (nav, modals, datepickers) → **page fragments / component objects**
+- Low-level driver access (DB connections, email, filesystem, complex mouse) → **helper**
+- Data creation/cleanup via API → **data objects** (REST/GraphQL helper + `_after()` cleanup), or `ApiDataFactory` (`I.have(...)`)
+- Don't overengineer: no page object until an abstraction is reused across tests.
+
+## Shortcuts
+
+- Login needed → `autoLogin` plugin or actor method, not inline steps per test
+- Test data → create via API before the test, not through the UI
+- Long test → break into several; long tests are fragile and hard to follow
+- Optional UI element / conditional flow → `await tryTo(...)` instead of `if (await I.grab...)` — keeps scenarios linear
+
+## Architecture
+
+- **Config**: `codecept.conf.{js,ts,mjs,cjs}` at repo root; multiple files selected via `--config <file>`.
+- **Helpers execute; `I` delegates.** Every `I.<method>` is routed to whichever active helper implements it (Playwright, WebDriver, Puppeteer, Appium share one API surface). Active helpers = keys under `helpers`. Tests call the actor, never the engine — backends stay swappable.
+- **DI container** — why it exists:
+  - Everything shared (actor, helpers, page objects, support objects) registers under one container; `include` maps names → modules
+  - Classes are auto-instantiated by the container — no `new`, no manual wiring
+  - **`inject()` returns lazy proxies**: destructuring at module top resolves at call time, so circular page-object references work where plain `import` would give `undefined`
+  - Access: destructure in Scenario signature (`Scenario('...', ({ I, loginPage }) => ...)`) or `const { I } = inject()` once per file
+- **Custom helpers** extend `Helper`, register under `helpers`, add new `I.*` methods.
+  - **`I` does not exist inside a helper.** Compose via `this.helpers['<HelperName>']` (e.g. `this.helpers['Playwright'].page`, `this.helpers['REST'].sendGetRequest(...)`).
+- **Plugins** are event listeners on lifecycle events (`suite.*`, `test.*`, `step.*`, `hook.*`, `multiple.*`). Full list: `node_modules/codeceptjs/lib/event.js`. Register under `plugins` with `enabled: true`.
+
+## Config mutation trap
+
+- `@codeceptjs/configure` mutates resolved config at load time (`setHeadlessWhen`, `setBrowser`, ...). Static values can lie — grep for its import before trusting `show:` / `browser:` fields.
+- `setCommonPlugins()`: **enables** `retryFailedStep` + `screenshot`; **registers** (off until `-p`) `pause`, `browser`, `aiTrace`, `heal`.
+
+## Plugins worth knowing
+
+- `retryFailedStep` — retries transient step failures
+- `screenshot` — screenshots on failure; `slides: true` → `output/records.html` slideshow
+- `pageInfo` — dumps URL/HTML/console on failure
+- `auth` — session reuse for login (see `codeceptjs-auth` skill)
+- `aiTrace` — per-step screenshots/HTML/ARIA/console for AI debugging
+- `pause` — interactive pause
+- `heal` — AI-suggested fixes for broken steps (off in `--debug`)
+- `screencast` — video of the run
+- `customLocator` — maps `$name` prefix to team's test attribute (`data-testid`, `data-qa`)
+- `browser` — CLI-only override of browser helper config (see below)
+
+Note: `tryTo`, `retryTo`, `eachElement` are not plugins in 4.x — import them from `codeceptjs/effects`.
+
+## Plugins from CLI
+
+Any plugin can be enabled/reconfigured per-run with `-p <plugin>`, args chained with `:`:
+
+```sh
+npx codeceptjs run -p aiTrace                    # enable for this run
+npx codeceptjs run -p screenshot:on=step         # reconfigure inline
+npx codeceptjs run -p pause:on=file:path=tests/login_test.js;line=43
+```
+
+- `screenshot`, `pause`, `aiTrace`, `heal` share an `on=` trigger: `fail` (default except aiTrace) | `step` | `test` | `file:path=...;line=N` | `url:pattern=<glob>`
+- `browser` plugin overrides without touching config — CI matrix legs, one-off env variants:
+  - `-p browser:hide` / `-p browser:show` / `-p browser:browser=firefox` / `-p browser:windowSize=1280x800`
+  - Requires `@codeceptjs/configure`
+
+## Effects (`codeceptjs/effects`)
+
+Flow-control functions imported from `codeceptjs/effects`. In 4.x these are no longer plugins/globals.
+
+- **`tryTo(() => ...)`** — runs steps that may fail without stopping the test; returns `boolean`.
+  - **Prefer `tryTo` over `if`:** scenarios should stay linear — instead of branching on a grabbed value to decide whether a UI state exists, attempt the optional steps and branch on the boolean result:
+    ```js
+    const banner = await tryTo(() => { I.see('Cookie banner'); I.click('Accept cookies') })
+    if (!banner) I.say('No cookie banner')
+    ```
+  - Auto-retries are disabled inside `tryTo` blocks.
+- **`retryTo(() => ..., maxTries, pollInterval = 200)`** — retries a step block until it succeeds (flaky elements, animations); callback receives the current attempt count.
+- **`hopeThat(() => ...)`** — soft assertions (see Assertions); end with `hopeThat.noErrors()`.
+- **`within(locator | { frame }, fn)`** — scopes resolution to subtree or iframe; can return values (`await`). Prefer the context parameter of individual actions (`I.click('Save', '.toolbar')`) when possible — reserve `within` for genuinely scoped blocks.
+
+All effects return Promises — `await` them.
+
+## Element-based API (`codeceptjs/els`)
+
+Hybrid style: mix `I.*` with direct element access. Import `{ element, eachElement, expectElement, expectAnyElement, expectAllElements } from 'codeceptjs/els'`.
+
+- `element(locator, async el => { ... })` — scoped access to one element; chain `el.$(locator)` into children without re-querying
+- `eachElement(locator, async (el, index) => ...)` — iterate collections
+- `expectElement` / `expectAnyElement` / `expectAllElements(locator, fn)` — custom conditions
+- Elements are `WebElement` wrappers — same API on all helpers: `getText()`, `getAttribute()`, `isVisible()`, `isEnabled()`, `getBoundingBox()`, `exists()`, `$$()`
+- Optional purpose string improves debug logs: `element('verify discount applied', '.price', ...)`
+- Use when built-ins don't cover it: collections, layout checks (`getBoundingBox`), per-element loops, chaining ops on one element. Prefer `I.*` for readability otherwise.
+
+## Writing tests
+
+- Structure: one `Feature(...)` per file, one or more `Scenario(...)` inside. No nested suites, no multiple Features per file.
+- Hooks: `Before`, `After`, `BeforeSuite`, `AfterSuite`, `Fail(...)`.
+- Page object lifecycle hooks: `_before()` (lazy, once per test, on first use), `_after()` (skipped if unused), `_beforeSuite()`, `_afterSuite()`.
+- **`await` required for**: `grab*` methods, imported functions, page-object methods containing async ops (elsewhere: unhandled rejections). Never for plain action steps — the recorder chains them.
+- Secrets: `I.fillField('Password', secret(process.env.PASSWORD))` — masks logs, traces, AI prompts. Import from `codeceptjs`.
+- Sessions: `session(name, fn)` — parallel browser context for multi-user Scenarios (chat, multi-tenant).
+
+## Locators
+
+- **ARIA locators are strongest** — resilient to CSS refactors, describe what the user sees:
+  - `I.click({ role: 'button', name: 'Save' })`
+- Actions accept plain strings (visible text, label, placeholder, `name`, `aria-label`) or objects (`{ css }`, `{ xpath }`, `{ id }`).
+- Plain string already matches `aria-label` — no `'aria-label=...'` prefix needed.
+- **Pass context as last argument** — scoped semantic locator beats long unscoped one:
+  - `I.click('Save', '.toolbar')` not `I.click('#toolbar .btn-save')`
+- Avoid style-based class names (`.bg-green`); prefer semantic ones (`.btn-save`).
+- `data-testid`/`data-qa` apps → enable `customLocator`, write `$name`.
+- No semantic name fits → `locate(...)` builder (`.withClass`, `.withText`, `.inside`, `.and`): `locate('.button').withText('Click me')`.
+
+## Waiting
+
+- Action steps auto-wait for existence + interactability. Add explicit `waitFor*` only when the condition isn't tied to an interaction (modal after network call, spinner hiding).
+- Avoid `I.wait(N)` — last resort.
+
+## Assertions
+
+Built-in browser assertions come first: `I.see`, `I.seeTextEquals`, `I.seeElement`, `I.seeInField`, `I.seeNumberOfElements`, `I.seeInCurrentUrl` (+ `dontSee*` counterparts). Clear failures, recorder-integrated. `see` matches *visible* text; hidden DOM content needs `seeInSource` / `seeElementInDOM`.
+
+For what built-ins don't cover, in order of preference:
+
+1. **Reusable custom assertion** in a helper — `I.seeTableIsOrdered('Price', 'desc')`; name positives `see*`, negatives `dontSee*`; use `codeceptjs/assertions` inside, never raw `throw new Error()`
+2. **ExpectHelper** (`@codeceptjs/expect-helper`) — chai matchers on `I`: `I.expectEqual`, `I.expectDeepEqualExcluding`, `I.expectMatchesPattern`, `I.expectJsonSchema`; appears in step log like other steps
+3. **`codeceptjs/assertions`** directly — dependency-free factories: `equals(subject).assert(actual, expected)` / `.negate(...)`; failure messages match `I.see` formatting
+4. **Any library** on grabbed data (`grab*` always needs `await`) — chai/jest/`node:assert`; fails the test but won't show as a step
+
+Soft assertions: `hopeThat(() => I.see(...))` from `codeceptjs/effects` — logs each failure and continues; end with `hopeThat.noErrors()` to fail if any were recorded.
+
+## Parallel runs
+
+- `run-workers <N>` — splits Scenarios across worker threads
+- `run-multiple <profile>` — profiles via `multiple` block in config (browsers, viewports)
+
+## Config organization (recommended)
+
+- Multiple config files per environment (`codecept.conf.js`, `codecept.ci.conf.js`, ...); share parts via modules in a `config/` dir
+- `.env` files + `dotenv` for secrets/env-specific values
+- Bulk-register page objects/components by spreading exported maps into `include`
+- Pass data from config/bootstrap into tests via `codeceptjs.container.append({ testUser })` — injectable by name
+
+## Discover this project
+
+In order; skipping steps produces wrong guesses:
+
+1. **Verify setup loads**: `npx codeceptjs check -c <config>` — validates everything; output doubles as inventory. Fix failures before continuing.
+2. **Read the active config**: helpers (+ browser/baseURL/viewport/env-driven values), plugins (incl. anything `setCommonPlugins()` injects), AI provider + required env var, env selection mechanism, page objects from `include`, custom helpers.
+3. **List actions**: `npx codeceptjs list -c <config>` (`--docs` adds JSDoc; `--action <name>` for one). The actual `I.*` surface differs from built-ins when custom helpers exist — always check before suggesting a method.
+4. **List tests**: `npx codeceptjs dry-run -c <config>` — `--steps` shows queued actions, `--grep` filters, `--numbers` gives per-test step indices matching MCP `pauseAt`.
+   - ⚠ `dry-run --grep` and `run --grep` do **not** select the same set (4.1.0): `run --grep` matches `Feature` + `Scenario`, `dry-run --grep` matches the Scenario title only. `dry-run --grep 'Dialogs'` lists 0 tests where `run --grep 'Dialogs'` executes all 11. Never size a run from a dry-run's grep, and target a whole Feature by file path (`run tests/foo_test.ts`) when the selection must be exact.
+
+Gherkin projects: `npx codeceptjs gherkin:steps -c <config>`.
+
+Reference docs live under `node_modules/codeceptjs/docs/` — read them instead of guessing APIs.
+
+## Report
+
+Short prose summary. Must include:
+
+- Env-driven values flagged as env-driven (`process.env.BROWSER || 'chromium'`, not just `'chromium'`)
+- Conflicts flagged (static `show: true` vs `setHeadlessWhen(CI)`; `auth` configured but credential env vars missing)
+- No config at root and no `--config` referenced → recommend `npx codeceptjs init .`, stop
+- 3.x/CommonJS detected → recommend `migrate-codeceptjs-4`, stop

package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: codeceptjs-run-analysis
+description: >
+  Use after running CodeceptJS tests with the `aiTrace` plugin enabled and the
+  results need review — verify a fix held, investigate a single failure, cluster
+  errors across a CI fail-storm, diagnose flakiness across reruns. Invoked by
+  other CodeceptJS skills whenever a run has happened. Trigger on "what failed",
+  "analyse the run", "cluster these errors", "is it flaky", "did the fix hold".
+---
+
+# CodeceptJS Run Analysis
+
+After `npx codeceptjs run`, trace artifacts land in `output/`. This skill reads them efficiently — right step, right file, right slice of a giant HTML snapshot — via bash tools rather than re-running through MCP.
+
+No single end goal: pick the use case matching the situation. The foundations apply to all of them.
+
+## Foundations
+
+- **aiTrace must be on.** Everything leans on `output/trace_<TestName>_<hash>/trace.md`. Confirm in the active config (run `codeceptjs-fundamentals` if unknown). Without it there are only screenshots and `pageInfo` dumps — suggest enabling and re-running before deep analysis.
+- `run_step_by_step` is interactive only; `aiTrace` is the sole source of per-step files. Ad-hoc `run_code` / `snapshot` still produce single-shot bundles under `output/trace_run_code_*` / `output/snapshot_*`.
+- **Locate traces**: reruns create new dirs — when unclear, most recent wins (`ls -dt output/trace_*`).
+- **Read trace.md first** — it's the index linking each step to its artifacts.
+  - Focus on the failed step; none marked → last step in the trace.
+  - Multiple failures marked → the **first** is usually the cause; the rest cascade.
+
+### Artifact order for the focus step (`NNNN_<step>`)
+
+| Artifact | When |
+|---|---|
+| `NNNN_*_aria.txt` | First read — lean, structured, easy to scan for duplicates |
+| `NNNN_*_screenshot.png` | Visual confirmation — layout, animation, "rendered but wrong" |
+| `NNNN_*_console.json` | JS errors, 4xx/5xx, deprecation warnings explaining vanished elements |
+| `NNNN_*_storage.json` | Cookies + localStorage at this step — first stop when auth suspected |
+| `NNNN_*_page.html` | Last resort, and only via `grep` |
+
+### Never read big files whole
+
+- HTML snapshots: search with `grep` (text, class, aria-label, `data-*`) — line numbers + context flags keep matches readable.
+- `console.json`: filter with `jq`, don't read every entry.
+- Scanning many files: `grep -l` for filenames only.
+
+## Use cases
+
+### Verify a fix held
+Locate latest trace → read trace.md → confirm no FAILED markers. Glance at `console.json` for warnings worth fixing while you're there.
+
+### Cluster errors across a CI batch
+Extract failing-step lines from every `trace.md`, group by signature (`grep` + `sort` + `uniq -c`), rank by frequency.
+- Same error in many tests = **systemic** (env var, auth, base URL, deploy regression) — fix root cause once, rerun the batch.
+- Different errors per test = local — triage one at a time.
+Start with the most frequent root cause; rerun to see how many tests came back with it.
+
+### Diagnose flakiness
+Rerun the same test 5–10 times; compare which step failed in each trace.
+- Different step each run → timing, environment, external service
+- Same step, different state → missing/wrong wait — `diff` the ARIA snapshots of that step between runs
+- `console.json` differs between runs → transient backend/network errors
+- Bounding box differs → layout reflow or late-loading content
+
+### Investigate a single failure
+Failed-or-last step → ARIA + screenshot first, console second, HTML last (grep only). Form a hypothesis. Trace not enough / page needs live poking → hand off to `debugging-codeceptjs-tests`.
+
+## After analysis
+
+- Systemic cause → fix root once (env, auth, deploy), not per test
+- Locator drift → `codeceptjs-exploration`
+- Timing/wait issue → Waiting guidance from fundamentals/writing skills; replace `I.wait(N)` with specific `waitFor*`
+- Resists static analysis → `debugging-codeceptjs-tests` (live MCP loop)
+- Clean pass → done, but glance at `console.json` anyway
+
+## Things to avoid
+
+- Reading large HTML or `console.json` whole — `grep` / `jq`.
+- Stopping at the last marked failure instead of the first.
+- Triaging individual failures before clustering.
+- Flakiness conclusions from a single run — needs 5+ reruns.
+- Deleting `output/` mid-investigation.
+
+## Related skills
+
+- `codeceptjs-fundamentals` — what's configured, aiTrace `-p` overrides
+- `codeceptjs-exploration` — locator drift fixes
+- `debugging-codeceptjs-tests` — live loop + offline locator resolution via `codeceptq`

package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md

@@ -0,0 +1,126 @@
+---
+name: debugging-codeceptjs-tests
+description: >
+  Use when a CodeceptJS 4 test fails, flakes, or behaves unexpectedly. Trigger
+  on run errors and stack traces from `npx codeceptjs run`, intermittent
+  failures, locator drift, timing issues, "works locally fails in CI", "why
+  does this fail", trace/screenshot/console mentions, and breakpoint /
+  step-through / "pause at step N" requests.
+---
+
+# Debugging CodeceptJS 4 Tests
+
+Failures lie — the error usually points at a step that's a side effect of something earlier (auth expired, frame switch missed, network call pending). Reproduce, capture state, read it, then fix.
+
+## Paths
+
+- **MCP-first (AI agents)**: `run_test` / in-test `pause()` yield control to the agent on the same `I` / browser the test uses. Inspect via `run_code` / `snapshot`; step through via `run_step_by_step` + `continue`.
+- **CLI fallback (humans / CI / framework internals)**: `--debug` → `--verbose` → `DEBUG="codeceptjs:*"`. The DEBUG escape hatch is only for framework-internal suspicion: recorder hangs, plugin races, event leaks, "step never ran". Namespaces: `codeceptjs:recorder`, `codeceptjs:pause`, `codeceptjs:ai`, `codeceptjs:plugin:<name>`.
+
+`pause()` adapts to who's driving: TTY → readline REPL; MCP → yields control to the agent; non-TTY non-MCP subprocess → prints a notice and resolves (no CI deadlock).
+
+## Workflow
+
+1. **Fundamentals** — run `codeceptjs-fundamentals`: helper, plugins (`aiTrace`, `screenshot`, `pageInfo`, `retryFailedStep`, `pause`, `auth`), env vars, MCP availability. If `aiTrace` isn't declared, add it once: `plugins: { aiTrace: { enabled: true } }` — most of this skill leans on its output.
+   - **Declare once; never edit config to change its trigger.** Override per-run with `-p aiTrace:on=step|fail|test|file|url` (fundamentals § Plugins from CLI). `on=fail` for lean CI repros, `on=step` while diagnosing. Flipping `on:` in config churns the repo and leaks into other runs.
+2. **Reproduce minimally**: `npx codeceptjs run --grep '<scenario>' --steps`. Add `-c codecept.ci.conf.js` if CI-specific. Confirm reproduction before instrumenting.
+3. **Pick tools**:
+   - MCP `run_test <test>` — runs in-process; returns reporter result or `{ status: 'paused', pausedAfter, page, suggestions }`
+   - MCP `run_step_by_step` + `continue` — pause after every step; for watching the whole flow
+   - MCP `run_code` — arbitrary code in the live session; works fresh *and* paused; returns produced values + console output + final-state snapshot
+   - MCP `snapshot` — current browser state without acting (URL, cookies, storage, HTML, ARIA, screenshot, console)
+   - MCP `list_actions` — sanity-check an `I.*` method exists
+   - CLI `npx codeceptjs run --grep '<scenario>' --debug` — first move without MCP; `--verbose` adds promise-queue/retry/timeout logs
+4. **Breakpoint**:
+   - In-test `pause()` — best when already editing or breaking inside `within`/loop/hook
+   - `pauseAt: N` on `run_test` — no test edit; pauses after the Nth leaf step
+   - Find N: `npx codeceptjs dry-run --debug --grep '<scenario>' --numbers --no-ansi` (1-based, per-test; number of the line to stop *after*)
+   - While paused: inspect with `run_code`, capture clean state with `snapshot`, walk prior steps in `output/trace_<TestName>_<hash>/trace.md`, release with `continue`
+5. **Read the trace** — hand off to `codeceptjs-run-analysis`; focus on the **first** failed step — late failures are usually side effects of an earlier silent miss. It also covers grepping large HTML, clustering errors across traces, comparing reruns.
+6. **Form a hypothesis**:
+
+   | Symptom | Likely cause |
+   |---|---|
+   | Element missing, page is `/login` | Auth: stale session cache, missing env var |
+   | Element in HTML but `display: none` | `waitForVisible`, not `waitForElement` |
+   | Locator matches 2+ (strict mode) | Scope it: context arg, then ARIA role, then `step.opts({ elementIndex })` |
+   | Element present in screenshot N+1, missing in N | Animation / lazy load → `waitForVisible(loc, t)` |
+   | 401/403 in console.json | API token expired or env var missing |
+   | Steps pass, next `I.see` fails | Frame switch missed → `within({ frame })` |
+   | Different result CI vs local | `setHeadlessWhen(CI)`, viewport, timing, env vars |
+   | Recorder hangs, step never fires | `DEBUG="codeceptjs:recorder"` |
+   | Plugin misbehaves | `DEBUG="codeceptjs:plugin:<name>"` |
+
+7. **Verify the fix on the live page** — try the candidate replacement step via `run_code` (works while paused, same `I`) **before editing the file**. Humans at a TTY get the same via in-test `pause()`.
+8. **Apply and re-run**: edit, then `npx codeceptjs run --grep '<scenario>' --steps`; confirm via `codeceptjs-run-analysis` that the failure didn't shift steps. Leave a one-line `Why:` comment when the fix introduces `waitFor*` or `step.opts` — those comments are worth keeping.
+
+## Query trace HTML with `codeceptq`
+
+`aiTrace` writes per-step `<NNNN>_<step>_page.html` snapshots (one element per line — line numbers map 1:1 to elements). `codeceptq` resolves any CodeceptJS locator against a saved snapshot.
+
+- **Never load page HTML into context manually** — snapshots are thousands of lines; `codeceptq` returns only matched elements with their line numbers.
+- Test candidate locators offline before applying via `run_code` — a hit is a green light to try live, not a guarantee (visibility, re-renders).
+- Multiple matches → don't write a brittler XPath; disambiguate with `step.opts({ elementIndex })` following the order `codeceptq` prints.
+
+```bash
+npx codeceptq '#submit-btn' --file output/trace_*/0007_*_page.html     # CSS
+npx codeceptq 'Email' --field --file output/trace_*/0003_*_page.html   # semantic field
+npx codeceptq 'Save' '.modal' --click --file output/trace_*/0005_*_page.html  # scoped clickable
+npx codeceptq 'Username' --field --json --file ...                     # machine-readable
+```
+
+Key flags: `--field/--click/--checkable/--select` force semantic strategies; `--xpath`/`--css` override auto-detection (a bare tag name like `select.foo` is treated as fuzzy text); exit codes `0` match / `1` none / `2` invalid input.
+
+> ⚠ **The `[context]` second positional does not scope** (CodeceptJS 4.1.0). It prints `N matches within '<ctx>'` but returns page-wide results — `lib/command/query.js` evaluates an absolute XPath (`//…`) against the context node, and `//foo` re-roots at the document. It will report a match inside a container that does not hold the element. To check a scoped locator offline, pass one composed selector (`codeceptq '.modal button[aria-label="Save"]'`) and compare its count against the unscoped form; a context-dependent locator is only truly verified by running the step.
+
+## Inspect deeper
+
+Hand off to `codeceptjs-exploration` when the trace says *what* failed but you need more page-state detail ("is this button actually disabled?", "are there really two Save buttons?"). Debug-specific reaches:
+
+- Button rendered but click had no effect → `grabWebElement('Submit')` + `isEnabled()` + `getBoundingBox()` (disabled? offscreen?)
+- Strict-mode multi-match → exploration's broad-XPath iterate-and-disambiguate pattern
+- Iframe content → wrap failing steps in `within({ frame })`
+
+Prefer this over `usePlaywrightTo`/`useWebDriverTo` for inspection — same code across helpers, less boilerplate.
+
+## Native helper escape hatch
+
+Only when the `I.*` surface truly doesn't cover it (network interception, storage manipulation, helper-only APIs):
+
+- Playwright: `I.usePlaywrightTo('label', async ({ browser, browserContext, page }) => { ... })`
+- Puppeteer: `I.usePuppeteerTo('label', async ({ page }) => { ... })`
+- WebDriver: `I.useWebDriverTo('label', async ({ browser }) => { ... })`
+
+The label shows up in step output and traces. Works inside MCP `run_code` too. These couple tests to a specific helper — last resort.
+
+## Helper gotchas
+
+- Playwright: `strict: true` throws on multi-match; prefer `'load'`/`'domcontentloaded'` over `'networkidle'`; `trace: 'on'` → `output/trace.zip` (`npx playwright show-trace`)
+- Puppeteer: `'networkidle0'` hangs on long-polling pages — use `'networkidle2'` or `'domcontentloaded'`
+- WebDriver: `smartWait` covers actions only, not assertions; `executeScript` args must be JSON-serializable
+
+## Auth-related failures
+
+Redirect to `/login` mid-test or 401/403 in console → fix **auth**, not the failing step: check the plugin's `check`, credential env vars, and stale cached session under `output/<role>_session.json` (delete to force re-login). Full pattern in `codeceptjs-auth`.
+
+## Flakiness and waits
+
+Most "intermittent" failures are missed waits. Use trace HTML/ARIA to find the actual gating element instead of adding generic delay (fundamentals § Waiting has the mapping). `I.wait(N)` confirms a timing hypothesis while debugging — replace with the specific `waitFor*` before committing.
+
+## Things to avoid
+
+- Fixing from the error message without reading the trace.
+- Editing the test before verifying the fix via `run_code`.
+- Committing `pause()` calls — debugging tool only.
+- Blind `waitFor*` instead of identifying the real gating element.
+- Leaving `I.wait(N)` in committed tests.
+- Editing `aiTrace`'s `on:` in config between runs — declare once, override per-run.
+- Skipping the config check — `setHeadlessWhen(CI)` / env-driven URLs explain many "works locally fails in CI".
+- Hiding failures with `retries` instead of fixing the cause.
+
+## Related skills
+
+- `codeceptjs-fundamentals` — effects, plugins-from-CLI, waiting rules
+- `codeceptjs-exploration` — WebElement inspection, broad-XPath disambiguation
+- `codeceptjs-run-analysis` — trace.md walking, error clustering, rerun comparison
+- `codeceptjs-auth` — auth failure patterns