explorbot 0.2.3 → 0.2.5

package/docs/web-testing/agents.md

@@ -0,0 +1,158 @@
+# AI Agents
+
+Explorbot splits the testing workflow across specialized AI agents. Each agent handles one part of the work, which keeps it focused and keeps token costs down.
+
+## Agent Overview
+
+```mermaid
+flowchart LR
+    A[Navigator] --> B[Researcher] --> C[Planner] --> D[Tester]
+    A -- "goes to page" --> B
+    B -- "analyzes UI" --> C
+    C -- "suggests tests" --> D
+    D -- "runs tests" --> A
+    E[Pilot] -.->|supervises| D
+```
+
+## Navigator Agent
+
+Handles browser interactions: clicks, form fills, and navigation.
+
+The Navigator runs CodeceptJS commands in the browser. When a selector fails, it tries other locator strategies and resolves the interaction without stopping the run. It remembers what worked and what didn't, so failed selectors don't keep killing your runs and tests survive UI changes.
+
+Commands that use Navigator:
+- `/navigate <target>`
+- `I.click()`, `I.fillField()`, `I.amOnPage()`, etc.
+
+## Researcher Agent
+
+Analyzes pages to find what's actually on them.
+
+The Researcher discovers every interactive element, including content hidden in accordions, dropdowns, and modals. It maps navigation paths and form structures, extracts data from tables and lists, and filters out noise like cookie banners and ads. The result is a complete map of what you can test, with form validation rules documented.
+
+Commands that use Researcher:
+- `npx explorbot research /path` (CLI)
+- `/research [path]` (TUI)
+- `/research --deep` — expand hidden elements
+- `/research --data` — extract page data
+
+Explicit research always captures a screenshot; a configured vision model analyzes it.
+
+See [Researcher Agent](./researcher.md) for configuration and usage.
+
+## Planner Agent
+
+Generates test scenarios from research findings.
+
+The Planner writes business-focused scenarios with priority levels (critical/important/high/normal/low) and expected outcomes for verification. It balances positive and negative cases, skips scenarios you already have, and cycles through planning styles (normal, psycho, curious) to broaden coverage across iterations. You can add your own styles and page-specific rules.
+
+Commands that use Planner:
+- `/plan [--focus <feature>]`
+- `/explore`
+
+See [Planner Agent](./planner.md) for planning styles, customization, and configuration.
+
+## Tester Agent
+
+Runs the planned scenarios.
+
+The Tester executes scenarios step by step and adapts when something goes wrong. It tracks state changes during execution, records actual results against expected ones, and uses research context to make decisions. It handles unexpected modals and popups, recovers from minor failures on its own, and produces detailed execution logs.
+
+Commands that use Tester:
+- `/test [scenario]`
+- `/explore`
+
+## Pilot Agent
+
+Supervises the Tester and steps in when a test gets stuck.
+
+The Pilot keeps a separate conversation to track progress over time. It detects stuck patterns — loops, repeated failures, no page changes — and decides what context the Tester needs next (HTML, ARIA, UI map). When automated recovery fails, it asks the user for help instead of giving up.
+
+Because the Pilot sees only tool summaries rather than raw HTML, you can run it on a smarter model without a token cost explosion.
+
+The Pilot intervenes when:
+- Actions succeed but the page doesn't change (wrong element)
+- The same action repeats several times (loop)
+- The same locator keeps failing (need a different approach)
+- Only research or context calls run, with no action tools (no progress)
+
+## Analyst Agent
+
+Produces a human-readable session report after `/explore` and `/freesail` runs.
+
+The Analyst reads every test in the session — scenario, expected outcome, final result, notes, and step log. It clusters findings by root cause: three tests that fail on the same control become one defect with three test references, not three rows. It separates product defects from automation problems, so an element the test could not click is not reported as a bug.
+
+The report is plain markdown: a one-to-two sentence headline about the feature state (no test counts), then `## Coverage`, `## What works`, `## Defects` (each with a `[High]`/`[Medium]`/`[Low]` severity tag, reproduce steps, and one-line evidence), `## UX issues`, and `## Execution Issues`. Empty sections are omitted.
+
+It runs automatically at the end of `/explore` (per-run) and on app exit (session-wide, across multiple `/explore` or `/freesail` runs). The same markdown goes to the console, to `output/reports/<mode>-<sessionName>.md`, and — when the reporter is enabled — to the Testomat.io run description.
+
+Configuration:
+
+```javascript
+export default {
+  ai: {
+    agents: {
+      analyst: {
+        // model: openai('gpt-4o'),       // override the model
+        // systemPrompt: 'Focus on...',   // append guidance to the prompt
+        // enabled: false,                // disable the analyst entirely
+      },
+    },
+  },
+};
+```
+
+The Analyst resolves its model like other agentic tasks: the agent-specific `model`, then `ai.agenticModel`, then `ai.model`.
+
+## Captain Agent
+
+Handles your direct requests in the TUI and recovers the session when something breaks.
+
+The Captain steps in when a slash command isn't enough — answering questions about your setup, inspecting tests and page states, and reading recent output before it replies. It works in four modes:
+
+- **idle** — plan management, project inspection, knowledge and experience files; available before any page loads
+- **web** — page interaction, navigation, and browser diagnostics
+- **test** — test timeline, state inspection, generated code and logs
+- **heal** — browser and test recovery when an active test loses its page or browser
+
+When a test hits a fatal browser error, the Captain tries to recover — reload, restart the browser, open a fresh tab, or close extra tabs — before the test is stopped, then tells the Tester how to continue. It runs on explicit TUI requests and on test interrupts (stop, pass, skip, or redirect a running test).
+
+## Per-Agent Model Configuration
+
+Use different models for different agents to control cost:
+
+```javascript
+export default {
+  ai: {
+    model: groq('openai/gpt-oss-20b'),
+    visionModel: groq('meta-llama/llama-4-scout-17b-16e-instruct'),
+    agents: {
+      navigator: { model: groq('openai/gpt-oss-20b') },
+      researcher: { model: groq('openai/gpt-oss-20b') },
+      planner: { model: groq('openai/gpt-oss-20b') },
+      tester: { model: groq('openai/gpt-oss-20b'), progressCheckInterval: 5 },
+      pilot: { stepsToReview: 5 },
+    },
+  },
+};
+```
+
+Typical choices:
+- Navigator needs fast responses for real-time interaction
+- Researcher benefits from vision
+- Planner can use a larger model for better test design
+- Tester needs tool use for execution
+- Pilot can use a smarter model — it only processes tool summaries, not HTML or ARIA
+
+## How Agents Communicate
+
+Agents share context through four channels:
+
+1. **State Manager** — tracks the current page, URL, and navigation history
+2. **Research Results** — structured page analysis available to Planner and Tester
+3. **Experience Files** — patterns learned across sessions. Injected as a compact table of contents (file tags plus section headings) rather than full bodies; agents pull individual sections on demand via the `learnExperience` tool
+4. **Knowledge Files** — domain knowledge you provide
+
+Each agent keeps its context minimal to hold costs down. Agents request specific information when they need it instead of carrying full conversation history.
+
+The Pilot keeps a separate conversation from the Tester. The Tester's conversation carries heavy HTML and ARIA context; the Pilot sees only tool execution summaries (what succeeded, what failed, what changed). That lets the Pilot run on expensive models without a token cost explosion.

package/docs/web-testing/automated-tests.md

@@ -0,0 +1,134 @@
+# Automated Tests
+
+When Explorbot runs a plan, it saves the executed scenarios to a runnable test file under `output/tests/`. Commit it, run it from CI, or edit it by hand.
+
+Pick the framework you use:
+
+- **Playwright** — set `ai.agents.historian.framework: 'playwright'` in your config. Output is a `.spec.ts` file.
+- **CodeceptJS** — the default. Output is a `.js` file.
+
+If you use neither, start with Playwright.
+
+## Playwright
+
+A file Explorbot produced from a plan called *Creating a plan*, trimmed for readability. See `example/output/tests/` for full examples.
+
+```ts
+import { test, expect } from '@playwright/test';
+
+test.describe('Creating a plan', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.goto('/projects/zyntra/plans/');
+    await page.waitForTimeout(1000);
+  });
+
+  test('Create a new manual plan and verify it appears in the list', async ({ page }) => {
+    await test.step("Click the 'New plan' button in toolbar", async () => {
+      await page.getByRole('button', { name: 'New plan' }).first().click();
+    });
+
+    await test.step('Select Manual plan type in modal', async () => {
+      await page.locator('#portal-container').getByRole('button', { name: 'Manual' }).click();
+    });
+
+    await test.step('Click Save', async () => {
+      await page.getByRole('button', { name: 'Save' }).click();
+    });
+
+    await test.step('Verification', async () => {
+      await expect(page).toContainText('Test Plan UI Creation 001');
+    });
+  });
+
+  // FAILED: Create a new automated plan and verify it appears under automated tab
+  test.skip('Create a new automated plan and verify it appears under automated tab', async ({ page }) => {
+    // ...
+  });
+
+  test.fixme('Delete a plan from the list', async ({ page }) => {
+    // open the action menu
+    // click Delete
+    // confirm in dialog
+  });
+});
+```
+
+Three things to notice:
+
+1. Every action is its own `test.step`. The label is the AI's description of that action, so the Playwright report tree reads like the plan it came from. Failures land on a single step.
+2. The locators are real. `page.getByRole('button', { name: 'New plan' })` is what Playwright ran during the session, not a translation. If it worked then, it works now.
+3. `test.beforeEach` reaches the starting state. It navigates to the plan's start URL and replays the `wait` / `waitForElement` knowledge you declared for that page. The closing `test.step('Verification', ...)` block holds whatever the Pilot verified.
+
+Run it the normal way:
+
+```bash
+npx playwright test output/tests/runs_archive_feature_testing.spec.ts
+```
+
+`npx explorbot rerun` does not run Playwright specs. It points you at `npx playwright test` instead.
+
+### Scenarios that didn't pass
+
+The file always runs, even when some scenarios failed:
+
+- Passed scenarios become plain `test(...)`.
+- Failed scenarios become `test.skip(...)` with a `// FAILED: <scenario>` comment above them, so you can see what broke and decide whether to fix or drop it.
+- Scenarios Explorbot didn't reach become `test.fixme(...)`, with the planned steps preserved as comments.
+
+Commit the file as-is. Passing scenarios run, broken ones are skipped and visible, planned ones wait for you.
+
+## CodeceptJS
+
+The default output is a `Feature` with one `Scenario` per plan entry, plus a `Before` block for setup:
+
+```js
+import step, { Section } from 'codeceptjs/steps';
+
+Feature('Runs Archive Feature Testing')
+
+Before(({ I }) => {
+  I.amOnPage('/projects/zyntra/runs/archive');
+  I.wait(1);
+});
+
+Scenario('Apply filters specific to archived runs and verify results', ({ I }) => {
+  Section('Open the filter panel');
+  I.click({ css: 'button.btn-only-icon.btn-lg:has(svg.md-icon-filter)' });
+
+  Section('Pick the Passed status');
+  I.click('Select status');
+  I.click('Passed');
+  I.click('Apply');
+
+  I.see('1 run found');
+});
+```
+
+Two things to notice:
+
+1. `Before` reaches the starting state. It calls `I.amOnPage` for the plan's start URL and replays the `wait` / `waitForElement` knowledge you declared for that page.
+2. Steps are grouped by `Section('...')`. Each label is the AI's description of that group, so the test reads top to bottom like the plan it came from.
+
+Run it with:
+
+```bash
+npx explorbot rerun output/tests/runs_archive_feature_testing.js
+```
+
+`npx explorbot rerun` heals broken steps automatically. See [Rerun](./rerun.md).
+
+### Scenarios that didn't pass
+
+The file always runs, even when some scenarios failed:
+
+- Passed scenarios become plain `Scenario(...)`.
+- Failed scenarios become `Scenario.skip(...)` with a `// FAILED: <scenario>` comment above them.
+- Scenarios Explorbot didn't reach become `Scenario.todo(...)`, with the planned steps preserved as comments.
+
+Commit the file as-is. Passing scenarios run, broken ones are skipped and visible, planned ones wait for you.
+
+## See also
+
+- [Test Plans](../workflow/test-plans.md) — the markdown plans that drive these test files
+- [Rerun](./rerun.md) — re-run CodeceptJS tests with auto-healing
+- [Configuration → Historian Agent Options](../reference/configuration.md#historian-agent-options) — the `framework` option

package/docs/web-testing/basics.md

@@ -0,0 +1,91 @@
+# Web Testing Basics
+
+Explorbot tests a web app the way a curious QA engineer would: open a page, figure out what it does, invent test scenarios, run them, and remember what happened. You write no test scripts up front — the tests come from the app itself.
+
+This page explains the concepts behind that loop and the minimum configuration to start. If you haven't installed Explorbot yet, do [Getting Started](../basics/getting-started.md) first.
+
+## What it does
+
+One exploration cycle has four parts:
+
+1. **Research.** The Researcher agent reads the current page — HTML, ARIA snapshot, and optionally a screenshot — and builds a map of what's on it: forms, buttons, tables, navigation.
+2. **Plan.** The Planner turns that map into test scenarios with priorities and expected outcomes, cycling through planning styles to broaden coverage.
+3. **Test.** The Tester runs each scenario step by step in a real browser, adapting when a click misses or a modal appears. The Pilot supervises from above and steps in when the test gets stuck.
+4. **Learn.** Outcomes are recorded — what worked, what failed, how failures were resolved — so the next run starts smarter.
+
+When you run `/explore`, this cycle repeats. After the start page is covered, Explorbot picks promising sub-pages linked from it and continues there, until it runs out of pages or hits your test limit.
+
+Two ideas hold this loop together, and you will meet them everywhere in these docs: states and learning.
+
+### States
+
+A page state is the URL plus the page's main headings (`h1` and `h2`). That's how Explorbot knows where it is. A URL alone isn't enough — a single-page app can show a list, an edit form, and a confirmation dialog all on one URL — so headings are part of the identity.
+
+States are the anchor points for everything else. Navigation history is a chain of state transitions, knowledge and experience are matched to the current state, and loop detection compares recent state hashes: if the bot keeps cycling through the same one or two states, it knows it's stuck and changes strategy instead of burning iterations.
+
+### Learning
+
+Explorbot keeps two directories of markdown files next to your config:
+
+- `knowledge/` — facts **you** teach it: credentials, quirks, hints, small automations. Each file targets pages by URL pattern, so knowledge for `/login` loads only on the login page and knowledge for `*` loads everywhere.
+- `experience/` — what it **learned by doing**: failed attempts, working resolutions, session notes. Files are named after state hashes, so lessons from a page are re-read the next time that page appears.
+
+Knowledge is the input you control; experience accumulates on its own. Both are plain markdown you can read and edit. See [Knowledge](../workflow/knowledge.md).
+
+## Configure
+
+Web testing needs one thing beyond the AI setup from Getting Started — the base URL of your app:
+
+```javascript
+export default {
+  web: {
+    url: 'http://localhost:3000',
+  },
+  ai: {
+    model: openrouter('openai/gpt-oss-20b:nitro'),
+    visionModel: openrouter('google/gemma-4-31b-it'),
+    agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
+  },
+};
+```
+
+The three models split the work by cost: `model` does the heavy page reading on every step, `visionModel` analyzes screenshots, and `agenticModel` makes the high-level decisions on short inputs, so it can be smarter without costing much. The full breakdown is in [Getting Started](../basics/getting-started.md); provider setup is in [Providers](../basics/providers.md). Every other option — browser settings, directories, per-agent tuning — lives in the [Configuration reference](../reference/configuration.md).
+
+## First run
+
+One tip from Getting Started is worth repeating: don't start on your homepage. Point Explorbot at one focused page with a clear CRUD interface — a list-and-edit screen, a settings page. It gives the bot an obvious job and you an obvious way to judge the result.
+
+The interactive route opens the TUI, where you watch the run and can step in:
+
+```bash
+npx explorbot start /admin/projects
+```
+
+Then type `/explore` to run the full loop, or go one step at a time with `/research` (analyze the page), `/plan` (propose scenarios), and `/test` (run the next one).
+
+The headless route runs the same loop without the TUI and exits when done — good for CI or overnight runs:
+
+```bash
+npx explorbot explore /admin/projects --max-tests 10 --focus "project management"
+```
+
+`--max-tests` caps how many tests run; `--focus` narrows planning to one feature. Both are optional.
+
+## Output files
+
+Every run leaves artifacts behind:
+
+- **Test plans** in `output/plans/` — the scenarios as markdown, with priorities and results. You can re-run, edit, or extend them. See [Test Plans](../workflow/test-plans.md).
+- **Runnable tests** in `output/tests/` — CodeceptJS/Playwright code generated from successful runs, ready to commit and run in CI. See [Automated Tests](./automated-tests.md).
+- **A session report** in `output/reports/` — a human-readable summary that clusters defects, UX issues, and execution problems by root cause. See [Reporting](../workflow/reporting.md).
+- **Experience files** in `experience/` — the lessons that make the next run faster and less error-prone.
+
+## Where to go next
+
+- [Customization](./customization.md) — make it work on *your* app: login, cookie banners, modals, test data.
+- [Planner](./planner.md) — tune what gets tested: styles, priorities, custom rules.
+- [Researcher](./researcher.md) — how pages are analyzed and what the UI map contains.
+- [Page Interaction](./page-interaction.md) — how Explorbot reads pages and picks locators.
+- [Automated Tests](./automated-tests.md) and [Rerun](./rerun.md) — the tests you keep, and re-running them with AI healing.
+- [Agents](./agents.md) — the agents behind the loop, and per-agent model configuration.
+- [Hooks](./hooks.md) — run your own code before and after agents.

package/docs/web-testing/customization.md

@@ -0,0 +1,131 @@
+# Customization
+
+Most real apps need a little setup before Explorbot can test them: a login, a cookie banner to dismiss, a slow page to wait for. This page shows the shortest recipe for each common case.
+
+You have two tools:
+
+- **Knowledge files** — markdown hints and small automations that run per URL. Start here.
+- **Hooks** — code that runs for one agent only. Use these when knowledge isn't enough.
+
+Each recipe below links to the full reference.
+
+## Log in
+
+Add a knowledge file for your login page and give Explorbot the credentials. Keep secrets in environment variables and reference them with `${env.NAME}`.
+
+`knowledge/login.md`:
+
+```markdown
+---
+url: /login
+---
+
+Log in with these credentials:
+- email: ${env.APP_EMAIL}
+- password: ${env.APP_PASSWORD}
+```
+
+Explorbot reads this when it opens the login page and signs in on its own. See [Knowledge](../workflow/knowledge.md).
+
+## Stay logged in between runs
+
+Logging in on every run is slow. Use `--session` to save cookies and local storage, then restore them next time:
+
+```bash
+npx explorbot start /login --session        # logs in, saves the session
+npx explorbot start /dashboard --session    # restores it, skips login
+```
+
+Pass a filename to keep more than one session: `--session auth.json`.
+
+## Dismiss a cookie banner
+
+A cookie bar blocks the page until you accept it. Dismiss one on every page with a knowledge file on `*`. Wrap the click in `tryTo` so it does nothing when the bar is absent:
+
+`knowledge/cookies.md`:
+
+```markdown
+---
+url: *
+code: |
+  await tryTo(() => I.click('Accept all'));
+---
+
+A cookie banner appears on first load. Accept it before interacting.
+```
+
+## Close modals and popups
+
+A modal can cover the page right after it loads. Close it with a `code` block that runs only if the modal is there:
+
+```markdown
+---
+url: /checkout
+code: |
+  await tryTo(() => I.click('[aria-label="Close"]'));
+---
+```
+
+To close a popup for one agent only — say, before research but not during a test — use a [hook](./hooks.md) instead.
+
+## Wait for slow or single-page apps
+
+Some pages show a spinner before the real content loads. Tell Explorbot to wait:
+
+```markdown
+---
+url: /dashboard
+wait: 2
+waitForElement: '.dashboard-ready'
+---
+```
+
+`wait` pauses for the given seconds. `waitForElement` waits for a selector to appear. For single-page apps where a full reload breaks state, add `statePush: true` so Explorbot navigates without reloading.
+
+## Set up and restore test data
+
+To seed data before tests and clean it up after, use agent hooks. `tester.beforeHook` runs before the test loop; `afterHook` runs after it:
+
+```javascript
+ai: {
+  agents: {
+    tester: {
+      beforeHook: {
+        type: 'codeceptjs',
+        hook: async ({ I }) => {
+          await I.executeScript(() => localStorage.setItem('cart', '[]'));
+        },
+      },
+      afterHook: {
+        type: 'playwright',
+        hook: async ({ page }) => {
+          await page.evaluate(() => localStorage.clear());
+        },
+      },
+    },
+  },
+}
+```
+
+See [Hooks](./hooks.md) for every agent and hook type.
+
+## Avoid fragile locators
+
+Some frameworks generate random IDs that change on every load — for example Ember's `#ember123`. Tell Explorbot to ignore them, and it will prefer stable locators like ARIA labels and visible text:
+
+```markdown
+---
+url: /projects/*
+---
+
+## Framework
+
+This app is built with Ember. Do not use auto-generated IDs like #ember123 in locators.
+Prefer ARIA labels and visible text.
+```
+
+## Knowledge or hooks?
+
+Reach for a knowledge file first. It is markdown, lives beside your other knowledge, and applies to every agent on matching pages. Use a hook when you need code to run for one agent only, or different behavior for navigation versus testing. For a case-by-case comparison, see the decision table in [Hooks](./hooks.md#when-to-use-hooks-vs-knowledge).
+
+Learn more in [Knowledge](../workflow/knowledge.md) and [Hooks](./hooks.md).