explorbot 0.2.2 → 0.2.4

package/docs/web-testing/automated-tests.md

@@ -0,0 +1,134 @@
+# Automated Tests
+
+When Explorbot runs a plan, it saves the executed scenarios to a runnable test file under `output/tests/`. Commit it, run it from CI, or edit it by hand.
+
+Pick the framework you use:
+
+- **Playwright** — set `ai.agents.historian.framework: 'playwright'` in your config. Output is a `.spec.ts` file.
+- **CodeceptJS** — the default. Output is a `.js` file.
+
+If you use neither, start with Playwright.
+
+## Playwright
+
+A file Explorbot produced from a plan called *Creating a plan*, trimmed for readability. See `example/output/tests/` for full examples.
+
+```ts
+import { test, expect } from '@playwright/test';
+
+test.describe('Creating a plan', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.goto('/projects/zyntra/plans/');
+    await page.waitForTimeout(1000);
+  });
+
+  test('Create a new manual plan and verify it appears in the list', async ({ page }) => {
+    await test.step("Click the 'New plan' button in toolbar", async () => {
+      await page.getByRole('button', { name: 'New plan' }).first().click();
+    });
+
+    await test.step('Select Manual plan type in modal', async () => {
+      await page.locator('#portal-container').getByRole('button', { name: 'Manual' }).click();
+    });
+
+    await test.step('Click Save', async () => {
+      await page.getByRole('button', { name: 'Save' }).click();
+    });
+
+    await test.step('Verification', async () => {
+      await expect(page).toContainText('Test Plan UI Creation 001');
+    });
+  });
+
+  // FAILED: Create a new automated plan and verify it appears under automated tab
+  test.skip('Create a new automated plan and verify it appears under automated tab', async ({ page }) => {
+    // ...
+  });
+
+  test.fixme('Delete a plan from the list', async ({ page }) => {
+    // open the action menu
+    // click Delete
+    // confirm in dialog
+  });
+});
+```
+
+Three things to notice:
+
+1. Every action is its own `test.step`. The label is the AI's description of that action, so the Playwright report tree reads like the plan it came from. Failures land on a single step.
+2. The locators are real. `page.getByRole('button', { name: 'New plan' })` is what Playwright ran during the session, not a translation. If it worked then, it works now.
+3. `test.beforeEach` reaches the starting state. It navigates to the plan's start URL and replays the `wait` / `waitForElement` knowledge you declared for that page. The closing `test.step('Verification', ...)` block holds whatever the Pilot verified.
+
+Run it the normal way:
+
+```bash
+npx playwright test output/tests/runs_archive_feature_testing.spec.ts
+```
+
+`npx explorbot rerun` does not run Playwright specs. It points you at `npx playwright test` instead.
+
+### Scenarios that didn't pass
+
+The file always runs, even when some scenarios failed:
+
+- Passed scenarios become plain `test(...)`.
+- Failed scenarios become `test.skip(...)` with a `// FAILED: <scenario>` comment above them, so you can see what broke and decide whether to fix or drop it.
+- Scenarios Explorbot didn't reach become `test.fixme(...)`, with the planned steps preserved as comments.
+
+Commit the file as-is. Passing scenarios run, broken ones are skipped and visible, planned ones wait for you.
+
+## CodeceptJS
+
+The default output is a `Feature` with one `Scenario` per plan entry, plus a `Before` block for setup:
+
+```js
+import step, { Section } from 'codeceptjs/steps';
+
+Feature('Runs Archive Feature Testing')
+
+Before(({ I }) => {
+  I.amOnPage('/projects/zyntra/runs/archive');
+  I.wait(1);
+});
+
+Scenario('Apply filters specific to archived runs and verify results', ({ I }) => {
+  Section('Open the filter panel');
+  I.click({ css: 'button.btn-only-icon.btn-lg:has(svg.md-icon-filter)' });
+
+  Section('Pick the Passed status');
+  I.click('Select status');
+  I.click('Passed');
+  I.click('Apply');
+
+  I.see('1 run found');
+});
+```
+
+Two things to notice:
+
+1. `Before` reaches the starting state. It calls `I.amOnPage` for the plan's start URL and replays the `wait` / `waitForElement` knowledge you declared for that page.
+2. Steps are grouped by `Section('...')`. Each label is the AI's description of that group, so the test reads top to bottom like the plan it came from.
+
+Run it with:
+
+```bash
+npx explorbot rerun output/tests/runs_archive_feature_testing.js
+```
+
+`npx explorbot rerun` heals broken steps automatically. See [Rerun](./rerun.md).
+
+### Scenarios that didn't pass
+
+The file always runs, even when some scenarios failed:
+
+- Passed scenarios become plain `Scenario(...)`.
+- Failed scenarios become `Scenario.skip(...)` with a `// FAILED: <scenario>` comment above them.
+- Scenarios Explorbot didn't reach become `Scenario.todo(...)`, with the planned steps preserved as comments.
+
+Commit the file as-is. Passing scenarios run, broken ones are skipped and visible, planned ones wait for you.
+
+## See also
+
+- [Test Plans](../workflow/test-plans.md) — the markdown plans that drive these test files
+- [Rerun](./rerun.md) — re-run CodeceptJS tests with auto-healing
+- [Configuration → Historian Agent Options](../reference/configuration.md#historian-agent-options) — the `framework` option

package/docs/web-testing/basics.md

@@ -0,0 +1,91 @@
+# Web Testing Basics
+
+Explorbot tests a web app the way a curious QA engineer would: open a page, figure out what it does, invent test scenarios, run them, and remember what happened. You write no test scripts up front — the tests come from the app itself.
+
+This page explains the concepts behind that loop and the minimum configuration to start. If you haven't installed Explorbot yet, do [Getting Started](../basics/getting-started.md) first.
+
+## What it does
+
+One exploration cycle has four parts:
+
+1. **Research.** The Researcher agent reads the current page — HTML, ARIA snapshot, and optionally a screenshot — and builds a map of what's on it: forms, buttons, tables, navigation.
+2. **Plan.** The Planner turns that map into test scenarios with priorities and expected outcomes, cycling through planning styles to broaden coverage.
+3. **Test.** The Tester runs each scenario step by step in a real browser, adapting when a click misses or a modal appears. The Pilot supervises from above and steps in when the test gets stuck.
+4. **Learn.** Outcomes are recorded — what worked, what failed, how failures were resolved — so the next run starts smarter.
+
+When you run `/explore`, this cycle repeats. After the start page is covered, Explorbot picks promising sub-pages linked from it and continues there, until it runs out of pages or hits your test limit.
+
+Two ideas hold this loop together, and you will meet them everywhere in these docs: states and learning.
+
+### States
+
+A page state is the URL plus the page's main headings (`h1` and `h2`). That's how Explorbot knows where it is. A URL alone isn't enough — a single-page app can show a list, an edit form, and a confirmation dialog all on one URL — so headings are part of the identity.
+
+States are the anchor points for everything else. Navigation history is a chain of state transitions, knowledge and experience are matched to the current state, and loop detection compares recent state hashes: if the bot keeps cycling through the same one or two states, it knows it's stuck and changes strategy instead of burning iterations.
+
+### Learning
+
+Explorbot keeps two directories of markdown files next to your config:
+
+- `knowledge/` — facts **you** teach it: credentials, quirks, hints, small automations. Each file targets pages by URL pattern, so knowledge for `/login` loads only on the login page and knowledge for `*` loads everywhere.
+- `experience/` — what it **learned by doing**: failed attempts, working resolutions, session notes. Files are named after state hashes, so lessons from a page are re-read the next time that page appears.
+
+Knowledge is the input you control; experience accumulates on its own. Both are plain markdown you can read and edit. See [Knowledge](../workflow/knowledge.md).
+
+## Configure
+
+Web testing needs one thing beyond the AI setup from Getting Started — the base URL of your app:
+
+```javascript
+export default {
+  web: {
+    url: 'http://localhost:3000',
+  },
+  ai: {
+    model: openrouter('openai/gpt-oss-20b:nitro'),
+    visionModel: openrouter('google/gemma-4-31b-it'),
+    agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
+  },
+};
+```
+
+The three models split the work by cost: `model` does the heavy page reading on every step, `visionModel` analyzes screenshots, and `agenticModel` makes the high-level decisions on short inputs, so it can be smarter without costing much. The full breakdown is in [Getting Started](../basics/getting-started.md); provider setup is in [Providers](../basics/providers.md). Every other option — browser settings, directories, per-agent tuning — lives in the [Configuration reference](../reference/configuration.md).
+
+## First run
+
+One tip from Getting Started is worth repeating: don't start on your homepage. Point Explorbot at one focused page with a clear CRUD interface — a list-and-edit screen, a settings page. It gives the bot an obvious job and you an obvious way to judge the result.
+
+The interactive route opens the TUI, where you watch the run and can step in:
+
+```bash
+npx explorbot start /admin/projects
+```
+
+Then type `/explore` to run the full loop, or go one step at a time with `/research` (analyze the page), `/plan` (propose scenarios), and `/test` (run the next one).
+
+The headless route runs the same loop without the TUI and exits when done — good for CI or overnight runs:
+
+```bash
+npx explorbot explore /admin/projects --max-tests 10 --focus "project management"
+```
+
+`--max-tests` caps how many tests run; `--focus` narrows planning to one feature. Both are optional.
+
+## Output files
+
+Every run leaves artifacts behind:
+
+- **Test plans** in `output/plans/` — the scenarios as markdown, with priorities and results. You can re-run, edit, or extend them. See [Test Plans](../workflow/test-plans.md).
+- **Runnable tests** in `output/tests/` — CodeceptJS/Playwright code generated from successful runs, ready to commit and run in CI. See [Automated Tests](./automated-tests.md).
+- **A session report** in `output/reports/` — a human-readable summary that clusters defects, UX issues, and execution problems by root cause. See [Reporting](../workflow/reporting.md).
+- **Experience files** in `experience/` — the lessons that make the next run faster and less error-prone.
+
+## Where to go next
+
+- [Customization](./customization.md) — make it work on *your* app: login, cookie banners, modals, test data.
+- [Planner](./planner.md) — tune what gets tested: styles, priorities, custom rules.
+- [Researcher](./researcher.md) — how pages are analyzed and what the UI map contains.
+- [Page Interaction](./page-interaction.md) — how Explorbot reads pages and picks locators.
+- [Automated Tests](./automated-tests.md) and [Rerun](./rerun.md) — the tests you keep, and re-running them with AI healing.
+- [Agents](./agents.md) — the agents behind the loop, and per-agent model configuration.
+- [Hooks](./hooks.md) — run your own code before and after agents.

package/docs/web-testing/customization.md

@@ -0,0 +1,131 @@
+# Customization
+
+Most real apps need a little setup before Explorbot can test them: a login, a cookie banner to dismiss, a slow page to wait for. This page shows the shortest recipe for each common case.
+
+You have two tools:
+
+- **Knowledge files** — markdown hints and small automations that run per URL. Start here.
+- **Hooks** — code that runs for one agent only. Use these when knowledge isn't enough.
+
+Each recipe below links to the full reference.
+
+## Log in
+
+Add a knowledge file for your login page and give Explorbot the credentials. Keep secrets in environment variables and reference them with `${env.NAME}`.
+
+`knowledge/login.md`:
+
+```markdown
+---
+url: /login
+---
+
+Log in with these credentials:
+- email: ${env.APP_EMAIL}
+- password: ${env.APP_PASSWORD}
+```
+
+Explorbot reads this when it opens the login page and signs in on its own. See [Knowledge](../workflow/knowledge.md).
+
+## Stay logged in between runs
+
+Logging in on every run is slow. Use `--session` to save cookies and local storage, then restore them next time:
+
+```bash
+npx explorbot start /login --session        # logs in, saves the session
+npx explorbot start /dashboard --session    # restores it, skips login
+```
+
+Pass a filename to keep more than one session: `--session auth.json`.
+
+## Dismiss a cookie banner
+
+A cookie bar blocks the page until you accept it. Dismiss one on every page with a knowledge file on `*`. Wrap the click in `tryTo` so it does nothing when the bar is absent:
+
+`knowledge/cookies.md`:
+
+```markdown
+---
+url: *
+code: |
+  await tryTo(() => I.click('Accept all'));
+---
+
+A cookie banner appears on first load. Accept it before interacting.
+```
+
+## Close modals and popups
+
+A modal can cover the page right after it loads. Close it with a `code` block that runs only if the modal is there:
+
+```markdown
+---
+url: /checkout
+code: |
+  await tryTo(() => I.click('[aria-label="Close"]'));
+---
+```
+
+To close a popup for one agent only — say, before research but not during a test — use a [hook](./hooks.md) instead.
+
+## Wait for slow or single-page apps
+
+Some pages show a spinner before the real content loads. Tell Explorbot to wait:
+
+```markdown
+---
+url: /dashboard
+wait: 2
+waitForElement: '.dashboard-ready'
+---
+```
+
+`wait` pauses for the given seconds. `waitForElement` waits for a selector to appear. For single-page apps where a full reload breaks state, add `statePush: true` so Explorbot navigates without reloading.
+
+## Set up and restore test data
+
+To seed data before tests and clean it up after, use agent hooks. `tester.beforeHook` runs before the test loop; `afterHook` runs after it:
+
+```javascript
+ai: {
+  agents: {
+    tester: {
+      beforeHook: {
+        type: 'codeceptjs',
+        hook: async ({ I }) => {
+          await I.executeScript(() => localStorage.setItem('cart', '[]'));
+        },
+      },
+      afterHook: {
+        type: 'playwright',
+        hook: async ({ page }) => {
+          await page.evaluate(() => localStorage.clear());
+        },
+      },
+    },
+  },
+}
+```
+
+See [Hooks](./hooks.md) for every agent and hook type.
+
+## Avoid fragile locators
+
+Some frameworks generate random IDs that change on every load — for example Ember's `#ember123`. Tell Explorbot to ignore them, and it will prefer stable locators like ARIA labels and visible text:
+
+```markdown
+---
+url: /projects/*
+---
+
+## Framework
+
+This app is built with Ember. Do not use auto-generated IDs like #ember123 in locators.
+Prefer ARIA labels and visible text.
+```
+
+## Knowledge or hooks?
+
+Reach for a knowledge file first. It is markdown, lives beside your other knowledge, and applies to every agent on matching pages. Use a hook when you need code to run for one agent only, or different behavior for navigation versus testing. For a case-by-case comparison, see the decision table in [Hooks](./hooks.md#when-to-use-hooks-vs-knowledge).
+
+Learn more in [Knowledge](../workflow/knowledge.md) and [Hooks](./hooks.md).

package/docs/web-testing/hooks.md

@@ -0,0 +1,238 @@
+# Agent Hooks
+
+Hooks run custom code before or after a specific agent. Use them to prepare and clean up the page per agent.
+
+> [!NOTE]
+> For simple page automation such as waiting or clicking cookie banners, use [Knowledge Files](../workflow/knowledge.md) with the `wait`, `waitForElement`, or `code` fields. Reach for hooks when you need different behavior for different agents.
+
+## When to Use Hooks vs Knowledge
+
+| Use Case | Solution |
+|----------|----------|
+| Wait for element on all page visits | Knowledge: `waitForElement` |
+| Dismiss cookie banner on page load | Knowledge: `code` |
+| Wait for network idle only during research | Hook: `researcher.beforeHook` |
+| Clean up test data after each test | Hook: `tester.afterHook` |
+| Different waits for navigation vs testing | Hooks for each agent |
+
+## Configuration
+
+Hooks are configured per-agent in `explorbot.config.js`:
+
+```javascript
+export default {
+  ai: {
+    model: myProvider('gpt-4o'),
+    agents: {
+      navigator: {
+        beforeHook: {
+          type: 'playwright',
+          hook: async ({ page, url }) => {
+            await page.waitForLoadState('networkidle');
+          }
+        }
+      },
+      tester: {
+        afterHook: {
+          type: 'codeceptjs',
+          hook: async ({ I, url }) => {
+            await I.executeScript(() => localStorage.clear());
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+## Hook Types
+
+### Playwright Hooks
+
+Get direct access to the Playwright page object:
+
+```javascript
+beforeHook: {
+  type: 'playwright',
+  hook: async ({ page, url }) => {
+    await page.waitForLoadState('networkidle');
+    await page.locator('.loading').waitFor({ state: 'hidden' });
+  }
+}
+```
+
+### CodeceptJS Hooks
+
+Use the CodeceptJS `I` actor:
+
+```javascript
+beforeHook: {
+  type: 'codeceptjs',
+  hook: async ({ I, url }) => {
+    await I.waitForElement('.page-ready');
+    await I.wait(1);
+  }
+}
+```
+
+## URL Pattern Matching
+
+Run different hooks for different URL patterns:
+
+```javascript
+researcher: {
+  beforeHook: {
+    '/login': {
+      type: 'codeceptjs',
+      hook: async ({ I }) => await I.waitForElement('#login-form')
+    },
+    '/admin/*': {
+      type: 'playwright',
+      hook: async ({ page }) => await page.waitForLoadState('networkidle')
+    },
+    '/api/*': {
+      type: 'codeceptjs',
+      hook: async ({ I }) => await I.wait(2)
+    }
+  }
+}
+```
+
+### Pattern Syntax
+
+| Pattern | Matches |
+|---------|---------|
+| `/login` | Exact path `/login` |
+| `/admin/*` | `/admin` and any path starting with `/admin/` |
+| `*` | All URLs (fallback) |
+| `^/users/\d+$` | Regex: `/users/` followed by digits |
+| `**/*.html` | Glob: any `.html` file |
+
+## Supported Agents
+
+| Agent | beforeHook | afterHook | Description |
+|-------|------------|-----------|-------------|
+| `navigator` | After navigation | After page capture | Browser navigation |
+| `researcher` | After navigation | After research complete | Page analysis |
+| `tester` | Before test loop | After test loop | Test execution |
+| `driller` | Before drilling starts | After drilling completes | Component drilling |
+| `captain` | Before handling command | After command complete | User commands |
+
+> [!WARNING]
+> The `planner` agent does not support hooks as it doesn't interact with the browser.
+
+## Examples
+
+### Wait for SPA to Load
+
+```javascript
+navigator: {
+  beforeHook: {
+    type: 'playwright',
+    hook: async ({ page }) => {
+      await page.waitForFunction(() => {
+        return window.__APP_READY__ === true;
+      });
+    }
+  }
+}
+```
+
+### Dismiss Modals Before Research
+
+```javascript
+researcher: {
+  beforeHook: {
+    type: 'codeceptjs',
+    hook: async ({ I }) => {
+      const modalVisible = await I.grabNumberOfVisibleElements('.modal-overlay');
+      if (modalVisible > 0) {
+        await I.click('.modal-close');
+        await I.wait(0.5);
+      }
+    }
+  }
+}
+```
+
+### Clean Up After Tests
+
+```javascript
+tester: {
+  afterHook: {
+    type: 'playwright',
+    hook: async ({ page }) => {
+      await page.evaluate(() => {
+        localStorage.clear();
+        sessionStorage.clear();
+      });
+    }
+  }
+}
+```
+
+### Different Behavior per URL
+
+```javascript
+tester: {
+  beforeHook: {
+    '/checkout': {
+      type: 'codeceptjs',
+      hook: async ({ I }) => {
+        // Ensure cart has items before checkout tests
+        await I.executeScript(() => {
+          if (!localStorage.getItem('cart')) {
+            localStorage.setItem('cart', JSON.stringify([{ id: 1, qty: 1 }]));
+          }
+        });
+      }
+    },
+    '/admin/*': {
+      type: 'codeceptjs',
+      hook: async ({ I }) => {
+        // Ensure admin session
+        await I.waitForElement('.admin-header', 5);
+      }
+    }
+  }
+}
+```
+
+## Error Handling
+
+A hook error is logged but does not stop the agent:
+
+```javascript
+beforeHook: {
+  type: 'codeceptjs',
+  hook: async ({ I }) => {
+    try {
+      await I.waitForElement('.optional-banner', 2);
+      await I.click('.dismiss');
+    } catch {
+      // Banner not present, continue
+    }
+  }
+}
+```
+
+## Execution Flow
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    Agent Execution                       │
+├─────────────────────────────────────────────────────────┤
+│  1. Agent starts                                         │
+│  2. Navigate to URL (if applicable)                      │
+│  3. ▶ beforeHook executes                               │
+│  4. Agent performs main work                             │
+│  5. ▶ afterHook executes                                │
+│  6. Agent completes                                      │
+└─────────────────────────────────────────────────────────┘
+```
+
+## See Also
+
+- [Knowledge Files](../workflow/knowledge.md) — page-level automation with `wait`, `waitForElement`, `code`
+- [Configuration](../reference/configuration.md) — full configuration reference
+- [Agents](./agents.md) — agent descriptions and workflows

package/docs/web-testing/page-interaction.md

@@ -0,0 +1,84 @@
+# Page Interaction
+
+This page explains how Explorbot agents read a web page and act on it — and the two things you can change to help them: filtering noisy HTML and marking custom components.
+
+## How agents see a page
+
+Agents read each page three ways at once:
+
+| Source | What it gives | Used for |
+|--------|---------------|----------|
+| **ARIA snapshot** | Roles, labels, states, hierarchy | Understanding structure, building locators |
+| **HTML** | Classes, IDs, data attributes, form fields | Precise locators, reading field values |
+| **Screenshot** | Layout, colors, icons, coordinates | Visual checks, click fallbacks |
+
+The accessibility tree is the primary source. HTML adds detail. The screenshot is a fallback when the first two aren't enough.
+
+## Filter noisy HTML
+
+Agents work from the `combined` HTML snapshot. Cookie banners, chat widgets, ads, and analytics tags add noise and burn tokens. Exclude them in your config:
+
+```javascript
+// explorbot.config.js
+html: {
+  combined: {
+    include: ['*'],
+    exclude: ['script', 'style', 'svg', '.cookie-banner', '.analytics-tracker'],
+  },
+}
+```
+
+Three snapshots exist, each configurable:
+
+| Snapshot | Purpose | Config key |
+|----------|---------|------------|
+| `combined` | Main HTML for agents | `html.combined` |
+| `minimal` | Interactive elements only | `html.minimal` |
+| `text` | Text content only | `html.text` |
+
+## Mark custom components
+
+Some components are interactive but have no ARIA role or semantic HTML, so agents miss them. Mark them with a `data-explorbot-*` attribute:
+
+```html
+<div data-explorbot-role="button" data-explorbot-label="Save Draft">
+  <svg>...</svg>
+  Save
+</div>
+```
+
+A marked element is always kept in snapshots, treated as interactive, and shown to agents. During processing, `data-explorbot-role="button"` becomes `role="button"`, so you add a hint without changing how your component behaves.
+
+Use this when standard locators fail to find an element, or when a custom control isn't detected as interactive.
+
+## Locator priority
+
+When an agent picks a locator, it prefers the most stable option first:
+
+```mermaid
+flowchart LR
+    A[ARIA, semantic text, and role] --> B[Scope with a container]
+    B --> C[HTML attributes, CSS, or XPath]
+    C --> D[Screenshot and visual coordinates]
+```
+
+The agent starts with short semantic locators from ARIA or visible text. When a match is ambiguous, it scopes that locator to a container before moving to HTML-based selectors and, finally, visual coordinates.
+
+1. Short ARIA or text — `I.click({ role: 'button', text: 'Save' })` or `I.click('Save')` when it is unique
+2. Text with a container — `I.click('Save', '.modal')` — simplest and preferred when a container is known
+3. ARIA with a container — `I.click({ role: 'button', text: 'Save' }, '.modal')` — for semantic disambiguation
+4. CSS or XPath — `I.click('#save-btn')`
+5. Coordinates — `I.clickXY(400, 300)` (last resort)
+
+When a locator fails, the agent tries the next strategy, then a visual click. Locators that worked are saved to experience and preferred on the next run.
+
+## What happens after each action
+
+After every action, Explorbot captures the new page state and compares it with the previous one. The resulting diff tells the agent what changed — the URL, the accessibility tree, or the HTML — so it can confirm the action worked and decide what to do next. The Researcher turns a page into a structured UI map of sections and elements; see [Researcher](./researcher.md).
+
+## See also
+
+- [Knowledge files](../workflow/knowledge.md) — teach Explorbot about your app
+- [Agent hooks](./hooks.md) — run code before or after an agent
+- [Configuration](../reference/configuration.md) — full configuration reference
+- [Researcher](./researcher.md) — how pages become UI maps