explorbot 0.2.3 → 0.2.5

package/docs/workflow/ci.md

@@ -0,0 +1,202 @@
+# Continuous Integration
+
+Once your Explorbot setup works locally — it logs in, explores, and produces sensible tests — the next step is to run it on a schedule, so the whole app gets re-tested continuously without anyone starting it by hand. This page shows what a CI run needs and gives a worked pipeline for the common providers. If you haven't run Explorbot headlessly yet, read [Running Explorbot](../basics/running.md) first.
+
+## What a CI run needs
+
+- **A provider API key from the environment.** Your local `.env` file is not in the repo; in CI, set the key (for example `OPENROUTER_API_KEY`) as a pipeline secret and pass it as an environment variable.
+- **Playwright browsers.** Install them in the job: `npx playwright install --with-deps chromium`.
+- **A headless browser.** That's the default — don't pass `--show` and there is nothing to configure.
+- **A start path and a test budget.** `npx explorbot explore / --max-tests 10` keeps the run bounded and predictable.
+- **A hard timeout as backstop.** AI runs can stall; a job-level timeout guarantees the pipeline never hangs.
+
+For login, commit your `knowledge/` directory to the repo — it's the input you control, and Explorbot reads it on every run. Keep credentials out of the files with `${env.NAME}` interpolation, as described in [Knowledge](./knowledge.md), and set those variables as pipeline secrets too. Add `--session` to the run command so Explorbot logs in once and reuses the saved session.
+
+## Cache, commit, or upload
+
+Three directories, three fates:
+
+- **Commit `knowledge/`.** It's authored by you and versioned like code.
+- **Cache `experience/` and `output/`.** `experience/` carries lessons between runs — fewer repeated failures, faster runs — so a cached CI job gets smarter every night instead of starting from zero. `output/` carries `session.json` plus previous plans and generated tests; with `--configure` the next run reloads a saved plan and re-runs its tests regression-style instead of planning everything fresh (see the [`--configure` reference](../reference/commands.md#explore)).
+- **Upload `output/reports/`, `output/tests/`, and `output/screencasts/` as artifacts.** Reports and screencasts are for humans to review after the run; generated tests are code you may want to review and commit. If you send results to Testomat.io with artifact storage, screencasts are already attached to tests there — see [Reporting on CI](#reporting-on-ci).
+
+## Exit codes and gating
+
+As covered in [Running Explorbot](../basics/running.md#exit-codes), `explore` exits `1` only when the run itself crashes — bad config, unreachable app, provider failure. Failing tests do not fail the job; they are findings, not crashes. So a green pipeline means "Explorbot ran", not "no bugs found".
+
+To see what it found, read the session report in `output/reports/` — the Analyst writes a markdown summary that clusters defects by root cause. For per-test pass/fail in machine-checkable form, enable the markdown test report or send results to Testomat.io with the `TESTOMATIO` project key; both are described in [Reporting](./reporting.md). API tests are stricter: `api test` and `api explore` exit `1` when any test fails, so they gate natively.
+
+## Reporting on CI
+
+A scheduled run is only worth having if its results are easy to review the next morning. The recommended stack gives you a report you read top-down — start at the overview, drill into a failing test, watch exactly what happened:
+
+1. **Send results to Testomat.io.** Add the `TESTOMATIO` project key as a pipeline secret. Every run lands in the dashboard with steps, statuses, and history — nothing to download from CI.
+2. **Enable artifact storage.** Connect an S3-compatible bucket under Settings > Artifacts in Testomat.io (credentials via `S3_*` pipeline variables — see [artifacts in cloud reports](./reporting.md#artifacts-in-cloud-reports)). Screenshots and screencasts attached to tests upload automatically.
+3. **Record screencasts.** Every test becomes a video with the executed actions and the current scenario step overlaid — reviewing one beats clicking through a screenshot trail.
+4. **Enable the Analyst.** Its summary becomes the run description on Testomat.io, so the run opens with defects clustered by root cause, coverage, and what works.
+
+The two config switches, next to your reporter settings:
+
+```js
+export default {
+  ai: {
+    agents: {
+      historian: { screencast: true },
+      analyst: { enabled: true },
+    },
+  },
+};
+```
+
+The review flow this buys: open the run, read the Analyst overview in the description, scan the failed tests, open one, watch its screencast.
+
+Without the cloud reporter, the fallback is `html: true` and `markdown: true` under `reporter` plus uploading `output/reports/` and `output/screencasts/` as CI artifacts — the pipelines below do. You lose history and the overview-to-video flow, but every run is still reviewable.
+
+**API runs** report through the same reporter, so the same `TESTOMATIO` key covers `api test` and `api explore`. There are no screencasts (no browser) and the Analyst covers web sessions only; upload `output/requests/` as a CI artifact instead — every request and response is logged there as YAML.
+
+## GitHub Actions
+
+A nightly run with caching, artifacts, and a manual trigger:
+
+```yaml
+name: Explorbot Nightly
+
+on:
+  schedule:
+    - cron: '0 3 * * *'
+  workflow_dispatch:
+
+jobs:
+  explore:
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 24
+
+      - run: npm ci
+      - run: npx playwright install --with-deps chromium
+
+      - uses: actions/cache@v4
+        with:
+          path: |
+            experience
+            output
+          key: explorbot-${{ github.run_id }}
+          restore-keys: |
+            explorbot-
+
+      - run: npx explorbot explore / --max-tests 10 --session
+        env:
+          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: explorbot-results
+          path: |
+            output/reports
+            output/tests
+            output/screencasts
+```
+
+The cache key includes the run id, so every run saves an updated cache and the next one restores the latest via `restore-keys`. To report runs to Testomat.io, add `TESTOMATIO: ${{ secrets.TESTOMATIO }}` to the `env` block — see [Reporting](./reporting.md).
+
+## GitLab CI
+
+Create the schedule under **CI/CD → Pipeline schedules** and set `OPENROUTER_API_KEY` as a masked CI/CD variable. The job itself:
+
+```yaml
+explorbot:
+  image: node:24
+  rules:
+    - if: $CI_PIPELINE_SOURCE == "schedule"
+  timeout: 1h
+  cache:
+    key: explorbot
+    paths:
+      - experience/
+      - output/
+  script:
+    - npm ci
+    - npx playwright install --with-deps chromium
+    - npx explorbot explore / --max-tests 10 --session
+  artifacts:
+    when: always
+    paths:
+      - output/reports/
+      - output/tests/
+      - output/screencasts/
+```
+
+## Jenkins
+
+Jenkins has no built-in cross-run cache, but the workspace persists between builds on the same agent — so `experience/` and `output/` carry over as long as you don't wipe the workspace (skip `cleanWs`, or use the Job Cacher plugin on those two directories).
+
+```groovy
+pipeline {
+  agent any
+  triggers { cron('H 3 * * *') }
+  options { timeout(time: 60, unit: 'MINUTES') }
+  environment { OPENROUTER_API_KEY = credentials('openrouter-api-key') }
+  stages {
+    stage('Explore') {
+      steps {
+        sh 'npm ci'
+        sh 'npx playwright install --with-deps chromium'
+        sh 'npx explorbot explore / --max-tests 10 --session'
+      }
+    }
+  }
+  post {
+    always {
+      archiveArtifacts artifacts: 'output/reports/**, output/tests/**, output/screencasts/**', allowEmptyArchive: true
+    }
+  }
+}
+```
+
+## Azure Pipelines
+
+Scheduled triggers are declared in the pipeline itself; `always: true` runs it even without new commits. Set the API key as a secret pipeline variable. Azure caches one path per `Cache@2` task and keys are immutable, so use two tasks with the same rolling-key pattern as the GitHub example.
+
+```yaml
+schedules:
+  - cron: '0 3 * * *'
+    branches:
+      include: [main]
+    always: true
+
+pool:
+  vmImage: ubuntu-latest
+
+steps:
+  - task: NodeTool@0
+    inputs:
+      versionSpec: '24.x'
+  - task: Cache@2
+    inputs:
+      key: 'explorbot | "$(Build.BuildId)"'
+      restoreKeys: 'explorbot'
+      path: experience
+  - task: Cache@2
+    inputs:
+      key: 'explorbot-output | "$(Build.BuildId)"'
+      restoreKeys: 'explorbot-output'
+      path: output
+  - script: |
+      npm ci
+      npx playwright install --with-deps chromium
+      npx explorbot explore / --max-tests 10 --session
+    timeoutInMinutes: 60
+    env:
+      OPENROUTER_API_KEY: $(OPENROUTER_API_KEY)
+  - task: PublishPipelineArtifact@1
+    condition: always()
+    inputs:
+      targetPath: output/reports
+      artifact: explorbot-reports
+```

package/docs/workflow/knowledge.md

@@ -0,0 +1,310 @@
+# Knowledge System
+
+Knowledge files tell explorbot facts about your app. Agents read them to make better decisions about authentication, special workflows, and app-specific behavior.
+
+## Adding Knowledge
+
+### Interactive Mode
+
+```bash
+npx explorbot learn
+```
+
+Opens a TUI form where you can:
+- Enter a URL pattern
+- See existing knowledge for that URL
+- Add new knowledge
+
+### CLI Mode
+
+```bash
+npx explorbot learn "<url-pattern>" "<description>"
+```
+
+Examples:
+
+```bash
+# Login credentials
+npx explorbot learn "/login" "Use credentials: admin@example.com / secret123"
+
+# General knowledge (applies to all pages)
+npx explorbot learn "*" "This is a React SPA. Wait for loading spinners to disappear."
+
+# Specific page behavior
+npx explorbot learn "/checkout" "Credit card field requires format: XXXX-XXXX-XXXX-XXXX"
+```
+
+### Inside TUI
+
+While exploring, use the `/learn` command.
+
+```
+/learn                              # Opens interactive form
+/learn Test user: test@example.com  # Adds to current page
+```
+
+### API Testing
+
+[API testing](../api-testing/basics.md) shares the same `knowledge/` directory. `npx explorbot api know <endpoint> "<description>"` adds endpoint-scoped notes, stored with an `endpoint:` frontmatter field instead of `url:`.
+
+## URL Patterns
+
+| Pattern | Matches |
+|---------|---------|
+| `/login` | Exact path `/login` |
+| `/admin/*` | Any path starting with `/admin/` |
+| `*` | All pages (general knowledge) |
+| `^/users/\d+` | Regex: `/users/` followed by digits |
+| `~dashboard~` | Regex: "dashboard" anywhere in URL (tilde on both sides) |
+
+## Knowledge File Format
+
+Knowledge lives in `./knowledge/` as markdown files with frontmatter:
+
+```markdown
+---
+url: /login
+title: Login Page
+---
+
+Test credentials:
+- email: admin@example.com
+- password: secret123
+
+Notes:
+- Submit button disabled until email validates
+- 3 failed attempts triggers captcha
+- "Remember me" checkbox persists session for 30 days
+```
+
+### Frontmatter Fields
+
+| Field | Purpose |
+|-------|---------|
+| `url` | URL pattern to match (optional, defaults to `*`) |
+| `title` | Human-readable title (optional) |
+| Custom fields | Any additional metadata for agents |
+
+## Variables
+
+Knowledge files support variable interpolation with `${namespace.key}` syntax. Explorbot resolves variables when it loads the knowledge.
+
+### Environment Variables
+
+Use `${env.VARNAME}` to reference environment variables. This keeps secrets out of knowledge files.
+
+```markdown
+---
+url: /login
+---
+
+Login credentials:
+- email: ${env.LOGIN}
+- password: ${env.PASSWORD}
+```
+
+Missing environment variables become an empty string.
+
+### Config Variables
+
+Use `${config.path}` to reference values from `explorbot.config.js` with dot notation.
+
+```markdown
+---
+url: *
+---
+
+Base URL: ${config.playwright.url}
+Browser: ${config.playwright.browser}
+```
+
+You can reference any scalar config value. Object values become an empty string.
+
+### Supported Namespaces
+
+| Namespace | Source | Example |
+|-----------|--------|---------|
+| `env` | `process.env` | `${env.API_KEY}` |
+| `config` | `explorbot.config.js` | `${config.playwright.url}` |
+
+Expressions with an unknown namespace (such as `${other.value}`) or no namespace (such as `${value}`) are left as-is.
+
+## Page Automation
+
+Knowledge files can run automation commands when explorbot navigates to a matching page. Use this for loading states, cookie banners, or page-specific setup.
+
+### Available Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `wait` | `number` | Wait for specified seconds after page load |
+| `waitForElement` | `string` | Wait for element to appear (CSS selector) |
+| `code` | `string` | Execute CodeceptJS code after navigation |
+| `statePush` | `boolean` | Use `history.pushState` instead of full navigation |
+
+### Wait for Page Load
+
+```markdown
+---
+url: /dashboard
+wait: 2
+waitForElement: '.dashboard-loaded'
+---
+
+Dashboard requires data to load before interaction.
+```
+
+### Execute Custom Code
+
+```markdown
+---
+url: /app/*
+code: |
+  I.waitForElement('.app-ready');
+  I.click('.cookie-accept');
+  I.wait(1);
+---
+
+App pages need cookie consent dismissed and loading complete.
+```
+
+### CodeceptJS Effects
+
+Knowledge code can use CodeceptJS effects for error handling and retries:
+
+| Effect | Purpose |
+|--------|---------|
+| `tryTo(fn)` | Execute without failing - returns `true`/`false` |
+| `retryTo(fn, maxTries, interval)` | Retry on failure with polling |
+| `within(context, fn)` | Execute within a specific element context |
+
+**Example with effects:**
+
+```markdown
+---
+url: /dashboard
+code: |
+  await tryTo(() => I.click('.cookie-dismiss'));
+  await retryTo(() => {
+    I.click('Reload Data');
+    I.waitForElement('.data-loaded');
+  }, 5, 500);
+---
+
+Dashboard may show cookie banner. Data loads asynchronously - retry reload if needed.
+```
+
+> [!NOTE]
+> Effects are async. Use `await` when you call them in knowledge code.
+
+### SPA Navigation
+
+For single-page apps where a full reload breaks state:
+
+```markdown
+---
+url: /settings/*
+statePush: true
+---
+
+Settings uses client-side routing. Use pushState to preserve app state.
+```
+
+> [!TIP]
+> Use knowledge automation for page-specific behavior. For agent-specific logic, such as code that runs only during testing, use [Agent Hooks](../web-testing/hooks.md) instead.
+
+### Execution Order
+
+When explorbot navigates to a page, automation runs in this order:
+
+1. Navigation (`I.amOnPage()` or `history.pushState`)
+2. `wait` (if specified)
+3. `waitForElement` (if specified)
+4. `code` (if specified)
+
+## What to Document
+
+### Authentication
+
+```markdown
+---
+url: /login
+---
+
+Credentials: test@example.com / testpass123
+OAuth: Use "Continue with Google" for SSO testing
+2FA: Code is always 123456 in test environment
+```
+
+### Form Behavior
+
+```markdown
+---
+url: /checkout
+---
+
+Required fields: name, email, card number, expiry, CVV
+Card format: XXXX-XXXX-XXXX-XXXX
+Test card: 4111-1111-1111-1111, any future expiry, any CVV
+Promo code "TEST10" gives 10% discount
+```
+
+### Navigation Quirks
+
+```markdown
+---
+url: *
+---
+
+- App uses React Router, wait for route transitions
+- Loading spinner class: .spinner-overlay
+- Modals block interaction until dismissed
+- Session expires after 15 minutes of inactivity
+```
+
+### Test Data
+
+```markdown
+---
+url: /users
+---
+
+Test users available:
+- admin@test.com (admin role)
+- user@test.com (standard user)
+- readonly@test.com (view-only permissions)
+```
+
+## How Agents Use Knowledge
+
+When an agent works on a page, it gets the knowledge whose URL pattern matches:
+
+1. **Navigator** — uses credentials and knows about special interactions
+2. **Researcher** — reads page structure and hidden elements
+3. **Planner** — adds edge cases and validation rules to test scenarios
+4. **Tester** — uses test data and expected behaviors
+
+## Best Practices
+
+1. **Start with auth** — add login credentials before exploring protected areas
+2. **Use `*` for globals** — document app-wide behavior such as loading states and timeouts
+3. **Be specific** — give exact selectors, formats, and values when you know them
+4. **Update as you learn** — add knowledge when agents struggle with an interaction
+
+## File Organization
+
+```
+./knowledge/
+├── login.md           # /login page
+├── checkout.md        # /checkout page
+├── general.md         # * (all pages)
+└── admin_users.md     # /admin/users/*
+```
+
+Files are named after the URL pattern. Multiple entries for the same URL append to the same file.
+
+## See Also
+
+- [Agent Hooks](../web-testing/hooks.md) — per-agent custom code execution
+- [Configuration](../reference/configuration.md) — full configuration reference
+- [Page Interaction](../web-testing/page-interaction.md) — how agents interact with pages

package/docs/workflow/planning-styles.md

@@ -0,0 +1,67 @@
+# Planning Styles
+
+A planning style is a markdown rule file that shapes what scenarios get planned. The whole file becomes the planning approach in the agent's prompt — written as instructions to a QA engineer on how to think about test scenarios.
+
+Two agents use styles through the same mechanism: the web [Planner](../web-testing/planner.md) and the API [Chief](../api-testing/basics.md).
+
+## Built-in Styles
+
+Bundled style files live in `rules/planner/styles/` (web) and `rules/chief/styles/` (API):
+
+| Style | Intent |
+|-------|--------|
+| `normal` | Complete user workflows: CRUD and full commit flows that end in a data or state change |
+| `curious` | Coverage gaps: mines previous test results and research to find paths earlier tests missed |
+| `psycho` | Stress tests: feeds empty, invalid, and extreme values to every reachable control, then commits |
+| `hacker` | API only: probes beyond the spec — hidden endpoints, undocumented fields, unprotected actions |
+
+## Cycling
+
+Each planning iteration uses the next style in the list, cycling by index.
+
+**Web Planner** — default order is `normal`, `curious`, `psycho`. The first `/plan` uses normal, the second curious, the third psycho, then the cycle repeats. Override the list and order with the `styles` option in [Planner configuration](../web-testing/planner.md#configuration); a name may appear more than once.
+
+**API Chief** — order is `normal`, `curious`, `psycho`, `hacker`. `api plan` cycles them the same way. `api explore` doesn't cycle: it runs every style once, generating and executing a plan per style.
+
+## Selecting a Style
+
+Force a style for a single run:
+
+```bash
+npx explorbot plan /users --style psycho        # web
+npx explorbot api plan /users --style hacker    # API
+```
+
+The web TUI accepts the same flag: `/plan --style psycho`. `api explore` has no `--style` flag — it always runs all styles.
+
+## Custom Styles
+
+Styles load from `rules/<agent>/styles/<name>.md` in your project first, falling back to the bundled file of the same name. To edit built-in styles, copy the bundled rules into your project:
+
+```bash
+npx explorbot extract-rules planner   # or: chief
+```
+
+This copies the agent's bundled rule files (including styles) to `./rules/<agent>/`, skipping files that already exist. Edit the copies — they take precedence over the bundled versions. Extract only what you want to change; anything you delete falls back to the bundled file.
+
+Style files are plain markdown with no frontmatter. Write the mindset, the patterns to test, what counts as a test, and what to skip.
+
+To add a new web style, create `rules/planner/styles/<name>.md` and add its name to the rotation:
+
+```javascript
+ai: {
+  agents: {
+    planner: {
+      styles: ['normal', 'curious', 'psycho', 'security'],
+    },
+  },
+}
+```
+
+The API Chief has no `styles` config option — customize it by overriding the built-in style files in `rules/chief/styles/`.
+
+## See Also
+
+- [Planner](../web-testing/planner.md) — how web test plans are generated
+- [API Testing](../api-testing/basics.md) — the Chief agent and API commands
+- [Test Plans](./test-plans.md) — the plan file format