explorbot 0.2.3 → 0.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/bin/explorbot-cli.ts +7 -1
- package/boat/api-tester/src/config.ts +4 -2
- package/boat/doc-collector/bin/doc-collector-cli.ts +2 -0
- package/boat/doc-collector/src/ai/documentarian.ts +61 -31
- package/boat/doc-collector/src/config.ts +4 -2
- package/boat/prima/bin/prima-cli.ts +0 -0
- package/boat/prima/src/activity-line.ts +33 -0
- package/boat/prima/src/cli.ts +115 -86
- package/boat/prima/src/envelope.ts +80 -52
- package/boat/prima/src/prima.ts +547 -128
- package/boat/prima/src/pw-parser.ts +11 -1
- package/boat/prima/src/pw-registry.ts +4 -5
- package/boat/prima/src/session-log.ts +126 -0
- package/dist/bin/explorbot-cli.js +7 -1
- package/dist/boat/api-tester/bin/apibot-cli.js +2 -0
- package/dist/boat/api-tester/src/config.js +4 -2
- package/dist/boat/doc-collector/bin/doc-collector-cli.js +2 -0
- package/dist/boat/doc-collector/src/ai/documentarian.js +44 -19
- package/dist/boat/doc-collector/src/config.js +4 -2
- package/dist/boat/prima/src/activity-line.js +30 -0
- package/dist/boat/prima/src/cli.js +97 -77
- package/dist/boat/prima/src/envelope.js +70 -44
- package/dist/boat/prima/src/prima.js +515 -119
- package/dist/boat/prima/src/pw-parser.js +13 -1
- package/dist/boat/prima/src/pw-registry.js +4 -5
- package/dist/boat/prima/src/session-log.js +108 -0
- package/dist/package.json +3 -2
- package/dist/rules/navigator/verification-actions.md +20 -0
- package/dist/src/action-result.d.ts +7 -0
- package/dist/src/action-result.js +4 -0
- package/dist/src/action.d.ts +2 -0
- package/dist/src/action.js +36 -1
- package/dist/src/ai/captain/web-mode.js +6 -3
- package/dist/src/ai/captain.js +2 -0
- package/dist/src/ai/navigator.d.ts +7 -0
- package/dist/src/ai/navigator.js +10 -6
- package/dist/src/ai/pilot.d.ts +4 -0
- package/dist/src/ai/pilot.js +48 -2
- package/dist/src/ai/provider.js +2 -2
- package/dist/src/ai/rerunner.js +2 -1
- package/dist/src/ai/researcher.js +1 -1
- package/dist/src/ai/rules.js +17 -10
- package/dist/src/ai/task-agent.js +4 -1
- package/dist/src/ai/tester.d.ts +1 -1
- package/dist/src/ai/tester.js +33 -33
- package/dist/src/ai/tools.d.ts +14 -0
- package/dist/src/ai/tools.js +113 -36
- package/dist/src/config.d.ts +1 -0
- package/dist/src/config.js +13 -4
- package/dist/src/execution-controller.d.ts +2 -0
- package/dist/src/execution-controller.js +6 -0
- package/dist/src/explorbot.d.ts +2 -1
- package/dist/src/explorbot.js +3 -1
- package/dist/src/explorer.js +2 -3
- package/dist/src/playwright-recorder.js +30 -0
- package/dist/src/remote.d.ts +54 -0
- package/dist/src/remote.js +229 -0
- package/dist/src/reporter.d.ts +1 -0
- package/dist/src/reporter.js +7 -1
- package/dist/src/state-manager.d.ts +2 -1
- package/dist/src/stats.d.ts +1 -0
- package/dist/src/stats.js +1 -0
- package/dist/src/test-plan.d.ts +2 -0
- package/dist/src/test-plan.js +7 -0
- package/dist/src/utils/aria.d.ts +2 -8
- package/dist/src/utils/aria.js +69 -40
- package/dist/src/utils/html.js +1 -0
- package/dist/src/utils/logger.d.ts +6 -0
- package/dist/src/utils/logger.js +24 -0
- package/dist/src/utils/page-readiness.js +18 -1
- package/dist/src/utils/url-matcher.js +3 -0
- package/dist/src/utils/web-element.d.ts +2 -0
- package/dist/src/utils/web-element.js +8 -0
- package/dist/src/utils/web-sandbox.d.ts +1 -1
- package/dist/src/utils/web-sandbox.js +2 -3
- package/docs/api-testing/basics.md +90 -0
- package/docs/api-testing/planning.md +57 -0
- package/docs/api-testing/running-tests.md +55 -0
- package/docs/assets/cloud-report.png +0 -0
- package/docs/assets/html-report.png +0 -0
- package/docs/assets/langfuse-trace.png +0 -0
- package/docs/assets/successful-explore-run.png +0 -0
- package/docs/basics/getting-started.md +140 -0
- package/docs/basics/prerequisites.md +63 -0
- package/docs/basics/providers.md +362 -0
- package/docs/basics/running.md +78 -0
- package/docs/contributing/ai-integration-tests.md +57 -0
- package/docs/contributing/contributing.md +90 -0
- package/docs/contributing/demo-videos.md +36 -0
- package/docs/contributing/npm-package.md +138 -0
- package/docs/contributing/observability.md +227 -0
- package/docs/contributing/regression-tests.md +103 -0
- package/docs/contributing/testing.md +95 -0
- package/docs/doc-collection/basics.md +128 -0
- package/docs/doc-collection/crawling.md +67 -0
- package/docs/doc-collection/interactive-mode.md +99 -0
- package/docs/index.json +86 -0
- package/docs/reference/commands.md +994 -0
- package/docs/reference/configuration.md +569 -0
- package/docs/reference/scripting.md +303 -0
- package/docs/superpowers/plans/2026-08-01-actor-boat.md +925 -0
- package/docs/superpowers/plans/2026-08-01-prima-boat.md +1120 -0
- package/docs/superpowers/reviews/2026-08-06-prima-vs-playwright-cli.md +268 -0
- package/docs/superpowers/specs/2026-08-01-actor-boat-design.md +204 -0
- package/docs/superpowers/specs/2026-08-01-prima-boat-design.md +242 -0
- package/docs/superpowers/specs/2026-08-03-global-config-design.md +138 -0
- package/docs/superpowers/specs/2026-08-07-prima-fixes-design.md +394 -0
- package/docs/web-testing/agents.md +158 -0
- package/docs/web-testing/automated-tests.md +134 -0
- package/docs/web-testing/basics.md +91 -0
- package/docs/web-testing/customization.md +131 -0
- package/docs/web-testing/hooks.md +238 -0
- package/docs/web-testing/page-interaction.md +84 -0
- package/docs/web-testing/planner.md +122 -0
- package/docs/web-testing/rerun.md +164 -0
- package/docs/web-testing/researcher.md +380 -0
- package/docs/workflow/agentic-usage.md +233 -0
- package/docs/workflow/application-spec.md +73 -0
- package/docs/workflow/ci.md +202 -0
- package/docs/workflow/knowledge.md +310 -0
- package/docs/workflow/planning-styles.md +67 -0
- package/docs/workflow/reporting.md +133 -0
- package/docs/workflow/test-plans.md +90 -0
- package/package.json +3 -2
- package/rules/navigator/verification-actions.md +20 -0
- package/src/action-result.ts +11 -0
- package/src/action.ts +38 -2
- package/src/ai/captain/web-mode.ts +6 -3
- package/src/ai/captain.ts +3 -0
- package/src/ai/navigator.ts +14 -8
- package/src/ai/pilot.ts +53 -2
- package/src/ai/provider.ts +2 -4
- package/src/ai/rerunner.ts +2 -1
- package/src/ai/researcher.ts +1 -1
- package/src/ai/rules.ts +17 -10
- package/src/ai/task-agent.ts +3 -1
- package/src/ai/tester.ts +32 -33
- package/src/ai/tools.ts +132 -36
- package/src/config.ts +15 -4
- package/src/execution-controller.ts +8 -0
- package/src/explorbot.ts +4 -2
- package/src/explorer.ts +2 -2
- package/src/playwright-recorder.ts +23 -0
- package/src/remote.ts +238 -0
- package/src/reporter.ts +7 -1
- package/src/state-manager.ts +2 -1
- package/src/stats.ts +1 -0
- package/src/test-plan.ts +9 -0
- package/src/utils/aria.ts +65 -45
- package/src/utils/html.ts +1 -0
- package/src/utils/logger.ts +24 -1
- package/src/utils/page-readiness.ts +24 -1
- package/src/utils/url-matcher.ts +3 -0
- package/src/utils/web-element.ts +9 -0
- package/src/utils/web-sandbox.ts +3 -4
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# Planner Agent
|
|
2
|
+
|
|
3
|
+
The Planner agent turns Researcher findings into test scenarios. Each scenario has steps, expected outcomes, and a priority, ready for the Tester to run.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
When you run `/plan` or `/explore`, the Planner:
|
|
8
|
+
|
|
9
|
+
1. Receives the Researcher's UI map of the current page.
|
|
10
|
+
2. Applies a planning style.
|
|
11
|
+
3. Generates 3 to 12 scenarios with steps and expected outcomes.
|
|
12
|
+
4. Assigns priorities based on business importance.
|
|
13
|
+
|
|
14
|
+
Run it again and the Planner adds new scenarios in a different style. It skips scenarios that already exist.
|
|
15
|
+
|
|
16
|
+
## Write a useful focus
|
|
17
|
+
|
|
18
|
+
Focus works best when it names one feature boundary, the user goal, and the behavior that matters. Treat it as a testing brief, not a keyword.
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
npx explorbot plan /checkout --focus "Guest checkout: complete an order with card payment; cover validation, declined payment, retry, and confirmation without testing account registration"
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
The same focus works in the TUI:
|
|
25
|
+
|
|
26
|
+
```
|
|
27
|
+
/plan --focus "Guest checkout: complete an order with card payment; cover validation, declined payment, retry, and confirmation without testing account registration"
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
`checkout` alone leaves the scope ambiguous. The fuller focus tells Planner where the flow starts and ends, which outcomes deserve scenarios, and what to leave out. Keep the focus observable from the current page; put durable product facts or credentials in [Knowledge](../workflow/knowledge.md), not in the focus.
|
|
31
|
+
|
|
32
|
+
## Configuration
|
|
33
|
+
|
|
34
|
+
```javascript
|
|
35
|
+
ai: {
|
|
36
|
+
agents: {
|
|
37
|
+
planner: {
|
|
38
|
+
model: groq('gpt-oss-20b'),
|
|
39
|
+
styles: ['normal', 'curious', 'psycho'],
|
|
40
|
+
rules: [
|
|
41
|
+
{ '/checkout/*': 'payment-focus' },
|
|
42
|
+
],
|
|
43
|
+
},
|
|
44
|
+
},
|
|
45
|
+
}
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
### Options Reference
|
|
49
|
+
|
|
50
|
+
| Option | Type | Default | Description |
|
|
51
|
+
|--------|------|---------|-------------|
|
|
52
|
+
| `model` | `LanguageModel` | default model | Override model for Planner |
|
|
53
|
+
| `styles` | `string[]` | `['normal', 'curious', 'psycho']` | Style names and cycling order |
|
|
54
|
+
| `rules` | `RuleEntry[]` | `[]` | URL-aware rule files from `rules/planner/` |
|
|
55
|
+
| `systemPrompt` | `string` | - | Inline instructions appended to the prompt |
|
|
56
|
+
|
|
57
|
+
## Planning Styles
|
|
58
|
+
|
|
59
|
+
A style is a testing approach that shapes which tests the Planner creates. Styles cycle on each planning iteration, so repeated runs produce different kinds of tests. See [Planning Styles](../workflow/planning-styles.md) for how cycling works, custom style files, and `extract-rules`.
|
|
60
|
+
|
|
61
|
+
### Built-in Styles
|
|
62
|
+
|
|
63
|
+
All three built-in styles rank scenarios by outcome strength, from strongest to weakest:
|
|
64
|
+
|
|
65
|
+
1. **Data change** — a record is created, edited, deleted; a setting is persisted; a message is sent; a job is triggered.
|
|
66
|
+
2. **State change** — a route change, a filter or sort applied to real data, a mode or auth change the app remembers.
|
|
67
|
+
3. **UI-only change** — something opens, closes, is cancelled, is hovered, or is toggled for display. The application registers nothing new.
|
|
68
|
+
|
|
69
|
+
The Planner prefers scenarios that end in category 1 or 2. It proposes category 3 only when the UI-only behaviour has a verifiable side effect, such as a warning prompt, a persisted draft, or a badge appearing.
|
|
70
|
+
|
|
71
|
+
| Style | Focus | What it generates |
|
|
72
|
+
|-------|-------|-------------------|
|
|
73
|
+
| **normal** | Complete user workflows | CRUD operations, full commit flows, filter+verify flows, distributed across feature areas. UI-only tests (tab switching, pagination, view toggles) come last. |
|
|
74
|
+
| **curious** | Coverage gaps | Cross-references previous test results with page research to find untested controls. Variation scenarios and dismissal scenarios are kept separate — the planner will not merge them by appending a cancel at the end. |
|
|
75
|
+
| **psycho** | Invalid and extreme inputs | Attacks **every reachable control in the same scenario** with a different strange value — empty, 10000 chars, unicode, SQL, script tags, invalid formats, conflicting toggles, out-of-range dates — then commits. Scenarios that enter bad data and cancel are rejected: the application never received the payload. |
|
|
76
|
+
|
|
77
|
+
### Style Cycling
|
|
78
|
+
|
|
79
|
+
The default cycle is: normal, curious, psycho, then back to normal. The 1st `/plan` uses normal, the 2nd curious, the 3rd psycho. Each iteration proposes only scenarios that aren't already in the plan. When all feature areas are covered, the Planner returns an empty list.
|
|
80
|
+
|
|
81
|
+
Force a specific style:
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
/plan --style psycho
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Set `styles` in the config to control the rotation. To customize or add style files, see [Planning Styles](../workflow/planning-styles.md).
|
|
88
|
+
|
|
89
|
+
## Page-Specific Rules
|
|
90
|
+
|
|
91
|
+
Use [rules](../reference/configuration.md#rules) to give the Planner extra instructions for specific pages:
|
|
92
|
+
|
|
93
|
+
```javascript
|
|
94
|
+
planner: {
|
|
95
|
+
rules: [
|
|
96
|
+
'no-delete-tests', // rules/planner/no-delete-tests.md — all pages
|
|
97
|
+
{ '/checkout/*': 'payment-rules' }, // rules/planner/payment-rules.md — checkout only
|
|
98
|
+
{ '/admin/*': 'admin-scenarios' }, // rules/planner/admin-scenarios.md — admin pages
|
|
99
|
+
],
|
|
100
|
+
}
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Rules are additive. The Planner concatenates all matching rules and appends them to its prompt alongside the active style.
|
|
104
|
+
|
|
105
|
+
## Test Priorities
|
|
106
|
+
|
|
107
|
+
The Planner assigns priorities by business importance:
|
|
108
|
+
|
|
109
|
+
| Priority | Meaning | Examples |
|
|
110
|
+
|----------|---------|---------|
|
|
111
|
+
| **critical** | Core business functionality | Login, checkout, primary CRUD |
|
|
112
|
+
| **important** | Key user flows | Profile edit, search, main filters |
|
|
113
|
+
| **high** | Secondary features | Edge cases for critical flows |
|
|
114
|
+
| **normal** | Supporting actions | Settings, configuration |
|
|
115
|
+
| **low** | Minor interactions | Cosmetic checks, boundary tests |
|
|
116
|
+
|
|
117
|
+
## See Also
|
|
118
|
+
|
|
119
|
+
- [Planning Styles](../workflow/planning-styles.md) — cycling, custom styles, `extract-rules`
|
|
120
|
+
- [Configuration: Rules](../reference/configuration.md#rules) — URL-aware rule files
|
|
121
|
+
- [Agents](./agents.md) — all agent descriptions
|
|
122
|
+
- [Commands](../reference/commands.md) — CLI and TUI commands
|
|
@@ -0,0 +1,164 @@
|
|
|
1
|
+
# Rerunning Tests
|
|
2
|
+
|
|
3
|
+
Explorbot generates CodeceptJS test files in `output/tests/` after an exploration session. Use the `runs` and `rerun` commands to list, inspect, and re-execute these tests with AI healing.
|
|
4
|
+
|
|
5
|
+
## Workflow
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
npx explorbot explore /dashboard # generates output/tests/dashboard.js
|
|
9
|
+
npx explorbot runs # list all generated tests with indices
|
|
10
|
+
npx explorbot runs output/tests/dashboard.js # preview steps (dry-run)
|
|
11
|
+
npx explorbot rerun output/tests/dashboard.js --session # run with healing
|
|
12
|
+
npx explorbot rerun output/tests/dashboard.js 3 --session # run test #3 only
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## Listing tests
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
npx explorbot runs
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
Lists all generated test files with numbered scenarios:
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
Dashboard Testing
|
|
25
|
+
output/tests/dashboard_testing.js
|
|
26
|
+
1. ❯ Create a new item
|
|
27
|
+
2. ─ Delete an item (skipped)
|
|
28
|
+
3. ❯ Edit item title
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Active tests show `❯`, skipped tests show `─`.
|
|
32
|
+
|
|
33
|
+
### Dry-run a file
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
npx explorbot runs output/tests/dashboard_testing.js
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Prints the CodeceptJS steps each test runs, including `Before` hooks, without launching a browser. Use it to check what a test does before running it.
|
|
40
|
+
|
|
41
|
+
## Re-running tests
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
npx explorbot rerun <file> [index] [--session]
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Runs tests through CodeceptJS. The Rerunner agent heals steps that fail.
|
|
48
|
+
|
|
49
|
+
### Index selection
|
|
50
|
+
|
|
51
|
+
| Syntax | Meaning |
|
|
52
|
+
|--------|---------|
|
|
53
|
+
| _(no index)_ | Run all tests in the file |
|
|
54
|
+
| `3` | Run test #3 only |
|
|
55
|
+
| `1-5` | Run tests 1 through 5 |
|
|
56
|
+
| `1,3,7` | Run specific tests |
|
|
57
|
+
|
|
58
|
+
Indices match the numbers shown by `npx explorbot runs`.
|
|
59
|
+
|
|
60
|
+
### What gets skipped
|
|
61
|
+
|
|
62
|
+
- **Scenario.skip / Scenario.todo** — already marked as skipped in the test file.
|
|
63
|
+
- **Tests without assertions** — tests with no `I.see`, `I.seeElement`, `I.dontSee`, or similar are skipped, because their results can't be verified.
|
|
64
|
+
|
|
65
|
+
## AI healing
|
|
66
|
+
|
|
67
|
+
When a step fails, the Rerunner agent diagnoses the problem and tries to fix it. It uses the same tools as the Tester agent: click, form, pressKey, xpathCheck, see, research, and bash.
|
|
68
|
+
|
|
69
|
+
### How it works
|
|
70
|
+
|
|
71
|
+
1. A step like `I.click("Save", ".modal")` fails.
|
|
72
|
+
2. The healer receives the current page state (URL, ARIA tree) and trace data.
|
|
73
|
+
3. It diagnoses the cause: wrong page, loading, broken locator, or missing data.
|
|
74
|
+
4. It runs a replacement action, for example waiting for the page to load and then clicking with a corrected locator.
|
|
75
|
+
5. On success, the test continues. On failure, it moves to the next test.
|
|
76
|
+
|
|
77
|
+
### Healing boundaries
|
|
78
|
+
|
|
79
|
+
AI healing repairs how an existing step reaches the same intended outcome. It is deliberately not a second exploratory test run.
|
|
80
|
+
|
|
81
|
+
| Healing can | Healing does not |
|
|
82
|
+
|-------------|------------------|
|
|
83
|
+
| Replace a stale or ambiguous locator | Change the scenario's business intent |
|
|
84
|
+
| Wait for loading or dismiss a blocking transient UI | Turn a failed assertion into a pass |
|
|
85
|
+
| Restore expected navigation or repeat an equivalent interaction | Invent missing credentials, permissions, or test data |
|
|
86
|
+
| Adapt to a small UI structure change | Work around a real product defect or unavailable service |
|
|
87
|
+
|
|
88
|
+
The healer only has `healLimit` attempts per test and `healMaxIterations` AI iterations per failed step. It should stop when the expected element or state no longer exists, required data is missing, access is denied, or the application behavior contradicts the assertion. In those cases the test remains failed and the trace preserves the evidence for review.
|
|
89
|
+
|
|
90
|
+
Successful healing updates the generated test file with the replacement step. Review that diff before committing it: healing shows that an equivalent interaction worked now, not that every UI change is safe or intentional.
|
|
91
|
+
|
|
92
|
+
### Healing output
|
|
93
|
+
|
|
94
|
+
During healing, the agent's actions show as substeps:
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
❯ Create a new test
|
|
98
|
+
I.amOnPage("/projects/testcaselabs/")
|
|
99
|
+
I.click("Test", ".sticky-header .first")
|
|
100
|
+
→ Healing: I.click("Test", ".sticky-header .first")
|
|
101
|
+
> ✔ Detected loading spinner, waiting
|
|
102
|
+
> ✔ Click Test button in toolbar
|
|
103
|
+
✔ Healed: I.click({"role":"button","text":"Test"})
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### Healing configuration
|
|
107
|
+
|
|
108
|
+
Configure the Rerunner agent in `explorbot.config.js`:
|
|
109
|
+
|
|
110
|
+
```javascript
|
|
111
|
+
export default {
|
|
112
|
+
ai: {
|
|
113
|
+
agents: {
|
|
114
|
+
rerunner: {
|
|
115
|
+
healLimit: 3, // max heals per test (default: 3)
|
|
116
|
+
healMaxIterations: 3, // max AI loop iterations per heal (default: 3)
|
|
117
|
+
recipes: {
|
|
118
|
+
// Custom healing recipes (CodeceptJS heal API)
|
|
119
|
+
waitForLoader: {
|
|
120
|
+
steps: ['amOnPage'],
|
|
121
|
+
fn: async () => {
|
|
122
|
+
return async ({ I }) => {
|
|
123
|
+
await I.waitForInvisible('.loader', 20);
|
|
124
|
+
};
|
|
125
|
+
},
|
|
126
|
+
},
|
|
127
|
+
},
|
|
128
|
+
},
|
|
129
|
+
},
|
|
130
|
+
},
|
|
131
|
+
};
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### Custom rules
|
|
135
|
+
|
|
136
|
+
Healing prompt rules live in `rules/rerunner/` as markdown files:
|
|
137
|
+
|
|
138
|
+
| File | Purpose |
|
|
139
|
+
|------|---------|
|
|
140
|
+
| `healing-approach.md` | Step-by-step diagnosis strategy |
|
|
141
|
+
|
|
142
|
+
To override a rule, place a file with the same name in your project's `rules/rerunner/` directory.
|
|
143
|
+
|
|
144
|
+
## Trace output
|
|
145
|
+
|
|
146
|
+
Each rerun creates a trace directory under `output/states/rerun_<timestamp>/` with per-test artifacts:
|
|
147
|
+
|
|
148
|
+
- `trace.md` — execution timeline with links to all artifacts
|
|
149
|
+
- `*_aria.txt` — ARIA snapshot per step
|
|
150
|
+
- `*_page.html` — full HTML per step
|
|
151
|
+
- `*_screenshot.png` — screenshot per step
|
|
152
|
+
- `*_console.json` — browser console logs per step
|
|
153
|
+
|
|
154
|
+
The healer reads these files to diagnose failures.
|
|
155
|
+
|
|
156
|
+
## After exploration
|
|
157
|
+
|
|
158
|
+
After `/explore` or `npx explorbot explore`, Explorbot shows the generated test files with rerun suggestions:
|
|
159
|
+
|
|
160
|
+
```
|
|
161
|
+
Generated: test_management.js
|
|
162
|
+
List tests: npx explorbot runs
|
|
163
|
+
Re-run with healing: npx explorbot rerun <filename> [index]
|
|
164
|
+
```
|
|
@@ -0,0 +1,380 @@
|
|
|
1
|
+
# Researcher Agent
|
|
2
|
+
|
|
3
|
+
The Researcher analyzes web pages to understand their structure, identify UI elements, and build UI maps with locators.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
Research gives other agents the context they need:
|
|
8
|
+
|
|
9
|
+
- **Planner** — calls the Researcher before generating test scenarios
|
|
10
|
+
- **Tester** — uses research results to understand page context during execution
|
|
11
|
+
|
|
12
|
+
You can also run research manually to inspect pages or debug locator issues.
|
|
13
|
+
|
|
14
|
+
## Configuration
|
|
15
|
+
|
|
16
|
+
> [!IMPORTANT]
|
|
17
|
+
> The Researcher processes large amounts of HTML and ARIA tokens on every call. Use a **fast, cheap model** — it does not need deep thinking, just accurate element extraction. Models like `gpt-oss-20b` via Groq or Cerebras at 100+ TPS work well. On reasoning models the Researcher runs at low reasoning effort by default — see [Reasoning Effort](#reasoning-effort).
|
|
18
|
+
|
|
19
|
+
```javascript
|
|
20
|
+
ai: {
|
|
21
|
+
agents: {
|
|
22
|
+
researcher: {
|
|
23
|
+
model: groq('openai/gpt-oss-20b'),
|
|
24
|
+
systemPrompt: 'Focus on form validation elements...',
|
|
25
|
+
sections: ['overlay', 'content', 'list'],
|
|
26
|
+
maxExpandableClicks: 10,
|
|
27
|
+
retries: 2,
|
|
28
|
+
},
|
|
29
|
+
},
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### Options Reference
|
|
34
|
+
|
|
35
|
+
| Option | Type | Default | Description |
|
|
36
|
+
|--------|------|---------|-------------|
|
|
37
|
+
| `model` | `string` | - | Override the default model for the Researcher |
|
|
38
|
+
| `systemPrompt` | `string` | - | Extra instructions appended to the research prompt |
|
|
39
|
+
| `sections` | `string[]` | all sections | Page sections to identify (order = priority) |
|
|
40
|
+
| `focusSections` | `string[]` | `[]` | CSS selectors used in the truncated-response fallback, when research is split into per-section requests (first match wins). The matching element becomes the container for the focused section — see [Handling Truncated Responses](#handling-truncated-responses). |
|
|
41
|
+
| `maxExpandableClicks` | `number` | `10` | Max expandable elements clicked during deep research |
|
|
42
|
+
| `errorPageTimeout` | `number` | `10` | Seconds to wait for the page to settle before research; error pages detected during this wait abort research. Set `0` to skip the wait |
|
|
43
|
+
| `retries` | `number` | `2` | Retries when most locators are broken in Stage 2 |
|
|
44
|
+
| `reasoning` | `string` | `'low'` | AI SDK v7 reasoning effort: `'none'`, `'minimal'`, `'low'`, `'medium'`, `'high'`, `'xhigh'`, `'provider-default'` |
|
|
45
|
+
| `providerOptions` | `object` | - | Provider-specific options. Reasoning keys here take precedence over `reasoning`. |
|
|
46
|
+
|
|
47
|
+
See [Configuration Examples](#configuration-examples) at the end of this page for common setups.
|
|
48
|
+
|
|
49
|
+
## Usage
|
|
50
|
+
|
|
51
|
+
### CLI Mode
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
# Research a specific path (relative to url in config)
|
|
55
|
+
npx explorbot research /login
|
|
56
|
+
npx explorbot research /admin/users
|
|
57
|
+
|
|
58
|
+
# Research with options
|
|
59
|
+
npx explorbot research /dashboard --deep
|
|
60
|
+
npx explorbot research /products --data
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### TUI Mode (Interactive)
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
# Research current page
|
|
67
|
+
/research
|
|
68
|
+
|
|
69
|
+
# Research with deep expansion (clicks dropdowns, tabs, etc.)
|
|
70
|
+
/research --deep
|
|
71
|
+
|
|
72
|
+
# Research with data extraction
|
|
73
|
+
/research --data
|
|
74
|
+
|
|
75
|
+
# Research specific path
|
|
76
|
+
/research /login
|
|
77
|
+
/research /admin/pages
|
|
78
|
+
|
|
79
|
+
# Skip locator validation and fixing
|
|
80
|
+
/research --no-fix
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Explicit research always runs fresh (bypassing the [cache](#caching)) and always captures a screenshot.
|
|
84
|
+
|
|
85
|
+
### Automatic Research
|
|
86
|
+
|
|
87
|
+
Research also runs as part of other commands:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
# Planner researches the page before planning
|
|
91
|
+
npx explorbot plan /dashboard
|
|
92
|
+
|
|
93
|
+
# Explorer researches each new page it discovers
|
|
94
|
+
npx explorbot explore /admin
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## How It Works
|
|
98
|
+
|
|
99
|
+
### Element Indexing (eidx)
|
|
100
|
+
|
|
101
|
+
Before research begins, Explorbot injects a `data-explorbot-eidx` attribute into every interactive element on the page (buttons, links, inputs, tabs, and so on). Each element gets a unique numeric index — its **eidx**.
|
|
102
|
+
|
|
103
|
+
The eidx is a stable bridge between three representations of the same element:
|
|
104
|
+
|
|
105
|
+
| Representation | What it provides | Where eidx appears |
|
|
106
|
+
|----------------|------------------|--------------------|
|
|
107
|
+
| **HTML** | Structure, attributes, CSS selectors | `<button eidx="5">Save</button>` |
|
|
108
|
+
| **ARIA tree** | Accessible roles, names | Mapped back via Playwright `getByRole` |
|
|
109
|
+
| **Screenshot** | Visual position, color, icon | Colored label `5` drawn above the element |
|
|
110
|
+
|
|
111
|
+
When the AI produces a research table with `eidx=5`, that index is used to:
|
|
112
|
+
- Test the element's CSS locator against the live DOM
|
|
113
|
+
- Look up its coordinates from the annotated screenshot
|
|
114
|
+
- Generate a fallback XPath if CSS is broken
|
|
115
|
+
|
|
116
|
+
Without eidx, there would be no reliable way to match "the third button in the HTML" with "the blue button at (400, 300) on the screenshot."
|
|
117
|
+
|
|
118
|
+
### The 5-Stage Pipeline
|
|
119
|
+
|
|
120
|
+
Research runs each page through five stages:
|
|
121
|
+
|
|
122
|
+
| Stage | Name | What happens |
|
|
123
|
+
|-------|------|--------------|
|
|
124
|
+
| 1 | **Research** (AI) | AI analyzes HTML and ARIA, produces a UI map with sections, containers, ARIA locators, CSS locators, and eidx references |
|
|
125
|
+
| 2 | **Test** | Test containers first, then element locators. Capture exact counts (`0 elements`, `3 elements`, `dynamic ID`). If all containers are broken or more than 80% of locators are broken, retry Stage 1 |
|
|
126
|
+
| 3 | **Fix** (AI, same conversation) | Continue the Stage 1 conversation with Playwright test results. AI fixes broken locators with full page context |
|
|
127
|
+
| 4 | **Visual** (optional) | Annotate the screenshot with eidx labels. AI extracts coordinates, colors, and icons. Merge into research by eidx |
|
|
128
|
+
| 5 | **Backfill** | Re-test all locators. For still-broken elements, look up the eidx in the DOM and generate an XPath from attributes. Nullify containers that are still broken |
|
|
129
|
+
|
|
130
|
+
Stage 3 reuses the Stage 1 conversation. The AI already has full context about the page HTML, so it fixes locators more accurately without extra token cost.
|
|
131
|
+
|
|
132
|
+
### Research Modes
|
|
133
|
+
|
|
134
|
+
#### Standard Research (HTML + ARIA)
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
/research
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Analyzes the page using HTML and the ARIA tree. Fast, and works with any model.
|
|
141
|
+
|
|
142
|
+
#### Deep Research
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
/research --deep
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
Expands hidden elements (dropdowns, accordions, tabs) to discover more UI. Clicks through interactive elements and documents what appears. Deep research also reuses what it found on previous runs — see [Reusing Previous Results](#reusing-previous-results).
|
|
149
|
+
|
|
150
|
+
#### Research with Data Extraction
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
/research --data
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Extracts domain-specific content (articles, products, users) as structured data.
|
|
157
|
+
|
|
158
|
+
## Page Sections
|
|
159
|
+
|
|
160
|
+
The Researcher breaks each page into sections by UI purpose. Sections are identified in priority order:
|
|
161
|
+
|
|
162
|
+
| Section | Description |
|
|
163
|
+
|---------|-------------|
|
|
164
|
+
| `overlay` | Dialog, modal, drawer, popup, or active form overlay |
|
|
165
|
+
| `list` | List area (items collection, table, cards, or list view) |
|
|
166
|
+
| `detail` | Detail area (selected item preview or full details) |
|
|
167
|
+
| `panes` | Screen is split into equal panes |
|
|
168
|
+
| `content` | Main area of the page |
|
|
169
|
+
| `menu` | Page menu (toolbar, context actions, filters, dropdowns) |
|
|
170
|
+
| `navigation` | Main navigation (top bar, sidebar, breadcrumbs) |
|
|
171
|
+
|
|
172
|
+
Each section includes:
|
|
173
|
+
- A **container CSS selector** that scopes all elements within it
|
|
174
|
+
- A **UI map table** listing interactive elements with ARIA and CSS locators
|
|
175
|
+
|
|
176
|
+
Override the default section list via `ai.agents.researcher.sections` — see [Configuration](#configuration).
|
|
177
|
+
|
|
178
|
+
## Vision Model Support
|
|
179
|
+
|
|
180
|
+
### Without Vision
|
|
181
|
+
|
|
182
|
+
The Researcher works with text-only models by analyzing HTML structure, the ARIA tree, and element roles and names. This is enough for most pages, and it is faster and cheaper.
|
|
183
|
+
|
|
184
|
+
### With Vision
|
|
185
|
+
|
|
186
|
+
With a vision model configured, the Researcher can analyze screenshots for visual elements, detect icons and visual indicators, and provide element coordinates for visual clicking.
|
|
187
|
+
|
|
188
|
+
Enable vision by configuring a vision model instance:
|
|
189
|
+
|
|
190
|
+
```javascript
|
|
191
|
+
ai: {
|
|
192
|
+
visionModel: openai('gpt-4o'),
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Explicit research always captures a screenshot; when a vision model is configured, the screenshot is analyzed in Stage 4.
|
|
197
|
+
|
|
198
|
+
Vision helps most on pages with icon-only buttons, canvas-based UIs, and when the HTML doesn't reflect the visual layout.
|
|
199
|
+
|
|
200
|
+
## Deep Exploration
|
|
201
|
+
|
|
202
|
+
Deep exploration (the `--deep` flag) discovers hidden UI by clicking through elements to find modals, dropdowns, tabs, and menus.
|
|
203
|
+
|
|
204
|
+
For each element, the Researcher:
|
|
205
|
+
1. Captures state before the click
|
|
206
|
+
2. Clicks the element
|
|
207
|
+
3. Detects what changed (navigation, modal, menu, UI change)
|
|
208
|
+
4. Restores the original state (Escape key or navigate back)
|
|
209
|
+
|
|
210
|
+
### Reusing Previous Results
|
|
211
|
+
|
|
212
|
+
Hidden sections discovered by deep research are saved under an **Extended Research** block in the page's research file, together with the action that revealed each one. On the next deep run for the same page — even in a later session — the researcher builds on that instead of starting from scratch:
|
|
213
|
+
|
|
214
|
+
1. **Replay** — it re-runs the saved action for every previously found section to check it still opens.
|
|
215
|
+
2. **Reuse** — sections that still open are kept as-is and are not explored again, so the run spends its click budget on what is actually new.
|
|
216
|
+
3. **Re-discover** — if a section's trigger no longer works (the button moved or was renamed), it is flagged to the AI as "this section existed before, find it again", so a relocated control is recovered rather than lost.
|
|
217
|
+
4. **Skip** — when every known section still opens and the click budget is already covered, the slow click-through exploration is skipped because the page is effectively unchanged.
|
|
218
|
+
|
|
219
|
+
This makes repeated deep runs faster and stops the researcher from silently losing hidden UI it had already mapped. The reuse reads the last saved research file directly, so it works across sessions and is not limited by the in-memory [cache window](#caching).
|
|
220
|
+
|
|
221
|
+
### Selecting Elements
|
|
222
|
+
|
|
223
|
+
Not every element should be explored. During deep analysis the AI itself discovers expandable candidates from the research results — and from the annotated screenshot when a vision model is configured — picking elements that hide content until clicked (menus, dropdowns, accordions, tabs) and skipping regular links and navigation. Repeated controls, like the same expand button on every list row, collapse to a single representative. When more candidates are found than the click budget allows, the AI selects the most promising ones. The budget is set by `maxExpandableClicks` (default 10).
|
|
224
|
+
|
|
225
|
+
## Output Format
|
|
226
|
+
|
|
227
|
+
Research results are saved to `output/research/{hash}.md`:
|
|
228
|
+
|
|
229
|
+
```markdown
|
|
230
|
+
## Summary
|
|
231
|
+
|
|
232
|
+
Brief description of the page purpose.
|
|
233
|
+
|
|
234
|
+
## Login Modal
|
|
235
|
+
|
|
236
|
+
Modal dialog for user login...
|
|
237
|
+
|
|
238
|
+
> Container: '[role="dialog"]'
|
|
239
|
+
> **Focused**
|
|
240
|
+
|
|
241
|
+
| Element | Type | ARIA | CSS |
|
|
242
|
+
|---------|------|------|-----|
|
|
243
|
+
| 'Email' | textbox | { role: 'textbox', text: 'Email' } | 'input#email' |
|
|
244
|
+
| 'Password' | textbox | { role: 'textbox', text: 'Password' } | 'input[name="password"]' |
|
|
245
|
+
| 'Sign In' | button | { role: 'button', text: 'Sign In' } | 'button[type="submit"]' |
|
|
246
|
+
|
|
247
|
+
## Content Section
|
|
248
|
+
|
|
249
|
+
Main content area...
|
|
250
|
+
|
|
251
|
+
> Container: '.main-content'
|
|
252
|
+
|
|
253
|
+
| Element | Type | ARIA | CSS | XPath | Coordinates |
|
|
254
|
+
|---------|------|------|-----|-------|-------------|
|
|
255
|
+
| 'Save' | button | { role: 'button', text: 'Save' } | 'button.save' | - | (400, 300) |
|
|
256
|
+
| 'Delete' | button | { role: 'button', text: 'Delete' } | - | '//button[@class="del"]' | (500, 300) |
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Notes:
|
|
260
|
+
- Sections are named after their content (never "Focus"); a focused overlay is marked with a `> **Focused**` blockquote under its container line
|
|
261
|
+
- The Type column is derived from the ARIA role during cleanup
|
|
262
|
+
- The XPath column appears only when CSS is broken and XPath was backfilled from the DOM
|
|
263
|
+
- Coordinates are backfilled from DOM positions for all indexed (eidx) elements; a vision model additionally contributes colors and icons
|
|
264
|
+
- The container is shown as a blockquote `> Container: '...'` before the table
|
|
265
|
+
|
|
266
|
+
## Caching
|
|
267
|
+
|
|
268
|
+
Research results are cached for 6 hours:
|
|
269
|
+
- In memory during the session
|
|
270
|
+
- On disk in `output/research/`
|
|
271
|
+
|
|
272
|
+
Separately, for up to 1 hour a page whose HTML fingerprint is at least 90% similar to an already-researched state reuses that state's research.
|
|
273
|
+
|
|
274
|
+
The cache applies to research triggered automatically by other agents. Explicit `/research` always bypasses it and runs fresh.
|
|
275
|
+
|
|
276
|
+
This cache controls when a fresh result is reused within a session. It is separate from how [deep research reuses previous results](#reusing-previous-results): a deep run always reloads the last saved research file from `output/research/` to replay and verify previously discovered hidden sections, regardless of the cache window or session.
|
|
277
|
+
|
|
278
|
+
## Configuration Examples
|
|
279
|
+
|
|
280
|
+
### Limit Sections
|
|
281
|
+
|
|
282
|
+
```javascript
|
|
283
|
+
ai: {
|
|
284
|
+
agents: {
|
|
285
|
+
researcher: {
|
|
286
|
+
// Only research these sections, skip navigation and menu
|
|
287
|
+
sections: ['overlay', 'content', 'list', 'detail'],
|
|
288
|
+
},
|
|
289
|
+
},
|
|
290
|
+
}
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
### Focus on a Single Element
|
|
294
|
+
|
|
295
|
+
`focusSections` applies when a truncated response forces the Researcher into per-section research (see [Handling Truncated Responses](#handling-truncated-responses)). It is a list of CSS selectors — the first one that matches on the current page wins, and the split research treats that element as the focused container instead of the whole page. Useful for apps that open a modal, drawer, or detail panel on top of the main layout:
|
|
296
|
+
|
|
297
|
+
```javascript
|
|
298
|
+
ai: {
|
|
299
|
+
agents: {
|
|
300
|
+
researcher: {
|
|
301
|
+
focusSections: [
|
|
302
|
+
'[role="dialog"]', // open modal
|
|
303
|
+
'.drawer-open', // expanded side drawer
|
|
304
|
+
'#focused-panel', // your app's detail panel
|
|
305
|
+
],
|
|
306
|
+
},
|
|
307
|
+
},
|
|
308
|
+
}
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
When none of the selectors match, per-section research covers the whole page.
|
|
312
|
+
|
|
313
|
+
### Handling Truncated Responses
|
|
314
|
+
|
|
315
|
+
The Researcher produces a lot of output for busy pages. If the model's response is cut off at `maxOutputTokens`, Explorbot retries by splitting the work into one request per section (focus, main, sidebar, and so on) and merging the results. This usually happens transparently in the logs; no configuration is needed.
|
|
316
|
+
|
|
317
|
+
If you see it often, consider:
|
|
318
|
+
- lowering reasoning effort (see [Reasoning Effort](#reasoning-effort) below),
|
|
319
|
+
- pinning the Researcher to a non-reasoning model with a larger output window,
|
|
320
|
+
- or narrowing the scope with `focusSections`.
|
|
321
|
+
|
|
322
|
+
### Custom Component Guidance
|
|
323
|
+
|
|
324
|
+
```javascript
|
|
325
|
+
ai: {
|
|
326
|
+
agents: {
|
|
327
|
+
researcher: {
|
|
328
|
+
systemPrompt: `
|
|
329
|
+
This app uses custom components:
|
|
330
|
+
- <DataGrid> renders as div with role="grid"
|
|
331
|
+
- <Modal> uses [data-modal] attribute
|
|
332
|
+
- Dropdowns have [data-dropdown] attribute
|
|
333
|
+
|
|
334
|
+
Look for data-testid attributes for reliable selectors.
|
|
335
|
+
`,
|
|
336
|
+
},
|
|
337
|
+
},
|
|
338
|
+
}
|
|
339
|
+
```
|
|
340
|
+
|
|
341
|
+
### Reasoning Effort
|
|
342
|
+
|
|
343
|
+
The Researcher runs with `reasoning: 'low'` by default. On reasoning models this keeps the chain-of-thought short, so the output budget goes to the UI map instead of thinking tokens. `reasoning` is the provider-agnostic setting from AI SDK v7 — the SDK maps it to the active provider's effort control, so the same value works across OpenAI, Anthropic, Google, Groq, and others:
|
|
344
|
+
|
|
345
|
+
```javascript
|
|
346
|
+
ai: {
|
|
347
|
+
agents: {
|
|
348
|
+
researcher: {
|
|
349
|
+
reasoning: 'none', // 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'provider-default'
|
|
350
|
+
},
|
|
351
|
+
},
|
|
352
|
+
}
|
|
353
|
+
```
|
|
354
|
+
|
|
355
|
+
For provider-specific control (such as an exact thinking-token budget), set the provider's own keys in `providerOptions` — they take precedence over `reasoning`.
|
|
356
|
+
|
|
357
|
+
If heavy pages still truncate the response (`AI response empty: output truncated at maxTokens`), lower `reasoning` to `'none'`, raise `maxOutputTokens`, or pin the Researcher to a non-reasoning model.
|
|
358
|
+
|
|
359
|
+
### Vision-Heavy Research
|
|
360
|
+
|
|
361
|
+
```javascript
|
|
362
|
+
ai: {
|
|
363
|
+
agents: {
|
|
364
|
+
researcher: {
|
|
365
|
+
systemPrompt: `
|
|
366
|
+
Pay attention to:
|
|
367
|
+
- Icon buttons without text labels
|
|
368
|
+
- Color indicators (red = error, green = success)
|
|
369
|
+
- Visual hierarchy and spacing
|
|
370
|
+
`,
|
|
371
|
+
},
|
|
372
|
+
},
|
|
373
|
+
}
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
## See Also
|
|
377
|
+
|
|
378
|
+
- [Configuration](../reference/configuration.md) - general configuration options
|
|
379
|
+
- [Agents](./agents.md) - all agent descriptions
|
|
380
|
+
- [Knowledge Files](../workflow/knowledge.md) - domain-specific hints
|