explorbot 0.1.28 → 0.1.29
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +83 -245
- package/bin/explorbot-cli.ts +1 -0
- package/dist/bin/explorbot-cli.js +1 -0
- package/dist/package.json +8 -6
- package/dist/rules/navigator/verification-actions.md +2 -0
- package/dist/src/ai/fisherman.js +14 -3
- package/dist/src/ai/pilot.js +19 -4
- package/dist/src/ai/planner.js +16 -5
- package/dist/src/ai/provider.js +53 -18
- package/dist/src/ai/researcher.js +7 -1
- package/dist/src/ai/rules.js +44 -0
- package/dist/src/ai/tester.js +70 -7
- package/dist/src/ai/tools.js +67 -1
- package/dist/src/explorbot.js +7 -2
- package/dist/src/stats.js +16 -0
- package/dist/src/utils/aria.js +66 -6
- package/package.json +8 -6
- package/rules/navigator/verification-actions.md +2 -0
- package/src/ai/fisherman.ts +14 -3
- package/src/ai/pilot.ts +19 -4
- package/src/ai/planner.ts +16 -5
- package/src/ai/provider.ts +51 -19
- package/src/ai/researcher.ts +8 -1
- package/src/ai/rules.ts +46 -0
- package/src/ai/tester.ts +74 -7
- package/src/ai/tools.ts +80 -1
- package/src/config.ts +1 -0
- package/src/explorbot.ts +6 -2
- package/src/stats.ts +18 -0
- package/src/utils/aria.ts +63 -6
package/README.md
CHANGED
|
@@ -1,336 +1,178 @@
|
|
|
1
|
-
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="assets/explorbot-logo.png" alt="Explorbot" width="560">
|
|
3
|
+
</p>
|
|
2
4
|
|
|
3
|
-
|
|
5
|
+
<p align="center"><b>The vibe-testing agent for web applications.</b></p>
|
|
4
6
|
|
|
5
7
|

|
|
6
8
|
|
|
7
|
-
Explorbot
|
|
9
|
+
Explorbot is an AI agent that investigates your product like your most relentless QA engineer — clicking around, filling forms, and finding bugs. It turns every discovery into a test you can keep. No test scripts required. Just point it at your app and let it work.
|
|
8
10
|
|
|
9
11
|
```bash
|
|
10
12
|
npx explorbot start https://your-app.com
|
|
11
13
|
```
|
|
12
14
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
+
It runs with no babysitting and reports back what it finds. This is vibe-testing.
|
|
16
|
+
|
|
17
|
+
New here? Read the [Getting Started guide](docs/getting-started.md).
|
|
15
18
|
|
|
16
19
|
## Use Cases
|
|
17
20
|
|
|
18
|
-
* Autonomously test web application or
|
|
21
|
+
* Autonomously test a web application or parts of it
|
|
19
22
|
* Discover test scenarios and get automated tests for them
|
|
20
|
-
* Write manual test cases from exploring website
|
|
21
|
-
*
|
|
23
|
+
* Write manual test cases from exploring a website
|
|
24
|
+
* 24/7 monkey-testing that reveals hidden errors
|
|
22
25
|
* Quick-test for MVPs and prototypes
|
|
23
26
|
|
|
24
|
-
Explorbot
|
|
27
|
+
Explorbot tests features that unit tests and scripted browser tests never reach.
|
|
25
28
|
|
|
26
29
|
## Demo
|
|
27
30
|
|
|
28
31
|

|
|
29
32
|
|
|
30
|
-
##
|
|
31
|
-
|
|
32
|
-
- NodeJS 24+ or **Bun**
|
|
33
|
-
- **AI provider API key** — OpenRouter recommended; Groq, Cerebras, OpenAI, Anthropic, or others via [Vercel AI SDK](https://sdk.vercel.ai/providers)
|
|
34
|
-
- **Modern terminal** — iTerm2, WARP, Kitty, Ghostty. WSL if running on Windows
|
|
35
|
-
- **Compatible web app** — Check [docs/prerequisites.md](docs/prerequisites.md) to verify your app works with Explorbot
|
|
36
|
-
|
|
37
|
-
## Quick Start
|
|
38
|
-
|
|
39
|
-
**1. Install dependencies**
|
|
40
|
-
|
|
41
|
-
```bash
|
|
42
|
-
npm i explorbot --save
|
|
43
|
-
npx playwright install
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
**2. Initialize config**
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
npx explorbot init
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
**3. Edit `explorbot.config.js`** — set your app URL and AI provider:
|
|
53
|
-
|
|
54
|
-
> [!IMPORTANT]
|
|
55
|
-
> **Explorbot uses three types of models:**
|
|
56
|
-
>
|
|
57
|
-
> | Type | Config key | Purpose | Recommendation |
|
|
58
|
-
> |------|-----------|---------|----------------|
|
|
59
|
-
> | **model** | `ai.model` | Standard model for HTML/ARIA processing. Used by Tester, Navigator, Researcher. Should be fast and cheap — these agents are token-hungry. | e.g. `openai/gpt-oss-20b` |
|
|
60
|
-
> | **visionModel** | `ai.visionModel` | Screenshot analysis. Used when agents need to visually inspect the page. | e.g. `meta-llama/llama-4-scout-17b-16e-instruct` |
|
|
61
|
-
> | **agenticModel** | `ai.agenticModel` | Exceptional decision making. Used by Captain and Pilot — agents that read compact action logs and make high-level decisions. Benefits from a smarter model. | Strong agentic models but fast (MiniMax 2.5, Grok Fast, Qwen, …) |
|
|
62
|
-
>
|
|
63
|
-
> See [OpenRouter](https://openrouter.ai/rankings#performance) for latency-focused model picks.
|
|
64
|
-
|
|
65
|
-
This example uses **OpenRouter** (one API key, many providers). Any Vercel AI SDK provider works; see [docs/providers.md](docs/providers.md).
|
|
66
|
-
|
|
67
|
-
```javascript
|
|
68
|
-
import { createOpenRouter } from '@openrouter/ai-sdk-provider';
|
|
69
|
-
|
|
70
|
-
const openrouter = createOpenRouter({
|
|
71
|
-
apiKey: process.env.OPENROUTER_API_KEY,
|
|
72
|
-
});
|
|
73
|
-
|
|
74
|
-
export default {
|
|
75
|
-
web: {
|
|
76
|
-
url: 'https://your-app.com',
|
|
77
|
-
},
|
|
78
|
-
ai: {
|
|
79
|
-
model: openrouter('openai/gpt-oss-20b'),
|
|
80
|
-
visionModel: openrouter('meta-llama/llama-4-scout-17b-16e-instruct'),
|
|
81
|
-
agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
|
|
82
|
-
},
|
|
83
|
-
};
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
> [!TIP]
|
|
87
|
-
> Captain and Pilot barely use tokens (just action summaries), so a smarter `agenticModel` costs very little while significantly improving test quality. You can also override any agent's model individually via `ai.agents.<name>.model`.
|
|
88
|
-
|
|
89
|
-
**4. Add knowledge** (optional but recommended)
|
|
90
|
-
|
|
91
|
-
If your app requires authentication, tell Explorbot how to log in:
|
|
92
|
-
|
|
93
|
-
```bash
|
|
94
|
-
# Interactive mode
|
|
95
|
-
npx explorbot learn
|
|
96
|
-
|
|
97
|
-
# Or via CLI
|
|
98
|
-
npx explorbot learn "/login" "Use credentials: admin@example.com / secret123"
|
|
99
|
-
```
|
|
100
|
-
|
|
101
|
-
> [!TIP]
|
|
102
|
-
> Use `--session` to persist browser cookies and localStorage between runs. Log in once, and Explorbot will restore the session on next start:
|
|
103
|
-
> ```bash
|
|
104
|
-
> npx explorbot start /login --session # saves to output/session.json
|
|
105
|
-
> npx explorbot start /dashboard --session # restores session, skips login
|
|
106
|
-
> npx explorbot start /app --session auth.json # custom session file
|
|
107
|
-
> ```
|
|
108
|
-
|
|
109
|
-
> [!NOTE]
|
|
110
|
-
> Use `*` as URL pattern to add general knowledge that applies to all pages. See [docs/knowledge.md](docs/knowledge.md) for more.
|
|
111
|
-
|
|
112
|
-
**5. Run**
|
|
113
|
-
|
|
114
|
-
```bash
|
|
115
|
-
npx explorbot start /admin/users
|
|
116
|
-
```
|
|
117
|
-
|
|
118
|
-
Start from a small functional area of your app (admin panel, settings, any CRUD section) so Explorbot can quickly understand its business purpose and context.
|
|
33
|
+
## A new layer of testing
|
|
119
34
|
|
|
120
|
-
|
|
35
|
+
Unit tests check a function. End-to-end tests replay fixed user journeys. **Exploratory tests** investigate the app the way a curious tester would — taking new paths every run and catching what no one thought to script.
|
|
121
36
|
|
|
122
|
-
|
|
123
|
-
npx explorbot start /settings --show
|
|
124
|
-
```
|
|
37
|
+
Explorbot makes that third layer routine. It runs on your CI next to the other two, and everything stays local — no cloud service touches your app.
|
|
125
38
|
|
|
126
|
-
|
|
39
|
+
## How a session works
|
|
127
40
|
|
|
128
|
-
|
|
41
|
+
Give Explorbot a goal and a URL. A crew of agents takes it from there — no scripts, no human in the loop.
|
|
129
42
|
|
|
130
|
-
|
|
43
|
+
1. **Research** — map the page into sections and index every element. No source or docs needed.
|
|
44
|
+
2. **Plan** — draft test scenarios across normal, curious, and edge styles.
|
|
45
|
+
3. **Execute** — drive the browser step by step, adapting as the app changes.
|
|
46
|
+
4. **Verify** — confirm each outcome, cluster findings by root cause, and capture evidence.
|
|
47
|
+
5. **Keep** — save passing flows as real tests, with reports and screencasts — and learn for next run.
|
|
131
48
|
|
|
132
49
|

|
|
133
50
|
|
|
134
|
-
|
|
51
|
+
## The crew
|
|
135
52
|
|
|
136
|
-
|
|
53
|
+
Cheap, fast workers do the clicking and reading; smart managers make the calls — so a full session costs cents, not dollars.
|
|
137
54
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
55
|
+
| | | |
|
|
56
|
+
|---|---|---|
|
|
57
|
+
| [Researcher](docs/reference/agents.md) | [Planner](docs/reference/agents.md) | [Tester](docs/reference/agents.md) |
|
|
58
|
+
| [Pilot](docs/reference/agents.md) | [Captain](docs/reference/agents.md) | [Navigator](docs/reference/agents.md) |
|
|
59
|
+
| [Analyst](docs/reference/agents.md) | [Historian](docs/reference/agents.md) | [Fisherman](docs/reference/agents.md) |
|
|
142
60
|
|
|
143
|
-
|
|
61
|
+
See [Agents](docs/reference/agents.md) for what each one does.
|
|
144
62
|
|
|
145
|
-
|
|
63
|
+
## Core Philosophy
|
|
146
64
|
|
|
147
|
-
|
|
148
|
-
/explore # Full cycle: research → plan → test
|
|
149
|
-
/research # Analyze current page
|
|
150
|
-
/plan # Generate test scenarios
|
|
151
|
-
/test # Run next test
|
|
152
|
-
/navigate /settings # Go to a page
|
|
153
|
-
```
|
|
65
|
+
**Strategic decisions are deterministic** — the workflow (research → plan → test) is predictable and consistent.
|
|
154
66
|
|
|
155
|
-
|
|
67
|
+
**Tactical decisions are AI-driven** — how to click a button, what to do when a modal appears, how to recover from an error.
|
|
156
68
|
|
|
157
|
-
|
|
158
|
-
I.click('Login')
|
|
159
|
-
I.fillField('email', 'test@example.com')
|
|
160
|
-
I.see('Welcome')
|
|
161
|
-
```
|
|
69
|
+
**Cheap workers, smart managers** — token-hungry agents run on a fast, cheap model. The decision-makers read only short action logs, so a smarter model there costs almost nothing.
|
|
162
70
|
|
|
163
|
-
|
|
71
|
+
**Explorbot learns from failure** — it reuses past experience with a page to make faster, better decisions next time.
|
|
164
72
|
|
|
165
|
-
|
|
166
|
-
> Most TUI commands also have CLI equivalents that run headless and exit. For example, `explorbot research <url>` and `explorbot plan <path>` work without launching TUI. See [docs/commands.md](docs/commands.md) for the full mapping.
|
|
73
|
+
**Explorbot needs your knowledge** — you guide it with plain-text notes and domain hints, loaded when the matching page opens.
|
|
167
74
|
|
|
168
|
-
|
|
75
|
+
When tuned, Explorbot **runs autonomously for hours**, trying new scenarios on its own. The more it runs, the more it learns.
|
|
169
76
|
|
|
170
|
-
|
|
171
|
-
|--------|----------|-------------|
|
|
172
|
-
| Test files | `output/tests/*.spec.ts` or `*.js` | Runnable Playwright or CodeceptJS tests |
|
|
173
|
-
| Test plans | `output/plans/*.md` | Markdown documentation of scenarios |
|
|
174
|
-
| Experience | `./experience/` | What Explorbot learned about your app |
|
|
77
|
+
## What you get
|
|
175
78
|
|
|
176
|
-
Every run
|
|
79
|
+
Every run leaves behind:
|
|
177
80
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
framework: 'playwright', // or 'codeceptjs' (default)
|
|
183
|
-
screencast: true, // record .webm video per scenario, chapters labelled with each step
|
|
184
|
-
// screencast: { size: { width: 1280, height: 720 }, quality: 95 }
|
|
185
|
-
},
|
|
186
|
-
},
|
|
187
|
-
}
|
|
188
|
-
```
|
|
81
|
+
- **Runnable tests** — Playwright or CodeceptJS specs for every flow, ready to commit and run in CI.
|
|
82
|
+
- **Reports** — a pass/fail breakdown with a written analysis, as HTML and Markdown, or in Testomat.io.
|
|
83
|
+
- **Videos** — step-by-step screencasts of every run.
|
|
84
|
+
- **Experience** — what Explorbot learned, reused to test smarter next time.
|
|
189
85
|
|
|
190
|
-
|
|
86
|
+
See [Automated Tests](docs/guides/automated-tests.md) for the test output and [Reporting](docs/guides/reporting.md) for reports.
|
|
191
87
|
|
|
192
|
-
|
|
88
|
+
## It works with your suite
|
|
193
89
|
|
|
194
|
-
|
|
195
|
-
test('Create a new manual plan', async ({ page }) => {
|
|
196
|
-
await test.step("Click the 'New plan' button in toolbar", async () => {
|
|
197
|
-
await page.getByRole('button', { name: 'New plan' }).first().click();
|
|
198
|
-
});
|
|
90
|
+
Explorbot won't replace your regression tests — it covers what they can't. Your Playwright or CodeceptJS suites replay the same fixed steps every build. Explorbot re-explores the same pages new ways, clicking UI and paths your scripts never touch. Point it at a brand-new feature with zero coverage, and it works out the basic test cases and runs them right away.
|
|
199
91
|
|
|
200
|
-
|
|
201
|
-
await page.locator('#portal-container').getByRole('button', { name: 'Manual' }).click();
|
|
202
|
-
});
|
|
92
|
+
## Requirements
|
|
203
93
|
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
```
|
|
94
|
+
- Node.js 24+ or **Bun**
|
|
95
|
+
- An **AI provider key** — OpenRouter recommended; Groq, Cerebras, OpenAI, Anthropic, and others via the [Vercel AI SDK](https://sdk.vercel.ai/providers)
|
|
96
|
+
- A **modern terminal** — iTerm2, WARP, Kitty, Ghostty, or Windows Terminal with WSL
|
|
97
|
+
- A **compatible web app** — CRUD-heavy apps fit best. See [Prerequisites](docs/reference/prerequisites.md)
|
|
209
98
|
|
|
210
|
-
|
|
99
|
+
If your CI runs Playwright, it runs Explorbot. No GPUs, no special runners.
|
|
211
100
|
|
|
212
|
-
##
|
|
101
|
+
## Quick Start
|
|
213
102
|
|
|
214
|
-
**
|
|
103
|
+
**1. Install dependencies**
|
|
215
104
|
|
|
216
105
|
```bash
|
|
217
|
-
|
|
106
|
+
npm i explorbot --save
|
|
107
|
+
npx playwright install
|
|
218
108
|
```
|
|
219
109
|
|
|
220
|
-
**
|
|
110
|
+
**2. Initialize config**
|
|
221
111
|
|
|
222
112
|
```bash
|
|
223
|
-
npx explorbot
|
|
113
|
+
npx explorbot init
|
|
224
114
|
```
|
|
225
115
|
|
|
226
|
-
**
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
npx explorbot freesail /admin # explore and test pages indefinitely
|
|
230
|
-
npx explorbot freesail /app --deep # depth-first: explore nearby pages first
|
|
231
|
-
npx explorbot freesail /app --shallow # breadth-first: spread across many pages
|
|
232
|
-
npx explorbot freesail /app --scope /admin # restrict to URLs under /admin
|
|
233
|
-
```
|
|
116
|
+
**3. Configure and run**
|
|
234
117
|
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
## API Testing
|
|
238
|
-
|
|
239
|
-
Explorbot also tests REST APIs. Add an `api` section to your config and point it at your API:
|
|
240
|
-
|
|
241
|
-
```javascript
|
|
242
|
-
export default {
|
|
243
|
-
web: {
|
|
244
|
-
url: 'http://localhost:3000',
|
|
245
|
-
},
|
|
246
|
-
ai: {
|
|
247
|
-
model: openrouter('openai/gpt-oss-20b'),
|
|
248
|
-
agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
|
|
249
|
-
},
|
|
250
|
-
api: {
|
|
251
|
-
baseEndpoint: 'http://localhost:3000/api/v1',
|
|
252
|
-
spec: ['http://localhost:3000/api/openapi.json'],
|
|
253
|
-
headers: {
|
|
254
|
-
'Authorization': 'Bearer <token>',
|
|
255
|
-
},
|
|
256
|
-
},
|
|
257
|
-
};
|
|
258
|
-
```
|
|
118
|
+
Add your AI provider key to `.env`, set your app URL in `explorbot.config.js`, then point Explorbot at a focused page — an admin panel, settings, or any CRUD section:
|
|
259
119
|
|
|
260
120
|
```bash
|
|
261
|
-
npx explorbot
|
|
262
|
-
npx explorbot api plan /users # generate test plan only
|
|
263
|
-
npx explorbot api test plans/users.md * # run all tests from a plan
|
|
121
|
+
npx explorbot start /admin/users
|
|
264
122
|
```
|
|
265
123
|
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
See [docs/api-testing.md](docs/api-testing.md) for setup, authentication hooks, and full command reference.
|
|
124
|
+
Type `/explore`, and Explorbot runs its loop on its own — research, plan, test, repeat — learning from every run.
|
|
269
125
|
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
**Strategic decisions are deterministic** — The workflow (research → plan → test) is predictable and consistent.
|
|
273
|
-
|
|
274
|
-
**Tactical decisions are AI-driven** — How to click that button, what to do when a modal appears, how to recover from errors.
|
|
275
|
-
|
|
276
|
-
**Cheap workers, smart managers** — Tester, Navigator, and Researcher are token-hungry agents that chew through HTML and ARIA on every step. They run on the fast, cheap `model`. Captain and Pilot are the decision-makers — they read only compact action logs and make high-level choices. Set `agenticModel` to a smarter model for better results at negligible extra cost.
|
|
126
|
+
That's the gist. The [**Getting Started guide**](docs/getting-started.md) walks through the full setup — choosing models, teaching Explorbot to log in, and picking the right feature to start on.
|
|
277
127
|
|
|
278
|
-
|
|
128
|
+
## Teaching Explorbot
|
|
279
129
|
|
|
280
|
-
|
|
130
|
+
Explorbot gets better when you tell it about your app:
|
|
281
131
|
|
|
282
|
-
|
|
132
|
+
- **Knowledge** — credentials, form rules, navigation quirks. See [Knowledge](docs/guides/knowledge.md).
|
|
133
|
+
- **Rules** — per-agent, per-page instructions. See [Configuration](docs/reference/configuration.md#rules).
|
|
134
|
+
- **Experience** — learned automatically from what works.
|
|
283
135
|
|
|
136
|
+
Handling logins, cookie banners, modals, and test data takes a few lines — see [Customization](docs/guides/customization.md).
|
|
284
137
|
|
|
285
|
-
##
|
|
138
|
+
## It also tests REST APIs
|
|
286
139
|
|
|
287
|
-
|
|
288
|
-
* **Rules** (`./rules/`) — Customize agent behavior with markdown files. Add page-specific instructions, override planning styles, or tune how agents work on different parts of your app. See [docs/configuration.md](docs/configuration.md#rules).
|
|
289
|
-
* **Experience** (`./experience/`) — Explorbot learns automatically from successful interactions and saves what works.
|
|
140
|
+
Point Explorbot at an OpenAPI spec and it plans and runs API tests too. See [API Testing](docs/guides/api-testing.md).
|
|
290
141
|
|
|
291
|
-
##
|
|
142
|
+
## Keep going
|
|
292
143
|
|
|
293
|
-
|
|
294
|
-
- [docs/commands.md](docs/commands.md) — Terminal command reference
|
|
295
|
-
- [docs/api-testing.md](docs/api-testing.md) — API testing setup and commands
|
|
296
|
-
- [docs/knowledge.md](docs/knowledge.md) — Knowledge system and URL patterns
|
|
297
|
-
- [docs/providers.md](docs/providers.md) — AI provider configuration
|
|
298
|
-
- [docs/agents.md](docs/agents.md) — Agent descriptions and capabilities
|
|
299
|
-
- [docs/planner.md](docs/planner.md) — Planner agent: planning styles and customization
|
|
300
|
-
- [docs/scripting.md](docs/scripting.md) — Building custom autonomous scripts
|
|
301
|
-
- [docs/observability.md](docs/observability.md) — Langfuse tracing and debugging
|
|
302
|
-
- [docs/page-interaction.md](docs/page-interaction.md) — How agents interact with pages
|
|
144
|
+
When you're ready to go deeper, the [full documentation](docs/README.md) covers everything, starting with the [Getting Started guide](docs/getting-started.md).
|
|
303
145
|
|
|
304
146
|
## FAQ
|
|
305
147
|
|
|
306
|
-
**Can I run it in Cursor
|
|
307
|
-
No, Explorbot is a separate application designed for constant testing. Cursor, Codex,
|
|
148
|
+
**Can I run it in Cursor or Claude Code?**
|
|
149
|
+
No, Explorbot is a separate application designed for constant testing. Cursor, Codex, and Claude Code are coding agents — not relevant here.
|
|
308
150
|
|
|
309
|
-
> However, Explorbot can be used as subagent or terminal command
|
|
151
|
+
> However, Explorbot can be used as a subagent or terminal command controlled by a coding agent.
|
|
310
152
|
|
|
311
|
-
**Can I bring Cursor or OpenAI
|
|
312
|
-
No
|
|
153
|
+
**Can I bring a Cursor or OpenAI subscription?**
|
|
154
|
+
No. Their models are too slow for the way Explorbot works. Use pay-per-token providers like Groq and OpenRouter.
|
|
313
155
|
|
|
314
156
|
**I want to use Opus!!!**
|
|
315
|
-
Opus is great for coding.
|
|
157
|
+
Opus is great for coding. Testing needs a simpler model that can safely consume lots of HTML tokens, fast. Save the expensive models for sophisticated decision-making.
|
|
316
158
|
|
|
317
|
-
**Is
|
|
159
|
+
**Is it expensive?**
|
|
318
160
|
No. With fast open models (e.g. `openai/gpt-oss-20b` on OpenRouter or Groq), expect roughly **~$1/hour of continuous run**, depending on provider and traffic.
|
|
319
161
|
|
|
320
162
|
**Does Explorbot have MCP?**
|
|
321
163
|
Not yet.
|
|
322
164
|
|
|
323
165
|
**Can I build my own agents with it?**
|
|
324
|
-
Yes, use the programmatic API. See [
|
|
166
|
+
Yes, use the programmatic API. See [Scripting](docs/contributing/scripting.md).
|
|
325
167
|
|
|
326
|
-
**
|
|
168
|
+
**Can I do the same in Cursor with Playwright MCP?**
|
|
327
169
|
Good luck running it on CI!
|
|
328
170
|
|
|
329
171
|
## Development
|
|
330
172
|
|
|
331
173
|
* Clone this repository
|
|
332
174
|
* Use **Bun** to run TS and TSX with no building
|
|
333
|
-
* Create a sample project under `example` directory:
|
|
175
|
+
* Create a sample project under the `example` directory:
|
|
334
176
|
|
|
335
177
|
```
|
|
336
178
|
./bin/explorbot-cli.ts init --path example
|
|
@@ -351,7 +193,3 @@ Explorbot is licensed under the [Elastic License 2.0 (ELv2)](LICENSE).
|
|
|
351
193
|
The only restriction: you may not offer Explorbot itself as a hosted/managed service (i.e., resell it as a product). This license is used by Elastic, Grafana, and other open-source companies.
|
|
352
194
|
|
|
353
195
|
Explorbot is built by [Testomat.io](https://testomat.io).
|
|
354
|
-
|
|
355
|
-
---
|
|
356
|
-
|
|
357
|
-
Explorbot learns as it explores. The more it tests your app, the better it gets at testing your app. That's vibe-testing.
|
package/bin/explorbot-cli.ts
CHANGED
|
@@ -338,6 +338,7 @@ addCommonOptions(program.command('test <planfile> [index]').description('Execute
|
|
|
338
338
|
const cmd = new TestCommand(explorBot);
|
|
339
339
|
await cmd.execute(args);
|
|
340
340
|
|
|
341
|
+
await explorBot.printSessionAnalysis();
|
|
341
342
|
await explorBot.stop();
|
|
342
343
|
await showStatsAndExit(0);
|
|
343
344
|
} catch (error) {
|
|
@@ -293,6 +293,7 @@ addCommonOptions(program.command('test <planfile> [index]').description('Execute
|
|
|
293
293
|
const { TestCommand } = await import('../src/commands/test-command.js');
|
|
294
294
|
const cmd = new TestCommand(explorBot);
|
|
295
295
|
await cmd.execute(args);
|
|
296
|
+
await explorBot.printSessionAnalysis();
|
|
296
297
|
await explorBot.stop();
|
|
297
298
|
await showStatsAndExit(0);
|
|
298
299
|
}
|
package/dist/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "explorbot",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.29",
|
|
4
4
|
"description": "CLI app built with React Ink, CodeceptJS, and Playwright",
|
|
5
5
|
"license": "Elastic-2.0",
|
|
6
6
|
"type": "module",
|
|
@@ -65,9 +65,10 @@
|
|
|
65
65
|
},
|
|
66
66
|
"author": "",
|
|
67
67
|
"dependencies": {
|
|
68
|
-
"@ai-sdk/anthropic": "^
|
|
69
|
-
"@ai-sdk/groq": "^
|
|
70
|
-
"@ai-sdk/openai": "^
|
|
68
|
+
"@ai-sdk/anthropic": "^4.0",
|
|
69
|
+
"@ai-sdk/groq": "^4.0",
|
|
70
|
+
"@ai-sdk/openai": "^4.0",
|
|
71
|
+
"@ai-sdk/otel": "^1.0.2",
|
|
71
72
|
"@axe-core/playwright": "^4.11.0",
|
|
72
73
|
"@codeceptjs/reflection": "^0.5.2",
|
|
73
74
|
"@faker-js/faker": "^10.4.0",
|
|
@@ -82,8 +83,8 @@
|
|
|
82
83
|
"@opentelemetry/sdk-trace-base": "^2.2.0",
|
|
83
84
|
"@opentelemetry/semantic-conventions": "^1.38.0",
|
|
84
85
|
"@scalar/openapi-parser": "^0.25.6",
|
|
85
|
-
"@testomatio/reporter": "^2.
|
|
86
|
-
"ai": "^
|
|
86
|
+
"@testomatio/reporter": "^2.9.1",
|
|
87
|
+
"ai": "^7.0.2",
|
|
87
88
|
"axe-core": "^4.11.1",
|
|
88
89
|
"bash-tool": "^1.3.15",
|
|
89
90
|
"cli-highlight": "^2.1.11",
|
|
@@ -110,6 +111,7 @@
|
|
|
110
111
|
"parse5": "^8.0.0",
|
|
111
112
|
"playwright": "^1.60",
|
|
112
113
|
"react": "^19.1.1",
|
|
114
|
+
"sambanova-ai-provider": "^1.2.2",
|
|
113
115
|
"strip-ansi": "^7.1.2",
|
|
114
116
|
"turndown": "^7.2.1",
|
|
115
117
|
"unique-names-generator": "^4.7.1",
|
|
@@ -113,6 +113,8 @@ For input field values, ALWAYS use I.seeInField() — never check value via CSS
|
|
|
113
113
|
Prefer text locators (label, name, placeholder) for form fields: I.seeInField('Search', 'value') over I.seeInField('input[name="search"]', 'value').
|
|
114
114
|
Only use locators that exist in the provided HTML or ARIA snapshot.
|
|
115
115
|
Verify exact conditions, not approximate matches.
|
|
116
|
+
When the claim contains a concrete quoted value, generated assertion code MUST include that whole value. Do not shorten names, IDs, titles, emails, URLs, or other user-created values.
|
|
117
|
+
For exact visible text, prefer a text assertion scoped to a specific container or an ARIA locator with the complete text. Partial text is not valid evidence for a claim about the full value.
|
|
116
118
|
NEVER use `:has-text(...)` inside a seeElement/dontSeeElement locator. Checking text inside an element is the job of I.see(text, context) — the `:has-text()` form duplicates that capability with a fragile selector.
|
|
117
119
|
NEVER emit two assertions that check the same fact with different shapes. `I.see(text, locator)` and `I.seeElement("<locator>:has-text('text')")` verify the same thing — pick one (prefer I.see). One claim, one assertion.
|
|
118
120
|
</verification_rules>
|
package/dist/src/ai/fisherman.js
CHANGED
|
@@ -4,6 +4,7 @@ import { createDebug, tag } from "../utils/logger.js";
|
|
|
4
4
|
const debugLog = createDebug('explorbot:fisherman');
|
|
5
5
|
import { loop } from "../utils/loop.js";
|
|
6
6
|
import { createFishermanTools } from "./fisherman-tools.js";
|
|
7
|
+
import { dataProtectionRules } from "./rules.js";
|
|
7
8
|
const MAX_ITERATIONS = 15;
|
|
8
9
|
const MAX_TOOL_ROUNDTRIPS = 5;
|
|
9
10
|
export class Fisherman {
|
|
@@ -65,7 +66,7 @@ export class Fisherman {
|
|
|
65
66
|
spec: this.spec,
|
|
66
67
|
baseEndpoint: this.baseEndpoint,
|
|
67
68
|
});
|
|
68
|
-
const conversation = this.provider.startConversation(this.buildSystemPrompt(endpointList, scopeUrl), 'fisherman');
|
|
69
|
+
const conversation = this.provider.startConversation(this.buildSystemPrompt(endpointList, Object.keys(tools), scopeUrl), 'fisherman');
|
|
69
70
|
conversation.addUserText(this.buildTaskPrompt(instructions));
|
|
70
71
|
await loop(async ({ stop, iteration }) => {
|
|
71
72
|
debugLog(`iteration ${iteration}`);
|
|
@@ -149,7 +150,7 @@ export class Fisherman {
|
|
|
149
150
|
}
|
|
150
151
|
return lines.join('\n');
|
|
151
152
|
}
|
|
152
|
-
buildSystemPrompt(endpointList, scopeUrl) {
|
|
153
|
+
buildSystemPrompt(endpointList, toolNames, scopeUrl) {
|
|
153
154
|
const scopeBlock = scopeUrl ? `\n\nSCOPE: You are operating within ${scopeUrl}.\nAll created items must belong to this scope.` : '';
|
|
154
155
|
return dedent `
|
|
155
156
|
You are Fisherman — a data preparation agent. You create test data by making API requests.
|
|
@@ -158,6 +159,11 @@ export class Fisherman {
|
|
|
158
159
|
${endpointList}
|
|
159
160
|
${scopeBlock}
|
|
160
161
|
|
|
162
|
+
AVAILABLE TOOLS:
|
|
163
|
+
${toolNames.join(', ')}.
|
|
164
|
+
Use tool names exactly as listed. Do not invent aliases, combined names, or names with channel markers such as "commentary".
|
|
165
|
+
Match each tool input schema exactly. Do not invent parameter names or pass extra fields.
|
|
166
|
+
|
|
161
167
|
WORKFLOW:
|
|
162
168
|
1. Call getEndpointSpec to see the request body example for the endpoint
|
|
163
169
|
2. Make requests — the response automatically extracts IDs, names, and status fields
|
|
@@ -169,6 +175,8 @@ export class Fisherman {
|
|
|
169
175
|
- Chain requests logically — create parent resources before children
|
|
170
176
|
- If a request fails, try once more with adjusted data before reporting failure
|
|
171
177
|
- Use realistic but unique data for each item (vary names, titles)
|
|
178
|
+
|
|
179
|
+
${dataProtectionRules}
|
|
172
180
|
`;
|
|
173
181
|
}
|
|
174
182
|
buildTaskPrompt(instructions) {
|
|
@@ -177,7 +185,10 @@ export class Fisherman {
|
|
|
177
185
|
|
|
178
186
|
${instructions}
|
|
179
187
|
|
|
180
|
-
|
|
188
|
+
${dataProtectionRules}
|
|
189
|
+
|
|
190
|
+
If data preparation is allowed by these rules, execute the necessary API requests to create this data.
|
|
191
|
+
When done, call finish with the summary. If data preparation is forbidden, call stop with the reason.
|
|
181
192
|
`;
|
|
182
193
|
}
|
|
183
194
|
}
|