explorbot 0.2.3 → 0.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/bin/explorbot-cli.ts +26 -8
- package/boat/api-tester/src/cli.ts +17 -0
- package/boat/api-tester/src/config.ts +4 -2
- package/boat/doc-collector/bin/doc-collector-cli.ts +2 -0
- package/boat/doc-collector/src/ai/documentarian.ts +61 -31
- package/boat/doc-collector/src/cli.ts +14 -1
- package/boat/doc-collector/src/config.ts +4 -2
- package/boat/prima/bin/prima-cli.ts +0 -0
- package/boat/prima/src/activity-line.ts +33 -0
- package/boat/prima/src/cli.ts +127 -86
- package/boat/prima/src/envelope.ts +102 -52
- package/boat/prima/src/prima.ts +567 -128
- package/boat/prima/src/pw-parser.ts +11 -1
- package/boat/prima/src/pw-registry.ts +4 -5
- package/boat/prima/src/session-log.ts +126 -0
- package/dist/bin/explorbot-cli.js +26 -8
- package/dist/boat/api-tester/bin/apibot-cli.js +2 -0
- package/dist/boat/api-tester/src/cli.js +17 -0
- package/dist/boat/api-tester/src/config.js +4 -2
- package/dist/boat/doc-collector/bin/doc-collector-cli.js +2 -0
- package/dist/boat/doc-collector/src/ai/documentarian.js +44 -19
- package/dist/boat/doc-collector/src/cli.js +14 -1
- package/dist/boat/doc-collector/src/config.js +4 -2
- package/dist/boat/prima/src/activity-line.js +30 -0
- package/dist/boat/prima/src/cli.js +109 -77
- package/dist/boat/prima/src/envelope.js +94 -44
- package/dist/boat/prima/src/prima.js +533 -119
- package/dist/boat/prima/src/pw-parser.js +13 -1
- package/dist/boat/prima/src/pw-registry.js +4 -5
- package/dist/boat/prima/src/session-log.js +108 -0
- package/dist/package.json +3 -2
- package/dist/rules/navigator/verification-actions.md +20 -0
- package/dist/src/action-result.d.ts +7 -0
- package/dist/src/action-result.js +4 -0
- package/dist/src/action.d.ts +2 -0
- package/dist/src/action.js +41 -2
- package/dist/src/ai/captain/web-mode.js +6 -3
- package/dist/src/ai/captain.js +2 -0
- package/dist/src/ai/navigator.d.ts +34 -0
- package/dist/src/ai/navigator.js +237 -181
- package/dist/src/ai/pilot.d.ts +7 -0
- package/dist/src/ai/pilot.js +90 -2
- package/dist/src/ai/provider.d.ts +2 -2
- package/dist/src/ai/provider.js +14 -23
- package/dist/src/ai/rerunner.js +2 -1
- package/dist/src/ai/researcher/cache.d.ts +2 -0
- package/dist/src/ai/researcher/cache.js +10 -2
- package/dist/src/ai/researcher.js +3 -2
- package/dist/src/ai/rules.js +17 -10
- package/dist/src/ai/session-analyst.js +2 -0
- package/dist/src/ai/task-agent.js +4 -1
- package/dist/src/ai/tester.d.ts +6 -3
- package/dist/src/ai/tester.js +50 -46
- package/dist/src/ai/tools.d.ts +14 -0
- package/dist/src/ai/tools.js +117 -37
- package/dist/src/commands/config-command.d.ts +51 -0
- package/dist/src/commands/config-command.js +117 -0
- package/dist/src/commands/index.js +2 -0
- package/dist/src/config.d.ts +9 -1
- package/dist/src/config.js +53 -4
- package/dist/src/execution-controller.d.ts +2 -0
- package/dist/src/execution-controller.js +6 -0
- package/dist/src/explorbot.d.ts +2 -1
- package/dist/src/explorbot.js +7 -2
- package/dist/src/explorer.js +2 -3
- package/dist/src/playwright-recorder.js +30 -0
- package/dist/src/remote.d.ts +55 -0
- package/dist/src/remote.js +235 -0
- package/dist/src/reporter.d.ts +1 -0
- package/dist/src/reporter.js +7 -1
- package/dist/src/state-manager.d.ts +2 -1
- package/dist/src/state-manager.js +3 -1
- package/dist/src/stats.d.ts +1 -0
- package/dist/src/stats.js +1 -0
- package/dist/src/test-plan.d.ts +3 -0
- package/dist/src/test-plan.js +26 -0
- package/dist/src/utils/aria.d.ts +2 -8
- package/dist/src/utils/aria.js +69 -40
- package/dist/src/utils/html.js +1 -0
- package/dist/src/utils/logger.d.ts +7 -1
- package/dist/src/utils/logger.js +32 -0
- package/dist/src/utils/page-readiness.js +18 -1
- package/dist/src/utils/url-matcher.js +3 -0
- package/dist/src/utils/web-element.d.ts +2 -0
- package/dist/src/utils/web-element.js +8 -0
- package/dist/src/utils/web-sandbox.d.ts +1 -1
- package/dist/src/utils/web-sandbox.js +2 -3
- package/docs/api-testing/basics.md +90 -0
- package/docs/api-testing/planning.md +57 -0
- package/docs/api-testing/running-tests.md +55 -0
- package/docs/assets/cloud-report.png +0 -0
- package/docs/assets/html-report.png +0 -0
- package/docs/assets/langfuse-trace.png +0 -0
- package/docs/assets/successful-explore-run.png +0 -0
- package/docs/basics/getting-started.md +140 -0
- package/docs/basics/prerequisites.md +63 -0
- package/docs/basics/providers.md +362 -0
- package/docs/basics/running.md +78 -0
- package/docs/contributing/ai-integration-tests.md +57 -0
- package/docs/contributing/contributing.md +90 -0
- package/docs/contributing/demo-videos.md +36 -0
- package/docs/contributing/npm-package.md +138 -0
- package/docs/contributing/observability.md +227 -0
- package/docs/contributing/regression-tests.md +103 -0
- package/docs/contributing/testing.md +95 -0
- package/docs/doc-collection/basics.md +128 -0
- package/docs/doc-collection/crawling.md +67 -0
- package/docs/doc-collection/interactive-mode.md +99 -0
- package/docs/index.json +87 -0
- package/docs/reference/commands.md +997 -0
- package/docs/reference/configuration.md +569 -0
- package/docs/reference/scripting.md +303 -0
- package/docs/reference/websocket.md +50 -0
- package/docs/superpowers/plans/2026-08-01-actor-boat.md +925 -0
- package/docs/superpowers/plans/2026-08-01-prima-boat.md +1120 -0
- package/docs/superpowers/reviews/2026-08-06-prima-vs-playwright-cli.md +268 -0
- package/docs/superpowers/specs/2026-08-01-actor-boat-design.md +204 -0
- package/docs/superpowers/specs/2026-08-01-prima-boat-design.md +242 -0
- package/docs/superpowers/specs/2026-08-03-global-config-design.md +138 -0
- package/docs/superpowers/specs/2026-08-07-prima-fixes-design.md +394 -0
- package/docs/superpowers/specs/2026-08-18-prima-false-verdicts.md +159 -0
- package/docs/web-testing/agents.md +158 -0
- package/docs/web-testing/automated-tests.md +134 -0
- package/docs/web-testing/basics.md +91 -0
- package/docs/web-testing/customization.md +131 -0
- package/docs/web-testing/hooks.md +238 -0
- package/docs/web-testing/page-interaction.md +84 -0
- package/docs/web-testing/planner.md +122 -0
- package/docs/web-testing/rerun.md +164 -0
- package/docs/web-testing/researcher.md +380 -0
- package/docs/workflow/agentic-usage.md +233 -0
- package/docs/workflow/application-spec.md +73 -0
- package/docs/workflow/ci.md +202 -0
- package/docs/workflow/knowledge.md +310 -0
- package/docs/workflow/planning-styles.md +67 -0
- package/docs/workflow/reporting.md +133 -0
- package/docs/workflow/test-plans.md +90 -0
- package/package.json +3 -2
- package/rules/navigator/verification-actions.md +20 -0
- package/src/action-result.ts +11 -0
- package/src/action.ts +43 -3
- package/src/ai/captain/web-mode.ts +6 -3
- package/src/ai/captain.ts +3 -0
- package/src/ai/navigator.ts +255 -186
- package/src/ai/pilot.ts +104 -2
- package/src/ai/provider.ts +14 -24
- package/src/ai/rerunner.ts +2 -1
- package/src/ai/researcher/cache.ts +12 -2
- package/src/ai/researcher.ts +3 -2
- package/src/ai/rules.ts +17 -10
- package/src/ai/session-analyst.ts +2 -0
- package/src/ai/task-agent.ts +3 -1
- package/src/ai/tester.ts +52 -45
- package/src/ai/tools.ts +136 -37
- package/src/commands/config-command.ts +146 -0
- package/src/commands/index.ts +2 -0
- package/src/config.ts +60 -5
- package/src/execution-controller.ts +8 -0
- package/src/explorbot.ts +7 -3
- package/src/explorer.ts +2 -2
- package/src/playwright-recorder.ts +23 -0
- package/src/remote.ts +244 -0
- package/src/reporter.ts +7 -1
- package/src/state-manager.ts +6 -2
- package/src/stats.ts +1 -0
- package/src/test-plan.ts +29 -0
- package/src/utils/aria.ts +65 -45
- package/src/utils/html.ts +1 -0
- package/src/utils/logger.ts +33 -2
- package/src/utils/page-readiness.ts +24 -1
- package/src/utils/url-matcher.ts +3 -0
- package/src/utils/web-element.ts +9 -0
- package/src/utils/web-sandbox.ts +3 -4
package/dist/src/utils/logger.js
CHANGED
|
@@ -52,13 +52,19 @@ const debugFilter = new DebugFilter();
|
|
|
52
52
|
class ConsoleDestination {
|
|
53
53
|
verboseMode = false;
|
|
54
54
|
forceEnabled = false;
|
|
55
|
+
quiet = false;
|
|
55
56
|
recentSteps = new RecentStepFilter();
|
|
56
57
|
isEnabled() {
|
|
58
|
+
if (this.quiet)
|
|
59
|
+
return false;
|
|
57
60
|
return this.forceEnabled || !process.env.INK_RUNNING;
|
|
58
61
|
}
|
|
59
62
|
forceEnable(enabled) {
|
|
60
63
|
this.forceEnabled = enabled;
|
|
61
64
|
}
|
|
65
|
+
setQuiet(enabled) {
|
|
66
|
+
this.quiet = enabled;
|
|
67
|
+
}
|
|
62
68
|
setVerboseMode(enabled) {
|
|
63
69
|
this.verboseMode = enabled;
|
|
64
70
|
}
|
|
@@ -113,6 +119,9 @@ class DebugDestination {
|
|
|
113
119
|
}
|
|
114
120
|
setVerboseMode(enabled) {
|
|
115
121
|
this.verboseMode = enabled;
|
|
122
|
+
// the debug package reads DEBUG when it loads, so a namespace turned on later needs enabling by hand
|
|
123
|
+
if (enabled)
|
|
124
|
+
debug.enable(process.env.DEBUG || 'explorbot:*');
|
|
116
125
|
}
|
|
117
126
|
write(namespace, ...args) {
|
|
118
127
|
if (!this.isNamespaceEnabled(namespace))
|
|
@@ -296,6 +305,7 @@ class Logger {
|
|
|
296
305
|
span = new SpanDestination();
|
|
297
306
|
react = new ReactDestination();
|
|
298
307
|
captain = new CaptainDestination();
|
|
308
|
+
extra = [];
|
|
299
309
|
truncateTags = ['page_html'];
|
|
300
310
|
constructor() { }
|
|
301
311
|
static getInstance() {
|
|
@@ -317,9 +327,17 @@ class Logger {
|
|
|
317
327
|
setPreserveConsoleLogs(enabled) {
|
|
318
328
|
this.console.forceEnable(enabled);
|
|
319
329
|
}
|
|
330
|
+
setQuietMode(enabled) {
|
|
331
|
+
this.console.setQuiet(enabled);
|
|
332
|
+
}
|
|
320
333
|
isVerboseMode() {
|
|
321
334
|
return this.debugDestination.isEnabled();
|
|
322
335
|
}
|
|
336
|
+
addDestination(destination) {
|
|
337
|
+
if (this.extra.includes(destination))
|
|
338
|
+
return;
|
|
339
|
+
this.extra.push(destination);
|
|
340
|
+
}
|
|
323
341
|
registerLogPane(addLog) {
|
|
324
342
|
this.react.registerLogPane(addLog);
|
|
325
343
|
}
|
|
@@ -388,6 +406,14 @@ class Logger {
|
|
|
388
406
|
}
|
|
389
407
|
return;
|
|
390
408
|
}
|
|
409
|
+
if (type === 'data') {
|
|
410
|
+
const entry = { type, content: String(args[0]), timestamp: new Date(), originalArgs: args };
|
|
411
|
+
for (const destination of this.extra) {
|
|
412
|
+
if (destination.isEnabled())
|
|
413
|
+
destination.write(entry);
|
|
414
|
+
}
|
|
415
|
+
return;
|
|
416
|
+
}
|
|
391
417
|
const options = this.extractLogOptions(type, args);
|
|
392
418
|
let content = this.processArgs(args);
|
|
393
419
|
if (type === 'step' && args[0]?.toCode) {
|
|
@@ -406,6 +432,10 @@ class Logger {
|
|
|
406
432
|
this.span.write(entry);
|
|
407
433
|
if (this.captain.isEnabled())
|
|
408
434
|
this.captain.write(entry);
|
|
435
|
+
for (const destination of this.extra) {
|
|
436
|
+
if (destination.isEnabled())
|
|
437
|
+
destination.write(entry);
|
|
438
|
+
}
|
|
409
439
|
if (process.env.INK_RUNNING) {
|
|
410
440
|
this.react.write(entry);
|
|
411
441
|
}
|
|
@@ -475,9 +505,11 @@ export const startLogCapture = () => logger.captain.startCapture();
|
|
|
475
505
|
export const stopLogCapture = () => logger.captain.stopCapture();
|
|
476
506
|
export const setVerboseMode = (enabled) => logger.setVerboseMode(enabled);
|
|
477
507
|
export const setPreserveConsoleLogs = (enabled) => logger.setPreserveConsoleLogs(enabled);
|
|
508
|
+
export const setQuietMode = (enabled) => logger.setQuietMode(enabled);
|
|
478
509
|
export const isVerboseMode = () => logger.isVerboseMode();
|
|
479
510
|
export const setDebugMode = (enabled) => logger.setDebugMode(enabled);
|
|
480
511
|
export const isDebugMode = () => logger.isDebugMode();
|
|
512
|
+
export const addDestination = (destination) => logger.addDestination(destination);
|
|
481
513
|
export const registerLogPane = (addLog) => logger.registerLogPane(addLog);
|
|
482
514
|
export const unregisterLogPane = (addLog) => logger.unregisterLogPane(addLog);
|
|
483
515
|
export const addTruncateTag = (tagName) => logger.addTruncateTag(tagName);
|
|
@@ -3,9 +3,26 @@ export async function waitForPageReadiness(page, options = {}) {
|
|
|
3
3
|
return;
|
|
4
4
|
const timeout = options.timeout ?? 6000;
|
|
5
5
|
await page.waitForLoadState?.('domcontentloaded', { timeout })?.catch(() => { });
|
|
6
|
-
await Promise.race([waitForNetworkIdle(page, timeout), waitForVisibleSpinnersHidden(page, options.spinnerSelectors || [], timeout), sleep(timeout)]).catch(() => { });
|
|
6
|
+
await Promise.race([waitForNetworkIdle(page, timeout), waitForDomQuiet(page, timeout), waitForVisibleSpinnersHidden(page, options.spinnerSelectors || [], timeout), sleep(timeout)]).catch(() => { });
|
|
7
7
|
await waitForPageBodyContent(page, timeout);
|
|
8
8
|
}
|
|
9
|
+
const DOM_QUIET_MS = 350;
|
|
10
|
+
function waitForDomQuiet(page, timeout) {
|
|
11
|
+
if (!page?.waitForFunction)
|
|
12
|
+
return new Promise(() => { });
|
|
13
|
+
return page
|
|
14
|
+
.waitForFunction((quiet) => {
|
|
15
|
+
const store = window;
|
|
16
|
+
if (!store.__explorbotDomQuiet) {
|
|
17
|
+
store.__explorbotDomQuiet = { last: Date.now() };
|
|
18
|
+
new MutationObserver(() => {
|
|
19
|
+
store.__explorbotDomQuiet.last = Date.now();
|
|
20
|
+
}).observe(document, { subtree: true, childList: true, attributes: true, characterData: true });
|
|
21
|
+
}
|
|
22
|
+
return Date.now() - store.__explorbotDomQuiet.last >= quiet;
|
|
23
|
+
}, DOM_QUIET_MS, { timeout, polling: 100 })
|
|
24
|
+
.catch(() => { });
|
|
25
|
+
}
|
|
9
26
|
function waitForNetworkIdle(page, timeout) {
|
|
10
27
|
if (!page?.waitForLoadState)
|
|
11
28
|
return Promise.resolve();
|
|
@@ -108,6 +108,9 @@ export function matchesNavigationUrl(expected, current) {
|
|
|
108
108
|
if (!expectedPath.includes('#')) {
|
|
109
109
|
currentPath = currentPath.split('#')[0];
|
|
110
110
|
}
|
|
111
|
+
if (!expectedPath.includes('?')) {
|
|
112
|
+
currentPath = currentPath.split('?')[0];
|
|
113
|
+
}
|
|
111
114
|
const normalize = (value) => value.replace(/^\/+|\/+$/g, '').toLowerCase();
|
|
112
115
|
return normalize(expectedPath) === normalize(currentPath);
|
|
113
116
|
}
|
|
@@ -36,6 +36,8 @@ export declare class WebElement {
|
|
|
36
36
|
static fromXPathMatch(m: XPathMatch): WebElement;
|
|
37
37
|
static fromPlaywrightLocator(locator: any): Promise<WebElement | null>;
|
|
38
38
|
static fromEidx(page: any, eidx: string): Promise<WebElement | null>;
|
|
39
|
+
static isAriaRef(ref: string): boolean;
|
|
40
|
+
static fromAriaRef(page: any, ref: string): Promise<WebElement | null>;
|
|
39
41
|
static fromEidxList(page: any, eidxList: string[]): Promise<WebElement[]>;
|
|
40
42
|
static commonAncestor(page: any, eidxList: string[]): Promise<WebElement | null>;
|
|
41
43
|
static findByXPath(html: string, xpath: string): Promise<{
|
|
@@ -111,6 +111,14 @@ export class WebElement {
|
|
|
111
111
|
static async fromEidx(page, eidx) {
|
|
112
112
|
return WebElement.fromPlaywrightLocator(page.locator(`[${EXPLORBOT_ATTRS.eidx}="${eidx}"]`));
|
|
113
113
|
}
|
|
114
|
+
static isAriaRef(ref) {
|
|
115
|
+
return /^(f\d+)?e\d+$/i.test(ref);
|
|
116
|
+
}
|
|
117
|
+
static async fromAriaRef(page, ref) {
|
|
118
|
+
if (!WebElement.isAriaRef(ref))
|
|
119
|
+
return null;
|
|
120
|
+
return WebElement.fromPlaywrightLocator(page.locator(`aria-ref=${ref}`));
|
|
121
|
+
}
|
|
114
122
|
static async fromEidxList(page, eidxList) {
|
|
115
123
|
const validEidxList = eidxList.filter((eidx) => /^e\d+$/i.test(eidx));
|
|
116
124
|
if (validEidxList.length === 0)
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
export declare function sanitizeCodeBlock(code: string): string;
|
|
2
2
|
export declare function hasPlaywrightCommands(code: string): boolean;
|
|
3
3
|
export declare function playwrightSandbox(page: any, code: string): Promise<any>;
|
|
4
|
-
export declare function codeceptJSSandbox(actor: any, codeOrFn: string | ((...args: any[]) => void)):
|
|
4
|
+
export declare function codeceptJSSandbox(actor: any, codeOrFn: string | ((...args: any[]) => void)): any;
|
|
@@ -25,11 +25,10 @@ export function playwrightSandbox(page, code) {
|
|
|
25
25
|
}
|
|
26
26
|
export function codeceptJSSandbox(actor, codeOrFn) {
|
|
27
27
|
if (typeof codeOrFn === 'function') {
|
|
28
|
-
codeOrFn(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
29
|
-
return;
|
|
28
|
+
return codeOrFn(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
30
29
|
}
|
|
31
30
|
const run = createSandbox(CODECEPT_ARG_NAMES, codeOrFn);
|
|
32
|
-
run(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
31
|
+
return run(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
33
32
|
}
|
|
34
33
|
function createSandbox(argNames, body) {
|
|
35
34
|
const fn = new Function(...argNames, ...SHADOWED_GLOBALS, `'use strict';\n${body}`);
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# API Testing Basics
|
|
2
|
+
|
|
3
|
+
Explorbot tests REST APIs the same way it tests web apps: it plans scenarios, runs them, and reports results — no test scripts. Two AI agents do the work.
|
|
4
|
+
|
|
5
|
+
**Chief** reads your endpoint, its OpenAPI spec, and any [knowledge](../workflow/knowledge.md) you've written, then plans test scenarios: what to send, and what a correct response looks like.
|
|
6
|
+
|
|
7
|
+
**Curler** takes each scenario and executes it as real HTTP requests, checking the responses with assertions.
|
|
8
|
+
|
|
9
|
+
The plans Chief writes are ordinary Explorbot [test plans](../workflow/test-plans.md) — plain markdown you can read, edit, and commit. The web and API sides share the same plan format and the same [reporting](../workflow/reporting.md).
|
|
10
|
+
|
|
11
|
+
## Configure
|
|
12
|
+
|
|
13
|
+
Point Explorbot at your API by adding an `api` key to your `explorbot.config.js`:
|
|
14
|
+
|
|
15
|
+
```javascript
|
|
16
|
+
export default {
|
|
17
|
+
ai: {
|
|
18
|
+
model: openrouter('openai/gpt-oss-20b:nitro'),
|
|
19
|
+
agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
|
|
20
|
+
},
|
|
21
|
+
api: {
|
|
22
|
+
baseEndpoint: 'http://localhost:3000/api/v1',
|
|
23
|
+
spec: ['http://localhost:3000/api/openapi.json'],
|
|
24
|
+
headers: {
|
|
25
|
+
Authorization: 'Bearer <token>',
|
|
26
|
+
},
|
|
27
|
+
},
|
|
28
|
+
};
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
- **`baseEndpoint`** (required) — the base URL prepended to every request. Test steps use relative paths like `/users`; Curler adds the base for you.
|
|
32
|
+
- **`spec`** (required) — one or more OpenAPI specs, given as HTTP(S) URLs or local file paths, in YAML or JSON. Chief uses the spec to plan; Curler uses it to look up schemas. Both agents refuse to run without one.
|
|
33
|
+
- **`headers`** — sent with every request. This is where API keys and auth tokens go.
|
|
34
|
+
|
|
35
|
+
See the [full configuration reference](../reference/configuration.md) for every option and [providers](../basics/providers.md) for choosing an AI model.
|
|
36
|
+
|
|
37
|
+
### Authenticating
|
|
38
|
+
|
|
39
|
+
If a static token in `headers` is enough, you're done. If you need to log in and fetch a token first, use the `bootstrap` hook — it runs once before any tests, and whatever headers it returns merge into every later request:
|
|
40
|
+
|
|
41
|
+
```javascript
|
|
42
|
+
api: {
|
|
43
|
+
baseEndpoint: 'http://localhost:3000/api/v1',
|
|
44
|
+
spec: ['http://localhost:3000/api/openapi.json'],
|
|
45
|
+
bootstrap: async ({ baseEndpoint }) => {
|
|
46
|
+
const res = await fetch(`${baseEndpoint}/auth/login`, {
|
|
47
|
+
method: 'POST',
|
|
48
|
+
headers: { 'Content-Type': 'application/json' },
|
|
49
|
+
body: JSON.stringify({ email: 'admin@test.com', password: 'secret' }),
|
|
50
|
+
});
|
|
51
|
+
const { token } = await res.json();
|
|
52
|
+
return { Authorization: `Bearer ${token}` };
|
|
53
|
+
},
|
|
54
|
+
},
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
A matching `teardown` hook runs after all tests finish — use it to clean up data.
|
|
58
|
+
|
|
59
|
+
### A dedicated API project
|
|
60
|
+
|
|
61
|
+
If you don't have a web `explorbot.config.js`, run `npx explorbot api init`. It asks for your base endpoint, spec, and a one-line description of the API, then writes a standalone `apibot.config.ts` (with an `ai` and `api` section) plus `output/` and `knowledge/` directories. When both files exist, `apibot.config.*` takes precedence over `explorbot.config.*`.
|
|
62
|
+
|
|
63
|
+
## Your first run
|
|
64
|
+
|
|
65
|
+
The minimal loop is plan, then test. Point Chief at an endpoint:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx explorbot api plan /users
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
On startup Explorbot does a health check — a `GET /` against your base endpoint — so a bad URL or token fails immediately. Then Chief fetches sample data, reads the spec, and writes scenarios to `output/plans/users.md`. Hand that file to Curler:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
npx explorbot api test output/plans/users.md
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Curler runs the scenarios and prints how many passed and failed.
|
|
78
|
+
|
|
79
|
+
## Output files
|
|
80
|
+
|
|
81
|
+
| Output | Location | What it is |
|
|
82
|
+
|--------|----------|------------|
|
|
83
|
+
| Test plans | `output/plans/*.md` | Chief's scenarios — priorities, steps, expected outcomes |
|
|
84
|
+
| Request logs | `output/requests/*.request.yaml` | Every HTTP request and response, for debugging |
|
|
85
|
+
| Reports | via the shared [reporter](../workflow/reporting.md) | Pass/fail results, optionally sent to Testomat.io |
|
|
86
|
+
|
|
87
|
+
## Next steps
|
|
88
|
+
|
|
89
|
+
- [Planning API tests](./planning.md) — give Chief context and steer what it tests.
|
|
90
|
+
- [Running API tests](./running-tests.md) — execute plans, read request logs, and run the full autonomous cycle.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Planning API Tests
|
|
2
|
+
|
|
3
|
+
Chief plans the scenarios; how good they are depends on the context you give it. Start by feeding it what it needs, then choose how aggressively it should probe.
|
|
4
|
+
|
|
5
|
+
## Give Chief context
|
|
6
|
+
|
|
7
|
+
**The API spec.** Chief reads the OpenAPI spec from your `spec` config to learn paths, methods, and request/response schemas. It also fetches live sample data from the endpoint — real IDs, enum values, field names — so the scenarios it writes reference data that actually exists instead of guessing. A good spec is the single biggest lever on plan quality.
|
|
8
|
+
|
|
9
|
+
**Endpoint knowledge.** The spec says what an endpoint accepts, not how your business rules work. Add that with `know`:
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
npx explorbot api know /users "CRUD for users. Admin role required for writes. IDs are UUIDs."
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
This writes a markdown file under `knowledge/` with `endpoint:` frontmatter:
|
|
16
|
+
|
|
17
|
+
```markdown
|
|
18
|
+
---
|
|
19
|
+
endpoint: "/users"
|
|
20
|
+
---
|
|
21
|
+
CRUD for users. Admin role required for writes. IDs are UUIDs.
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Chief loads knowledge matching the endpoint it's planning. Running `know` again on the same endpoint appends to the file. See [knowledge](../workflow/knowledge.md) for how matching and files work.
|
|
25
|
+
|
|
26
|
+
## Choose a planning style
|
|
27
|
+
|
|
28
|
+
A style tells Chief what kind of tests to prioritize. Four ship by default:
|
|
29
|
+
|
|
30
|
+
| Style | Focus |
|
|
31
|
+
|-------|-------|
|
|
32
|
+
| `normal` | Standard CRUD and happy paths — create, read, update, delete, verify status codes and schemas |
|
|
33
|
+
| `curious` | Maximum coverage — every field, every enum value, arrays, defaults |
|
|
34
|
+
| `psycho` | Malformed and extreme input — missing fields, injection payloads, wrong content types, boundary values |
|
|
35
|
+
| `hacker` | Security probing — infers hidden endpoints and undocumented fields from responses, then tries privilege escalation, IDOR, and auth-bypass against them |
|
|
36
|
+
|
|
37
|
+
Pick one with `--style`:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
npx explorbot api plan /users --style hacker
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Without `--style`, Chief uses `normal`. Styles are markdown files in `rules/chief/styles/`, so you can edit them or drop in your own. The cycling and customization mechanism is shared with web planning — see [planning styles](../workflow/planning-styles.md).
|
|
44
|
+
|
|
45
|
+
## Replan from scratch or add to a plan
|
|
46
|
+
|
|
47
|
+
By default a `plan` run generates a fresh set of scenarios. When a plan already exists in the same run, Chief compares against it and adds only new, non-duplicate scenarios — it won't re-propose behavior it already covered, even under a different style. Pass `--fresh` to discard the in-progress plan and start clean:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
npx explorbot api plan /users --fresh
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
This is what [`explore`](./running-tests.md) uses to give each style its own clean plan.
|
|
54
|
+
|
|
55
|
+
## Where plans land
|
|
56
|
+
|
|
57
|
+
Chief saves each plan to `output/plans/`, named after the endpoint (for example `output/plans/users.md`). The file is a standard Explorbot [test plan](../workflow/test-plans.md): a suite of scenarios, each with steps, expected outcomes, and a priority. Read it, edit it, or commit it — then run it as described in [Running API tests](./running-tests.md).
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Running API Tests
|
|
2
|
+
|
|
3
|
+
Once Chief has written a plan, Curler runs it. Curler reads each scenario and drives it to a pass or fail by making real HTTP requests and asserting on the responses.
|
|
4
|
+
|
|
5
|
+
## Run a plan
|
|
6
|
+
|
|
7
|
+
Point Curler at a plan file:
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npx explorbot api test output/plans/users.md
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
With no index, Curler runs every pending test. Add an index to run a subset:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
npx explorbot api test output/plans/users.md 1 # the first test
|
|
17
|
+
npx explorbot api test output/plans/users.md 1-3 # tests 1 through 3
|
|
18
|
+
npx explorbot api test output/plans/users.md 1,3,5 # specific tests
|
|
19
|
+
npx explorbot api test output/plans/users.md * # all pending tests
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Curler prints a running log of requests and, at the end, how many tests passed and failed.
|
|
23
|
+
|
|
24
|
+
## What Curler can do
|
|
25
|
+
|
|
26
|
+
Curler works through AI tool calls. Its toolset:
|
|
27
|
+
|
|
28
|
+
- **`request`** — make an HTTP request (any method: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS) with a body, headers, or query params. Returns status, timing, and a preview of the response; the full body is saved to disk. This is how Curler chains requests — it reads an ID from one response and feeds it into the next.
|
|
29
|
+
- **`verifyStructure`** — check the response shape against a Zod schema. On success it reports the actual structure, which Curler uses to write correct value assertions.
|
|
30
|
+
- **`verifyData`** — assert specific values with `expect()` (`toBe`, `toHaveProperty`, `toHaveLength`, and so on).
|
|
31
|
+
- **`schemaFor`** — search the OpenAPI spec for related endpoints when a test needs to set up prerequisite data or discover a dependency.
|
|
32
|
+
- **`record`** — note a finding or observation as the test runs.
|
|
33
|
+
- **`finish`** / **`stop`** — mark the test complete, or abandon it when the scenario is impossible.
|
|
34
|
+
|
|
35
|
+
Curler favors verification over trust: for writes, it follows up with a `GET` to confirm the data actually persisted rather than believing the write response alone.
|
|
36
|
+
|
|
37
|
+
## Debugging with request logs
|
|
38
|
+
|
|
39
|
+
Every request Curler makes is saved to `output/requests/` as a `.request.yaml` file — full URL, headers, body, status, and response. When a test fails and the log isn't enough, open these to see exactly what went over the wire, including a reproducible curl command.
|
|
40
|
+
|
|
41
|
+
## Explore an endpoint end to end
|
|
42
|
+
|
|
43
|
+
`explore` runs the whole cycle autonomously across every planning style:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
npx explorbot api explore /users
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
For each style — `normal`, `curious`, `psycho`, `hacker` — it plans a fresh set of scenarios, runs them with Curler, and saves one plan file per style (`users_normal.md`, `users_hacker.md`, and so on). Scenarios are de-duplicated across styles, so the same test won't run twice. When every style is done, it prints the combined totals.
|
|
50
|
+
|
|
51
|
+
Use `explore` to hammer an endpoint from every angle in one command; use `plan` plus `test` when you want to review or edit scenarios before running them.
|
|
52
|
+
|
|
53
|
+
## Results and reporting
|
|
54
|
+
|
|
55
|
+
Pass/fail results flow through Explorbot's shared reporter — the same one the web side uses. See [reporting](../workflow/reporting.md) for local reports and sending runs to Testomat.io.
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# Getting Started
|
|
2
|
+
|
|
3
|
+
Explorbot explores your web app, plans tests, and runs them — no test scripts. This guide gets you from zero to your first test in about ten minutes.
|
|
4
|
+
|
|
5
|
+
The path is short: install, configure, tell it how to log in, then point it at one feature and let it work.
|
|
6
|
+
|
|
7
|
+
## 1. Install
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npm i explorbot --save
|
|
11
|
+
npx playwright install
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
You need Node.js 24+ (or Bun), an AI provider key, and a modern terminal — iTerm2, WARP, Kitty, Ghostty, or Windows Terminal with WSL. For the full compatibility checklist, see [Prerequisites](./prerequisites.md).
|
|
15
|
+
|
|
16
|
+
## 2. Configure
|
|
17
|
+
|
|
18
|
+
Create the config files:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
npx explorbot init
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
This writes `explorbot.config.js`, an `.env` file for your keys, and an `output/` folder.
|
|
25
|
+
|
|
26
|
+
Open `.env` and add your provider key:
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
OPENROUTER_API_KEY=sk-...
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Then open `explorbot.config.js` and set your app's base URL — the host only, no path:
|
|
33
|
+
|
|
34
|
+
```javascript
|
|
35
|
+
import { createOpenRouter } from '@openrouter/ai-sdk-provider';
|
|
36
|
+
|
|
37
|
+
const openrouter = createOpenRouter({
|
|
38
|
+
apiKey: process.env.OPENROUTER_API_KEY,
|
|
39
|
+
});
|
|
40
|
+
|
|
41
|
+
export default {
|
|
42
|
+
web: {
|
|
43
|
+
url: 'http://localhost:3000',
|
|
44
|
+
},
|
|
45
|
+
ai: {
|
|
46
|
+
model: openrouter('openai/gpt-oss-20b:nitro'),
|
|
47
|
+
visionModel: openrouter('google/gemma-4-31b-it'),
|
|
48
|
+
agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
|
|
49
|
+
},
|
|
50
|
+
};
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Explorbot uses three models. Pick each one for speed and cost:
|
|
54
|
+
|
|
55
|
+
| Model | Config key | Used by | Pick |
|
|
56
|
+
|-------|-----------|---------|------|
|
|
57
|
+
| `model` | `ai.model` | Tester, Navigator, Researcher — they read HTML and ARIA on every step | a fast, cheap model (e.g. `openai/gpt-oss-20b:nitro`) |
|
|
58
|
+
| `visionModel` | `ai.visionModel` | screenshot analysis | a vision model (e.g. `google/gemma-4-31b-it`) |
|
|
59
|
+
| `agenticModel` | `ai.agenticModel` | Captain and Pilot — they read short action logs and make the big decisions | a smarter model (e.g. MiniMax 2.5, Grok Fast) |
|
|
60
|
+
|
|
61
|
+
Captain and Pilot barely use tokens, so a smarter `agenticModel` improves results for almost no extra cost. OpenRouter is the simplest start — one key, many models. To use OpenAI, Anthropic, Groq, or others, see [Providers](./providers.md). For every config option, see [Configuration](../reference/configuration.md).
|
|
62
|
+
|
|
63
|
+
## 3. Tell Explorbot how to log in
|
|
64
|
+
|
|
65
|
+
Most apps need a login. Give Explorbot the credentials once, and it signs in on its own:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx explorbot learn "/login" "Use credentials: admin@example.com / secret123"
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
This saves a knowledge file under `knowledge/`. Explorbot reads it whenever it opens the login page. Use `*` as the URL pattern for knowledge that applies to every page.
|
|
72
|
+
|
|
73
|
+
To skip the login on later runs, add `--session`. Explorbot logs in once and restores the saved cookies next time:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
npx explorbot start /login --session # logs in, saves the session
|
|
77
|
+
npx explorbot start /dashboard --session # restores it, skips login
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Keep real secrets in environment variables, and handle cookie banners, modals, and test data the same way — see [Customization](../web-testing/customization.md).
|
|
81
|
+
|
|
82
|
+
## 4. Pick one feature to test
|
|
83
|
+
|
|
84
|
+
Don't point Explorbot at your homepage. Start it on a single focused feature — a page with a clear, visible CRUD interface it can work with. Good first targets:
|
|
85
|
+
|
|
86
|
+
- `/admin/projects`
|
|
87
|
+
- `/posts`
|
|
88
|
+
- `/admin/users`
|
|
89
|
+
- any list-and-edit or settings page
|
|
90
|
+
|
|
91
|
+
A page where you can create, edit, and delete items gives Explorbot an obvious job and a clear way to tell whether it worked.
|
|
92
|
+
|
|
93
|
+
## 5. Run
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
npx explorbot start /admin/projects
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
The browser runs hidden by default. Add `--show` to watch it:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
npx explorbot start /admin/projects --show
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
When the terminal UI opens, type `/explore`. Explorbot researches the page, plans tests, runs them, and repeats. To go one step at a time:
|
|
106
|
+
|
|
107
|
+
> [!WARNING]
|
|
108
|
+
> Run your first `/explore` against staging, a disposable workspace, or another isolated environment with non-production data. Explorbot can create, edit, and delete records while testing. Make sure the data is safe to change and easy to restore.
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
/research # analyze the current page
|
|
112
|
+
/plan # propose test scenarios
|
|
113
|
+
/test # run the next test
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
### What a successful run looks like
|
|
117
|
+
|
|
118
|
+
A completed exploration shows the test totals, a session analysis, the covered features, and any execution issues that need review:
|
|
119
|
+
|
|
120
|
+

|
|
121
|
+
|
|
122
|
+
## The concepts
|
|
123
|
+
|
|
124
|
+
You have now touched everything Explorbot is built on. Here is the whole vocabulary, once:
|
|
125
|
+
|
|
126
|
+
- **State** — where the bot is: the page URL plus its main headings (`h1`, `h2`). States anchor navigation, learning, and loop detection.
|
|
127
|
+
- **Research** — reading a page. The Researcher agent maps forms, buttons, tables, and navigation into a UI map the other agents work from. Saved under `output/research/`. See [Researcher](../web-testing/researcher.md).
|
|
128
|
+
- **Plan** — test scenarios invented from research, with priorities and expected outcomes. Markdown you can read and edit, in `output/plans/`. See [Test plans](../workflow/test-plans.md).
|
|
129
|
+
- **Test** — one scenario executed step by step in the real browser. Passing tests are saved as runnable Playwright or CodeceptJS code in `output/tests/`. See [Automated tests](../web-testing/automated-tests.md).
|
|
130
|
+
- **Knowledge** — facts you teach Explorbot: credentials, quirks, hints. Markdown files in `knowledge/`, matched to pages by URL — you wrote your first one in step 3. See [Knowledge](../workflow/knowledge.md).
|
|
131
|
+
- **Experience** — what Explorbot learns by doing: failed attempts and the fixes that worked, saved in `experience/` and reused on every later run. Knowledge you write; experience it earns. See [Learning](../web-testing/basics.md#learning).
|
|
132
|
+
- **Agents** — the AI workers behind each step: Researcher, Planner, Tester, Pilot, and more, each with its own job and model. See [Agents](../web-testing/agents.md).
|
|
133
|
+
- **Report** — the end-of-session summary of defects, UX findings, and coverage in `output/reports/`. See [Reporting](../workflow/reporting.md).
|
|
134
|
+
|
|
135
|
+
## Next steps
|
|
136
|
+
|
|
137
|
+
- [Running Explorbot](./running.md) — the TUI you just used, the headless CLI, and when to use each.
|
|
138
|
+
- [Customization](../web-testing/customization.md) — login, cookie bars, modals, and test data.
|
|
139
|
+
- [Commands](../reference/commands.md) — every command, in the terminal and on the CLI.
|
|
140
|
+
- [Knowledge](../workflow/knowledge.md) — teach Explorbot more about your app.
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Application prerequisites
|
|
2
|
+
|
|
3
|
+
## Security
|
|
4
|
+
|
|
5
|
+
Give Explorbot an isolated environment that can never touch production data. This is your responsibility.
|
|
6
|
+
|
|
7
|
+
Explorbot logs in with a user session and predefined credentials. Give that user limited permissions so it cannot harm the environment.
|
|
8
|
+
|
|
9
|
+
Do not put real sensitive data in that environment. Explorbot has no secret redaction: anything the browser sees can end up in logs, AI prompts, and saved artifacts. Use disposable test data.
|
|
10
|
+
|
|
11
|
+
Explorbot runs with few privileges. Explorbot cannot:
|
|
12
|
+
|
|
13
|
+
- read or write local files (except the Captain agent, and only in the `knowledge/`, `experience/`, and `output/` folders)
|
|
14
|
+
- fetch content from external websites (it stays on the configured site)
|
|
15
|
+
- run Bash or CLI tools like `git` or `rm -rf` (except the Captain agent, for a limited set of actions)
|
|
16
|
+
|
|
17
|
+
Explorbot follows a predefined script and a plan. It does not act on its own. Its main actions are web and API requests, which you should restrict.
|
|
18
|
+
|
|
19
|
+
Run Explorbot on local, dev, or staging environments. You can still run it on production for regression testing, as long as its user has narrow access and works in a single project or workspace.
|
|
20
|
+
|
|
21
|
+
## Data management
|
|
22
|
+
|
|
23
|
+
Run Explorbot against an isolated workspace: a separate project inside your app, a staging environment, or similar. Pre-populate it with data. Explorbot reads that data to learn what the application does and proposes more meaningful tests.
|
|
24
|
+
|
|
25
|
+
Explorbot can change or delete data through the web interface. Make sure you can reload that data if something breaks.
|
|
26
|
+
|
|
27
|
+
To prepare:
|
|
28
|
+
|
|
29
|
+
- Add a dataset to the pages Explorbot will visit, so it learns the app faster.
|
|
30
|
+
- Keep that data non-critical and easy to restore.
|
|
31
|
+
|
|
32
|
+
Explorbot works best on CRUD-heavy web applications:
|
|
33
|
+
|
|
34
|
+
- SaaS platforms
|
|
35
|
+
- ecommerce
|
|
36
|
+
- ERP
|
|
37
|
+
- admin panels
|
|
38
|
+
- internal tools
|
|
39
|
+
|
|
40
|
+
It is not a good fit for landing pages, blogs, CMS, or static sites.
|
|
41
|
+
|
|
42
|
+
## Page management
|
|
43
|
+
|
|
44
|
+
Explorbot uses URLs as anchor points when it navigates. Each change in the URL or the `h1`/`h2` headings creates a new state. This is how Explorbot tracks where it is and analyzes each transition. The `title` and `h3`/`h4` headings are captured too, but they do not define state identity.
|
|
45
|
+
|
|
46
|
+
If your app does not change the URL on navigation, or skips `h1`/`h2` headings, navigation gets harder. Use URLs to identify your application states.
|
|
47
|
+
|
|
48
|
+
Set edge cases and domain rules in [Knowledge files](../workflow/knowledge.md). These attach to page URLs.
|
|
49
|
+
|
|
50
|
+
## Web elements
|
|
51
|
+
|
|
52
|
+
Explorbot identifies elements through HTML, ARIA, and the screenshot. When one strategy fails, another usually works.
|
|
53
|
+
|
|
54
|
+
Avoid long scrolling pages. They make visual identification harder.
|
|
55
|
+
|
|
56
|
+
Explorbot reads ARIA attributes first, then falls back to HTML when ARIA elements are empty. An ARIA tree like this still works:
|
|
57
|
+
|
|
58
|
+
```
|
|
59
|
+
- role: button
|
|
60
|
+
text:
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Following A11y standards across your site improves results. Explorbot makes fewer failed attempts on common elements.
|