explorbot 0.2.2 → 0.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/bin/explorbot-cli.ts +59 -38
- package/boat/api-tester/src/apibot.ts +4 -2
- package/boat/api-tester/src/cli.ts +2 -2
- package/boat/api-tester/src/config.ts +43 -10
- package/boat/doc-collector/bin/doc-collector-cli.ts +2 -0
- package/boat/doc-collector/src/ai/documentarian.ts +61 -31
- package/boat/doc-collector/src/cli.ts +1 -0
- package/boat/doc-collector/src/config.ts +4 -2
- package/boat/doc-collector/src/docs-renderer.ts +18 -4
- package/boat/doc-collector/src/state-diagram.ts +61 -14
- package/boat/prima/bin/prima-cli.ts +5 -0
- package/boat/prima/package.json +16 -0
- package/boat/prima/src/activity-line.ts +33 -0
- package/boat/prima/src/cli.ts +251 -0
- package/boat/prima/src/envelope.ts +169 -0
- package/boat/prima/src/prima.ts +1124 -0
- package/boat/prima/src/pw-parser.ts +27 -0
- package/boat/prima/src/pw-registry.ts +74 -0
- package/boat/prima/src/session-log.ts +126 -0
- package/dist/bin/explorbot-cli.js +51 -32
- package/dist/boat/api-tester/bin/apibot-cli.js +2 -0
- package/dist/boat/api-tester/src/apibot.js +3 -2
- package/dist/boat/api-tester/src/cli.js +2 -2
- package/dist/boat/api-tester/src/config.js +40 -10
- package/dist/boat/doc-collector/bin/doc-collector-cli.js +2 -0
- package/dist/boat/doc-collector/src/ai/documentarian.js +44 -19
- package/dist/boat/doc-collector/src/cli.js +1 -0
- package/dist/boat/doc-collector/src/config.js +4 -2
- package/dist/boat/doc-collector/src/docs-renderer.js +17 -3
- package/dist/boat/doc-collector/src/state-diagram.js +57 -13
- package/dist/boat/prima/bin/prima-cli.js +4 -0
- package/dist/boat/prima/src/activity-line.js +30 -0
- package/dist/boat/prima/src/cli.js +220 -0
- package/dist/boat/prima/src/envelope.js +142 -0
- package/dist/boat/prima/src/prima.js +1031 -0
- package/dist/boat/prima/src/pw-parser.js +30 -0
- package/dist/boat/prima/src/pw-registry.js +65 -0
- package/dist/boat/prima/src/session-log.js +108 -0
- package/dist/models.json +3 -0
- package/dist/package.json +8 -3
- package/dist/rules/navigator/verification-actions.md +20 -0
- package/dist/src/action-result.d.ts +7 -0
- package/dist/src/action-result.js +4 -0
- package/dist/src/action.d.ts +7 -2
- package/dist/src/action.js +41 -6
- package/dist/src/ai/captain/mixin.js +3 -4
- package/dist/src/ai/captain/web-mode.js +7 -4
- package/dist/src/ai/captain.js +2 -0
- package/dist/src/ai/navigator.d.ts +11 -0
- package/dist/src/ai/navigator.js +21 -12
- package/dist/src/ai/pilot.d.ts +4 -0
- package/dist/src/ai/pilot.js +48 -2
- package/dist/src/ai/planner.d.ts +1 -0
- package/dist/src/ai/planner.js +6 -0
- package/dist/src/ai/provider.js +2 -2
- package/dist/src/ai/rerunner.js +2 -1
- package/dist/src/ai/researcher.js +2 -2
- package/dist/src/ai/rules.js +17 -10
- package/dist/src/ai/task-agent.js +5 -2
- package/dist/src/ai/tester.d.ts +1 -0
- package/dist/src/ai/tester.js +33 -20
- package/dist/src/ai/tools.d.ts +14 -0
- package/dist/src/ai/tools.js +113 -36
- package/dist/src/application-spec-contract.d.ts +8 -0
- package/dist/src/application-spec-contract.js +8 -0
- package/dist/src/application-spec.d.ts +15 -0
- package/dist/src/application-spec.js +71 -0
- package/dist/src/browser-server.d.ts +12 -6
- package/dist/src/browser-server.js +74 -19
- package/dist/src/commands/clean-command.js +2 -7
- package/dist/src/commands/init-command.d.ts +5 -0
- package/dist/src/commands/init-command.js +119 -1
- package/dist/src/commands/navigate-command.js +1 -1
- package/dist/src/commands/research-command.js +1 -1
- package/dist/src/commands/sites-command.d.ts +6 -0
- package/dist/src/commands/sites-command.js +23 -0
- package/dist/src/components/InitWizard.d.ts +10 -0
- package/dist/src/components/InitWizard.js +133 -0
- package/dist/src/components/InputReadline.d.ts +1 -0
- package/dist/src/components/InputReadline.js +7 -4
- package/dist/src/config.d.ts +25 -5
- package/dist/src/config.js +158 -40
- package/dist/src/execution-controller.d.ts +2 -0
- package/dist/src/execution-controller.js +6 -0
- package/dist/src/explorbot.d.ts +11 -1
- package/dist/src/explorbot.js +27 -6
- package/dist/src/explorer.d.ts +4 -1
- package/dist/src/explorer.js +42 -9
- package/dist/src/global-config.d.ts +22 -0
- package/dist/src/global-config.js +117 -0
- package/dist/src/knowledge-tracker.d.ts +5 -1
- package/dist/src/knowledge-tracker.js +14 -1
- package/dist/src/playwright-recorder.js +30 -0
- package/dist/src/remote.d.ts +54 -0
- package/dist/src/remote.js +229 -0
- package/dist/src/reporter.d.ts +1 -0
- package/dist/src/reporter.js +7 -1
- package/dist/src/state-manager.d.ts +2 -1
- package/dist/src/stats.d.ts +1 -0
- package/dist/src/stats.js +1 -0
- package/dist/src/test-plan.d.ts +2 -0
- package/dist/src/test-plan.js +7 -0
- package/dist/src/utils/aria.d.ts +2 -8
- package/dist/src/utils/aria.js +69 -40
- package/dist/src/utils/cli-name.js +6 -2
- package/dist/src/utils/html.js +1 -0
- package/dist/src/utils/logger.d.ts +6 -0
- package/dist/src/utils/logger.js +24 -0
- package/dist/src/utils/page-readiness.js +18 -1
- package/dist/src/utils/test-files.js +1 -2
- package/dist/src/utils/url-matcher.d.ts +1 -0
- package/dist/src/utils/url-matcher.js +12 -0
- package/dist/src/utils/web-element.d.ts +2 -0
- package/dist/src/utils/web-element.js +8 -0
- package/dist/src/utils/web-sandbox.d.ts +1 -1
- package/dist/src/utils/web-sandbox.js +2 -3
- package/docs/api-testing/basics.md +90 -0
- package/docs/api-testing/planning.md +57 -0
- package/docs/api-testing/running-tests.md +55 -0
- package/docs/assets/cloud-report.png +0 -0
- package/docs/assets/html-report.png +0 -0
- package/docs/assets/langfuse-trace.png +0 -0
- package/docs/assets/successful-explore-run.png +0 -0
- package/docs/basics/getting-started.md +140 -0
- package/docs/basics/prerequisites.md +63 -0
- package/docs/basics/providers.md +362 -0
- package/docs/basics/running.md +78 -0
- package/docs/contributing/ai-integration-tests.md +57 -0
- package/docs/contributing/contributing.md +90 -0
- package/docs/contributing/demo-videos.md +36 -0
- package/docs/contributing/npm-package.md +138 -0
- package/docs/contributing/observability.md +227 -0
- package/docs/contributing/regression-tests.md +103 -0
- package/docs/contributing/testing.md +95 -0
- package/docs/doc-collection/basics.md +128 -0
- package/docs/doc-collection/crawling.md +67 -0
- package/docs/doc-collection/interactive-mode.md +99 -0
- package/docs/index.json +86 -0
- package/docs/reference/commands.md +994 -0
- package/docs/reference/configuration.md +569 -0
- package/docs/reference/scripting.md +303 -0
- package/docs/superpowers/plans/2026-08-01-actor-boat.md +925 -0
- package/docs/superpowers/plans/2026-08-01-prima-boat.md +1120 -0
- package/docs/superpowers/reviews/2026-08-06-prima-vs-playwright-cli.md +268 -0
- package/docs/superpowers/specs/2026-08-01-actor-boat-design.md +204 -0
- package/docs/superpowers/specs/2026-08-01-prima-boat-design.md +242 -0
- package/docs/superpowers/specs/2026-08-03-global-config-design.md +138 -0
- package/docs/superpowers/specs/2026-08-07-prima-fixes-design.md +394 -0
- package/docs/web-testing/agents.md +158 -0
- package/docs/web-testing/automated-tests.md +134 -0
- package/docs/web-testing/basics.md +91 -0
- package/docs/web-testing/customization.md +131 -0
- package/docs/web-testing/hooks.md +238 -0
- package/docs/web-testing/page-interaction.md +84 -0
- package/docs/web-testing/planner.md +122 -0
- package/docs/web-testing/rerun.md +164 -0
- package/docs/web-testing/researcher.md +380 -0
- package/docs/workflow/agentic-usage.md +233 -0
- package/docs/workflow/application-spec.md +73 -0
- package/docs/workflow/ci.md +202 -0
- package/docs/workflow/knowledge.md +310 -0
- package/docs/workflow/planning-styles.md +67 -0
- package/docs/workflow/reporting.md +133 -0
- package/docs/workflow/test-plans.md +90 -0
- package/models.json +3 -0
- package/package.json +8 -3
- package/rules/navigator/verification-actions.md +20 -0
- package/src/action-result.ts +11 -0
- package/src/action.ts +47 -7
- package/src/ai/captain/mixin.ts +3 -3
- package/src/ai/captain/web-mode.ts +7 -4
- package/src/ai/captain.ts +3 -0
- package/src/ai/navigator.ts +26 -15
- package/src/ai/pilot.ts +53 -2
- package/src/ai/planner.ts +7 -0
- package/src/ai/provider.ts +2 -4
- package/src/ai/rerunner.ts +2 -1
- package/src/ai/researcher.ts +2 -2
- package/src/ai/rules.ts +17 -10
- package/src/ai/task-agent.ts +4 -2
- package/src/ai/tester.ts +32 -18
- package/src/ai/tools.ts +132 -36
- package/src/application-spec-contract.ts +10 -0
- package/src/application-spec.ts +87 -0
- package/src/browser-server.ts +74 -19
- package/src/commands/clean-command.ts +1 -6
- package/src/commands/init-command.ts +146 -1
- package/src/commands/navigate-command.ts +1 -1
- package/src/commands/research-command.ts +1 -1
- package/src/commands/sites-command.ts +27 -0
- package/src/components/InitWizard.tsx +166 -0
- package/src/components/InputReadline.tsx +8 -4
- package/src/config.ts +177 -43
- package/src/execution-controller.ts +8 -0
- package/src/explorbot.ts +34 -7
- package/src/explorer.ts +47 -9
- package/src/global-config.ts +148 -0
- package/src/knowledge-tracker.ts +17 -1
- package/src/playwright-recorder.ts +23 -0
- package/src/remote.ts +238 -0
- package/src/reporter.ts +7 -1
- package/src/state-manager.ts +2 -1
- package/src/stats.ts +1 -0
- package/src/test-plan.ts +9 -0
- package/src/utils/aria.ts +65 -45
- package/src/utils/cli-name.ts +5 -2
- package/src/utils/html.ts +1 -0
- package/src/utils/logger.ts +24 -1
- package/src/utils/page-readiness.ts +24 -1
- package/src/utils/test-files.ts +1 -2
- package/src/utils/url-matcher.ts +13 -0
- package/src/utils/web-element.ts +9 -0
- package/src/utils/web-sandbox.ts +3 -4
package/dist/src/utils/aria.js
CHANGED
|
@@ -78,8 +78,7 @@ const parseLabel = (label) => {
|
|
|
78
78
|
rest = rest.slice(nameMatch[0].length);
|
|
79
79
|
}
|
|
80
80
|
const attributes = {};
|
|
81
|
-
const attrMatch
|
|
82
|
-
if (attrMatch) {
|
|
81
|
+
for (const attrMatch of rest.matchAll(/\[([^\]]*)\]/g)) {
|
|
83
82
|
for (const tok of attrMatch[1].split(/[\s,]+/).filter(Boolean)) {
|
|
84
83
|
const eq = tok.indexOf('=');
|
|
85
84
|
if (eq === -1) {
|
|
@@ -189,7 +188,8 @@ const dropEmpty = (nodes, opts = {}) => nodes.flatMap((node) => {
|
|
|
189
188
|
// STEP 3 · Render: AriaNode[] → text or flat entries
|
|
190
189
|
// ─────────────────────────────────────────────────────────────────
|
|
191
190
|
// One-line representation of a node. Stable attr order so diff comparisons are deterministic.
|
|
192
|
-
const
|
|
191
|
+
const MAX_INLINE_VALUE = 400;
|
|
192
|
+
const formatNode = (node, offload) => {
|
|
193
193
|
let line = node.role;
|
|
194
194
|
if (node.name?.trim())
|
|
195
195
|
line += ` "${node.name.trim()}"`;
|
|
@@ -209,11 +209,20 @@ const formatNode = (node) => {
|
|
|
209
209
|
line += ` [${attrStr}]`;
|
|
210
210
|
if (node.value !== undefined && node.value !== null) {
|
|
211
211
|
const text = String(node.value).trim();
|
|
212
|
-
if (text)
|
|
212
|
+
if (text && text.length <= MAX_INLINE_VALUE)
|
|
213
213
|
line += `: ${text}`;
|
|
214
|
+
if (text && text.length > MAX_INLINE_VALUE)
|
|
215
|
+
line += `: ${offloadValue(text, offload)}`;
|
|
214
216
|
}
|
|
215
217
|
return line;
|
|
216
218
|
};
|
|
219
|
+
const offloadValue = (text, offload) => {
|
|
220
|
+
const head = `${text.slice(0, MAX_INLINE_VALUE)}…`;
|
|
221
|
+
const reference = offload?.(text);
|
|
222
|
+
if (!reference)
|
|
223
|
+
return `${head} (${text.length} chars, truncated)`;
|
|
224
|
+
return `${head} (${text.length} chars) [Full Text: ${reference}]`;
|
|
225
|
+
};
|
|
217
226
|
// Group consecutive same-role siblings. [a,a,b,a,a,a] → [[a,a],[b],[a,a,a]]
|
|
218
227
|
const groupByConsecutiveRole = (nodes) => nodes.reduce((groups, node) => {
|
|
219
228
|
const last = groups[groups.length - 1];
|
|
@@ -236,16 +245,16 @@ const collapseGroup = (group, depth) => {
|
|
|
236
245
|
};
|
|
237
246
|
const collapseSiblingGroups = (nodes, depth) => groupByConsecutiveRole(nodes).flatMap((group) => collapseGroup(group, depth));
|
|
238
247
|
// Tree → indented YAML text.
|
|
239
|
-
const renderTree = (nodes, depth = 0) => collapseSiblingGroups(nodes, depth)
|
|
248
|
+
const renderTree = (nodes, depth = 0, offload) => collapseSiblingGroups(nodes, depth)
|
|
240
249
|
.map((entry) => {
|
|
241
250
|
if ('placeholder' in entry)
|
|
242
251
|
return entry.placeholder;
|
|
243
252
|
const { node } = entry;
|
|
244
253
|
const indent = ' '.repeat(depth);
|
|
245
|
-
const head = `${indent}- ${formatNode(node)}`;
|
|
254
|
+
const head = `${indent}- ${formatNode(node, offload)}`;
|
|
246
255
|
if (node.children.length === 0)
|
|
247
256
|
return head;
|
|
248
|
-
return `${head}:\n${renderTree(node.children, depth + 1)}`;
|
|
257
|
+
return `${head}:\n${renderTree(node.children, depth + 1, offload)}`;
|
|
249
258
|
})
|
|
250
259
|
.join('\n');
|
|
251
260
|
// Build the structured "entry" object for an interactive node, or null if not worth keeping.
|
|
@@ -384,6 +393,42 @@ const detectToggles = (prev, curr) => {
|
|
|
384
393
|
}
|
|
385
394
|
return { toggled, togglePaths };
|
|
386
395
|
};
|
|
396
|
+
const detectValueChanges = (prev, curr) => {
|
|
397
|
+
const typed = [];
|
|
398
|
+
const typedPaths = new Set();
|
|
399
|
+
const currByPath = new Map(curr.map((e) => [e.path, e]));
|
|
400
|
+
for (const before of prev) {
|
|
401
|
+
const after = currByPath.get(before.path);
|
|
402
|
+
if (!after)
|
|
403
|
+
continue;
|
|
404
|
+
if (before.entry.role !== after.entry.role)
|
|
405
|
+
continue;
|
|
406
|
+
if (before.entry.name !== after.entry.name)
|
|
407
|
+
continue;
|
|
408
|
+
const was = valueWord(before.entry.value);
|
|
409
|
+
const now = valueWord(after.entry.value);
|
|
410
|
+
if (was === now)
|
|
411
|
+
continue;
|
|
412
|
+
typedPaths.add(before.path);
|
|
413
|
+
let label = String(after.entry.role);
|
|
414
|
+
const name = after.entry.name;
|
|
415
|
+
if (typeof name === 'string' && name.trim())
|
|
416
|
+
label += ` "${name.trim()}"`;
|
|
417
|
+
typed.push(`${label}: ${was} -> ${now}`);
|
|
418
|
+
}
|
|
419
|
+
return { typed, typedPaths };
|
|
420
|
+
};
|
|
421
|
+
const VALUE_EXCERPT = 60;
|
|
422
|
+
const valueWord = (value) => {
|
|
423
|
+
if (value === undefined || value === null)
|
|
424
|
+
return 'empty';
|
|
425
|
+
const text = String(value).trim();
|
|
426
|
+
if (!text)
|
|
427
|
+
return 'empty';
|
|
428
|
+
if (text.length <= VALUE_EXCERPT)
|
|
429
|
+
return JSON.stringify(text);
|
|
430
|
+
return `${JSON.stringify(text.slice(0, VALUE_EXCERPT))}… (${text.length} chars)`;
|
|
431
|
+
};
|
|
387
432
|
const TOP_DIFF_ITEMS = 10;
|
|
388
433
|
const formatDiffSection = (label, items) => {
|
|
389
434
|
const summary = countBy(items);
|
|
@@ -407,10 +452,15 @@ const formatDiffSection = (label, items) => {
|
|
|
407
452
|
}
|
|
408
453
|
return lines;
|
|
409
454
|
};
|
|
410
|
-
const formatDiff = (added, removed, toggled) => {
|
|
411
|
-
if (added.length === 0 && removed.length === 0 && toggled.length === 0)
|
|
455
|
+
const formatDiff = (added, removed, toggled, typed = []) => {
|
|
456
|
+
if (added.length === 0 && removed.length === 0 && toggled.length === 0 && typed.length === 0)
|
|
412
457
|
return null;
|
|
413
458
|
const sections = ['ariaDiff:'];
|
|
459
|
+
if (typed.length > 0) {
|
|
460
|
+
sections.push(' typed:');
|
|
461
|
+
for (const line of typed)
|
|
462
|
+
sections.push(` - ${line}`);
|
|
463
|
+
}
|
|
414
464
|
if (toggled.length > 0) {
|
|
415
465
|
sections.push(' toggled:');
|
|
416
466
|
for (const line of toggled)
|
|
@@ -461,17 +511,14 @@ const findDialogOrModal = (nodes) => {
|
|
|
461
511
|
}
|
|
462
512
|
return null;
|
|
463
513
|
};
|
|
464
|
-
|
|
465
|
-
// Public API — pipelines composed visibly, top-to-bottom
|
|
466
|
-
// ─────────────────────────────────────────────────────────────────
|
|
467
|
-
export const compactAriaSnapshot = (snapshot, keepNamed = false) => {
|
|
514
|
+
export const compactAriaSnapshot = (snapshot, keepNamed = false, offload) => {
|
|
468
515
|
if (!snapshot)
|
|
469
516
|
return '';
|
|
470
517
|
let tree = parseSnapshot(snapshot);
|
|
471
518
|
tree = unwrapIgnored(tree);
|
|
472
519
|
tree = nameIconButtons(tree);
|
|
473
520
|
tree = dropEmpty(tree, { keepNamed });
|
|
474
|
-
return renderTree(tree);
|
|
521
|
+
return renderTree(tree, 0, offload);
|
|
475
522
|
};
|
|
476
523
|
export const diffAriaSnapshots = (previous, current) => {
|
|
477
524
|
const flat = (snap) => {
|
|
@@ -484,15 +531,17 @@ export const diffAriaSnapshots = (previous, current) => {
|
|
|
484
531
|
const prevAll = flat(previous);
|
|
485
532
|
const currAll = flat(current);
|
|
486
533
|
const { toggled, togglePaths } = detectToggles(prevAll, currAll);
|
|
487
|
-
const
|
|
488
|
-
const
|
|
534
|
+
const { typed, typedPaths } = detectValueChanges(prevAll, currAll);
|
|
535
|
+
const skip = (entry) => togglePaths.has(entry.path) || typedPaths.has(entry.path);
|
|
536
|
+
const prev = prevAll.filter((e) => !skip(e));
|
|
537
|
+
const curr = currAll.filter((e) => !skip(e));
|
|
489
538
|
const prevTotals = countBy(prev.map((e) => e.summary));
|
|
490
539
|
const currTotals = countBy(curr.map((e) => e.summary));
|
|
491
540
|
const byCount = diffByCount(prevTotals, currTotals);
|
|
492
541
|
const renames = detectRenames(prev, curr, prevTotals, currTotals);
|
|
493
542
|
const added = [...byCount.added, ...renames.added];
|
|
494
543
|
const removed = [...byCount.removed, ...renames.removed];
|
|
495
|
-
return { text: formatDiff(added, removed, toggled), count: added.length + removed.length + toggled.length };
|
|
544
|
+
return { text: formatDiff(added, removed, toggled, typed), count: added.length + removed.length + toggled.length + typed.length };
|
|
496
545
|
};
|
|
497
546
|
export const detectFocusArea = (snapshot) => {
|
|
498
547
|
let tree = parseSnapshot(snapshot);
|
|
@@ -513,29 +562,9 @@ export const collectInteractiveNodes = (snapshot) => {
|
|
|
513
562
|
tree = dropEmpty(tree);
|
|
514
563
|
return flatten(tree).map((e) => e.entry);
|
|
515
564
|
};
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
const focusedMatch = ariaSnapshot.match(/-\s*(\w+)\s+"([^"]*)"([^:\n]*)\[focused\](?::\s*(.*))?/);
|
|
520
|
-
if (!focusedMatch)
|
|
521
|
-
return null;
|
|
522
|
-
const [, role, name, attributesStr, value] = focusedMatch;
|
|
523
|
-
const attributes = [];
|
|
524
|
-
if (attributesStr) {
|
|
525
|
-
const attrMatches = attributesStr.matchAll(/\[([^\]]+)\]/g);
|
|
526
|
-
for (const match of attrMatches) {
|
|
527
|
-
if (match[1] !== 'focused') {
|
|
528
|
-
attributes.push(match[1]);
|
|
529
|
-
}
|
|
530
|
-
}
|
|
531
|
-
}
|
|
532
|
-
const result = { role, name };
|
|
533
|
-
if (value)
|
|
534
|
-
result.value = value.trim();
|
|
535
|
-
if (attributes.length > 0)
|
|
536
|
-
result.attributes = attributes;
|
|
537
|
-
return result;
|
|
538
|
-
}
|
|
565
|
+
// ─────────────────────────────────────────────────────────────────
|
|
566
|
+
// Standalone helpers (regex on raw strings — not part of the pipeline)
|
|
567
|
+
// ─────────────────────────────────────────────────────────────────
|
|
539
568
|
export function parseAriaLocator(ariaStr) {
|
|
540
569
|
const trimmed = ariaStr.trim();
|
|
541
570
|
if (trimmed === '-' || trimmed === '' || trimmed === '"-"')
|
|
@@ -8,8 +8,12 @@ export function getCliName() {
|
|
|
8
8
|
cached = 'bunx explorbot';
|
|
9
9
|
else if (ua.includes('npm'))
|
|
10
10
|
cached = 'npx explorbot';
|
|
11
|
-
else if (process.argv[1]?.endsWith('.ts'))
|
|
12
|
-
|
|
11
|
+
else if (process.argv[1]?.endsWith('.ts')) {
|
|
12
|
+
let script = path.relative(process.cwd(), process.argv[1]);
|
|
13
|
+
if (script.startsWith('..'))
|
|
14
|
+
script = process.argv[1];
|
|
15
|
+
cached = `bun ${script}`;
|
|
16
|
+
}
|
|
13
17
|
else
|
|
14
18
|
cached = 'explorbot';
|
|
15
19
|
return cached;
|
package/dist/src/utils/html.js
CHANGED
|
@@ -1376,6 +1376,7 @@ function cleanElement(element) {
|
|
|
1376
1376
|
attr.value = attr.value
|
|
1377
1377
|
.split(/\s+/)
|
|
1378
1378
|
.filter((className) => !/\d/.test(className))
|
|
1379
|
+
.filter((className) => !className.includes(':'))
|
|
1379
1380
|
.filter((className) => !TAILWIND_CLASS_PATTERNS.some((pattern) => pattern.test(className)))
|
|
1380
1381
|
.join(' ');
|
|
1381
1382
|
if (!attr.value) {
|
|
@@ -9,6 +9,10 @@ export interface TaggedLogEntry {
|
|
|
9
9
|
maxLines?: number;
|
|
10
10
|
}
|
|
11
11
|
type LogEntry = TaggedLogEntry;
|
|
12
|
+
export interface LogDestination {
|
|
13
|
+
isEnabled(): boolean;
|
|
14
|
+
write(entry: TaggedLogEntry): void;
|
|
15
|
+
}
|
|
12
16
|
export declare const tag: (type: LogType) => {
|
|
13
17
|
log: (...args: any[]) => void;
|
|
14
18
|
};
|
|
@@ -23,9 +27,11 @@ export declare const startLogCapture: () => void;
|
|
|
23
27
|
export declare const stopLogCapture: () => string[];
|
|
24
28
|
export declare const setVerboseMode: (enabled: boolean) => void;
|
|
25
29
|
export declare const setPreserveConsoleLogs: (enabled: boolean) => void;
|
|
30
|
+
export declare const setQuietMode: (enabled: boolean) => void;
|
|
26
31
|
export declare const isVerboseMode: () => boolean;
|
|
27
32
|
export declare const setDebugMode: (enabled: boolean) => void;
|
|
28
33
|
export declare const isDebugMode: () => boolean;
|
|
34
|
+
export declare const addDestination: (destination: LogDestination) => void;
|
|
29
35
|
export declare const registerLogPane: (addLog: (entry: LogEntry) => void) => void;
|
|
30
36
|
export declare const unregisterLogPane: (addLog: (entry: LogEntry) => void) => void;
|
|
31
37
|
export declare const addTruncateTag: (tagName: string) => void;
|
package/dist/src/utils/logger.js
CHANGED
|
@@ -52,13 +52,19 @@ const debugFilter = new DebugFilter();
|
|
|
52
52
|
class ConsoleDestination {
|
|
53
53
|
verboseMode = false;
|
|
54
54
|
forceEnabled = false;
|
|
55
|
+
quiet = false;
|
|
55
56
|
recentSteps = new RecentStepFilter();
|
|
56
57
|
isEnabled() {
|
|
58
|
+
if (this.quiet)
|
|
59
|
+
return false;
|
|
57
60
|
return this.forceEnabled || !process.env.INK_RUNNING;
|
|
58
61
|
}
|
|
59
62
|
forceEnable(enabled) {
|
|
60
63
|
this.forceEnabled = enabled;
|
|
61
64
|
}
|
|
65
|
+
setQuiet(enabled) {
|
|
66
|
+
this.quiet = enabled;
|
|
67
|
+
}
|
|
62
68
|
setVerboseMode(enabled) {
|
|
63
69
|
this.verboseMode = enabled;
|
|
64
70
|
}
|
|
@@ -113,6 +119,9 @@ class DebugDestination {
|
|
|
113
119
|
}
|
|
114
120
|
setVerboseMode(enabled) {
|
|
115
121
|
this.verboseMode = enabled;
|
|
122
|
+
// the debug package reads DEBUG when it loads, so a namespace turned on later needs enabling by hand
|
|
123
|
+
if (enabled)
|
|
124
|
+
debug.enable(process.env.DEBUG || 'explorbot:*');
|
|
116
125
|
}
|
|
117
126
|
write(namespace, ...args) {
|
|
118
127
|
if (!this.isNamespaceEnabled(namespace))
|
|
@@ -296,6 +305,7 @@ class Logger {
|
|
|
296
305
|
span = new SpanDestination();
|
|
297
306
|
react = new ReactDestination();
|
|
298
307
|
captain = new CaptainDestination();
|
|
308
|
+
extra = [];
|
|
299
309
|
truncateTags = ['page_html'];
|
|
300
310
|
constructor() { }
|
|
301
311
|
static getInstance() {
|
|
@@ -317,9 +327,17 @@ class Logger {
|
|
|
317
327
|
setPreserveConsoleLogs(enabled) {
|
|
318
328
|
this.console.forceEnable(enabled);
|
|
319
329
|
}
|
|
330
|
+
setQuietMode(enabled) {
|
|
331
|
+
this.console.setQuiet(enabled);
|
|
332
|
+
}
|
|
320
333
|
isVerboseMode() {
|
|
321
334
|
return this.debugDestination.isEnabled();
|
|
322
335
|
}
|
|
336
|
+
addDestination(destination) {
|
|
337
|
+
if (this.extra.includes(destination))
|
|
338
|
+
return;
|
|
339
|
+
this.extra.push(destination);
|
|
340
|
+
}
|
|
323
341
|
registerLogPane(addLog) {
|
|
324
342
|
this.react.registerLogPane(addLog);
|
|
325
343
|
}
|
|
@@ -406,6 +424,10 @@ class Logger {
|
|
|
406
424
|
this.span.write(entry);
|
|
407
425
|
if (this.captain.isEnabled())
|
|
408
426
|
this.captain.write(entry);
|
|
427
|
+
for (const destination of this.extra) {
|
|
428
|
+
if (destination.isEnabled())
|
|
429
|
+
destination.write(entry);
|
|
430
|
+
}
|
|
409
431
|
if (process.env.INK_RUNNING) {
|
|
410
432
|
this.react.write(entry);
|
|
411
433
|
}
|
|
@@ -475,9 +497,11 @@ export const startLogCapture = () => logger.captain.startCapture();
|
|
|
475
497
|
export const stopLogCapture = () => logger.captain.stopCapture();
|
|
476
498
|
export const setVerboseMode = (enabled) => logger.setVerboseMode(enabled);
|
|
477
499
|
export const setPreserveConsoleLogs = (enabled) => logger.setPreserveConsoleLogs(enabled);
|
|
500
|
+
export const setQuietMode = (enabled) => logger.setQuietMode(enabled);
|
|
478
501
|
export const isVerboseMode = () => logger.isVerboseMode();
|
|
479
502
|
export const setDebugMode = (enabled) => logger.setDebugMode(enabled);
|
|
480
503
|
export const isDebugMode = () => logger.isDebugMode();
|
|
504
|
+
export const addDestination = (destination) => logger.addDestination(destination);
|
|
481
505
|
export const registerLogPane = (addLog) => logger.registerLogPane(addLog);
|
|
482
506
|
export const unregisterLogPane = (addLog) => logger.unregisterLogPane(addLog);
|
|
483
507
|
export const addTruncateTag = (tagName) => logger.addTruncateTag(tagName);
|
|
@@ -3,9 +3,26 @@ export async function waitForPageReadiness(page, options = {}) {
|
|
|
3
3
|
return;
|
|
4
4
|
const timeout = options.timeout ?? 6000;
|
|
5
5
|
await page.waitForLoadState?.('domcontentloaded', { timeout })?.catch(() => { });
|
|
6
|
-
await Promise.race([waitForNetworkIdle(page, timeout), waitForVisibleSpinnersHidden(page, options.spinnerSelectors || [], timeout), sleep(timeout)]).catch(() => { });
|
|
6
|
+
await Promise.race([waitForNetworkIdle(page, timeout), waitForDomQuiet(page, timeout), waitForVisibleSpinnersHidden(page, options.spinnerSelectors || [], timeout), sleep(timeout)]).catch(() => { });
|
|
7
7
|
await waitForPageBodyContent(page, timeout);
|
|
8
8
|
}
|
|
9
|
+
const DOM_QUIET_MS = 350;
|
|
10
|
+
function waitForDomQuiet(page, timeout) {
|
|
11
|
+
if (!page?.waitForFunction)
|
|
12
|
+
return new Promise(() => { });
|
|
13
|
+
return page
|
|
14
|
+
.waitForFunction((quiet) => {
|
|
15
|
+
const store = window;
|
|
16
|
+
if (!store.__explorbotDomQuiet) {
|
|
17
|
+
store.__explorbotDomQuiet = { last: Date.now() };
|
|
18
|
+
new MutationObserver(() => {
|
|
19
|
+
store.__explorbotDomQuiet.last = Date.now();
|
|
20
|
+
}).observe(document, { subtree: true, childList: true, attributes: true, characterData: true });
|
|
21
|
+
}
|
|
22
|
+
return Date.now() - store.__explorbotDomQuiet.last >= quiet;
|
|
23
|
+
}, DOM_QUIET_MS, { timeout, polling: 100 })
|
|
24
|
+
.catch(() => { });
|
|
25
|
+
}
|
|
9
26
|
function waitForNetworkIdle(page, timeout) {
|
|
10
27
|
if (!page?.waitForLoadState)
|
|
11
28
|
return Promise.resolve();
|
|
@@ -57,8 +57,7 @@ export async function dryRunTestFile(filePath) {
|
|
|
57
57
|
return;
|
|
58
58
|
}
|
|
59
59
|
const config = ConfigParser.getInstance().getConfig();
|
|
60
|
-
const
|
|
61
|
-
const projectRoot = configPath ? path.dirname(configPath) : process.cwd();
|
|
60
|
+
const projectRoot = ConfigParser.getInstance().getProjectRoot();
|
|
62
61
|
const codeceptConfig = {
|
|
63
62
|
helpers: {
|
|
64
63
|
Playwright: { browser: config.playwright.browser, url: config.playwright.url },
|
|
@@ -4,3 +4,4 @@ export declare function generalizeSegment(segment: string): string;
|
|
|
4
4
|
export declare function generalizeUrl(url: string): string;
|
|
5
5
|
export declare function matchesUrl(pattern: string, path: string): boolean;
|
|
6
6
|
export declare function extractStatePath(url: string): string;
|
|
7
|
+
export declare function matchesNavigationUrl(expected: string, current: string): boolean;
|
|
@@ -102,3 +102,15 @@ export function extractStatePath(url) {
|
|
|
102
102
|
return url;
|
|
103
103
|
}
|
|
104
104
|
}
|
|
105
|
+
export function matchesNavigationUrl(expected, current) {
|
|
106
|
+
const expectedPath = extractStatePath(expected);
|
|
107
|
+
let currentPath = extractStatePath(current);
|
|
108
|
+
if (!expectedPath.includes('#')) {
|
|
109
|
+
currentPath = currentPath.split('#')[0];
|
|
110
|
+
}
|
|
111
|
+
if (!expectedPath.includes('?')) {
|
|
112
|
+
currentPath = currentPath.split('?')[0];
|
|
113
|
+
}
|
|
114
|
+
const normalize = (value) => value.replace(/^\/+|\/+$/g, '').toLowerCase();
|
|
115
|
+
return normalize(expectedPath) === normalize(currentPath);
|
|
116
|
+
}
|
|
@@ -36,6 +36,8 @@ export declare class WebElement {
|
|
|
36
36
|
static fromXPathMatch(m: XPathMatch): WebElement;
|
|
37
37
|
static fromPlaywrightLocator(locator: any): Promise<WebElement | null>;
|
|
38
38
|
static fromEidx(page: any, eidx: string): Promise<WebElement | null>;
|
|
39
|
+
static isAriaRef(ref: string): boolean;
|
|
40
|
+
static fromAriaRef(page: any, ref: string): Promise<WebElement | null>;
|
|
39
41
|
static fromEidxList(page: any, eidxList: string[]): Promise<WebElement[]>;
|
|
40
42
|
static commonAncestor(page: any, eidxList: string[]): Promise<WebElement | null>;
|
|
41
43
|
static findByXPath(html: string, xpath: string): Promise<{
|
|
@@ -111,6 +111,14 @@ export class WebElement {
|
|
|
111
111
|
static async fromEidx(page, eidx) {
|
|
112
112
|
return WebElement.fromPlaywrightLocator(page.locator(`[${EXPLORBOT_ATTRS.eidx}="${eidx}"]`));
|
|
113
113
|
}
|
|
114
|
+
static isAriaRef(ref) {
|
|
115
|
+
return /^(f\d+)?e\d+$/i.test(ref);
|
|
116
|
+
}
|
|
117
|
+
static async fromAriaRef(page, ref) {
|
|
118
|
+
if (!WebElement.isAriaRef(ref))
|
|
119
|
+
return null;
|
|
120
|
+
return WebElement.fromPlaywrightLocator(page.locator(`aria-ref=${ref}`));
|
|
121
|
+
}
|
|
114
122
|
static async fromEidxList(page, eidxList) {
|
|
115
123
|
const validEidxList = eidxList.filter((eidx) => /^e\d+$/i.test(eidx));
|
|
116
124
|
if (validEidxList.length === 0)
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
export declare function sanitizeCodeBlock(code: string): string;
|
|
2
2
|
export declare function hasPlaywrightCommands(code: string): boolean;
|
|
3
3
|
export declare function playwrightSandbox(page: any, code: string): Promise<any>;
|
|
4
|
-
export declare function codeceptJSSandbox(actor: any, codeOrFn: string | ((...args: any[]) => void)):
|
|
4
|
+
export declare function codeceptJSSandbox(actor: any, codeOrFn: string | ((...args: any[]) => void)): any;
|
|
@@ -25,11 +25,10 @@ export function playwrightSandbox(page, code) {
|
|
|
25
25
|
}
|
|
26
26
|
export function codeceptJSSandbox(actor, codeOrFn) {
|
|
27
27
|
if (typeof codeOrFn === 'function') {
|
|
28
|
-
codeOrFn(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
29
|
-
return;
|
|
28
|
+
return codeOrFn(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
30
29
|
}
|
|
31
30
|
const run = createSandbox(CODECEPT_ARG_NAMES, codeOrFn);
|
|
32
|
-
run(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
31
|
+
return run(actor, tryTo, retryTo, within, hopeThat, step, faker);
|
|
33
32
|
}
|
|
34
33
|
function createSandbox(argNames, body) {
|
|
35
34
|
const fn = new Function(...argNames, ...SHADOWED_GLOBALS, `'use strict';\n${body}`);
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# API Testing Basics
|
|
2
|
+
|
|
3
|
+
Explorbot tests REST APIs the same way it tests web apps: it plans scenarios, runs them, and reports results — no test scripts. Two AI agents do the work.
|
|
4
|
+
|
|
5
|
+
**Chief** reads your endpoint, its OpenAPI spec, and any [knowledge](../workflow/knowledge.md) you've written, then plans test scenarios: what to send, and what a correct response looks like.
|
|
6
|
+
|
|
7
|
+
**Curler** takes each scenario and executes it as real HTTP requests, checking the responses with assertions.
|
|
8
|
+
|
|
9
|
+
The plans Chief writes are ordinary Explorbot [test plans](../workflow/test-plans.md) — plain markdown you can read, edit, and commit. The web and API sides share the same plan format and the same [reporting](../workflow/reporting.md).
|
|
10
|
+
|
|
11
|
+
## Configure
|
|
12
|
+
|
|
13
|
+
Point Explorbot at your API by adding an `api` key to your `explorbot.config.js`:
|
|
14
|
+
|
|
15
|
+
```javascript
|
|
16
|
+
export default {
|
|
17
|
+
ai: {
|
|
18
|
+
model: openrouter('openai/gpt-oss-20b:nitro'),
|
|
19
|
+
agenticModel: openrouter('minimax/minimax-m2.5:nitro'),
|
|
20
|
+
},
|
|
21
|
+
api: {
|
|
22
|
+
baseEndpoint: 'http://localhost:3000/api/v1',
|
|
23
|
+
spec: ['http://localhost:3000/api/openapi.json'],
|
|
24
|
+
headers: {
|
|
25
|
+
Authorization: 'Bearer <token>',
|
|
26
|
+
},
|
|
27
|
+
},
|
|
28
|
+
};
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
- **`baseEndpoint`** (required) — the base URL prepended to every request. Test steps use relative paths like `/users`; Curler adds the base for you.
|
|
32
|
+
- **`spec`** (required) — one or more OpenAPI specs, given as HTTP(S) URLs or local file paths, in YAML or JSON. Chief uses the spec to plan; Curler uses it to look up schemas. Both agents refuse to run without one.
|
|
33
|
+
- **`headers`** — sent with every request. This is where API keys and auth tokens go.
|
|
34
|
+
|
|
35
|
+
See the [full configuration reference](../reference/configuration.md) for every option and [providers](../basics/providers.md) for choosing an AI model.
|
|
36
|
+
|
|
37
|
+
### Authenticating
|
|
38
|
+
|
|
39
|
+
If a static token in `headers` is enough, you're done. If you need to log in and fetch a token first, use the `bootstrap` hook — it runs once before any tests, and whatever headers it returns merge into every later request:
|
|
40
|
+
|
|
41
|
+
```javascript
|
|
42
|
+
api: {
|
|
43
|
+
baseEndpoint: 'http://localhost:3000/api/v1',
|
|
44
|
+
spec: ['http://localhost:3000/api/openapi.json'],
|
|
45
|
+
bootstrap: async ({ baseEndpoint }) => {
|
|
46
|
+
const res = await fetch(`${baseEndpoint}/auth/login`, {
|
|
47
|
+
method: 'POST',
|
|
48
|
+
headers: { 'Content-Type': 'application/json' },
|
|
49
|
+
body: JSON.stringify({ email: 'admin@test.com', password: 'secret' }),
|
|
50
|
+
});
|
|
51
|
+
const { token } = await res.json();
|
|
52
|
+
return { Authorization: `Bearer ${token}` };
|
|
53
|
+
},
|
|
54
|
+
},
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
A matching `teardown` hook runs after all tests finish — use it to clean up data.
|
|
58
|
+
|
|
59
|
+
### A dedicated API project
|
|
60
|
+
|
|
61
|
+
If you don't have a web `explorbot.config.js`, run `npx explorbot api init`. It asks for your base endpoint, spec, and a one-line description of the API, then writes a standalone `apibot.config.ts` (with an `ai` and `api` section) plus `output/` and `knowledge/` directories. When both files exist, `apibot.config.*` takes precedence over `explorbot.config.*`.
|
|
62
|
+
|
|
63
|
+
## Your first run
|
|
64
|
+
|
|
65
|
+
The minimal loop is plan, then test. Point Chief at an endpoint:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
npx explorbot api plan /users
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
On startup Explorbot does a health check — a `GET /` against your base endpoint — so a bad URL or token fails immediately. Then Chief fetches sample data, reads the spec, and writes scenarios to `output/plans/users.md`. Hand that file to Curler:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
npx explorbot api test output/plans/users.md
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Curler runs the scenarios and prints how many passed and failed.
|
|
78
|
+
|
|
79
|
+
## Output files
|
|
80
|
+
|
|
81
|
+
| Output | Location | What it is |
|
|
82
|
+
|--------|----------|------------|
|
|
83
|
+
| Test plans | `output/plans/*.md` | Chief's scenarios — priorities, steps, expected outcomes |
|
|
84
|
+
| Request logs | `output/requests/*.request.yaml` | Every HTTP request and response, for debugging |
|
|
85
|
+
| Reports | via the shared [reporter](../workflow/reporting.md) | Pass/fail results, optionally sent to Testomat.io |
|
|
86
|
+
|
|
87
|
+
## Next steps
|
|
88
|
+
|
|
89
|
+
- [Planning API tests](./planning.md) — give Chief context and steer what it tests.
|
|
90
|
+
- [Running API tests](./running-tests.md) — execute plans, read request logs, and run the full autonomous cycle.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# Planning API Tests
|
|
2
|
+
|
|
3
|
+
Chief plans the scenarios; how good they are depends on the context you give it. Start by feeding it what it needs, then choose how aggressively it should probe.
|
|
4
|
+
|
|
5
|
+
## Give Chief context
|
|
6
|
+
|
|
7
|
+
**The API spec.** Chief reads the OpenAPI spec from your `spec` config to learn paths, methods, and request/response schemas. It also fetches live sample data from the endpoint — real IDs, enum values, field names — so the scenarios it writes reference data that actually exists instead of guessing. A good spec is the single biggest lever on plan quality.
|
|
8
|
+
|
|
9
|
+
**Endpoint knowledge.** The spec says what an endpoint accepts, not how your business rules work. Add that with `know`:
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
npx explorbot api know /users "CRUD for users. Admin role required for writes. IDs are UUIDs."
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
This writes a markdown file under `knowledge/` with `endpoint:` frontmatter:
|
|
16
|
+
|
|
17
|
+
```markdown
|
|
18
|
+
---
|
|
19
|
+
endpoint: "/users"
|
|
20
|
+
---
|
|
21
|
+
CRUD for users. Admin role required for writes. IDs are UUIDs.
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Chief loads knowledge matching the endpoint it's planning. Running `know` again on the same endpoint appends to the file. See [knowledge](../workflow/knowledge.md) for how matching and files work.
|
|
25
|
+
|
|
26
|
+
## Choose a planning style
|
|
27
|
+
|
|
28
|
+
A style tells Chief what kind of tests to prioritize. Four ship by default:
|
|
29
|
+
|
|
30
|
+
| Style | Focus |
|
|
31
|
+
|-------|-------|
|
|
32
|
+
| `normal` | Standard CRUD and happy paths — create, read, update, delete, verify status codes and schemas |
|
|
33
|
+
| `curious` | Maximum coverage — every field, every enum value, arrays, defaults |
|
|
34
|
+
| `psycho` | Malformed and extreme input — missing fields, injection payloads, wrong content types, boundary values |
|
|
35
|
+
| `hacker` | Security probing — infers hidden endpoints and undocumented fields from responses, then tries privilege escalation, IDOR, and auth-bypass against them |
|
|
36
|
+
|
|
37
|
+
Pick one with `--style`:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
npx explorbot api plan /users --style hacker
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Without `--style`, Chief uses `normal`. Styles are markdown files in `rules/chief/styles/`, so you can edit them or drop in your own. The cycling and customization mechanism is shared with web planning — see [planning styles](../workflow/planning-styles.md).
|
|
44
|
+
|
|
45
|
+
## Replan from scratch or add to a plan
|
|
46
|
+
|
|
47
|
+
By default a `plan` run generates a fresh set of scenarios. When a plan already exists in the same run, Chief compares against it and adds only new, non-duplicate scenarios — it won't re-propose behavior it already covered, even under a different style. Pass `--fresh` to discard the in-progress plan and start clean:
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
npx explorbot api plan /users --fresh
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
This is what [`explore`](./running-tests.md) uses to give each style its own clean plan.
|
|
54
|
+
|
|
55
|
+
## Where plans land
|
|
56
|
+
|
|
57
|
+
Chief saves each plan to `output/plans/`, named after the endpoint (for example `output/plans/users.md`). The file is a standard Explorbot [test plan](../workflow/test-plans.md): a suite of scenarios, each with steps, expected outcomes, and a priority. Read it, edit it, or commit it — then run it as described in [Running API tests](./running-tests.md).
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Running API Tests
|
|
2
|
+
|
|
3
|
+
Once Chief has written a plan, Curler runs it. Curler reads each scenario and drives it to a pass or fail by making real HTTP requests and asserting on the responses.
|
|
4
|
+
|
|
5
|
+
## Run a plan
|
|
6
|
+
|
|
7
|
+
Point Curler at a plan file:
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npx explorbot api test output/plans/users.md
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
With no index, Curler runs every pending test. Add an index to run a subset:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
npx explorbot api test output/plans/users.md 1 # the first test
|
|
17
|
+
npx explorbot api test output/plans/users.md 1-3 # tests 1 through 3
|
|
18
|
+
npx explorbot api test output/plans/users.md 1,3,5 # specific tests
|
|
19
|
+
npx explorbot api test output/plans/users.md * # all pending tests
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Curler prints a running log of requests and, at the end, how many tests passed and failed.
|
|
23
|
+
|
|
24
|
+
## What Curler can do
|
|
25
|
+
|
|
26
|
+
Curler works through AI tool calls. Its toolset:
|
|
27
|
+
|
|
28
|
+
- **`request`** — make an HTTP request (any method: GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS) with a body, headers, or query params. Returns status, timing, and a preview of the response; the full body is saved to disk. This is how Curler chains requests — it reads an ID from one response and feeds it into the next.
|
|
29
|
+
- **`verifyStructure`** — check the response shape against a Zod schema. On success it reports the actual structure, which Curler uses to write correct value assertions.
|
|
30
|
+
- **`verifyData`** — assert specific values with `expect()` (`toBe`, `toHaveProperty`, `toHaveLength`, and so on).
|
|
31
|
+
- **`schemaFor`** — search the OpenAPI spec for related endpoints when a test needs to set up prerequisite data or discover a dependency.
|
|
32
|
+
- **`record`** — note a finding or observation as the test runs.
|
|
33
|
+
- **`finish`** / **`stop`** — mark the test complete, or abandon it when the scenario is impossible.
|
|
34
|
+
|
|
35
|
+
Curler favors verification over trust: for writes, it follows up with a `GET` to confirm the data actually persisted rather than believing the write response alone.
|
|
36
|
+
|
|
37
|
+
## Debugging with request logs
|
|
38
|
+
|
|
39
|
+
Every request Curler makes is saved to `output/requests/` as a `.request.yaml` file — full URL, headers, body, status, and response. When a test fails and the log isn't enough, open these to see exactly what went over the wire, including a reproducible curl command.
|
|
40
|
+
|
|
41
|
+
## Explore an endpoint end to end
|
|
42
|
+
|
|
43
|
+
`explore` runs the whole cycle autonomously across every planning style:
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
npx explorbot api explore /users
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
For each style — `normal`, `curious`, `psycho`, `hacker` — it plans a fresh set of scenarios, runs them with Curler, and saves one plan file per style (`users_normal.md`, `users_hacker.md`, and so on). Scenarios are de-duplicated across styles, so the same test won't run twice. When every style is done, it prints the combined totals.
|
|
50
|
+
|
|
51
|
+
Use `explore` to hammer an endpoint from every angle in one command; use `plan` plus `test` when you want to review or edit scenarios before running them.
|
|
52
|
+
|
|
53
|
+
## Results and reporting
|
|
54
|
+
|
|
55
|
+
Pass/fail results flow through Explorbot's shared reporter — the same one the web side uses. See [reporting](../workflow/reporting.md) for local reports and sending runs to Testomat.io.
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|