explorbot 0.4.0 → 0.4.2

package/docs/workflow/knowledge.md

@@ -45,7 +45,48 @@ While exploring, use the `/learn` command.
 
 ### API Testing
 
-[API testing](../api-testing/basics.md) shares the same `knowledge/` directory. `npx explorbot api know <endpoint> "<description>"` adds endpoint-scoped notes, stored with an `endpoint:` frontmatter field instead of `url:`.
+[API testing](../api-testing/basics.md) shares the same `knowledge/` directory. `npx explorbot api know <endpoint> "<description>"` adds endpoint-scoped notes, stored with an `endpoint:` frontmatter field instead of `url:`. Chief reads them when planning an endpoint and Curler reads them when running its tests, so auth headers and payload rules reach both.
+
+## Per-Session Knowledge
+
+`--knowledge` passes facts to a single run. Nothing is written to `knowledge/`, so credentials and one-off test data stay out of the repository.
+
+```bash
+npx explorbot explore /pay --knowledge 'My credit card is 4111 1111 1111 1111'
+```
+
+Plain text applies to every page. Add frontmatter to scope it, with the same URL patterns knowledge files use:
+
+```bash
+npx explorbot explore / --knowledge '---
+url: /pay
+---
+Use the sandbox card 4111 1111 1111 1111 with any future expiry'
+```
+
+Repeat the flag for several facts:
+
+```bash
+npx explorbot explore / \
+  --knowledge '---
+url: /login
+---
+Log in as admin@example.com / secret123' \
+  --knowledge 'Dismiss the cookie banner before anything else'
+```
+
+Everything a knowledge file supports works here: `${env.VAR}` interpolation, and page automation fields such as `wait` and `waitForElement`.
+
+The flag works on every command of `explorbot`, `explorbot api`, `explorbot docs` and `prima`, and can go anywhere on the line. Scope API knowledge with `endpoint:` instead of `url:`:
+
+```bash
+npx explorbot api explore /orders --knowledge '---
+endpoint: /orders/*
+---
+Send X-Api-Key: ${env.API_KEY} on every request'
+```
+
+For runs driven from the environment — config-free, or on the global configuration — `EXPLORBOT_KNOWLEDGE` does the same job for the length of one run. See [Agentic usage](./agentic-usage.md).
 
 ## URL Patterns
 
@@ -81,10 +122,13 @@ Notes:
 
 | Field | Purpose |
 |-------|---------|
-| `url` | URL pattern to match (optional, defaults to `*`) |
+| `url` | Page URL pattern to match |
+| `endpoint` | API endpoint pattern to match |
 | `title` | Human-readable title (optional) |
 | Custom fields | Any additional metadata for agents |
 
+`url` scopes a file to browser pages and `endpoint` scopes it to API endpoints. A file with neither applies to both.
+
 ## Variables
 
 Knowledge files support variable interpolation with `${namespace.key}` syntax. Explorbot resolves variables when it loads the knowledge.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "explorbot",
-  "version": "0.4.0",
+  "version": "0.4.2",
   "description": "CLI app built with React Ink, CodeceptJS, and Playwright",
   "license": "Elastic-2.0",
   "type": "module",

package/rules/planner/styles/normal.md

@@ -1,7 +1,7 @@
 Study the page and figure out its business purpose. What is this page FOR? What would a user come here to do?
 
 Based on the page type, propose tests for COMPLETE user workflows:
-- If this is a data page (lists, tables): test CRUD operations end-to-end (create item → verify in list, edit item → verify changes saved, delete item → verify removed)
+- If this is a data page (lists, tables): test create, edit and delete as separate tests, each preparing the item it acts on
 - If the page has inputs to fill in: test the full commit flow, not just that the controls render
 - If this has filters and search: test filtering AND verify results change, not just "filter tab clicked"
 - If this has modals/dropdowns: test the ACTION inside them, not just opening/closing them

package/src/action-result.ts

@@ -8,6 +8,7 @@ import { type HtmlDiffPart, type HtmlDiffResult, htmlDiff, liveRegionMessages }
 import { extractHeadings, extractLinks, extractTargetedHtml, htmlCombinedSnapshot, htmlMinimalUISnapshot, htmlTextSnapshot, minifyHtml } from './utils/html.ts';
 import { createDebug } from './utils/logger.ts';
 import { Overlay } from './utils/overlay.ts';
+import { Region } from './utils/region.ts';
 import { slugify } from './utils/strings.ts';
 import { extractStatePath, matchesUrl } from './utils/url-matcher.ts';
 
@@ -88,7 +89,7 @@ export class ActionResult implements ActionResultData {
   notes: string[] = [];
   public links: Link[] = [];
   public verifications?: Record<string, boolean>;
-  public overlay: Overlay = new Overlay();
+  public overlay: Region = new Region();
   private _diffCache: { previousId: number | undefined; diff: Diff } | null = null;
 
   constructor(data: ActionResultData) {
@@ -264,7 +265,7 @@ export class ActionResult implements ActionResultData {
     if (!record.url || !this.url) return false;
     if (record.region && this.overlay.name !== record.region) return false;
     if (record.root) {
-      if (!this.overlay.present) return false;
+      if (!this.overlay.isOpen) return false;
       if (this.overlay.root && this.overlay.root !== record.root) return false;
     }
     if (this.isMatchedBy(record)) return true;
@@ -545,16 +546,13 @@ export class ActionResult implements ActionResultData {
       pageDiff.ariaChangeCount = diff.ariaChangeCount;
     }
 
-    if (this.overlay.present && (!previousState.overlay.present || previousState.overlay.name !== this.overlay.name)) {
+    if (this.overlay.isOpen && (!previousState.overlay.isOpen || previousState.overlay.name !== this.overlay.name)) {
       pageDiff.areaOfInterest = this.overlay.describe();
     }
 
     if (pageDiff.areaOfInterest && this.overlay.html && this.overlay.root) {
       const htmlConfig = ConfigParser.getInstance().getConfig().html;
-      let subtree = await minifyHtml(htmlCombinedSnapshot(this.overlay.html, htmlConfig?.combined));
-      if (subtree.length > HTML_PART_SUBTREE_BUDGET) {
-        subtree = `${subtree.slice(0, HTML_PART_SUBTREE_BUDGET)}...<!-- truncated -->`;
-      }
+      const subtree = trimSubtree(await minifyHtml(htmlCombinedSnapshot(this.overlay.html, htmlConfig?.combined)));
       pageDiff.htmlParts = [{ container: this.overlay.root, subtree, rawSize: subtree.length, added: [], removed: [] }];
     } else if (diff.isSameUrl() && diff.htmlParts.length > 0) {
       const collapsed = collapseHtmlParts(await diff.cleanedHtmlParts());
@@ -581,7 +579,7 @@ export class ActionResult implements ActionResultData {
 
     if (this.h1) parts.push(`h1_${this.h1}`);
     if (this.h2) parts.push(`h2_${this.h2}`);
-    if (includeRegion && this.overlay.present && this.overlay.name) parts.push(`region_${this.overlay.name}`);
+    if (includeRegion && this.overlay.isOpen && this.overlay.name) parts.push(`region_${this.overlay.name}`);
 
     let stateString = slugify(parts.map((part) => part.substring(0, 100)).join('_'));
 
@@ -617,6 +615,7 @@ const CONSOLE_ERROR_LIMIT = 3;
 const HTML_PARTS_TOTAL_BUDGET = 8000;
 const HTML_PARTS_COUNT_LIMIT = 8;
 const HTML_PART_SUBTREE_BUDGET = 2000;
+const SUBTREE_TAIL_BUDGET = 600;
 
 function collapseHtmlParts(parts: HtmlDiffPart[]): HtmlDiffPart[] {
   const total = parts.reduce((sum, p) => sum + p.subtree.length, 0);
@@ -631,14 +630,14 @@ function collapseHtmlParts(parts: HtmlDiffPart[]): HtmlDiffPart[] {
       }));
   }
 
-  return parts.map((part) => {
-    if (part.subtree.length <= HTML_PART_SUBTREE_BUDGET) return part;
-    const head = part.subtree.slice(0, HTML_PART_SUBTREE_BUDGET);
-    return {
-      ...part,
-      subtree: `${head}...<!-- truncated ${part.subtree.length - HTML_PART_SUBTREE_BUDGET} chars -->`,
-    };
-  });
+  return parts.map((part) => ({ ...part, subtree: trimSubtree(part.subtree) }));
+}
+
+function trimSubtree(subtree: string): string {
+  if (subtree.length <= HTML_PART_SUBTREE_BUDGET) return subtree;
+  const head = subtree.slice(0, HTML_PART_SUBTREE_BUDGET - SUBTREE_TAIL_BUDGET);
+  const tail = subtree.slice(-SUBTREE_TAIL_BUDGET);
+  return `${head}...<!-- truncated ${subtree.length - HTML_PART_SUBTREE_BUDGET} chars -->${tail}`;
 }
 
 export class Diff {

package/src/action.ts

@@ -14,6 +14,7 @@ import { browserErrorMessage, isFatalBrowserError, isNavigationTransitionError }
 import { captureHtmlForSnapshot, htmlCombinedSnapshot, minifyHtml } from './utils/html.js';
 import { createDebug, setStepSpanParent, tag } from './utils/logger.js';
 import { Overlay, OverlayPage } from './utils/overlay.js';
+import type { Region } from './utils/region.js';
 import { sleep, waitForPageReadiness } from './utils/page-readiness.ts';
 import { safeFilename } from './utils/strings.ts';
 import { codeceptJSSandbox, hasPlaywrightCommands, playwrightSandbox, sanitizeCodeBlock } from './utils/web-sandbox.ts';
@@ -181,7 +182,7 @@ class Action {
         focusedElement,
         iframeURL: frame ? frame.url?.() || 'iframe' : undefined,
       });
-      if (!frame) await this.detectRegionOfInterest(result).catch((err: Error) => debugLog('Region detection failed:', err.message));
+      if (!frame) await this.detectRegion(result).catch((err: Error) => debugLog('Region detection failed:', err.message));
       this.stateManager.updateState(result, codeBlock);
       return result;
     } catch (err) {
@@ -193,36 +194,36 @@ class Action {
     }
   }
 
-  private async detectRegionOfInterest(result: ActionResult): Promise<void> {
+  private async detectRegion(result: ActionResult): Promise<void> {
     const previousState = this.stateManager.getCurrentState();
     if (!previousState) return;
     const previous = ActionResult.fromState(previousState);
-    const previousOverlay = previous.overlay;
+    const previousRegion = previous.overlay;
     const sameUrl = !!previous.url && result.isSameUrl({ url: previous.url });
     const overlayPage = new OverlayPage(this.playwrightHelper.page);
 
-    if (result.overlay.detected && previousOverlay.detected && previousOverlay.root && previousOverlay.type === result.overlay.type && previousOverlay.name === result.overlay.name) {
-      result.overlay = previousOverlay;
+    if (result.overlay.isModal && previousRegion.isModal && previousRegion.root && previousRegion.type === result.overlay.type && previousRegion.name === result.overlay.name) {
+      result.overlay = previousRegion;
       return;
     }
 
     if (!previous.html) return;
 
     if (previous.html === result.html) {
-      if (sameUrl && previousOverlay.present && previousOverlay.xpath && !result.overlay.detected) result.overlay = previousOverlay;
+      if (sameUrl && previousRegion.isOpen && previousRegion.xpath && !result.overlay.isModal) result.overlay = previousRegion;
       return;
     }
 
-    let carried: Overlay | null = null;
-    if (sameUrl && previousOverlay.present && previousOverlay.xpath) {
-      if (await overlayPage.isStillOpen(previousOverlay)) {
-        carried = previousOverlay;
+    let carried: Region | null = null;
+    if (sameUrl && previousRegion.isOpen && previousRegion.xpath) {
+      if (await overlayPage.isStillOpen(previousRegion)) {
+        carried = previousRegion;
       } else {
-        debugLog(`Region closed: ${previousOverlay.name || previousOverlay.type}`);
-        if (!result.overlay.detected) {
-          const parent = previousOverlay.parent;
+        debugLog(`Region closed: ${previousRegion.name || previousRegion.type}`);
+        if (!result.overlay.isModal) {
+          const parent = previousRegion.parent;
           if (parent?.xpath) {
-            const restored = new Overlay(parent);
+            const restored = Overlay.resolve({ overlay: parent });
             if (await overlayPage.isStillOpen(restored)) result.overlay = restored;
           }
           return;
@@ -239,7 +240,7 @@ class Action {
       previousHtml: previous.html,
     });
 
-    if (result.overlay.detected) {
+    if (result.overlay instanceof Overlay) {
       if (detected) result.overlay = result.overlay.withGeometry(detected);
       return;
     }

package/src/ai/captain.ts

@@ -160,7 +160,7 @@ export class Captain extends CaptainBase implements Agent {
     const headingsBlock = headingLines.join('\n');
 
     let pageSummary = '';
-    const cachedResearch = Researcher.getCachedResearch(state);
+    const cachedResearch = Researcher.getCachedResearch(actionResult);
     if (cachedResearch) {
       pageSummary = `<page_summary>\n${this.explorBot.agentResearcher().extractBrief(cachedResearch)}\n</page_summary>`;
     }

package/src/ai/fisherman/request-haul.ts

@@ -0,0 +1,32 @@
+import type { RequestResult } from '../../api/request-result.ts';
+import { type RequestStore, isFailedRequest } from '../../api/request-store.ts';
+
+export class RequestHaul {
+  private start: number;
+
+  constructor(private store: RequestStore) {
+    this.start = store.getMadeRequests().length;
+  }
+
+  requests(): RequestResult[] {
+    return this.store.getMadeRequests().slice(this.start);
+  }
+
+  failed(): RequestResult[] {
+    return this.requests().filter(isFailedRequest);
+  }
+
+  successfulWrites(): RequestResult[] {
+    return this.requests().filter((r) => r.isWrite && !r.error && r.status >= 200 && r.status < 400);
+  }
+
+  byId(): Map<string, RequestResult> {
+    const map = new Map<string, RequestResult>();
+    for (const request of this.successfulWrites()) {
+      const { id } = request.extractIdAndTitle();
+      if (id === undefined) continue;
+      map.set(String(id), request);
+    }
+    return map;
+  }
+}

package/src/ai/fisherman-tools.ts

@@ -6,21 +6,18 @@ import type { RequestResult } from '../api/request-result.ts';
 import type { RequestStore } from '../api/request-store.ts';
 import { extractEndpointDefinition } from '../api/spec-reader.ts';
 import { tag } from '../utils/logger.ts';
-import { RequestMap } from '../utils/request-map.ts';
 import { isDynamicSegment } from '../utils/url-matcher.ts';
+import type { RequestHaul } from './fisherman/request-haul.ts';
 
-export function createFishermanTools(apiClient: ApiClient, requestStore: RequestStore, opts: { spec?: any; baseEndpoint?: string }) {
+export function createFishermanTools(apiClient: ApiClient, requestStore: RequestStore, haul: RequestHaul, opts: { spec?: any; baseEndpoint?: string }) {
   let finished = false;
   let result: FishermanResult | null = null;
-  const ledgerStart = requestStore.getMadeRequests().length;
 
-  const runRequests = () => requestStore.getMadeRequests().slice(ledgerStart);
-  const successfulWrites = () => runRequests().filter((r) => r.isWrite && !r.error && r.status >= 200 && r.status < 400);
-  const getResult = () => result ?? synthesizeResult(runRequests(), successfulWrites(), false);
+  const getResult = () => result ?? synthesizeResult(haul, false);
   const isFinished = () => finished;
   const finishFromText = (text?: string) => {
     finished = true;
-    const synthesized = synthesizeResult(runRequests(), successfulWrites(), true);
+    const synthesized = synthesizeResult(haul, true);
     if (text && synthesized.success) synthesized.summary = text;
     result = synthesized;
   };
@@ -160,32 +157,12 @@ export function createFishermanTools(apiClient: ApiClient, requestStore: Request
           .describe('List of items that could not be created'),
       }),
       execute: async ({ summary, created, failed }) => {
-        const writes = successfulWrites();
-        if (writes.length === 0) {
-          tag('warning').log('Fisherman: finish rejected — no successful write request in this run');
-          return { finished: false, error: 'No successful write request was made in this run, so nothing was created. Keep working, or call stop if the data cannot be prepared.' };
-        }
-
-        const createdRequests = new RequestMap(writes);
-
-        const verified: FishermanResult['created'] = [];
-        for (const item of created) {
-          if (item.id === undefined) {
-            verified.push(item);
-            continue;
-          }
-          const request = createdRequests.get(item.id);
-          if (!request) {
-            tag('warning').log(`Fisherman: dropped unverified created item ${item.type} (id: ${item.id})`);
-            continue;
-          }
-          verified.push({ ...item, request: request.toEndpoint() });
-        }
-        if (verified.length === 0) verified.push(...writes.map(toCreatedItem));
+        const { result: verified, error } = verifyFinish(haul, { summary, created, failed });
+        if (!verified) return { finished: false, error };
 
         tag('success').log(`Fisherman done: ${summary}`);
         finished = true;
-        result = { success: true, summary, created: verified, failed: failed || [] };
+        result = verified;
         return { finished: true };
       },
     }),
@@ -207,8 +184,37 @@ export function createFishermanTools(apiClient: ApiClient, requestStore: Request
   return { tools, getResult, isFinished, finishFromText };
 }
 
-function synthesizeResult(made: RequestResult[], writes: RequestResult[], declaredDone: boolean): FishermanResult {
-  const failures = made.filter((r) => r.status >= 400 || r.error);
+export function verifyFinish(haul: RequestHaul, input: { summary: string; created: FishermanResult['created']; failed?: FishermanResult['failed'] }): { result: FishermanResult | null; error?: string } {
+  const writes = haul.successfulWrites();
+  if (writes.length === 0) {
+    tag('warning').log('Fisherman: finish rejected — no successful write request in this run');
+    return { result: null, error: 'No successful write request was made in this run, so nothing was created. Keep working, or call stop if the data cannot be prepared.' };
+  }
+
+  const createdRequests = haul.byId();
+
+  const verified: FishermanResult['created'] = [];
+  for (const item of input.created) {
+    if (item.id === undefined) {
+      verified.push(item);
+      continue;
+    }
+    const request = createdRequests.get(String(item.id));
+    if (!request) {
+      tag('warning').log(`Fisherman: dropped unverified created item ${item.type} (id: ${item.id})`);
+      continue;
+    }
+    verified.push({ ...item, request: request.toEndpoint() });
+  }
+  if (verified.length === 0) verified.push(...writes.map(toCreatedItem));
+
+  return { result: { success: true, summary: input.summary, created: verified, failed: input.failed || [] } };
+}
+
+function synthesizeResult(haul: RequestHaul, declaredDone: boolean): FishermanResult {
+  const made = haul.requests();
+  const writes = haul.successfulWrites();
+  const failures = haul.failed();
   let summary = `Stopped before finishing: ${made.length} requests, ${writes.length} successful writes, ${failures.length} failed`;
   const lastFailure = failures[failures.length - 1];
   if (lastFailure) summary += `; last failure: ${lastFailure.toSummary()}`;

package/src/ai/fisherman.ts

@@ -1,6 +1,6 @@
 import dedent from 'dedent';
 import type { ApiClient } from '../api/api-client.ts';
-import type { RequestStore } from '../api/request-store.ts';
+import { type RequestStore, isFailedRequest } from '../api/request-store.ts';
 import { listAllEndpoints } from '../api/spec-reader.ts';
 import { createDebug, tag } from '../utils/logger.ts';
 
@@ -8,6 +8,7 @@ const debugLog = createDebug('explorbot:fisherman');
 import { loop } from '../utils/loop.ts';
 import type { Agent } from './agent.ts';
 import { type FishermanResult, createFishermanTools } from './fisherman-tools.ts';
+import { RequestHaul } from './fisherman/request-haul.ts';
 import type { Provider } from './provider.ts';
 import { dataProtectionRules } from './rules.ts';
 
@@ -79,11 +80,11 @@ export class Fisherman implements Agent {
     await this.refreshAuth();
     debugLog(`auth headers: ${Object.keys(this.apiClient.getHeaders()).join(', ')}`);
 
-    const { tools, getResult, isFinished, finishFromText } = createFishermanTools(this.apiClient, this.requestStore, {
+    const haul = new RequestHaul(this.requestStore);
+    const { tools, getResult, isFinished, finishFromText } = createFishermanTools(this.apiClient, this.requestStore, haul, {
       spec: this.spec,
       baseEndpoint: this.baseEndpoint,
     });
-    const ledgerStart = this.requestStore.getMadeRequests().length;
 
     const conversation = this.provider.startConversation(this.buildSystemPrompt(endpointList, Object.keys(tools), scopeUrl), 'fisherman');
     conversation.addUserText(this.buildTaskPrompt(instructions));
@@ -109,7 +110,7 @@ export class Fisherman implements Agent {
           return;
         }
 
-        if (this.isStuckOnEndpoint(ledgerStart)) {
+        if (this.isStuckOnEndpoint(haul)) {
           tag('warning').log('Fisherman: repeated failures on the same endpoint — stopping');
           stop();
           return;
@@ -228,12 +229,12 @@ export class Fisherman implements Agent {
     `;
   }
 
-  private isStuckOnEndpoint(ledgerStart: number): boolean {
-    const made = this.requestStore.getMadeRequests().slice(ledgerStart);
+  private isStuckOnEndpoint(haul: RequestHaul): boolean {
+    const made = haul.requests();
     if (made.length < REPEATED_FAILURE_LIMIT) return false;
     const recent = made.slice(-REPEATED_FAILURE_LIMIT);
     const first = recent[0];
-    return recent.every((r) => (r.status >= 400 || r.error) && r.method === first.method && r.path === first.path);
+    return recent.every((r) => isFailedRequest(r) && r.method === first.method && r.path === first.path);
   }
 
   private buildTaskPrompt(instructions: string): string {

package/src/ai/navigator.ts

@@ -575,7 +575,7 @@ class Navigator implements Agent {
     }
 
     const currentActionResult = actionResult || ActionResult.fromState(state);
-    const research = Researcher.getCachedResearch(state) || '';
+    const research = Researcher.getCachedResearch(currentActionResult) || '';
     const combinedHtml = await currentActionResult.combinedHtml();
 
     const history = stateManager.getStateHistory();

package/src/ai/pilot.ts

@@ -31,6 +31,7 @@ const META_TOOLS = ['record', 'reset', 'stop', 'finish'];
 const PILOT_REASONING_LIMIT = 500;
 const PILOT_MESSAGE_LIMIT = 2;
 const PILOT_MESSAGE_MAX_LENGTH = 160;
+const PILOT_REQUEST_LIMIT = 5;
 
 export class Pilot implements Agent {
   emoji = '🧭';
@@ -826,17 +827,17 @@ export class Pilot implements Agent {
     lines.push(`h3: ${state.h3 || ''}`);
     lines.push(`h4: ${state.h4 || ''}`);
 
-    const focusArea = state.overlay;
-    if (focusArea.detected) {
-      let line = `modal: ${focusArea.name || focusArea.type}`;
-      if (focusArea.root) line += ` (root: ${focusArea.root})`;
+    const region = state.overlay;
+    if (region.isModal) {
+      let line = `overlay: ${region.name || region.type}`;
+      if (region.root) line += ` (root: ${region.root})`;
       lines.push(line);
-    } else if (focusArea.present) {
-      let line = `region: ${focusArea.name || 'unnamed'} (inline`;
-      if (focusArea.root) line += `, root: ${focusArea.root}`;
+    } else if (region.isOpen) {
+      let line = `region: ${region.name || 'unnamed'} (inline`;
+      if (region.root) line += `, root: ${region.root}`;
       lines.push(`${line})`);
     } else {
-      lines.push('modal: none');
+      lines.push('overlay: none');
     }
 
     const tabs = this.stateManager.otherTabs;
@@ -1071,9 +1072,13 @@ export class Pilot implements Agent {
 
         if (t.output?.pageDiff?.urlChanged) line += `\n   moved: ${t.output.pageDiff.previousUrl} → ${t.output.pageDiff.currentUrl}`;
 
-        const failedRequests = (t.output?.pageDiff?.requests ?? []).filter((r: any) => r.status >= 400);
-        if (failedRequests.length > 0) {
-          line += `\n   requests: ${failedRequests.map((r: any) => `${r.method} ${r.path} → ${r.status}`).join(', ')}`;
+        const pageRequests = t.output?.pageDiff?.requests ?? [];
+        const requests = pageRequests
+          .filter((r: any) => r.status >= 400)
+          .concat(pageRequests.filter((r: any) => r.status < 400))
+          .slice(0, PILOT_REQUEST_LIMIT);
+        if (requests.length > 0) {
+          line += `\n   requests: ${requests.map((r: any) => `${r.method} ${r.path} → ${r.status}`).join(', ')}`;
         }
 
         const messages = (t.output?.pageDiff?.messages ?? []).slice(0, PILOT_MESSAGE_LIMIT);
@@ -1139,15 +1144,15 @@ export class Pilot implements Agent {
 
       Diagnostic patterns (use <state>, executed/element/skipped fields, ariaDiff):
       - Click failed + button in "disabled buttons" → required field missing. Instruct fill first.
-      - "modal: none" but Tester targets a modal → modal closed; re-trigger.
+      - "overlay: none" but Tester targets an overlay → overlay closed; re-trigger.
       - "region:" in <state> → a large area appeared in place without navigation (subview, wizard step, panel). Direct Tester to act inside it; the rest of the page is still usable.
       - Action SUCCESS but ariaDiff empty → may have worked without visible DOM change; check result message.
-      - MultipleElementsFound → xpathCheck() to identify the right one, then precise locator or visualClick().
+      - MultipleElementsFound → nothing was clicked. Tell Tester to reuse the same locator with step.opts({ elementIndex: N }) from the numbered elements list.
       - Wrong page (settings vs feature) → getVisitedStates() then back() or reset(). Don't try breadcrumbs (SPA back-nav is unreliable).
       - Click SUCCESS but executed locator ≠ explanation intent, or "skipped" attempts present → wrong element clicked.
       - form(I.type()) SUCCESS but "element" shows a button/link → keys went to wrong element; click the input first.
       - ariaDiff shows 5+ added/removed → page entered new mode (editor/modal); call context() before guessing selectors.
-      - Empty dropdown/list when items expected → missing data; call precondition() to create it.
+      - Empty dropdown/list when items expected → wait explicitly, then check the state changed: ariaDiff and any GET that loaded data. If still nothing loaded, confirm the empty state with verify().
       - Search-and-select needs SEQUENCE: focus trigger → type to filter → click option. Tell Tester to split into separate tool calls.
       - Multi-action explanation in one tool call → instruct Tester to split.
 

package/src/ai/planner.ts

@@ -114,11 +114,15 @@ export class Planner extends PlannerBase implements Agent {
       Tests must be relevant to the page
       Tests must be achievable from UI
       Tests must be verifiable from UI
-      NEVER split one workflow into multiple tests. Each test must be a complete end-to-end flow.
+      One test verifies ONE business operation: the steps that reach it, the action itself, and its verification.
+      Steps that only reach the action — opening a form, expanding a panel, creating or locating the item to act on — belong to that test. A second operation with its own verification does not.
       Bad: "Open delete dropdown" + "Confirm deletion" — these are ONE test, not two.
       Bad: "Search for X" + "Verify search results" — searching and verifying is ONE test.
       Bad: "Leave field empty" + "Click submit" — that's one negative test, not two.
-      If two scenarios cannot run independently (one requires the other to run first), merge them into one.${featureDirective}${focusExistingDataDirective}
+      Bad: "Create a record, rename it, delete it" — three verified operations, so THREE tests, not one.
+      Good: "Rename existing record and verify the new title" — ONE test; creating is skipped, we assume record already exists, only the rename is verified.
+      You may rely on another test having run first in case we deal with empty state and no relevant data was created yet and we expect another our test creates it
+      When the page reports a record is missing or unavailable, it is not a testable surface — plan list-level or recovery behavior instead of operations on that record.${featureDirective}${focusExistingDataDirective}
     </task>
 
     ${customPrompt || ''}
@@ -195,10 +199,6 @@ export class Planner extends PlannerBase implements Agent {
         throw new Error('No tasks were created successfully');
       }
 
-      if (aiResult.object.scenarios.length === 0 && !this.currentPlan) {
-        throw new Error('No tasks were created successfully');
-      }
-
       const defaultStartUrl = this.getDefaultStartUrl(state);
       const fromPlanning = aiResult.object.scenarios.map((s: any) => new Test(s.scenario, s.priority, s.expectedOutcomes, s.startUrl || defaultStartUrl, s.steps || []));
 
@@ -335,6 +335,7 @@ export class Planner extends PlannerBase implements Agent {
       <task>
       Based on the page research, create ${this.MIN_TASKS}-${this.MAX_TASKS} exploratory testing scenarios.
       For each scenario provide specific steps and expected outcomes.
+      Exception: if the page reports the requested resource is missing, shows a failure state, or holds no content and no controls, return an empty scenarios list. Never invent tests for a page with nothing to exercise.
       </task>
 
       <rules>
@@ -348,13 +349,13 @@ export class Planner extends PlannerBase implements Agent {
       Focus on error or success messages as outcome.
       Focus on URL page change or data persistency after page reload.
       If there are subpages (pages with same URL path) plan testing of those subpages as well
-      If you plan to test CRUD operations, plan them in correct order: create, read, update.
+      Plan CRUD operations in order: create, read, update, delete.
       Do not invent specific route names, success messages, validation texts, badge counts, or welcome messages unless they are visible in research, visited pages, or prior observed flows.
       When validation placement or wording was not observed, require feedback associated with the invalid input without inventing a specific location or message.
       If exact wording is unknown, describe the expected result generically, for example "an authentication error is shown" or "the user stays on the login page" instead of guessing the literal text.
       If exact redirect destination is unknown, describe the destination by visible page identity, for example "the dashboard page opens" or "the current workspace home page opens" instead of inventing a URL slug.
       Only propose scenarios whose prerequisites are evident from page research, visited pages, or API data preparation context.
-      If a scenario needs existing records, recipients, results, notifications, or other target data, propose it only when that data is visible or API preconditions can create it.
+      If a scenario needs existing records, recipients, results, notifications, or other target data, propose it only when that data is visible, API preconditions can create it, or the scenario itself creates the record as its setup.
       If the page appears read-only, degraded, demo-limited, maintenance-like, or lacks write controls, prefer read-only scenarios such as opening panels, inspecting visible lists, filtering, searching, or verifying current state.
       Do not assume hidden data exists just because a control is present.
       For scenarios that act on existing items or search/filter by existing values, use only item names or values visible in research, visited pages, or prior observed flows.