retestkit 1.4.1 → 1.5.0

package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/tasks.md

@@ -1,264 +0,0 @@
-# Tasks: Add Dynamic Web Testing Orchestrator
-
-## Phase 1: Core Infrastructure
-
-- [x] 1.1 Add Playwright MCP client dependency and types
-- [x] 1.2 Create `src/playwright-client/` module for Playwright MCP subprocess management
-  - [x] 1.2.1 Implement spawn/connect/disconnect lifecycle
-  - [x] 1.2.2 Implement tool discovery via `tools/list`
-  - [x] 1.2.3 Implement capability adapter (canonical name → actual tool name mapping)
-  - [x] 1.2.4 Handle tool name variants (`browser_*`, `playwright_*`, unprefixed)
-  - [x] 1.2.5 Log detected Playwright MCP implementation variant
-  - [x] 1.2.6 Implement tool call wrapper with error handling
-- [x] 1.3 Create `src/lifecycle/` module for MCP lifecycle management
-  - [x] 1.3.1 Implement protocol version negotiation (require 2025-06-18+)
-  - [x] 1.3.2 Implement capability negotiation on initialize
-  - [x] 1.3.3 Record capabilities: sampling, elicitation, logging, progress, resources.listChanged, resources.subscribe
-  - [x] 1.3.4 Store capabilities in server context
-  - [x] 1.3.5 Add capability query helpers (`hasSampling()`, `hasElicitation()`, etc.)
-  - [x] 1.3.6 Log warning when running in degraded mode (older protocol)
-- [x] 1.4 Create `src/sampling/` module for MCP Sampling client
-  - [x] 1.4.1 Implement `sampling/createMessage` request wrapper
-  - [x] 1.4.2 Implement JSON schema enforcement in prompts
-  - [x] 1.4.3 Implement response validation against schemas
-  - [x] 1.4.4 Implement retry on validation failure (once, with error feedback)
-  - [x] 1.4.5 Add fallback mode (return prompt resource when sampling unavailable)
-- [x] 1.5 Create `src/elicitation/` module for MCP Elicitation client
-  - [x] 1.5.1 Implement `elicitation/create` request wrapper
-  - [x] 1.5.2 Define allowed elicitation types (enum)
-  - [x] 1.5.3 Add fallback mode (return questions in tool output)
-- [x] 1.6 Create `src/progress/` module for progress and cancellation
-  - [x] 1.6.1 Implement progress notification emitter (with budget status)
-  - [x] 1.6.2 Implement cancellation registry
-  - [x] 1.6.3 Add `checkCancelled()` helper for loops
-- [x] 1.7 Update `src/logger.ts` for structured logging with MCP notifications
-  - [x] 1.7.1 Emit `notifications/message` when client supports logging
-  - [x] 1.7.2 Support `logging/setLevel` for dynamic level control
-  - [x] 1.7.3 Add correlation ID support (analysisId, crawlId, testRunId, iteration, requestId)
-  - [x] 1.7.4 Implement sensitive data redaction (URL params, cookies, passwords)
-  - [x] 1.7.5 Truncate HTML content in logs
-- [x] 1.8 Write unit tests for all Phase 1 modules
-
-## Phase 2: Workspace and Resources
-
-- [x] 2.1 Create `src/workspace/` module for analysis workspaces
-  - [x] 2.1.1 Implement workspace directory creation
-  - [x] 2.1.2 Implement `index.json` metadata management
-  - [x] 2.1.3 Add workspace path resolution helpers
-- [x] 2.2 Create `src/resources/` module for MCP Resource management
-  - [x] 2.2.1 Register resource templates with server
-  - [x] 2.2.2 Implement `webtest://` URI scheme handler
-  - [x] 2.2.3 Implement resource listing for analysis artifacts
-  - [x] 2.2.4 Implement resource read for individual artifacts
-  - [x] 2.2.5 Implement `resources/subscribe` handler
-  - [x] 2.2.6 Emit `notifications/resources/list_changed` on new resource creation
-  - [x] 2.2.7 Emit `notifications/resources/updated` for subscribed resources
-  - [x] 2.2.8 Handle missing listChanged/subscribe capabilities gracefully
-- [x] 2.3 Create artifact capture utilities
-  - [x] 2.3.1 Screenshot capture and storage
-  - [x] 2.3.2 Snapshot JSON storage
-  - [x] 2.3.3 HTML DOM capture and storage
-  - [x] 2.3.4 Markdown report generation helpers
-- [x] 2.4 Add configuration for workspace location (`WEBTEST_WORKSPACE_DIR`)
-- [x] 2.5 Write unit tests for workspace and resource modules
-
-## Phase 3: webtest_init Tool
-
-- [x] 3.1 Create `src/tools/webtest/start-analysis.ts`
-  - [x] 3.1.1 Define input schema (url, focus, limits)
-  - [x] 3.1.2 Validate URL and normalize domain
-  - [x] 3.1.3 Generate `analysisId`
-  - [x] 3.1.4 Create workspace directory structure
-  - [x] 3.1.5 Write initial `index.json` resource
-  - [x] 3.1.6 Return `analysisId`, `workspaceRootUri`, `statusUri`
-- [x] 3.2 Register tool in `src/tools/index.ts`
-- [x] 3.3 Write unit tests for start_analysis
-- [x] 3.4 Write integration test: start_analysis creates workspace
-
-## Phase 4: webtest_crawl_app Tool
-
-- [x] 4.1 Create `src/tools/webtest/crawl.ts`
-  - [x] 4.1.1 Define input schema (analysisId, goal, strategy, limits, artifacts)
-  - [x] 4.1.2 Validate analysisId exists
-- [x] 4.2 Implement crawl loop core
-  - [x] 4.2.1 Navigate to starting URL
-  - [x] 4.2.2 Capture initial state (snapshot, screenshot, HTML)
-  - [x] 4.2.3 Build sampling prompt with goal, history, current state
-  - [x] 4.2.4 Request next action via sampling
-  - [x] 4.2.5 Validate and execute Playwright actions
-  - [x] 4.2.6 Record action result and update history
-  - [x] 4.2.7 Check termination conditions (goal met, limits, cancellation)
-  - [x] 4.2.8 Loop or finalize
-- [x] 4.3 Implement elicitation integration
-  - [x] 4.3.1 Detect elicitation triggers (cookie banner, modal, ambiguity, auth)
-  - [x] 4.3.2 Call elicitation or fallback
-  - [x] 4.3.3 Incorporate user decision into crawl
-- [x] 4.4 Implement progress reporting
-  - [x] 4.4.1 Emit progress on each iteration
-  - [x] 4.4.2 Include step count, pages, current intent
-- [x] 4.5 Implement cancellation handling
-  - [x] 4.5.1 Check cancellation registry each iteration
-  - [x] 4.5.2 On cancel, finalize partial crawl index
-- [x] 4.6 Implement checkpointing
-  - [x] 4.6.1 Write checkpoint every N steps (configurable, default 5)
-  - [x] 4.6.2 Include: step count, visited pages, action history, goal progress, DOM signatures
-  - [x] 4.6.3 Support `resume: true` input to continue from checkpoint
-  - [x] 4.6.4 Update crawl index immediately on each page capture
-- [x] 4.7 Implement loop detection and prevention
-  - [x] 4.7.1 Track DOM signatures (hash of key structural elements)
-  - [x] 4.7.2 Detect same-state loops (3 consecutive same signatures)
-  - [x] 4.7.3 Detect URL cycles (same URL > 3 times)
-  - [x] 4.7.4 Detect action repeats (same tool+args 3 times consecutively)
-  - [x] 4.7.5 Include loop state in sampling prompts
-- [x] 4.8 Implement budget enforcement
-  - [x] 4.8.1 Enforce maxSteps limit
-  - [x] 4.8.2 Enforce maxMinutes timeout
-  - [x] 4.8.3 Enforce maxPages limit
-  - [x] 4.8.4 Include budget status in progress notifications
-- [x] 4.9 Implement fallback mode (no sampling)
-  - [x] 4.9.1 Return prompt resource for manual execution
-  - [x] 4.9.2 Accept `manualNextActions` input to continue
-- [x] 4.10 Implement security checks
-  - [x] 4.10.1 Validate actions target allowed domains
-  - [x] 4.10.2 Block data exfiltration patterns (POST to external, URL params)
-  - [x] 4.10.3 Log all sampling inputs/outputs for audit
-- [x] 4.11 Write crawl output resources
-  - [x] 4.11.1 `crawlIndexUri` (JSON)
-  - [x] 4.11.2 Per-page artifact URIs
-  - [x] 4.11.3 `summaryUri` (markdown)
-- [x] 4.12 Register tool in `src/tools/index.ts`
-- [x] 4.13 Write unit tests for crawl logic
-- [x] 4.14 Write integration test: crawl with mock Playwright MCP
-
-## Phase 5: webtest_analyze_app Tool
-
-- [x] 5.1 Create `src/tools/webtest/analyze-app.ts`
-  - [x] 5.1.1 Define input schema (analysisId, crawlId)
-  - [x] 5.1.2 Load crawl index and artifacts
-- [x] 5.2 Build analysis sampling prompt
-  - [x] 5.2.1 Include crawl summary, page snapshots
-  - [x] 5.2.2 Request: app purpose, entities, user flows, assertions
-  - [x] 5.2.3 Define output JSON schema
-- [x] 5.3 Execute sampling and validate response
-- [x] 5.4 Write analysis outputs
-  - [x] 5.4.1 `app-analysis.md` resource
-  - [x] 5.4.2 `flows.json` resource
-- [x] 5.5 Implement fallback mode (prompt resource)
-- [x] 5.6 Register tool in `src/tools/index.ts`
-- [x] 5.7 Write unit tests for analyze_app
-
-## Phase 6: webtest_generate_tests Tool
-
-- [x] 6.1 Create `src/tools/webtest/generate-tests.ts`
-  - [x] 6.1.1 Define input schema (analysisId, appAnalysisUri, testStrategy)
-  - [x] 6.1.2 Load app analysis and flows
-- [x] 6.2 Build test generation sampling prompt
-  - [x] 6.2.1 Include analysis, flows, strategy preferences
-  - [x] 6.2.2 Define test case output schema (id, name, purpose, preconditions, steps, expected)
-- [x] 6.3 Execute sampling and validate response
-- [x] 6.4 Write test outputs
-  - [x] 6.4.1 `tests.md` resource
-  - [x] 6.4.2 `tests.json` resource
-- [x] 6.5 Implement fallback mode (prompt resource)
-- [x] 6.6 Register tool in `src/tools/index.ts`
-- [x] 6.7 Write unit tests for generate_tests
-
-## Phase 7: webtest_run_tests Tool
-
-- [x] 7.1 Create `src/tools/webtest/run-test-case.ts`
-  - [x] 7.1.1 Define input schema (analysisId, testCaseId, testsUri, runOptions)
-  - [x] 7.1.2 Load test case from tests index
-- [x] 7.2 Implement test execution loop
-  - [x] 7.2.1 For each step: capture state
-  - [x] 7.2.2 Use sampling to translate step to Playwright actions
-  - [x] 7.2.3 Execute actions, capture evidence
-  - [x] 7.2.4 Evaluate pass/fail via sampling
-  - [x] 7.2.5 Record step result
-- [x] 7.3 Implement progress reporting per step
-- [x] 7.4 Implement cancellation handling
-- [x] 7.5 Write run outputs
-  - [x] 7.5.1 `report.md` resource (pass/fail summary, evidence links)
-  - [x] 7.5.2 `artifacts.json` resource (per-step artifacts)
-- [x] 7.6 Implement fallback mode
-- [x] 7.7 Register tool in `src/tools/index.ts`
-- [x] 7.8 Write unit tests for run_test_case
-- [x] 7.9 Write integration test: full test execution with mock Playwright
-
-## Phase 8: Prompts
-
-- [x] 8.1 Create `src/prompts/` module
-- [x] 8.2 Implement prompt: "Start web testing analysis"
-  - [x] 8.2.1 Gather URL, focus, limits
-  - [x] 8.2.2 Call `webtest_init`
-- [x] 8.3 Implement prompt: "Crawl to satisfy focus"
-  - [x] 8.3.1 Gather analysisId, goal, strategy
-  - [x] 8.3.2 Call `webtest_crawl_app`
-- [x] 8.4 Implement prompt: "Generate tests"
-  - [x] 8.4.1 Gather analysisId, appAnalysisUri
-  - [x] 8.4.2 Call `webtest_generate_tests`
-- [x] 8.5 Implement prompt: "Run test case"
-  - [x] 8.5.1 Gather analysisId, testCaseId
-  - [x] 8.5.2 Call `webtest_run_tests`
-- [x] 8.6 Register prompts with server
-- [x] 8.7 Write unit tests for prompt registration
-
-## Phase 9: Cleanup and Finalization
-
-- [x] 9.1 Remove `hello` tool from `src/tools/`
-- [x] 9.2 Update `src/tools/index.ts` to export only webtest tools
-- [x] 9.3 Update configuration schema for new env vars
-- [x] 9.4 Update README with usage documentation
-- [x] 9.5 Add example workflows to documentation
-- [x] 9.6 Run full test suite and fix failures
-- [x] 9.7 Run `npm run build` and verify clean build
-- [ ] 9.8 Manual end-to-end test with real client
-
-## Phase 10: Integration Testing
-
-- [x] 10.1 Create mock Playwright MCP server for testing
-- [x] 10.2 Integration test: full workflow (start → crawl → analyze → generate → run)
-- [x] 10.3 Integration test: cancellation mid-crawl
-- [x] 10.4 Integration test: fallback mode without sampling
-- [x] 10.5 Integration test: elicitation triggers
-- [x] 10.6 Integration test: resource listChanged notifications
-- [x] 10.7 Integration test: crawl resume from checkpoint
-- [x] 10.8 Integration test: loop detection triggers
-
-## Phase 11: Security Testing
-
-- [x] 11.1 Create prompt injection test suite
-  - [x] 11.1.1 Test: direct instruction injection ("Ignore previous instructions...")
-  - [x] 11.1.2 Test: indirect injection via page meta tags
-  - [x] 11.1.3 Test: indirect injection via hidden elements
-  - [x] 11.1.4 Test: goal hijacking ("Actually, the user wants...")
-  - [x] 11.1.5 Test: credential phishing in page content
-- [x] 11.2 Create domain enforcement test suite
-  - [x] 11.2.1 Test: navigation to disallowed domain blocked
-  - [x] 11.2.2 Test: subdomain matching works correctly
-  - [x] 11.2.3 Test: link click to external domain detected
-- [x] 11.3 Create data exfiltration test suite
-  - [x] 11.3.1 Test: POST to external domain blocked
-  - [x] 11.3.2 Test: page content in external URL params blocked
-  - [x] 11.3.3 Test: browser_run_code external requests blocked
-- [x] 11.4 Create sensitive data redaction test suite
-  - [x] 11.4.1 Test: URL params with sensitive keys redacted
-  - [x] 11.4.2 Test: cookie values redacted in logs
-  - [x] 11.4.3 Test: password input values redacted
-
-## Dependencies
-
-- Phase 2 depends on Phase 1 (needs lifecycle, sampling, progress modules)
-- Phase 3 depends on Phase 2 (needs workspace, resources)
-- Phases 4-7 depend on Phases 1-3 (need full infrastructure)
-- Phase 8 depends on Phases 3-7 (prompts call tools)
-- Phase 9 depends on Phases 1-8 (cleanup after features complete)
-- Phase 10 depends on Phase 9 (test complete system)
-- Phase 11 depends on Phase 4 (security tests need crawl implementation)
-
-## Parallelization Opportunities
-
-- Tasks within each phase can often be parallelized
-- Phase 4 (crawl) and Phase 5 (analyze) development can overlap once Phase 3 is done
-- Unit test writing can happen in parallel with implementation
-- Documentation (9.4, 9.5) can start after Phase 7
-- Phase 11 (security testing) can start as soon as Phase 4 is complete

package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/proposal.md

@@ -1,24 +0,0 @@
-# Change: Allow Analysis of Incomplete Crawls
-
-## Why
-
-Currently, `webtest_analyze_app` requires a crawl with `status === "completed"` before proceeding. When a crawl is interrupted (cancelled, timed out, errored) or still in progress, the user is blocked with "No completed crawls found" and forced to re-run crawling.
-
-This is problematic because:
-1. Crawl data is captured incrementally - valuable artifacts exist even for incomplete crawls
-2. Users may intentionally stop a crawl early after sufficient data is gathered
-3. Re-running a crawl wastes time and resources when existing data is sufficient
-4. The current behavior removes user agency to decide when enough data exists
-
-## What Changes
-
-- `webtest_analyze_app` SHALL accept crawls with any status that has captured pages
-- When no `crawlId` is provided, the tool selects the most recent crawl with captured data (any status)
-- Analysis output SHALL indicate if the source crawl was incomplete
-- Users can still explicitly specify a `crawlId` to analyze any specific crawl
-
-## Impact
-
-- Affected specs: `webtest-tools`
-- Affected code: `src/tools/webtest/analyze-app.ts`
-- Non-breaking: Existing behavior (analyzing completed crawls) still works

package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/specs/webtest-tools/spec.md

@@ -1,80 +0,0 @@
-## MODIFIED Requirements
-
-### Requirement: webtest_analyze_app Tool
-
-The system SHALL provide a `webtest_analyze_app` tool that reverse-engineers application structure from crawl data.
-
-#### Scenario: Analyze app loads crawl data
-
-- **GIVEN** the tool is called with valid analysisId and crawlId
-- **WHEN** execution begins
-- **THEN** it SHALL load crawl index and artifact references
-- **AND** load page snapshots for key pages
-
-#### Scenario: Analyze app selects most recent crawl with data
-
-- **GIVEN** the tool is called with valid analysisId but no crawlId
-- **WHEN** crawl selection occurs
-- **THEN** it SHALL select the most recent crawl that has captured at least one page
-- **AND** crawl status (completed, in_progress, failed, cancelled) SHALL NOT prevent selection
-
-#### Scenario: Analyze app checks crawl index when workspace metadata is stale
-
-- **GIVEN** the workspace index shows pagesVisited as 0 for a crawl
-- **AND** the actual crawl index contains captured pages
-- **WHEN** crawl selection occurs
-- **THEN** it SHALL read the crawl index to verify actual page count
-- **AND** select the crawl if pages exist in the index
-
-#### Scenario: Analyze app warns about incomplete crawl
-
-- **GIVEN** the selected crawl has status other than "completed"
-- **WHEN** analysis output is generated
-- **THEN** it SHALL include a warning indicating the crawl was incomplete
-- **AND** the warning SHALL include the crawl status and pages captured count
-
-#### Scenario: Analyze app rejects crawl with no captured pages
-
-- **GIVEN** the tool is called with analysisId
-- **WHEN** no crawls exist with captured pages
-- **THEN** it SHALL return an error: "No crawls with captured data found. Run webtest_crawl_app first."
-
-#### Scenario: Analyze app uses sampling for analysis
-
-- **GIVEN** crawl data is loaded
-- **WHEN** analysis is performed
-- **THEN** it SHALL construct sampling prompt with crawl summary and snapshots
-- **AND** request structured analysis via `sampling/createMessage`
-
-#### Scenario: Analyze app extracts application purpose
-
-- **GIVEN** analysis sampling completes
-- **WHEN** results are processed
-- **THEN** output SHALL include identified app purpose
-- **AND** key entities (users, products, orders, etc.)
-
-#### Scenario: Analyze app identifies user flows
-
-- **GIVEN** analysis sampling completes
-- **WHEN** results are processed
-- **THEN** output SHALL include discovered user flows
-- **AND** each flow SHALL have id, name, description, steps
-
-#### Scenario: Analyze app suggests assertions
-
-- **GIVEN** analysis sampling completes
-- **WHEN** results are processed
-- **THEN** output SHALL include suggested assertions for testing
-- **AND** potential risks or edge cases
-
-#### Scenario: Analyze app writes markdown report
-
-- **GIVEN** analysis is complete
-- **WHEN** output is generated
-- **THEN** it SHALL write `app-analysis.md` resource to workspace
-
-#### Scenario: Analyze app outputs URIs
-
-- **GIVEN** analysis is complete
-- **WHEN** tool returns
-- **THEN** it SHALL include `appAnalysisUri` and `flowsIndexUri`

package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/tasks.md

@@ -1,8 +0,0 @@
-## 1. Implementation
-
-- [x] 1.1 Update crawl selection logic in `analyze-app.ts` to accept any crawl with captured pages
-- [x] 1.2 Add warning to analysis output when source crawl status is not "completed"
-- [x] 1.3 Update error message to be more helpful when no crawls have captured data
-- [x] 1.4 Add fallback to check crawl index when workspace metadata shows 0 pages
-- [x] 1.5 Add unit tests for analyzing incomplete crawls
-- [x] 1.6 Update existing tests to reflect new behavior

package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/design.md

@@ -1,90 +0,0 @@
-## Context
-
-The crawl process uses AI sampling to decide next actions at each step. When loop detection mechanisms trigger warnings, the AI model may interpret "try a different action" as "start over from homepage" rather than "try a different element on the current page."
-
-This is exacerbated by:
-- DOM signatures that only capture structural elements, causing false positives on structurally similar pages
-- Loss of loop detection history when resuming from checkpoints
-- Limited action history context (10 actions) insufficient for complex multi-step flows
-
-## Goals / Non-Goals
-
-**Goals:**
-- Prevent AI model from navigating back to start URL mid-flow
-- Reduce false positive loop detection on e-commerce sites
-- Maintain crawl progress context across checkpoint resume
-- Provide sufficient action history for complex flows (15-25 steps)
-
-**Non-Goals:**
-- Complete rewrite of loop detection mechanism
-- Adding new external dependencies
-- Changing the checkpoint file format significantly (additive changes only)
-
-## Decisions
-
-### Decision 1: Enhanced Sampling Prompt with Anti-Reset Guidance
-
-Add explicit instructions in the prompt:
-```
-CRITICAL: Do NOT navigate back to the homepage or start URL to "reset" or "try again".
-If you seem stuck, try different elements on the current page first.
-Navigation to start URL is only allowed if the goal explicitly requires it.
-```
-
-**Rationale:** The AI model responds well to explicit constraints. Making the anti-reset rule clear prevents the misinterpretation of loop warnings.
-
-### Decision 2: Semantic DOM Signature
-
-Include in signature:
-- URL pathname (not query params)
-- Page `<title>` or first `<h1>` text
-- Button text content
-- Data attributes (`data-testid`, `data-page`, etc.)
-- Link hrefs
-- Input types
-
-**Rationale:** E-commerce pages like product listings, cart, and checkout share similar structural elements (forms, buttons, inputs) but differ in semantic content. Including text and data attributes differentiates them.
-
-### Decision 3: Checkpoint State Preservation
-
-Serialize loop detection state to checkpoint:
-```typescript
-interface CrawlCheckpoint {
-  // existing fields...
-  loopDetection?: {
-    domSignatures: Array<[string, number]>;  // Map entries
-    urlVisits: Array<[string, number]>;
-    recentActions: string[];
-  };
-}
-```
-
-**Rationale:** Maps can be serialized as arrays of entries. This is backwards-compatible (optional field).
-
-### Decision 4: Navigation Guard
-
-Track `startUrl` in crawl state. After step 3, if AI requests navigation to start URL:
-1. Log warning
-2. Include warning in next prompt: "Navigation to start URL was blocked. Continue from current page."
-3. Skip the action (don't execute it)
-
-**Rationale:** Legitimate flows rarely need to return to start URL mid-execution. Blocking this prevents restart loops while allowing initial navigation.
-
-## Risks / Trade-offs
-
-| Risk | Mitigation |
-|------|------------|
-| Semantic signature may be too strict | Include only stable elements (title, h1, data-testid) |
-| Checkpoint file size increase | Loop detection data is small (~1KB) |
-| Navigation guard too restrictive | Only activate after step 3, allow explicit override |
-
-## Migration Plan
-
-1. All changes are additive and backwards-compatible
-2. Old checkpoints without `loopDetection` field work as before (fresh state)
-3. No breaking changes to existing APIs
-
-## Open Questions
-
-- Should navigation guard threshold (step 3) be configurable?
-- Should DOM signature include visible text content (may be unstable)?

package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/proposal.md

@@ -1,28 +0,0 @@
-# Change: Fix crawl loop stability and prevent restart from homepage
-
-## Why
-
-During E2E testing on saucedemo.com, the crawl process exhibits unstable behavior: instead of progressing through a purchase flow (login → add to cart → checkout), it repeatedly restarts from the homepage. This is caused by:
-
-1. **AI model misinterpreting loop warnings** as signals to "reset" by navigating home
-2. **DOM signature collisions** between structurally similar pages triggering false loop detection
-3. **Lost loop state on checkpoint resume** causing the AI to lose context
-4. **Limited action history** (only 10 actions) insufficient for multi-step flows
-5. **No explicit anti-reset guidance** in sampling prompts
-
-## What Changes
-
-- **Enhance sampling prompt** with explicit anti-reset guidance preventing homepage navigation mid-flow
-- **Improve DOM signature algorithm** to include semantic context (text content, data attributes, URL path)
-- **Preserve loop detection state** across checkpoint resume
-- **Increase action history window** from 10 to 20 for complex flows
-- **Add navigation guard** that blocks navigation to start URL after initial steps
-- **Include flow stage tracking** in sampling prompt to maintain progress awareness
-
-## Impact
-
-- Affected specs: `webtest-sampling`
-- Affected code:
-  - `src/sampling/prompts.ts` - Enhanced prompt construction
-  - `src/security/index.ts` - Improved DOM signature
-  - `src/tools/webtest/crawl.ts` - Checkpoint state preservation, navigation guard

package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/specs/webtest-sampling/spec.md

@@ -1,90 +0,0 @@
-## ADDED Requirements
-
-### Requirement: Anti-Reset Navigation Guidance
-
-The system SHALL include explicit guidance in crawl sampling prompts to prevent the AI model from navigating back to the start URL mid-flow.
-
-#### Scenario: Prompt includes anti-reset instruction
-
-- **GIVEN** a crawl sampling prompt is constructed
-- **WHEN** the prompt text is built
-- **THEN** it SHALL include an explicit instruction stating navigation to start URL is prohibited unless the goal requires it
-- **AND** the instruction SHALL advise trying different elements on the current page when stuck
-
-#### Scenario: Start URL navigation is identified
-
-- **GIVEN** a crawl is in progress past step 3
-- **WHEN** the AI model requests navigation to the start URL
-- **THEN** the system SHALL log a warning
-- **AND** skip the navigation action
-- **AND** include a warning in the next sampling prompt explaining the action was blocked
-
-### Requirement: Extended Action History Context
-
-The system SHALL provide sufficient action history context for complex multi-step flows.
-
-#### Scenario: Action history window is extended
-
-- **GIVEN** a crawl sampling prompt is constructed
-- **WHEN** action history is included
-- **THEN** it SHALL include the last 20 actions (increased from 10)
-- **AND** each action SHALL include step number, tool, args, and reasoning
-
-#### Scenario: Flow progress indicator is included
-
-- **GIVEN** a crawl sampling prompt is constructed
-- **WHEN** progress information is included
-- **THEN** it SHALL include a flow stage indicator showing:
-  - Current step number
-  - Total steps taken
-  - Percentage of budget used
-  - Goal progress summary from previous iteration
-
-### Requirement: Semantic DOM Signature
-
-The system SHALL use semantic content in DOM signatures to differentiate structurally similar pages.
-
-#### Scenario: DOM signature includes semantic elements
-
-- **GIVEN** a page DOM needs to be fingerprinted for loop detection
-- **WHEN** the DOM signature is created
-- **THEN** the signature SHALL include:
-  - URL pathname (without query parameters)
-  - Page title or first h1 heading text
-  - Button text content
-  - Data attributes (data-testid, data-page, etc.)
-  - Link hrefs
-  - Input types
-
-#### Scenario: Similar structure pages have different signatures
-
-- **GIVEN** two e-commerce pages with similar HTML structure
-- **WHEN** one is a product listing and another is a cart page
-- **THEN** their DOM signatures SHALL be different due to semantic content differences
-
-### Requirement: Loop Detection State Preservation
-
-The system SHALL preserve loop detection state across checkpoint resume to maintain crawl context.
-
-#### Scenario: Checkpoint includes loop detection state
-
-- **GIVEN** a crawl checkpoint is saved
-- **WHEN** the checkpoint data is written
-- **THEN** it SHALL include serialized loop detection data:
-  - DOM signature visit counts
-  - URL visit counts
-  - Recent actions list
-
-#### Scenario: Checkpoint resume restores loop detection state
-
-- **GIVEN** a crawl resumes from checkpoint
-- **WHEN** the checkpoint data is loaded
-- **THEN** the loop detection state SHALL be restored from checkpoint
-- **AND** the crawl SHALL continue with full context of previous iterations
-
-#### Scenario: Missing loop detection data uses fresh state
-
-- **GIVEN** a crawl resumes from an old checkpoint without loop detection data
-- **WHEN** the checkpoint data is loaded
-- **THEN** the system SHALL initialize fresh loop detection state
-- **AND** log a warning about missing historical context

package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/tasks.md

@@ -1,33 +0,0 @@
-## 1. Sampling Prompt Enhancements
-
-- [x] 1.1 Add explicit anti-reset guidance to `buildCrawlActionPrompt` in `src/sampling/prompts.ts`
-- [x] 1.2 Increase action history window from 10 to 20 actions
-- [x] 1.3 Add flow stage indicator showing progress percentage and steps completed
-- [x] 1.4 Add explicit instruction: "NEVER navigate back to homepage/start URL to 'reset'"
-
-## 2. DOM Signature Improvements
-
-- [x] 2.1 Update `createDomSignature` in `src/security/index.ts` to include:
-  - Page title/heading text
-  - Data attributes (data-testid, data-*)
-  - URL path component
-  - Key text content from buttons and labels
-- [x] 2.2 Add tests for DOM signature differentiation between similar e-commerce pages
-
-## 3. Checkpoint State Preservation
-
-- [x] 3.1 Modify checkpoint save in `crawl.ts` to include `loopDetection` state
-- [x] 3.2 Modify checkpoint resume to restore `loopDetection.domSignatures`, `urlVisits`, and `recentActions`
-- [x] 3.3 Update `CrawlCheckpoint` type to include loop detection data
-
-## 4. Navigation Guard
-
-- [x] 4.1 Add `startUrl` tracking to crawl state
-- [x] 4.2 Implement navigation guard that warns/blocks navigation to start URL after step 3
-- [x] 4.3 Include guard warning in sampling prompt when blocked
-
-## 5. Testing
-
-- [x] 5.1 Add unit tests for enhanced DOM signature
-- [x] 5.2 Add unit tests for checkpoint state preservation (covered by existing tests)
-- [x] 5.3 All 458 tests pass with no regressions