retestkit 1.4.1

package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/design.md

@@ -0,0 +1,558 @@
+# Design: Markdown Artifacts Architecture
+
+## Overview
+
+This document describes the architectural approach for converting all JSON artifacts to markdown format while preserving programmatic access through YAML frontmatter.
+
+## Core Design Decisions
+
+### 1. YAML Frontmatter as Structured Data Layer
+
+All markdown artifacts that require programmatic access will use YAML frontmatter delimited by `---`:
+
+```markdown
+---
+# Structured data here (parseable)
+key: value
+nested:
+  field: data
+---
+
+# Human-Readable Content
+
+Content formatted for reading...
+```
+
+**Rationale**:
+- Standard pattern used by static site generators, documentation tools
+- Well-supported parsing libraries in Node.js (`gray-matter`, `yaml`)
+- Separates machine-readable from human-readable content
+- Maintains single source of truth (no sync between .json and .md)
+
+### 2. Artifact-Specific Formats
+
+#### 2.1 Workspace Index (`index.md`)
+
+```markdown
+---
+analysisId: "uuid"
+createdAt: "2025-01-15T10:30:00Z"
+url: "https://example.com"
+focus: "checkout flow"
+status: "in_progress" | "completed"
+limits:
+  maxSteps: 100
+  maxPages: 20
+  maxMinutes: 30
+allowedDomains:
+  - "example.com"
+crawls:
+  - id: "crawl-001"
+    status: "completed"
+    pagesDiscovered: 15
+    createdAt: "2025-01-15T10:31:00Z"
+analysis:
+  createdAt: "2025-01-15T10:45:00Z"
+  appAnalysisUri: "webtest://uuid/analysis/app-analysis.md"
+tests:
+  createdAt: "2025-01-15T10:50:00Z"
+  testsUri: "webtest://uuid/tests/tests.md"
+  testCount: 12
+runs:
+  - id: "run-001"
+    testCaseId: "test-001"
+    status: "passed"
+    createdAt: "2025-01-15T11:00:00Z"
+---
+
+# Analysis Workspace
+
+**Target**: https://example.com
+**Focus**: checkout flow
+**Status**: In Progress
+**Created**: January 15, 2025 at 10:30 AM
+
+## Configuration
+
+| Setting | Value |
+|---------|-------|
+| Max Steps | 100 |
+| Max Pages | 20 |
+| Max Minutes | 30 |
+| Allowed Domains | example.com |
+
+## Crawls
+
+### crawl-001
+- **Status**: Completed
+- **Pages**: 15 discovered
+- **Started**: 10:31 AM
+
+## Analysis
+
+Analysis completed at 10:45 AM.
+- [View Analysis Report](./analysis/app-analysis.md)
+
+## Tests
+
+12 test cases generated at 10:50 AM.
+- [View Test Cases](./tests/tests.md)
+
+## Test Runs
+
+| Run | Test | Status | Time |
+|-----|------|--------|------|
+| run-001 | test-001 | Passed | 11:00 AM |
+```
+
+#### 2.2 Crawl Index (`crawls/{id}/index.md`)
+
+```markdown
+---
+crawlId: "uuid"
+analysisId: "parent-uuid"
+status: "completed" | "in_progress" | "cancelled" | "limits_reached"
+goal: "Explore the checkout flow"
+createdAt: "2025-01-15T10:31:00Z"
+completedAt: "2025-01-15T10:44:00Z"
+budget:
+  maxSteps: 100
+  stepsUsed: 45
+  maxPages: 20
+  pagesDiscovered: 15
+  maxMinutes: 30
+  minutesElapsed: 13
+pages:
+  - id: "page-001"
+    url: "https://example.com/"
+    title: "Home"
+    capturedAt: "2025-01-15T10:31:05Z"
+  - id: "page-002"
+    url: "https://example.com/products"
+    title: "Products"
+    capturedAt: "2025-01-15T10:32:10Z"
+actionHistory:
+  - step: 1
+    tool: "browser_click"
+    element: "Products link"
+    result: "success"
+loopDetection:
+  urlVisitCounts:
+    "https://example.com/": 2
+  recentActions: []
+  domSignatures: []
+---
+
+# Crawl Report
+
+**Goal**: Explore the checkout flow
+**Status**: Completed
+**Duration**: 13 minutes (10:31 AM - 10:44 AM)
+
+## Budget Usage
+
+| Metric | Used | Limit | % |
+|--------|------|-------|---|
+| Steps | 45 | 100 | 45% |
+| Pages | 15 | 20 | 75% |
+| Time | 13m | 30m | 43% |
+
+## Pages Discovered
+
+| # | Page | URL | Captured |
+|---|------|-----|----------|
+| 1 | Home | https://example.com/ | 10:31:05 |
+| 2 | Products | https://example.com/products | 10:32:10 |
+...
+
+## Action History
+
+1. **Click** Products link → success
+2. **Click** Add to Cart button → success
+...
+```
+
+#### 2.3 Checkpoint (`crawls/{id}/checkpoint.md`)
+
+```markdown
+---
+crawlId: "uuid"
+step: 45
+savedAt: "2025-01-15T10:40:00Z"
+currentUrl: "https://example.com/cart"
+currentPageId: "page-012"
+visitedUrls:
+  - "https://example.com/"
+  - "https://example.com/products"
+  - "https://example.com/cart"
+actionHistory:
+  - step: 43
+    tool: "browser_click"
+    args:
+      ref: "btn-123"
+    result: "success"
+loopDetection:
+  urlVisitCounts:
+    "https://example.com/": 2
+  recentActions:
+    - "browser_click:{\"ref\":\"btn-123\"}"
+  domSignatures:
+    - "hash-abc"
+    - "hash-def"
+goalProgress: "Added item to cart, navigating to checkout"
+---
+
+# Crawl Checkpoint
+
+**Step**: 45
+**Saved**: 10:40 AM
+**Current URL**: https://example.com/cart
+
+## Progress
+
+Added item to cart, navigating to checkout
+
+## Recent Actions
+
+| Step | Action | Result |
+|------|--------|--------|
+| 43 | Click btn-123 | Success |
+| 44 | Navigate cart | Success |
+| 45 | Snapshot | Success |
+
+## Loop Detection State
+
+- URL `/` visited 2 times
+- Recent action signatures tracked: 3
+- DOM signatures in window: 2
+```
+
+#### 2.4 Page Snapshot (`pages/{id}/snapshot.md`)
+
+```markdown
+---
+pageId: "page-001"
+url: "https://example.com/"
+title: "Example Store - Home"
+capturedAt: "2025-01-15T10:31:05Z"
+domSignature: "hash-abc123"
+snapshot:
+  role: "WebArea"
+  name: "Example Store - Home"
+  children:
+    - role: "navigation"
+      name: "Main navigation"
+      children:
+        - role: "link"
+          name: "Home"
+          ref: "nav-home"
+        - role: "link"
+          name: "Products"
+          ref: "nav-products"
+    - role: "main"
+      name: ""
+      children:
+        - role: "heading"
+          name: "Welcome to Example Store"
+          level: 1
+---
+
+# Page Snapshot: Example Store - Home
+
+**URL**: https://example.com/
+**Captured**: 10:31:05 AM
+**DOM Signature**: hash-abc123
+
+## Accessibility Tree
+
+```
+WebArea "Example Store - Home"
+├── navigation "Main navigation"
+│   ├── link "Home" [ref: nav-home]
+│   └── link "Products" [ref: nav-products]
+└── main
+    └── heading "Welcome to Example Store" (h1)
+```
+
+## Interactive Elements
+
+| Element | Type | Ref |
+|---------|------|-----|
+| Home | link | nav-home |
+| Products | link | nav-products |
+```
+
+#### 2.5 Flows (`analysis/flows.md`)
+
+Instead of separate `flows.json`, embed flows in `app-analysis.md` or create a dedicated `flows.md`:
+
+```markdown
+---
+flows:
+  - id: "flow-checkout"
+    name: "Checkout Flow"
+    entryPoint: "https://example.com/cart"
+    description: "Complete purchase from cart to confirmation"
+    steps:
+      - "View cart contents"
+      - "Click proceed to checkout"
+      - "Enter shipping info"
+      - "Select payment method"
+      - "Confirm order"
+  - id: "flow-search"
+    name: "Product Search"
+    entryPoint: "https://example.com/"
+    description: "Search and filter products"
+    steps:
+      - "Enter search term"
+      - "Apply filters"
+      - "View results"
+---
+
+# User Flows
+
+## Checkout Flow
+
+**ID**: flow-checkout
+**Entry Point**: /cart
+**Description**: Complete purchase from cart to confirmation
+
+### Steps
+
+1. View cart contents
+2. Click proceed to checkout
+3. Enter shipping info
+4. Select payment method
+5. Confirm order
+
+---
+
+## Product Search
+
+**ID**: flow-search
+**Entry Point**: /
+**Description**: Search and filter products
+
+### Steps
+
+1. Enter search term
+2. Apply filters
+3. View results
+```
+
+#### 2.6 Test Definitions (`tests/tests.md`)
+
+Consolidate `tests.json` and `tests.md` into single file:
+
+```markdown
+---
+generatedAt: "2025-01-15T10:50:00Z"
+strategy: "comprehensive"
+coverage:
+  flowsCovered:
+    - "flow-checkout"
+    - "flow-search"
+  estimatedCoverage: "85%"
+tests:
+  - id: "test-checkout-happy"
+    name: "Checkout Happy Path"
+    category: "happy_path"
+    purpose: "Verify complete checkout flow works"
+    preconditions:
+      - "User has items in cart"
+      - "User is not logged in"
+    steps:
+      - stepNumber: 1
+        action: "click"
+        element: "Proceed to Checkout button"
+        target: "checkout-btn"
+        expected: "Shipping form appears"
+      - stepNumber: 2
+        action: "fill"
+        element: "Shipping form"
+        value: "Test address data"
+        expected: "Form accepts input"
+    expectedOutcomes:
+      - "Order confirmation page displayed"
+      - "Order number generated"
+    tags:
+      - "checkout"
+      - "critical"
+---
+
+# Generated Test Cases
+
+**Generated**: January 15, 2025 at 10:50 AM
+**Strategy**: Comprehensive
+**Coverage**: 85% (2 flows)
+
+## Test Summary
+
+| ID | Name | Category | Tags |
+|----|------|----------|------|
+| test-checkout-happy | Checkout Happy Path | Happy Path | checkout, critical |
+
+---
+
+## test-checkout-happy: Checkout Happy Path
+
+**Category**: Happy Path
+**Purpose**: Verify complete checkout flow works
+
+### Preconditions
+
+- User has items in cart
+- User is not logged in
+
+### Steps
+
+| # | Action | Element | Expected |
+|---|--------|---------|----------|
+| 1 | Click | Proceed to Checkout button | Shipping form appears |
+| 2 | Fill | Shipping form | Form accepts input |
+
+### Expected Outcomes
+
+- Order confirmation page displayed
+- Order number generated
+```
+
+#### 2.7 Test Run Report (`runs/{id}/report.md`)
+
+```markdown
+---
+runId: "run-001"
+analysisId: "parent-uuid"
+testCaseId: "test-checkout-happy"
+testCaseName: "Checkout Happy Path"
+status: "passed" | "failed" | "cancelled"
+startedAt: "2025-01-15T11:00:00Z"
+completedAt: "2025-01-15T11:02:30Z"
+durationMs: 150000
+steps:
+  - stepNumber: 1
+    status: "passed"
+    action: "click"
+    element: "Proceed to Checkout button"
+    expected: "Shipping form appears"
+    actual: "Shipping form displayed"
+    evidence:
+      screenshotUri: "webtest://uuid/runs/run-001/steps/1/screenshot.png"
+      snapshotUri: "webtest://uuid/runs/run-001/steps/1/snapshot.md"
+  - stepNumber: 2
+    status: "passed"
+    action: "fill"
+    element: "Shipping form"
+    expected: "Form accepts input"
+    actual: "Form filled successfully"
+summary:
+  totalSteps: 2
+  passedSteps: 2
+  failedSteps: 0
+---
+
+# Test Run Report
+
+**Test**: Checkout Happy Path (test-checkout-happy)
+**Status**: PASSED
+**Duration**: 2m 30s
+**Run ID**: run-001
+
+## Summary
+
+| Metric | Value |
+|--------|-------|
+| Total Steps | 2 |
+| Passed | 2 |
+| Failed | 0 |
+| Success Rate | 100% |
+
+## Step Results
+
+### Step 1: Click Proceed to Checkout button
+
+**Status**: PASSED
+**Expected**: Shipping form appears
+**Actual**: Shipping form displayed
+
+Evidence:
+- [Screenshot](./steps/1/screenshot.png)
+- [Snapshot](./steps/1/snapshot.md)
+
+---
+
+### Step 2: Fill Shipping form
+
+**Status**: PASSED
+**Expected**: Form accepts input
+**Actual**: Form filled successfully
+
+Evidence:
+- [Screenshot](./steps/2/screenshot.png)
+- [Snapshot](./steps/2/snapshot.md)
+```
+
+### 3. Workspace Manager Changes
+
+The `WorkspaceManager` in `src/workspace/index.ts` needs these modifications:
+
+```typescript
+// New helper for markdown with frontmatter
+async function writeMarkdownWithFrontmatter(
+  filePath: string,
+  frontmatter: Record<string, unknown>,
+  content: string
+): Promise<void> {
+  const yaml = YAML.stringify(frontmatter);
+  const markdown = `---\n${yaml}---\n\n${content}`;
+  await writeFile(filePath, markdown, 'utf-8');
+}
+
+// New helper to read frontmatter
+async function readMarkdownFrontmatter<T>(filePath: string): Promise<T> {
+  const content = await readFile(filePath, 'utf-8');
+  const { data } = matter(content);
+  return data as T;
+}
+```
+
+### 4. Resource Manager Changes
+
+Update `src/resources/index.ts` to:
+- Map `.md` extensions for former JSON resources
+- Return `text/markdown` MIME type
+- Parse frontmatter when returning structured content
+
+### 5. Tool Output Changes
+
+Each tool's return value updates:
+- `webtest_init`: Return `statusUri` as `index.md`
+- `webtest_crawl_app`: Return `crawlIndexUri` as `index.md`
+- `webtest_analyze_app`: Remove `flowsUri` (embedded in analysis), or update to `flows.md`
+- `webtest_generate_tests`: Return single `testsUri` pointing to `tests.md`
+- `webtest_run_tests`: Return `reportUri` as `report.md`
+
+## Implementation Sequence
+
+1. **Add markdown utilities** - frontmatter read/write helpers
+2. **Update workspace index** - migrate to `.md` format
+3. **Update crawl index** - migrate to `.md` format
+4. **Update checkpoint** - migrate to `.md` format
+5. **Update page snapshot** - migrate to `.md` format with tree visualization
+6. **Consolidate flows** - embed in analysis or create `flows.md`
+7. **Consolidate test definitions** - single `tests.md` file
+8. **Update test run report** - migrate to `report.md`
+9. **Update resource manager** - handle new paths and MIME types
+10. **Update tool outputs** - reference new file paths
+
+## Dependencies
+
+- `gray-matter` - Parse YAML frontmatter from markdown
+- `yaml` - Serialize objects to YAML (already likely present)
+
+## Testing Strategy
+
+1. **Unit tests**: Frontmatter read/write utilities
+2. **Integration tests**: Full workflow produces valid markdown
+3. **Snapshot tests**: Verify markdown output format stability
+4. **Manual testing**: Browse workspace folder, verify readability

package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/proposal.md

@@ -0,0 +1,119 @@
+# Proposal: Use Markdown for All Artifacts
+
+## Change ID
+`use-markdown-artifacts`
+
+## Status
+Draft
+
+## Problem Statement
+
+Currently, the testing-mcp server stores and exposes artifacts using a mix of formats:
+- **Markdown (`.md`)**: Analysis reports, test case documentation
+- **JSON (`.json`)**: Workspace indexes, crawl indexes, flows data, test definitions, test run reports, checkpoints, snapshots
+
+This creates friction for consumers who want human-readable artifacts:
+1. **Index files** (workspace, crawl, test run) are JSON - hard to read without tooling
+2. **Flows data** is JSON - not directly usable for documentation or review
+3. **Test definitions** have both `.json` and `.md` but tools return the JSON path as primary
+4. **Test run results** are JSON only - no human-readable report
+5. **Checkpoints** are JSON - difficult to debug crawl state
+6. **Snapshots** are JSON - accessibility trees not easily inspectable
+
+The user wants **all artifacts** to be stored and handed over as readable markdown files, making the workspace self-documenting and easy to inspect without specialized tools.
+
+## Proposed Solution
+
+Convert all JSON-based artifacts to markdown format while preserving structured data through YAML frontmatter where needed. This creates a consistent, human-readable workspace.
+
+### Artifact Format Changes
+
+| Artifact | Current | Proposed |
+|----------|---------|----------|
+| Workspace index | `index.json` | `index.md` with YAML frontmatter |
+| Crawl index | `crawls/{id}/index.json` | `crawls/{id}/index.md` with YAML frontmatter |
+| Crawl checkpoint | `checkpoint.json` | `checkpoint.md` with YAML frontmatter |
+| Page snapshot | `pages/{id}/snapshot.json` | `pages/{id}/snapshot.md` (formatted tree) |
+| Analysis flows | `analysis/flows.json` | `analysis/flows.md` (embedded in analysis or separate) |
+| Test definitions | `tests/tests.json` + `tests.md` | `tests/tests.md` only (with YAML frontmatter for structured data) |
+| Test run index | `runs/{id}/index.json` | `runs/{id}/report.md` with YAML frontmatter |
+| Test step snapshot | `steps/{n}/snapshot.json` | `steps/{n}/snapshot.md` |
+
+### YAML Frontmatter for Structured Data
+
+For artifacts that require programmatic access (workspace index, crawl index, test runs), use YAML frontmatter to embed structured data:
+
+```markdown
+---
+analysisId: abc-123
+createdAt: 2025-01-15T10:30:00Z
+url: https://example.com
+status: completed
+crawls:
+  - id: crawl-001
+    status: completed
+    pagesDiscovered: 15
+---
+
+# Analysis Workspace
+
+**Target URL**: https://example.com
+**Created**: January 15, 2025 at 10:30 AM
+**Status**: Completed
+
+## Crawls
+
+### Crawl: crawl-001
+- **Status**: Completed
+- **Pages Discovered**: 15
+...
+```
+
+### Benefits
+
+1. **Self-documenting workspace**: Any developer can browse the workspace folder and understand state
+2. **Version control friendly**: Markdown diffs are readable in PRs
+3. **No tooling required**: Standard text editors can view all artifacts
+4. **Consistent format**: One format (markdown) for all human-facing artifacts
+5. **Programmatic access preserved**: YAML frontmatter can be parsed when needed
+
+## Scope
+
+### In Scope
+- Workspace index format change
+- Crawl index format change
+- Checkpoint format change
+- Page snapshot format change
+- Flows output format change
+- Test definition format consolidation
+- Test run report format change
+- Test step snapshot format change
+- Resource URI updates (`.json` → `.md`)
+- Tool output URI/path updates
+
+### Out of Scope
+- Binary artifacts (screenshots remain PNG/JPEG)
+- DOM HTML (remains `.html`)
+- Backward compatibility migration (clean break)
+
+## Affected Specs
+
+1. **webtest-resources**: Update resource URIs from `.json` to `.md`, update MIME types
+2. **webtest-tools**: Update tool outputs to reference `.md` files
+
+## Risks and Mitigations
+
+| Risk | Mitigation |
+|------|------------|
+| YAML frontmatter parsing complexity | Use well-established libraries (gray-matter, yaml) |
+| Larger file sizes for structured data | Markdown is still compact; compression available if needed |
+| Breaking existing clients | This is a breaking change; document in release notes |
+| Snapshot readability for large trees | Use collapsible sections or truncation with full data in frontmatter |
+
+## Success Criteria
+
+1. All workspace artifacts are `.md` files (except binary screenshots and HTML DOM)
+2. Workspace folder is browsable and understandable without any tools
+3. Structured data remains accessible via YAML frontmatter parsing
+4. All tool outputs reference markdown file paths
+5. All resource URIs use `.md` extension for converted artifacts