retestkit 1.4.1 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +59 -40
- package/dist/config.js +8 -8
- package/dist/config.js.map +1 -1
- package/dist/logger.js +1 -1
- package/dist/logger.js.map +1 -1
- package/dist/prompts/index.d.ts +1 -1
- package/dist/prompts/index.d.ts.map +1 -1
- package/dist/prompts/index.js +21 -21
- package/dist/prompts/index.js.map +1 -1
- package/dist/prompts/templates/mcp/retest-crawl.md +7 -0
- package/{src/prompts/templates/mcp/webtest-discover-flows.md → dist/prompts/templates/mcp/retest-discover-flows.md} +1 -1
- package/{src/prompts/templates/mcp/webtest-discover.md → dist/prompts/templates/mcp/retest-discover.md} +2 -2
- package/dist/prompts/templates/mcp/retest-full-workflow.md +12 -0
- package/{src/prompts/templates/mcp/webtest-generate-tests.md → dist/prompts/templates/mcp/retest-generate-tests.md} +1 -1
- package/{src/prompts/templates/mcp/webtest-run-test.md → dist/prompts/templates/mcp/retest-run-test.md} +1 -1
- package/{src/prompts/templates/mcp/webtest-start.md → dist/prompts/templates/mcp/retest-start.md} +1 -1
- package/{src → dist}/prompts/templates/sampling/system-prefix.md +1 -1
- package/dist/resources/index.js +7 -7
- package/dist/resources/index.js.map +1 -1
- package/dist/schemas/config.js +2 -2
- package/dist/schemas/config.js.map +1 -1
- package/dist/security/index.js +1 -1
- package/dist/security/index.js.map +1 -1
- package/dist/server.js +3 -3
- package/dist/server.js.map +1 -1
- package/dist/test-utils/mock-context.js +22 -22
- package/dist/test-utils/mock-context.js.map +1 -1
- package/dist/tools/index.d.ts +1 -1
- package/dist/tools/index.d.ts.map +1 -1
- package/dist/tools/index.js +5 -5
- package/dist/tools/index.js.map +1 -1
- package/dist/tools/retest/crawl.d.ts.map +1 -0
- package/dist/tools/{webtest → retest}/crawl.js +7 -7
- package/dist/tools/retest/crawl.js.map +1 -0
- package/dist/tools/retest/discover-features.d.ts.map +1 -0
- package/dist/tools/{webtest → retest}/discover-features.js +6 -6
- package/dist/tools/retest/discover-features.js.map +1 -0
- package/dist/tools/retest/discover-flows.d.ts.map +1 -0
- package/dist/tools/{webtest → retest}/discover-flows.js +6 -6
- package/dist/tools/retest/discover-flows.js.map +1 -0
- package/dist/tools/retest/generate-tests.d.ts.map +1 -0
- package/dist/tools/{webtest → retest}/generate-tests.js +5 -5
- package/dist/tools/retest/generate-tests.js.map +1 -0
- package/dist/tools/retest/index.d.ts.map +1 -0
- package/dist/tools/retest/index.js.map +1 -0
- package/dist/tools/retest/run-test-case.d.ts.map +1 -0
- package/dist/tools/{webtest → retest}/run-test-case.js +3 -3
- package/dist/tools/retest/run-test-case.js.map +1 -0
- package/dist/tools/retest/schemas.d.ts.map +1 -0
- package/dist/tools/retest/schemas.js.map +1 -0
- package/dist/tools/retest/start-analysis.d.ts.map +1 -0
- package/dist/tools/{webtest → retest}/start-analysis.js +5 -5
- package/dist/tools/retest/start-analysis.js.map +1 -0
- package/dist/workspace/index.js +8 -8
- package/dist/workspace/index.js.map +1 -1
- package/dist/workspace/types.d.ts +2 -2
- package/dist/workspace/types.d.ts.map +1 -1
- package/package.json +6 -2
- package/.claude/commands/openspec/apply.md +0 -23
- package/.claude/commands/openspec/archive.md +0 -27
- package/.claude/commands/openspec/proposal.md +0 -28
- package/.gemini/commands/openspec/apply.toml +0 -21
- package/.gemini/commands/openspec/archive.toml +0 -25
- package/.gemini/commands/openspec/proposal.toml +0 -26
- package/.github/prompts/openspec-apply.prompt.md +0 -22
- package/.github/prompts/openspec-archive.prompt.md +0 -26
- package/.github/prompts/openspec-proposal.prompt.md +0 -27
- package/.github/workflows/release.yml +0 -33
- package/.kilocode/workflows/openspec-apply.md +0 -17
- package/.kilocode/workflows/openspec-archive.md +0 -21
- package/.kilocode/workflows/openspec-proposal.md +0 -22
- package/.mcp.json +0 -23
- package/.opencode/command/openspec-apply.md +0 -25
- package/.opencode/command/openspec-archive.md +0 -28
- package/.opencode/command/openspec-proposal.md +0 -30
- package/.roo/commands/openspec-apply.md +0 -20
- package/.roo/commands/openspec-archive.md +0 -24
- package/.roo/commands/openspec-proposal.md +0 -25
- package/.vscode/mcp.json +0 -23
- package/AGENTS.md +0 -18
- package/CLAUDE.md +0 -18
- package/dist/tools/webtest/crawl.d.ts.map +0 -1
- package/dist/tools/webtest/crawl.js.map +0 -1
- package/dist/tools/webtest/discover-features.d.ts.map +0 -1
- package/dist/tools/webtest/discover-features.js.map +0 -1
- package/dist/tools/webtest/discover-flows.d.ts.map +0 -1
- package/dist/tools/webtest/discover-flows.js.map +0 -1
- package/dist/tools/webtest/generate-tests.d.ts.map +0 -1
- package/dist/tools/webtest/generate-tests.js.map +0 -1
- package/dist/tools/webtest/index.d.ts.map +0 -1
- package/dist/tools/webtest/index.js.map +0 -1
- package/dist/tools/webtest/run-test-case.d.ts.map +0 -1
- package/dist/tools/webtest/run-test-case.js.map +0 -1
- package/dist/tools/webtest/schemas.d.ts.map +0 -1
- package/dist/tools/webtest/schemas.js.map +0 -1
- package/dist/tools/webtest/start-analysis.d.ts.map +0 -1
- package/dist/tools/webtest/start-analysis.js.map +0 -1
- package/openspec/AGENTS.md +0 -456
- package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/proposal.md +0 -33
- package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/specs/webtest-resources/spec.md +0 -27
- package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/specs/webtest-tools/spec.md +0 -304
- package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/tasks.md +0 -43
- package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/design.md +0 -209
- package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/proposal.md +0 -41
- package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/specs/mcp-server-core/spec.md +0 -183
- package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/tasks.md +0 -112
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/design.md +0 -333
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/proposal.md +0 -66
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/mcp-server-core/spec.md +0 -129
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-lifecycle/spec.md +0 -138
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-logging/spec.md +0 -211
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-prompts/spec.md +0 -157
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-resources/spec.md +0 -213
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-sampling/spec.md +0 -257
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-tools/spec.md +0 -501
- package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/tasks.md +0 -264
- package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/proposal.md +0 -24
- package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/specs/webtest-tools/spec.md +0 -80
- package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/tasks.md +0 -8
- package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/design.md +0 -90
- package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/proposal.md +0 -28
- package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/specs/webtest-sampling/spec.md +0 -90
- package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/tasks.md +0 -33
- package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/design.md +0 -558
- package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/proposal.md +0 -119
- package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/specs/webtest-resources/spec.md +0 -109
- package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/specs/webtest-tools/spec.md +0 -121
- package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/tasks.md +0 -133
- package/openspec/changes/extract-prompts-to-markdown/design.md +0 -86
- package/openspec/changes/extract-prompts-to-markdown/proposal.md +0 -50
- package/openspec/changes/extract-prompts-to-markdown/specs/webtest-prompts/spec.md +0 -74
- package/openspec/changes/extract-prompts-to-markdown/tasks.md +0 -40
- package/openspec/changes/refactor-webtest-naming/design.md +0 -95
- package/openspec/changes/refactor-webtest-naming/proposal.md +0 -66
- package/openspec/changes/refactor-webtest-naming/specs/webtest-prompts/spec.md +0 -79
- package/openspec/changes/refactor-webtest-naming/specs/webtest-resources/spec.md +0 -80
- package/openspec/changes/refactor-webtest-naming/specs/webtest-sampling/spec.md +0 -122
- package/openspec/changes/refactor-webtest-naming/specs/webtest-tools/spec.md +0 -113
- package/openspec/changes/refactor-webtest-naming/tasks.md +0 -119
- package/openspec/changes/rename-package-to-retest/proposal.md +0 -52
- package/openspec/changes/rename-package-to-retest/specs/mcp-server-core/spec.md +0 -53
- package/openspec/changes/rename-package-to-retest/specs/retest-lifecycle/spec.md +0 -68
- package/openspec/changes/rename-package-to-retest/specs/retest-logging/spec.md +0 -35
- package/openspec/changes/rename-package-to-retest/specs/retest-prompts/spec.md +0 -159
- package/openspec/changes/rename-package-to-retest/specs/retest-resources/spec.md +0 -251
- package/openspec/changes/rename-package-to-retest/specs/retest-sampling/spec.md +0 -99
- package/openspec/changes/rename-package-to-retest/specs/retest-tools/spec.md +0 -295
- package/openspec/changes/rename-package-to-retest/tasks.md +0 -71
- package/openspec/project.md +0 -31
- package/openspec/specs/mcp-server-core/spec.md +0 -178
- package/openspec/specs/webtest-lifecycle/spec.md +0 -136
- package/openspec/specs/webtest-logging/spec.md +0 -209
- package/openspec/specs/webtest-prompts/spec.md +0 -155
- package/openspec/specs/webtest-resources/spec.md +0 -248
- package/openspec/specs/webtest-sampling/spec.md +0 -344
- package/openspec/specs/webtest-tools/spec.md +0 -282
- package/release.config.js +0 -9
- package/src/config.test.ts +0 -96
- package/src/config.ts +0 -32
- package/src/elicitation/index.test.ts +0 -399
- package/src/elicitation/index.ts +0 -171
- package/src/elicitation/types.ts +0 -68
- package/src/index.ts +0 -83
- package/src/lifecycle/index.test.ts +0 -260
- package/src/lifecycle/index.ts +0 -101
- package/src/logger.redaction.test.ts +0 -322
- package/src/logger.test.ts +0 -123
- package/src/logger.ts +0 -229
- package/src/playwright-client/index.ts +0 -392
- package/src/playwright-client/types.ts +0 -99
- package/src/progress/index.test.ts +0 -327
- package/src/progress/index.ts +0 -170
- package/src/progress/types.ts +0 -25
- package/src/prompts/index.test.ts +0 -451
- package/src/prompts/index.ts +0 -246
- package/src/prompts/loader.test.ts +0 -100
- package/src/prompts/loader.ts +0 -59
- package/src/prompts/templates/mcp/webtest-crawl.md +0 -7
- package/src/prompts/templates/mcp/webtest-full-workflow.md +0 -12
- package/src/resources/index.ts +0 -250
- package/src/resources/subscriptions.ts +0 -37
- package/src/sampling/index.test.ts +0 -414
- package/src/sampling/index.ts +0 -286
- package/src/sampling/prompts.ts +0 -194
- package/src/sampling/types.ts +0 -60
- package/src/schemas/config.ts +0 -39
- package/src/security/index.test.ts +0 -441
- package/src/security/index.ts +0 -361
- package/src/security/security-scenarios.test.ts +0 -468
- package/src/server.ts +0 -211
- package/src/test-utils/index.ts +0 -6
- package/src/test-utils/mock-context.ts +0 -426
- package/src/test-utils/mock-playwright-client.ts +0 -422
- package/src/tools/index.ts +0 -11
- package/src/tools/webtest/crawl.test.ts +0 -834
- package/src/tools/webtest/crawl.ts +0 -901
- package/src/tools/webtest/discover-features.ts +0 -412
- package/src/tools/webtest/discover-flows.ts +0 -408
- package/src/tools/webtest/generate-tests.test.ts +0 -532
- package/src/tools/webtest/generate-tests.ts +0 -425
- package/src/tools/webtest/index.ts +0 -7
- package/src/tools/webtest/integration.test.ts +0 -536
- package/src/tools/webtest/run-test-case.test.ts +0 -659
- package/src/tools/webtest/run-test-case.ts +0 -508
- package/src/tools/webtest/schemas.ts +0 -201
- package/src/tools/webtest/start-analysis.test.ts +0 -151
- package/src/tools/webtest/start-analysis.ts +0 -158
- package/src/transports/http.ts +0 -19
- package/src/transports/index.ts +0 -30
- package/src/transports/stdio.ts +0 -7
- package/src/types/capabilities.test.ts +0 -193
- package/src/types/capabilities.ts +0 -50
- package/src/types/context.ts +0 -21
- package/src/types/tool.ts +0 -11
- package/src/workspace/index.ts +0 -945
- package/src/workspace/markdown.ts +0 -272
- package/src/workspace/types.ts +0 -186
- package/tests/integration/server.test.ts +0 -89
- package/tests/integration/tools.test.ts +0 -99
- package/tsconfig.json +0 -20
- package/vitest.config.ts +0 -9
- package/vitest.integration.config.ts +0 -10
- /package/{src → dist}/prompts/templates/sampling/crawl-action.md +0 -0
- /package/{src → dist}/prompts/templates/sampling/feature-discovery.md +0 -0
- /package/{src → dist}/prompts/templates/sampling/flow-discovery.md +0 -0
- /package/{src → dist}/prompts/templates/sampling/page-content-wrapper.md +0 -0
- /package/{src → dist}/prompts/templates/sampling/test-evaluation.md +0 -0
- /package/{src → dist}/prompts/templates/sampling/test-generation.md +0 -0
- /package/dist/tools/{webtest → retest}/crawl.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/discover-features.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/discover-flows.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/generate-tests.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/index.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/index.js +0 -0
- /package/dist/tools/{webtest → retest}/run-test-case.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/schemas.d.ts +0 -0
- /package/dist/tools/{webtest → retest}/schemas.js +0 -0
- /package/dist/tools/{webtest → retest}/start-analysis.d.ts +0 -0
package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/specs/webtest-resources/spec.md
DELETED
|
@@ -1,109 +0,0 @@
|
|
|
1
|
-
# webtest-resources Spec Delta
|
|
2
|
-
|
|
3
|
-
## MODIFIED Requirements
|
|
4
|
-
|
|
5
|
-
### Requirement: Resource URI Scheme
|
|
6
|
-
|
|
7
|
-
The system SHALL expose all webtest artifacts using a `webtest://` URI scheme with hierarchical paths, using markdown format for all human-readable artifacts.
|
|
8
|
-
|
|
9
|
-
#### Scenario: Analysis root resource is accessible
|
|
10
|
-
|
|
11
|
-
- **GIVEN** an analysis has been started with analysisId "abc123"
|
|
12
|
-
- **WHEN** client requests resource `webtest://abc123/`
|
|
13
|
-
- **THEN** it SHALL return the analysis `index.md` metadata as markdown with YAML frontmatter
|
|
14
|
-
|
|
15
|
-
#### Scenario: Crawl index resource is accessible
|
|
16
|
-
|
|
17
|
-
- **GIVEN** a crawl has completed with crawlId "crawl-001"
|
|
18
|
-
- **WHEN** client requests resource `webtest://abc123/crawls/crawl-001/index.md`
|
|
19
|
-
- **THEN** it SHALL return the crawl index as markdown with YAML frontmatter containing page list and metadata
|
|
20
|
-
|
|
21
|
-
#### Scenario: Page artifacts are accessible by type
|
|
22
|
-
|
|
23
|
-
- **GIVEN** a page was captured with pageId "page-001"
|
|
24
|
-
- **WHEN** client requests `webtest://abc123/crawls/crawl-001/pages/page-001/screenshot.png`
|
|
25
|
-
- **THEN** it SHALL return the screenshot image
|
|
26
|
-
- **AND** `snapshot.md` returns accessibility tree as formatted markdown with YAML frontmatter
|
|
27
|
-
- **AND** `dom.html` returns HTML content
|
|
28
|
-
|
|
29
|
-
#### Scenario: Crawl checkpoint is accessible
|
|
30
|
-
|
|
31
|
-
- **GIVEN** a crawl is in progress with checkpoint saved
|
|
32
|
-
- **WHEN** client requests `webtest://abc123/crawls/crawl-001/checkpoint.md`
|
|
33
|
-
- **THEN** it SHALL return the checkpoint as markdown with YAML frontmatter containing crawl state
|
|
34
|
-
|
|
35
|
-
#### Scenario: Analysis report is accessible
|
|
36
|
-
|
|
37
|
-
- **GIVEN** analyze_app has completed
|
|
38
|
-
- **WHEN** client requests `webtest://abc123/analysis/app-analysis.md`
|
|
39
|
-
- **THEN** it SHALL return the markdown analysis report
|
|
40
|
-
|
|
41
|
-
#### Scenario: Flows are accessible
|
|
42
|
-
|
|
43
|
-
- **GIVEN** analyze_app has completed
|
|
44
|
-
- **WHEN** client requests `webtest://abc123/analysis/flows.md`
|
|
45
|
-
- **THEN** it SHALL return user flows as markdown with YAML frontmatter containing structured flow definitions
|
|
46
|
-
|
|
47
|
-
#### Scenario: Tests are accessible
|
|
48
|
-
|
|
49
|
-
- **GIVEN** generate_tests has completed
|
|
50
|
-
- **WHEN** client requests `webtest://abc123/tests/tests.md`
|
|
51
|
-
- **THEN** it SHALL return the test cases as markdown with YAML frontmatter containing structured test definitions
|
|
52
|
-
|
|
53
|
-
#### Scenario: Test run report is accessible
|
|
54
|
-
|
|
55
|
-
- **GIVEN** a test run has completed with runId "run-001"
|
|
56
|
-
- **WHEN** client requests `webtest://abc123/runs/run-001/report.md`
|
|
57
|
-
- **THEN** it SHALL return the test execution report as markdown with YAML frontmatter containing structured results
|
|
58
|
-
|
|
59
|
-
#### Scenario: Test step snapshot is accessible
|
|
60
|
-
|
|
61
|
-
- **GIVEN** a test step has captured evidence
|
|
62
|
-
- **WHEN** client requests `webtest://abc123/runs/run-001/steps/1/snapshot.md`
|
|
63
|
-
- **THEN** it SHALL return the accessibility snapshot as formatted markdown with YAML frontmatter
|
|
64
|
-
|
|
65
|
-
### Requirement: Resource Template Registration
|
|
66
|
-
|
|
67
|
-
The system SHALL register resource templates with the MCP server for discovery, using markdown extensions for all index and report resources.
|
|
68
|
-
|
|
69
|
-
#### Scenario: Templates are listed on resources/list
|
|
70
|
-
|
|
71
|
-
- **GIVEN** a client calls `resources/list`
|
|
72
|
-
- **WHEN** the response is returned
|
|
73
|
-
- **THEN** it SHALL include templates for:
|
|
74
|
-
- `webtest://{analysisId}/index.md` (Analysis index)
|
|
75
|
-
- `webtest://{analysisId}/crawls/{crawlId}/index.md` (Crawl index)
|
|
76
|
-
- `webtest://{analysisId}/crawls/{crawlId}/checkpoint.md` (Crawl checkpoint)
|
|
77
|
-
- `webtest://{analysisId}/crawls/{crawlId}/pages/{pageId}/snapshot.md` (Page snapshot)
|
|
78
|
-
- `webtest://{analysisId}/crawls/{crawlId}/pages/{pageId}/screenshot.png` (Page screenshot)
|
|
79
|
-
- `webtest://{analysisId}/crawls/{crawlId}/pages/{pageId}/dom.html` (Page DOM)
|
|
80
|
-
- `webtest://{analysisId}/analysis/app-analysis.md` (Analysis report)
|
|
81
|
-
- `webtest://{analysisId}/analysis/flows.md` (User flows)
|
|
82
|
-
- `webtest://{analysisId}/tests/tests.md` (Test definitions)
|
|
83
|
-
- `webtest://{analysisId}/runs/{runId}/report.md` (Test run report)
|
|
84
|
-
- `webtest://{analysisId}/runs/{runId}/steps/{stepNumber}/snapshot.md` (Step snapshot)
|
|
85
|
-
- `webtest://{analysisId}/runs/{runId}/steps/{stepNumber}/screenshot.png` (Step screenshot)
|
|
86
|
-
|
|
87
|
-
### Requirement: Resource Content Types
|
|
88
|
-
|
|
89
|
-
The system SHALL return appropriate MIME types for different artifact types.
|
|
90
|
-
|
|
91
|
-
#### Scenario: Markdown resources have correct type
|
|
92
|
-
|
|
93
|
-
- **GIVEN** client reads a `.md` resource
|
|
94
|
-
- **WHEN** response is returned
|
|
95
|
-
- **THEN** mimeType SHALL be `text/markdown`
|
|
96
|
-
|
|
97
|
-
#### Scenario: Screenshot resources have correct type
|
|
98
|
-
|
|
99
|
-
- **GIVEN** client reads a `.png` resource
|
|
100
|
-
- **WHEN** response is returned
|
|
101
|
-
- **THEN** mimeType SHALL be `image/png`
|
|
102
|
-
- **AND** content SHALL be base64 encoded
|
|
103
|
-
|
|
104
|
-
#### Scenario: HTML resources have correct type
|
|
105
|
-
|
|
106
|
-
- **GIVEN** client reads a `.html` resource
|
|
107
|
-
- **WHEN** response is returned
|
|
108
|
-
- **THEN** mimeType SHALL be `text/html`
|
|
109
|
-
|
package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/specs/webtest-tools/spec.md
DELETED
|
@@ -1,121 +0,0 @@
|
|
|
1
|
-
# webtest-tools Spec Delta
|
|
2
|
-
|
|
3
|
-
## MODIFIED Requirements
|
|
4
|
-
|
|
5
|
-
### Requirement: webtest_init Tool
|
|
6
|
-
|
|
7
|
-
The system SHALL provide a `webtest_init` tool that initializes an analysis workspace for a target URL and focus, storing metadata in markdown format.
|
|
8
|
-
|
|
9
|
-
#### Scenario: Start analysis with valid URL
|
|
10
|
-
|
|
11
|
-
- **GIVEN** the tool is called with a valid URL and focus
|
|
12
|
-
- **WHEN** execution completes
|
|
13
|
-
- **THEN** it SHALL generate a unique `analysisId`
|
|
14
|
-
- **AND** create workspace directories
|
|
15
|
-
- **AND** write initial `index.md` metadata with YAML frontmatter
|
|
16
|
-
- **AND** return `{ analysisId, workspaceRootPath, workspaceRootUri, statusUri }` where statusUri points to `index.md`
|
|
17
|
-
|
|
18
|
-
### Requirement: webtest_crawl_app Tool
|
|
19
|
-
|
|
20
|
-
The system SHALL provide a `webtest_crawl_app` tool that dynamically explores a web application, storing all crawl artifacts in markdown format.
|
|
21
|
-
|
|
22
|
-
#### Scenario: Crawl captures artifacts at each checkpoint
|
|
23
|
-
|
|
24
|
-
- **GIVEN** a crawl iteration completes an action
|
|
25
|
-
- **WHEN** state is captured
|
|
26
|
-
- **THEN** it SHALL call Playwright MCP `browser_snapshot` for accessibility tree
|
|
27
|
-
- **AND** call `browser_take_screenshot` for visual evidence
|
|
28
|
-
- **AND** optionally extract HTML DOM
|
|
29
|
-
- **AND** store snapshot as `snapshot.md` with formatted accessibility tree and YAML frontmatter
|
|
30
|
-
- **AND** store screenshot as PNG
|
|
31
|
-
- **AND** store DOM as HTML
|
|
32
|
-
|
|
33
|
-
#### Scenario: Crawl outputs complete results
|
|
34
|
-
|
|
35
|
-
- **GIVEN** crawl has finalized
|
|
36
|
-
- **WHEN** output is returned
|
|
37
|
-
- **THEN** it SHALL include `crawlId`, `crawlIndexFilePath`, `crawlIndexUri` (pointing to `index.md`), `pages[]`, `summaryUri`
|
|
38
|
-
|
|
39
|
-
### Requirement: Crawl Checkpointing
|
|
40
|
-
|
|
41
|
-
The system SHALL implement checkpointing during crawl using markdown format to enable resumption and provide human-readable partial results on failure.
|
|
42
|
-
|
|
43
|
-
#### Scenario: Checkpoint is written periodically
|
|
44
|
-
|
|
45
|
-
- **GIVEN** a crawl is in progress
|
|
46
|
-
- **WHEN** N steps have completed (configurable, default 5)
|
|
47
|
-
- **THEN** it SHALL write a checkpoint to `webtest://{analysisId}/crawls/{crawlId}/checkpoint.md`
|
|
48
|
-
- **AND** the checkpoint SHALL be markdown with YAML frontmatter including: current step, visited pages, action history, goal progress
|
|
49
|
-
- **AND** the checkpoint body SHALL contain human-readable progress summary
|
|
50
|
-
|
|
51
|
-
#### Scenario: Crawl can resume from checkpoint
|
|
52
|
-
|
|
53
|
-
- **GIVEN** a crawl was interrupted (cancelled, error, timeout)
|
|
54
|
-
- **AND** a checkpoint exists as `checkpoint.md`
|
|
55
|
-
- **WHEN** `webtest_crawl_app` is called with `resume: true`
|
|
56
|
-
- **THEN** it SHALL parse the checkpoint YAML frontmatter
|
|
57
|
-
- **AND** continue from the last recorded state
|
|
58
|
-
|
|
59
|
-
### Requirement: webtest_analyze_app Tool
|
|
60
|
-
|
|
61
|
-
The system SHALL provide a `webtest_analyze_app` tool that reverse-engineers application structure from crawl data, outputting all results in markdown format.
|
|
62
|
-
|
|
63
|
-
#### Scenario: Analyze app writes markdown report
|
|
64
|
-
|
|
65
|
-
- **GIVEN** analysis is complete
|
|
66
|
-
- **WHEN** output is generated
|
|
67
|
-
- **THEN** it SHALL write `app-analysis.md` resource to workspace
|
|
68
|
-
|
|
69
|
-
#### Scenario: Analyze app writes flows
|
|
70
|
-
|
|
71
|
-
- **GIVEN** analysis is complete
|
|
72
|
-
- **WHEN** output is generated
|
|
73
|
-
- **THEN** it SHALL write `flows.md` resource to workspace as markdown with YAML frontmatter containing structured flow definitions
|
|
74
|
-
|
|
75
|
-
#### Scenario: Analyze app outputs URIs
|
|
76
|
-
|
|
77
|
-
- **GIVEN** analysis is complete
|
|
78
|
-
- **WHEN** tool returns
|
|
79
|
-
- **THEN** it SHALL include `appAnalysisFilePath`, `appAnalysisUri`, `flowsFilePath`, `flowsUri` all pointing to `.md` files
|
|
80
|
-
|
|
81
|
-
### Requirement: webtest_generate_tests Tool
|
|
82
|
-
|
|
83
|
-
The system SHALL provide a `webtest_generate_tests` tool that produces test cases from application analysis in a single markdown format.
|
|
84
|
-
|
|
85
|
-
#### Scenario: Generate tests outputs structured format
|
|
86
|
-
|
|
87
|
-
- **GIVEN** test generation completes
|
|
88
|
-
- **WHEN** results are written
|
|
89
|
-
- **THEN** it SHALL produce `tests.md` with human-readable format AND YAML frontmatter containing structured test definitions
|
|
90
|
-
- **AND** there SHALL NOT be a separate `tests.json` file
|
|
91
|
-
|
|
92
|
-
#### Scenario: Generate tests outputs URIs
|
|
93
|
-
|
|
94
|
-
- **GIVEN** generation is complete
|
|
95
|
-
- **WHEN** tool returns
|
|
96
|
-
- **THEN** it SHALL include `testsFilePath` and `testsUri` pointing to `tests.md`
|
|
97
|
-
|
|
98
|
-
### Requirement: webtest_run_tests Tool
|
|
99
|
-
|
|
100
|
-
The system SHALL provide a `webtest_run_tests` tool that executes a test case with evidence capture, storing all results in markdown format.
|
|
101
|
-
|
|
102
|
-
#### Scenario: Run test case captures evidence
|
|
103
|
-
|
|
104
|
-
- **GIVEN** a step is executed
|
|
105
|
-
- **WHEN** evidence is captured
|
|
106
|
-
- **THEN** it SHALL take screenshot after action (stored as PNG)
|
|
107
|
-
- **AND** capture accessibility snapshot (stored as `snapshot.md` with formatted tree and YAML frontmatter)
|
|
108
|
-
- **AND** store with step identifier
|
|
109
|
-
|
|
110
|
-
#### Scenario: Run test case outputs report
|
|
111
|
-
|
|
112
|
-
- **GIVEN** test execution completes
|
|
113
|
-
- **WHEN** output is generated
|
|
114
|
-
- **THEN** it SHALL write `report.md` with pass/fail summary, step details, evidence links, and YAML frontmatter containing structured run data
|
|
115
|
-
- **AND** there SHALL NOT be a separate `index.json` or `artifacts.json` file
|
|
116
|
-
|
|
117
|
-
#### Scenario: Run test case returns URIs
|
|
118
|
-
|
|
119
|
-
- **GIVEN** execution is complete
|
|
120
|
-
- **WHEN** tool returns
|
|
121
|
-
- **THEN** it SHALL include `testRunId`, `reportFilePath`, `reportUri` pointing to `report.md`
|
|
@@ -1,133 +0,0 @@
|
|
|
1
|
-
# Tasks: Use Markdown for All Artifacts
|
|
2
|
-
|
|
3
|
-
## Prerequisites
|
|
4
|
-
|
|
5
|
-
- [x] **Add gray-matter dependency** - Install `gray-matter` package for YAML frontmatter parsing
|
|
6
|
-
- `npm install gray-matter`
|
|
7
|
-
- Add to package.json dependencies
|
|
8
|
-
|
|
9
|
-
## Phase 1: Core Utilities
|
|
10
|
-
|
|
11
|
-
- [x] **Create markdown utilities module** - `src/workspace/markdown.ts`
|
|
12
|
-
- Implement `writeMarkdownWithFrontmatter(path, frontmatter, content)`
|
|
13
|
-
- Implement `readMarkdownFrontmatter<T>(path)` returning parsed YAML data
|
|
14
|
-
- Implement `formatAccessibilityTree(snapshot)` for human-readable tree output
|
|
15
|
-
- Add unit tests for utilities
|
|
16
|
-
|
|
17
|
-
## Phase 2: Workspace Index
|
|
18
|
-
|
|
19
|
-
- [x] **Update workspace index to markdown** - `src/workspace/index.ts`
|
|
20
|
-
- Change `createWorkspace` to write `index.md` instead of `index.json`
|
|
21
|
-
- Update `readWorkspaceIndex` to parse from markdown frontmatter
|
|
22
|
-
- Update `updateWorkspaceIndex` to preserve markdown body while updating frontmatter
|
|
23
|
-
- Generate human-readable workspace summary in markdown body
|
|
24
|
-
|
|
25
|
-
- [x] **Update workspace types** - `src/workspace/types.ts`
|
|
26
|
-
- Update `WorkspaceIndex` type if needed for markdown-specific fields
|
|
27
|
-
|
|
28
|
-
## Phase 3: Crawl Artifacts
|
|
29
|
-
|
|
30
|
-
- [x] **Update crawl index to markdown** - `src/workspace/index.ts`
|
|
31
|
-
- Change `createCrawl` to write `index.md`
|
|
32
|
-
- Update `readCrawlIndex` to parse from markdown frontmatter
|
|
33
|
-
- Update `updateCrawlIndex` to regenerate markdown body
|
|
34
|
-
- Include formatted action history and budget status in body
|
|
35
|
-
|
|
36
|
-
- [x] **Update checkpoint to markdown** - `src/workspace/index.ts`
|
|
37
|
-
- Change `saveCheckpoint` to write `checkpoint.md`
|
|
38
|
-
- Update `loadCheckpoint` to parse from markdown frontmatter
|
|
39
|
-
- Include human-readable progress summary in body
|
|
40
|
-
|
|
41
|
-
- [x] **Update page snapshot to markdown** - `src/workspace/index.ts`
|
|
42
|
-
- Change `savePage` to write `snapshot.md` instead of `snapshot.json`
|
|
43
|
-
- Format accessibility tree as indented text tree in body
|
|
44
|
-
- Keep full structured snapshot in YAML frontmatter
|
|
45
|
-
- Update return paths/URIs to `.md` extension
|
|
46
|
-
|
|
47
|
-
## Phase 4: Analysis Artifacts
|
|
48
|
-
|
|
49
|
-
- [x] **Update flows to markdown** - `src/tools/webtest/analyze-app.ts`
|
|
50
|
-
- Change `saveAnalysis` to write `flows.md` instead of `flows.json`
|
|
51
|
-
- Format flows as readable markdown sections
|
|
52
|
-
- Include structured flow data in YAML frontmatter
|
|
53
|
-
- Update return paths/URIs
|
|
54
|
-
|
|
55
|
-
- [x] **Update analyze-app tool outputs** - `src/tools/webtest/analyze-app.ts`
|
|
56
|
-
- Update `flowsFilePath` and `flowsUri` to point to `flows.md`
|
|
57
|
-
|
|
58
|
-
## Phase 5: Test Artifacts
|
|
59
|
-
|
|
60
|
-
- [x] **Consolidate test definitions** - `src/tools/webtest/generate-tests.ts`
|
|
61
|
-
- Remove separate `tests.json` generation
|
|
62
|
-
- Update `saveTests` to write only `tests.md` with YAML frontmatter
|
|
63
|
-
- Include full test case structures in frontmatter
|
|
64
|
-
- Keep human-readable test documentation in body
|
|
65
|
-
- Update return paths/URIs (remove `testsFilePath` for JSON)
|
|
66
|
-
|
|
67
|
-
- [x] **Update generate-tests tool outputs** - `src/tools/webtest/generate-tests.ts`
|
|
68
|
-
- Return single `testsUri` pointing to `tests.md`
|
|
69
|
-
- Update `testsFilePath` to point to `tests.md`
|
|
70
|
-
|
|
71
|
-
## Phase 6: Test Run Artifacts
|
|
72
|
-
|
|
73
|
-
- [x] **Update test run index to report** - `src/workspace/index.ts`
|
|
74
|
-
- Change `createTestRun` to write `report.md` instead of `index.json`
|
|
75
|
-
- Update `readTestRunIndex` to parse from markdown frontmatter
|
|
76
|
-
- Update `updateTestRunIndex` to regenerate markdown body
|
|
77
|
-
|
|
78
|
-
- [x] **Update test step snapshots** - `src/workspace/index.ts`
|
|
79
|
-
- Change `saveTestStepEvidence` to write `snapshot.md`
|
|
80
|
-
- Format accessibility tree in body
|
|
81
|
-
- Update return paths/URIs
|
|
82
|
-
|
|
83
|
-
- [x] **Update run-test-case tool outputs** - `src/tools/webtest/run-test-case.ts`
|
|
84
|
-
- Update `reportUri` to point to `report.md`
|
|
85
|
-
- Update step evidence URIs to point to `snapshot.md`
|
|
86
|
-
|
|
87
|
-
## Phase 7: Resource Manager
|
|
88
|
-
|
|
89
|
-
- [x] **Update resource URI mappings** - `src/resources/index.ts`
|
|
90
|
-
- Update all `.json` paths to `.md` in URI templates
|
|
91
|
-
- Handle markdown MIME type for new paths
|
|
92
|
-
- Update resource listing to discover `.md` files
|
|
93
|
-
|
|
94
|
-
- [x] **Update resource reading** - `src/resources/index.ts`
|
|
95
|
-
- Return raw markdown content for `.md` resources
|
|
96
|
-
- Update MIME type to `text/markdown`
|
|
97
|
-
|
|
98
|
-
## Phase 8: Integration & Testing
|
|
99
|
-
|
|
100
|
-
- [x] **Update existing tests** - Test files in `src/` or `tests/`
|
|
101
|
-
- Update file path expectations from `.json` to `.md`
|
|
102
|
-
- Update content parsing to use frontmatter extraction
|
|
103
|
-
- Add tests for markdown format validity
|
|
104
|
-
|
|
105
|
-
- [x] **Manual integration test**
|
|
106
|
-
- Run full workflow: init → crawl → analyze → generate tests → run tests
|
|
107
|
-
- Verify all workspace files are `.md` (except screenshots and DOM)
|
|
108
|
-
- Verify files are readable without tooling
|
|
109
|
-
- Verify programmatic access via frontmatter works
|
|
110
|
-
|
|
111
|
-
## Phase 9: Documentation
|
|
112
|
-
|
|
113
|
-
- [x] **Update CLAUDE.md or README** (if exists)
|
|
114
|
-
- Document new artifact format
|
|
115
|
-
- Provide examples of frontmatter structure
|
|
116
|
-
- Note breaking change for existing workspaces
|
|
117
|
-
|
|
118
|
-
## Verification Checklist
|
|
119
|
-
|
|
120
|
-
After implementation, verify:
|
|
121
|
-
|
|
122
|
-
- [x] `index.md` at workspace root with valid frontmatter
|
|
123
|
-
- [x] `crawls/{id}/index.md` with action history
|
|
124
|
-
- [x] `crawls/{id}/checkpoint.md` with progress state
|
|
125
|
-
- [x] `crawls/{id}/pages/{id}/snapshot.md` with formatted tree
|
|
126
|
-
- [x] `analysis/app-analysis.md` (unchanged)
|
|
127
|
-
- [x] `analysis/flows.md` with flow definitions
|
|
128
|
-
- [x] `tests/tests.md` with all test data (no tests.json)
|
|
129
|
-
- [x] `runs/{id}/report.md` with execution results
|
|
130
|
-
- [x] `runs/{id}/steps/{n}/snapshot.md` with formatted tree
|
|
131
|
-
- [x] All tool outputs reference `.md` paths
|
|
132
|
-
- [x] All resource URIs use `.md` extensions
|
|
133
|
-
- [x] Workspace is browsable and human-readable
|
|
@@ -1,86 +0,0 @@
|
|
|
1
|
-
## Context
|
|
2
|
-
|
|
3
|
-
Prompts are currently embedded as template literal strings in TypeScript files. This change extracts them to markdown files for better maintainability while preserving runtime variable interpolation.
|
|
4
|
-
|
|
5
|
-
## Goals / Non-Goals
|
|
6
|
-
|
|
7
|
-
**Goals:**
|
|
8
|
-
- Make prompts easy to read, edit, and review as standalone files
|
|
9
|
-
- Preserve `${varName}` syntax for variable substitution
|
|
10
|
-
- Maintain type safety for required variables
|
|
11
|
-
- Keep runtime performance acceptable (load once, cache)
|
|
12
|
-
|
|
13
|
-
**Non-Goals:**
|
|
14
|
-
- Complex templating features (loops, conditionals) - keep it simple
|
|
15
|
-
- Hot-reloading of templates in production
|
|
16
|
-
- User-configurable prompts at runtime
|
|
17
|
-
|
|
18
|
-
## Decisions
|
|
19
|
-
|
|
20
|
-
### Decision: Use simple `${varName}` interpolation
|
|
21
|
-
|
|
22
|
-
Keep the existing JavaScript template literal syntax. The loader will:
|
|
23
|
-
1. Read the markdown file content
|
|
24
|
-
2. Replace `${varName}` patterns with provided values
|
|
25
|
-
3. Return the interpolated string
|
|
26
|
-
|
|
27
|
-
**Alternatives considered:**
|
|
28
|
-
- Mustache/Handlebars: Adds dependency, overkill for simple substitution
|
|
29
|
-
- Tagged template literals: Would require keeping prompts in JS files
|
|
30
|
-
- JSON with placeholders: Less readable for long-form content
|
|
31
|
-
|
|
32
|
-
### Decision: Directory structure
|
|
33
|
-
|
|
34
|
-
```
|
|
35
|
-
src/prompts/
|
|
36
|
-
├── templates/
|
|
37
|
-
│ ├── mcp/ # User-facing MCP prompts
|
|
38
|
-
│ │ ├── webtest-start.md
|
|
39
|
-
│ │ ├── webtest-crawl.md
|
|
40
|
-
│ │ └── ...
|
|
41
|
-
│ └── sampling/ # Internal AI sampling prompts
|
|
42
|
-
│ ├── system-prefix.md
|
|
43
|
-
│ ├── crawl-action.md
|
|
44
|
-
│ └── ...
|
|
45
|
-
├── loader.ts # Template loading utility
|
|
46
|
-
├── index.ts # MCP prompt definitions (uses loader)
|
|
47
|
-
└── index.test.ts # Tests
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
**Rationale:** Separating `mcp/` and `sampling/` makes it clear which prompts are user-facing vs internal.
|
|
51
|
-
|
|
52
|
-
### Decision: Synchronous file loading with caching
|
|
53
|
-
|
|
54
|
-
Load templates synchronously at module initialization and cache them. This:
|
|
55
|
-
- Avoids async complexity in prompt builders
|
|
56
|
-
- Ensures templates are available immediately
|
|
57
|
-
- Is acceptable since templates are static assets
|
|
58
|
-
|
|
59
|
-
### Decision: TypeScript interface for variables
|
|
60
|
-
|
|
61
|
-
Each prompt template will have a corresponding TypeScript type defining its variables:
|
|
62
|
-
|
|
63
|
-
```typescript
|
|
64
|
-
interface CrawlActionVars {
|
|
65
|
-
goal: string;
|
|
66
|
-
currentUrl: string;
|
|
67
|
-
pageSnapshot: string;
|
|
68
|
-
actionHistory: string;
|
|
69
|
-
allowedDomains: string;
|
|
70
|
-
// ... optional vars
|
|
71
|
-
}
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
This maintains type safety when calling the interpolation function.
|
|
75
|
-
|
|
76
|
-
## Risks / Trade-offs
|
|
77
|
-
|
|
78
|
-
| Risk | Mitigation |
|
|
79
|
-
|------|------------|
|
|
80
|
-
| File system access at startup | Templates are small, load is fast |
|
|
81
|
-
| Missing variable errors at runtime | Type definitions + tests catch this |
|
|
82
|
-
| Bundler complications | Ensure templates are included in build output |
|
|
83
|
-
|
|
84
|
-
## Open Questions
|
|
85
|
-
|
|
86
|
-
- None at this time
|
|
@@ -1,50 +0,0 @@
|
|
|
1
|
-
# Change: Extract prompts to separate markdown files
|
|
2
|
-
|
|
3
|
-
## Why
|
|
4
|
-
|
|
5
|
-
Currently, all prompt text is embedded inline within TypeScript files (`src/prompts/index.ts` and `src/sampling/prompts.ts`). This makes prompts difficult to maintain, review, and iterate on since:
|
|
6
|
-
- Prompt changes require modifying TypeScript code
|
|
7
|
-
- Large multi-line strings with template literals are hard to read and edit
|
|
8
|
-
- Non-developers cannot easily review or suggest prompt improvements
|
|
9
|
-
- No syntax highlighting or markdown preview for prompt content
|
|
10
|
-
|
|
11
|
-
## What Changes
|
|
12
|
-
|
|
13
|
-
- **Extract MCP prompts** from `src/prompts/index.ts` to separate `.md` files
|
|
14
|
-
- **Extract sampling prompts** from `src/sampling/prompts.ts` to separate `.md` files
|
|
15
|
-
- Store all prompt templates in `src/prompts/templates/`
|
|
16
|
-
- Keep `${varName}` interpolation syntax for dynamic values
|
|
17
|
-
- Create a utility to load and interpolate markdown templates at runtime
|
|
18
|
-
- Update existing code to use the template loader
|
|
19
|
-
|
|
20
|
-
### Prompts to Extract
|
|
21
|
-
|
|
22
|
-
**MCP Prompts (user-facing):**
|
|
23
|
-
- `webtest-start.md`
|
|
24
|
-
- `webtest-crawl.md`
|
|
25
|
-
- `webtest-discover.md`
|
|
26
|
-
- `webtest-generate-tests.md`
|
|
27
|
-
- `webtest-run-test.md`
|
|
28
|
-
- `webtest-full-workflow.md`
|
|
29
|
-
|
|
30
|
-
**Sampling Prompts (internal AI prompts):**
|
|
31
|
-
- `system-prefix.md` - Security-hardened system prompt
|
|
32
|
-
- `page-content-wrapper.md` - Untrusted content delimiters
|
|
33
|
-
- `crawl-action.md` - Crawl navigation decisions
|
|
34
|
-
- `feature-discovery.md` - Discover app features/modules
|
|
35
|
-
- `flow-discovery.md` - Discover user flows within features
|
|
36
|
-
- `test-generation.md` - Generate test cases
|
|
37
|
-
- `test-evaluation.md` - Evaluate test step results
|
|
38
|
-
|
|
39
|
-
**Cleanup:**
|
|
40
|
-
- Remove deprecated `buildAnalysisPrompt` function
|
|
41
|
-
- Update `analyze-app.ts` to use `buildFeatureDiscoveryPrompt` instead
|
|
42
|
-
|
|
43
|
-
## Impact
|
|
44
|
-
|
|
45
|
-
- Affected specs: `webtest-prompts`
|
|
46
|
-
- Affected code:
|
|
47
|
-
- `src/prompts/index.ts` - Refactor to load from markdown
|
|
48
|
-
- `src/sampling/prompts.ts` - Refactor to load from markdown
|
|
49
|
-
- New: `src/prompts/templates/` directory with `.md` files
|
|
50
|
-
- New: `src/prompts/loader.ts` - Template loading utility
|
|
@@ -1,74 +0,0 @@
|
|
|
1
|
-
## ADDED Requirements
|
|
2
|
-
|
|
3
|
-
### Requirement: Prompt Template Files
|
|
4
|
-
|
|
5
|
-
The system SHALL store prompt text in separate markdown files under `src/prompts/templates/`.
|
|
6
|
-
|
|
7
|
-
#### Scenario: MCP prompts stored in templates/mcp/
|
|
8
|
-
|
|
9
|
-
- **GIVEN** a user-facing MCP prompt (e.g., `webtest-start`)
|
|
10
|
-
- **WHEN** the prompt content is needed
|
|
11
|
-
- **THEN** it SHALL be loaded from `src/prompts/templates/mcp/<prompt-name>.md`
|
|
12
|
-
|
|
13
|
-
#### Scenario: Sampling prompts stored in templates/sampling/
|
|
14
|
-
|
|
15
|
-
- **GIVEN** an internal sampling prompt (e.g., `crawl-action`)
|
|
16
|
-
- **WHEN** the prompt content is needed
|
|
17
|
-
- **THEN** it SHALL be loaded from `src/prompts/templates/sampling/<prompt-name>.md`
|
|
18
|
-
|
|
19
|
-
### Requirement: Template Variable Interpolation
|
|
20
|
-
|
|
21
|
-
The system SHALL support `${varName}` syntax for variable substitution in markdown templates.
|
|
22
|
-
|
|
23
|
-
#### Scenario: Simple variable substitution
|
|
24
|
-
|
|
25
|
-
- **GIVEN** a template containing `${url}` placeholder
|
|
26
|
-
- **WHEN** the template is rendered with `{ url: "https://example.com" }`
|
|
27
|
-
- **THEN** the output SHALL contain `https://example.com` in place of `${url}`
|
|
28
|
-
|
|
29
|
-
#### Scenario: Multiple variables in template
|
|
30
|
-
|
|
31
|
-
- **GIVEN** a template containing `${goal}`, `${currentUrl}`, and `${allowedDomains}`
|
|
32
|
-
- **WHEN** the template is rendered with all variables provided
|
|
33
|
-
- **THEN** all placeholders SHALL be replaced with their corresponding values
|
|
34
|
-
|
|
35
|
-
#### Scenario: Preserve unmatched placeholders
|
|
36
|
-
|
|
37
|
-
- **GIVEN** a template containing `${undefinedVar}`
|
|
38
|
-
- **WHEN** the template is rendered without that variable
|
|
39
|
-
- **THEN** the placeholder `${undefinedVar}` SHALL remain in the output unchanged
|
|
40
|
-
- **AND** no error SHALL be thrown
|
|
41
|
-
|
|
42
|
-
### Requirement: Template Loading and Caching
|
|
43
|
-
|
|
44
|
-
The system SHALL load templates efficiently with appropriate caching.
|
|
45
|
-
|
|
46
|
-
#### Scenario: Templates loaded at initialization
|
|
47
|
-
|
|
48
|
-
- **GIVEN** the server starts
|
|
49
|
-
- **WHEN** templates are first accessed
|
|
50
|
-
- **THEN** they SHALL be loaded from the filesystem
|
|
51
|
-
- **AND** cached for subsequent access
|
|
52
|
-
|
|
53
|
-
#### Scenario: Cached templates returned on repeated access
|
|
54
|
-
|
|
55
|
-
- **GIVEN** a template has been loaded previously
|
|
56
|
-
- **WHEN** the same template is requested again
|
|
57
|
-
- **THEN** the cached version SHALL be returned without re-reading the file
|
|
58
|
-
|
|
59
|
-
### Requirement: Template File Format
|
|
60
|
-
|
|
61
|
-
Markdown template files SHALL follow a consistent format for maintainability.
|
|
62
|
-
|
|
63
|
-
#### Scenario: Template uses raw markdown content
|
|
64
|
-
|
|
65
|
-
- **GIVEN** a prompt template file
|
|
66
|
-
- **WHEN** the file is read
|
|
67
|
-
- **THEN** the entire file content SHALL be used as the prompt text
|
|
68
|
-
- **AND** no special parsing or metadata extraction is required
|
|
69
|
-
|
|
70
|
-
#### Scenario: Template preserves whitespace and formatting
|
|
71
|
-
|
|
72
|
-
- **GIVEN** a template with specific indentation and line breaks
|
|
73
|
-
- **WHEN** the template is loaded and interpolated
|
|
74
|
-
- **THEN** all whitespace and formatting SHALL be preserved exactly
|
|
@@ -1,40 +0,0 @@
|
|
|
1
|
-
## 1. Template Infrastructure
|
|
2
|
-
|
|
3
|
-
- [x] 1.1 Create `src/prompts/templates/` directory structure
|
|
4
|
-
- [x] 1.2 Implement `src/prompts/loader.ts` with template loading and interpolation
|
|
5
|
-
- [x] 1.3 Add tests for template loader (variable substitution, missing vars, file not found)
|
|
6
|
-
|
|
7
|
-
## 2. Extract Sampling Prompts
|
|
8
|
-
|
|
9
|
-
- [x] 2.1 Create `src/prompts/templates/sampling/system-prefix.md`
|
|
10
|
-
- [x] 2.2 Create `src/prompts/templates/sampling/page-content-wrapper.md`
|
|
11
|
-
- [x] 2.3 Create `src/prompts/templates/sampling/crawl-action.md`
|
|
12
|
-
- [x] 2.4 Create `src/prompts/templates/sampling/feature-discovery.md`
|
|
13
|
-
- [x] 2.5 Create `src/prompts/templates/sampling/flow-discovery.md`
|
|
14
|
-
- [x] 2.6 Create `src/prompts/templates/sampling/test-generation.md`
|
|
15
|
-
- [x] 2.7 Create `src/prompts/templates/sampling/test-evaluation.md`
|
|
16
|
-
- [x] 2.8 Update `src/sampling/prompts.ts` to use template loader
|
|
17
|
-
- [x] 2.9 Verify existing sampling tests pass
|
|
18
|
-
|
|
19
|
-
## 3. Remove Deprecated Code
|
|
20
|
-
|
|
21
|
-
- [x] 3.1 Remove `buildAnalysisPrompt` from `src/sampling/prompts.ts` (already removed)
|
|
22
|
-
- [x] 3.2 Update `src/tools/webtest/analyze-app.ts` to use `buildFeatureDiscoveryPrompt` (already done)
|
|
23
|
-
- [x] 3.3 Update or remove related tests in `analyze-app.test.ts` (already done)
|
|
24
|
-
|
|
25
|
-
## 4. Extract MCP Prompts
|
|
26
|
-
|
|
27
|
-
- [x] 4.1 Create `src/prompts/templates/mcp/webtest-start.md`
|
|
28
|
-
- [x] 4.2 Create `src/prompts/templates/mcp/webtest-crawl.md`
|
|
29
|
-
- [x] 4.3 Create `src/prompts/templates/mcp/webtest-discover.md`
|
|
30
|
-
- [x] 4.4 Create `src/prompts/templates/mcp/webtest-discover-flows.md`
|
|
31
|
-
- [x] 4.5 Create `src/prompts/templates/mcp/webtest-generate-tests.md`
|
|
32
|
-
- [x] 4.6 Create `src/prompts/templates/mcp/webtest-run-test.md`
|
|
33
|
-
- [x] 4.7 Create `src/prompts/templates/mcp/webtest-full-workflow.md`
|
|
34
|
-
- [x] 4.8 Update `src/prompts/index.ts` to use template loader
|
|
35
|
-
- [x] 4.9 Verify existing MCP prompt tests pass
|
|
36
|
-
|
|
37
|
-
## 5. Validation
|
|
38
|
-
|
|
39
|
-
- [x] 5.1 Run full test suite (`npm test`) - 448 tests pass
|
|
40
|
-
- [x] 5.2 Run build (`npm run build`) - success
|