retestkit 1.4.1 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (238) hide show
  1. package/README.md +59 -40
  2. package/dist/config.js +8 -8
  3. package/dist/config.js.map +1 -1
  4. package/dist/logger.js +1 -1
  5. package/dist/logger.js.map +1 -1
  6. package/dist/prompts/index.d.ts +1 -1
  7. package/dist/prompts/index.d.ts.map +1 -1
  8. package/dist/prompts/index.js +21 -21
  9. package/dist/prompts/index.js.map +1 -1
  10. package/dist/prompts/templates/mcp/retest-crawl.md +7 -0
  11. package/{src/prompts/templates/mcp/webtest-discover-flows.md → dist/prompts/templates/mcp/retest-discover-flows.md} +1 -1
  12. package/{src/prompts/templates/mcp/webtest-discover.md → dist/prompts/templates/mcp/retest-discover.md} +2 -2
  13. package/dist/prompts/templates/mcp/retest-full-workflow.md +12 -0
  14. package/{src/prompts/templates/mcp/webtest-generate-tests.md → dist/prompts/templates/mcp/retest-generate-tests.md} +1 -1
  15. package/{src/prompts/templates/mcp/webtest-run-test.md → dist/prompts/templates/mcp/retest-run-test.md} +1 -1
  16. package/{src/prompts/templates/mcp/webtest-start.md → dist/prompts/templates/mcp/retest-start.md} +1 -1
  17. package/{src → dist}/prompts/templates/sampling/system-prefix.md +1 -1
  18. package/dist/resources/index.js +7 -7
  19. package/dist/resources/index.js.map +1 -1
  20. package/dist/schemas/config.js +2 -2
  21. package/dist/schemas/config.js.map +1 -1
  22. package/dist/security/index.js +1 -1
  23. package/dist/security/index.js.map +1 -1
  24. package/dist/server.js +3 -3
  25. package/dist/server.js.map +1 -1
  26. package/dist/test-utils/mock-context.js +22 -22
  27. package/dist/test-utils/mock-context.js.map +1 -1
  28. package/dist/tools/index.d.ts +1 -1
  29. package/dist/tools/index.d.ts.map +1 -1
  30. package/dist/tools/index.js +5 -5
  31. package/dist/tools/index.js.map +1 -1
  32. package/dist/tools/retest/crawl.d.ts.map +1 -0
  33. package/dist/tools/{webtest → retest}/crawl.js +7 -7
  34. package/dist/tools/retest/crawl.js.map +1 -0
  35. package/dist/tools/retest/discover-features.d.ts.map +1 -0
  36. package/dist/tools/{webtest → retest}/discover-features.js +6 -6
  37. package/dist/tools/retest/discover-features.js.map +1 -0
  38. package/dist/tools/retest/discover-flows.d.ts.map +1 -0
  39. package/dist/tools/{webtest → retest}/discover-flows.js +6 -6
  40. package/dist/tools/retest/discover-flows.js.map +1 -0
  41. package/dist/tools/retest/generate-tests.d.ts.map +1 -0
  42. package/dist/tools/{webtest → retest}/generate-tests.js +5 -5
  43. package/dist/tools/retest/generate-tests.js.map +1 -0
  44. package/dist/tools/retest/index.d.ts.map +1 -0
  45. package/dist/tools/retest/index.js.map +1 -0
  46. package/dist/tools/retest/run-test-case.d.ts.map +1 -0
  47. package/dist/tools/{webtest → retest}/run-test-case.js +3 -3
  48. package/dist/tools/retest/run-test-case.js.map +1 -0
  49. package/dist/tools/retest/schemas.d.ts.map +1 -0
  50. package/dist/tools/retest/schemas.js.map +1 -0
  51. package/dist/tools/retest/start-analysis.d.ts.map +1 -0
  52. package/dist/tools/{webtest → retest}/start-analysis.js +5 -5
  53. package/dist/tools/retest/start-analysis.js.map +1 -0
  54. package/dist/workspace/index.js +8 -8
  55. package/dist/workspace/index.js.map +1 -1
  56. package/dist/workspace/types.d.ts +2 -2
  57. package/dist/workspace/types.d.ts.map +1 -1
  58. package/package.json +6 -2
  59. package/.claude/commands/openspec/apply.md +0 -23
  60. package/.claude/commands/openspec/archive.md +0 -27
  61. package/.claude/commands/openspec/proposal.md +0 -28
  62. package/.gemini/commands/openspec/apply.toml +0 -21
  63. package/.gemini/commands/openspec/archive.toml +0 -25
  64. package/.gemini/commands/openspec/proposal.toml +0 -26
  65. package/.github/prompts/openspec-apply.prompt.md +0 -22
  66. package/.github/prompts/openspec-archive.prompt.md +0 -26
  67. package/.github/prompts/openspec-proposal.prompt.md +0 -27
  68. package/.github/workflows/release.yml +0 -33
  69. package/.kilocode/workflows/openspec-apply.md +0 -17
  70. package/.kilocode/workflows/openspec-archive.md +0 -21
  71. package/.kilocode/workflows/openspec-proposal.md +0 -22
  72. package/.mcp.json +0 -23
  73. package/.opencode/command/openspec-apply.md +0 -25
  74. package/.opencode/command/openspec-archive.md +0 -28
  75. package/.opencode/command/openspec-proposal.md +0 -30
  76. package/.roo/commands/openspec-apply.md +0 -20
  77. package/.roo/commands/openspec-archive.md +0 -24
  78. package/.roo/commands/openspec-proposal.md +0 -25
  79. package/.vscode/mcp.json +0 -23
  80. package/AGENTS.md +0 -18
  81. package/CLAUDE.md +0 -18
  82. package/dist/tools/webtest/crawl.d.ts.map +0 -1
  83. package/dist/tools/webtest/crawl.js.map +0 -1
  84. package/dist/tools/webtest/discover-features.d.ts.map +0 -1
  85. package/dist/tools/webtest/discover-features.js.map +0 -1
  86. package/dist/tools/webtest/discover-flows.d.ts.map +0 -1
  87. package/dist/tools/webtest/discover-flows.js.map +0 -1
  88. package/dist/tools/webtest/generate-tests.d.ts.map +0 -1
  89. package/dist/tools/webtest/generate-tests.js.map +0 -1
  90. package/dist/tools/webtest/index.d.ts.map +0 -1
  91. package/dist/tools/webtest/index.js.map +0 -1
  92. package/dist/tools/webtest/run-test-case.d.ts.map +0 -1
  93. package/dist/tools/webtest/run-test-case.js.map +0 -1
  94. package/dist/tools/webtest/schemas.d.ts.map +0 -1
  95. package/dist/tools/webtest/schemas.js.map +0 -1
  96. package/dist/tools/webtest/start-analysis.d.ts.map +0 -1
  97. package/dist/tools/webtest/start-analysis.js.map +0 -1
  98. package/openspec/AGENTS.md +0 -456
  99. package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/proposal.md +0 -33
  100. package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/specs/webtest-resources/spec.md +0 -27
  101. package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/specs/webtest-tools/spec.md +0 -304
  102. package/openspec/changes/archive/2025-12-18-add-hybrid-artifact-paths/tasks.md +0 -43
  103. package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/design.md +0 -209
  104. package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/proposal.md +0 -41
  105. package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/specs/mcp-server-core/spec.md +0 -183
  106. package/openspec/changes/archive/2025-12-18-add-mcp-server-foundation/tasks.md +0 -112
  107. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/design.md +0 -333
  108. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/proposal.md +0 -66
  109. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/mcp-server-core/spec.md +0 -129
  110. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-lifecycle/spec.md +0 -138
  111. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-logging/spec.md +0 -211
  112. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-prompts/spec.md +0 -157
  113. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-resources/spec.md +0 -213
  114. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-sampling/spec.md +0 -257
  115. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/specs/webtest-tools/spec.md +0 -501
  116. package/openspec/changes/archive/2025-12-18-add-webtest-orchestrator/tasks.md +0 -264
  117. package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/proposal.md +0 -24
  118. package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/specs/webtest-tools/spec.md +0 -80
  119. package/openspec/changes/archive/2025-12-18-allow-analysis-of-incomplete-crawls/tasks.md +0 -8
  120. package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/design.md +0 -90
  121. package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/proposal.md +0 -28
  122. package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/specs/webtest-sampling/spec.md +0 -90
  123. package/openspec/changes/archive/2025-12-18-fix-crawl-loop-stability/tasks.md +0 -33
  124. package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/design.md +0 -558
  125. package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/proposal.md +0 -119
  126. package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/specs/webtest-resources/spec.md +0 -109
  127. package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/specs/webtest-tools/spec.md +0 -121
  128. package/openspec/changes/archive/2025-12-18-use-markdown-artifacts/tasks.md +0 -133
  129. package/openspec/changes/extract-prompts-to-markdown/design.md +0 -86
  130. package/openspec/changes/extract-prompts-to-markdown/proposal.md +0 -50
  131. package/openspec/changes/extract-prompts-to-markdown/specs/webtest-prompts/spec.md +0 -74
  132. package/openspec/changes/extract-prompts-to-markdown/tasks.md +0 -40
  133. package/openspec/changes/refactor-webtest-naming/design.md +0 -95
  134. package/openspec/changes/refactor-webtest-naming/proposal.md +0 -66
  135. package/openspec/changes/refactor-webtest-naming/specs/webtest-prompts/spec.md +0 -79
  136. package/openspec/changes/refactor-webtest-naming/specs/webtest-resources/spec.md +0 -80
  137. package/openspec/changes/refactor-webtest-naming/specs/webtest-sampling/spec.md +0 -122
  138. package/openspec/changes/refactor-webtest-naming/specs/webtest-tools/spec.md +0 -113
  139. package/openspec/changes/refactor-webtest-naming/tasks.md +0 -119
  140. package/openspec/changes/rename-package-to-retest/proposal.md +0 -52
  141. package/openspec/changes/rename-package-to-retest/specs/mcp-server-core/spec.md +0 -53
  142. package/openspec/changes/rename-package-to-retest/specs/retest-lifecycle/spec.md +0 -68
  143. package/openspec/changes/rename-package-to-retest/specs/retest-logging/spec.md +0 -35
  144. package/openspec/changes/rename-package-to-retest/specs/retest-prompts/spec.md +0 -159
  145. package/openspec/changes/rename-package-to-retest/specs/retest-resources/spec.md +0 -251
  146. package/openspec/changes/rename-package-to-retest/specs/retest-sampling/spec.md +0 -99
  147. package/openspec/changes/rename-package-to-retest/specs/retest-tools/spec.md +0 -295
  148. package/openspec/changes/rename-package-to-retest/tasks.md +0 -71
  149. package/openspec/project.md +0 -31
  150. package/openspec/specs/mcp-server-core/spec.md +0 -178
  151. package/openspec/specs/webtest-lifecycle/spec.md +0 -136
  152. package/openspec/specs/webtest-logging/spec.md +0 -209
  153. package/openspec/specs/webtest-prompts/spec.md +0 -155
  154. package/openspec/specs/webtest-resources/spec.md +0 -248
  155. package/openspec/specs/webtest-sampling/spec.md +0 -344
  156. package/openspec/specs/webtest-tools/spec.md +0 -282
  157. package/release.config.js +0 -9
  158. package/src/config.test.ts +0 -96
  159. package/src/config.ts +0 -32
  160. package/src/elicitation/index.test.ts +0 -399
  161. package/src/elicitation/index.ts +0 -171
  162. package/src/elicitation/types.ts +0 -68
  163. package/src/index.ts +0 -83
  164. package/src/lifecycle/index.test.ts +0 -260
  165. package/src/lifecycle/index.ts +0 -101
  166. package/src/logger.redaction.test.ts +0 -322
  167. package/src/logger.test.ts +0 -123
  168. package/src/logger.ts +0 -229
  169. package/src/playwright-client/index.ts +0 -392
  170. package/src/playwright-client/types.ts +0 -99
  171. package/src/progress/index.test.ts +0 -327
  172. package/src/progress/index.ts +0 -170
  173. package/src/progress/types.ts +0 -25
  174. package/src/prompts/index.test.ts +0 -451
  175. package/src/prompts/index.ts +0 -246
  176. package/src/prompts/loader.test.ts +0 -100
  177. package/src/prompts/loader.ts +0 -59
  178. package/src/prompts/templates/mcp/webtest-crawl.md +0 -7
  179. package/src/prompts/templates/mcp/webtest-full-workflow.md +0 -12
  180. package/src/resources/index.ts +0 -250
  181. package/src/resources/subscriptions.ts +0 -37
  182. package/src/sampling/index.test.ts +0 -414
  183. package/src/sampling/index.ts +0 -286
  184. package/src/sampling/prompts.ts +0 -194
  185. package/src/sampling/types.ts +0 -60
  186. package/src/schemas/config.ts +0 -39
  187. package/src/security/index.test.ts +0 -441
  188. package/src/security/index.ts +0 -361
  189. package/src/security/security-scenarios.test.ts +0 -468
  190. package/src/server.ts +0 -211
  191. package/src/test-utils/index.ts +0 -6
  192. package/src/test-utils/mock-context.ts +0 -426
  193. package/src/test-utils/mock-playwright-client.ts +0 -422
  194. package/src/tools/index.ts +0 -11
  195. package/src/tools/webtest/crawl.test.ts +0 -834
  196. package/src/tools/webtest/crawl.ts +0 -901
  197. package/src/tools/webtest/discover-features.ts +0 -412
  198. package/src/tools/webtest/discover-flows.ts +0 -408
  199. package/src/tools/webtest/generate-tests.test.ts +0 -532
  200. package/src/tools/webtest/generate-tests.ts +0 -425
  201. package/src/tools/webtest/index.ts +0 -7
  202. package/src/tools/webtest/integration.test.ts +0 -536
  203. package/src/tools/webtest/run-test-case.test.ts +0 -659
  204. package/src/tools/webtest/run-test-case.ts +0 -508
  205. package/src/tools/webtest/schemas.ts +0 -201
  206. package/src/tools/webtest/start-analysis.test.ts +0 -151
  207. package/src/tools/webtest/start-analysis.ts +0 -158
  208. package/src/transports/http.ts +0 -19
  209. package/src/transports/index.ts +0 -30
  210. package/src/transports/stdio.ts +0 -7
  211. package/src/types/capabilities.test.ts +0 -193
  212. package/src/types/capabilities.ts +0 -50
  213. package/src/types/context.ts +0 -21
  214. package/src/types/tool.ts +0 -11
  215. package/src/workspace/index.ts +0 -945
  216. package/src/workspace/markdown.ts +0 -272
  217. package/src/workspace/types.ts +0 -186
  218. package/tests/integration/server.test.ts +0 -89
  219. package/tests/integration/tools.test.ts +0 -99
  220. package/tsconfig.json +0 -20
  221. package/vitest.config.ts +0 -9
  222. package/vitest.integration.config.ts +0 -10
  223. /package/{src → dist}/prompts/templates/sampling/crawl-action.md +0 -0
  224. /package/{src → dist}/prompts/templates/sampling/feature-discovery.md +0 -0
  225. /package/{src → dist}/prompts/templates/sampling/flow-discovery.md +0 -0
  226. /package/{src → dist}/prompts/templates/sampling/page-content-wrapper.md +0 -0
  227. /package/{src → dist}/prompts/templates/sampling/test-evaluation.md +0 -0
  228. /package/{src → dist}/prompts/templates/sampling/test-generation.md +0 -0
  229. /package/dist/tools/{webtest → retest}/crawl.d.ts +0 -0
  230. /package/dist/tools/{webtest → retest}/discover-features.d.ts +0 -0
  231. /package/dist/tools/{webtest → retest}/discover-flows.d.ts +0 -0
  232. /package/dist/tools/{webtest → retest}/generate-tests.d.ts +0 -0
  233. /package/dist/tools/{webtest → retest}/index.d.ts +0 -0
  234. /package/dist/tools/{webtest → retest}/index.js +0 -0
  235. /package/dist/tools/{webtest → retest}/run-test-case.d.ts +0 -0
  236. /package/dist/tools/{webtest → retest}/schemas.d.ts +0 -0
  237. /package/dist/tools/{webtest → retest}/schemas.js +0 -0
  238. /package/dist/tools/{webtest → retest}/start-analysis.d.ts +0 -0
@@ -1,109 +0,0 @@
1
- # webtest-resources Spec Delta
2
-
3
- ## MODIFIED Requirements
4
-
5
- ### Requirement: Resource URI Scheme
6
-
7
- The system SHALL expose all webtest artifacts using a `webtest://` URI scheme with hierarchical paths, using markdown format for all human-readable artifacts.
8
-
9
- #### Scenario: Analysis root resource is accessible
10
-
11
- - **GIVEN** an analysis has been started with analysisId "abc123"
12
- - **WHEN** client requests resource `webtest://abc123/`
13
- - **THEN** it SHALL return the analysis `index.md` metadata as markdown with YAML frontmatter
14
-
15
- #### Scenario: Crawl index resource is accessible
16
-
17
- - **GIVEN** a crawl has completed with crawlId "crawl-001"
18
- - **WHEN** client requests resource `webtest://abc123/crawls/crawl-001/index.md`
19
- - **THEN** it SHALL return the crawl index as markdown with YAML frontmatter containing page list and metadata
20
-
21
- #### Scenario: Page artifacts are accessible by type
22
-
23
- - **GIVEN** a page was captured with pageId "page-001"
24
- - **WHEN** client requests `webtest://abc123/crawls/crawl-001/pages/page-001/screenshot.png`
25
- - **THEN** it SHALL return the screenshot image
26
- - **AND** `snapshot.md` returns accessibility tree as formatted markdown with YAML frontmatter
27
- - **AND** `dom.html` returns HTML content
28
-
29
- #### Scenario: Crawl checkpoint is accessible
30
-
31
- - **GIVEN** a crawl is in progress with checkpoint saved
32
- - **WHEN** client requests `webtest://abc123/crawls/crawl-001/checkpoint.md`
33
- - **THEN** it SHALL return the checkpoint as markdown with YAML frontmatter containing crawl state
34
-
35
- #### Scenario: Analysis report is accessible
36
-
37
- - **GIVEN** analyze_app has completed
38
- - **WHEN** client requests `webtest://abc123/analysis/app-analysis.md`
39
- - **THEN** it SHALL return the markdown analysis report
40
-
41
- #### Scenario: Flows are accessible
42
-
43
- - **GIVEN** analyze_app has completed
44
- - **WHEN** client requests `webtest://abc123/analysis/flows.md`
45
- - **THEN** it SHALL return user flows as markdown with YAML frontmatter containing structured flow definitions
46
-
47
- #### Scenario: Tests are accessible
48
-
49
- - **GIVEN** generate_tests has completed
50
- - **WHEN** client requests `webtest://abc123/tests/tests.md`
51
- - **THEN** it SHALL return the test cases as markdown with YAML frontmatter containing structured test definitions
52
-
53
- #### Scenario: Test run report is accessible
54
-
55
- - **GIVEN** a test run has completed with runId "run-001"
56
- - **WHEN** client requests `webtest://abc123/runs/run-001/report.md`
57
- - **THEN** it SHALL return the test execution report as markdown with YAML frontmatter containing structured results
58
-
59
- #### Scenario: Test step snapshot is accessible
60
-
61
- - **GIVEN** a test step has captured evidence
62
- - **WHEN** client requests `webtest://abc123/runs/run-001/steps/1/snapshot.md`
63
- - **THEN** it SHALL return the accessibility snapshot as formatted markdown with YAML frontmatter
64
-
65
- ### Requirement: Resource Template Registration
66
-
67
- The system SHALL register resource templates with the MCP server for discovery, using markdown extensions for all index and report resources.
68
-
69
- #### Scenario: Templates are listed on resources/list
70
-
71
- - **GIVEN** a client calls `resources/list`
72
- - **WHEN** the response is returned
73
- - **THEN** it SHALL include templates for:
74
- - `webtest://{analysisId}/index.md` (Analysis index)
75
- - `webtest://{analysisId}/crawls/{crawlId}/index.md` (Crawl index)
76
- - `webtest://{analysisId}/crawls/{crawlId}/checkpoint.md` (Crawl checkpoint)
77
- - `webtest://{analysisId}/crawls/{crawlId}/pages/{pageId}/snapshot.md` (Page snapshot)
78
- - `webtest://{analysisId}/crawls/{crawlId}/pages/{pageId}/screenshot.png` (Page screenshot)
79
- - `webtest://{analysisId}/crawls/{crawlId}/pages/{pageId}/dom.html` (Page DOM)
80
- - `webtest://{analysisId}/analysis/app-analysis.md` (Analysis report)
81
- - `webtest://{analysisId}/analysis/flows.md` (User flows)
82
- - `webtest://{analysisId}/tests/tests.md` (Test definitions)
83
- - `webtest://{analysisId}/runs/{runId}/report.md` (Test run report)
84
- - `webtest://{analysisId}/runs/{runId}/steps/{stepNumber}/snapshot.md` (Step snapshot)
85
- - `webtest://{analysisId}/runs/{runId}/steps/{stepNumber}/screenshot.png` (Step screenshot)
86
-
87
- ### Requirement: Resource Content Types
88
-
89
- The system SHALL return appropriate MIME types for different artifact types.
90
-
91
- #### Scenario: Markdown resources have correct type
92
-
93
- - **GIVEN** client reads a `.md` resource
94
- - **WHEN** response is returned
95
- - **THEN** mimeType SHALL be `text/markdown`
96
-
97
- #### Scenario: Screenshot resources have correct type
98
-
99
- - **GIVEN** client reads a `.png` resource
100
- - **WHEN** response is returned
101
- - **THEN** mimeType SHALL be `image/png`
102
- - **AND** content SHALL be base64 encoded
103
-
104
- #### Scenario: HTML resources have correct type
105
-
106
- - **GIVEN** client reads a `.html` resource
107
- - **WHEN** response is returned
108
- - **THEN** mimeType SHALL be `text/html`
109
-
@@ -1,121 +0,0 @@
1
- # webtest-tools Spec Delta
2
-
3
- ## MODIFIED Requirements
4
-
5
- ### Requirement: webtest_init Tool
6
-
7
- The system SHALL provide a `webtest_init` tool that initializes an analysis workspace for a target URL and focus, storing metadata in markdown format.
8
-
9
- #### Scenario: Start analysis with valid URL
10
-
11
- - **GIVEN** the tool is called with a valid URL and focus
12
- - **WHEN** execution completes
13
- - **THEN** it SHALL generate a unique `analysisId`
14
- - **AND** create workspace directories
15
- - **AND** write initial `index.md` metadata with YAML frontmatter
16
- - **AND** return `{ analysisId, workspaceRootPath, workspaceRootUri, statusUri }` where statusUri points to `index.md`
17
-
18
- ### Requirement: webtest_crawl_app Tool
19
-
20
- The system SHALL provide a `webtest_crawl_app` tool that dynamically explores a web application, storing all crawl artifacts in markdown format.
21
-
22
- #### Scenario: Crawl captures artifacts at each checkpoint
23
-
24
- - **GIVEN** a crawl iteration completes an action
25
- - **WHEN** state is captured
26
- - **THEN** it SHALL call Playwright MCP `browser_snapshot` for accessibility tree
27
- - **AND** call `browser_take_screenshot` for visual evidence
28
- - **AND** optionally extract HTML DOM
29
- - **AND** store snapshot as `snapshot.md` with formatted accessibility tree and YAML frontmatter
30
- - **AND** store screenshot as PNG
31
- - **AND** store DOM as HTML
32
-
33
- #### Scenario: Crawl outputs complete results
34
-
35
- - **GIVEN** crawl has finalized
36
- - **WHEN** output is returned
37
- - **THEN** it SHALL include `crawlId`, `crawlIndexFilePath`, `crawlIndexUri` (pointing to `index.md`), `pages[]`, `summaryUri`
38
-
39
- ### Requirement: Crawl Checkpointing
40
-
41
- The system SHALL implement checkpointing during crawl using markdown format to enable resumption and provide human-readable partial results on failure.
42
-
43
- #### Scenario: Checkpoint is written periodically
44
-
45
- - **GIVEN** a crawl is in progress
46
- - **WHEN** N steps have completed (configurable, default 5)
47
- - **THEN** it SHALL write a checkpoint to `webtest://{analysisId}/crawls/{crawlId}/checkpoint.md`
48
- - **AND** the checkpoint SHALL be markdown with YAML frontmatter including: current step, visited pages, action history, goal progress
49
- - **AND** the checkpoint body SHALL contain human-readable progress summary
50
-
51
- #### Scenario: Crawl can resume from checkpoint
52
-
53
- - **GIVEN** a crawl was interrupted (cancelled, error, timeout)
54
- - **AND** a checkpoint exists as `checkpoint.md`
55
- - **WHEN** `webtest_crawl_app` is called with `resume: true`
56
- - **THEN** it SHALL parse the checkpoint YAML frontmatter
57
- - **AND** continue from the last recorded state
58
-
59
- ### Requirement: webtest_analyze_app Tool
60
-
61
- The system SHALL provide a `webtest_analyze_app` tool that reverse-engineers application structure from crawl data, outputting all results in markdown format.
62
-
63
- #### Scenario: Analyze app writes markdown report
64
-
65
- - **GIVEN** analysis is complete
66
- - **WHEN** output is generated
67
- - **THEN** it SHALL write `app-analysis.md` resource to workspace
68
-
69
- #### Scenario: Analyze app writes flows
70
-
71
- - **GIVEN** analysis is complete
72
- - **WHEN** output is generated
73
- - **THEN** it SHALL write `flows.md` resource to workspace as markdown with YAML frontmatter containing structured flow definitions
74
-
75
- #### Scenario: Analyze app outputs URIs
76
-
77
- - **GIVEN** analysis is complete
78
- - **WHEN** tool returns
79
- - **THEN** it SHALL include `appAnalysisFilePath`, `appAnalysisUri`, `flowsFilePath`, `flowsUri` all pointing to `.md` files
80
-
81
- ### Requirement: webtest_generate_tests Tool
82
-
83
- The system SHALL provide a `webtest_generate_tests` tool that produces test cases from application analysis in a single markdown format.
84
-
85
- #### Scenario: Generate tests outputs structured format
86
-
87
- - **GIVEN** test generation completes
88
- - **WHEN** results are written
89
- - **THEN** it SHALL produce `tests.md` with human-readable format AND YAML frontmatter containing structured test definitions
90
- - **AND** there SHALL NOT be a separate `tests.json` file
91
-
92
- #### Scenario: Generate tests outputs URIs
93
-
94
- - **GIVEN** generation is complete
95
- - **WHEN** tool returns
96
- - **THEN** it SHALL include `testsFilePath` and `testsUri` pointing to `tests.md`
97
-
98
- ### Requirement: webtest_run_tests Tool
99
-
100
- The system SHALL provide a `webtest_run_tests` tool that executes a test case with evidence capture, storing all results in markdown format.
101
-
102
- #### Scenario: Run test case captures evidence
103
-
104
- - **GIVEN** a step is executed
105
- - **WHEN** evidence is captured
106
- - **THEN** it SHALL take screenshot after action (stored as PNG)
107
- - **AND** capture accessibility snapshot (stored as `snapshot.md` with formatted tree and YAML frontmatter)
108
- - **AND** store with step identifier
109
-
110
- #### Scenario: Run test case outputs report
111
-
112
- - **GIVEN** test execution completes
113
- - **WHEN** output is generated
114
- - **THEN** it SHALL write `report.md` with pass/fail summary, step details, evidence links, and YAML frontmatter containing structured run data
115
- - **AND** there SHALL NOT be a separate `index.json` or `artifacts.json` file
116
-
117
- #### Scenario: Run test case returns URIs
118
-
119
- - **GIVEN** execution is complete
120
- - **WHEN** tool returns
121
- - **THEN** it SHALL include `testRunId`, `reportFilePath`, `reportUri` pointing to `report.md`
@@ -1,133 +0,0 @@
1
- # Tasks: Use Markdown for All Artifacts
2
-
3
- ## Prerequisites
4
-
5
- - [x] **Add gray-matter dependency** - Install `gray-matter` package for YAML frontmatter parsing
6
- - `npm install gray-matter`
7
- - Add to package.json dependencies
8
-
9
- ## Phase 1: Core Utilities
10
-
11
- - [x] **Create markdown utilities module** - `src/workspace/markdown.ts`
12
- - Implement `writeMarkdownWithFrontmatter(path, frontmatter, content)`
13
- - Implement `readMarkdownFrontmatter<T>(path)` returning parsed YAML data
14
- - Implement `formatAccessibilityTree(snapshot)` for human-readable tree output
15
- - Add unit tests for utilities
16
-
17
- ## Phase 2: Workspace Index
18
-
19
- - [x] **Update workspace index to markdown** - `src/workspace/index.ts`
20
- - Change `createWorkspace` to write `index.md` instead of `index.json`
21
- - Update `readWorkspaceIndex` to parse from markdown frontmatter
22
- - Update `updateWorkspaceIndex` to preserve markdown body while updating frontmatter
23
- - Generate human-readable workspace summary in markdown body
24
-
25
- - [x] **Update workspace types** - `src/workspace/types.ts`
26
- - Update `WorkspaceIndex` type if needed for markdown-specific fields
27
-
28
- ## Phase 3: Crawl Artifacts
29
-
30
- - [x] **Update crawl index to markdown** - `src/workspace/index.ts`
31
- - Change `createCrawl` to write `index.md`
32
- - Update `readCrawlIndex` to parse from markdown frontmatter
33
- - Update `updateCrawlIndex` to regenerate markdown body
34
- - Include formatted action history and budget status in body
35
-
36
- - [x] **Update checkpoint to markdown** - `src/workspace/index.ts`
37
- - Change `saveCheckpoint` to write `checkpoint.md`
38
- - Update `loadCheckpoint` to parse from markdown frontmatter
39
- - Include human-readable progress summary in body
40
-
41
- - [x] **Update page snapshot to markdown** - `src/workspace/index.ts`
42
- - Change `savePage` to write `snapshot.md` instead of `snapshot.json`
43
- - Format accessibility tree as indented text tree in body
44
- - Keep full structured snapshot in YAML frontmatter
45
- - Update return paths/URIs to `.md` extension
46
-
47
- ## Phase 4: Analysis Artifacts
48
-
49
- - [x] **Update flows to markdown** - `src/tools/webtest/analyze-app.ts`
50
- - Change `saveAnalysis` to write `flows.md` instead of `flows.json`
51
- - Format flows as readable markdown sections
52
- - Include structured flow data in YAML frontmatter
53
- - Update return paths/URIs
54
-
55
- - [x] **Update analyze-app tool outputs** - `src/tools/webtest/analyze-app.ts`
56
- - Update `flowsFilePath` and `flowsUri` to point to `flows.md`
57
-
58
- ## Phase 5: Test Artifacts
59
-
60
- - [x] **Consolidate test definitions** - `src/tools/webtest/generate-tests.ts`
61
- - Remove separate `tests.json` generation
62
- - Update `saveTests` to write only `tests.md` with YAML frontmatter
63
- - Include full test case structures in frontmatter
64
- - Keep human-readable test documentation in body
65
- - Update return paths/URIs (remove `testsFilePath` for JSON)
66
-
67
- - [x] **Update generate-tests tool outputs** - `src/tools/webtest/generate-tests.ts`
68
- - Return single `testsUri` pointing to `tests.md`
69
- - Update `testsFilePath` to point to `tests.md`
70
-
71
- ## Phase 6: Test Run Artifacts
72
-
73
- - [x] **Update test run index to report** - `src/workspace/index.ts`
74
- - Change `createTestRun` to write `report.md` instead of `index.json`
75
- - Update `readTestRunIndex` to parse from markdown frontmatter
76
- - Update `updateTestRunIndex` to regenerate markdown body
77
-
78
- - [x] **Update test step snapshots** - `src/workspace/index.ts`
79
- - Change `saveTestStepEvidence` to write `snapshot.md`
80
- - Format accessibility tree in body
81
- - Update return paths/URIs
82
-
83
- - [x] **Update run-test-case tool outputs** - `src/tools/webtest/run-test-case.ts`
84
- - Update `reportUri` to point to `report.md`
85
- - Update step evidence URIs to point to `snapshot.md`
86
-
87
- ## Phase 7: Resource Manager
88
-
89
- - [x] **Update resource URI mappings** - `src/resources/index.ts`
90
- - Update all `.json` paths to `.md` in URI templates
91
- - Handle markdown MIME type for new paths
92
- - Update resource listing to discover `.md` files
93
-
94
- - [x] **Update resource reading** - `src/resources/index.ts`
95
- - Return raw markdown content for `.md` resources
96
- - Update MIME type to `text/markdown`
97
-
98
- ## Phase 8: Integration & Testing
99
-
100
- - [x] **Update existing tests** - Test files in `src/` or `tests/`
101
- - Update file path expectations from `.json` to `.md`
102
- - Update content parsing to use frontmatter extraction
103
- - Add tests for markdown format validity
104
-
105
- - [x] **Manual integration test**
106
- - Run full workflow: init → crawl → analyze → generate tests → run tests
107
- - Verify all workspace files are `.md` (except screenshots and DOM)
108
- - Verify files are readable without tooling
109
- - Verify programmatic access via frontmatter works
110
-
111
- ## Phase 9: Documentation
112
-
113
- - [x] **Update CLAUDE.md or README** (if exists)
114
- - Document new artifact format
115
- - Provide examples of frontmatter structure
116
- - Note breaking change for existing workspaces
117
-
118
- ## Verification Checklist
119
-
120
- After implementation, verify:
121
-
122
- - [x] `index.md` at workspace root with valid frontmatter
123
- - [x] `crawls/{id}/index.md` with action history
124
- - [x] `crawls/{id}/checkpoint.md` with progress state
125
- - [x] `crawls/{id}/pages/{id}/snapshot.md` with formatted tree
126
- - [x] `analysis/app-analysis.md` (unchanged)
127
- - [x] `analysis/flows.md` with flow definitions
128
- - [x] `tests/tests.md` with all test data (no tests.json)
129
- - [x] `runs/{id}/report.md` with execution results
130
- - [x] `runs/{id}/steps/{n}/snapshot.md` with formatted tree
131
- - [x] All tool outputs reference `.md` paths
132
- - [x] All resource URIs use `.md` extensions
133
- - [x] Workspace is browsable and human-readable
@@ -1,86 +0,0 @@
1
- ## Context
2
-
3
- Prompts are currently embedded as template literal strings in TypeScript files. This change extracts them to markdown files for better maintainability while preserving runtime variable interpolation.
4
-
5
- ## Goals / Non-Goals
6
-
7
- **Goals:**
8
- - Make prompts easy to read, edit, and review as standalone files
9
- - Preserve `${varName}` syntax for variable substitution
10
- - Maintain type safety for required variables
11
- - Keep runtime performance acceptable (load once, cache)
12
-
13
- **Non-Goals:**
14
- - Complex templating features (loops, conditionals) - keep it simple
15
- - Hot-reloading of templates in production
16
- - User-configurable prompts at runtime
17
-
18
- ## Decisions
19
-
20
- ### Decision: Use simple `${varName}` interpolation
21
-
22
- Keep the existing JavaScript template literal syntax. The loader will:
23
- 1. Read the markdown file content
24
- 2. Replace `${varName}` patterns with provided values
25
- 3. Return the interpolated string
26
-
27
- **Alternatives considered:**
28
- - Mustache/Handlebars: Adds dependency, overkill for simple substitution
29
- - Tagged template literals: Would require keeping prompts in JS files
30
- - JSON with placeholders: Less readable for long-form content
31
-
32
- ### Decision: Directory structure
33
-
34
- ```
35
- src/prompts/
36
- ├── templates/
37
- │ ├── mcp/ # User-facing MCP prompts
38
- │ │ ├── webtest-start.md
39
- │ │ ├── webtest-crawl.md
40
- │ │ └── ...
41
- │ └── sampling/ # Internal AI sampling prompts
42
- │ ├── system-prefix.md
43
- │ ├── crawl-action.md
44
- │ └── ...
45
- ├── loader.ts # Template loading utility
46
- ├── index.ts # MCP prompt definitions (uses loader)
47
- └── index.test.ts # Tests
48
- ```
49
-
50
- **Rationale:** Separating `mcp/` and `sampling/` makes it clear which prompts are user-facing vs internal.
51
-
52
- ### Decision: Synchronous file loading with caching
53
-
54
- Load templates synchronously at module initialization and cache them. This:
55
- - Avoids async complexity in prompt builders
56
- - Ensures templates are available immediately
57
- - Is acceptable since templates are static assets
58
-
59
- ### Decision: TypeScript interface for variables
60
-
61
- Each prompt template will have a corresponding TypeScript type defining its variables:
62
-
63
- ```typescript
64
- interface CrawlActionVars {
65
- goal: string;
66
- currentUrl: string;
67
- pageSnapshot: string;
68
- actionHistory: string;
69
- allowedDomains: string;
70
- // ... optional vars
71
- }
72
- ```
73
-
74
- This maintains type safety when calling the interpolation function.
75
-
76
- ## Risks / Trade-offs
77
-
78
- | Risk | Mitigation |
79
- |------|------------|
80
- | File system access at startup | Templates are small, load is fast |
81
- | Missing variable errors at runtime | Type definitions + tests catch this |
82
- | Bundler complications | Ensure templates are included in build output |
83
-
84
- ## Open Questions
85
-
86
- - None at this time
@@ -1,50 +0,0 @@
1
- # Change: Extract prompts to separate markdown files
2
-
3
- ## Why
4
-
5
- Currently, all prompt text is embedded inline within TypeScript files (`src/prompts/index.ts` and `src/sampling/prompts.ts`). This makes prompts difficult to maintain, review, and iterate on since:
6
- - Prompt changes require modifying TypeScript code
7
- - Large multi-line strings with template literals are hard to read and edit
8
- - Non-developers cannot easily review or suggest prompt improvements
9
- - No syntax highlighting or markdown preview for prompt content
10
-
11
- ## What Changes
12
-
13
- - **Extract MCP prompts** from `src/prompts/index.ts` to separate `.md` files
14
- - **Extract sampling prompts** from `src/sampling/prompts.ts` to separate `.md` files
15
- - Store all prompt templates in `src/prompts/templates/`
16
- - Keep `${varName}` interpolation syntax for dynamic values
17
- - Create a utility to load and interpolate markdown templates at runtime
18
- - Update existing code to use the template loader
19
-
20
- ### Prompts to Extract
21
-
22
- **MCP Prompts (user-facing):**
23
- - `webtest-start.md`
24
- - `webtest-crawl.md`
25
- - `webtest-discover.md`
26
- - `webtest-generate-tests.md`
27
- - `webtest-run-test.md`
28
- - `webtest-full-workflow.md`
29
-
30
- **Sampling Prompts (internal AI prompts):**
31
- - `system-prefix.md` - Security-hardened system prompt
32
- - `page-content-wrapper.md` - Untrusted content delimiters
33
- - `crawl-action.md` - Crawl navigation decisions
34
- - `feature-discovery.md` - Discover app features/modules
35
- - `flow-discovery.md` - Discover user flows within features
36
- - `test-generation.md` - Generate test cases
37
- - `test-evaluation.md` - Evaluate test step results
38
-
39
- **Cleanup:**
40
- - Remove deprecated `buildAnalysisPrompt` function
41
- - Update `analyze-app.ts` to use `buildFeatureDiscoveryPrompt` instead
42
-
43
- ## Impact
44
-
45
- - Affected specs: `webtest-prompts`
46
- - Affected code:
47
- - `src/prompts/index.ts` - Refactor to load from markdown
48
- - `src/sampling/prompts.ts` - Refactor to load from markdown
49
- - New: `src/prompts/templates/` directory with `.md` files
50
- - New: `src/prompts/loader.ts` - Template loading utility
@@ -1,74 +0,0 @@
1
- ## ADDED Requirements
2
-
3
- ### Requirement: Prompt Template Files
4
-
5
- The system SHALL store prompt text in separate markdown files under `src/prompts/templates/`.
6
-
7
- #### Scenario: MCP prompts stored in templates/mcp/
8
-
9
- - **GIVEN** a user-facing MCP prompt (e.g., `webtest-start`)
10
- - **WHEN** the prompt content is needed
11
- - **THEN** it SHALL be loaded from `src/prompts/templates/mcp/<prompt-name>.md`
12
-
13
- #### Scenario: Sampling prompts stored in templates/sampling/
14
-
15
- - **GIVEN** an internal sampling prompt (e.g., `crawl-action`)
16
- - **WHEN** the prompt content is needed
17
- - **THEN** it SHALL be loaded from `src/prompts/templates/sampling/<prompt-name>.md`
18
-
19
- ### Requirement: Template Variable Interpolation
20
-
21
- The system SHALL support `${varName}` syntax for variable substitution in markdown templates.
22
-
23
- #### Scenario: Simple variable substitution
24
-
25
- - **GIVEN** a template containing `${url}` placeholder
26
- - **WHEN** the template is rendered with `{ url: "https://example.com" }`
27
- - **THEN** the output SHALL contain `https://example.com` in place of `${url}`
28
-
29
- #### Scenario: Multiple variables in template
30
-
31
- - **GIVEN** a template containing `${goal}`, `${currentUrl}`, and `${allowedDomains}`
32
- - **WHEN** the template is rendered with all variables provided
33
- - **THEN** all placeholders SHALL be replaced with their corresponding values
34
-
35
- #### Scenario: Preserve unmatched placeholders
36
-
37
- - **GIVEN** a template containing `${undefinedVar}`
38
- - **WHEN** the template is rendered without that variable
39
- - **THEN** the placeholder `${undefinedVar}` SHALL remain in the output unchanged
40
- - **AND** no error SHALL be thrown
41
-
42
- ### Requirement: Template Loading and Caching
43
-
44
- The system SHALL load templates efficiently with appropriate caching.
45
-
46
- #### Scenario: Templates loaded at initialization
47
-
48
- - **GIVEN** the server starts
49
- - **WHEN** templates are first accessed
50
- - **THEN** they SHALL be loaded from the filesystem
51
- - **AND** cached for subsequent access
52
-
53
- #### Scenario: Cached templates returned on repeated access
54
-
55
- - **GIVEN** a template has been loaded previously
56
- - **WHEN** the same template is requested again
57
- - **THEN** the cached version SHALL be returned without re-reading the file
58
-
59
- ### Requirement: Template File Format
60
-
61
- Markdown template files SHALL follow a consistent format for maintainability.
62
-
63
- #### Scenario: Template uses raw markdown content
64
-
65
- - **GIVEN** a prompt template file
66
- - **WHEN** the file is read
67
- - **THEN** the entire file content SHALL be used as the prompt text
68
- - **AND** no special parsing or metadata extraction is required
69
-
70
- #### Scenario: Template preserves whitespace and formatting
71
-
72
- - **GIVEN** a template with specific indentation and line breaks
73
- - **WHEN** the template is loaded and interpolated
74
- - **THEN** all whitespace and formatting SHALL be preserved exactly
@@ -1,40 +0,0 @@
1
- ## 1. Template Infrastructure
2
-
3
- - [x] 1.1 Create `src/prompts/templates/` directory structure
4
- - [x] 1.2 Implement `src/prompts/loader.ts` with template loading and interpolation
5
- - [x] 1.3 Add tests for template loader (variable substitution, missing vars, file not found)
6
-
7
- ## 2. Extract Sampling Prompts
8
-
9
- - [x] 2.1 Create `src/prompts/templates/sampling/system-prefix.md`
10
- - [x] 2.2 Create `src/prompts/templates/sampling/page-content-wrapper.md`
11
- - [x] 2.3 Create `src/prompts/templates/sampling/crawl-action.md`
12
- - [x] 2.4 Create `src/prompts/templates/sampling/feature-discovery.md`
13
- - [x] 2.5 Create `src/prompts/templates/sampling/flow-discovery.md`
14
- - [x] 2.6 Create `src/prompts/templates/sampling/test-generation.md`
15
- - [x] 2.7 Create `src/prompts/templates/sampling/test-evaluation.md`
16
- - [x] 2.8 Update `src/sampling/prompts.ts` to use template loader
17
- - [x] 2.9 Verify existing sampling tests pass
18
-
19
- ## 3. Remove Deprecated Code
20
-
21
- - [x] 3.1 Remove `buildAnalysisPrompt` from `src/sampling/prompts.ts` (already removed)
22
- - [x] 3.2 Update `src/tools/webtest/analyze-app.ts` to use `buildFeatureDiscoveryPrompt` (already done)
23
- - [x] 3.3 Update or remove related tests in `analyze-app.test.ts` (already done)
24
-
25
- ## 4. Extract MCP Prompts
26
-
27
- - [x] 4.1 Create `src/prompts/templates/mcp/webtest-start.md`
28
- - [x] 4.2 Create `src/prompts/templates/mcp/webtest-crawl.md`
29
- - [x] 4.3 Create `src/prompts/templates/mcp/webtest-discover.md`
30
- - [x] 4.4 Create `src/prompts/templates/mcp/webtest-discover-flows.md`
31
- - [x] 4.5 Create `src/prompts/templates/mcp/webtest-generate-tests.md`
32
- - [x] 4.6 Create `src/prompts/templates/mcp/webtest-run-test.md`
33
- - [x] 4.7 Create `src/prompts/templates/mcp/webtest-full-workflow.md`
34
- - [x] 4.8 Update `src/prompts/index.ts` to use template loader
35
- - [x] 4.9 Verify existing MCP prompt tests pass
36
-
37
- ## 5. Validation
38
-
39
- - [x] 5.1 Run full test suite (`npm test`) - 448 tests pass
40
- - [x] 5.2 Run build (`npm run build`) - success