csspilot 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/bin.js +31762 -0
- package/dist/index.js +31767 -0
- package/dist/skills/testpilot/SKILL.md +64 -0
- package/dist/skills/testpilot/manifest.yaml +37 -0
- package/dist/skills/testpilot/rules/.gitkeep +0 -0
- package/dist/skills/testpilot/rules/assertion.md +6 -0
- package/dist/skills/testpilot/rules/case-design.md +29 -0
- package/dist/skills/testpilot/rules/dsl.md +42 -0
- package/dist/skills/testpilot/rules/locator.md +14 -0
- package/dist/skills/testpilot/workflows/.gitkeep +0 -0
- package/dist/skills/testpilot/workflows/analyze-result.md +12 -0
- package/dist/skills/testpilot/workflows/create-case.md +13 -0
- package/dist/skills/testpilot/workflows/run.md +19 -0
- package/dist/skills/testpilot/workflows/validate.md +21 -0
- package/package.json +45 -0
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# TestPilot
|
|
2
|
+
|
|
3
|
+
## What is TestPilot
|
|
4
|
+
|
|
5
|
+
TestPilot is an AI Native cross-platform testing infrastructure.
|
|
6
|
+
|
|
7
|
+
It lets AI understand, generate, validate and run business tests for Web and Mini Program apps.
|
|
8
|
+
|
|
9
|
+
TestPilot is a **tool, not a business test project**. Business cases always belong to the project (`tests/e2e/cases/`), never to TestPilot itself.
|
|
10
|
+
|
|
11
|
+
## When to use
|
|
12
|
+
|
|
13
|
+
Use TestPilot when the user wants to:
|
|
14
|
+
|
|
15
|
+
- create business test cases
|
|
16
|
+
- validate test cases
|
|
17
|
+
- run tests
|
|
18
|
+
- inspect test results
|
|
19
|
+
|
|
20
|
+
## Core workflow
|
|
21
|
+
|
|
22
|
+
1. Understand user requirement
|
|
23
|
+
2. Inspect project
|
|
24
|
+
3. Identify test target
|
|
25
|
+
4. Create TestPilot Case (`tests/e2e/cases/*.yaml`)
|
|
26
|
+
5. Validate Case (`npx testpilot validate`)
|
|
27
|
+
6. Run Case (`npx testpilot run`)
|
|
28
|
+
7. Analyze result (`npx testpilot report`)
|
|
29
|
+
|
|
30
|
+
## Important rules
|
|
31
|
+
|
|
32
|
+
- Never directly control Playwright.
|
|
33
|
+
- Never directly control WeChatIDE.
|
|
34
|
+
- Use TestPilot DSL to describe tests.
|
|
35
|
+
- Business cases belong to the project.
|
|
36
|
+
- Prefer existing project conventions.
|
|
37
|
+
- Never invent locators without evidence.
|
|
38
|
+
- Validate before running.
|
|
39
|
+
|
|
40
|
+
## CLI
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
npx testpilot init # install TestPilot into the current project
|
|
44
|
+
npx testpilot validate # validate cases against the DSL
|
|
45
|
+
npx testpilot list # list cases (--tag smoke)
|
|
46
|
+
npx testpilot run # run cases (file paths / --tag)
|
|
47
|
+
npx testpilot report # generate JSON + HTML report
|
|
48
|
+
npx testpilot doctor # check environment
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
## Directory map (business project)
|
|
52
|
+
|
|
53
|
+
- `.ai/skills/testpilot/` — this skill (AI usage spec)
|
|
54
|
+
- `.testpilot/` — TestPilot runtime state and artifacts
|
|
55
|
+
- `tests/e2e/cases/` — business cases
|
|
56
|
+
- `testpilot.yaml` — project-level TestPilot config
|
|
57
|
+
|
|
58
|
+
## Further reading
|
|
59
|
+
|
|
60
|
+
- `rules/case-design.md` — what a case is, evidence-first generation
|
|
61
|
+
- `rules/dsl.md` — case structure and the V0.1 action set
|
|
62
|
+
- `rules/locator.md` — locator forms and the no-invention rule
|
|
63
|
+
- `rules/assertion.md` — assertions and variable references
|
|
64
|
+
- `workflows/` — step-by-step workflows (create / validate / run / analyze)
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# TestPilot Skill 清单(源文件)
|
|
2
|
+
# `npx testpilot init` 会把整个 skills/testpilot/ 安装到业务项目 .ai/skills/testpilot/
|
|
3
|
+
id: testpilot
|
|
4
|
+
name: TestPilot
|
|
5
|
+
version: 0.1.0
|
|
6
|
+
|
|
7
|
+
description: >
|
|
8
|
+
AI Native cross-platform testing skill.
|
|
9
|
+
|
|
10
|
+
capabilities:
|
|
11
|
+
- create-case
|
|
12
|
+
- validate
|
|
13
|
+
- run
|
|
14
|
+
- list
|
|
15
|
+
- report
|
|
16
|
+
|
|
17
|
+
runtime:
|
|
18
|
+
cli: testpilot
|
|
19
|
+
|
|
20
|
+
permissions:
|
|
21
|
+
filesystem:
|
|
22
|
+
read:
|
|
23
|
+
- src/**
|
|
24
|
+
- pages/**
|
|
25
|
+
- tests/**
|
|
26
|
+
- testpilot.yaml
|
|
27
|
+
|
|
28
|
+
write:
|
|
29
|
+
- tests/e2e/**
|
|
30
|
+
- .testpilot/**
|
|
31
|
+
|
|
32
|
+
shell:
|
|
33
|
+
allow:
|
|
34
|
+
- npx
|
|
35
|
+
- pnpm
|
|
36
|
+
- npm
|
|
37
|
+
- testpilot
|
|
File without changes
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
# 断言规范
|
|
2
|
+
|
|
3
|
+
- 断言使用 `action: assert`,配合 `locator` + `expected`。
|
|
4
|
+
- `expected` 支持 `${variable}` 变量引用,变量来自之前的 `extract` 步骤(存于 ExecutionContext,可跨端传递,如小程序提取订单号 → Web 端断言)。
|
|
5
|
+
- 断言必须对应真实业务预期(如订单号出现在 Web 后台),不写恒真断言。
|
|
6
|
+
- 断言失败不一定是 Case 错误:先分析 Evidence(截图 / trace / log),判断是用例问题还是产品 bug,再决定修改 Case 还是上报问题。
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Case 设计规范
|
|
2
|
+
|
|
3
|
+
## Case 属于业务项目
|
|
4
|
+
|
|
5
|
+
- Case 只存在于业务项目的 `tests/e2e/cases/`,永远不要写进 TestPilot 仓库的 `packages/`。
|
|
6
|
+
- 一个 Case 描述一条完整的业务流程(如:创建订单、提交退款),不是单个页面操作。
|
|
7
|
+
- `id` 使用 kebab-case,与文件名一致:`order-create.yaml` → `id: order-create`。
|
|
8
|
+
|
|
9
|
+
## AI 不能凭空生成测试
|
|
10
|
+
|
|
11
|
+
正确路径:
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
用户需求 → 代码 / 页面 / 配置 → Evidence → 测试步骤 → Case
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
禁止路径:
|
|
18
|
+
|
|
19
|
+
```text
|
|
20
|
+
用户需求 → AI 猜 → Case
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
看到"测试登录"就直接写 `locator: { text: 登录 }` 是错误的。
|
|
24
|
+
必须先阅读项目真实页面/代码,确认元素存在、文案准确,再生成步骤。
|
|
25
|
+
|
|
26
|
+
## 业务数据
|
|
27
|
+
|
|
28
|
+
- 不要把业务账号、密码硬编码进 Case。
|
|
29
|
+
- 测试数据放业务项目 `tests/e2e/fixtures/` 或 `tests/e2e/data/`,Case 中通过变量引用。
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# DSL 规范
|
|
2
|
+
|
|
3
|
+
Case 是 YAML 文件,V0.1 顶层结构:
|
|
4
|
+
|
|
5
|
+
```yaml
|
|
6
|
+
id: order-create # kebab-case,与文件名一致
|
|
7
|
+
name: 用户创建订单 # 人类可读名称
|
|
8
|
+
steps: # 有序步骤列表
|
|
9
|
+
- target: miniapp # 执行端:miniapp | web
|
|
10
|
+
action: click # 见下方 Action 表
|
|
11
|
+
locator: # 定位器(见 rules/locator.md)
|
|
12
|
+
text: 提交订单
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## V0.1 稳定 Action
|
|
16
|
+
|
|
17
|
+
| action | 用途 | 关键字段 |
|
|
18
|
+
| --- | --- | --- |
|
|
19
|
+
| `launch` | 启动应用(miniapp) | — |
|
|
20
|
+
| `navigate` | 打开页面(web) | `url` |
|
|
21
|
+
| `click` | 点击 | `locator` |
|
|
22
|
+
| `input` | 输入 | `locator`, `value` |
|
|
23
|
+
| `select` | 选择 | `locator`, `value` |
|
|
24
|
+
| `wait` | 等待 | 条件(以 @testpilot/dsl schema 为准) |
|
|
25
|
+
| `assert` | 断言 | `locator`, `expected` |
|
|
26
|
+
| `extract` | 提取内容为变量 | `locator`, 变量名(以 schema 为准) |
|
|
27
|
+
| `screenshot` | 截图取证 | — |
|
|
28
|
+
|
|
29
|
+
字段定义最终以 `packages/dsl` 的 zod schema 为准;不要使用表中之外的字段。
|
|
30
|
+
|
|
31
|
+
## 原则:Action 是通用测试原语,不是业务能力
|
|
32
|
+
|
|
33
|
+
```text
|
|
34
|
+
login / order / payment ❌ 业务动作不是 Action
|
|
35
|
+
click / input / assert ✓ 由基础 Action 组合表达业务流程
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
登录、下单等业务流程由多个基础 Action 组合而成,不要设计业务 Action。
|
|
39
|
+
|
|
40
|
+
## 示例
|
|
41
|
+
|
|
42
|
+
见仓库 `examples/cases/order-create.yaml`(小程序下单 → Web 后台验证的黄金路径)。
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Locator 规范
|
|
2
|
+
|
|
3
|
+
## V0.1 支持形式
|
|
4
|
+
|
|
5
|
+
- `text:` 按可见文案匹配(仅 web,如 `text: 提交订单`)
|
|
6
|
+
- `css:` 按 CSS 选择器匹配(web 与小程序均支持,如 `css: .order-id`;小程序匹配 WXML 的 class/id)
|
|
7
|
+
|
|
8
|
+
## 铁律:不凭空发明 locator
|
|
9
|
+
|
|
10
|
+
- locator 必须来自真实证据:页面源码、DOM、组件代码,或 screenshot / extract 的结果。
|
|
11
|
+
- 禁止根据业务名词猜测文案或类名。
|
|
12
|
+
- 文案类 locator 要与真实渲染完全一致,注意空格与标点。
|
|
13
|
+
- 优先选择稳定且语义化的目标:自定义属性 > 语义类名 > 文案;避免依赖动态生成的类名和元素索引。
|
|
14
|
+
- 小程序端只能用 css:从 WXML / WXSS 中确认 class 或 id 后再写步骤。
|
|
File without changes
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Workflow:分析测试结果
|
|
2
|
+
|
|
3
|
+
1. 执行 `npx testpilot report` 生成报告。
|
|
4
|
+
2. 产物位于 `.testpilot/artifacts/runs/<run-id>/`:
|
|
5
|
+
- `report.json` —— 结构化结果,AI 优先读这个:`{ generatedAt, summary: { runId, status, totals, cases: [{ caseId, status, steps: [{ index, target, action, status, durationMs, error?, screenshot?, extracted? }] }] } }`
|
|
6
|
+
- `report.html` —— 人类可读报告
|
|
7
|
+
- `screenshots/`、`traces/`、`logs/` —— 失败取证
|
|
8
|
+
3. 失败分析顺序:report.json 中失败步骤的 `error` 信息 → 对应 `screenshot` → `logs/run.log` → `events.ndjson`。
|
|
9
|
+
4. 得出结论:
|
|
10
|
+
- Case 问题(locator 过期、步骤错误)→ 修改 Case,重新 validate → run
|
|
11
|
+
- 产品 bug → 停止修改 Case,向用户报告证据与结论
|
|
12
|
+
5. 不要在未分析 Evidence 的情况下盲目重跑或修改 Case。
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Workflow:创建测试 Case
|
|
2
|
+
|
|
3
|
+
输入:用户的一句业务需求(如"帮我测试用户下单")。
|
|
4
|
+
|
|
5
|
+
1. 阅读本 Skill:`rules/dsl.md`、`rules/case-design.md`、`rules/locator.md`。
|
|
6
|
+
2. 分析项目结构:确定被测端(miniapp / web)与业务入口页面。
|
|
7
|
+
3. 找到业务路径上的真实页面与组件,收集证据(读代码 / DOM / 现有测试)。
|
|
8
|
+
4. 确定测试路径:从入口到业务结果页的完整步骤。
|
|
9
|
+
5. 按 DSL 规范生成 Case,写入 `tests/e2e/cases/<case-id>.yaml`。
|
|
10
|
+
6. 执行 `npx testpilot validate tests/e2e/cases/<case-id>.yaml`。
|
|
11
|
+
7. 校验失败则修复 Case 后重新 validate,直到通过。
|
|
12
|
+
|
|
13
|
+
禁止:跳过第 2-3 步直接生成 Case(AI 不能凭空生成测试)。
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Workflow:执行测试
|
|
2
|
+
|
|
3
|
+
```bash
|
|
4
|
+
npx testpilot run # 运行项目配置(testpilot.yaml)中的 Case
|
|
5
|
+
npx testpilot run <file> # 运行指定 Case
|
|
6
|
+
npx testpilot run --tag smoke # 按标签运行
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
执行链路:
|
|
10
|
+
|
|
11
|
+
```text
|
|
12
|
+
CLI → Case Loader → DSL Validator → Execution Engine → Adapter(Playwright / WeChatIDE)→ Evidence → Reporter
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
规则:
|
|
16
|
+
|
|
17
|
+
- 只通过 testpilot CLI 执行测试,不要直接调用 playwright 或 wechat-devtools。
|
|
18
|
+
- 被测服务需先就绪(如业务后端、本地页面服务),`navigate` 的相对 url 基于 testpilot.yaml 的 `web.baseUrl` 解析。
|
|
19
|
+
- 运行产物写入 `.testpilot/artifacts/runs/<run-id>/`(screenshots / traces / logs)。
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Workflow:校验 Case
|
|
2
|
+
|
|
3
|
+
任何 Case 在 run 之前必须先 validate。
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
npx testpilot validate # 全量
|
|
7
|
+
npx testpilot validate <file> # 指定文件
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
检查内容:
|
|
11
|
+
|
|
12
|
+
- YAML 格式
|
|
13
|
+
- DSL 结构
|
|
14
|
+
- action 是否存在
|
|
15
|
+
- target 是否存在
|
|
16
|
+
- locator 是否合法
|
|
17
|
+
- 必填字段
|
|
18
|
+
- 变量引用
|
|
19
|
+
- 当前 adapter 是否支持该 action
|
|
20
|
+
|
|
21
|
+
输出 `ValidationResult`;有错误时修复 Case 后重新校验,全部通过才允许 run。
|
package/package.json
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "csspilot",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "TestPilot — AI Native 跨端业务测试基础设施 CLI",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"repository": {
|
|
7
|
+
"type": "git",
|
|
8
|
+
"url": "git+https://github.com/3lin9/css-test-pilot.git"
|
|
9
|
+
},
|
|
10
|
+
"type": "module",
|
|
11
|
+
"engines": {
|
|
12
|
+
"node": ">=22"
|
|
13
|
+
},
|
|
14
|
+
"bin": {
|
|
15
|
+
"csspilot": "dist/bin.js"
|
|
16
|
+
},
|
|
17
|
+
"files": [
|
|
18
|
+
"dist"
|
|
19
|
+
],
|
|
20
|
+
"scripts": {
|
|
21
|
+
"build": "node build.mjs",
|
|
22
|
+
"dev": "tsx src/bin.ts",
|
|
23
|
+
"start": "tsx src/bin.ts",
|
|
24
|
+
"typecheck": "tsc --noEmit"
|
|
25
|
+
},
|
|
26
|
+
"dependencies": {
|
|
27
|
+
"miniprogram-automator": "^0.12.0",
|
|
28
|
+
"playwright": "^1.53.0"
|
|
29
|
+
},
|
|
30
|
+
"devDependencies": {
|
|
31
|
+
"@testpilot/adapter-core": "workspace:^",
|
|
32
|
+
"@testpilot/adapter-playwright": "workspace:^",
|
|
33
|
+
"@testpilot/adapter-wechatide": "workspace:^",
|
|
34
|
+
"@testpilot/core": "workspace:^",
|
|
35
|
+
"@testpilot/dsl": "workspace:^",
|
|
36
|
+
"@testpilot/evidence": "workspace:^",
|
|
37
|
+
"@testpilot/execution-engine": "workspace:^",
|
|
38
|
+
"@testpilot/reporter": "workspace:^",
|
|
39
|
+
"@types/node": "^22.0.0",
|
|
40
|
+
"commander": "^12.0.0",
|
|
41
|
+
"esbuild": "^0.25.0",
|
|
42
|
+
"tsx": "^4.19.0",
|
|
43
|
+
"typescript": "^5.5.0"
|
|
44
|
+
}
|
|
45
|
+
}
|