@pzy560117/opentest 0.1.9 → 0.1.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. package/assets/manifest.json +12 -1
  2. package/assets/skills/opentest/references/api-testing.md +77 -0
  3. package/assets/skills/opentest/references/codex-harness-coverage-heuristics.md +17 -1
  4. package/assets/skills/opentest/references/complete-testing-workflow.md +27 -0
  5. package/assets/skills/opentest/references/desktop-gui-testing.md +52 -0
  6. package/assets/skills/opentest/references/matrix-format.md +12 -2
  7. package/assets/skills/opentest/references/opentest-driven-development.md +10 -0
  8. package/assets/skills/opentest/references/test-asset-layout.md +64 -0
  9. package/assets/skills/opentest/references/test-surfaces.md +101 -0
  10. package/assets/skills/opentest/references/web-browser-testing.md +40 -0
  11. package/assets/skills/opentest/templates/acceptance-template.md +3 -1
  12. package/assets/skills/opentest/templates/api-acceptance-template.md +44 -0
  13. package/assets/skills/opentest/templates/desktop-gui-acceptance-template.md +43 -0
  14. package/assets/skills/opentest/templates/matrix-template.md +12 -11
  15. package/assets/skills/opentest/templates/plan-template.md +2 -2
  16. package/assets/skills/opentest/templates/web-acceptance-template.md +27 -0
  17. package/assets/skills/opentest-accept/SKILL.md +15 -6
  18. package/assets/skills/opentest-api/SKILL.md +25 -0
  19. package/assets/skills/opentest-author/SKILL.md +7 -5
  20. package/assets/skills/opentest-desktop-gui/SKILL.md +24 -0
  21. package/assets/skills/opentest-plan/SKILL.md +15 -9
  22. package/assets/skills/opentest-run/SKILL.md +14 -8
  23. package/assets/skills/opentest-web-browser/SKILL.md +26 -0
  24. package/assets/skills-zh/opentest/references/api-testing.md +77 -0
  25. package/assets/skills-zh/opentest/references/codex-harness-coverage-heuristics.md +17 -1
  26. package/assets/skills-zh/opentest/references/complete-testing-workflow.md +27 -0
  27. package/assets/skills-zh/opentest/references/desktop-gui-testing.md +52 -0
  28. package/assets/skills-zh/opentest/references/matrix-format.md +12 -2
  29. package/assets/skills-zh/opentest/references/opentest-driven-development.md +10 -0
  30. package/assets/skills-zh/opentest/references/test-asset-layout.md +64 -0
  31. package/assets/skills-zh/opentest/references/test-surfaces.md +101 -0
  32. package/assets/skills-zh/opentest/references/web-browser-testing.md +40 -0
  33. package/assets/skills-zh/opentest/templates/acceptance-template.md +3 -1
  34. package/assets/skills-zh/opentest/templates/api-acceptance-template.md +44 -0
  35. package/assets/skills-zh/opentest/templates/desktop-gui-acceptance-template.md +43 -0
  36. package/assets/skills-zh/opentest/templates/matrix-template.md +12 -11
  37. package/assets/skills-zh/opentest/templates/plan-template.md +2 -2
  38. package/assets/skills-zh/opentest/templates/web-acceptance-template.md +27 -0
  39. package/assets/skills-zh/opentest-accept/SKILL.md +14 -5
  40. package/assets/skills-zh/opentest-api/SKILL.md +25 -0
  41. package/assets/skills-zh/opentest-author/SKILL.md +7 -5
  42. package/assets/skills-zh/opentest-desktop-gui/SKILL.md +24 -0
  43. package/assets/skills-zh/opentest-plan/SKILL.md +13 -7
  44. package/assets/skills-zh/opentest-run/SKILL.md +13 -7
  45. package/assets/skills-zh/opentest-web-browser/SKILL.md +26 -0
  46. package/package.json +1 -1
  47. package/scripts/smoke-test.js +309 -1
@@ -1,13 +1,14 @@
1
1
  # Acceptance-to-Test Matrix
2
2
 
3
- | ID | Intent | Coverage dimension | Trigger/Input | Expected behavior | Risk | Evidence layer | Framework/command | Required evidence | Gap/blocker | Status |
4
- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
5
- | ACC-001 | create succeeds | create | valid fixture entity | entity is created and visible in list/detail | high | integration + acceptance | `python -m pytest` + real workflow | create evidence, UI/API/DB assertion, fixture used | none | pending |
6
- | ACC-002 | read/list/detail succeeds | read/list/detail | seeded entity | list, search/filter, and detail show correct data | high | integration + acceptance | `python -m pytest` + real workflow | read/list/detail evidence and data consistency check | none | pending |
7
- | ACC-003 | update succeeds | update | edited fixture entity | updated values persist after read back | high | integration + acceptance | `python -m pytest` + real workflow | update evidence and read-back assertion | none | pending |
8
- | ACC-004 | delete succeeds safely | delete | existing fixture entity | confirm/cancel behavior works; deleted item disappears after confirmation | high | integration + acceptance | `python -m pytest` + real workflow | delete evidence, cancel evidence, post-delete read check | none | pending |
9
- | ACC-005 | failure and boundary paths are handled | failure/boundary | invalid, empty, duplicate, unauthorized, or stale fixture data | clear feedback without corrupting data | high | unit/integration/acceptance | `python -m pytest` + acceptance | validation, permission, duplicate, stale-state evidence | none | pending |
10
- | ACC-006 | data consistency holds | data consistency | create/update/delete flow | UI, API, database/storage, files, and logs agree | high | integration | project command or `python -m pytest` | consistency evidence across surfaces | none | pending |
11
- | ACC-007 | end-to-end CRUD flow works | end-to-end CRUD | create -> list -> detail -> update -> read back -> delete | full user workflow completes and leaves clean state | high | E2E/acceptance | browser/API workflow | full-chain steps, screenshots/logs, cleanup evidence | none | pending |
12
- | ACC-008 | smoke gate passes | smoke | app starts and core entry points open | core route/API/CRUD happy path does not crash | high | smoke | project smoke command or targeted workflow | smoke_report path | none | pending |
13
- | ACC-009 | pre-push gate passes | pre-push | staged change before push | format/lint/type/unit/integration/smoke/diff checks pass or block push | high | pre-push | project command sequence | pre_push_report path | none | pending |
3
+ | ID | Requirement source | Intent | Execution surface | Acceptance mode | Coverage dimension | Trigger/Input | Expected behavior | Risk | Evidence layer | Framework/command | Required evidence | Gap/blocker | Status |
4
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
5
+ | ACC-001 | REQ-001 / user story | create succeeds in web UI | web-browser | instant-acceptance | create | valid fixture entity | entity is created and visible in list/detail | high | browser-acceptance + integration | Playwright MCP / Playwright CLI + project command | create evidence, UI/API/DB assertion, fixture used | none | pending |
6
+ | ACC-002 | REQ-WEB-REG-001 / regression need | web CRUD has durable regression | web-browser | durable-regression | read/list/detail + end-to-end CRUD | committed E2E test | create -> list -> detail -> update -> read back -> delete is repeatable | high | e2e | `npx playwright test` or project E2E command | test file, trace/report, stable locator assertions | none | pending |
7
+ | ACC-003 | REQ-WEB-VIS-001 / visual risk | visual web state is verified | web-browser | visual-ai-assist | visual behavior | weak selector or canvas UI | Midscene helps locate UI and deterministic read-back proves result | medium | visual-acceptance | Midscene + Playwright route | Midscene report, screenshot, read/assert result | none | pending |
8
+ | ACC-004 | REQ-ANDROID-001 / mobile flow | create succeeds in Android app | android-app | n/a | create | APK installed on emulator/device | task is created and visible after app refresh | high | visual-acceptance + e2e | `android-midscene-pytest` / `python -m pytest tests_py -v` | pytest report, ADB smoke, Midscene HTML report, screenshot, logcat, `midscene_run` logs | missing ADB/APK/device/package/model env if unavailable | pending |
9
+ | ACC-005 | REQ-DESKTOP-001 / settings workflow | desktop setting saves | desktop-gui | n/a | update | desktop app window is open | setting persists after save and reopen | high | gui-acceptance + integration | `opentest-desktop-gui` / project GUI automation / `@midscene/computer` | screenshot or recording, GUI action log, app/window metadata, deterministic read-back, Midscene/computer report when used | missing display/RDP/app launch/window identity/model env/result surface if unavailable | pending |
10
+ | ACC-006 | REQ-API-001 / contract | API create succeeds | api | n/a | create | valid request payload | response status and payload confirm created entity | high | contract + integration | `opentest-api` / project API command / `python -m pytest tests/api -v` | request/response record, status code, payload/schema assertion, read-after-write, data consistency, cleanup/teardown | missing base URL/auth/fixture/seed/teardown/dependency/schema/result surface if unavailable | pending |
11
+ | ACC-007 | Risk: invalid and unauthorized input | failure and boundary paths are handled | api | n/a | failure/boundary | invalid, empty, duplicate, unauthorized, expired, forbidden, or stale fixture data | clear error contract without corrupting data | high | contract + security-review | `opentest-api` / project command / `python -m pytest tests/api -v` | validation, auth/permission, duplicate/idempotency, stale-state, error schema, sensitive-field evidence | none | pending |
12
+ | ACC-007A | Risk: list contract drift | API list/search behavior is stable | api | n/a | list/filter/sort/pagination | seeded collection and query params | pagination, filtering, sorting, and empty result follow contract | medium | contract + integration | `opentest-api` / project API command | request/response record, schema assertion, pagination metadata, fixture isolation | not applicable if no list endpoint changed | pending |
13
+ | ACC-008 | Quality gate requirement | smoke gate passes | web-browser | instant-acceptance | smoke | app starts and core entry points open | core route/API/CRUD happy path does not crash | high | smoke | project smoke command or targeted workflow | smoke_report path | none | pending |
14
+ | ACC-009 | Delivery gate requirement | pre-push gate passes | api | n/a | pre-push | staged change before push | format/lint/type/unit/integration/smoke/diff checks pass or block push | high | pre-push | project command sequence | pre_push_report path | none | pending |
@@ -24,5 +24,5 @@
24
24
 
25
25
  ## Evidence Plan
26
26
 
27
- | Evidence layer | Applicable scenario | Command or execution surface | Artifact path | Status |
28
- | --- | --- | --- | --- | --- |
27
+ | Execution surface | Acceptance mode | Evidence layer | Applicable scenario | Command/tool | Artifact path | Status |
28
+ | --- | --- | --- | --- | --- | --- | --- |
@@ -0,0 +1,27 @@
1
+ # Web Browser Acceptance
2
+
3
+ - ACC ID:
4
+ - execution surface: web-browser
5
+ - acceptance mode: instant-acceptance | durable-regression | visual-ai-assist
6
+ - tool route: Playwright MCP | Playwright CLI | @playwright/test | Midscene
7
+ - page/route:
8
+ - actor/role:
9
+ - fixture:
10
+
11
+ ## Steps
12
+
13
+ 1. open:
14
+ 2. snapshot:
15
+ 3. fill/input:
16
+ 4. submit/confirm:
17
+ 5. snapshot after submit:
18
+ 6. read/assert changed result:
19
+ 7. screenshot/report:
20
+
21
+ ## Evidence
22
+
23
+ - status:
24
+ - changed result asserted:
25
+ - artifact paths:
26
+ - console/network notes:
27
+ - blocked reason:
@@ -5,19 +5,28 @@ description: "OpenTest phase 4: execute natural language, MCP, or real workflow
5
5
 
6
6
  # OpenTest Accept
7
7
 
8
- Execute required acceptance items and write PASS, FAIL, or blocked evidence back to cases and the matrix.
8
+ Write PASS, FAIL, or blocked evidence to cases and matrix.
9
9
 
10
10
  ## Required references
11
11
 
12
12
  - `opentest/references/acceptance-evidence.md`
13
13
  - `opentest/references/complete-testing-workflow.md`
14
+ - `opentest/references/test-surfaces.md`
15
+ - `opentest/references/web-browser-testing.md`
16
+ - `opentest/references/desktop-gui-testing.md`
17
+ - `opentest/references/api-testing.md`
14
18
  - `opentest/templates/acceptance-template.md`
19
+ - `opentest/templates/desktop-gui-acceptance-template.md`
20
+ - `opentest/templates/api-acceptance-template.md`
15
21
 
16
22
  ## Steps
17
23
 
18
24
  1. Read the matrix, fixtures, and `docs/opentest/acceptance/`.
19
- 2. For frontend interactions, prefer Chrome DevTools MCP and observe the real rendered UI.
20
- 3. For API/backend workflows, use project commands or direct API checks.
21
- 4. For CRUD/data changes, execute the full chain from the workflow reference.
22
- 5. Record feedback location/shape, artifacts, blocked evidence, and matching ACC IDs.
23
- 6. Update acceptance records and run `bash "$OPENTEST_GUARD" accept --apply`.
25
+ 2. Select the acceptance tool from the matrix execution surface.
26
+ 3. For `web-browser`, use `opentest-web-browser`: Playwright MCP/CLI, `@playwright/test` for durable regression, Midscene only for visual assist.
27
+ 4. For `android-app`, use `android-midscene-pytest`: `python -m pytest tests_py -v`, ADB smoke, Midscene HTML, logcat, and `midscene_run`; block missing prerequisites.
28
+ 5. For `desktop-gui`, use `opentest-desktop-gui`: project GUI automation or `@midscene/computer`, screenshot/recording, metadata, and read-back.
29
+ 6. For `api`, use `opentest-api`: project API command or `pytest` with `httpx`/`requests`, schema checks, fixtures, read-after-write, and cleanup/teardown.
30
+ 7. For CRUD/data changes, execute the full chain from the workflow reference.
31
+ 8. Record feedback location/shape, artifacts, blocked evidence, and matching ACC IDs.
32
+ 9. Update acceptance records and run `bash "$OPENTEST_GUARD" accept --apply`.
@@ -0,0 +1,25 @@
1
+ ---
2
+ name: opentest-api
3
+ description: "OpenTest API execution surface adapter. Use when planning, authoring, running, or accepting HTTP API, RPC, backend workflow, contract, auth, data consistency, idempotency, and integration evidence."
4
+ ---
5
+
6
+ # OpenTest API
7
+
8
+ Use this adapter for `api` matrix rows.
9
+
10
+ ## Required references
11
+
12
+ - `opentest/references/api-testing.md`
13
+ - `opentest/templates/api-acceptance-template.md`
14
+
15
+ ## Route
16
+
17
+ 1. Prefer the repository's existing API/integration test command when it is explicit and repeatable.
18
+ 2. If no project command exists, default to `pytest` with `httpx` or `requests`, schema checks via `jsonschema` or existing Pydantic/DTO models, and fixtures for seed/teardown.
19
+ 3. Use OpenAPI, protobuf, or existing contract docs as the contract source when present; otherwise write the expected status, headers, response schema, and business payload in the acceptance case.
20
+ 4. Mock or stub third-party APIs unless the requirement explicitly needs a live external dependency.
21
+ 5. Mark blocked when base URL, auth token, fixture data, seed/teardown, dependency service, or stable read-back surface is missing.
22
+
23
+ ## Evidence Contract
24
+
25
+ PASS requires request/response records, status code, response payload or schema assertion, auth/permission result when applicable, data consistency/read-after-write evidence, and cleanup or teardown proof. API smoke only proves liveness; it does not replace contract, boundary, permission, or data consistency evidence for high-risk changes.
@@ -11,14 +11,16 @@ Turn the matrix into executable tests, fixtures, seed/teardown notes, and accept
11
11
 
12
12
  - `opentest/references/opentest-driven-development.md`
13
13
  - `opentest/references/complete-testing-workflow.md`
14
+ - `opentest/references/test-asset-layout.md`
14
15
  - `opentest/templates/fixtures-template.md`
15
16
  - `opentest/templates/acceptance-template.md`
16
17
 
17
18
  ## Steps
18
19
 
19
20
  1. Read `matrix` and `fixtures` from `.opentest.yaml`.
20
- 2. For code evidence, use the project framework; if none exists, Default to pytest with tests under `tests/` runnable by `python -m pytest`.
21
- 3. Create/update fixtures, seed, teardown, users, roles, entities, files/images, and assertion surfaces.
22
- 4. For CRUD/data changes, author the full acceptance flow: create -> list -> detail -> update -> read back -> delete -> confirm absence -> teardown.
23
- 5. Record any gap/blocker with reason and risk.
24
- 6. Write `.opentest.yaml` fields: `fixtures`, `acceptance`, then run `bash "$OPENTEST_GUARD" author --apply`.
21
+ 2. Preserve each row's requirement source and expected behavior. Do not rewrite acceptance cases around current implementation names, component internals, or existing test files.
22
+ 3. Place assets in the fixed layout from `test-asset-layout.md`; default to pytest under `tests/` when no project framework exists. Missing implementation means evidence stays pending.
23
+ 4. Create/update fixtures, seed, teardown, users, roles, entities, files/images, and assertion surfaces.
24
+ 5. For CRUD/data changes, author the full acceptance flow: create -> list -> detail -> update -> read back -> delete -> confirm absence -> teardown.
25
+ 6. Record any gap/blocker with reason and risk.
26
+ 7. Write `.opentest.yaml` fields: `fixtures`, `acceptance`, then run `bash "$OPENTEST_GUARD" author --apply`.
@@ -0,0 +1,24 @@
1
+ ---
2
+ name: opentest-desktop-gui
3
+ description: "OpenTest desktop GUI execution surface adapter. Use when planning, authoring, running, or accepting native desktop, Electron, Tauri, Windows, macOS, Linux, or RDP workflows and deciding between project GUI automation, @midscene/computer, accessibility metadata, screenshots, and scripted manual acceptance."
4
+ ---
5
+
6
+ # OpenTest Desktop GUI
7
+
8
+ Use this adapter for `desktop-gui` matrix rows.
9
+
10
+ ## Required references
11
+
12
+ - `opentest/references/desktop-gui-testing.md`
13
+ - `opentest/templates/desktop-gui-acceptance-template.md`
14
+
15
+ ## Route
16
+
17
+ 1. Prefer the repository's existing GUI automation command when it is explicit and repeatable.
18
+ 2. Use `@midscene/computer` for visual desktop automation, weak selectors, native controls, cross-window flows, or remote Windows RDP when deterministic project automation is unavailable.
19
+ 3. For Electron or Tauri apps that expose a browser context, prefer `web-browser` or Playwright routes for DOM-verifiable flows, and use `desktop-gui` only for native shell, tray, file picker, menu, OS dialog, or multi-window behavior.
20
+ 4. If model credentials, desktop access, display/RDP availability, app launch command, or stable result surface is missing, record `blocked` instead of PASS.
21
+
22
+ ## Evidence Contract
23
+
24
+ PASS requires the executed steps, screenshot or screen recording, window/app metadata, GUI action log, and a deterministic read-back result after writes. `@midscene/computer` assertions are visual assistance; they must not replace persisted-result checks such as reopening a settings screen, reading a file/config value, querying an app state surface, or confirming the expected window state after restart.
@@ -1,27 +1,33 @@
1
1
  ---
2
2
  name: opentest-plan
3
- description: "OpenTest phase 1: analyze the change, risks, project facts, and create a test strategy plus acceptance-to-test matrix."
3
+ description: "OpenTest phase 1: create test strategy and acceptance matrix."
4
4
  ---
5
5
 
6
6
  # OpenTest Plan
7
7
 
8
- Create `docs/opentest/plans/`, `docs/opentest/matrices/`, and a fixtures plan before implementation.
8
+ Create plan, matrix, and fixtures before implementation.
9
9
 
10
10
  ## Required references
11
11
 
12
12
  - `opentest/references/codex-harness-coverage-heuristics.md`
13
13
  - `opentest/references/matrix-format.md`
14
14
  - `opentest/references/complete-testing-workflow.md`
15
+ - `opentest/references/test-asset-layout.md`
16
+ - `opentest/references/test-surfaces.md`
17
+ - `opentest/references/web-browser-testing.md`
18
+ - `opentest/references/desktop-gui-testing.md`
19
+ - `opentest/references/api-testing.md`
15
20
 
16
21
  ## Steps
17
22
 
18
- 1. Read project rules, requirements/design/diff, existing commands, and detection output.
19
- 2. Classify risk, applicable coverage, test data, and whether code tests use an existing framework or default `pytest`.
20
- 3. Apply the CRUD baseline and test data requirements from the workflow reference for data-writing/API/form/file/stateful changes.
21
- 4. Produce a matrix with coverage dimension, framework/command, required evidence, gap/blocker, and status.
22
- 5. Write `.opentest.yaml` fields: `plan`, `matrix`, `fixtures`.
23
- 6. Update handoff if present, then run `bash "$OPENTEST_GUARD" plan --apply`.
23
+ 1. Read rules, requirements/diff, commands, and detection output.
24
+ 2. Treat requirements and risks as sources; inspect current code only for project facts.
25
+ 3. Apply CRUD baseline and test data rules for data-writing/API/form/file/stateful changes.
26
+ 4. Classify execution surface and evidence layer separately: `web-browser`, `android-app`, `desktop-gui`, or `api`; use web modes, `opentest-desktop-gui`, and `opentest-api` where applicable.
27
+ 5. Produce a requirement-first matrix with source, behavior, surface, mode, evidence, command, gap/blocker, status, and fixed layout.
28
+ 6. Write `.opentest.yaml` fields: `plan`, `matrix`, `fixtures`.
29
+ 7. Update handoff if present, then run `bash "$OPENTEST_GUARD" plan --apply`.
24
30
 
25
31
  ## Gate
26
32
 
27
- Every applicable behavior, failure path, boundary, and risk surface needs evidence or a written gap/blocker. CRUD baseline and test data are default requirements unless the matrix says why they are not applicable.
33
+ Every behavior, failure path, boundary, and risk needs evidence or gap/blocker. Every row cites a source and includes surface plus evidence layer; web rows include acceptance mode. Do not use unit/component/integration/contract/smoke as the execution surface. Do not drop or narrow acceptance because current code has no matching file.
@@ -5,12 +5,17 @@ description: "OpenTest phase 3: run project verification commands in targeted, f
5
5
 
6
6
  # OpenTest Run
7
7
 
8
- Run matrix-driven commands and write evidence reports under `docs/opentest/runs/`.
8
+ Run matrix-driven commands and write reports under `docs/opentest/runs/`.
9
9
 
10
10
  ## Required references
11
11
 
12
12
  - `opentest/references/command-routing.md`
13
13
  - `opentest/references/complete-testing-workflow.md`
14
+ - `opentest/references/test-asset-layout.md`
15
+ - `opentest/references/test-surfaces.md`
16
+ - `opentest/references/web-browser-testing.md`
17
+ - `opentest/references/desktop-gui-testing.md`
18
+ - `opentest/references/api-testing.md`
14
19
 
15
20
  ## Modes
16
21
 
@@ -22,10 +27,11 @@ Run matrix-driven commands and write evidence reports under `docs/opentest/runs/
22
27
 
23
28
  ## Steps
24
29
 
25
- 1. Read `run_mode`, matrix, fixtures, and required evidence.
26
- 2. Prefer explicit project commands; otherwise use `python -m pytest` for code-level tests.
27
- 3. For coverage, prefer `python -m pytest --cov=. --cov-report=term-missing`.
28
- 4. Smoke evidence is required unless the matrix says not applicable.
29
- 5. For `pre-push`, run or record format/check, lint, type, unit, targeted integration, smoke, and `git diff --check`.
30
- 6. Write `run_report`, plus `coverage_report`, `smoke_report`, or `pre_push_report` when produced/required.
31
- 7. Run `bash "$OPENTEST_GUARD" run --apply`.
30
+ 1. Read `run_mode`, matrix, fixtures, required evidence, and fixed asset layout.
31
+ 2. Choose by matrix execution surface: MCP/CLI or `npx playwright test` for `web-browser`; `python -m pytest tests_py -v` for `android-app`, with `npm run test:android` only when model env is ready or debugging Midscene; `opentest-desktop-gui` for `desktop-gui`; `opentest-api` or `python -m pytest tests/api -v` for `api`.
32
+ 3. Prefer explicit project commands; otherwise use `python -m pytest` for code-level tests.
33
+ 4. For coverage, prefer `python -m pytest --cov=. --cov-report=term-missing`.
34
+ 5. Smoke evidence is required unless the matrix says not applicable.
35
+ 6. For `pre-push`, run or record format/check, lint, type, unit, targeted integration, smoke, and `git diff --check`.
36
+ 7. Write `run_report`, `coverage_report`, `smoke_report`, or `pre_push_report` when required.
37
+ 8. Run `bash "$OPENTEST_GUARD" run --apply`.
@@ -0,0 +1,26 @@
1
+ ---
2
+ name: opentest-web-browser
3
+ description: "OpenTest web-browser execution surface adapter. Use when planning, authoring, running, or accepting browser-rendered web workflows and deciding between Playwright MCP, Playwright CLI, @playwright/test, and Midscene."
4
+ ---
5
+
6
+ # OpenTest Web Browser
7
+
8
+ Use this adapter for `web-browser` matrix rows.
9
+
10
+ ## Required references
11
+
12
+ - `opentest/references/web-browser-testing.md`
13
+ - `opentest/references/complete-testing-workflow.md`
14
+ - `opentest/templates/web-acceptance-template.md`
15
+
16
+ ## Steps
17
+
18
+ 1. Decide `acceptance_mode`: `instant-acceptance`, `durable-regression`, or `visual-ai-assist`.
19
+ 2. For `instant-acceptance`, use Playwright MCP first; if browser MCP is unavailable or unstable, use Playwright CLI.
20
+ 3. For `durable-regression`, write or run `@playwright/test` or the repository's existing E2E framework.
21
+ 4. Use Midscene only for visual, weak-selector, canvas, cross-frame, or other UI surfaces selectors cannot prove well.
22
+ 5. For writes, run the full chain: open -> snapshot -> fill/input -> submit -> snapshot -> read/assert changed result -> screenshot -> PASS/FAIL.
23
+
24
+ ## Gate
25
+
26
+ MCP or Playwright CLI evidence proves this acceptance run only. Do not count it as durable regression unless the repository has a committed repeatable test, command, and report path. Midscene must not replace read-after-write assertions.
@@ -0,0 +1,77 @@
1
+ # API 测试
2
+
3
+ 用于 `api` 执行面的矩阵行。
4
+
5
+ ## 默认架构
6
+
7
+ 优先使用仓库已有 API 测试框架。项目没有明确 API 测试命令时,默认采用:
8
+
9
+ ```text
10
+ pytest
11
+ -> httpx 或 requests client
12
+ -> pytest fixtures 管 seed/teardown
13
+ -> jsonschema 或项目已有 Pydantic/DTO 模型做契约断言
14
+ -> 可选 DB/存储/日志回读
15
+ -> pytest report 或 JUnit XML
16
+ ```
17
+
18
+ 依赖需要隔离时,用 Docker Compose、testcontainers 或仓库已有本地服务启动方式。第三方 API 默认 mock/stub;打 live 外部服务必须有明确需求和记录的风险说明。
19
+
20
+ 稳定 API 资产放入 `opentest/references/test-asset-layout.md` 约定的 `tests/api/`:client 放 `tests/api/clients/`,fixtures 放 `tests/api/fixtures/`,schemas 放 `tests/api/schemas/`,可重复入口使用 `scripts/opentest-run-api.ps1` 或仓库等价命令。
21
+
22
+ ## 证据层级
23
+
24
+ | 层级 | 证明内容 | 常见命令/工具 |
25
+ | --- | --- | --- |
26
+ | contract | 状态码、headers、响应字段、schema、错误形态 | 项目 contract tests、`pytest`、OpenAPI 检查 |
27
+ | integration | API handler、service、数据库/存储、队列/日志副作用 | 项目 integration 命令、`pytest`、本地服务 |
28
+ | smoke | base URL 和关键 endpoint 存活 | 项目 smoke 命令、小型 `pytest`/curl 脚本 |
29
+ | security-review | 鉴权、授权、敏感字段泄漏、注入风险 | 定向测试 + review 记录 |
30
+
31
+ ## 必需 API 用例
32
+
33
+ API 变更应按适用性写入矩阵:
34
+
35
+ - 主路径:期望状态码、payload、headers 和业务状态
36
+ - 校验失败:非法、空、边界、格式错误、不支持字段
37
+ - 鉴权和权限:未登录、token 过期、角色错误、对象级授权
38
+ - 未找到和过期状态:资源不存在、已删除资源、stale version
39
+ - 冲突和幂等:重复创建、重复提交、带 idempotency key 的 retry、并发修改
40
+ - 适用时的限流或节流
41
+ - 列表 endpoint 变更时的分页、过滤、排序和空结果
42
+ - 数据一致性:响应、DB/存储、事件、队列消息、文件或日志
43
+ - teardown/cleanup:创建资源已删除,或隔离 fixture namespace 已重置
44
+
45
+ ## 契约来源
46
+
47
+ 契约来源优先级:
48
+
49
+ 1. 仓库提交的 OpenAPI/protobuf/schema 文件。
50
+ 2. 既有 request/response DTO、Pydantic model、serializer 或 typed client。
51
+ 3. 明确写出字段和错误的需求/设计文档。
52
+ 4. 验收用例中手写 schema。
53
+
54
+ 当当前实现行为与需求冲突时,不得只按实现反推契约。
55
+
56
+ ## 阻塞规则
57
+
58
+ 缺少任一必需前置条件时记录 `blocked`:
59
+
60
+ - base URL 或服务启动命令
61
+ - auth token、角色或测试用户
62
+ - fixture seed/teardown 路径
63
+ - 依赖服务、数据库、队列或 mock server
64
+ - 稳定契约来源或期望 schema
65
+ - 确定性写后读结果面
66
+
67
+ 不得只凭 2xx 响应把 API 验收标为 PASS。写操作必须包含可信结果面的写后读证据,例如 API 查询 endpoint、DB/存储记录、队列/事件/日志,或另一个项目自有状态面。
68
+
69
+ ## 矩阵要求
70
+
71
+ `api` 行必须包含:
72
+
73
+ - `执行面`:`api`
74
+ - `验收模式`:`n/a`
75
+ - `证据层级`:`contract`、`integration`、`smoke` 或 `security-review`
76
+ - `框架/命令`:项目 API 命令、`python -m pytest tests/api -v`、curl/httpie 脚本、项目已使用时的 Postman/Newman,或契约工具
77
+ - `必需证据`:请求/响应记录、状态码、payload/schema 断言、适用时的鉴权/权限断言、写后读/数据一致性,以及 cleanup/teardown 证明
@@ -13,6 +13,9 @@ OpenTest 应根据变更类型、风险和项目事实选择适用覆盖面。
13
13
  - `tdd-workflow`
14
14
  - `e2e-runner`
15
15
  - `browser-e2e-testing`
16
+ - `android-midscene-pytest`
17
+ - `opentest-desktop-gui`
18
+ - `opentest-api`
16
19
  - `verification-loop`
17
20
  - `code-reviewer`
18
21
  - `speckit-checklist`
@@ -28,6 +31,19 @@ OpenTest 应根据变更类型、风险和项目事实选择适用覆盖面。
28
31
  | 权限、支付、安全、数据写入、跨页面闭环 | high-risk 验收或 E2E 证据 |
29
32
  | 文案、配置、小范围无行为变更 | targeted review 或轻量证据 |
30
33
 
34
+ ## 执行面选择
35
+
36
+ 执行面和证据层级必须分开选择:
37
+
38
+ | 执行面 | 默认路由 |
39
+ | --- | --- |
40
+ | `web-browser` | Chrome DevTools MCP、Playwright CLI 或浏览器验收 |
41
+ | `android-app` | 已安装时使用 `android-midscene-pytest`;`python -m pytest tests_py -v` 通过 ADB 驱动 Midscene Android |
42
+ | `desktop-gui` | `opentest-desktop-gui`;优先项目 GUI 自动化,视觉/原生/RDP GUI 流程用 `@midscene/computer`,无自动化时才用脚本化人工 GUI 验收 |
43
+ | `api` | `opentest-api`;优先项目 API/integration 命令,否则用 `pytest` + `httpx`/`requests`、schema 校验、fixtures、写后读和 cleanup/teardown |
44
+
45
+ 不得把 unit、component、integration、contract、smoke 或 security review 这类代码检查归类成执行面;它们只能作为证据层级。
46
+
31
47
  ## 前端验收维度
32
48
 
33
49
  前端或真实链路验收可从以下维度中选择适用项,不要求每次全部覆盖:
@@ -74,7 +90,7 @@ OpenTest plan 阶段默认检查以下问题;适用则进入矩阵,不适用
74
90
  | ACC-001 | 用户保存后看到成功反馈 | medium | UI 验收 | pending |
75
91
  ```
76
92
 
77
- 只有当风险或变更类型需要时,才增加覆盖维度、命令、证据路径和阻塞原因列。
93
+ 当矩阵需要驱动验收执行或风险要求时,再增加执行面、证据层级、命令/工具、证据路径和阻塞原因列。
78
94
 
79
95
  ## 质量门启发
80
96
 
@@ -8,6 +8,33 @@
8
8
  plan -> matrix -> fixtures -> tests -> run -> accept -> smoke -> pre-push -> verify -> archive
9
9
  ```
10
10
 
11
+ ## 需求先行验收
12
+
13
+ `plan` 和 `author` 发生在实现前,必须把需求转成验收契约。来源只能是需求、设计说明、用户流程、业务规则和风险边界。当前代码只用于发现执行事实,例如命令、已有框架、路由、fixtures 和可复用 helper。
14
+
15
+ unit、component、integration、contract、E2E、smoke、browser acceptance 都是证据层级。它们描述实现期间或实现后如何证明需求,不能决定需求是什么。如果代码还没出现,保留验收用例,并把依赖代码的证据标成 pending 或 blocked 且写明原因。
16
+
17
+ ## 执行面
18
+
19
+ 每条矩阵行必须同时写执行面和证据层级。执行面表示需求从哪里被实际操作;证据层级表示如何证明结果。
20
+
21
+ 编写测试前必须先按 `opentest/references/test-asset-layout.md` 选定固定资产目录。默认使用方案 B 的标准测试框架骨架;一次性脚本只能用于明确的非稳定验收或 blocked 排查。
22
+
23
+ 主执行面只有:
24
+
25
+ - `web-browser`:浏览器渲染页面和 Web App
26
+ - `android-app`:模拟器或真机上的 Android APK/App GUI
27
+ - `desktop-gui`:原生桌面 GUI、Electron、Tauri 或类似 App UI
28
+ - `api`:HTTP API、RPC、后端工作流、契约或服务端点
29
+
30
+ 不得把 unit、component、integration、contract、smoke 或 security review 当成执行面;它们是证据层级或运行门禁。如果存在 Android GUI 需求,已安装 `android-midscene-pytest` 时用它执行验收,并要求 pytest/Midscene/截图/logcat 证据。如果存在原生桌面 GUI 行为,用 `opentest-desktop-gui` 执行验收,并要求项目 GUI 自动化或 `@midscene/computer` 证据,加上截图、GUI 操作日志、窗口/App 元数据和确定性回读。如果存在 API 行为,用 `opentest-api` 执行验收,并按适用性要求契约、状态码、payload/schema、鉴权/权限、写后读和 cleanup/teardown 证据。
31
+
32
+ `web-browser` 必须按 `opentest/references/web-browser-testing.md` 选择验收模式。MCP 和 Playwright CLI 是现场验收路径;稳定回归必须有已提交、可重复运行的测试,例如 `@playwright/test`。
33
+
34
+ `desktop-gui` 使用 `opentest/references/desktop-gui-testing.md`。Electron/Tauri 的 DOM 可验证流程可以保留在 `web-browser`;原生外壳、托盘、文件选择器、菜单、系统弹窗、安装器、更新器、RDP 和多窗口行为保留在 `desktop-gui`。
35
+
36
+ `api` 使用 `opentest/references/api-testing.md`。优先项目 API/integration 命令;没有项目命令时,用 `pytest` + `httpx` 或 `requests`、schema 校验、fixtures 和确定性回读。
37
+
11
38
  ## 测试数据
12
39
 
13
40
  变更涉及数据、文件、角色、权限、API 或有状态流程时,创建 `docs/opentest/fixtures/`。
@@ -0,0 +1,52 @@
1
+ # 桌面 GUI 测试
2
+
3
+ 用于 `desktop-gui` 执行面的矩阵行。
4
+
5
+ 稳定桌面 GUI 资产遵循 `opentest/references/test-asset-layout.md`:脚本放 `tests/desktop/scripts/`,Midscene 资产放 `tests/desktop/midscene/`,元数据采集放 `tests/desktop/metadata/`,可重复入口使用 `scripts/opentest-run-desktop.ps1` 或项目 GUI 命令。
6
+
7
+ ## 工具路线
8
+
9
+ | 路线 | 适用场景 | 必需证据 |
10
+ | --- | --- | --- |
11
+ | 项目 GUI 自动化 | 仓库已有可重复的桌面自动化命令 | 命令、报告/日志、截图或录屏、操作后回读 |
12
+ | `@midscene/computer` | 原生桌面控件、弱选择器、视觉流程、多窗口流程,或 Windows RDP 需要 AI 视觉辅助 | Midscene/computer 运行日志、截图、模型环境状态、窗口/App 元数据、确定性回读 |
13
+ | 可访问性/窗口元数据 | 原生控件能暴露稳定可访问性树、标题、进程、窗口句柄或菜单状态 | 元数据 dump、操作日志、期望状态断言 |
14
+ | 脚本化人工 GUI 验收 | 没有可靠自动化且验收是一次性的 | 精确步骤、截图、窗口/App 元数据、观察结果、阻塞/风险说明 |
15
+
16
+ ## Midscene 桌面路线
17
+
18
+ `@midscene/computer` 是 Midscene 的桌面自动化包,可控制本地 Windows、macOS 和 Linux 桌面,也可在配置后通过 RDP 控制远程 Windows 桌面。
19
+
20
+ 在 OpenTest 中把它作为视觉自动化层,而不是整个质量门:
21
+
22
+ ```text
23
+ desktop-gui 矩阵行
24
+ -> 项目启动 / 环境检查
25
+ -> @midscene/computer 或项目 GUI 自动化
26
+ -> 截图 + GUI 操作日志 + 窗口/App 元数据
27
+ -> 确定性 read/assert changed result
28
+ ```
29
+
30
+ Electron 或 Tauri 先判断需求是否能用 DOM 验证。DOM 可验证流程归入 `web-browser`;原生外壳、托盘、文件选择器、原生菜单、系统弹窗、安装器、更新器和多窗口行为归入 `desktop-gui`。
31
+
32
+ ## 阻塞规则
33
+
34
+ 缺少任一必需前置条件时记录 `blocked`:
35
+
36
+ - Midscene 视觉自动化的模型凭据
37
+ - 桌面访问、显示器或 RDP 会话
38
+ - App 启动命令或目标进程/窗口标识
39
+ - 稳定 fixture 数据或 reset/teardown 路径
40
+ - 写操作后的确定性结果面
41
+
42
+ 不得只凭 AI 视觉断言把 `desktop-gui` 验收标为 PASS。新增/修改/删除/保存后,必须重新读取可信结果面:重开的窗口状态、文件/配置值、App 存储/API、可访问性元数据、进程/窗口元数据,或重启后仍可见的持久化值。
43
+
44
+ ## 矩阵要求
45
+
46
+ `desktop-gui` 行必须包含:
47
+
48
+ - `执行面`:`desktop-gui`
49
+ - `验收模式`:`n/a`
50
+ - `证据层级`:`gui-acceptance`、`visual-acceptance`、`integration` 或 `smoke`
51
+ - `框架/命令`:项目 GUI 命令、`@midscene/computer`、可访问性/窗口元数据脚本,或脚本化人工 GUI 路线
52
+ - `必需证据`:截图或录屏、GUI 操作日志、窗口/App 元数据、确定性回读,以及不可用时的 blocked 前置条件
@@ -2,8 +2,14 @@
2
2
 
3
3
  ## 最小列
4
4
 
5
- | ID | 意图 | 触发/输入 | 期望行为 | 风险 | 证据层级 | 必需证据 | 状态 |
6
- | --- | --- | --- | --- | --- | --- | --- | --- |
5
+ | ID | 需求来源 | 意图 | 执行面 | 验收模式 | 触发/输入 | 期望行为 | 风险 | 证据层级 | 必需证据 | 状态 |
6
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
7
+
8
+ `需求来源` 是必填项。可填写需求 ID、设计章节、用户故事、业务规则、风险说明、issue 或用户明确请求。不要把函数名、组件路径或已有测试文件当成验收来源。
9
+
10
+ `执行面` 是必填项,应使用 `web-browser`、`android-app`、`desktop-gui` 或 `api` 之一。
11
+
12
+ `web-browser` 行必须填写 `验收模式`:`instant-acceptance`、`durable-regression` 或 `visual-ai-assist`。
7
13
 
8
14
  ## 可选列
9
15
 
@@ -17,6 +23,8 @@
17
23
 
18
24
  ## 证据层级
19
25
 
26
+ 证据层级只描述如何证明需求,不能产生或限制需求本身。
27
+
20
28
  - `unit`:纯函数、校验规则、状态计算。
21
29
  - `component`:表单反馈、按钮状态、局部 UI 状态。
22
30
  - `integration`:模块协作、API client、状态管理、mock server。
@@ -24,4 +32,6 @@
24
32
  - `e2e`:跨页面、登录、权限、关键业务闭环。
25
33
  - `smoke`:关键页面或主路径不崩。
26
34
  - `browser-acceptance`:真实浏览器交互、反馈位置、响应式和视觉状态。
35
+ - `visual-acceptance`:Android App 或桌面 GUI 执行面上的视觉 GUI 行为。
36
+ - `gui-acceptance`:桌面 GUI 行为、窗口状态、弹窗、菜单和原生控件。
27
37
  - `security-review`:权限、敏感信息、越权、重复提交、注入风险。
@@ -16,6 +16,14 @@ OpenTest 驱动开发不是传统 TDD 的替代名词。它把 TDD 放进更大
16
16
 
17
17
  这些场景需要先进入验收到测试矩阵,再决定证据层级。
18
18
 
19
+ ## 需求先行契约
20
+
21
+ OpenTest 的 `plan` 和 `author` 阶段是在实现前生成需求验收契约。验收用例必须来自需求、设计说明、用户流程、业务规则、风险边界和预期交互反馈。
22
+
23
+ 可以读取当前代码来发现项目事实,例如已有测试框架、命令、路由、fixtures 或可复用 helper。当前代码不能决定某个需求是否需要验收、用户可见行为应该是什么,或某个需求是否可以删除。
24
+
25
+ `unit`、`component`、`integration`、`contract`、`e2e` 等证据层级只描述实现期间或实现后如何证明需求,不是需求来源。如果代码还没出现,矩阵仍然要记录必需行为,并把依赖实现的证据标为 pending。
26
+
19
27
  ## 推荐顺序
20
28
 
21
29
  ```text
@@ -42,6 +50,8 @@ OpenTest 驱动开发不是传统 TDD 的替代名词。它把 TDD 放进更大
42
50
 
43
51
  ## 输出要求
44
52
 
53
+ - 每条矩阵行必须引用需求来源,例如需求 ID、设计章节、用户故事、业务规则、风险说明或用户明确请求。
54
+ - 验收表述必须与实现无关:描述用户可观察行为和业务结果,不描述当前函数名、组件内部结构或已有测试文件。
45
55
  - 每个 required 场景必须有证据层级和执行面。
46
56
  - blocked 不能直接等于 pass;必须写原因和恢复路径。
47
57
  - high-risk 场景缺证据默认 fail,除非用户明确接受风险并写明理由。