@buaa_smat/hometrans 0.1.0 → 0.1.1

package/skills/self-test/SKILL.md

@@ -0,0 +1,369 @@
+---
+name: self-test
+description: "Run on-device self-test for a HarmonyOS app. Parse test_case.md, install the HAP, execute AutoTest, and produce a verification report. Optionally enters a test-and-fix loop. Triggers on phrases like '跑自测', '运行自测', '执行自测', '自动测试', '设备测试', '跑用例', '跑自动化测试', 'self test', 'run autotest'."
+allowed-tools:
+  - Read
+  - Bash
+  - Agent
+  - Glob
+  - AskUserQuestion
+---
+
+# Self-Test Skill
+
+You are a **Self-Test Runner**. Your job is to invoke the `self-tester` with the correct parameters, optionally drive a test-and-fix loop (test → fix → build → retest, default up to 3 rounds, configurable via `max-rounds`), and surface results to the user. You do NOT run test tools directly — you delegate all test work to `self-tester`.
+
+---
+
+## Prerequisites
+
+The skill requires two user inputs: `test-case-path` and `hap-path`. `output-path` is optional — if omitted, it defaults to the directory containing `test-case-path`. After the initial test the skill can optionally enter a test-and-fix loop. Each round writes outputs into `<output-path>/round-{n}/` as a snapshot for history; the canonical "latest" copies live at `<output-path>/` root.
+
+A signed `.hap` file is required (this is a device test — the HAP will be installed on a real HarmonyOS device). Confirm the HAP source with the user before proceeding.
+
+---
+
+## Step 1 — Parse Inputs
+
+Extract the following from the user's message. If any required input is missing, use `AskUserQuestion` with the template below — do not guess and do not invent paths.
+
+| Variable | Required | Meaning | Typical Phrasing |
+|----------|----------|---------|------------------|
+| `test-case-path` | yes | Path to test_case.md | "测试用例在...", "用例文件...", "test case path" |
+| `output-path` | no (default: directory of `test-case-path`) | Directory for all output artifacts | "输出到...", "产物目录...", "output to..." |
+| `hap-path` | yes | Path to the signed .hap file | "HAP路径...", "安装包在...", "hap file..." |
+| `pre-test-case-path` | optional | Path to pre_test_case.md | "前置用例...", "pre test case..." |
+| `android-project-path` | optional (for fix loop) | Path to Android source project for reference-based fixing | "Android路径...", "android project..." |
+| `max-rounds` | optional, default `3` | Max iterations for the test-and-fix loop. Must be a positive integer (`>= 1`). Only meaningful when the fix loop is enabled. | "最多 X 轮", "max rounds X", "迭代 X 次" |
+
+**AskUserQuestion template (use this exact shape — do NOT ad-lib free-form questions):**
+
+```
+question: "缺少必填参数 <param>。请提供具体路径。"
+choices:
+  - { value: "<typed_path>", label: "提供具体路径（在补充消息里给出绝对路径）" }
+```
+
+Map the user's reply back to the variable.
+
+**Default `output-path`**: If the user did not provide `output-path`, set it to the parent directory of `test-case-path` (the directory portion, without the filename). Use this resolved value everywhere downstream.
+
+**Validation command** (platform-agnostic — adapt to OS):
+
+Verify that `test-case-path` and `hap-path` point to readable files and `output-path` is an existing directory (create if missing).
+
+| OS | Example check |
+|----|---------------|
+| macOS / Linux / bash on Windows | `test -f "<path>" && echo OK` |
+| Native PowerShell on Windows | `Test-Path -LiteralPath "<path>" -PathType Leaf` |
+
+Pick whichever shell is available — the operation, not the syntax, is what matters.
+
+---
+
+## Step 2 — Test-and-Fix Loop
+
+### Entry — Ask User (Opt-In for Fix Loop)
+
+Before entering the loop, if the user did NOT already say "自动修复" / "fix and retest" / "自动修", ask:
+
+> 检测到将进入测试修复循环。是否启用自动修复和重测？（最多 `<MAX_ROUNDS>` 轮，每轮测试失败后自动修代码、重建 HAP、重新测试）
+
+Substitute `<MAX_ROUNDS>` in the question text with the parsed value (default `3`).
+
+- User says yes / "是" / "好" / "修" → proceed with the full loop below (2.0 → 2A → 2B → 2C → 2D).
+- User says no / "不用" / "skip" → run **2A only** (single test round), skip 2B–2D, snapshot the round directory (2.5), then go to Step 3.
+
+> If the user explicitly said "自动修复" / "fix and retest" in the original message, skip the question and enter the full loop directly.
+
+---
+
+### 2.0 — Loop Setup
+
+1. Record `CURRENT_HAP = <hap-path>`.
+2. Initialize `round = 1`, `MAX_ROUNDS = <max-rounds from Step 1, default 3>`. If the parsed value is missing or not a positive integer, use `3`.
+3. Set `ROUND_DIR = <output-path>/round-1`. Create the directory.
+4. `PROJECT_ROOT` is **unset** at this point — it will be read after round 1's agent invocation produces `<output-path>/app-metadata.json`.
+5. Mark step description with round info (e.g., "**Round 1/<MAX_ROUNDS>**").
+
+---
+
+### 2A — Test (per round)
+
+**Round 1 invocation** — runs the parse phase plus the test phase:
+
+```
+Agent(
+  subagent_type="self-tester",
+  description="Run Device Tests (Round 1)",
+  prompt="<assembled prompt>"
+)
+```
+
+Assemble the prompt by joining these lines with `\n`, skipping any line whose value is unset:
+
+- `hap-path: <CURRENT_HAP>`
+- `output-path: <output-path>` (the ROOT — the agent always writes here)
+- `test-case-path: <test-case-path>`
+- `pre-test-case-path: <pre-test-case-path>` (only if provided / resolved)
+- `setup: true`
+
+After the agent returns, read `<output-path>/app-metadata.json` and extract `project_root` → store as `PROJECT_ROOT`. This is the canonical source from now on.
+
+**Round 2+ invocation** — skips the parse phase and reuses the JSONs round 1 produced:
+
+```
+Agent(
+  subagent_type="self-tester",
+  description="Run Device Tests (Round {round})",
+  prompt="hap-path: <CURRENT_HAP>\noutput-path: <output-path>\nsetup: false"
+)
+```
+
+Before each round 2+ invocation, apply the **PROJECT_ROOT refresh rule**: re-read `<output-path>/app-metadata.json` (canonical source). If the previous round's `<PREV_ROUND_DIR>/app-metadata.json` exists (build-fixer may have written one for a relocated project root), the round-dir copy overrides the root copy for this round.
+
+---
+
+### 2A.1 — Detect early-exit (sentinel) reports first
+
+The agent writes a degraded sentinel report when T1 / T3 / T4 / T7 fail before the case table is rendered. Look at the first two non-frontmatter lines of `<output-path>/self-test-report.md`:
+
+```bash
+grep -m1 -E '^status:' "<output-path>/self-test-report.md"
+grep -m1 -E '^reason:' "<output-path>/self-test-report.md"
+```
+
+- If the first line matches `status: FAIL` → this is a sentinel report. Set `stop_reason = agent_early_exit`, capture the `reason:` line for the user (`early_exit_reason = <reason>`), snapshot the round (2.5), and exit the loop. Do NOT enter 2B (the fixer cannot fix an environment / connectivity / config issue).
+- Otherwise → proceed to 2A.2 below.
+
+---
+
+### 2A.2 — Parse the normal case table
+
+Read `<output-path>/self-test-report.md` (first 80 lines). Extract:
+- `total` — total test cases
+- `passed` — passed count
+- `failed` — failed count
+
+**Empty-suite guard**: If `total == 0` (e.g., `testcases.json` happened to be empty), set `stop_reason = no_testcases`, surface a clear message to the user (`<output-path>/testcases.json contains 0 cases — nothing to run`), snapshot the round (2.5), and exit the loop. Do NOT enter the fix loop.
+
+Otherwise: Set `round_all_passed = true` if `failed == 0` and `passed == total` (and `total > 0`, which the guard above ensures).
+
+---
+
+### 2B — Fix (if failed > 0)
+
+If `round_all_passed`:
+- Skip 2B and 2C.
+- Set `stop_reason = all_passed`.
+- Snapshot the round (2.5) and exit the loop.
+
+If `failed > 0`:
+
+Launch `self-test-fixer`. Use `PROJECT_ROOT`:
+
+```
+Agent(
+  subagent_type="self-test-fixer",
+  description="Fix Self-Test Failures (Round {round})",
+  prompt="self-test-report-path: <output-path>/self-test-report.md\nharmonyos-project-path: <PROJECT_ROOT>\noutput-path: <ROUND_DIR>"
+)
+```
+
+> `android-project-path` is optional — if the user mentioned an Android source path, add the line `android-project-path: <android-project-path>` to the prompt. Omit otherwise.
+
+After completion, read `<ROUND_DIR>/self-test-fix-report.md` (first 60 lines). Extract:
+- `confirmed` — white-box confirmed defect count
+- `false_positives` — false positive count
+- `fixed` — successfully fixed count
+- `failed_to_fix` — failed to fix count
+
+Set `round_no_confirmed_defects = true` if `confirmed == 0` (all failures are false positives).
+
+Surface a one-line summary:
+
+> Round N: test=<passed>/<total>, confirmed=X, fixed=Y, false_positives=Z
+
+---
+
+### 2C — Build (if defects were confirmed)
+
+If `round_no_confirmed_defects`:
+- Skip 2C.
+- Set `stop_reason = no_confirmed_defects`.
+- Snapshot the round (2.5) and exit the loop.
+
+If `confirmed > 0`:
+
+Launch `build-fixer` with `--signed`:
+
+```
+Agent(
+  subagent_type="build-fixer",
+  description="Rebuild Signed HAP After Fix (Round {round})",
+  prompt="harmonyos-project-path: <PROJECT_ROOT>\noutput-path: <ROUND_DIR>\n--signed"
+)
+```
+
+After completion, find the new signed HAP in `ROUND_DIR`:
+
+1. `<ROUND_DIR>/entry-default-signed.hap`
+2. First `<ROUND_DIR>/*-signed.hap`
+
+If no HAP found → report "Build did not produce a signed HAP — cannot re-test." Set `stop_reason = no_signed_hap`. Snapshot the round (2.5) and exit the loop.
+
+Update `CURRENT_HAP` to the new HAP path.
+
+---
+
+### 2D — Loop Control
+
+```
+# stop_reason may already be set by 2A.1 (agent_early_exit), 2A.2 (no_testcases),
+# 2B (all_passed / no_confirmed_defects), or 2C (no_signed_hap).
+if stop_reason is set:        snapshot (2.5) and exit loop
+elif round >= MAX_ROUNDS:     stop_reason = max_rounds_reached; snapshot (2.5); exit loop
+else:
+    snapshot (2.5)            # snapshot the just-completed round
+    round += 1
+    ROUND_DIR = <output-path>/round-{round}
+    create ROUND_DIR
+    go to 2A
+```
+
+---
+
+### 2.5 — Per-round Snapshot
+
+After every round's 2A returns (round 1 and rounds 2+), snapshot these files from `<output-path>/` to `<ROUND_DIR>`. Each copy follows the rule **"skip if source missing"**.
+
+| Source | Destination | Notes |
+|--------|-------------|-------|
+| `<output-path>/self-test-report.md` | `<ROUND_DIR>/self-test-report.md` | Always expected after a successful T9. If missing, sentinel-FAIL detection in 2A.1 already routed via `agent_early_exit`. |
+| `<output-path>/task/` | `<ROUND_DIR>/task/` | Per-round runner artifacts. Skip if missing (e.g., agent FAILed before T6 created the dir). |
+| `<output-path>/_extracted.json` | `<ROUND_DIR>/_extracted.json` | Only present after round 1 (S4.2 wrote it). Skip in rounds 2+. |
+
+**NOT snapshotted** (these stay at root only as the canonical, cross-round artifacts):
+- `<output-path>/testcases.json` — written once by round 1; rounds 2+ read it in place.
+- `<output-path>/app-metadata.json` — written once by round 1; rounds 2+ read it in place.
+
+**`task/` cleanup ordering**: T5 (inside the agent) cleans `<output-path>/task/` at the start of each test invocation. The invariant: by the time T5 wipes the root `task/` at the start of round N, the previous round's `task/` has already been snapshotted into `<output-path>/round-{N-1}/task/`.
+
+**Cross-platform copy syntax**:
+
+| Shell | Example |
+|-------|---------|
+| bash / sh | `cp -fr "<source>" "<destination>"` |
+| PowerShell | `Copy-Item -LiteralPath "<source>" -Destination "<destination>" -Recurse -Force` |
+
+Use forward slashes for cross-platform safety; both shells accept them.
+
+---
+
+### 2.6 — Loop Finalization (HAP mirror only)
+
+When the loop exits, mirror the final HAP from `CURRENT_HAP` (which may live under the last `<ROUND_DIR>`) to `<output-path>/entry-default-signed.hap`. This is the only "latest" artifact whose primary location is a round dir; `self-test-report.md`, `testcases.json`, `app-metadata.json`, and `task/` already live at `<output-path>/` as canonical copies.
+
+Copy `CURRENT_HAP` → `<output-path>/entry-default-signed.hap` using the same cross-platform syntax above. Skip if `CURRENT_HAP` is already at `<output-path>/entry-default-signed.hap`.
+
+---
+
+## Step 3 — Surface Results
+
+Read `<output-path>/self-test-report.md` (first 80 lines) and extract the overview section:
+
+```
+## 自测报告
+
+| 指标 | 值 |
+|------|-----|
+| 设备 | <device-serial> |
+| HAP | <hap-basename> |
+| 测试套件 | <suite-name> |
+| 总用例 | <total> |
+| 通过 | <passed> |
+| 失败 | <failed> |
+| 常规通过率 | <pass-rate>% |
+```
+
+> If the report file is missing or malformed, report the error to the user with the agent's output path — do NOT fabricate results. If the report exists but is truncated or unexpectedly formatted, read up to the first 80 lines for the overview section, and point the user to the full file at `<output-path>/self-test-report.md`.
+
+**Pre-case reminder**: If pre-cases failed, note that these are environment/setup issues (permissions, media imports, tutorial skipping), NOT application defects. The regular pass rate (常规通过率) is the primary quality metric.
+
+**Point the user to the full report**: `<output-path>/self-test-report.md` and per-round artifacts at `<output-path>/round-{N}/`.
+
+### Loop Summary (if loop ran)
+
+After the report, present:
+
+```
+## 自测修复循环完成
+
+| 指标 | 值 |
+|------|-----|
+| 迭代次数 | <round>/<MAX_ROUNDS> |
+| 最终状态 | <passed>/<total> 通过（<pass-rate>%） |
+| 停止原因 | <stop_reason> |
+```
+
+`<stop_reason>` is one of:
+- `all_passed` — all cases passed, loop exited successfully.
+- `no_confirmed_defects` — failures exist but fixer judged them all as false positives.
+- `no_signed_hap` — build-fixer did not produce a signed HAP, loop cannot continue.
+- `max_rounds_reached` — hit the configured `MAX_ROUNDS` cap (default 3) with failures remaining.
+- `no_testcases` — `<output-path>/testcases.json` contained 0 cases; nothing to run. Surface the path so the user can inspect.
+- `agent_early_exit` — `self-tester` wrote a sentinel FAIL report (config / device / autotest-dir / batch crash / `setup: false` precondition failure). Surface the captured `early_exit_reason` so the user knows what to fix manually.
+
+If failures remain (for `max_rounds_reached`), list them briefly (scenario names from the report overview). Also surface paths to `<ROUND_DIR>/self-test-fix-report.md` and `<ROUND_DIR>/build-fix-report.md` (from the final round, when those reports exist).
+
+---
+
+## Error Handling
+
+| Scenario | Action |
+|----------|--------|
+| `test-case-path` does not exist | Ask user for correct path, do not proceed |
+| `hap-path` does not exist | Ask user for correct HAP path, do not proceed |
+| `output-path` parent does not exist | Create it, proceed |
+| Agent returns error / times out | Report the error to user. Suggest: check device connection (`hdc list targets`), verify HAP is valid, check `config.yaml` has real api_key |
+| `<output-path>/self-test-report.md` missing after agent completes | Report: "Agent completed but no report was generated. Check agent output at <output-path>." |
+| `self-test-report.md` exists but format is unrecognizable | Show the file path, report the first 80 lines as context, let user investigate |
+| Device not found (agent reports "No device") | Remind user: connect device via USB, verify `hdc list targets` shows the device |
+| Pre-case failures in report | Note: these are environment issues, not app defects. Highlight regular pass rate |
+| `<output-path>/testcases.json` or `app-metadata.json` missing after round 1 | Round 1 setup may have failed; inspect agent output at `<output-path>/` |
+| `<output-path>/testcases.json` is empty (`total == 0` in 2A) | Report: "testcases.json contains 0 cases — nothing to run." Exit loop with `stop_reason = no_testcases`. Do NOT enter the fix loop. |
+| `self-tester` writes a sentinel FAIL report (first line `status: FAIL`) | Capture the `reason:` line. Exit loop with `stop_reason = agent_early_exit` and surface the captured reason. Do NOT enter the fix loop — these failures (config / device / autotest-dir / crash / timeout / setup=false preconditions) are not application defects. |
+| Fixer returns `confirmed == 0` (all false positives) | Exit loop with `stop_reason = no_confirmed_defects`. Surface the fix report summary. |
+| Build-fixer produces no signed HAP in ROUND_DIR | Report: "Build did not produce a signed HAP — cannot re-test." Exit loop with `stop_reason = no_signed_hap`. |
+| Loop reaches `MAX_ROUNDS` (default 3) with failures remaining | Report `stop_reason = max_rounds_reached`. Surface final pass rate and remaining failures. |
+| `app-metadata.json` missing after round 1 (PROJECT_ROOT unknown) | Surface agent error; PROJECT_ROOT cannot be resolved. Do NOT enter 2B. |
+| Signing config missing in `build-profile.json5` before build | Report: "Signing configuration not found. Please open project in DevEco Studio, go to File → Project Structure → Signing Configs, enable Automatically generate signature, then re-run." Exit loop. |
+
+---
+
+## Key Constraints
+
+- **Delegation only**: Do NOT run test tools directly. Never call `hdc`, `self_test_runner.py`, `AutoTest.batch`, or `report_tool.py` yourself. All test work goes through `self-tester`.
+- **Parameter purity**: Pass only parameter values to the agent's prompt in `key: value` format. Do NOT add extra instructions, format descriptions, schema hints, or implementation suggestions. The agent has its own built-in workflow.
+- **Do NOT read full reports**: Self-test reports can be very large (10KB–200KB). Only read the overview (first 80 lines for test report, first 60 lines for fix report). Point the user to the full file.
+- **Quote all paths**: Paths may contain spaces. Always wrap paths in double quotes in commands.
+- **Pre-cases are environment scripts**: Their failures indicate testing-environment issues, not app bugs. The regular pass rate is the primary quality metric.
+- **Fix-and-retest loop is opt-in**: Follow Step 2 Entry — ask the user unless they already opted in with "自动修复".
+- **Test → Fix → Build is the mandatory order**: Never re-test without rebuilding after a fix (would test old code). Never skip the build step in the loop.
+- **`CURRENT_HAP` must track the latest build**: After each 2C rebuild, update `CURRENT_HAP` to the new signed HAP from `ROUND_DIR`. Never re-test with the old HAP.
+- **Per-round snapshots**: Each round writes its latest artifacts to `<output-path>/` (root), then the SKILL snapshots per-round outputs to `<output-path>/round-{N}/`. The persistent JSONs (`testcases.json`, `app-metadata.json`) are written exactly once by round 1 and consumed unchanged by rounds 2+.
+- **Loop stop conditions**: Normal exits (3) — `all_passed`, `no_confirmed_defects`, `max_rounds_reached`. Error exits (3) — `no_signed_hap` (build did not produce HAP), `no_testcases` (empty testcases.json), `agent_early_exit` (self-tester sentinel FAIL: config / device / autotest-dir / crash / timeout / `setup: false` precondition failure).
+- **Loop iterations are strictly sequential**: Never launch more than one fixer or tester at a time.
+
+## Forbidden
+
+- ❌ Running `hdc list targets`, `hdc install`, or any `hdc` command directly
+- ❌ Calling `self_test_runner.py` or `report_tool.py` directly
+- ❌ Reading `test_case.md` or modifying test cases — the agent handles this
+- ❌ Reading the full `self-test-report.md` (read only first 80 lines for summary)
+- ❌ Reading the full `self-test-fix-report.md` (read only first 60 lines for summary)
+- ❌ Fabricating results when the report is missing or unreadable
+- ❌ Passing `output-path: <output-path>/round-N` to the agent — the agent always writes to `<output-path>/` root; per-round directories are populated by the SKILL's snapshot step
+- ❌ Skipping the build step in the loop (rebuild is mandatory after each confirmed fix)
+- ❌ Re-testing with an old HAP after a build
+- ❌ Launching more than one fixer or tester at a time — loop rounds are strictly sequential