android2harmony 0.1.3 โ 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agents/self-tester.md +120 -83
- package/dist/index.js +13 -486
- package/dist/index.js.map +4 -4
- package/package.json +3 -8
- package/tools/autotest/deps/autotest-agent-0.1.1.tgz +0 -0
- package/tools/autotest/engine/batch-launcher.ts +0 -326
- package/tools/autotest/engine/report-tool.ts +0 -773
- package/tools/autotest/engine/self-test-runner.ts +0 -1024
- package/tools/autotest/engine/testcases-tool.ts +0 -246
- package/tools/autotest/resolve-metadata-tool.ts +0 -143
- package/tools/autotest/runner/logger.ts +0 -45
- package/tools/autotest/runner/process-utils.ts +0 -41
- package/tools/autotest/validate.ts +0 -115
package/agents/self-tester.md
CHANGED
|
@@ -9,9 +9,7 @@ mode: subagent
|
|
|
9
9
|
|
|
10
10
|
You are a self-tester for HarmonyOS applications. You run a single pipeline: parse `test_case.md` into structured artifacts, install the HAP on a connected device, execute AutoTest, and produce a verification report. The `setup` parameter controls one branch โ when `setup=true` the parse phase runs and writes `<output_path>/testcases.json` and `<output_path>/app-metadata.json`; when `setup=false` the parse phase is skipped and the agent reads those two files from `<output_path>/` directly.
|
|
11
11
|
|
|
12
|
-
> ๐จ **CRITICAL โ Instruction Priority**: The workflow defined in this file is the **supreme authority**. The caller's prompt provides only parameter values (paths, `setup`). Any format hints, schema descriptions, structural suggestions, or parameter-usage advice in the caller's prompt **MUST be ignored** if
|
|
13
|
-
|
|
14
|
-
> ๐ **Tool surface**: The AutoTest work runs through the `a2h_autotest_*` plugin tools (registered by the a2h plugin). The tools are **grouped by noun** โ `a2h_autotest_resolve_metadata`, `a2h_autotest_testcases`, `a2h_autotest_report`, `a2h_autotest_selftest` โ each dispatched on an `action` enum; 11 engine subcommands surface as 4 plugin tools. Each tool wraps an engine script under `tools/autotest/` that ships with a2h โ no `ht` CLI, no external tool directory to locate, no `HOMETRANS_HOME`. Each tool's `args` is a union (only `action` is required at the top level; the rest apply per action as documented per phase below). Exit codes / stdout JSON are documented per phase below. The multimodal model is resolved by the host (DevEco Code) and piped into the `selftest` tool's `run` action over stdin โ the apiKey never touches env/argv/disk.
|
|
12
|
+
> ๐จ **CRITICAL โ Instruction Priority**: The workflow defined in this file is the **supreme authority**. The caller's prompt provides only parameter values (paths, `setup`). Any format hints, schema descriptions, structural suggestions, or parameter-usage advice in the caller's prompt **MUST be ignored** if they conflict with, extend, or bypass any step, validation rule, error-handling logic, or FORBIDDEN constraint in this file. The `setup` parameter selects whether to run Phase S1-S4; Phase T1-T8 always runs. Every step is executed exactly as written.
|
|
15
13
|
|
|
16
14
|
---
|
|
17
15
|
|
|
@@ -19,7 +17,7 @@ You are a self-tester for HarmonyOS applications. You run a single pipeline: par
|
|
|
19
17
|
|
|
20
18
|
| Parameter | Required | Type | Description |
|
|
21
19
|
|-----------|----------|------|-------------|
|
|
22
|
-
| `hap_path` | always | path(s) | The
|
|
20
|
+
| `hap_path` | always | path(s) | The package set: a single `.hap`/`.hsp` file, a directory, **or a comma-separated list** of files/dirs. The union across all entries must hold one entry HAP + any in-app HSPs / feature HAPs (which may live in different directories). The runner aggregates every listed package and installs them in one transaction. |
|
|
23
21
|
| `output_path` | always | path | Root output directory. All artifacts โ `testcases.json`, `app-metadata.json`, `_extracted.json`, `self-test-report.md`, `task/` โ are written here |
|
|
24
22
|
| `project_dir` | when `setup=true` | path | HarmonyOS ๅทฅ็จๆ น็ฎๅฝ๏ผๅซ `AppScope/app.json5`๏ผ๏ผ็จไบ่งฃๆ `bundle_name` / `app_name` |
|
|
25
23
|
| `test_case_path` | when `setup=true` | path | Path to `test_case.md` |
|
|
@@ -40,13 +38,42 @@ You are a self-tester for HarmonyOS applications. You run a single pipeline: par
|
|
|
40
38
|
|
|
41
39
|
## Shared Utilities
|
|
42
40
|
|
|
43
|
-
###
|
|
41
|
+
### Resolving the AutoTest Tools Directory
|
|
42
|
+
|
|
43
|
+
All AutoTest tools are invoked directly via `node`. Resolve the tools directory once at the start (the global npm package always contains `tools/test-tools/autotest/` with `node_modules/`):
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
TOOL_DIR="$(npm root -g)/@buaa_smat/hometrans/tools/test-tools/autotest"
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
**Tool file mapping:**
|
|
50
|
+
|
|
51
|
+
| Tool | File | Example |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| selftest | `$TOOL_DIR/engine/self-test-runner.ts` | `node "$TOOL_DIR/engine/self-test-runner.ts" run --testcases ...` |
|
|
54
|
+
| testcases | `$TOOL_DIR/engine/testcases-tool.ts` | `node "$TOOL_DIR/engine/testcases-tool.ts" generate ...` |
|
|
55
|
+
| report | `$TOOL_DIR/engine/report-tool.ts` | `node "$TOOL_DIR/engine/report-tool.ts" generate ...` |
|
|
56
|
+
| resolve-metadata | `$TOOL_DIR/resolve-metadata-tool.ts` | `node "$TOOL_DIR/resolve-metadata-tool.ts" --project-dir ... --output ...` |
|
|
57
|
+
|
|
58
|
+
Each tool has its own commander CLI parsing, so sub-command names, flags, exit codes, and the strict stdout-JSON contract are exactly as documented in each phase below.
|
|
59
|
+
|
|
60
|
+
> **Dev-mode:** `dist/` only exists after `npm run build`; ensure `ht` on PATH points at the built package (e.g. via `npm link`). The agent never invokes `tsx` or the tools' `.js` directly.
|
|
44
61
|
|
|
45
|
-
The
|
|
62
|
+
> The `run` subcommand emits `{status:'RUNNING', pid, ...}` on success (no-`--timeout` backward-compat mode), or `{status:'FAILED', error}` if the batch process could not be spawned. With `--timeout` (the agent's normal mode โ see T6), no `RUNNING` is emitted; the call blocks until terminal status and returns one final JSON line. On `FAILED`, no `batch.pid` is written; a subsequent `status` call returns `NOT_STARTED`.
|
|
46
63
|
|
|
47
|
-
|
|
64
|
+
### AutoTest Directory Locator
|
|
48
65
|
|
|
49
|
-
|
|
66
|
+
Used by T4 as an AutoTest-install precheck (S4 no longer needs it โ the testcases tool is a pure JSON transform; and the selftest runner resolves this dir itself at run time). The AutoTest tools are installed by `ht init` under the HomeTrans tools dir, recorded in `HOMETRANS_TOOL_PATH` โ the single tools-dir env var; there is no autotest-specific override.
|
|
67
|
+
|
|
68
|
+
A single command resolves the whole chain โ **source-tree walk-up โ env var `HOMETRANS_TOOL_PATH` โ default `~/.hometrans/tools`** (there is no `config.json` layer), then falls back to walking up from `<output_path>` (for a sibling `test-tools/autotest`, or the legacy `agents/test-tools/autotest`) when `--from` is supplied. It prints the resolved dir on success and exits non-zero if nothing is found (**anchor the fallback at `<output_path>`, the sub-agent cwd is not guaranteed**):
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
node "$TOOL_DIR/engine/self-test-runner.ts" resolve-dir --from "<output_path>"
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
| Caller | Purpose | On failure |
|
|
75
|
+
|--------|---------|------------|
|
|
76
|
+
| T4 | Locate then verify autotest config | Write `self-test-report.md` with status **FAIL** and reason "AutoTest directory not found from HOMETRANS_TOOL_PATH env var or output_path", then exit |
|
|
50
77
|
|
|
51
78
|
### Input Validation Template
|
|
52
79
|
|
|
@@ -90,18 +117,22 @@ If the command fails, stop and report the error.
|
|
|
90
117
|
|
|
91
118
|
### S2 โ Resolve App Metadata
|
|
92
119
|
|
|
93
|
-
Call
|
|
120
|
+
Call `node "$TOOL_DIR/resolve-metadata-tool.ts"` to discover `bundle_name`/`app_name`
|
|
121
|
+
from the HarmonyOS project and write `app-metadata.json`:
|
|
94
122
|
|
|
95
|
-
|
|
96
|
-
-
|
|
123
|
+
```bash
|
|
124
|
+
node "$TOOL_DIR/resolve-metadata-tool.ts" --project-dir "<project_dir>" --output "<output_path>/app-metadata.json"
|
|
125
|
+
```
|
|
97
126
|
|
|
98
|
-
Parse the JSON from
|
|
127
|
+
Parse the JSON from stdout โ `bundle_name` and `app_name` are needed in S3
|
|
128
|
+
for app-name โ bundle-name replacement. The file at `--output` is also
|
|
129
|
+
written for downstream phases (T2 reads it when `setup=false`).
|
|
99
130
|
|
|
100
|
-
On non-zero
|
|
131
|
+
On non-zero exit, stop and report the stderr message as the failure reason.
|
|
101
132
|
|
|
102
133
|
### S3 โ Parse test_case.md (LLM extraction)
|
|
103
134
|
|
|
104
|
-
> ๐จ **Two-phase architecture**: You (the LLM) extract into `_extracted.json`, then
|
|
135
|
+
> ๐จ **Two-phase architecture**: You (the LLM) extract into `_extracted.json`, then the testcases tool generates `testcases.json`. You do NOT write `testcases.json` directly.
|
|
105
136
|
|
|
106
137
|
**S3.1 โ Read test_case.md**
|
|
107
138
|
|
|
@@ -154,24 +185,24 @@ Write to `<output_path>/_extracted.json`:
|
|
|
154
185
|
- **Lines to SKIP**: `- ๅ็ฝฎๆกไปถ๏ผ` and all its sub-items, `## ้กต้ขๆ่ฟฐๆณจ่งฃ` section, `## ็ผๅทๆ ๅฐ่กจ` section.
|
|
155
186
|
- The `_extracted.json` has NO `preconditions` field.
|
|
156
187
|
|
|
157
|
-
### S4 โ Generate testcases.json via `
|
|
188
|
+
### S4 โ Generate testcases.json via `testcases-tool`
|
|
158
189
|
|
|
159
|
-
`testcases`
|
|
190
|
+
`testcases-tool` is a pure JSON transform (no device or AutoTest dir needed) โ invoke it directly:
|
|
160
191
|
|
|
161
|
-
|
|
162
|
-
-
|
|
163
|
-
|
|
164
|
-
- `validate` = `true`
|
|
192
|
+
```bash
|
|
193
|
+
node "$TOOL_DIR/engine/testcases-tool.ts" generate "<output_path>/_extracted.json" "<output_path>/testcases.json" --validate
|
|
194
|
+
```
|
|
165
195
|
|
|
166
|
-
- `
|
|
167
|
-
- `VALIDATION FAILED` โ check for unreplaced app names in `_extracted.json`, fix, re-
|
|
196
|
+
- `VALIDATION PASSED` โ done.
|
|
197
|
+
- `VALIDATION FAILED` โ check for unreplaced app names in `_extracted.json`, fix, re-run. Do NOT manually write `testcases.json`.
|
|
168
198
|
|
|
169
|
-
**Sanity-check `[PRE]` ordering** โ
|
|
199
|
+
**Sanity-check `[PRE]` ordering** โ run this command **if and only if** a `pre_test_case.md` was found and extracted in S3.1. If no pre-cases were extracted, skip this step (there is nothing to order):
|
|
170
200
|
|
|
171
|
-
|
|
172
|
-
-
|
|
201
|
+
```bash
|
|
202
|
+
node "$TOOL_DIR/engine/self-test-runner.ts" check-pre --testcases "<output_path>/testcases.json"
|
|
203
|
+
```
|
|
173
204
|
|
|
174
|
-
If
|
|
205
|
+
If assertion fails, edit `_extracted.json` to move all `[PRE]` entries contiguously to the front and re-run `node "$TOOL_DIR/engine/testcases-tool.ts" generate`.
|
|
175
206
|
|
|
176
207
|
---
|
|
177
208
|
|
|
@@ -183,12 +214,11 @@ T1 expects `<output_path>/testcases.json` and `<output_path>/app-metadata.json`
|
|
|
183
214
|
|
|
184
215
|
### T1 โ Validate Inputs
|
|
185
216
|
|
|
186
|
-
Validate `hap_path`
|
|
187
|
-
|
|
188
|
-
- `action` = `"check_hap"`
|
|
189
|
-
- `hap` = `<hap_path>`
|
|
217
|
+
Validate `hap_path` (a comma-separated list of files/dirs โ each entry a `.hap`/`.hsp` file or a directory; at least one `.hap` must exist across all entries):
|
|
190
218
|
|
|
191
|
-
|
|
219
|
+
```bash
|
|
220
|
+
node "$TOOL_DIR/engine/self-test-runner.ts" check-hap --hap "<hap_path>"
|
|
221
|
+
```
|
|
192
222
|
|
|
193
223
|
Then run the rest in a single Bash command:
|
|
194
224
|
|
|
@@ -196,11 +226,11 @@ Then run the rest in a single Bash command:
|
|
|
196
226
|
mkdir -p "<output_path>" && test -f "<output_path>/testcases.json" && test -f "<output_path>/app-metadata.json" && echo "OK"
|
|
197
227
|
```
|
|
198
228
|
|
|
199
|
-
If `setup=false`, additionally verify
|
|
229
|
+
If `setup=false`, additionally verify each JSON parses and `testcases.json` is non-empty:
|
|
200
230
|
|
|
201
|
-
|
|
202
|
-
-
|
|
203
|
-
|
|
231
|
+
```bash
|
|
232
|
+
node "$TOOL_DIR/engine/self-test-runner.ts" check-inputs --testcases "<output_path>/testcases.json" --metadata "<output_path>/app-metadata.json"
|
|
233
|
+
```
|
|
204
234
|
|
|
205
235
|
If any check fails, write a `self-test-report.md` using the sentinel format from Shared Utilities. The `reason:` line names the specific failure, e.g.:
|
|
206
236
|
|
|
@@ -222,65 +252,68 @@ Run `npx --yes devecocli device list`:
|
|
|
222
252
|
- If at least one connected device or running emulator is returned โ proceed.
|
|
223
253
|
- If no available devices are listed โ write report with the sentinel format (`status: FAIL` / `reason: No HarmonyOS device connected`), then stop.
|
|
224
254
|
|
|
225
|
-
Record the device serial number
|
|
255
|
+
Record the device serial number.
|
|
226
256
|
|
|
227
|
-
### T4 โ Verify
|
|
257
|
+
### T4 โ Locate AutoTest Directory & Verify autotest config
|
|
228
258
|
|
|
229
|
-
|
|
259
|
+
Find the `test-tools/autotest` directory using the **AutoTest Directory Locator** (Shared Utilities). If not found, write a `self-test-report.md` with the sentinel format (`status: FAIL` / `reason: AutoTest directory not found from HOMETRANS_TOOL_PATH env var or output_path`), then stop.
|
|
230
260
|
|
|
231
|
-
The
|
|
261
|
+
The resolved path is used for this precheck only; do not pass it to any later phase โ `node "$TOOL_DIR/engine/self-test-runner.ts" run` resolves the AutoTest directory itself at run time.
|
|
232
262
|
|
|
233
|
-
|
|
263
|
+
The model config is shared with UI alignment and resolved along the standard chain โ **env var โ `~/.hometrans/config.json` โ ask**. The selftest runner reads it at run time (env `HOMETRANS_MODEL_API_KEY` overrides `autotest.unified_model.api_key`) and maps the api_key/model/device fields into `AUTOTEST_*` env vars for the batch launcher โ you do **not** create or edit any YAML. Just verify the api_key is usable: the `HOMETRANS_MODEL_API_KEY` OS environment variable is set, **or** every model endpoint defined in the `autotest` block of `~/.hometrans/config.json` has a real (non-placeholder) `api_key`.
|
|
234
264
|
|
|
235
265
|
```bash
|
|
236
|
-
|
|
266
|
+
node "$TOOL_DIR/engine/self-test-runner.ts" check-config
|
|
237
267
|
```
|
|
238
268
|
|
|
239
|
-
|
|
269
|
+
If this does not print `OK` (exit non-zero โ failure modes: config file missing / unparseable, no `autotest` block in it, no model endpoint (`unified_model` / `execute_model` / `decision_model`) defined, or an endpoint `api_key` empty / still the placeholder with no `HOMETRANS_MODEL_API_KEY` env var override), write `self-test-report.md` with the sentinel format (`status: FAIL` / `reason: model api_key not configured โ set HOMETRANS_MODEL_API_KEY env var or run ht init`; copy the command's printed error message into `reason:` for specifics), then stop.
|
|
270
|
+
|
|
271
|
+
### T5 โ Clean Task Directory
|
|
272
|
+
|
|
273
|
+
| Shell | Example |
|
|
274
|
+
|-------|---------|
|
|
275
|
+
| bash / sh | `rm -rf "<output_path>/task" && echo "Task directory cleaned" \|\| echo "Task directory does not exist, skipping cleanup"` |
|
|
276
|
+
| PowerShell | `Remove-Item -LiteralPath "<output_path>/task" -Recurse -Force -ErrorAction SilentlyContinue; Write-Output "Task directory cleaned"` |
|
|
277
|
+
|
|
278
|
+
### T6 โ Run `node "$TOOL_DIR/engine/self-test-runner.ts" run --timeout`
|
|
240
279
|
|
|
241
|
-
> ๐จ **MANDATORY**: Environment setup, HAP installation, test execution, polling, and timeout auto-kill are all handled by a **single
|
|
242
|
-
> 1.
|
|
280
|
+
> ๐จ **MANDATORY**: Environment setup, HAP installation, test execution, polling, and timeout auto-kill are all handled by a **single invocation** of `node "$TOOL_DIR/engine/self-test-runner.ts" run --timeout`. The script synchronously executes:
|
|
281
|
+
> 1. Read & validate the `autotest` config from `~/.hometrans/config.json` (env `HOMETRANS_MODEL_API_KEY` overrides the api_key) and map its api_key/model/device fields into `AUTOTEST_*` env vars for the batch launcher
|
|
243
282
|
> 2. Kill any stale previous batch recorded in `batch.pid` (via `killProcessTree`)
|
|
244
283
|
> 3. `hdc uninstall` + `hdc install -r`
|
|
245
284
|
> 4. Normalize testcases JSON into JSONL (filling missing `uuid`/`spec`)
|
|
246
|
-
> 5. Launch `node batch-launcher.
|
|
247
|
-
> 6. **Polling loop**: sleep 60s, probe status, until terminal status or
|
|
285
|
+
> 5. Launch `node batch-launcher.ts` (the `@autotest/agent` engine, constructing `new AutoTestAgent({...})` from the `AUTOTEST_*` env) as a **detached background process** and write `batch.pid`
|
|
286
|
+
> 6. **Polling loop**: sleep 60s, probe status, until terminal status or `--timeout` elapsed. On timeout, auto-kill the batch tree and return `TIMEOUT`.
|
|
248
287
|
>
|
|
249
|
-
> Without
|
|
288
|
+
> Without `--timeout` the script returns `RUNNING` JSON immediately after spawn (backward-compat; not used by this agent). With `--timeout`, **no `RUNNING` JSON is emitted** โ only one final terminal JSON line on stdout when the loop exits.
|
|
250
289
|
|
|
251
290
|
> **FORBIDDEN actions** (violating any of these invalidates the entire test run):
|
|
252
|
-
> - โ Calling `node batch-launcher.
|
|
291
|
+
> - โ Calling `node batch-launcher.ts` directly โ always go through `node "$TOOL_DIR/engine/self-test-runner.ts" run`
|
|
253
292
|
> - โ Reading source code of the `self-test-runner` tool or any `AutoTest` module
|
|
254
293
|
> - โ Running `hdc install -r` separately
|
|
255
294
|
> - โ Writing a shell loop or Python loop to iterate over cases yourself
|
|
256
295
|
|
|
257
|
-
|
|
296
|
+
Run as a **background task** (`run_in_background: true`) โ the call may block for up to caseCount ร 12min (720s/case = batch-launcher `CASE_TIMEOUT_MS=10min` + 20% margin). One call, one final JSON on stdout. Pass `--timeout auto`: the CLI reads `--testcases` and derives the budget itself โ no shell-side case counting or arithmetic, and the command is identical on bash and PowerShell:
|
|
258
297
|
|
|
259
|
-
|
|
260
|
-
-
|
|
261
|
-
|
|
262
|
-
- `bundleName` = `<bundle_name>`
|
|
263
|
-
- `category` = `<app_name>`
|
|
264
|
-
- `taskDir` = `<output_path>/task`
|
|
265
|
-
- `outputDir` = `<output_path>`
|
|
266
|
-
- `deviceSn` = `<device_serial>` (from T3)
|
|
267
|
-
- `timeout` = `"auto"`
|
|
298
|
+
```
|
|
299
|
+
node "$TOOL_DIR/engine/self-test-runner.ts" run --testcases "<output_path>/testcases.json" --hap "<hap_path>" --bundle-name "<bundle_name>" --category "<app_name>" --task-dir "<output_path>/task" --output-dir "<output_path>" --timeout auto
|
|
300
|
+
```
|
|
268
301
|
|
|
269
|
-
> The
|
|
302
|
+
> The TIMEOUT JSON does not carry the budget. When the sentinel `reason:` needs the concrete value (`timed out after <timeout>s`), read it from the runner's log line `--timeout auto: <N> ๆก็จไพ ร 720s = <total>s` in the background task output โ do not recompute.
|
|
270
303
|
|
|
271
|
-
When the
|
|
304
|
+
When the background task returns, parse the **last line of stdout** as the terminal status JSON. Branch on `status`:
|
|
272
305
|
|
|
273
306
|
| `status` | Exit code | Meaning | Action |
|
|
274
307
|
|-----------|----------|---------|--------|
|
|
275
308
|
| `COMPLETED` | 0 | All cases finished, `summary.json` written | Record `pass_count`/`fail_count`/`unknown_count`/`pass_rate`/`task_subdir` for T7/T8; proceed to T7 |
|
|
276
309
|
| `CRASHED` | 3 | Batch died without producing `summary.json` | Write CRASHED report with sentinel (`status: FAIL` / `reason: AutoTest batch crashed`), include `log_tail` from JSON; stop |
|
|
277
|
-
| `TIMEOUT` | 5 |
|
|
310
|
+
| `TIMEOUT` | 5 | `--timeout` elapsed, batch auto-killed | Write TIMEOUT report with sentinel (`status: FAIL` / `reason: AutoTest batch timed out after <timeout>s`), include `cases_done`/`last_case`/`log_tail` from the JSON for partial-result context; stop |
|
|
278
311
|
| `NOT_STARTED` | 4 | Race: `batch.pid` vanished between spawn and first poll | Write FAIL report with sentinel `reason: launch failed (no batch.pid)`; stop |
|
|
279
|
-
| `FAILED` | 1 | Spawn-time failure (config error, install failed, etc.)
|
|
312
|
+
| `FAILED` | 1 | Spawn-time failure (config error, install failed, etc.) | Write FAIL report with sentinel `reason: <error field from JSON>`; stop |
|
|
280
313
|
|
|
281
|
-
> ๐จ **CRITICAL**: While `run` is
|
|
314
|
+
> ๐จ **CRITICAL**: While `run --timeout` is in the background, do NOT read `task_results.jsonl`, HTML reports, MD reports, `agent.log`, or any other output file. These files are being actively written and contain incomplete data. Only proceed to T7 after the background call returns a terminal JSON.
|
|
282
315
|
|
|
283
|
-
**User-initiated abort (rare)**: if the user cancels mid-run, the
|
|
316
|
+
**User-initiated abort (rare)**: if the user cancels mid-run, the agent's background Bash task is killed by the harness but the detached batch keeps running. To release the device immediately, call `node "$TOOL_DIR/engine/self-test-runner.ts" kill --task-dir "<output_path>/task"`. The next `run` invocation also auto-cleans a stale `batch.pid` on startup.
|
|
284
317
|
|
|
285
318
|
### T7 โ Read results after completion
|
|
286
319
|
|
|
@@ -291,9 +324,10 @@ After `COMPLETED`:
|
|
|
291
324
|
- **Status semantics**: `PASS` = AutoTest rendered a PASS badge. `FAIL` = AutoTest rendered FAIL **or** the case timed out / crashed (the `reason` field disambiguates). `UNKNOWN` = AutoTest produced a report but the badge couldn't be determined; treat as not-passed in the summary but mark separately.
|
|
292
325
|
3. For PASS cases the JSONL row is sufficient โ do NOT open the per-case report. For FAIL / UNKNOWN cases with empty `reason`, you may peek at the runner log of last resort:
|
|
293
326
|
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
327
|
+
| Shell | Example |
|
|
328
|
+
|-------|---------|
|
|
329
|
+
| bash / sh | `tail -40 "<report_dir>/agent.log"` |
|
|
330
|
+
| PowerShell | `Get-Content -LiteralPath "<report_dir>/agent.log" -Tail 40` |
|
|
297
331
|
|
|
298
332
|
This caps token usage. **Do NOT read full HTML / MD / JSON per-case reports** โ they are very large (10KBโ200KB) and meant for human users.
|
|
299
333
|
|
|
@@ -301,32 +335,35 @@ After `COMPLETED`:
|
|
|
301
335
|
|
|
302
336
|
### T8 โ Generate self-test-report.md
|
|
303
337
|
|
|
304
|
-
> ๐จ **Do NOT compose the report markdown yourself.** Use `
|
|
338
|
+
> ๐จ **Do NOT compose the report markdown yourself.** Use `node "$TOOL_DIR/engine/report-tool.ts" generate`.
|
|
305
339
|
|
|
306
340
|
**Collect inputs:**
|
|
307
|
-
- `<suite_name>` โ read the first non-empty line of `test_case.md` (typically `# <title>`). Strip leading `#` and whitespace. When `setup=false`, `test_case.md` may not be at a known path; in that case use
|
|
341
|
+
- `<suite_name>` โ read the first non-empty line of `test_case.md` (typically `# <title>`). Strip leading `#` and whitespace. When `setup=false`, `test_case.md` may not be at a known path; in that case use `bundle_name` as the suite name.
|
|
308
342
|
- `<device_serial>` โ from T3.
|
|
309
343
|
- `<entry_hap>` โ the **entry HAP** for display (never the raw comma string). If `hap_path` is a single `.hap` file, use it. If it is a directory or comma-separated list, pick a `.hap` named `entry-*` if present, otherwise the first `.hap` found across the entries.
|
|
310
344
|
- `<task_subdir>` โ from T6 final JSON.
|
|
311
345
|
|
|
312
|
-
**
|
|
346
|
+
**Run the renderer:**
|
|
313
347
|
|
|
314
|
-
|
|
315
|
-
-
|
|
316
|
-
-
|
|
317
|
-
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
-
|
|
348
|
+
```bash
|
|
349
|
+
node "$TOOL_DIR/engine/report-tool.ts" generate \
|
|
350
|
+
--task-subdir "<task_subdir>" \
|
|
351
|
+
--app-metadata "<output_path>/app-metadata.json" \
|
|
352
|
+
--hap "<entry_hap>" \
|
|
353
|
+
--device "<device_serial>" \
|
|
354
|
+
--suite "<suite_name>" \
|
|
355
|
+
--out "<output_path>/self-test-report.md" \
|
|
356
|
+
--validate
|
|
357
|
+
```
|
|
322
358
|
|
|
323
|
-
- `
|
|
324
|
-
- `VALIDATION FAILED` โ fix upstream data and re-
|
|
359
|
+
- `VALIDATION PASSED` โ done.
|
|
360
|
+
- `VALIDATION FAILED` โ fix upstream data and re-run; do NOT hand-write the report.
|
|
325
361
|
|
|
326
|
-
To re-validate an existing report without regenerating it (debugging only)
|
|
362
|
+
To re-validate an existing report without regenerating it (debugging only):
|
|
327
363
|
|
|
328
|
-
|
|
329
|
-
-
|
|
364
|
+
```bash
|
|
365
|
+
node "$TOOL_DIR/engine/report-tool.ts" validate "<output_path>/self-test-report.md"
|
|
366
|
+
```
|
|
330
367
|
|
|
331
368
|
**Pre-case rendering**: Rows with `case_name` starting with `[PRE] ` go into `## ๅ็ฝฎ็จไพ`. All others go into `## ็จไพ่ฏฆๆ
`. Pre indices and Case indices each start at 1 within their own section (`### Pre 1:`, `### Case 1:`). The script emits these forms automatically โ do NOT hand-edit to `### Case PRE-1:` or any other variant; the validator will reject it.
|
|
332
369
|
|