@buaa_smat/hometrans 0.1.0 → 0.1.2

package/agents/test-tools/autotest/testcases_schema.md

@@ -0,0 +1,143 @@
+# testcases.json 生成规范
+
+> 本文件是 `testcases.json` 的**唯一权威格式定义**。
+> 生成 `testcases.json` 时，必须严格遵循本规范。
+
+---
+
+## 1. 文件结构
+
+根元素必须是 **JSON 数组 `[]`**，每个元素是一个测试用例对象。
+
+```json
+[
+  { ... },
+  { ... }
+]
+```
+
+## 2. 测试用例对象（4 个必填字段）
+
+| 字段 | 类型 | 说明 | 示例 |
+|------|------|------|------|
+| `case_name` | `string` | 测试场景标题，来自 `### Scenario:` | `"冷启动初始化后正常进入主页面"` |
+| `bundle_name` | `string` | HarmonyOS 应用包名 | `"com.example.tuku"` |
+| `app_name` | `string` | 应用显示名称（不要翻译，保持原始值） | `"Tuku"` |
+| `test_steps` | `string` | 测试步骤文本（见下方模板） | 见第 3 节 |
+
+> **不允许**出现其他字段（如 `id`、`priority`、`preconditions`、`spec_id` 等）。
+
+## 3. `test_steps` 字符串模板
+
+每个测试用例的 `test_steps` 必须严格按以下模板生成：
+
+```
+动作：{actions}
+预期结果：{expected_results}
+```
+
+`test_steps` 中**只包含** `动作：` 和 `预期结果：` 两行，不包含任何段落头（如 `【说明】`、`【待测应用信息】`、`【测试用例】` 等）。
+
+### 变量说明
+
+| 变量 | 来源 | 处理规则 |
+|------|------|----------|
+| `{actions}` | `test_case.md` 中的 `- 动作：` 行 | 所有应用名引用（如"图库应用"、"Tuku"等）替换为 `bundle_name` |
+| `{expected_results}` | `test_case.md` 中的 `- 预期结果：` 行 | 原样使用，同样执行 app_name → bundle_name 替换 |
+
+## 4. 完整示例
+
+```json
+[
+  {
+    "case_name": "冷启动初始化后正常进入主页面",
+    "bundle_name": "com.example.tuku",
+    "app_name": "Tuku",
+    "test_steps": "动作：打开com.example.tuku\n预期结果：文件夹网格主页面成功展示文件夹列表"
+  },
+  {
+    "case_name": "首次打开应用-授权通过-展示所有文件夹",
+    "bundle_name": "com.example.tuku",
+    "app_name": "Tuku",
+    "test_steps": "动作：打开com.example.tuku -> 点击允许按钮授予媒体访问权限\n预期结果：权限授予成功，跳转到文件夹网格主页面，可见所有包含媒体的文件夹"
+  }
+]
+```
+
+## 5. 禁止字符串
+
+以下字符串**不得出现在 `test_steps` 中的任何位置**。如果 `test_case.md` 源文件中包含这些内容，提取时必须删除或改写：
+
+| 禁止字符串 | 原因 | 处理方式 |
+|------------|------|----------|
+| `前置条件` | 前置条件不应进入测试步骤 | 删除含该词的句子 |
+| `Spec:` | Spec 引用不应出现 | 删除 |
+| `Scenario:` | Scenario 引用不应出现 | 删除 |
+| `ability_name:` | 不允许出现在 test_steps 中 | 删除 |
+| `module_name:` | 不允许出现在 test_steps 中 | 删除 |
+
+> **不得出现任何 `【xxx】` 段落头**（如 `【说明】`、`【待测应用信息】`、`【测试用例】`、`【前置条件】` 等）。`test_steps` 只包含 `动作：` 和 `预期结果：` 两行。
+
+## 6. 错误格式示例
+
+### 错误 1：使用结构化对象数组代替纯文本字符串
+
+```json
+{
+  "test_steps": [
+    { "step": 1, "action": "launch_app", "params": { "bundle_name": "..." } }
+  ]
+}
+```
+**正确做法**: `test_steps` 必须是字符串，不是数组。
+
+### 错误 2：根元素是对象而非数组
+
+```json
+{
+  "spec_id": "SPEC-01",
+  "test_cases": [ ... ]
+}
+```
+**正确做法**: 根元素必须是 `[ ... ]`。
+
+### 错误 3：包含段落头
+
+```json
+{
+  "test_steps": "【说明】\n...\n\n【待测应用信息】\napp_name: Tuku\nbundle_name: com.example.tuku\n\n【测试用例】\n动作：...\n预期结果：..."
+}
+```
+**正确做法**: `test_steps` 中不允许任何 `【xxx】` 段落头，只包含 `动作：` 和 `预期结果：` 两行。
+
+### 错误 4：包含额外字段
+
+```json
+{
+  "id": "TC-001",
+  "priority": "P0",
+  "preconditions": ["..."],
+  "case_name": "...",
+  "test_steps": "..."
+}
+```
+**正确做法**: 只允许 `case_name`、`bundle_name`、`app_name`、`test_steps` 四个字段。
+
+### 错误 5：缺少必要行
+
+```json
+{
+  "test_steps": "打开图库应用 -> 检查主页面"
+}
+```
+**正确做法**: 必须包含 `动作：` 和 `预期结果：` 两行。
+
+## 7. 验证
+
+生成后必须运行验证脚本：
+
+```bash
+cd "<autotest_dir>" && uv run python testcases_tool.py validate "<path-to-testcases.json>"
+```
+
+只有输出 `VALIDATION PASSED` 才算生成成功。

package/agents/test-tools/autotest/testcases_tool.py

@@ -0,0 +1,215 @@
+#!/usr/bin/env python3
+"""Generate or validate testcases.json for the Self-Testing Agent.
+
+Subcommands:
+    generate <input-json> <output-json> [--validate]
+        Rewrite app_name → bundle_name and compose `test_steps` strictly
+        ("动作：..." / "预期结果：..."). Optionally re-validate the output.
+
+    validate <testcases-json>
+        Check a `testcases.json` (array of cases) against the schema in
+        `testcases_schema.md`. Exit 0 = PASSED, exit 1 = FAILED.
+
+The forbidden-strings and section-header rules live in this file as the single
+source of truth — generate enforces them at write time, validate re-checks them
+on existing files.
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+
+# Ensure UTF-8 output on Windows
+os.environ.setdefault("PYTHONIOENCODING", "utf-8")
+
+
+# =============================================================================
+# Shared schema invariants — used by both generate and validate
+# =============================================================================
+
+# Substrings that must not appear anywhere in `test_steps`. Keep the validator
+# in lock-step with the generator: anything that the generator refuses to emit
+# must also be rejected by the validator.
+FORBIDDEN_STRINGS = ("前置条件", "Spec:", "Scenario:", "ability_name:", "module_name:")
+
+# `test_steps` must contain only the two lines starting with `动作：` and
+# `预期结果：`. `【…】` section headers (e.g. 【说明】, 【待测应用信息】) are forbidden.
+SECTION_HEADER_RE = re.compile(r"【(.+?)】")
+
+REQUIRED_LINES = ("动作：", "预期结果：")
+
+TEST_STEPS_TEMPLATE = """\
+动作：{actions}
+预期结果：{expected_results}"""
+
+
+# =============================================================================
+# Generate
+# =============================================================================
+
+def rewrite_app_name(text: str, app_name: str, bundle_name: str) -> str:
+    """Safety net: replace remaining app_name references with bundle_name."""
+    if app_name and app_name in text:
+        text = text.replace(app_name, bundle_name)
+    return text
+
+
+def compose_test_steps(app_name: str, bundle_name: str,
+                       actions: str, expected_results: str) -> str:
+    """Compose a compliant test_steps string with only 动作 and 预期结果."""
+    rewritten_actions = rewrite_app_name(actions, app_name, bundle_name)
+    rewritten_expected = rewrite_app_name(expected_results, app_name, bundle_name)
+
+    test_steps = TEST_STEPS_TEMPLATE.format(
+        actions=rewritten_actions,
+        expected_results=rewritten_expected,
+    )
+    # Final safety: rewrite once more in case app_name only became visible after composition.
+    test_steps = rewrite_app_name(test_steps, app_name, bundle_name)
+
+    for forbidden in FORBIDDEN_STRINGS:
+        if forbidden in test_steps:
+            raise ValueError(
+                f"FATAL: forbidden string '{forbidden}' found in generated test_steps. "
+                f"This is a bug in testcases_tool.py."
+            )
+    for section in SECTION_HEADER_RE.findall(test_steps):
+        raise ValueError(
+            f"FATAL: unexpected section header 【{section}】 in generated test_steps. "
+            f"test_steps should only contain 动作 and 预期结果."
+        )
+    return test_steps
+
+
+def generate(input_data: dict) -> list[dict]:
+    """Generate compliant testcases array from input data."""
+    bundle_name = input_data["bundle_name"]
+    app_name = input_data["app_name"]
+    cases = input_data["cases"]
+
+    result = []
+    for case in cases:
+        test_steps = compose_test_steps(
+            app_name=app_name,
+            bundle_name=bundle_name,
+            actions=case["actions"],
+            expected_results=case["expected_results"],
+        )
+        result.append({
+            "case_name": case["case_name"],
+            "bundle_name": bundle_name,
+            "app_name": app_name,
+            "test_steps": test_steps,
+        })
+    return result
+
+
+# =============================================================================
+# Validate
+# =============================================================================
+
+def validate_testcases(filepath: str) -> list[str]:
+    """Return a list of validation error messages. Empty list means PASSED."""
+    with open(filepath, "r", encoding="utf-8") as f:
+        cases = json.load(f)
+
+    if not isinstance(cases, list):
+        return ["Root element is not a JSON array"]
+
+    errors: list[str] = []
+    for i, c in enumerate(cases):
+        ts = c.get("test_steps", "")
+        name = c.get("case_name", f"case_{i}")
+        app_name = c.get("app_name", "")
+        bundle_name = c.get("bundle_name", "")
+        prefix = f"Case {i + 1} [{name}]"
+
+        # Required lines
+        for required in REQUIRED_LINES:
+            if required not in ts:
+                errors.append(f"{prefix}: MISSING {required} line")
+
+        # Forbidden substrings (single source of truth shared with the generator)
+        for forbidden in FORBIDDEN_STRINGS:
+            if forbidden in ts:
+                errors.append(f"{prefix}: CONTAINS {forbidden} (must be excluded)")
+
+        # Forbidden section headers
+        for section in SECTION_HEADER_RE.findall(ts):
+            errors.append(
+                f"{prefix}: CONTAINS unexpected section header 【{section}】 "
+                f"(test_steps should only contain 动作 and 预期结果)"
+            )
+
+        # App name must be rewritten to bundle_name
+        if app_name and bundle_name and app_name != bundle_name and app_name in ts:
+            errors.append(
+                f"{prefix}: test_steps contains app_name '{app_name}' instead of bundle_name "
+                f"'{bundle_name}'. All app name references in test_steps must use bundle_name."
+            )
+
+    return errors
+
+
+# =============================================================================
+# CLI
+# =============================================================================
+
+def cmd_generate(args) -> None:
+    with open(args.input, "r", encoding="utf-8") as f:
+        input_data = json.load(f)
+
+    testcases = generate(input_data)
+
+    with open(args.output, "w", encoding="utf-8") as f:
+        json.dump(testcases, f, ensure_ascii=False, indent=2)
+
+    print(f"Generated {len(testcases)} test cases -> {args.output}")
+
+    if args.validate:
+        errors = validate_testcases(args.output)
+        if errors:
+            print("VALIDATION FAILED:")
+            for e in errors:
+                print(f"  [FAIL] {e}")
+            sys.exit(1)
+        print(f"VALIDATION PASSED: all {len(testcases)} cases conform to the schema")
+
+
+def cmd_validate(args) -> None:
+    errors = validate_testcases(args.path)
+    if errors:
+        print("VALIDATION FAILED:")
+        for e in errors:
+            print(f"  [FAIL] {e}")
+        sys.exit(1)
+    with open(args.path, "r", encoding="utf-8") as f:
+        cases = json.load(f)
+    print(f"VALIDATION PASSED: all {len(cases)} cases have correct test_steps format")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Generate or validate testcases.json for the Self-Testing Agent."
+    )
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    p_gen = sub.add_parser("generate", help="Generate compliant testcases.json from an extracted input")
+    p_gen.add_argument("input", help="Path to the extracted input JSON")
+    p_gen.add_argument("output", help="Path to write the compliant testcases.json")
+    p_gen.add_argument("--validate", action="store_true",
+                       help="After writing, re-validate the output against the schema")
+    p_gen.set_defaults(func=cmd_generate)
+
+    p_val = sub.add_parser("validate", help="Validate an existing testcases.json against the schema")
+    p_val.add_argument("path", help="Path to the testcases.json to validate")
+    p_val.set_defaults(func=cmd_validate)
+
+    args = parser.parse_args()
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()