symtest-cli 1.4.1__tar.gz → 1.5.0__tar.gz

{symtest_cli-1.4.1/src/symtest_cli.egg-info → symtest_cli-1.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: symtest-cli
-Version: 1.4.1
+Version: 1.5.0
 Summary: Regression testing for CLI applications, with multi-step workflows and numerical file comparison.
 Home-page: https://github.com/ozil111/symtest-cli
 Author: Xiaotong Wang

{symtest_cli-1.4.1 → symtest_cli-1.5.0}/docs/design.md

@@ -175,24 +175,57 @@ PathResolver 解析（系统命令直通、shell builtin 平台包装、复合
 
 ## 6. 文件比较子系统
 
-### 6.1 比较器分层
+### 6.1 三泳道比较器架构
 
-- 文本系比较器共享 difflib 行级基底，json / csv / xml 为其结构化特化（键对齐 / 列结构 / DOM 对齐）
-- h5 面向科学数据集；binary 流式分块 + LCS 相似度；script 委托外部脚本
-- 统一返回 ComparisonResult（identical / differences / error / script command_output），支持 text / json / html 渲染；支持行列窗口范围参数截取后比较
+所有比较器共享根契约 `ComparatorBase.compare(ctx) -> ComparisonResult`：
+框架构建 `CompareContext`（workspace / actual / baseline / params / error_analysis）；
+`actual`/`baseline` 以及插件 `path_params` 声明的构造参数在**比较器构造之前**
+按 workspace 统一解析，插件不得自行对 CWD resolve。配置单一事实源：
+比较器配置只存在于构造器捕获的实例状态，`ctx` 仅承载调用级上下文。
+三条泳道职责互斥：
 
-### 6.2 工厂与插件发现
+| 泳道 | 基类 | 契约 | verdict 归属 | 内置实例 |
+|---|---|---|---|---|
+| 文件泳道 | `FileComparator` | `read_content` + `compare_content`（两文件模型，行列窗口、line N 偏移、chunk_size、similarity 均属本泳道） | 插件 | text / json / csv / xml / h5 / binary |
+| 数据泳道 | `ExtractorComparator` | `extract(ctx) -> {channel: ChannelData}`；框架逐通道跑 `compare_numeric`，per-channel 容差（`channels` / `default_channel`），聚合 `ChannelResult` | **框架**（容差语义对 AI 消费端可预期） | script_extract、extractor 插件 |
+| 自主泳道 | 直接继承根 | `compare(ctx)` 全权判定 | 插件 | script、hourglass 式分析插件 |
 
-- `file_type` 取值：`text` / `json` / `csv` / `xml` / `h5` / `binary` / `script`；工厂按类型分发，支持动态注册与全局 reset（测试用）
+- 文本系共享 difflib 行级基底，json / csv / xml 为其结构化特化；h5 面向科学数据集；binary 流式分块 + LCS 相似度
+- 统一返回 `ComparisonResult`（identical / differences / error / error_stats / command_output / channels），支持 text / json / html 渲染
+- 数据泳道插件可通过 `ChannelData.extra_stats` 附带自定义误差指标，存放在独立命名空间（`ChannelResult.extra_stats`），不可覆盖框架规范指标；但 verdict 仍由框架容差持有——需要自定 verdict 的走自主泳道
+
+### 6.2 通道协议（数据泳道）
+
+- 通道判定：`identical = all(channels.passed)`；差异 position 带通道前缀（`channel S33`）；`error_stats` 按通道名嵌套
+- 内置 `script_extract` 类型：子进程执行用户脚本（零改动接入），脚本 stdout 输出约定 JSON：
+
+```json
+{"channels": {"S11": {"expected": [...], "actual": [...], "extra_stats": {...}}}}
+```
+
+  非零退出、超时、畸形 JSON 一律判为比较 error（绝不静默通过）；`actual`/`baseline` 按惯例以 baseline 在前的顺序追加为尾部 argv 槽位（可缺省）
+- 结果粒度：一条 compareSpec = 一条断言；`ComparisonResult.channels` 携带通道级子结果，报告逐通道展示 pass/fail + stats；JSON 输出中通道 differences 按 per-channel 配额截断
+
+### 6.3 工厂与插件发现
+
+- `file_type` 取值：`text` / `json` / `csv` / `xml` / `h5` / `binary` / `script` / `script_extract`；工厂按类型分发，支持动态注册与全局 reset（测试用）
 - 插件发现四处来源：内置 `*_comparator.py` 自动发现、`workspace/comparators/` 自动扫描、`--plugin-dir` CLI 参数、`CLITEST_PLUGIN_DIRS` 环境变量（供进程模式 worker 使用）
-- 命名约定：`*_comparator.py` + `*Comparator` 类名
+- 命名约定：`*_comparator.py` + `*Comparator` 类名；可用类属性 `comparator_type` 显式指定类型名（如 `script_extract`）；抽象基类自动跳过注册
+- 配置传递与严格校验：`actual`/`baseline`/`type`/`options` 之外的键转发给比较器构造函数（kwargs）；构造器参数是**严格**的——未声明的键（拼写错误）在构造时大声失败并给出支持参数清单。`options` 是框架持有的插件配置命名空间（并入构造参数，显式顶层键优先），核心 schema 不为单个插件增加专属字段；`channels`/`default_channel` 属数据泳道框架结构
 
-### 6.3 script 比较协议（对外契约）
+### 6.4 script 比较协议（自主泳道对外契约）
 
-- 子进程方式执行 `<interpreter> <script> <actual> <baseline>`
+- 子进程方式执行 `<interpreter> <script> <baseline> <actual>`（文件槽位可缺省）
 - 默认 exit code 0 → 通过；可选 `pass_pattern` / `fail_pattern` 正则匹配 stdout 细化判定
 - 超时可配
 
+### 6.5 插件公共 API
+
+`symtest.file_comparator` 包导出：`ComparatorBase` / `CompareContext` /
+`FileComparator` / `ExtractorComparator` / `ChannelData` / `ChannelResult` /
+`compare_numeric` / `NumericComparisonStats` / `parse_data_filter`。
+数据泳道插件只需依赖 `ExtractorComparator` + `ChannelData`，数值判定全部复用框架核心。
+
 ## 7. TUI 子系统
 
 基于 Textual 的终端交互界面：App + Controller（load / create / update /
@@ -210,7 +243,7 @@ TUI 与 runner 共用同一解析器；宽松的展示形态在 TUI 侧自行处
 | 新配置格式 | `BaseRunner` | 支持新的测试定义格式（如 XML、TOML） |
 | 自定义 Setup | `BaseSetup` | 数据库初始化、服务启停等 |
 | 自定义断言 | 扩展 `Assertions` | 特定业务校验逻辑 |
-| 新比较器 | `BaseComparator` | 支持新的文件格式比较，放入 `comparators/` 目录自动发现 |
+| 新比较器 | `FileComparator` / `ExtractorComparator` / `ComparatorBase` | 按泳道选择基类（文件格式 / 数据提取+通道 / 全权分析），放入 `comparators/` 目录自动发现 |
 | 新 Runner | `ParallelRunner` / `BaseRunner` | 自定义并行调度策略 |
 | TUI 扩展 | `CaseController` / Widgets | 扩展终端管理界面 |
 

{symtest_cli-1.4.1 → symtest_cli-1.5.0}/docs/user_manual.md

@@ -1972,11 +1972,31 @@ class MySetup(BaseSetup):
 
 ### 自定义文件比较器
 
-框架支持三种方式扩展比较能力：
+#### 三泳道架构（v2）
 
-#### 方式一：工作区插件目录（推荐）
+比较器插件体系围绕根契约 `ComparatorBase.compare(ctx) -> ComparisonResult` 组织，
+按判定权归属分为三条泳道：
 
-在 workspace 下创建 `comparators/` 目录，放入 `*_comparator.py` 文件（命名与内置比较器一致），框架会在首次使用时自动发现并注册其中的 `*Comparator` 类。
+| 泳道 | 基类 | 适用场景 | verdict 归属 |
+|---|---|---|---|
+| **文件泳道** | `FileComparator` | 比较两个同类文件（text/json/csv/xml/h5/binary 内置即此类） | 插件 |
+| **数据泳道** | `ExtractorComparator` | 从任意上游（文件/求解器输出/跨文件整理）提取数值数据，**复用框架数值核心**做通道化容差判定 | **框架** |
+| **自主泳道** | 直接继承 `ComparatorBase` | 独特判定逻辑（阈值组合、标签解析、渐近分析等） | 插件 |
+
+选型决策：
+
+- 数据本来就是 `(expected, actual)` 数值数组、只需 rtol/atol 判定 → **数据泳道**（可享受通道、per-channel 容差、统一 error_stats 与报告渲染）
+- 需要自定 verdict（比如"主分量紧、EAS 分量松"的组合判定、符号/渐近检查）→ **自主泳道**
+- 只是想让现成脚本快速接入 → 优先考虑内置 `script`（自定判定）或 `script_extract`（框架容差判定），零 Python 插件代码
+
+**路径解析约定**：`actual`/`baseline` 以及插件用 `path_params` 类属性声明的路径参数，
+由框架在**比较器构造之前**统一按 workspace 解析为绝对路径——构造器捕获的状态
+（`self.script`、`self.cwd` 等）持有的已是解析后的绝对路径。插件内部
+**禁止**对 CWD 做 `Path.resolve()`——并行/process 模式下 CWD 不可靠。
+
+#### 插件发现方式（三种泳道通用）
+
+在 workspace 下创建 `comparators/` 目录，放入 `*_comparator.py` 文件，框架会在首次使用时自动发现并注册其中的 `*Comparator` 类。
 
 ```
 your-workspace/
@@ -1993,26 +2013,148 @@ symtest run test_config.json --plugin-dir ./extra_plugins
 
 插件也会通过环境变量 `CLITEST_PLUGIN_DIRS` 自动继承到 process 模式子进程。
 
-**插件开发注意事项**：
-- 继承 `symtest.file_comparator.BaseComparator`
+**命名约定**：
+- 文件名必须以 `_comparator.py` 结尾（如 `my_analysis_comparator.py`）
 - 类名必须以 `Comparator` 结尾（如 `MyAnalysisComparator`）
-- 注册的 type 名 = 类名去掉 `Comparator` 再小写（如 `myanalysis`）
-- 推荐重写 `compare_files(file1, file2, **kwargs)` 方法而非 `read_content`/`compare_content`（若比较器不使用两文件模型）
-- 通过 `from symtest.file_comparator import ComparisonResult, Difference` 构造结构化结果
-- `extra_kwargs` 自动从 config `compareSpec` 透传
+- 注册的 type 名 = 类名去掉 `Comparator` 再小写（如 `myanalysis`）；也可用类属性 `comparator_type = "my_analysis"` 显式指定
+- 抽象基类（未实现全部抽象方法）自动跳过注册
 
-在配置中直接使用注册的类型名：
+在配置中直接使用注册的类型名。比较器构造参数是**严格**的：`actual`/`baseline`/
+`type` 以外的键会透传给插件构造函数，但插件未声明的参数（如拼写错误
+`pass_threhsold`）会在构造时**大声失败**（错误信息包含比较器类型名与支持的
+参数清单），绝不静默回退到默认值。插件自有配置建议放入 `options` 命名空间
+（其条目并入构造参数，显式顶层键优先），避免向核心 schema 增加插件专属字段：
 
 ```json
 {
   "type": "myanalysis",
   "actual": "optional_for_plugins",
   "baseline": "optional_for_plugins",
-  "param1": "value1"
+  "options": {"param1": "value1", "param2": 42}
+}
+```
+
+#### 数据泳道：通道提取器插件
+
+适用场景：一个脚本/插件产出多通道数据（如 CSV 的不同列、不同物理量），
+每通道需要独立的容差与误差分析。插件只负责"提取"，判定完全由框架完成。
+
+```python
+# comparators/uel_stress_comparator.py
+import numpy as np
+from symtest.file_comparator import (
+    ExtractorComparator, ChannelData, CompareContext,
+)
+
+class UelStressComparator(ExtractorComparator):
+    path_params = ("uel_csv", "ref_csv")   # 由框架按 workspace 解析
+
+    def __init__(self, uel_csv="", ref_csv="", **kwargs):
+        super().__init__(**kwargs)
+        self.uel_csv = uel_csv
+        self.ref_csv = ref_csv
+
+    def extract(self, ctx: CompareContext):
+        uel = np.loadtxt(self.uel_csv, delimiter=",")
+        ref = np.loadtxt(self.ref_csv, delimiter=",")
+        return {
+            "S11": ChannelData(expected=ref[:, 0], actual=uel[:, 0]),
+            "S33": ChannelData(
+                expected=ref[:, 2], actual=uel[:, 2],
+                extra_stats={"hydrostatic_shift": float(uel[:, 2].mean())},
+            ),
+        }
+```
+
+配置（per-channel 容差按名路由，未声明的通道用 `default_channel`）：
+
+```json
+{
+  "type": "uelstress",
+  "uel_csv": "out/uel.csv",
+  "ref_csv": "out/native.csv",
+  "channels": {
+    "S33": {"atol": 600.0, "data_filter": "abs>1e-12"}
+  },
+  "default_channel": {"rtol": 1e-5, "atol": 1e-8}
+}
+```
+
+**判定与结果语义**：
+- 每通道独立跑 `compare_numeric`（对称容差 `|a-b| <= max(rtol*max(|a|,|b|), atol)`）
+- `identical = 所有通道全过`；一条 compareSpec 仍是一条断言
+- 差异 position 带通道前缀（`channel S33`）；`error_stats` 按通道名嵌套
+- 报告逐通道展示 pass/fail、容差与统计；JSON 输出中通道差异按 per-channel 配额截断
+- 自定义误差指标通过 `ChannelData.extra_stats` 附带，存放在**独立命名空间**
+  （`ChannelResult.extra_stats`），不可能覆盖框架规范指标（`max_abs_error`、
+  `total` 等）；报告逐通道分别渲染两个命名空间
+- verdict 不可由插件干预——需要自定 verdict 请走自主泳道
+
+#### 数据泳道：内置 `script_extract` 类型（零改动脚本接入）
+
+现成分析脚本无需包装成 Python 插件：脚本以子进程执行，向 stdout 打印约定
+JSON，各通道即进入上述框架判定管线。
+
+```json
+{
+  "type": "script_extract",
+  "script": "extract_channels.py",
+  "args": ["--frame", "10"],
+  "cwd": "case/subdir",
+  "channels": {"S33": {"atol": 600.0}},
+  "default_channel": {"rtol": 1e-5, "atol": 1e-8},
+  "timeout": 600
+}
+```
+
+**stdout JSON 协议**：
+
+```json
+{
+  "channels": {
+    "S11": {"expected": [1.0, 2.0], "actual": [1.0, 2.0]},
+    "S33": {"expected": [...], "actual": [...], "extra_stats": {"asymmetry": 2e-13}}
+  }
 }
 ```
 
-#### 方式二：内置 `script` 类型比较器
+**错误语义**：非零退出码、超时、畸形 JSON、缺 `channels` 键或通道缺
+`expected`/`actual` 数组 → 一律判为比较 error（绝不静默通过）。
+`actual`/`baseline`（若配置）以 baseline 在前的顺序追加为脚本尾部参数，可缺省。
+
+#### 自主泳道：全权判定插件
+
+适用场景：独特判定逻辑。插件实现 `compare(ctx)` 并直接返回结果。
+
+```python
+# comparators/my_analysis_comparator.py
+from symtest.file_comparator import ComparatorBase, CompareContext, ComparisonResult, Difference
+
+class MyAnalysisComparator(ComparatorBase):
+    path_params = ("script", "case_dir")
+
+    def __init__(self, script="", case_dir=None, pass_threshold=1e-6):
+        super().__init__()   # 参数严格：未声明/拼错的配置键在构造时报错
+        self.script = script      # 构造前已由框架按 workspace 解析
+        self.case_dir = case_dir
+        self.pass_threshold = pass_threshold
+
+    def compare(self, ctx: CompareContext) -> ComparisonResult:
+        result = ComparisonResult(
+            file1=ctx.baseline,   # 无文件输入时保持 None，不伪造空串
+            file2=ctx.actual,
+        )
+        # ... 执行分析、解析指标 ...
+        result.identical = True  # 或 False + differences
+        result.error_stats = {"full_rel": 8e-8}   # 自由 dict，报告通用渲染
+        return result
+```
+
+`ctx` 字段（仅调用级上下文）：`workspace` / `actual` / `baseline`（可空，
+无文件输入时为 `None`）/ `params`（文件泳道窗口参数）/ `error_analysis`。
+比较器配置的唯一权威在构造器捕获的实例状态中——`ctx` 不携带配置副本。
+
+#### 内置 `script` 类型比较器（自主泳道开箱即用）
 
 适用于独立分析脚本快速接入，无需编写比较器类：
 
@@ -2035,8 +2177,8 @@ symtest run test_config.json --plugin-dir ./extra_plugins
 | 参数 | 必填 | 默认值 | 说明 |
 |---|---|---|---|
 | `script` | 是 | — | 脚本路径（相对 workspace 或绝对） |
-| `actual` | 否 | — | 传给脚本的第一个文件参数 |
-| `baseline` | 否 | — | 传给脚本的第二个文件参数 |
+| `actual` | 否 | — | 传给脚本的文件参数（尾部第二位） |
+| `baseline` | 否 | — | 传给脚本的文件参数（尾部第一位） |
 | `cwd` | 否 | — | 脚本工作目录 |
 | `interpreter` | 否 | `sys.executable` | Python 解释器 |
 | `pass_exit_code` | 否 | `0` | 判定为通过的退出码 |
@@ -2052,15 +2194,15 @@ symtest run test_config.json --plugin-dir ./extra_plugins
 
 脚本的 stdout 和 stderr 会完整捕获到 `Comparator Output` 区块中，在报告渲染时限 20 行展示。
 
-#### 方式三：手动注册（编程方式）
+#### 手动注册（编程方式）
 
 ```python
-from symtest.file_comparator import ComparatorFactory
-from symtest.file_comparator.base_comparator import BaseComparator
+from symtest.file_comparator import ComparatorFactory, ComparatorBase, CompareContext
+from symtest.file_comparator.result import ComparisonResult
 
-class FooComparator(BaseComparator):
-    # 实现 read_content / compare_content 等方法
-    pass
+class FooComparator(ComparatorBase):
+    def compare(self, ctx: CompareContext) -> ComparisonResult:
+        ...
 
 ComparatorFactory.register_comparator("foo", FooComparator)
 
@@ -2070,7 +2212,7 @@ comparator = ComparatorFactory.create_comparator("foo")
 
 #### 专用插件示例：hourglass 切线刚度分析
 
-`examples/plugins/hourglass_tangent_comparator.py` 是一个完整的工作区插件示例，展示了如何将专用的 `analyze_*_tangent.py` 分析脚本接入框架：
+`examples/plugins/hourglass_tangent_comparator.py` 是一个完整的自主泳道工作区插件示例，展示了如何将专用的 `analyze_*_tangent.py` 分析脚本接入框架：
 
 ```json
 {
@@ -2085,6 +2227,7 @@ comparator = ComparatorFactory.create_comparator("foo")
 **特点**：
 - 通过 subprocess 调分析脚本（**零改动** analyze 代码），捕获 stdout 后用正则解析 `RESULT:` 行和 `full_rel`/`aa_rel`/`hh_rel`/`asymmetry` 等数值指标
 - 构造结构化 `ComparisonResult`：`identical` 基于 `full_rel < pass_threshold` 判定；`differences` 列出超限指标；`error_stats` 包含全部数值
+- `path_params` 声明 `script`/`case_dir`，路径由框架按 workspace 解析
 - 脚本 stdout 进入 `Comparator Output` 区块
 
 使用方法：将插件文件复制到 workspace 的 `comparators/` 目录下即可自动发现，无需改框架代码。

{symtest_cli-1.4.1 → symtest_cli-1.5.0}/docs/user_manual_en.md

@@ -1970,11 +1970,35 @@ class MySetup(BaseSetup):
 
 ### Custom File Comparator
 
-The framework supports three ways to extend comparison capabilities:
+#### Three-Lane Architecture (v2)
 
-#### Method 1: Workspace Plugin Directory (Recommended)
+The comparator plugin system is organized around the root contract
+`ComparatorBase.compare(ctx) -> ComparisonResult`, split by verdict ownership
+into three lanes:
 
-Create a `comparators/` directory under your workspace and place `*_comparator.py` files inside (following the same naming convention as built-in comparators). The framework auto-discovers and registers the `*Comparator` classes on first use.
+| Lane | Base class | Use case | Verdict owned by |
+|---|---|---|---|
+| **File lane** | `FileComparator` | Comparing two same-kind files (built-in text/json/csv/xml/h5/binary) | Plugin |
+| **Data lane** | `ExtractorComparator` | Extracting numeric data from any upstream (files / solver output / cross-file aggregation), reusing the framework numeric core for per-channel tolerance verdicts | **Framework** |
+| **Autonomous lane** | extend `ComparatorBase` directly | Custom verdict logic (composite thresholds, label parsing, sign/asymptotic checks) | Plugin |
+
+Selection guide:
+
+- Data is already `(expected, actual)` numeric arrays needing only rtol/atol → **data lane** (channels, per-channel tolerances, unified error_stats and report rendering)
+- Custom verdict needed (e.g. "tight on main components, loose on EAS components", sign checks) → **autonomous lane**
+- Just want to wire an existing script in quickly → built-in `script` (custom verdict) or `script_extract` (framework tolerance verdict), zero plugin code
+
+**Path resolution convention**: `actual`/`baseline` and any constructor params
+declared in the plugin's `path_params` class attribute are resolved against
+the workspace by the framework **before the comparator is constructed** —
+constructor-captured state (`self.script`, `self.cwd`, ...) already holds
+absolute paths. Plugins must
+**never** resolve paths against CWD — it is unreliable under parallel/process
+execution.
+
+#### Plugin Discovery (common to all lanes)
+
+Create a `comparators/` directory under your workspace and place `*_comparator.py` files inside. The framework auto-discovers and registers the `*Comparator` classes on first use.
 
 ```
 your-workspace/
@@ -1991,26 +2015,155 @@ symtest run test_config.json --plugin-dir ./extra_plugins
 
 Plugins are also automatically inherited by process-mode child processes via the `CLITEST_PLUGIN_DIRS` environment variable.
 
-**Plugin development notes**:
-- Inherit from `symtest.file_comparator.BaseComparator`
+**Naming conventions**:
+- File name must end with `_comparator.py` (e.g., `my_analysis_comparator.py`)
 - Class name must end with `Comparator` (e.g., `MyAnalysisComparator`)
-- The registered type name = class name without `Comparator`, lowercased (e.g., `myanalysis`)
-- Prefer overriding the `compare_files(file1, file2, **kwargs)` method over `read_content`/`compare_content` (if your comparator doesn't use the two-file model)
-- Construct structured results via `from symtest.file_comparator import ComparisonResult, Difference`
-- `extra_kwargs` are auto-passed from the config `compareSpec`
-
-Use the registered type name directly in your config:
+- The registered type name = class name without `Comparator`, lowercased (e.g., `myanalysis`); a `comparator_type = "my_analysis"` class attribute overrides it
+- Abstract base classes (with unimplemented abstract methods) are skipped automatically
+
+Use the registered type name directly in your config. Comparator constructor
+parameters are **strict**: keys other than `actual`/`baseline`/`type` are
+forwarded to the plugin constructor, but a parameter the plugin does not
+declare (e.g. a typo like `pass_threhsold`) **fails loudly** at construction
+(the error names the comparator type and its supported parameters) — it never
+silently falls back to defaults. Plugin-owned configuration should go into
+the `options` namespace (entries merged into constructor kwargs; explicit
+top-level keys take precedence), keeping the core schema generic:
 
 ```json
 {
   "type": "myanalysis",
   "actual": "optional_for_plugins",
   "baseline": "optional_for_plugins",
-  "param1": "value1"
+  "options": {"param1": "value1", "param2": 42}
 }
 ```
 
-#### Method 2: Built-in `script` Type Comparator
+#### Data Lane: Channel Extractor Plugin
+
+Use case: one script/plugin produces multi-channel data (e.g. different CSV
+columns, different physical quantities), each channel needing independent
+tolerances and error analysis. The plugin only EXTRACTS; the framework
+owns the verdict entirely.
+
+```python
+# comparators/uel_stress_comparator.py
+import numpy as np
+from symtest.file_comparator import (
+    ExtractorComparator, ChannelData, CompareContext,
+)
+
+class UelStressComparator(ExtractorComparator):
+    path_params = ("uel_csv", "ref_csv")   # workspace-resolved by framework
+
+    def __init__(self, uel_csv="", ref_csv="", **kwargs):
+        super().__init__(**kwargs)
+        self.uel_csv = uel_csv
+        self.ref_csv = ref_csv
+
+    def extract(self, ctx: CompareContext):
+        uel = np.loadtxt(self.uel_csv, delimiter=",")
+        ref = np.loadtxt(self.ref_csv, delimiter=",")
+        return {
+            "S11": ChannelData(expected=ref[:, 0], actual=uel[:, 0]),
+            "S33": ChannelData(
+                expected=ref[:, 2], actual=uel[:, 2],
+                extra_stats={"hydrostatic_shift": float(uel[:, 2].mean())},
+            ),
+        }
+```
+
+Config (per-channel tolerances routed by name; unlisted channels use `default_channel`):
+
+```json
+{
+  "type": "uelstress",
+  "uel_csv": "out/uel.csv",
+  "ref_csv": "out/native.csv",
+  "channels": {
+    "S33": {"atol": 600.0, "data_filter": "abs>1e-12"}
+  },
+  "default_channel": {"rtol": 1e-5, "atol": 1e-8}
+}
+```
+
+**Verdict and result semantics**:
+- Each channel runs `compare_numeric` independently (symmetric tolerance `|a-b| <= max(rtol*max(|a|,|b|), atol)`)
+- `identical = all channels pass`; one compareSpec is still one assertion
+- Difference positions carry the channel prefix (`channel S33`); `error_stats` nests by channel name
+- The report shows per-channel pass/fail, tolerances and stats; channel differences are trimmed per channel in JSON output
+- Custom error metrics ride along via `ChannelData.extra_stats`, kept in a SEPARATE namespace (`ChannelResult.extra_stats`) that can never overwrite framework canonical metrics (`max_abs_error`, `total`, ...); the report renders both namespaces per channel
+- The verdict itself cannot be altered by the plugin — custom verdicts belong to the autonomous lane
+
+#### Data Lane: Built-in `script_extract` Type (Zero-Modification Script Access)
+
+Existing analysis scripts need no Python plugin wrapper: the script runs as a
+subprocess and prints a JSON channel payload on stdout; each channel then
+enters the framework-owned tolerance pipeline described above.
+
+```json
+{
+  "type": "script_extract",
+  "script": "extract_channels.py",
+  "args": ["--frame", "10"],
+  "cwd": "case/subdir",
+  "channels": {"S33": {"atol": 600.0}},
+  "default_channel": {"rtol": 1e-5, "atol": 1e-8},
+  "timeout": 600
+}
+```
+
+**stdout JSON protocol**:
+
+```json
+{
+  "channels": {
+    "S11": {"expected": [1.0, 2.0], "actual": [1.0, 2.0]},
+    "S33": {"expected": [...], "actual": [...], "extra_stats": {"asymmetry": 2e-13}}
+  }
+}
+```
+
+**Error semantics**: non-zero exit code, timeout, malformed JSON, a missing
+`channels` key, or a channel missing `expected`/`actual` arrays → always a
+comparison error (never a silent pass). `actual`/`baseline` (if configured)
+are appended as trailing script arguments with baseline first; both optional.
+
+#### Autonomous Lane: Full-Verdict Plugin
+
+Use case: custom verdict logic. The plugin implements `compare(ctx)` and
+returns the result directly.
+
+```python
+# comparators/my_analysis_comparator.py
+from symtest.file_comparator import ComparatorBase, CompareContext, ComparisonResult, Difference
+
+class MyAnalysisComparator(ComparatorBase):
+    path_params = ("script", "case_dir")
+
+    def __init__(self, script="", case_dir=None, pass_threshold=1e-6):
+        super().__init__()   # strict: undeclared/misspelled config keys fail loudly
+        self.script = script      # workspace-resolved by the framework before construction
+        self.case_dir = case_dir
+        self.pass_threshold = pass_threshold
+
+    def compare(self, ctx: CompareContext) -> ComparisonResult:
+        result = ComparisonResult(
+            file1=ctx.baseline,   # stays None when there is no file input — no fake ""
+            file2=ctx.actual,
+        )
+        # ... run analysis, parse metrics ...
+        result.identical = True  # or False + differences
+        result.error_stats = {"full_rel": 8e-8}   # free-form dict, generic rendering
+        return result
+```
+
+`ctx` fields (invocation-level context only): `workspace` / `actual` /
+`baseline` (`None` when absent) / `params` (file-lane window ranges) /
+`error_analysis`.  The single authoritative copy of comparator configuration
+lives in constructor-captured state — `ctx` never duplicates it.
+
+#### Built-in `script` Type Comparator (Autonomous Lane Out of the Box)
 
 For quickly integrating standalone analysis scripts without writing a comparator class:
 
@@ -2033,8 +2186,8 @@ For quickly integrating standalone analysis scripts without writing a comparator
 | Parameter | Required | Default | Description |
 |---|---|---|---|
 | `script` | Yes | — | Script path (relative to workspace or absolute) |
-| `actual` | No | — | First file argument passed to the script |
-| `baseline` | No | — | Second file argument passed to the script |
+| `actual` | No | — | File argument passed to the script (trailing second slot) |
+| `baseline` | No | — | File argument passed to the script (trailing first slot) |
 | `cwd` | No | — | Script working directory |
 | `interpreter` | No | `sys.executable` | Python interpreter |
 | `pass_exit_code` | No | `0` | Exit code considered a pass |
@@ -2050,15 +2203,15 @@ For quickly integrating standalone analysis scripts without writing a comparator
 
 The script's stdout and stderr are fully captured in the `Comparator Output` section, displayed up to 20 lines in the rendered report.
 
-#### Method 3: Manual Registration (Programmatic)
+#### Manual Registration (Programmatic)
 
 ```python
-from symtest.file_comparator import ComparatorFactory
-from symtest.file_comparator.base_comparator import BaseComparator
+from symtest.file_comparator import ComparatorFactory, ComparatorBase, CompareContext
+from symtest.file_comparator.result import ComparisonResult
 
-class FooComparator(BaseComparator):
-    # Implement read_content / compare_content etc.
-    pass
+class FooComparator(ComparatorBase):
+    def compare(self, ctx: CompareContext) -> ComparisonResult:
+        ...
 
 ComparatorFactory.register_comparator("foo", FooComparator)
 
@@ -2068,7 +2221,7 @@ comparator = ComparatorFactory.create_comparator("foo")
 
 #### Specialized Plugin Example: Hourglass Tangent Stiffness Analysis
 
-`examples/plugins/hourglass_tangent_comparator.py` is a complete workspace plugin example demonstrating how to integrate a dedicated `analyze_*_tangent.py` analysis script into the framework:
+`examples/plugins/hourglass_tangent_comparator.py` is a complete autonomous-lane workspace plugin example demonstrating how to integrate a dedicated `analyze_*_tangent.py` analysis script into the framework:
 
 ```json
 {
@@ -2083,6 +2236,7 @@ comparator = ComparatorFactory.create_comparator("foo")
 **Features**:
 - Calls the analysis script via subprocess (**zero changes** to analyze code), parses `RESULT:` lines and numerical metrics like `full_rel`/`aa_rel`/`hh_rel`/`asymmetry` from stdout using regex
 - Constructs a structured `ComparisonResult`: `identical` determined by `full_rel < pass_threshold`; `differences` lists exceeded metrics; `error_stats` contains all numeric values
+- `path_params` declares `script`/`case_dir`, resolved against the workspace by the framework
 - Script stdout goes into the `Comparator Output` section
 
 Usage: copy the plugin file into your workspace's `comparators/` directory for auto-discovery — no framework code changes needed.

{symtest_cli-1.4.1 → symtest_cli-1.5.0}/src/symtest/__init__.py

@@ -16,7 +16,7 @@ runtime.  Library users can enable console logging explicitly::
     setup_console_logging(level=logging.DEBUG)
 """
 
-__version__ = "1.4.1"
+__version__ = "1.5.0"
 __author__ = "Xiaotong Wang"
 __email__ = "xiaotongwang98@gmail.com"