@rpamis/comet 0.3.9 → 0.4.0-beta.10

package/eval/langsmith/README.md

@@ -0,0 +1,93 @@
+# LangSmith 评估套件
+
+这个套件复用 local 的任务集，并把每次 `(task, treatment)` 运行同步到 LangSmith：
+
+- 每个任务是一个 dataset example（`inputs` = 任务/treatment，`reference_outputs` = 预期产物 / 必需 Skill / rubric 判据）。
+- 每个 treatment 是一个 experiment；`CONTROL` 与注入 Skill 的 treatment 可在实验对比页并排看。
+- 每个 run 记录 `outputs`（轮次、工具调用、耗时、token、成本、调用的 Skill）和 feedback（每个 rubric 维度分 + `checks_pass_rate`）。
+- 可选：Claude Code 的完整轨迹（trajectory）通过官方 `langsmith-tracing` 插件嵌套在同一个 run 下。
+
+默认复用：
+
+- `../local/tasks`
+- `../local/treatments`
+- `../local/skills`
+
+报告和产物写入：
+
+```text
+logs/experiments/
+```
+
+## 安装
+
+```bash
+cd eval
+uv sync --extra langsmith
+```
+
+## 运行
+
+结果上报（rubric + treatment 对比）只需要 `LANGSMITH_API_KEY`。推荐在 `eval/.env` 或 `eval/langsmith/.env` 中设置：
+
+```bash
+LANGSMITH_API_KEY=lsv2_pt_...
+LANGSMITH_PROJECT=comet-skill-eval
+LANGSMITH_TRACING=true
+```
+
+Comet 会从这组 `LANGSMITH_*` 配置自动派生 Claude Code 官方插件需要的 `TRACE_TO_LANGSMITH`、`CC_LANGSMITH_API_KEY` 和 `CC_LANGSMITH_PROJECT`。只有需要覆盖插件行为时，才需要显式设置这些 `CC_*` / `TRACE_TO_LANGSMITH` 变量。
+
+从 `eval/` 目录运行：
+
+```bash
+# 一个 treatment = 一个 experiment；用 LANGSMITH_EXPERIMENT 命名便于对比
+LANGSMITH_EXPERIMENT=COMET_FULL_040_BETA \
+  uv run pytest langsmith/tests/tasks/test_tasks.py \
+  --task=comet-fix-median --treatment=COMET_FULL_040_BETA -v
+
+# 再跑 CONTROL 作为对照
+LANGSMITH_EXPERIMENT=CONTROL \
+  uv run pytest langsmith/tests/tasks/test_tasks.py \
+  --task=comet-fix-median --treatment=CONTROL -v
+```
+
+生成 Skill manifest 示例：
+
+```bash
+uv run pytest langsmith/tests/tasks/test_tasks.py \
+  --eval-manifest=/path/to/my-skill/comet/eval.yaml -v
+```
+
+## 轨迹追踪（可选，官方 Claude Code 插件）
+
+轨迹追踪用官方 [`langsmith-tracing`](https://github.com/langchain-ai/langsmith-claude-code-plugins) 插件，容器内以 headless（`claude -p`）方式启用。默认情况下，LangSmith suite 会在首次运行时用 `node:20` 一次性构建插件到 `eval/.cache/langsmith-cc-plugin`，后续运行复用这个缓存目录并挂载到任务容器的 `/opt/langsmith-cc-plugin`。
+
+因为 eval 在 Linux 容器里运行，不要在 Windows 宿主机直接 `pnpm build` 后挂载，避免跨平台 `node_modules` 失效。你通常只需要设置：
+
+```bash
+LANGSMITH_TRACING=true
+```
+
+如果要提前手动构建缓存，可以从 `eval/` 目录运行：
+
+```bash
+docker run --rm -v "$PWD/.cache:/out" node:20 sh -c \
+  "cd /tmp && git clone https://github.com/langchain-ai/langsmith-claude-code-plugins && cd langsmith-claude-code-plugins && corepack enable && pnpm install && pnpm build && cp -r . /out/langsmith-cc-plugin"
+```
+
+PowerShell 中可用 `${PWD}`：
+
+```powershell
+docker run --rm -v "${PWD}\.cache:/out" node:20 sh -c "cd /tmp && git clone https://github.com/langchain-ai/langsmith-claude-code-plugins && cd langsmith-claude-code-plugins && corepack enable && pnpm install && pnpm build && cp -r . /out/langsmith-cc-plugin"
+```
+
+如果要覆盖默认缓存目录，可以把构建好的插件目录显式指给评估（宿主机路径）：
+
+```bash
+export CC_LANGSMITH_PLUGIN_DIR=/abs/path/to/comet/eval/.cache/langsmith-cc-plugin
+```
+
+- 设置后，插件目录会以 `/opt/langsmith-cc-plugin` 只读挂载进容器，`claude` 加 `--plugin-dir` 启用；轨迹通过自动派生的 Claude Code 插件配置和 `CC_LANGSMITH_PARENT_DOTTED_ORDER` 嵌套在对应 pytest run 下。
+- 未设置时，轨迹追踪自动跳过（不报错），rubric 与 treatment 对比照常上报。
+- 如果不希望 suite 自动构建插件，设置 `CC_LANGSMITH_PLUGIN_AUTO_BUILD=false`。

package/eval/langsmith/skills/README.md

@@ -0,0 +1,5 @@
+# LangSmith Skills
+
+这个套件当前通过 `BENCH_SKILLS_DIR` 复用 `../../local/skills`。
+
+如果实验需要 LangSmith 专用的 Skill 变体，可以放在这里。

package/eval/langsmith/tasks/README.md

@@ -0,0 +1,5 @@
+# LangSmith Tasks
+
+这个套件当前通过 `BENCH_TASKS_DIR` 复用 `../../local/tasks`。
+
+如果需要不同的任务集，可以在这里添加 LangSmith 专用任务，然后更新 `tests/conftest.py`，让 `BENCH_TASKS_DIR` 指向该目录。

package/eval/langsmith/tests/conftest.py

@@ -0,0 +1,187 @@
+"""LangSmith eval suite pytest configuration.
+
+This suite reuses the local task corpus by default and writes reports under
+eval/langsmith/logs. LangSmith-specific environment checks live here so local
+runs stay free of LangSmith requirements.
+"""
+
+import importlib.util
+import os
+import subprocess
+from pathlib import Path
+
+import pytest
+from dotenv import load_dotenv
+from scaffold.python.utils import _to_bash_path
+
+LANGSMITH_ROOT = Path(__file__).resolve().parents[1]
+EVAL_ROOT = LANGSMITH_ROOT.parent
+LOCAL_ROOT = EVAL_ROOT / "local"
+DEFAULT_LANGSMITH_PLUGIN_DIR = EVAL_ROOT / ".cache" / "langsmith-cc-plugin"
+
+os.environ.setdefault("BENCH_SUITE_ROOT", str(LANGSMITH_ROOT))
+os.environ.setdefault("BENCH_TASKS_DIR", str(LOCAL_ROOT / "tasks"))
+os.environ.setdefault("BENCH_TREATMENTS_DIR", str(LOCAL_ROOT / "treatments"))
+os.environ.setdefault("BENCH_SKILLS_DIR", str(LOCAL_ROOT / "skills"))
+os.environ.setdefault("BENCH_LOGS_DIR", str(LANGSMITH_ROOT / "logs"))
+# Group every run in this suite under one LangSmith dataset/test-suite by default.
+os.environ.setdefault("LANGSMITH_TEST_SUITE", "comet-skill-eval")
+
+load_dotenv(EVAL_ROOT / ".env")
+load_dotenv(LANGSMITH_ROOT / ".env", override=True)
+
+
+def configure_langsmith_environment() -> None:
+    """Derive Claude Code plugin env from the eval suite's LangSmith config."""
+    os.environ.setdefault("LANGSMITH_TRACING", "true")
+    os.environ.setdefault(
+        "TRACE_TO_LANGSMITH",
+        "true" if os.environ.get("LANGSMITH_TRACING", "").lower() == "true" else "false",
+    )
+    os.environ.setdefault("LANGSMITH_PROJECT", "comet-skill-eval")
+
+    api_key = os.environ.get("LANGSMITH_API_KEY", "")
+    if api_key:
+        os.environ.setdefault("CC_LANGSMITH_API_KEY", api_key)
+    os.environ.setdefault("CC_LANGSMITH_PROJECT", os.environ["LANGSMITH_PROJECT"])
+
+
+configure_langsmith_environment()
+
+_LOCAL_CONFTEST = LOCAL_ROOT / "tests" / "conftest.py"
+_spec = importlib.util.spec_from_file_location("_comet_local_conftest", _LOCAL_CONFTEST)
+_local_conftest = importlib.util.module_from_spec(_spec)
+assert _spec and _spec.loader
+_spec.loader.exec_module(_local_conftest)
+
+for _name in dir(_local_conftest):
+    if not _name.startswith("__") and _name not in globals():
+        globals()[_name] = getattr(_local_conftest, _name)
+
+
+@pytest.fixture(scope="session", autouse=True)
+def verify_langsmith_environment(request):
+    """Require LangSmith credentials only for non-unit LangSmith eval runs."""
+    if _local_conftest._is_unit_tests_only(request.config):
+        return
+    if not os.environ.get("LANGSMITH_API_KEY"):
+        pytest.skip("LANGSMITH_API_KEY not set")
+
+
+def _build_default_langsmith_plugin(target_dir: Path) -> bool:
+    """Build the official Claude Code LangSmith plugin into eval/.cache."""
+    target_dir.parent.mkdir(parents=True, exist_ok=True)
+    mount_arg = f"{target_dir.parent}:/out"
+    command = (
+        "set -e; "
+        "cd /tmp; "
+        "git clone --depth 1 https://github.com/langchain-ai/langsmith-claude-code-plugins; "
+        "cd langsmith-claude-code-plugins; "
+        "corepack enable; "
+        "pnpm install; "
+        "pnpm build; "
+        "rm -rf /out/langsmith-cc-plugin; "
+        "cp -r . /out/langsmith-cc-plugin"
+    )
+    result = subprocess.run(
+        ["docker", "run", "--rm", "-v", mount_arg, "node:20", "sh", "-c", command],
+        capture_output=True,
+        text=True,
+        timeout=600,
+    )
+    if result.returncode == 0:
+        return True
+    print("[langsmith] failed to auto-build Claude Code tracing plugin.")
+    if result.stdout.strip():
+        print(result.stdout.strip())
+    if result.stderr.strip():
+        print(result.stderr.strip())
+    return False
+
+
+def _plugin_dir_env_value(plugin_dir: Path) -> str:
+    """Return a plugin path value that docker.sh can test and mount from bash."""
+    return _to_bash_path(plugin_dir)
+
+
+def provision_langsmith_plugin_dir() -> Path | None:
+    """Resolve or auto-build the official Claude Code LangSmith plugin directory."""
+    configure_langsmith_environment()
+    if os.environ.get("TRACE_TO_LANGSMITH", "").lower() != "true":
+        return None
+
+    explicit_dir = os.environ.get("CC_LANGSMITH_PLUGIN_DIR", "").strip()
+    if explicit_dir:
+        plugin_dir = Path(explicit_dir)
+        if plugin_dir.is_dir():
+            os.environ["CC_LANGSMITH_PLUGIN_DIR"] = _plugin_dir_env_value(plugin_dir)
+            return plugin_dir
+        print(
+            f"[langsmith] CC_LANGSMITH_PLUGIN_DIR not a directory ({plugin_dir}); "
+            "falling back to the eval plugin cache."
+        )
+        os.environ.pop("CC_LANGSMITH_PLUGIN_DIR", None)
+
+    plugin_dir = DEFAULT_LANGSMITH_PLUGIN_DIR
+    if plugin_dir.is_dir():
+        os.environ["CC_LANGSMITH_PLUGIN_DIR"] = _plugin_dir_env_value(plugin_dir)
+        return plugin_dir
+
+    if os.environ.get("CC_LANGSMITH_PLUGIN_AUTO_BUILD", "true").lower() in {
+        "0",
+        "false",
+        "no",
+    }:
+        print(
+            "[langsmith] trajectory tracing disabled: "
+            "CC_LANGSMITH_PLUGIN_AUTO_BUILD=false and no plugin directory was provided."
+        )
+        return None
+
+    print(
+        "[langsmith] building Claude Code tracing plugin into "
+        f"{plugin_dir} via node:20..."
+    )
+    if _build_default_langsmith_plugin(plugin_dir) and plugin_dir.is_dir():
+        os.environ["CC_LANGSMITH_PLUGIN_DIR"] = _plugin_dir_env_value(plugin_dir)
+        return plugin_dir
+
+    os.environ.pop("CC_LANGSMITH_PLUGIN_DIR", None)
+    return None
+
+
+@pytest.fixture(scope="session", autouse=True)
+def provision_langsmith_tracing(request):
+    """Wire the official langsmith-tracing Claude Code plugin, best-effort.
+
+    Results logging (rubric scores + treatment comparison) only needs a
+    LANGSMITH_API_KEY. Trajectory tracing additionally needs the prebuilt
+    ``langsmith-tracing`` plugin, provided via ``CC_LANGSMITH_PLUGIN_DIR``
+    (host path). When it is missing we log once and continue without the
+    nested Claude Code trajectory instead of failing the run.
+    """
+    if _local_conftest._is_unit_tests_only(request.config):
+        return
+
+    plugin_dir = provision_langsmith_plugin_dir()
+    if plugin_dir:
+        print(f"[langsmith] trajectory tracing enabled via plugin: {plugin_dir}")
+    else:
+        print(
+            "[langsmith] trajectory tracing disabled "
+            "(rubric + comparison still logged)."
+        )
+
+
+@pytest.fixture(scope="session")
+def langsmith_experiment_metadata():
+    """Attach suite metadata to each LangSmith experiment (project).
+
+    Read by the LangSmith pytest plugin (requires langsmith>=0.7.13); ignored by
+    older versions.
+    """
+    return {
+        "suite": "comet-skill-eval",
+        "cc_model": os.environ.get("BENCH_CC_MODEL", "cli-default"),
+        "cc_version": os.environ.get("BENCH_CC_VERSION", "latest"),
+    }

package/eval/langsmith/tests/tasks/test_tasks.py

@@ -0,0 +1,287 @@
+"""LangSmith suite task runner.
+
+Wraps the local task runner with LangSmith's pytest integration. Each
+``(task, treatment)`` run syncs to a LangSmith dataset example + experiment:
+
+- ``log_inputs`` / ``log_reference_outputs`` capture the task prompt and the
+  ground-truth expectations (expected artifacts, required skills, rubric).
+- ``log_outputs`` captures the run's efficiency metrics and invoked skills.
+- ``log_feedback`` reports each rubric dimension score (and the check pass rate),
+  so CONTROL vs skill-injected treatments compare directly in the experiment view.
+- Trajectory tracing (official ``langsmith-tracing`` Claude Code plugin) nests
+  under the same run via ``CC_LANGSMITH_PARENT_DOTTED_ORDER`` when a prebuilt
+  plugin dir is provided through ``CC_LANGSMITH_PLUGIN_DIR``.
+
+The local suite stays completely free of any LangSmith import; this module only
+adds a thin logging wrapper around the unchanged local runner.
+"""
+
+import importlib.util
+import json
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+from scaffold.python.tasks import load_task
+from scaffold.python.logging import TreatmentResult
+
+# ---------------------------------------------------------------------------
+# Load the local runner as a module (do NOT re-export its test_task_treatment;
+# we wrap it below). pytest_generate_tests and any non-task unit tests are
+# re-exported unchanged so parametrization and coverage stay identical.
+# ---------------------------------------------------------------------------
+_LOCAL_TEST_TASKS = (
+    Path(__file__).resolve().parents[3] / "local" / "tests" / "tasks" / "test_tasks.py"
+)
+_spec = importlib.util.spec_from_file_location("_comet_local_test_tasks", _LOCAL_TEST_TASKS)
+_local_test_tasks = importlib.util.module_from_spec(_spec)
+assert _spec and _spec.loader
+sys.modules[_spec.name] = _local_test_tasks
+_spec.loader.exec_module(_local_test_tasks)
+
+pytest_generate_tests = _local_test_tasks.pytest_generate_tests
+for _name in dir(_local_test_tasks):
+    if _name.startswith("test_") and _name != "test_task_treatment":
+        globals()[_name] = getattr(_local_test_tasks, _name)
+
+PYTEST_TIMEOUT = _local_test_tasks.PYTEST_TIMEOUT
+_run_local_task_treatment = _local_test_tasks.test_task_treatment
+
+# LangSmith helpers are optional; degrade to a plain pass-through when the
+# ``langsmith`` extra is not installed so collection never crashes.
+try:
+    from langsmith import testing as ls_testing
+    from langsmith import get_current_run_tree
+    from langsmith.testing import _internal as ls_testing_internal
+
+    _LANGSMITH_AVAILABLE = True
+except Exception:  # pragma: no cover - import guard
+    ls_testing = None
+    get_current_run_tree = None
+    ls_testing_internal = None
+    _LANGSMITH_AVAILABLE = False
+
+try:
+    from scaffold.python.logging import _rubric_scores as _extract_rubric_scores
+except Exception:  # pragma: no cover - fallback if private helper moves
+    import re as _re
+
+    _RUBRIC_RE = _re.compile(r"\[RUBRIC\]\s+(\S+):\s*([0-9.]+)")
+
+    def _extract_rubric_scores(result):
+        scores = {}
+        for check in getattr(result, "checks_passed", []):
+            match = _RUBRIC_RE.search(check)
+            if match:
+                try:
+                    scores[match.group(1)] = float(match.group(2))
+                except ValueError:
+                    continue
+        return scores
+
+
+# =============================================================================
+# LANGSMITH LOGGING HELPERS (all best-effort; never break the eval run)
+# =============================================================================
+
+
+def _safe(fn, label=None):
+    """Call a LangSmith logging fn, swallowing errors so tracing never fails a run."""
+    try:
+        fn()
+    except Exception as exc:  # pragma: no cover - defensive
+        if label:
+            print(f"[langsmith] {label} failed: {exc}", file=sys.stderr)
+
+
+def _install_failed_end_run_outputs_patch():
+    """Preserve log_outputs payloads when LangSmith ends a failed pytest run."""
+    if not _LANGSMITH_AVAILABLE or ls_testing_internal is None:
+        return
+    test_case_cls = getattr(ls_testing_internal, "_TestCase", None)
+    if test_case_cls is None:
+        return
+    original = getattr(test_case_cls, "end_run", None)
+    if original is None or getattr(original, "_comet_preserve_logged_outputs", False):
+        return
+
+    def end_run(self, run_tree, outputs):
+        if outputs is None and getattr(self, "_logged_outputs", None) is not None:
+            outputs = self._logged_outputs
+        return original(self, run_tree, outputs)
+
+    end_run._comet_preserve_logged_outputs = True
+    end_run._comet_original = original
+    test_case_cls.end_run = end_run
+
+
+_install_failed_end_run_outputs_patch()
+
+
+def _log_inputs_and_reference(task_name, treatment_name):
+    if not _LANGSMITH_AVAILABLE:
+        return
+    try:
+        task = load_task(task_name)
+        evaluation = task.config.evaluation
+        _safe(
+            lambda: ls_testing.log_inputs(
+                {
+                    "task": task_name,
+                    "treatment": treatment_name,
+                    "difficulty": getattr(task.config.metadata, "difficulty", None),
+                }
+            )
+        )
+        _safe(
+            lambda: ls_testing.log_reference_outputs(
+                {
+                    "expected_artifacts": list(evaluation.expected_artifacts or []),
+                    "required_skills": list(evaluation.required_skills or []),
+                    "rubric_criteria": list(evaluation.rubric_criteria or []),
+                }
+            )
+        )
+    except Exception:  # pragma: no cover - defensive
+        pass
+
+
+def _set_parent_run_env():
+    """Point the Claude Code plugin at this test's run so its trajectory nests here."""
+    if not _LANGSMITH_AVAILABLE or get_current_run_tree is None:
+        return None
+    try:
+        run_tree = get_current_run_tree()
+        dotted_order = getattr(run_tree, "dotted_order", None) if run_tree else None
+    except Exception:  # pragma: no cover - defensive
+        dotted_order = None
+    if dotted_order:
+        os.environ["CC_LANGSMITH_PARENT_DOTTED_ORDER"] = dotted_order
+    return dotted_order
+
+
+def _result_from_logger(logger, treatment_name):
+    runs = logger.results.get(treatment_name) or []
+    if runs:
+        return runs[-1]
+
+    parametrized_suffix = f"-{treatment_name}-"
+    for result_name, result_runs in reversed(list(logger.results.items())):
+        if result_name == treatment_name or parametrized_suffix in result_name:
+            return result_runs[-1] if result_runs else None
+    return None
+
+
+def _iter_result_loggers():
+    seen = set()
+    candidates = [
+        getattr(_local_test_tasks, "conftest", None),
+        sys.modules.get("conftest"),
+        sys.modules.get("_comet_local_conftest"),
+    ]
+    for module in candidates:
+        plugin = getattr(module, "_plugin", None) if module else None
+        logger = getattr(plugin, "logger", None) if plugin else None
+        if logger is not None and id(logger) not in seen:
+            seen.add(id(logger))
+            yield logger
+
+
+def _latest_report_result(treatment_name):
+    logs_dir = Path(os.environ.get("BENCH_LOGS_DIR", ""))
+    if not logs_dir:
+        return None
+    reports_root = logs_dir / "experiments"
+    if not reports_root.exists():
+        return None
+    parametrized_suffix = f"-{treatment_name}-"
+    report_files = sorted(
+        reports_root.glob("*/reports/*_report.json"),
+        key=lambda path: path.stat().st_mtime,
+        reverse=True,
+    )
+    for report_file in report_files:
+        try:
+            report = json.loads(report_file.read_text())
+        except Exception:
+            continue
+        result_name = report.get("name", "")
+        if result_name != treatment_name and parametrized_suffix not in result_name:
+            continue
+        return TreatmentResult(
+            name=result_name,
+            passed=report.get("passed", False),
+            checks_passed=report.get("checks_passed", []),
+            checks_failed=report.get("checks_failed", []),
+            events_summary=report.get("events_summary", {}),
+            run_id=report.get("run_id", ""),
+        )
+    return None
+
+
+def _latest_result(treatment_name):
+    for logger in _iter_result_loggers():
+        result = _result_from_logger(logger, treatment_name)
+        if result is not None:
+            return result
+    return _latest_report_result(treatment_name)
+
+
+def _log_outputs_and_feedback(result):
+    if not _LANGSMITH_AVAILABLE:
+        return
+    if result is None:
+        print("[langsmith] no local eval result found; outputs/feedback not logged.", file=sys.stderr)
+        return
+    summary = getattr(result, "events_summary", {}) or {}
+    outputs = {
+        "run_id": getattr(result, "run_id", ""),
+        "passed": getattr(result, "passed", None),
+        "checks_passed": len(getattr(result, "checks_passed", [])),
+        "checks_failed": len(getattr(result, "checks_failed", [])),
+        "num_turns": summary.get("num_turns"),
+        "tool_calls": summary.get("tool_calls"),
+        "duration_seconds": summary.get("duration_seconds"),
+        "total_tokens": summary.get("total_tokens"),
+        "total_cost_usd": summary.get("total_cost_usd"),
+        "skills_invoked": summary.get("skills_invoked", []),
+    }
+    _safe(
+        lambda: ls_testing.log_outputs(outputs)
+    )
+
+    passed = len(getattr(result, "checks_passed", []))
+    failed = len(getattr(result, "checks_failed", []))
+    total = passed + failed
+    if total:
+        _safe(
+            lambda: ls_testing.log_feedback(key="checks_pass_rate", score=passed / total),
+            label="log_feedback checks_pass_rate",
+        )
+
+    for dim, score in _extract_rubric_scores(result).items():
+        _safe(
+            lambda d=dim, s=score: ls_testing.log_feedback(key=f"rubric.{d}", score=s),
+            label=f"log_feedback rubric.{dim}",
+        )
+
+
+# =============================================================================
+# TEST
+# =============================================================================
+
+
+@pytest.mark.timeout(PYTEST_TIMEOUT)
+@pytest.mark.langsmith
+def test_task_treatment(task_name, treatment_name):
+    """Run a task+treatment via the local runner and log results to LangSmith."""
+    _log_inputs_and_reference(task_name, treatment_name)
+    _set_parent_run_env()
+    try:
+        _run_local_task_treatment(task_name, treatment_name)
+    finally:
+        _log_outputs_and_feedback(_latest_result(treatment_name))
+        os.environ.pop("CC_LANGSMITH_PARENT_DOTTED_ORDER", None)
+

package/eval/langsmith/treatments/README.md

@@ -0,0 +1,5 @@
+# LangSmith Treatments
+
+这个套件当前通过 `BENCH_TREATMENTS_DIR` 复用 `../../local/treatments`。
+
+如果需要不同的 Skill 组合方式，可以在这里添加 LangSmith 专用 treatment。