design-research-experiments 0.1.0__tar.gz

design_research_experiments-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 The Design Research Collective
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

design_research_experiments-0.1.0/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: design-research-experiments
+Version: 0.1.0
+Summary: Study-definition and orchestration layer for the cmudrc design research ecosystem
+Author: The Design Research Collective
+Maintainer-email: "Christopher C. McComb" <ccm@cmu.edu>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/cmudrc/design-research-experiments
+Project-URL: Repository, https://github.com/cmudrc/design-research-experiments
+Project-URL: Issues, https://github.com/cmudrc/design-research-experiments/issues
+Keywords: design,research,experiments,orchestration,doe
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Scientific/Engineering
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: PyYAML<7,>=6.0
+Provides-Extra: dev
+Requires-Dist: build<2,>=1.2; extra == "dev"
+Requires-Dist: mypy<2,>=1.10; extra == "dev"
+Requires-Dist: pre-commit<5,>=3.7; extra == "dev"
+Requires-Dist: pytest<9,>=8.2; extra == "dev"
+Requires-Dist: pytest-cov<8,>=7.0; extra == "dev"
+Requires-Dist: ruff<1,>=0.6.0; extra == "dev"
+Requires-Dist: sphinx<9,>=7.4; extra == "dev"
+Requires-Dist: sphinx-rtd-theme<4,>=2.0; extra == "dev"
+Requires-Dist: twine<7,>=5.1; extra == "dev"
+Requires-Dist: types-PyYAML<7,>=6.0; extra == "dev"
+Requires-Dist: uv<1,>=0.6; extra == "dev"
+Dynamic: license-file
+
+# design-research-experiments
+[![CI](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml/badge.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml)
+[![Docs](https://github.com/cmudrc/design-research-experiments/actions/workflows/docs-pages.yml/badge.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/docs-pages.yml)
+
+`design-research-experiments` is the hypothesis-first study-definition and
+experiment-orchestration layer in the cmudrc design research ecosystem.
+
+It composes sibling libraries rather than reimplementing them:
+
+- `design-research-agents` for executable agent behavior, workflows, and traces
+- `design-research-problems` for problem catalogs, registries, and evaluators
+- `design-research-analysis` for downstream unified-table analysis and reporting
+
+## Overview
+
+This package centers on reproducible experiment structure and execution:
+
+- typed schemas for studies, factors, blocks, hypotheses, outcomes, and analysis plans
+- design-of-experiments materialization (full/constrained factorial, randomized block,
+  repeated measures, latin square, custom matrices)
+- run orchestration with deterministic seeding, checkpointing, and resume support
+- canonical artifact exports (`study.yaml`, `manifest.json`, `conditions.csv`,
+  `runs.csv`, `events.csv`, `evaluations.csv`, and machine-readable hypothesis/plan files)
+- thin adapters that connect to the public APIs of sibling agent/problem/analysis libraries
+
+## Quickstart
+
+Requires Python 3.12+.
+Reproducible release installs are pinned to Python `3.12.12` (`.python-version`).
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+make dev
+make test
+```
+
+Run a basic example:
+
+```bash
+make run-example
+```
+
+## CLI
+
+The package installs a `drexp` CLI:
+
+```bash
+drexp validate-study path/to/study.yaml
+drexp materialize-design path/to/study.yaml
+drexp generate-doe --kind lhs --factors-json '{"x": [0, 1], "y": [10, 20]}' --n-samples 12 --out artifacts/doe.csv
+drexp run-study path/to/study.yaml
+drexp resume-study path/to/study.yaml
+drexp export-analysis path/to/study.yaml
+drexp bundle-results path/to/output_dir
+```
+
+## Examples
+
+See [examples/README.md](examples/README.md) for runnable scripts, including
+end-to-end recipe executions.
+
+## Docs
+
+See the [published documentation](https://cmudrc.github.io/design-research-experiments/)
+for guides and API reference.
+
+Build docs locally with:
+
+```bash
+make docs
+```
+
+## Public API
+
+Top-level exports are intentionally small:
+
+- `Study`, `Factor`, `Level`, `Constraint`, `Condition`, `Block`
+- `RecipeStudyConfig`, recipe-specific typed config classes
+- `Hypothesis`, `OutcomeSpec`, `AnalysisPlan`
+- `RunSpec`, `RunResult`, `BenchmarkBundle`
+- `build_design`, `generate_doe`, `materialize_conditions`
+- `build_prompt_framing_study`, `build_optimization_benchmark_study`, and other recipe builders
+- `run_study`, `resume_study`
+- `export_analysis_tables`, `validate_study`
+
+## Contributing
+
+Contribution workflow and quality gates are documented in `CONTRIBUTING.md`.

design_research_experiments-0.1.0/README.md

@@ -0,0 +1,89 @@
+# design-research-experiments
+[![CI](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml/badge.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/ci.yml)
+[![Docs](https://github.com/cmudrc/design-research-experiments/actions/workflows/docs-pages.yml/badge.svg)](https://github.com/cmudrc/design-research-experiments/actions/workflows/docs-pages.yml)
+
+`design-research-experiments` is the hypothesis-first study-definition and
+experiment-orchestration layer in the cmudrc design research ecosystem.
+
+It composes sibling libraries rather than reimplementing them:
+
+- `design-research-agents` for executable agent behavior, workflows, and traces
+- `design-research-problems` for problem catalogs, registries, and evaluators
+- `design-research-analysis` for downstream unified-table analysis and reporting
+
+## Overview
+
+This package centers on reproducible experiment structure and execution:
+
+- typed schemas for studies, factors, blocks, hypotheses, outcomes, and analysis plans
+- design-of-experiments materialization (full/constrained factorial, randomized block,
+  repeated measures, latin square, custom matrices)
+- run orchestration with deterministic seeding, checkpointing, and resume support
+- canonical artifact exports (`study.yaml`, `manifest.json`, `conditions.csv`,
+  `runs.csv`, `events.csv`, `evaluations.csv`, and machine-readable hypothesis/plan files)
+- thin adapters that connect to the public APIs of sibling agent/problem/analysis libraries
+
+## Quickstart
+
+Requires Python 3.12+.
+Reproducible release installs are pinned to Python `3.12.12` (`.python-version`).
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+make dev
+make test
+```
+
+Run a basic example:
+
+```bash
+make run-example
+```
+
+## CLI
+
+The package installs a `drexp` CLI:
+
+```bash
+drexp validate-study path/to/study.yaml
+drexp materialize-design path/to/study.yaml
+drexp generate-doe --kind lhs --factors-json '{"x": [0, 1], "y": [10, 20]}' --n-samples 12 --out artifacts/doe.csv
+drexp run-study path/to/study.yaml
+drexp resume-study path/to/study.yaml
+drexp export-analysis path/to/study.yaml
+drexp bundle-results path/to/output_dir
+```
+
+## Examples
+
+See [examples/README.md](examples/README.md) for runnable scripts, including
+end-to-end recipe executions.
+
+## Docs
+
+See the [published documentation](https://cmudrc.github.io/design-research-experiments/)
+for guides and API reference.
+
+Build docs locally with:
+
+```bash
+make docs
+```
+
+## Public API
+
+Top-level exports are intentionally small:
+
+- `Study`, `Factor`, `Level`, `Constraint`, `Condition`, `Block`
+- `RecipeStudyConfig`, recipe-specific typed config classes
+- `Hypothesis`, `OutcomeSpec`, `AnalysisPlan`
+- `RunSpec`, `RunResult`, `BenchmarkBundle`
+- `build_design`, `generate_doe`, `materialize_conditions`
+- `build_prompt_framing_study`, `build_optimization_benchmark_study`, and other recipe builders
+- `run_study`, `resume_study`
+- `export_analysis_tables`, `validate_study`
+
+## Contributing
+
+Contribution workflow and quality gates are documented in `CONTRIBUTING.md`.

design_research_experiments-0.1.0/pyproject.toml

@@ -0,0 +1,97 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "design-research-experiments"
+version = "0.1.0"
+description = "Study-definition and orchestration layer for the cmudrc design research ecosystem"
+readme = "README.md"
+requires-python = ">=3.12"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "The Design Research Collective" }]
+maintainers = [{ name = "Christopher C. McComb", email = "ccm@cmu.edu" }]
+keywords = ["design", "research", "experiments", "orchestration", "doe"]
+classifiers = [
+  "Development Status :: 3 - Alpha",
+  "Intended Audience :: Science/Research",
+  "Programming Language :: Python :: 3 :: Only",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.12",
+  "Operating System :: OS Independent",
+  "Topic :: Scientific/Engineering",
+  "Typing :: Typed",
+]
+dependencies = [
+  "PyYAML>=6.0,<7",
+]
+
+[project.optional-dependencies]
+dev = [
+  "build>=1.2,<2",
+  "mypy>=1.10,<2",
+  "pre-commit>=3.7,<5",
+  "pytest>=8.2,<9",
+  "pytest-cov>=7.0,<8",
+  "ruff>=0.6.0,<1",
+  "sphinx>=7.4,<9",
+  "sphinx-rtd-theme>=2.0,<4",
+  "twine>=5.1,<7",
+  "types-PyYAML>=6.0,<7",
+  "uv>=0.6,<1",
+]
+
+[project.urls]
+Homepage = "https://github.com/cmudrc/design-research-experiments"
+Repository = "https://github.com/cmudrc/design-research-experiments"
+Issues = "https://github.com/cmudrc/design-research-experiments/issues"
+
+[project.scripts]
+drexp = "design_research_experiments.cli:main"
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+include-package-data = true
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+design_research_experiments = ["py.typed"]
+
+[tool.pytest.ini_options]
+addopts = "-q"
+testpaths = ["tests"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py312"
+src = ["src", "tests", "examples", "scripts"]
+
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+skip-magic-trailing-comma = false
+line-ending = "auto"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM", "RUF", "D"]
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"
+
+[tool.ruff.lint.per-file-ignores]
+"tests/*.py" = ["D"]
+
+[tool.mypy]
+packages = ["design_research_experiments"]
+python_version = "3.12"
+warn_unused_configs = true
+check_untyped_defs = true
+disallow_untyped_defs = true
+no_implicit_optional = true
+warn_redundant_casts = true
+warn_unused_ignores = true
+warn_return_any = true
+strict_equality = true

design_research_experiments-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

design_research_experiments-0.1.0/src/design_research_experiments/__init__.py

@@ -0,0 +1,87 @@
+"""Public API exports for design-research-experiments."""
+
+from .adapters.analysis import export_analysis_tables
+from .adapters.problems import ProblemPacket
+from .bundles import (
+    BenchmarkBundle,
+    grammar_problem_bundle,
+    human_vs_agent_bundle,
+    ideation_bundle,
+    optimization_bundle,
+)
+from .conditions import Condition, Constraint, Factor, FactorKind, Level, materialize_conditions
+from .designs import build_design, generate_doe
+from .hypotheses import AnalysisPlan, Hypothesis, OutcomeSpec
+from .recipes import (
+    AgentArchitectureComparisonConfig,
+    DiversityAndExplorationConfig,
+    GrammarScaffoldConfig,
+    HumanVsAgentProcessConfig,
+    OptimizationBenchmarkConfig,
+    PromptFramingConfig,
+    RecipeStudyConfig,
+    build_agent_architecture_comparison_study,
+    build_diversity_and_exploration_study,
+    build_grammar_scaffold_study,
+    build_human_vs_agent_process_study,
+    build_optimization_benchmark_study,
+    build_prompt_framing_study,
+)
+from .reporting import (
+    render_codebook,
+    render_markdown_summary,
+    render_methods_scaffold,
+    render_significance_brief,
+    write_markdown_report,
+)
+from .runners import resume_study, run_study
+from .schemas import RunBudget, SeedPolicy
+from .study import Block, RunResult, RunSpec, Study, validate_study
+
+__all__ = [
+    "AgentArchitectureComparisonConfig",
+    "AnalysisPlan",
+    "BenchmarkBundle",
+    "Block",
+    "Condition",
+    "Constraint",
+    "DiversityAndExplorationConfig",
+    "Factor",
+    "FactorKind",
+    "GrammarScaffoldConfig",
+    "HumanVsAgentProcessConfig",
+    "Hypothesis",
+    "Level",
+    "OptimizationBenchmarkConfig",
+    "OutcomeSpec",
+    "ProblemPacket",
+    "PromptFramingConfig",
+    "RecipeStudyConfig",
+    "RunBudget",
+    "RunResult",
+    "RunSpec",
+    "SeedPolicy",
+    "Study",
+    "build_agent_architecture_comparison_study",
+    "build_design",
+    "build_diversity_and_exploration_study",
+    "build_grammar_scaffold_study",
+    "build_human_vs_agent_process_study",
+    "build_optimization_benchmark_study",
+    "build_prompt_framing_study",
+    "export_analysis_tables",
+    "generate_doe",
+    "grammar_problem_bundle",
+    "human_vs_agent_bundle",
+    "ideation_bundle",
+    "materialize_conditions",
+    "optimization_bundle",
+    "render_codebook",
+    "render_markdown_summary",
+    "render_methods_scaffold",
+    "render_significance_brief",
+    "resume_study",
+    "run_study",
+    "validate_study",
+    "write_markdown_report",
+]

design_research_experiments-0.1.0/src/design_research_experiments/adapters/__init__.py

@@ -0,0 +1,16 @@
+"""Integration adapters for agents, problems, and downstream analysis."""
+
+from .agents import AgentExecution, execute_agent, resolve_agent
+from .analysis import export_analysis_tables
+from .problems import ProblemPacket, evaluate_problem, resolve_problem, sample_problem_packets
+
+__all__ = [
+    "AgentExecution",
+    "ProblemPacket",
+    "evaluate_problem",
+    "execute_agent",
+    "export_analysis_tables",
+    "resolve_agent",
+    "resolve_problem",
+    "sample_problem_packets",
+]

design_research_experiments-0.1.0/src/design_research_experiments/adapters/agents.py

@@ -0,0 +1,248 @@
+"""Agent-layer adapter utilities built on public agent APIs."""
+
+from __future__ import annotations
+
+import importlib
+import inspect
+from collections.abc import Callable, Mapping, Sequence
+from dataclasses import dataclass, field
+from typing import Any
+
+from ..conditions import Condition
+from ..schemas import Observation, ObservationLevel, ValidationError, hash_identifier, utc_now_iso
+from ..study import RunSpec
+from .problems import ProblemPacket
+
+
+@dataclass(slots=True)
+class AgentExecution:
+    """Normalized agent execution bundle."""
+
+    output: dict[str, Any] = field(default_factory=dict)
+    metrics: dict[str, Any] = field(default_factory=dict)
+    events: list[Observation] = field(default_factory=list)
+    trace_refs: list[str] = field(default_factory=list)
+    metadata: dict[str, Any] = field(default_factory=dict)
+
+
+def resolve_agent(
+    agent_spec_ref: Any,
+    *,
+    condition: Condition,
+    factories: Mapping[str, Callable[[Condition], Any]] | None = None,
+) -> Any:
+    """Resolve an agent reference into an executable object."""
+    if isinstance(agent_spec_ref, str):
+        if factories and agent_spec_ref in factories:
+            return factories[agent_spec_ref](condition)
+
+        maybe_agent = _resolve_from_design_research_agents(agent_spec_ref)
+        if maybe_agent is not None:
+            return maybe_agent
+
+        raise ValidationError(
+            "Unknown agent spec "
+            f"'{agent_spec_ref}'. Register a factory or pass an executable object."
+        )
+
+    return agent_spec_ref
+
+
+def execute_agent(
+    *,
+    agent_spec_ref: Any,
+    run_spec: RunSpec,
+    condition: Condition,
+    problem_packet: ProblemPacket,
+    factories: Mapping[str, Callable[[Condition], Any]] | None = None,
+) -> AgentExecution:
+    """Execute one agent run and normalize outputs, events, and trace refs."""
+    executable = resolve_agent(agent_spec_ref, condition=condition, factories=factories)
+    raw = _invoke_agent(
+        executable=executable,
+        run_spec=run_spec,
+        condition=condition,
+        problem_packet=problem_packet,
+    )
+    return _normalize_agent_execution(raw=raw, run_spec=run_spec, condition=condition)
+
+
+def _resolve_from_design_research_agents(agent_id: str) -> Any | None:
+    """Attempt loading a public constructor from design-research-agents."""
+    try:
+        module = importlib.import_module("design_research_agents")
+    except ImportError:
+        return None
+
+    if hasattr(module, agent_id):
+        constructor = getattr(module, agent_id)
+        if callable(constructor):
+            try:
+                return constructor()
+            except Exception:
+                return constructor
+    return None
+
+
+def _invoke_agent(
+    *,
+    executable: Any,
+    run_spec: RunSpec,
+    condition: Condition,
+    problem_packet: ProblemPacket,
+) -> Any:
+    """Invoke an agent object or callable with a best-effort argument mapping."""
+    if hasattr(executable, "run") and callable(executable.run):
+        return _invoke_callable(
+            callable_obj=executable.run,
+            run_spec=run_spec,
+            condition=condition,
+            problem_packet=problem_packet,
+        )
+
+    if callable(executable):
+        return _invoke_callable(
+            callable_obj=executable,
+            run_spec=run_spec,
+            condition=condition,
+            problem_packet=problem_packet,
+        )
+
+    raise ValidationError("Resolved agent object is not executable.")
+
+
+def _invoke_callable(
+    *,
+    callable_obj: Callable[..., Any],
+    run_spec: RunSpec,
+    condition: Condition,
+    problem_packet: ProblemPacket,
+) -> Any:
+    """Invoke a callable by matching supported keyword parameters."""
+    parameters = inspect.signature(callable_obj).parameters
+    kwargs: dict[str, Any] = {}
+
+    if "problem_packet" in parameters:
+        kwargs["problem_packet"] = problem_packet
+    if "problem" in parameters:
+        kwargs["problem"] = problem_packet
+    if "brief" in parameters:
+        kwargs["brief"] = problem_packet.brief
+    if "run_spec" in parameters:
+        kwargs["run_spec"] = run_spec
+    if "condition" in parameters:
+        kwargs["condition"] = condition
+    if "seed" in parameters:
+        kwargs["seed"] = run_spec.seed
+
+    if kwargs:
+        return callable_obj(**kwargs)
+
+    try:
+        return callable_obj(problem_packet, run_spec.seed)
+    except TypeError:
+        return callable_obj(problem_packet)
+
+
+def _normalize_agent_execution(
+    *,
+    raw: Any,
+    run_spec: RunSpec,
+    condition: Condition,
+) -> AgentExecution:
+    """Normalize raw execution output to canonical adapter shape."""
+    if isinstance(raw, Mapping):
+        output = dict(raw.get("output", raw.get("outputs", {})))
+        if not output and "text" in raw:
+            output = {"text": raw["text"]}
+
+        metrics = dict(raw.get("metrics", {}))
+        trace_refs = [str(value) for value in raw.get("trace_refs", [])]
+        metadata = dict(raw.get("metadata", {}))
+        events = _normalize_events(
+            raw_events=raw.get("events", []),
+            run_spec=run_spec,
+            condition=condition,
+            output=output,
+        )
+        return AgentExecution(
+            output=output,
+            metrics=metrics,
+            events=events,
+            trace_refs=trace_refs,
+            metadata=metadata,
+        )
+
+    output = {"text": str(raw)}
+    events = _normalize_events(
+        raw_events=[],
+        run_spec=run_spec,
+        condition=condition,
+        output=output,
+    )
+    return AgentExecution(output=output, metrics={}, events=events, trace_refs=[], metadata={})
+
+
+def _normalize_events(
+    *,
+    raw_events: Sequence[Any],
+    run_spec: RunSpec,
+    condition: Condition,
+    output: Mapping[str, Any],
+) -> list[Observation]:
+    """Normalize external event payloads into unified observation records."""
+    events: list[Observation] = []
+
+    for index, raw_event in enumerate(raw_events):
+        if not isinstance(raw_event, Mapping):
+            continue
+        events.append(
+            Observation(
+                timestamp=str(raw_event.get("timestamp", utc_now_iso())),
+                record_id=str(
+                    raw_event.get(
+                        "record_id",
+                        hash_identifier(
+                            "evt",
+                            {
+                                "run_id": run_spec.run_id,
+                                "index": index,
+                                "event_type": raw_event.get("event_type", "event"),
+                            },
+                        ),
+                    )
+                ),
+                text=str(raw_event.get("text", "")),
+                session_id=str(raw_event.get("session_id", run_spec.run_id)),
+                actor_id=str(raw_event.get("actor_id", "agent")),
+                event_type=str(raw_event.get("event_type", "event")),
+                meta_json=dict(raw_event.get("meta_json", {})),
+                level=ObservationLevel(str(raw_event.get("level", ObservationLevel.STEP.value))),
+                study_id=run_spec.study_id,
+                run_id=run_spec.run_id,
+                condition_id=condition.condition_id,
+                trial_id=raw_event.get("trial_id"),
+                step_id=raw_event.get("step_id"),
+                tool_name=raw_event.get("tool_name"),
+                evaluation_id=raw_event.get("evaluation_id"),
+            )
+        )
+
+    if events:
+        return events
+
+    return [
+        Observation(
+            timestamp=utc_now_iso(),
+            record_id=hash_identifier("evt", {"run_id": run_spec.run_id, "index": 0}),
+            text=str(output.get("text", "")),
+            session_id=run_spec.run_id,
+            actor_id="agent",
+            event_type="assistant_output",
+            meta_json={"auto_generated": True},
+            level=ObservationLevel.STEP,
+            study_id=run_spec.study_id,
+            run_id=run_spec.run_id,
+            condition_id=condition.condition_id,
+        )
+    ]