Orcca 0.1.0__tar.gz

orcca-0.1.0/Orcca.egg-info/PKG-INFO

@@ -0,0 +1,269 @@
+Metadata-Version: 2.4
+Name: Orcca
+Version: 0.1.0
+Summary: Manifest-driven clinical workflow orchestration framework
+Author: Athenkosi Nkonyeni
+License-Expression: MIT
+Keywords: clinical-trials,workflow,orchestration,manifest,data-pipeline
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: jsonschema>=4.0
+Requires-Dist: linkml>=1.8
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+
+# Orcca
+
+Orcca is a manifest-driven orchestration framework for clinical programming workflows.
+It is designed for teams that need reproducible execution, explicit dependency modeling,
+and clear run reporting.
+
+## Current State
+
+- Package version: `0.1.0` (alpha)
+- Python: `>=3.11`
+- Core dependencies: `PyYAML`, `jsonschema`, `linkml`
+- Entry point: `orcca` CLI (`orcca.cli:main`)
+
+This project is currently optimized for practical workflow execution and validation with a
+strong traceability model, not for broad general-purpose scheduling.
+
+## What Orcca Does
+
+Orcca provides:
+
+- declarative workflow modeling in YAML
+- manifest include/config resolution
+- LinkML + business-rule validation
+- executable task orchestration with dependency-aware scheduling
+- runtime preflight for package-manager and container environments
+- run reporting (`run_report.html`) and task logs
+- validation task execution for independent double programming (IDP)
+- `blinding_mode`-based blinded/unblinded workflow control
+
+## Core Architecture
+
+Main modules:
+
+- `orcca/resolver.py`: include/config merge and placeholder interpolation
+- `orcca/validator.py`: LinkML validation + business rules
+- `orcca/executor.py`: task execution engine + runtime preflight
+- `orcca/workflow_runner.py`: end-to-end orchestration pipeline and run reports
+- `orcca/runtime_launchers.py`: built-in + custom launcher resolution
+- `orcca/dagger.py`: generated-artifact DAG utilities
+- `orcca/cli.py`: user-facing commands
+
+Packaged schema/rules:
+
+- `orcca/resources/schema/flow-linkml.yaml`
+- `orcca/resources/rules/business-rules.yaml`
+
+## Manifest Resolution Semantics
+
+Orcca resolves manifests in this order:
+
+1. Resolve `includes` recursively in depth-first, declaration order.
+2. Merge include payloads into a single root payload.
+3. Merge the root manifest payload on top.
+4. Resolve `configs` in declaration order.
+5. Interpolate `${config['output_dir']}`-style placeholders after merging is complete.
+6. Placeholder substitutions must resolve to scalar values, not lists or dictionaries.
+
+Merge behavior:
+
+- dict values replace completely
+- list-valued keys inside dicts: merged with the same list rules
+- lists of scalars: append + dedupe while preserving first-seen order
+- lists of `{id: ...}` entities: overwrite by `id`
+- scalars/type mismatch: latest wins
+
+Config files use the same merge rules as manifest root slots, so later files can override
+earlier ones without changing the merge model.
+
+Declarative slots such as `id`, `name`, `description`, `notes`, `envvar_name`, and
+`param_name` must be written as explicit literals and may not use
+`${config['output_dir']}`-style interpolation.
+Placeholder substitutions must resolve to scalar values, not lists or dictionaries.
+
+This matters for override design: unblinded manifests should include base first, then
+override files that only change allowed slots.
+
+## Workflow Model
+
+Primary slots:
+
+- `study`
+- `standards`
+- `artifacts`
+- `artifact_groups`
+- `runtimes`
+- `tasks`
+- `validations`
+- `deliveries`
+- `execution`
+- `blinding_mode`
+
+Important modeling rules:
+
+- `tasks.outputs` must reference declared artifacts.
+- `tasks.inputs` can reference declared artifacts or transient IDs used internally.
+- `required_tasks` provides explicit execution sequencing independent of artifact lineage.
+
+## Blinded/Unblinded Mode
+
+Run mode:
+
+```yaml
+blinding_mode: blinded # or unblinded
+```
+
+Artifact variants:
+
+- `variant` is optional and only needed for paired blinded/unblinded artifacts.
+
+```yaml
+artifacts:
+  - id: rand_map
+    variant: blinded
+    name: Randomization Map
+    type: lookup_dataset
+    origin: external
+    path: outputs/rand_map_blinded.csv
+```
+
+Unblinded pattern:
+
+```yaml
+includes:
+  - workflow_blinded.yaml
+  - manifests/artifacts_unblinded.yaml
+
+blinding_mode: unblinded
+```
+
+Declarative slots must remain literal and may not use quoted config interpolation placeholders.
+
+## CLI
+
+Top-level commands:
+
+- `orcca validate <manifest.yaml>`
+- `orcca plan <manifest.yaml>`
+- `orcca run <manifest.yaml> [--project-root <path>]`
+- `orcca env-cache list|prune [--all]`
+- `orcca init [path]`
+
+Useful run flags:
+
+- `--task <id>` (repeatable)
+- `--artifact <id>` (repeatable)
+- `--run-validations`
+- `--validation-id <id>` (repeatable; requires `--run-validations`)
+- `--no-execute`
+- `--no-env-cache`
+- `--refresh-env`
+
+Run root behavior:
+
+- `--project-root` controls execution root and run artifact root
+- runs are stored under `<project-root>/.orcca/runs`
+
+## Execution and Reports
+
+`run_workflow(...)` pipeline:
+
+1. resolve manifest
+2. validate manifest
+3. execute tasks (optional)
+4. package deliveries (when configured)
+5. optionally execute selected validation tasks
+6. write run artifacts
+
+Per-run artifacts:
+
+- `resolved_manifest.yaml`
+- `run_report.html`
+- task logs under `logs/`
+- task context payloads under `task-context/`
+
+## Runtime Model
+
+Built-in launchers:
+
+- `python`
+- `rscript`
+- `julia`
+- `container`
+
+Custom launchers:
+
+- auto-loaded from `launchers/*.py`
+- optional `launchers/register.py`
+- launcher file rule: filename stem must match exported function name
+
+Package-manager environments:
+
+- cached under `.orcca/cache/environments/`
+- managers: `pip`, `venv`, `renv`, `julia_pkg` (+ explicit restore command for others)
+
+Container environments:
+
+- source types: `registry`, `archive`, `build`
+- archive load and local build workflows supported
+
+## Validation and IDP
+
+Validation uses:
+
+- LinkML schema validation
+- business-rule validation
+- cross-reference checks and dependency checks
+
+IDP (`check_type: independent_double_programming`) supports:
+
+- artifact/group targets
+- explicit `validation_task`
+- validation targets and validation task references
+- tool metadata via `tool`
+
+Recommended IDP pattern:
+
+- keep `targets` for business ownership
+- use `validation_task` for executable provenance
+- keep validation outputs easy to inspect in reports and logs
+
+## Examples in Repo
+
+Reference workflows:
+
+- `testing/e2e_python_native/workflow.yaml`
+- `testing/e2e_r_native/workflow.yaml`
+- `testing/e2e_julia_native/workflow.yaml`
+- `testing/e2e_sdtm/workflow.yaml`
+
+Blinding handoff example:
+
+- `testing/blinding_handoff/README.md`
+
+## Development
+
+Install:
+
+```bash
+pip install -e .[dev]
+```
+
+Run tests:
+
+```bash
+pytest -q
+```

orcca-0.1.0/Orcca.egg-info/SOURCES.txt

@@ -0,0 +1,38 @@
+README.md
+pyproject.toml
+Orcca.egg-info/PKG-INFO
+Orcca.egg-info/SOURCES.txt
+Orcca.egg-info/dependency_links.txt
+Orcca.egg-info/entry_points.txt
+Orcca.egg-info/requires.txt
+Orcca.egg-info/top_level.txt
+orcca/__init__.py
+orcca/cli.py
+orcca/click_cli.py
+orcca/dagger.py
+orcca/executor.py
+orcca/io_utils.py
+orcca/launchers.py
+orcca/resolver.py
+orcca/runtime_launchers.py
+orcca/validator.py
+orcca/workflow.py
+orcca/workflow_runner.py
+orcca/resources/__init__.py
+orcca/resources/rules/business-rules.yaml
+orcca/resources/schema/business-rules-linkml.yaml
+orcca/resources/schema/flow-linkml.yaml
+tests/test_cli_cache_init.py
+tests/test_cli_launcher_autoload.py
+tests/test_container_image_source_types_integration.py
+tests/test_container_image_source_types_podman_integration.py
+tests/test_dagger.py
+tests/test_executor.py
+tests/test_io_utils.py
+tests/test_native_language_templates.py
+tests/test_package_manager_integration.py
+tests/test_resolver.py
+tests/test_runtime_adapters.py
+tests/test_sdtm_fixture.py
+tests/test_validator.py
+tests/test_workflow_runner.py

orcca-0.1.0/Orcca.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

orcca-0.1.0/Orcca.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+orcca = orcca.cli:main

orcca-0.1.0/Orcca.egg-info/requires.txt

@@ -0,0 +1,6 @@
+PyYAML>=6.0
+jsonschema>=4.0
+linkml>=1.8
+
+[dev]
+pytest>=8.0

orcca-0.1.0/Orcca.egg-info/top_level.txt

@@ -0,0 +1 @@
+orcca

orcca-0.1.0/PKG-INFO

@@ -0,0 +1,269 @@
+Metadata-Version: 2.4
+Name: Orcca
+Version: 0.1.0
+Summary: Manifest-driven clinical workflow orchestration framework
+Author: Athenkosi Nkonyeni
+License-Expression: MIT
+Keywords: clinical-trials,workflow,orchestration,manifest,data-pipeline
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: PyYAML>=6.0
+Requires-Dist: jsonschema>=4.0
+Requires-Dist: linkml>=1.8
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+
+# Orcca
+
+Orcca is a manifest-driven orchestration framework for clinical programming workflows.
+It is designed for teams that need reproducible execution, explicit dependency modeling,
+and clear run reporting.
+
+## Current State
+
+- Package version: `0.1.0` (alpha)
+- Python: `>=3.11`
+- Core dependencies: `PyYAML`, `jsonschema`, `linkml`
+- Entry point: `orcca` CLI (`orcca.cli:main`)
+
+This project is currently optimized for practical workflow execution and validation with a
+strong traceability model, not for broad general-purpose scheduling.
+
+## What Orcca Does
+
+Orcca provides:
+
+- declarative workflow modeling in YAML
+- manifest include/config resolution
+- LinkML + business-rule validation
+- executable task orchestration with dependency-aware scheduling
+- runtime preflight for package-manager and container environments
+- run reporting (`run_report.html`) and task logs
+- validation task execution for independent double programming (IDP)
+- `blinding_mode`-based blinded/unblinded workflow control
+
+## Core Architecture
+
+Main modules:
+
+- `orcca/resolver.py`: include/config merge and placeholder interpolation
+- `orcca/validator.py`: LinkML validation + business rules
+- `orcca/executor.py`: task execution engine + runtime preflight
+- `orcca/workflow_runner.py`: end-to-end orchestration pipeline and run reports
+- `orcca/runtime_launchers.py`: built-in + custom launcher resolution
+- `orcca/dagger.py`: generated-artifact DAG utilities
+- `orcca/cli.py`: user-facing commands
+
+Packaged schema/rules:
+
+- `orcca/resources/schema/flow-linkml.yaml`
+- `orcca/resources/rules/business-rules.yaml`
+
+## Manifest Resolution Semantics
+
+Orcca resolves manifests in this order:
+
+1. Resolve `includes` recursively in depth-first, declaration order.
+2. Merge include payloads into a single root payload.
+3. Merge the root manifest payload on top.
+4. Resolve `configs` in declaration order.
+5. Interpolate `${config['output_dir']}`-style placeholders after merging is complete.
+6. Placeholder substitutions must resolve to scalar values, not lists or dictionaries.
+
+Merge behavior:
+
+- dict values replace completely
+- list-valued keys inside dicts: merged with the same list rules
+- lists of scalars: append + dedupe while preserving first-seen order
+- lists of `{id: ...}` entities: overwrite by `id`
+- scalars/type mismatch: latest wins
+
+Config files use the same merge rules as manifest root slots, so later files can override
+earlier ones without changing the merge model.
+
+Declarative slots such as `id`, `name`, `description`, `notes`, `envvar_name`, and
+`param_name` must be written as explicit literals and may not use
+`${config['output_dir']}`-style interpolation.
+Placeholder substitutions must resolve to scalar values, not lists or dictionaries.
+
+This matters for override design: unblinded manifests should include base first, then
+override files that only change allowed slots.
+
+## Workflow Model
+
+Primary slots:
+
+- `study`
+- `standards`
+- `artifacts`
+- `artifact_groups`
+- `runtimes`
+- `tasks`
+- `validations`
+- `deliveries`
+- `execution`
+- `blinding_mode`
+
+Important modeling rules:
+
+- `tasks.outputs` must reference declared artifacts.
+- `tasks.inputs` can reference declared artifacts or transient IDs used internally.
+- `required_tasks` provides explicit execution sequencing independent of artifact lineage.
+
+## Blinded/Unblinded Mode
+
+Run mode:
+
+```yaml
+blinding_mode: blinded # or unblinded
+```
+
+Artifact variants:
+
+- `variant` is optional and only needed for paired blinded/unblinded artifacts.
+
+```yaml
+artifacts:
+  - id: rand_map
+    variant: blinded
+    name: Randomization Map
+    type: lookup_dataset
+    origin: external
+    path: outputs/rand_map_blinded.csv
+```
+
+Unblinded pattern:
+
+```yaml
+includes:
+  - workflow_blinded.yaml
+  - manifests/artifacts_unblinded.yaml
+
+blinding_mode: unblinded
+```
+
+Declarative slots must remain literal and may not use quoted config interpolation placeholders.
+
+## CLI
+
+Top-level commands:
+
+- `orcca validate <manifest.yaml>`
+- `orcca plan <manifest.yaml>`
+- `orcca run <manifest.yaml> [--project-root <path>]`
+- `orcca env-cache list|prune [--all]`
+- `orcca init [path]`
+
+Useful run flags:
+
+- `--task <id>` (repeatable)
+- `--artifact <id>` (repeatable)
+- `--run-validations`
+- `--validation-id <id>` (repeatable; requires `--run-validations`)
+- `--no-execute`
+- `--no-env-cache`
+- `--refresh-env`
+
+Run root behavior:
+
+- `--project-root` controls execution root and run artifact root
+- runs are stored under `<project-root>/.orcca/runs`
+
+## Execution and Reports
+
+`run_workflow(...)` pipeline:
+
+1. resolve manifest
+2. validate manifest
+3. execute tasks (optional)
+4. package deliveries (when configured)
+5. optionally execute selected validation tasks
+6. write run artifacts
+
+Per-run artifacts:
+
+- `resolved_manifest.yaml`
+- `run_report.html`
+- task logs under `logs/`
+- task context payloads under `task-context/`
+
+## Runtime Model
+
+Built-in launchers:
+
+- `python`
+- `rscript`
+- `julia`
+- `container`
+
+Custom launchers:
+
+- auto-loaded from `launchers/*.py`
+- optional `launchers/register.py`
+- launcher file rule: filename stem must match exported function name
+
+Package-manager environments:
+
+- cached under `.orcca/cache/environments/`
+- managers: `pip`, `venv`, `renv`, `julia_pkg` (+ explicit restore command for others)
+
+Container environments:
+
+- source types: `registry`, `archive`, `build`
+- archive load and local build workflows supported
+
+## Validation and IDP
+
+Validation uses:
+
+- LinkML schema validation
+- business-rule validation
+- cross-reference checks and dependency checks
+
+IDP (`check_type: independent_double_programming`) supports:
+
+- artifact/group targets
+- explicit `validation_task`
+- validation targets and validation task references
+- tool metadata via `tool`
+
+Recommended IDP pattern:
+
+- keep `targets` for business ownership
+- use `validation_task` for executable provenance
+- keep validation outputs easy to inspect in reports and logs
+
+## Examples in Repo
+
+Reference workflows:
+
+- `testing/e2e_python_native/workflow.yaml`
+- `testing/e2e_r_native/workflow.yaml`
+- `testing/e2e_julia_native/workflow.yaml`
+- `testing/e2e_sdtm/workflow.yaml`
+
+Blinding handoff example:
+
+- `testing/blinding_handoff/README.md`
+
+## Development
+
+Install:
+
+```bash
+pip install -e .[dev]
+```
+
+Run tests:
+
+```bash
+pytest -q
+```