snakemake-contracts 0.1.0__tar.gz

snakemake_contracts-0.1.0/.gitignore

@@ -0,0 +1 @@
+**/__pycache__

snakemake_contracts-0.1.0/.gitlab-ci.yml

@@ -0,0 +1,26 @@
+stages:
+  - build
+  - publish
+
+build:
+  stage: build
+  image: python:3.12
+  script:
+    - pip install flit
+    - flit build
+  artifacts:
+    paths:
+      - dist/
+
+publish-pypi:
+  stage: publish
+  image: python:3.12
+  script:
+    - pip install flit
+    - flit publish
+  environment: pypi
+  rules:
+    - if: $CI_COMMIT_TAG =~ /^\d+\.\d+\.\d+$/
+  variables:
+    FLIT_USERNAME: __token__
+    FLIT_PASSWORD: $PYPI_TOKEN

snakemake_contracts-0.1.0/PKG-INFO

@@ -0,0 +1,121 @@
+Metadata-Version: 2.4
+Name: snakemake-contracts
+Version: 0.1.0
+Summary: Lightweight contract helpers for Snakemake modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: PyYAML>=6.0
+
+# snakemake-contracts
+
+A small helper library for Snakemake workflows that use
+`provides` / `requires`-style contracts between modules.
+
+## Motivation
+
+A lot of hidden complexities can stem from splitting a snakemake-based
+project into multiple independent submodules: how do you handle
+changing independent submodule outputs ?
+
+This package exists for workflows that are split into many Snakemake submodules
+and need a disciplined way to connect them.
+
+In practice, the goal is not just to reuse code. It is to make module boundaries
+explicit and stable:
+
+- each module should declare what it consumes and what it exposes;
+- each module should keep its own outputs defined in one place;
+- the root workflow should not become the only place that knows where every
+  generated file lives.
+
+That matters once a workflow grows beyond a single Snakefile. At that point the
+top-level repo tends to accumulate accidental knowledge about other modules:
+hard-coded file paths, config overrides for foreign outputs, and ad hoc
+assumptions about directory layouts. Those shortcuts make the graph harder to
+reason about and much easier to break when a module changes shape.
+
+The contract model used here is intentionally close to the `requires` /
+`provides` style found in systems like nREPL middleware chains: producers
+declare the interface they offer, consumers ask for a named capability, and the
+resolver stays small and mechanical. The point is to treat the dependency
+boundary as a contract, not as an implicit convention spread across the whole
+repository.
+
+Using `provides.yaml` as the source of truth also has a practical benefit: the
+module itself remains the authoritative owner of its exported outputs. The repo
+root can still orchestrate modules, but it does not need to mirror every output
+name or path in a central registry. That reduces duplication and keeps the
+dependency metadata next to the module that owns it.
+
+## Scope
+
+- resolve declared outputs by logical key
+- locate module output directories
+- validate `provides.yaml` files
+- optionally compare declared contracts against a repo layout
+
+## Library Usage
+
+The core API lives in `snakemake_contracts.contracts` and expects a repo layout
+like:
+
+```text
+repo/
+  modules/
+    my_module/
+      Snakefile
+      provides.yaml
+      out/
+```
+
+Typical usage:
+
+```python
+from snakemake_contracts.contracts import load_provides, provides, validate_repo
+
+mapping = load_provides("my_module", repo_root="/path/to/repo")
+report = provides("my_module", "report", repo_root="/path/to/repo")
+problems = validate_repo("/path/to/repo")
+```
+
+`provides.yaml` is the source of truth for logical output names, and `provides()`
+resolves them relative to the module's `out/` directory.
+
+## Wrapper Layer
+
+The repository now also includes a separate `wrapper/` package,
+`snakemake_contracts_wrapper`, for Snakemake-facing command construction.
+That layer is intentionally thin: it reads the named `input`, `output`, and
+`params` objects already present in a rule and turns them into ordinary script
+arguments.
+
+The script itself still stays unaware of Snakemake. It only sees CLI flags.
+The wrapper only removes the repetitive `--flag {input.foo}` wiring in the
+Snakefile.
+
+## Current Surface
+
+- `load_provides(module, repo_root=None)`
+- `provides(module, key, repo_root=None)`
+- `out_dir(module, repo_root=None)`
+- `list_modules(repo_root=None)`
+- `mangle(name)`
+- `validate_repo(repo_root=None)`
+
+Wrapper surface:
+
+- `build_argv(command, input=None, output=None, params=None, exclude_input=("script",))`
+- `render_shell(command, input=None, output=None, params=None, exclude_input=("script",))`
+
+## Tests
+
+- `tests/test_contracts.py` exercises the library directly against a temporary
+  repo layout.
+- `tests/test_wrapper.py` covers the wrapper command helpers.
+
+## Status
+
+This is a small utility package, not a general Snakemake extension. It is meant
+to support workflows that already use `provides.yaml`-style contracts and want a
+single place to validate and resolve them.
+

snakemake_contracts-0.1.0/README.md

@@ -0,0 +1,112 @@
+# snakemake-contracts
+
+A small helper library for Snakemake workflows that use
+`provides` / `requires`-style contracts between modules.
+
+## Motivation
+
+A lot of hidden complexities can stem from splitting a snakemake-based
+project into multiple independent submodules: how do you handle
+changing independent submodule outputs ?
+
+This package exists for workflows that are split into many Snakemake submodules
+and need a disciplined way to connect them.
+
+In practice, the goal is not just to reuse code. It is to make module boundaries
+explicit and stable:
+
+- each module should declare what it consumes and what it exposes;
+- each module should keep its own outputs defined in one place;
+- the root workflow should not become the only place that knows where every
+  generated file lives.
+
+That matters once a workflow grows beyond a single Snakefile. At that point the
+top-level repo tends to accumulate accidental knowledge about other modules:
+hard-coded file paths, config overrides for foreign outputs, and ad hoc
+assumptions about directory layouts. Those shortcuts make the graph harder to
+reason about and much easier to break when a module changes shape.
+
+The contract model used here is intentionally close to the `requires` /
+`provides` style found in systems like nREPL middleware chains: producers
+declare the interface they offer, consumers ask for a named capability, and the
+resolver stays small and mechanical. The point is to treat the dependency
+boundary as a contract, not as an implicit convention spread across the whole
+repository.
+
+Using `provides.yaml` as the source of truth also has a practical benefit: the
+module itself remains the authoritative owner of its exported outputs. The repo
+root can still orchestrate modules, but it does not need to mirror every output
+name or path in a central registry. That reduces duplication and keeps the
+dependency metadata next to the module that owns it.
+
+## Scope
+
+- resolve declared outputs by logical key
+- locate module output directories
+- validate `provides.yaml` files
+- optionally compare declared contracts against a repo layout
+
+## Library Usage
+
+The core API lives in `snakemake_contracts.contracts` and expects a repo layout
+like:
+
+```text
+repo/
+  modules/
+    my_module/
+      Snakefile
+      provides.yaml
+      out/
+```
+
+Typical usage:
+
+```python
+from snakemake_contracts.contracts import load_provides, provides, validate_repo
+
+mapping = load_provides("my_module", repo_root="/path/to/repo")
+report = provides("my_module", "report", repo_root="/path/to/repo")
+problems = validate_repo("/path/to/repo")
+```
+
+`provides.yaml` is the source of truth for logical output names, and `provides()`
+resolves them relative to the module's `out/` directory.
+
+## Wrapper Layer
+
+The repository now also includes a separate `wrapper/` package,
+`snakemake_contracts_wrapper`, for Snakemake-facing command construction.
+That layer is intentionally thin: it reads the named `input`, `output`, and
+`params` objects already present in a rule and turns them into ordinary script
+arguments.
+
+The script itself still stays unaware of Snakemake. It only sees CLI flags.
+The wrapper only removes the repetitive `--flag {input.foo}` wiring in the
+Snakefile.
+
+## Current Surface
+
+- `load_provides(module, repo_root=None)`
+- `provides(module, key, repo_root=None)`
+- `out_dir(module, repo_root=None)`
+- `list_modules(repo_root=None)`
+- `mangle(name)`
+- `validate_repo(repo_root=None)`
+
+Wrapper surface:
+
+- `build_argv(command, input=None, output=None, params=None, exclude_input=("script",))`
+- `render_shell(command, input=None, output=None, params=None, exclude_input=("script",))`
+
+## Tests
+
+- `tests/test_contracts.py` exercises the library directly against a temporary
+  repo layout.
+- `tests/test_wrapper.py` covers the wrapper command helpers.
+
+## Status
+
+This is a small utility package, not a general Snakemake extension. It is meant
+to support workflows that already use `provides.yaml`-style contracts and want a
+single place to validate and resolve them.

snakemake_contracts-0.1.0/pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "snakemake-contracts"
+version = "0.1.0"
+description = "Lightweight contract helpers for Snakemake modules"
+readme = "README.md"
+requires-python = ">=3.10"
+dependencies = ["PyYAML>=6.0"]
+
+[project.scripts]
+snakemake-contracts = "snakemake_contracts.cli:main"
+
+[tool.setuptools]
+package-dir = {"" = "src", "snakemake_contracts_wrapper" = "wrapper/snakemake_contracts_wrapper"}
+
+[tool.setuptools.packages.find]
+where = ["src", "wrapper"]

snakemake_contracts-0.1.0/src/snakemake_contracts/__init__.py

@@ -0,0 +1 @@
+from .contracts import ContractError, load_provides, out_dir, provides, list_modules, mangle, validate_repo

snakemake_contracts-0.1.0/src/snakemake_contracts/cli.py

@@ -0,0 +1,67 @@
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+import sys
+
+from .contracts import list_modules, load_provides, provides, validate_repo
+
+
+def _cmd_list_modules(args: argparse.Namespace) -> int:
+    for module in list_modules(args.repo_root):
+        print(module)
+    return 0
+
+
+def _cmd_list_keys(args: argparse.Namespace) -> int:
+    data = load_provides(args.module, args.repo_root)
+    for key in sorted(data):
+        print(f"{key}\t{data[key]}")
+    return 0
+
+
+def _cmd_resolve(args: argparse.Namespace) -> int:
+    print(provides(args.module, args.key, args.repo_root))
+    return 0
+
+
+def _cmd_validate(args: argparse.Namespace) -> int:
+    problems = validate_repo(args.repo_root)
+    if problems:
+        for problem in problems:
+            print(problem, file=sys.stderr)
+        return 1
+    return 0
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="snakemake-contracts")
+    parser.add_argument("--repo-root", type=Path, default=None)
+    sub = parser.add_subparsers(dest="cmd", required=True)
+
+    p = sub.add_parser("list-modules")
+    p.set_defaults(func=_cmd_list_modules)
+
+    p = sub.add_parser("list-keys")
+    p.add_argument("module")
+    p.set_defaults(func=_cmd_list_keys)
+
+    p = sub.add_parser("resolve")
+    p.add_argument("module")
+    p.add_argument("key")
+    p.set_defaults(func=_cmd_resolve)
+
+    p = sub.add_parser("validate")
+    p.set_defaults(func=_cmd_validate)
+
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = build_parser()
+    args = parser.parse_args(argv)
+    return args.func(args)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

snakemake_contracts-0.1.0/src/snakemake_contracts/contracts.py

@@ -0,0 +1,75 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+
+@dataclass(frozen=True)
+class ContractError(Exception):
+    message: str
+
+    def __str__(self) -> str:
+        return self.message
+
+
+def _repo_root(repo_root: str | Path | None = None) -> Path:
+    if repo_root is None:
+        return Path(__file__).resolve().parents[2]
+    return Path(repo_root)
+
+
+def modules_dir(repo_root: str | Path | None = None) -> Path:
+    return _repo_root(repo_root) / "modules"
+
+
+def load_provides(module: str, repo_root: str | Path | None = None) -> dict[str, str]:
+    path = modules_dir(repo_root) / module / "provides.yaml"
+    if not path.exists():
+        raise FileNotFoundError(f"{path} not found")
+    with path.open() as f:
+        data = yaml.safe_load(f) or {}
+    if not isinstance(data, dict):
+        raise ContractError(f"{path} must contain a mapping of logical keys to relative paths")
+    bad = [k for k, v in data.items() if not isinstance(k, str) or not isinstance(v, str)]
+    if bad:
+        raise ContractError(f"{path} contains non-string keys/values: {bad}")
+    return data
+
+
+def out_dir(module: str, repo_root: str | Path | None = None) -> Path:
+    return modules_dir(repo_root) / module / "out"
+
+
+def provides(module: str, key: str, repo_root: str | Path | None = None) -> str:
+    mapping = load_provides(module, repo_root=repo_root)
+    if key not in mapping:
+        raise KeyError(f"{module}:{key} not declared; available keys: {sorted(mapping)}")
+    return str(out_dir(module, repo_root=repo_root) / mapping[key])
+
+
+def list_modules(repo_root: str | Path | None = None) -> list[str]:
+    root = modules_dir(repo_root)
+    if not root.exists():
+        return []
+    return sorted(p.name for p in root.iterdir() if p.is_dir() and (p / "Snakefile").exists())
+
+
+def mangle(name: str) -> str:
+    return name.replace("-", "_").replace(".", "_")
+
+
+def validate_repo(repo_root: str | Path | None = None) -> list[str]:
+    root = _repo_root(repo_root)
+    problems: list[str] = []
+    for module in list_modules(root):
+        p = modules_dir(root) / module / "provides.yaml"
+        if not p.exists():
+            continue
+        try:
+            load_provides(module, root)
+        except Exception as exc:  # noqa: BLE001
+            problems.append(f"{module}: {exc}")
+    return problems

snakemake_contracts-0.1.0/tests/test_contracts.py

@@ -0,0 +1,50 @@
+from __future__ import annotations
+
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / "src"))
+
+from snakemake_contracts.contracts import list_modules, load_provides, provides, validate_repo
+
+
+class ContractLibraryTests(unittest.TestCase):
+    def test_load_provides_and_resolve_output_path(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            repo_root = Path(tmp)
+            module_root = repo_root / "modules" / "alpha"
+            module_root.mkdir(parents=True)
+            (module_root / "Snakefile").write_text("rule all:\n    pass\n")
+            (module_root / "provides.yaml").write_text("reads: reads.fastq\nreport: results/report.tsv\n")
+
+            self.assertEqual(
+                load_provides("alpha", repo_root),
+                {"reads": "reads.fastq", "report": "results/report.tsv"},
+            )
+            self.assertEqual(
+                provides("alpha", "report", repo_root),
+                str(module_root / "out" / "results/report.tsv"),
+            )
+
+    def test_list_modules_and_validate_repo_use_real_repo_layout(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            repo_root = Path(tmp)
+
+            good = repo_root / "modules" / "good"
+            good.mkdir(parents=True)
+            (good / "Snakefile").write_text("rule all:\n    pass\n")
+            (good / "provides.yaml").write_text("x: out/x.txt\n")
+
+            ignored = repo_root / "modules" / "ignored"
+            ignored.mkdir(parents=True)
+            (ignored / "provides.yaml").write_text("bad: 123\n")
+
+            self.assertEqual(list_modules(repo_root), ["good"])
+            self.assertEqual(validate_repo(repo_root), [])
+
+
+if __name__ == "__main__":
+    unittest.main()

snakemake_contracts-0.1.0/tests/test_wrapper.py

@@ -0,0 +1,91 @@
+from __future__ import annotations
+
+import unittest
+import sys
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / "wrapper"))
+sys.path.insert(0, str(ROOT / "src"))
+
+from snakemake_contracts_wrapper import build_argv, namespace_flag, render_shell, snake_to_kebab
+
+
+class WrapperCommandTests(unittest.TestCase):
+    def test_snake_to_kebab(self) -> None:
+        self.assertEqual(snake_to_kebab("rp2019_f09_87"), "rp2019-f09-87")
+
+    def test_namespace_flag(self) -> None:
+        self.assertEqual(namespace_flag("input", "rp_stock"), "--rp-stock")
+        self.assertEqual(namespace_flag("output", "m2_86_inv"), "--output-m2-86-inv")
+
+    def test_build_argv_excludes_script_and_expands_names(self) -> None:
+        argv = build_argv(
+            ["python3", "/tmp/build_joint.py"],
+            input={"script": "/tmp/build_joint.py", "rp_stock": "/data/rp_stock.csv"},
+            output={"f09": "/tmp/out.parquet"},
+            params={"mc_samples": 10},
+        )
+        self.assertEqual(
+            argv,
+            [
+                "python3",
+                "/tmp/build_joint.py",
+                "--rp-stock",
+                "/data/rp_stock.csv",
+                "--output-f09",
+                "/tmp/out.parquet",
+                "--mc-samples",
+                "10",
+            ],
+        )
+
+    def test_build_argv_handles_log_namespace(self) -> None:
+        argv = build_argv(
+            "python3",
+            input={"sample": "/tmp/sample.csv"},
+            output={"result": "/tmp/out.csv"},
+            log={"log": "/tmp/run.log"},
+            output_prefix="",
+        )
+        self.assertEqual(
+            argv,
+            [
+                "python3",
+                "--sample",
+                "/tmp/sample.csv",
+                "--result",
+                "/tmp/out.csv",
+                "--log",
+                "/tmp/run.log",
+            ],
+        )
+
+    def test_build_argv_expands_sequences(self) -> None:
+        argv = build_argv(
+            "python3",
+            input={"reads": [Path("/tmp/r1.fastq"), Path("/tmp/r2.fastq")]},
+        )
+        self.assertEqual(
+            argv,
+            [
+                "python3",
+                "--reads",
+                "/tmp/r1.fastq",
+                "--reads",
+                "/tmp/r2.fastq",
+            ],
+        )
+
+    def test_render_shell_quotes(self) -> None:
+        command = render_shell(
+            ["python3", "/tmp/build joint.py"],
+            input={"sample_name": "a b"},
+        )
+        self.assertIn("'/tmp/build joint.py'", command)
+        self.assertIn("--sample-name", command)
+        self.assertIn("'a b'", command)
+
+
+if __name__ == "__main__":
+    unittest.main()

snakemake_contracts-0.1.0/wrapper/snakemake_contracts_wrapper/__init__.py

@@ -0,0 +1,3 @@
+from .command import build_argv, namespace_flag, render_shell, snake_to_kebab
+
+__all__ = ["build_argv", "namespace_flag", "render_shell", "snake_to_kebab"]

snakemake_contracts-0.1.0/wrapper/snakemake_contracts_wrapper/command.py

@@ -0,0 +1,123 @@
+from __future__ import annotations
+
+from collections.abc import Iterable, Mapping, Sequence
+from pathlib import Path
+import shlex
+from typing import Any
+
+
+def snake_to_kebab(name: str) -> str:
+    return name.replace("_", "-")
+
+
+def namespace_flag(namespace: str, name: str, *, output_prefix: str = "output-") -> str:
+    flag = snake_to_kebab(name)
+    if namespace == "output":
+        flag = f"{output_prefix}{flag}"
+    return f"--{flag}"
+
+
+def _is_scalar(value: Any) -> bool:
+    return isinstance(value, (str, bytes, Path)) or not isinstance(value, Iterable)
+
+
+def _iter_items(namespace: Any) -> list[tuple[str, Any]]:
+    if namespace is None:
+        return []
+    if isinstance(namespace, Mapping):
+        return list(namespace.items())
+    items = getattr(namespace, "items", None)
+    if callable(items):
+        return list(items())
+    keys = getattr(namespace, "keys", None)
+    if callable(keys):
+        return [(key, namespace[key]) for key in keys()]
+    raise TypeError(f"Unsupported namespace object: {type(namespace)!r}")
+
+
+def _extend_namespace(
+    argv: list[str],
+    *,
+    namespace: str,
+    values: Any,
+    exclude: set[str] | None = None,
+    output_prefix: str = "output-",
+) -> None:
+    excluded = exclude or set()
+    for name, value in _iter_items(values):
+        if name in excluded:
+            continue
+        flag = namespace_flag(namespace, name, output_prefix=output_prefix)
+        if _is_scalar(value):
+            argv.extend([flag, str(value)])
+            continue
+        for item in value:
+            argv.extend([flag, str(item)])
+
+
+def build_argv(
+    command: str | Sequence[Any],
+    *,
+    input: Any = None,
+    output: Any = None,
+    log: Any = None,
+    params: Any = None,
+    exclude_input: Iterable[str] = ("script",),
+    output_prefix: str = "output-",
+) -> list[str]:
+    argv: list[str] = []
+    if isinstance(command, str):
+        argv.append(command)
+    else:
+        argv.extend(str(item) for item in command)
+
+    _extend_namespace(
+        argv,
+        namespace="input",
+        values=input,
+        exclude=set(exclude_input),
+        output_prefix=output_prefix,
+    )
+    _extend_namespace(
+        argv,
+        namespace="output",
+        values=output,
+        output_prefix=output_prefix,
+    )
+    _extend_namespace(
+        argv,
+        namespace="log",
+        values=log,
+        output_prefix=output_prefix,
+    )
+    _extend_namespace(
+        argv,
+        namespace="params",
+        values=params,
+        output_prefix=output_prefix,
+    )
+    return argv
+
+
+def render_shell(
+    command: str | Sequence[Any],
+    *,
+    input: Any = None,
+    output: Any = None,
+    log: Any = None,
+    params: Any = None,
+    exclude_input: Iterable[str] = ("script",),
+    output_prefix: str = "output-",
+) -> str:
+    return " ".join(
+        shlex.quote(arg)
+        for arg in build_argv(
+            command,
+            input=input,
+            output=output,
+            log=log,
+            params=params,
+            exclude_input=exclude_input,
+            output_prefix=output_prefix,
+        )
+    )