toolrack-cli 0.1.0__tar.gz → 0.2.0__tar.gz

{toolrack_cli-0.1.0/src/toolrack_cli.egg-info → toolrack_cli-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: toolrack-cli
-Version: 0.1.0
+Version: 0.2.0
 Summary: Turn a folder of small scripts plus YAML sidecars into one typed, completable CLI
 Author: Jan Chimiak
 License-Expression: MIT
@@ -33,6 +33,9 @@ and builds a typed Click CLI at runtime.
 If you want the starter repository that uses this engine, see
 [`toolrack-template`](https://github.com/anfit/toolrack-template).
 
+[![PyPI](https://img.shields.io/pypi/v/toolrack-cli)](https://pypi.org/project/toolrack-cli/)
+[![Unit tests](https://github.com/anfit/toolrack/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/anfit/toolrack/actions/workflows/unit-tests.yml)
+
 ![Animated toolrack walkthrough: register, discover, and run scripts](docs/assets/toolrack-demo.gif)
 
 ## Quickstart
@@ -127,10 +130,72 @@ Which also makes the AI prompt much nicer:
 
 > Hey AI, make me one like this, but mauve.
 
+## Testing Your Tools
+
+Installing `toolrack-cli` also gives your test suite a pytest plugin. It runs the
+real CLI in a subprocess against a repository, so tests cover what users run:
+sidecar parsing, argument mapping, exit codes and the script itself. Python,
+Bash and SQL scripts are all supported.
+
+**Check the whole repo in CI.** `core validate` is read-only and exits non-zero
+on a missing script or sidecar, an invalid sidecar, a bad `aliases.cfg`, or two
+scripts that map to the same command. Unregistered sidecar pairs and a stale
+cache are warnings; `--strict` makes them failures.
+
+```sh
+my core validate --strict
+```
+
+**Or from pytest**, against your real repo (`toolrack_project`) or a throwaway
+one (`toolrack_repo`):
+
+```python
+def test_repo_is_valid(toolrack_project):
+    toolrack_project.validate(strict=True)   # AssertionError lists every problem
+
+def test_python_tool(toolrack_repo):
+    toolrack_repo.add_script(
+        "greet/hello.py",
+        "import sys\nprint('hi', sys.argv[2])\n",
+        {"description": "Say hi.", "args": [{"name": "name", "required": True}]},
+    )
+    assert toolrack_repo.run("greet", "hello", "--name", "Ada").check().stdout == "hi Ada\n"
+
+def test_bash_tool(toolrack_repo, toolrack_require_bash):      # skips without bash
+    toolrack_repo.add_script("ops/fail.sh", "exit 3\n", {"description": "Fails."})
+    assert toolrack_repo.run("ops", "fail").returncode == 3
+
+def test_sql_tool(toolrack_repo, toolrack_require_posix):      # no database needed
+    psql = toolrack_repo.stub_executable("psql", record_env=("PGDATABASE",))
+    script = toolrack_repo.add_script("db/ping.sql", "SELECT 1;", {"description": "Ping."})
+    toolrack_repo.run("db", "ping", env={"PGDATABASE": "dev"}).check()
+    assert psql.calls()[0]["argv"] == ["-f", str(script)]
+```
+
+SQL scripts run as `psql -f <file>`, so `stub_executable` puts a recording fake
+`psql` first on `PATH` instead of needing a database. It works on POSIX hosts
+only. `ToolrackRepo` is in `toolrack.testing` if you prefer to use it without
+the fixtures.
+
+## Migrating From a Checked-Out toolrack
+
+If you installed the engine editable from a clone (`pip install -e path/to/toolrack`,
+distribution name `toolrack`), switch your scripts repo to the PyPI release:
+
+```sh
+python3 ./sync_toolrack.py                              # refresh setup_toolrack.py
+python3 ./setup_toolrack.py --yes --force --cli-name <your-command>
+```
+
+Setup removes the old editable install, installs `toolrack-cli`, rewrites the
+wrappers, and regenerates completion. Full steps, plus the manual route, are in
+the [toolrack-template README](https://github.com/anfit/toolrack-template#migrating-from-a-checked-out-toolrack).
+
 ## Deployment Model
 
-`toolrack` is meant to be installed into a scripts repository and launched
-through a repo-local wrapper in `bin/`.
+`toolrack` is meant to be installed into a scripts repository's virtualenv
+(`pip install toolrack-cli`) and launched through a repo-local wrapper in `bin/`.
+The PyPI distribution is `toolrack-cli`; the package and command are `toolrack`.
 
 The wrapper is responsible for:
 

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/README.md

@@ -7,6 +7,9 @@ and builds a typed Click CLI at runtime.
 If you want the starter repository that uses this engine, see
 [`toolrack-template`](https://github.com/anfit/toolrack-template).
 
+[![PyPI](https://img.shields.io/pypi/v/toolrack-cli)](https://pypi.org/project/toolrack-cli/)
+[![Unit tests](https://github.com/anfit/toolrack/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/anfit/toolrack/actions/workflows/unit-tests.yml)
+
 ![Animated toolrack walkthrough: register, discover, and run scripts](docs/assets/toolrack-demo.gif)
 
 ## Quickstart
@@ -101,10 +104,72 @@ Which also makes the AI prompt much nicer:
 
 > Hey AI, make me one like this, but mauve.
 
+## Testing Your Tools
+
+Installing `toolrack-cli` also gives your test suite a pytest plugin. It runs the
+real CLI in a subprocess against a repository, so tests cover what users run:
+sidecar parsing, argument mapping, exit codes and the script itself. Python,
+Bash and SQL scripts are all supported.
+
+**Check the whole repo in CI.** `core validate` is read-only and exits non-zero
+on a missing script or sidecar, an invalid sidecar, a bad `aliases.cfg`, or two
+scripts that map to the same command. Unregistered sidecar pairs and a stale
+cache are warnings; `--strict` makes them failures.
+
+```sh
+my core validate --strict
+```
+
+**Or from pytest**, against your real repo (`toolrack_project`) or a throwaway
+one (`toolrack_repo`):
+
+```python
+def test_repo_is_valid(toolrack_project):
+    toolrack_project.validate(strict=True)   # AssertionError lists every problem
+
+def test_python_tool(toolrack_repo):
+    toolrack_repo.add_script(
+        "greet/hello.py",
+        "import sys\nprint('hi', sys.argv[2])\n",
+        {"description": "Say hi.", "args": [{"name": "name", "required": True}]},
+    )
+    assert toolrack_repo.run("greet", "hello", "--name", "Ada").check().stdout == "hi Ada\n"
+
+def test_bash_tool(toolrack_repo, toolrack_require_bash):      # skips without bash
+    toolrack_repo.add_script("ops/fail.sh", "exit 3\n", {"description": "Fails."})
+    assert toolrack_repo.run("ops", "fail").returncode == 3
+
+def test_sql_tool(toolrack_repo, toolrack_require_posix):      # no database needed
+    psql = toolrack_repo.stub_executable("psql", record_env=("PGDATABASE",))
+    script = toolrack_repo.add_script("db/ping.sql", "SELECT 1;", {"description": "Ping."})
+    toolrack_repo.run("db", "ping", env={"PGDATABASE": "dev"}).check()
+    assert psql.calls()[0]["argv"] == ["-f", str(script)]
+```
+
+SQL scripts run as `psql -f <file>`, so `stub_executable` puts a recording fake
+`psql` first on `PATH` instead of needing a database. It works on POSIX hosts
+only. `ToolrackRepo` is in `toolrack.testing` if you prefer to use it without
+the fixtures.
+
+## Migrating From a Checked-Out toolrack
+
+If you installed the engine editable from a clone (`pip install -e path/to/toolrack`,
+distribution name `toolrack`), switch your scripts repo to the PyPI release:
+
+```sh
+python3 ./sync_toolrack.py                              # refresh setup_toolrack.py
+python3 ./setup_toolrack.py --yes --force --cli-name <your-command>
+```
+
+Setup removes the old editable install, installs `toolrack-cli`, rewrites the
+wrappers, and regenerates completion. Full steps, plus the manual route, are in
+the [toolrack-template README](https://github.com/anfit/toolrack-template#migrating-from-a-checked-out-toolrack).
+
 ## Deployment Model
 
-`toolrack` is meant to be installed into a scripts repository and launched
-through a repo-local wrapper in `bin/`.
+`toolrack` is meant to be installed into a scripts repository's virtualenv
+(`pip install toolrack-cli`) and launched through a repo-local wrapper in `bin/`.
+The PyPI distribution is `toolrack-cli`; the package and command are `toolrack`.
 
 The wrapper is responsible for:
 

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/pyproject.toml

@@ -39,6 +39,10 @@ dev = [
 [project.scripts]
 toolrack = "toolrack.entrypoint:main"
 
+# Fixtures for testing your own scripts repo; inert unless a test requests them.
+[project.entry-points.pytest11]
+toolrack = "toolrack.pytest_plugin"
+
 [tool.setuptools.dynamic]
 version = { attr = "toolrack.__version__" }
 

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/src/toolrack/__init__.py

@@ -1,3 +1,3 @@
 """toolrack - typed dispatcher for a user-owned scripts repository."""
 
-__version__ = "0.1.0"
+__version__ = "0.2.0"

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/src/toolrack/cli.py

@@ -853,6 +853,122 @@ def cmd_reregister():
     click.echo(f"\n{ok} ok, {errors} errors.")
 
 
+@dataclass(frozen=True)
+class Diagnostic:
+    level: str  # "error" or "warning"
+    subject: str
+    message: str
+
+
+def _check_repository(repo: RepositoryContext) -> tuple[list[str], list[Diagnostic]]:
+    """Inspect the whole repository without writing anything.
+
+    Returns the registry entries that are fully valid and the diagnostics for
+    everything else. Unlike discovery at runtime (which skips broken entries
+    with a warning), this reports every problem so CI can fail on it.
+    """
+    import yaml
+
+    diagnostics: list[Diagnostic] = []
+    ok: list[str] = []
+    registry = _read_registry(repo)
+
+    try:
+        aliases = _load_aliases(repo)
+    except RuntimeError as exc:
+        diagnostics.append(Diagnostic("error", os.path.basename(repo.aliases_file), str(exc)))
+        aliases = {}
+
+    claimed: dict[tuple[str, ...], str] = {}
+    for entry in registry:
+        abs_path = _abs_script_path(entry, repo)
+        if not os.path.isfile(abs_path):
+            diagnostics.append(Diagnostic("error", entry, "registered script not found"))
+            continue
+        ext = os.path.splitext(abs_path)[1].lower()
+        if ext not in _RUNNABLE_EXTS:
+            diagnostics.append(Diagnostic("error", entry, f"unsupported extension '{ext}'"))
+            continue
+        try:
+            sidecar = _load_sidecar(abs_path)
+            if sidecar is None:
+                diagnostics.append(
+                    Diagnostic("error", entry, f"sidecar missing (run: {CLI_NAME} core register {entry})")
+                )
+                continue
+            _validate_sidecar(sidecar, abs_path)
+            group_parts, cmd_name, _ = _group_path_and_cmd(entry, aliases, repo)
+            _command_spec(abs_path, cmd_name, sidecar)
+        except (RuntimeError, ValueError, yaml.YAMLError) as exc:
+            diagnostics.append(Diagnostic("error", entry, str(exc)))
+            continue
+
+        key = (*group_parts, cmd_name)
+        if key in claimed:
+            diagnostics.append(
+                Diagnostic(
+                    "error",
+                    entry,
+                    f"command '{' '.join(key)}' is already provided by {claimed[key]}",
+                )
+            )
+            continue
+        claimed[key] = entry
+        ok.append(entry)
+
+    registered = set(registry)
+    for dirpath, dirnames, filenames in os.walk(repo.scripts_root):
+        dirnames[:] = sorted(
+            name for name in dirnames if name not in _SKIP_DIRS and not name.startswith(".")
+        )
+        for filename in sorted(filenames):
+            if not filename.endswith(".yml"):
+                continue
+            stem = os.path.join(dirpath, os.path.splitext(filename)[0])
+            for script_ext in sorted(_RUNNABLE_EXTS):
+                if os.path.isfile(stem + script_ext):
+                    canonical = _resolve_script_path(stem + script_ext, repo)
+                    if canonical not in registered:
+                        diagnostics.append(
+                            Diagnostic(
+                                "warning",
+                                canonical,
+                                f"has a sidecar but is not registered (run: {CLI_NAME} core auto-register)",
+                            )
+                        )
+                    break
+
+    if registry and _read_cache(registry, repo) is None:
+        diagnostics.append(
+            Diagnostic(
+                "warning",
+                os.path.basename(repo.cache_file),
+                f"cache is missing or stale (run: {CLI_NAME} core refresh-cache)",
+            )
+        )
+    return ok, diagnostics
+
+
+@core.command("validate")
+@click.option("--strict", is_flag=True, help="Treat warnings as errors.")
+def cmd_validate(strict: bool):
+    """Check registry, sidecars and aliases without changing anything (for CI)."""
+    ok, diagnostics = _check_repository(_current_repo())
+    for entry in ok:
+        click.echo(f"  OK       {entry}")
+    errors = warnings = 0
+    for item in diagnostics:
+        if item.level == "error":
+            errors += 1
+            click.echo(f"  ERROR    {item.subject}: {item.message}", err=True)
+        else:
+            warnings += 1
+            click.echo(f"  WARNING  {item.subject}: {item.message}", err=True)
+    click.echo(f"\n{len(ok)} ok, {errors} errors, {warnings} warnings.")
+    if errors or (strict and warnings):
+        raise click.exceptions.Exit(1)
+
+
 @core.command("refresh-cache")
 def cmd_refresh_cache():
     """Rebuild the command metadata cache next to the registry."""

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/src/toolrack/completion.py

@@ -54,6 +54,17 @@ CORE_COMMANDS: dict[str, dict] = {
         "args": [{"name": "script_path", "positional": True, "required": True}],
     },
     "reregister": {"description": "Re-validate all registered commands."},
+    "validate": {
+        "description": "Check registry, sidecars and aliases without changing anything (for CI).",
+        "args": [
+            {
+                "name": "strict",
+                "flag": True,
+                "option": "--strict",
+                "help": "Treat warnings as errors.",
+            }
+        ],
+    },
     "refresh-cache": {"description": "Rebuild the command metadata cache next to the registry."},
     "list": {"description": "List all registered scripts."},
     "install-completion": {

toolrack_cli-0.2.0/src/toolrack/pytest_plugin.py

@@ -0,0 +1,55 @@
+"""pytest fixtures for testing a toolrack scripts repository.
+
+Registered through the ``pytest11`` entry point, so installing ``toolrack-cli``
+makes these fixtures available in any project's tests:
+
+``toolrack_project``
+    The real repository around the test run (found by walking up from the
+    current directory, like the CLI does). Use it to validate and smoke-test
+    your actual scripts, e.g. ``toolrack_project.validate(strict=True)``.
+
+``toolrack_repo``
+    A fresh, empty temporary repository for tests that need controlled
+    scripts and sidecars.
+
+Both return :class:`toolrack.testing.ToolrackRepo`, which runs the real CLI in
+a subprocess.
+
+``toolrack_require_bash``
+    Request it (or use ``@pytest.mark.usefixtures``) in tests of ``.sh`` /
+    ``.bash`` scripts: skips the test when no working Bash is available.
+
+``toolrack_require_posix``
+    Skips on Windows; for tests that use ``stub_executable`` (for example a
+    fake ``psql`` to test ``.sql`` scripts).
+"""
+
+from __future__ import annotations
+
+import os
+
+import pytest
+
+from toolrack.testing import ToolrackRepo, bash_available
+
+
+@pytest.fixture()
+def toolrack_project(request) -> ToolrackRepo:
+    return ToolrackRepo.discover(request.config.rootpath)
+
+
+@pytest.fixture()
+def toolrack_repo(tmp_path) -> ToolrackRepo:
+    return ToolrackRepo.create(tmp_path / "repo")
+
+
+@pytest.fixture(scope="session")
+def toolrack_require_bash() -> None:
+    if not bash_available():
+        pytest.skip("no working bash available")
+
+
+@pytest.fixture()
+def toolrack_require_posix() -> None:
+    if os.name == "nt":
+        pytest.skip("requires a POSIX host")

toolrack_cli-0.2.0/src/toolrack/testing.py

@@ -0,0 +1,262 @@
+"""Helpers for testing a toolrack scripts repository.
+
+These run the real CLI in a subprocess (``python -m toolrack``) with the
+``TOOLRACK_*`` environment pointing at a repository, so a test exercises exactly
+what a user would run, including sidecar parsing, argument handling and the
+script itself. Nothing is monkeypatched.
+
+    repo = ToolrackRepo.create(tmp_path)
+    repo.add_script("greet/hello.py", "print('hi')", {"description": "Say hi."})
+    assert repo.run("greet", "hello").stdout.strip() == "hi"
+    repo.validate()          # raises AssertionError listing every problem
+
+The pytest fixtures in :mod:`toolrack.pytest_plugin` wrap these.
+"""
+
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+from typing import NamedTuple
+
+
+class RunResult(NamedTuple):
+    args: tuple[str, ...]
+    returncode: int
+    stdout: str
+    stderr: str
+
+    @property
+    def ok(self) -> bool:
+        return self.returncode == 0
+
+    def check(self) -> "RunResult":
+        """Return self, or raise AssertionError with full output on failure."""
+        if not self.ok:
+            raise AssertionError(
+                f"toolrack {' '.join(self.args)} exited {self.returncode}\n"
+                f"--- stdout ---\n{self.stdout}\n--- stderr ---\n{self.stderr}"
+            )
+        return self
+
+
+_DEFAULT_CONTENT = {
+    ".py": "print('ok')\n",
+    ".sh": "#!/usr/bin/env bash\necho ok\n",
+    ".bash": "#!/usr/bin/env bash\necho ok\n",
+    ".sql": "SELECT 1;\n",
+}
+
+
+def bash_available() -> bool:
+    """True when a working ``bash`` can run scripts.
+
+    Checks by running it: on Windows ``bash`` on PATH is often the WSL stub,
+    which exists but fails when no distribution is installed.
+    """
+    from toolrack import cli
+
+    try:
+        done = subprocess.run(
+            [cli._resolve_bash_exe(), "-c", "echo toolrack"],
+            capture_output=True,
+            text=True,
+            timeout=15,
+            check=False,
+        )
+    except (OSError, subprocess.TimeoutExpired):
+        return False
+    return done.returncode == 0 and done.stdout.strip() == "toolrack"
+
+
+class StubCommand:
+    """A fake executable on the test PATH that records how it was called."""
+
+    def __init__(self, path: Path, log: Path) -> None:
+        self.path = path
+        self._log = log
+
+    def calls(self) -> list[dict]:
+        """Every invocation so far: ``{"argv": [...], "env": {...}}``."""
+        import json
+
+        if not self._log.is_file():
+            return []
+        return [json.loads(line) for line in self._log.read_text(encoding="utf-8").splitlines() if line]
+
+
+class ToolrackRepo:
+    """A scripts repository (real or temporary) driven through the real CLI.
+
+    Supports every script kind toolrack runs: Python (``.py``), Bash
+    (``.sh``/``.bash``) and SQL (``.sql``, which toolrack runs as
+    ``psql -f <file>``; fake ``psql`` with :meth:`stub_executable`).
+    """
+
+    def __init__(self, root: str | os.PathLike[str], cli_name: str = "toolrack") -> None:
+        self.root = Path(root).resolve()
+        self.cli_name = cli_name
+
+    # -- construction -----------------------------------------------------
+
+    @classmethod
+    def create(cls, root: str | os.PathLike[str]) -> "ToolrackRepo":
+        """Make an empty repository (``scripts/`` plus an empty registry)."""
+        repo = cls(root)
+        repo.scripts.mkdir(parents=True, exist_ok=True)
+        repo.registry.touch()
+        return repo
+
+    @classmethod
+    def discover(cls, start: str | os.PathLike[str] | None = None) -> "ToolrackRepo":
+        """Find the repository containing ``start`` (default: cwd), like the CLI does."""
+        from toolrack import runtime
+
+        cwd = os.getcwd()
+        try:
+            if start is not None:
+                os.chdir(start)
+            return cls(runtime.resolve_repository().repo_root)
+        finally:
+            os.chdir(cwd)
+
+    # -- layout -----------------------------------------------------------
+
+    @property
+    def scripts(self) -> Path:
+        return self.root / "scripts"
+
+    @property
+    def registry(self) -> Path:
+        return self.root / ".toolrack"
+
+    @property
+    def aliases(self) -> Path:
+        return self.root / "aliases.cfg"
+
+    def registered(self) -> list[str]:
+        if not self.registry.is_file():
+            return []
+        lines = (line.strip() for line in self.registry.read_text(encoding="utf-8").splitlines())
+        return [line for line in lines if line and not line.startswith("#")]
+
+    # -- building a repo --------------------------------------------------
+
+    def add_script(
+        self,
+        rel_path: str,
+        content: str | None = None,
+        sidecar: dict | None = None,
+        register: bool = True,
+    ) -> Path:
+        """Write a script (and optional YAML sidecar) under ``scripts/``.
+
+        ``sidecar`` is dumped as the adjacent ``.yml``. With ``register=True``
+        the repo-relative path is appended to the registry. Writing files
+        directly keeps this independent of the commands under test.
+        """
+        import yaml
+
+        path = self.scripts / rel_path
+        path.parent.mkdir(parents=True, exist_ok=True)
+        if content is None:
+            content = _DEFAULT_CONTENT.get(path.suffix.lower(), "")
+        # newline="\n": Bash must never see CRLF, whatever the host OS.
+        path.write_text(content, encoding="utf-8", newline="\n")
+        if sidecar is not None:
+            path.with_suffix(".yml").write_text(
+                yaml.safe_dump(sidecar, sort_keys=False, allow_unicode=True), encoding="utf-8"
+            )
+        if register:
+            entry = path.relative_to(self.root).as_posix()
+            if entry not in self.registered():
+                with open(self.registry, "a", encoding="utf-8") as handle:
+                    handle.write(entry + "\n")
+        return path
+
+    def write_aliases(self, groups: dict[str, str]) -> Path:
+        lines = ["[groups]"] + [f"{name} = {alias}" for name, alias in groups.items()]
+        self.aliases.write_text("\n".join(lines) + "\n", encoding="utf-8")
+        return self.aliases
+
+    def stub_executable(
+        self, name: str, stdout: str = "", exit_code: int = 0, record_env: tuple[str, ...] = ()
+    ) -> StubCommand:
+        """Put a fake ``name`` first on PATH for :meth:`run`, recording each call.
+
+        Use it for tools your scripts or toolrack shell out to, most notably
+        ``psql`` for ``.sql`` scripts, so tests need no real database.
+        ``record_env`` lists environment variables to capture per call (for
+        example ``("PGDATABASE",)``). POSIX only.
+        """
+        if os.name == "nt":
+            raise NotImplementedError(
+                "stub_executable needs a POSIX host: Windows cannot launch a script shim "
+                "by bare name without a compiled .exe"
+            )
+        bin_dir = self.root / ".toolrack-test-bin"
+        bin_dir.mkdir(exist_ok=True)
+        log = bin_dir / f"{name}.calls.jsonl"
+        shim = bin_dir / name
+        shim.write_text(
+            f"#!{sys.executable}\n"
+            "import json, os, sys\n"
+            f"record = {{'argv': sys.argv[1:], 'env': {{k: os.environ.get(k) for k in {list(record_env)!r}}}}}\n"
+            f"open({str(log)!r}, 'a', encoding='utf-8').write(json.dumps(record) + '\\n')\n"
+            f"sys.stdout.write({stdout!r})\n"
+            f"sys.exit({exit_code})\n",
+            encoding="utf-8",
+            newline="\n",
+        )
+        shim.chmod(0o755)
+        return StubCommand(shim, log)
+
+    # -- running ----------------------------------------------------------
+
+    def env(self, extra: dict[str, str] | None = None) -> dict[str, str]:
+        env = {
+            key: value
+            for key, value in os.environ.items()
+            if not key.startswith("TOOLRACK_") and not key.endswith("_COMPLETE")
+        }
+        env.update(
+            TOOLRACK_CLI_NAME=self.cli_name,
+            TOOLRACK_REPO_ROOT=str(self.root),
+            TOOLRACK_SCRIPTS_ROOT=str(self.scripts),
+            TOOLRACK_REGISTRY_FILE=str(self.registry),
+            PYTHONIOENCODING="utf-8",
+        )
+        stubs = self.root / ".toolrack-test-bin"
+        if stubs.is_dir():
+            env["PATH"] = str(stubs) + os.pathsep + env.get("PATH", "")
+        env.update(extra or {})
+        return env
+
+    def run(
+        self,
+        *args: str,
+        input: str | None = None,
+        env: dict[str, str] | None = None,
+        timeout: float = 60,
+    ) -> RunResult:
+        """Run ``toolrack <args>`` against this repository."""
+        completed = subprocess.run(
+            [sys.executable, "-m", "toolrack", *args],
+            input=input,
+            capture_output=True,
+            text=True,
+            encoding="utf-8",
+            errors="replace",
+            env=self.env(env),
+            cwd=self.root,
+            timeout=timeout,
+            check=False,
+        )
+        return RunResult(args, completed.returncode, completed.stdout, completed.stderr)
+
+    def validate(self, strict: bool = False) -> RunResult:
+        """Run ``core validate``; raise AssertionError listing problems if it fails."""
+        args = ["core", "validate", *(["--strict"] if strict else [])]
+        return self.run(*args).check()

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0/src/toolrack_cli.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: toolrack-cli
-Version: 0.1.0
+Version: 0.2.0
 Summary: Turn a folder of small scripts plus YAML sidecars into one typed, completable CLI
 Author: Jan Chimiak
 License-Expression: MIT
@@ -33,6 +33,9 @@ and builds a typed Click CLI at runtime.
 If you want the starter repository that uses this engine, see
 [`toolrack-template`](https://github.com/anfit/toolrack-template).
 
+[![PyPI](https://img.shields.io/pypi/v/toolrack-cli)](https://pypi.org/project/toolrack-cli/)
+[![Unit tests](https://github.com/anfit/toolrack/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/anfit/toolrack/actions/workflows/unit-tests.yml)
+
 ![Animated toolrack walkthrough: register, discover, and run scripts](docs/assets/toolrack-demo.gif)
 
 ## Quickstart
@@ -127,10 +130,72 @@ Which also makes the AI prompt much nicer:
 
 > Hey AI, make me one like this, but mauve.
 
+## Testing Your Tools
+
+Installing `toolrack-cli` also gives your test suite a pytest plugin. It runs the
+real CLI in a subprocess against a repository, so tests cover what users run:
+sidecar parsing, argument mapping, exit codes and the script itself. Python,
+Bash and SQL scripts are all supported.
+
+**Check the whole repo in CI.** `core validate` is read-only and exits non-zero
+on a missing script or sidecar, an invalid sidecar, a bad `aliases.cfg`, or two
+scripts that map to the same command. Unregistered sidecar pairs and a stale
+cache are warnings; `--strict` makes them failures.
+
+```sh
+my core validate --strict
+```
+
+**Or from pytest**, against your real repo (`toolrack_project`) or a throwaway
+one (`toolrack_repo`):
+
+```python
+def test_repo_is_valid(toolrack_project):
+    toolrack_project.validate(strict=True)   # AssertionError lists every problem
+
+def test_python_tool(toolrack_repo):
+    toolrack_repo.add_script(
+        "greet/hello.py",
+        "import sys\nprint('hi', sys.argv[2])\n",
+        {"description": "Say hi.", "args": [{"name": "name", "required": True}]},
+    )
+    assert toolrack_repo.run("greet", "hello", "--name", "Ada").check().stdout == "hi Ada\n"
+
+def test_bash_tool(toolrack_repo, toolrack_require_bash):      # skips without bash
+    toolrack_repo.add_script("ops/fail.sh", "exit 3\n", {"description": "Fails."})
+    assert toolrack_repo.run("ops", "fail").returncode == 3
+
+def test_sql_tool(toolrack_repo, toolrack_require_posix):      # no database needed
+    psql = toolrack_repo.stub_executable("psql", record_env=("PGDATABASE",))
+    script = toolrack_repo.add_script("db/ping.sql", "SELECT 1;", {"description": "Ping."})
+    toolrack_repo.run("db", "ping", env={"PGDATABASE": "dev"}).check()
+    assert psql.calls()[0]["argv"] == ["-f", str(script)]
+```
+
+SQL scripts run as `psql -f <file>`, so `stub_executable` puts a recording fake
+`psql` first on `PATH` instead of needing a database. It works on POSIX hosts
+only. `ToolrackRepo` is in `toolrack.testing` if you prefer to use it without
+the fixtures.
+
+## Migrating From a Checked-Out toolrack
+
+If you installed the engine editable from a clone (`pip install -e path/to/toolrack`,
+distribution name `toolrack`), switch your scripts repo to the PyPI release:
+
+```sh
+python3 ./sync_toolrack.py                              # refresh setup_toolrack.py
+python3 ./setup_toolrack.py --yes --force --cli-name <your-command>
+```
+
+Setup removes the old editable install, installs `toolrack-cli`, rewrites the
+wrappers, and regenerates completion. Full steps, plus the manual route, are in
+the [toolrack-template README](https://github.com/anfit/toolrack-template#migrating-from-a-checked-out-toolrack).
+
 ## Deployment Model
 
-`toolrack` is meant to be installed into a scripts repository and launched
-through a repo-local wrapper in `bin/`.
+`toolrack` is meant to be installed into a scripts repository's virtualenv
+(`pip install toolrack-cli`) and launched through a repo-local wrapper in `bin/`.
+The PyPI distribution is `toolrack-cli`; the package and command are `toolrack`.
 
 The wrapper is responsible for:
 

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/src/toolrack_cli.egg-info/SOURCES.txt

@@ -6,7 +6,9 @@ src/toolrack/__main__.py
 src/toolrack/cli.py
 src/toolrack/completion.py
 src/toolrack/entrypoint.py
+src/toolrack/pytest_plugin.py
 src/toolrack/runtime.py
+src/toolrack/testing.py
 src/toolrack_cli.egg-info/PKG-INFO
 src/toolrack_cli.egg-info/SOURCES.txt
 src/toolrack_cli.egg-info/dependency_links.txt
@@ -15,4 +17,6 @@ src/toolrack_cli.egg-info/requires.txt
 src/toolrack_cli.egg-info/top_level.txt
 tests/test_cli.py
 tests/test_completion.py
-tests/test_cross_shell.py
+tests/test_cross_shell.py
+tests/test_testing.py
+tests/test_validate.py

{toolrack_cli-0.1.0 → toolrack_cli-0.2.0}/src/toolrack_cli.egg-info/entry_points.txt

@@ -1,2 +1,5 @@
 [console_scripts]
 toolrack = toolrack.entrypoint:main
+
+[pytest11]
+toolrack = toolrack.pytest_plugin

toolrack_cli-0.2.0/tests/test_testing.py

@@ -0,0 +1,130 @@
+"""The public test helpers: ToolrackRepo (real CLI in a subprocess) and its fixtures."""
+
+import pytest
+
+from toolrack.testing import RunResult, ToolrackRepo
+
+
+def test_add_script_registers_and_runs(tmp_path):
+    repo = ToolrackRepo.create(tmp_path)
+    repo.add_script(
+        "greet/hello.py",
+        "import sys\nprint('hello,', sys.argv[2])\n",
+        {"description": "Say hi.", "args": [{"name": "name", "required": True}]},
+    )
+
+    assert repo.registered() == ["scripts/greet/hello.py"]
+    result = repo.run("greet", "hello", "--name", "Ada").check()
+    assert result.stdout.strip() == "hello, Ada"
+
+
+def test_run_returns_script_exit_code(tmp_path):
+    repo = ToolrackRepo.create(tmp_path)
+    repo.add_script("t/fail.py", "raise SystemExit(7)\n", {"description": "fails"})
+    assert repo.run("t", "fail").returncode == 7
+
+
+def test_validate_passes_then_fails_with_listing(tmp_path):
+    repo = ToolrackRepo.create(tmp_path)
+    repo.add_script("t/ok.py", sidecar={"description": "ok"})
+    repo.run("core", "refresh-cache").check()
+    repo.validate(strict=True)
+
+    repo.add_script("t/nosidecar.py")
+    with pytest.raises(AssertionError) as info:
+        repo.validate()
+    assert "scripts/t/nosidecar.py" in str(info.value)
+
+
+def test_environment_is_isolated_from_the_callers(tmp_path, monkeypatch):
+    other = tmp_path / "other"
+    other.mkdir()
+    monkeypatch.setenv("TOOLRACK_REPO_ROOT", str(other))
+    repo = ToolrackRepo.create(tmp_path / "mine")
+    repo.add_script("t/x.py", sidecar={"description": "x"})
+    assert "scripts/t/x.py" in repo.run("core", "list").stdout
+
+
+def test_write_aliases(tmp_path):
+    repo = ToolrackRepo.create(tmp_path)
+    repo.add_script("long_group_name/x.py", sidecar={"description": "x"})
+    repo.write_aliases({"long_group_name": "lg"})
+    assert repo.run("lg", "x").ok
+
+
+def test_discover_finds_enclosing_repo(tmp_path):
+    repo = ToolrackRepo.create(tmp_path)
+    nested = repo.scripts / "deep"
+    nested.mkdir()
+    assert ToolrackRepo.discover(nested).root == repo.root
+
+
+def test_run_result_check_includes_output():
+    failed = RunResult(("x",), 2, "out", "err")
+    with pytest.raises(AssertionError, match="exited 2"):
+        failed.check()
+
+
+def test_default_content_per_script_kind(tmp_path):
+    repo = ToolrackRepo.create(tmp_path)
+    assert "print" in repo.add_script("a/p.py").read_text()
+    assert "echo" in repo.add_script("a/s.sh").read_text()
+    assert "SELECT" in repo.add_script("a/q.sql").read_text()
+    assert b"\r" not in repo.add_script("a/s.sh").read_bytes()
+
+
+def test_bash_script_runs_with_typed_args(toolrack_repo, toolrack_require_bash):
+    toolrack_repo.add_script(
+        "ops/say.sh",
+        'echo "got: $2 $3"\n',
+        {
+            "description": "Echo args.",
+            "args": [
+                {"name": "name", "required": True},
+                {"name": "loud", "flag": True},
+            ],
+        },
+    )
+    out = toolrack_repo.run("ops", "say", "--name", "Ada", "--loud").check().stdout
+    assert "got: Ada --loud" in out
+
+
+def test_bash_exit_code_propagates(toolrack_repo, toolrack_require_bash):
+    toolrack_repo.add_script("ops/fail.sh", "exit 3\n", {"description": "fails"})
+    assert toolrack_repo.run("ops", "fail").returncode == 3
+
+
+def test_sql_script_is_run_through_psql(toolrack_repo, toolrack_require_posix):
+    psql = toolrack_repo.stub_executable("psql", stdout="1\n", record_env=("PGDATABASE",))
+    script = toolrack_repo.add_script(
+        "db/ping.sql",
+        "SELECT 1;\n",
+        {"description": "Ping.", "env": [{"name": "PGDATABASE", "required": True}]},
+    )
+
+    result = toolrack_repo.run("db", "ping", env={"PGDATABASE": "analytics"}).check()
+
+    assert result.stdout == "1\n"
+    (call,) = psql.calls()
+    assert call["argv"] == ["-f", str(script)]
+    assert call["env"] == {"PGDATABASE": "analytics"}
+
+
+def test_sql_sidecar_with_args_fails_validation(toolrack_repo):
+    toolrack_repo.add_script(
+        "db/bad.sql", sidecar={"description": "x", "args": [{"name": "p", "positional": True}]}
+    )
+    with pytest.raises(AssertionError, match="SQL scripts must not declare"):
+        toolrack_repo.validate()
+
+
+def test_stub_executable_is_rejected_on_windows(tmp_path, monkeypatch):
+    monkeypatch.setattr("toolrack.testing.os.name", "nt")
+    with pytest.raises(NotImplementedError):
+        ToolrackRepo.create(tmp_path).stub_executable("psql")
+
+
+def test_pytest_fixtures_are_provided_by_the_plugin(toolrack_repo, toolrack_project):
+    toolrack_repo.add_script("t/ok.py", sidecar={"description": "ok"})
+    assert toolrack_repo.run("t", "ok").ok
+    assert toolrack_project.root.is_dir()

toolrack_cli-0.2.0/tests/test_validate.py

@@ -0,0 +1,106 @@
+"""`core validate`: CI-oriented, read-only repository check."""
+
+from toolrack import cli
+
+
+def _validate(runner, *extra):
+    return runner.invoke(cli.cli, ["core", "validate", *extra])
+
+
+def _register(repo, *entries):
+    repo["registry"].write_text("".join(f"{e}\n" for e in entries), encoding="utf-8")
+
+
+def test_validate_passes_on_clean_repo(repo, runner, make_script, make_sidecar):
+    make_script("tools/hello.py")
+    make_sidecar("tools/hello.py", {"description": "hi"})
+    _register(repo, "scripts/tools/hello.py")
+    cli._refresh_cache()
+
+    result = _validate(runner, "--strict")
+
+    assert result.exit_code == 0, result.output
+    assert "1 ok, 0 errors, 0 warnings" in result.output
+
+
+def test_validate_empty_repo_is_ok(repo, runner):
+    assert _validate(runner).exit_code == 0
+
+
+def test_validate_fails_on_missing_script(repo, runner):
+    _register(repo, "scripts/gone.py")
+    result = _validate(runner)
+    assert result.exit_code == 1
+    assert "registered script not found" in result.output
+
+
+def test_validate_fails_on_missing_sidecar(repo, runner, make_script):
+    make_script("tools/bare.py")
+    _register(repo, "scripts/tools/bare.py")
+    result = _validate(runner)
+    assert result.exit_code == 1
+    assert "sidecar missing" in result.output
+
+
+def test_validate_fails_on_invalid_sidecar(repo, runner, make_script, make_sidecar):
+    make_script("tools/bad.py")
+    make_sidecar("tools/bad.py", {"args": [{"name": "x", "variadic": True}]})
+    _register(repo, "scripts/tools/bad.py")
+    result = _validate(runner)
+    assert result.exit_code == 1
+    assert "variadic" in result.output
+
+
+def test_validate_fails_on_malformed_yaml(repo, runner, make_script):
+    make_script("tools/broken.py")
+    (repo["scripts"] / "tools" / "broken.yml").write_text("description: [unclosed\n", encoding="utf-8")
+    _register(repo, "scripts/tools/broken.py")
+    result = _validate(runner)
+    assert result.exit_code == 1
+    assert "scripts/tools/broken.py" in result.output
+
+
+def test_validate_reports_every_problem_not_just_the_first(repo, runner):
+    _register(repo, "scripts/a.py", "scripts/b.py")
+    result = _validate(runner)
+    assert "scripts/a.py" in result.output and "scripts/b.py" in result.output
+    assert "0 ok, 2 errors" in result.output
+
+
+def test_validate_fails_on_alias_collision(repo, runner, aliases_cfg):
+    aliases_cfg({"one": "x", "two": "x"})
+    result = _validate(runner)
+    assert result.exit_code == 1
+    assert "collision" in result.output
+
+
+def test_validate_fails_on_duplicate_command_names(repo, runner, make_script, make_sidecar):
+    for name in ("hello_world.py", "hello-world.py"):
+        make_script(f"tools/{name}")
+        make_sidecar(f"tools/{name}", {"description": "x"})
+    _register(repo, "scripts/tools/hello_world.py", "scripts/tools/hello-world.py")
+    result = _validate(runner)
+    assert result.exit_code == 1
+    assert "already provided by" in result.output
+
+
+def test_validate_warns_about_unregistered_pair(repo, runner, make_script, make_sidecar):
+    make_script("tools/loose.py")
+    make_sidecar("tools/loose.py", {"description": "x"})
+    result = _validate(runner)
+    assert result.exit_code == 0
+    assert "not registered" in result.output
+    assert _validate(runner, "--strict").exit_code == 1
+
+
+def test_validate_warns_on_stale_cache_but_does_not_write(repo, runner, make_script, make_sidecar):
+    make_script("tools/hello.py")
+    make_sidecar("tools/hello.py", {"description": "hi"})
+    _register(repo, "scripts/tools/hello.py")
+
+    result = _validate(runner)
+
+    assert result.exit_code == 0
+    assert "cache is missing or stale" in result.output
+    assert not repo["cache"].exists()
+    assert _validate(runner, "--strict").exit_code == 1