@amsterdamdatalabs/adl-agent-control-linux-x64 0.1.12 → 0.1.28

package/packages/repo-controls/package/adl-compliance/src/adl_compliance/checks/repository/lockfile_integrity/check.py

@@ -26,11 +26,26 @@ declared. `<rust>` resolves to `backend` and `<typescript>` resolves to
 defaults, just reached through `ctx.inputs` now, the same as every other
 check that names a consumer path.
 
+A tree declared absent (`rust = ""` or `typescript = ""` in
+`[controls.compliance.trees]`) drops the inputs naming it, and each half of this
+row is then graded on its own: a repository declaring no Rust tree is graded on
+its TypeScript lockfile alone and is never asked for a `Cargo.lock`. With
+neither tree left the row is could-not-run (see `structural`) -- it used to raise
+`KeyError`.
+
+Retired 2026-10-01: `coverage.backend-ratchet` and `coverage.backend-crate`,
+named here as precedents, were deleted with the `coverage` domain
+(COV-001..005); the references record the pattern this check followed.
+
   LCK-001   a required lockfile is missing or empty (`[ -s FILE ]` in check.sh
             — MISSING and EMPTY are the same finding, worded the same way).
   LCK-002   `cargo metadata --locked` rejects `backend/Cargo.lock` against
             `backend/Cargo.toml` — the lock exists but does not satisfy the
-            manifest.
+            manifest — or the lock's CONTENT fails the validation in
+            `cargo_lock.py` (see "`--no-deps` never reads the lockfile" below):
+            not TOML, an unsupported `version`, no packages, a package listed
+            as a dependency but absent, a workspace member or a direct
+            dependency of one missing from it.
 
 **check.sh stops at the first problem.** Each of the three failure branches
 ends in `report_summary; exit` — a missing frontend lockfile is never reported
@@ -56,29 +71,112 @@ finding (`LCK-002`) here exactly as it is in check.sh — this check trusts
 `cargo --version` having already ruled out "cargo cannot run at all" for the
 same reason check.sh does.
 
+**`--no-deps` never reads the lockfile (measured 2026-10-01).** check.sh's command,
+kept unchanged, passes `--no-deps`, so cargo never resolves and never opens
+`Cargo.lock`: a lock that was `version = 4` alone, one that was not TOML, and one
+with `version = 99` all came out CLEAN through this check, and only a missing or
+empty lock was ever reported. Dropping `--no-deps` is not the fix: cargo then loads
+every dependency's manifest, which fails offline whenever the crate sources are not
+in the registry cache, so the gate would pass on one machine and fail or be
+could-not-run on another. The lock's CONTENT is therefore judged in Python, in
+addition to the cargo call and after it (`cargo_lock.CargoLock`, `tomllib`, no cargo,
+no registry, no network; what it checks, and what it cannot judge and so refuses with
+could-not-run, is that module's docstring). Per lock the order is: `cargo --version`;
+`cargo metadata`, whose rejection is the one finding for that lock (cargo is the
+authority on a manifest it refuses, and a lock cannot be compared with one); only when
+cargo accepts, the content validation, one `LCK-002` per problem with the same remedy.
+The limit, stated plainly: version-REQUIREMENT drift (a manifest raising a requirement
+past the locked version) is not detected offline; `cargo metadata` without `--no-deps`
+is the authority for that and needs the registry.
+
 cargo resolves its toolchain from the WORKING DIRECTORY — rustup applies
 `backend/rust-toolchain.toml` as a directory override, and `--manifest-path`
 does not change that: toolchain selection precedes reading the manifest. So
 both cargo calls run with `cwd` set to the workspace directory, not the
 manifest's own directory, matching check.sh's identical comment.
+
+**A repository can declare exactly which paths this gate scans and grades**
+(Tarun, 2026-10-01: "it should be configurable and declarative on what paths
+should be scanned"). `[controls.compliance.lockfiles]` in `adl-config.toml`
+reaches the engine as four comma-list declarations, a HALF (Bun, Cargo) being
+declared by its manifests list and its lockfiles list together:
+
+  bun-manifests     / bun-lockfiles       `package.json` paths or globs / `bun.lock` paths
+  cargo-manifests   / cargo-lockfiles     `Cargo.toml` paths or globs  / `Cargo.lock` paths
+
+Why: the registry scope is `<typescript>/package.json` and `<typescript>/bun.lock`
+(and the `<rust>` pair), a tree placeholder joined to a ONE-level file name.
+enact-combined's `typescript` tree is a brace union of directories and its
+manifests sit at the repository ROOT and under `apps/*/` and `packages/*/`, so on
+a whole-tree run `engine.scope` matched no file and the row was could-not-run
+(measured live: a narrowed run passed, the exhaustive run could not). The
+per-gate `bun_lock` input moved the grade and never the scope.
+
+A declared half REPLACES that half's tree defaults, in both places the paths are
+used:
+
+  * scope (`declared_scope`): which changed files select the gate, and, because a
+    restated scope is graded whole, which files `graded` is handed;
+  * grade (`structural`): each declared lockfile must exist and be non-empty
+    (LCK-001, same wording and remedy as before); each declared Cargo lockfile's
+    directory is the cargo workspace and `<that directory>/Cargo.toml` its
+    manifest (LCK-002 as before).
+
+`graded` makes the declaration honest: every declared manifest entry must match
+at least one file of the complete in-scope set, or the row is could-not-run
+naming the declaration and the entry. A declared glob that matches nothing would
+otherwise shrink what the gate scans to nothing and say OK.
+
+An undeclared half is exactly what it was: tree defaults, the `inputs`, the same
+early-return shape. A malformed declaration (empty, absolute, backslash, `.`/`..`
+segments, placeholder or brace or negation syntax, duplicate, the wrong file
+name, a glob in a lockfile entry, or a half with only one of its two lists) is
+could-not-run naming the declaration; the engine never grades against a guess at
+what was meant. `adl` refuses the same values when it reads `adl-config.toml`.
 """
 
 from __future__ import annotations
 
 import shutil
 import subprocess
-from collections.abc import Iterable
-from typing import ClassVar
+from collections.abc import Iterable, Mapping
+from pathlib import Path
+from typing import ClassVar, NamedTuple
 
 from ....core.check import Check, FileState
-from ....core.context import Context
+from ....core.context import Context, FileSet
 from ....core.errors import CouldNotRun
 from ....core.finding import Report
+from ....core.scope import matches
+from .cargo_lock import CargoLock, Undecidable
+
+
+class Half(NamedTuple):
+    """One of the two ecosystems this row grades, and the names that declare it."""
+
+    manifests: str
+    """The `--declare` name carrying the manifest paths or globs."""
+    lockfiles: str
+    """The `--declare` name carrying the literal lockfile paths."""
+    manifest_name: str
+    """The file name every manifest entry must end in."""
+    tree: str
+    """Prefix of the registry scope entries a declaration of this half replaces."""
+
+
+class Declared(NamedTuple):
+    manifests: tuple[str, ...]
+    lockfiles: tuple[str, ...]
 
 
 class LockfileIntegrity(Check):
     id: ClassVar[str] = "REP-002"
 
+    CARGO: ClassVar[Half] = Half("cargo-manifests", "cargo-lockfiles", "Cargo.toml", "<rust>/")
+    BUN: ClassVar[Half] = Half("bun-manifests", "bun-lockfiles", "package.json", "<typescript>/")
+    # Cargo first, as the registry lists it and as `structural` grades it.
+    HALVES: ClassVar[tuple[Half, ...]] = (CARGO, BUN)
+
     inputs: ClassVar[dict[str, str]] = {
         "workspace": "<rust>",
         "cargo_lock": "<rust>/Cargo.lock",
@@ -94,33 +192,208 @@ class LockfileIntegrity(Check):
     MISSING: ClassVar[str] = "LCK-001"
     STALE: ClassVar[str] = "LCK-002"
 
+
+    # ------------------------------------------------------------ declarations
+
+    @classmethod
+    def _entries(
+        cls, declarations: Mapping[str, str], name: str, manifest_name: str | None
+    ) -> tuple[str, ...] | None:
+        """The validated entries of one comma-list declaration, `None` when absent.
+
+        `manifest_name` is the file name a manifest entry must end in; `None`
+        marks a lockfile list, whose entries are literal paths. Present but blank
+        is not absent: an empty entry is refused, never read as "declared nothing".
+        """
+        raw = declarations.get(name)
+        if raw is None:
+            return None
+        entries = tuple(part.strip() for part in raw.split(","))
+        for entry in entries:
+            why = cls._refusal(entry, manifest_name)
+            if why is not None:
+                raise CouldNotRun(
+                    f"{cls.id}: declaration `{name}` entry {entry!r} {why}; fix "
+                    f"[controls.compliance.lockfiles].{name.replace('-', '_')} in "
+                    "adl-config.toml -- the gate does not guess which path was meant"
+                )
+        for index, entry in enumerate(entries):
+            if entry in entries[:index]:
+                raise CouldNotRun(
+                    f"{cls.id}: declaration `{name}` lists {entry!r} twice; fix "
+                    f"[controls.compliance.lockfiles].{name.replace('-', '_')} in adl-config.toml"
+                )
+        return entries
+
+    @staticmethod
+    def _refusal(entry: str, manifest_name: str | None) -> str | None:
+        """Why `entry` is not a repository-relative path this row can scan, or None.
+
+        The same set `adl` refuses when it reads `adl-config.toml`. Placeholder,
+        brace and negation syntax are scope SYNTAX, and a declared entry is a
+        path: letting one through would be a pattern that means more or less than
+        the repository wrote.
+        """
+        if entry == "":
+            return "is empty"
+        if entry.startswith("/"):
+            return "is absolute"
+        if "\\" in entry:
+            return "contains a backslash"
+        if entry.startswith("!") or any(char in entry for char in "<>{}"):
+            return "contains scope syntax (a leading '!', a <placeholder>, or braces)"
+        if any(segment in ("", ".", "..") for segment in entry.split("/")):
+            return "has a `.`, `..` or empty path segment"
+        if manifest_name is not None:
+            if entry.rsplit("/", 1)[-1] != manifest_name:
+                return f"does not end in {manifest_name}"
+        elif any(char in entry for char in "*?[]"):
+            return (
+                "is a glob, but a lockfile is a literal path: a missing one is "
+                "reported by name"
+            )
+        return None
+
+    @classmethod
+    def declared(cls, half: Half, declarations: Mapping[str, str]) -> Declared | None:
+        """What the repository declares for one half, or `None` when it declares nothing.
+
+        A half is declared by BOTH lists. One without the other is refused: the
+        missing list would have to fall back to a tree default, and which default
+        is exactly the guess this declaration exists to remove.
+        """
+        manifests = cls._entries(declarations, half.manifests, half.manifest_name)
+        lockfiles = cls._entries(declarations, half.lockfiles, None)
+        if manifests is None and lockfiles is None:
+            return None
+        if manifests is None or lockfiles is None:
+            present, missing = (
+                (half.lockfiles, half.manifests) if manifests is None else (half.manifests, half.lockfiles)
+            )
+            raise CouldNotRun(
+                f"{cls.id}: declaration `{present}` is set without `{missing}`; a half is "
+                "declared as a pair -- set both under [controls.compliance.lockfiles] in "
+                "adl-config.toml"
+            )
+        return Declared(manifests, lockfiles)
+
+    @classmethod
+    def declared_scope(
+        cls, scope: tuple[str, ...], declarations: Mapping[str, str]
+    ) -> tuple[str, ...] | None:
+        """The registry scope with each declared half's tree defaults replaced.
+
+        A half's defaults are the registry entries under its tree prefix
+        (`<rust>/...`, `<typescript>/...`); `test_the_registry_scope_the_hook_
+        replaces_is_two_entries_per_tree` pins that shape. The other half, and
+        anything else the row lists, is kept as it was.
+        """
+        restated = scope
+        any_declared = False
+        for half in cls.HALVES:
+            declared = cls.declared(half, declarations)
+            if declared is None:
+                continue
+            any_declared = True
+            restated = (
+                *(pattern for pattern in restated if not pattern.startswith(half.tree)),
+                *declared.manifests,
+                *declared.lockfiles,
+            )
+        return restated if any_declared else None
+
     # ------------------------------------------------------------- structural
 
+    # The three inputs that all name the `<rust>` tree. They are resolved or
+    # dropped together, so seeing only some of them means a per-gate
+    # `--input REP-002:<name>=...` replaced part of the triple.
+    CARGO_INPUTS: ClassVar[tuple[str, ...]] = ("workspace", "cargo_lock", "cargo_manifest")
+
     def structural(self, ctx: Context) -> Iterable[Report]:
-        cargo_lock = ctx.inputs["cargo_lock"]
-        bun_lock = ctx.inputs["bun_lock"]
-
-        if self.state(ctx, cargo_lock) is not FileState.PRESENT:
-            shown = ctx.relative(cargo_lock)
-            yield self.report(
-                shown,
-                f"missing {shown}",
-                f"run: (cd {ctx.relative(ctx.inputs['workspace'])} && "
-                "cargo generate-lockfile) and commit the result",
-                rule=self.MISSING,
+        # Decision for a tree declared absent (`rust = ""` / `typescript = ""`,
+        # which drops every input naming the tree). Unlike an exclusion path, a
+        # lockfile IS what this row grades, so each half is judged on its own:
+        #
+        #   * a half whose tree is declared absent is not graded -- the owner
+        #     stated there is no such lockfile to require, which is the narrower,
+        #     honest verdict `resolve_inputs` documents ("grades a smaller scope");
+        #   * with NEITHER half left there is nothing to grade, and an empty set of
+        #     findings would read as a pass -- `CouldNotRun`, naming both inputs and
+        #     trees;
+        #   * a half-resolved Cargo triple is `CouldNotRun`, naming what is missing.
+        #
+        # This read `ctx.inputs["cargo_lock"]` and `["bun_lock"]` unguarded: a
+        # `KeyError` traceback (exit 1), not a verdict, whenever either tree was
+        # declared absent.
+        #
+        # A half the repository DECLARES (see the module docstring) is graded from
+        # its declaration and never from the tree or the inputs; the rules above
+        # apply to the halves it did not declare.
+        cargo_declared = self.declared(self.CARGO, ctx.declarations)
+        bun_declared = self.declared(self.BUN, ctx.declarations)
+
+        present_cargo = [name for name in self.CARGO_INPUTS if name in ctx.inputs]
+        if cargo_declared is None and present_cargo and len(present_cargo) != len(self.CARGO_INPUTS):
+            missing = [name for name in self.CARGO_INPUTS if name not in ctx.inputs]
+            raise CouldNotRun(
+                f"{self.id}: the Cargo inputs resolve together from <rust>, but only "
+                f"{present_cargo} are set; missing {missing} -- declare "
+                "[controls.compliance.trees].rust, or override all three"
             )
-            return
 
-        if self.state(ctx, bun_lock) is not FileState.PRESENT:
-            shown = ctx.relative(bun_lock)
-            typescript_dir = shown.rsplit("/", 1)[0] if "/" in shown else "."
-            yield self.report(
-                shown,
-                f"missing {shown}",
-                f"run: (cd {typescript_dir} && bun install) to generate the "
-                "lockfile and commit the result",
-                rule=self.MISSING,
+        # (lockfile, cargo workspace, manifest) per Cargo lockfile to grade.
+        cargo: list[tuple[Path, Path, Path]] = []
+        if cargo_declared is not None:
+            for relative in cargo_declared.lockfiles:
+                lock = ctx.path(relative)
+                cargo.append((lock, lock.parent, lock.parent / "Cargo.toml"))
+        elif "cargo_lock" in ctx.inputs:
+            cargo.append(
+                (ctx.inputs["cargo_lock"], ctx.inputs["workspace"], ctx.inputs["cargo_manifest"])
+            )
+
+        bun: list[Path] = []
+        if bun_declared is not None:
+            bun = [ctx.path(relative) for relative in bun_declared.lockfiles]
+        elif "bun_lock" in ctx.inputs:
+            bun = [ctx.inputs["bun_lock"]]
+
+        if not cargo and not bun:
+            raise CouldNotRun(
+                f"{self.id}: no lockfile to grade -- `cargo_lock` (<rust>/Cargo.lock) and "
+                "`bun_lock` (<typescript>/bun.lock) are both absent because the trees "
+                "<rust> and <typescript> are both declared absent in "
+                "[controls.compliance.trees]; a row that grades nothing is could-not-run, "
+                "never a clean pass"
             )
+
+        for lock, workspace, _manifest in cargo:
+            if self.state(ctx, lock) is not FileState.PRESENT:
+                shown = ctx.relative(lock)
+                yield self.report(
+                    shown,
+                    f"missing {shown}",
+                    f"run: (cd {ctx.relative(workspace)} && "
+                    "cargo generate-lockfile) and commit the result",
+                    rule=self.MISSING,
+                )
+                return
+
+        for lock in bun:
+            if self.state(ctx, lock) is not FileState.PRESENT:
+                shown = ctx.relative(lock)
+                typescript_dir = shown.rsplit("/", 1)[0] if "/" in shown else "."
+                yield self.report(
+                    shown,
+                    f"missing {shown}",
+                    f"run: (cd {typescript_dir} && bun install) to generate the "
+                    "lockfile and commit the result",
+                    rule=self.MISSING,
+                )
+                return
+
+        if not cargo:
+            # Rust declared absent: the TypeScript lockfile above was the whole row.
             return
 
         if shutil.which("cargo") is None:
@@ -129,39 +402,79 @@ class LockfileIntegrity(Check):
                 "not a finding"
             )
 
-        workspace = ctx.inputs["workspace"]
-        version_check = subprocess.run(
-            ["cargo", "--version"],
-            cwd=workspace,
-            capture_output=True,
-            text=True,
-            check=False,
-        )
-        if version_check.returncode != 0:
-            raise CouldNotRun(
-                f"repository.lockfile-integrity: cargo could not run in "
-                f"{ctx.relative(workspace)}; tool failure, not a finding"
+        for lock, workspace, manifest in cargo:
+            version_check = subprocess.run(
+                ["cargo", "--version"],
+                cwd=workspace,
+                capture_output=True,
+                text=True,
+                check=False,
             )
+            if version_check.returncode != 0:
+                raise CouldNotRun(
+                    f"repository.lockfile-integrity: cargo could not run in "
+                    f"{ctx.relative(workspace)}; tool failure, not a finding"
+                )
 
-        manifest = ctx.inputs["cargo_manifest"]
-        metadata_run = subprocess.run(
-            [
-                "cargo", "metadata", "--locked", "--offline", "--no-deps",
-                "--format-version", "1", "--manifest-path", str(manifest),
-            ],
-            cwd=workspace,
-            capture_output=True,
-            text=True,
-            check=False,
-        )
-        if metadata_run.returncode != 0:
-            lock_shown = ctx.relative(cargo_lock)
+            metadata_run = subprocess.run(
+                [
+                    "cargo", "metadata", "--locked", "--offline", "--no-deps",
+                    "--format-version", "1", "--manifest-path", str(manifest),
+                ],
+                cwd=workspace,
+                capture_output=True,
+                text=True,
+                check=False,
+            )
+            lock_shown = ctx.relative(lock)
             manifest_shown = ctx.relative(manifest)
-            yield self.report(
-                lock_shown,
-                f"{lock_shown} does not satisfy {manifest_shown}",
+            remedy = (
                 f"run: (cd {ctx.relative(workspace)} && cargo update) to "
                 f"resync {lock_shown} with {manifest_shown}, then commit "
-                "the result",
-                rule=self.STALE,
+                "the result"
             )
+            if metadata_run.returncode != 0:
+                yield self.report(
+                    lock_shown,
+                    f"{lock_shown} does not satisfy {manifest_shown}",
+                    remedy,
+                    rule=self.STALE,
+                )
+                continue
+
+            # `--no-deps` never reads the lockfile (see the module docstring), so cargo
+            # accepting the manifest says nothing about the lock. The content is judged
+            # here, deterministically and without cargo; a lock this cannot judge is
+            # could-not-run, never a pass.
+            try:
+                problems = CargoLock(lock, manifest, ctx.relative).problems()
+            except Undecidable as exc:
+                raise CouldNotRun(f"{self.id}: cannot judge {lock_shown}: {exc}") from exc
+            for problem in problems:
+                yield self.report(lock_shown, problem, remedy, rule=self.STALE)
+
+    # ----------------------------------------------------------------- graded
+
+    def graded(self, ctx: Context, files: FileSet) -> Iterable[Report]:
+        """A declared manifest entry that matches no file is could-not-run.
+
+        `files` is the COMPLETE in-scope set here: a declared half restates the
+        row's scope, and a restated scope is graded whole (`Check.declared_scope`),
+        so "matches nothing" is a statement about the repository and not about
+        which files happened to change. Matched with the scope's own matcher.
+        """
+        paths = tuple(files)
+        for half in self.HALVES:
+            declared = self.declared(half, ctx.declarations)
+            if declared is None:
+                continue
+            for pattern in declared.manifests:
+                if not any(matches(pattern, path) for path in paths):
+                    raise CouldNotRun(
+                        f"{self.id}: declaration `{half.manifests}` names {pattern!r}, which "
+                        "matches no file in the repository; fix "
+                        f"[controls.compliance.lockfiles].{half.manifests.replace('-', '_')} "
+                        "in adl-config.toml -- a declared manifest that is not there would "
+                        "shrink what this gate scans to nothing and still read OK"
+                    )
+        return ()

package/packages/repo-controls/package/adl-compliance/src/adl_compliance/checks/repository/lockfile_integrity/test_check.py

@@ -14,7 +14,7 @@ Ported here as two COMMITTED fixture directories built the identical way (see
 file for the exact commands that produced them), rather than reconstructed at
 test time — the same shift every other ported check.sh/test.sh pair in this
 package already made, and the one this task calls out `coverage.backend-ratchet`
-(`COV-002`) as its concrete precedent for.
+(`COV-002`, retired 2026-10-01) as its concrete precedent for.
 
 A real Rust toolchain and a real `cargo` invocation are both required to prove
 the `LCK-002` branch (`cargo metadata --locked` rejecting a stale lock), but
@@ -100,7 +100,9 @@ def test_a_redeclared_rust_tree_is_honoured(tmp_path: Path) -> None:
     graded against `rust-workspace/Cargo.lock`, never the packaged default.
     """
     trees = effective_trees({"rust": "rust-workspace", "typescript": "ts-workspace"})
-    verdict = runner.run(ROW, tmp_path, corpus_files(tmp_path), trees=trees)
+    # The subject here is an EMPTY tree: the lockfile is reported missing because
+    # nothing exists, so the empty corpus is opted into explicitly.
+    verdict = runner.run(ROW, tmp_path, corpus_files(tmp_path, allow_empty=True), trees=trees)
     assert verdict.outcome is Outcome.FINDINGS, verdict.render()
     assert len(verdict.findings) == 1
     finding = verdict.findings[0]