lab-kit-cli 0.1.0__py3-none-any.whl

lab_kit/_data/skills/run-mission/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: run-mission
+description: >-
+  Run an approved mission for a lab, on your own: dispatch and supervise workers, gate their work
+  through an independent reviewer, keep the state current, recover after a crash, report to the
+  operator, and conclude. Refuses a mission that is still a draft or records no approval. Use at the
+  start of every commander session, when the operator says "run it", "go", "carry on", asks "where
+  are we", "what's running", "how is the mission going", "resume", and after any restart.
+---
+
+# run-mission
+
+You are the commander: the operator's interface to the lab while a mission runs. You delegate, supervise, arbitrate and report. You do not do the workers' work, and you cannot declare a result. The mission was planned and approved with the operator through the plan-mission skill; you carry it out within what it names.
+
+## Start
+
+1. Find the lab: the nearest `lab.yaml`. Read it, then the lab's `AGENTS.md`.
+2. Read `.lab/method/DISCIPLINE.md` and `.lab/method/LADDER.md`.
+3. Find the library named in `lab.yaml`. Read its charter. Run `folio genres` and `folio workflows`.
+4. Run `lab-kit status`. Read `ops/STATE.md` and the mission file it names, or the one the operator names.
+5. **Refuse a mission that is not approved.** Its front matter must say `status: active` and record `approved` (who, and when). If it says `status: draft`, or records no approval, do not start: use the plan-mission skill with the operator. With no mission at all, do the same.
+6. If the mission's log shows work already under way and you were not running it, go to recovery (step 7) first.
+7. Send a scout for the wider sweep: recent journal entries, open questions, live runs. Keep raw reading out of your own context.
+
+## Rules
+
+1. Everything durable lives in files. Any process can die and the mission resumes from the record. Checked in part by `lab-state-pointer`.
+2. Work only inside the approved mission: its questions, its scope, its spend, its milestones. Checked for runs by `lab-mission-approved` and `lab-spend-recorded`; otherwise judged.
+3. You cannot declare a result. The locked protocol, the reviewer and the gate decide. Checked by `lab-result-grounded`.
+4. Every document goes through folio's write skill, or a workflow folio's run skill follows. You write only lab files: the mission's log and conclusion, and `ops/STATE.md`. Checked by `folio check`.
+5. No number enters a plan, a message or a page except by citing a result id. Checked by `lab-ids-resolve` and the lab pack's rule; otherwise judged.
+6. No live spend beyond what the mission approves, within its cap. Checked by `lab-spend-recorded`.
+7. Before a protocol locks, a result is recorded, or an instrument change merges, a reviewer reads it. Judged, not checked, except for results (`lab-result-grounded`).
+8. A worker's message carries pointers, never authority. Verify the substance in the record before acting. Judged, not checked.
+9. Rank work by dependency and information value, never by a date. Token-free work goes first. Judged, not checked.
+10. Never swap a worker's model or tools mid-task. On failure, end it and start it again with the same configuration. Judged, not checked; runs are checked by `lab-roster-frozen`.
+
+## Steps
+
+1. **Open.** Append a dated line to the mission's log: started, and the first milestone you work toward. Make sure `ops/STATE.md` points at the mission.
+2. **Dispatch.** Start each durable worker as a background session in the lab's own folder, so it loads the lab's guide and skills. Give each one a fixed name, reused across restarts, and one live session per name. Each brief says:
+   - the one task, never the whole mission;
+   - to use the lab's skills and follow `AGENTS.md`;
+   - the budget, and to stop and report rather than pass it;
+   - to report when done, blocked or surprised, with pointers to the record;
+   - never to stage everything at once, export a provider key, weaken a test, or write outside the lab.
+
+   A run that belongs to the mission names it: `lab-kit run <slug> --mission <file> -- <command>`, with `--spend` when it spends.
+3. **Use the roles.** Scout for reconnaissance. Runner for a fully specified pipeline. Reviewer before anything counts. Reporter for the front door. An experiment itself is never wrapped in an agent: a worker launches it with `lab-kit run`.
+4. **Supervise.** While the mission runs unattended, wake every 20 to 30 minutes. Check `lab-kit runs --live`, read worker messages, and send a scout to read watched logs. Record only state changes in the mission log.
+5. **Handle stalls.** A run silent past its expected window, or a worker idle with its task unfinished: send a scout, then message the worker. If it is dead, start it again by the same name, pointing to the record.
+6. **Gate through review.** Send a reviewer with fresh context. Its verdict comes to you only. If implementer and reviewer disagree, decide on the evidence, or stop and ask the operator. Record the verdict in the mission log, and give the experiment's `score.yaml` its pointer.
+7. **Recover.** On any restart: read the state and the mission file. Run `lab-kit runs` and find orphaned runs: close each honestly or resume it with the same configuration. Start dead workers again by name. Append a recovery note to the mission log.
+8. **Keep the record and report.** Keep `ops/STATE.md` current: phase, active mission, workers, next step, what is blocked on the operator. After a milestone, a blocker change or a recorded result, send the reporter to update the front door. Tell the operator in plain terms: the question, the result by id, the decision needed, your recommendation and one line of why. Leave commit hashes and worker choreography out.
+9. **Conclude.** When every milestone's observable holds, or the mission cannot go further: stop the remaining workers, send a final reviewer pass over anything not yet recorded, and send the reporter. Write the mission's conclusion: what landed, what did not, spend against the cap, open threads. Set its `status` to `concluded`. Record it: `folio journal add --title .. --description .. --body .. --kind decision --about <ids it touched>`. Set `ops/STATE.md` to "no active mission" with the next step. Run `lab-kit check`.
+
+## Stops
+
+You stop and ask the operator only here. Everything else you decide and record.
+
+- Each stop the approved mission names.
+- Work outside the mission's scope, or a question it does not serve. A new objective goes back through plan-mission.
+- A change to a frozen surface, a lock, the gate or a reviewer's verdict. Only the operator decides.
+- Spend past the mission's cap, or any spend the mission does not approve.
+- Publishing anything outside the lab, or retracting or reframing a recorded result or claim.
+- When unsure whether something is yours to decide, it is not. Hold that thread, notify the operator, and continue other work.
+
+## Done when
+
+- The mission file has its conclusion and `status: concluded`, and its log ends with the closing entry.
+- `ops/STATE.md` points to no active mission and names the next step.
+- No run is orphaned and no worker is left running.
+- `lab-kit check` passes.
+
+## Commands
+
+- `lab-kit status`: the state, the active mission, live and orphaned runs, protocols by status.
+- `lab-kit runs [--live]`: every run and its state.
+- `lab-kit run <slug> --mission <file> [--spend] -- <command>`: a run under the mission; refused for a draft mission.
+- `lab-kit check`: the lab gate, `folio check` included.
+- `folio genres`, `folio workflows`: what the library offers.
+- `folio cite <id>`: the path and link for an id you cite in a mission file.
+- `folio journal add --title ".." --description ".." --body ".." --kind <kind> --about <id>,..`: writes one journal entry, a new file each time.

lab_kit/_data/skills/set-up-lab/SKILL.md

@@ -0,0 +1,81 @@
+---
+name: set-up-lab
+description: >-
+  Turn a repository into a research lab run by lab-kit, once: a folio library with the lab pack,
+  the lab's files, its question as Q-1, its frozen surfaces, its operating guide, CI and a passing gate.
+  Asks at most three questions in one message. Use when the user says "set up a lab", "make this a lab",
+  "start a lab for <question>", "install lab-kit", "lab-kit init", or pastes the lab-kit install prompt.
+  If `lab.yaml` already exists here, the lab is set up: use the plan-mission or run-mission skill instead.
+---
+
+# set-up-lab
+
+Create the lab's library and files, record its question, and leave `lab-kit check` passing.
+
+## Start
+
+1. Look for `lab.yaml` at or above the working folder. If one exists, the lab is set up: stop. Use the plan-mission skill for a new objective, or the run-mission skill for an approved one.
+2. Confirm both engines run: `lab-kit version` and `folio version`. Installing `lab-kit-cli` brings folio as a dependency, but a tool installer may not put the `folio` command on the path. If `folio version` fails, install it too: `uv tool install folio-kb` or `pipx install folio-kb`.
+3. Confirm this skill is in `.agents/skills/set-up-lab/`. If it is not, run `lab-kit init` once with no arguments: before a library exists, it installs only lab-kit's method, skills and roles.
+4. Look for a library: a `folio.yaml` in the repository root or in `docs/`. If one exists, reuse it: skip the location question, and in step 3 below run only the parts of folio's set-up skill it lacks.
+
+## Rules
+
+1. Ask at most three questions, all in one message, each with a default. Skip any the request already answered, so setup takes one round.
+2. Every document goes through folio's skills: set-up for the charter, write for the front door and the question. Never write a document by hand.
+3. The library's `root` is the repository root, so a result's paths and lab-kit's paths are read from the same place. `lab-kit init` refuses otherwise.
+4. Only the operator names frozen surfaces. Never freeze a path the operator did not name.
+5. Record the setup once, in one journal entry of kind `decision`, so the lab's first decision is on the record.
+6. Do not run an experiment, spend tokens on a live run, or push anything. Setup ends at a passing gate and a commit.
+
+## Steps
+
+1. **Ask the questions.** In one message, ask only what you do not know yet:
+   1. What is the lab's question, in one sentence? And its name? Default name: the repository's folder name.
+   2. Where should the library live? Default: `docs/`, with the lab's files at the repository root. The other choice is the repository root itself.
+   3. Which paths, if any, are frozen surfaces: inputs no one may edit in place, such as a vendored benchmark or a dataset? Default: none.
+2. **Make sure it is a repository.** If the folder is not inside a git work tree, run `git init`. folio installs its skills at the work tree's root, beside lab-kit's.
+3. **Create the library with folio.** Run `folio init <dir> --name "<name>" --purpose "<the question, in one sentence>"`. It writes the charter, the home page and folio's seven skills. Then read `.agents/skills/set-up/SKILL.md`, folio's set-up skill, and follow it for the rest of the charter, with these answers:
+   - the reader: whoever will read the lab's results, and what they already know;
+   - the pack: `lab`;
+   - `root`: `..` when the library is in `docs/`, nothing when it is the repository root.
+
+   Skip its journal entry and its commit: step 10 records and commits the whole setup once.
+4. **Install the lab.** Run `lab-kit init --library <dir>`. It writes `lab.yaml`, `ops/STATE.md`, `ops/missions/`, `experiments/` and `.lab/`, restores the method, the skills and the roles to this version, links `.claude/agents` to `.agents/agents`, and switches the lab pack on if set-up did not.
+5. **Add the front door.** Through folio's write skill, add a project document `lab`. Its description states the question. Its `reviewed` date is today, its state says the lab is set up and no mission is active, and its next step is the operator's first objective. Create a top-level map, `research`, through the write skill, keep it `live`, add the front door to it, and set it on the home page: `folio config set home.maps research`. Set `front: lab` in `lab.yaml`.
+6. **Add the question.** Through the write skill, add the question as `Q-1`, with rank 1, why it matters, what would settle it, and what would make the lab drop it. Add it to the `research` map.
+7. **Freeze the surfaces.** List each path the operator named under `frozen:` in `lab.yaml`, then run `lab-kit freeze <path>` for each. With none, leave `frozen: []`.
+8. **Write the operating guide.** Write `AGENTS.md` at the repository root, and make `CLAUDE.md` the one line `@AGENTS.md`. `AGENTS.md` states the question and its id, says to read `.lab/method/DISCIPLINE.md` and `.lab/method/LADDER.md` before any work, says where the library is, names the frozen surfaces, and names `lab-kit check` as the gate. Keep it under a screen: the method holds the rules.
+9. **Add CI.** Make CI run `lab-kit check` on every push and pull request. On GitHub, a workflow in `.github/workflows/` that installs `lab-kit` with pip and runs `lab-kit check` from the root.
+10. **Record and check.** Run `folio journal add --title "Lab set up" --description "<the question, in one line>" --body "<where the library is, the frozen surfaces>" --kind decision --about lab,Q-1` in the library. Run `folio index` there, then `lab-kit check` from the root. Fix what it names until it reports no error.
+11. **Commit.** Add exactly what setup wrote: the library (`folio.yaml`, `.gitignore`, `content/`, `assets/`, `.folio/`), `lab.yaml`, `ops/`, `experiments/.gitkeep`, `.lab/`, `.agents/`, `.claude/`, `AGENTS.md`, `CLAUDE.md`, the CI workflow, and any `.gitignore` setup changed. Never `git add -A`. Commit with one line saying the lab was set up.
+12. **Tell the operator, in three lines,** where the library is, the question's id and the frozen surfaces, and end with "Tell me the first mission and we will plan it together." Example: "The lab is set up: the library is in docs/, the question is Q-1, nothing is frozen. Tell me the first mission and we will plan it together."
+
+## Stops
+
+- `lab.yaml` already exists: the lab is set up. Use the plan-mission or run-mission skill.
+- `lab-kit version` or `folio version` fails and installing does not fix it. Say so and stop.
+- A library exists but its `root` is not the repository root, and changing it would break its documents' paths. Ask the operator.
+- The operator names a frozen surface that does not exist yet. Ask whether to freeze it once it does.
+
+## Done when
+
+- The library has the lab pack on and its `root` at the repository root.
+- `lab.yaml` names the library, `front: lab`, and the frozen surfaces, each recorded in `.lab/frozen.sha256`.
+- The front door `lab` and the question `Q-1` exist, on the `research` map.
+- `AGENTS.md`, `CLAUDE.md` and the CI workflow exist.
+- One `decision` journal entry records the setup.
+- `lab-kit check` reports no error, and the setup is committed.
+
+## Commands
+
+- `lab-kit version` and `folio version` confirm the engines.
+- `lab-kit init`: before a library exists, installs the method, the skills and the roles only.
+- `folio init <dir> --name ".." --purpose ".."` creates the library and installs folio's skills.
+- `folio config set <key> <value>` writes the charter: `reader`, `root`, `home.maps`.
+- `folio pack on lab` switches the lab pack on.
+- `lab-kit init --library <dir>` installs the lab.
+- `lab-kit freeze <path>` records a frozen surface's hashes.
+- `folio journal add --title ".." --description ".." --body ".." --kind decision --about lab,Q-1` records the setup.
+- `folio index` regenerates the indices.
+- `lab-kit check` runs the lab gate: `folio check`, then the lab checks.

lab_kit/checks/__init__.py

@@ -0,0 +1 @@
+"""The lab gate's checks."""

lab_kit/checks/base.py

@@ -0,0 +1,41 @@
+"""What every lab check shares: its name, its default severity, and how it reports a problem."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Callable
+
+from folio.checks.problems import Problem
+
+from ..errors import LabError
+from ..lab import Lab
+
+
+@dataclass
+class Reporter:
+    check: str
+    severity: str  # the check's default: "error" or "warning"
+    problems: list[Problem] = field(default_factory=list)
+
+    def add(self, path: str, message: str, severity: str | None = None) -> None:
+        self.problems.append(Problem(path, severity or self.severity, self.check, message))
+
+    def warn(self, path: str, message: str) -> None:
+        self.add(path, message, "warning")
+
+    def guard(self, path: str, action: Callable[[], None]) -> None:
+        """Run one item's part of a check; a file lab-kit cannot read becomes a problem on that file."""
+        try:
+            action()
+        except LabError as exc:
+            self.add(path, str(exc))
+
+
+CheckFn = Callable[[Lab, Reporter], None]
+
+
+@dataclass(frozen=True)
+class LabCheck:
+    name: str
+    severity: str
+    fn: CheckFn

lab_kit/checks/files.py

@@ -0,0 +1,312 @@
+"""Checks on the lab's own files: ids, frozen surfaces, selftests, spend, history, method and state."""
+
+from __future__ import annotations
+
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+from folio.checks.rules import measurements
+
+from .. import __version__, data, frozen, gitlog, ops, records
+from ..records import script_env
+from ..errors import LabError
+from ..lab import Lab
+from .base import Reporter
+
+STATE_WORDS = 300
+
+
+def _id_pattern(lab: Lab) -> re.Pattern[str] | None:
+    prefixes = sorted(lab.library().prefixes)
+    if not prefixes:
+        return None
+    return re.compile(r"(?<![\w-])(?:" + "|".join(re.escape(p) for p in prefixes) + r")-\d+\b")
+
+
+def ids_resolve(lab: Lab, rep: Reporter) -> None:
+    """Every id cited in a lab file resolves to a document in the library."""
+    lib = lab.library()
+    pattern = _id_pattern(lab)
+
+    def resolve(where: str, ident: object, what: str) -> None:
+        if not isinstance(ident, str) or not ident or not lib.find(ident):
+            rep.add(where, f"{what} `{ident}` names no document in the library")
+
+    def text_ids(where: str, text: str) -> None:
+        if pattern is None:
+            return
+        for ident in sorted(set(pattern.findall(text))):
+            resolve(where, ident, "cites")
+
+    if lab.settings.front:
+        resolve("lab.yaml", lab.settings.front, "`front`")
+    state = lab.root / ops.STATE
+    if state.is_file():
+        text_ids(ops.STATE, state.read_text(encoding="utf-8"))
+    for path in ops.mission_files(lab):
+        where = lab.rel(path)
+
+        def mission(path=path, where=where) -> None:
+            m = ops.read_mission(lab, path)
+            rests = m.meta.get("rests_on") or []
+            for ident in rests if isinstance(rests, list) else [rests]:
+                resolve(where, ident, "`rests_on`")
+            text_ids(where, m.text)
+        rep.guard(where, mission)
+    for slug in lab.experiment_slugs():
+        lock_where = f"experiments/{slug}/{records.LOCK}"
+
+        def lock(slug=slug, where=lock_where) -> None:
+            record = records.read_lock(lab, slug)
+            if record is not None:
+                resolve(where, record.protocol, "`protocol`")
+        rep.guard(lock_where, lock)
+        for run in records.runs(lab, slug):
+            resolve(run.path, run.data.get("protocol"), "`protocol`")
+        score_path = records.score_path(lab, slug)
+        if score_path.is_file():
+            where = lab.rel(score_path)
+
+            def score(score_path=score_path, where=where) -> None:
+                card = records.read_yaml(score_path, where)
+                resolve(where, card.get("protocol"), "`protocol`")
+                for name, entry in (card.get("record") or {}).items():
+                    if isinstance(entry, dict):
+                        if "protocol" in entry:
+                            resolve(where, entry["protocol"], f"`record.{name}.protocol`")
+                        if entry.get("supersedes"):
+                            resolve(where, entry["supersedes"], f"`record.{name}.supersedes`")
+                text_ids(where, score_path.read_text(encoding="utf-8"))
+            rep.guard(where, score)
+
+
+def frozen_intact(lab: Lab, rep: Reporter) -> None:
+    """Every frozen surface matches its hashes in .lab/frozen.sha256."""
+    try:
+        recorded = frozen.recorded(lab)
+    except LabError as exc:
+        rep.add(frozen.FROZEN, str(exc))
+        return
+    surfaces = [s.rstrip("/") for s in lab.settings.frozen]
+    for path in sorted(recorded):
+        if not any(frozen.belongs(path, s) for s in surfaces):
+            rep.add(frozen.FROZEN, f"records {path}, which no surface under `frozen:` in lab.yaml covers")
+    for surface in surfaces:
+        mine = {p: d for p, d in recorded.items() if frozen.belongs(p, surface)}
+        if not mine:
+            rep.add("lab.yaml", f"frozen surface `{surface}` has no recorded hash; the operator runs "
+                                f"`lab-kit freeze {surface}`")
+            continue
+        target = lab.root / surface
+        actual = records.tree_hashes(target, lab.root) if target.exists() else {}
+        for path in sorted(set(mine) | set(actual)):
+            if path not in actual:
+                rep.add(path, f"is gone from frozen surface `{surface}`")
+            elif path not in mine:
+                rep.add(path, f"was added to frozen surface `{surface}` after it was frozen")
+            elif mine[path] != actual[path]:
+                rep.add(path, f"changed: frozen surface `{surface}` is never edited in place")
+
+
+def _command(path: Path) -> list[str] | None:
+    if path.suffix == ".py":
+        return [sys.executable, str(path)]
+    if path.suffix == ".sh":
+        return ["bash", str(path)]
+    if path.stat().st_mode & 0o111:
+        return [str(path)]
+    return None
+
+
+def scripts(lab: Lab) -> list[Path]:
+    found: list[Path] = []
+    for slug in lab.experiment_slugs():
+        folder = lab.experiments / slug / "bin"
+        if folder.is_dir():
+            found.extend(p for p in sorted(folder.iterdir())
+                         if p.is_file() and not p.name.startswith(".") and p.suffix not in (".pyc", ".md"))
+    found.extend(lab.root / t for t in lab.settings.tools)
+    return found
+
+
+def selftest(lab: Lab, rep: Reporter) -> None:
+    """Every script in experiments/*/bin/ and under `tools:` passes --selftest."""
+    for path in scripts(lab):
+        where = lab.rel(path) if path.exists() else path.relative_to(lab.root).as_posix()
+        if not path.is_file():
+            rep.add("lab.yaml", f"tool `{where}` does not exist")
+            continue
+        command = _command(path)
+        if command is None:
+            rep.add(where, "is neither a .py or .sh script nor executable, so it cannot run --selftest")
+            continue
+        try:
+            done = subprocess.run([*command, "--selftest"], cwd=lab.root, capture_output=True, text=True,
+                                  timeout=lab.settings.rederive_timeout, check=False, env=script_env())
+        except subprocess.TimeoutExpired:
+            rep.add(where, f"--selftest ran past {lab.settings.rederive_timeout}s")
+            continue
+        if done.returncode != 0:
+            tail = (done.stderr.strip() or done.stdout.strip()).splitlines()[-1:] or [""]
+            rep.add(where, f"--selftest exited {done.returncode}: {tail[0]}".rstrip(": "))
+
+
+def mission_approved(lab: Lab, rep: Reporter) -> None:
+    """Missions are draft, active or concluded; an active or concluded one records who approved it and when;
+    nothing runs under a draft, and the state never names a draft as active."""
+    statuses: dict[str, str] = {}
+    for path in ops.mission_files(lab):
+        where = lab.rel(path)
+        try:
+            mission = ops.read_mission(lab, path)
+        except LabError as exc:
+            rep.add(where, str(exc))
+            continue
+        statuses[where] = mission.status
+        if mission.status not in ops.STATUSES:
+            rep.add(where, f"`status: {mission.status or '(none)'}` is not one of {', '.join(ops.STATUSES)}")
+            continue
+        if mission.status == "draft":
+            continue
+        if not mission.approved:
+            rep.add(where, f"is {mission.status} but records no approval: `approved:` names who approved "
+                           "the mission and when (the plan-mission skill records it)")
+        elif not ops.approved_names_a_date(mission.approved):
+            rep.add(where, f"`approved: {mission.approved}` must name who approved the mission and the date, "
+                           "as YYYY-MM-DD")
+    for run in records.runs(lab):
+        named = run.data.get("mission")
+        if isinstance(named, str) and statuses.get(named) == "draft":
+            rep.add(run.path, f"runs under {named}, a draft mission: nothing runs before the operator approves it")
+    state = lab.root / ops.STATE
+    if state.is_file():
+        active = ops.active_mission(state.read_text(encoding="utf-8"))
+        if active and statuses.get(active) == "draft":
+            rep.add(ops.STATE, f"names {active} as the active mission, but it is a draft awaiting approval")
+
+
+def spend_recorded(lab: Lab, rep: Reporter) -> None:
+    """A spending run names a mission with the operator's approval and a cap, and stays within the cap."""
+    spent_by_mission: dict[str, float] = {}
+    caps: dict[str, float] = {}
+    for run in records.runs(lab):
+        if run.data.get("spend") is not True:
+            continue
+        mission_path = run.data.get("mission")
+        if not isinstance(mission_path, str) or not mission_path:
+            rep.add(run.path, "spends but names no mission; a spending run needs one with the operator's approval")
+            continue
+        target = lab.root / mission_path
+        if not target.is_file():
+            rep.add(run.path, f"spends under mission {mission_path}, which does not exist")
+            continue
+        try:
+            mission = ops.read_mission(lab, target)
+        except LabError as exc:
+            rep.add(mission_path, str(exc))
+            continue
+        if not mission.approved:
+            rep.add(mission_path, f"records no operator approval (`approved:`), yet run {run.path} spends")
+        if mission.cap is None:
+            rep.add(mission_path, f"records no numeric `cap:`, yet run {run.path} spends")
+            continue
+        caps[mission_path] = mission.cap
+        spent = run.data.get("spent")
+        if run.ended:
+            if not isinstance(spent, (int, float)) or isinstance(spent, bool):
+                rep.add(run.path, "spent but recorded no spend: its command writes "
+                                  f"out/{records.SPEND} with `spent`")
+                continue
+        if isinstance(spent, (int, float)) and not isinstance(spent, bool):
+            spent_by_mission[mission_path] = spent_by_mission.get(mission_path, 0.0) + float(spent)
+    for mission_path, total in sorted(spent_by_mission.items()):
+        if total > caps[mission_path]:
+            rep.add(mission_path, f"its runs spent {total:g}, past its cap of {caps[mission_path]:g}")
+
+
+def _lab_files(lab: Lab) -> list[str]:
+    """The append-only lab files: missions, lock records and run records, now or ever committed."""
+    current = [lab.rel(p) for p in ops.mission_files(lab)]
+    for slug in lab.experiment_slugs():
+        if records.lock_path(lab, slug).is_file():
+            current.append(f"experiments/{slug}/{records.LOCK}")
+    current.extend(run.path for run in records.runs(lab))
+    pattern = re.compile(r"^(ops/missions/[^/]+\.md|experiments/[^/]+/lock\.json|"
+                         r"experiments/[^/]+/runs/[^/]+/run\.json)$")
+    ever = [p for folder in ("ops/missions", "experiments") for p in gitlog.tracked(lab.root, folder)
+            if pattern.match(p)]
+    return sorted(set(current) | set(ever))
+
+
+def append_only(lab: Lab, rep: Reporter) -> None:
+    """Committed mission log entries, lock records and run records are unchanged in later commits."""
+    if not gitlog.in_git(lab.root):
+        rep.warn("lab.yaml", "the lab is not in a git repository, so lab-append-only could not be checked")
+        return
+    for path in _lab_files(lab):
+        history = gitlog.versions(lab.root, path)
+        committed = [(sha, text) for sha, text in history if sha != gitlog.WORKING]
+        if not committed:
+            continue
+        first_sha, first = committed[0]
+        if path.startswith("ops/missions/"):
+            previous = ops.log_of(first or "")
+            for sha, text in history[1:]:
+                label = "the working tree" if sha == gitlog.WORKING else f"commit {sha[:7]}"
+                if text is None:
+                    rep.add(path, f"was deleted in {label}; a mission file is never deleted")
+                    break
+                log = ops.log_of(text)
+                if previous is not None and (log is None or not log.startswith(previous.rstrip())):
+                    rep.add(path, f"its log was changed in {label}: entries are only ever added at the bottom")
+                    break
+                previous = log
+            continue
+        for sha, text in history[1:]:
+            label = "the working tree" if sha == gitlog.WORKING else f"commit {sha[:7]}"
+            if text != first:
+                what = "deleted" if text is None else "changed"
+                rep.add(path, f"was {what} in {label} after its first commit {first_sha[:7]}; "
+                              "lock and run records never change once committed")
+                break
+
+
+def method_current(lab: Lab, rep: Reporter) -> None:
+    """The method, the five skills and the four roles match the lab-kit version in lab.yaml."""
+    if lab.settings.version != __version__:
+        rep.add("lab.yaml", f"`lab-kit: {lab.settings.version}`, but this is lab-kit {__version__}; "
+                            "install that version, or run `lab-kit init` to move the lab to this one")
+        return
+    for src, dest in data.shipped_files():
+        target = lab.root / dest
+        if not target.is_file():
+            rep.add(dest, f"is missing; lab-kit {__version__} ships it (`lab-kit init` restores it)")
+        elif target.read_bytes() != src.read_bytes():
+            rep.add(dest, f"differs from what lab-kit {__version__} ships; it is never hand-edited "
+                          "(`lab-kit init` restores it)")
+
+
+def state_pointer(lab: Lab, rep: Reporter) -> None:
+    """ops/STATE.md names an active mission or none, stays short, and cites ids for its numbers."""
+    path = lab.root / ops.STATE
+    if not path.is_file():
+        rep.add(ops.STATE, "does not exist; it says what is true now and points to the active mission")
+        return
+    text = path.read_text(encoding="utf-8")
+    active = ops.active_mission(text)
+    if active is None:
+        rep.add(ops.STATE, "has no `Active mission:` line naming a mission file or `none`")
+    elif active.lower() != "none" and not (lab.root / active).is_file():
+        rep.add(ops.STATE, f"names active mission `{active}`, which does not exist")
+    elif active.lower() != "none" and not active.startswith(ops.MISSIONS + "/"):
+        rep.add(ops.STATE, f"names active mission `{active}`, which is not under {ops.MISSIONS}/")
+    words = len(text.split())
+    if words > STATE_WORDS:
+        rep.add(ops.STATE, f"has {words} words, past {STATE_WORDS}: it is a pointer, never a record")
+    pattern = _id_pattern(lab)
+    for number, line in enumerate(text.splitlines(), 1):
+        found = measurements(line)
+        if found and (pattern is None or not pattern.search(line)):
+            rep.add(ops.STATE, f"line {number}: `{found[0]}` cites no id; a number in the state cites its result")

lab_kit/checks/locks.py

@@ -0,0 +1,134 @@
+"""Checks on locks and runs: the lock is recorded and intact, and every run follows it."""
+
+from __future__ import annotations
+
+from .. import protocol as protocol_mod
+from .. import records
+from ..errors import LabError
+from ..lab import Lab
+from .base import Reporter
+
+LOCKED_STATES = ("locked", "abandoned")
+
+
+def lock_recorded(lab: Lab, rep: Reporter) -> None:
+    """Every locked protocol has a lock.json naming it; every lock.json names a locked or abandoned protocol."""
+    lib = lab.library()
+    for doc in lib.documents:
+        if not doc.is_a("protocol") or doc.status != "locked":
+            continue
+        where = lab.doc_path(doc.main.path) if doc.main else doc.key
+
+        def one(doc=doc, where=where) -> None:
+            lock = records.read_lock(lab, doc.id)
+            if lock is None:
+                rep.add(where, f"is locked but experiments/{doc.id}/{records.LOCK} does not exist; "
+                               f"run `lab-kit lock {doc.id}`")
+            elif lock.protocol.casefold() != doc.id.casefold():
+                rep.add(lock.path, f"names protocol `{lock.protocol}`, not `{doc.id}`, the experiment it sits in")
+        rep.guard(where, one)
+    for slug in lab.experiment_slugs():
+        path = f"experiments/{slug}/{records.LOCK}"
+
+        def check_lock(slug=slug, path=path) -> None:
+            lock = records.read_lock(lab, slug)
+            if lock is None:
+                return
+            proto = protocol_mod.find(lab, lock.protocol)
+            if proto is None:
+                rep.add(path, f"names protocol `{lock.protocol}`, which is not in the library")
+            elif proto.status not in LOCKED_STATES:
+                rep.add(path, f"names protocol `{lock.protocol}`, whose status is `{proto.status}`, "
+                              "not locked or abandoned")
+        rep.guard(path, check_lock)
+
+
+def lock_intact(lab: Lab, rep: Reporter) -> None:
+    """A locked protocol, its status set aside, hashes to the value in its lock.json."""
+    for slug in lab.experiment_slugs():
+        path = f"experiments/{slug}/{records.LOCK}"
+
+        def one(slug=slug, path=path) -> None:
+            lock = records.read_lock(lab, slug)
+            if lock is None:
+                return
+            proto = protocol_mod.find(lab, lock.protocol)
+            if proto is None:
+                return  # lab-lock-recorded names it
+            actual = protocol_mod.hash_text(proto.text())
+            if actual != lock.sha256:
+                rep.add(proto.path, f"changed since its lock: its hash, status set aside, is {actual[:12]}, "
+                                    f"and {path} records {lock.sha256[:12]}")
+        rep.guard(path, one)
+
+
+def run_after_lock(lab: Lab, rep: Reporter) -> None:
+    """Every run belongs to a locked experiment, records the lock's hash, and started after the lock."""
+    for where in records.runs_without_record(lab):
+        rep.add(where, f"is a run folder with no {records.RUN}")
+    for slug in lab.experiment_slugs():
+        def per_experiment(slug=slug) -> None:
+            lock = records.read_lock(lab, slug)
+            for run in records.runs(lab, slug):
+                data = run.data
+                if lock is None:
+                    rep.add(run.path, f"belongs to experiment `{slug}`, which has no {records.LOCK}: "
+                                      "no run starts before the lock")
+                    continue
+                if data.get("protocol") != lock.protocol:
+                    rep.add(run.path, f"names protocol `{data.get('protocol')}`, not `{lock.protocol}`")
+                if data.get("lock") != lock.sha256:
+                    rep.add(run.path, "records a lock hash that is not the one in "
+                                      f"experiments/{slug}/{records.LOCK}")
+                try:
+                    started = records.parse_moment(data.get("started"), run.path + ": `started`")
+                except LabError as exc:
+                    rep.add(run.path, str(exc))
+                    continue
+                if started < lock.locked:
+                    rep.add(run.path, f"started at {data.get('started')}, before the lock at "
+                                      f"{lock.data.get('locked')}")
+        rep.guard(f"experiments/{slug}", per_experiment)
+
+
+def roster_frozen(lab: Lab, rep: Reporter) -> None:
+    """Every run's configuration equals its locked protocol's pinned configuration."""
+    for slug in lab.experiment_slugs():
+        def per_experiment(slug=slug) -> None:
+            lock = records.read_lock(lab, slug)
+            proto = protocol_mod.find(lab, lock.protocol if lock else slug)
+            if proto is None:
+                return
+            pinned = protocol_mod.pinned_config(proto.text(), proto.path)
+            for run in records.runs(lab, slug):
+                if run.data.get("config") != pinned:
+                    rep.add(run.path, f"its `config` differs from the yaml block under `## Pinned configuration` "
+                                      f"in {proto.path}: a run uses exactly the pinned configuration")
+        rep.guard(f"experiments/{slug}", per_experiment)
+
+
+def evidence_sealed(lab: Lab, rep: Reporter) -> None:
+    """Every finished run's out/ matches its manifest; a run with no end and no live process is orphaned."""
+    for run in records.runs(lab):
+        def one(run=run) -> None:
+            state = run.state
+            if state == "orphaned":
+                rep.warn(run.path, "has no end and no live process: it is orphaned; close it honestly or "
+                                   "start a fresh run with the same configuration")
+                return
+            if state == "running":
+                return
+            manifest_path = run.path.rsplit("/", 1)[0] + "/" + records.MANIFEST
+            if not run.manifest.is_file():
+                rep.add(manifest_path, "is missing for a run that ended")
+                return
+            recorded = records.parse_hashes(run.manifest.read_text(encoding="utf-8"), manifest_path)
+            actual = records.tree_hashes(run.out, run.out)
+            for name in sorted(set(recorded) | set(actual)):
+                if name not in actual:
+                    rep.add(manifest_path, f"lists out/{name}, which is gone")
+                elif name not in recorded:
+                    rep.add(manifest_path, f"does not list out/{name}, added after the run ended")
+                elif recorded[name] != actual[name]:
+                    rep.add(manifest_path, f"out/{name} changed after the run ended")
+        rep.guard(run.path, one)