results-cli 0.5.2__tar.gz → 0.5.4__tar.gz

{results_cli-0.5.2 → results_cli-0.5.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: results-cli
-Version: 0.5.2
+Version: 0.5.4
 Summary: Seal a run, record what it produced, and verify the chain
 Project-URL: Changelog, https://github.com/elliottower/reproducible-science/blob/main/CHANGELOG.md
 Project-URL: Homepage, https://github.com/elliottower/reproducible-science
@@ -24,11 +24,11 @@ Description-Content-Type: text/markdown
 [![pypi](https://img.shields.io/pypi/v/results-cli)](https://pypi.org/project/results-cli/)
 [![python](https://img.shields.io/pypi/pyversions/results-cli)](https://pypi.org/project/results-cli/)
 [![license](https://img.shields.io/pypi/l/results-cli)](https://github.com/elliottower/reproducible-science/blob/main/LICENSE)
-[![docs](https://img.shields.io/badge/docs-live-blue)](https://elliottower.github.io/reproducible-science/)
+[![docs](https://img.shields.io/badge/docs-live-blue)](https://reproducible.science/)
 
 Seal inputs, record outputs, and bind a paper's claims to the runs behind them.
 
-Part of [reproducible-science](https://github.com/elliottower/reproducible-science) alongside `repro`, `citations` and `prereg` — see the [documentation](https://elliottower.github.io/reproducible-science/tools/results/).
+Part of [reproducible-science](https://github.com/elliottower/reproducible-science) alongside `repro`, `citations` and `prereg` — see the [documentation](https://reproducible.science/tools/results/).
 
 ## Install
 
@@ -76,6 +76,8 @@ all checks passed.
 | `results access <note>` | Record a data-access event |
 | `results run <file>...` | Record outputs after a run |
 | `results claim <text>` | Bind a manuscript claim to a run |
+| `results review <file>...` | Record that someone examined these files, and what they found |
+| `results annotate <file>...` | Link these files to something outside the ledger; linked, not checked |
 | `results verify` | Check the ledger chain and every hash it names |
 | `results timestamp` | Date the ledger's head outside the repository, and check earlier dates |
 
@@ -91,6 +93,74 @@ manuscript  →  claim  →  run  →  output file  →  sha256
 
 `results verify --files` walks the whole thing and tells you what moved.
 
+## Reviews
+
+Some checks are done by reading: a person, or a model, compares a table with the file behind it.
+`results review` records that the reading happened, and of which bytes.
+
+```bash
+results review results/table3.csv paper/table3.pdf \
+    --scope "every cell of Table 3 against the regenerated file" \
+    --procedure "read the rendered page beside the csv" \
+    --verdict partial --reviewer "A. Reader" --model claude-opus-5-5 \
+    --note "30 cells agree at the certified revision; 4 differ"
+```
+
+Each file is recorded by its sha256, so the review is of those bytes and no others.
+`results verify --files` lists every review and marks a subject `STALE` once the file has
+changed, without failing: a reviewed file that then changes is the ordinary result of acting on
+a review. `--scope`, `--procedure`, `--verdict` and `--reviewer` are required, and the verdict
+is one of `pass`, `fail`, `partial` and `uncertain`.
+
+The record is self-reported and unsigned. The ledger shows that the review was stated and when;
+nothing checks who stated it. `--model` is the identifier the caller gives. Inside Claude Code
+the command also records the harness, its version and the session identifier from the
+environment, which does not say which model ran, so a model reviewer passes `--model`.
+
+## Annotations
+
+Some of what a paper rests on lives outside the repository: a Weights & Biases artifact, a
+deposited dataset, an OSF node, a row in another group's table. `results annotate` records that
+a file here corresponds to that outside thing.
+
+```bash
+results annotate checkpoints/model.pt \
+    --kind wandb:artifact --ref "lab/study/model:v3" \
+    --digest md5:0123456789abcdef0123456789abcdef \
+    --by "A. Reader" --note "the checkpoint behind Table 2"
+
+results annotate results/table2.csv \
+    --kind doi:dataset --ref 10.5281/zenodo.1234567 --location "Table 2"
+```
+
+An annotation is a statement of a link and not a check: the tool fetches nothing and compares
+nothing, and the event carries the status `linked, not checked`. Each file is recorded by its
+sha256, so the link is stated of those bytes.
+
+`--kind` and `--ref` are required. `--kind` is `namespace:type` in lowercase, such as
+`wandb:artifact`, `doi:dataset`, `osf:node`, `url:page`, `zenodo:record` or `hf:dataset`; any
+kind of that shape is accepted, and a malformed kind writes nothing. `--ref` is the artifact
+path, DOI, URL or identifier. `--digest` records the digest the outside system reports, as
+`algorithm:hex`. `--by` names who states the link, `--note` says anything else, and `--location`
+names where in the manuscript the linked number or sentence appears. No session information is
+recorded.
+
+```text
+annotations (linked, not checked):
+  2026-10-11T16:54:21  wandb:artifact  lab/study/model:v3  [A. Reader]
+    digest as stated: md5:0123456789abcdef0123456789abcdef
+    STALE        checkpoints/model.pt
+      annotated 1b465fa6b6bcbc06…, now 7e4fa2eb8c7ac089…
+  2026-10-11T16:54:21  doi:dataset  10.5281/zenodo.1234567
+    appears in: Table 2
+    current      results/table2.csv
+  1 annotated file(s) are no longer the bytes the link was stated of.
+```
+
+`results verify` lists every annotation. With `--files` each file is `current`, `STALE` once it
+has changed, or `MISSING`; without `--files` it is `not checked`. None of the three changes the
+exit code.
+
 ## Data-access levels
 
 The access timeline is what makes the confirmatory/exploratory distinction verifiable.

{results_cli-0.5.2 → results_cli-0.5.4}/README.md

@@ -3,11 +3,11 @@
 [![pypi](https://img.shields.io/pypi/v/results-cli)](https://pypi.org/project/results-cli/)
 [![python](https://img.shields.io/pypi/pyversions/results-cli)](https://pypi.org/project/results-cli/)
 [![license](https://img.shields.io/pypi/l/results-cli)](https://github.com/elliottower/reproducible-science/blob/main/LICENSE)
-[![docs](https://img.shields.io/badge/docs-live-blue)](https://elliottower.github.io/reproducible-science/)
+[![docs](https://img.shields.io/badge/docs-live-blue)](https://reproducible.science/)
 
 Seal inputs, record outputs, and bind a paper's claims to the runs behind them.
 
-Part of [reproducible-science](https://github.com/elliottower/reproducible-science) alongside `repro`, `citations` and `prereg` — see the [documentation](https://elliottower.github.io/reproducible-science/tools/results/).
+Part of [reproducible-science](https://github.com/elliottower/reproducible-science) alongside `repro`, `citations` and `prereg` — see the [documentation](https://reproducible.science/tools/results/).
 
 ## Install
 
@@ -55,6 +55,8 @@ all checks passed.
 | `results access <note>` | Record a data-access event |
 | `results run <file>...` | Record outputs after a run |
 | `results claim <text>` | Bind a manuscript claim to a run |
+| `results review <file>...` | Record that someone examined these files, and what they found |
+| `results annotate <file>...` | Link these files to something outside the ledger; linked, not checked |
 | `results verify` | Check the ledger chain and every hash it names |
 | `results timestamp` | Date the ledger's head outside the repository, and check earlier dates |
 
@@ -70,6 +72,74 @@ manuscript  →  claim  →  run  →  output file  →  sha256
 
 `results verify --files` walks the whole thing and tells you what moved.
 
+## Reviews
+
+Some checks are done by reading: a person, or a model, compares a table with the file behind it.
+`results review` records that the reading happened, and of which bytes.
+
+```bash
+results review results/table3.csv paper/table3.pdf \
+    --scope "every cell of Table 3 against the regenerated file" \
+    --procedure "read the rendered page beside the csv" \
+    --verdict partial --reviewer "A. Reader" --model claude-opus-5-5 \
+    --note "30 cells agree at the certified revision; 4 differ"
+```
+
+Each file is recorded by its sha256, so the review is of those bytes and no others.
+`results verify --files` lists every review and marks a subject `STALE` once the file has
+changed, without failing: a reviewed file that then changes is the ordinary result of acting on
+a review. `--scope`, `--procedure`, `--verdict` and `--reviewer` are required, and the verdict
+is one of `pass`, `fail`, `partial` and `uncertain`.
+
+The record is self-reported and unsigned. The ledger shows that the review was stated and when;
+nothing checks who stated it. `--model` is the identifier the caller gives. Inside Claude Code
+the command also records the harness, its version and the session identifier from the
+environment, which does not say which model ran, so a model reviewer passes `--model`.
+
+## Annotations
+
+Some of what a paper rests on lives outside the repository: a Weights & Biases artifact, a
+deposited dataset, an OSF node, a row in another group's table. `results annotate` records that
+a file here corresponds to that outside thing.
+
+```bash
+results annotate checkpoints/model.pt \
+    --kind wandb:artifact --ref "lab/study/model:v3" \
+    --digest md5:0123456789abcdef0123456789abcdef \
+    --by "A. Reader" --note "the checkpoint behind Table 2"
+
+results annotate results/table2.csv \
+    --kind doi:dataset --ref 10.5281/zenodo.1234567 --location "Table 2"
+```
+
+An annotation is a statement of a link and not a check: the tool fetches nothing and compares
+nothing, and the event carries the status `linked, not checked`. Each file is recorded by its
+sha256, so the link is stated of those bytes.
+
+`--kind` and `--ref` are required. `--kind` is `namespace:type` in lowercase, such as
+`wandb:artifact`, `doi:dataset`, `osf:node`, `url:page`, `zenodo:record` or `hf:dataset`; any
+kind of that shape is accepted, and a malformed kind writes nothing. `--ref` is the artifact
+path, DOI, URL or identifier. `--digest` records the digest the outside system reports, as
+`algorithm:hex`. `--by` names who states the link, `--note` says anything else, and `--location`
+names where in the manuscript the linked number or sentence appears. No session information is
+recorded.
+
+```text
+annotations (linked, not checked):
+  2026-10-11T16:54:21  wandb:artifact  lab/study/model:v3  [A. Reader]
+    digest as stated: md5:0123456789abcdef0123456789abcdef
+    STALE        checkpoints/model.pt
+      annotated 1b465fa6b6bcbc06…, now 7e4fa2eb8c7ac089…
+  2026-10-11T16:54:21  doi:dataset  10.5281/zenodo.1234567
+    appears in: Table 2
+    current      results/table2.csv
+  1 annotated file(s) are no longer the bytes the link was stated of.
+```
+
+`results verify` lists every annotation. With `--files` each file is `current`, `STALE` once it
+has changed, or `MISSING`; without `--files` it is `not checked`. None of the three changes the
+exit code.
+
 ## Data-access levels
 
 The access timeline is what makes the confirmatory/exploratory distinction verifiable.

{results_cli-0.5.2 → results_cli-0.5.4}/plugin/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "results",
   "description": "Seal a run's inputs, record what it produced, bind every number in a manuscript to the run behind it as it is written, and verify the chain",
-  "version": "0.5.2",
+  "version": "0.5.4",
   "author": {
     "name": "Elliot Tower",
     "email": "elliot@elliottower.ai"

{results_cli-0.5.2 → results_cli-0.5.4}/plugin/skills/results/SKILL.md

@@ -25,6 +25,8 @@ results seal <file>... [--role input]   # hash inputs before a run
 results access <note> [--level ...]     # record a data-access event
 results run <file>... --run-id <id>     # record outputs after a run
 results claim <text> --run-id <id>      # bind a manuscript claim to a run
+results review <file>... --scope ... --procedure ... --verdict ... --reviewer ...  # record a reading
+results annotate <file>... --kind <namespace:type> --ref <reference>   # link to an outside artifact; not checked
 results verify [--files]                # check the chain and every hash it names
 results coverage <manuscript>           # which of a paper's numbers are bound
 ```
@@ -104,6 +106,28 @@ With `--files`, every sealed input and recorded output is re-hashed against its
 **`CHAIN BROKEN` means the ledger was tampered with.** It does not mean the results are wrong —
 it means the evidence that they are right was damaged. Restore from git history.
 
+## Reviews
+
+When you check something by reading it (a table against its file, a mismatch list against the
+paper), record it: `results review <file>... --scope "<what was checked>" --procedure "<how>"
+--verdict pass|fail|partial|uncertain --reviewer "<who answers for it>"`. Pass `--model` with
+your own model identifier when you did the reading; the harness, its version and the session
+are read from the environment. Each file is recorded by digest, and `results verify --files`
+marks the review `STALE` once a file changes. The record is self-reported and unsigned, so
+describe it as a recorded review and never as verified.
+
+## Annotations
+
+When a file corresponds to something outside the repository (a W&B artifact, a dataset DOI, an
+OSF node, a URL), record the link: `results annotate <file>... --kind <namespace:type> --ref
+"<artifact path, DOI, URL or id>"`, with `--kind` such as `wandb:artifact`, `doi:dataset`,
+`osf:node` or `url:page`. Optional: `--digest <algorithm:hex>` for the digest the outside system
+reports, `--by`, `--note`, and `--location "<where in the manuscript>"`. An annotation is a
+statement of a link and not a check: the tool fetches nothing, and the event's status is
+`linked, not checked`. `results verify --files` marks the file `STALE` once it changes and never
+fails on it. Describe an annotated file as linked and never as verified, and do not use an
+annotation in place of `results run` for an output the project produced.
+
 ## Claims
 
 A claim names a run. A run names its outputs. The outputs were hashed when they were recorded.

{results_cli-0.5.2 → results_cli-0.5.4}/pyproject.toml

@@ -4,7 +4,7 @@ requires = [ "hatchling" ]
 
 [project]
 name = "results-cli"
-version = "0.5.2"
+version = "0.5.4"
 description = "Seal a run, record what it produced, and verify the chain"
 readme = "README.md"
 keywords = [ "open science", "provenance", "reproducibility", "results" ]

{results_cli-0.5.2 → results_cli-0.5.4}/src/results/audit.py

@@ -19,6 +19,7 @@ from provenance_core import anchor, sha256_of_tree, try_run
 
 from results import ledger, manuscript, stamps
 from results.paths import ledger_path, require_root
+from results.record import LINKED
 from results.timeline import first_outcomes_seen, first_run_timestamp, precedes
 
 
@@ -44,6 +45,75 @@ def in_history(ledger: pathlib.Path) -> str | None:
     return None
 
 
+def reviews(events: list[dict], base: pathlib.Path, check_files: bool) -> list[str]:
+    """One block per review, and with `check_files` whether each subject is still those bytes.
+
+    A review is of a digest. Where the file has changed since, the review says nothing about
+    what is there now, and printing the verdict without that would carry it forward. Reported
+    and never counted as drift: a reviewed file that then changes is the ordinary result of
+    acting on a review.
+    """
+    lines: list[str] = []
+    for e in events:
+        if e.get("event") != "review":
+            continue
+        ran_in = e.get("session") or {}
+        harness = " ".join(v for k in ("harness", "harness_version") if (v := ran_in.get(k)))
+        named = ", ".join(v for v in (ran_in.get("model"), harness) if v)
+        lines.append(
+            f"  {e['timestamp'][:19]}  {e.get('verdict', '?'):<10}  {e.get('reviewer', '?')}"
+            + (f"  [{named}]" if named else "")
+        )
+        lines.append(f"    scope: {e.get('scope', '')}")
+        lines += subjects(e, base, check_files, "reviewed")
+    return lines
+
+
+def annotation_lines(events: list[dict], base: pathlib.Path, check_files: bool) -> list[str]:
+    """One block per annotation: what it links to, and with `check_files` whether each subject
+    is still the bytes the link was stated of.
+
+    An annotation is a statement that these bytes correspond to something outside the ledger.
+    Nothing here reads the target, so the most this can say is whether the local side of the
+    link still exists as recorded. Reported and never counted as drift, as a review is.
+    """
+    lines: list[str] = []
+    for e in events:
+        if e.get("event") != "annotation":
+            continue
+        target = e.get("target") or {}
+        lines.append(
+            f"  {e['timestamp'][:19]}  {target.get('kind', '?')}  {target.get('ref', '?')}"
+            + (f"  [{by}]" if (by := e.get("by")) else "")
+        )
+        if stated := target.get("digest"):
+            lines.append(f"    digest as stated: {stated}")
+        if location := e.get("location"):
+            lines.append(f"    appears in: {location}")
+        lines += subjects(e, base, check_files, "annotated")
+    return lines
+
+
+def subjects(event: dict, base: pathlib.Path, check_files: bool, was: str) -> list[str]:
+    """Each subject of a review or an annotation, and whether it is still the recorded bytes."""
+    lines: list[str] = []
+    for subject in event.get("subjects", []):
+        path, recorded = subject["path"], subject["sha256"]
+        p = base / path
+        if not check_files:
+            lines.append(f"    not checked  {path}")
+        elif not p.exists():
+            lines.append(f"    MISSING      {path}")
+        else:
+            now = sha256_of_tree(p)[0] if p.is_dir() else ledger.sha256_of_file(p)
+            if now == recorded:
+                lines.append(f"    current      {path}")
+            else:
+                lines.append(f"    STALE        {path}")
+                lines.append(f"      {was} {recorded[:16]}…, now {now[:16]}…")
+    return lines
+
+
 def coverage(manuscript_path: str, limit: int, strict: bool) -> int:
     root = require_root()
     events = ledger.read_ledger(ledger_path(root))
@@ -292,6 +362,22 @@ def verify(check_files: bool) -> int:
         for e in access_events:
             print(f"  {e['timestamp'][:19]}  {e['level']:<20}  {e.get('note', '')}")
 
+    if review_lines := reviews(events, root.parent, check_files):
+        print("\nreviews (self-reported, unsigned):")
+        for line in review_lines:
+            print(line)
+        stale = sum(1 for line in review_lines if line.lstrip().startswith(("STALE", "MISSING")))
+        if stale:
+            print(f"  {stale} reviewed file(s) are no longer the bytes that were reviewed.")
+
+    if linked := annotation_lines(events, root.parent, check_files):
+        print(f"\nannotations ({LINKED}):")
+        for line in linked:
+            print(line)
+        stale = sum(1 for line in linked if line.lstrip().startswith(("STALE", "MISSING")))
+        if stale:
+            print(f"  {stale} annotated file(s) are no longer the bytes the link was stated of.")
+
     claims = [e for e in events if e.get("event") == "claim"]
     unlinked = []
     contested = []

{results_cli-0.5.2 → results_cli-0.5.4}/src/results/cli.py

@@ -5,6 +5,8 @@ results seal <file>...    hash inputs before a run (prereg, script, data)
 results access <note>     record a data-access event (what you looked at, when)
 results run <file>...     record outputs after a run completes
 results claim <text>      bind a manuscript claim to a run's output
+results review <file>...  record that someone examined these files, and what they found
+results annotate <file>...  link these files to something outside the ledger; linked, not checked
 results coverage <paper>  how many of a manuscript's numbers are bound to a run
 results verify            check the ledger chain and every hash it names
 results reanchor          record the ledger's current length as authoritative
@@ -31,7 +33,7 @@ from results.paths import find_root as find_root
 from results.paths import ledger_path as ledger_path
 from results.paths import record_path as record_path
 from results.paths import require_root as require_root
-from results.record import ACCESS_LEVELS
+from results.record import ACCESS_LEVELS, VERDICTS
 from results.timeline import first_outcomes_seen as first_outcomes_seen
 from results.timeline import first_run_timestamp as first_run_timestamp
 from results.timeline import freeze_timestamp as freeze_timestamp
@@ -65,6 +67,14 @@ def cmd_claim(a) -> int:
     )
 
 
+def cmd_review(a) -> int:
+    return record.review(a.files, a.scope, a.procedure, a.verdict, a.reviewer, a.model, a.note)
+
+
+def cmd_annotate(a) -> int:
+    return record.annotate(a.files, a.kind, a.ref, a.digest, a.by, a.note, a.location)
+
+
 def cmd_coverage(a) -> int:
     return audit.coverage(a.manuscript, a.limit, a.strict)
 
@@ -147,6 +157,45 @@ def _main(argv: list[str] | None = None) -> int:
     cl.add_argument("--location", help="where in the manuscript: Table 2, Section 4.1, etc.")
     cl.set_defaults(fn=cmd_claim)
 
+    rv = sub.add_parser(
+        "review", help="record that someone examined these files, and what they found"
+    )
+    rv.add_argument("files", nargs="+", help="what was examined; each is recorded by digest")
+    rv.add_argument("--scope", required=True, help="what was checked about them")
+    rv.add_argument("--procedure", required=True, help="how it was checked")
+    rv.add_argument("--verdict", required=True, help=f"one of: {', '.join(VERDICTS)}")
+    rv.add_argument("--reviewer", required=True, help="who answers for the review")
+    rv.add_argument(
+        "--model",
+        help="the model that did the reading, where one did: its identifier, as reported",
+    )
+    rv.add_argument("--note", help="anything the verdict needs said beside it")
+    rv.set_defaults(fn=cmd_review)
+
+    an = sub.add_parser(
+        "annotate", help="link these files to something outside the ledger; linked, not checked"
+    )
+    an.add_argument(
+        "files", nargs="+", help="what the link is stated of; each is recorded by digest"
+    )
+    an.add_argument(
+        "--kind",
+        required=True,
+        help="what the outside thing is, as namespace:type: wandb:artifact, doi:dataset, "
+        "osf:node, url:page",
+    )
+    an.add_argument(
+        "--ref", required=True, help="where it is: an artifact path, a DOI, a URL, an id"
+    )
+    an.add_argument(
+        "--digest",
+        help="the digest the outside system states, as algorithm:hex; recorded, never compared",
+    )
+    an.add_argument("--by", help="who states the link")
+    an.add_argument("--note", help="anything the link needs said beside it")
+    an.add_argument("--location", help="where in the manuscript: Table 2, Section 4.1, etc.")
+    an.set_defaults(fn=cmd_annotate)
+
     v = sub.add_parser("verify", help="check the ledger and every hash it names")
     v.add_argument(
         "--files",

{results_cli-0.5.2 → results_cli-0.5.4}/src/results/record.py

@@ -1,16 +1,20 @@
 """The commands that put an event in the ledger.
 
-`init`, `seal`, `access`, `run` and `claim` share a shape: find the governing `.results/`,
-build one event, append it, and report what was written. What separates them is the refusal
-each makes first -- a path that is not a file, an access level outside the four, a run id
-already in use, a confirmatory claim resting on a run recorded after the outcomes were seen.
+`init`, `seal`, `access`, `run`, `claim`, `review` and `annotate` share a shape: find the
+governing `.results/`, build one event, append it, and report what was written. What separates
+them is the refusal each makes first -- a path that is not a file, an access level outside the
+four, a run id already in use, a confirmatory claim resting on a run recorded after the outcomes
+were seen, a review with a verdict outside the four, an annotation whose kind is not
+`namespace:type`.
 
 Each returns the process's exit code. `cli.py` unpacks the argparse namespace and calls in.
 """
 
 from __future__ import annotations
 
+import os
 import pathlib
+import re
 
 from provenance_core import sha256_of_tree
 
@@ -24,6 +28,22 @@ ACCESS_LEVELS = [
     "outcomes seen",
 ]
 
+VERDICTS = ["pass", "fail", "partial", "uncertain"]
+
+#: What an annotation's target is, as `namespace:type`: `wandb:artifact`, `doi:dataset`,
+#: `osf:node`, `url:page`. Open-ended, because the outside systems a project links to are not
+#: this tool's to enumerate; only the shape is fixed, so that a reader can group by namespace.
+KIND = re.compile(r"[a-z0-9._-]+:[a-z0-9._-]+")
+
+#: A digest as the outside system states it: `algorithm:hex`.
+DIGEST = re.compile(r"([a-z0-9][a-z0-9_-]*):([0-9a-fA-F]+)")
+
+#: Hex digits in the digests whose length is fixed. Any other algorithm is taken at any length.
+DIGEST_LENGTHS = {"md5": 32, "sha1": 40, "sha256": 64, "sha512": 128}
+
+#: Written into every annotation and printed wherever one is shown.
+LINKED = "linked, not checked"
+
 
 #: Written into `.results/` so the ledger is tracked without anyone editing the project's own
 #: ignore file. The ledger and its anchor are the record a claim appeals to, and one that exists
@@ -250,3 +270,144 @@ def claim(
     elif retrospective:
         print(f"  recorded after outcomes were seen at {retrospective[:19]}; verify will report it")
     return 0
+
+
+def session(model: str | None) -> dict[str, str]:
+    """Where the command ran and which model the caller names, for a review's record.
+
+    Read off the environment and the `--model` flag, and checked against nothing. Claude Code
+    exports its own version and session but not the model, so the model is whatever the caller
+    passes. A person who runs the command from inside a session is recorded the same way as a
+    model that does, which is why the reviewer is a separate, required field.
+    """
+    found: dict[str, str] = {}
+    if model:
+        found["model"] = model
+    if os.environ.get("CLAUDECODE") == "1":
+        found["harness"] = "Claude Code"
+        version = pathlib.PurePath(os.environ.get("CLAUDE_CODE_EXECPATH", "")).name
+        if re.fullmatch(r"\d+(\.\d+)+", version):
+            found["harness_version"] = version
+        if session_id := os.environ.get("CLAUDE_CODE_SESSION_ID"):
+            found["session_id"] = session_id
+    return found
+
+
+def digests_of(files: list[str], root: pathlib.Path) -> list[dict[str, str]] | None:
+    """Each named file or directory by its digest, or None once one of them is neither.
+
+    What a review and an annotation both record under `subjects`. Named `subjects`, never
+    `files`: `verify --files` reads `files` as seals, and a subject that later changes would
+    then report the path as sealed twice.
+    """
+    subjects = []
+    for name in files:
+        p = pathlib.Path(name).resolve()
+        if p.is_dir():
+            digest = sha256_of_tree(p)[0]
+            subjects.append({"path": record_path(p, root), "sha256": digest, "kind": "tree"})
+            continue
+        if not p.is_file():
+            print(f"not a file: {name}")
+            return None
+        digest = ledger.sha256_of_file(p)
+        subjects.append({"path": record_path(p, root), "sha256": digest, "kind": "file"})
+    return subjects
+
+
+def review(
+    files: list[str],
+    scope: str,
+    procedure: str,
+    verdict: str,
+    reviewer: str,
+    model: str | None,
+    note: str | None,
+) -> int:
+    root = require_root()
+    lp = ledger_path(root)
+    if verdict not in VERDICTS:
+        print(f"verdict must be one of: {', '.join(VERDICTS)}")
+        return 1
+    subjects = digests_of(files, root)
+    if subjects is None:
+        return 1
+    ran_in = session(model)
+    ledger.append_event(
+        lp,
+        {
+            "event": "review",
+            "subjects": subjects,
+            "scope": scope,
+            "procedure": procedure,
+            "verdict": verdict,
+            "reviewer": reviewer,
+            "session": ran_in,
+            "note": note or "",
+        },
+    )
+    print(f"review ({verdict}) by {reviewer}: {scope[:72]}")
+    for subject in subjects:
+        print(f"  {subject['sha256'][:16]}…  {subject['path']}")
+    if ran_in:
+        print(f"  session: {', '.join(f'{k} {v}' for k, v in ran_in.items())}")
+    if "harness" in ran_in and "model" not in ran_in:
+        print("  ran inside Claude Code with no --model; pass it if a model did the review.")
+    print("self-reported and unsigned: the ledger records that this was said, and of which bytes.")
+    return 0
+
+
+def annotate(
+    files: list[str],
+    kind: str,
+    ref: str,
+    digest: str | None,
+    by: str | None,
+    note: str | None,
+    location: str | None,
+) -> int:
+    root = require_root()
+    lp = ledger_path(root)
+    if not KIND.fullmatch(kind):
+        print(f"kind must be namespace:type, such as wandb:artifact or doi:dataset; got '{kind}'")
+        print("lowercase letters, digits, '-', '_' and '.' on each side of one colon.")
+        return 1
+    if not ref.strip():
+        print("ref must name the outside thing: an artifact path, a DOI, a URL, an id.")
+        return 1
+    if digest is not None:
+        stated = DIGEST.fullmatch(digest)
+        length = DIGEST_LENGTHS.get(stated[1]) if stated else None
+        if not stated or (length is not None and len(stated[2]) != length):
+            print(f"digest must be algorithm:hex, such as sha256:<64 hex digits>; got '{digest}'")
+            return 1
+    subjects = digests_of(files, root)
+    if subjects is None:
+        return 1
+    # No session is read here. A review says who or what did the reading, so it records where
+    # the command ran; an annotation makes no such statement, and `--by` is whatever was typed.
+    target = {"kind": kind, "ref": ref}
+    if digest:
+        target["digest"] = digest
+    event: dict[str, object] = {
+        "event": "annotation",
+        "subjects": subjects,
+        "target": target,
+        "status": LINKED,
+        "note": note or "",
+    }
+    if by:
+        event["by"] = by
+    if location:
+        event["location"] = location
+    ledger.append_event(lp, event)
+    print(f"annotation: {kind}  {ref[:72]}" + (f"  by {by}" if by else ""))
+    for subject in subjects:
+        print(f"  {subject['sha256'][:16]}…  {subject['path']}")
+    if digest:
+        print(f"  digest as stated: {digest}")
+    if location:
+        print(f"  appears in: {location}")
+    print(f"{LINKED}: the ledger records that this link was stated, and of which bytes.")
+    print("Nothing was fetched or compared.")
+    return 0

results_cli-0.5.4/tests/test_annotate.py

@@ -0,0 +1,244 @@
+"""`results annotate`: a stated link from named bytes to something outside the ledger."""
+
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+
+import pytest
+from results import ledger
+
+LINK = ("--kind", "wandb:artifact", "--ref", "lab/study/model:v3")
+SHA256 = "sha256:" + "0123456789abcdef" * 4
+
+
+def run_cli(*args, cwd, env=None):
+    base = {k: v for k, v in os.environ.items() if not k.startswith(("CLAUDE", "ANTHROPIC"))}
+    return subprocess.run(
+        [sys.executable, "-m", "results.cli", *args],
+        cwd=cwd,
+        capture_output=True,
+        text=True,
+        env={**base, **(env or {})},
+    )
+
+
+def project(tmp_path):
+    run_cli("init", cwd=tmp_path)
+    (tmp_path / "model.pt").write_text("weights\n")
+    return tmp_path
+
+
+def ledger_bytes(tmp_path):
+    return (tmp_path / ".results" / "ledger.jsonl").read_bytes()
+
+
+def last_event(tmp_path):
+    return ledger.read_ledger(tmp_path / ".results" / "ledger.jsonl")[-1]
+
+
+def test_an_annotation_records_the_digest_of_the_subject_and_the_target(tmp_path):
+    project(tmp_path)
+    r = run_cli("annotate", "model.pt", *LINK, cwd=tmp_path)
+    assert r.returncode == 0, r.stdout + r.stderr
+    event = last_event(tmp_path)
+    assert {k: v for k, v in event.items() if k not in ("seq", "timestamp", "prev_hash")} == {
+        "event": "annotation",
+        "subjects": [
+            {
+                "path": "model.pt",
+                "sha256": ledger.sha256_of_file(tmp_path / "model.pt"),
+                "kind": "file",
+            }
+        ],
+        "target": {"kind": "wandb:artifact", "ref": "lab/study/model:v3"},
+        "status": "linked, not checked",
+        "note": "",
+    }
+    assert "linked, not checked: the ledger records that this link was stated" in r.stdout
+    assert "Nothing was fetched or compared." in r.stdout
+
+
+def test_by_note_and_location_are_recorded_as_typed(tmp_path):
+    project(tmp_path)
+    run_cli(
+        "annotate",
+        "model.pt",
+        "--kind",
+        "doi:dataset",
+        "--ref",
+        "10.5281/zenodo.1234567",
+        "--by",
+        "A. Reader",
+        "--note",
+        "the checkpoint behind Table 2",
+        "--location",
+        "Table 2",
+        cwd=tmp_path,
+    )
+    event = last_event(tmp_path)
+    assert event["target"] == {"kind": "doi:dataset", "ref": "10.5281/zenodo.1234567"}
+    assert event["by"] == "A. Reader"
+    assert event["note"] == "the checkpoint behind Table 2"
+    assert event["location"] == "Table 2"
+
+
+@pytest.mark.parametrize(
+    "kind", ["wandb", "wandb:", ":artifact", "WandB:artifact", "wandb:art ifact", "a:b:c", "a/b:c"]
+)
+def test_a_malformed_kind_writes_nothing(tmp_path, kind):
+    project(tmp_path)
+    before = ledger_bytes(tmp_path)
+    r = run_cli("annotate", "model.pt", "--kind", kind, "--ref", "x", cwd=tmp_path)
+    assert r.returncode == 1
+    assert "namespace:type" in r.stdout
+    assert ledger_bytes(tmp_path) == before
+
+
+@pytest.mark.parametrize(
+    "kind", ["osf:node", "url:page", "zenodo:record", "hf:dataset", "x-1.y:z_2"]
+)
+def test_any_well_formed_kind_is_accepted(tmp_path, kind):
+    project(tmp_path)
+    r = run_cli("annotate", "model.pt", "--kind", kind, "--ref", "x", cwd=tmp_path)
+    assert r.returncode == 0, r.stdout + r.stderr
+    assert last_event(tmp_path)["target"]["kind"] == kind
+
+
+def test_kind_and_ref_are_required(tmp_path):
+    project(tmp_path)
+    before = ledger_bytes(tmp_path)
+    for dropped in ("--kind", "--ref"):
+        at = LINK.index(dropped)
+        args = LINK[:at] + LINK[at + 2 :]
+        r = run_cli("annotate", "model.pt", *args, cwd=tmp_path)
+        assert r.returncode == 2, dropped
+        assert dropped in r.stderr
+    blank = run_cli("annotate", "model.pt", "--kind", "url:page", "--ref", "  ", cwd=tmp_path)
+    assert blank.returncode == 1
+    assert ledger_bytes(tmp_path) == before
+
+
+def test_a_path_that_is_not_a_file_writes_nothing(tmp_path):
+    project(tmp_path)
+    before = ledger_bytes(tmp_path)
+    r = run_cli("annotate", "absent.pt", *LINK, cwd=tmp_path)
+    assert r.returncode == 1
+    assert ledger_bytes(tmp_path) == before
+
+
+def test_no_session_is_recorded_even_inside_claude_code(tmp_path):
+    project(tmp_path)
+    env = {
+        "CLAUDECODE": "1",
+        "CLAUDE_CODE_EXECPATH": "/somewhere/claude/versions/2.1.295",
+        "CLAUDE_CODE_SESSION_ID": "0f0f0f0f-aaaa-bbbb-cccc-121212121212",
+    }
+    r = run_cli("annotate", "model.pt", *LINK, "--by", "A. Reader", cwd=tmp_path, env=env)
+    assert r.returncode == 0, r.stdout + r.stderr
+    event = last_event(tmp_path)
+    assert not {"session", "model", "harness", "harness_version", "session_id"} & set(event)
+    line = ledger_bytes(tmp_path).splitlines()[-1].decode()
+    for leaked in ("Claude", "2.1.295", "0f0f0f0f", "session", "harness", 'model"'):
+        assert leaked not in line, leaked
+
+
+def test_a_well_formed_digest_is_stored_as_stated(tmp_path):
+    project(tmp_path)
+    for stated in (SHA256, "md5:" + "ab" * 16, "blake3:" + "CD" * 20):
+        r = run_cli("annotate", "model.pt", *LINK, "--digest", stated, cwd=tmp_path)
+        assert r.returncode == 0, r.stdout + r.stderr
+        assert last_event(tmp_path)["target"]["digest"] == stated
+    assert last_event(tmp_path)["subjects"][0]["sha256"] != stated.split(":")[1]
+
+
+@pytest.mark.parametrize(
+    "digest",
+    ["0123456789abcdef" * 4, "sha256:" + "0" * 63, "sha256:" + "g" * 64, "sha256:", "md5:abc"],
+)
+def test_a_malformed_digest_writes_nothing(tmp_path, digest):
+    project(tmp_path)
+    before = ledger_bytes(tmp_path)
+    r = run_cli("annotate", "model.pt", *LINK, "--digest", digest, cwd=tmp_path)
+    assert r.returncode == 1
+    assert "algorithm:hex" in r.stdout
+    assert ledger_bytes(tmp_path) == before
+
+
+def test_verify_reports_an_annotated_file_that_changed_as_stale_and_does_not_fail(tmp_path):
+    project(tmp_path)
+    run_cli("annotate", "model.pt", *LINK, "--by", "A. Reader", cwd=tmp_path)
+    recorded = ledger.sha256_of_file(tmp_path / "model.pt")
+
+    current = run_cli("verify", "--files", cwd=tmp_path)
+    assert current.returncode == 0, current.stdout
+    assert "annotations (linked, not checked):" in current.stdout
+    assert "wandb:artifact  lab/study/model:v3  [A. Reader]" in current.stdout
+    assert "current      model.pt" in current.stdout
+    assert "STALE" not in current.stdout
+
+    (tmp_path / "model.pt").write_text("other weights\n")
+    now = ledger.sha256_of_file(tmp_path / "model.pt")
+    stale = run_cli("verify", "--files", cwd=tmp_path)
+    assert stale.returncode == 0, stale.stdout
+    assert "STALE        model.pt" in stale.stdout
+    assert f"annotated {recorded[:16]}…, now {now[:16]}…" in stale.stdout
+    assert "1 annotated file(s) are no longer the bytes the link was stated of" in stale.stdout
+
+    (tmp_path / "model.pt").unlink()
+    missing = run_cli("verify", "--files", cwd=tmp_path)
+    assert missing.returncode == 0, missing.stdout
+    assert "MISSING      model.pt" in missing.stdout
+
+
+def test_an_annotation_is_not_a_seal(tmp_path):
+    project(tmp_path)
+    run_cli("annotate", "model.pt", *LINK, cwd=tmp_path)
+    (tmp_path / "model.pt").write_text("other weights\n")
+    run_cli("annotate", "model.pt", *LINK, cwd=tmp_path)
+    r = run_cli("verify", "--files", cwd=tmp_path)
+    assert r.returncode == 0, r.stdout
+    assert "RESEALED" not in r.stdout
+    assert "CHANGED" not in r.stdout
+    assert "  ok " not in r.stdout
+
+
+def test_without_files_verify_lists_the_annotation_and_says_the_subject_was_not_checked(tmp_path):
+    project(tmp_path)
+    run_cli(
+        "annotate", "model.pt", *LINK, "--digest", SHA256, "--location", "Table 2", cwd=tmp_path
+    )
+    r = run_cli("verify", cwd=tmp_path)
+    assert r.returncode == 0, r.stdout
+    assert "annotations (linked, not checked):" in r.stdout
+    assert "wandb:artifact  lab/study/model:v3" in r.stdout
+    assert f"digest as stated: {SHA256}" in r.stdout
+    assert "appears in: Table 2" in r.stdout
+    assert "not checked  model.pt" in r.stdout
+
+
+def test_the_chain_stays_intact_with_annotations_among_the_other_events(tmp_path):
+    project(tmp_path)
+    lp = tmp_path / ".results" / "ledger.jsonl"
+    run_cli("seal", "model.pt", cwd=tmp_path)
+    run_cli("annotate", "model.pt", *LINK, cwd=tmp_path)
+    run_cli("run", "model.pt", "--run-id", "r1", cwd=tmp_path)
+    run_cli("annotate", "model.pt", "--kind", "osf:node", "--ref", "abc12", cwd=tmp_path)
+    run_cli("claim", "accuracy 0.91", "--run-id", "r1", cwd=tmp_path)
+    assert ledger.verify(lp) == (ledger.ChainStatus.INTACT, [])
+    assert [e["event"] for e in ledger.read_ledger(lp)] == [
+        "init",
+        "seal",
+        "annotation",
+        "run",
+        "annotation",
+        "claim",
+    ]
+    r = run_cli("verify", "--files", cwd=tmp_path)
+    assert r.returncode == 0, r.stdout
+    assert "all checks passed." in r.stdout
+
+    tampered = lp.read_text().replace("lab/study/model:v3", "lab/study/model:v4")
+    lp.write_text(tampered)
+    assert ledger.verify(lp)[0] is ledger.ChainStatus.EDITED

results_cli-0.5.4/tests/test_review.py

@@ -0,0 +1,152 @@
+"""`results review`: a record that someone examined named bytes, and what `verify` says of it."""
+
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+
+from results import ledger
+
+REVIEW = (
+    "--scope",
+    "each mismatch against the paper's table",
+    "--procedure",
+    "read the rendered page beside the file",
+    "--reviewer",
+    "A. Reader",
+)
+
+
+def run_cli(*args, cwd, env=None):
+    base = {k: v for k, v in os.environ.items() if not k.startswith(("CLAUDE", "ANTHROPIC"))}
+    return subprocess.run(
+        [sys.executable, "-m", "results.cli", *args],
+        cwd=cwd,
+        capture_output=True,
+        text=True,
+        env={**base, **(env or {})},
+    )
+
+
+def project(tmp_path):
+    run_cli("init", cwd=tmp_path)
+    (tmp_path / "table.csv").write_text("a,b\n1,2\n")
+    return tmp_path
+
+
+def last_event(tmp_path):
+    return ledger.read_ledger(tmp_path / ".results" / "ledger.jsonl")[-1]
+
+
+def test_a_review_records_the_digest_of_what_was_examined(tmp_path):
+    project(tmp_path)
+    r = run_cli("review", "table.csv", *REVIEW, "--verdict", "partial", cwd=tmp_path)
+    assert r.returncode == 0, r.stdout + r.stderr
+    event = last_event(tmp_path)
+    assert event["event"] == "review"
+    assert event["verdict"] == "partial"
+    assert event["reviewer"] == "A. Reader"
+    assert event["subjects"] == [
+        {
+            "path": "table.csv",
+            "sha256": ledger.sha256_of_file(tmp_path / "table.csv"),
+            "kind": "file",
+        }
+    ]
+    assert "self-reported and unsigned" in r.stdout
+
+
+def test_a_verdict_outside_the_four_writes_nothing(tmp_path):
+    project(tmp_path)
+    before = (tmp_path / ".results" / "ledger.jsonl").read_bytes()
+    r = run_cli("review", "table.csv", *REVIEW, "--verdict", "verified", cwd=tmp_path)
+    assert r.returncode == 1
+    assert (tmp_path / ".results" / "ledger.jsonl").read_bytes() == before
+
+
+def test_scope_procedure_and_reviewer_are_required(tmp_path):
+    project(tmp_path)
+    for dropped in ("--scope", "--procedure", "--reviewer"):
+        at = REVIEW.index(dropped)
+        args = REVIEW[:at] + REVIEW[at + 2 :]
+        r = run_cli("review", "table.csv", *args, "--verdict", "pass", cwd=tmp_path)
+        assert r.returncode == 2, dropped
+        assert dropped in r.stderr
+
+
+def test_the_session_is_recorded_as_the_environment_and_the_flag_give_it(tmp_path):
+    project(tmp_path)
+    env = {
+        "CLAUDECODE": "1",
+        "CLAUDE_CODE_EXECPATH": "/somewhere/claude/versions/2.1.295",
+        "CLAUDE_CODE_SESSION_ID": "0f0f0f0f-aaaa-bbbb-cccc-121212121212",
+    }
+    r = run_cli(
+        "review",
+        "table.csv",
+        *REVIEW,
+        "--verdict",
+        "pass",
+        "--model",
+        "claude-opus-5-5",
+        cwd=tmp_path,
+        env=env,
+    )
+    assert r.returncode == 0, r.stdout + r.stderr
+    assert last_event(tmp_path)["session"] == {
+        "model": "claude-opus-5-5",
+        "harness": "Claude Code",
+        "harness_version": "2.1.295",
+        "session_id": "0f0f0f0f-aaaa-bbbb-cccc-121212121212",
+    }
+
+
+def test_outside_a_session_and_with_no_model_the_session_is_empty(tmp_path):
+    project(tmp_path)
+    run_cli("review", "table.csv", *REVIEW, "--verdict", "pass", cwd=tmp_path)
+    assert last_event(tmp_path)["session"] == {}
+
+
+def test_a_session_with_no_model_named_says_so(tmp_path):
+    project(tmp_path)
+    r = run_cli(
+        "review", "table.csv", *REVIEW, "--verdict", "pass", cwd=tmp_path, env={"CLAUDECODE": "1"}
+    )
+    assert "no --model" in r.stdout
+    assert "model" not in last_event(tmp_path)["session"]
+
+
+def test_verify_reports_a_reviewed_file_that_changed_as_stale_and_does_not_fail(tmp_path):
+    project(tmp_path)
+    run_cli("review", "table.csv", *REVIEW, "--verdict", "pass", cwd=tmp_path)
+
+    current = run_cli("verify", "--files", cwd=tmp_path)
+    assert current.returncode == 0, current.stdout
+    assert "current      table.csv" in current.stdout
+    assert "STALE" not in current.stdout
+
+    (tmp_path / "table.csv").write_text("a,b\n1,3\n")
+    stale = run_cli("verify", "--files", cwd=tmp_path)
+    assert stale.returncode == 0, stale.stdout
+    assert "STALE        table.csv" in stale.stdout
+    assert "1 reviewed file(s) are no longer the bytes that were reviewed" in stale.stdout
+
+
+def test_a_review_is_not_a_seal(tmp_path):
+    project(tmp_path)
+    run_cli("review", "table.csv", *REVIEW, "--verdict", "pass", cwd=tmp_path)
+    (tmp_path / "table.csv").write_text("a,b\n1,3\n")
+    run_cli("review", "table.csv", *REVIEW, "--verdict", "pass", cwd=tmp_path)
+    r = run_cli("verify", "--files", cwd=tmp_path)
+    assert "RESEALED" not in r.stdout
+    assert "CHANGED" not in r.stdout
+
+
+def test_without_files_verify_lists_the_review_and_says_the_subject_was_not_checked(tmp_path):
+    project(tmp_path)
+    run_cli("review", "table.csv", *REVIEW, "--verdict", "fail", cwd=tmp_path)
+    r = run_cli("verify", cwd=tmp_path)
+    assert "reviews (self-reported, unsigned):" in r.stdout
+    assert "fail" in r.stdout and "A. Reader" in r.stdout
+    assert "not checked  table.csv" in r.stdout