shuttlecheck 0.2.1__tar.gz → 0.3.0__tar.gz

{shuttlecheck-0.2.1/shuttlecheck.egg-info → shuttlecheck-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: shuttlecheck
-Version: 0.2.1
+Version: 0.3.0
 Summary: Check media filenames, folder structure and deliverables against a written convention. Read-only.
 Author-email: MVRK Systems <hello@mvrk.systems>
 License: Proprietary — free to use, no redistribution
@@ -26,6 +26,7 @@ Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: PyYAML>=6.0
 Requires-Dist: jsonschema>=4.20
+Requires-Dist: xxhash>=3.0
 Requires-Dist: tzdata>=2024.1; sys_platform == "win32"
 Provides-Extra: dev
 Requires-Dist: pytest>=8.0; extra == "dev"
@@ -65,8 +66,8 @@ FIELDS RECOVERED
   {title}        slug        85%  opening-remarks, keynote
 ```
 
-It handles conventions that disagree with themselves — several accepted forms
-per field, optional elements, free-text titles containing the separator —
+It handles conventions that disagree with themselves: several accepted forms
+per field, optional elements, free-text titles containing the separator, all
 because real ones do.
 
 ## Then it tells you what breaks it
@@ -101,14 +102,14 @@ conflated, because a noisy report does not get read.
 | Unicode | Names that compare equal on one platform and not another |
 
 Media inspection uses `ffprobe` when it is on your PATH. When it is not, those
-checks are reported as **skipped** — never as passed. A silent skip is how a
+checks are reported as **skipped**, never as passed. A silent skip is how a
 conformance tool starts lying.
 
 ## For live events, the running order does the work
 
 An event agenda is structured data that exists weeks before anyone turns a
-camera on. Joined against time-of-day timecode, every recording names itself —
-stage, session, speaker, sponsor — with no inference of any kind.
+camera on. Joined against time-of-day timecode, every recording names itself,
+with its stage, session, speaker and sponsor, and no inference of any kind.
 
 It will also tell you when a filename is well-formed and still wrong: `S007`
 recorded during `S011` is a valid shape carrying a false claim, and only the

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/PYPI.md

@@ -32,8 +32,8 @@ FIELDS RECOVERED
   {title}        slug        85%  opening-remarks, keynote
 ```
 
-It handles conventions that disagree with themselves — several accepted forms
-per field, optional elements, free-text titles containing the separator —
+It handles conventions that disagree with themselves: several accepted forms
+per field, optional elements, free-text titles containing the separator, all
 because real ones do.
 
 ## Then it tells you what breaks it
@@ -68,14 +68,14 @@ conflated, because a noisy report does not get read.
 | Unicode | Names that compare equal on one platform and not another |
 
 Media inspection uses `ffprobe` when it is on your PATH. When it is not, those
-checks are reported as **skipped** — never as passed. A silent skip is how a
+checks are reported as **skipped**, never as passed. A silent skip is how a
 conformance tool starts lying.
 
 ## For live events, the running order does the work
 
 An event agenda is structured data that exists weeks before anyone turns a
-camera on. Joined against time-of-day timecode, every recording names itself —
-stage, session, speaker, sponsor — with no inference of any kind.
+camera on. Joined against time-of-day timecode, every recording names itself,
+with its stage, session, speaker and sponsor, and no inference of any kind.
 
 It will also tell you when a filename is well-formed and still wrong: `S007`
 recorded during `S011` is a valid shape carrying a false claim, and only the

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/README.md

@@ -1,4 +1,4 @@
-# ShuttleCheck — validator
+# ShuttleCheck, the validator
 
 Naming conformance for a folder tree: infer a spec from it, check it, fix it,
 undo the fix, and send the convention with the media. Installs as `shuttlecheck`;
@@ -16,7 +16,7 @@ talked into running on a live job, because running it cannot cost them anything.
 python -m pip install -e .
 ```
 
-`tzdata` is a real runtime dependency on Windows — the platform ships no system
+`tzdata` is a real runtime dependency on Windows, because the platform ships no system
 timezone database, so `zoneinfo` cannot resolve the agenda timezone without it.
 
 ## Use
@@ -109,7 +109,7 @@ Delivery rules, for contexts that declare a `deliverable:` profile:
 | `missing-sidecar` | error | Captions or checksum did not arrive with the file |
 | `no-video-stream`, `no-audio-stream`, `unreadable-media` | error | Not the kind of file it claims to be |
 | `color-*-mismatch` | error | Tagged with the wrong colour space |
-| `color-*-unknown` | warning | Not tagged at all — absence is not proof of wrongness |
+| `color-*-unknown` | warning | Not tagged at all. Absence is not proof of wrongness |
 | `incomplete-package` | error | A session or episode is short a deliverable |
 | `spec-unknown-deliverable` | error | A context points at a profile that is not defined |
 
@@ -120,7 +120,7 @@ Severities are overridable per rule in the spec's `rules:` block.
 `shuttlecheck ui` is a local web UI, not Electron and not Qt. The page ships
 inside the wheel and is served by the standard library's HTTP server, so the
 install stays `pip install shuttlecheck`, nothing is downloaded at runtime, and
-it works with no connection at all — which matters, because the machine this
+it works with no connection at all, which matters, because the machine this
 runs on is often in a truck. No web fonts either; the stylesheet falls back to
 whatever the OS has.
 
@@ -134,7 +134,7 @@ in the URL the browser is opened with, and never leaves the machine.
 
 **Applying a plan requires echoing back the move count.** The page states how
 many moves it believes it is approving. If the folder changed since the plan
-was drawn, the numbers disagree and nothing runs — a stale tab cannot apply a
+was drawn, the numbers disagree and nothing runs. A stale tab cannot apply a
 plan that no longer matches reality.
 
 ## How inference works
@@ -150,14 +150,14 @@ to try.
    has a different field count, anchor from the left and the right while the
    columns still look like fixed codes and treat the remainder as one variable
    field. Names structurally unrelated to the rest are set aside first, and
-   reported — they are usually the existing violations.
+   reported. They are usually the existing violations.
 3. **Profile** each column: constant, prefixed counter, enum, date, slug, or
    free text, each with a confidence and a plain-English reason.
 4. **Reconcile** value sets across groups, so a column that reads as the
    constant `HALLA` in one group and the enum `{HALLA, HALLB}` in another
    becomes one token rather than two.
-5. **Name** the fields from prefix semantics rather than exact widths — `D01`
-   and `DAY1` are both `{day}` — falling back to `{field3}` and flagging it.
+5. **Name** the fields from prefix semantics rather than exact widths, so `D01`
+   and `DAY1` are both `{day}`, falling back to `{field3}` and flagging it.
 6. **Self-validate**: run the validator over the same tree with the spec just
    inferred, and show the result. That is the thirty-second demonstration.
 
@@ -175,14 +175,14 @@ comments of the file it writes.
 It renames only what it can *derive*. For each non-conforming file:
 
 1. Parse the folder and the filename against the context's patterns.
-2. If both parse but disagree on a token, **skip** — there is no safe way to
+2. If both parse but disagree on a token, **skip**. There is no safe way to
    tell which one is the mistake, so it is a question for a human.
 3. If the name fails on a single field, recover that one field: from the folder
    path, by matching a known alias, by re-applying the token's own transforms
    (which fixes casing, spaces and punctuation), or from its declared source.
 4. If a context sets `schedule_check`, cross-check the schedule-bound tokens
    against the agenda and correct them. A name can parse perfectly and still be
-   wrong — `S001` recorded during `S005` is a valid shape carrying a false
+   wrong: `S001` recorded during `S005` is a valid shape carrying a false
    claim. An ambiguous clock window corrects nothing rather than guessing.
 5. If the correct name still cannot be derived, **skip and say why**. Inventing
    a session number is worse than leaving a bad filename.
@@ -194,7 +194,7 @@ Safety, none of it optional:
 | Dry run | The default. `--apply` is a separate, explicit act. |
 | Originals | `rename: never` contexts are never planned, moved, or renamed. |
 | Overwrite | Never. A destination that exists aborts the whole plan. |
-| Collisions | Two files resolving to one name abort everything — a half-renamed job is worse than an unrenamed one. |
+| Collisions | Two files resolving to one name abort everything. A half-renamed job is worse than an unrenamed one. |
 | Stale plans | Every source is re-checked for size and mtime before the first move. |
 | Case-only renames | Routed through a temporary name, as are swaps and cycles. |
 | Mid-run failure | Every completed move is rolled back, and the run is marked `rolled_back` in the journal. |
@@ -251,7 +251,7 @@ WHAT YOU ARE DELIVERING
     with      Subtitles ({stem}.srt)
 ```
 
-The pack is **derived**, never authored separately — a vendor cannot drift from
+The pack is **derived**, never authored separately, so a vendor cannot drift from
 a document they did not write. `vendors` is the other direction: point it at the
 incoming deliveries and it reports who is clear, who has errors, and who has
 sent nothing at all.
@@ -260,7 +260,7 @@ Two things worth knowing:
 
 **Out-of-scope contexts are kept as inert stubs, not deleted.** Contexts are
 first-match-wins, so removing an early one silently re-routes its files into a
-later catch-all — the ISO rushes would come back reported as malformed masters.
+later catch-all, and the ISO rushes would come back reported as malformed masters.
 The stubs preserve ordering and say plainly: not yours.
 
 **A vendor is never shown a rule they cannot act on.** Packages narrow to the
@@ -275,7 +275,7 @@ intended.
 
 Its second job matters more than undo. Every file carries an `original_path`,
 recorded the first time it is touched and never rewritten. When a spec is
-edited and the tree re-conformed, names are computed from that identity — never
+edited and the tree re-conformed, names are computed from that identity, never
 from the current name. Without it, tokens compound on every pass: the second
 run re-prefixes what the first already prefixed, and by the third edit the
 names are unrecoverable. That failure is silent and looks like it worked.
@@ -294,7 +294,7 @@ drive out of a Peli case. `seal` writes `_SPEC/` onto the drive itself:
 | `README.txt` | plain-language instructions for someone who has never heard of this tool |
 | `journal/` | what has been renamed, and by which spec revision |
 
-The receiving end runs `shuttlecheck verify .` — no spec to be told about, no
+The receiving end runs `shuttlecheck verify .` with no spec to be told about, no
 account, no connectivity. Sealing never touches the media; it only writes
 inside `_SPEC/`.
 
@@ -302,19 +302,19 @@ What `verify` reports:
 
 | | |
 |---|---|
-| `missing` | error — on the manifest, not on the drive; the copy is incomplete |
-| `size-mismatch` | error — damaged or replaced in transit |
-| `hash-mismatch` | error — same size, different contents (needs `--hash` at both ends) |
-| `added` | warning — on the drive but not the manifest |
-| `mtime-drift` | info — most copy tools do not preserve timestamps |
-| `clock-did-not-survive-copy` | warning — see below |
+| `missing` | error: on the manifest, not on the drive; the copy is incomplete |
+| `size-mismatch` | error: damaged or replaced in transit |
+| `hash-mismatch` | error: same size, different contents (needs `--hash` at both ends) |
+| `added` | warning: on the drive but not the manifest |
+| `mtime-drift` | info: most copy tools do not preserve timestamps |
+| `clock-did-not-survive-copy` | warning: see below |
 
 Two judgement calls worth knowing about.
 
 **Timestamp drift is not damage.** Most copy tools do not preserve mtime, so
 treating that as corruption would make every genuine transfer look broken.
 *But* if the spec joins the agenda on `file_mtime`, drift silently moves every
-recording to the wrong session and the naming findings become fiction — so that
+recording to the wrong session and the naming findings become fiction, so that
 specific combination is raised as a warning rather than left to be discovered.
 A spec using embedded timecode does not have the problem.
 
@@ -328,7 +328,7 @@ timestamps are all that can be checked, and the sidecar says so in writing.
 **It parses, it does not render.** The validator compiles each naming pattern
 into a regex whose groups are constrained by each token's own type, enum values,
 and transforms, then matches the existing name against it. That is why it needs
-no prompting, no schedule, and no media parser to do its core job — and why
+no prompting, no schedule, and no media parser to do its core job, and why
 `prompt`-sourced tokens validate fine.
 
 **Mismatches name the field.** On a failure it recompiles the pattern with each
@@ -363,7 +363,7 @@ shuttlecheck validate ./fixture --spec ./fixture/EXC26/_SPEC/spec.yaml
 ```
 
 `tests/make_infer_fixture.py` writes a tree using a convention the code has
-never seen — dashes instead of underscores, `DAY1` instead of `D01`, `SES001`
+never seen: dashes instead of underscores, `DAY1` instead of `D01`, `SES001`
 instead of `S001`, variable-length titles containing the separator, and
 camera-native ISO names. If inference recovers that, it is reading the tree
 rather than recognising something baked into the source:
@@ -379,7 +379,7 @@ errors in the files it learned from.
 
 Portability rules are unit-tested against the check functions rather than real
 files, because Windows cannot create a file named `NUL.mov`, `bad<name>.mov`, or
-`trailing..mov.` — which is the point of the check.
+`trailing..mov.`, which is the point of the check.
 
 ## Not built yet
 
@@ -388,5 +388,5 @@ files, because Windows cannot create a file named `NUL.mov`, `bad<name>.mov`, or
   creation time via `ffprobe` and degrades honestly without it.
 - Agenda formats other than CSV. `ics`, `sched`, `swapcard`, and `cvent` are in
   the schema and raise `ScheduleUnavailable` rather than failing the run.
-- The conformer, its journal, and revert. Deliberately absent — see the
+- The conformer, its journal, and revert. Deliberately absent: see the
   read-only boundary above.

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "shuttlecheck"
-version = "0.2.1"
+version = "0.3.0"
 description = "Check media filenames, folder structure and deliverables against a written convention. Read-only."
 readme = { file = "PYPI.md", content-type = "text/markdown" }
 requires-python = ">=3.11"
@@ -29,6 +29,13 @@ classifiers = [
 dependencies = [
     "PyYAML>=6.0",
     "jsonschema>=4.20",
+    # xxh64 is the checksum the ASC MHL specification is usually written with
+    # and the one DailiesDrop records, and it is not in the standard library.
+    # A dependency in the read-only half looks like the wrong place for it
+    # until you notice that verifying a drive the appliance sealed is exactly
+    # what this half is for: without it, the common case degrades to "this
+    # installation cannot check those checksums".
+    "xxhash>=3.0",
     # Windows ships no system tz database, so zoneinfo cannot resolve the
     # agenda timezone without this. Required, not optional, on a target platform.
     "tzdata>=2024.1; sys_platform == 'win32'",

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/shuttlecheck/__init__.py

@@ -1,18 +1,18 @@
-"""ShuttleCheck -- naming and delivery conformance.
+"""ShuttleCheck: naming and delivery conformance.
 
 Three entry points, in the order you would use them:
 
     infer_spec / walkthrough   read an existing tree and write the spec it follows
     Validator / validate       report what does not conform, in name and in
-                               substance -- codec, resolution, audio, packages
+                               substance: codec, resolution, audio, packages
     verify                     check a drive against the sidecar it arrived with
 
 **Nothing in this package modifies media.** That is the whole of it, and it is
 deliberate: a checker that cannot alter anything is a checker a stranger will
 run on a live job, because running it cannot cost them anything.
 
-The operations that write -- conforming, the journal, sealing a drive, vendor
-packs -- are licensed separately and ship as `shuttlecheck_pro`. When it is
+The operations that write (conforming, the journal, sealing a drive, vendor
+packs) are licensed separately and ship as `shuttlecheck_pro`. When it is
 installed, `extensions.load()` finds it and the CLI and app grow the extra
 commands; nothing here imports it, and nothing here requires it.
 """

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/shuttlecheck/cli.py

@@ -10,8 +10,8 @@ None of the above modifies media. That is the whole of this package, and it is
 deliberate: a checker that cannot alter anything is a checker a stranger will
 run on a live job.
 
-Installing ShuttleCheck Pro adds the operations that write -- conform, revert,
-history, seal, pack, vendors -- and they appear here as extra subcommands. When
+Installing ShuttleCheck Pro adds the operations that write (conform, revert,
+history, seal, pack, vendors) and they appear here as extra subcommands. When
 it is absent those names are still registered, so asking for one gets an
 explanation rather than "invalid choice".
 """
@@ -55,6 +55,18 @@ def build_parser() -> argparse.ArgumentParser:
     c = sub.add_parser("check-spec", help="validate the spec itself and exit")
     c.add_argument("--spec", required=True)
 
+    b = sub.add_parser(
+        "support",
+        help="write one file to send us when something has gone wrong")
+    b.add_argument("--names", action="store_true",
+                   help="replace every filename with a stand-in, for media "
+                        "you are not allowed to share")
+    b.add_argument("--job", action="append", dest="jobs", default=None,
+                   help="a specific job id; repeatable (default: the most recent)")
+    b.add_argument("--out", help="where to write it")
+    b.add_argument("--yes", action="store_true",
+                   help="write it without printing the contents first")
+
     u = sub.add_parser("ui", help="open the local app in a browser")
     u.add_argument("root", nargs="?", help="folder to open on (optional)")
     u.add_argument("--port", type=int, default=8765)
@@ -100,7 +112,7 @@ def _add_licensed_stubs(sub) -> None:
     installed. These accept anything and fail with a sentence.
     """
     for name in LICENSED:
-        stub = sub.add_parser(name, help="(ShuttleCheck Pro -- not installed)",
+        stub = sub.add_parser(name, help="(ShuttleCheck Pro, not installed)",
                               add_help=False)
         stub.add_argument("args", nargs=argparse.REMAINDER)
 
@@ -215,8 +227,52 @@ def _verify(args) -> int:
     return 1 if result.naming is not None and result.naming.errors else 0
 
 
+def _support(args) -> int:
+    """Write the bundle, after saying what is going in it.
+
+    Printed first and by default. Somebody running this is about to email a
+    file to a company, and "it only contains diagnostics" is the kind of
+    sentence that is always said and rarely checked. So it is not said: the
+    contents are listed, and the flag to skip the listing is opt-in.
+    """
+    from . import support
+
+    mode = support.NO_NAMES if args.names else support.FULL
+    items = support.manifest(args.jobs, mode)
+
+    where = "names removed" if args.names else "full"
+    print(f"\n  A support bundle in {where} mode would contain:\n")
+    for item in items:
+        mark = "names " if item.names else "      "
+        print(f"    {mark} {item.name}")
+        print(f"            {item.what}")
+    if mode == support.FULL and any(i.names for i in items):
+        print("\n  The lines marked 'names' carry filenames from your media."
+              "\n  Run it again with --names if you cannot send those.")
+
+    if not args.yes:
+        try:
+            answer = input("\n  Write it? [y/N] ").strip().lower()
+        except EOFError:
+            # Run from a script, or with no terminal attached. Declining is
+            # the safe reading of silence here: this writes a file somebody is
+            # about to email. `--yes` is how a script says yes.
+            print("\n  Nothing was written: there is nobody here to ask. "
+                  "Pass --yes to write it without the question.")
+            return 0
+        if answer not in ("y", "yes"):
+            print("  Nothing was written.")
+            return 0
+
+    bundle = support.build(Path(args.out) if args.out else None, args.jobs, mode)
+    print(f"\n  {bundle.bytes:,} bytes written to\n    {bundle.path}")
+    print("\n  Nothing was uploaded. Send it to hello@mvrk.systems when you "
+          "are ready.")
+    return 0
+
+
 COMMANDS = {"ui": _ui, "validate": _validate, "check-spec": _check_spec,
-            "init": _init, "verify": _verify}
+            "init": _init, "verify": _verify, "support": _support}
 
 
 def _handlers() -> dict:

shuttlecheck-0.3.0/shuttlecheck/completeness.py

@@ -0,0 +1,227 @@
+"""Reconciling what was shot against what arrived.
+
+Every other check in this tool reasons about files that exist. That is a whole
+class of failure it cannot see, and it is the expensive one: a reel that was
+shot and never ingested produces no files, so it forms no group, matches no
+context, and breaks no rule. The delivery passes. Nobody finds out until an
+edit assistant goes looking for B_R004 three weeks later and it has been wiped
+off the card.
+
+`check_packages` in `delivery.py` is the closest existing thing and it has the
+same blind spot by construction: it groups the records it was given and asks
+whether each group is short. A group with no members is not short, it is
+absent, and absence is invisible from the inside.
+
+The only way out is a declaration from outside the media: a camera report, a
+sound report, a shooting log. The production already has one; it is the
+paperwork that says which rolls exist. This module reads it and subtracts.
+
+**Where it runs matters as much as what it does.** The declaration names reels,
+and a reel identifier carries the production, the camera and the roll. It is
+reconciled here, on the destination, from paperwork the production has already
+handed over, never on our servers. See `docs/DATA_BOUNDARY.md` section 4.
+"""
+
+from __future__ import annotations
+
+import csv
+from dataclasses import dataclass, field
+from pathlib import Path
+
+from .report import Finding
+
+# A row that declares nothing is not a declaration, and a camera report often
+# ends with a blank line or a totals row. Both would otherwise become a reel
+# that was never delivered, which is a false alarm about missing footage --
+# the most expensive kind of false alarm there is.
+SKIP_EMPTY = True
+
+
+class ManifestUnavailable(Exception):
+    """The declaration could not be read. Never silently treated as empty.
+
+    An empty declaration and an unreadable one look identical afterwards: both
+    produce no missing reels. One of them means everything arrived and the
+    other means nobody checked, and this tool does not let a check that could
+    not run report as a pass.
+    """
+
+
+@dataclass
+class Declared:
+    """One unit of work somebody says exists."""
+
+    key: tuple
+    fields: dict = field(default_factory=dict)
+    row: int = 0
+    source: str = ""
+
+    @property
+    def label(self) -> str:
+        return " ".join(f"{k}={v}" for k, v in self.fields.items())
+
+
+@dataclass
+class Ledger:
+    """What was declared, what arrived, and the difference in both directions."""
+
+    source: str = ""
+    key_tokens: list = field(default_factory=list)
+    declared: list = field(default_factory=list)
+    delivered: dict = field(default_factory=dict)      # key -> file count
+    missing: list = field(default_factory=list)        # Declared, nothing arrived
+    undeclared: list = field(default_factory=list)     # arrived, nothing declared
+
+    @property
+    def complete(self) -> bool:
+        return not self.missing
+
+    @property
+    def arrived(self) -> int:
+        """Declared work that something arrived for.
+
+        Counted against the declaration, not against everything that turned
+        up. Counting every key with files in it read "6 ingested of 6
+        declared" on a job with one reel missing and one nobody declared --
+        two errors cancelling into a reassuring number, which is the worst
+        arithmetic this screen could do.
+        """
+        keys = {d.key for d in self.declared}
+        return len([k for k, n in self.delivered.items() if n and k in keys])
+
+    def summary(self) -> dict:
+        return {
+            "source": self.source,
+            "keyTokens": list(self.key_tokens),
+            "declared": len(self.declared),
+            "delivered": self.arrived,
+            "files": sum(self.delivered.values()),
+            "missing": [{"label": d.label, "fields": d.fields, "row": d.row}
+                        for d in self.missing],
+            "undeclared": [{"label": " ".join(f"{k}={v}" for k, v in dict(
+                zip(self.key_tokens, key)).items()), "files": self.delivered[key]}
+                for key in self.undeclared],
+            "complete": self.complete,
+        }
+
+
+# --------------------------------------------------------------------------
+# Reading the declaration
+# --------------------------------------------------------------------------
+
+def load_manifest(source: dict, spec_dir: Path, name: str = "") -> list[Declared]:
+    """Read a camera report, or anything else shaped like one.
+
+    CSV only for now, and it says so rather than pretending. A camera report
+    is a table with one row per roll; the map says which of its columns are
+    which of the convention's fields, because nobody names their columns the
+    same way twice.
+    """
+    imp = source.get("import") or {}
+    fmt = imp.get("format")
+    if fmt != "csv":
+        raise ManifestUnavailable(
+            f"manifest format '{fmt}' is not implemented yet (only csv)")
+
+    raw = imp.get("path", "")
+    path = (spec_dir / raw).resolve()
+    if not path.is_file():
+        raise ManifestUnavailable(f"manifest file not found: {path}")
+
+    key_tokens = list(source.get("key") or [])
+    if not key_tokens:
+        raise ManifestUnavailable(
+            f"manifest '{name}' declares no key, so there is nothing to "
+            f"reconcile against: add `key: [unit, reel]` or similar")
+
+    column = imp.get("map") or {}
+    out: list[Declared] = []
+    try:
+        with path.open(encoding="utf-8-sig", newline="") as fh:
+            for number, row in enumerate(csv.DictReader(fh), start=2):
+                fields = {}
+                for token in key_tokens:
+                    value = (row.get(column.get(token, token)) or "").strip()
+                    fields[token] = value
+                if SKIP_EMPTY and not all(fields.values()):
+                    continue
+                out.append(Declared(key=tuple(fields[t] for t in key_tokens),
+                                    fields=fields, row=number, source=name))
+    except OSError as exc:
+        raise ManifestUnavailable(f"manifest could not be read: {exc}")
+
+    if not out:
+        raise ManifestUnavailable(
+            f"manifest '{name}' has no usable rows. Check the column names in "
+            f"`map` against the file's header: {path.name}")
+    return out
+
+
+# --------------------------------------------------------------------------
+# Subtracting
+# --------------------------------------------------------------------------
+
+def reconcile(declared: list[Declared], records, key_tokens: list,
+              source: str = "") -> Ledger:
+    """What arrived, against what was declared.
+
+    Both directions are reported. A declared reel that never arrived is the
+    expensive one. A reel that arrived and was never declared is cheaper and
+    still worth a look: usually a typo in one place or the other, occasionally
+    somebody's second camera that nobody told the office about.
+    """
+    ledger = Ledger(source=source, key_tokens=list(key_tokens),
+                    declared=list(declared))
+
+    for record in records:
+        if not all(t in record.tokens for t in key_tokens):
+            continue
+        key = tuple(record.tokens[t] for t in key_tokens)
+        ledger.delivered[key] = ledger.delivered.get(key, 0) + 1
+
+    declared_keys = {d.key for d in declared}
+    for entry in declared:
+        if not ledger.delivered.get(entry.key):
+            ledger.delivered.setdefault(entry.key, 0)
+            ledger.missing.append(entry)
+
+    ledger.undeclared = sorted(k for k, n in ledger.delivered.items()
+                               if n and k not in declared_keys)
+    return ledger
+
+
+def findings(ledger: Ledger, severities) -> list[Finding]:
+    """A missing reel is an error in the report, not a note on a side screen.
+
+    It belongs beside the other errors for the same reason the package check
+    does: this is the failure that costs somebody their deadline, and a person
+    who never opens the fleet view has still been told.
+    """
+    out = []
+    for entry in ledger.missing:
+        out.append(Finding(
+            rule="declared-never-delivered",
+            severity=severities("declared-never-delivered", "error"),
+            path=entry.label or "(unnamed)", context=ledger.source,
+            message=(f"{entry.label} is listed in {ledger.source or 'the manifest'} "
+                     f"(row {entry.row}) and nothing was ever delivered for it"),
+            detail={"source": ledger.source, "row": entry.row,
+                    "fields": dict(entry.fields)},
+        ))
+    for key in ledger.undeclared:
+        described = " ".join(f"{t}={v}" for t, v in zip(ledger.key_tokens, key))
+        out.append(Finding(
+            rule="delivered-never-declared",
+            severity=severities("delivered-never-declared", "warning"),
+            path=described, context=ledger.source,
+            message=(f"{described} was delivered but is not listed in "
+                     f"{ledger.source or 'the manifest'}"),
+            detail={"source": ledger.source, "files": ledger.delivered[key]},
+        ))
+    return out
+
+
+def manifests(spec) -> dict:
+    """The manifest sources declared in a convention, if any."""
+    return {name: source for name, source in (spec.sources or {}).items()
+            if source.get("type") == "manifest"}

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/shuttlecheck/delivery.py

@@ -9,7 +9,7 @@ The same engine as the naming rules, one level up: a written spec, a read-only
 check, and a report that names the exact field that is wrong.
 
 Media inspection goes through `Prober`. When no prober is available every media
-check reports as **skipped**, never as passed -- the rule that matters more here
+check reports as **skipped**, never as passed, the rule that matters more here
 than anywhere else in this tool, because a delivery report that silently checked
 nothing is worse than no report at all.
 """
@@ -429,7 +429,7 @@ def _check_audio(rule: dict, info: MediaInfo, rel, label, severity, fail):
 
 def check_sidecars(profile: dict, path: Path, root: Path, rel: str, *,
                    label: str) -> list[Finding]:
-    """Files that must arrive alongside the deliverable -- captions, checksums."""
+    """Files that must arrive alongside the deliverable: captions, checksums."""
     out: list[Finding] = []
     for entry in profile.get("sidecars") or []:
         pattern = entry.get("pattern", "")

{shuttlecheck-0.2.1 → shuttlecheck-0.3.0}/shuttlecheck/extensions.py

@@ -2,7 +2,7 @@
 
 The open package is the read-only half: work out a convention, check a tree
 against it, check a drive that arrived. Everything that writes to media --
-conforming, the journal, sealing, vendor packs -- ships separately and is
+conforming, the journal, sealing, vendor packs, ships separately and is
 licensed.
 
 The base package must never require the extension, must never import it at
@@ -42,7 +42,7 @@ def require(what: str):
         raise ExtensionMissing(
             f"{what} is part of ShuttleCheck Pro, which is not installed on this "
             f"machine.\n"
-            f"The open package covers init, validate, verify and the app -- "
+            f"The open package covers init, validate, verify and the app. "
             f"everything that only reads.\n"
             f"Conforming, sealing, the journal and vendor packs are licensed: "
             f"https://mvrk.systems/shuttlecheck")