mostlyright-data 0.31.8__tar.gz → 0.31.9__tar.gz

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.31.8
+Version: 0.31.9
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/skills/mr-data-build/references/commands.md

@@ -47,7 +47,7 @@ run in full.
 | `mr-data open` | Mint a single-use, short-lived address that continues this session in a browser at one dashboard path, and print it. `--open` hands it to the platform opener. The credential decides who mints: a device credential — what `mr-data login` stores — mints at Cloud, and a personal access token mints at Studio. |
 | `mr-data promote` | Record the cadence one table refreshes on and the reasoning behind it. A table's first succeeded run goes live on its own, so this is not what makes it readable; it is idempotent on a table that is already live, and it is how a withdrawn table is put back. The Harness worker does not perform the catch-up or scheduled refresh; report those only when another deployed component returns durable evidence that it did. |
 | `mr-data reschedule` | Change how often one live table refreshes, without the demote and promote that would reset its bookkeeping and buy a catch-up run. `--cadence` takes a cron expression or an interval such as `every 6h`; `--why` records the reasoning. It starts no run. `--lock` freezes the schedule so Studio stops adjusting it; `--unlock` lets it follow the source again. A table that cannot refresh answers `table_schedule_held` and exits 2, naming the two repairs: declare `rebuild`, or `table resync`. |
-| `mr-data table` | Report what one table is serving and how its refresh schedule is going: the state in plain words, the schedule in force, the next refresh, the schedule that was asked for and the reason given for it, and — once Studio has settled on one — the interval it measured and how many source updates that rests on. Where Studio reports it, it also prints the table's refresh strategy: "rebuilds every 7d; next T" or "held: blocker". `table rename TABLE --name NAME` writes a new name, lowercase snake_case and unique within its dataset, fenced on that dataset's version and retried once if somebody else wrote first; nothing is rebuilt, because every run, version and schedule hangs off the identifier, which does not move. `table archive TABLE --confirm-name NAME` retires one table you name by the name it is called, and frees that name; a live table is refused until you `demote` it, and nothing built is deleted. `table archive TABLE [TABLE ...] --confirm DELETE` retires as many tables as you name at once: `DELETE` is the literal word, typed exactly, and it confirms the identifiers you typed rather than a name, so each table is retired under the name Studio is holding for it. Every table is attempted whatever the one before it answered, one line is printed for each, and the command exits non-zero if any was not retired — so read the outcome of each rather than the exit code alone. `--confirm-name` is refused when you name more than one table. `table view TABLE` says whether a live table has a saved view (`view_saved`) and lists the view kinds (`table`, `cards`, `places`, `products`, `events`, `jobs`, `people`, `articles`) and the roles each binds. There is no map or chart kind, and `places` are location cards. `table view TABLE --set VIEW.json` saves `{"version": 1, "template": KIND, "fields": {ROLE: COLUMN}}`, with optional `facts` (up to 4 columns) and `imageStyle`. `--dry-run` has Cloud validate it against the live table and save nothing. The file is checked here before anything is sent. Replacing a saved view needs its revision as `--expected-revision`, which MCP `get_table` reports, because Cloud serves no CLI read of it. A stale revision is refused `THIN_TABLE_VIEW_VIEW_CHANGED`. The command uses the device key. |
+| `mr-data table` | Report what one table is serving and how its refresh schedule is going: the state in plain words, the schedule in force, the next refresh, the schedule that was asked for and the reason given for it, and — once Studio has settled on one — the interval it measured and how many source updates that rests on. Where Studio reports it, it also prints the table's refresh strategy: "rebuilds every 7d; next T" or "held: blocker". `table rename TABLE --name NAME` writes a new name, lowercase snake_case and unique within its dataset, fenced on that dataset's version and retried once if somebody else wrote first; nothing is rebuilt, because every run, version and schedule hangs off the identifier, which does not move. `table archive TABLE --confirm-name NAME` retires one table you name by the name it is called, and frees that name; a live table is refused until you `demote` it, and nothing built is deleted. `table archive TABLE [TABLE ...] --confirm DELETE` retires as many tables as you name at once: `DELETE` is the literal word, typed exactly, and it confirms the identifiers you typed rather than a name, so each table is retired under the name Studio is holding for it. Every table is attempted whatever the one before it answered, one line is printed for each, and the command exits non-zero if any was not retired — so read the outcome of each rather than the exit code alone. `--confirm-name` is refused when you name more than one table. `table view TABLE` says whether a live table has a saved view (`view_saved`) and lists the view kinds (`table`, `cards`, `places`, `products`, `events`, `jobs`, `people`, `articles`) and the roles each binds. There is no map or chart kind, and `places` are location cards. `table view TABLE --set VIEW.json` saves `{"version": 1, "template": KIND, "fields": {ROLE: COLUMN}}`, with optional `facts` (up to 4 columns) and `imageStyle`. `--dry-run` has Cloud validate it against the live table and save nothing. The file is checked here before anything is sent. Replacing a saved view needs its revision as `--expected-revision`, which MCP `get_table` reports, because Cloud serves no CLI read of it. A stale revision is refused `THIN_TABLE_VIEW_VIEW_CHANGED`. The command uses the device key. When the newest version is held from live, `table show` says `The newest version X is held: <reason>; serving Y`. `table accept-version TABLE --version VERSION_ID` releases that hold; live then derives from the newest unheld version. |
 | `mr-data pin` | Ask Studio to freeze one table pointer on an exact version, for a rollback or a hold. |
 | `mr-data unpin` | Ask Studio to resume pointer tracking; read the returned state before claiming it resumed. |
 | `mr-data demote` | Ask Studio to withdraw the pointer and any schedule it owns, for one table or for as many as you name: `demote TABLE [TABLE ...]` withdraws them one after another, attempts every one of them whatever the one before it answered, prints a line for each, and exits non-zero if any is still live. Pair it with `table archive` when you are retiring a set: a live table cannot be archived, so it is withdraw-then-archive, two commands rather than a loop. |

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/skills/mr-data-build/references/promote.md

@@ -83,6 +83,17 @@ or a hold, `mr-data unpin TABLE_ID` resumes tracking, and `mr-data demote TABLE_
 schedule and withdraws the live pointer. Read the resulting table state, but do not infer a
 schedule, catch-up, or current bytes from the command name or from route acceptance.
 
+**A succeeded run can be held from live.** Studio stamps every succeeded run with `verification`:
+`verified`, or `degraded` with its reasons. When the new version would replace a better one, it
+also stamps `live_hold`: the version is sealed and readable but not serving, and the table keeps
+the version it had. `completeness` means the new version is degraded and the old one is verified.
+`shrink` means it holds under half the old version's rows. `retention` means it sealed fewer rows
+than the recipe's `verification.min_retained_ratio` allows. `mr-data status`, `watch` and
+`receipt` say so, and `mr-data table show TABLE_ID` names the held version and the one serving.
+Never report a held version as live. Find out why it is smaller or less complete first. If it is
+right, `mr-data table accept-version TABLE_ID --version VERSION_ID` releases the hold. Live then
+derives from the newest unheld version, so a newer version that has advanced since stays live.
+
 To retire several tables, do it in two commands rather than a loop: `mr-data demote TABLE_ID
 TABLE_ID ...` withdraws them, then `mr-data table archive TABLE_ID TABLE_ID ... --confirm DELETE`
 retires them. `DELETE` is the literal word and nothing else is accepted; it confirms the

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/src/mostlyright/data_harness/thin/router.py

@@ -164,8 +164,8 @@ HOSTED_SUMMARY = dict(
         ("reschedule", "change how often one table refreshes, without starting a run"),
         (
             "table",
-            "report what one table is serving, explicitly resync it, rename it, or retire one or "
-            "many",
+            "report what one table is serving, explicitly resync it, rename it, accept a version "
+            "held from live, or retire one or many",
         ),
         ("pin", "ask Studio to pin the live pointer and report the returned state"),
         ("unpin", "ask Studio to remove a pin and report the returned pointer state"),

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/src/mostlyright/data_harness/thin/v4.py

@@ -277,9 +277,9 @@ RUN_EVENTS_PATH = "/v4/runs/{run_id}/events"
 #: bounded window of the run's log, with the sequence of the newest such event in that window. It
 #: reads the sealed record without mutating it or its ETag.
 #:
-#: ⚠ DECLARED WITH NO CALLER YET, on the same terms as :data:`LIST_TABLE_VERSIONS_PATH`. This
-#: client renders progress by following :data:`RUN_EVENTS_PATH` and folding the frames itself, so
-#: a command that wants one bounded snapshot rather than the stream lands against this constant.
+#: ⚠ DECLARED WITH NO CALLER YET. This client renders progress by following
+#: :data:`RUN_EVENTS_PATH` and folding the frames itself, so a command that wants one bounded
+#: snapshot rather than the stream lands against this constant.
 RUN_PROGRESS_PATH = "/v4/runs/{run_id}/progress"
 
 #: ``GET`` -- one row per attempt of a run, oldest first and at most 32: how it ended, when it was
@@ -462,15 +462,9 @@ ARCHIVE_TABLE_PATH = "/v4/tables/{table_id}:archive"
 #: ``POST`` -- retire one whole dataset, under its typed name, and release the name it held.
 ARCHIVE_DATASET_PATH = "/v4/datasets/{dataset_id}:archive"
 
-#: ``GET`` -- every version ``:pin`` accepts, newest first, with the digest each one sealed.
-#:
-#: ⚠ DECLARED WITH NO CALLER YET, WHICH IS A SCHEDULE RATHER THAN A DECISION. ``pin`` takes a
-#: version the caller names, and the set of nameable versions is server-side: a caller never holds
-#: a run, so without this route it had no way to enumerate what it was allowed to name and rollback
-#: was reachable only by guessing. Studio's 406-03 shipped it after the whole v4 command surface
-#: had been written, so the command that reads it lands later against exactly this constant --
-#: which is why it is here and not in :data:`UNUSED_STUDIO_PATHS`: "no command reaches it yet" is
-#: not a reason for never calling it.
+#: ``GET`` -- every version ``:pin`` accepts, newest first, with the digest each one sealed and a
+#: ``live_hold`` on each version Studio held from live. ``mr-data table show`` reads its newest row
+#: to say whether the newest version is held.
 LIST_TABLE_VERSIONS_PATH = "/v4/tables/{table_id}/versions"
 
 #: ``GET`` -- one table version's manifest, projected from Studio's own part mirror rather than
@@ -765,19 +759,15 @@ DECLARED_V4_PATHS: frozenset[str] = frozenset(
         # (Every line in this block narrates the promotion it sits on. What the pending set holds
         # TODAY is stated once, above `PENDING_V4_PATHS` itself, and today it holds nothing.)
         SESSION_HANDOFF_PATH,
-        # ⚠ AND ONE ROUTE NO COMMAND READS YET. See `LIST_TABLE_VERSIONS_PATH`: it is declared
-        # because the parity gate demands the whole public surface be accounted for, and because
-        # the reason it has no caller today is a schedule rather than a decision never to call it.
         LIST_TABLE_VERSIONS_PATH,
         # ⚠ AND THE THREE PARTITIONED-TABLE ROUTES, promoted out of `PENDING_V4_PATHS` once the
         # parity gate proved Studio had been serving them all along. They were recorded as pending
         # under the placeholder name `{version}`; the contract spells it `{table_version_id}`, so
         # the entries asserted the absence of paths that do not exist and were true for the wrong
         # reason. `mr-data parts` calls the manifest, and `mr-data download` calls the session
-        # minting for every part of a version its own run did not seal. The export is declared on
-        # the same terms as `LIST_TABLE_VERSIONS_PATH` above -- the parity gate accounts for the
-        # whole public surface, and its caller lands against exactly this constant -- not because
-        # anything calls it yet.
+        # minting for every part of a version its own run did not seal. The export is declared
+        # because the parity gate accounts for the whole public surface, and its caller lands
+        # against exactly this constant -- not because anything calls it yet.
         TABLE_VERSION_MANIFEST_PATH,
         TABLE_VERSION_PARTS_SESSIONS_PATH,
         TABLE_VERSION_EXPORT_PATH,

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/src/mostlyright/data_harness/thin/v4_artifacts.py

@@ -97,6 +97,7 @@ from mostlyright.data_harness.thin.v4_runs import (
     StudioV4RunClient,
     failure_from,
     failure_sentence,
+    verification_members,
 )
 from mostlyright.data_harness.thin.v4_stream import check_results, terminal_event
 from mostlyright.data_harness.ux.credentials import (
@@ -857,8 +858,9 @@ def _head(
     """The three facts every answer in this module opens with.
 
     The address is the run's Dataset page when the command read the run record, and the team's
-    Dataset listing when it did not: ``download``, ``receipt`` and ``peek`` read nothing but
-    artifacts, so they hold no Dataset identifier to name a page with.
+    Dataset listing when it did not: ``download`` and ``peek`` read nothing but artifacts, so they
+    hold no Dataset identifier to name a page with. ``receipt`` reads the run record and passes it
+    when the read succeeds.
     """
 
     return {
@@ -1670,14 +1672,21 @@ def receipt(
     # receipt's own `artifact_id` is on the payload, so which one was read is never a guess.
     row = _rows_of_kind(_listing(selected, run_id), ("receipt",), run_id=run_id)[0]
     document = _document(selected, run_id, row)
+    # The sealed receipt says how the run was built; whether Studio found its version complete,
+    # and whether that version is serving, is on the run record. A refused read leaves it out.
+    try:
+        record: Mapping[str, Any] | None = selected.run(run_id)
+    except ThinLaneError:
+        record = None
     return {
         "schema_version": RECEIPT_SCHEMA,
         "status": "receipt_reported",
-        **_head(selected.session, run_id),
+        **_head(selected.session, run_id, record),
         "artifact_id": _artifact_id(row),
         "digest": row.get("digest"),
         "size_bytes": row.get("size_bytes"),
         "receipt": document,
+        **(verification_members(record) if record is not None else {}),
     }
 
 

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/src/mostlyright/data_harness/thin/v4_runs.py

@@ -2739,6 +2739,7 @@ def status(args: argparse.Namespace, *, client: StudioV4RunClient | None = None)
     )
     if delivered is not None:
         payload["delivery"] = delivered
+    payload.update(verification_members(run))
     if diagnosing:
         # Imported here because the diagnosis reads this module's triple and room-fault reading,
         # and a module-level import in both directions is a cycle.
@@ -2860,6 +2861,144 @@ def status_exit_code(payload: Mapping[str, Any]) -> int:
     return 2 if payload.get("run", {}).get("status") == "failed" else 0
 
 
+# --------------------------------------------------------------------------------------------
+# Verification and the live hold
+# --------------------------------------------------------------------------------------------
+
+
+#: The run-record members Studio stamps on a succeeded run about whether its version is complete
+#: and whether it became live. Carried verbatim; absent on a record written before they existed.
+VERIFICATION_MEMBERS: tuple[str, ...] = ("verification", "live_hold", "live_hold_accepted")
+
+#: What each ``verification.reasons`` token means, in the order Studio lists them.
+DEGRADED_REASONS: dict[str, str] = {
+    "ROW_CLAMP_TRUNCATED": "a row clamp cut the table short",
+    "DISCOVERY_INCOMPLETE": "a page collection's listing stopped before its end",
+    "DISCOVERY_FAILED": "a page collection's listing was refused",
+    "PAGES_FAILED": "pages whose last fetch failed are missing",
+    "SOURCE_COUNT_SHORT": "a source delivered fewer rows or pages than it states it holds",
+}
+
+#: What each ``live_hold.reason`` means for the version it is stamped on.
+LIVE_HOLD_REASONS: dict[str, str] = {
+    "completeness": "it is degraded and the version it would have replaced is verified",
+    "shrink": "it holds fewer than half the rows of the version it would have replaced",
+    "retention": (
+        "it sealed a smaller share of the rows its sources supplied than the recipe's "
+        "verification.min_retained_ratio allows"
+    ),
+}
+
+
+def accept_version_command(table_id: Any, version_id: Any) -> str:
+    """The command that releases one held version."""
+
+    return shlex.join(
+        [
+            "mr-data",
+            "table",
+            "accept-version",
+            str(table_id or "TABLE_ID"),
+            "--version",
+            str(version_id or "VERSION_ID"),
+        ]
+    )
+
+
+def live_hold_reason(reason: Any) -> str:
+    """One hold reason in words, or the token itself when this client does not know it."""
+
+    if reason is None:
+        return "Studio held it without naming a reason"
+    return LIVE_HOLD_REASONS.get(
+        str(reason), f"Studio held it for a reason this client does not know ({reason})"
+    )
+
+
+def _tokens(value: Any) -> list[str]:
+    """The string members of a list, or nothing when the member is not a list."""
+
+    if not isinstance(value, list):
+        return []
+    return [item for item in value if isinstance(item, str) and item]
+
+
+def verification_sentence(verification: Mapping[str, Any]) -> str:
+    """``verified`` or ``degraded`` with the reasons, and any warning, as one sentence."""
+
+    state = verification.get("status")
+    reasons = _tokens(verification.get("reasons"))
+    if state == "degraded":
+        said = "Degraded: the run succeeded but the version is short of what its sources hold"
+        if reasons:
+            said += (
+                " ("
+                + "; ".join(
+                    f"{reason}: {DEGRADED_REASONS.get(reason, reason)}" for reason in reasons
+                )
+                + ")"
+            )
+        said += "."
+    elif state == "verified":
+        said = "Verified: Studio found nothing missing from this version."
+    else:
+        said = f"Studio reports verification {state!r}."
+    sealed, acquired = verification.get("sealed_rows"), verification.get("acquired_rows")
+    for warning in _tokens(verification.get("warnings")):
+        if warning == "SEALED_BELOW_ACQUIRED":
+            said += " Warning SEALED_BELOW_ACQUIRED: the table holds under a tenth of the rows "
+            said += "its sources supplied"
+            if isinstance(sealed, int) and isinstance(acquired, int):
+                said += f" ({sealed:,} of {acquired:,})"
+            said += "."
+        else:
+            said += f" Warning {warning}."
+    return said
+
+
+def live_hold_sentence(run: Mapping[str, Any]) -> str | None:
+    """What a held version means for the table, and the command that releases it."""
+
+    hold = run.get("live_hold")
+    if isinstance(hold, Mapping):
+        version = run.get("table_version_id")
+        named = f"Version {version}" if version else "This version"
+        return (
+            f"{named} is sealed but not serving: Studio held it from live because "
+            f"{live_hold_reason(hold.get('reason'))}. Live kept version "
+            f"{hold.get('against_version_id')}, the one it would have replaced; "
+            f"mr-data table show {run.get('table_id') or 'TABLE_ID'} says what the table serves "
+            f"now. If this version is right, accept it with "
+            f"{accept_version_command(run.get('table_id'), version)}."
+        )
+    accepted = run.get("live_hold_accepted")
+    if isinstance(accepted, Mapping):
+        return (
+            f"This version was held from live ({accepted.get('reason')}) and accepted at "
+            f"{accepted.get('accepted_at')}."
+        )
+    return None
+
+
+def verification_members(run: Mapping[str, Any]) -> dict[str, Any]:
+    """The verification and live-hold members a run receipt carries, or nothing.
+
+    The record's own members travel verbatim; the two sentences are what a person reads. A record
+    written before Studio stamped them yields an empty mapping, so older runs print as before.
+    """
+
+    members: dict[str, Any] = {
+        member: run[member] for member in VERIFICATION_MEMBERS if isinstance(run.get(member), dict)
+    }
+    verification = members.get("verification")
+    if isinstance(verification, Mapping):
+        members["verification_said"] = verification_sentence(verification)
+    held = live_hold_sentence(run)
+    if held is not None:
+        members["live_hold_said"] = held
+    return members
+
+
 # --------------------------------------------------------------------------------------------
 # Listing them
 # --------------------------------------------------------------------------------------------
@@ -3176,11 +3315,13 @@ def _matches(
 __all__ = [
     "CONFIRM_REQUIRED_CODE",
     "DEFAULT_RUN_LIMIT",
+    "DEGRADED_REASONS",
     "DISCARD_HISTORY_FLAG",
     "FAILURE_MEMBERS",
     "FRESH_READ_CONTROL",
     "GRAMMAR_NOT_SERVED_CODE",
     "LIST_PAGE_SIZE",
+    "LIVE_HOLD_REASONS",
     "MAX_CLAMP_BYTES",
     "MAX_PAGE_HALVINGS",
     "MAX_RUN_PAGES",
@@ -3202,9 +3343,11 @@ __all__ = [
     "STEP_UP_CODES",
     "TERMINAL_RUN_STATUSES",
     "TEST_STOPPED_STATUSES",
+    "VERIFICATION_MEMBERS",
     "WINDOW_MEMBERS",
     "AdaptivePage",
     "StudioV4RunClient",
+    "accept_version_command",
     "approve_full",
     "confirm_held",
     "confirm_held_command",
@@ -3219,6 +3362,8 @@ __all__ = [
     "fresh_full_record",
     "is_room_fault",
     "listing_page_size",
+    "live_hold_reason",
+    "live_hold_sentence",
     "pages_sentence",
     "registered_digest",
     "release_full_command",
@@ -3236,4 +3381,6 @@ __all__ = [
     "status",
     "status_exit_code",
     "table_capabilities",
+    "verification_members",
+    "verification_sentence",
 ]

{mostlyright_data-0.31.8 → mostlyright_data-0.31.9}/src/mostlyright/data_harness/thin/v4_stream.py

@@ -62,6 +62,7 @@ from mostlyright.data_harness.thin.v4_runs import (
     _preflight_run,
     failure_from,
     failure_sentence,
+    verification_members,
 )
 from mostlyright.data_harness.ux.credentials import resolve_cloud_credentials
 
@@ -246,11 +247,12 @@ def watch(
     live view is that it is live -- and the returned payload is the summary, so ``--json`` gets one
     object and a person gets a timeline followed by that object.
 
-    ⚠ THE RUN RECORD IS READ ONCE, BEFORE THE TIMELINE, AND NOT AFTERWARDS. It is read first
-    because it names the Dataset whose page this run is followed on. The terminal event says what
-    happened and carries the whole failure triple, so a second GET would be asking a second
-    authority the question the stream just answered -- and the two can disagree by exactly the
-    width of a race.
+    ⚠ THE RUN RECORD IS READ BEFORE THE TIMELINE, AND AFTERWARDS ONLY FOR A SUCCESS. It is read
+    first because it names the Dataset whose page this run is followed on. The terminal event says
+    what happened and carries the whole failure triple, so a failed run is not read again: a second
+    GET would be asking a second authority the question the stream just answered. A succeeded run
+    is read once more for its ``verification`` and ``live_hold``, which the terminal event does
+    not carry.
 
     ⚠ THE STREAM READS THROUGH ``sessions`` AND THE ONE RECORD READ DOES NOT NEED TO. The record
     is read at once, on the token just minted; the stream is what outlives it, so the stream is
@@ -341,9 +343,28 @@ def watch(
     )
     if delivered is not None:
         payload["delivery"] = delivered
+    if terminal.get("event_type") == "run_succeeded":
+        payload.update(_succeeded_verification(current, selected, run_id))
     return payload
 
 
+def _succeeded_verification(
+    current: StudioSession, selected: StudioV4RunClient, run_id: str
+) -> dict[str, Any]:
+    """The verification and live-hold members of a run that has just succeeded, or nothing.
+
+    The terminal event does not carry them: Studio stamps them on the run record in the same
+    write, so the record is read once more after ``run_succeeded``. A refused read leaves them
+    out rather than failing a watch whose run did succeed.
+    """
+
+    try:
+        record = StudioV4RunClient(current, transport=selected.transport).run(run_id)
+    except ThinLaneError:
+        return {}
+    return verification_members(record) if isinstance(record, Mapping) else {}
+
+
 def watch_many(
     args: argparse.Namespace,
     *,