mostlyright-data 0.31.13__tar.gz → 0.31.14__tar.gz

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.31.13
+Version: 0.31.14
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/skills/mr-data-build/references/commands.md

@@ -17,7 +17,7 @@ run in full.
 | `mr-data clarify` | Check whether clarification is allowed. Use `--attended` for an active user chat and `--unattended` for a scheduled invocation; an explicit operator veto such as `MOSTLYRIGHT_ATTENDED=0` still wins. A non-TTY subprocess does not prove the user is absent. The command does not ask the person in chat or wait for a reply; the agent must do that. It exits non-zero when nobody can answer or the recipe is already registered. |
 | `mr-data probe` | `probe SOURCE_ID QUESTION_ID --dataset DATASET_ID [--kind source_inspect\|sample_rows\|profile_columns\|evaluate_expression]` asks one already-registered source one question and prints the answer. **It rides the frozen `/v3/sessions` routes, which a deployment may have switched off, so do not plan a source inspection around it.** Both positionals are identifiers. `QUESTION_ID` is a question identifier, which no v4 registration receipt returns. The v4 way to look at a source before registering it is `mr-data source inspect`, in the next row. The other way to read one is a bounded run of a scratch table, never of the real table, as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. |
 | `mr-data source` | `source inspect` takes a bounded look at one source, and it is the v4 path where `probe` is not. [Stage 3](3-probe-read-a-source-before-committing-to-it.md) says when to use it. It starts one of three ways, and each prints the `session_id`, `probe_id` and `source_id`, plus the `dataset_id` and `question_id` when it has them. For a new public HTTPS CSV that needs no key, give `--dataset-id`, `--question`, `--source-name`, `--source-url`, `--claimed-basis` and `--claim-evidence-digest`: it prepares the source and opens a session. For a source already prepared, give `--dataset-id`, `--question-id` and `--source-id`, which opens a session. In a session you already hold, give `--session-id` with `--source-id`. `--context-source-id` adds a source to a session it opens. `--session-id` with `--probe-id` reads the answer back. A look is bounded to 65,536 bytes, 1,000 rows and 300 seconds. `queued` and `truncated` are not complete answers, and `capability_unavailable` means this deployment cannot inspect, so read the source by a bounded run of a scratch table instead, as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. What it finds is exploratory evidence, not a run and not a receipt. |
-| `mr-data reader` | `reader normalize --family-id ID --family-version VERSION --options-file FILE [--workspace-id ID]` asks Studio's Reader service to apply one certified family's own defaults and option rules to a JSON object of options, at most 64 KiB, and prints the normalized `decode_options` and `decode_options_json`. The second is the text a recipe carries as `reader.decode_options`, pinned as [Readers](readers.md) describes. It changes nothing, and this client does not validate or invent a family's options: Studio decides. Run it for every Reader pin before you register, so a wrong option is refused here and not inside a run. |
+| `mr-data reader` | `reader normalize --family-id ID --family-version VERSION --options-file FILE [--workspace-id ID]` asks Studio's Reader service to apply one certified family's own defaults and option rules to a JSON object of options, at most 64 KiB, and prints the normalized `decode_options`, `decode_options_json` and `options_digest`. The second is the text a recipe carries as `reader.decode_options`, and the third is the `reader.options_digest` registration accepts for exactly those options, pinned as [Readers](readers.md) describes. It changes nothing, and this client does not validate or invent a family's options: Studio decides. Run it for every Reader pin before you register, so a wrong option is refused here and not inside a run. |
 | `mr-data catalog` | `catalog search "QUESTION"` asks the sealed public-source catalogue which of its entries might answer a question and ranks them best first. Every ranked entry comes back with the disposition its own facts earned — `admitted`, `human_escalation_required` or `refused` — because an entry the catalogue could not vouch for is still a finding. `--format csv` states the one data format the question requires — one token per question, never repeated — and it is not a filter: an entry that does not declare that format still comes back ranked, with disposition `refused` and `filters_match` false, so nothing is held back; `--limit N` ranks at most N, up to 25. It fetches nothing and registers nothing. The catalogue holds one provider (Data.gov) and only part of it, so it is never exhaustive and an empty answer is not evidence that no such source exists. When the answer is `catalog_unavailable`, this deployment has no catalogue to search: record the lane and carry on. |
 | `mr-data public` | Read Mostly Right's public dataset catalogue. `public search "QUERY"` ranks published datasets (`--topic`, `--limit` and `--cursor` narrow and page it), `show SLUG` reads one overview, `tables SLUG` lists its table identifiers, `schema TABLE_ID` reads one table's schema, `sample TABLE_ID` prints its published preview rows, `lookup "QUERY"` is the connector search shape and `fetch SLUG` returns the dataset's Markdown document. Those seven need no login. `query TABLE_ID`, with `--body` or `--body-file`, and `download-url TABLE_ID` do need `mr-data login`: they ask Cloud, with the device key, to run one query over a public table and to say how it can be downloaded. Preview rows are not a query, and no table bytes pass through the command. |
 | `mr-data connected` | `connected list` shows the public datasets connected to this workspace and `connected add SLUG` connects one by its slug. It needs `mr-data login`: both ask Cloud, with the device key, over the routes MCP's `list_connected_datasets` and `connect_dataset` use. |

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/skills/mr-data-build/references/patterns.md

@@ -225,8 +225,9 @@ for the days that can still change, through the two request `parameters` it rend
 unit reads its own page count, so size `limits.max_requests` from a measured unit.
 [Paging a keyed API in parallel](sources.md#paging-a-keyed-api-in-parallel) has the rules.
 
-`options_digest` is the SHA-256 of the canonical JSON of `decode_options`. Registration names it
-when the two disagree.
+`options_digest` is the SHA-256 of the canonical JSON of `decode_options`, and
+`mr-data reader normalize` prints it for the options it returns. Registration names the digest it
+expected when the two disagree.
 
 ```json
 {

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/skills/mr-data-build/references/readers.md

@@ -57,9 +57,17 @@ mr-data reader normalize --family-id json.tabular --family-version 1.1.0 \
 
 `options.json` holds the keys the source needs, at most 64 KiB. Studio applies the family's own
 defaults and option rules. The answer's `decode_options_json` is the text to write as
-`reader.decode_options`. A refusal arrives here, before any run is spent. Run it for every
+`reader.decode_options`, and its `options_digest` is the `reader.options_digest` to pin beside it.
+That digest binds the default-filled options the answer returns, not the subset in `options.json`,
+so copy both together. A refusal arrives here, before any run is spent. Run it for every
 Reader pin before you register.
 
+`normalize` checks the options against the family, not against the place the pin goes. A
+Generic API `generic_api.response.reader` admits fewer coordinates: `json.tabular` `1.0.0`,
+`1.1.0` or `1.2.0`, `delimited_text` `1.0.0` or `1.1.0`, and `parquet.tabular` `1.0.0`.
+`json.tabular` `1.3.0` and `1.4.0` normalize but are refused there. The highest-version advice
+below applies inside that set.
+
 A column that may be absent from a record must state `"required": false`. Publishers routinely omit
 a key rather than sending null, and a required column that is missing refuses the record.
 
@@ -87,6 +95,7 @@ decoder. Zero matches and multiple matches both refuse.
 | `archive.zip@1.0.0` | `member`; `member_format` (`csv` or `tsv`); optional `header: true`. |
 | `archive.zip@1.1.0` | Same, plus `member_format: semicolon` for publishers such as DWD. |
 | `archive.zip@1.2.0` | Same as `archive.zip@1.1.0`; also accepts a response served as `application/octet-stream`. |
+| `archive.zip@4.0.0` | Exact safe `member`, `member_format: lines` and `encoding: utf-8 or cp1252`; no header. Streams the member's physical lines as line_number, raw_line, as `archive.tar@2.0.0` does for one member. |
 | `archive.gzip@1.0.0` | `member_format`; optional `header: true`. `member` is invalid. |
 | `archive.gzip@1.1.0` | Same; also accepts a response served as `application/octet-stream`. |
 | `archive.tar@1.0.0` | Same as `archive.zip@1.0.0`. |
@@ -181,7 +190,8 @@ Use `public.https.api@1.0.0` only when `keyless_window` is literal `true`. Other
 The new adapter uses the existing `generic_api.response` Reader contract. For Binance klines,
 state `format: json`, `records_pointer: ""`, and `reader.family_id: json.tabular` with
 `family_version: 1.0.0` (or the already admitted `1.1.0`). Pin `options_digest` to the SHA-256
-of the canonical decode options, rather than inserting a placeholder digest. These decode
+of the canonical decode options, the `options_digest` `mr-data reader normalize` prints, rather
+than inserting a placeholder digest. These decode
 options project a root JSON array of arrays, with RFC 6901 numeric indexes into each row:
 
 ```json

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/skills/mr-data-build/references/sources.md

@@ -335,6 +335,10 @@ to separate them. No per-row collection may derive its rows from a `closed` or `
 `pagination` is a `page` or `offset` walk with a required `parallel` member, in one of two forms.
 Prefer the counted form.
 
+`@2.1.0` refuses `{"kind": "none", "max_pages": 1}`, and `cursor` and `link` walks too. For one
+request, use `@2.0.0`. Use `authenticated.https.connector@3.0.0` instead when the request
+declares a `per_call` cost, which `@2.0.0` does not admit.
+
 | Form | Members | Use it when |
 | --- | --- | --- |
 | counted | `total_pages_pointer`, `max_page_bytes`, `min_interval_milliseconds`, optional `concurrency` | the first page states how many pages exist. A `page` walk over a `json` response only |

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/recipe.py

@@ -170,12 +170,12 @@ FILE_DECODER_COORDINATES: frozenset[tuple[str, str]] = frozenset(
     for family, versions in {
         "archive.gzip": ("1.0.0", "1.1.0", "2.0.0", "2.1.0"),
         "archive.tar": ("1.0.0", "1.1.0", "3.0.0", "4.0.0"),
-        "archive.zip": ("1.0.0", "1.1.0", "1.2.0", "1.3.0", "1.4.0", "1.5.0", "3.0.0"),
+        "archive.zip": ("1.0.0", "1.1.0", "1.2.0", "1.3.0", "1.4.0", "1.5.0", "3.0.0", "4.0.0"),
         "delimited_text": ("1.0.0", "1.1.0", "2.0.0"),
         "geoparquet.vector": ("1.0.0",),
         "json.tabular": ("1.1.0", "1.2.0"),
         "meteorology.bufr": ("1.0.0",),
-        "meteorology.grib": ("1.0.0",),
+        "meteorology.grib": ("1.0.0", "2.0.0"),
         "microdata.sas_xpt": ("1.0.0",),
         "microdata.spss_sav": ("1.0.0",),
         "microdata.stata_dta": ("1.0.0",),

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/session.py

@@ -587,6 +587,19 @@ def sessions_for(session: StudioSession | StudioSessionProvider) -> StudioSessio
 #: a sentence with spaces in it.
 CLOUD_REFUSAL_CODE = re.compile(r"[A-Z][A-Z0-9_]{0,63}")
 
+#: Cloud's refusal of a web chat sandbox key on a route that would change something
+#: (Cloud ``lib/sandbox-key-refusal.ts``). The key is valid and reads; every change it
+#: would make goes through the chat's own approval-gated tools or a person on the dashboard.
+#: Folded onto ``THIN_AUTHORIZATION_DENIED`` it told an agent in the web chat to run
+#: ``mr-data login``, which cannot help, so it keeps its own code -- and Cloud's sentence,
+#: which names the chat tool that does the job.
+SANDBOX_KEY_READ_ONLY_CODE = "SANDBOX_KEY_READ_ONLY"
+THIN_SANDBOX_KEY_READ_ONLY = f"THIN_CLOUD_{SANDBOX_KEY_READ_ONLY_CODE}"
+SANDBOX_KEY_READ_ONLY_SENTENCE = (
+    "This is a web chat key: it reads, and changes go through the chat's own tools or a "
+    "person on the dashboard; signing in again will not change that"
+)
+
 #: The family of Studio's refusals of a call that would skip Cloud's usage meter
 #: (``DATA_EGRESS_THROUGH_CLOUD``). From a Cloud route it means Studio refused CLOUD's call, which
 #: is Mostly Right's misconfiguration and never the caller's key.
@@ -844,6 +857,18 @@ def cloud_refusal(
             "this computer's or the device key's; nothing here needs to change. Run the command "
             "again later",
         )
+    elif status == 403 and named == SANDBOX_KEY_READ_ONLY_CODE:
+        # NOT A MISSING OR EXPIRED KEY, so not the shared authorization code: the chat's
+        # key is valid and only this change is not its to make. Cloud's sentence names the
+        # tool that makes it, so it is quoted (bounded, printable, attributed) even though a
+        # code was named, which no other named refusal here does.
+        stated_alternative = _cloud_sentence(parsed.get("error"))
+        refusal = ThinLaneError(
+            THIN_SANDBOX_KEY_READ_ONLY,
+            f"{SANDBOX_KEY_READ_ONLY_SENTENCE}. Cloud said: {stated_alternative}"
+            if stated_alternative
+            else SANDBOX_KEY_READ_ONLY_SENTENCE,
+        )
     elif status in {401, 403}:
         if named:
             suffix = f" (Cloud said {named}, HTTP {status})"

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/support.py

@@ -31,7 +31,11 @@ from typing import Any
 
 from mostlyright.data_harness.thin import THIN_SCHEMA_PREFIX
 from mostlyright.data_harness.thin.backoff import jittered
-from mostlyright.data_harness.thin.session import CloudClient, require_device_key
+from mostlyright.data_harness.thin.session import (
+    THIN_SANDBOX_KEY_READ_ONLY,
+    CloudClient,
+    require_device_key,
+)
 from mostlyright.data_harness.thin.transport import ThinLaneError, UrlLibTransport
 from mostlyright.data_harness.ux.credentials import resolve_cloud_credentials
 
@@ -61,6 +65,14 @@ SETTLED = frozenset({"answered", "budget_exhausted", "failed"})
 SETTLED_RECEIPTS = frozenset(f"support_{state}" for state in SETTLED)
 STATUS_VERB = "status"
 
+#: What a web chat key is told when Cloud refuses it this route. Said here as well as by
+#: Cloud, so an agent in the chat learns the tool even if Cloud's sentence did not survive
+#: the bound on quoted text.
+CHAT_SUPPORT_TOOLS = (
+    "In the web chat, ask with the chat's ask_support tool, then read the answer with "
+    "get_support_answer"
+)
+
 
 def declare_arguments(parser: argparse.ArgumentParser) -> None:
     """Add `ask`'s arguments to the shared hosted parser."""
@@ -106,17 +118,29 @@ def declare_arguments(parser: argparse.ArgumentParser) -> None:
     )
 
 
+def _name_chat_tools(failure: ThinLaneError) -> ThinLaneError:
+    """A web chat key refused here is pointed at the chat's own support tools."""
+
+    if failure.code == THIN_SANDBOX_KEY_READ_ONLY and "ask_support" not in failure.detail:
+        failure.detail = f"{failure.detail}. {CHAT_SUPPORT_TOOLS}"
+        failure.args = (f"{failure.code}: {failure.detail}",)
+    return failure
+
+
 class SupportClient(CloudClient):
     """The two support routes over the shared device-key Cloud client."""
 
     def submit(self, body: Mapping[str, Any]) -> dict[str, Any]:
-        return self.call(
-            "POST",
-            CLOUD_SUPPORT_QUESTIONS_PATH,
-            act="ask the support agent a question",
-            body=body,
-            expected=(202,),
-        )
+        try:
+            return self.call(
+                "POST",
+                CLOUD_SUPPORT_QUESTIONS_PATH,
+                act="ask the support agent a question",
+                body=body,
+                expected=(202,),
+            )
+        except ThinLaneError as failure:
+            raise _name_chat_tools(failure) from None
 
     def read(self, question_id: str, *, run: bool) -> dict[str, Any]:
         suffix = "" if run else "?wait=0"

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/table_search_contract.json

@@ -1,6 +1,6 @@
 {
  "schema_version": "mostlyright-thin-table-search-contract.v1",
- "source_commit": "5109da1f92eaaf4fc41ed35a03e5d010268d7c7b",
+ "source_commit": "b296c21484f40cce334e1b7c0f6c7c03b44a41e9",
  "recipe_search_definition": "recipe_search",
  "$defs": {
   "common.safe_integer": {
@@ -234,6 +234,12 @@
          "asc",
          "desc"
         ]
+       },
+       "quality": {
+        "description": "Optional owner quality note (4.55.0). A string of 1 to 200 characters. Shown on the order question only when this phrase occurs in the query. It does not change the sort. A boolean is invalid.",
+        "type": "string",
+        "minLength": 1,
+        "maxLength": 200
        }
       },
       "required": [
@@ -242,7 +248,7 @@
        "direction"
       ]
      },
-     "description": "Order phrases (4.45.0: at most 256), one package order key each, under the aliases' shared byte budget."
+     "description": "Order phrases (4.45.0: at most 256), one package order key each, under the aliases' shared byte budget. Optional quality (4.55.0) is the owner's note for that phrase."
     },
     "default_order": {
      "description": "The order of a search with nothing to rank by (4.44.0): a browse or an entity plan with no order of its own walks the rows in this order instead of the package's row order (popularity). Mojirama's `known` (vote count descending) for a film table; the column must be sortable (declared sortable, a sort alias's, or the popularity or votes role).",

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4.py

@@ -149,8 +149,8 @@ from mostlyright.data_harness.thin.v4_matching import CONTRACT_PATHS as MATCHING
 #: ``JOB_INVALID`` outright, which in production was every refresh a collection epoch was offered
 #: for rather than only the collection runs. This package must not reach a Studio older than the
 #: commit it pins; ``docs/V4-WORKER-PROTOCOL.md`` states it beside the layout's own ordering rule.
-PINNED_V4_OPENAPI_SOURCE_SHA256 = "e7bd4e994b32d62ae00b4d7fc2f8936e4964cd4160b7aa30ae010528ff57b1a8"
-PINNED_V4_CONTRACT_VERSION = "4.54.0"
+PINNED_V4_OPENAPI_SOURCE_SHA256 = "b378d4f70d580aed094b00edeec5acc39cae5c79f19dc4e869fe40304fccf654"
+PINNED_V4_CONTRACT_VERSION = "4.57.0"
 
 #: The ``schema_version`` every command body that predates 4.14.0 stamps: ``dataset activity``,
 #: ``table rename``, ``table archive`` and ``dataset archive``. It is the number the 0.25 line

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4_datasets.py

@@ -136,6 +136,7 @@ from mostlyright.data_harness.thin.session import (
     CLOUD_REFUSAL_CODE,
     MAX_JSON_RESPONSE_BYTES,
     MAX_STATED_MESSAGE_CHARS,
+    THIN_SANDBOX_KEY_READ_ONLY,
     CloudClient,
     StudioSession,
     StudioSessionProvider,
@@ -1615,18 +1616,30 @@ def _sync_written_identity(dataset_id: str, cloud_url: str) -> dict[str, Any]:
             )
         return {"public_projection_synced": True}
     except (ThinLaneError, LoginError, OSError) as error:
-        return {
+        stated = isinstance(error, ThinLaneError) and getattr(error, "http_status", None) != 404
+        result: dict[str, Any] = {
             "public_projection_synced": False,
-            "public_projection_error": (
-                error.code
-                if isinstance(error, ThinLaneError) and getattr(error, "http_status", None) != 404
-                else "THIN_IDENTITY_SYNC_UNAVAILABLE"
-            ),
+            "public_projection_error": (error.code if stated else "THIN_IDENTITY_SYNC_UNAVAILABLE"),
             "public_projection_note": (
                 "The dataset was saved, but its public page could not be updated. "
                 f"Retry only the sync with: mr-data dataset sync {dataset_id}"
             ),
         }
+        # ⚠ THE CODE ALONE SENT PEOPLE TO THE WRONG FIX. Every refusal on this path left
+        # as a bare code, so a key Cloud would never let update the page and an expired one
+        # read the same. The refusal's own sentence -- this lane's words, with Cloud's
+        # quoted inside them already bounded and printable by ``cloud_refusal`` -- rides
+        # along. Nothing branches on it; the code beside it is what a script reads.
+        if stated:
+            result["public_projection_message"] = error.detail
+        if stated and error.code == THIN_SANDBOX_KEY_READ_ONLY:
+            # Retrying with this key is refused the same way: the chat does it.
+            result["public_projection_note"] = (
+                "The dataset was saved, but this web chat key cannot update its public "
+                "page. Use the chat's update_dataset tool with sync_only: true, or a "
+                "person syncs it from the dashboard."
+            )
+        return result
 
 
 def _held(record: Mapping[str, Any], member: str) -> Any:

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4_query.py

@@ -43,6 +43,7 @@ import os
 import re
 import time
 from collections.abc import Callable, Mapping
+from datetime import datetime
 from http.client import HTTPException
 from pathlib import Path
 from typing import Any
@@ -124,6 +125,20 @@ MAX_QUERY_WAIT_SECONDS = 120.0
 QUERY_POLL_INITIAL_SECONDS = 0.2
 QUERY_POLL_MAX_SECONDS = 2.0
 
+#: How long past the query's OWN deadline this command keeps reading before it stops.
+#:
+#: ⚠ THE SERVER'S DEADLINE BOUNDS THE WAIT TOO, NOT ONLY THIS CLIENT'S PATIENCE. Studio's deadline
+#: (``deadline_at - submitted_at``, thirty seconds by default) is shorter than the two ceilings
+#: above, so a query no worker took used to be read back for over a minute after Studio had stopped
+#: caring about it, and then reported as merely "still running". The window is measured on this
+#: computer's monotonic clock from the first read, using the server's own two stamps for its
+#: length, so a skewed wall clock here can neither cut it short nor stretch it. The grace covers
+#: a settle that lands on the deadline itself.
+QUERY_DEADLINE_GRACE_SECONDS = 5.0
+
+#: The refusal a query gets when Studio still reports it unsettled after its own deadline.
+QUERY_DEADLINE_PASSED_CODE = "THIN_QUERY_DEADLINE_PASSED"
+
 #: The submit body's bound, in canonical-JSON bytes, and the same number Studio enforces. Checked
 #: here so an oversized submission is a sentence at the terminal rather than a 422 after a token
 #: has been minted -- the reason every early refusal in this repository exists.
@@ -1071,7 +1086,7 @@ def _settled(
     query_id = _record_identifier(record, run_id)
     checkpoint_content_digest = _checkpoint_binding(record, checkpoint_id)
     try:
-        record, read = _poll(
+        record, read, overdue = _poll(
             selected,
             run_id,
             query_id,
@@ -1082,6 +1097,8 @@ def _settled(
         )
     except ThinLaneError as refusal:
         raise _while_reading_back(refusal, query_id) from None
+    if overdue and record.get("state") not in TERMINAL_QUERY_STATES:
+        raise _deadline_passed(record, query_id, read)
     head: dict[str, Any] = {
         "schema_version": QUERY_SCHEMA,
         "lane": "hosted",
@@ -1134,6 +1151,58 @@ def _settled(
     return payload
 
 
+def _server_window_seconds(record: Mapping[str, Any]) -> float | None:
+    """The deadline Studio gave this query, in seconds after its submission, or ``None``.
+
+    Read from the record's own two stamps rather than compared with this computer's wall clock,
+    which can be minutes off from Studio's.
+    """
+
+    submitted = record.get("submitted_at")
+    deadline = record.get("deadline_at")
+    if not isinstance(submitted, str) or not isinstance(deadline, str):
+        return None
+    try:
+        ends = datetime.fromisoformat(deadline.replace("Z", "+00:00"))
+        began = datetime.fromisoformat(submitted.replace("Z", "+00:00"))
+    except ValueError:
+        return None
+    # A stamp without an offset is not an instant, and subtracting a naive one from an aware one
+    # raises TypeError: either way the record states no window, and the old ceilings decide.
+    if ends.tzinfo is None or began.tzinfo is None:
+        return None
+    try:
+        window = (ends - began).total_seconds()
+    except (TypeError, OverflowError):
+        return None
+    return window if window >= 0 else None
+
+
+def _deadline_passed(record: Mapping[str, Any], query_id: str, read: int) -> ThinLaneError:
+    """A query Studio still calls unsettled after its own deadline, as the terminal refusal it is.
+
+    Not "still running": past ``deadline_at`` nothing will answer this query, and an agent told to
+    wait would wait for nothing. What it can do is said: run the command again, which queues the
+    same question afresh because a query past its deadline is re-run rather than replayed, and, if
+    that also stalls, read the table without a worker.
+    """
+
+    state = record.get("state")
+    refusal = ThinLaneError(
+        QUERY_DEADLINE_PASSED_CODE,
+        f"query {query_id} was still {state} {int(QUERY_DEADLINE_GRACE_SECONDS)} seconds after "
+        f"its deadline {record.get('deadline_at')} ({read} reads), so no worker answered it in "
+        "time and nothing was returned. Running this same command again queues the question "
+        "afresh under the same identifier. If it stalls again, no session worker is taking this "
+        "workspace's queries: mr-data download "
+        f"{record.get('run_id') or 'RUN'} brings the table here to query locally",
+    )
+    refusal.query_id = query_id  # type: ignore[attr-defined]
+    refusal.deadline_at = record.get("deadline_at")  # type: ignore[attr-defined]
+    refusal.state = state  # type: ignore[attr-defined]
+    return refusal
+
+
 #: The file the queries this computer has asked are remembered in, beside the release check's.
 QUERY_MEMO_NAME = "queries.json"
 
@@ -1450,15 +1519,20 @@ def _poll(
     checkpoint_content_digest: str | None,
     sleep: Callable[[float], None],
     clock: Callable[[], float],
-) -> tuple[dict[str, Any], int]:
-    """Read the record back until it settles, or until one of the two ceilings arrives.
+) -> tuple[dict[str, Any], int, bool]:
+    """Read the record back until it settles, until one of the two ceilings arrives, or until
+    the query's own deadline (plus :data:`QUERY_DEADLINE_GRACE_SECONDS`) has passed.
 
     The first read is taken immediately rather than after a sleep: what is in hand on entry is the
     SUBMIT's own answer, written before any worker could have claimed it, so a query that settled
     while the submit was returning would otherwise be reported a whole interval after the fact.
+
+    The third member says the server deadline is what ended the wait.
     """
 
-    deadline = clock() + MAX_QUERY_WAIT_SECONDS
+    started = clock()
+    deadline = started + MAX_QUERY_WAIT_SECONDS
+    overdue_at: float | None = None
     interval = QUERY_POLL_INITIAL_SECONDS
     record: dict[str, Any] = {}
     read = 0
@@ -1466,12 +1540,21 @@ def _poll(
         if attempt:
             if clock() >= deadline:
                 break
-            sleep(interval)
+            if overdue_at is not None and clock() >= overdue_at:
+                return record, read, True
+            pause = interval
+            if overdue_at is not None:
+                pause = max(0.0, min(pause, overdue_at - clock()))
+            sleep(pause)
             interval = min(interval * 2, QUERY_POLL_MAX_SECONDS)
             if clock() >= deadline:
                 break
         record = client.run_query(run_id, query_id)
         read += 1
+        if overdue_at is None:
+            window = _server_window_seconds(record)
+            if window is not None:
+                overdue_at = started + window + QUERY_DEADLINE_GRACE_SECONDS
         if record.get("run_id") not in (None, run_id):
             raise ThinLaneError(
                 "THIN_RESPONSE_INVALID", "Studio answered with a query belonging to another run"
@@ -1479,7 +1562,9 @@ def _poll(
         _checkpoint_binding(record, checkpoint_id, checkpoint_content_digest)
         if record.get("state") in TERMINAL_QUERY_STATES:
             break
-    return record, read
+        if overdue_at is not None and clock() >= overdue_at:
+            return record, read, True
+    return record, read, False
 
 
 def query_exit_code(payload: Mapping[str, Any]) -> int:
@@ -1499,6 +1584,8 @@ __all__ = [
     "MAX_QUERY_WAIT_SECONDS",
     "MAX_STATEMENT_BYTES",
     "PYTZ_NOT_IMPORTED",
+    "QUERY_DEADLINE_GRACE_SECONDS",
+    "QUERY_DEADLINE_PASSED_CODE",
     "QUERY_MEMO_ENTRIES",
     "QUERY_MEMO_NAME",
     "QUERY_POLL_INITIAL_SECONDS",

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4_reader.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import argparse
+import hashlib
 import re
 from collections.abc import Mapping
 from typing import Any
@@ -141,6 +142,7 @@ class StudioV4ReaderClient(StudioV4RunClient):
                 "family_version",
                 "decode_options",
                 "decode_options_json",
+                "options_digest",
             }
         ):
             raise ThinLaneError(
@@ -160,6 +162,30 @@ class StudioV4ReaderClient(StudioV4RunClient):
             raise ThinLaneError(
                 "THIN_RESPONSE_INVALID", "Studio returned inconsistent reader JSON"
             ) from error
+        # The recipe's `reader.options_digest`: SHA-256 of the canonical JSON of the returned
+        # (default-filled) options, the exact value registration recomputes for a recipe that
+        # freezes this `decode_options`. Hashed from THIS client's canonical bytes for the object,
+        # never from Studio's string, and those bytes must equal Studio's `decode_options_json`:
+        # a string that parses equal but is spelled differently would hash to a digest
+        # registration refuses.
+        local = canonical_json_bytes(response["decode_options"])
+        if local != response["decode_options_json"].encode("utf-8"):
+            raise ThinLaneError(
+                "THIN_RESPONSE_INVALID",
+                "Studio's decode_options_json is not the canonical JSON of its decode_options, "
+                "so no options_digest can be stated for it",
+            )
+        digest = hashlib.sha256(local).hexdigest()
+        # ⚠ FORWARD-LOOKING ONLY. Studio 4.57.0 states `options_digest` when the command sets
+        # `include_options_digest: true`, but every older Studio's normalize command is
+        # `additionalProperties: false` and would 422 the flag, and this client negotiates no
+        # contract version before the call. So the flag is not sent and this cross-check runs only
+        # against a Studio that states the digest unasked.
+        if "options_digest" in response and response["options_digest"] != digest:
+            raise ThinLaneError(
+                "THIN_RESPONSE_INVALID", "Studio returned a reader options_digest for other options"
+            )
+        response["options_digest"] = digest
         return response
 
     def open_research_session(self, body: Mapping[str, Any]) -> dict[str, Any]:

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4_search.py

@@ -1529,7 +1529,7 @@ def _workbench_read(
         # rows: only a failure is settled from the handle alone, and every other state is polled.
         if handle.get("state") != "failed":
             try:
-                record, polls = v4_query._poll(
+                record, polls, overdue = v4_query._poll(
                     selected,
                     run_id,
                     query_id,
@@ -1547,6 +1547,8 @@ def _workbench_read(
                     f"{_poll_path(run_id, query_id)}. Running this same command again converges "
                     "on that same query rather than queueing a second",
                 ) from None
+            if overdue and record.get("state") not in v4_query.TERMINAL_QUERY_STATES:
+                raise v4_query._deadline_passed(record, query_id, polls)
             if record.get("query_id") not in (None, query_id):
                 raise ThinLaneError(
                     "THIN_RESPONSE_INVALID", "Studio answered the poll with another query"

{mostlyright_data-0.31.13 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/ux/remediation.py

@@ -11,7 +11,7 @@ neither affordable nor honest:
 * **Families.** Every code prefix is assigned to a family -- 224 of them, plus 21 compound
   prefixes, across 39 families. A family is one paragraph: what this class of failure is about,
   where to look, and what to do. Every prefix has one, so no code can fail to say something useful.
-* **Curated codes.** The 120 failures a person actually meets get their own exact sentences, and
+* **Curated codes.** The 121 failures a person actually meets get their own exact sentences, and
   where one command is the whole answer, the command itself.
 
 Those four counts are not a note somebody remembered to update:
@@ -2308,6 +2308,17 @@ CODES: dict[str, Remediation] = {
             "account counts toward that account's monthly allowance instead.",
         ),
     ),
+    # A web chat key refused a change (Cloud lib/sandbox-key-refusal.ts). The hosted-lane
+    # paragraph would send it to mr-data login, which cannot help.
+    "THIN_CLOUD_SANDBOX_KEY_READ_ONLY": Remediation(
+        lines=(
+            "This command ran with a web chat key. It reads, and the key is valid; changes "
+            "go through the chat's own tools, which the person approves there, or a person "
+            "makes them on the dashboard.",
+            "The message names the chat tool that does this. Signing in again or replacing the "
+            "key will not change the answer.",
+        ),
+    ),
     "THIN_CLOUD_UNREACHABLE": Remediation(
         lines=(
             "Mostly Right Cloud did not answer: the connection was refused, cut, or timed out.",