mostlyright-data 0.31.12__tar.gz → 0.31.14__tar.gz

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.31.12
+Version: 0.31.14
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/
@@ -46,6 +46,13 @@ Once a day, after a command has printed its answer, `mr-data` asks the Python pa
 a newer release is published and says so in one line on stderr. `MOSTLYRIGHT_RELEASE_CHECK=0`
 switches that off.
 
+Every call to Mostly Right names the coding agent running `mr-data` in `X-Mostlyright-Agent`
+(`claude-code`, `codex`, `gemini-cli`, `cursor`, `ci` or `none`, read from the variables those
+hosts set), and a program that imports this package sends it too. An `mr-data` command also sends
+one random id in `X-Mostlyright-Invocation`, so the requests of one command can be grouped; a
+library call sends none. Neither is sent to storage hosts. `MOSTLYRIGHT_NO_AGENT_HEADERS=1`
+switches both off.
+
 ## Sign in
 
 ```bash

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/README.md

@@ -31,6 +31,13 @@ Once a day, after a command has printed its answer, `mr-data` asks the Python pa
 a newer release is published and says so in one line on stderr. `MOSTLYRIGHT_RELEASE_CHECK=0`
 switches that off.
 
+Every call to Mostly Right names the coding agent running `mr-data` in `X-Mostlyright-Agent`
+(`claude-code`, `codex`, `gemini-cli`, `cursor`, `ci` or `none`, read from the variables those
+hosts set), and a program that imports this package sends it too. An `mr-data` command also sends
+one random id in `X-Mostlyright-Invocation`, so the requests of one command can be grouped; a
+library call sends none. Neither is sent to storage hosts. `MOSTLYRIGHT_NO_AGENT_HEADERS=1`
+switches both off.
+
 ## Sign in
 
 ```bash

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/skills/mr-data-build/references/commands.md

@@ -17,7 +17,7 @@ run in full.
 | `mr-data clarify` | Check whether clarification is allowed. Use `--attended` for an active user chat and `--unattended` for a scheduled invocation; an explicit operator veto such as `MOSTLYRIGHT_ATTENDED=0` still wins. A non-TTY subprocess does not prove the user is absent. The command does not ask the person in chat or wait for a reply; the agent must do that. It exits non-zero when nobody can answer or the recipe is already registered. |
 | `mr-data probe` | `probe SOURCE_ID QUESTION_ID --dataset DATASET_ID [--kind source_inspect\|sample_rows\|profile_columns\|evaluate_expression]` asks one already-registered source one question and prints the answer. **It rides the frozen `/v3/sessions` routes, which a deployment may have switched off, so do not plan a source inspection around it.** Both positionals are identifiers. `QUESTION_ID` is a question identifier, which no v4 registration receipt returns. The v4 way to look at a source before registering it is `mr-data source inspect`, in the next row. The other way to read one is a bounded run of a scratch table, never of the real table, as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. |
 | `mr-data source` | `source inspect` takes a bounded look at one source, and it is the v4 path where `probe` is not. [Stage 3](3-probe-read-a-source-before-committing-to-it.md) says when to use it. It starts one of three ways, and each prints the `session_id`, `probe_id` and `source_id`, plus the `dataset_id` and `question_id` when it has them. For a new public HTTPS CSV that needs no key, give `--dataset-id`, `--question`, `--source-name`, `--source-url`, `--claimed-basis` and `--claim-evidence-digest`: it prepares the source and opens a session. For a source already prepared, give `--dataset-id`, `--question-id` and `--source-id`, which opens a session. In a session you already hold, give `--session-id` with `--source-id`. `--context-source-id` adds a source to a session it opens. `--session-id` with `--probe-id` reads the answer back. A look is bounded to 65,536 bytes, 1,000 rows and 300 seconds. `queued` and `truncated` are not complete answers, and `capability_unavailable` means this deployment cannot inspect, so read the source by a bounded run of a scratch table instead, as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. What it finds is exploratory evidence, not a run and not a receipt. |
-| `mr-data reader` | `reader normalize --family-id ID --family-version VERSION --options-file FILE [--workspace-id ID]` asks Studio's Reader service to apply one certified family's own defaults and option rules to a JSON object of options, at most 64 KiB, and prints the normalized `decode_options` and `decode_options_json`. The second is the text a recipe carries as `reader.decode_options`, pinned as [Readers](readers.md) describes. It changes nothing, and this client does not validate or invent a family's options: Studio decides. Run it for every Reader pin before you register, so a wrong option is refused here and not inside a run. |
+| `mr-data reader` | `reader normalize --family-id ID --family-version VERSION --options-file FILE [--workspace-id ID]` asks Studio's Reader service to apply one certified family's own defaults and option rules to a JSON object of options, at most 64 KiB, and prints the normalized `decode_options`, `decode_options_json` and `options_digest`. The second is the text a recipe carries as `reader.decode_options`, and the third is the `reader.options_digest` registration accepts for exactly those options, pinned as [Readers](readers.md) describes. It changes nothing, and this client does not validate or invent a family's options: Studio decides. Run it for every Reader pin before you register, so a wrong option is refused here and not inside a run. |
 | `mr-data catalog` | `catalog search "QUESTION"` asks the sealed public-source catalogue which of its entries might answer a question and ranks them best first. Every ranked entry comes back with the disposition its own facts earned — `admitted`, `human_escalation_required` or `refused` — because an entry the catalogue could not vouch for is still a finding. `--format csv` states the one data format the question requires — one token per question, never repeated — and it is not a filter: an entry that does not declare that format still comes back ranked, with disposition `refused` and `filters_match` false, so nothing is held back; `--limit N` ranks at most N, up to 25. It fetches nothing and registers nothing. The catalogue holds one provider (Data.gov) and only part of it, so it is never exhaustive and an empty answer is not evidence that no such source exists. When the answer is `catalog_unavailable`, this deployment has no catalogue to search: record the lane and carry on. |
 | `mr-data public` | Read Mostly Right's public dataset catalogue. `public search "QUERY"` ranks published datasets (`--topic`, `--limit` and `--cursor` narrow and page it), `show SLUG` reads one overview, `tables SLUG` lists its table identifiers, `schema TABLE_ID` reads one table's schema, `sample TABLE_ID` prints its published preview rows, `lookup "QUERY"` is the connector search shape and `fetch SLUG` returns the dataset's Markdown document. Those seven need no login. `query TABLE_ID`, with `--body` or `--body-file`, and `download-url TABLE_ID` do need `mr-data login`: they ask Cloud, with the device key, to run one query over a public table and to say how it can be downloaded. Preview rows are not a query, and no table bytes pass through the command. |
 | `mr-data connected` | `connected list` shows the public datasets connected to this workspace and `connected add SLUG` connects one by its slug. It needs `mr-data login`: both ask Cloud, with the device key, over the routes MCP's `list_connected_datasets` and `connect_dataset` use. |

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/skills/mr-data-build/references/patterns.md

@@ -225,8 +225,9 @@ for the days that can still change, through the two request `parameters` it rend
 unit reads its own page count, so size `limits.max_requests` from a measured unit.
 [Paging a keyed API in parallel](sources.md#paging-a-keyed-api-in-parallel) has the rules.
 
-`options_digest` is the SHA-256 of the canonical JSON of `decode_options`. Registration names it
-when the two disagree.
+`options_digest` is the SHA-256 of the canonical JSON of `decode_options`, and
+`mr-data reader normalize` prints it for the options it returns. Registration names the digest it
+expected when the two disagree.
 
 ```json
 {

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/skills/mr-data-build/references/readers.md

@@ -57,9 +57,17 @@ mr-data reader normalize --family-id json.tabular --family-version 1.1.0 \
 
 `options.json` holds the keys the source needs, at most 64 KiB. Studio applies the family's own
 defaults and option rules. The answer's `decode_options_json` is the text to write as
-`reader.decode_options`. A refusal arrives here, before any run is spent. Run it for every
+`reader.decode_options`, and its `options_digest` is the `reader.options_digest` to pin beside it.
+That digest binds the default-filled options the answer returns, not the subset in `options.json`,
+so copy both together. A refusal arrives here, before any run is spent. Run it for every
 Reader pin before you register.
 
+`normalize` checks the options against the family, not against the place the pin goes. A
+Generic API `generic_api.response.reader` admits fewer coordinates: `json.tabular` `1.0.0`,
+`1.1.0` or `1.2.0`, `delimited_text` `1.0.0` or `1.1.0`, and `parquet.tabular` `1.0.0`.
+`json.tabular` `1.3.0` and `1.4.0` normalize but are refused there. The highest-version advice
+below applies inside that set.
+
 A column that may be absent from a record must state `"required": false`. Publishers routinely omit
 a key rather than sending null, and a required column that is missing refuses the record.
 
@@ -87,6 +95,7 @@ decoder. Zero matches and multiple matches both refuse.
 | `archive.zip@1.0.0` | `member`; `member_format` (`csv` or `tsv`); optional `header: true`. |
 | `archive.zip@1.1.0` | Same, plus `member_format: semicolon` for publishers such as DWD. |
 | `archive.zip@1.2.0` | Same as `archive.zip@1.1.0`; also accepts a response served as `application/octet-stream`. |
+| `archive.zip@4.0.0` | Exact safe `member`, `member_format: lines` and `encoding: utf-8 or cp1252`; no header. Streams the member's physical lines as line_number, raw_line, as `archive.tar@2.0.0` does for one member. |
 | `archive.gzip@1.0.0` | `member_format`; optional `header: true`. `member` is invalid. |
 | `archive.gzip@1.1.0` | Same; also accepts a response served as `application/octet-stream`. |
 | `archive.tar@1.0.0` | Same as `archive.zip@1.0.0`. |
@@ -181,7 +190,8 @@ Use `public.https.api@1.0.0` only when `keyless_window` is literal `true`. Other
 The new adapter uses the existing `generic_api.response` Reader contract. For Binance klines,
 state `format: json`, `records_pointer: ""`, and `reader.family_id: json.tabular` with
 `family_version: 1.0.0` (or the already admitted `1.1.0`). Pin `options_digest` to the SHA-256
-of the canonical decode options, rather than inserting a placeholder digest. These decode
+of the canonical decode options, the `options_digest` `mr-data reader normalize` prints, rather
+than inserting a placeholder digest. These decode
 options project a root JSON array of arrays, with RFC 6901 numeric indexes into each row:
 
 ```json

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/skills/mr-data-build/references/sources.md

@@ -335,6 +335,10 @@ to separate them. No per-row collection may derive its rows from a `closed` or `
 `pagination` is a `page` or `offset` walk with a required `parallel` member, in one of two forms.
 Prefer the counted form.
 
+`@2.1.0` refuses `{"kind": "none", "max_pages": 1}`, and `cursor` and `link` walks too. For one
+request, use `@2.0.0`. Use `authenticated.https.connector@3.0.0` instead when the request
+declares a `per_call` cost, which `@2.0.0` does not admit.
+
 | Form | Members | Use it when |
 | --- | --- | --- |
 | counted | `total_pages_pointer`, `max_page_bytes`, `min_interval_milliseconds`, optional `concurrency` | the first page states how many pages exist. A `page` walk over a `json` response only |

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/recipe.py

@@ -170,12 +170,12 @@ FILE_DECODER_COORDINATES: frozenset[tuple[str, str]] = frozenset(
     for family, versions in {
         "archive.gzip": ("1.0.0", "1.1.0", "2.0.0", "2.1.0"),
         "archive.tar": ("1.0.0", "1.1.0", "3.0.0", "4.0.0"),
-        "archive.zip": ("1.0.0", "1.1.0", "1.2.0", "1.3.0", "1.4.0", "1.5.0", "3.0.0"),
+        "archive.zip": ("1.0.0", "1.1.0", "1.2.0", "1.3.0", "1.4.0", "1.5.0", "3.0.0", "4.0.0"),
         "delimited_text": ("1.0.0", "1.1.0", "2.0.0"),
         "geoparquet.vector": ("1.0.0",),
         "json.tabular": ("1.1.0", "1.2.0"),
         "meteorology.bufr": ("1.0.0",),
-        "meteorology.grib": ("1.0.0",),
+        "meteorology.grib": ("1.0.0", "2.0.0"),
         "microdata.sas_xpt": ("1.0.0",),
         "microdata.spss_sav": ("1.0.0",),
         "microdata.stata_dta": ("1.0.0",),

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/router.py

@@ -52,6 +52,7 @@ from collections.abc import Callable, Mapping, Sequence
 from typing import Any
 
 from mostlyright.data_harness import skill_assets
+from mostlyright.data_harness.thin import user_agent
 from mostlyright.data_harness.thin.transport import ThinLaneError
 from mostlyright.data_harness.thin.vocabulary import (
     CLASSIFICATION,
@@ -781,6 +782,7 @@ def main(argv: Sequence[str] | None = None) -> int:
     # not write, it never raises, and `MOSTLYRIGHT_SKILL_AUTOINSTALL=0` turns it off. A skill that
     # arrived only when somebody knew to ask for it helped exactly the people who did not need it.
     skill_assets.ensure_installed()
+    user_agent.begin_invocation()
 
     arguments = list(sys.argv[1:] if argv is None else argv)
     hosted_requested = _hosted_flag_present(arguments)

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/runs.py

@@ -46,6 +46,7 @@ from mostlyright.data_harness.thin.transport import (
     UrlLibTransport,
     signed_transfer_url,
 )
+from mostlyright.data_harness.thin.user_agent import agent_headers
 
 #: The contract version every payload below is written against. ``studio_boundary`` pins the
 #: generated client to the same 3.0.0 and to one OpenAPI source digest; this constant is the thin
@@ -279,6 +280,7 @@ class StudioApiClient:
         headers = {
             "Accept": "application/json",
             CLIENT_HEADER: CLIENT_NAME,
+            **agent_headers(),
         }
         if extra_headers:
             headers.update(extra_headers)

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/session.py

@@ -42,6 +42,7 @@ from mostlyright.data_harness.thin.transport import (
     UrlLibTransport,
     service_origin,
 )
+from mostlyright.data_harness.thin.user_agent import agent_headers
 from mostlyright.data_harness.ux.credentials import ResolvedCloudCredentials
 
 #: Cloud's token-exchange route. Held identical to ``hosted_deploy.CLOUD_TOKEN_PATH`` by a test.
@@ -396,7 +397,7 @@ def open_studio_session(
             status, raw, headers = selected.request(
                 "POST",
                 f"{cloud_url}{CLOUD_TOKEN_PATH}",
-                {"Accept": "application/json", "x-api-key": credentials.raw_key},
+                {"Accept": "application/json", "x-api-key": credentials.raw_key, **agent_headers()},
                 b"",
                 MAX_TOKEN_RESPONSE_BYTES,
             )
@@ -586,6 +587,19 @@ def sessions_for(session: StudioSession | StudioSessionProvider) -> StudioSessio
 #: a sentence with spaces in it.
 CLOUD_REFUSAL_CODE = re.compile(r"[A-Z][A-Z0-9_]{0,63}")
 
+#: Cloud's refusal of a web chat sandbox key on a route that would change something
+#: (Cloud ``lib/sandbox-key-refusal.ts``). The key is valid and reads; every change it
+#: would make goes through the chat's own approval-gated tools or a person on the dashboard.
+#: Folded onto ``THIN_AUTHORIZATION_DENIED`` it told an agent in the web chat to run
+#: ``mr-data login``, which cannot help, so it keeps its own code -- and Cloud's sentence,
+#: which names the chat tool that does the job.
+SANDBOX_KEY_READ_ONLY_CODE = "SANDBOX_KEY_READ_ONLY"
+THIN_SANDBOX_KEY_READ_ONLY = f"THIN_CLOUD_{SANDBOX_KEY_READ_ONLY_CODE}"
+SANDBOX_KEY_READ_ONLY_SENTENCE = (
+    "This is a web chat key: it reads, and changes go through the chat's own tools or a "
+    "person on the dashboard; signing in again will not change that"
+)
+
 #: The family of Studio's refusals of a call that would skip Cloud's usage meter
 #: (``DATA_EGRESS_THROUGH_CLOUD``). From a Cloud route it means Studio refused CLOUD's call, which
 #: is Mostly Right's misconfiguration and never the caller's key.
@@ -843,6 +857,18 @@ def cloud_refusal(
             "this computer's or the device key's; nothing here needs to change. Run the command "
             "again later",
         )
+    elif status == 403 and named == SANDBOX_KEY_READ_ONLY_CODE:
+        # NOT A MISSING OR EXPIRED KEY, so not the shared authorization code: the chat's
+        # key is valid and only this change is not its to make. Cloud's sentence names the
+        # tool that makes it, so it is quoted (bounded, printable, attributed) even though a
+        # code was named, which no other named refusal here does.
+        stated_alternative = _cloud_sentence(parsed.get("error"))
+        refusal = ThinLaneError(
+            THIN_SANDBOX_KEY_READ_ONLY,
+            f"{SANDBOX_KEY_READ_ONLY_SENTENCE}. Cloud said: {stated_alternative}"
+            if stated_alternative
+            else SANDBOX_KEY_READ_ONLY_SENTENCE,
+        )
     elif status in {401, 403}:
         if named:
             suffix = f" (Cloud said {named}, HTTP {status})"
@@ -953,6 +979,7 @@ class CloudClient:
         headers = {
             "Accept": "application/json",
             "Authorization": f"Bearer {self._credentials.raw_key}",
+            **agent_headers(),
         }
         encoded: bytes | None = None
         if body is not None:

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/stream.py

@@ -49,6 +49,7 @@ from mostlyright.data_harness.thin.session import (
     stated_sentence,
 )
 from mostlyright.data_harness.thin.transport import ThinLaneError, Transport, UrlLibTransport
+from mostlyright.data_harness.thin.user_agent import agent_headers
 
 #: Upstream's frame names. The first two are the tick/proof partition; ``end`` is its
 #: end-of-stream notice.
@@ -593,6 +594,7 @@ class RunEventStream:
         headers = {
             "Accept": "text/event-stream",
             CLIENT_HEADER: CLIENT_NAME,
+            **agent_headers(),
             **session.authorization(),
         }
         # Resume in Studio's own cursor space when we have one. `from_seq` is only ever the

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/support.py

@@ -31,7 +31,11 @@ from typing import Any
 
 from mostlyright.data_harness.thin import THIN_SCHEMA_PREFIX
 from mostlyright.data_harness.thin.backoff import jittered
-from mostlyright.data_harness.thin.session import CloudClient, require_device_key
+from mostlyright.data_harness.thin.session import (
+    THIN_SANDBOX_KEY_READ_ONLY,
+    CloudClient,
+    require_device_key,
+)
 from mostlyright.data_harness.thin.transport import ThinLaneError, UrlLibTransport
 from mostlyright.data_harness.ux.credentials import resolve_cloud_credentials
 
@@ -61,6 +65,14 @@ SETTLED = frozenset({"answered", "budget_exhausted", "failed"})
 SETTLED_RECEIPTS = frozenset(f"support_{state}" for state in SETTLED)
 STATUS_VERB = "status"
 
+#: What a web chat key is told when Cloud refuses it this route. Said here as well as by
+#: Cloud, so an agent in the chat learns the tool even if Cloud's sentence did not survive
+#: the bound on quoted text.
+CHAT_SUPPORT_TOOLS = (
+    "In the web chat, ask with the chat's ask_support tool, then read the answer with "
+    "get_support_answer"
+)
+
 
 def declare_arguments(parser: argparse.ArgumentParser) -> None:
     """Add `ask`'s arguments to the shared hosted parser."""
@@ -106,17 +118,29 @@ def declare_arguments(parser: argparse.ArgumentParser) -> None:
     )
 
 
+def _name_chat_tools(failure: ThinLaneError) -> ThinLaneError:
+    """A web chat key refused here is pointed at the chat's own support tools."""
+
+    if failure.code == THIN_SANDBOX_KEY_READ_ONLY and "ask_support" not in failure.detail:
+        failure.detail = f"{failure.detail}. {CHAT_SUPPORT_TOOLS}"
+        failure.args = (f"{failure.code}: {failure.detail}",)
+    return failure
+
+
 class SupportClient(CloudClient):
     """The two support routes over the shared device-key Cloud client."""
 
     def submit(self, body: Mapping[str, Any]) -> dict[str, Any]:
-        return self.call(
-            "POST",
-            CLOUD_SUPPORT_QUESTIONS_PATH,
-            act="ask the support agent a question",
-            body=body,
-            expected=(202,),
-        )
+        try:
+            return self.call(
+                "POST",
+                CLOUD_SUPPORT_QUESTIONS_PATH,
+                act="ask the support agent a question",
+                body=body,
+                expected=(202,),
+            )
+        except ThinLaneError as failure:
+            raise _name_chat_tools(failure) from None
 
     def read(self, question_id: str, *, run: bool) -> dict[str, Any]:
         suffix = "" if run else "?wait=0"

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/table_search_contract.json

@@ -1,6 +1,6 @@
 {
  "schema_version": "mostlyright-thin-table-search-contract.v1",
- "source_commit": "5109da1f92eaaf4fc41ed35a03e5d010268d7c7b",
+ "source_commit": "b296c21484f40cce334e1b7c0f6c7c03b44a41e9",
  "recipe_search_definition": "recipe_search",
  "$defs": {
   "common.safe_integer": {
@@ -234,6 +234,12 @@
          "asc",
          "desc"
         ]
+       },
+       "quality": {
+        "description": "Optional owner quality note (4.55.0). A string of 1 to 200 characters. Shown on the order question only when this phrase occurs in the query. It does not change the sort. A boolean is invalid.",
+        "type": "string",
+        "minLength": 1,
+        "maxLength": 200
        }
       },
       "required": [
@@ -242,7 +248,7 @@
        "direction"
       ]
      },
-     "description": "Order phrases (4.45.0: at most 256), one package order key each, under the aliases' shared byte budget."
+     "description": "Order phrases (4.45.0: at most 256), one package order key each, under the aliases' shared byte budget. Optional quality (4.55.0) is the owner's note for that phrase."
     },
     "default_order": {
      "description": "The order of a search with nothing to rank by (4.44.0): a browse or an entity plan with no order of its own walks the rows in this order instead of the package's row order (popularity). Mojirama's `known` (vote count descending) for a film table; the column must be sortable (declared sortable, a sort alias's, or the popularity or votes role).",

mostlyright_data-0.31.14/src/mostlyright/data_harness/thin/user_agent.py

@@ -0,0 +1,234 @@
+"""The identity every first-party request to Mostly Right states.
+
+Without it the standard library sends ``Python-urllib/3.x``, which is what every unattended
+script on the internet sends: the CLI's own traffic could not be told apart from anything else in
+a server log, so nothing downstream could measure it, rate-limit it separately, or exempt it.
+
+The name follows the convention the two crawler lanes already use -- a product token, a version,
+and the contact segment that says where to reach the people running it -- so an origin and our own
+logs see one family of identities from this deployment rather than one per code path.
+
+Two more headers ride beside it on first-party calls only (:func:`agent_headers`): which coding
+agent, if any, is driving this process, and, under the ``mr-data`` command, one id shared by every
+request that command makes. They go where ``X-Mostlyright-Client`` goes and on Cloud's own
+routes, never through the transport, for the reason ``thin.runs.CLIENT_HEADER`` gives: the
+transport also carries signed transfers to storage nobody here owns.
+"""
+
+from __future__ import annotations
+
+import os
+import secrets
+from collections.abc import Mapping
+from functools import cache
+from importlib.metadata import PackageNotFoundError, version
+from typing import NamedTuple
+
+#: The contact segment ``hosted_worker.CRAWLER_CONTACT_SEGMENT`` states for the keyed-source
+#: agents and ``hosted_crawler_fetch.PUBLIC_HTTPS_USER_AGENT`` carries for public sources, with
+#: the measurement that makes it load-bearing recorded at the first of those.
+#:
+#: It is spelled again here rather than imported because both of those modules import ``pyarrow``
+#: at module scope and this package installs with no compiled extension at all -- and they are
+#: Studio's now, so importing them is not a choice this repository has. The spellings are held
+#: against each other by ``tests/test_thin_user_agent.py`` under
+#: ``make verify-cross-repo STUDIO_ROOT=...``, which reads the other one out of Studio's source.
+CRAWLER_CONTACT_SEGMENT = "(+https://mostlyright.md/crawler; crawler@mostlyright.md)"
+
+#: The distribution this client is installed as, and the product token it therefore reports.
+DISTRIBUTION_NAME = "mostlyright-data"
+
+#: The version reported from a source tree, where no distribution metadata exists to read. A
+#: number rather than a word, so every value this header can carry parses the same way.
+UNINSTALLED_VERSION = "0.0.0"
+
+_USER_AGENT_HEADER = "User-Agent"
+
+#: The header naming the agent driving this process, one of :data:`AGENTS`.
+AGENT_HEADER = "X-Mostlyright-Agent"
+
+#: The header carrying :func:`invocation_id`, sent only after :func:`begin_invocation`.
+INVOCATION_HEADER = "X-Mostlyright-Invocation"
+
+#: Set to anything but ``0`` or empty, and neither header above is sent.
+NO_AGENT_HEADERS_ENV = "MOSTLYRIGHT_NO_AGENT_HEADERS"
+
+AGENT_CLAUDE_CODE = "claude-code"
+AGENT_CODEX = "codex"
+AGENT_GEMINI_CLI = "gemini-cli"
+AGENT_CURSOR = "cursor"
+AGENT_CI = "ci"
+AGENT_NONE = "none"
+
+
+class AgentMarker(NamedTuple):
+    """One variable an agent host exports into the commands it runs.
+
+    ``conversation`` says whether the attendance check (``ux.attendance``) reads the variable as
+    a conversation existing. Only a marker every headless run of that host lacks may say so:
+    ``CODEX_SANDBOX`` is set by ``codex exec`` on a cron job too, so it names Codex here and is
+    not read as a person being there.
+    """
+
+    agent: str
+    variable: str
+    conversation: bool
+
+
+#: Every marker, grouped by agent in precedence order.
+#:
+#: Presence of a non-empty value is the signal and the value is never read: these are the hosts'
+#: variables, and their values are theirs to change. A coding agent outranks ``ci``, so
+#: ``claude -p`` inside a GitHub Action reports ``claude-code``. Nothing that every terminal of an
+#: editor exports is listed, because a person typing there would read as the agent.
+AGENT_MARKERS: tuple[AgentMarker, ...] = (
+    AgentMarker(AGENT_CLAUDE_CODE, "CLAUDECODE", True),
+    AgentMarker(AGENT_CLAUDE_CODE, "CLAUDE_CODE_SESSION", True),
+    AgentMarker(AGENT_CODEX, "CODEX_SESSION", True),
+    AgentMarker(AGENT_CODEX, "CODEX_SANDBOX", False),
+    AgentMarker(AGENT_CODEX, "CODEX_SANDBOX_NETWORK_DISABLED", False),
+    AgentMarker(AGENT_GEMINI_CLI, "GEMINI_CLI", False),
+    AgentMarker(AGENT_CURSOR, "CURSOR_AGENT", False),
+    AgentMarker(AGENT_CI, "CI", False),
+    AgentMarker(AGENT_CI, "GITHUB_ACTIONS", False),
+)
+
+#: Every value :data:`AGENT_HEADER` can carry.
+AGENTS: tuple[str, ...] = (*dict.fromkeys(marker.agent for marker in AGENT_MARKERS), AGENT_NONE)
+
+#: What :func:`invocation_id` returns: 16 random bytes as 32 lowercase hex digits.
+INVOCATION_ID_PATTERN = r"^[0-9a-f]{32}$"
+
+_invocation: str | None = None
+
+
+@cache
+def client_user_agent() -> str:
+    """Return the ``User-Agent`` this client presents to Mostly Right.
+
+    Cached because the answer is a property of the installed environment, and reading distribution
+    metadata walks ``sys.path``; a command that makes twenty calls would otherwise pay for it
+    twenty times.
+    """
+
+    try:
+        installed = version(DISTRIBUTION_NAME)
+    except PackageNotFoundError:
+        installed = UNINSTALLED_VERSION
+    return f"{DISTRIBUTION_NAME}/{installed} {CRAWLER_CONTACT_SEGMENT}"
+
+
+def with_user_agent(headers: Mapping[str, str]) -> dict[str, str]:
+    """Return ``headers`` as a new dict carrying this client's identity.
+
+    A caller that stated its own ``User-Agent`` keeps it, under whatever spelling it used: HTTP
+    header names are case-insensitive, so adding a second one under a different capitalization
+    would send two identities rather than override one.
+    """
+
+    resolved = dict(headers)
+    if any(name.lower() == "user-agent" for name in resolved):
+        return resolved
+    resolved[_USER_AGENT_HEADER] = client_user_agent()
+    return resolved
+
+
+def agent_env(agent: str) -> tuple[str, ...]:
+    """Return the variables that name ``agent``; empty for ``none`` and for a name not listed."""
+
+    return tuple(marker.variable for marker in AGENT_MARKERS if marker.agent == agent)
+
+
+def conversation_env() -> tuple[str, ...]:
+    """Return the markers the attendance check reads as a conversation existing."""
+
+    return tuple(marker.variable for marker in AGENT_MARKERS if marker.conversation)
+
+
+def first_set(names: tuple[str, ...], environ: Mapping[str, str] | None = None) -> str | None:
+    """Return the first of ``names`` set to a non-empty value, or ``None``."""
+
+    table = os.environ if environ is None else environ
+    return next((name for name in names if (table.get(name) or "").strip()), None)
+
+
+def detect_agent(environ: Mapping[str, str] | None = None) -> str:
+    """Return the agent driving this process, one of :data:`AGENTS`."""
+
+    table = os.environ if environ is None else environ
+    for marker in AGENT_MARKERS:
+        if (table.get(marker.variable) or "").strip():
+            return marker.agent
+    return AGENT_NONE
+
+
+def begin_invocation() -> str:
+    """Draw a fresh invocation id for the command starting now, and return it.
+
+    Called by the ``mr-data`` entry point once per command. A program importing this package
+    never calls it, so its requests carry no invocation id rather than one that lasts as long as
+    the process does.
+    """
+
+    global _invocation
+    _invocation = secrets.token_hex(16)
+    return _invocation
+
+
+def end_invocation() -> None:
+    """Forget the current invocation id, so later requests carry none."""
+
+    global _invocation
+    _invocation = None
+
+
+def invocation_id() -> str | None:
+    """Return the current command's id (see :data:`INVOCATION_ID_PATTERN`), or ``None``."""
+
+    return _invocation
+
+
+def agent_headers(environ: Mapping[str, str] | None = None) -> dict[str, str]:
+    """Return the agent and invocation headers, or none when :data:`NO_AGENT_HEADERS_ENV` is set.
+
+    The invocation header is present only inside a command :func:`begin_invocation` started. A
+    first-party call adds these itself; the transport never does.
+    """
+
+    table = os.environ if environ is None else environ
+    if (table.get(NO_AGENT_HEADERS_ENV) or "").strip() not in ("", "0"):
+        return {}
+    headers = {AGENT_HEADER: detect_agent(table)}
+    if _invocation is not None:
+        headers[INVOCATION_HEADER] = _invocation
+    return headers
+
+
+__all__ = [
+    "AGENTS",
+    "AGENT_CI",
+    "AGENT_CLAUDE_CODE",
+    "AGENT_CODEX",
+    "AGENT_CURSOR",
+    "AGENT_GEMINI_CLI",
+    "AGENT_HEADER",
+    "AGENT_MARKERS",
+    "AGENT_NONE",
+    "CRAWLER_CONTACT_SEGMENT",
+    "DISTRIBUTION_NAME",
+    "INVOCATION_HEADER",
+    "INVOCATION_ID_PATTERN",
+    "NO_AGENT_HEADERS_ENV",
+    "UNINSTALLED_VERSION",
+    "AgentMarker",
+    "agent_env",
+    "agent_headers",
+    "begin_invocation",
+    "client_user_agent",
+    "conversation_env",
+    "detect_agent",
+    "end_invocation",
+    "first_set",
+    "invocation_id",
+    "with_user_agent",
+]

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4.py

@@ -149,8 +149,8 @@ from mostlyright.data_harness.thin.v4_matching import CONTRACT_PATHS as MATCHING
 #: ``JOB_INVALID`` outright, which in production was every refresh a collection epoch was offered
 #: for rather than only the collection runs. This package must not reach a Studio older than the
 #: commit it pins; ``docs/V4-WORKER-PROTOCOL.md`` states it beside the layout's own ordering rule.
-PINNED_V4_OPENAPI_SOURCE_SHA256 = "e7bd4e994b32d62ae00b4d7fc2f8936e4964cd4160b7aa30ae010528ff57b1a8"
-PINNED_V4_CONTRACT_VERSION = "4.54.0"
+PINNED_V4_OPENAPI_SOURCE_SHA256 = "b378d4f70d580aed094b00edeec5acc39cae5c79f19dc4e869fe40304fccf654"
+PINNED_V4_CONTRACT_VERSION = "4.57.0"
 
 #: The ``schema_version`` every command body that predates 4.14.0 stamps: ``dataset activity``,
 #: ``table rename``, ``table archive`` and ``dataset archive``. It is the number the 0.25 line

{mostlyright_data-0.31.12 → mostlyright_data-0.31.14}/src/mostlyright/data_harness/thin/v4_datasets.py

@@ -136,6 +136,7 @@ from mostlyright.data_harness.thin.session import (
     CLOUD_REFUSAL_CODE,
     MAX_JSON_RESPONSE_BYTES,
     MAX_STATED_MESSAGE_CHARS,
+    THIN_SANDBOX_KEY_READ_ONLY,
     CloudClient,
     StudioSession,
     StudioSessionProvider,
@@ -1615,18 +1616,30 @@ def _sync_written_identity(dataset_id: str, cloud_url: str) -> dict[str, Any]:
             )
         return {"public_projection_synced": True}
     except (ThinLaneError, LoginError, OSError) as error:
-        return {
+        stated = isinstance(error, ThinLaneError) and getattr(error, "http_status", None) != 404
+        result: dict[str, Any] = {
             "public_projection_synced": False,
-            "public_projection_error": (
-                error.code
-                if isinstance(error, ThinLaneError) and getattr(error, "http_status", None) != 404
-                else "THIN_IDENTITY_SYNC_UNAVAILABLE"
-            ),
+            "public_projection_error": (error.code if stated else "THIN_IDENTITY_SYNC_UNAVAILABLE"),
             "public_projection_note": (
                 "The dataset was saved, but its public page could not be updated. "
                 f"Retry only the sync with: mr-data dataset sync {dataset_id}"
             ),
         }
+        # ⚠ THE CODE ALONE SENT PEOPLE TO THE WRONG FIX. Every refusal on this path left
+        # as a bare code, so a key Cloud would never let update the page and an expired one
+        # read the same. The refusal's own sentence -- this lane's words, with Cloud's
+        # quoted inside them already bounded and printable by ``cloud_refusal`` -- rides
+        # along. Nothing branches on it; the code beside it is what a script reads.
+        if stated:
+            result["public_projection_message"] = error.detail
+        if stated and error.code == THIN_SANDBOX_KEY_READ_ONLY:
+            # Retrying with this key is refused the same way: the chat does it.
+            result["public_projection_note"] = (
+                "The dataset was saved, but this web chat key cannot update its public "
+                "page. Use the chat's update_dataset tool with sync_only: true, or a "
+                "person syncs it from the dashboard."
+            )
+        return result
 
 
 def _held(record: Mapping[str, Any], member: str) -> Any: