mostlyright-data 0.31.11__tar.gz → 0.31.13__tar.gz

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.31.11
+Version: 0.31.13
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/
@@ -46,6 +46,13 @@ Once a day, after a command has printed its answer, `mr-data` asks the Python pa
 a newer release is published and says so in one line on stderr. `MOSTLYRIGHT_RELEASE_CHECK=0`
 switches that off.
 
+Every call to Mostly Right names the coding agent running `mr-data` in `X-Mostlyright-Agent`
+(`claude-code`, `codex`, `gemini-cli`, `cursor`, `ci` or `none`, read from the variables those
+hosts set), and a program that imports this package sends it too. An `mr-data` command also sends
+one random id in `X-Mostlyright-Invocation`, so the requests of one command can be grouped; a
+library call sends none. Neither is sent to storage hosts. `MOSTLYRIGHT_NO_AGENT_HEADERS=1`
+switches both off.
+
 ## Sign in
 
 ```bash

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/README.md

@@ -31,6 +31,13 @@ Once a day, after a command has printed its answer, `mr-data` asks the Python pa
 a newer release is published and says so in one line on stderr. `MOSTLYRIGHT_RELEASE_CHECK=0`
 switches that off.
 
+Every call to Mostly Right names the coding agent running `mr-data` in `X-Mostlyright-Agent`
+(`claude-code`, `codex`, `gemini-cli`, `cursor`, `ci` or `none`, read from the variables those
+hosts set), and a program that imports this package sends it too. An `mr-data` command also sends
+one random id in `X-Mostlyright-Invocation`, so the requests of one command can be grouped; a
+library call sends none. Neither is sent to storage hosts. `MOSTLYRIGHT_NO_AGENT_HEADERS=1`
+switches both off.
+
 ## Sign in
 
 ```bash

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/skills/mr-data-build/references/preparing-table-search.md

@@ -194,7 +194,46 @@ column names, the descriptions and the planner. Never bind a word that two colum
 such as "growth", "change" or "best", unless the brief names the metric. "Highest growth"
 pointed at a revenue column turns "highest growth last 30 days" into a revenue sort.
 
-Studio caps a spec at 128 filter aliases and 64 sort aliases, and `search validate` refuses more.
+### Declare a floor for every rate
+
+A growth, change, ratio or per-unit column ranks nonsense first when its base is tiny. Revenue
+that grew from $5,000 to $80 million is 1,600,000% growth, and a film with three votes has a
+perfect rating. Sorting by such a column shows shells and placeholders, not the rows the reader
+asked for. For each sortable rate column, declare the column it is a rate of, and the smallest
+base that makes the rate meaningful, in `order_floors` (contract 4.54.0, up to 32 entries):
+
+```json
+"order_floors": [
+  {"column": "revenue_growth_pct", "base_column": "revenue_prior_year_usd", "min": 10000000},
+  {"column": "revenue_cagr_5y_pct", "base_column": "revenue_5y_ago_usd", "min": 10000000},
+  {"column": "vote_average", "base_column": "vote_count", "min": 100}
+]
+```
+
+- `column` is in `sortable_columns` or is a sort alias's column. `base_column` is another numeric
+  column in `filterable_columns`. List each base early in `filterable_columns`: a package keeps 64
+  filters, role and alias columns first and the rest in spec order, and a base past that cut
+  loses its floor. `search validate TABLE --spec` names such a floor in `warnings` under
+  `/order_floors/`; treat that warning as a floor that will not apply.
+- `min` is a whole number in the base column's own unit, written without a decimal point or
+  exponent: `10000000` on a column in dollars, `10` on a column in millions.
+- Floors are declared on a search revision with `mr-data search configure --spec`, not in the
+  recipe's `table.search`, which refuses the member. A revision configured from the recipe's own
+  spec carries none, and a new version of a private table keeps them only when you configure the
+  revision again. A plan made without the planner, such as a keystroke search, applies no floor.
+- A Studio older than contract 4.54.0 refuses the member as unknown. Configure without it there.
+- When a plan orders by `column`, in either direction, Studio adds `base_column >= min` as a
+  filter. The reader sees it in the search summary and can remove it. A query that already filters
+  `base_column` keeps its own filter.
+- Prefer the true base (the earlier period's revenue, the vote count) over a size proxy such as
+  market cap. When the table has no base column, add one in the transform: it is cheaper than a
+  wrong ranking.
+- Choose `min` from the built table with `mr-data query`: the smallest base at which the rate stops
+  being dominated by placeholders. State the floor at the start of the base column's
+  description, since the planner reads only the first 160 characters of a description.
+
+A recipe's spec takes 128 filter aliases and 64 sort aliases, and a search revision 512 and 256.
+`search validate` refuses more.
 Rank the phrases and keep the top of the list. Put first the words that name the kind of row, the
 facets users type most, the popularity sorts and the product's own vocabulary. A phrase you leave
 out maps to nothing. The planner may still recover it from the column names and values, and
@@ -225,8 +264,13 @@ roles. It never sees rows.
 `planner_detail.reason`. `external_context_not_allowed` means the recipe withholds consent.
 `private_source` means a source is not public.
 
-Write each column description as if the planner reads it. Say what the column holds, its unit,
-and the words people use for it.
+Write each column description as if the planner reads it, because it does, and the page shows it
+to the reader beside results ordered by that column. Say what the column holds, the period it
+covers ("trailing twelve months", "latest fiscal year against the year before", "over the last
+five fiscal years"), its unit, and the words people use for it, abbreviations included
+("compound annual growth rate, CAGR"). The planner picks the column a query asks about, named
+bare ("revenue") or in a question ("what is the P/E ratio?"), from these descriptions. It reads
+the first 160 characters of each, so lead with what the column measures and its period.
 
 ### Name and describe columns for search
 

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/router.py

@@ -52,6 +52,7 @@ from collections.abc import Callable, Mapping, Sequence
 from typing import Any
 
 from mostlyright.data_harness import skill_assets
+from mostlyright.data_harness.thin import user_agent
 from mostlyright.data_harness.thin.transport import ThinLaneError
 from mostlyright.data_harness.thin.vocabulary import (
     CLASSIFICATION,
@@ -781,6 +782,7 @@ def main(argv: Sequence[str] | None = None) -> int:
     # not write, it never raises, and `MOSTLYRIGHT_SKILL_AUTOINSTALL=0` turns it off. A skill that
     # arrived only when somebody knew to ask for it helped exactly the people who did not need it.
     skill_assets.ensure_installed()
+    user_agent.begin_invocation()
 
     arguments = list(sys.argv[1:] if argv is None else argv)
     hosted_requested = _hosted_flag_present(arguments)

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/runs.py

@@ -46,6 +46,7 @@ from mostlyright.data_harness.thin.transport import (
     UrlLibTransport,
     signed_transfer_url,
 )
+from mostlyright.data_harness.thin.user_agent import agent_headers
 
 #: The contract version every payload below is written against. ``studio_boundary`` pins the
 #: generated client to the same 3.0.0 and to one OpenAPI source digest; this constant is the thin
@@ -279,6 +280,7 @@ class StudioApiClient:
         headers = {
             "Accept": "application/json",
             CLIENT_HEADER: CLIENT_NAME,
+            **agent_headers(),
         }
         if extra_headers:
             headers.update(extra_headers)

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/session.py

@@ -42,6 +42,7 @@ from mostlyright.data_harness.thin.transport import (
     UrlLibTransport,
     service_origin,
 )
+from mostlyright.data_harness.thin.user_agent import agent_headers
 from mostlyright.data_harness.ux.credentials import ResolvedCloudCredentials
 
 #: Cloud's token-exchange route. Held identical to ``hosted_deploy.CLOUD_TOKEN_PATH`` by a test.
@@ -396,7 +397,7 @@ def open_studio_session(
             status, raw, headers = selected.request(
                 "POST",
                 f"{cloud_url}{CLOUD_TOKEN_PATH}",
-                {"Accept": "application/json", "x-api-key": credentials.raw_key},
+                {"Accept": "application/json", "x-api-key": credentials.raw_key, **agent_headers()},
                 b"",
                 MAX_TOKEN_RESPONSE_BYTES,
             )
@@ -953,6 +954,7 @@ class CloudClient:
         headers = {
             "Accept": "application/json",
             "Authorization": f"Bearer {self._credentials.raw_key}",
+            **agent_headers(),
         }
         encoded: bytes | None = None
         if body is not None:

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/stream.py

@@ -49,6 +49,7 @@ from mostlyright.data_harness.thin.session import (
     stated_sentence,
 )
 from mostlyright.data_harness.thin.transport import ThinLaneError, Transport, UrlLibTransport
+from mostlyright.data_harness.thin.user_agent import agent_headers
 
 #: Upstream's frame names. The first two are the tick/proof partition; ``end`` is its
 #: end-of-stream notice.
@@ -593,6 +594,7 @@ class RunEventStream:
         headers = {
             "Accept": "text/event-stream",
             CLIENT_HEADER: CLIENT_NAME,
+            **agent_headers(),
             **session.authorization(),
         }
         # Resume in Studio's own cursor space when we have one. `from_seq` is only ever the

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/table_search_contract.json

@@ -1,8 +1,14 @@
 {
  "schema_version": "mostlyright-thin-table-search-contract.v1",
- "source_commit": "0dcecc1b8dfdd3f8960497fdfcffeeb72e240abd",
+ "source_commit": "5109da1f92eaaf4fc41ed35a03e5d010268d7c7b",
  "recipe_search_definition": "recipe_search",
  "$defs": {
+  "common.safe_integer": {
+   "description": "An integer exactly representable by both Python JSON integers and TypeScript numbers.",
+   "type": "integer",
+   "minimum": -9007199254740991,
+   "maximum": 9007199254740991
+  },
   "evaluation_case": {
    "type": "object",
    "additionalProperties": false,
@@ -119,6 +125,14 @@
       ]
      },
      "description": "Keel's recipe reader does not admit long_text_columns yet (4.42.0): a search revision declares it."
+    },
+    {
+     "not": {
+      "required": [
+       "order_floors"
+      ]
+     },
+     "description": "The recipe readers do not admit order_floors (4.54.0): a search revision declares it."
     }
    ]
   },
@@ -252,6 +266,35 @@
       "direction"
      ]
     },
+    "order_floors": {
+     "description": "Order floors (4.54.0): for a sortable column whose order means little on rows with a tiny or missing base (a growth rate on almost no revenue, a rating from a handful of votes), the base column and the least value of it at which a row answers that order. When a plan orders by `column`, either direction, and states no predicate on `base_column`, the planner adds `base_column >= min` (origin `parser`), which the search node runs and reports as any predicate. `column` is declared sortable (in `sortable_columns` or a sort alias's, even where a v3 package sorts numbers by default), `base_column` is another declared numeric column in `filterable_columns`, and each `column` has at most one floor. `min` is an integer (a request body carries no fractional or exponent number), in the base column's own unit. Admitted on a search revision; a recipe's `table.search` may not name it yet.",
+     "type": "array",
+     "maxItems": 32,
+     "items": {
+      "type": "object",
+      "additionalProperties": false,
+      "properties": {
+       "column": {
+        "type": "string",
+        "minLength": 1,
+        "maxLength": 128
+       },
+       "base_column": {
+        "type": "string",
+        "minLength": 1,
+        "maxLength": 128
+       },
+       "min": {
+        "$ref": "#/$defs/common.safe_integer"
+       }
+      },
+      "required": [
+       "column",
+       "base_column",
+       "min"
+      ]
+     }
+    },
     "planner": {
      "type": "object",
      "additionalProperties": false,

mostlyright_data-0.31.13/src/mostlyright/data_harness/thin/user_agent.py

@@ -0,0 +1,234 @@
+"""The identity every first-party request to Mostly Right states.
+
+Without it the standard library sends ``Python-urllib/3.x``, which is what every unattended
+script on the internet sends: the CLI's own traffic could not be told apart from anything else in
+a server log, so nothing downstream could measure it, rate-limit it separately, or exempt it.
+
+The name follows the convention the two crawler lanes already use -- a product token, a version,
+and the contact segment that says where to reach the people running it -- so an origin and our own
+logs see one family of identities from this deployment rather than one per code path.
+
+Two more headers ride beside it on first-party calls only (:func:`agent_headers`): which coding
+agent, if any, is driving this process, and, under the ``mr-data`` command, one id shared by every
+request that command makes. They go where ``X-Mostlyright-Client`` goes and on Cloud's own
+routes, never through the transport, for the reason ``thin.runs.CLIENT_HEADER`` gives: the
+transport also carries signed transfers to storage nobody here owns.
+"""
+
+from __future__ import annotations
+
+import os
+import secrets
+from collections.abc import Mapping
+from functools import cache
+from importlib.metadata import PackageNotFoundError, version
+from typing import NamedTuple
+
+#: The contact segment ``hosted_worker.CRAWLER_CONTACT_SEGMENT`` states for the keyed-source
+#: agents and ``hosted_crawler_fetch.PUBLIC_HTTPS_USER_AGENT`` carries for public sources, with
+#: the measurement that makes it load-bearing recorded at the first of those.
+#:
+#: It is spelled again here rather than imported because both of those modules import ``pyarrow``
+#: at module scope and this package installs with no compiled extension at all -- and they are
+#: Studio's now, so importing them is not a choice this repository has. The spellings are held
+#: against each other by ``tests/test_thin_user_agent.py`` under
+#: ``make verify-cross-repo STUDIO_ROOT=...``, which reads the other one out of Studio's source.
+CRAWLER_CONTACT_SEGMENT = "(+https://mostlyright.md/crawler; crawler@mostlyright.md)"
+
+#: The distribution this client is installed as, and the product token it therefore reports.
+DISTRIBUTION_NAME = "mostlyright-data"
+
+#: The version reported from a source tree, where no distribution metadata exists to read. A
+#: number rather than a word, so every value this header can carry parses the same way.
+UNINSTALLED_VERSION = "0.0.0"
+
+_USER_AGENT_HEADER = "User-Agent"
+
+#: The header naming the agent driving this process, one of :data:`AGENTS`.
+AGENT_HEADER = "X-Mostlyright-Agent"
+
+#: The header carrying :func:`invocation_id`, sent only after :func:`begin_invocation`.
+INVOCATION_HEADER = "X-Mostlyright-Invocation"
+
+#: Set to anything but ``0`` or empty, and neither header above is sent.
+NO_AGENT_HEADERS_ENV = "MOSTLYRIGHT_NO_AGENT_HEADERS"
+
+AGENT_CLAUDE_CODE = "claude-code"
+AGENT_CODEX = "codex"
+AGENT_GEMINI_CLI = "gemini-cli"
+AGENT_CURSOR = "cursor"
+AGENT_CI = "ci"
+AGENT_NONE = "none"
+
+
+class AgentMarker(NamedTuple):
+    """One variable an agent host exports into the commands it runs.
+
+    ``conversation`` says whether the attendance check (``ux.attendance``) reads the variable as
+    a conversation existing. Only a marker every headless run of that host lacks may say so:
+    ``CODEX_SANDBOX`` is set by ``codex exec`` on a cron job too, so it names Codex here and is
+    not read as a person being there.
+    """
+
+    agent: str
+    variable: str
+    conversation: bool
+
+
+#: Every marker, grouped by agent in precedence order.
+#:
+#: Presence of a non-empty value is the signal and the value is never read: these are the hosts'
+#: variables, and their values are theirs to change. A coding agent outranks ``ci``, so
+#: ``claude -p`` inside a GitHub Action reports ``claude-code``. Nothing that every terminal of an
+#: editor exports is listed, because a person typing there would read as the agent.
+AGENT_MARKERS: tuple[AgentMarker, ...] = (
+    AgentMarker(AGENT_CLAUDE_CODE, "CLAUDECODE", True),
+    AgentMarker(AGENT_CLAUDE_CODE, "CLAUDE_CODE_SESSION", True),
+    AgentMarker(AGENT_CODEX, "CODEX_SESSION", True),
+    AgentMarker(AGENT_CODEX, "CODEX_SANDBOX", False),
+    AgentMarker(AGENT_CODEX, "CODEX_SANDBOX_NETWORK_DISABLED", False),
+    AgentMarker(AGENT_GEMINI_CLI, "GEMINI_CLI", False),
+    AgentMarker(AGENT_CURSOR, "CURSOR_AGENT", False),
+    AgentMarker(AGENT_CI, "CI", False),
+    AgentMarker(AGENT_CI, "GITHUB_ACTIONS", False),
+)
+
+#: Every value :data:`AGENT_HEADER` can carry.
+AGENTS: tuple[str, ...] = (*dict.fromkeys(marker.agent for marker in AGENT_MARKERS), AGENT_NONE)
+
+#: What :func:`invocation_id` returns: 16 random bytes as 32 lowercase hex digits.
+INVOCATION_ID_PATTERN = r"^[0-9a-f]{32}$"
+
+_invocation: str | None = None
+
+
+@cache
+def client_user_agent() -> str:
+    """Return the ``User-Agent`` this client presents to Mostly Right.
+
+    Cached because the answer is a property of the installed environment, and reading distribution
+    metadata walks ``sys.path``; a command that makes twenty calls would otherwise pay for it
+    twenty times.
+    """
+
+    try:
+        installed = version(DISTRIBUTION_NAME)
+    except PackageNotFoundError:
+        installed = UNINSTALLED_VERSION
+    return f"{DISTRIBUTION_NAME}/{installed} {CRAWLER_CONTACT_SEGMENT}"
+
+
+def with_user_agent(headers: Mapping[str, str]) -> dict[str, str]:
+    """Return ``headers`` as a new dict carrying this client's identity.
+
+    A caller that stated its own ``User-Agent`` keeps it, under whatever spelling it used: HTTP
+    header names are case-insensitive, so adding a second one under a different capitalization
+    would send two identities rather than override one.
+    """
+
+    resolved = dict(headers)
+    if any(name.lower() == "user-agent" for name in resolved):
+        return resolved
+    resolved[_USER_AGENT_HEADER] = client_user_agent()
+    return resolved
+
+
+def agent_env(agent: str) -> tuple[str, ...]:
+    """Return the variables that name ``agent``; empty for ``none`` and for a name not listed."""
+
+    return tuple(marker.variable for marker in AGENT_MARKERS if marker.agent == agent)
+
+
+def conversation_env() -> tuple[str, ...]:
+    """Return the markers the attendance check reads as a conversation existing."""
+
+    return tuple(marker.variable for marker in AGENT_MARKERS if marker.conversation)
+
+
+def first_set(names: tuple[str, ...], environ: Mapping[str, str] | None = None) -> str | None:
+    """Return the first of ``names`` set to a non-empty value, or ``None``."""
+
+    table = os.environ if environ is None else environ
+    return next((name for name in names if (table.get(name) or "").strip()), None)
+
+
+def detect_agent(environ: Mapping[str, str] | None = None) -> str:
+    """Return the agent driving this process, one of :data:`AGENTS`."""
+
+    table = os.environ if environ is None else environ
+    for marker in AGENT_MARKERS:
+        if (table.get(marker.variable) or "").strip():
+            return marker.agent
+    return AGENT_NONE
+
+
+def begin_invocation() -> str:
+    """Draw a fresh invocation id for the command starting now, and return it.
+
+    Called by the ``mr-data`` entry point once per command. A program importing this package
+    never calls it, so its requests carry no invocation id rather than one that lasts as long as
+    the process does.
+    """
+
+    global _invocation
+    _invocation = secrets.token_hex(16)
+    return _invocation
+
+
+def end_invocation() -> None:
+    """Forget the current invocation id, so later requests carry none."""
+
+    global _invocation
+    _invocation = None
+
+
+def invocation_id() -> str | None:
+    """Return the current command's id (see :data:`INVOCATION_ID_PATTERN`), or ``None``."""
+
+    return _invocation
+
+
+def agent_headers(environ: Mapping[str, str] | None = None) -> dict[str, str]:
+    """Return the agent and invocation headers, or none when :data:`NO_AGENT_HEADERS_ENV` is set.
+
+    The invocation header is present only inside a command :func:`begin_invocation` started. A
+    first-party call adds these itself; the transport never does.
+    """
+
+    table = os.environ if environ is None else environ
+    if (table.get(NO_AGENT_HEADERS_ENV) or "").strip() not in ("", "0"):
+        return {}
+    headers = {AGENT_HEADER: detect_agent(table)}
+    if _invocation is not None:
+        headers[INVOCATION_HEADER] = _invocation
+    return headers
+
+
+__all__ = [
+    "AGENTS",
+    "AGENT_CI",
+    "AGENT_CLAUDE_CODE",
+    "AGENT_CODEX",
+    "AGENT_CURSOR",
+    "AGENT_GEMINI_CLI",
+    "AGENT_HEADER",
+    "AGENT_MARKERS",
+    "AGENT_NONE",
+    "CRAWLER_CONTACT_SEGMENT",
+    "DISTRIBUTION_NAME",
+    "INVOCATION_HEADER",
+    "INVOCATION_ID_PATTERN",
+    "NO_AGENT_HEADERS_ENV",
+    "UNINSTALLED_VERSION",
+    "AgentMarker",
+    "agent_env",
+    "agent_headers",
+    "begin_invocation",
+    "client_user_agent",
+    "conversation_env",
+    "detect_agent",
+    "end_invocation",
+    "first_set",
+    "invocation_id",
+    "with_user_agent",
+]

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/v4.py

@@ -149,8 +149,8 @@ from mostlyright.data_harness.thin.v4_matching import CONTRACT_PATHS as MATCHING
 #: ``JOB_INVALID`` outright, which in production was every refresh a collection epoch was offered
 #: for rather than only the collection runs. This package must not reach a Studio older than the
 #: commit it pins; ``docs/V4-WORKER-PROTOCOL.md`` states it beside the layout's own ordering rule.
-PINNED_V4_OPENAPI_SOURCE_SHA256 = "b8f2ac008a714a12b88516d4c723ef8a7af18100c2678831fbdafd50fc6a5f20"
-PINNED_V4_CONTRACT_VERSION = "4.49.0"
+PINNED_V4_OPENAPI_SOURCE_SHA256 = "e7bd4e994b32d62ae00b4d7fc2f8936e4964cd4160b7aa30ae010528ff57b1a8"
+PINNED_V4_CONTRACT_VERSION = "4.54.0"
 
 #: The ``schema_version`` every command body that predates 4.14.0 stamps: ``dataset activity``,
 #: ``table rename``, ``table archive`` and ``dataset archive``. It is the number the 0.25 line

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/v4_public.py

@@ -52,7 +52,7 @@ from mostlyright.data_harness.thin.transport import (
     UrlLibTransport,
     service_origin,
 )
-from mostlyright.data_harness.thin.user_agent import with_user_agent
+from mostlyright.data_harness.thin.user_agent import agent_headers, with_user_agent
 from mostlyright.data_harness.ux.credentials import (
     CLOUD_URL_ENV,
     DEFAULT_CLOUD_URL,
@@ -245,6 +245,7 @@ class PublicCatalogClient:
             {
                 "Accept": accept,
                 CLIENT_HEADER: CLIENT_NAME,
+                **agent_headers(),
             }
         )
 
@@ -960,6 +961,7 @@ class PublicQueryClient(CloudClient):
         headers = {
             "Accept": "application/json",
             "Authorization": f"Bearer {self._credentials.raw_key}",
+            **agent_headers(),
         }
         status, raw, response_headers = self._transport.request(
             "GET",

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/v4_query.py

@@ -75,6 +75,7 @@ from mostlyright.data_harness.thin.transport import (
     Transport,
     UrlLibTransport,
 )
+from mostlyright.data_harness.thin.user_agent import agent_headers
 from mostlyright.data_harness.thin.v4 import (
     GET_RUN_PATH,
     GET_RUN_QUERY_PATH,
@@ -329,6 +330,7 @@ class CloudRunQueryClient(CloudClient):
         headers = {
             "Accept": "application/json",
             "Authorization": f"Bearer {self._credentials.raw_key}",
+            **agent_headers(),
         }
         encoded: bytes | None = None
         if body is not None:

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/v4_runs.py

@@ -1494,7 +1494,16 @@ def _retry_refusal_step(run: Mapping[str, Any]) -> str:
 #: A source that kept refusing. Studio admits a person's named retry of these once its wait has
 #: passed and answers ``RUN_RETRY_TOO_SOON`` with a Retry-After before then (planning #1234), so
 #: this command sends them and lets Studio say when.
-PERSON_RETRY_CODES = frozenset({"SOURCE_RATE_LIMITED", "SOURCE_HTTP_5XX", "SOURCE_PROXY_BUSY"})
+PERSON_RETRY_CODES = frozenset(
+    {
+        "SOURCE_RATE_LIMITED",
+        "SOURCE_HTTP_5XX",
+        "SOURCE_PROXY_BUSY",
+        "TRANSPORT_FAILURE",
+        "TOTAL_TIMEOUT",
+        "DNS_TEMPORARY_FAILURE",
+    }
+)
 
 #: Failure codes a named retry may take without a recipe change. A hand copy of Studio's
 #: ``agent_recovery`` retry and platform sets. The server refuses ``retry_of_run_id`` for anything

{mostlyright_data-0.31.11 → mostlyright_data-0.31.13}/src/mostlyright/data_harness/thin/v4_search_contract.py

@@ -54,10 +54,57 @@ def _recipe_search_definition(recipe: dict[str, Any]) -> dict[str, Any]:
     return localize(recipe["$defs"]["table_plan"]["properties"]["search"])
 
 
-def build_bundle(table_search: dict[str, Any], recipe: dict[str, Any], pin: dict[str, Any]) -> dict:
-    """The closure of the definitions the client checks, from the vendored documents."""
+#: A reference into the common schema is carried into the bundle as a local definition named
+#: with this prefix, so the bundle stays one self-contained document.
+COMMON_REF = "common.schema.json#/$defs/"
+COMMON_PREFIX = "common."
 
-    definitions = table_search["$defs"]
+
+def _localize_common(value: Any) -> Any:
+    """``value`` with every common-schema reference pointing at its bundled local copy."""
+
+    if isinstance(value, list):
+        return [_localize_common(item) for item in value]
+    if not isinstance(value, dict):
+        return value
+    result = {}
+    for key, item in value.items():
+        if key == "$ref" and isinstance(item, str) and item.startswith(COMMON_REF):
+            result[key] = "#/$defs/" + COMMON_PREFIX + item[len(COMMON_REF) :]
+        else:
+            result[key] = _localize_common(item)
+    return result
+
+
+def build_bundle(
+    table_search: dict[str, Any],
+    recipe: dict[str, Any],
+    pin: dict[str, Any],
+    common: dict[str, Any] | None = None,
+) -> dict:
+    """The closure of the definitions the client checks, from the vendored documents. A
+    definition the search schema takes from the common schema is copied in under
+    :data:`COMMON_PREFIX`."""
+
+    def within_common(value: Any) -> Any:
+        """A common definition with its own local references pointing at their copies."""
+
+        if isinstance(value, list):
+            return [within_common(item) for item in value]
+        if not isinstance(value, dict):
+            return value
+        return {
+            key: (
+                "#/$defs/" + COMMON_PREFIX + _ref_name(item)
+                if key == "$ref" and isinstance(item, str) and item.startswith("#/$defs/")
+                else within_common(item)
+            )
+            for key, item in value.items()
+        }
+
+    definitions = dict(table_search["$defs"])
+    for name, definition in ((common or {}).get("$defs") or {}).items():
+        definitions[COMMON_PREFIX + name] = within_common(definition)
     recipe_member = "recipe_search"
     wanted = [SPEC, recipe_member, CASES]
     selected: dict[str, Any] = {recipe_member: _recipe_search_definition(recipe)}
@@ -66,6 +113,8 @@ def build_bundle(table_search: dict[str, Any], recipe: dict[str, Any], pin: dict
         if isinstance(value, dict):
             ref = value.get("$ref")
             if isinstance(ref, str):
+                if ref.startswith(COMMON_REF):
+                    ref = "#/$defs/" + COMMON_PREFIX + ref[len(COMMON_REF) :]
                 name = _ref_name(ref)
                 if name not in selected:
                     selected[name] = definitions[name]
@@ -81,6 +130,7 @@ def build_bundle(table_search: dict[str, Any], recipe: dict[str, Any], pin: dict
         if name not in selected:
             selected[name] = definitions[name]
             collect(definitions[name])
+    selected = {name: _localize_common(definition) for name, definition in selected.items()}
     return {
         "schema_version": BUNDLE_SCHEMA_VERSION,
         "source_commit": pin["source_commit"],
@@ -95,6 +145,7 @@ def bundle_from_vendor(vendor: Path = VENDOR) -> dict[str, Any]:
         json.loads((schemas / "table-search.schema.json").read_text(encoding="utf-8")),
         json.loads((schemas / "recipe.schema.json").read_text(encoding="utf-8")),
         json.loads((vendor / "client-v4-pin.json").read_text(encoding="utf-8")),
+        json.loads((schemas / "common.schema.json").read_text(encoding="utf-8")),
     )