mostlyright-data 0.29.3__tar.gz → 0.29.5__tar.gz

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.29.3
+Version: 0.29.5
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.29.3"
+version = "0.29.5"
 description = "Mostly Right hosted CLI for reviewed datasets"
 readme = "README.md"
 requires-python = ">=3.11"

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/skills/mr-data-build/references/3-probe-read-a-source-before-committing-to-it.md

@@ -108,10 +108,25 @@ not an answer yet, so read it back again later. A status of `truncated` is an an
 a limit: keep its limits and its evidence with whatever you record, and never call it complete. The
 command takes no key and no Reader options, so a source that needs either, or that is not a CSV, is
 read by the run below instead. So is any source that answers `capability_unavailable`, which means
-this deployment cannot inspect it: do not retry it, and do not guess in its place. What the look did
-observe belongs in that feed's `source` block, in `coverage`, as one plain line: what the sample
-showed (the format and the columns) and how much of the file it covered (the rows and bytes read,
-and whether it was cut short).
+this deployment cannot inspect it: do not retry it, and do not guess in its place.
+
+**Write down what each look found.** An inspection is not a run, and nothing on the page shows what
+it found unless you write it down. Once a look has shown you the start of a file, write one
+`mr-data dataset note DATASET_ID` for that source. The heading names the source and the outcome;
+writing the same heading again revises that cell instead of adding one, so two sources need two
+headings. The body says what the sample showed (the format and the columns) and how much of the file
+it covered (the rows and bytes read, and whether it was cut short). Then count the source in
+`--fact sources_probed` the next time you set the activity line.
+
+```sh
+mr-data dataset note DATASET_ID \
+  --heading "Gauge file A is a CSV with four columns, cut short at 1,000 rows" \
+  --json <<'EOF'
+The look read the first 1,000 rows (41,208 bytes) of the file and stopped at the row limit, so it
+describes the start of the file and not the whole file. It is a CSV with four columns: station_id,
+observed_at, stage_m and quality_flag.
+EOF
+```
 
 **Both obey the ordinary gate.** Registering needs the brief answered or delegated, so the run
 happens AFTER stage 2's answers arrive — or at once under a delegation — and an inspection, which

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/skills/mr-data-build/references/making-a-table-incremental.md

@@ -560,3 +560,70 @@ again. Ask the user before passing either flag. Never pass one to clear a refusa
 | calls on the first full after the change | every hit whose answer is not held | each hit that passes the filter, once, because that full reads fresh |
 | calls on each weekly rebuild | every hit whose answer is not held | only new hits that pass the filter; the held answers are carried forward |
 | `pages.max_fetches_per_run` | 36,874 | the forecast's suggestion after the first filtered run |
+
+#### Keyless API windows (Studio 4.26)
+
+Check `studio_capabilities.keyless_window` is literal `true` before using this grammar.
+An absent or false capability means "not available on this Studio yet". Registration and
+file explanation refuse with `THIN_GRAMMAR_NOT_SERVED` before local lint or registration;
+an older API's schema 422 on the explanation also refuses. No dataset delegation is written.
+The capability covers `public.https.api@1.0.0`, `generic_api.auth.kind: none`, endpoint
+`epoch_millis`, the `epoch_millis_value` partition key, and the two request controls below. Existing credential-free file and collection
+adapters do not require this capability.
+
+For a public JSON/CSV API use `credential_mode: none`, an HTTPS `origin`, and `generic_api`
+with exact `auth: {"kind":"none"}`. Requests are GET-only, with no body or credential reference.
+Authenticated adapters retain their existing credential requirements. Declare both window
+endpoints with `location: parameter` and corresponding required request parameters. Origin,
+host allowlists and cursor next-URL restrictions still apply.
+
+For Binance BTCUSDT one-minute klines, start the backfill at `2017-08-17T00:00:00Z` and request
+`/api/v3/klines` with `symbol=BTCUSDT`, `interval=1m`, `limit=1000`, and parameter templates for
+`startTime` and `endTime`. Use this request window:
+
+```json
+{
+  "request": {
+    "start": {"location": "parameter", "encoding": "epoch_millis", "bound": "inclusive", "parameter": "startTime"},
+    "end": {"location": "parameter", "encoding": "epoch_millis", "bound": "inclusive", "parameter": "endTime"},
+    "max_slice_seconds": 43200,
+    "min_interval_milliseconds": 1000
+  }
+}
+```
+
+Keep `granularity: day`, `timezone: UTC` and `merge.materialization: partition_replace`.
+For the acquired `open_time_ms` column declare
+`merge.partition: {"column":"open_time_ms","key":"epoch_millis_value"}` and
+`merge.row_identity: ["open_time_ms"]`. This encoding maps the integer millisecond cell to
+its UTC calendar day before transforms; it floors negative epochs correctly too.
+The 43,200-second maximum cuts each UTC day into two 12-hour requests, each with at most 720
+one-minute bars under the publisher's 1,000-row cap. Slices are requests, never new partitions.
+An inclusive millisecond end renders the next slice boundary minus 1 ms; adjacent slices do not
+overlap. An exclusive start renders the start minus 1 ms. The older date/epoch-second encodings
+retain their day-step conversions. Slice bounds require both endpoints to use milliseconds.
+`max_slice_seconds` is 1–86,400; `window.request.min_interval_milliseconds` is 1–3,600,000 and is supported on
+the new public generic adapter, defaulting to 1,000 when omitted. Pacing applies across the
+entire walk, including pagination, rather than resetting at each day.
+
+After the full backfill, choose an hourly cadence in the agreed brief. Refresh re-reads the
+current UTC day, flooring a catch-up start to midnight when prior coverage reached the next
+midnight; it replaces that day's partition. Do not set `complete_period_only` when the open day
+is needed. Size `limits.max_requests` for every slice and every pagination page across the full
+walk; the global budget must allow it to finish. The 4,096-partition ceiling still counts days.
+Python and Keel plan the same windows; large declared relations use the admitted Keel route,
+with each day still subject to its in-memory ceiling. A declaration is not evidence of a
+completed large ingestion: verify actual coverage from the run.
+
+The [Reader example](readers.md#needs-studio-426) maps Binance's root array of 12-column arrays.
+An explicit `table.partition` may use `{"column":"open_time_ms","encoding":"epoch_millis_value"}`;
+that table encoding also requires `keyless_window`. The native refresh subset preserves this
+partition column with exact `CAST(open_time_ms AS BIGINT) AS open_time_ms`; its current subset
+requires `checks: []`. Verify the built values and coverage with bounded queries and receipts.
+Bind the real dataset ID before Harness registration, following the brief workflow; the fixture
+omits an ID rather than inventing an existing dataset.
+The raw millisecond timestamp uses `epoch_millis_value`, never `iso_date_prefix`;
+a transform does not run before source partition merging. Existing JSON cursor pagination is reused unchanged. XML readers
+do not add extraction of arXiv OAI-PMH XML continuation tokens to JSON-pointer pagination.
+Wikipedia's existing `public.https` date-in-path source remains a literal full fetch; this
+example does not prove an entire historical date-path backfill.

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/skills/mr-data-build/references/readers.md

@@ -119,7 +119,7 @@ decoder. Zero matches and multiple matches both refuse.
 | `raster.geospatial@1.0.0` | `format`, exact native `crs`, and an `x`/`y`/`width`/`height` window. |
 | `satellite.sentinel_safe@1.0.0` | SAFE ZIP with exact spectral/quality member names, native CRS, quality layer, and an `x`/`y`/`width`/`height` window. |
 | `image.rendered@1.0.0` | `format` plus a bounded `x`/`y`/`width`/`height` rendered-image crop. |
-| `scientific.fits@1.0.0` | `band` plus an exact native `x`/`y`/`width`/`height` image window. |
+| `scientific.fits@1.0.0` | `band_id: "195"` plus an exact native `x`/`y`/`width`/`height` image window. |
 | `satellite.himawari_hsd@1.0.0` | Certified HSD v1.3 single-segment B13 plus an exact native `x`/`y`/`width`/`height` radiance window. |
 | `satellite.himawari_hsd_complete@1.0.0` | Complete HSD v1.3 scan ZIP: `band_id`, exact ordered `segment_members`, and a native `x`/`y`/`width`/`height` radiance window. |
 | `satellite.xrit_image@1.0.0` | Exact ZIP prologue, eight ordered HRIT/LRIT image segments and epilogue; sealed channel/grid/navigation plus a bounded raw-count window. |
@@ -140,3 +140,48 @@ settings. Spreadsheet receipts also identify stored calculated values.
 Pinned HTTPS Reader sources retrieve, decode, and parse in one attested Clean room call. JSON,
 NDJSON, ZIP, TAR, GZIP, XLSX, delimited text, and single-record GRIB2 are supported when the
 response matches the family's exact media-type, settings, and budget contract.
+
+### Needs Studio 4.26
+
+Check first: run `mr-data recipe explain RECIPE.json --json` and read `studio_capabilities`.
+Use `public.https.api@1.0.0` only when `keyless_window` is literal `true`; otherwise report
+"not available on this Studio yet". Registration and file explanation refuse with
+`THIN_GRAMMAR_NOT_SERVED`, including an older API's schema 422 before capabilities.
+
+The new adapter uses the existing `generic_api.response` Reader contract. For Binance klines,
+state `format: json`, `records_pointer: ""`, and `reader.family_id: json.tabular` with
+`family_version: 1.0.0` (or the already admitted `1.1.0`). Pin `options_digest` to the SHA-256
+of the canonical decode options, rather than inserting a placeholder digest. These decode
+options project a root JSON array of arrays, with RFC 6901 numeric indexes into each row:
+
+```json
+{
+  "document_format": "json",
+  "records_pointer": "",
+  "expand": [],
+  "columns": [
+    {"name": "open_time_ms", "pointer": "/0"},
+    {"name": "open", "pointer": "/1"},
+    {"name": "high", "pointer": "/2"},
+    {"name": "low", "pointer": "/3"},
+    {"name": "close", "pointer": "/4"},
+    {"name": "volume", "pointer": "/5"},
+    {"name": "close_time_ms", "pointer": "/6"},
+    {"name": "quote_volume", "pointer": "/7"},
+    {"name": "trades", "pointer": "/8"},
+    {"name": "taker_buy_base_volume", "pointer": "/9"},
+    {"name": "taker_buy_quote_volume", "pointer": "/10"},
+    {"name": "unused", "pointer": "/11"}
+  ]
+}
+```
+
+Every column is required by default. The Reader projects values; it does not turn an epoch
+millisecond cell into an ISO date. Declare numeric/decimal types in the table transform and
+declare `window.merge.partition: {"column":"open_time_ms","key":"epoch_millis_value"}`.
+The key maps the acquired integer millisecond value to its UTC day before transforms;
+`merge.row_identity` is `["open_time_ms"]`. It also needs `keyless_window`. See
+[the cap-safe window](making-a-table-incremental.md#keyless-api-windows-studio-426) for the
+backfill, hourly cadence, shared Python/Keel windows and request limits. Cursor pagination
+continues to use its existing JSON next-token pointer; XML OAI-PMH token parsing is not supplied
+by this mapping.

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/skills/mr-data-build/references/sources.md

@@ -271,3 +271,27 @@ its upstream cannot refresh incrementally. A `closed` or `reference` source may
 table, and a refresh restores it from the predecessor (ADR 0061). Do not split a table
 to separate them. No per-row collection may derive its rows from a `closed` or `reference` source.
 [Which plans refresh](making-a-table-incremental.md#which-plans-refresh) has the rest.
+
+### Needs Studio 4.26
+
+Check first: run `mr-data recipe explain RECIPE.json --json` and read `studio_capabilities`.
+Use the following only when `keyless_window` is literal `true`; otherwise report "not available
+on this Studio yet". The client refuses with `THIN_GRAMMAR_NOT_SERVED` before registration,
+including when an older API rejects the explanation document with schema 422.
+
+`public.https.api@1.0.0` is the credential-free generic HTTPS API adapter. State
+`credential_mode: none`, `origin`, and `generic_api` with exact `auth: {"kind":"none"}`.
+Use GET only: no request body, credential reference, token, secret header or secret query value.
+This adapter does not relax authentication on the keyed adapters. The owned HTTPS origin,
+host allowlist, bounded request budgets and cursor next-URL restrictions remain in force.
+The existing generic response mapping and JSON/CSV pagination apply; a cursor needs the
+publisher's actual next-token pointer. JSON-pointer pagination does not extract XML OAI-PMH tokens.
+
+For Binance klines use parameter-windowed `startTime`/`endTime`, millisecond endpoints and
+12-hour slices at one request per second. Follow the backfill and hourly-refresh example in
+[making a table incremental](making-a-table-incremental.md#keyless-api-windows-studio-426)
+and the root-array projection in [Readers](readers.md#needs-studio-426). Keep the day partition
+with `epoch_millis_value` on the raw millisecond time column, and size the request ceiling
+for all slices and pages. Do not substitute a whole-day query:
+1,440 one-minute bars exceed Binance's 1,000-row response cap. Existing credential-free
+`public.https` file fetches and `public.https.collection@2.0.0` keep their own contracts.

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/canonical.py

@@ -14,6 +14,7 @@ from typing import Any, NoReturn
 
 MAX_CANONICAL_DEPTH = 64
 MAX_CANONICAL_MEMBERS = 100_000
+_MAX_INTEGER_LITERAL_LENGTH = len(str(-(2**63)))
 MAX_CANONICAL_STRING_LENGTH = 1_000_000
 MAX_CANONICAL_INTEGER = (1 << 63) - 1
 
@@ -35,7 +36,10 @@ def parse_json(value: str | bytes | bytearray | memoryview) -> Any:
 
 
 def parse_foreign_json(
-    value: str | bytes | bytearray | memoryview, *, max_members: int = MAX_CANONICAL_MEMBERS
+    value: str | bytes | bytearray | memoryview,
+    *,
+    max_members: int = MAX_CANONICAL_MEMBERS,
+    wide_integers: bool = False,
 ) -> Any:
     """Parse a third-party document with the same hardening, but accept fractional numbers.
 
@@ -59,9 +63,14 @@ def parse_foreign_json(
 
     ``max_members`` raises the member ceiling for one document whose size a route already bounds,
     such as a table manifest listing thousands of parts.
+
+    ``wide_integers`` reads an integer literal outside the signed 64-bit range as a finite float
+    rather than refusing the document. It is for an answer that carries a table's own values from
+    a publisher that writes a whole-valued double without a fraction or an exponent, so ``1e20``
+    arrives as ``100000000000000000000``. An integer inside the range is still an ``int``.
     """
 
-    return _parse(value, fractions="accept", max_members=max_members)
+    return _parse(value, fractions="accept", max_members=max_members, wide_integers=wide_integers)
 
 
 def _parse(
@@ -69,6 +78,7 @@ def _parse(
     *,
     fractions: str,
     max_members: int = MAX_CANONICAL_MEMBERS,
+    wide_integers: bool = False,
 ) -> Any:
     if isinstance(value, str):
         text = value
@@ -98,7 +108,7 @@ def _parse(
         parsed = json.loads(
             text,
             object_pairs_hook=_reject_duplicate_pairs,
-            parse_int=_parse_integer,
+            parse_int=_parse_wide_integer if wide_integers else _parse_integer,
             parse_float=_parse_fraction if accept_fractions else _reject_fraction,
             parse_constant=_reject_constant,
         )
@@ -318,6 +328,16 @@ def _parse_integer(value: str) -> int:
     return parsed
 
 
+def _parse_wide_integer(value: str) -> int | float:
+    """Read an integer literal, as a float when it is outside the signed 64-bit range."""
+
+    if len(value) <= _MAX_INTEGER_LITERAL_LENGTH:
+        parsed = int(value)
+        if -MAX_CANONICAL_INTEGER - 1 <= parsed <= MAX_CANONICAL_INTEGER:
+            return parsed
+    return _parse_fraction(value)
+
+
 def _parse_fraction(value: str) -> float:
     """Read a fractional literal from a foreign document, refusing a non-finite one."""
 

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/table_manifest.py

@@ -101,7 +101,7 @@ _SUPPORTED_MANIFEST_SCHEMAS = frozenset((TABLE_MANIFEST_SCHEMA, TABLE_MANIFEST_S
 #: ``iso_month_prefix`` tables read as TABLE_MANIFEST_INVALID on 0.28.0 (planning#807).
 #: ``tests/test_thin_table_manifest_encodings.py`` holds this equal to the vendored recipe schema.
 MANIFEST_PARTITION_ENCODINGS: frozenset[str] = frozenset(
-    {"value", "iso_date_prefix", "iso_date_value", "iso_month_prefix"}
+    {"value", "iso_date_prefix", "iso_date_value", "iso_month_prefix", "epoch_millis_value"}
 )
 
 #: The two media types the format adds to the closed artifact vocabulary.

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/recipe.py

@@ -1181,13 +1181,21 @@ def register(
 ) -> dict[str, Any]:
     """``mr-data recipe FILE``: register one document and print what the server derived from it.
 
-    The order is the order the answers cost what they cost. The document is read and linted with
-    no credential resolved and nothing sent, so an author whose checks are malformed learns it
-    before a token is minted. The brief gate comes next, because it is one GET and it decides
-    whether this registration should happen at all. The registration itself is last.
+    Existing documents are linted before a credential is resolved. A keyless-window document
+    asks what Studio serves before local lint, because an older schema may reject the grammar
+    before reporting capabilities. The brief and delegation gates follow; registration is last.
     """
 
     document = read_document(Path(args.target))
+    # New grammar must be refused as unserved before local lint can diagnose its shape.
+    keyless = any(
+        capability == "keyless_window" for _, capability in recipe_explain.gated_grammar(document)
+    )
+    selected = client
+    served = []
+    if keyless:
+        selected = selected or _client(args)
+        served = recipe_explain.registration_gate(selected, document)
     check_document(document)
     if not getattr(args, "no_lint", False):
         refusal = lint_refusal(
@@ -1239,13 +1247,14 @@ def register(
         )
     delegated = getattr(args, "delegated", None)
     blocks = delegation_blocks(delegated) if delegated is not None else None
-    selected = client or _client(args)
+    selected = selected or _client(args)
     # ⚠ THE CAPABILITY GATE COMES BEFORE ANYTHING IS WRITTEN. A document using grammar this
     # Studio does not serve (`row_filter`, a per-row `interval` revisit) is refused here, before
     # the delegation append below and before the registration, so a refusal leaves nothing
     # behind. It asks Studio only when the document uses such grammar or a snapshot window, so
     # every other registration makes exactly the calls it made before.
-    served = recipe_explain.registration_gate(selected, document)
+    if not keyless:
+        served = recipe_explain.registration_gate(selected, document)
     warnings += served
     warnings += sampled
     warning_codes = [recipe_explain.SNAPSHOT_WILL_NOT_REFRESH_YET] if served else []

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/recipe_explain.py

@@ -125,6 +125,7 @@ CAPABILITY_NAMES: tuple[str, ...] = (
     "fresh_read_control",
     "snapshot_refresh",
     "row_request_batch",
+    "keyless_window",
 )
 
 #: What a Studio that answers 404 or 405 from the explanation routes serves (README D27).
@@ -1246,6 +1247,17 @@ def explain_document_at_studio(
     except ThinLaneError as refusal:
         if route_unavailable(refusal):
             return None
+        keyless = [pointer for pointer, capability in gated_grammar(document)
+                   if capability == "keyless_window"]
+        if keyless and getattr(refusal, "http_status", None) == 422:
+            # Older API schemas reject the document before they can answer capabilities.
+            # A schema rejection is never evidence that this grammar is served.
+            raise ThinLaneError(
+                GRAMMAR_NOT_SERVED_CODE,
+                f"not available on this Studio yet: {', '.join(keyless)} "
+                "(Studio rejected the explanation document before reporting capabilities). "
+                "Nothing was registered. Wait for the Studio deploy, or remove the member",
+            ) from refusal
         raise
     return _explanation(answer, EXPLAIN_REFRESH_PATH)
 
@@ -1286,14 +1298,42 @@ def gated_grammar(document: Mapping[str, Any]) -> list[tuple[str, str]]:
 
     ``row_filter`` on any collection discovery, and a per-row request collection whose
     ``pages.revisit`` states ``interval`` (4.14.0). A revisit on any other collection is old
-    grammar. ``batch`` on a per-row request (4.15.0).
+    grammar. ``batch`` on a per-row request (4.15.0). The public generic API, no-auth kind,
+    millisecond endpoints and slice/pacing members (4.26.0) each need ``keyless_window``.
     """
 
     gated: list[tuple[str, str]] = []
+    table = document.get("table")
+    partition = table.get("partition") if isinstance(table, Mapping) else None
+    if isinstance(partition, Mapping) and partition.get("encoding") == "epoch_millis_value":
+        gated.append(("/table/partition/encoding", "keyless_window"))
     sources = document.get("sources")
     for index, source in enumerate(sources if isinstance(sources, list) else ()):
         if not isinstance(source, Mapping):
             continue
+        prefix = f"/sources/{index}"
+        connector = source.get("connector")
+        if isinstance(connector, Mapping):
+            if connector.get("adapter_id") == "public.https.api@1.0.0":
+                gated.append((f"{prefix}/connector/adapter_id", "keyless_window"))
+            generic = connector.get("generic_api")
+            auth = generic.get("auth") if isinstance(generic, Mapping) else None
+            if isinstance(auth, Mapping) and auth.get("kind") == "none":
+                gated.append((f"{prefix}/connector/generic_api/auth/kind", "keyless_window"))
+        window = source.get("window")
+        merge = window.get("merge") if isinstance(window, Mapping) else None
+        partition = merge.get("partition") if isinstance(merge, Mapping) else None
+        if isinstance(partition, Mapping) and partition.get("key") == "epoch_millis_value":
+            gated.append((f"{prefix}/window/merge/partition/key", "keyless_window"))
+        window_request = window.get("request") if isinstance(window, Mapping) else None
+        if isinstance(window_request, Mapping):
+            for endpoint in ("start", "end"):
+                value = window_request.get(endpoint)
+                if isinstance(value, Mapping) and value.get("encoding") == "epoch_millis":
+                    gated.append((f"{prefix}/window/request/{endpoint}/encoding", "keyless_window"))
+            for member in ("max_slice_seconds", "min_interval_milliseconds"):
+                if member in window_request:
+                    gated.append((f"{prefix}/window/request/{member}", "keyless_window"))
         collection = source.get("collection")
         if not isinstance(collection, Mapping):
             continue
@@ -1348,6 +1388,16 @@ def registration_gate(client: StudioV4Client, document: Mapping[str, Any]) -> li
     if not gated and not snapshots:
         return []
     answer = explain_document_at_studio(client, document)
+    return grammar_gate(document, answer)
+
+
+def grammar_gate(
+    document: Mapping[str, Any], answer: Mapping[str, Any] | None
+) -> list[str]:
+    """Admit only grammar explicitly served by an explanation; perform no writes."""
+
+    gated = gated_grammar(document)
+    snapshots = snapshot_sources(document)
     capabilities = served_capabilities(answer)
     missing = [pointer for pointer, capability in gated if not capabilities[capability]]
     if missing:
@@ -1455,10 +1505,15 @@ def explain(args: argparse.Namespace, *, client: StudioV4Client | None = None) -
         answer = explain_table_at_studio(selected, registered["table_id"])
     else:
         document = recipe_module.read_document(Path(target))
+        selected = client or recipe_module._client(args)
+        keyless = any(capability == "keyless_window" for _, capability in gated_grammar(document))
+        if keyless:
+            answer = explain_document_at_studio(selected, document)
+            grammar_gate(document, answer)
         recipe_module.check_document(document)
         subject = {"kind": "document", "table_id": None, "recipe_id": None, "recipe_digest": None}
-        selected = client or recipe_module._client(args)
-        answer = explain_document_at_studio(selected, document)
+        if not keyless:
+            answer = explain_document_at_studio(selected, document)
     capabilities = served_capabilities(answer)
     # The mirror is judged under the fence Studio states, so the two can only disagree about
     # the document; with no Studio to ask the fence reads closed, as a pre-4.14.0 Studio runs.

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/session.py

@@ -26,7 +26,12 @@ from typing import Any
 from urllib.parse import quote, urlencode
 from uuid import UUID
 
-from mostlyright.data_harness.canonical import CanonicalJSONError, parse_json
+from mostlyright.data_harness.canonical import (
+    MAX_CANONICAL_MEMBERS,
+    CanonicalJSONError,
+    parse_foreign_json,
+    parse_json,
+)
 from mostlyright.data_harness.thin.transport import (
     MAX_JSON_RESPONSE_BYTES,
     ThinLaneError,
@@ -70,6 +75,9 @@ CLOUD_DATASETS_LIST_PATH = "/api/cli/datasets"
 # Projection repair after a Studio metadata write, without a publication mutation.
 CLOUD_DATASET_IDENTITY_PATH = "/api/cli/datasets/{dataset_id}/identity"
 
+# Shared UI/CLI archive orchestration, including public projection withdrawal.
+CLOUD_DATASET_ARCHIVE_PATH = "/api/cli/datasets/{dataset_id}/archive"
+
 #: The one Cloud address a Dataset has. ``mostlyright-cloud`` serves it at this route with
 #: section suffixes, and a run has no page of its own: its evidence lives on this page's Harness
 #: card, so a receipt names its run with :func:`run_dashboard_url` as a query member here rather
@@ -620,6 +628,8 @@ class CloudClient:
         expected: tuple[int, ...] = (200,),
         absent: str | None = None,
         maximum: int = MAX_TOKEN_RESPONSE_BYTES,
+        table_values: bool = False,
+        max_members: int = MAX_CANONICAL_MEMBERS,
     ) -> dict[str, Any]:
         """Make one Cloud call and return the object it answered with, or raise the refusal.
 
@@ -633,6 +643,14 @@ class CloudClient:
 
         ``absent`` is what this route's own 404 means, in the caller's words; see
         :func:`cloud_refusal` for why one sentence could not serve both routes.
+
+        ``table_values`` is for a route whose answer carries a table's own values. It admits a
+        finite fractional number, because a DOUBLE column or an average is not an integer, and it
+        reads an integer literal outside the signed 64-bit range as a float, because Cloud writes
+        a whole-valued double without a fraction. ``max_members`` is that answer's member ceiling,
+        for a route that states how large a page it may be handed. Every other hardening is
+        unchanged, and the default refuses all of it as it does for every answer this lane reads
+        as a document of its own.
         """
 
         headers = {
@@ -651,7 +669,11 @@ class CloudClient:
             maximum,
         )
         try:
-            parsed = parse_json(raw)
+            parsed = (
+                parse_foreign_json(raw, max_members=max_members, wide_integers=True)
+                if table_values
+                else parse_json(raw)
+            )
         except CanonicalJSONError:
             parsed = None
         if status not in expected:
@@ -668,6 +690,7 @@ __all__ = [
     "CLOUD_APPROVAL_DASHBOARD_PATH",
     "CLOUD_DATASETS_LIST_PATH",
     "CLOUD_DATASET_ACCESS_PATH",
+    "CLOUD_DATASET_ARCHIVE_PATH",
     "CLOUD_DATASET_DASHBOARD_PATH",
     "CLOUD_DATASET_RESOLVED_SCHEMA",
     "CLOUD_DATASET_RESOLVE_PATH",

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/studio_capabilities.py

@@ -36,12 +36,14 @@ FRESH_READ_CONTROL = "fresh_read_control"
 SNAPSHOT_REFRESH = "snapshot_refresh"
 #: Served by every 4.15.0 API, which admits ``batch`` on a per-row request. An older API omits it.
 ROW_REQUEST_BATCH = "row_request_batch"
+KEYLESS_WINDOW = "keyless_window"
 CAPABILITY_NAMES: tuple[str, ...] = (
     ROW_FILTER,
     PER_ROW_REVISIT_INTERVAL,
     FRESH_READ_CONTROL,
     SNAPSHOT_REFRESH,
     ROW_REQUEST_BATCH,
+    KEYLESS_WINDOW,
 )
 
 #: What a Studio that serves neither explanation route serves.
@@ -96,6 +98,14 @@ def document_capabilities(client: StudioV4Client, document: Mapping[str, Any]) -
     read. Studio judges the document alone and writes nothing.
     """
 
+    from mostlyright.data_harness.thin.recipe_explain import (
+        explain_document_at_studio,
+        gated_grammar,
+    )
+
+    if any(capability == KEYLESS_WINDOW for _, capability in gated_grammar(document)):
+        # Share the older-schema 422 translation with registration and file explanation.
+        return served_capabilities(explain_document_at_studio(client, document))
     try:
         answer = client.explain_recipe_refresh(document)
     except ThinLaneError as refusal:
@@ -109,6 +119,7 @@ __all__ = [
     "CAPABILITY_NAMES",
     "FRESH_READ_CONTROL",
     "GRAMMAR_NOT_SERVED_CODE",
+    "KEYLESS_WINDOW",
     "NO_CAPABILITIES",
     "PER_ROW_REVISIT_INTERVAL",
     "ROW_FILTER",

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/v4.py

@@ -147,8 +147,8 @@ from mostlyright.data_harness.thin.transport import ThinLaneError
 #: ``JOB_INVALID`` outright, which in production was every refresh a collection epoch was offered
 #: for rather than only the collection runs. This package must not reach a Studio older than the
 #: commit it pins; ``docs/V4-WORKER-PROTOCOL.md`` states it beside the layout's own ordering rule.
-PINNED_V4_OPENAPI_SOURCE_SHA256 = "80ff61057468d16bd97a071ba8f0acfff0050deb834ca979a9489bfc5ef1da13"
-PINNED_V4_CONTRACT_VERSION = "4.19.0"
+PINNED_V4_OPENAPI_SOURCE_SHA256 = "1f30e109b81f6aa55c00c8999fd877b6678b7e0bea617707d26e7a040380f98b"
+PINNED_V4_CONTRACT_VERSION = "4.27.0"
 
 #: The ``schema_version`` every command body that predates 4.14.0 stamps: ``dataset activity``,
 #: ``table rename``, ``table archive`` and ``dataset archive``. It is the number the 0.25 line

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/v4_datasets.py

@@ -127,6 +127,7 @@ from mostlyright.data_harness.thin.narrative import (
 from mostlyright.data_harness.thin.runs import new_idempotency_key
 from mostlyright.data_harness.thin.session import (
     CLOUD_DATASET_ACCESS_PATH,
+    CLOUD_DATASET_ARCHIVE_PATH,
     CLOUD_DATASET_IDENTITY_PATH,
     CLOUD_DATASET_RESOLVE_PATH,
     CLOUD_DATASET_RESOLVED_SCHEMA,
@@ -149,7 +150,6 @@ from mostlyright.data_harness.thin.stream import RunEventStream
 from mostlyright.data_harness.thin.transport import ThinLaneError, Transport, UrlLibTransport
 from mostlyright.data_harness.thin.v4 import (
     ARCHIVE_DATASET_PATH,
-    COMMAND_REQUEST_VERSION_FLOOR,
     CREATE_DATASET_PATH,
     DATASET_CURRENT_STATE_PATH,
     DATASET_EVENT_HISTORY_PATH,
@@ -448,6 +448,14 @@ class CloudDatasetAccessClient(CloudClient):
     credential header and the refusal folding to drift.
     """
 
+    def archive_dataset(self, dataset_id: str, confirmation_name: str) -> dict[str, Any]:
+        return self.call(
+            "POST",
+            CLOUD_DATASET_ARCHIVE_PATH.format(dataset_id=dataset_id),
+            act="archive this dataset and withdraw its public listings",
+            body={"confirmation_name": confirmation_name},
+        )
+
     def sync_dataset_identity(self, dataset_id: str) -> dict[str, Any]:
         """Carry Studio's current metadata into Cloud's existing public projection."""
 
@@ -458,7 +466,6 @@ class CloudDatasetAccessClient(CloudClient):
             body={},
         )
 
-
     def list_datasets(self, *, limit: int | None = None) -> dict[str, Any]:
         """Name every dataset this workspace holds, most recently updated first.
 
@@ -1838,75 +1845,46 @@ def write(
 
 
 def archive(
-    args: argparse.Namespace, *, client: StudioV4DatasetClient | None = None
+    args: argparse.Namespace, *, client: CloudDatasetAccessClient | None = None
 ) -> dict[str, Any]:
-    """``mr-data dataset archive DATASET_ID --confirm-name TITLE``: retire the page, free the name.
-
-    ⚠ THE TYPED TITLE IS COMPARED HERE AS WELL AS BY STUDIO, WHICH IS THE OPPOSITE CHOICE FROM
-    ``mr-data table archive`` AND THE REASON IS THAT THE READ IS ALREADY HAPPENING. This write is
-    fenced on the dataset's version, so the page has to be read before it can be sent; the title
-    arrives in the same body, the comparison costs nothing, and the fence is what makes it safe --
-    a title that changed after the read moves the version, and the conditional refuses. Where a
-    read would have to be bought only to give a second opinion, this lane does not buy it.
-
-    ⚠ AND IT IS NOT A DELETION. Runs, artifacts, versions and the decision record all stay. What
-    the command ends is the page's claim on its title and its place in the workspace's listing.
-    """
-
+    """Use the UI's Cloud archive ceremony; a Studio-only tombstone is incomplete."""
     dataset_id = identifier(getattr(args, "dataset_id", None) or "", "dataset identifier")
     confirmation = getattr(args, "confirm_name", None)
     if not isinstance(confirmation, str) or not confirmation.strip():
         raise ThinLaneError(
             "THIN_REQUEST_INVALID",
             "mr-data dataset archive needs the dataset's current title typed out: "
-            f"--confirm-name TITLE. mr-data dataset show {dataset_id} prints the title Studio "
-            "holds",
+            "--confirm-name TITLE",
         )
     stated = confirmation.strip()
-    selected = client or _client(args)
-    record = selected.dataset(dataset_id)
-    version = _version(record)
-    held = record.get("name")
-    if isinstance(held, str) and held.strip() != stated:
+    if client is None:
+        credentials = resolve_cloud_credentials()
+        require_device_key(credentials)
+        selected = CloudDatasetAccessClient(credentials)
+    elif isinstance(client, CloudDatasetAccessClient):
+        selected = client
+    else:
         raise ThinLaneError(
-            "THIN_CONFIRMATION_MISMATCH",
-            f"this dataset is called {held!r} and --confirm-name said {stated!r}. Archiving "
-            "releases the title, so the title is what confirms it: type the one on the page, or "
-            "check you have the dataset you meant",
-        )
-    try:
-        archived = selected.archive_dataset(
-            dataset_id,
-            body={
-                "schema_version": COMMAND_REQUEST_VERSION_FLOOR,
-                "workspace_id": str(selected.session.workspace_id),
-                "confirmation_name": stated,
-            },
-            expected_version=version,
+            "THIN_REQUEST_INVALID", "dataset archive requires the Cloud archive client"
         )
-    except ThinLaneError as refusal:
-        if not stale_version(refusal):
-            raise
-        # ⚠ NAMED, AND NOT RETRIED, WHICH IS THE ONE PLACE THIS DIFFERS FROM `write` AND `rename`.
-        # Those two replay at the new version because what they write does not depend on what
-        # changed underneath them. This does: the title typed to confirm the archive is exactly
-        # what a racing `dataset set` may have just rewritten, so a silent retry could archive a
-        # page under a title that is no longer its own. What it gets instead is the same sentence
-        # the other two end on, because it is the same thing that happened.
+    answer = selected.archive_dataset(dataset_id, stated)
+    if (
+        answer.get("schema_version") != "mostlyright-dataset-archive.v1"
+        or answer.get("status") != "dataset_archived"
+        or answer.get("dataset_id") != dataset_id
+        or answer.get("name") != stated
+    ):
         raise ThinLaneError(
-            "THIN_DATASET_VERSION_CONFLICT",
-            f"the dataset moved between the read at version {version} and this archive; somebody "
-            f"else is editing it. Read it with mr-data dataset show {dataset_id}, check the title "
-            "is still the one you typed, and archive again",
-        ) from refusal
+            "THIN_RESPONSE_INVALID",
+            "Cloud did not confirm this dataset's complete archive",
+        )
     return {
         "schema_version": ARCHIVE_SCHEMA,
         "status": "dataset_archived",
         "lane": "hosted",
-        "dataset_id": _dataset_id(archived, verb="archive") if archived else dataset_id,
+        "dataset_id": dataset_id,
         "name": stated,
-        "version": archived.get("version"),
-        "dashboard_url": dataset_dashboard_url(selected.session.cloud_url, dataset_id),
+        "dashboard_url": dataset_dashboard_url(selected.cloud_url, dataset_id),
         "note": (
             f"The dataset is retired and the title {stated!r} is free for another one. Nothing "
             "built was deleted: every run, artifact, version and decision cell stays where it is."
@@ -2488,7 +2466,6 @@ def watch(
     }
 
 
-
 def listing(
     args: argparse.Namespace, *, client: CloudDatasetAccessClient | None = None
 ) -> dict[str, Any]:

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/thin/v4_public.py

@@ -867,6 +867,20 @@ CLOUD_PUBLIC_TABLE_DOWNLOAD_URL_PATH = "/api/cli/public/tables/{table_id}/downlo
 QUERY_SCHEMA = f"{THIN_SCHEMA_PREFIX}-v4-public-query.v1"
 DOWNLOAD_URL_SCHEMA = f"{THIN_SCHEMA_PREFIX}-v4-public-download-url.v1"
 
+#: Cloud bounds one credentialled query answer at 10 000 rows of at most 64 columns, and at 8 MiB
+#: of JSON for the rows. The ceilings here admit the widest page those bounds allow: the row bytes
+#: plus room for the members around them, and one member per value, per row and per member around
+#: the rows.
+MAX_PUBLIC_QUERY_ROWS = 10_000
+MAX_PUBLIC_QUERY_COLUMNS = 64
+MAX_PUBLIC_QUERY_ROW_BYTES = 8 * 1024 * 1024
+MAX_PUBLIC_QUERY_ENVELOPE_BYTES = 1024 * 1024
+MAX_PUBLIC_QUERY_ENVELOPE_MEMBERS = 16_384
+MAX_PUBLIC_QUERY_RESPONSE_BYTES = MAX_PUBLIC_QUERY_ROW_BYTES + MAX_PUBLIC_QUERY_ENVELOPE_BYTES
+MAX_PUBLIC_QUERY_RESPONSE_MEMBERS = (
+    MAX_PUBLIC_QUERY_ROWS * (MAX_PUBLIC_QUERY_COLUMNS + 1) + MAX_PUBLIC_QUERY_ENVELOPE_MEMBERS
+)
+
 
 class PublicQueryClient(CloudClient):
     """Authenticated public table query + download-url over the device-key client."""
@@ -878,6 +892,9 @@ class PublicQueryClient(CloudClient):
             act="query a public table",
             body=body,
             absent="no public table has that id",
+            maximum=MAX_PUBLIC_QUERY_RESPONSE_BYTES,
+            table_values=True,
+            max_members=MAX_PUBLIC_QUERY_RESPONSE_MEMBERS,
         )
 
     def download_url(self, table_id: str) -> dict[str, Any]:
@@ -1091,6 +1108,8 @@ __all__ = [
     "FETCH_SCHEMA",
     "LOOKUP_SCHEMA",
     "MAX_LIMIT",
+    "MAX_PUBLIC_QUERY_RESPONSE_BYTES",
+    "MAX_PUBLIC_QUERY_RESPONSE_MEMBERS",
     "MAX_SAMPLE_LIMIT",
     "QUERY_SCHEMA",
     "SAMPLE_SCHEMA",

{mostlyright_data-0.29.3 → mostlyright_data-0.29.5}/src/mostlyright/data_harness/ux/credential_store.py

@@ -910,6 +910,11 @@ def _validate_private_file(
 def _windows_acl_is_private(path: Path, *, require_confidential: bool = True) -> bool:
     """Prove only the current user, SYSTEM, or Administrators can mutate a Windows path.
 
+    The path must be owned by the current user, which is what lets an OWNER RIGHTS entry count as
+    the current user: it names whoever owns the path.  CPython 3.12.4 and later give a folder made
+    with ``mkdir(mode=0o700)`` three entries, for SYSTEM, Administrators and OWNER RIGHTS, and none
+    for the user by name.
+
     With ``require_confidential`` they must also be the only accounts that can read it.  Only the
     explicit secure-file credential needs that, when it is read or staged; replacing or removing it
     exposes nothing.  The folder, the lock file, the backend manifest and the DPAPI blob hold no
@@ -957,7 +962,8 @@ def _windows_acl_is_private(path: Path, *, require_confidential: bool = True) ->
                 return False
             allowed = [current_sid]
             allocated: list[ctypes.c_void_p] = []
-            for text in ("S-1-5-18", "S-1-5-32-544"):  # SYSTEM, Administrators
+            # SYSTEM, Administrators, and OWNER RIGHTS, which stands for the owner proved above.
+            for text in ("S-1-5-18", "S-1-5-32-544", "S-1-3-4"):
                 sid = ctypes.c_void_p()
                 if not advapi32.ConvertStringSidToSidW(text, ctypes.byref(sid)) or not sid.value:
                     return False