mostlyright-data 0.29.9__tar.gz → 0.29.11__tar.gz

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.29.9
+Version: 0.29.11
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.29.9"
+version = "0.29.11"
 description = "Mostly Right hosted CLI for reviewed datasets"
 readme = "README.md"
 requires-python = ">=3.11"

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/SKILL.md

@@ -9,6 +9,67 @@ Build the dataset the user requested, verify its contents, and report its actual
 Use hosted commands with `--json` and returned identifiers. This core workflow governs the
 references. Load the reference for the current stage only, not the entire library before starting.
 
+## Four gates
+
+Every build passes these four gates in order. Do not skip one. The numbered steps below carry
+the detail.
+
+**Gate 1. Read what Studio serves and what the recipe misses.** Draft the recipe. Run
+`mr-data recipe explain RECIPE.json --json` before you register. Read `studio_capabilities`: a
+capability that is not literal `true` is unavailable on this Studio. Read `fast_features`. On a
+new table, fix every entry marked `missed`, or say in the build message why it stays.
+On a table that is already live, list the missed entries and ask the user first. Each fix changes
+the recipe digest, so the next build is a full resync. A paid source is called again for every
+per-row request whose saved answer no longer applies.
+
+**Gate 2. State a pace and leave concurrency out.** A page collection that states no pace sends
+one request a second. State `pages.min_interval_milliseconds` as the gap the publisher documents.
+No documented limit: state 1000 ms and say so. That is a valid reason. Registration refuses a gap
+under 16 ms. That is a platform bound, never a target. Where the deployment spreads requests over
+its egress addresses, the gap is kept per address, so the publisher can see a multiple of the
+stated rate. No command prints how many addresses there are. So when the publisher's limit is
+per key or per account, state 1000 ms unless the user confirms the deployment does not spread
+requests. Leave `pages.concurrency` out: the worker then sizes the width, and a stated value is
+only a ceiling. Keep a stated one only when a run's refusal told you to lower it or the publisher
+documents a limit on concurrent connections, and record the reason in a dataset note so the next
+session finds it. Then choose the source shape that fits.
+
+| Source shape | Use | Always state |
+| --- | --- | --- |
+| One public file that never changes | `public.https` with a Reader pin | `closed: true` |
+| One large public Parquet file whose publisher answers byte ranges | a `parquet.tabular@1.1.0` pin | `columns` |
+| Keyed JSON API paged by page number, whose first page states the page count | `authenticated.https.api_key@2.1.0` with the counted `pagination.parallel` | `total_pages_pointer`, `max_page_bytes`, the gap |
+| API that takes a date range | `window.request` | `lookback_seconds` |
+| Many pages behind one index | `public.https.collection@2.0.0` | the gap |
+| One request per row of another source | a per-row `request`, with `batch` when a `GET` takes a list of ids | the gap |
+| Bulk dump beside a windowed source | `reference` | `recheck_seconds` |
+| Live `wss://` feed | a recorded stream | see [Streams](references/recording-a-stream-venue.md) |
+
+[Sources](references/sources.md#pace-and-width) has the pace rules and the limits of parallel
+pages. [Readers](references/readers.md) has the Reader pins and the limits of a ranged read.
+
+**Gate 3. Run a bounded test on a scratch table.** Never test on the real table: its first
+succeeded run goes live, truncated or not. For a public CSV that needs no key, run
+`mr-data source inspect`. For anything else follow [the probe rule](references/3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table): copy the drafted
+recipe under another `table.name`, drop `table.search` from the copy, write small budgets into
+it, and take one bounded run.
+A sample's `--max-rows` cuts rows after the fetch. It does not shorten a collection walk, so the
+scratch recipe states a small `pages.max_fetches_per_run`. A per-row request collection also
+states `pages.stop_at_budget: true`, or it fails at that budget. A source whose provider bills
+per call spends real calls on the test, and the real table cannot reuse them. Forecast it and get
+the user's decision before the test. Retire the scratch table with `mr-data demote TABLE_ID` and
+then `mr-data table archive TABLE_ID --confirm DELETE`.
+
+**Gate 4. Forecast, then one full run.** Run
+`mr-data recipe forecast RECIPE_ID --digest DIGEST --json`. Exit 3 means resize before you run.
+Read the registration `warnings`. Then run `--full` once and follow it with `mr-data watch`.
+
+The steps below are numbered 1 to 10. The reference files are numbered by stage, 1 to 9. Steps 1
+and 2 are stage 1. Step 3 is stage 2. Step 4 is stage 3. Step 5 is stages 4 and 5. Steps 6 and 7
+are stage 6. Step 8 is stage 7. Step 9 is stage 8. Step 10 is stage 9. Gates 1 and 2 belong to
+step 5, where the recipe is drafted. Gate 3 needs that draft, so it runs after step 5 and before
+step 6. The stage 3 reference holds its rule. Gate 4 is step 6.
+
 ## Start and agree the brief
 
 1. Run `mr-data whoami --json` once. If authentication needs repair, read
@@ -60,12 +121,20 @@ references. Load the reference for the current stage only, not the entire librar
    Decide how each source continues before you register it, and whether the table needs a
    declared `rebuild`: read [making a table incremental](references/making-a-table-incremental.md).
    Run `mr-data recipe explain RECIPE.json --json` first. It names every reason the table could
-   not refresh, not only the first. Its `fast_features` names each fast feature the recipe
-   misses, with a one-line fix. Write `table.partition` for any table no window partitions that
+   not refresh, not only the first. Its `fast_features` has eight entries, and each missed one
+   carries a one-line fix. Write `table.partition` for any table no window partitions that
    has a date-like column, `batch` for a per-row GET whose API takes a list of ids, and
    `reference` for a bulk dump beside a window
    ([fast builds](references/making-a-table-incremental.md#fast-builds-tablepartition-batch-and-reference)).
-   Put them in the first revision, because each later change costs one resync. Adopting
+   State the gap, drop a stated `pages.concurrency`, page a keyed API in parallel and pin the
+   ranged Parquet Reader where the entries `pace`, `concurrency`, `parallel_pagination` and
+   `ranged_parquet` say so and the source meets each condition
+   ([fetching fast](references/making-a-table-incremental.md#fetching-fast-pace-width-parallel-pages-and-ranged-parquet)).
+   Put them in the first revision. Registering a revision is free, but building one is a full
+   resync, so each later change costs one resync.
+   On a table that is already live, list the missed entries and ask the user first. Each fix
+   changes the recipe digest, so the next build is a full resync. A paid source is called again
+   for every per-row request whose saved answer no longer applies. Adopting
    `table.partition` on an existing table costs one full resync, so for a paid source ask the
    user before that resync, as the brief and forecast rules require. Before any
    `--full` of a paid collection, run `mr-data recipe forecast RECIPE_ID --digest DIGEST --json`
@@ -75,7 +144,8 @@ references. Load the reference for the current stage only, not the entire librar
    Use `--full` for the requested build: it is progressive by default, with a five-minute
    inspection checkpoint while the same acquisition continues. Choose bounds from measured
    source size; row and byte clamps apply per source, not elapsed time. Use explicit `--sample`
-   only for a deliberate standalone inspection. Preserve the agreed semantics and quality checks.
+   only for a bounded test on a scratch table, as gate 3 says. Preserve the agreed semantics and
+   quality checks.
 7. Follow the returned run with `mr-data watch RUN_ID --json`. Its `source_rates` and
    `mr-data status RUN_ID --json` on a live run give each collection source's rate and time left.
    Quote those numbers rather than doing the arithmetic. Read `run.status`: returning an existing

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/3-probe-read-a-source-before-committing-to-it.md

@@ -79,11 +79,12 @@ refusal to the provider and never bypass it.
 reason codes, only once the discovery lanes and the comparisons are complete — and claim
 `unsupported` only after every typed-refusal chain ends `lawful_routes_exhausted`.
 
-**Reading the bytes is a bounded run, not `mr-data probe`.** `probe SOURCE_ID QUESTION_ID
+**Reading the bytes is a bounded test run, not `mr-data probe`.** `probe SOURCE_ID QUESTION_ID
 --dataset DATASET_ID` takes two identifiers rather than a question in words, and no v4 registration
 receipt returns a question identifier; it also rides the frozen `/v3/sessions` routes, which a
-deployment may have switched off. So the reading path on a hosted workspace is the ordinary loop,
-run early: register the recipe and take one run without `--window`.
+deployment may have switched off. So the reading path on a hosted workspace is
+[the probe rule](#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) below: one
+bounded run of a scratch table.
 
 **A public CSV can also be looked at before anything is registered.** `mr-data source inspect` reads
 the start of one HTTPS file that needs no key and answers with what it saw, with no recipe and no
@@ -107,7 +108,7 @@ start of the file and not the file. Its `evidence` carries the `format`, the `co
 not an answer yet, so read it back again later. A status of `truncated` is an answer that stopped at
 a limit: keep its limits and its evidence with whatever you record, and never call it complete. The
 command takes no key and no Reader options, so a source that needs either, or that is not a CSV, is
-read by the run below instead. So is any source that answers `capability_unavailable`, which means
+read by the test run below instead. So is any source that answers `capability_unavailable`, which means
 this deployment cannot inspect it: do not retry it, and do not guess in its place.
 
 **Write down what each look found.** An inspection is not a run, and nothing on the page shows what
@@ -128,33 +129,79 @@ observed_at, stage_m and quality_flag.
 EOF
 ```
 
-**Both obey the ordinary gate.** Registering needs the brief answered or delegated, so the run
-happens AFTER stage 2's answers arrive — or at once under a delegation — and an inspection, which
-writes a question and a source onto the dataset, waits for the same answers. What continues while
+**Both obey the ordinary gate.** Registering needs the brief answered or delegated. So the test
+run happens after stage 2's answers arrive, or at once under a delegation. An inspection writes a
+question and a source onto the dataset, so it waits for the same answers. What continues while
 you wait is research that submits nothing: documentation, publisher comparison, catalog.
 
+#### The probe rule: test on a scratch table, never on the real table
+
+Test a recipe on a scratch table with small budgets. Never test on the real table. A table's
+first succeeded run goes live, truncated or not. A test run on the real table would serve a cut
+table on the dataset page. This stage starts no full build of the real table. Stage 6 starts
+that one build. The test needs a drafted recipe, so run it once stage 5 has a draft and before
+stage 6.
+
+1. Copy the recipe document. Give the copy its own `table.name`. Keep the same `dataset.id`.
+   Drop `table.search` from the copy, so the test prepares no search index.
+2. Write small budgets into the copy, as the list below says.
+3. Register the copy with `mr-data recipe SCRATCH.json --json`. It returns a recipe id, a digest
+   and a table id of its own.
+4. Take one bounded run and read it back.
+5. Retire the scratch table. A scratch table goes live too, so withdraw it first.
+
 ```sh
-mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --json
-mr-data status RUN --inspection --json  # inspect while the same full acquisition continues
+mr-data run --recipe SCRATCH_RECIPE_ID --digest SCRATCH_DIGEST --sample --max-rows 2000 --json
 mr-data peek RUN --json      # the columns and types the run sealed
 mr-data query RUN "select count(*) from run_table" --json
 mr-data receipt RUN --json   # what was fetched and what those bytes hashed to
+mr-data status RUN --diagnose --json  # progress, plan counts and how each attempt ended
+mr-data demote SCRATCH_TABLE_ID --json
+mr-data table archive SCRATCH_TABLE_ID --confirm DELETE --json
 ```
 
-**Inspect the full build while it runs.** Its progressive checkpoint appears after about five
-minutes without stopping acquisition. Use an explicit `--sample` only for a deliberate standalone
-experiment: a row ceiling can truncate output after every collection page was fetched and does
-not promise a short acquisition. Never derive source size from a truncated sample's ceiling.
-
-**A reading run that came back whole IS the build, so stage 6 adopts it rather than repeating
-it.** Untruncated everywhere is the build, whichever stage started the run: keep this `run_id`.
-The run meets Studio's ordinary size confirmation before acquisition when it is large (a size
-check, not a charge); its checkpoint requires no
-second approval or second full build.
-
-That is stages 5 through 7 run once to settle the shape. Where the deployment does answer a source
-inspection or a probe, keep its status, reason code and exit value internal and describe only what
-it settled.
+**Know what a sample bounds and what it does not.**
+
+- `--max-rows` cuts each source's rows after that source is fetched and decoded. It does not
+  shorten a collection walk. Every page the budgets admit is still fetched.
+- A collection's walk is bounded by `pages.max_fetches_per_run` and `discovery.max_requests`.
+  Write both small in the scratch recipe. A listing collection that spends its fetch budget
+  still succeeds and reports `complete: false`.
+- A per-row request collection fails with `ROW_REQUESTS_EXCEED_BUDGET` at its fetch budget.
+  State `pages.stop_at_budget: true` beside the small budget in the scratch recipe. The run
+  then seals the rows it answered and reports `complete: false`.
+- A row clamp does not truncate a per-row request collection. Answers past the clamp fail the
+  run with `ROW_REQUESTS_EXCEED_BUDGET`. Keep `--max-rows` above the rows its budget can answer.
+- A document projection source cannot run under a row clamp. The run fails with
+  `CLAMP_UNSUPPORTED_FORMAT`. Test it with a full run of the scratch table:
+  `mr-data run --recipe SCRATCH_RECIPE_ID --digest SCRATCH_DIGEST --full --json`.
+- A source with a request window is read for its latest `lookback_seconds` only.
+  [Stage 6](6-build-one-run-sized-to-acquire-every-measured-source-whole.md) has the exact days.
+- `--max-source-bytes` and `limits.max_source_bytes` refuse in every mode. They never truncate.
+- A sample must state a clamp. Studio refuses a sample with none as `CLAMPS_REQUIRED`. This
+  client refuses it first as `THIN_REQUEST_INVALID`. Give `--max-rows`, `--max-source-bytes` or
+  `--window`.
+
+**A paid source spends real calls on the test.** A per-row request whose provider bills per call
+makes a paid call for every request the scratch run sends. The real table cannot reuse those answers,
+because saved answers belong to the table that holds them. Run `mr-data recipe forecast` on the
+scratch recipe and get the user's decision before the test.
+
+**Retire a scratch table by what happened to it.** A scratch table whose run succeeded is live:
+`demote` it, then `table archive` it. One whose run failed never went live, and `demote` answers
+`TABLE_NOT_PROMOTED` for it. Go straight to `table archive`. To test a revision, register it
+under the same scratch `table.name` before you archive. After the archive the name is free
+again, and registering it makes a new table.
+
+**Keep the activity line truthful during the test.** The page shows the scratch table going live
+as a table event. That event does not write your activity line. Say in the line that you are
+testing on a scratch table, and do not report `--phase done` for it.
+
+Never derive a source's size from a truncated sample's ceiling. A check on the whole table, such
+as a row floor, can fail on a sample of a sound recipe. Read the registration `warnings` first.
+
+Where the deployment does answer a source inspection or a probe, keep its status, reason code and
+exit value internal and describe only what it settled.
 
 **An acquisition's answer is evidence, and cheap to reuse.** Acquisitions are cacheable by
 source, query digest and window, so a fetch feeds a later build in the same warm session without

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/6-build-one-run-sized-to-acquire-every-measured-source-whole.md

@@ -1,13 +1,17 @@
 ### 6. Build — one run, sized to acquire every measured source whole
 
-**Stage 3's reading run is usually already this run.** Where it came back with
-`coverage.truncated` false everywhere, that run is the build: keep it, present it at stage 7, start
-nothing here. Re-running the coordinate replays it; a different ceiling buys only another
-acquisition.
-
-**Otherwise start one progressive full build.** Use the registered recipe and measured bounds
+**Stage 3 tested the recipe on a scratch table. This stage starts the one full build of the real
+table.** Do not test on the real table: its first succeeded run goes live, truncated or not.
+[The probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table)
+has the scratch-table steps. Before the full build, run
+`mr-data recipe forecast RECIPE_ID --digest RECIPE_DIGEST --json`. Exit 3 means the run as declared
+would outlast one attempt or overrun a quota, so resize it first. Read the registration
+`warnings` too.
+
+**Then start one progressive full build.** Use the registered recipe and measured bounds
 for the requested coverage. Every full run is progressive by default; agents do not choose
-between progressive and sample-first acquisition.
+between progressive and sample-first acquisition. Re-running the same coordinate replays its
+stored result. A different ceiling buys only another acquisition.
 
 ```sh
 mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --json
@@ -19,11 +23,12 @@ limit. A short build can finish before a checkpoint is needed. Do not submit ano
 inspecting it. A required size confirmation happens before acquisition; it is not a charge. The
 older `--progressive` flag is accepted for compatibility but is unnecessary.
 
-**A standalone sample is an explicit inspection choice.** Use `--sample --max-rows 2000` only
-when a separate bounded experiment is needed. Rows are clamped after acquisition, so a collection
-may still fetch every page and take longer than five minutes. For an already authorized full
-build, use the progressive checkpoint. A truncated sample is a preview: report its actual
-coverage and the full build's proposed coverage before requesting any missing authorization.
+**A standalone sample belongs on a scratch table.** Use `--sample --max-rows 2000` only for a
+separate bounded experiment, and run it as the probe rule says. Rows are clamped after
+acquisition, so a collection still fetches every page its budgets admit and can take longer than
+five minutes. For an already authorized full build, use the progressive checkpoint. A truncated
+sample is a preview: report its actual coverage and the full build's proposed coverage before
+requesting any missing authorization.
 
 > The preview stopped at 2,000 rows on `asos_observations`, which is the ceiling rather than the
 > end of the feed; the other three sources came back whole. The full build would cover 2000 to 2026

mostlyright_data-0.29.11/skills/mr-data-build/references/8-fix-revise-the-document-and-register-it-again.md

@@ -0,0 +1,16 @@
+### 8. Fix — revise the document and register it again
+
+Change the document and send it again. Same dataset, same table, new digest:
+
+```sh
+mr-data recipe RECIPE.json --json
+```
+
+**Registering a revision is free and abandons nothing. Building it costs one resync.**
+Registration starts no run. Table identity is stable across revisions: no `_v2.._vN` renaming, no
+proposal wedged on an id somebody has to reclaim, no cleanup after a failed run. The revision has
+a new digest, so the sealed predecessor no longer matches it. Its next build reads every source
+again. A per-row request collection still reuses the answers the live version holds while its
+request, Reader and binding stay the same. Put every change in one revision so the table pays
+once. Forecast a source whose provider bills per call, and ask the user before that resync.
+Build again, interrogate again, until the checks and coverage say what you need.

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/9-present-only-what-survived-inspection-with-caveats.md

@@ -12,9 +12,9 @@ you did not read off `coverage`.
 every coverage entry, nothing was cut and the table this run sealed is the table a full run would
 have sealed. When it is true anywhere, what you have is a shape rather than an answer.
 
-**Untruncated everywhere is the build. Present it as one.** The mode a run started in is not what
-makes a table finished, and calling a complete table "a preview" tells somebody their dataset does
-not exist yet when it does. Say what was built, and link the page:
+**The real table's full run, untruncated everywhere, is the build. Present it as one.** A
+scratch-table test is never the build, whatever its coverage says. Calling a complete table "a
+preview" tells somebody their dataset does not exist yet when it does. Say what was built, and link the page:
 
 > Built 43,812 rows from four sources, every source acquired whole and nothing truncated. All four
 > declared checks passed. Live at <link>. It refreshes every 6 hours, because the publisher posts
@@ -39,8 +39,9 @@ refresh delegation described above. Stage 6 carries the run commands.
 the dataset page, so before acting on this recipe again read `mr-data runs --json --mode full` and,
 for any run newer than the preview that you did not start, `mr-data status RUN_ID --json` to see
 whether it names this recipe and digest. One that does is theirs, and its state says what to do.
-`queued`, `running` or `succeeded` is the full build: adopt it as if you had started it — write its
-run cell, `mr-data watch RUN_ID --json` if it has not ended, then stage 7. `awaiting_confirmation`
+`queued`, `running` or `succeeded` is the full build of the real table: adopt it as if you had
+started it. Write its run cell, `mr-data watch RUN_ID --json` if it has not ended, then stage 7. A
+run of a scratch table is never adopted as the build. `awaiting_confirmation`
 is the size gate (not a charge), which the same command settles with `--confirm`.
 `awaiting_sample_approval` is stage 6's held full: read its `sample_run_id`, interrogate that
 preview, and release it with `mr-data run --approve-full`. `failed` is the full build too: read its

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/commands.md

@@ -9,16 +9,16 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data login` | Complete device approval and store a device credential. `login --force` is the compatibility alias for safe `auth rotate`, never an in-place truncate. |
 | `mr-data whoami` | Use the compatibility alias for the remotely validated `auth status` answer. |
 | `mr-data clarify` | Check whether clarification is allowed. Use `--attended` for an active user chat and `--unattended` for a scheduled invocation; an explicit operator veto such as `MOSTLYRIGHT_ATTENDED=0` still wins. A non-TTY subprocess does not prove the user is absent. The command does not ask the person in chat or wait for a reply; the agent must do that. It exits non-zero when nobody can answer or the recipe is already registered. |
-| `mr-data probe` | `probe SOURCE_ID QUESTION_ID --dataset DATASET_ID [--kind source_inspect\|sample_rows\|profile_columns\|evaluate_expression]` asks one already-registered source one question and prints the answer. **It rides the frozen `/v3/sessions` routes, which a deployment may have switched off, so do not plan a source inspection around it.** Both positionals are identifiers — `QUESTION_ID` is a question identifier, which no v4 registration receipt returns. The v4 way to look at a source before registering it is `mr-data source inspect`, in the next row; the other way to read one is to register the recipe and take one run without `--window` under `--max-rows`, then `peek`, `query` and `receipt`. |
-| `mr-data source` | `source inspect` takes a bounded look at one source, and it is the v4 path where `probe` is not; [stage 3](3-probe-read-a-source-before-committing-to-it.md) says when to use it. It starts one of three ways, and each prints the `session_id`, `probe_id` and `source_id`, plus the `dataset_id` and `question_id` when it has them. For a new public HTTPS CSV that needs no key, give `--dataset-id`, `--question`, `--source-name`, `--source-url`, `--claimed-basis` and `--claim-evidence-digest`: it prepares the source and opens a session. For a source already prepared, give `--dataset-id`, `--question-id` and `--source-id`, which opens a session. In a session you already hold, give `--session-id` with `--source-id`. `--context-source-id` adds a source to a session it opens. `--session-id` with `--probe-id` reads the answer back. A look is bounded to 65,536 bytes, 1,000 rows and 300 seconds; `queued` and `truncated` are not complete answers, and `capability_unavailable` means this deployment cannot inspect, so read the source by a bounded run instead. What it finds is exploratory evidence, not a run and not a receipt. |
-| `mr-data reader` | `reader normalize --family-id ID --family-version VERSION --options-file FILE [--workspace-id ID]` asks Studio's Reader service to apply one certified family's own defaults and option rules to a JSON object of options, at most 64 KiB, and prints the normalized `decode_options` and `decode_options_json`. The second is the text a recipe carries as `reader.decode_options`, pinned as [Readers](readers.md) describes. It changes nothing, and this client does not validate or invent a family's options: Studio decides. |
+| `mr-data probe` | `probe SOURCE_ID QUESTION_ID --dataset DATASET_ID [--kind source_inspect\|sample_rows\|profile_columns\|evaluate_expression]` asks one already-registered source one question and prints the answer. **It rides the frozen `/v3/sessions` routes, which a deployment may have switched off, so do not plan a source inspection around it.** Both positionals are identifiers. `QUESTION_ID` is a question identifier, which no v4 registration receipt returns. The v4 way to look at a source before registering it is `mr-data source inspect`, in the next row. The other way to read one is a bounded run of a scratch table, never of the real table, as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. |
+| `mr-data source` | `source inspect` takes a bounded look at one source, and it is the v4 path where `probe` is not. [Stage 3](3-probe-read-a-source-before-committing-to-it.md) says when to use it. It starts one of three ways, and each prints the `session_id`, `probe_id` and `source_id`, plus the `dataset_id` and `question_id` when it has them. For a new public HTTPS CSV that needs no key, give `--dataset-id`, `--question`, `--source-name`, `--source-url`, `--claimed-basis` and `--claim-evidence-digest`: it prepares the source and opens a session. For a source already prepared, give `--dataset-id`, `--question-id` and `--source-id`, which opens a session. In a session you already hold, give `--session-id` with `--source-id`. `--context-source-id` adds a source to a session it opens. `--session-id` with `--probe-id` reads the answer back. A look is bounded to 65,536 bytes, 1,000 rows and 300 seconds. `queued` and `truncated` are not complete answers, and `capability_unavailable` means this deployment cannot inspect, so read the source by a bounded run of a scratch table instead, as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. What it finds is exploratory evidence, not a run and not a receipt. |
+| `mr-data reader` | `reader normalize --family-id ID --family-version VERSION --options-file FILE [--workspace-id ID]` asks Studio's Reader service to apply one certified family's own defaults and option rules to a JSON object of options, at most 64 KiB, and prints the normalized `decode_options` and `decode_options_json`. The second is the text a recipe carries as `reader.decode_options`, pinned as [Readers](readers.md) describes. It changes nothing, and this client does not validate or invent a family's options: Studio decides. Run it for every Reader pin before you register, so a wrong option is refused here and not inside a run. |
 | `mr-data catalog` | `catalog search "QUESTION"` asks the sealed public-source catalogue which of its entries might answer a question and ranks them best first. Every ranked entry comes back with the disposition its own facts earned — `admitted`, `human_escalation_required` or `refused` — because an entry the catalogue could not vouch for is still a finding. `--format csv` states the one data format the question requires — one token per question, never repeated — and it is not a filter: an entry that does not declare that format still comes back ranked, with disposition `refused` and `filters_match` false, so nothing is held back; `--limit N` ranks at most N, up to 25. It fetches nothing and registers nothing. The catalogue holds one provider (Data.gov) and only part of it, so it is never exhaustive and an empty answer is not evidence that no such source exists. When the answer is `catalog_unavailable`, this deployment has no catalogue to search: record the lane and carry on. |
 | `mr-data public` | Read Mostly Right's public dataset catalogue. `public search "QUERY"` ranks published datasets (`--topic`, `--limit` and `--cursor` narrow and page it), `show SLUG` reads one overview, `tables SLUG` lists its table identifiers, `schema TABLE_ID` reads one table's schema, `sample TABLE_ID` prints its published preview rows, `lookup "QUERY"` is the connector search shape and `fetch SLUG` returns the dataset's Markdown document. Those seven need no login. `query TABLE_ID`, with `--body` or `--body-file`, and `download-url TABLE_ID` do need `mr-data login`: they ask Cloud, with the device key, to run one query over a public table and to say how it can be downloaded. Preview rows are not a query, and no table bytes pass through the command. |
 | `mr-data connected` | `connected list` shows the public datasets connected to this workspace and `connected add SLUG` connects one by its slug. It needs `mr-data login`: both ask Cloud, with the device key, over the routes MCP's `list_connected_datasets` and `connect_dataset` use. |
 | `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`, unless the user's own request already has an empty dataset open, in which case it answers `open_request_reused` with that `dataset_id` and mints nothing (`--separate` mints a new one for a different dataset); `dataset show ID` reads it back; `dataset set ID --name TEXT --topics "a,b,c" --license ID --description-file F` writes the title, the descriptive tags, the SPDX licence and the description under the version it was read at, retrying once if somebody else wrote first, and an empty `--topics` or `--license` takes that value off the page; send `--name` and `--topics` whenever you send `--description-file`, because a description sent on its own can make Studio write a new title and new tags, and read `changed` and `written_by_studio` on the receipt for every field the write moved; a saved write whose public sync fails exits 2 and reports `public_projection_synced: false` — use `dataset sync ID` to retry that sync without rewriting Studio; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream; `dataset activity ID --phase P --message TEXT` says what is happening right now, silently, and is never a chat message and never a cell; `dataset publish ID [--mode public|link|private]` says who can read the dataset — `public` lists it in the public directory and serves it at an address anybody can read, `link` serves it at an unlisted address, `private` takes it back to the workspace — and `dataset publish ID --show` reads that back without changing it; `dataset archive ID --confirm-name TITLE` retires the page and frees its title, deleting nothing. |
-| `mr-data recipe` | Register one recipe document — dataset, question, table plan, sources, transform, checks and units — in one call, and print the identifiers the server derived. The document is read first and every fault comes back in one refusal, each with the JSON pointer that names it; `--no-lint` sends it as written instead. Registration refuses `THIN_BRIEF_MISSING` when the dataset's record carries no answered question and no delegation, and `THIN_RECIPE_DATASET_UNBOUND` when the document names no `dataset.id`; `--delegated "their words"` writes the delegation onto the dataset and registers in the same command, and clears the first of those two and never the second. `mr-data recipe show ID` reads one back. `mr-data recipe explain FILE` (or a recipe identifier) lists every reason its table can or cannot refresh, each with a rule code and Studio's suggested repair, and exits 2 when one blocks it. `mr-data recipe forecast RECIPE_ID --digest DIGEST` says what one run would ask of each source before it runs, and exits 3 when the run would overrun today's platform quota or one attempt. `mr-data recipe readiness --table TABLE_ID --current-only` reads the revision one table runs now. [Making a table incremental](making-a-table-incremental.md#the-loop-explain-forecast-run-diagnose) says when to use each. |
+| `mr-data recipe` | Register one recipe document (dataset, question, table plan, sources, transform, checks and units) in one call, and print the identifiers the server derived. The document is read first and every fault comes back in one refusal, each with the JSON pointer that names it. `--no-lint` sends it as written instead. Registration refuses `THIN_BRIEF_MISSING` when the dataset's record carries no answered question and no delegation, and `THIN_RECIPE_DATASET_UNBOUND` when the document names no `dataset.id`. `--delegated "their words"` writes the delegation onto the dataset and registers in the same command, and clears the first of those two and never the second. `mr-data recipe show ID` reads one back. `mr-data recipe explain FILE` (or a recipe identifier) lists every reason its table can or cannot refresh, each with a rule code and Studio's suggested repair, and exits 2 when one blocks it. `mr-data recipe forecast RECIPE_ID --digest DIGEST` says what one run would ask of each source before it runs, and exits 3 when the run would overrun today's platform quota or one attempt. `mr-data recipe readiness --table TABLE_ID --current-only` reads the revision one table runs now. The explanation's `fast_features` names each fast feature the recipe uses or misses, with a one-line fix. [Making a table incremental](making-a-table-incremental.md#the-loop-explain-forecast-run-diagnose) says when to use each. |
 | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
-| `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID --digest RECIPE_DIGEST` — both required, neither positional, and the digest is the bare hex the registration receipt printed. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh executes only Studio's persisted source-action plan, and a plan shape it does not refresh returns `RESYNC_REQUIRED` before acquisition; [which plans refresh](making-a-table-incremental.md#which-plans-refresh) lists them, and `mr-data recipe explain` says which one a table is. Use `mr-data table resync TABLE_ID --request-id UUID` for an explicit full source reread, retaining that UUID if the response is uncertain. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal — report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. A large run is held for a size confirmation (not a charge), which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried. `--approve-full RUN_ID` releases the full an existing legacy sample-first pair is holding once its preview has succeeded, naming either half of the pair; where the deployment still wants a person at a browser it answers `THIN_INTERACTIVE_HUMAN_REQUIRED` and names the run page. `--retry RUN_ID` tries one failed run again when its triple carries `room_fault: true`, re-stating that run's own coordinate and ceilings and naming the failed run so Studio starts a new attempt, and refuses with the triple when the recipe was at fault. `--cancel RUN` stops one. |
+| `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID --digest RECIPE_DIGEST`. Both are required, neither is positional, and the digest is the bare hex the registration receipt printed. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh executes only Studio's persisted source-action plan, and a plan shape it does not refresh returns `RESYNC_REQUIRED` before acquisition. [Which plans refresh](making-a-table-incremental.md#which-plans-refresh) lists them, and `mr-data recipe explain` says which one a table is. Use `mr-data table resync TABLE_ID --request-id UUID` for an explicit full source reread, retaining that UUID if the response is uncertain. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal. Report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. `--max-rows` cuts rows after a source is fetched and decoded, so it does not shorten a collection walk. A sample with no clamp is refused: `CLAMPS_REQUIRED` from Studio, and `THIN_REQUEST_INVALID` from this client first. A table's first succeeded run goes live, truncated or not, so test on a scratch table as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. A large run is held for a size confirmation (not a charge), which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried. `--approve-full RUN_ID` releases the full an existing legacy sample-first pair is holding once its preview has succeeded, naming either half of the pair. Where the deployment still wants a person at a browser it answers `THIN_INTERACTIVE_HUMAN_REQUIRED` and names the run page. `--retry RUN_ID` tries one failed run again when its triple carries `room_fault: true`, re-stating that run's own coordinate and ceilings and naming the failed run so Studio starts a new attempt, and refuses with the triple when the recipe was at fault. `--cancel RUN` stops one. |
 | `mr-data status` | Report which of the seven states one run is in, what it delivered and whether a ceiling cut it short, and — on a failure — what failed, where, and whether the fault was the execution room's rather than the recipe's. Exits non-zero on a failed run. `--diagnose` adds everything else Studio can say about the run: its progress, a per-row source's plan counts, its checkpoints, and how each attempt ended, with a lost attempt's cause marked as an inference. |
 | `mr-data runs` | Report this workspace's own runs: identifier, mode, state and creation time, plus the failure triple of any that failed and whether that failure was the execution room's. `--dataset DATASET_ID` narrows it to one dataset's runs, so "what have I run for this dataset?" is one scoped read, and a value that is not an identifier is refused `THIN_ARGUMENT_INVALID`. `--status` and `--mode` narrow it too and are refused `THIN_ARGUMENT_INVALID` for a word outside the seven states or the six modes; the three combine. Studio narrows before it cuts a page, so a filtered page holds the newest runs that match however many runs the workspace has. `--limit` says how many, and pages are followed to reach it; a listing that does not end says how many were read and suggests a smaller `--limit` or a narrower filter. The answer prints `next_cursor`, and `--cursor` continues from it. |
 | `mr-data watch` | Stream one run's live progress under the durable event type of each stage, resuming across stream cuts. Exits non-zero when the run failed. |

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/live-run.md

@@ -73,9 +73,11 @@ product decision is needed and give the person the single action that lets the b
 covers nothing and says so. A `succeeded` one reports what `coverage.truncated` recorded. Anything
 else states the bound it ran under — a `sample` its ceilings, a `full` the whole dataset, and a
 `full` at `awaiting_sample_approval` the whole dataset plus the preview it waits on. The mode
-alone never establishes a truncation, which is the rule reference 6 states: a sample sized above
-every measured source cut nothing and is the build. A source with a request window is the
-exception: a sample reads its latest lookback span only, and the sentence says so. It is absent
+alone never establishes a truncation. A sample sized above every measured source cut nothing.
+It is still a test, and it belongs on a scratch table, as
+[the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table)
+says. A source with a request window is the exception to the size rule. A sample reads its latest
+lookback span only, and the sentence says so. It is absent
 on `refresh`, `backfill`, `compact` and `replay`, and it is never evidence that a run is
 executing; `status` is.
 

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/making-a-table-incremental.md

@@ -20,10 +20,16 @@ after every revision.
    `suggestions`. After the first full, run `mr-data recipe explain RECIPE_ID --json` again: the
    table route also judges the predecessor. The command exits 0 when the table is ready to refresh
    and 2 when a finding blocks it. An advisory finding (`blocks_refresh: false`) never changes the
-   exit code. The receipt's `fast_features` says which of `table.partition`, `batch` and
-   `reference` the recipe uses or misses, and which sources every run reads whole. Each missed
-   feature carries a one-line fix. `fast_features_missed` lists them for a script. See
-   [Fast builds](#fast-builds-tablepartition-batch-and-reference).
+   exit code. The receipt's `fast_features` has eight entries. Four say what a run can keep:
+   `table.partition`, `batch`, `reference`, and `resync_only_sources` for the sources every run
+   reads whole. Four say how fast a run fetches: `pace`, `concurrency`, `parallel_pagination` and
+   `ranged_parquet`. Each missed feature carries a one-line fix. `fast_features_missed` lists
+   them for a script. On a new table, fix each one, or say in the build message why it stays.
+   On a table that is already live, list the missed entries and ask the user first. Each fix
+   changes the recipe digest, so the next build is a full resync. A paid source is called again
+   for every per-row request whose saved answer no longer applies. See
+   [Fast builds](#fast-builds-tablepartition-batch-and-reference) and
+   [Fetching fast](#fetching-fast-pace-width-parallel-pages-and-ranged-parquet).
 2. `mr-data recipe forecast RECIPE_ID --digest DIGEST --json` before any `--full` that pays for
    calls. It sends nothing. Per source, it reports the request ceiling the document allows, the
    count the last run of the same source definition actually sent, the pace, and the runtime at
@@ -32,8 +38,9 @@ after every revision.
    collection rather than start the run. The forecast also reads observed limits. Each source
    with a history run gets `observed`: the rate that run actually reached and the runtime of the
    expected requests at that rate. A line flags a run that went under half its declared pace,
-   because some other limit held it. On a Studio without the forecast route, a platform source is
-   paced at its binding's stated rate and provider limits. `--no-observed` skips both reads.
+   because some other limit held it. On a Studio without the forecast route, the forecast bounds
+   a platform source by its binding's stated rate and provider limits. The declared gap still
+   applies under them. `--no-observed` skips both reads.
 3. `mr-data run --recipe RECIPE_ID --digest DIGEST --full --json`, followed as
    [the run lifecycle](live-run.md) describes. `mr-data watch RUN_ID` prints a `rate` line for
    each collection source about once a minute: members done, members a minute, and the time left
@@ -267,12 +274,14 @@ from the forecast.** `mr-data recipe forecast` prints a suggested
 plus a quarter. A cap near the real count makes a corpus that suddenly doubles fail with
 `ROW_REQUESTS_EXCEED_BUDGET` before any call is sent, rather than pay for the doubling.
 `limits.max_requests` must still be at least `pages.max_fetches_per_run` times one more than
-`pages.retries`. With no history the forecast says `EXPECTED_UNKNOWN`. Build a small `--sample`
-first, read its plan counts with `mr-data status RUN_ID --diagnose --json`, and size from them.
+`pages.retries`. With no history the forecast says `EXPECTED_UNKNOWN`. Run a small bounded test
+on a scratch table first, as
+[the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table)
+says. Read its plan counts with `mr-data status RUN_ID --diagnose --json`, and size from them.
 
 ### Fast builds: `table.partition`, `batch` and `reference`
 
-`mr-data recipe explain` lists each of these under `fast_features`. Each is a recipe member, so
+`mr-data recipe explain` lists each of these three under `fast_features`. Each is a recipe member, so
 adding one changes the digest and the table's next full is a resync. Make every change in one
 revision so the table pays once. Adopting `table.partition` on an existing table costs one full
 resync. For a paid source, ask the user before that resync, as the brief and forecast rules
@@ -342,6 +351,28 @@ call:
   reads it whole. The same digest changes nothing. A new digest makes that refresh recompute the
   whole table, so every version joins exactly one edition of the dump.
 
+### Fetching fast: pace, width, parallel pages and ranged Parquet
+
+These four `fast_features` entries are about the rate a run fetches at. Each is a recipe member
+too, so change them in the first revision.
+
+| Entry | It reports | The fix |
+| --- | --- | --- |
+| `pace` | a page collection or a keyless request window that states no gap | state the gap the publisher documents. No documented limit: state 1000 ms and say so |
+| `concurrency` | any stated `pages.concurrency` | remove it, unless a run's refusal told you to lower it or the publisher documents a limit on concurrent connections. Record the reason in a dataset note |
+| `parallel_pagination` | a serial `GET` page walk over JSON on `authenticated.https.api_key@2.0.0`, on a source with no window | move to `@2.1.0` with the counted `pagination.parallel`, when the first page states its page count |
+| `ranged_parquet` | a `parquet.tabular@1.0.0` pin on a public file with no request window, in a recipe that states no `execution_policy` | pin `parquet.tabular@1.1.0`, when the publisher answers byte ranges and the limits in [Readers](readers.md) fit |
+
+A stated gap is never reported, whatever its size. A slow gap that is the publisher's limit is
+correct. The entries read the document alone. They cannot know a publisher's limit, so every fix
+is yours to check against the source.
+On a table that is already live, list the missed entries and ask the user first. Each fix changes
+the recipe digest, so the next build is a full resync. A paid source is called again for every
+per-row request whose saved answer no longer applies.
+[Pace and width](sources.md#pace-and-width),
+[parallel pages](sources.md#paging-a-keyed-api-in-parallel) and [Readers](readers.md) have the
+rules.
+
 ### Probing a request window without an executing probe
 
 `mr-data recipe explain` suggests `declare_request_window` for a source that reads whole. Its
@@ -366,7 +397,10 @@ bounded runs over two disjoint windows:
 5. Retire the scratch tables: `mr-data demote TABLE_A TABLE_B`, then
    `mr-data table archive TABLE_A TABLE_B --confirm DELETE`.
 
-Both runs spend real requests, so keep the clamp small. Nothing can be probed yet about response
+These are scratch tables, as
+[the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table)
+requires: never run this test on the real table. Both runs spend real requests, so keep the clamp
+small. The row clamp cuts rows after the fetch. It does not shorten a collection walk. Nothing can be probed yet about response
 headers, `ETag` or `Last-Modified`, sort order, cursors, or how the publisher answers under 429 or
 5xx. No shipped command can vary a request or report those facts. Do not claim any of them from
 documentation alone.
@@ -595,6 +629,10 @@ For Binance BTCUSDT one-minute klines, start the backfill at `2017-08-17T00:00:0
 }
 ```
 
+The 1000 above is one request a second. State the gap the publisher documents. With no documented
+limit, state 1000 and say so. `mr-data recipe explain` reports a keyless window that states no
+gap under `fast_features` as `pace`.
+
 Keep `granularity: day`, `timezone: UTC` and `merge.materialization: partition_replace`.
 For the acquired `open_time_ms` column declare
 `merge.partition: {"column":"open_time_ms","key":"epoch_millis_value"}` and

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/one-install.md

@@ -9,7 +9,7 @@ resolves no third-party package at all, and the engine, the Readers and the work
 Mostly Right Studio. `--hosted` is accepted and does nothing, because the lane it names is the only
 one; a script that already carries it keeps working.
 
-`mr-data --help` lists every command this product has — thirty-four of them, and the page is the
-whole list. A name that is not on it is not a command: typing one answers `THIN_UNKNOWN_COMMAND`,
+`mr-data --help` lists every command this product has. There are thirty-eight, and the page is
+the whole list. A name that is not on it is not a command: typing one answers `THIN_UNKNOWN_COMMAND`,
 exactly as a misspelling does. Do not retry it under a different install, and do not report its
 absence as a defect.

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/preparing-table-search.md

@@ -15,8 +15,9 @@ mr-data search inspect --recipe recipe.json --json
 
 Local CLI help describes installed commands; it does not prove the deployed server serves them.
 For the recipe-target probe above, read JSON refusals from stderr: `schema_version` is
-`mr-data-error.v1`, `code` is the typed discriminator, and `error` is the detail. There is no
-public `http_status` field in that envelope. Treat `code: THIN_NOT_FOUND` as absent search
+`mr-data-error.v1`, `code` is the typed discriminator, and `error` is the detail. Cloud refusals
+also expose `http_status`, `request_id`, `detail` and `retry_after_seconds` when supplied;
+Studio inspection refusals retain their typed code. Treat `code: THIN_NOT_FOUND` as absent search
 registration support only when `error` begins exactly with
 `Studio has no such resource for /v4/search/recipe-inspection`; the CLI maps this route's HTTP 404
 that way. An explicit `code: THIN_SEARCH_CAPABILITY_UNAVAILABLE` also states a capability limit.
@@ -141,3 +142,28 @@ an exact filter.
 
 This workflow opts in newly authored recipes only. It does not migrate old datasets, reauthor users'
 existing recipes, launch a full data rebuild or imply successful indexing of every table.
+
+## Diagnose preparation and query a published table
+
+`preparation_state: unavailable` can mean a never-built table, a failed or cancelled job,
+or an incompatible specification. Keep `job_id` from the search build receipt and run
+`mr-data search status --job JOB_ID --json` to inspect the job's status and failure.
+Studio is adding `preparation_detail`; print it when present rather than inferring a reason
+from `unavailable`. Table status passes this additive field through unchanged.
+
+For a published table, first run `mr-data public schema TABLE_ID --json`.
+The table schema's `capabilities.search` contains `table_version_id`, `spec_digest`,
+`keyword_columns`, `semantic_columns`, and optional stored `lexical`/`semantic` preparation values.
+Use `keyword_columns` for `search.columns`; top-level `columns` is the non-empty output projection.
+The following lexical example uses a syntactically valid illustrative digest:
+
+```sh
+mr-data public query TABLE_ID --body '{"columns":["title"],"search":{"text":"example","columns":["title"],"spec_digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000","mode":"lexical"},"limit":10}' --json
+```
+
+Replace the illustrative columns and digest with the actual schema values. Hybrid mode additionally
+needs `generation_digest` (`sha256:` plus 64 lowercase hex digits) from readiness/explain;
+it is not in `capabilities.search`. Omit it for this lexical example. Public query prints
+Cloud stage durations in milliseconds under `result.timings` when Server-Timing is present,
+and reports HTTP status, request id and retry timing. Keep refusal `detail` with the request id.
+Missing timing metadata does not establish that a stage did no work.

{mostlyright_data-0.29.9 → mostlyright_data-0.29.11}/skills/mr-data-build/references/promote.md

@@ -1,7 +1,9 @@
 ## Promote
 
-A table's first succeeded run goes live on its own, whatever mode that run was — a run whose
-coverage is untruncated everywhere counts. There is no command that makes it readable and no click
+A table's first succeeded run goes live on its own, whatever mode that run was and whether or
+not a clamp truncated it. A sample counts. So test on a scratch table and never on the real one,
+as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table)
+says. There is no command that makes a table readable and no click
 to wait for, and no catch-up run is dispatched for it. Everything before it is workspace scratch
 too: probing, building, downloading and interrogating all run with no human ceremony at all.