mostlyright-data 0.10.7__tar.gz → 0.11.0__tar.gz

{mostlyright_data-0.10.7 → mostlyright_data-0.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.10.7
+Version: 0.11.0
 Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.10.7 → mostlyright_data-0.11.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.10.7"
+version = "0.11.0"
 description = "Mostly Right CLI for reproducible, reviewed Parquet datasets"
 readme = "README.md"
 requires-python = ">=3.13,<3.14"

{mostlyright_data-0.10.7 → mostlyright_data-0.11.0}/skills/mr-data-build/SKILL.md

@@ -78,14 +78,14 @@ the one command with no implementation behind it.
 There is no second profile and no lane to choose. `--hosted` is accepted and does nothing, because
 the lane it names is the only one; a script that already carries it keeps working.
 
-`mr-data --help` lists every command this product has — twenty-five of them, and the page is the
+`mr-data --help` lists every command this product has — twenty-six of them, and the page is the
 whole list. A name that is not on it is not a command: typing one answers `THIN_UNKNOWN_COMMAND`,
 exactly as a misspelling does. Do not retry it under a different install, and do not report its
 absence as a defect.
 
 ## Commands
 
-These twenty-five are the whole surface, alongside `acquire-slices`, which names a capability
+These twenty-six are the whole surface, alongside `acquire-slices`, which names a capability
 gap rather than doing anything, and `export-hosted-candidate`, which is a backend operator step.
 
 | Command | What it does |
@@ -95,6 +95,7 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data whoami` | Use the compatibility alias for the remotely validated `auth status` answer. |
 | `mr-data clarify` | Say whether one clarifying question may be put to the user right now, and compose the record of their answer. It reaches nothing, asks nobody and waits for nothing. Exits non-zero when no person can answer or when a recipe is already registered. |
 | `mr-data probe` | Ask one registered source one question and print the answer. **Currently unusable:** it reaches `/v3/sessions`, which Studio refuses, and it takes a question identifier that no v4 receipt returns. Do not plan a source inspection around it. |
+| `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`; `dataset show ID` reads it back; `dataset set ID --name TEXT --description-file F` writes the title and the description under the version it was read at, retrying once if somebody else wrote first; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream. |
 | `mr-data recipe` | Register one recipe document — dataset, question, table plan, sources, transform, checks and units — in one call, and print the identifiers the server derived. `mr-data recipe show ID` reads one back. |
 | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
 | `mr-data run` | Start one run against a registered recipe. The command accepts `--sample`, `--full`, `--refresh`, `--backfill` and `--window`, and the worker executes `--sample`, `--full` and `--refresh` (a refresh re-acquires the declared addresses over the previous version and claims no window); `--backfill` and a `--window` on a sample or full are refused before acquisition. `--max-rows` and `--max-source-bytes` remain supported. A large run is held for a spend confirmation, which `--confirm` settles; a sample-first held full requires Studio's interactive human approval instead. `--cancel RUN` stops one. |
@@ -110,17 +111,17 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data diff` | Compare two runs and say what changed. |
 | `mr-data keys` | Enrol a source credential by name, list the names with their created and rotated times, and delete one. `keys set` reads the value from a file or from standard input and never from the command line; no command prints a value back. |
 | `mr-data stream` | Record a public `wss://` venue so a build can read it. `stream document register --file F` admits the connector document, `stream registry create --digest D` admits a set of them, `stream probe` listens briefly and seals nothing, `stream capture --seconds N` records a bounded window, `stream subscribe` keeps recording, `stream subscription {show|pause|resume|stop} ID` moves one, `stream status ID` reads a capture or a probe, and `stream recordings --document-digest D` names what a build could read and until when. `--seconds` is at most 600. `mr-data run` takes no duration: a build reads sealed batches and is bounded by them. |
-| `mr-data open` | Mint a single-use, short-lived address that continues this session in a browser at one dashboard path, and print it. `--open` hands it to the platform opener. A handoff is minted only for a personal access token; a device credential is refused with `THIN_STUDIO_HANDOFF_REQUIRES_PERSONAL_ACCESS_TOKEN`, so give the user the plain dashboard address instead. |
+| `mr-data open` | Mint a single-use, short-lived address that continues this session in a browser at one dashboard path, and print it. `--open` hands it to the platform opener. The credential decides who mints: a device credential — what `mr-data login` stores — mints at Cloud, and a personal access token mints at Studio. |
 | `mr-data promote` | Ask Studio to publish one table after a person's decision. The Harness worker does not perform the catch-up or scheduled refresh; report those only when another deployed component returns durable evidence that it did. |
 | `mr-data pin` | Ask Studio to freeze one table pointer on an exact version, for a rollback or a hold. |
 | `mr-data unpin` | Ask Studio to resume pointer tracking; read the returned state before claiming it resumed. |
 | `mr-data demote` | Ask Studio to withdraw the pointer and any schedule it owns. |
 
 `auth`, `login`, `whoami` and `clarify` are the same implementation in both profiles, and
-`clarify` alone reaches nothing at all. The other twenty-one answer from Studio. Eight only
-read: `status`, `runs`, `watch`, `peek`, `receipt`, `checks`, `download` and `diff`. Thirteen
-start or change something: `recipe`, `cover`, `run`, `query`, `probe`, `keys`, `stream`, `open`,
-`note`, `promote`, `pin`, `unpin` and `demote`.
+`clarify` alone reaches nothing at all. The other twenty-two answer from Studio. Eight only
+read: `status`, `runs`, `watch`, `peek`, `receipt`, `checks`, `download` and `diff`. Fourteen
+start or change something: `dataset`, `recipe`, `cover`, `run`, `query`, `probe`, `keys`,
+`stream`, `open`, `note`, `promote`, `pin`, `unpin` and `demote`.
 
 **Any successful run is readable straight away.** There is no approval, no admission and no release
 step between a run finishing and reading it: `download`, `receipt`, `checks`, `peek` and `query`
@@ -205,12 +206,20 @@ live data key is refused with `THIN_AUTHENTICATION_FAILED` rather than failing t
 
 ## Required protocol
 
-There is one loop, and it is short: probe a source before committing to it, draft one recipe
-document, build a sample, interrogate what came back, fix the document, build again. Then present
-only what you inspected, with its caveats. Nothing between a run finishing and reading it is held
-for a person, so there is no approval to seek, no separate body of evidence to read back, and no
-state to wait for; the only human gate in the product is [Promote](#promote), which comes after the
-loop has produced a table worth making live.
+There is one loop, and it is short: open the page before you have anything to put on it, probe the
+sources, say out loud what you chose and what you refused, draft one recipe document, build a
+sample, interrogate what came back, fix the document, build again. Then present only what you
+inspected, with its caveats. Nothing between a run finishing and reading it is held for a person,
+so there is no approval to seek, no separate body of evidence to read back, and no state to wait
+for; the only human gate in the product is [Promote](#promote), which comes after the loop has
+produced a table worth making live.
+
+**The page is the work, and it exists first.** A dataset used to come into being as a side effect
+of registering a recipe, which meant nothing was visible until every decision that fills the page
+had already been taken: it appeared at the end, complete, as an assertion rather than as work.
+`mr-data dataset create` mints it from a working title before any research starts, so somebody
+watching sees it fill in. Everything after that is an edit to something already on screen, and
+every stage below writes to it as it goes rather than composing the page at the end.
 
 Authenticate first: run the Cloud authentication preflight above once, before the first cloud
 command of the session.
@@ -245,7 +254,25 @@ called `reader` — is invisible to a search for its own name. One run that isol
 settles it. Where a run is too expensive, at least check the failure details on existing runs for a
 message that contradicts you.
 
-### 1. Probe — read a source before committing to it
+### 1. Open — the page, before there is anything on it
+
+Create the dataset from a working title and put it in front of whoever is watching, **before any
+research**. It costs two commands and it is what turns a build into something a person can follow:
+
+```sh
+mr-data dataset create --name "Kathmandu monsoon flooding" --json
+mr-data open /datasets/DATASET_ID
+```
+
+Navigate the host's in-app browser to that address immediately, and keep that one tab for the whole
+build. The working title is allowed to be provisional — stage 4 writes the title it settles on —
+and a title that never changes is fine too. Where the address cannot be minted, carry on with the
+plain dashboard address rather than blocking.
+
+Hold the `dataset_id` for the rest of the session: stage 3 writes the decision record to it, stage
+4 writes its title, description and cover, and the recipe binds to it by naming it as `dataset.id`.
+
+### 2. Probe — read a source before committing to it
 
 **Judge the sources.** This stays with the external agent and is the substance of the task. Search
 every available discovery lane with no lane priority: a configured sealed catalog, host public-web
@@ -303,11 +330,56 @@ Clean room be the first thing that opens the bytes. Named messages out of one la
 
 Hold the comparison in the shape [Narrating the run](#narrating-the-run) takes: one `source` block
 per feed evaluated, rejections included with the clause that ruled each one out. Write them to a
-blocks file and **do not call `mr-data note` yet** — a cell goes into one run's own log, no run
-exists until stage 3, and a run's narrative closes the moment the run reaches a terminal state.
-Stage 3 is where the file is written and says when.
+blocks file; stage 3 is what puts them on the page.
+
+### 3. Decide — say what you chose, what you refused, and what unblocks it
+
+**Post one Sources and decisions message before you register anything.** It goes to two places, in
+the same words: the dataset's own decision record, and the chat. This is the decision point that
+replaced the old one-question rule, and the reason for the change is that a question is not what a
+person mostly needs — what they need is to see the search, while there is still time to disagree
+with it.
+
+```sh
+mr-data dataset note DATASET_ID --heading "Sources and decisions" \
+  --blocks-file BLOCKS.json --json <<'EOF'
+Two feeds carry hourly stage; one is licensed and cannot be used here.
+EOF
+```
+
+The cell carries a `source` block for every feed evaluated — the ones being used, the ones
+considered, and the ones rejected with the clause that ruled each out — and a `decision` block for
+every grain, key, join and cutoff settled. Post the same content in the chat as one short message.
+Then **keep going**: this is a statement, not a gate, and waiting for a reply to it is the pause
+this document does not allow.
+
+**Blocked sources: always say what unlocks it and how to enrol the credential.** A source you could
+not reach because it needs an API key, an account or an approval is not a dead end and must never
+be reported as one. Name the concrete unblock step in the block's `reason` and in the chat message:
+who issues the credential, where to apply for it, and the exact command that enrols it —
+`mr-data keys set NAME --value-file PATH`, with the name the recipe will reference. A rejected
+source with no route back is a decision the user cannot revisit; one with a route back is a
+decision they can.
+
+**Ask only at a genuine fork.** A fork is a question whose two answers produce two different tables
+and which no evidence can settle — what an ambiguous word in the request means, how wide the
+population is, how far back it goes. Write it as a `clarification` block carrying `question` and no
+`answer`, which is what the page shows as a pending question, and ask it in the chat in the same
+words. `mr-data clarify --question TEXT` says whether there is anybody to ask and whether it is
+still early enough. When the answer comes back, write it as the answer on that block:
+
+```sh
+mr-data dataset note DATASET_ID --heading "The stage grain is settled" \
+  --revise "one-fork-to-settle" --blocks-file ANSWERED.json --json <<'EOF'
+Daily maxima, on the user's answer.
+EOF
+```
 
-### 2. Draft — one recipe document, one call
+Everything under [Autonomous delivery](#autonomous-delivery) still holds: which source, Reader,
+engine, format, workspace, retry tactic or cleaning operation to use is settled by the evidence and
+is never a fork. If nothing is genuinely forked, ask nothing and carry on.
+
+### 4. Draft — one recipe document, one call
 
 Write ONE document — dataset identity, question, sources with their rights claims and per-source
 limits, table plan, transform plan, declared checks, declared units and timezone — and register it:
@@ -321,9 +393,27 @@ question, the sources, the connector configs and the table plan in one transacti
 `recipe_id`, `recipe_digest`, `dataset_id`, `table_id` and `source_ids`. Read
 [The recipe document](../../docs/RECIPE-DOCUMENT.md) for the shape of every member.
 
-Every grain, key, join and cutoff settled here is a `decision` block — see
-[Narrating the run](#narrating-the-run). Append them to the same blocks file the source blocks
-went into. There is still no run to write them into; stage 3 writes the file.
+**Name the dataset the recipe binds to.** The document's `dataset` member takes an optional `id`,
+and it is the identifier stage 1 printed. With it, the registration attaches to the page somebody
+has been watching; without it, Studio resolves a container from the name and mints a second page
+behind them. Always carry it.
+
+Every grain, key, join and cutoff settled after stage 3 is a further `decision` block on the
+DATASET's record — `mr-data dataset note DATASET_ID` — because the fact outlives every run.
+
+**Write the title and the description before the first run, one call each.** The page has been
+open since stage 1 under a working title; this is where it settles:
+
+```sh
+mr-data dataset set DATASET_ID --name "Kathmandu monsoon flooding, 2015-2026" --json
+mr-data dataset set DATASET_ID --description-file DESCRIPTION.md --json
+```
+
+Two calls rather than one, so the person watching sees the title land and then the text arrive.
+Each call reads the version first and writes under it, so a description you wrote against a page
+somebody else has since edited is retried once and then reported rather than silently overwriting
+them. Write the description COMPLETE the first time — it is documentation, not a placeholder — and
+do not send it in pieces.
 
 **The dataset description is the documentation, and it is read by people and by search engines.**
 `dataset.description` is Markdown, rendered on the dataset page with headings and lists, and it
@@ -361,8 +451,8 @@ show, so they must stand alone. A one-line description is a defect to fix in the
 style choice. Give `table.description` the grain and the window, and give every column a
 description a reader can act on.
 
-**Every registered dataset gets a cover before its first run is presented.** After
-`mr-data recipe` answers with the `dataset_id`, generate one with
+**Every dataset gets a cover before its first run is presented.** Once the description is
+written, generate one with
 `mr-data cover DATASET_ID "what the image should depict" --json`, describing the subject in plain
 language — the place, the instrument, the phenomenon — and nothing about style, colour or size,
 which Studio fixes. Do it once per dataset, silently, and do not repeat it for a revision unless the
@@ -428,35 +518,42 @@ from the accepted subset — `cm`, `ug/m3`, `km/h` and the rest of it are writte
 the construct that put it there. Say `none` where a column carries no physical unit. Remember that
 the two temperature scales are offsets rather than multipliers.
 
-### 3. Build — take an unwindowed sample first
+### 5. Build — take an unwindowed sample first
 
 Take an unwindowed sample before a full run. Choose row and source-byte bounds that exercise the
 shape without pretending they selected a time range. The current worker does not render a run
 window into any source request, so any stated `--window` is refused before acquisition.
 
 ```sh
-mr-data run RECIPE_ID --sample --max-rows 50000 --json
+mr-data run RECIPE_ID --sample --max-rows 2000 --json
 ```
 
-**Write the record as soon as the run exists, and before it finishes.** `mr-data run` prints the
-run identifier and returns; the run itself then reaches a terminal state on its own, often within
-seconds on a sample. A run's narrative closes with the run, so a cell appended after that answers
-`409 RUN_NARRATIVE_CLOSED` and is lost. This is the only window there is, and it is the reason
-the two stages above wrote to a file instead of calling anything. So the first thing to do with a
-new run identifier is write the blocks stages 1 and 2 settled:
+**Two thousand rows, and the first run is always this one.** A sample is for establishing shape,
+and a usable one has to arrive within minutes: two thousand rows exercises every column, every
+cast and every declared check, and it comes back while the person who asked is still watching the
+page. A larger first sample buys nothing the checks do not already report and costs the one thing
+the page cannot get back, which is the first few minutes. The full run comes later, in stage 8,
+and only when the user says so.
+
+**The dataset's record is the primary one; a run's own record is for what happened inside that
+run.** The sources, the decisions and the questions went onto the dataset in stage 3, where they
+outlive every run and are still there next week. What belongs on the RUN is what is true of that
+run alone — a clamp that truncated, a check that did not pass, the exact revision this attempt
+tested:
 
 ```sh
-mr-data note --run RUN_ID --heading "Using FAA ASPM hourly over the state feed" \
-  --blocks-file BLOCKS.json --phase acquire --json <<'EOF'
-Hourly airport throughput, joined to hourly weather.
+mr-data note --run RUN_ID --heading "Row ceiling truncated the sample" \
+  --phase acquire --json <<'EOF'
+The clamp cut acquisition at 2,000 rows; coverage.truncated is true on both sources.
 EOF
 ```
 
-Do this on every run you start, the throwaway sample included: the record belongs to the run whose
-decisions it explains, and by the time a full run starts from a settled recipe there is no search
-left to write down. Anything settled later — a clamp that truncated, a check that failed, the
-revision stage 5 makes — is a further cell on the run it is about, written while that run is
-still going. [Narrating the run](#narrating-the-run) is the reference for what a cell may carry.
+Write it as soon as the run exists, and before it finishes. `mr-data run` prints the run identifier
+and returns; the run itself then reaches a terminal state on its own, often within seconds on a
+sample. A run's narrative closes with the run, so a cell appended after that answers
+`409 RUN_NARRATIVE_CLOSED` and is lost. The dataset's record has no such window — it never closes —
+which is the other reason the search belongs there.
+[Narrating the run](#narrating-the-run) is the reference for what either cell may carry.
 
 **The protocol's three clamps do not behave alike, and the executable subset is narrower.** In the
 protocol, on `--sample`, `max_rows` and `window` truncate: the run succeeds at the clamp boundary
@@ -482,7 +579,7 @@ no coverage window -- it is how an enabled table's catch-up and its scheduled re
 `backfill` is defined by the exact window it covers, which this worker cannot narrow a source
 request to, and is refused before acquisition.
 
-### 4. Interrogate — ask the run what it actually delivered
+### 6. Interrogate — ask the run what it actually delivered
 
 Four reads, and none of them waits on anything:
 
@@ -525,7 +622,7 @@ required check that fails ends the run with `failure_code: CHECK_FAILED`.
 `mr-data diff RUN_A RUN_B` compares two runs — columns added, removed and retyped, row counts, the
 coverage window and the data fingerprint. `--columns-only` narrows what is compared to the schema.
 
-### 5. Fix — revise the document and register it again
+### 7. Fix — revise the document and register it again
 
 Change the document and send it again. Same dataset, same table, new digest:
 
@@ -538,15 +635,15 @@ revisions, so there is no `_v2.._vN` renaming, no proposal left wedged on an id
 reclaim, and no cleanup after a failed run. Sample again, interrogate again, and keep going until
 the checks and the coverage say what you need them to say.
 
-### 6. Present — only what survived inspection, with caveats
+### 8. Present — only what survived inspection, with caveats
 
 Report what you inspected, and nothing you did not. Nothing else checks this run: `verified` is not
 a word some other worker said about it, it is what you established by reading it. So the honest
-sentence names the reading:
+sentence names the reading, and it says the word **sample** out loud:
 
-> Inspected 50,000 rows from the acquired source snapshot. All four declared checks passed.
-> Coverage was truncated at the row ceiling, and no run window was applied, so this is a sample
-> rather than evidence for a stated calendar interval.
+> Inspected 2,000 rows from the acquired source snapshot — a sample. All four declared checks
+> passed. Coverage was truncated at the row ceiling, and no run window was applied, so this is not
+> evidence for a stated calendar interval.
 
 Three facts carry that sentence and all three are on the run: how many rows you actually looked at,
 what the declared checks reported, and whether `coverage.truncated` was true. A null coverage
@@ -554,6 +651,25 @@ window means the acquisition was unwindowed, not that it covered all time. Never
 row as a whole-population value, never call a run verified because it succeeded, and never report
 coverage you did not read off `coverage`.
 
+**Offer the sample's own bytes.** The sample is a sealed table like any other and its Parquet is
+downloadable — `mr-data download RUN --output DIR --json` — so somebody who wants to look at it
+before agreeing to a full run can. Say so.
+
+**Then stop, and ask whether to run the full build.** This is a pause, and it is the second one
+this document allows alongside the promote decision. A full run over a real window costs real time
+and real money, and it is a decision about the user's data rather than about the recipe, so it is
+theirs and not yours. State what the sample established, what the full run would cover, and ask.
+Only when they say so:
+
+```sh
+mr-data run RECIPE_ID --full --json
+```
+
+Then interrogate that run as stage 6 says, and present it with the sample's caveats removed and
+its own read off `coverage`. Do not start a full run because the sample looked good, because the
+checks passed, or because the user asked for the dataset in the first place — the request was for a
+dataset, and the decision in front of them is whether THIS is the recipe to spend a full run on.
+
 ## Live run
 
 `mr-data run` answers with the run identifier and Studio's own run record. These are machine-facing
@@ -628,26 +744,43 @@ not depend on which command you happened to ask.
 
 ## Narrating the run
 
-The run's own record is where the source comparison and the decision lines belong, not only the
-reply. A cell is one outcome-led heading, a markdown body, and typed blocks the Dataset page lays
-out as rows a reader can compare.
+There are two decision records and they are not interchangeable. A cell is the same document in
+both: one outcome-led heading, a markdown body, and typed blocks the Dataset page lays out as rows
+a reader can compare. What differs is which log it lands in and when that log closes.
 
-**Write it while the run is going, which is a narrower window than it sounds.** A run's narrative
-closes when the run reaches a terminal state, and an append after that answers
-`409 RUN_NARRATIVE_CLOSED` — a refusal no client check can soften, because only the backend knows
-the run has finished. A sample run reaches its terminal state in seconds, so "while the run is
-going" means immediately after `mr-data run` returns the identifier, not after `status` says the
-run succeeded. Stage 3 of the [Required protocol](#required-protocol) is where that lands in the
-loop, and it is why stages 1 and 2 draft their blocks into a file rather than calling this
-command.
+**The DATASET's record is the primary one, and it never closes.**
+`mr-data dataset note DATASET_ID` writes it. The sources compared and refused, the grain argued
+over, the question a person answered — those are facts about the dataset, they are true a week
+later, and a reader opening the page long after every run has scrolled away should still find them.
+Stage 3 of the [Required protocol](#required-protocol) is where the first cell goes, before
+anything is registered.
 
 ```sh
-mr-data note --run RUN_ID --heading "Using FAA ASPM hourly" \
-  --blocks-file BLOCKS.json --phase acquire --json <<'EOF'
-Hourly airport throughput, joined to hourly weather.
+mr-data dataset note DATASET_ID --heading "Sources and decisions" \
+  --blocks-file BLOCKS.json --json <<'EOF'
+Two feeds carry hourly stage; one is licensed and cannot be used here.
 EOF
 ```
 
+**A RUN's record is secondary, and it closes with the run.** `mr-data note --run RUN_ID` writes it,
+and what belongs there is what is true of that run alone: a clamp that truncated, a check that did
+not pass, the exact revision this attempt tested. A run's narrative closes when the run reaches a
+terminal state, and an append after that answers `409 RUN_NARRATIVE_CLOSED` — a refusal no client
+check can soften, because only the backend knows the run has finished. A sample run reaches its
+terminal state in seconds, so "while the run is going" means immediately after `mr-data run`
+returns the identifier, not after `status` says the run succeeded.
+
+```sh
+mr-data note --run RUN_ID --heading "Row ceiling truncated the sample" \
+  --phase acquire --json <<'EOF'
+The clamp cut acquisition at 2,000 rows.
+EOF
+```
+
+Writing the search into a run's record instead was the old shape, and the reason it changed is that
+it put the most durable facts in the shortest-lived place: the record of why a dataset looks the
+way it does died with a throwaway sample run, and had to be written again on every attempt.
+
 `BLOCKS.json` is a JSON array of at most 64 objects, of three kinds:
 
 | Kind | Required | Optional |
@@ -667,13 +800,16 @@ where you did not establish the fact; never guess one.
 where there was a rival worth naming, and `because`. A decision without its `because` is a
 changelog entry, and the run's own log is already that.
 
-A `clarification` records an exchange that already happened, and only the two pauses this document
-allows: the promote decision, and a spend confirmation over the threshold. `answered_at` is
-admitted only beside `answer` — a time with no answer records that the question was settled by
-nobody — and is a UTC timestamp ending in `Z`, as `2026-09-01T10:00:00Z`. It is never a way to
-ask. Everything under [Autonomous delivery](#autonomous-delivery) still holds — do not put a
-source, Reader, engine, format, workspace, retry tactic, cleaning operation or repair to the user,
-in a block or anywhere else.
+**A `clarification` block with a `question` and no `answer` is a PENDING question**, and it is the
+one shape here that is a way to ask: the Dataset page shows it, and the chat asks it in the same
+words. Write one only at a genuine fork, as stage 3 defines one. The answer is a REVISION of that
+cell — `--revise CELL_ID`, the same blocks again with `answer` and `answered_at` filled in — so the
+page shows the question settled rather than a second cell nothing ties to the first. `answered_at`
+is admitted only beside `answer`, because a time with no answer records that the question was
+settled by nobody, and is a UTC timestamp ending in `Z`, as `2026-09-01T10:00:00Z`. Everything
+under [Autonomous delivery](#autonomous-delivery) still holds — do not put a source, Reader,
+engine, format, workspace, retry tactic, cleaning operation or repair to the user, in a block or
+anywhere else.
 
 `--phase` binds the cell to one build stage — `acquire`, `parse`, `transform`, `check`, `persist` —
 so a reader can group a run's record by stage. `--checkpoint` places the cell beside the event it
@@ -683,9 +819,10 @@ itself changes with the conclusion, and `--revise CELL_ID` where the successor i
 named cell — the superseded one stays in the log and is flagged. A cell revises itself by reusing
 its own identifier, so `--revise` naming the cell you are writing is refused.
 
-`mr-data note --run RUN_ID --list --json` reads back what is written, superseded revisions
-included and flagged. It reads and writes nothing, so it refuses any flag that describes a cell —
-`--heading`, `--blocks-file`, `--phase` and the rest — rather than ignoring it.
+`mr-data note --run RUN_ID --list --json` and `mr-data dataset note DATASET_ID --list --json`
+read back what is written, superseded revisions included and flagged. Each reads and writes
+nothing, so it refuses any flag that describes a cell — `--heading`, `--blocks-file`, `--phase` and
+the rest — rather than ignoring it.
 
 The bounds are refused before the request rather than after it, in the words the API would have
 used: 64 blocks a cell, 200 characters a label, 255 a host, 1024 an explanation, 200 characters of
@@ -695,10 +832,10 @@ tab and newline and nothing else. Where `--blocks-file` is given it must hold at
 an empty array is refused rather than read as "no blocks". The markdown body is always required
 and is what a reader gets when a block kind is unknown to it.
 
-One bound is the backend's alone, because it needs the log: a run's whole narrative holds 512
-appended cells, revisions counted, and the 513th answers `429 NARRATIVE_BUDGET_EXCEEDED`. It is a
-ceiling on a decision record rather than on a commentary, and nothing that respects the paragraph
-below will come near it.
+One bound is the backend's alone, because it needs the log: a narrative holds 512 appended cells,
+revisions counted, and the 513th answers `429 NARRATIVE_BUDGET_EXCEEDED`. It is a ceiling on a
+decision record rather than on a commentary, and nothing that respects the paragraph below will
+come near it.
 
 Blocks are facts, not narration. A `source` block whose `grain` is `one row per hour per airport`
 is a row; `I evaluated four candidate feeds and settled on the FAA one` is the sentence it
@@ -872,23 +1009,29 @@ schedule, catch-up, or current bytes from the command name or from route accepta
 Own the run until the requested outcome is delivered: a run that succeeded, its declared checks
 read, its coverage inspected, and a publication request made when the user asked for it. Promotion
 is complete only when Studio returns the resulting live state; catch-up and continuing freshness
-require separate durable evidence from the deployed component that owns them. The only allowed
-user pauses are the promote decision and a spend confirmation on a run over the threshold. Do
-not ask the user to choose sources, Readers, engines, formats, workspaces, retry tactics, cleaning
-operations, or validation repairs. Make those operational decisions from the question, observed
-evidence, registered capabilities, and deterministic receipts.
-
-One clarifying question is not a pause and does not weaken that ban. It is allowed for a fact the
-user holds and no source can supply: what an ambiguous word in their request means, how wide the
-population is, or how far back they need it. `Throughput: movements or passengers?` is such a fact.
-Which Reader parses the feed is not, and neither is anything else named above; those stay banned
-because the evidence settles them. Ask before probing, while nothing is registered, nothing is
-running and nothing has been spent: a question at that point costs a sentence, and the guess it
-replaces costs a whole build. Ask ONE question, carrying the answer you will assume if none comes
-back. One, not two: `mr-data clarify` records exactly one question per invocation and the block it
-composes has no field relating a question to another, so a second question is a second record with
-nothing tying it to the first. If two things are genuinely unsettled, the one that survives the
-evidence is the one to ask about.
+require separate durable evidence from the deployed component that owns them. The allowed user pauses
+are the promote decision, the decision to start a full run after the sample, and a spend
+confirmation on a run over the threshold. Do not ask the user to choose sources, Readers, engines,
+formats, workspaces, retry tactics, cleaning operations, or validation repairs. Make those
+operational decisions from the question, observed evidence, registered capabilities, and
+deterministic receipts.
+
+**The decision point replaces the one-question rule.** There used to be a budget of exactly one
+clarifying question, and it was the wrong instrument: it made the agent hoard its one question
+against a fork that might arrive later, and it left the user with no view of the search at all
+until the page was finished. What stage 3 asks for instead is a STATEMENT — sources chosen, sources
+rejected or blocked with the concrete unblock step, the assumption taken about window and grain —
+posted to the dataset's record and to the chat, after which the build continues without waiting.
+A person who disagrees with it says so; a person who does not is not made to answer anything.
+
+A question is still asked, and it is asked at a genuine FORK: two answers that produce two
+different tables, and no evidence that can settle it. `Throughput: movements or passengers?` is
+such a fork. Which Reader parses the feed is not, and neither is anything else named above; those
+stay banned because the evidence settles them. There is no budget of one — a build with two real
+forks asks about both — but a fork is rare, and a question that the evidence could have answered is
+still the failure this section is about. Ask before registering, while nothing is running and
+nothing has been spent, write it as a pending `clarification` block on the dataset, and carry the
+answer back as a revision of that cell.
 
 `mr-data clarify --question TEXT` says whether to ask at all. It exits non-zero when no person can
 answer, which is what a scheduled or headless session reports, and when `--recipe RECIPE_ID` names
@@ -900,6 +1043,10 @@ block. Carry its wording into the recipe document the answer settles — the que
 description, the column description — so the assumption the table rests on reads the same in the
 document, in the Dataset, and in the report.
 
+`clarify` treats the session as attended when `--attended` is passed, or when the environment
+carries an agent-host marker. Pass `--attended` explicitly whenever a person is in the chat: the
+marker is a fallback for a caller that did not say so, not a substitute for saying it.
+
 On a hosted install the delivered outcome is concrete: a run in `succeeded`, `mr-data checks` reporting
 every declared check, the coverage read off the run rather than guessed, and the artifacts brought
 here by `mr-data download` when the user asked for the bytes. Link to the canonical Dataset page and

{mostlyright_data-0.10.7 → mostlyright_data-0.11.0}/src/mostlyright/data_harness/declared_checks.py

@@ -77,7 +77,7 @@ SPECIFICATION cannot be parsed and ``"check"`` when an EVALUATION cannot proceed
 are members of ``docs/V4-WORKER-PROTOCOL.md``'s ``failed_stage`` vocabulary. The first was
 ``"transform"`` until the driver was corrected to parse before acquiring: a specification refusal
 happens before all five stages, so it reports at the first, and ``"transform"`` would send an
-operator looking for a build that never began (``run_execution_v4._DOCUMENT_STAGE`` states the same
+operator looking for a build that never began (``run_execution_v4.DOCUMENT_STAGE`` states the same
 rule for the job document). Nothing about WHICH refusals are raised changed with it. A check that
 merely FAILS is not a refusal and raises nothing — it returns a ``CheckResult`` with
 ``passed=False``, and ``required_failure`` is the separate query the driver asks to learn
@@ -139,7 +139,7 @@ DECLARED_CHECK_FAILURE_CODES: tuple[str, ...] = (
 )
 
 #: The stage a SPECIFICATION refusal reports under. ``acquire``, the first, for the reason
-#: ``run_execution_v4._DOCUMENT_STAGE`` records about the job document: a refusal that happens
+#: ``run_execution_v4.DOCUMENT_STAGE`` records about the job document: a refusal that happens
 #: before every one of the five stages reports at the first, because "nothing was fetched" is what
 #: a reader of the run record needs to know. This used to be ``transform``, which was true of WHERE
 #: the driver called the parser and never of what the parser does -- and the driver now calls it