mostlyright-data 0.11.0__tar.gz → 0.11.1__tar.gz

{mostlyright_data-0.11.0 → mostlyright_data-0.11.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.11.0
+Version: 0.11.1
 Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.11.0 → mostlyright_data-0.11.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.11.0"
+version = "0.11.1"
 description = "Mostly Right CLI for reproducible, reviewed Parquet datasets"
 readme = "README.md"
 requires-python = ">=3.13,<3.14"

{mostlyright_data-0.11.0 → mostlyright_data-0.11.1}/skills/mr-data-build/SKILL.md

@@ -49,6 +49,16 @@ assessment is complete. Review the proposed schema and coverage in the Dataset.`
 routine stages, predict duration or throughput, or expose raw state names. Write in short,
 professional product language and lead with the outcome or decision.
 
+**Whatever is said in chat is written to the page, in the same words, at the same moment.** Every
+sentence sent to the user during a build is also a cell on the dataset's record: write it with
+`mr-data dataset note`, taking the message's first sentence as the heading and the whole message as
+the body, before or in the same breath as sending it. The two are one act, not a message and a
+later summary of it. A person who joins by opening the page reads exactly what the person in the
+chat read, and a person who scrolls the chat away still has it. Nothing is said in one place and
+not the other: silent work stays silent in both, and a message that is not worth writing down is
+not worth sending. This is a rule about the messages this contract already allows — it does not
+license one extra update.
+
 For hosted work, register the Dataset first and open its canonical page immediately in the host's
 in-app browser: `mr-data open /datasets/DATASET_ID` mints a single-use address for it, and
 when that address cannot be minted, carry on without it rather than blocking.
@@ -98,15 +108,15 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`; `dataset show ID` reads it back; `dataset set ID --name TEXT --description-file F` writes the title and the description under the version it was read at, retrying once if somebody else wrote first; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream. |
 | `mr-data recipe` | Register one recipe document — dataset, question, table plan, sources, transform, checks and units — in one call, and print the identifiers the server derived. `mr-data recipe show ID` reads one back. |
 | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
-| `mr-data run` | Start one run against a registered recipe. The command accepts `--sample`, `--full`, `--refresh`, `--backfill` and `--window`, and the worker executes `--sample`, `--full` and `--refresh` (a refresh re-acquires the declared addresses over the previous version and claims no window); `--backfill` and a `--window` on a sample or full are refused before acquisition. `--max-rows` and `--max-source-bytes` remain supported. A large run is held for a spend confirmation, which `--confirm` settles; a sample-first held full requires Studio's interactive human approval instead. `--cancel RUN` stops one. |
+| `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID --digest RECIPE_DIGEST` — both required, neither positional, and the digest is the bare hex the registration receipt printed. The command accepts `--sample`, `--full`, `--refresh`, `--backfill` and `--window`, and the worker executes `--sample`, `--full` and `--refresh` (a refresh re-acquires the declared addresses over the previous version and claims no window); `--backfill` and a `--window` on a sample or full are refused before acquisition. `--max-rows` and `--max-source-bytes` remain supported, and each bounds ONE SOURCE rather than the finished table. A large run is held for a spend confirmation, which `--confirm` settles; a sample-first held full requires Studio's interactive human approval instead. `--cancel RUN` stops one. |
 | `mr-data status` | Report which of the seven states one run is in, what it delivered and whether a ceiling cut it short, and — on a failure — what failed and where. Exits non-zero on a failed run. |
 | `mr-data runs` | Report this workspace's own runs: identifier, mode, state and creation time, plus the failure triple of any that failed. `--status` and `--mode` narrow it and are refused for a word outside the seven or the four; `--limit` says how many, and pages are followed to reach it. |
 | `mr-data watch` | Stream one run's live progress under the durable event type of each stage, resuming across stream cuts. Exits non-zero when the run failed. |
 | `mr-data query` | Run one bounded read-only SQL statement over a finished run's own table and print the rows. Bounded on rows, on answer bytes and on time, and refused by name when a bound is crossed. |
-| `mr-data peek` | Print the columns, types and first rows of one run's table. |
+| `mr-data peek` | Print the columns, their logical types and the first rows of one run's table. The type comes off the run's own column profile where the preview carries none, so `type` is null only for a run that sealed no profile. |
 | `mr-data receipt` | Print what one run recorded about how it was built: every source it fetched and what those bytes hashed to, the coverage it delivered, and the members of its raw snapshot. |
 | `mr-data checks` | Report how each check the recipe declared came out — the name, whether it passed, what it found. Exits non-zero when one did not pass. |
-| `mr-data note` | Write one cell of a run's decision record — a heading, a body, and the typed blocks that carry a source, a decision or a clarification onto the stage its phase names. `--list` reads back what is written. |
+| `mr-data note` | Write one cell of a run's decision record — a heading, a body, and the typed blocks that carry a source, a decision or a clarification onto the stage its phase names. The body comes from `--markdown-file FILE`, which is the form to use; standard input is read when no file is named and is refused when it is a terminal or empty. `--list` reads back what is written. |
 | `mr-data download` | Bring one run's artifacts back to this computer, each one's bytes checked against the digest Studio sealed them under. `--kind` selects out of `table_parquet`, `column_profile`, `preview`, `receipt` and `raw_snapshot`; without it, the table and the receipt. |
 | `mr-data diff` | Compare two runs and say what changed. |
 | `mr-data keys` | Enrol a source credential by name, list the names with their created and rotated times, and delete one. `keys set` reads the value from a file or from standard input and never from the command line; no command prints a value back. |
@@ -224,8 +234,16 @@ every stage below writes to it as it goes rather than composing the page at the
 Authenticate first: run the Cloud authentication preflight above once, before the first cloud
 command of the session.
 
-**Then spend two cheap lookups before the first run.** Both regularly decide the task, and skipping
-them is what turns a half-hour build into an afternoon.
+**Then stage 1, and nothing before it.** `mr-data dataset create` and `mr-data open` are the first
+two commands of the session after the preflight. They come before the durable notes are read,
+before `mr-data runs`, before a catalog is searched, before a provider's documentation is opened
+and before any URL is fetched. The lookups below are research, research is what fills the page, and
+a page created after the research has already happened is the assertion this stage exists to
+prevent. Nothing in stage 1 depends on knowing the sources: the title is a working one and stage 4
+settles it.
+
+**Then spend two cheap lookups before the first run**, with the page already open. Both regularly
+decide the task, and skipping them is what turns a half-hour build into an afternoon.
 
 Read whatever durable notes you hold for this project. Dead endpoints, retired response formats,
 working URL recipes, the Reader a given publisher needs, and known-open defects are usually already
@@ -257,7 +275,9 @@ message that contradicts you.
 ### 1. Open — the page, before there is anything on it
 
 Create the dataset from a working title and put it in front of whoever is watching, **before any
-research**. It costs two commands and it is what turns a build into something a person can follow:
+source lookup**. These are the first two commands after the preflight — nothing reads a note,
+searches a catalog, opens a provider page or fetches a URL until they have both returned. It costs
+two commands and it is what turns a build into something a person can follow:
 
 ```sh
 mr-data dataset create --name "Kathmandu monsoon flooding" --json
@@ -398,6 +418,14 @@ and it is the identifier stage 1 printed. With it, the registration attaches to
 has been watching; without it, Studio resolves a container from the name and mints a second page
 behind them. Always carry it.
 
+**A document that carries `dataset.id` leaves `dataset.description` out.** The description on the
+page belongs to whoever wrote it — you at stage 4 below, or a person editing the page afterwards —
+and a registration that also states one writes over what is there, so a revision registered an hour
+later silently replaces a description somebody rewrote by hand.
+`mr-data dataset set DATASET_ID --description-file F` is the only way this protocol writes a
+dataset description, and it writes under the version it read. Leave `dataset.name` in: the title is the recipe's own subject
+and is settled once, at stage 4.
+
 Every grain, key, join and cutoff settled after stage 3 is a further `decision` block on the
 DATASET's record — `mr-data dataset note DATASET_ID` — because the fact outlives every run.
 
@@ -525,15 +553,27 @@ shape without pretending they selected a time range. The current worker does not
 window into any source request, so any stated `--window` is refused before acquisition.
 
 ```sh
-mr-data run RECIPE_ID --sample --max-rows 2000 --json
+mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --sample --max-rows 2000 --json
 ```
 
+**Both flags are required and neither is a positional.** `mr-data run RECIPE_ID` is refused:
+`--recipe` names the registered recipe and `--digest` states which bytes of it you mean to run,
+as the bare lowercase hex `recipe_digest` the registration receipt printed — no `sha256:` prefix.
+A digest that disagrees with the registered recipe is a refusal rather than the silent execution of
+a revision nobody read, which is why the client will not derive it for you. Hold both identifiers
+from the registration receipt: every run of that recipe needs the pair.
+
+**`--max-rows` is a per-source ceiling, not a table size.** Each acquired source is truncated at
+that many rows, so a recipe with eight sources delivers up to eight times the number you asked for
+— 2,000 becomes 16,000. Read the delivered size off the run's `coverage` and say the number the run
+reports, never the number you passed.
+
 **Two thousand rows, and the first run is always this one.** A sample is for establishing shape,
-and a usable one has to arrive within minutes: two thousand rows exercises every column, every
-cast and every declared check, and it comes back while the person who asked is still watching the
-page. A larger first sample buys nothing the checks do not already report and costs the one thing
-the page cannot get back, which is the first few minutes. The full run comes later, in stage 8,
-and only when the user says so.
+and a usable one has to arrive within minutes: two thousand rows a source exercises every column,
+every cast and every declared check, and it comes back while the person who asked is still watching
+the page. A larger first sample buys nothing the checks do not already report and costs the one
+thing the page cannot get back, which is the first few minutes. The full run comes later, in stage
+8, and only when the user says so.
 
 **The dataset's record is the primary one; a run's own record is for what happened inside that
 run.** The sources, the decisions and the questions went onto the dataset in stage 3, where they
@@ -543,11 +583,14 @@ tested:
 
 ```sh
 mr-data note --run RUN_ID --heading "Row ceiling truncated the sample" \
-  --phase acquire --json <<'EOF'
-The clamp cut acquisition at 2,000 rows; coverage.truncated is true on both sources.
-EOF
+  --phase acquire --markdown-file BODY.md --json
 ```
 
+**Give a run cell its body with `--markdown-file`.** `note` will read a body on standard input, but
+a caller that cannot guarantee standard input is a pipe gets an empty stream or a terminal, and both
+are refused after the cell is otherwise composed. Write the body to a file and name it; the file is
+the shape that works from any caller.
+
 Write it as soon as the run exists, and before it finishes. `mr-data run` prints the run identifier
 and returns; the run itself then reaches a terminal state on its own, often within seconds on a
 sample. A run's narrative closes with the run, so a cell appended after that answers
@@ -569,9 +612,10 @@ complete acquired response is told from one cut at fifty thousand rows without o
 receipt: read `coverage` off the run, not the row count off the table.
 
 A run over the spend threshold is not started. It answers with the projected bytes, runtime and
-cost, and waits; `mr-data run --confirm RUN` settles it against the number already on the record.
-Any authenticated surface of the paying workspace may settle it. `--cancel RUN` stops a run that
-is queued or running.
+cost, and waits; the same command again with `--confirm` settles it against the number already on
+the record — `mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --confirm`, which the
+held answer prints back as `confirm_command`. Any authenticated surface of the paying workspace may
+settle it. `mr-data run --cancel RUN_ID` stops a run that is queued or running.
 
 Do not treat the accepted command vocabulary as execution evidence. A `refresh` restores the
 previous version, re-acquires the declared addresses in full and seals the next version, claiming
@@ -584,12 +628,18 @@ request to, and is refused before acquisition.
 Four reads, and none of them waits on anything:
 
 ```sh
-mr-data peek RUN --json                       # columns, types, first rows
+mr-data peek RUN --json                       # columns, logical types, first rows
 mr-data query RUN "select count(*) from t" --json
 mr-data checks RUN --json                     # every check the recipe declared
 mr-data receipt RUN --json                    # sources, digests, coverage, snapshot members
 ```
 
+**`peek` reports the logical type the run sealed.** The preview artifact carries column names
+alone, so the type is read from the run's own `column_profile` — the same artifact `diff` compares
+schemas with, which is why the two agree about what a column is. A `type` of null means the run
+sealed no profile for that column, not that the column is untyped; check the run reached `persist`
+before reporting a schema from it.
+
 `query` is bounded on rows, on answer bytes and on time, and each bound has its own named refusal.
 The refusals are cheap and legible by design, so read the code rather than retrying:
 
@@ -662,7 +712,7 @@ theirs and not yours. State what the sample established, what the full run would
 Only when they say so:
 
 ```sh
-mr-data run RECIPE_ID --full --json
+mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --json
 ```
 
 Then interrogate that run as stage 6 says, and present it with the sample's caveats removed and
@@ -730,7 +780,8 @@ cite a progress event where a receipt is what is being asked for.
 **Seven states, one machine.** A run is `queued`, `running`, `awaiting_confirmation`,
 `awaiting_sample_approval`, `succeeded`, `failed` or `cancelled`. Three of them are terminal and
 end a watch: `succeeded`, `failed` and `cancelled`. `awaiting_confirmation` is the ordinary spend
-gate: nothing has been executed, and `mr-data run --confirm RUN` releases it.
+gate: nothing has been executed, and the run command again with `--confirm` on the end of it
+releases it: `--recipe`, `--digest` and the mode are all stated a second time.
 `awaiting_sample_approval` is different: a bounded sample must seal a downloadable
 `table_parquet`, then a person completes Studio's fresh operation-bound interactive approval before
 the linked full run dispatches. Do not present `--confirm` as an alternative for that state. Keep
@@ -748,6 +799,20 @@ There are two decision records and they are not interchangeable. A cell is the s
 both: one outcome-led heading, a markdown body, and typed blocks the Dataset page lays out as rows
 a reader can compare. What differs is which log it lands in and when that log closes.
 
+**The record and the chat carry the same sentences.** Every message sent to the user during a build
+is written to the dataset's record as a cell at the same moment and in the same words — the
+message's first sentence is the `--heading`, the whole message is the body. Not a paraphrase, not a
+summary written afterwards, and not a shorter version for the page: the page is what somebody
+watching reads instead of the chat, so a sentence that exists in only one of them is a fact one of
+the two readers does not have. The rule runs both ways: do not write a cell about work this
+document says to keep silent, and do not send a message you are not willing to write down.
+
+```sh
+mr-data dataset note DATASET_ID --heading "Two feeds carry hourly stage" --json <<'EOF'
+Two feeds carry hourly stage; one is licensed and cannot be used here. Building on the other.
+EOF
+```
+
 **The DATASET's record is the primary one, and it never closes.**
 `mr-data dataset note DATASET_ID` writes it. The sources compared and refused, the grain argued
 over, the question a person answered — those are facts about the dataset, they are true a week
@@ -772,9 +837,7 @@ returns the identifier, not after `status` says the run succeeded.
 
 ```sh
 mr-data note --run RUN_ID --heading "Row ceiling truncated the sample" \
-  --phase acquire --json <<'EOF'
-The clamp cut acquisition at 2,000 rows.
-EOF
+  --phase acquire --markdown-file BODY.md --json
 ```
 
 Writing the search into a run's record instead was the old shape, and the reason it changed is that
@@ -830,7 +893,9 @@ heading and 65536 UTF-8 **bytes** of body. Every block field is one line of plai
 control character, no Unicode line separator, no bidirectional override — while the body admits
 tab and newline and nothing else. Where `--blocks-file` is given it must hold at least one object:
 an empty array is refused rather than read as "no blocks". The markdown body is always required
-and is what a reader gets when a block kind is unknown to it.
+and is what a reader gets when a block kind is unknown to it. It comes from `--markdown-file FILE`,
+or from standard input when no file is named; a terminal and an empty stream are both refused, so
+name the file wherever the caller cannot guarantee a pipe.
 
 One bound is the backend's alone, because it needs the log: a narrative holds 512 appended cells,
 revisions counted, and the 513th answers `429 NARRATIVE_BUDGET_EXCEEDED`. It is a ceiling on a
@@ -859,7 +924,7 @@ workspace exactly as it found it.
 | `mr-data receipt RUN --json` | How the run was built: every source it fetched and what those bytes hashed to, the coverage it delivered, and the members of its raw snapshot. |
 | `mr-data checks RUN --json` | How each declared check came out. |
 | `mr-data download RUN --output DIR --json` | The artifacts, each verified against the digest Studio sealed it under before it is written. |
-| `mr-data peek RUN --json` | The columns, types and first rows. `--rows` narrows it and cannot widen it. |
+| `mr-data peek RUN --json` | The columns, their logical types off the run's column profile, and the first rows. `--rows` narrows it and cannot widen it. |
 | `mr-data diff LEFT RIGHT --json` | What changed between two runs. `--columns-only` narrows what is compared and never what is counted. |
 
 These receipt fields are internal verification evidence. Read them only when verification needs