mostlyright-data 0.19.4__tar.gz → 0.19.6__tar.gz

{mostlyright_data-0.19.4 → mostlyright_data-0.19.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.19.4
+Version: 0.19.6
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.19.4 → mostlyright_data-0.19.6}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.19.4"
+version = "0.19.6"
 description = "Mostly Right hosted CLI for reviewed datasets"
 readme = "README.md"
 requires-python = ">=3.13"

{mostlyright_data-0.19.4 → mostlyright_data-0.19.6}/skills/mr-data-build/SKILL.md

@@ -155,7 +155,7 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data clarify` | Say whether one clarifying question may be put to the user right now, and compose the record of their answer. It reaches nothing, asks nobody and waits for nothing. Exits non-zero when no person can answer or when a recipe is already registered. |
 | `mr-data probe` | `probe SOURCE_ID QUESTION_ID --dataset DATASET_ID [--kind source_inspect\|sample_rows\|profile_columns\|evaluate_expression]` asks one already-registered source one question and prints the answer. **Do not plan a source inspection around it.** Both positionals are identifiers — `QUESTION_ID` is a question identifier, which no v4 registration receipt returns — and the command rides the frozen `/v3/sessions` routes, which a deployment may have switched off. Read a source instead by registering the recipe and taking an unwindowed sample under `--max-rows`, then `peek`, `query` and `receipt`. |
 | `mr-data catalog` | `catalog search "QUESTION"` asks the sealed public-source catalogue which of its entries might answer a question and ranks them best first. Every ranked entry comes back with the disposition its own facts earned — `admitted`, `human_escalation_required` or `refused` — because an entry the catalogue could not vouch for is still a finding. `--format csv` states the one data format the question requires — one token per question, never repeated — and it is not a filter: an entry that does not declare that format still comes back ranked, with disposition `refused` and `filters_match` false, so nothing is held back; `--limit N` ranks at most N, up to 25. It fetches nothing and registers nothing. The catalogue holds one provider (Data.gov) and only part of it, so it is never exhaustive and an empty answer is not evidence that no such source exists. When the answer is `catalog_unavailable`, this deployment has no catalogue to search: record the lane and carry on. |
-| `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`; `dataset show ID` reads it back; `dataset set ID --name TEXT --topics "a,b,c" --license ID --description-file F` writes the title, the descriptive tags, the SPDX licence and the description under the version it was read at, retrying once if somebody else wrote first, and an empty `--topics` or `--license` takes that value off the page; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream; `dataset activity ID --phase P --message TEXT` says what is happening right now, silently, and is never a chat message and never a cell; `dataset publish ID [--mode public|link|private]` says who can read the dataset — `public` lists it in the public directory and serves it at an address anybody can read, `link` serves it at an unlisted address, `private` takes it back to the workspace — and `dataset publish ID --show` reads that back without changing it; `dataset archive ID --confirm-name TITLE` retires the page and frees its title, deleting nothing. |
+| `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`; `dataset show ID` reads it back; `dataset set ID --name TEXT --topics "a,b,c" --license ID --description-file F` writes the title, the descriptive tags, the SPDX licence and the description under the version it was read at, retrying once if somebody else wrote first, and an empty `--topics` or `--license` takes that value off the page; a saved write whose public sync fails exits 2 and reports `public_projection_synced: false` — use `dataset sync ID` to retry that sync without rewriting Studio; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream; `dataset activity ID --phase P --message TEXT` says what is happening right now, silently, and is never a chat message and never a cell; `dataset publish ID [--mode public|link|private]` says who can read the dataset — `public` lists it in the public directory and serves it at an address anybody can read, `link` serves it at an unlisted address, `private` takes it back to the workspace — and `dataset publish ID --show` reads that back without changing it; `dataset archive ID --confirm-name TITLE` retires the page and frees its title, deleting nothing. |
 | `mr-data recipe` | Register one recipe document — dataset, question, table plan, sources, transform, checks and units — in one call, and print the identifiers the server derived. `mr-data recipe show ID` reads one back. |
 | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
 | `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID --digest RECIPE_DIGEST` — both required, neither positional, and the digest is the bare hex the registration receipt printed. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh narrows each source that declares a supported address window and merges its returned relation into retained source state; unwindowed sources are conditionally revalidated or re-acquired whole, then the full declared transform runs. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal — report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. A large run is held for a spend confirmation, which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried; a sample-first held full requires Studio's interactive human approval instead. `--cancel RUN` stops one. |
@@ -640,16 +640,16 @@ seconds -- not a paragraph, and not a report. Use exactly this shape, in this or
 section out only when there is genuinely nothing to say (shown indented here; the document
 itself carries the headings at column one):
 
-    Two or three plain sentences: what the data is about, why it is worth a look, and what is
-    in it.
+    A plain opening paragraph: what the data contains, what question or model it supports, and
+    the facts a reader needs before using it.
 
     ## Coverage
-    - **Window:** the exact dates and the time zone.
-    - **Grain:** one row per what.
-    - **Cadence:** how often rows arrive, and how many per day or month.
+    - Window: the exact dates and the time zone.
+    - Grain: one row per what.
+    - Cadence: how often rows arrive, and how many per day or month.
 
     ## Columns
-    - `column_name` — what it measures, in which unit.
+    - `column_name`: what it measures, in which unit.
     - (one bullet per column; group unit-sharing columns on one line where that reads better)
 
     ## Missing values
@@ -665,43 +665,33 @@ That is six sections: the opening paragraph, and the five headings under it.
 
 **Opening paragraph** (everything before the first `##` heading)
 
-Two or three short sentences, 120 to 280 characters, written for somebody who has never heard of
-the source, the venue or the format. Sentence one says what the data is about and what makes it
-worth a look, in at most 110 characters. The rest says, in the same plain words, what is in it.
-
-1. No tickers, station codes, series codes, format names, publisher names, row definitions (the
-   sentence that begins `Each row is`) or row counts here. Every one of those facts is in the
-   sections below, which is where a reader who wants it will look.
-2. No Markdown: no bold, no links, no bullets, no code.
-3. A hook is an interesting fact stated plainly, never a pitch. Banned outright: comprehensive,
-   rich, valuable, powerful, unlock, explore, dive into, perfect for, insights, "this dataset", a
-   rhetorical question, an exclamation mark.
-4. Every number in it must be one the run established. "About every half hour" and "thousands of
-   markets" are claims, and a claim that is not on the run does not go here.
-5. The stranger test, before anything else is written: read sentence one alone under the title. If
-   a curious non-specialist could not say what the data is about and would not want the next line,
-   rewrite it.
-6. Plain and short. Every sentence under about twenty words, every sentence a fact: the venue, the
-   subject, the horizon, the cadence, the window. Nothing about what a reader might do with it: no
-   "so you can", "a way to", "ready to", "for seeing". Say what is there and stop.
-
-One airport weather table, the shape to avoid and the shape to write. To avoid:
-
-    Each row is one METAR or SPECI weather report from Beijing (ZBAA), since January 2020, via
-    Iowa Environmental Mesonet. 117,876 reports of temperature, wind, pressure, visibility with
-    the day's highest and lowest reading.
-
-To write:
-
-    Beijing airport weather since 2020, a report about every half hour, with the day's high and
-    low on each row. Temperature, wind, pressure and visibility as the airport recorded them.
-
-The station code, the report types, the publisher and the count all moved down to Coverage and to
-Source and rights, where a reader who wants them looks. "About every half hour" stays only because
-Coverage puts a month at 1,302 to 1,591 rows, which is one row every 27 to 33 minutes. Three ways
-to fail: the pitch (`Unlock real-time insights into how prediction markets price
-the weather!`), the promise (`ready to test a forecast against what actually happened`) and the
-bare subject with no reason to care (`Beijing weather data.`).
+Write as an engineer explaining a useful table to another engineer. The opening may have more than
+two sentences. Start with the concrete records the dataset contains. Then say which question the
+records help answer or which model they can support. Include geography, time range, row grain,
+update cadence and material limitations when those facts are known. Put a join, transformation or
+source caveat here only when it changes how somebody should use the data.
+
+1. The first two sentences must distinguish this dataset from every other dataset in the set. If
+   both sentences could describe another dataset unchanged, rewrite them.
+2. Use ordinary verbs such as contains, tracks, joins, updates, records and returns. Vary sentence
+   length, and use short sentences for the important facts.
+3. Preserve every established fact. Never invent coverage, freshness, quality, licensing or an
+   intended use.
+4. Write `Mostly Right`, with a space, except inside a literal identifier that uses another form.
+5. Use no Markdown in the opening paragraph. Do not use generic openings such as `This dataset
+   provides`, marketing claims such as `comprehensive`, `powerful` or `high-quality`, or abstract
+   phrases such as `enables insights`, `facilitates analysis` or `serves as a valuable resource`.
+6. Do not explain page design or metadata fields. Do not use em dashes, semicolons, bold labels or
+   fake quotations. Explain the data before implementation details.
+
+After the build establishes each fact, an airport-weather opening can read:
+
+    Hourly weather observations from 20 airport stations across the United States. Each row records
+    one station report with temperature, dew point, wind, visibility, and observation time. Use it
+    to compare observed conditions across stations and time or train a short-term weather model.
+    The table keeps multiple reports within an hour when conditions change, so aggregate it before
+    using one row per station-hour. Mostly Right refreshes the recent window and preserves older
+    observations.
 
 **Body rules** (the five sections under the headings)
 1. Every sentence must stay true and complete when quoted alone, away from the page. Never
@@ -715,21 +705,26 @@ bare subject with no reason to care (`Beijing weather data.`).
 5. Everything in the paragraph below still applies.
 
 Rules that keep it readable: short sentences; no sentence over about twenty-five words; bullets
-rather than comma lists; no tables, no code blocks, no links to internal tools, no headings beyond
+rather than comma lists; no bold labels, em dashes, semicolons or fake quotations; no tables, no
+code blocks, no links to internal tools, no headings beyond
 the ones above. Spell out the names a reader would search for -- the place, the identifiers in
 every common form, the programme and the publisher -- once each, under Coverage and under
 Source and rights, and never as a keyword list. A one-line description is a defect to fix in the
 revision, not a style choice. Give `table.description` the grain and the window.
+Every statement in the opening and body must preserve an established fact. Never add unsupported
+coverage, freshness, quality, licensing or intended use.
 
 **Check before writing.** Fix and re-check until every line passes:
 - title: at most 60 characters, contains the place, contains no colon, pipe or exclamation mark,
   no brand, no "dataset" or "data", no version number, no run date
-- opening paragraph: two or three sentences, each under about twenty words, 120 to 280 characters,
-  first full stop before character 110, no Markdown, no identifier, publisher name, row count or
-  row definition, no banned word, no clause about what a reader might do
+- opening paragraph: at least two distinct sentences, with the concrete records first and the
+  question or model they support next; include known geography, time range, grain, update cadence
+  and material limitations; no Markdown, banned style or invented fact
 - category: exactly one fixed ID saved and verified before recipe registration
 - topics: 3 to 8 descriptive tags, no duplicates, lowercase, at most 40 characters each
 - licence: an SPDX identifier the sources actually grant, or `--license` left off
+- whole description: every statement preserves an established fact; no bold labels, em dashes,
+  semicolons, fake quotations or unsupported claims
 - every `##` heading in DESCRIPTION.md is one of the five, in that relative order, none repeated
 
 **Every dataset gets a cover before its first run is presented.** Once the description is

{mostlyright_data-0.19.4 → mostlyright_data-0.19.6}/src/mostlyright/data_harness/thin/router.py

@@ -461,7 +461,7 @@ def _exit_codes() -> dict[str, Callable[[dict[str, Any]], int]]:
     ``ux.commands.CommandSpec`` gives: an exit code that can disagree with the printed answer is
     an exit code somebody will trust over the answer.
 
-    Eight entries, and all eight are answers a script acts on rather than reports it reads: a run
+    These are answers a script acts on rather than reports it reads: a run
     held at the spend gate has not started, a run that failed is not a build whichever surface it
     was learned from, a table whose declared checks did not all pass is not one to go on with, a
     question this command stopped waiting on has not been answered, and a probe that settled
@@ -476,7 +476,14 @@ def _exit_codes() -> dict[str, Callable[[dict[str, Any]], int]]:
     """
 
     from mostlyright.data_harness.thin import probe as probe_module
-    from mostlyright.data_harness.thin import v4_artifacts, v4_query, v4_runs, v4_stream, v4_tables
+    from mostlyright.data_harness.thin import (
+        v4_artifacts,
+        v4_datasets,
+        v4_query,
+        v4_runs,
+        v4_stream,
+        v4_tables,
+    )
 
     return {
         "run": v4_runs.run_exit_code,
@@ -488,6 +495,7 @@ def _exit_codes() -> dict[str, Callable[[dict[str, Any]], int]]:
         "probe": probe_module.probe_exit_code,
         "table": v4_tables.lifecycle_exit_code,
         "demote": v4_tables.lifecycle_exit_code,
+        "dataset": v4_datasets.dataset_exit_code,
     }
 
 

{mostlyright_data-0.19.4 → mostlyright_data-0.19.6}/src/mostlyright/data_harness/thin/session.py

@@ -63,6 +63,9 @@ CLOUD_SESSION_HANDOFF_PATH = "/api/cli/session-handoffs"
 #: this client sends a mode and renders what came back rather than deciding any part of it.
 CLOUD_DATASET_ACCESS_PATH = "/api/cli/datasets/{dataset_id}/access"
 
+# Projection repair after a Studio metadata write, without a publication mutation.
+CLOUD_DATASET_IDENTITY_PATH = "/api/cli/datasets/{dataset_id}/identity"
+
 #: The one Cloud address a Dataset has. ``mostlyright-cloud`` serves it at this route with
 #: section suffixes, and a run has no page of its own: its evidence lives on this page's Harness
 #: card, so a receipt names its run with :func:`run_dashboard_url` as a query member here rather

{mostlyright_data-0.19.4 → mostlyright_data-0.19.6}/src/mostlyright/data_harness/thin/v4_datasets.py

@@ -117,6 +117,7 @@ from mostlyright.data_harness.thin.narrative import (
 from mostlyright.data_harness.thin.runs import new_idempotency_key
 from mostlyright.data_harness.thin.session import (
     CLOUD_DATASET_ACCESS_PATH,
+    CLOUD_DATASET_IDENTITY_PATH,
     CLOUD_REFUSAL_CODE,
     MAX_STATED_MESSAGE_CHARS,
     CloudClient,
@@ -152,7 +153,7 @@ from mostlyright.data_harness.thin.v4_narrative import (
     refuse,
     v4_cell_receipt,
 )
-from mostlyright.data_harness.ux.credentials import resolve_cloud_credentials
+from mostlyright.data_harness.ux.credentials import LoginError, resolve_cloud_credentials
 from mostlyright.data_harness.ux.plain_file import PlainFileRefusal, read_plain_file
 
 CREATE_SCHEMA = f"{THIN_SCHEMA_PREFIX}-v4-dataset-created.v1"
@@ -191,6 +192,7 @@ ACTIONS: tuple[str, ...] = (
     "categories",
     "create",
     "set",
+    "sync",
     "draft",
     "show",
     "state",
@@ -205,6 +207,7 @@ ACTIONS: tuple[str, ...] = (
 #: what produces one.
 _NEEDS_DATASET: tuple[str, ...] = (
     "set",
+    "sync",
     "draft",
     "show",
     "state",
@@ -353,13 +356,23 @@ CLOUD_COMPOSED_REFUSAL = "step_up_required"
 
 
 class CloudDatasetAccessClient(CloudClient):
-    """The one Cloud route this command speaks, over the shared device-key Cloud client.
+    """Dataset access and identity routes over the shared device-key Cloud client.
 
     Adds two routes and no transport, for the reason :class:`StudioV4DatasetClient` gives about
     its own: a second ``call`` would be a second place for the origin check, the byte ceiling, the
     credential header and the refusal folding to drift.
     """
 
+    def sync_dataset_identity(self, dataset_id: str) -> dict[str, Any]:
+        """Carry Studio's current metadata into Cloud's existing public projection."""
+
+        return self.call(
+            "POST",
+            CLOUD_DATASET_IDENTITY_PATH.format(dataset_id=dataset_id),
+            act="update the dataset's public page",
+            body={},
+        )
+
     def dataset_access(self, dataset_id: str) -> dict[str, Any]:
         """Report how this dataset is shared now, changing nothing."""
 
@@ -551,7 +564,8 @@ def declare_arguments(parser: argparse.ArgumentParser) -> None:
         "Bring a dataset page into existence before there is anything on it, then fill it in "
         "while somebody watches. `create` mints the page from a working title; `show` reads it "
         "back; `set` writes the final title, the topics, the licence and the description, "
-        "conditional on the version it was read at; `note` writes one cell of the decision "
+        "conditional on the version it was read at; `sync` retries only public-page sync; "
+        "`note` writes one cell of the decision "
         "record that outlives every run; `activity` says in one line what is happening right "
         "now, which the page shows while it happens and forgets afterwards; `watch` follows the "
         "page's own event stream; `publish` says who can read the dataset -- anybody, whoever "
@@ -1148,6 +1162,49 @@ def draft(
     }
 
 
+def _sync_written_identity(dataset_id: str, cloud_url: str) -> dict[str, Any]:
+    """Report partial success honestly: a saved Studio write must not be lost in a sync error."""
+
+    try:
+        credentials = resolve_cloud_credentials()
+        require_device_key(credentials)
+        cloud = CloudDatasetAccessClient(credentials)
+        if cloud.cloud_url != cloud_url.rstrip("/"):
+            raise ThinLaneError(
+                "THIN_CLOUD_ORIGIN_CHANGED", "Cloud origin changed during the write"
+            )
+        answer = cloud.sync_dataset_identity(dataset_id)
+        if (
+            answer.get("schema_version") != "mostlyright-dataset-identity-sync.v1"
+            or answer.get("status") != "synced"
+            or answer.get("dataset_id") != dataset_id
+            or "dataset_version" not in answer
+            or (
+                answer["dataset_version"] is not None
+                and (type(answer["dataset_version"]) is not int or answer["dataset_version"] < 1)
+            )
+            or "dataset_slug" not in answer
+            or (answer["dataset_slug"] is not None and not isinstance(answer["dataset_slug"], str))
+        ):
+            raise ThinLaneError(
+                "THIN_IDENTITY_SYNC_INVALID", "Cloud did not confirm the projection"
+            )
+        return {"public_projection_synced": True}
+    except (ThinLaneError, LoginError, OSError) as error:
+        return {
+            "public_projection_synced": False,
+            "public_projection_error": (
+                error.code
+                if isinstance(error, ThinLaneError) and getattr(error, "http_status", None) != 404
+                else "THIN_IDENTITY_SYNC_UNAVAILABLE"
+            ),
+            "public_projection_note": (
+                "The dataset was saved, but its public page could not be updated. "
+                f"Retry only the sync with: mr-data dataset sync {dataset_id}"
+            ),
+        }
+
+
 def write(
     args: argparse.Namespace,
     *,
@@ -1242,8 +1299,14 @@ def write(
                 f"{version}; somebody else is editing it. Read it with mr-data dataset show "
                 f"{dataset_id} and write again",
             ) from second
+    # Validate the write receipt before making a second request with another credential.
+    written_id = _dataset_id(updated, verb="write")
+    if written_id != dataset_id:
+        raise ThinLaneError("THIN_DATASET_COORDINATE_MISMATCH", "Studio returned another dataset")
+    projection = _sync_written_identity(dataset_id, selected.session.cloud_url)
     return {
         "schema_version": SET_SCHEMA,
+        **projection,
         "status": "dataset_updated",
         "lane": "hosted",
         "dataset_id": _dataset_id(updated, verb="write"),
@@ -1598,6 +1661,7 @@ def note(
 #: belongs to both; that is one verb short of the whole truth and still points at the pair of
 #: commands the flag is for, which is what somebody who mistyped needs.
 _VERB_FLAGS: Mapping[str, tuple[tuple[str, str], ...]] = {
+    "sync": (),
     "set": (("name", "--name"), ("description_file", "--description-file")),
     "note": (
         ("heading", "--heading"),
@@ -1923,6 +1987,18 @@ def dataset(
         return draft(args, client=client, stdin=stdin)
     if action == "set":
         return write(args, client=client, stdin=stdin)
+    if action == "sync":
+        dataset_id = identifier(args.dataset_id, "dataset identifier")
+        credentials = resolve_cloud_credentials()
+        result = _sync_written_identity(dataset_id, credentials.cloud_url)
+        return {
+            "schema_version": "mostlyright-dataset-identity-reconciled.v1",
+            "status": "dataset_identity_synced"
+            if result["public_projection_synced"]
+            else "dataset_identity_sync_pending",
+            "dataset_id": dataset_id,
+            **result,
+        }
     if action == "note":
         return note(args, client=client, stdin=stdin)
     if action == "activity":
@@ -1930,8 +2006,7 @@ def dataset(
     if action == "watch":
         return watch(args, client=client)
     if action == "publish":
-        # ⚠ THE ONE VERB WHOSE CLIENT IS CLOUD'S. Everything else on this surface speaks Studio's
-        # dataset routes; who may read a dataset is Cloud's decision and Cloud's route, so the
+        # This verb and identity sync use Cloud credentials. Access is Cloud's decision, so the
         # injected client for this verb is the Cloud one and cannot be a Studio session.
         if client is not None and not isinstance(client, CloudDatasetAccessClient):
             raise ThinLaneError(
@@ -1946,6 +2021,11 @@ def dataset(
     )
 
 
+def dataset_exit_code(payload: dict[str, Any]) -> int:
+    """Preserve the saved-write receipt but signal an incomplete public sync."""
+    return 2 if payload.get("public_projection_synced") is False else 0
+
+
 __all__ = [
     "ACCESS_MEMBERS",
     "ACCESS_REFUSALS",
@@ -1973,6 +2053,7 @@ __all__ = [
     "checked_name",
     "create",
     "dataset",
+    "dataset_exit_code",
     "declare_arguments",
     "event_stream",
     "note",