mostlyright-data 0.9.0__tar.gz

mostlyright_data-0.9.0/.gitignore

@@ -0,0 +1,34 @@
+.venv/
+__pycache__/
+.pytest_cache/
+.ruff_cache/
+*.py[cod]
+dist/
+build/
+*.egg-info/
+.env
+.DS_Store
+.work/
+
+# scripts/generate_registry_tables.py stages a rewrite beside the file it is replacing and moves
+# it into place. Nothing should survive that move; if an interruption leaves one behind, it is
+# not a source file somebody should be offered the chance to commit.
+*.generated
+
+# Vendored third-party source is content, not build output. The `build/` rule above is meant for
+# Python build artifacts and would otherwise swallow vendor/grib-rs/build/, which is the crate's
+# own build script and is part of what the recorded tree digest covers.
+!vendor/grib-rs/build/
+!vendor/grib-rs/build/**
+
+# The performance suite builds its fixture inside the repository root rather than in a temporary
+# directory, so an interrupted or parallel run leaves one of these behind. Nothing in one is a
+# source file: it is a generated build folder holding a hundred thousand rows of synthetic data,
+# under directories the build seals read-only, and `git add -A` in that window commits the lot.
+# Recorded as issue #311; ignored here so the window cannot cost anybody a rewritten branch in
+# the meantime.
+.mostlyright-performance-*/
+
+# Cargo build output for the vendored decoder binding. The wheels it produces are committed
+# under vendor/wheels/; the intermediate object tree that produced them is not.
+/vendor/mostlyright-grib/target/

mostlyright_data-0.9.0/PKG-INFO

@@ -0,0 +1,607 @@
+Metadata-Version: 2.4
+Name: mostlyright-data
+Version: 0.9.0
+Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
+Requires-Python: <3.14,>=3.13
+Provides-Extra: build
+Requires-Dist: hatchling==1.28.0; extra == 'build'
+Requires-Dist: uv-build==0.9.26; extra == 'build'
+Provides-Extra: dev
+Requires-Dist: cmarkgfm>=2025.10.22; extra == 'dev'
+Requires-Dist: croniter==6.2.4; extra == 'dev'
+Requires-Dist: cryptography<51,>=50.0.0; extra == 'dev'
+Requires-Dist: et-xmlfile==2.0.0; extra == 'dev'
+Requires-Dist: idna<4,>=3.10; extra == 'dev'
+Requires-Dist: jsonschema[format]<5,>=4.23; extra == 'dev'
+Requires-Dist: numpy==2.5.1; extra == 'dev'
+Requires-Dist: openpyxl==3.1.5; extra == 'dev'
+Requires-Dist: packaging<27,>=24; extra == 'dev'
+Requires-Dist: pandas==3.0.3; extra == 'dev'
+Requires-Dist: polars-runtime-32==1.42.1; extra == 'dev'
+Requires-Dist: polars==1.42.1; extra == 'dev'
+Requires-Dist: pyarrow==23.0.1; extra == 'dev'
+Requires-Dist: pytest-xdist>=3.6; extra == 'dev'
+Requires-Dist: pytest>=8.3; extra == 'dev'
+Requires-Dist: referencing<1,>=0.35; extra == 'dev'
+Requires-Dist: ruff>=0.9; extra == 'dev'
+Provides-Extra: hosted
+Requires-Dist: mostlyright-studio-client-v3==3.0.0; extra == 'hosted'
+Provides-Extra: local
+Requires-Dist: croniter==6.2.4; extra == 'local'
+Requires-Dist: cryptography<51,>=50.0.0; extra == 'local'
+Requires-Dist: et-xmlfile==2.0.0; extra == 'local'
+Requires-Dist: idna<4,>=3.10; extra == 'local'
+Requires-Dist: numpy==2.5.1; extra == 'local'
+Requires-Dist: openpyxl==3.1.5; extra == 'local'
+Requires-Dist: packaging<27,>=24; extra == 'local'
+Requires-Dist: pandas==3.0.3; extra == 'local'
+Requires-Dist: polars-runtime-32==1.42.1; extra == 'local'
+Requires-Dist: polars==1.42.1; extra == 'local'
+Requires-Dist: pyarrow==23.0.1; extra == 'local'
+Provides-Extra: local-search-neural
+Requires-Dist: anyio==4.14.2; extra == 'local-search-neural'
+Requires-Dist: certifi==2026.7.22; extra == 'local-search-neural'
+Requires-Dist: click==8.4.2; extra == 'local-search-neural'
+Requires-Dist: filelock==3.32.2; extra == 'local-search-neural'
+Requires-Dist: flatbuffers==25.12.19; extra == 'local-search-neural'
+Requires-Dist: fsspec==2026.7.0; extra == 'local-search-neural'
+Requires-Dist: h11==0.16.0; extra == 'local-search-neural'
+Requires-Dist: hf-xet==1.6.0; extra == 'local-search-neural'
+Requires-Dist: httpcore==1.0.9; extra == 'local-search-neural'
+Requires-Dist: httpx==0.28.1; extra == 'local-search-neural'
+Requires-Dist: huggingface-hub==1.27.0; extra == 'local-search-neural'
+Requires-Dist: idna==3.18; extra == 'local-search-neural'
+Requires-Dist: numpy==2.5.1; extra == 'local-search-neural'
+Requires-Dist: onnxruntime==1.28.0; extra == 'local-search-neural'
+Requires-Dist: packaging==26.3; extra == 'local-search-neural'
+Requires-Dist: protobuf==7.35.1; extra == 'local-search-neural'
+Requires-Dist: pyyaml==6.0.3; extra == 'local-search-neural'
+Requires-Dist: tokenizers==0.23.1; extra == 'local-search-neural'
+Requires-Dist: tqdm==4.70.0; extra == 'local-search-neural'
+Requires-Dist: typing-extensions==4.16.0; extra == 'local-search-neural'
+Provides-Extra: release
+Requires-Dist: croniter==6.2.4; extra == 'release'
+Requires-Dist: cryptography<51,>=50.0.0; extra == 'release'
+Requires-Dist: et-xmlfile==2.0.0; extra == 'release'
+Requires-Dist: idna<4,>=3.10; extra == 'release'
+Requires-Dist: jsonschema[format]<5,>=4.23; extra == 'release'
+Requires-Dist: numpy==2.5.1; extra == 'release'
+Requires-Dist: openpyxl==3.1.5; extra == 'release'
+Requires-Dist: packaging<27,>=24; extra == 'release'
+Requires-Dist: pandas==3.0.3; extra == 'release'
+Requires-Dist: polars-runtime-32==1.42.1; extra == 'release'
+Requires-Dist: polars==1.42.1; extra == 'release'
+Requires-Dist: pyarrow==23.0.1; extra == 'release'
+Provides-Extra: viewer
+Requires-Dist: nbconvert==7.17.1; extra == 'viewer'
+Requires-Dist: nbformat==5.10.4; extra == 'viewer'
+Description-Content-Type: text/markdown
+
+# mostlyright-data
+
+`mostlyright-data` converts a question and selected sources into a reproducible Parquet
+dataset. An external agent may select sources and author a declarative recipe. A human approves the
+exact recipe digest. Deterministic workers acquire bytes, transform data, run checks, and seal an
+immutable Build.
+
+For question-to-dataset work, the external host agent owns orchestration through delivery of a
+verified Build and its table notebook. Exact proposed-contract or Recipe-digest approval is the
+only user pause: typed refusals and failed checks drive bounded retry, supported substitution, or
+a replacement Recipe. Workers remain fail-closed, changed Recipe digests require new approval,
+and Studio alone owns release. If every registered lawful alternative is unsupported, the agent
+reports that typed impossibility instead of bypassing a refusal or fabricating a dataset.
+
+Source discovery is a host-owned, multi-lane search. `network_search_bundled: false` is a truthful
+capability disclosure: the credential-isolated Harness does not contain or execute a browser. It is
+not a switch that disables the host's public-web tools. For question-to-dataset work, the host must
+search every available lane—registered Harness acquisition adapters, a configured sealed catalog,
+public web search, official provider catalogs or APIs, and user-supplied sources—with no preferred
+lane. It then merges and deduplicates source options, inspects them through registered boundaries, and
+compares them on the canonical `harness-source-fitness.v1` dimensions. Every lane cites an immutable
+host capability receipt, every query cites a host-tool receipt, and every comparison cites source
+or receipt evidence. A host receipt is a strict JSON object whose SHA-256 is calculated over its
+canonical UTF-8 JSON bytes; preserve those bytes in `research.ipynb` or record the exact durable
+host-result reference beside the digest. Those receipts and the report remain untrusted host input: `mr-data
+source-audit` checks their structure and internal bindings, but does not claim a browser ran. It
+still performs no search, fetch, approval, or acquisition itself.
+
+Selected and eligible sources also record the exact promoted inspection coordinate—locator kind,
+credential-free locator, certified Reader identity plus decode-options digest (or `null` for native
+CSV handling), format, and host platform—and its immutable inspection-receipt digest. Typed
+acquisition refusals are not fitness failures. Non-final retries remain rejected; recovery and
+lawful exhaustion are terminal, and an unsupported decision requires every typed-refusal route to
+be lawfully exhausted.
+
+The repository includes a complete report example and the canonical host-receipt objects whose
+digests it cites. Audit the report with:
+
+```bash
+uv run --extra dev mr-data source-audit \
+  --question examples/city-temperature/question.json \
+  --requirements examples/city-temperature/requirements.json \
+  --source-proposals examples/city-temperature/source-proposals.json \
+  --discovery-report examples/city-temperature/source-discovery-report.json \
+  --json
+```
+
+The package version is `0.9.0` and requires Python `>=3.13,<3.14`.
+
+## Two install profiles
+
+`mr-data` is one binary with two install profiles, and the command names are the same in both.
+
+| Profile | Install | Where it runs | What it is |
+| --- | --- | --- | --- |
+| Hosted (the default) | `pip install mostlyright-data` | Any operating system with CPython 3.13 -- Linux, macOS and Windows alike | Pure Python. One `py3-none-any` wheel, no compiled extension in the install closure and, today, no third-party package at all. It signs in, submits work to the Mostly Right backend, streams a run's live progress, downloads the dataset it produced, and shows the receipts. |
+| Local | `pip install "mostlyright-data[local]"` | Linux; installable on macOS, untested there | Everything above plus the engines, the certified Readers, the Clean room, and every build command below. Not a product surface: ADR 0021 makes every lane hosted, and this extra is operator and developer tooling for the machines that build the workers. |
+
+One wheel serves every platform, and that is checked rather than asserted: `thin-client-smoke`
+builds exactly one artifact and installs **that same file** on ubuntu, macOS and Windows before
+running the hosted command surface on each.
+
+Windows gets the hosted profile and not the local one, and the reason is not that nobody has
+ported it: the local lane runs every acquisition and every Reader behind a kernel boundary built
+from POSIX process control -- `fork` and cgroups v2 -- and Windows has neither. Since ADR 0021
+that is also true of macOS: Seatbelt was the second boundary implementation, it existed so a
+person's Mac could run a lane, and it left with the product surface that needed it. Note that
+`pip install "mostlyright-data[local]"` *succeeds* there, because
+the engines all publish Windows wheels, so what you installed is not what decides it. `mr-data`
+asks the machine, and every local command refuses **by name** with that reason rather than
+running into a traceback later. One consequence is recorded rather than hidden: the credential
+`mr-data login` stores is written at mode `0600` on Linux and macOS, and on Windows is protected
+by the ACL your user profile directory already carries, which is a weaker statement.
+
+`mr-data` chooses the lane by what is installed. With `[local]` present every invocation behaves
+exactly as it always has; `--hosted` on `build`, `run`, `watch`, `status` or `deploy-dataset`
+selects the backend instead. Without `[local]`, the hosted lane is the only lane, and a command
+that runs on this machine refuses **by name** and says what to install rather than failing on a
+missing import.
+
+Where a hosted command runs is not a variance a person has to track: a submission prints the
+dashboard address for the run it queued, and `mr-data watch <run-id>` follows that run's event
+log live, resuming from its last cursor when the connection is cut rather than replaying it.
+
+## Supported platforms
+
+| Platform | Installed capability |
+| --- | --- |
+| Anything CPython 3.13 runs on -- Linux, macOS, Windows | The hosted profile, complete: `login`, `whoami`, and every hosted command. This is the product, and it is the row that matters. |
+| Ubuntu 24.04 x86_64, CPython 3.13.14 | Additionally the `[local]` extra: every certified Reader over local files and public HTTPS, including bounded archives, workbooks, and optional GRIB. The Clean room uses a private network namespace and an operator-bounded cgroup root. This is the platform the hosted workers are, which is why it is the one that is tested. |
+| macOS 14 arm64 | The hosted profile, and `[local]` installs. Nothing under `[local]` is tested there any more: ADR 0021 retired `parity-macos`, `sandbox-macos`, and the macOS legs of the `local-*` workflows along with the local lane. Clean-room execution refuses -- `SANDBOX_MEMORY_BOUNDARY` for a Reader, `SANDBOX_OS_BOUNDARY` for anything else -- because the Seatbelt boundary that used to admit it is gone. A logged-in single-address credential-free public HTTPS acquisition still runs in Studio's reviewed Linux/amd64 hosted Clean room. Bounded collections require both local attestations, so they need a host that can produce them. Catalogue authoring, which is local by design, still works. |
+| Windows | The hosted profile. `pip install "mostlyright-data[local]"` succeeds, because the engines all publish Windows wheels, so what you installed is not what decides it: every local command refuses **by name** and says why. |
+
+The GRIB Reader is optional. If a compatible `mostlyright-grib` binding is unavailable, first use
+fails with `READER_DEPENDENCY_UNAVAILABLE`. It accepts one extracted GRIB2 record and rejects a
+multi-record GFS or HRRR file.
+
+## Install from a checkout
+
+Update the persistent uv tool and install the complete matching skill for both Codex and Claude
+Code with one command from the checkout:
+
+```bash
+uv run --no-project --python 3.13 python scripts/install_checkout.py
+```
+
+The installer resolves the checkout from its own path, reinstalls `mr-data` from that exact
+directory and installs the exact pinned generated-client wheel alongside it, copies the complete `skills/mr-data-build` tree to both
+personal skill directories, and then verifies `mr-data --version`, exact source-to-installed
+package bytes, and exact source-to-installed skill-tree parity. The separately owned generated
+Studio client is also exposed as the published distribution's `hosted` extra. The base Harness wheel remains client-free for
+credential-free Courier images. It also binds every installed launcher to the
+`[project.scripts]` entry point it declares.
+
+`PATH` is checked last, after the verified generation is committed, because `PATH` belongs to your
+shell rather than to the installation and an ambient shell problem must never restore the code the
+install just replaced. If the uv tool bin directory is not on `PATH` at all, which is the ordinary
+first-install case, the installer keeps the install, exits 0, and tells you to run
+`uv tool update-shell` and open a new shell. If a different `mr-data` is found earlier on `PATH`,
+the installer keeps the install and exits 2 with a warning naming both paths: typing `mr-data`
+would otherwise still run the stale program. Put the uv tool bin directory earlier on `PATH`, or
+remove the other one.
+
+So read the exit code rather than testing it for zero: `0` and `2` both mean the new generation is
+installed and verified, and only `1` means the install was refused and the prior generation
+restored. Rerunning on `2` produces the same `2` until `PATH` changes.
+
+It preflights both skill targets before changing the uv tool. If an installed skill differs, it
+refuses with no mutation; inspect the named path and rerun with `--force` only when replacement is
+intended. The installer retains private backups of the exact prior uv tool environment, all
+Harness launchers, and both prior skill trees until final parity succeeds. Any failure restores
+that whole prior generation, or removes every new artifact when no prior generation existed. A
+single per-user lock serializes this entire operation with direct skill installs. Once exact final
+parity is verified, the new generation is successful; failed best-effort backup cleanup emits a
+warning and leaves the named private backup for manual recovery instead of undoing that generation.
+
+For an isolated wheel smoke test instead of a persistent global tool update:
+
+```bash
+artifact_dir="$(mktemp -d)"
+uv build --wheel --out-dir "$artifact_dir"
+set -- "$artifact_dir"/mostlyright_data-*.whl
+test "$#" -eq 1 && test -f "$1"
+wheel="$1"
+uv venv .work/harness-install --python 3.13
+uv pip install --python .work/harness-install/bin/python "${wheel}[local]"
+.work/harness-install/bin/mr-data --version
+.work/harness-install/bin/mr-data inventory --json
+```
+
+### Run ordinary CSV with the installed binary
+
+```bash
+mkdir -p .work
+.work/harness-install/bin/mr-data init .work/installed-city-temperature \
+  --run-id installed-city-temperature \
+  --input-root examples/city-temperature \
+  --question examples/city-temperature/question.json \
+  --requirements examples/city-temperature/requirements.json \
+  --source-proposals examples/city-temperature/source-proposals.json \
+  --plan examples/city-temperature/question-plan.json \
+  --semantics examples/city-temperature/semantics.json \
+  --json
+.work/harness-install/bin/mr-data plan .work/installed-city-temperature --json
+.work/harness-install/bin/mr-data run .work/installed-city-temperature --json
+.work/harness-install/bin/mr-data verify .work/installed-city-temperature --json
+```
+
+The workspace must not exist before `init`. `run` seals the Build under `<workspace>/result` and
+writes `table.ipynb` beside it.
+
+Create a research directory, then start the local notebook viewer in a second terminal before
+`init`. Keep the future workspace absent:
+
+```bash
+mkdir -p .work/installed-city-temperature-research
+.work/harness-install/bin/mr-data view \
+  --workspace .work/installed-city-temperature \
+  --research-dir .work/installed-city-temperature-research \
+  --json
+```
+
+The receipt names `url`, `research_notebook`, and `table_notebook`. Codex and Claude Code agents
+always open `url` in their in-app browser and never pass `--open`; a host that is genuinely
+terminal-only and is not Codex or Claude Code may use `--open`. For a direct Build, use
+`mr-data view --run-dir PATH --research-dir RESEARCH_DIR`. `view` stays on loopback and starts
+before source selection or Recipe approval. It streams atomic `research.ipynb` updates and adds
+`table.ipynb` when the Build finishes. Keep an in-app viewer tab across approval turns. If a
+successful Build reports an unavailable sidecar, run `mr-data notebook RUN_DIR --json`; the viewer
+writes `table.ipynb` only when there is none and never replaces one that is there.
+`mr-data watch` remains an opt-in operational diagnostic.
+
+### Test the packaged Reader boundary
+
+On Ubuntu 24.04 with cgroup v2, this command runs the installed Reader in the required memory and
+network boundaries:
+
+```bash
+uv sync --locked --extra dev
+sudo sh -eu -c '
+  echo "+memory +pids" > /sys/fs/cgroup/cgroup.subtree_control
+  mkdir /sys/fs/cgroup/mostlyright-stock-reader-test
+  echo 536870912 > /sys/fs/cgroup/mostlyright-stock-reader-test/memory.max
+  echo 0 > /sys/fs/cgroup/mostlyright-stock-reader-test/memory.swap.max
+  echo 64 > /sys/fs/cgroup/mostlyright-stock-reader-test/pids.max
+  echo 1 > /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.max.depth
+  echo 2 > /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.max.descendants
+  echo "+memory +pids" > \
+    /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.subtree_control
+  mkdir /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
+'
+sudo chown "$(id -u):$(id -g)" \
+  /sys/fs/cgroup/mostlyright-stock-reader-test \
+  /sys/fs/cgroup/mostlyright-stock-reader-test/cgroup.procs
+test ! -w /sys/fs/cgroup/mostlyright-stock-reader-test/memory.max
+test ! -w /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator/cgroup.procs
+test ! -w /sys/fs/cgroup/cgroup.procs
+runner_uid="$(id -u)"
+runner_gid="$(id -g)"
+sudo env \
+  GITHUB_WORKSPACE="$PWD" \
+  RUNNER_UID="$runner_uid" \
+  RUNNER_GID="$runner_gid" \
+  bash <<'STOCK_READER'
+set -eu
+cd "${GITHUB_WORKSPACE}"
+echo "$$" > /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator/cgroup.procs
+exec 3</sys/fs/cgroup/mostlyright-stock-reader-test
+exec 4</proc/self/ns/net
+exec unshare --net -- setpriv \
+  --reuid="${RUNNER_UID}" \
+  --regid="${RUNNER_GID}" \
+  --init-groups \
+  .venv/bin/python -m pytest -q tests/test_stock_reader_workflow.py
+STOCK_READER
+find /sys/fs/cgroup/mostlyright-stock-reader-test -mindepth 1 -maxdepth 1 \
+  -type d -name 'mostlyright-sandbox-*' -exec sudo rmdir {} +
+test -z "$(find /sys/fs/cgroup/mostlyright-stock-reader-test -mindepth 1 -maxdepth 1 \
+  -type d -name 'mostlyright-sandbox-*' -print -quit)"
+sudo rmdir /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
+test ! -e /sys/fs/cgroup/mostlyright-stock-reader-test/coordinator
+sudo rmdir /sys/fs/cgroup/mostlyright-stock-reader-test
+test ! -e /sys/fs/cgroup/mostlyright-stock-reader-test
+```
+
+For development, run the same workflow from the checkout:
+
+```bash
+uv sync --locked --extra dev
+mkdir -p .work
+uv run --extra dev mr-data init .work/city-temperature \
+  --run-id golden-city-temperature \
+  --input-root examples/city-temperature \
+  --question examples/city-temperature/question.json \
+  --requirements examples/city-temperature/requirements.json \
+  --source-proposals examples/city-temperature/source-proposals.json \
+  --plan examples/city-temperature/question-plan.json \
+  --semantics examples/city-temperature/semantics.json
+uv run --extra dev mr-data plan .work/city-temperature
+uv run --extra dev mr-data run .work/city-temperature
+uv run --extra dev mr-data verify .work/city-temperature
+```
+
+Add `--json` for one JSON object on success or failure. Failures exit nonzero. The command result is
+the receipt; paths named by a receipt remain mutable after the command returns.
+
+`--input-root` and `--output` may be relative paths. Relative paths are resolved from the current
+working directory.
+
+## Commands
+
+| Command | Behavior |
+| --- | --- |
+| `mr-data catalog-search` | Search a sealed source catalog and return one entry with its query trace. |
+| `mr-data catalog-fill` | Stage and publish one bounded credential-free CKAN, STAC, or SDMX catalog. |
+| `mr-data catalog-harvest` | Stage a resumable Data.gov provider sweep, or verify its key boundary without making a request. |
+| `mr-data catalog-author` | Convert one completed private harvest into bounded immutable authoring shards. |
+| `mr-data catalog-publish` | Validate harvest and authoring receipts, then publish and independently read back one packed catalog generation. |
+| `mr-data catalog-update` | Install one published catalog release after checking its exact archive digest, its store digest, and its descent from the generation already installed. |
+| `mr-data index` | Rebuild the private local Build index under explicit discovery and byte limits. |
+| `mr-data search` | Search exact public-catalog and private-Build snapshots under caller-supplied retrieval limits. |
+| `mr-data source-audit` | Validate the host's multi-lane discovery report, source comparisons, proposals, and citations without searching or fetching. |
+| `mr-data peek` | Report columns, types, and sample rows for a local file or public HTTPS resource, including an exact certified Reader selected with `--reader FAMILY@VERSION --reader-options JSON`. |
+| `mr-data record` | Seal messages already recorded from a live source into bounded digest-named batches a build can read, writing down each dropped connection. It opens no connection and reaches no network. |
+| `mr-data author` | Convert YAML into a Recipe. |
+| `mr-data approve` | Bind approval to an exact Recipe digest. |
+| `mr-data plan-check` | Execute a dataset plan in a temporary directory and report the result. |
+| `mr-data show` | Inspect one Build. |
+| `mr-data list` | List Builds under a root. |
+| `mr-data diff` | Compare two Builds. |
+| `mr-data review --local` | Run local checks without granting approval; exit 1 when a check fails. |
+| `mr-data deploy-plan` | Validate a reviewed Build and write a deployment request without deploying it. |
+| `mr-data deploy RUN_DIR` | Stage an exact verified Build, record and confirm its internal approval as the authenticated Editor, then activate and bind the Cloud Dataset. |
+| `mr-data deploy-status RUN_DIR` | Report the state Studio holds for the run that deployment queued, optionally waiting for it with `--watch-seconds N`. It reads; it changes nothing Studio holds. |
+| `mr-data deploy-dataset RUN_DIR` | Bring the Build that deployment's run released back into `RUN_DIR/hosted`, checked against the fingerprints the release sealed and checked again here, and write its dataset notebook. |
+| `mr-data review-setup` | Generate review enrollment, keys, role tokens, and anchor input. |
+| `mr-data notebook RUN_DIR` | Create or refresh `table.ipynb` for an existing Harness Build. |
+| `mr-data view --workspace PATH` / `--run-dir PATH` | Serve the local notebook viewer, optionally pairing it with an explicit `--research-dir`. |
+| `mr-data watch` | Serve the read-only event view. |
+| `mr-data workflow` | Print the supported workflow stages and their authority boundaries. |
+| `mr-data preflight` | Report build blockers for the host, for named options, or for a named workload measured against the engine's own ceilings. |
+| `mr-data inventory` | Report installed capabilities. |
+| `mr-data auth` | Validate the effective device credential with Cloud; manage metadata-only device revocation, remote-first logout, safe rotation, and explicit recovery; explain the ephemeral token boundary. |
+| `mr-data login` | Sign in to the Mostly Right cloud with a device code and store a device credential. `login --force` is the compatibility alias for safe `auth rotate`, never an in-place truncate. |
+| `mr-data whoami` | Compatibility alias for the remotely validated `mr-data auth status` answer. |
+
+### The hosted lane
+
+Twenty-five commands run on the Mostly Right backend. On a `[local]` install they take `--hosted`
+to select that lane; on the hosted-only install they are what the command already does.
+[Hosted parity](docs/HOSTED-PARITY.md) is the full ledger, generated from the classification the
+software itself branches on; the table below is the submit-and-follow half of it, plus the
+commands that write something other than a run.
+
+| Command | Behavior |
+| --- | --- |
+| `mr-data build --hosted` / `mr-data run --hosted` | Queue one hosted initial build from an approved recipe proposal (`--recipe-proposal`, `--recipe-digest`, optionally `--resource-class standard\|large`) and print the dashboard address to watch it at. |
+| `mr-data watch RUN_ID --hosted` | Stream that run's event log live, rendering the facts the worker measured. It resumes from its last cursor across a stream cut instead of replaying, and stops when the run reaches a state that appends nothing further. |
+| `mr-data status RUN_ID --hosted` | Report the state Studio holds for one run, and with `--receipts` its execution summary and the sealed evidence for the Build it produced. |
+| `mr-data deploy-dataset RUN_ID --hosted --output DIR` | Fetch that run's dataset through a short-lived signed URL, refuse it unless the bytes hash to the digest the release sealed, and write it once into `DIR`. |
+| `mr-data acquire --hosted` | Fetch one public address in Studio's clean room and print the Receipt Studio bound, naming under `reader_binding` which party resolved the pinned Reader. A source that refused answers `status: "refused"` with its own sealed code and exits 1. |
+| `mr-data acquire-cancel --workspace-id WORKSPACE_ID --crawler-session-id CRAWLER_SESSION_ID --hosted` | Cancel one still-active public acquisition using the coordinate `acquire` printed. By default it writes a private, per-coordinate receipt journal under the local mr-data state directory; a transport-loss retry reuses its idempotency key, and success prints Studio's exact cancellation receipt. Use `--journal FILE` only to override that location. |
+| `mr-data approve --hosted` / `mr-data recipe-approve --hosted` | Record the decision you are asking for against one approval Studio holds, print the address where the person who must grant it does so, then wait and read the decision receipt back. The command never decides, and exits 2 unless a human recorded the decision that was asked for. |
+| `mr-data note --run RUN_ID --hosted` | Append one cell of the operator-facing decision record to that run's own log, where the rendered notebook and the live view both read it: `--heading` is its one outcome-led line, the body arrives on standard input or from `--markdown-file`, and `--checkpoint` places it against the run-event it explains. Writing the same `--cell-id` again revises the cell in place. A cell is unsealed prose and is never attestation evidence. `--list` reads them all back. |
+| `mr-data research-open --dataset DATASET_ID --question QUESTION_ID --source SOURCE_ID` | Open one research session and print each state as its worker warms up: `opening`, `warming`, `ready`. A workspace already holding its bound of live sessions is refused with the retry window Studio stated, never waited through. |
+| `mr-data research-probe SESSION_ID --kind KIND --request JSON` | Ask the session's warm worker one question about a source, checked against the probe vocabulary before it is sent, follow the session run's stream until the probe settles, and print the answer read from the probe record. Exits 0 only for an answered probe; a probe answer is exploration and never evidence. |
+| `mr-data research-status (SESSION_ID \| --run RUN_ID) [--probe PROBE_ID]` | Report Studio's own reading of one session, and of one probe with its answer. `--run` resolves the owning session directly in Studio; it never scans sessions client-side. |
+| `mr-data research-close (SESSION_ID \| --run RUN_ID)` | Close the session, cancelling any probe still queued, so its worker exits. `--run` asks Studio to resolve and close the owning session directly. |
+
+## Current local workflow
+
+The supported local workflow is:
+
+```text
+question and output grain
+  -> selected sources and rights evidence
+  -> declarative dataset plan
+  -> confined source snapshots
+  -> deterministic transform and validation
+  -> immutable Parquet Build
+  -> two assigned reviews
+  -> Checker-signed decision
+  -> reviewed_candidate | fixes_required
+  -> Studio source staging and immutable Recipe proposal
+  -> authenticated Editor confirmation of the internal approval record
+  -> Dataset activation Run and Cloud Current binding
+```
+
+For the controlled acceptance walkthrough that starts from a changing synthetic public CSV and,
+after the hosted rollout is separately marked ready, ends at Cloud Current after a scheduled
+Dataset Version, see [Local Build to Cloud refresh demo](docs/DEMO-REAL-REFRESH.md). Local authoring
+does not require Docker; governed Linux execution remains in Studio.
+
+The local engine supports string trimming, exact empty-string-to-null conversion, collision-safe
+rename, closed-type casts, stable left equality joins, output selection, and deterministic quality
+checks. `local-plan.v2` runs a wider closed set of ordered operations over the same shared
+arithmetic: it adds exact decimal division, wide-to-long reshaping of columns the plan names, inner
+and anti joins, and bounded text normalization for keys that will not otherwise match. Every one of
+those is admitted by a written test that separates cleaning from modeling choices, and the same
+test is what refuses a general window, a filled-in value, and the reverse of the reshape. See
+[Transforms a plan may perform](docs/TRANSFORMS.md).
+
+`local-plan.v2` also carries one retired operation, `prediction_label@1.0.0`. It named a value at a
+future date, and the horizon it required is a modeling choice rather than a property of the
+sources, so no newly authored plan may name it: authoring, plan checking, validation, export and
+approval all refuse one that does. It remains readable so that a Build sealed with it still
+verifies and a Recipe approved with it still runs. See
+[Prediction labels](docs/PREDICTION-LABELS.md).
+
+The installed stock source-to-Reader path supports certified Readers for local files,
+credential-free public HTTPS sources, and bounded ordered public HTTPS collections. Each HTTPS
+retrieval, Reader decode, and parse shares one attested clean-room operation; JSON and NDJSON
+projection, ZIP, TAR, GZIP, XLSX, delimited text, and single-record GRIB2 are available under their
+exact Reader settings and budgets. A collection can combine 1–16 explicit objects or a fixed
+page-number range when every member uses the same Reader and normalized schema; see
+[Deterministic source collections](docs/SOURCE-COLLECTIONS.md).
+
+The review runtime handles initial Builds only. It records whether the Build passed review or needs
+fixes, rejects nonzero failed-cycle counts, and does not verify repaired successors or derive a
+human-direction status.
+
+The package also contains a pure canonical empirical source-observation chain and deterministic
+cadence inference with shared cross-system vectors. Both governed paths record that evidence
+through one emitter: a hosted probe submits it to Studio, and `mr-data acquire --cadence-history`
+records a local probe in an append-only folder. For one acquisition outcome the two produce the
+same document byte for byte. It does not yet schedule probes or claim that a manually selected cron
+was learned; Studio owns persistence and adaptive scheduling.
+
+The weather Reader opens one extracted GRIB2 record. It refuses a full GFS or HRRR file containing
+a collection of hundreds of records.
+
+## Recipes
+
+`frozen-recipe.v1` binds the question, requirements, selected source evidence, connector versions,
+query templates, credential-reference names, transformation plan, validation policy, Bookmark
+strategy, and engine versions. Its digest is lowercase SHA-256.
+
+The checked-in example contains `recipe.json` and `recipe-execution.json`:
+
+```bash
+uv sync --locked --extra dev
+mkdir -p .work/recipe
+uv run --extra dev mr-data recipe-export \
+  --recipe examples/city-temperature/recipe.json \
+  --output .work/recipe/frozen-recipe.json
+uv run --extra dev mr-data recipe-approve \
+  --recipe .work/recipe/frozen-recipe.json \
+  --approval-id approval-city-temperature-v1 \
+  --approved-by local-operator \
+  --approved-at 2026-07-19T12:00:00Z \
+  --approval-policy-digest 0000000000000000000000000000000000000000000000000000000000000000 \
+  --output .work/recipe/recipe-approval.json
+uv run --extra dev mr-data recipe-run \
+  --recipe .work/recipe/frozen-recipe.json \
+  --approval .work/recipe/recipe-approval.json \
+  --execution examples/city-temperature/recipe-execution.json \
+  --input-root examples/city-temperature \
+  --output .work/recipe/version-1 \
+  --producer-principal local-recipe-producer \
+  --producer-attempt initial-v1
+uv run --extra dev mr-data recipe-verify .work/recipe/version-1
+```
+
+The output parent must exist. Export, approval, and run commands do not overwrite existing output.
+Refresh and backfill executions require `--previous-run-dir` and the exact predecessor Build.
+
+## Authority and security
+
+- External agents may interpret questions, propose sources, author declarative inputs, and report
+  findings. They do not calculate authoritative metrics or approve releases.
+- The deterministic worker owns confinement, budgets, snapshots, transforms, checks, lineage,
+  hashes, and artifact sealing.
+- The trusted coordinator owns roots, assignments, attempt identities, enrolled signing keys, and
+  immutable cycle lineage.
+- The Harness owns deterministic public-catalog provider semantics, including Data.gov endpoint,
+  pagination, catalog identity, revisions, normalization, and provenance. Catalog discovery never
+  authorizes or acquires the underlying dataset resource.
+- Studio owns hosted contracts, orchestration, persistence, scheduling, credentials, approval, and
+  release.
+
+Source bytes and dataset plans are untrusted. The local-file worker accepts normalized relative
+paths to regular, single-link CSV files below an approved input root. It rejects traversal,
+absolute paths, symlinks, hardlinks, special files, input/output overlap, source mutation, and
+budget violations.
+
+Builds are content-addressed and non-overwritable. Canonical payloads exclude ambient clocks and
+absolute filesystem paths. Model output and source content cannot grant human approval.
+
+The package does not provide local publication authority, local release authority, modeling,
+general feature engineering, prediction serving, authenticated source access, scheduling, or
+arbitrary code execution from an agent response. The one narrow exception to general feature
+engineering is deterministic, terminal-only prediction-label construction described above.
+
+## Hosted workers
+
+`mr-data export-hosted-candidate` verifies a local Build and writes the canonical hosted
+Builder-input envelope:
+
+```bash
+mr-data export-hosted-candidate /absolute/path/to/run \
+  --output /absolute/path/to/new-candidate-input.json \
+  --json
+```
+
+The hosted executables are:
+
+- `mr-data-bootstrap`: exchanges workload identity for one Studio job and starts the fixed role.
+- `mr-data-producer`: reconstructs and verifies a Build, then uploads reservation-bound artifacts.
+- `mr-data-verifier`: independently verifies artifacts and submits a signed report.
+- `mr-data-crawler-job`: performs allowlisted public acquisition and passes bounded bytes to a
+  networkless Clean room.
+- `mr-data-ingestion-job`: redeems one opaque workload-identity-bound ingestion dispatch and runs
+  only a reviewed bounded delivery processor; see [Hosted ingestion delivery](docs/INGESTION-DELIVERY.md).
+
+These commands cannot select or release a Build, create a dataset version, advance `latest`, or
+access Studio databases, queues, secrets, or storage metadata directly.
+
+## Verification
+
+For ordinary edit-loop feedback, run `make test-fast`. This is a non-authoritative test tier that
+omits semantically expensive scale, isolated-install, and live-repository-scan coverage;
+`make verify` is required before review or merge.
+
+```bash
+uv sync --extra dev
+make verify-offline-slice
+make verify-hosted-workers
+make verify-viewer
+make verify-viewer-browser
+make verify
+```
+
+`verify-hosted-workers` requires a sibling `mostlyright-studio` checkout. Set
+`STUDIO_ROOT=/absolute/path` when it is elsewhere.
+
+`make verify-thin-client` proves the hosted profile the way a stranger meets it: it builds the
+base wheel, installs it into a fresh environment from an empty cache, refuses any compiled
+extension it finds there, and runs the hosted command surface. The same body runs in CI as the
+reusable `thin-client-smoke` workflow, fanned out over GitHub-hosted ubuntu, macOS and Windows:
+one `py3-none-any` wheel is built once and installed by all three legs, on every pull request and
+nightly.
+
+There is no documented known-failing set; every failing node is a verification failure.
+
+## Documentation
+
+- [Architecture](docs/ARCHITECTURE.md)
+- [Offline Vertical Slice](docs/OFFLINE-VERTICAL-SLICE.md)
+- [Recipes](docs/FROZEN-RECIPES.md)
+- [Sources and Acquisition](docs/SOURCES-AND-ACQUISITION.md)
+- [Empirical source cadence](docs/SOURCE-CADENCE.md)
+- [Readers](docs/READERS.md)
+- [Deterministic source collections](docs/SOURCE-COLLECTIONS.md)
+- [Notebook View](docs/NOTEBOOK-VIEW.md)
+- [Deployment request](docs/DEPLOYMENT.md)
+- [Local Build to Cloud refresh demo](docs/DEMO-REAL-REFRESH.md)
+- [Hosted parity](docs/HOSTED-PARITY.md)
+- [Catalogue release channel](docs/CATALOG-CHANNEL.md)
+- [Hosted workers](docs/HOSTED-WORKERS.md)
+- [Hosted worker publication](docs/HOSTED-WORKER-PUBLICATION.md)
+- [Research-session worker](docs/SESSION-WORKER.md)
+- [Transforms a plan may perform](docs/TRANSFORMS.md)
+- [Review policy](docs/REVIEW-LOOP.md)
+- [Contract ownership](docs/CONTRACTS.md)
+- [Engine governance](docs/ENGINE-GOVERNANCE.md)
+
+`.planning/` contains repository work records. Its contents are not product contracts, merge
+authorization, approval, or release evidence.