mostlyright-data 0.10.3__tar.gz → 0.10.4__tar.gz

{mostlyright_data-0.10.3 → mostlyright_data-0.10.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.10.3
+Version: 0.10.4
 Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/

{mostlyright_data-0.10.3 → mostlyright_data-0.10.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.10.3"
+version = "0.10.4"
 description = "Mostly Right CLI for reproducible, reviewed Parquet datasets"
 readme = "README.md"
 requires-python = ">=3.13,<3.14"

{mostlyright_data-0.10.3 → mostlyright_data-0.10.4}/skills/mr-data-build/SKILL.md

@@ -85,7 +85,7 @@ absence as a defect.
 
 ## Commands
 
-These twenty-three are the whole surface, alongside `acquire-slices`, which names a capability
+These twenty-four are the whole surface, alongside `acquire-slices`, which names a capability
 gap rather than doing anything, and `export-hosted-candidate`, which is a backend operator step.
 
 | Command | What it does |
@@ -93,6 +93,7 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data auth` | Validate the effective device credential with Cloud; manage metadata-only device revocation, remote-first logout, safe rotation, and explicit recovery; explain the ephemeral token boundary. |
 | `mr-data login` | Complete device approval and store a device credential. `login --force` is the compatibility alias for safe `auth rotate`, never an in-place truncate. |
 | `mr-data whoami` | Use the compatibility alias for the remotely validated `auth status` answer. |
+| `mr-data clarify` | Say whether one clarifying question may be put to the user right now, and compose the record of their answer. It reaches nothing, asks nobody and waits for nothing. Exits non-zero when no person can answer or when a recipe is already registered. |
 | `mr-data probe` | Ask one registered source one question and print the answer. **Currently unusable:** it reaches `/v3/sessions`, which Studio refuses, and it takes a question identifier that no v4 receipt returns. Do not plan a source inspection around it. |
 | `mr-data recipe` | Register one recipe document — dataset, question, table plan, sources, transform, checks and units — in one call, and print the identifiers the server derived. `mr-data recipe show ID` reads one back. |
 | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
@@ -114,10 +115,11 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
 | `mr-data unpin` | Ask Studio to resume pointer tracking; read the returned state before claiming it resumed. |
 | `mr-data demote` | Ask Studio to withdraw the pointer and any schedule it owns. |
 
-`auth`, `login` and `whoami` are the same implementation in both profiles. The other twenty
-answer from Studio. Eight only read: `status`, `runs`, `watch`, `peek`, `receipt`, `checks`,
-`download` and `diff`. Twelve start or change something: `recipe`, `cover`, `run`, `query`,
-`probe`, `keys`, `open`, `note`, `promote`, `pin`, `unpin` and `demote`.
+`auth`, `login`, `whoami` and `clarify` are the same implementation in both profiles, and
+`clarify` alone reaches nothing at all. The other twenty answer from Studio. Eight only
+read: `status`, `runs`, `watch`, `peek`, `receipt`, `checks`, `download` and `diff`. Twelve
+start or change something: `recipe`, `cover`, `run`, `query`, `probe`, `keys`, `open`,
+`note`, `promote`, `pin`, `unpin` and `demote`.
 
 **Any successful run is readable straight away.** There is no approval, no admission and no release
 step between a run finishing and reading it: `download`, `receipt`, `checks`, `peek` and `query`
@@ -226,6 +228,12 @@ again, and the failure triples on recent runs tell you what the backend is refus
 `SANDBOX_MEMORY_BOUNDARY` on somebody else's recipe an hour ago means the room is unwell and your
 first failure will not be your document's fault. Keep this check silent.
 
+**Then settle anything only the user can settle, before probing.** Read the request for a word with
+two defensible readings, a population whose edges are unstated, or a window nobody named.
+`mr-data clarify --question TEXT` says whether there is anybody to ask and whether it is still early
+enough; [Autonomous delivery](#autonomous-delivery) is the rule for what may be asked and what stays
+yours to decide.
+
 Pass `--json` to automated commands. A command receipt is the only record that a Harness action
 succeeded.
 
@@ -316,6 +324,50 @@ Every grain, key, join and cutoff settled here is a `decision` block — see
 [Narrating the run](#narrating-the-run). Append them to the same blocks file the source blocks
 went into. There is still no run to write them into; stage 3 writes the file.
 
+**The dataset description is the documentation, and it is read by people and by search engines.**
+`dataset.description` is Markdown, rendered on the dataset page with headings and lists, and it
+admits up to 4000 characters. Write it as a short reference entry a stranger can scan in thirty
+seconds -- not a paragraph, and not a report. Use exactly this shape, in this order, and leave a
+section out only when there is genuinely nothing to say (shown indented here; the document
+itself carries the headings at column one):
+
+    One or two plain sentences: what each row is, where it comes from, and the period it covers.
+
+    ## Coverage
+    - **Window:** the exact dates and the time zone.
+    - **Grain:** one row per what.
+    - **Cadence:** how often rows arrive, and how many per day or month.
+
+    ## Columns
+    - `column_name` — what it measures, in which unit.
+    - (one bullet per column; group unit-sharing columns on one line where that reads better)
+
+    ## Missing values
+    One or two sentences: how absent, trace and sentinel values are represented.
+
+    ## Suitable for
+    - Two to four concrete uses, each a short phrase.
+
+    ## Source and rights
+    The publisher, the programme, and the rights basis, in one or two sentences.
+
+Rules that keep it readable: short sentences; no sentence over about twenty-five words; bullets
+rather than comma lists; no tables, no code blocks, no links to internal tools, no headings beyond
+the ones above. Spell out the names a reader would search for -- the place, the identifiers in
+every common form, the programme and the publisher -- once each, in the opening sentences and
+never as a keyword list. The opening sentences double as the summary cards and search engines
+show, so they must stand alone. A one-line description is a defect to fix in the revision, not a
+style choice. Give `table.description` the grain and the window, and give every column a
+description a reader can act on.
+
+**Every registered dataset gets a cover before its first run is presented.** After
+`mr-data recipe` answers with the `dataset_id`, generate one with
+`mr-data cover DATASET_ID "what the image should depict" --json`, describing the subject in plain
+language — the place, the instrument, the phenomenon — and nothing about style, colour or size,
+which Studio fixes. Do it once per dataset, silently, and do not repeat it for a revision unless the
+subject changed. A dataset presented without a cover is a defect, the same as a one-line
+description.
+
 ### What the statement must actually return
 
 Every acquired source is bound as all-text CSV — `read_csv(header = true, auto_detect = false,
@@ -769,7 +821,29 @@ not ask the user to choose sources, Readers, engines, formats, workspaces, retry
 operations, or validation repairs. Make those operational decisions from the question, observed
 evidence, registered capabilities, and deterministic receipts.
 
-On a hosted install that outcome is concrete: a run in `succeeded`, `mr-data checks` reporting
+One clarifying question is not a pause and does not weaken that ban. It is allowed for a fact the
+user holds and no source can supply: what an ambiguous word in their request means, how wide the
+population is, or how far back they need it. `Throughput: movements or passengers?` is such a fact.
+Which Reader parses the feed is not, and neither is anything else named above; those stay banned
+because the evidence settles them. Ask before probing, while nothing is registered, nothing is
+running and nothing has been spent: a question at that point costs a sentence, and the guess it
+replaces costs a whole build. Ask ONE question, carrying the answer you will assume if none comes
+back. One, not two: `mr-data clarify` records exactly one question per invocation and the block it
+composes has no field relating a question to another, so a second question is a second record with
+nothing tying it to the first. If two things are genuinely unsettled, the one that survives the
+evidence is the one to ask about.
+
+`mr-data clarify --question TEXT` says whether to ask at all. It exits non-zero when no person can
+answer, which is what a scheduled or headless session reports, and when `--recipe RECIPE_ID` names
+a registered recipe, because the answer to a question asked after the contract is a revision and
+goes through the contract approval instead. On a non-zero exit ask nothing: decide from the
+evidence, write the assumption into the recipe document, and state it as a limitation in the final
+report. `mr-data clarify --question TEXT --answer TEXT` composes the exchange as one `clarification`
+block. Carry its wording into the recipe document the answer settles — the question text, the table
+description, the column description — so the assumption the table rests on reads the same in the
+document, in the Dataset, and in the report.
+
+On a hosted install the delivered outcome is concrete: a run in `succeeded`, `mr-data checks` reporting
 every declared check, the coverage read off the run rather than guessed, and the artifacts brought
 here by `mr-data download` when the user asked for the bytes. Link to the canonical Dataset page and
 state what is live and what is not, rather than claiming a promotion or reporting internal run

{mostlyright_data-0.10.3 → mostlyright_data-0.10.4}/src/mostlyright/data_harness/cli.py

@@ -574,12 +574,18 @@ def _dispatch(
         selected_command == "note"
         or selected_command == "propose"
         or selected_command == "acquire-cancel"
+        or selected_command == "clarify"
         or selected_command in _RESEARCH_COMMANDS
     ):
         # `note`, `propose` and the research commands are the hosted lane's own implementation, run
         # here unchanged, so they refuse in the hosted lane's own typed vocabulary. Handling it here
         # is what turns `THIN_...` into the same code-plus-remediation block every other refusal in
         # this command line prints.
+        #
+        # `clarify` is here for the same reason and is not a hosted command at all: it reaches
+        # nothing and is identical in both profiles. It refuses in the thin lane's typed vocabulary
+        # because that is the vocabulary the agent driving it already branches on, and a fifth
+        # refusal class for four sentences would be a fifth remediation family for no reader.
         from mostlyright.data_harness.thin.transport import ThinLaneError
 
         handled_errors += (ThinLaneError,)

{mostlyright_data-0.10.3 → mostlyright_data-0.10.4}/src/mostlyright/data_harness/thin/router.py

@@ -91,7 +91,12 @@ REFUSAL_CODE = {
 #: The portable commands, and the ``ux.commands`` module each resolves to. Both are stdlib-only
 #: today, so the thin profile runs the SAME implementation the local profile runs rather than a
 #: second one written to look like it.
-PORTABLE_MODULES = {"auth": "auth", "login": "login", "whoami": "whoami"}
+PORTABLE_MODULES = {
+    "auth": "auth",
+    "clarify": "clarify",
+    "login": "login",
+    "whoami": "whoami",
+}
 
 #: One line per hosted command, for the help the thin profile prints. Short on purpose: the local
 #: profile's ``--help`` is ``cli``'s and is not restated here, so these cannot drift out of sync

{mostlyright_data-0.10.3 → mostlyright_data-0.10.4}/src/mostlyright/data_harness/thin/vocabulary.py

@@ -112,6 +112,11 @@ CLASSIFICATION: dict[str, Lane] = {
         "it reports how each check one run's recipe declared came out -- the name, whether it "
         "passed and what it found -- and exits non-zero when one of them did not",
     ),
+    "clarify": Lane(
+        PORTABLE,
+        "it reads whether a person is here to answer one question and turns their answer into "
+        "the block it is recorded as; it reaches nothing and asks nobody",
+    ),
     "cover": Lane(
         HOSTED,
         "it generates and attaches one branded dataset cover through Studio",

mostlyright_data-0.10.4/src/mostlyright/data_harness/ux/attendance.py

@@ -0,0 +1,206 @@
+"""Whether a person is available to answer a question right now, decided in one place.
+
+WHAT WAS MISSING, AND WHY IT IS THE LOAD-BEARING PART. This command line has never had a model of
+who -- if anyone -- is on the other end of it. It could refuse, report and wait, and every one of
+those is a thing you do AT somebody rather than a thing you ask them. That was survivable while the
+only pauses were the promote decision and a spend confirmation, because both of those are held by
+the backend and answered in a browser: nothing on this machine had to know whether a human existed.
+
+It stops being survivable the moment an agent may ask a clarifying question. A question asked into
+a scheduled run is not a slow answer, it is no answer, and an agent that waits for one has replaced
+a build with a hang. So the question "is there a person here" has to be answerable BEFORE anything
+is asked, and it has to be answerable the same way every time.
+
+⚠ THE ANSWER IS NOT ``isatty``, AND THAT IS THE WHOLE DIFFICULTY. The process that runs ``mr-data``
+in the case this exists for is an agent, and an agent is a pipe whether or not a person is reading
+its chat. Taking a terminal's absence for a person's absence would answer "unattended" for exactly
+the population the feature is for. A terminal's PRESENCE is still sound in the other direction --
+somebody typed this at a shell -- so the terminal is read as a positive signal only, and never as
+a negative one. That asymmetry is the reason this is a resolver with six ordered steps rather than
+one call to :func:`os.isatty`.
+
+THE SIX STEPS, IN ORDER, AND WHY THAT ORDER.
+
+1. ``--unattended``, or :data:`ATTENDANCE_ENV` set to ``0``. Either party may say nobody is here,
+   and a no is final. ⚠ THIS IS THE ONE ASYMMETRY IN THE TABLE AND IT IS THE POINT OF IT. The agent
+   inside the process is the party whose judgement this exists to bound -- it is always in a
+   conversation and will always believe somebody is reading it -- while the process that SPAWNED it
+   is the only party that knows whether that conversation has a person on the other end. So the
+   outer party is given a veto and not a grant: a scheduler that exports ``MOSTLYRIGHT_ATTENDED=0``
+   cannot have that overruled from inside by an agent passing ``--attended``, and an agent that
+   knows it is running unwatched can still say so on a machine where the variable says otherwise.
+2. ``--attended``. Nobody vetoed and the caller stated it.
+3. :data:`ATTENDANCE_ENV` set to ``1``. The spawning process stated it, for every command it runs.
+4. :data:`ATTENDANCE_ENV` set to anything else. Unattended, and named as unreadable rather than
+   silently folded into the default, so a caller who wrote ``true`` learns that from the receipt
+   instead of from a build that quietly stopped asking.
+5. A controlling terminal on BOTH standard input and standard output. Sound one way only, as above.
+6. Nothing said: unattended.
+
+⚠ EVERY UNKNOWN RESOLVES TO UNATTENDED. The failure this guards against is a scheduled run stopping
+to ask a question nobody will ever read, and the failure it accepts in exchange is an agent that
+had a person available and did not use them -- which costs one guess that could have been a
+question, and is recoverable. They are not the same size.
+
+⚠ IT READS THE ENVIRONMENT AND TWO ``isatty`` CALLS, AND NOTHING ELSE. No file is opened, no
+credential is touched, no clock is read and nothing is written. The answer is a fact about this
+process, so it is derived from this process alone; ``environ`` and ``streams`` are injectable for
+the same reason ``router.local_execution_enabled`` takes an ``environ`` -- a guard should be able
+to ask what this function would answer without rewriting the process it is running in.
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from typing import Any
+
+#: The variable the process that spawns ``mr-data`` sets to say whether the session it is driving
+#: has a person in it. Exactly ``"1"`` is attended and exactly ``"0"`` is unattended -- and the
+#: ``"0"`` is a veto no flag overrides, for the reason the module docstring gives. Every other value
+#: is unattended and is reported as unrecognised rather than guessed at.
+#:
+#: ⚠ THE EXACT-``"1"`` COMPARISON IS THE ONE THIS REPOSITORY ALREADY USES, in
+#: ``router.local_execution_enabled``. Two opt-in variables that disagree about whether ``yes``
+#: counts is how one of them eventually gets set wrong in a way nobody notices.
+ATTENDANCE_ENV = "MOSTLYRIGHT_ATTENDED"
+
+#: What settled it. Reported in every receipt, because "there is nobody to ask" and "nobody said,
+#: so I assumed there is nobody to ask" are different situations to be in and only one of them is
+#: fixed by setting a variable.
+BY_FLAG = "flag"
+BY_ENVIRONMENT = "environment"
+BY_TERMINAL = "terminal"
+BY_DEFAULT = "default"
+
+#: The fifth value, and the only one that is a complaint. The variable was set to something this
+#: module does not read as either answer, so the answer is the fail-closed one and the receipt says
+#: the variable is the reason it is not the other one.
+BY_UNREADABLE_ENVIRONMENT = "unreadable_environment"
+
+DECIDED_BY = (BY_FLAG, BY_ENVIRONMENT, BY_TERMINAL, BY_DEFAULT, BY_UNREADABLE_ENVIRONMENT)
+
+#: The two values :data:`ATTENDANCE_ENV` is read for.
+_ATTENDED_VALUE = "1"
+_UNATTENDED_VALUE = "0"
+
+
+@dataclass(frozen=True)
+class Attendance:
+    """Whether a person can be asked something right now, and what settled it.
+
+    ``reason`` is one sentence that survives being printed on its own: it is what an agent reads
+    when it is deciding whether to ask, and what a person reads when they are wondering why it
+    did not.
+    """
+
+    attended: bool
+    decided_by: str
+    reason: str
+
+    def to_dict(self) -> dict[str, Any]:
+        return {"attended": self.attended, "decided_by": self.decided_by, "reason": self.reason}
+
+
+def _terminal(streams: Sequence[Any]) -> bool:
+    """Whether every named stream is a terminal.
+
+    ⚠ EVERY ONE OF THEM, NOT ANY. A command whose output was redirected into a file keeps a
+    terminal on standard input with nothing watching standard output, and a question printed into
+    that file is a question nobody read. Requiring both is what makes this signal mean "a person is
+    looking at this shell" rather than "a shell exists".
+    """
+
+    for stream in streams:
+        if not callable(getattr(stream, "isatty", None)):
+            return False
+        try:
+            if not stream.isatty():
+                return False
+        except (OSError, ValueError):
+            # A closed or detached stream is not a terminal, and it is not an error either: the
+            # question being asked is whether a person is there, and this is an answer to it.
+            return False
+    return True
+
+
+def resolve_attendance(
+    *,
+    flag: bool | None = None,
+    environ: Mapping[str, str] | None = None,
+    streams: Sequence[Any] | None = None,
+) -> Attendance:
+    """Decide, once, whether a person can be asked something in this invocation."""
+
+    table = os.environ if environ is None else environ
+    stated = table.get(ATTENDANCE_ENV)
+
+    # Step 1: the veto, from either party, before anything that could grant.
+    if flag is False:
+        return Attendance(
+            attended=False,
+            decided_by=BY_FLAG,
+            reason="you said no person is here to answer",
+        )
+    if stated == _UNATTENDED_VALUE:
+        return Attendance(
+            attended=False,
+            decided_by=BY_ENVIRONMENT,
+            reason=(
+                f"{ATTENDANCE_ENV} is 0, so the session that started this has nobody in it; that "
+                "is the one reading no argument here overrides"
+            ),
+        )
+    # Steps 2 and 3: the grants, the narrower statement first.
+    if flag is True:
+        return Attendance(
+            attended=True,
+            decided_by=BY_FLAG,
+            reason="you said a person is here to answer",
+        )
+    if stated == _ATTENDED_VALUE:
+        return Attendance(
+            attended=True,
+            decided_by=BY_ENVIRONMENT,
+            reason=f"{ATTENDANCE_ENV} is 1, so the session that started this has a person in it",
+        )
+    # Step 4: set, and readable as neither.
+    if stated is not None:
+        return Attendance(
+            attended=False,
+            decided_by=BY_UNREADABLE_ENVIRONMENT,
+            reason=(
+                f"{ATTENDANCE_ENV} is set to something this reads as neither 1 nor 0, so nobody "
+                "is treated as available; set it to 1 or to 0"
+            ),
+        )
+    # Steps 5 and 6.
+    if _terminal((sys.stdin, sys.stdout) if streams is None else streams):
+        return Attendance(
+            attended=True,
+            decided_by=BY_TERMINAL,
+            reason="this was typed at a terminal, so somebody is looking at it",
+        )
+    return Attendance(
+        attended=False,
+        decided_by=BY_DEFAULT,
+        reason=(
+            f"nothing said a person is here and this is not a terminal; set {ATTENDANCE_ENV}=1 "
+            "when a person can answer"
+        ),
+    )
+
+
+__all__ = [
+    "ATTENDANCE_ENV",
+    "BY_DEFAULT",
+    "BY_ENVIRONMENT",
+    "BY_FLAG",
+    "BY_TERMINAL",
+    "BY_UNREADABLE_ENVIRONMENT",
+    "DECIDED_BY",
+    "Attendance",
+    "resolve_attendance",
+]