mostlyright-data 0.9.4__tar.gz → 0.9.6__tar.gz

{mostlyright_data-0.9.4 → mostlyright_data-0.9.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.9.4
+Version: 0.9.6
 Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/
@@ -81,12 +81,11 @@ Description-Content-Type: text/markdown
 
 # mostlyright-data
 
-Mostly Right turns historical and live public data—weather, economic releases, financial reports,
-and satellite data—into consistent datasets for analysis and prediction. Each dataset includes
-source links, timestamps, and check results.
+Browse datasets built by our community or create your own with agents that find the sources and
+configure the pipeline. Mostly Right keeps the dataset live.
 
-`mostlyright-data` is the command-line client. It starts hosted research and build work, follows
-run status, and downloads completed Parquet datasets.
+`mostlyright-data` is the command-line client. It opens hosted research sessions, starts and
+follows hosted dataset runs, and downloads verified datasets after they finish.
 
 ## Requirements
 
@@ -119,20 +118,20 @@ mr-data --help
 Use `mr-data research-open`, `mr-data research-status`, and `mr-data research-close` for hosted
 research sessions. `research-status` shows the current state.
 
-## Local tools
+## Local execution
 
-The optional local tools require Linux:
+The local execution lane runs only on Linux:
 
 ```bash
 python -m pip install "mostlyright-data[local]"
 ```
 
-They build and check worker inputs. They are not required for the hosted client.
+It builds and checks worker inputs. It is not required for the hosted client.
 
 ## Documentation
 
 - [CLI guide](https://mostlyright.md/docs/guides/cli/)
-- [Research sessions](https://mostlyright.md/docs/guides/research/)
+- [Hosted research sessions](https://mostlyright.md/docs/guides/hosted-research/)
 - [Credentials](https://mostlyright.md/docs/guides/credentials/)
 - [Public datasets](https://mostlyright.md/docs/guides/use-public-datasets/)
 
@@ -141,11 +140,11 @@ Use `mr-data --help` for the full command list and options. Commands that suppor
 
 For local builds, `--input-root` and `--output` may use relative paths.
 
-## Limits and behavior
+## Behavior and limits
 
 - `mr-data review --local RUN_DIR` exits 1 when a check fails.
-- The weather Reader accepts one extracted GRIB2 record. It refuses a multi-record GFS or HRRR file.
-- Hosted credential-free acquisition is limited to one single-address public HTTPS object. Bounded collections require both local attestations.
+- The weather reader accepts one extracted GRIB2 record. It refuses a multi-record GFS or HRRR file.
+- Hosted credential-free acquisition supports one single-address public HTTPS object. Bounded collections require both local attestations.
 
 ## Verification
 

{mostlyright_data-0.9.4 → mostlyright_data-0.9.6}/README.md

@@ -1,11 +1,10 @@
 # mostlyright-data
 
-Mostly Right turns historical and live public data—weather, economic releases, financial reports,
-and satellite data—into consistent datasets for analysis and prediction. Each dataset includes
-source links, timestamps, and check results.
+Browse datasets built by our community or create your own with agents that find the sources and
+configure the pipeline. Mostly Right keeps the dataset live.
 
-`mostlyright-data` is the command-line client. It starts hosted research and build work, follows
-run status, and downloads completed Parquet datasets.
+`mostlyright-data` is the command-line client. It opens hosted research sessions, starts and
+follows hosted dataset runs, and downloads verified datasets after they finish.
 
 ## Requirements
 
@@ -38,20 +37,20 @@ mr-data --help
 Use `mr-data research-open`, `mr-data research-status`, and `mr-data research-close` for hosted
 research sessions. `research-status` shows the current state.
 
-## Local tools
+## Local execution
 
-The optional local tools require Linux:
+The local execution lane runs only on Linux:
 
 ```bash
 python -m pip install "mostlyright-data[local]"
 ```
 
-They build and check worker inputs. They are not required for the hosted client.
+It builds and checks worker inputs. It is not required for the hosted client.
 
 ## Documentation
 
 - [CLI guide](https://mostlyright.md/docs/guides/cli/)
-- [Research sessions](https://mostlyright.md/docs/guides/research/)
+- [Hosted research sessions](https://mostlyright.md/docs/guides/hosted-research/)
 - [Credentials](https://mostlyright.md/docs/guides/credentials/)
 - [Public datasets](https://mostlyright.md/docs/guides/use-public-datasets/)
 
@@ -60,11 +59,11 @@ Use `mr-data --help` for the full command list and options. Commands that suppor
 
 For local builds, `--input-root` and `--output` may use relative paths.
 
-## Limits and behavior
+## Behavior and limits
 
 - `mr-data review --local RUN_DIR` exits 1 when a check fails.
-- The weather Reader accepts one extracted GRIB2 record. It refuses a multi-record GFS or HRRR file.
-- Hosted credential-free acquisition is limited to one single-address public HTTPS object. Bounded collections require both local attestations.
+- The weather reader accepts one extracted GRIB2 record. It refuses a multi-record GFS or HRRR file.
+- Hosted credential-free acquisition supports one single-address public HTTPS object. Bounded collections require both local attestations.
 
 ## Verification
 

{mostlyright_data-0.9.4 → mostlyright_data-0.9.6}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "mostlyright-data"
-version = "0.9.4"
+version = "0.9.6"
 description = "Mostly Right CLI for reproducible, reviewed Parquet datasets"
 readme = "README.md"
 requires-python = ">=3.13,<3.14"

{mostlyright_data-0.9.4 → mostlyright_data-0.9.6}/src/mostlyright/data_harness/acquisition/url_policy.py

@@ -188,6 +188,62 @@ class ValidatedTarget:
         return urlunsplit(("", "", parsed.path or "/", parsed.query, ""))
 
 
+def approved_public_addresses(
+    hostname: str,
+    port: int,
+    *,
+    resolver: Resolver | None = None,
+) -> tuple[str, ...]:
+    """Return every address a hostname answers with, once each has been proved publicly routable.
+
+    The part of :func:`validate_public_https_url` that is about *where a name leads* rather than
+    about a URL or an allowlist, so that a transport which is not fetching a URL can hold the same
+    rule without holding a second copy of it.  A client that hands a hostname to the operating
+    system and lets it connect has delegated the decision to DNS: a document naming
+    ``stream.example.test`` reaches whatever that name resolves to, and inside a network that is
+    an internal service that never expected a stranger.
+
+    Every answer is checked, and one bad answer refuses the whole resolution.  Approving the public
+    members of a mixed set would be deciding *after* a resolver has already demonstrated that it
+    can put an address of its choosing in front of the caller.
+
+    A caller that connects to one of these addresses -- rather than handing the name back to the
+    operating system, which would resolve it a second time and need not agree -- has the answer it
+    checked.
+    """
+
+    if _looks_like_obfuscated_ip(hostname):
+        raise AcquisitionSecurityError(
+            "OBFUSCATED_IP",
+            "non-canonical numeric IP hostnames are denied",
+        )
+    literal = _parse_ip_literal(hostname)
+    if literal is not None:
+        addresses: tuple[str, ...] = (literal.compressed,)
+    else:
+        addresses = (SystemResolver() if resolver is None else resolver).resolve(hostname, port)
+    approved: list[str] = []
+    for raw in addresses:
+        try:
+            address = ipaddress.ip_address(raw)
+        except ValueError:
+            raise AcquisitionSecurityError(
+                "DNS_ADDRESS",
+                "resolver returned a non-IP address",
+            ) from None
+        _require_globally_routable(address)
+        approved.append(address.compressed)
+    approved_ips = tuple(sorted(dict.fromkeys(approved)))
+    if not approved_ips:
+        raise AcquisitionSecurityError("DNS_EMPTY", "target has no approved public addresses")
+    if len(approved_ips) > MAX_DNS_ANSWERS:
+        raise AcquisitionSecurityError(
+            "DNS_ANSWER_LIMIT",
+            f"target has more than {MAX_DNS_ANSWERS} approved addresses",
+        )
+    return approved_ips
+
+
 def validate_public_https_url(
     url: str,
     *,
@@ -222,34 +278,7 @@ def validate_public_https_url(
         raise AcquisitionSecurityError("METADATA_DESTINATION", "metadata/internal hosts are denied")
 
     literal = _parse_ip_literal(hostname)
-    if literal is not None:
-        addresses = (literal.compressed,)
-    else:
-        if _looks_like_obfuscated_ip(hostname):
-            raise AcquisitionSecurityError(
-                "OBFUSCATED_IP",
-                "non-canonical numeric IP hostnames are denied",
-            )
-        addresses = resolver.resolve(hostname, effective_port)
-    approved: list[str] = []
-    for raw in addresses:
-        try:
-            address = ipaddress.ip_address(raw)
-        except ValueError:
-            raise AcquisitionSecurityError(
-                "DNS_ADDRESS",
-                "resolver returned a non-IP address",
-            ) from None
-        _require_globally_routable(address)
-        approved.append(address.compressed)
-    approved_ips = tuple(sorted(dict.fromkeys(approved)))
-    if not approved_ips:
-        raise AcquisitionSecurityError("DNS_EMPTY", "target has no approved public addresses")
-    if len(approved_ips) > MAX_DNS_ANSWERS:
-        raise AcquisitionSecurityError(
-            "DNS_ANSWER_LIMIT",
-            f"target has more than {MAX_DNS_ANSWERS} approved addresses",
-        )
+    approved_ips = approved_public_addresses(hostname, effective_port, resolver=resolver)
 
     netloc = hostname
     if ":" in hostname and literal is not None:

{mostlyright_data-0.9.4 → mostlyright_data-0.9.6}/src/mostlyright/data_harness/cli.py

@@ -23,7 +23,7 @@ import sys
 import tempfile
 import time
 from collections.abc import Callable, Iterator, Mapping, Sequence
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
 from datetime import datetime
 from pathlib import Path
 from typing import Any
@@ -96,7 +96,6 @@ from mostlyright.data_harness.offline import (
 )
 from mostlyright.data_harness.pipeline import (
     _VERIFY_ALLOWLISTED_ENGINES,
-    MAX_ROWS,
     SOURCE_PROPOSAL_SET_VERSION,
     BuildError,
     ResourceLimits,
@@ -106,7 +105,10 @@ from mostlyright.data_harness.pipeline import (
     verify_candidate,  # noqa: F401 - retained for compatibility/security-route monkeypatch tests
 )
 from mostlyright.data_harness.plan_graph import (
+    HARD_MAX_INTERMEDIATE_ROWS,
+    HARD_MAX_OUTPUT_ROWS,
     HARD_MAX_RETAINED_BYTES,
+    HARD_MAX_SOURCE_ROWS,
     GraphResourcePolicy,
 )
 from mostlyright.data_harness.readers.contracts import ReaderError
@@ -209,6 +211,9 @@ from mostlyright.data_harness.sources.stream_recorder import (
 from mostlyright.data_harness.sources.stream_recorder import (
     ledger_root as stream_ledger_root,
 )
+from mostlyright.data_harness.sources.stream_registry import (
+    load_stream_connector_registry,
+)
 from mostlyright.data_harness.ux.commands import discovered_commands
 from mostlyright.data_harness.ux.credentials import LoginError, resolve_cloud_credentials
 from mostlyright.data_harness.ux.headline import (
@@ -390,17 +395,44 @@ def _selected_graph_limits(
     max_intermediate_rows: int | None,
     max_retained_bytes: int | None,
 ) -> ResourceLimits | None:
-    """Select only explicitly raised graph dimensions; keep every other row cap at 100k."""
+    """Narrow the trusted graph policy by exactly the dimensions the operator named.
+
+    These flags are a NARROWING instrument and nothing else. The baseline is what a build with no
+    flags at all already runs under -- ``GraphResourcePolicy()``, whose defaults are the engine's
+    own hard ceilings -- so an unnamed dimension keeps that ceiling and a named one lowers only
+    itself. The invariant is worth stating because it is what makes the flags safe to reach for:
+    for every dimension, the policy this returns is at most the no-flag default, and equals it
+    unless the flag naming that dimension was supplied.
+
+    It used to default every row dimension to ``pipeline.MAX_ROWS`` instead. That is the v1 plan's
+    row policy, not the graph engine's, and it is 1/100th of ``HARD_MAX_SOURCE_ROWS``: naming a
+    byte budget alone silently cut a v2 build's row ceiling to 100k, which is the opposite of what
+    an operator raising a budget is asking for. ``build`` only reaches this function for a v2 plan
+    -- a graph flag on a v1 plan is refused as ``GRAPH_POLICY_V1`` before the build starts -- so
+    there is no v1 caller whose 100k cap this could be speaking for.
+
+    ``--graph-max-rows`` sets the source AND output ceilings, as its help says: one number for
+    "how many rows this build may read and release". The intermediate ceiling is separate because
+    the engine's own contract separates them (``HARD_MAX_INTERMEDIATE_ROWS`` is four times
+    ``HARD_MAX_SOURCE_ROWS``, sized for a union of several year-scale sources), so an operator who
+    wants a narrower working set says so with the flag that means it.
+    """
 
     if max_rows is None and max_intermediate_rows is None and max_retained_bytes is None:
         return None
-    selected_rows = max_rows or MAX_ROWS
+    selected_rows = max_rows or HARD_MAX_SOURCE_ROWS
     return ResourceLimits(
+        # The v1 field, carried for a v2 build so `_replay_graph_derivations` streams sources under
+        # the policy rather than under the 100k default. It re-derives the same value from
+        # `graph_policy` anyway; agreeing here keeps the two from reading differently.
         max_rows=selected_rows,
         graph_policy=GraphResourcePolicy(
             max_source_rows=selected_rows,
-            max_intermediate_rows=max_intermediate_rows or selected_rows,
-            max_output_rows=selected_rows,
+            max_intermediate_rows=max_intermediate_rows or HARD_MAX_INTERMEDIATE_ROWS,
+            # `min` because the two ceilings are equal today and are not the same contract: a
+            # source ceiling that ever rose above the output one must not raise the output one
+            # with it.
+            max_output_rows=min(selected_rows, HARD_MAX_OUTPUT_ROWS),
             max_retained_bytes=max_retained_bytes or HARD_MAX_RETAINED_BYTES,
         ),
     )
@@ -1014,6 +1046,7 @@ def _dispatch(
                     producer_principal=args.producer_principal,
                     producer_attempt=args.producer_attempt,
                     previous_run_dir=args.previous_run_dir,
+                    connector_registry=args.connector_registry,
                     visual_store=visual_store,
                 )
                 payload = result.to_dict()
@@ -1830,22 +1863,29 @@ def _parser(
     build.add_argument("--producer-principal", required=True, help=_BUILT_BY)
     build.add_argument("--producer-attempt", required=True, help=_BUILD_ATTEMPT)
     build.add_argument("--research-dir", type=Path, help="existing visual research directory")
+    # ⚠ THE ARGPARSE MAXIMA ARE THE ENGINE'S OWN HARD CEILINGS, and deliberately not a second,
+    # smaller policy. `GraphResourcePolicy.__post_init__` is the single enforcement point -- it
+    # refuses anything above a ceiling whatever route the value arrived by -- so a smaller number
+    # here would not be a safety property, only a cap the CLI applies and the library does not.
+    # The previous maxima (1M rows, 2M intermediate, 256 MiB retained) were 1/10th, 1/20th and
+    # 1/64th of the ceilings they front, with nothing written down deriving them, and they made
+    # the year-scale capacity the graph engine documents unreachable from the command line.
     build.add_argument(
         "--graph-max-rows",
-        type=_integer_between(1, 1_000_000),
-        metavar="1..1000000",
+        type=_integer_between(1, HARD_MAX_SOURCE_ROWS),
+        metavar=f"1..{HARD_MAX_SOURCE_ROWS}",
         help="trusted graph source/output row ceiling (v2 plans only)",
     )
     build.add_argument(
         "--graph-max-intermediate-rows",
-        type=_integer_between(1, 2_000_000),
+        type=_integer_between(1, HARD_MAX_INTERMEDIATE_ROWS),
         default=None,
-        metavar="1..2000000",
+        metavar=f"1..{HARD_MAX_INTERMEDIATE_ROWS}",
         help="trusted graph intermediate row ceiling",
     )
     build.add_argument(
         "--graph-max-retained-bytes",
-        type=_integer_between(1, 256 * 1024 * 1024),
+        type=_integer_between(1, HARD_MAX_RETAINED_BYTES),
         default=None,
         metavar="BYTES",
         help="trusted graph retained-byte ceiling",
@@ -1979,6 +2019,14 @@ def _parser(
         help="the run settings: which recipe fingerprint and engine versions this run is pinned to",
     )
     recipe_run.add_argument("--input-root", type=Path, required=True, help=_INPUT_ROOT)
+    recipe_run.add_argument(
+        "--connector-registry",
+        type=Path,
+        help=(
+            "the folder of stream source documents the captures under the input folder were "
+            "recorded under; without it, only the ones this build ships are held"
+        ),
+    )
     recipe_run.add_argument("--output", type=Path, required=True, help=_OUTPUT_NEW)
     recipe_run.add_argument("--previous-run-dir", type=Path, help=_PREVIOUS_BUILD)
     recipe_run.add_argument("--producer-principal", required=True, help=_BUILT_BY)
@@ -3058,6 +3106,7 @@ def _local_recipe_registry(
     sandbox: CrawlerSandbox,
     stock_sources: tuple[Any, ...],
     stream_sources: tuple[Any, ...],
+    connector_registry: Path | None = None,
 ) -> AdapterRegistry:
     """Build the exact allowlist this local run is permitted to use.
 
@@ -3069,6 +3118,14 @@ def _local_recipe_registry(
     host allowlist against the address in the query and the other hands the fetch to Studio's
     hosted clean room.  A recorded capture has no address and no hosted home yet, so an adapter
     registered in either of those would claim a capability it cannot honestly provide.
+
+    ``connector_registry`` is the folder of source documents an operator wrote, the same one they
+    recorded the capture under.  ``None`` means the versions this build ships.  It has to be
+    sayable here: a sealed capture pins the digest of the source document it was recorded under,
+    and a build that could only look in the documents it ships would refuse every capture taken
+    from a source somebody wrote -- which is every live capture, because the one source this build
+    ships may not be captured live at all.  It is not a way to reinterpret a capture: the manifest
+    still names its own digest, and this only says where that version may be looked up.
     """
 
     adapters: list[Any] = []
@@ -3082,19 +3139,37 @@ def _local_recipe_registry(
         )
     if stream_sources:
         captures_root = input_root / _CAPTURES_DIRECTORY_NAME
-        gateway = RecordedStreamGateway(
+        # Read once, here, before any Build work: a folder that is not there, or is a link to
+        # one, is refused with STREAM_CONNECTOR_REGISTRY_ROOT at the start of the run rather
+        # than found missing halfway through it. With no folder named, this is the registry of
+        # documents the build ships, which is what every run did before the folder could be said.
+        held = load_stream_connector_registry(
+            None if connector_registry is None else connector_registry.absolute()
+        )
+        reader = RecordedStreamGateway(
             captures_root=captures_root,
             # Coordinator times, not capture times. The adapter checks the batch it gets back
             # against the observation evidence recorded here, and a gateway that answered with
             # times it read out of a file would make that check a tautology.
             observed_at=execution.requested_at,
             available_at=execution.requested_at,
+            registry=held,
         )
+        governance = _recorded_stream_governance(execution, stream_sources, reader)
+        # The same gateway, stamping a batch with the availability this coordinator just wrote
+        # down rather than with the moment the run started. The two are not the same fact: a
+        # capture stops being extended when its Recorder stops, and the observation above says
+        # so deliberately, so a gateway still stamping "now" would hand the adapter a batch its
+        # own evidence contradicts and every build over a capture older than an instant would
+        # be refused with STREAM_TEMPORAL_BINDING. This is still a coordinator input -- the
+        # value is read once, into the evidence, and handed on -- and not the gateway reading a
+        # time out of the file it is serving.
+        gateway = replace(reader, available_at=governance.observation.available_at)
         adapters.append(
             StreamBatchAdapter(
                 gateway=gateway,
                 sandbox=sandbox,
-                governance=_recorded_stream_governance(execution, stream_sources, gateway),
+                governance=governance,
                 watermark_store=StreamWatermarkStore(stream_ledger_root(captures_root)),
             )
         )
@@ -3111,6 +3186,7 @@ def _execute_public_recipe_run(
     producer_principal: str,
     producer_attempt: str,
     previous_run_dir: Path | None,
+    connector_registry: Path | None = None,
     visual_store: VisualRunStore | None = None,
 ) -> Any:
     stock_sources = _stock_local_reader_sources(recipe, execution)
@@ -3147,6 +3223,7 @@ def _execute_public_recipe_run(
             sandbox=sandbox,
             stock_sources=stock_sources,
             stream_sources=stream_sources,
+            connector_registry=connector_registry,
         )
 
         def _context(source: Any, operation: str) -> AcquisitionContext:

{mostlyright_data-0.9.4 → mostlyright_data-0.9.6}/src/mostlyright/data_harness/hosted_crawler_job.py

@@ -35,6 +35,7 @@ from mostlyright.data_harness.canonical import (
 from mostlyright.data_harness.hosted_crawler import (
     CRAWLER_EGRESS_POLICY_ENV,
     WARM_UP_RESULT_SCHEMA,
+    HostedCrawlerError,
     encode_normalization_input,
 )
 from mostlyright.data_harness.hosted_crawler_fetch import fetch_crawler_source
@@ -80,6 +81,24 @@ PRE_BOOTSTRAP_FAILURE_CODES = frozenset(
         "CRAWLER_BOOTSTRAP_RESPONSE_TOO_LARGE",
     }
 )
+# Wrapper deaths after bootstrap: the fetch may have succeeded, so these must
+# never become sealed refusals (a refusal is the source's answer, and the
+# source never answered "no").  Without a control-plane report the session
+# dangles until the producer's poll deadline expires and the run dies blind
+# (harness#400).  Keep this closed in lock-step with Studio's
+# ``WRAPPER_FAILURE_CODES``; a Studio that predates that set refuses the
+# report with 400 and the job still exits non-zero, so deploy order is free.
+WRAPPER_FAILURE_CODES = frozenset(
+    {
+        "CRAWLER_CHILD_FAILED",
+        "CRAWLER_CHILD_INVALID",
+        "CRAWLER_EXECUTABLE_UNAVAILABLE",
+        "CRAWLER_JOB_FAILED",
+        "CRAWLER_RESULTS_BUCKET_REQUIRED",
+        "CRAWLER_UPLOAD_FAILED",
+        "CRAWLER_WORKLOAD_IDENTITY_UNAVAILABLE",
+    }
+)
 # Sized for an EPA-scale year: the streamed decode, the streamed verification pass, and
 # the chunked result encode are each one pure-Python walk over ~200M cells, which a live
 # run spends 12-15 minutes on.  The old 600 was calibrated to decode alone and killed a
@@ -474,7 +493,7 @@ def _report_crawler_bootstrap_failure(
     can cross back into the crawler through this path.
     """
 
-    if failure_code not in PRE_BOOTSTRAP_FAILURE_CODES:
+    if failure_code not in PRE_BOOTSTRAP_FAILURE_CODES | WRAPPER_FAILURE_CODES:
         return False
     token: bytearray | None = None
     response: bytearray | None = None
@@ -1028,19 +1047,23 @@ def _validate_public_https_result(
         from mostlyright.data_harness.readers.contracts import ReaderPin
 
         try:
+            from mostlyright.data_harness.readers.registry import TOOLBOX
+
+            family = TOOLBOX.resolve(reader["family_id"], reader["family_version"])
+            # The child digests the ADMITTED options -- ``validate_options``
+            # fills family defaults -- so the expected identity must digest the
+            # same bytes.  Digesting the sealed pin raw made every recipe that
+            # omitted a defaulted key fail ``reader_identity`` on every crawl.
             pin = ReaderPin(
                 reader["family_id"],
                 reader["family_version"],
-                reader["decode_options"],
+                dict(family.validate_options(reader["decode_options"])),
             )
         except (KeyError, TypeError, ValueError):
             _fail("CRAWLER_CHILD_INVALID", "public HTTPS Reader identity is invalid")
         reader_identity = (pin.family_id, pin.family_version, pin.options_digest)
         resource_caps = query.get("resource_caps")
         try:
-            from mostlyright.data_harness.readers.registry import TOOLBOX
-
-            family = TOOLBOX.resolve(pin.family_id, pin.family_version)
             expected_reader_budgets = asdict(
                 family.default_budgets.narrowed_by(
                     {
@@ -1324,6 +1347,10 @@ def crawler_job_main(argv: list[str] | None = None) -> int:
         return 0
     except (
         CanonicalJSONError,
+        # HostedCrawlerError is a RuntimeError, not a ValueError; leaving it out
+        # let every request/normalization refusal escape as a raw traceback with
+        # no sealed failure and no session report (harness#400).
+        HostedCrawlerError,
         HostedCrawlerJobError,
         HostedCrawlerProtocolError,
         TypeError,
@@ -1366,6 +1393,28 @@ def crawler_job_main(argv: list[str] | None = None) -> int:
                 )
                 sys.stderr.write("\n")
                 return 0
+        if code in WRAPPER_FAILURE_CODES:
+            # A post-bootstrap wrapper death has no sealed-refusal lane (the
+            # source never refused), so terminalize the session through the
+            # ``:fail`` control-plane route instead of leaving it to dangle
+            # into the producer's poll deadline.  Best-effort: the job exits
+            # non-zero either way, and an older Studio refuses the code.
+            reported = _report_crawler_bootstrap_failure(
+                crawler_session_id=crawler_session_id,
+                failure_code=code,
+            )
+            sys.stderr.write(
+                canonical_json_bytes(
+                    {
+                        "status": "failed",
+                        "code": code,
+                        "detail": detail,
+                        "session_failure_reported": reported,
+                    }
+                ).decode()
+            )
+            sys.stderr.write("\n")
+            return 1
         sys.stderr.write(
             canonical_json_bytes({"status": "failed", "code": code, "detail": detail}).decode()
         )