mostlyright-data 0.31.5__tar.gz → 0.31.6__tar.gz

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mostlyright-data
-Version: 0.31.5
+Version: 0.31.6
 Summary: Mostly Right hosted CLI for reviewed datasets
 Project-URL: Homepage, https://mostlyright.md/
 Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/
@@ -103,7 +103,8 @@ explicit standalone bounded inspection; row ceilings do not impose a five-minute
 Run submission is idempotent: sending identical recipe, mode and bounds returns the same run,
 including a previous failure. The CLI reports a non-queued result as `run_returned` and exits
 with code 2 when that run failed. Read its failure before retrying; `mr-data run --retry RUN_ID`
-can reissue supported platform execution failures without changing the recipe.
+resumes a failed or cancelled progressive full from the pages it saved, and can reissue a
+supported platform execution failure of any other run without changing the recipe.
 
 Use the identifiers returned by registration and run submission. `mr-data dataset create`
 creates a dataset page before a recipe is ready. `mr-data watch RUN_ID` follows a submitted run.

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/README.md

@@ -88,7 +88,8 @@ explicit standalone bounded inspection; row ceilings do not impose a five-minute
 Run submission is idempotent: sending identical recipe, mode and bounds returns the same run,
 including a previous failure. The CLI reports a non-queued result as `run_returned` and exits
 with code 2 when that run failed. Read its failure before retrying; `mr-data run --retry RUN_ID`
-can reissue supported platform execution failures without changing the recipe.
+resumes a failed or cancelled progressive full from the pages it saved, and can reissue a
+supported platform execution failure of any other run without changing the recipe.
 
 Use the identifiers returned by registration and run submission. `mr-data dataset create`
 creates a dataset page before a recipe is ready. `mr-data watch RUN_ID` follows a submitted run.

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/SKILL.md

@@ -27,17 +27,21 @@ On a table that is already live, list the missed entries and ask the user first.
 the recipe digest, so the next build is a full resync. A paid source is called again for every
 per-row request whose saved answer no longer applies.
 
-**Gate 2. State a pace and leave concurrency out.** A page collection that states no pace sends
-one request a second. State `pages.min_interval_milliseconds` as the gap the publisher documents.
-No documented limit: state 1000 ms and say so. That is a valid reason. Registration refuses a gap
-under 16 ms. That is a platform bound, never a target. Where the deployment spreads requests over
-its egress addresses, the gap is kept per address, so the publisher can see a multiple of the
-stated rate. No command prints how many addresses there are. So when the publisher's limit is
-per key or per account, state 1000 ms unless the user confirms the deployment does not spread
-requests. Leave `pages.concurrency` out: the worker then sizes the width, and a stated value is
-only a ceiling. Keep a stated one only when a run's refusal told you to lower it or the publisher
-documents a limit on concurrent connections, and record the reason in a dataset note so the next
-session finds it. Then choose the source shape that fits.
+**Gate 2. Omit an undocumented pace, and leave a low concurrency out.** A page collection that
+states no pace starts at 1000 ms and adapts down to 250 ms. A stated gap is a hard floor that
+never adapts. No documented limit: omit pages.min_interval_milliseconds. State a number only when the
+publisher documents one. Registration refuses a gap under 4 ms, which is 256 requests a second
+to one host from a collection. That is a platform bound, never a target. Where the deployment
+spreads requests over its egress addresses, an omitted pace is held to 4 requests a second per
+address and to max(4, M) a second host-wide when spread over M addresses. A stated gap is kept
+per address, so the publisher can see a multiple of that rate. No command prints how many
+addresses there are. When the publisher's limit is per key or per account, state the gap it
+documents. Leave `pages.concurrency` out, or state it only at 64 or above. A stated width
+strictly under 64 is a ceiling on the width the worker sizes, and `run --full` refuses it
+unless `--accept-slow` says why. `concurrency: 1` stays that miss. Keep a lower ceiling only
+when a run's refusal told you to lower it or the publisher documents a limit on concurrent
+connections, and record the reason in a dataset note so the next session finds it. Then choose
+the source shape that fits.
 
 | Source shape | Use | Always state | Recipe |
 | --- | --- | --- | --- |
@@ -149,7 +153,7 @@ step 6. The stage 3 reference holds its rule. Gate 4 is step 6.
    has a date-like column, `batch` for a per-row GET whose API takes a list of ids, and
    `reference` for a bulk dump beside a window
    ([fast builds](references/making-a-table-incremental.md#fast-builds-tablepartition-batch-and-reference)).
-   State the gap, drop a stated `pages.concurrency`, page a keyed API in parallel and pin the
+   Omit an undocumented gap, drop a stated `pages.concurrency` under 64, page a keyed API in parallel and pin the
    ranged Parquet Reader where the entries `pace`, `concurrency`, `parallel_pagination` and
    `ranged_parquet` say so and the source meets each condition
    ([fetching fast](references/making-a-table-incremental.md#fetching-fast-pace-width-parallel-pages-and-ranged-parquet)).
@@ -237,7 +241,9 @@ These failures look like recipe faults and are not:
 | `EXECUTION_CAPACITY_CLAIM_FAILED` during a platform release | Start the run again after the release. Keep the recipe. |
 | "Schedule held" on the dashboard | Read what holds it with `mr-data table show TABLE_ID`. A recipe with no refreshing plan and no `rebuild` stays held. [Promotion and cadence](references/promote.md) has the repair. |
 
-Legacy `room_fault: true` can identify a supported `mr-data run --retry RUN_ID` case.
+Legacy `room_fault: true` can identify a supported `mr-data run --retry RUN_ID` case. A failed
+or cancelled progressive full is resumed by the same command from the pages it saved, unless the
+failure says the recipe must change.
 **`room_fault: false` does not prove the recipe is wrong.** Repeating a terminal coordinate may
 only return its stored failure. Retry at most three times when supported by evidence, then report
 what remains unresolved. Never narrow coverage, drop checks, guess encodings or switch sources

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/5-draft-one-recipe-document-one-call.md

@@ -33,12 +33,11 @@ the identifier stage 1 printed. With it, the registration attaches to the page s
 watching; without it there is no page whose record could carry the brief, and the command refuses
 -- `THIN_RECIPE_DATASET_UNBOUND`, below, whether or not `--no-lint` was passed.
 
-**A document that carries `dataset.id` leaves `dataset.description` out.** Registration FILLS an
-empty page's description and never replaces one, so this destroys nothing; what it does is write
-prose composed for the recipe onto the page as its documentation, from the wrong writer. The page's
-description is written once, below, in the one `mr-data dataset set` call that carries the title
-and the tags with it, under the version it read. The preflight refuses a document carrying both.
-Leave `dataset.name` in: the title is the recipe's own subject, settled below.
+**A document that carries `dataset.id` may also carry `dataset.description`.** Registration FILLS
+an empty page's description and never replaces a stored one. The page's own sentence is still
+written, below, in the `mr-data dataset set` call that carries the title and the tags with it,
+under the version it read. The preflight does not refuse the pair. Leave `dataset.name` in: the
+title is the recipe's own subject, settled below.
 
 **Registration refuses a dataset nobody was asked about**, with `THIN_BRIEF_MISSING`. The record
 has to carry either an answered `clarification` block or a delegation `decision` block first. Where

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/autonomous-delivery.md

@@ -27,8 +27,9 @@ state what is live and what is not, rather than claiming a promotion or reportin
 coordinates.
 
 A typed refusal is recovery evidence, not a terminal outcome while a safe alternative remains.
-Record it and continue with the first supported action that fits the evidence: retry a room fault
-with `mr-data run --retry RUN_ID`, select another supported source, Reader, format or capacity
+Record it and continue with the first supported action that fits the evidence: retry a room fault,
+or resume a failed or cancelled progressive full, with `mr-data run --retry RUN_ID`, select another
+supported source, Reader, format or capacity
 class, or revise the recipe document. Retry an unchanged transient failure at most three times and
 never repeat a permanent refusal without changing the condition that caused it. Re-run source fitness and feasibility classification whenever the selected
 alternative could change their result. Preserve the requested

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/commands.md

@@ -23,9 +23,9 @@ run in full.
 | `mr-data connected` | `connected list` shows the public datasets connected to this workspace and `connected add SLUG` connects one by its slug. It needs `mr-data login`: both ask Cloud, with the device key, over the routes MCP's `list_connected_datasets` and `connect_dataset` use. |
 | `mr-data ask` | `ask "QUESTION" [--run RUN_ID ...] [--failure-code CODE]` asks the Mostly Right support agent one question and waits for the answer; `--no-wait` prints the question id at once and `ask status QUESTION_ID` reads it later. It needs `mr-data login` and a workspace the agent is switched on for: while it is off the answer is `THIN_CLOUD_SUPPORT_AGENT_DISABLED`, which is final. The agent may file a problem for the Mostly Right team and prints its reference (`SR-N`). |
 | `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`, unless the user's own request already has an empty dataset open, in which case it answers `open_request_reused` with that `dataset_id` and mints nothing (`--separate` mints a new one for a different dataset); `dataset show ID` reads it back; `dataset set ID --name TEXT --topics "a,b,c" --license ID --description-file F` writes the title, the descriptive tags, the SPDX licence and the description under the version it was read at, retrying once if somebody else wrote first, and an empty `--topics` or `--license` takes that value off the page; send `--name` and `--topics` whenever you send `--description-file`, because a description sent on its own can make Studio write a new title and new tags, and read `changed` and `written_by_studio` on the receipt for every field the write moved; a saved write whose public sync fails exits 2 and reports `public_projection_synced: false` — use `dataset sync ID` to retry that sync without rewriting Studio; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream; `dataset activity ID --phase P --message TEXT` says what is happening right now, silently, and is never a chat message and never a cell; `dataset publish ID [--mode public|link|private]` says who can read the dataset — `public` lists it in the public directory and serves it at an address anybody can read, `link` serves it at an unlisted address, `private` takes it back to the workspace — and `dataset publish ID --show` reads that back without changing it; `dataset archive ID --confirm-name TITLE` retires the page and frees its title, deleting nothing. |
-| `mr-data recipe` | Register one recipe document (dataset, question, table plan, sources, transform, checks and units) in one call, and print the identifiers the server derived. The document is read first and every fault comes back in one refusal, each with the JSON pointer that names it. `--no-lint` sends it as written instead. A recipe that misses a feature that sets how fast a run fetches (an unstated pace, a stated `pages.concurrency`, a per-row request open to `batch`, a page walk open to `pagination.parallel`, or a Parquet file open to a ranged read) is refused `THIN_RECIPE_SLOW`, with each feature's pointer and fix, before anything is sent. Apply the fixes, or pass `--accept-slow "REASON"` with one sentence on why the slow path is right for this source. The receipt prints the reason and the features it covered. Registration refuses `THIN_BRIEF_MISSING` when the dataset's record carries no answered question and no delegation, and `THIN_RECIPE_DATASET_UNBOUND` when the document names no `dataset.id`. `--delegated "their words"` writes the delegation onto the dataset and registers in the same command, and clears the first of those two and never the second. `mr-data recipe show ID` reads one back. `mr-data recipe explain FILE` (or a recipe identifier) lists every reason its table can or cannot refresh, each with a rule code and Studio's suggested repair, and exits 2 when one blocks it. `mr-data recipe forecast RECIPE_ID` says what one run would ask of each source before it runs, and exits 3 when the run would overrun today's platform quota or one attempt. `--digest` is optional there, and without it the digest the recipe was registered with is forecast. `mr-data recipe readiness --table TABLE_ID --current-only` reads the revision one table runs now. The explanation's `fast_features` names each fast feature the recipe uses or misses, with a one-line fix. [Making a table incremental](making-a-table-incremental.md#the-loop-explain-forecast-run-diagnose) says when to use each. |
+| `mr-data recipe` | Register one recipe document (dataset, question, table plan, sources, transform, checks and units) in one call, and print the identifiers the server derived. The document is read first and every fault comes back in one refusal, each with the JSON pointer that names it. `--no-lint` sends it as written instead. A recipe that misses a feature that sets how fast a run fetches (a stated `pages.concurrency` under 64, a per-row request open to `batch`, a page walk open to `pagination.parallel`, or a Parquet file open to a ranged read) is refused `THIN_RECIPE_SLOW`, with each feature's pointer and fix, before anything is sent. Apply the fixes, or pass `--accept-slow "REASON"` with one sentence on why the slow path is right for this source. The receipt prints the reason and the features it covered. Registration refuses `THIN_BRIEF_MISSING` when the dataset's record carries no answered question and no delegation, and `THIN_RECIPE_DATASET_UNBOUND` when the document names no `dataset.id`. `--delegated "their words"` writes the delegation onto the dataset and registers in the same command, and clears the first of those two and never the second. `mr-data recipe show ID` reads one back. `mr-data recipe explain FILE` (or a recipe identifier) lists every reason its table can or cannot refresh, each with a rule code and Studio's suggested repair, and exits 2 when one blocks it. `mr-data recipe forecast RECIPE_ID` says what one run would ask of each source before it runs, and exits 3 when the run would overrun today's platform quota or one attempt. `--digest` is optional there, and without it the digest the recipe was registered with is forecast. `mr-data recipe readiness --table TABLE_ID --current-only` reads the revision one table runs now. The explanation's `fast_features` names each fast feature the recipe uses or misses, with a one-line fix. [Making a table incremental](making-a-table-incremental.md#the-loop-explain-forecast-run-diagnose) says when to use each. |
 | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
-| `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID`, with `--digest RECIPE_DIGEST` when you want the run to assert which bytes it means. Neither is positional. The digest is the bare hex the registration receipt printed. Left out, the run reads it back from the registered recipe, which is the same value, because a recipe identifier names one revision. A full of a recipe that misses a fetch-speed feature is refused `THIN_RECIPE_SLOW`, as registration is, unless `--accept-slow "REASON"` is passed. Other modes print the missed features and are not refused. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh executes only Studio's persisted source-action plan, and a plan shape it does not refresh returns `RESYNC_REQUIRED` before acquisition. [Which plans refresh](making-a-table-incremental.md#which-plans-refresh) lists them, and `mr-data recipe explain` says which one a table is. Use `mr-data table resync TABLE_ID --request-id UUID` for an explicit full source reread, retaining that UUID if the response is uncertain. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal. Report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. `--max-rows` cuts rows after a source is fetched and decoded, so it does not shorten a collection walk. A sample with no clamp is refused: `CLAMPS_REQUIRED` from Studio, and `THIN_REQUEST_INVALID` from this client first. A table's first succeeded run goes live, truncated or not, so test on a scratch table as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. A large run is held for a size confirmation (not a charge), which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried. `--approve-full RUN_ID` releases the full an existing legacy sample-first pair is holding once its preview has succeeded, naming either half of the pair. Where the deployment still wants a person at a browser it answers `THIN_INTERACTIVE_HUMAN_REQUIRED` and names the run page. `--retry RUN_ID` tries one failed run again when its triple carries `room_fault: true`, re-stating that run's own coordinate and ceilings and naming the failed run so Studio starts a new attempt, and refuses with the triple and the next step Studio's recovery category names when it is not. `room_fault` is Studio's `recovery.room_fault` where Studio states one. A source that kept refusing (`SOURCE_RATE_LIMITED`, `SOURCE_HTTP_5XX`, `SOURCE_PROXY_BUSY`) is sent anyway: Studio admits it 120 seconds after the failure and refuses `RUN_RETRY_TOO_SOON` with a Retry-After before then, so wait that long and run `--retry` again. `--cancel RUN` stops one. |
+| `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID`, with `--digest RECIPE_DIGEST` when you want the run to assert which bytes it means. Neither is positional. The digest is the bare hex the registration receipt printed. Left out, the run reads it back from the registered recipe, which is the same value, because a recipe identifier names one revision. A full of a recipe that misses a fetch-speed feature is refused `THIN_RECIPE_SLOW`, as registration is, unless `--accept-slow "REASON"` is passed. Other modes print the missed features and are not refused. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh executes only Studio's persisted source-action plan, and a plan shape it does not refresh returns `RESYNC_REQUIRED` before acquisition. [Which plans refresh](making-a-table-incremental.md#which-plans-refresh) lists them, and `mr-data recipe explain` says which one a table is. Use `mr-data table resync TABLE_ID --request-id UUID` for an explicit full source reread, retaining that UUID if the response is uncertain. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal. Report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. `--max-rows` cuts rows after a source is fetched and decoded, so it does not shorten a collection walk. A sample with no clamp is refused: `CLAMPS_REQUIRED` from Studio, and `THIN_REQUEST_INVALID` from this client first. A table's first succeeded run goes live, truncated or not, so test on a scratch table as [the probe rule](3-probe-read-a-source-before-committing-to-it.md#the-probe-rule-test-on-a-scratch-table-never-on-the-real-table) says. A large run is held for a size confirmation (not a charge), which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried. `--approve-full RUN_ID` releases the full an existing legacy sample-first pair is holding once its preview has succeeded, naming either half of the pair. Where the deployment still wants a person at a browser it answers `THIN_INTERACTIVE_HUMAN_REQUIRED` and names the run page. `--retry RUN_ID` resumes a failed or cancelled progressive full from the pages it saved (`resume_capture_run_id` and the ceilings it carried; `retry_of_run_id` as well only when Studio admits a named retry). A failure that says the recipe must change is refused before anything is posted. Any other failed run is tried again when its triple carries `room_fault: true`, re-stating that run's own coordinate and ceilings and naming the failed run so Studio starts a new attempt, and is refused with the triple and the next step Studio's recovery category names when it is not. `room_fault` is Studio's `recovery.room_fault` where Studio states one. A source that kept refusing (`SOURCE_RATE_LIMITED`, `SOURCE_HTTP_5XX`, `SOURCE_PROXY_BUSY`) is sent anyway: Studio admits it 120 seconds after the failure and refuses `RUN_RETRY_TOO_SOON` with a Retry-After before then, so wait that long and run `--retry` again. `--cancel RUN` stops one. |
 | `mr-data status` | Report which of the seven states one run is in, what it delivered and whether a ceiling cut it short, and — on a failure — what failed, where, and whether the fault was the execution room's rather than the recipe's. Exits non-zero on a failed run. `--diagnose` adds everything else Studio can say about the run: its progress, a per-row source's plan counts, its checkpoints, and how each attempt ended, with a lost attempt's cause marked as an inference. |
 | `mr-data runs` | Report this workspace's own runs: identifier, mode, state and creation time, plus the failure triple of any that failed and whether that failure was the execution room's. `--dataset DATASET_ID` narrows it to one dataset's runs, so "what have I run for this dataset?" is one scoped read, and a value that is not an identifier is refused `THIN_ARGUMENT_INVALID`. `--status` and `--mode` narrow it too and are refused `THIN_ARGUMENT_INVALID` for a word outside the seven states or the six modes; the three combine. Studio narrows before it cuts a page, so a filtered page holds the newest runs that match however many runs the workspace has. `--limit` says how many, and pages are followed to reach it; a listing that does not end says how many were read and suggests a smaller `--limit` or a narrower filter. The answer prints `next_cursor`, and `--cursor` continues from it. |
 | `mr-data watch` | Stream one run's live progress under the durable event type of each stage, resuming across stream cuts. Exits non-zero when the run failed. `mr-data watch RUN_ID RUN_ID ...` follows up to 16 runs at once, prints one line as each ends, and returns once all have finished with one receipt per run (`failed_run_ids`, `refused_run_ids`); it exits 2 when any run failed and 1 when a run could not be watched. `--from-seq` and `--last-event-id` resume one run only. |

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/live-run.md

@@ -30,7 +30,8 @@ appends again, and `watch` stops rather than reconnecting.
 coordinate replays the stored result rather than making a new attempt — a stored failure included —
 which is why `mr-data run --retry RUN_ID` names the failed run to Studio rather than re-sending the
 request as it was; the ceilings stay the same unless Studio does not admit the named retry, and
-then the byte ceiling is raised as it was before. It is also why `SANDBOX_MEMORY_BOUNDARY` reads as a source problem and is not one: it names
+then the byte ceiling is raised as it was before. A failed or cancelled progressive full is
+resumed from the pages it saved, and names the run only when Studio admits that name. It is also why `SANDBOX_MEMORY_BOUNDARY` reads as a source problem and is not one: it names
 the source being fetched, the cause is a clean room with no delegated memory cgroup, it is
 intermittent and per-instance, and `room_fault` is true on that run so nothing has to be inferred
 from the wording. Report the failure once, retry it when supported, then report the replacement

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/making-a-table-incremental.md

@@ -359,14 +359,14 @@ too, so change them in the first revision.
 
 | Entry | It reports | The fix |
 | --- | --- | --- |
-| `pace` | a page collection or a keyless request window that states no gap | state the gap the publisher documents. No documented limit: state 1000 ms and say so |
-| `concurrency` | any stated `pages.concurrency` | remove it, unless a run's refusal told you to lower it or the publisher documents a limit on concurrent connections. Record the reason in a dataset note |
+| `pace` | a stated gap under the registration floor | registration refuses it. An omitted gap starts at 1000 ms and adapts down to 250 ms, and a stated gap is a hard floor that never adapts. Omit the member when the publisher documents no limit |
+| `concurrency` | a stated `pages.concurrency` strictly under 64 | remove it, unless a run's refusal told you to lower it or the publisher documents a limit on concurrent connections. At 64 and above the stated width stands. Record the reason in a dataset note |
 | `parallel_pagination` | a serial `GET` page walk over JSON on `authenticated.https.api_key@2.0.0`, on a source with no window | move to `@2.1.0` with the counted `pagination.parallel`, when the first page states its page count |
 | `ranged_parquet` | a `parquet.tabular@1.0.0` pin on a public file with no request window, in a recipe that states no `execution_policy` | pin `parquet.tabular@1.1.0`, when the publisher answers byte ranges and the limits in [Readers](readers.md) fit |
 
-A stated gap is never reported, whatever its size. A slow gap that is the publisher's limit is
-correct. The entries read the document alone. They cannot know a publisher's limit, so every fix
-is yours to check against the source.
+A stated gap at or above the registration floor is in use, and it never adapts. An omitted gap
+is in use too. A slow gap that is the publisher's limit is correct. The entries read the document
+alone. They cannot know a publisher's limit, so every fix is yours to check against the source.
 On a table that is already live, list the missed entries and ask the user first. Each fix changes
 the recipe digest, so the next build is a full resync. A paid source is called again for every
 per-row request whose saved answer no longer applies.
@@ -630,9 +630,10 @@ For Binance BTCUSDT one-minute klines, start the backfill at `2017-08-17T00:00:0
 }
 ```
 
-The 1000 above is one request a second. State the gap the publisher documents. With no documented
-limit, state 1000 and say so. `mr-data recipe explain` reports a keyless window that states no
-gap under `fast_features` as `pace`.
+The 1000 above is a stated hard floor that never adapts. Omit
+`window.request.min_interval_milliseconds` when the publisher documents no limit: an omitted gap
+starts at 1000 ms and adapts down to 250 ms. State a number only when the publisher documents one.
+`mr-data recipe explain` reports that window under `fast_features` as `pace`, in use.
 
 Keep `granularity: day`, `timezone: UTC` and `merge.materialization: partition_replace`.
 For the acquired `open_time_ms` column declare

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/patterns.md

@@ -14,8 +14,9 @@ Before you register a copy, replace every placeholder:
   Reader settings, read from the source itself.
 - `rights_claim` holds the basis you found, the digest of the terms you read, and a note.
 - `credential.secret_name` names a key the user enrolled with `mr-data keys set`.
-- A gap of 1000 ms means the publisher documents no limit. State the gap the publisher allows,
-  and say which you used and why.
+- A gap of 1000 ms in a worked recipe is a hard floor that never adapts. When the publisher
+  documents no limit, omit `pages.min_interval_milliseconds` instead of copying 1000. State a
+  number only when the publisher documents one, and say which you used and why.
 
 Then run `mr-data recipe explain RECIPE.json --json`, and take Gate 3's bounded test on a scratch
 table before any full run.

{mostlyright_data-0.31.5 → mostlyright_data-0.31.6}/skills/mr-data-build/references/sources.md

@@ -58,9 +58,10 @@ Write it as a source whose connector is that coordinate plus a `collection` memb
   (up to 50 000, or up to 4 000 000 on a per-row request collection), `concurrency` (up to
   10 000), `request_timeout_seconds`, `retries` and a `revisit` rule. `retries` is 0 to 3 and 2
   when left out. Registration refuses 4 or more. Its pace is `min_interval_milliseconds` in whole
-  milliseconds, 16 or more and 1000 when left out, or the legacy whole-second
-  `min_interval_seconds`. Only `allowed_origins` is required. Every other member has a stated
-  meaning when it is left out. [Pace and width](#pace-and-width) says which to state.
+  milliseconds. Left out, a collection starts at 1000 ms and adapts down to 250 ms. The legacy
+  whole-second `min_interval_seconds` is the other spelling. Only `allowed_origins` is required.
+  Every other member has a stated meaning when it is left out. [Pace and width](#pace-and-width)
+  says which to state.
 - `limits.max_requests` is REQUIRED and bounds every request the run makes, discovery and detail
   pages together. `limits.max_source_bytes` bounds the fetched bytes; `limits.max_rows` clamps the
   merged relation.
@@ -71,37 +72,43 @@ Write it as a source whose connector is that coordinate plus a `collection` memb
 
 **The pace sets the rate. Concurrency does not.**
 
-- A collection that states no pace sends one request a second. At that rate 10 000 pages take
-  about 2.8 hours.
-- State the gap the publisher documents. No documented limit: state 1000 ms and say so. That is
-  a valid reason. Never guess a smaller gap.
-- A stated gap is your declaration. `mr-data recipe explain` reports only an omitted one.
-- Registration refuses a gap under 16 ms by name. That is a platform bound, never a target.
-- Where the deployment spreads a collection's requests over its egress addresses, the gap is
-  kept per address. The publisher can then see a multiple of the stated rate. That holds for
-  every per-row request with no platform credential, your own key included. No command prints
-  how many addresses there are. When the publisher's limit is per key or per account, state
-  1000 ms unless the user confirms the deployment does not spread requests.
+- A collection that states no pace starts at 1000 ms and adapts down to 250 ms. A stated gap is
+  a hard floor that never adapts. At 1000 ms, 10 000 pages take about 2.8 hours.
+- No documented limit: omit pages.min_interval_milliseconds. State a number only when the
+  publisher documents one. Never guess a gap the publisher did not publish.
+- A stated gap at or above the registration floor is in use. `mr-data recipe explain` names a
+  gap under that floor, and does not ask you to fill in an omitted one.
+- Registration refuses a gap under 4 ms by name, which is 256 requests a second to one host
+  from a collection. That is a platform bound, never a target.
+- Where the deployment spreads a collection's requests over its egress addresses, an omitted
+  pace is held to 4 requests a second per address and to max(4, M) a second host-wide when
+  spread over M addresses. A stated gap is kept per address, so the publisher can see a multiple
+  of that rate. That holds for every per-row request with no platform credential, your own key
+  included. No command prints how many addresses there are. When the publisher's limit is per
+  key or per account, state the gap it documents.
 - The worker holds the gap between two requests to one host whatever `pages.concurrency` admits.
   A wider window never sends faster than the pace.
-- Leave `pages.concurrency` out. Left out, the worker uses the capacity it is admitted and picks
-  the width. A stated value is a ceiling on that width. State one only when a run's refusal told
-  you to lower it or the publisher documents a limit on concurrent connections. Record the
-  reason in a dataset note, so the next session finds it.
+- Leave `pages.concurrency` out, or state it only at 64 or above. Left out, the worker uses the
+  capacity it is admitted and picks the width. A stated width strictly under 64 is a ceiling on
+  that width, and `run --full` refuses it unless `--accept-slow` says why. `concurrency: 1`
+  stays that miss. State a lower ceiling only when a run's refusal told you to lower it or the
+  publisher documents a limit on concurrent connections. Record the reason in a dataset note,
+  so the next session finds it.
 - Above 10 000 fetches in one run, `pages.max_fetches_per_run` times the gap must stay within
   25 200 seconds. That is the eight-hour run lease less a one-hour reserve. Registration refuses
-  a pair that waits longer, by name. With no documented limit the gap stays 1000 ms, which
-  admits 25 200 fetches. State fewer fetches per run. Do not state a smaller gap to fit more.
+  a pair that waits longer, by name. An omitted gap is priced at 1000 ms for that check, which
+  admits 25 200 fetches. A larger budget states the gap the publisher documents, or fewer
+  fetches per run.
 - A platform credential's binding bounds the rate from above. The declared gap still applies
-  under it, and an omitted one still means one request a second. `mr-data recipe forecast`
-  prints the binding's rate and `effective_rps` once the recipe is registered. State the gap
-  down to that rate. When the forecast prints no binding rate, state 1000 ms.
+  under it, and an omitted pace still starts at 1000 ms and adapts down to 250 ms under that
+  cap. `mr-data recipe forecast` prints the binding's rate and `effective_rps` once the recipe
+  is registered.
 - State one spelling. Remove the legacy `min_interval_seconds` when you state
   `min_interval_milliseconds`.
 
-`mr-data recipe explain` reports both under `fast_features`. `pace` names a collection that states
-no gap. `concurrency` names any stated width. `mr-data recipe forecast` prints the rate and the
-runtime at the declared ceiling.
+`mr-data recipe explain` reports both under `fast_features`. `pace` names a stated gap under the
+registration floor. An omitted gap is in use. `concurrency` names a stated width strictly under
+64. `mr-data recipe forecast` prints the rate and the runtime at the declared ceiling.
 
 **`limits.max_requests` is charged per HTTP request, redirect hops and retries included** — a page
 that redirects once costs two. For a collection that lists pages, registration only checks that