mostlyright-data 0.28.1__tar.gz → 0.29.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (122) hide show
  1. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/PKG-INFO +11 -11
  2. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/README.md +10 -10
  3. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/pyproject.toml +1 -1
  4. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/SKILL.md +29 -13
  5. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/2-brief-two-to-four-questions-each-with-a-recommended-answer.md +3 -3
  6. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/3-probe-read-a-source-before-committing-to-it.md +2 -1
  7. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/6-build-one-run-sized-to-acquire-every-measured-source-whole.md +18 -15
  8. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/9-present-only-what-survived-inspection-with-caveats.md +10 -10
  9. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/autonomous-delivery.md +7 -7
  10. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/cloud-authentication-preflight.md +12 -0
  11. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/commands.md +18 -11
  12. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/live-run.md +2 -1
  13. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/making-a-table-incremental.md +6 -5
  14. mostlyright_data-0.29.1/skills/mr-data-build/references/preparing-table-search.md +143 -0
  15. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/promote.md +1 -1
  16. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/sources.md +1 -1
  17. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/recipe.py +6 -4
  18. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/router.py +7 -0
  19. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4.py +16 -4
  20. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_query.py +2 -1
  21. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_runs.py +11 -9
  22. mostlyright_data-0.29.1/src/mostlyright/data_harness/thin/v4_search.py +279 -0
  23. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_tables.py +6 -6
  24. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/vocabulary.py +9 -4
  25. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/attendance.py +1 -1
  26. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/clarification.py +1 -1
  27. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/commands/auth.py +70 -11
  28. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/commands/login.py +11 -16
  29. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/login.py +32 -0
  30. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/remediation.py +18 -1
  31. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/render.py +8 -7
  32. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/.gitignore +0 -0
  33. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/scripts/hatch_build.py +0 -0
  34. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/agents/openai.yaml +0 -0
  35. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/1-open-the-page-and-the-link-to-it-in-the-first-message.md +0 -0
  36. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/4-decide-say-what-you-chose-what-you-refused-and-ask-one-question.md +0 -0
  37. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/5-draft-one-recipe-document-one-call.md +0 -0
  38. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/7-interrogate-ask-the-run-what-it-actually-delivered.md +0 -0
  39. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/8-fix-revise-the-document-and-register-it-again.md +0 -0
  40. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/agent-protocol.md +0 -0
  41. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/before-the-first-tool-call.md +0 -0
  42. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/boundaries.md +0 -0
  43. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/cross-repository-protocol-reference.md +0 -0
  44. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/installation-parity.md +0 -0
  45. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/narrating-the-run.md +0 -0
  46. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/not-hosted-yet.md +0 -0
  47. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/one-install.md +0 -0
  48. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/prediction-labels.md +0 -0
  49. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/readers.md +0 -0
  50. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/receipts.md +0 -0
  51. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/recording-a-stream-venue.md +0 -0
  52. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/recovering-an-import-failure.md +0 -0
  53. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/reference-pages.md +0 -0
  54. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/required-protocol.md +0 -0
  55. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/source-credentials.md +0 -0
  56. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/the-one-thing-to-say-about-the-skill-itself.md +0 -0
  57. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/transforms.md +0 -0
  58. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/user-communication-contract.md +0 -0
  59. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/references/writing-a-decision-record.md +0 -0
  60. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/skills/mr-data-build/scripts/write_research_notebook.py +0 -0
  61. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/__init__.py +0 -0
  62. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/agent_protocol.py +0 -0
  63. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/canonical.py +0 -0
  64. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/formats.py +0 -0
  65. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/hosted_crawler_protocol.py +0 -0
  66. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/key_seam.py +0 -0
  67. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/page_coverage.py +0 -0
  68. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/part_check_evidence.py +0 -0
  69. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/session_probes.py +0 -0
  70. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/skill_assets.py +0 -0
  71. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/table_manifest.py +0 -0
  72. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/__init__.py +0 -0
  73. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/acquire.py +0 -0
  74. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/acquire_cancel.py +0 -0
  75. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/activity.py +0 -0
  76. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/approvals.py +0 -0
  77. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/categories.py +0 -0
  78. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/commands.py +0 -0
  79. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/dataset-categories-v1.json +0 -0
  80. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/download.py +0 -0
  81. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/fast_features.py +0 -0
  82. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/narrative.py +0 -0
  83. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/parity.py +0 -0
  84. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/probe.py +0 -0
  85. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/progress_rate.py +0 -0
  86. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/progress_vocabulary.py +0 -0
  87. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/propose.py +0 -0
  88. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/recipe_brief.py +0 -0
  89. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/recipe_explain.py +0 -0
  90. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/recipe_forecast.py +0 -0
  91. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/recipe_lint.py +0 -0
  92. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/research.py +0 -0
  93. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/run_diagnose.py +0 -0
  94. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/runs.py +0 -0
  95. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/session.py +0 -0
  96. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/stream.py +0 -0
  97. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/stream_venue.py +0 -0
  98. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/studio_capabilities.py +0 -0
  99. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/transport.py +0 -0
  100. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/user_agent.py +0 -0
  101. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_artifacts.py +0 -0
  102. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_catalog.py +0 -0
  103. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_connected.py +0 -0
  104. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_connections.py +0 -0
  105. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_dataset_covers.py +0 -0
  106. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_datasets.py +0 -0
  107. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_handoff.py +0 -0
  108. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_narrative.py +0 -0
  109. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_public.py +0 -0
  110. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_reader.py +0 -0
  111. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_secrets.py +0 -0
  112. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/thin/v4_stream.py +0 -0
  113. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/__init__.py +0 -0
  114. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/cloud_auth.py +0 -0
  115. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/commands/__init__.py +0 -0
  116. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/commands/clarify.py +0 -0
  117. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/commands/whoami.py +0 -0
  118. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/credential_native.py +0 -0
  119. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/credential_store.py +0 -0
  120. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/credentials.py +0 -0
  121. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/path_kind.py +0 -0
  122. {mostlyright_data-0.28.1 → mostlyright_data-0.29.1}/src/mostlyright/data_harness/ux/plain_file.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: mostlyright-data
3
- Version: 0.28.1
3
+ Version: 0.29.1
4
4
  Summary: Mostly Right hosted CLI for reviewed datasets
5
5
  Project-URL: Homepage, https://mostlyright.md/
6
6
  Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/
@@ -82,16 +82,16 @@ mr-data checks RUN_ID
82
82
  mr-data download RUN_ID --output ./out
83
83
  ```
84
84
 
85
- Full builds are progressive by default: one acquisition exposes an inspection checkpoint after
86
- about five minutes and continues toward the finished table. Inspect it with
85
+ Full builds are progressive by default: one acquisition exposes an inspection checkpoint after about
86
+ five minutes and continues toward the finished table. Inspect it with
87
87
  `mr-data status RUN_ID --inspection --json`; the checkpoint does not pause for approval or start
88
- another full acquisition. Any required spend confirmation happens before acquisition.
89
- When a run has sealed queryable table checkpoints, `mr-data status RUN_ID --checkpoints --json`
90
- lists their exact immutable identifiers and `mr-data query RUN_ID "SELECT …" --checkpoint
91
- CHECKPOINT_ID` reads one without advancing underneath the query. These workspace-private snapshots
92
- remain incomplete, keep checks pending, and never become the serving table.
93
- `--progressive` remains a compatibility alias. Use `--sample` only for an explicit standalone
94
- bounded inspection; row ceilings do not impose a five-minute acquisition limit.
88
+ another full acquisition. Any required size confirmation happens before acquisition; it is not a
89
+ charge (Mostly Right does not charge per build). When a run has sealed queryable table checkpoints,
90
+ `mr-data status RUN_ID --checkpoints --json` lists their exact immutable identifiers and
91
+ `mr-data query RUN_ID "SELECT …" --checkpoint CHECKPOINT_ID` reads one without advancing underneath
92
+ the query. These workspace-private snapshots remain incomplete, keep checks pending, and never
93
+ become the serving table. `--progressive` remains a compatibility alias. Use `--sample` only for an
94
+ explicit standalone bounded inspection; row ceilings do not impose a five-minute acquisition limit.
95
95
 
96
96
  Run submission is idempotent: sending identical recipe, mode and bounds returns the same run,
97
97
  including a previous failure. The CLI reports a non-queued result as `run_returned` and exits
@@ -104,7 +104,7 @@ A scheduled refresh runs only Studio's persisted bounded source-action plan; it
104
104
  falls back to a mutable whole-source fetch. `mr-data table resync TABLE_ID --request-id UUID`
105
105
  explicitly requests a full source reread when that is needed. Choose and retain the UUID before
106
106
  submitting: if the response is lost, repeat that exact command with the same UUID rather than
107
- starting another reread. A persisted spend hold is still an accepted resync: its receipt names
107
+ starting another reread. A persisted size hold is still an accepted resync: its receipt names
108
108
  the run, projection and exact `mr-data run --confirm-held RUN_ID` action. New full resyncs use
109
109
  progressive acquisition too. Existing legacy sample-first pairs can still finish through their
110
110
  original confirmation and approval actions. `mr-data recipe readiness` reads Studio's paginated,
@@ -70,16 +70,16 @@ mr-data checks RUN_ID
70
70
  mr-data download RUN_ID --output ./out
71
71
  ```
72
72
 
73
- Full builds are progressive by default: one acquisition exposes an inspection checkpoint after
74
- about five minutes and continues toward the finished table. Inspect it with
73
+ Full builds are progressive by default: one acquisition exposes an inspection checkpoint after about
74
+ five minutes and continues toward the finished table. Inspect it with
75
75
  `mr-data status RUN_ID --inspection --json`; the checkpoint does not pause for approval or start
76
- another full acquisition. Any required spend confirmation happens before acquisition.
77
- When a run has sealed queryable table checkpoints, `mr-data status RUN_ID --checkpoints --json`
78
- lists their exact immutable identifiers and `mr-data query RUN_ID "SELECT …" --checkpoint
79
- CHECKPOINT_ID` reads one without advancing underneath the query. These workspace-private snapshots
80
- remain incomplete, keep checks pending, and never become the serving table.
81
- `--progressive` remains a compatibility alias. Use `--sample` only for an explicit standalone
82
- bounded inspection; row ceilings do not impose a five-minute acquisition limit.
76
+ another full acquisition. Any required size confirmation happens before acquisition; it is not a
77
+ charge (Mostly Right does not charge per build). When a run has sealed queryable table checkpoints,
78
+ `mr-data status RUN_ID --checkpoints --json` lists their exact immutable identifiers and
79
+ `mr-data query RUN_ID "SELECT …" --checkpoint CHECKPOINT_ID` reads one without advancing underneath
80
+ the query. These workspace-private snapshots remain incomplete, keep checks pending, and never
81
+ become the serving table. `--progressive` remains a compatibility alias. Use `--sample` only for an
82
+ explicit standalone bounded inspection; row ceilings do not impose a five-minute acquisition limit.
83
83
 
84
84
  Run submission is idempotent: sending identical recipe, mode and bounds returns the same run,
85
85
  including a previous failure. The CLI reports a non-queued result as `run_returned` and exits
@@ -92,7 +92,7 @@ A scheduled refresh runs only Studio's persisted bounded source-action plan; it
92
92
  falls back to a mutable whole-source fetch. `mr-data table resync TABLE_ID --request-id UUID`
93
93
  explicitly requests a full source reread when that is needed. Choose and retain the UUID before
94
94
  submitting: if the response is lost, repeat that exact command with the same UUID rather than
95
- starting another reread. A persisted spend hold is still an accepted resync: its receipt names
95
+ starting another reread. A persisted size hold is still an accepted resync: its receipt names
96
96
  the run, projection and exact `mr-data run --confirm-held RUN_ID` action. New full resyncs use
97
97
  progressive acquisition too. Existing legacy sample-first pairs can still finish through their
98
98
  original confirmation and approval actions. `mr-data recipe readiness` reads Studio's paginated,
@@ -1,6 +1,6 @@
1
1
  [project]
2
2
  name = "mostlyright-data"
3
- version = "0.28.1"
3
+ version = "0.29.1"
4
4
  description = "Mostly Right hosted CLI for reviewed datasets"
5
5
  readme = "README.md"
6
6
  requires-python = ">=3.11"
@@ -46,6 +46,14 @@ references. Load the reference for the current stage only, not the entire librar
46
46
  5. State the selected sources, joins, coverage and material limitations. Ask for a decision only
47
47
  if existing authorization does not cover it. Read the
48
48
  [recipe reference](references/5-draft-one-recipe-document-one-call.md) before authoring fields.
49
+ For newly authored tables, read [search preparation](references/preparing-table-search.md).
50
+ Check availability with `mr-data search inspect --recipe RECIPE.json --json`; when supported,
51
+ propose/validate and add explicit `table.search` intent before registration. A brief that does
52
+ not mention search treats it as optional while the deployment lacks this capability; when
53
+ supported, default search intent remains. Follow the reference's fallback and decision-note
54
+ rules. If semantic search or JEV interpretation into filters/sorts is explicitly required,
55
+ do not register or queue a recipe that omits it; report the capability limit and wait.
56
+ Do not add search to existing recipes or rebuild existing datasets without an explicit request.
49
57
  Bind `dataset.id`, describe columns, declare checks and units, and register with
50
58
  `mr-data recipe RECIPE.json --json`. Repair all lint findings before queuing work.
51
59
  The server returns the recipe digest; do not compute or invent it.
@@ -70,18 +78,21 @@ references. Load the reference for the current stage only, not the entire librar
70
78
  only for a deliberate standalone inspection. Preserve the agreed semantics and quality checks.
71
79
  7. Follow the returned run with `mr-data watch RUN_ID --json`. Its `source_rates` and
72
80
  `mr-data status RUN_ID --json` on a live run give each collection source's rate and time left.
73
- Quote those numbers rather than doing the arithmetic. Read `run.status`: returning an
74
- existing run does not mean new work was queued. Resume the known run after an interruption,
75
- using the saved cursor where available. Read [run lifecycle](references/live-run.md) for
76
- holds and continuation commands. Confirm projected spend before acquisition when required;
77
- the progressive checkpoint needs no approval or second full run. Honor the user's
78
- authorization; do not request an existing delegation again. Send a
79
- concise chat update for each newly observed meaningful lifecycle transition: accepted or
80
- queued, released after confirmation, started, failed, retried or replaced, completed, and
81
- verified. Include the run ID so the user can tell attempts apart; state the supported failure
82
- cause, name both the old and new run IDs for a retry or replacement, and include elapsed time or
83
- throughput when the user requested a benchmark. Do not repeat an unchanged poll, and never
84
- treat the app activity pill as a substitute for these updates.
81
+ Quote those numbers rather than doing the arithmetic. Read `run.status`: returning an existing
82
+ run does not mean new work was queued. Resume the known run after an interruption, using the
83
+ saved cursor where available. Read [run lifecycle](references/live-run.md) for holds and
84
+ continuation commands. When Studio holds a large run for a size confirmation, settle it with
85
+ `--confirm`. The hold is a size check, not a charge: Mostly Right never charges per build, so
86
+ never ask the user to pay or confirm a cost for the hold. A source whose own provider bills per
87
+ call is a separate, real cost: forecast it and ask before re-reading it, as the paid-source rules
88
+ above say. The progressive checkpoint needs no approval or second full run. Honor the user's
89
+ authorization; do not request an existing delegation again. Send a concise chat update for each
90
+ newly observed meaningful lifecycle transition: accepted or queued, released after confirmation,
91
+ started, failed, retried or replaced, completed, and verified. Include the run ID so the user can
92
+ tell attempts apart; state the supported failure cause, name both the old and new run IDs for a
93
+ retry or replacement, and include elapsed time or throughput when the user requested a benchmark.
94
+ Do not repeat an unchanged poll, and never treat the app activity pill as a substitute for these
95
+ updates.
85
96
 
86
97
  ## Recover without guesswork
87
98
 
@@ -133,7 +144,11 @@ merely to obtain a green run. Read [recovery details](references/agent-protocol.
133
144
  repair, which is why the decision belongs in step 5. A table left not refresh-ready never
134
145
  refreshes unless it declares `rebuild`; it does not become ready later.
135
146
  Download and verify artifacts when the user requested bytes.
136
- 10. Report rows and coverage actually verified, checks, limitations and the stable dataset link.
147
+ 10. For new tables with search intent, inspect search readiness and test meaningful queries against
148
+ the observed version as [search preparation](references/preparing-table-search.md) describes.
149
+ Search preparation runs independently: pending or failed indexing never means the data build
150
+ failed, and declared metadata alone never proves observed search quality.
151
+ Report rows and coverage actually verified, checks, limitations and the stable dataset link.
137
152
  Read [delivery](references/9-present-only-what-survived-inspection-with-caveats.md) for details.
138
153
 
139
154
  ## Source-specific references
@@ -151,6 +166,7 @@ merely to obtain a green run. Read [recovery details](references/agent-protocol.
151
166
  | How each source continues, `rebuild`, and the cost of a refresh | [Making a table incremental](references/making-a-table-incremental.md) |
152
167
  | Refresh cadence | [Promotion and cadence](references/promote.md) |
153
168
  | Dataset notes and evidence blocks | [Decision records](references/writing-a-decision-record.md) |
169
+ | Search configuration, example queries and readiness | [Search preparation](references/preparing-table-search.md) |
154
170
  | Exact schemas and worked recipes | [Reference index](references/reference-pages.md) |
155
171
 
156
172
  ## Needs Studio 4.14
@@ -65,9 +65,9 @@ mr-data dataset note DATASET_ID --heading "The brief is delegated to the agent"
65
65
  the same identifier, revising it rather than stacking a second — the one-command form for a
66
66
  delegation given after stage 2.
67
67
 
68
- **Do not re-ask delegated decisions.** State choices within that scope and the spend projection
69
- when authorized. Ask only what the delegation did not settle; a narrower delegation must not
70
- silently become permission to schedule recurring work.
68
+ **Do not re-ask delegated decisions.** State choices within that scope and the projected size when
69
+ authorized (never a price: Mostly Right does not charge per build). Ask only what the delegation did
70
+ not settle; a narrower delegation must not silently become permission to schedule recurring work.
71
71
 
72
72
  **Registration refuses a dataset nobody was asked about.** `mr-data recipe` reads the dataset's own
73
73
  record and answers `THIN_BRIEF_MISSING` when it holds no answered `clarification` and no delegation
@@ -104,7 +104,8 @@ not promise a short acquisition. Never derive source size from a truncated sampl
104
104
 
105
105
  **A reading run that came back whole IS the build, so stage 6 adopts it rather than repeating
106
106
  it.** Untruncated everywhere is the build, whichever stage started the run: keep this `run_id`.
107
- The run meets the ordinary spend confirmation before acquisition; its checkpoint requires no
107
+ The run meets Studio's ordinary size confirmation before acquisition when it is large (a size
108
+ check, not a charge); its checkpoint requires no
108
109
  second approval or second full build.
109
110
 
110
111
  That is stages 5 through 7 run once to settle the shape. Where the deployment does answer a probe,
@@ -15,9 +15,9 @@ mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --json
15
15
 
16
16
  After about five minutes, inspect `mr-data status RUN_ID --inspection --json` while the same
17
17
  acquisition continues. This checkpoint is read-only, not a second approval gate or a hard runtime
18
- limit. A short build can finish before a checkpoint is needed. Do not submit another full run
19
- after inspecting it. Required spend confirmation happens before acquisition. The older
20
- `--progressive` flag is accepted for compatibility but is unnecessary.
18
+ limit. A short build can finish before a checkpoint is needed. Do not submit another full run after
19
+ inspecting it. A required size confirmation happens before acquisition; it is not a charge. The
20
+ older `--progressive` flag is accepted for compatibility but is unnecessary.
21
21
 
22
22
  **A standalone sample is an explicit inspection choice.** Use `--sample --max-rows 2000` only
23
23
  when a separate bounded experiment is needed. Rows are clamped after acquisition, so a collection
@@ -48,8 +48,9 @@ sample. A full run reads such a source from its `start_at`. A source whose `star
48
48
  current UTC day has no completed day until that day is over, so neither a sample nor a full run of
49
49
  it reads anything before then.
50
50
 
51
- If existing authorization or a stage-2 delegation covers this full build and its spend, do not
52
- ask again: state the projection and run it. A narrower delegation does not authorize the full run.
51
+ If existing authorization or a stage-2 delegation covers this full build, do not ask again: state
52
+ the projected size and run it. There is no per-build payment to ask about. A narrower delegation
53
+ does not authorize the full run.
53
54
 
54
55
  ```sh
55
56
  mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --json
@@ -105,14 +106,14 @@ mr-data run --retry RUN_ID --json
105
106
 
106
107
  It re-states that run's own recipe, digest, mode and ceilings under one byte ceiling above what
107
108
  the recipe declares, because Studio replays a terminal result for an identical command and the
108
- coordinate has to move — and the number that ceiling must clear is what the run is projected to
109
- cost, not just what it may fetch. It refuses, with the triple, when the failure was the recipe's,
109
+ coordinate has to move — and the number that ceiling must clear is how large the run is projected
110
+ to be, not just what it may fetch. It refuses, with the triple, when the failure was the recipe's,
110
111
  which is the signal to repair the document instead. It also refuses three runs with no coordinate
111
112
  left to move: a replay, whose ceilings must equal its source run's; a run stating a byte ceiling
112
- of its own below that floor, because raising it would fetch bytes the first attempt could not and could
113
- carry the run under a spend gate it was held at; and half of a sample-first pair, whose preview
114
- retried is a standalone run the held full never learns about. Retry a room fault at most three
115
- times; report each new run once and keep unchanged polling quiet.
113
+ of its own below that floor, because raising it would fetch bytes the first attempt could not and
114
+ could carry the run under a size gate it was held at; and half of a sample-first pair, whose
115
+ preview retried is a standalone run the held full never learns about. Retry a room fault at most
116
+ three times; report each new run once and keep unchanged polling quiet.
116
117
  - **Anything that changes what the data MEANS** — a source that cannot be reached at all, a grain
117
118
  the feeds cannot deliver, a column that has to be dropped — is a stage 2 question, asked the
118
119
  stage 2 way, and never a silent substitution.
@@ -199,11 +200,13 @@ make every source or connector incremental.
199
200
  complete acquired response is told from one cut at fifty thousand rows without opening the
200
201
  receipt: read `coverage` off the run, not the row count off the table.
201
202
 
202
- A run over the spend threshold is not started. It answers with the projected bytes, runtime and
203
- cost, and waits; the same command again with `--confirm` settles it against the number already on
203
+ A run over Studio's size threshold is not started. It answers with the projected bytes and runtime
204
+ (and Studio's internal compute estimate, which is not billed: Mostly Right does not charge per
205
+ build), and waits; the same command again with `--confirm` settles it against the number already on
204
206
  the record — `mr-data run --recipe RECIPE_ID --digest RECIPE_DIGEST --full --confirm`, which the
205
- held answer prints back as `confirm_command`. Any authenticated surface of the paying workspace may
206
- settle it. `mr-data run --cancel RUN_ID` stops a run that is queued or running.
207
+ held answer prints back as `confirm_command`. Any authenticated surface of the subscribed workspace
208
+ may settle it. Never present it to the user as a cost or a payment. `mr-data run --cancel RUN_ID`
209
+ stops a run that is queued or running.
207
210
 
208
211
  Do not treat the accepted command vocabulary as execution evidence. Before proposing a schedule,
209
212
  classify every source under one truthful continuation strategy and record the supporting evidence,
@@ -30,29 +30,29 @@ was explicitly delegated. Report the effective schedule and whether it is still
30
30
 
31
31
  **Truncated anywhere is a preview, and this is the one place the build pauses.** State what the
32
32
  preview established, name the source that was cut and the row it stopped at, say what a full build
33
- would cover, and ask: it costs real time and money and it is a decision about the user's data. That
34
- decision and spend confirmation may already be settled by a delegation at stage 2; honor its
35
- scope. Cadence requires the update answer or explicit refresh delegation described above. Stage 6
36
- carries the run commands.
33
+ would cover, and ask: it costs real time and it is a decision about the user's data (not a payment:
34
+ Mostly Right does not charge per build). That decision and any size confirmation may already be
35
+ settled by a delegation at stage 2; honor its scope. Cadence requires the update answer or explicit
36
+ refresh delegation described above. Stage 6 carries the run commands.
37
37
 
38
38
  **The same decision can arrive as a run you did not start.** A person can start the full build from
39
39
  the dataset page, so before acting on this recipe again read `mr-data runs --json --mode full` and,
40
40
  for any run newer than the preview that you did not start, `mr-data status RUN_ID --json` to see
41
41
  whether it names this recipe and digest. One that does is theirs, and its state says what to do.
42
42
  `queued`, `running` or `succeeded` is the full build: adopt it as if you had started it — write its
43
- run cell, `mr-data watch RUN_ID --json` if it has not ended, then stage 7.
44
- `awaiting_confirmation` is the spend gate, which the same command settles with `--confirm`.
43
+ run cell, `mr-data watch RUN_ID --json` if it has not ended, then stage 7. `awaiting_confirmation`
44
+ is the size gate (not a charge), which the same command settles with `--confirm`.
45
45
  `awaiting_sample_approval` is stage 6's held full: read its `sample_run_id`, interrogate that
46
46
  preview, and release it with `mr-data run --approve-full`. `failed` is the full build too: read its
47
47
  triple and structured recovery. A known retry case can use `mr-data run --retry RUN_ID`; otherwise
48
48
  preserve the document until the evidence identifies a repair. A bare run of the same coordinate
49
- replays that failure. `cancelled` means they
50
- withdrew the decision, so say so and ask nothing further unless they raise it. Ask, then give the
51
- turn back; do not hold a stream open waiting for either answer.
49
+ replays that failure. `cancelled` means they withdrew the decision, so say so and ask nothing
50
+ further unless they raise it. Ask, then give the turn back; do not hold a stream open waiting for
51
+ either answer.
52
52
 
53
53
  Then interrogate that run as stage 7 says, and present it with the preview's caveats removed and
54
54
  its own read off `coverage`. Do not start a full build because the preview looked good or because a
55
- dataset was asked for — the decision is whether THIS is the recipe to spend one on.
55
+ dataset was asked for — the decision is whether THIS is the recipe to build.
56
56
 
57
57
  **Either way, offer the bytes.** A sealed table is downloadable whatever mode produced it —
58
58
  `mr-data download RUN --output DIR --json` — so somebody who wants to look before agreeing to
@@ -4,13 +4,13 @@ Own the run until the requested outcome is delivered: a run that succeeded, its
4
4
  read, its coverage inspected against the shape stage 2 settled, and its refresh cadence recorded.
5
5
  What is live is what Studio returns as live; catch-up and continuing freshness require separate
6
6
  durable evidence from the deployed component that owns them. The brief's questions, a material
7
- semantic revision, the cadence, a full build after a truncated preview and a spend confirmation are
8
- the valid user pauses. A delegation settles only decisions within its stated scope; ongoing
9
- refresh requires an update answer or explicit refresh delegation. Own the
10
- operational choices: Readers, engines, parsing formats, workspace plumbing, retry tactics, and
11
- every repair that preserves the meaning of the data. Source preferences and cleaning choices that
12
- change the meaning belong in the user conversation, even when the agent could technically choose
13
- for them.
7
+ semantic revision, the cadence, a full build after a truncated preview and a size confirmation of a
8
+ large run are the valid user pauses (none of them is a payment: Mostly Right does not charge per
9
+ build). A delegation settles only decisions within its stated scope; ongoing refresh requires an
10
+ update answer or explicit refresh delegation. Own the operational choices: Readers, engines, parsing
11
+ formats, workspace plumbing, retry tactics, and every repair that preserves the meaning of the data.
12
+ Source preferences and cleaning choices that change the meaning belong in the user conversation,
13
+ even when the agent could technically choose for them.
14
14
 
15
15
  **`mr-data clarify` does not replace the user conversation.** Pass `--attended` when a person
16
16
  is available in chat, or `--unattended` for a scheduled invocation,
@@ -19,6 +19,18 @@ present, carry on to the cloud command and let that command's credential resolve
19
19
  override; when it is absent, run `mr-data login`, complete the device approval, rerun
20
20
  `mr-data whoami --json`, and require an authenticated status before the cloud command.
21
21
 
22
+ A credential belongs to one workspace for its whole life. An account in more than one workspace
23
+ is asked which on an interactive terminal; with `--json`, or with nobody at the terminal, login is
24
+ refused `LOGIN_WORKSPACE_REQUIRED` and the message lists the workspace names. Ask the user which
25
+ workspace this machine is for, then rerun with `--workspace NAME` — `mr-data login` and
26
+ `mr-data auth login` both take it, with the same meaning. Never choose a workspace for the user.
27
+ `LOGIN_DEVICE_KEY_LIMIT` means the user already holds as many active CLI keys in that workspace
28
+ as the cloud allows: tell them to revoke one they no longer use (in the dashboard, with that
29
+ workspace selected, under Account → Security; or `mr-data auth logout` on a machine they are
30
+ finished with), and do not retry until they have. Then rerun the command that was refused:
31
+ `mr-data login` for a first sign-in, `mr-data auth rotate` or `mr-data login --force` for a
32
+ replacement, because a rotation that hits the limit leaves this machine's credential in place.
33
+
22
34
  The hosted lane requires an `mr_cli_` device credential; a read-only live data key is refused with
23
35
  `THIN_AUTHENTICATION_FAILED` rather than failing three calls later.
24
36
  `mr-data export-hosted-candidate` is a backend operator step and needs neither this preflight nor a
@@ -14,7 +14,7 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
14
14
  | `mr-data dataset` | Bring the dataset page into existence before there is anything on it, then fill it in while somebody watches. `dataset create --name TEXT` mints it and prints the `dataset_id`, unless the user's own request already has an empty dataset open, in which case it answers `open_request_reused` with that `dataset_id` and mints nothing (`--separate` mints a new one for a different dataset); `dataset show ID` reads it back; `dataset set ID --name TEXT --topics "a,b,c" --license ID --description-file F` writes the title, the descriptive tags, the SPDX licence and the description under the version it was read at, retrying once if somebody else wrote first, and an empty `--topics` or `--license` takes that value off the page; send `--name` and `--topics` whenever you send `--description-file`, because a description sent on its own can make Studio write a new title and new tags, and read `changed` and `written_by_studio` on the receipt for every field the write moved; a saved write whose public sync fails exits 2 and reports `public_projection_synced: false` — use `dataset sync ID` to retry that sync without rewriting Studio; `dataset note ID --heading H --blocks-file B` writes one cell of the decision record that OUTLIVES every run, and `--list` reads it back; `dataset watch ID` follows the page's own event stream; `dataset activity ID --phase P --message TEXT` says what is happening right now, silently, and is never a chat message and never a cell; `dataset publish ID [--mode public|link|private]` says who can read the dataset — `public` lists it in the public directory and serves it at an address anybody can read, `link` serves it at an unlisted address, `private` takes it back to the workspace — and `dataset publish ID --show` reads that back without changing it; `dataset archive ID --confirm-name TITLE` retires the page and frees its title, deleting nothing. |
15
15
  | `mr-data recipe` | Register one recipe document — dataset, question, table plan, sources, transform, checks and units — in one call, and print the identifiers the server derived. The document is read first and every fault comes back in one refusal, each with the JSON pointer that names it; `--no-lint` sends it as written instead. Registration refuses `THIN_BRIEF_MISSING` when the dataset's record carries no answered question and no delegation, and `THIN_RECIPE_DATASET_UNBOUND` when the document names no `dataset.id`; `--delegated "their words"` writes the delegation onto the dataset and registers in the same command, and clears the first of those two and never the second. `mr-data recipe show ID` reads one back. `mr-data recipe explain FILE` (or a recipe identifier) lists every reason its table can or cannot refresh, each with a rule code and Studio's suggested repair, and exits 2 when one blocks it. `mr-data recipe forecast RECIPE_ID --digest DIGEST` says what one run would ask of each source before it runs, and exits 3 when the run would overrun today's platform quota or one attempt. `mr-data recipe readiness --table TABLE_ID --current-only` reads the revision one table runs now. [Making a table incremental](making-a-table-incremental.md#the-loop-explain-forecast-run-diagnose) says when to use each. |
16
16
  | `mr-data cover` | Generate and attach one branded 1200×630 dataset cover. Give it the dataset ID and what the image should depict; Studio fixes the model, single-color style, curated random palette, dimensions and storage. |
17
- | `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID --digest RECIPE_DIGEST` — both required, neither positional, and the digest is the bare hex the registration receipt printed. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh executes only Studio's persisted source-action plan, and a plan shape it does not refresh returns `RESYNC_REQUIRED` before acquisition; [which plans refresh](making-a-table-incremental.md#which-plans-refresh) lists them, and `mr-data recipe explain` says which one a table is. Use `mr-data table resync TABLE_ID --request-id UUID` for an explicit full source reread, retaining that UUID if the response is uncertain. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal — report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. A large run is held for a spend confirmation, which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried. `--approve-full RUN_ID` releases the full an existing legacy sample-first pair is holding once its preview has succeeded, naming either half of the pair; where the deployment still wants a person at a browser it answers `THIN_INTERACTIVE_HUMAN_REQUIRED` and names the run page. `--retry RUN_ID` tries one failed run again when its triple carries `room_fault: true`, re-stating that run's own coordinate under a byte ceiling that cannot narrow it, and refuses with the triple when the recipe was at fault. `--cancel RUN` stops one. |
17
+ | `mr-data run` | Start one run against a registered recipe, named as `--recipe RECIPE_ID --digest RECIPE_DIGEST` — both required, neither positional, and the digest is the bare hex the registration receipt printed. The mode is one of six: `sample`, `full`, `refresh`, `backfill`, `compact` and `replay`. Four have a shorthand flag (`--sample`, `--full`, `--refresh`, `--backfill`) and two do not, so write `--mode MODE`, which is accepted for every one of them. A refresh executes only Studio's persisted source-action plan, and a plan shape it does not refresh returns `RESYNC_REQUIRED` before acquisition; [which plans refresh](making-a-table-incremental.md#which-plans-refresh) lists them, and `mr-data recipe explain` says which one a table is. Use `mr-data table resync TABLE_ID --request-id UUID` for an explicit full source reread, retaining that UUID if the response is uncertain. `--backfill` states the exact window with `--window START END`. `--mode replay --sources-from RUN_ID` asks Studio to run the registered revision against the RETAINED RAW INPUTS of one named successful run of the same table: nothing is fetched from this computer, nothing is acquired again, the result is compared against that run and never becomes the live version, and it neither asks for nor records an approval. Where Studio has replay switched off, or the named run is not successful, not the same table, or no longer retains its inputs, it answers a typed refusal — report the code rather than retrying in another mode. `--sources-from` on any other mode is refused `THIN_ARGUMENT_INVALID` before anything is sent. `--max-rows` and `--max-source-bytes` bound ONE SOURCE rather than the finished table. A large run is held for a size confirmation (not a charge), which `--confirm` settles and whose printed `confirm_command` re-states every argument the request carried. `--approve-full RUN_ID` releases the full an existing legacy sample-first pair is holding once its preview has succeeded, naming either half of the pair; where the deployment still wants a person at a browser it answers `THIN_INTERACTIVE_HUMAN_REQUIRED` and names the run page. `--retry RUN_ID` tries one failed run again when its triple carries `room_fault: true`, re-stating that run's own coordinate under a byte ceiling that cannot narrow it, and refuses with the triple when the recipe was at fault. `--cancel RUN` stops one. |
18
18
  | `mr-data status` | Report which of the seven states one run is in, what it delivered and whether a ceiling cut it short, and — on a failure — what failed, where, and whether the fault was the execution room's rather than the recipe's. Exits non-zero on a failed run. `--diagnose` adds everything else Studio can say about the run: its progress, a per-row source's plan counts, its checkpoints, and how each attempt ended, with a lost attempt's cause marked as an inference. |
19
19
  | `mr-data runs` | Report this workspace's own runs: identifier, mode, state and creation time, plus the failure triple of any that failed and whether that failure was the execution room's. `--status` and `--mode` narrow it and are refused `THIN_ARGUMENT_INVALID` for a word outside the seven states or the six modes; `--limit` says how many, and pages are followed to reach it. A narrow filter over a workspace of thousands of runs reads a long way to find its matches, and a listing that does not end says how many were read and suggests a smaller `--limit`. The answer prints `next_cursor`, and `--cursor` continues from it. |
20
20
  | `mr-data watch` | Stream one run's live progress under the durable event type of each stage, resuming across stream cuts. Exits non-zero when the run failed. |
@@ -38,16 +38,16 @@ gap rather than doing anything, and `export-hosted-candidate`, which is a backen
38
38
  | `mr-data unpin` | Ask Studio to resume pointer tracking; read the returned state before claiming it resumed. |
39
39
  | `mr-data demote` | Ask Studio to withdraw the pointer and any schedule it owns, for one table or for as many as you name: `demote TABLE [TABLE ...]` withdraws them one after another, attempts every one of them whatever the one before it answered, prints a line for each, and exits non-zero if any is still live. Pair it with `table archive` when you are retiring a set: a live table cannot be archived, so it is withdraw-then-archive, two commands rather than a loop. |
40
40
 
41
- Every `mr-data run --full` or `--mode full` build is progressive by default. The old
42
- `--progressive` flag remains a compatibility alias. Studio admits one full run and keeps acquisition moving while the
43
- owner inspects its five-minute checkpoint. A spend confirmation, when required, still comes
44
- before acquisition. Read `mr-data status RUN_ID --inspection --json` for the declared columns,
45
- missing-data policy, every source's bounded progress, and any observed table evidence. Source
46
- relation rows are not finished table rows; unknown transformed values remain null. A failed or
47
- cancelled progressive run may be followed by a new
48
- `--mode full --resume-capture-run-id RUN_ID` request under a registered compatible
49
- recipe. Studio decides which sealed captures can be reused. Keep the returned new run ID and
50
- verify its final receipt.
41
+ Every `mr-data run --full` or `--mode full` build is progressive by default. The old `--progressive`
42
+ flag remains a compatibility alias. Studio admits one full run and keeps acquisition moving while
43
+ the owner inspects its five-minute checkpoint. A size confirmation, when Studio requires one, still
44
+ comes before acquisition; it is not a charge. Read `mr-data status RUN_ID --inspection --json` for
45
+ the declared columns, missing-data policy, every source's bounded progress, and any observed table
46
+ evidence. Source relation rows are not finished table rows; unknown transformed values remain null.
47
+ A failed or cancelled progressive run may be followed by a new
48
+ `--mode full --resume-capture-run-id RUN_ID` request under a registered compatible recipe. Studio
49
+ decides which sealed captures can be reused. Keep the returned new run ID and verify its final
50
+ receipt.
51
51
 
52
52
  ### Strict refresh controls
53
53
 
@@ -122,3 +122,10 @@ performs. Most of the survivors take a run identifier where the deleted local tw
122
122
  a hosted build has no folder, and the run identifier is what the submission printed and what the
123
123
  dashboard address contains. The deleted names and their replacements are written down only in the
124
124
  Harness repository's own `docs/HOSTED-PARITY.md`, which is not installed beside this document.
125
+
126
+ ## Search preparation
127
+
128
+ `mr-data search <verb>` prepares and tests search on new opted-in tables. See
129
+ [search preparation](preparing-table-search.md) and `mr-data search --help` for the supported
130
+ operations. `inspect`, `propose` and `validate` accept `--recipe FILE` before registration; these
131
+ report declared schema only. Index work runs in Studio and does not hold up data publication.
@@ -59,7 +59,8 @@ its row ceiling does not bound acquisition time.
59
59
 
60
60
  **Seven states, including legacy holds.** A run is `queued`, `running`, `awaiting_confirmation`,
61
61
  `awaiting_sample_approval`, `succeeded`, `failed` or `cancelled`; three are terminal and end a
62
- watch. `awaiting_confirmation` is the ordinary spend gate: nothing has been executed, and the same
62
+ watch. `awaiting_confirmation` is the ordinary size gate (not a charge: Mostly Right does not charge
63
+ per build): nothing has been executed, and the same
63
64
  run command with `--confirm` on the end of it releases it. `awaiting_sample_approval` is the held
64
65
  full of an existing legacy sample-first pair: its bounded preview must seal a downloadable table
65
66
  first, and then
@@ -212,7 +212,8 @@ run evidence support a claim that a table is refreshing.
212
212
  ### Rebuild: the fifth continuation
213
213
 
214
214
  The other continuations say how to fetch less. `rebuild` says truthfully that there is no less. It
215
- is a table-level member, and it is the only consent Studio takes to spend on a scheduled full read:
215
+ is a table-level member, and it is the only standing consent Studio takes to acquire a scheduled
216
+ full read:
216
217
 
217
218
  ```json
218
219
  "rebuild": {"mode": "whole_table", "max_rebuild_bytes": 270000000, "min_interval_seconds": 604800}
@@ -223,7 +224,7 @@ is a table-level member, and it is the only consent Studio takes to spend on a s
223
224
  - **Size `max_rebuild_bytes` at 1.5 times a measured run.** Take the bytes the last full acquired:
224
225
  the sum of `bytes_fetched` over the receipt's sources, or `totals.projected_bytes` in
225
226
  `mr-data recipe forecast`. If they sum to 180 MB, state 270,000,000. Studio starts a scheduled
226
- rebuild without a spend confirmation only while its projected acquisition fits inside this
227
+ rebuild without a size confirmation only while its projected acquisition fits inside this
227
228
  number. A revision that grows past it is refused `REBUILD_BUDGET_EXCEEDED` before any run
228
229
  exists.
229
230
  - **`min_interval_seconds`** is from 3,600 to 31,536,000. Choose it from how often the source
@@ -497,9 +498,9 @@ before you register:
497
498
  - if the horizon turns out short, register the corrected revision before the first refresh, while
498
499
  there is only one day to lose.
499
500
 
500
- Until a run of the source has been observed, Studio prices the first full at the declared
501
- `limits.max_source_bytes`, so a large ceiling can make that one full ask for a spend confirmation.
502
- Later runs are priced from what was observed.
501
+ Until a run of the source has been observed, Studio sizes the first full at the declared
502
+ `limits.max_source_bytes`, so a large ceiling can make that one full ask for a size confirmation.
503
+ Later runs are sized from what was observed.
503
504
 
504
505
  Watch `mr-data recipe forecast`. Its `snapshot_history` reports the days held, the declared
505
506
  horizon and `days_remaining`, and it warns `SNAPSHOT_HISTORY_CEILING_NEAR` at 30 days or fewer. At
@@ -0,0 +1,143 @@
1
+ # Prepare search for a newly authored table
2
+
3
+ Use `mr-data search --help` and the relevant verb's help. These tools call Studio; no embedding
4
+ model, database, index or workload credential is installed on the agent's machine.
5
+
6
+ ## Before registration: declared intent
7
+
8
+ For each new table, describe searchable text, exact categorical aliases and meaningful sort terms
9
+ using the table's purpose, column descriptions and source evidence. First check the deployed
10
+ recipe authoring capability with a recipe-target request, before adding `table.search`:
11
+
12
+ ```sh
13
+ mr-data search inspect --recipe recipe.json --json
14
+ ```
15
+
16
+ Local CLI help describes installed commands; it does not prove the deployed server serves them.
17
+ For the recipe-target probe above, read JSON refusals from stderr: `schema_version` is
18
+ `mr-data-error.v1`, `code` is the typed discriminator, and `error` is the detail. There is no
19
+ public `http_status` field in that envelope. Treat `code: THIN_NOT_FOUND` as absent search
20
+ registration support only when `error` begins exactly with
21
+ `Studio has no such resource for /v4/search/recipe-inspection`; the CLI maps this route's HTTP 404
22
+ that way. An explicit `code: THIN_SEARCH_CAPABILITY_UNAVAILABLE` also states a capability limit.
23
+ A generic refusal or a rendered HTTP 405 alone is not this typed evidence; preserve the error
24
+ and resolve it rather than guessing that the deployment lacks search.
25
+
26
+ This rule applies only to `search inspect --recipe`, which needs no table ID. A missing table
27
+ or version on a table-target request is not capability evidence. Neither are authorization
28
+ failures (401/403), rate limits (429), timeouts, invalid response schemas, or invalid recipe
29
+ diagnostics: resolve the actual error. Missing capability metadata is not evidence either.
30
+
31
+ When the agreed brief does not mention search, treat search as optional only while this
32
+ capability is unavailable. When supported, preserve the default of useful search intent for
33
+ new tables. For an optional-search build on an unsupported deployment, omit `table.search`,
34
+ keep the drafted search intent separately alongside the recipe, and record the capability
35
+ limit in the dataset's Sources and decisions note and the matching chat message. Do not put
36
+ this operational limitation into the dataset description, title, or topics. Preserve ordinary
37
+ recipe checks and use supported data inspection tools. Do not emulate search locally, claim
38
+ readiness, or automatically backfill later; adding that retained intent to the resulting table
39
+ requires an explicit request.
40
+
41
+ If the user explicitly requires semantic retrieval or JEV interpretation into filters/sorts,
42
+ do not register or queue a build for any recipe that omits the required search. Report the
43
+ capability limit and wait for backend support. Do not silently downgrade, including an interim
44
+ baseline build. Independent source research and recipe drafting may continue.
45
+
46
+ When inspection is supported, continue with propose and the validate step below; both require
47
+ the deployed search authoring capability:
48
+
49
+ ```sh
50
+ mr-data search propose --recipe recipe.json --json
51
+ ```
52
+
53
+ The replies have `scope=declared_schema`. They show declarations, not observed vocabulary, rows,
54
+ completeness, latency or index quality. Propose does not rewrite the file. Explicitly insert the
55
+ returned spec into the relevant `table.search`, refine it, then validate the resulting recipe:
56
+
57
+ ```sh
58
+ mr-data search validate --recipe recipe-with-search.json --json
59
+ mr-data recipe recipe-with-search.json --json
60
+ ```
61
+
62
+ Repair diagnostics before registration. Keep column descriptions and units in their normal recipe
63
+ fields. Do not invent meanings such as “best” or “largest” when the brief does not identify a
64
+ metric. Keep that wording unresolved, or define separate explicit aliases such as “highest revenue”
65
+ and “most employees”. Never manufacture observed category values from documentation.
66
+
67
+ The search spec uses `schema_version: table-search-spec.v1`, `keyword_columns`, `semantic_columns`,
68
+ `aliases` (`phrase`, `column`, `operator`, `value`), `sort_aliases` (`phrase`, `column`, `direction`),
69
+ and `planner` (`enabled`, `external_context_allowed`). Read current validation diagnostics rather
70
+ than assuming that arbitrary operations, columns or model options are supported.
71
+
72
+ Default to useful keyword columns plus exact filters and sorting. For new tables with descriptive
73
+ prose, include useful descriptions/content/notes in `semantic_columns` for meaning-based retrieval.
74
+ Keep it empty for identifiers, measurements, dates and tables without useful prose. Embeddings run
75
+ locally on the hosted worker with a pinned multilingual model; they do not transmit private table
76
+ text externally. Preparation is budgeted and asynchronous. Never claim readiness from a declaration.
77
+ Semantic-only tables may use an empty `keyword_columns` list; their lexical capability stays
78
+ `unavailable` even after semantic preparation succeeds.
79
+
80
+ JEV is conditional. Exact identifiers, empty input, explicit plans, manual chip edits and already
81
+ resolved simple filters do not need a planner call. Reuse interpretation when its table/spec
82
+ coordinates still match. Only enable planner refinement where it adds meaning and deployed policy
83
+ permits it. `external_context_allowed` must stay false unless the user or established workspace
84
+ policy authorizes sending the relevant context externally; never infer consent from data-read
85
+ permission. Source cells and model responses remain untrusted.
86
+ Queries containing negation or explicit disjunction stay unresolved rather than becoming guessed
87
+ hard predicates. Verify those cases with explicit supported filters.
88
+
89
+ ## Capability availability
90
+
91
+ This CLI release supports inspect, propose, configure, validate, explain, estimate, build, status and cancel.
92
+ Other preparation verbs return `THIN_SEARCH_CAPABILITY_UNAVAILABLE` until the hosted implementation
93
+ is available. Treat that as a capability limit, never as successful preparation. Use the existing
94
+ bounded `peek`/`query` tools for data verification meanwhile; do not emulate semantic search locally.
95
+
96
+ ## After a successful data build: observed evidence
97
+
98
+ Read table and version IDs from receipts. Inspect with `--version VERSION_ID`. Use the existing
99
+ bounded `peek`/`query` tools to verify real categorical values and search expectations. Preserve
100
+ coverage/truncation and exact/sample scope; never report a sampled vocabulary as all categories.
101
+
102
+ Run `search status TABLE_ID --version VERSION_ID`. Recipe completion prepares an index in the
103
+ background for newly opted-in tables. When keyword fields are configured, lexical capability is
104
+ `scan` while no compatible index is ready, or `indexed` when a complete compatible lexical index
105
+ is verified. A complete generation can retain lexical `scan` when full keyword cells exceed the
106
+ index representation budget; use the reported capability rather than inferring it from job success.
107
+ Semantic capability becomes `ready` only with a verified compatible generation covering every
108
+ source row. Preparation resumes across bounded immutable shards; the shard size is a resource
109
+ bound, not a table population limit. Admission or preparation failures preserve the baseline and
110
+ must not be described as partially ready. Data publication continues independently of preparation.
111
+ Use the current estimate and job receipts for resource admission, progress and failure evidence.
112
+
113
+ Meaning-based retrieval searches every prepared shard and returns a bounded relevance window.
114
+ Approximate ranking does not imply an exhaustive similarity ordering or an exact matching count.
115
+ Hard filters apply before ranking; explicit sorting retains its declared meaning. Keep these
116
+ distinctions in verification notes rather than inferring corpus coverage from the result count.
117
+ Do not claim readiness merely because the data build completed, or enqueue duplicate work.
118
+
119
+ Use `search explain TABLE_ID --version VERSION_ID --query '...'` to check exact identity,
120
+ category, numeric boundaries, sort meanings and ambiguous language. Keep a few meaningful checks
121
+ in the dataset's verification notes with actual results from the hosted query tools. Search sample,
122
+ values, cases, preview and evaluation commands are not available in this CLI release.
123
+
124
+ ## Draft revisions
125
+
126
+ For authorized search-only work on a registered table, inspect, configure, validate and explain a
127
+ draft revision using returned immutable coordinates. Configure requires `--version`, `--spec`
128
+ and `--expected-revision`; `none` denotes an explicitly empty draft selection. Metadata exposes the
129
+ selected draft coordinate for the next compare-and-swap. Configure creates a draft and does not
130
+ activate an index or change data publication. Use an explicit `--idempotency-key` when retrying the
131
+ same configuration; never change its body under that key.
132
+
133
+ For explicitly requested preparation, `search build TABLE_ID --version VERSION_ID` queues the
134
+ immutable opted-in version's own recipe configuration, then activates only complete compatible
135
+ coverage automatically. It cannot build or activate a draft revision. Follow its receipt with
136
+ `search status --job JOB_ID`; pending work exits with code 3. `search cancel JOB_ID` revokes search
137
+ work only; it does not undo an already successful generation. Activation and rollback commands
138
+ remain unavailable. Do not compute authoritative digests or bypass server resource admission. Preserve
139
+ unresolved choices for the user; reinterpretation cannot silently replace manual edits or broaden
140
+ an exact filter.
141
+
142
+ This workflow opts in newly authored recipes only. It does not migrate old datasets, reauthor users'
143
+ existing recipes, launch a full data rebuild or imply successful indexing of every table.
@@ -40,7 +40,7 @@ and — once there is one — the interval Studio measured and how many source u
40
40
  **Never describe a schedule as learned, settled or confirmed while that command says otherwise.**
41
41
 
42
42
  `mr-data reschedule TABLE_ID --cadence ... --why "..."` changes the schedule of a table that is
43
- already live without starting a run or spending anything. Adding `--lock` freezes the schedule so
43
+ already live without starting a run or acquiring anything. Adding `--lock` freezes the schedule so
44
44
  Studio stops adjusting it, which is how a person overrides the evidence; `--unlock` lets it follow
45
45
  the source again. Do not lock a schedule on your own judgement — it is the same kind of decision as
46
46
  the cadence itself, and it belongs to the user. An explicit request for a fixed schedule is