@researai/deepscientist 1.5.17 → 1.6.0

package/src/skills/baseline/SKILL.md

@@ -6,446 +6,226 @@ skill_role: stage
 
 # Baseline
 
-This skill establishes the reference system the quest will compare against.
-The target is one trustworthy baseline line, not an endless reproduction diary.
+Use this skill to secure one trustworthy comparator and then get out of the way.
+The target is one accepted baseline line, not an endless reproduction diary.
 
-## Interaction discipline
-
-- Follow the shared interaction contract injected by the system prompt.
-- Keep ordinary setup and debugging updates concise.
-- Use richer milestone updates only when the baseline becomes trusted, caveated, blocked, waived, or route-changing.
-- Hard execution rule: every terminal command in this stage must go through `bash_exec`; do not use any other terminal path for setup, reproduction, monitoring, verification, Git, Python, package-manager, or file-inspection commands.
-- Prefer `bash_exec` for setup, reproduction, monitoring, and verification commands so the baseline line stays durable and auditable.
-
-## Tool discipline
-
-- **Do not use native `shell_command` / `command_execution` in this skill.**
-- **All shell, CLI, Python, bash, node, git, npm, uv, and environment work must go through `bash_exec(...)`.**
-- **For git work inside the current quest repository or worktree, prefer `artifact.git(...)` before raw shell git commands.**
-- **If a generic git smoke test is needed outside the quest repo, use `bash_exec(...)` in an isolated scratch repository.**
-
-## Non-negotiable rules
-
-- no fabricated metrics, logs, run status, or success claims
-- do not skip baseline steps or silently simplify the route when that would change trust or comparability
-- do not claim a baseline is ready before verification is complete
-- do not infer missing commands, scripts, or parameters when the uncertainty could change the result
-- any unavoidable guess must be written down explicitly with expected impact
-- use web search for discovering papers or repos, but use `artifact.arxiv(paper_id=..., full_text=False)` for actually reading a source arXiv paper when it exists
-- set `full_text=True` only when the short form is insufficient
-- for Python baselines, environment setup should be standardized around `uv`
-
-## Stage purpose
-
-The baseline stage should produce a usable reference point through one of four routes:
-
-1. attach an existing reusable baseline
-2. import a reusable baseline package
-3. reproduce a baseline from source
-4. repair a broken or stale baseline
-
-Keep the classic control flow:
-
-1. analysis
-2. setup
-3. execution
-4. verification
-
-These are control gates, not paperwork walls.
-
-## Quick workflow
-
-1. Read the source paper and source repo first, or record exactly what is missing and why.
-2. Choose the lightest trustworthy route: attach, import, reproduce, or repair.
-3. Start with the fast path whenever the current baseline object, command path, and acceptance target are already clear enough to validate cheaply.
-4. Before substantial baseline setup, code edits, or a real baseline run, create `PLAN.md` and `CHECKLIST.md`; short-form files are enough for simple fast-path work.
-5. Keep one dominant phase visible: analysis -> setup -> execution -> verification.
-6. Prefer one clean implementation pass, one smoke test, and then one normal baseline run.
-7. Retry only when smoke, verification, or runtime evidence shows a concrete failure or incompatibility.
-8. Close the stage by confirming or waiving the gate, then hand off with a concise `1-2` sentence summary of trust status and next anchor.
-
-## Fast-path first
-
-Default to the lightest baseline path that can still establish a trustworthy comparison.
-Default to a fast path when it can establish trust with less work.
-
-Fast path is the default when any of the following is true:
-
-- `requested_baseline_ref` or `confirmed_baseline_ref` already points to the active baseline object
-- the route is clearly `attach` or `import`
-- the repo entrypoint, dataset or split, and metric contract are already concrete enough to validate cheaply
-- reproduction requires no meaningful code changes and the main uncertainty is only whether the command still runs
-
-Fast path means:
-
-- do not restart broad baseline discovery by default
-- do not front-load a full codebase audit when the entrypoint is already concrete
-- use a minimal `PLAN.md`, a minimal `CHECKLIST.md`, one bounded smoke test when needed, and then one real validation or run
-- default to reuse-and-verify when runtime already attached a concrete baseline
-
-Escalate from fast path to fuller audit only when:
-
-- the paper and repo disagree materially
-- the real run or eval entrypoint is unclear
-- code changes are likely required
-- the contract spans multiple metrics, datasets, subtasks, or splits that still need interpretation
-- the same failure class reappears after one documented autonomous fix
-- the quest is trying to publish a reusable global baseline rather than only clear the current gate
+## Match signals
 
-## Use when
+Use `baseline` when:
 
 - no credible baseline exists yet
 - the current baseline is unverified or stale
 - the user already has a baseline package that should be attached or imported
+- a local code path or local service should be verified as the comparator
 - a reproduction failed earlier and now needs repair
 - the quest resumed and the baseline trust state is unclear
 
-## Do not use when
-
-- the quest already has a verified active baseline and the next move is ideation or execution
-- the user explicitly waived the baseline gate and that waiver is durably recorded
-
-## Stage gate
-
-Do not proceed to comparison-heavy downstream work unless one of the following is durably true:
-
-- a baseline has been attached and accepted
-- a baseline has been imported and accepted
-- a baseline reproduction has completed and been verified
-- an explicit waiver decision exists with a clear reason
-
-Operationally:
-
-- call `artifact.confirm_baseline(...)` once the accepted baseline root and trusted comparison contract are clear
-- call `artifact.waive_baseline(...)` when the quest must continue without a baseline
-- attach, import, or publish alone do not open the downstream gate
-
-## Required plan and checklist
-
-Before substantial baseline setup, code edits, or a real baseline run, create a quest-visible `PLAN.md` and `CHECKLIST.md`.
-
-- Use `references/baseline-plan-template.md` as the canonical structure for `PLAN.md`.
-- Use `references/baseline-checklist-template.md` as the canonical structure for `CHECKLIST.md`.
-- `analysis_plan.md` and `REPRO_CHECKLIST.md` remain acceptable compatibility alias files when an older quest already depends on them.
-- For fast-path attach/import/prebound validation or a simple reproduce path with no expected code changes, short-form `PLAN.md` and `CHECKLIST.md` are enough.
-- The plan should put the user's explicit requirements and non-negotiable constraints first.
-- Then record the chosen route, source identity, command path, expected outputs, acceptance condition, safe efficiency levers, main risks, and fallback.
-- If the route, commands, source package, fallback path, or trust judgment changes materially, revise `PLAN.md` before continuing.
-- Once the route is concrete, stop reshaping code and commands speculatively.
-
-Default retry discipline:
-
-- do not rerun the same unchanged smoke command just to reconfirm the same fact
-- treat one autonomous retry for the same failure class as the normal upper bound
-- if the same failure class appears again, switch explicitly into `repair`, record `blocked`, or route through `decision`
-
-## Required durable outputs
-
-The baseline stage should usually leave behind:
-
-- a baseline directory under `baselines/local/` or `baselines/imported/`
-- `PLAN.md` and `CHECKLIST.md`
-- a verification note or report
-- command, config, environment, and metrics pointers
-- a baseline artifact
-- a confirmed baseline gate via `artifact.confirm_baseline(...)`, or an explicit waiver via `artifact.waive_baseline(...)`
-- an optional registry publication if the baseline is reusable beyond this quest
-
-For simple attach/import flows or a straightforward reproduce flow, do not stall just to precreate every optional note file.
-
-Useful optional notes:
-
-- `setup.md`
-- `execution.md`
-- `verification.md`
-- `STRUCTURE.md` when the layout is non-obvious
-
-## File-by-file contract
-
-- `PLAN.md` or compatibility alias `analysis_plan.md` is the required route contract before substantial setup, code edits, or a real run; it should state the route, source identity, command path, expected outputs, acceptance condition, main risks, and fallback.
-- `CHECKLIST.md` or compatibility alias `REPRO_CHECKLIST.md` is the required living state tracker; it should show whether the baseline object, smoke decision, real run decision, and final accept / block / waive outcome are explicit.
-- `setup.md` is optional unless environment or layout choices are non-trivial; if used, record the working directory, environment route, important config paths, source revision, and notable setup deviations.
-- `execution.md` is optional unless the run is long, multi-step, or rerun-heavy; if used, record the launched commands, durable log paths, checkpoints, exit state, and any reruns or repairs.
-- `verification.md` is optional as a filename but required in substance before acceptance or blocked closeout; either this file or an equivalent report should record trusted metrics, expected-versus-observed comparison, caveats, canonical output paths, and the next anchor.
-- `STRUCTURE.md` becomes required when the workspace layout, mounts, symlinks, or generated outputs are non-obvious or meant for reuse; it should map the important directories and say which paths are canonical.
-- `attachment.yaml` is required for attached or imported baselines under `baselines/imported/`; preserve source identity, selected variant when relevant, and attachment provenance there.
-- `<baseline_root>/json/metric_contract.json` is the canonical accepted comparison contract; once the baseline is accepted, do not leave the authoritative metric surface only in chat, memory, or prose.
-- `Result/metric.md` is scratch-only; it may help during execution, but it is never the final source of truth.
-
-Minimum stability rules:
-
-- before the first real run, leave one durable note with the chosen route, expected command path, target outputs, and main risks
-- after each smoke test or real run, record what actually happened and whether the route still looks viable
-- before acceptance, leave a clear verification note and baseline gate decision
-- every accepted baseline should leave one accepted baseline artifact
-- every blocked baseline line should leave one blocked report and one next-step decision
-- if one rolling note is enough for a simple baseline line, use it
-
-## Durable path contract
-
-Use the real runtime paths consistently.
-
-Quest-local paths:
-
-- reproduced baseline root: `<quest_root>/baselines/local/<baseline_id>/`
-- attached or imported baseline root: `<quest_root>/baselines/imported/<baseline_id>/`
-- attachment record: `<quest_root>/baselines/imported/<baseline_id>/attachment.yaml`
-- canonical baseline metric contract JSON: `<baseline_root>/json/metric_contract.json`
-- baseline artifact record: `<quest_root>/artifacts/baselines/<artifact_id>.json`
-- baseline reports: `<quest_root>/artifacts/reports/<artifact_id>.json`
-- confirmed baseline reference: `quest.yaml -> confirmed_baseline_ref`
-
-Global reusable registry paths:
-
-- baseline registry index: `~/DeepScientist/config/baselines/index.jsonl`
-- canonical baseline entry: `~/DeepScientist/config/baselines/entries/<baseline_id>.yaml`
+Do not use `baseline` when:
 
-## Baseline id and variant rules
-
-- `baseline_id` should be short, stable, and filesystem-safe
-- use letters, digits, `.`, `_`, or `-`
-- do not use spaces, `/`, `\\`, or `..`
-- if one codebase contains multiple comparable baselines, prefer one `baseline_id` with structured variants instead of inventing many near-duplicate entries
-- when variants exist, keep `default_variant_id`, `baseline_variants`, and per-variant metric summaries stable enough that later `experiment` and `write` stages can cite them directly
-
-Do not invent parallel durable locations when these runtime contracts already exist.
-Do not leave the authoritative metric contract only in chat, memory, or prose once the baseline is accepted.
-
-If a baseline is reproduced only because an analysis campaign needs an extra comparator:
+- a verified active baseline already exists and the next move is obviously `idea`, `experiment`, `write`, or `finalize`
+- the baseline gate was already explicitly waived for the current route
 
-- still place it under the normal baseline roots
-- treat it as a supplementary analysis baseline unless the quest explicitly promotes it into the canonical gate
-- do not call `artifact.confirm_baseline(...)` for that supplementary case unless the quest truly intends to replace the canonical baseline
+## One-sentence summary
 
-## Multi-baseline policy
+Secure the lightest trustworthy comparator, make the comparison contract explicit, then confirm, waive, or block the baseline and stop.
 
-One quest may legitimately need more than one baseline.
+## Control workflow
 
-- explicitly mark which baseline is the primary downstream comparator
-- distinguish primary comparison baselines from fallback or infrastructure baselines
-- if several baselines are credible, record why the chosen primary baseline is the fairest paper-facing comparator
-- do not leave later stages guessing which baseline is authoritative
+1. Choose the current acceptance target and the lightest route that can satisfy it.
+   Prefer `attach`, `import`, or `verify-local-existing` before full reproduction.
+2. Make the comparator identity and core metric contract explicit.
+   Record task, dataset, split, evaluation path, required metric ids, metric directions, source identity, and known deviations.
+3. Collect only the evidence needed to establish comparability.
+   Do not widen into broad codebase audit or heavy reruns unless the lighter route cannot be trusted.
+4. Verify before acceptance.
+   Check that outputs are real, metrics trace to real evidence, and the intended dataset/split and metric definitions match the contract.
+   Explicitly verify the comparator and metric contract before treating the baseline gate as open.
+5. Close the gate explicitly.
+   Call `artifact.confirm_baseline(...)`, call `artifact.waive_baseline(...)`, or record an explicit blocker and next route.
+   When an already accepted baseline needs a deliberate second-pass refresh after verified code, variant, or canonical metric changes, prefer `artifact.overwrite_baseline(...)` over pretending the update is just a first confirmation.
 
-## Route order
+## AVOID / pitfalls
 
-Prefer this order:
+- Do not default to full source reproduction when reuse or verify-local-existing is already sufficient.
+- Do not treat attach, import, or publish alone as baseline acceptance.
+- Do not accept metrics that are fabricated, copied from the paper, or not traceable to real outputs, logs, or service responses.
+- Do not silently normalize away deviations in dataset, split, metric definition, evaluation path, or source identity.
+- Do not keep doing baseline work after the current acceptance target is already satisfied.
+- Do not repeat the same failure class without new evidence, code changes, environment changes, or a route change.
 
-1. attach
-2. import
-3. reproduce
-4. repair
+## Constraints
 
-Prefer reuse over redundant reproduction.
+- Routes, templates, filenames, smoke tests, and environment choices are tactics; the hard requirement is objective evidence sufficient to accept, waive, block, or switch the route.
+- Do not treat templates, filenames, `uv`, smoke tests, detached runs, or the phase order as required paths.
+- Durable records are required in substance, not in fixed filenames.
+- `PLAN.md`, `CHECKLIST.md`, `setup.md`, `execution.md`, `verification.md`, `analysis_plan.md`, and `REPRO_CHECKLIST.md` are allowed compatibility surfaces, not mandatory success paths.
+- `<baseline_root>/json/metric_contract.json` is the canonical accepted comparison contract.
+- Accepted baselines still require `artifact.confirm_baseline(...)`.
+- Waived baselines still require `artifact.waive_baseline(...)`.
+- Attach/import/publish alone do not open the downstream gate.
+- Later stages must not need to guess the active comparator, trusted metrics, or main caveats.
 
-## Workflow
+## Validation
 
-### Phase 1. Analysis
+Before `baseline` can end, all applicable checks should be true:
 
-Before running anything substantial, determine:
+- comparator identity is explicit and stable enough to cite later
+- task, dataset, split, evaluation path, required metric ids, metric directions, source identity, and known deviations are durably recorded
+- trusted metric values or trusted output pointers trace to real files, logs, service responses, or source artifacts
+- verification checked the intended dataset/split and metric definitions
+- the accepted comparison contract exists at `<baseline_root>/json/metric_contract.json`
+- the route ended in `artifact.confirm_baseline(...)`, `artifact.waive_baseline(...)`, or an explicit blocked state with next-step routing
 
-- exact task
-- dataset and split contract
-- metric contract
-- source baseline identity
-- source code path
-- expected run command or evaluation path
-- expected paper or repo numbers when they exist
-- local resource constraints
-
-Default analysis discipline:
+## Interaction discipline
 
-- read the source paper and source repo first
-- if runtime already exposes a matching `requested_baseline_ref` or `confirmed_baseline_ref`, validate that concrete object before restarting broad discovery
-- identify the real run or evaluation entrypoint
-- identify the dataset or split and metric contract
-- identify likely environment blockers
-- define the cheapest credible smoke test
+Follow the shared interaction contract injected by the system prompt.
+Keep baseline updates brief unless trust state, blocker state, route, cost, or user-facing risk changed materially.
 
-Escalate to a fuller audit only when the command path is unclear, the repo is large or confusing, repair mode is active, or custom code changes look likely.
+## Tool discipline
 
-When the fuller audit is necessary, capture only what later stages truly need:
+- **Do not use native `shell_command` / `command_execution` in this skill.**
+- **All shell, CLI, Python, bash, node, git, npm, uv, and environment work must go through `bash_exec(...)`.**
+- **For git work inside the current quest repository or worktree, prefer `artifact.git(...)` before raw shell git commands.**
+- **If a generic git smoke test is needed outside the quest repo, use `bash_exec(...)` in an isolated scratch repository.**
+- Use web search for discovering papers or repos, but use `artifact.arxiv(paper_id=..., full_text=False)` for actually reading a source arXiv paper when it exists.
+- Set `full_text=True` only when the short form is insufficient.
 
-- major entry scripts, configs, and modules
-- end-to-end data flow
-- evaluation path and metric computation path
-- obvious environment assumptions
-- obvious bottlenecks or incompatibilities
+## Authority and freedom
 
-If the source paper is available, record:
+The agent owns the execution path.
+It may choose the workspace layout, environment manager, command order, debugging route, smoke strategy, local paths, and whether the best route is attach, import, verify-local-existing, reproduce, or repair.
 
-- the core algorithm in compact, implementation-faithful form
-- the main reported numbers
-- the main weaknesses or bottlenecks likely to matter for this quest
+Ask the user only when the next move depends on a real scope, cost, permission, data-access, or scientific-preference decision that cannot be inferred from the quest contract.
+Ordinary route, path, environment, and debugging choices are autonomous unless they change the accepted comparison meaning.
 
-You may inspect local feasibility with shell-based checks for OS, GPU, CPU, RAM, disk, Python version, and whether `uv` is available.
+## Comparator-first rule
 
-The analysis phase should leave behind a concrete plan rather than only conversational intent.
+The baseline stage is comparator-first, not reproduction-first.
+For `comparison_ready`, the default question is:
 
-## Phase 2. Setup
+- what is the lightest trustworthy comparator?
 
-Prepare the selected route:
+not:
 
-- attach: validate the selected baseline id and variant
-- import: place the imported baseline metadata under the quest and confirm the package is readable
-- reproduce: prepare the baseline work directory, commands, config pointers, and environment notes
-- repair: identify the precise broken point before rerunning blindly
+- how do I reproduce the whole source package most completely?
 
-For Python baselines, standardize environment setup around `uv`.
+Default to the lightest baseline path that can still support a fair downstream comparison.
+Default to a fast path when it can establish trust with less work.
+Do not restart broad discovery or front-load a full codebase audit when the comparator, command path, and metric contract are already concrete.
+When this applies, do not front-load a full codebase audit.
+In that fast-path state, do not restart broad baseline discovery by default.
+Do not require a fresh memory pass for every fast-path validation; use memory when it prevents repeated work or clarifies stale route state.
+In short, do not require a fresh memory pass for every fast-path validation.
+A bounded smoke test is usually helpful only when command path, environment viability, evaluator wiring, or output schema is still unclear.
+Treat smoke/pilot work as a `0-2` default budget, and remember not to repeat an unchanged check without new evidence.
+When resuming a previously blocked or ambiguous route, recover the relevant memory before trusting the old path again.
 
-### Python environment rule: use `uv`
+If runtime already exposes `requested_baseline_ref` or a matching `confirmed_baseline_ref`, default to reuse-and-verify.
+Escalate to fuller audit, reproduction, or repair only when no concrete comparator, command path, or core comparability surface can be trusted yet.
 
-- if the repo already contains `uv.lock` or a solid `pyproject.toml`, use `uv sync`
-- otherwise create a local virtual environment with `uv venv`
-- install dependencies with `uv pip install ...`
-- run setup, smoke tests, and real commands through `uv run ...`
+For route examples and boundary cases, read `references/route-selection.md`, `references/artifact-flow-examples.md`, and `references/boundary-cases.md`.
+Use `references/baseline-plan-template.md` and `references/baseline-checklist-template.md` when a baseline route is complex enough to need durable planning surfaces.
 
-Practical rules:
+## Acceptance targets
 
-- prefer a quest-local or baseline-local `.venv`
-- prefer `uv run python ...` or `uv run bash ...` over relying on shell activation state
-- if a specific interpreter is required, make it explicit with `uv venv --python 3.11` or `uv run --python 3.11 ...`
-- if CUDA, PyTorch, JAX, or custom wheels require a special index URL, keep that install under `uv pip`
-- only accept a non-`uv` route when there is a concrete blocker that cannot be resolved locally
+- `comparison_ready`: the default target; one comparator is trustworthy enough for downstream comparison, and the core metric contract is durably recorded
+- `paper_repro_ready`: the baseline is strong enough to support paper-facing reproduction or comparison claims
+- `registry_publishable`: the baseline package is reusable and clean enough to publish as a durable baseline package
+- `blocked`: the current route cannot clear the gate cleanly, and the next move is explicit
+- `waived`: the quest must continue without a baseline, and the reason is durably recorded
 
-Common `uv` patterns:
+Not every baseline needs paper-grade exact reproduction.
+A verified attached, imported, or local-existing comparator can be enough when the acceptance target is only `comparison_ready`.
 
-- `uv sync`
-- `uv venv --python 3.11`
-- `uv pip install -r requirements.txt`
-- `uv run python scripts/smoke_test.py`
-- `uv run python train.py --config ...`
+## Hard acceptance gates
 
-Setup should record:
+Baseline success means later stages can compare against one accepted comparator without guessing task, data, split, metric, source, command or evaluation path, provenance, or caveats.
 
-- baseline id and source identity
-- working directory
-- config files
-- command template
-- expected outputs
-- known deviations from paper or source
-- the chosen `uv` route and Python version
+A baseline is successful only when all applicable gates are true:
 
-Fallbacks:
+- the comparator identity is explicit and stable enough for later stages to cite
+- the task, dataset, split, evaluation path, required metric ids, metric directions, source identity, and known deviations are durably recorded
+- trusted metric values or trusted output pointers are traceable to real files, logs, service responses, source artifacts, or an accepted registry/package record
+- verification checked that the evidence came from the intended dataset/split and metric definitions
+- the accepted comparison contract is written to `<baseline_root>/json/metric_contract.json`
+- the baseline gate is opened with `artifact.confirm_baseline(...)`, or intentionally bypassed with `artifact.waive_baseline(...)`
 
-- if Hugging Face access is blocked, record and try an approved local mirror such as ModelScope when that does not change the comparison meaning
-- if a quest already depends on `analysis_plan.md` or `REPRO_CHECKLIST.md`, keep the compatibility alias explicit rather than splitting truth across two active plans
+Once a comparison-ready baseline is durably confirmed, baseline should usually stop immediately.
+Once a comparison-ready baseline is durably confirmed, baseline should usually stop immediately and hand off to the next scientific step.
+Any extra baseline work after that must name one explicit unresolved comparison risk it is meant to remove.
 
-## Phase 3. Execution
+## Route success criteria
 
-Run only the work required to establish the baseline credibly.
+Choose the route that maximizes trust per unit time and compute; do not follow a fixed ritual.
+Keep one dominant baseline route active at a time.
+If a lighter route already satisfies the current acceptance target, stop there.
 
-Execution rules:
+- `attach` succeeds when baseline identity, provenance, trusted outputs pointer, core metric contract, and accepted baseline artifact are explicit
+- `import` succeeds when the package is materialized/readable inside the quest, `attachment.yaml` or equivalent provenance exists, and trusted outputs or metrics are traceable
+- `verify-local-existing` succeeds when the concrete local path or service, exact command or evaluation endpoint, output location, required metrics, and core metric contract are verified
+- `reproduce` succeeds when source identity, command or evaluation path, expected outputs, verification evidence, deviations, and metric contract are explicit
+- `repair` succeeds when the broken point is identified, a bounded fix or route change is made, rerun or re-read evidence supports the new trust state, and the result is accepted or blocked
 
-- keep commands auditable
-- keep logs durable
-- avoid uncontrolled side experiments during baseline establishment
-- checkpoint only explainable, minimal code changes
-- prefer equivalence-preserving efficiency gains such as larger safe batch size, cache reuse, checkpoint resume, and parallel downloads or workers
-- do not use an efficiency lever if it changes accepted baseline meaning, effective evaluation contract, or trust judgment
+Prefer reuse over redundant reproduction, but prefer reproduction or repair when reuse would still leave the baseline incomparable.
+Do not replace a working comparison-ready comparator with a heavier route merely because the heavier route feels cleaner or more complete.
 
-Long-running execution discipline:
+## Objective evidence requirements
 
-- run one bounded smoke test before a substantial baseline reproduction
-- once the smoke test passes, launch the real baseline reproduction with `bash_exec(mode='detach', ...)`
-- monitor by forward progress instead of by short-window completion anxiety
-- do not report final success until the command actually finished and the expected result files exist
-- if you need to recover ids or inspect session state, use `bash_exec(mode='history')` or `bash_exec(mode='list')`
-- `bash_exec(mode='read', id=...)` returns the full saved log when it is `2000 lines or fewer`; for longer logs, inspect omitted middle windows with `start` and `tail`
-- during monitoring, prefer `bash_exec(mode='read', id=..., tail_limit=..., order='desc')`, and after the first read prefer incremental checks with `after_seq=last_seen_seq`
-- use `silent_seconds`, `progress_age_seconds`, `signal_age_seconds`, and `watchdog_overdue` as the default staleness clues
-- if a run is clearly invalid, wedged, or superseded, stop it with `bash_exec(mode='kill', id=..., wait=true, timeout_seconds=...)`, document why, and relaunch cleanly
-- do not let more than the `30-minute visibility bound` pass without a real inspection and a `next expected update time`
-- when the baseline code is under your control, prefer a throttled `tqdm` progress reporter and periodic `__DS_PROGRESS__` markers when feasible
+The final evidence should cover these facts before acceptance:
 
-Keep retries bounded:
+- comparator candidate and baseline id
+- source paper, source repo, source commit/version/tag, local service identity, or registry/package identity as applicable
+- task identity
+- dataset identity and split contract
+- evaluation script, evaluation endpoint, or evaluation path
+- required metric keys for the current downstream comparison
+- metric directions
+- metric values or trusted output pointers
+- environment and hardware facts that materially affect comparability
+- known deviations from the paper, source package, local reference, or selected target
+- verification verdict and caveats
 
-- one smoke test is the default
-- one autonomous fix-and-retry for the same failure class is the normal upper bound
-- if the same failure class returns, stop looping
+Unless the user explicitly specifies otherwise, treat the original paper's evaluation protocol as the canonical starting point.
+If later `experiment` work would still have to guess the comparison contract, the baseline is not ready.
+For a compact verdict rubric, read `references/comparability-contract.md`.
 
-## Phase 4. Verification
+## Verification
 
 Verification is mandatory before baseline acceptance.
 
 Verify:
 
-- the run actually finished
+- the run, service call, package import, or trusted-output inspection actually finished
 - the reported metrics came from the intended dataset and split
-- the metric definitions match the quest contract
-- the result is comparable to the paper, source repo, or selected target
-- any deviations are explicitly stated
+- metric definitions and directions match the quest contract
+- the result is comparable to the paper, source repo, local comparator, registry package, or selected target
+- deviations are explicitly stated rather than silently normalized away
 
 Classify the outcome as one of:
 
 - `verified_match`
 - `verified_close`
 - `verified_diverged`
+- `trusted_with_caveats`
 - `broken`
 
-Verification must explicitly separate:
+Verification should explicitly separate likely implementation mismatch, environment mismatch, data or split mismatch, expected stochastic variance, and unexplained divergence when those distinctions matter.
 
-- likely implementation mismatch
-- environment mismatch
-- data or split mismatch
-- expected stochastic variance
-- unexplained divergence
+## Core metric contract
 
-Verification should answer:
-
-- whether the baseline is trustworthy enough for downstream comparison
-- whether the result is reusable beyond this quest
-- whether another repair or rerun is justified
-- whether the line should stop here and hand off
-
-A verification report should be self-contained enough that a later stage can answer:
-
-- what was used
-- how it was obtained: attach, import, reproduce, or repair
-- what commands and configs were used
-- what metrics are trusted
-- what caveats remain
-- whether the result is reusable beyond this quest
-
-## Baseline comparability contract
-
-The baseline stage is not complete just because something ran.
-It is complete when later stages can compare against it fairly.
-
-Before declaring a baseline usable, make the comparability contract explicit:
+Before declaring a baseline usable, make the core comparison contract explicit:
 
 - task identity
-- dataset identity and version
-- split contract
-- preprocessing boundary
+- dataset identity and split contract
 - evaluation script or evaluation path
-- required metric keys
+- required metric keys for the current downstream comparison
 - metric directions
-- seed policy when relevant
 - source commit or source package identity
 - known deviations from the source reference
 
-Unless the user explicitly specifies otherwise, treat the original paper's evaluation protocol as the canonical baseline contract.
-If any of these fields are still materially unknown, do not pretend the baseline is a clean downstream reference.
-For the fuller checklist and verdict meanings, read `references/comparability-contract.md`.
-
-## Feasibility and trust classes
-
-Before acceptance, classify feasibility as one of:
-
-- `full_reproducible`
-- `degraded_but_acceptable`
-- `blocked`
-
-And classify downstream trust as one of:
-
-- `verified`
-- `partially_verified`
-- `operational_but_incomparable`
-- `failed`
-
-Do not silently upgrade a degraded or merely operational result into a normal trusted baseline.
-
-## Minimum baseline artifact content
+`<baseline_root>/json/metric_contract.json` is the canonical accepted comparison contract.
+The comparison-ready minimum still requires `<baseline_root>/json/metric_contract.json`.
+A core contract is enough to confirm a `comparison_ready` baseline; expand it later when paper claims, registry publication, or variant-heavy comparison need more coverage.
 
 The accepted baseline artifact should include at least:
 
@@ -460,111 +240,46 @@ The accepted baseline artifact should include at least:
 - `source`
 - `summary`
 
-If variants exist, also include:
-
-- `default_variant_id`
-- `baseline_variants`
-
 Metric-contract rules:
 
-- if the accepted baseline contract includes multiple metrics, datasets, subtasks, or splits, record all of them in `<baseline_root>/json/metric_contract.json`
 - keep `primary_metric` as the headline metric only; do not let it erase the rest of the comparison surface
-- when confirming a baseline, submit the canonical `metrics_summary` as a flat top-level dictionary keyed by the paper-facing metric ids
+- submit canonical `metrics_summary` as a flat top-level dictionary keyed by the paper-facing metric ids
 - every canonical baseline metric entry should include `description`, either `derivation` or `origin_path`, and `source_ref`
+- mark only the currently required canonical metrics as required; additional metrics can be added later or kept supplementary
+- if the accepted baseline contract already needs multiple metrics, datasets, subtasks, or splits, record them in `<baseline_root>/json/metric_contract.json`
 - if the paper reports both aggregate and per-dataset or per-task results, preserve both whenever feasible through `metrics_summary` plus structured rows rather than one cherry-picked scalar
 - if the source package already has a richer leaderboard table, structured result file, or `json/metric_contract.json`, reuse that richer contract instead of hand-writing a thinner one that keeps only one averaged scalar
 - `Result/metric.md` is optional temporary scratch memory only; reconcile against it before calling `artifact.confirm_baseline(...)`, but do not treat it as a required durable file
+- for stable accepted payload shapes, read `references/artifact-payload-examples.md`
 
-## Publication and reuse
-
-Use the registry deliberately, not as an afterthought.
-
-If the result is reusable beyond the current quest:
-
-- publish it through `artifact.publish_baseline(...)`
-- ensure the payload includes identity, provenance, trusted metrics, and any variant structure
-- set `publish_global: true` only when verification is complete and reuse is justified
-
-If the current quest should reuse an existing baseline:
-
-- attach it through `artifact.attach_baseline(...)`
-- preserve the selected `baseline_id`
-- preserve the selected `variant_id` when one is used
-- keep the attachment durable under `baselines/imported/`
-
-If runtime state already includes `requested_baseline_ref` or a matching `confirmed_baseline_ref`:
-
-- default to reuse-and-verify, not rediscovery
-- treat a creation-time pre-bound baseline as the active starting point unless you find a concrete incompatibility
-- do not rerun broad baseline scouting or full reproduction just because the stage name is `baseline`
-
-For a clearer attach/import/reproduce/repair rubric, read `references/route-selection.md`.
-For reusable-package expectations, read `references/publishable-baseline-package.md`.
-
-## Workspace and branch rules
-
-- treat the baseline workspace as a system-managed reproduction surface, not an unrelated sandbox
-- avoid creating a nested authoritative Git lifecycle inside the baseline workspace
-- use the quest branch unless isolation is genuinely needed
-- if baseline setup is risky or intrusive, prepare an isolated branch or worktree first and record why
-- do not proliferate branches without a reason
-
-## Memory rules
-
-Stage-start requirement:
-
-- by default, begin every baseline pass with `memory.list_recent(scope='quest', limit=5)`
-- then run at least one baseline-relevant `memory.search(...)` before new baseline analysis, repair, or rerun work
-- fast-path exception: if the quest already exposes a clear `requested_baseline_ref` or `confirmed_baseline_ref` and the immediate task is only to validate or reattach that concrete baseline, you may skip broad retrieval
+## Operational guidance
 
-Write memory only for reusable lessons such as:
+The main skill keeps the control surface in front.
+For the longer operational notes, read `references/operational-guidance.md`.
 
-- paper-to-code mismatch notes
-- environment incidents
-- dataset quirks
-- verification caveats
-- attach vs import vs reproduce vs repair rationale
+- use it when you need the exact durable route record shape
+- use it when you need detailed execution tactics or environment tactics
+- use it when reuse or memory handling materially affects the route
 
-When calling `memory.write(...)`, pass `tags` as an array like `["stage:baseline", "baseline:<baseline_id>", "type:repro-lesson"]`, not as one comma-joined string.
+## Negative cases and stop rules
 
-Stage-end requirement:
+Do not accept a baseline when:
 
-- if baseline work produced a durable reproduction lesson, verification caveat, environment incident, or route rationale, write at least one `memory.write(...)` before leaving the stage
-
-## Artifact rules
-
-Typical artifact sequence:
-
-- `progress` for long-running setup or execution checkpoints
-- `report` for analysis notes or verification notes
-- `decision` for route choice, blocked routing, or accept/reject/rerun/repair calls
-- `baseline` only for an accepted baseline record
-
-For stable field shapes, read `references/artifact-payload-examples.md`.
-
-The baseline handoff should make these items obvious:
-
-- `baseline_id`
-- `baseline_variant_id` when relevant
-- route used: attach, import, reproduce, or repair
-- trusted metrics
-- canonical metric contract JSON path
-- verification outcome
-- reusable or quest-local only
-- canonical output paths
-- main caveats
-- recommended next anchor
-
-If this packet is not obvious from the accepted artifact plus verification note, the baseline line is not stable enough yet.
-
-## Failure and blocked handling
+- metrics are fabricated, copied, or paraphrased without provenance
+- metrics are copied from a paper while the acceptance target requires local verification
+- dataset, split, metric direction, or evaluation path is materially unknown
+- outputs exist but cannot be tied to the intended command, source, comparator, package, or service
+- a local run completed but used a materially different protocol without a recorded caveat
+- source code was modified in a way that changes baseline scope without recording the deviation
+- a package imports but trusted metrics or outputs are not traceable
+- later experiment work would still need to guess the required baseline metric ids
+- the same failure class reappears without new evidence, code changes, environment changes, or a route change
 
+If the same failure class appears again without new evidence, code changes, environment changes, or a route change, stop looping and route through `repair`, `decision`, `blocked`, `waive`, or one bounded clarification.
 Do not hide failures.
-
-If blocked, record the class explicitly:
+If blocked, record the class explicitly when possible:
 
 - `missing_source`
-- `missing_code`
 - `missing_metric_contract`
 - `environment_infeasible`
 - `command_unknown`
@@ -576,29 +291,36 @@ A blocked result must state:
 - what failed
 - what was tried
 - which paths or logs show the issue
-- whether the next best move is attach, import, retry, repair, reset, or ask the user
+- whether the next best move is attach, import, retry, repair, reset, waive, or ask the user
 
-Reasonable autonomous fixes before escalation:
+Bounded autonomous fixes are acceptable only when they do not change confirmed scope, metrics, permissions, resource assumptions, or scientific meaning.
+Reasonable bounded fixes include missing dependency installs, wrong dataset paths, permission fixes on scripts, obvious environment activation mistakes, and conservative batch-size reductions for OOM.
 
-- missing module or dependency
-- wrong dataset path
-- permission errors on scripts
-- reasonable batch-size reductions for OOM
-- obvious environment activation mistakes
+## Baseline id and variant rules
+
+Keep baseline identifiers and variant names stable enough that later stages can cite the same comparator without guesswork.
 
-If a fix would change confirmed scope, metrics, permissions, or resource assumptions, stop and return to analysis rather than applying it silently.
+- keep `baseline_id` short, stable, and filesystem-safe
+- prefer one baseline id with stable variant names over many near-duplicate ids
+- if multiple comparators exist, mark which one is the primary downstream baseline
 
 ## Exit criteria
 
-Exit the baseline stage once one of the following is durably true:
+Exit once one of these is durably true:
 
 - a baseline is attached and accepted
 - an imported baseline is accepted
+- a verified local-existing comparator is accepted
 - a reproduced baseline is verified and accepted
+- a repaired baseline is verified and accepted
 - a broken route has been declared blocked and a next decision is recorded
+- a waiver decision explicitly leaves the baseline gate
+- a route change is recorded because the previous route is no longer the best trust-per-cost path
 
 Typical next anchors:
 
 - `idea`
 - `experiment` in tightly scoped follow-on cases
 - `decision` if the baseline line remains contested
+
+A good baseline pass leaves one trusted comparator, one explicit blocker, or one explicit route change, not a vague promise to keep rechecking baseline.