@researai/deepscientist 1.5.1 → 1.5.3

package/src/skills/baseline/SKILL.md

@@ -13,12 +13,12 @@ It absorbs the essential old DeepScientist reproducer discipline into one stage
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing baseline work.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest baseline progress update before assuming the task changed completely.
 - Prefer `bash_exec` for setup, reproduction, and verification commands so each baseline action keeps a durable quest-local session id and log trail.
 - When the baseline route is durably chosen, confirmed, waived, or blocked with a clear next action, send one richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` update that says whether the baseline is trusted, blocked, or waived, why that matters, and what the next stage is.
@@ -42,16 +42,20 @@ It absorbs the essential old DeepScientist reproducer discipline into one stage
 
 ## Priority workflow
 
-The baseline stage should follow this priority order and should not reorder it casually:
+Default to the lightest baseline path that can still establish a trustworthy comparison.
+Do not front-load a full reproduction dossier when a faster truth-finding step would tell you whether the route is even viable.
+
+The ordinary baseline order is:
 
 1. confirm quest binding and current baseline state
-2. acquire or validate the baseline workspace
-3. analyze the code, paper, and resource constraints
-4. choose the route: attach, import, reproduce, or repair
-5. write down the concrete execution plan
-6. execute only after the plan is concrete enough
-7. verify before accepting
-8. archive, publish, or attach the result when appropriate
+2. look for the cheapest trustworthy route in order: attach, import, reproduce, repair
+3. capture the minimum viable contract: task, dataset or split, metric, source identity, expected command path, and main risks
+4. run a bounded smoke test as soon as that contract is concrete enough
+5. only after the smoke test is credible, expand setup notes and launch the real run
+6. verify before accepting
+7. archive, publish, or attach the result when appropriate
+
+Escalate to the heavier baseline path only when the baseline is ambiguous, broken, multi-variant, paper-to-repo mismatched, or likely to be reused beyond the current quest.
 
 If the quest is not yet bound to a stable baseline context, do not pretend the stage is ready just because some code exists locally.
 
@@ -75,16 +79,17 @@ Do not casually skip these gates.
 
 ## Phase routing rule
 
-Treat the baseline stage as a strict internal sub-workflow.
-At any moment, the work should be clearly in one of:
+Treat `analysis`, `setup`, `execution`, and `verification` as logical control gates, not paperwork walls.
+At any moment, the work should have one dominant phase among:
 
 - `analysis`
 - `setup`
 - `execution`
 - `verification`
 
-Do not blur several phases together.
-Finish the current phase, update its durable notes, then move to the next phase intentionally.
+Keep the dominant phase explicit, but allow small backtracks and lightweight overlap when they reduce wasted work.
+Do not delay an early smoke test just because a fuller write-up is not done yet.
+Before a real long run, make sure the minimum viable contract is explicit and the active phase is still easy to reconstruct.
 
 ## Use when
 
@@ -140,14 +145,15 @@ Do not treat memory alone as sufficient evidence for baseline readiness.
 The baseline line should also maintain a durable working-record area outside the execution surface.
 Recommended quest-visible records include:
 
-- `analysis_plan.md`
+- `analysis_plan.md` or a compact equivalent section in `execution.md`
 - `setup.md`
 - `execution.md`
 - `verification.md`
-- `STRUCTURE.md`
-- `REPRO_CHECKLIST.md`
+- `STRUCTURE.md` only when the workspace layout is non-obvious or later reuse depends on it
+- `REPRO_CHECKLIST.md` only when the route is complex, repair-heavy, multi-variant, or publication-facing
 
-These should live in a quest-visible baseline artifact area so later stages can read them without replaying the whole reproduction process.
+For a simple attach/import flow or a straightforward reproduce flow, do not stall just to precreate every one of these files.
+Start with the smallest durable note that preserves the route, command path, target outputs, and main risks; expand it only after the route proves real.
 
 ## Required durable outputs
 
@@ -163,20 +169,25 @@ The baseline stage should usually leave behind:
 ## Stable execution contract
 
 To keep baseline work stable across different quests, do not stop at loose prose.
-Use the same durable structure every time unless the quest has a strong reason to differ.
+But also do not confuse stability with ceremony.
+Use the lightest durable structure that keeps the baseline auditable and reusable.
 
 Minimum stability rules:
 
-- every phase should leave one clearly named durable note
+- before the first real run, leave one durable note with the chosen route, expected command path, target outputs, and main risks
+- after each smoke test or real run, record what actually happened and whether the route still looks viable
+- before acceptance, leave a clear verification note and baseline gate decision
 - every route selection should leave one explicit reasoned decision record
 - every accepted baseline should leave one accepted baseline artifact
 - every blocked baseline line should leave one blocked report and one next-step decision
 - every handoff should name the active baseline reference and trusted metric set explicitly
+- do not require every optional checklist or template before the first smoke test
+- if one rolling note is enough for a simple baseline line, use it
 
 Recommended phase-to-output mapping:
 
-- `analysis` -> `analysis_plan.md` plus optional route decision artifact
-- `setup` -> `setup.md`
+- `analysis` -> a brief `analysis_plan.md` or equivalent compact route note, plus optional route decision artifact
+- `setup` -> `setup.md` when setup choices are non-trivial
 - `execution` -> `execution.md` plus progress artifacts when long-running
 - `verification` -> `verification.md` plus accepted baseline artifact and `artifact.confirm_baseline(...)`, or a blocked report plus `artifact.waive_baseline(...)` when skipping is intentional
 
@@ -204,6 +215,12 @@ Global reusable registry paths:
 Do not invent parallel durable locations when these runtime contracts already exist.
 Do not leave the authoritative metric contract only in chat, memory, or prose once the baseline is accepted.
 
+If a baseline is reproduced only because an analysis campaign needs an extra comparator:
+
+- still place it under `<quest_root>/baselines/local/<baseline_id>/` or `<quest_root>/baselines/imported/<baseline_id>/`
+- treat it as a supplementary analysis baseline unless the quest explicitly promotes it into the canonical gate
+- do not call `artifact.confirm_baseline(...)` for that supplementary case unless the quest truly intends to replace the canonical baseline
+
 ## Baseline id and variant rules
 
 Baseline identity should be stable and path-safe.
@@ -342,8 +359,16 @@ Before running anything substantial, determine:
 - expected paper or repo numbers, if any
 - local resource constraints
 
-Do not stop at a loose reproduction intent.
-Run a structured baseline codebase audit and capture at least:
+For straightforward baseline work, start with a quick viability pass:
+
+- find the real run or evaluation entrypoint
+- identify the dataset/split and metric contract
+- identify likely environment blockers
+- define the cheapest credible smoke test
+
+Escalate from that quick pass to a fuller baseline codebase audit when the command path is unclear, the repo is large or confusing, the paper and code diverge materially, repair mode is active, or custom code changes look likely.
+
+When the fuller audit is necessary, capture at least:
 
 - major modules and files
 - end-to-end data flow
@@ -390,7 +415,7 @@ At minimum, the plan should capture:
 - key risks
 - verification targets
 
-When possible, structure `analysis_plan.md` with headings close to:
+When the analysis note becomes substantial, structure `analysis_plan.md` with headings close to:
 
 - executive summary
 - codebase analysis
@@ -433,6 +458,10 @@ Prepare the selected route:
 - reproduce: prepare the baseline work directory, commands, config pointers, and environment notes
 - repair: identify the precise broken point before rerunning blindly
 
+For a fast-path reproduction, setup can stay lightweight.
+Confirm the working directory, environment, config, output paths, smoke command, and long-run command, then move forward.
+Do not manufacture a fresh workspace tree or copy the repo just to satisfy a template if the existing layout is already workable and auditable.
+
 Capture:
 
 - baseline identifier
@@ -450,8 +479,8 @@ Setup should also confirm:
 - required dependencies or environments are known
 - the execution plan is realistic for the detected hardware
 
-Setup should establish a clear baseline workspace layout.
-Recommended structure:
+If a dedicated baseline workspace is needed, establish a clear layout.
+One workable structure is:
 
 ```text
 <baseline_root>/
@@ -465,7 +494,7 @@ Recommended structure:
     <run_id>/
 ```
 
-And the quest-visible audit area should contain at least:
+If the baseline becomes long-lived, shared, or non-obvious, the quest-visible audit area may contain:
 
 ```text
 <quest_root>/
@@ -505,8 +534,10 @@ Execution rules:
 - if a run is long, emit progress artifacts at meaningful checkpoints
 - if setup required code changes, checkpoint only explainable, minimal changes
 
-Execution should rely on explicit scripts or command paths where possible.
-If a wrapper or entry script is needed, it should support most of the following:
+Execution should rely on existing explicit scripts or command paths where possible.
+Prefer the smallest runnable command that proves the baseline route.
+Do not build a new wrapper, registry, or result-export scaffold unless existing commands are missing, repeated reruns justify it, or later automation clearly needs it.
+If a wrapper or entry script is truly needed, it should support most of the following:
 
 - run mode for missing combinations
 - print-only mode that summarizes existing results without rerunning everything
@@ -543,10 +574,18 @@ If a result backup is useful for audit or recovery, create it explicitly rather
 
 Long-running execution rules:
 
+- before a substantial baseline reproduction, run a bounded smoke test first so command paths, output locations, and metric plumbing are validated cheaply
+- once the smoke test passes, launch the real baseline reproduction with `bash_exec(mode='detach', ...)` and normally leave `timeout_seconds` unset for the long run itself
+- when monitoring that detached run, prefer `bash_exec(mode='read', id=..., tail_limit=..., order='desc')` so you inspect the newest log evidence first
+- after the first read, prefer incremental checks with `bash_exec(mode='read', id=..., after_seq=last_seen_seq, tail_limit=..., order='asc')` so you only inspect newly appended evidence
+- if you need to recover ids or confirm the newest session quickly, use `bash_exec(mode='history')` or `bash_exec(mode='list')` rather than guessing
+- include a structured `comment` on long-running bash sessions with fields such as `stage`, `goal`, `action`, `expected_signal`, and `next_check`
+- use `silent_seconds`, `progress_age_seconds`, `signal_age_seconds`, and `watchdog_overdue` from `bash_exec(mode='list'|'read', ...)` as the default staleness checks
+- when the reproduction code is under your control, prefer a throttled `tqdm` progress reporter and, when feasible, pair it with periodic `__DS_PROGRESS__` JSON lines carrying phase and ETA
 - if a command is expected to run for a long time, monitor it as a real background task rather than assuming success
 - do not write final summaries or accepted metrics until the command has actually completed
 - verify that the expected result files exist before treating the run as finished
-- if a task fails, diagnose and either retry with a documented fix or record the failure durably
+- if a task is invalid, wedged, or failed, stop it with `bash_exec(mode='kill', id=..., wait=true, timeout_seconds=...)`; if it must die immediately, add `force=true`, then diagnose the reason and either retry with a documented fix or record the failure durably
 
 Recommended monitoring cadence for long-running work:
 
@@ -557,7 +596,7 @@ Recommended monitoring cadence for long-running work:
 - fifth check after about 1800 seconds
 - after that, keep checking about every 1800 seconds while the run is still active
 
-The exact mechanism should prefer `bash_exec(mode='await' | 'detach' | 'read' | 'list' | 'kill', ...)`, but the behavioral rule stays the same:
+The exact mechanism should prefer `bash_exec(mode='await' | 'detach' | 'read' | 'list' | 'history' | 'kill', ...)`, with `read` usually using a tailed or incremental window during monitoring, but the behavioral rule stays the same:
 do not report completion until the run is actually done and the outputs are real.
 After each meaningful check, notify the user through `artifact.interact(kind='progress', ...)` with current status, latest evidence, and the next monitoring point.
 Do this after every completed wait cycle for important long-running work; do not skip several sleep windows without reporting.
@@ -664,6 +703,8 @@ If variants exist, also include:
 ## Durable note templates
 
 Use compact but structured notes so later stages do not need to reconstruct baseline state from chat history.
+The templates below are references, not prerequisites for the first smoke test.
+For simple baseline lines, keep them short and fill only the sections that matter.
 
 ### `analysis_plan.md`
 

package/src/skills/decision/SKILL.md

@@ -12,14 +12,14 @@ Use this skill whenever continuation is non-trivial.
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before making the next decision.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: a meaningful checkpoint, a route-shaping update, or an occasional keepalive during truly long decision analysis. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: a meaningful checkpoint, a route-shaping update, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Message templates are references only. Adapt to context and vary wording so updates feel natural and non-robotic.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - If the runtime starts an auto-continue turn with no new user message, continue from the active requirements and durable quest state instead of replaying the previous user turn.
 - If `startup_contract.decision_policy = autonomous`, do not emit ordinary `artifact.interact(kind='decision_request', ...)` calls; decide the route yourself, record the reason, and continue.
 - Use `reply_mode='blocking'` for the actual decision request only when the user must choose before safe continuation and the quest contract still allows a user-gated decision.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest decision or progress interaction before assuming the task changed completely.
 - Quest completion is a special terminal decision: first ask for explicit completion approval with `artifact.interact(kind='decision_request', reply_mode='blocking', reply_schema={'decision_type': 'quest_completion_approval'}, ...)`, and only after an explicit approval reply should you call `artifact.complete_quest(...)`.
 
@@ -319,7 +319,7 @@ When asking, use a structured decision request with:
 - tradeoffs, including the main pros and cons for each option
 - recommended option first
 - explicit reply format
-- a stated timeout window; normally wait up to 1 day before self-resolving if no user reply arrives
+- a stated timeout window; normally wait up to 1 day before self-resolving if no user reply arrives, except when the only blocker is a missing external credential or secret that only the user can provide
 
 ### 6. Record the decision durably
 
@@ -327,6 +327,7 @@ Use `artifact.record(kind='decision', ...)` for the final decision.
 
 If user input is needed, also use `artifact.interact(kind='decision_request', ...)`.
 If the timeout expires without a user reply, choose the best option yourself, record why, and notify the user of the chosen option before moving on.
+This does not apply when the only blocker is a missing external credential or secret that only the user can provide; in that case keep the interaction waiting and, if resumed without the credential, you may park with `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` instead of busy-looping.
 
 If `startup_contract.decision_policy = autonomous`, ordinary route ambiguity is not by itself grounds to request user input.
 In that mode, only explicit approval-style exceptions such as quest completion should normally become blocking user decisions.

package/src/skills/experiment/SKILL.md

@@ -12,7 +12,7 @@ Use this skill for the main evidence-producing runs of the quest.
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing the run plan.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Keep ordinary subtask completions concise. When a main experiment actually finishes or reaches a stage-significant checkpoint, upgrade to a richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` report rather than another short progress line.
@@ -43,7 +43,7 @@ Use this skill for the main evidence-producing runs of the quest.
 - If the runtime starts an auto-continue turn with no new user message, continue from the current run state, logs, artifacts, and active requirements instead of replaying the previous user turn.
 - Progress message templates are references only. Adapt to the actual context and vary wording so messages feel human, respectful, and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest experiment progress update before assuming the task changed completely.
 - Prefer `bash_exec` for experiment commands so each run gets a durable session id, quest-local log folder, and later `read/list/kill` control.
 
@@ -377,9 +377,14 @@ Last-known-good rule:
 
 For commands that may run longer than a few minutes:
 
-- launch with `bash_exec(mode='detach', ...)`
+- before the real long run, execute a bounded smoke test or pilot that validates command paths, outputs, and basic metrics
+- once the smoke test passes, launch the real run with `bash_exec(mode='detach', ...)` and normally leave `timeout_seconds` unset for that long run
 - monitor through durable logs rather than only live terminal output
-- use `bash_exec(mode='list')` and `bash_exec(mode='read', id=...)` to monitor or revisit managed commands
+- use `bash_exec(mode='list')` and `bash_exec(mode='read', id=..., tail_limit=..., order='desc')` to monitor or revisit managed commands while focusing on the newest evidence first
+- after the first read, prefer `bash_exec(mode='read', id=..., after_seq=last_seen_seq, tail_limit=..., order='asc')` so later checks only fetch new evidence
+- if you need to recover ids or sanity-check the active session ordering, use `bash_exec(mode='history')`
+- launch important runs with a structured `comment` such as `{stage, goal, action, expected_signal, next_check}`
+- use `silent_seconds`, `progress_age_seconds`, `signal_age_seconds`, and `watchdog_overdue` from `bash_exec(mode='list'|'read', ...)` as your default watchdog signals
 - use an explicit wait-and-check loop such as:
   - wait about `60s`, then inspect logs
   - wait about `120s`, then inspect logs
@@ -387,9 +392,10 @@ For commands that may run longer than a few minutes:
   - wait about `600s`, then inspect logs
   - wait about `1800s`, then inspect logs
   - then keep checking about every `1800s` while the run is still active
-- if needed, use shell `sleep` between checks or an equivalent bounded `bash_exec(mode='await', id=..., timeout_seconds=...)`
+- if needed, use an explicit bounded wait such as `bash_exec(command='sleep 60', mode='await', timeout_seconds=70)` or `bash_exec(mode='await', id=..., timeout_seconds=...)` between checks
 - after every completed sleep / await cycle, inspect logs and send `artifact.interact(kind='progress', ...)` with the latest real status, latest evidence, the next checkpoint, and the estimated next reply time
 - after the first meaningful signal and then at real checkpoints (e.g., completion, or roughly every ~30 minutes if still running), keep those progress updates going rather than waiting silently
+- if the run is clearly invalid, wedged, or superseded, stop it with `bash_exec(mode='kill', id=..., wait=true, timeout_seconds=...)`; if it must die immediately, add `force=true`, record the reason, fix the issue, and relaunch cleanly
 - do not report completion until logs and output files both confirm completion
 
 Always preserve the managed `bash_exec` log and export it into the experiment artifact directory when the run artifact is written.
@@ -404,7 +410,7 @@ Long loops should emit structured progress markers rather than noisy raw progres
 - do not paste raw progress lines into summaries
 - when possible include `eta` in seconds and `next_reply_at` or `next_check_at` so web/TUI can show the next expected update
 
-If the codebase uses `tqdm` or similar tooling, disable or redirect the native bar and emit concise structured progress instead.
+If you control the code, prefer a throttled `tqdm`-style progress reporter for the run itself and pair it with concise structured `__DS_PROGRESS__` lines when feasible so monitoring remains machine-readable.
 
 ### 6. Validate the outputs
 
@@ -466,6 +472,22 @@ That call is responsible for writing:
 - evidence paths
 - changed files
 - relevant config paths when applicable
+- `evaluation_summary` with exactly these six fields:
+  - `takeaway`
+  - `claim_update`
+  - `baseline_relation`
+  - `comparability`
+  - `failure_mode`
+  - `next_action`
+
+Use `evaluation_summary` as the short structured judgment layer on top of the longer narrative fields:
+
+- `takeaway`: one sentence the next reader can reuse directly
+- `claim_update`: `strengthens`, `weakens`, `narrows`, or `neutral`
+- `baseline_relation`: `better`, `worse`, `mixed`, or `not_comparable`
+- `comparability`: `high`, `medium`, or `low`
+- `failure_mode`: `none`, `implementation`, `evaluation`, `environment`, or `direction`
+- `next_action`: the immediate route such as `continue`, `revise_idea`, `analysis_campaign`, `write`, or `stop`
 
 After `artifact.record_main_experiment(...)` succeeds, do not assume the same branch should absorb the next round by default.
 Interpret the measured result first, then either:

package/src/skills/finalize/SKILL.md

@@ -12,12 +12,12 @@ Use this skill to close or pause a quest responsibly.
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before closing or pausing the quest.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - If the runtime starts an auto-continue turn with no new user message, keep finalizing from the durable quest state and active requirements instead of replaying the previous user turn.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest finalize progress update before assuming the task changed completely.
 - When finalize reaches a real closure state, pause-ready packet, or route-back decision, send one threaded `artifact.interact(kind='milestone', ...)` update that names the recommendation, why it is the right call, and any reopen condition that still matters.
 - True quest completion still requires explicit user approval through the runtime completion flow before calling `artifact.complete_quest(...)`.
@@ -124,9 +124,12 @@ When a paper bundle exists, verify the manifest inventory explicitly, including:
 - referenced `writing_plan_path`
 - referenced `references_path`
 - referenced `claim_evidence_map_path`
+- referenced `baseline_inventory_path`
 - referenced `compile_report_path`
 - referenced `pdf_path`
 - referenced `latex_root_path`
+- `release/open_source/manifest.json` when open-source preparation has started
+- `release/open_source/cleanup_plan.md` when the paper line is being prepared for a public code release
 
 ### 2. Build the final claim ledger
 

package/src/skills/idea/SKILL.md

@@ -12,7 +12,7 @@ Use this skill to turn the current baseline and problem frame into concrete, lit
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before selecting or refining ideas.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Keep ordinary subtask completions concise. When the idea stage actually finishes a meaningful deliverable such as a selected idea package, a rejected-ideas summary, or a route-shaping ideation checkpoint, upgrade to a richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` report.
@@ -21,7 +21,7 @@ Use this skill to turn the current baseline and problem frame into concrete, lit
 - If the runtime starts an auto-continue turn with no new user message, keep advancing from the active requirements and current durable state instead of re-answering the previous user turn.
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest idea progress update before assuming the task changed completely.
 
 ## Stage purpose

package/src/skills/intake-audit/SKILL.md

@@ -12,12 +12,12 @@ Use this skill when the quest already has meaningful state and the first job is
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing the audit.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of the audit, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of the audit, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest intake-audit progress update before assuming the task changed completely.
 - When the audit reaches a durable route recommendation, send one richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` update that says what state is trusted, what still needs work, and which anchor should run next.
 

package/src/skills/rebuttal/SKILL.md

@@ -16,12 +16,12 @@ The task is “respond to concrete reviewer pressure with the smallest honest se
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing the rebuttal pass.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of the rebuttal pass, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of the rebuttal pass, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest rebuttal progress update before assuming the task changed completely.
 - When the rebuttal plan, the main supplementary-evidence package, or the final response bundle becomes durable, send one richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` update that says what reviewer concerns are now addressed, what still remains open, and what happens next.
 
@@ -87,11 +87,13 @@ Use, in roughly this order:
 - the current paper or draft
 - the selected outline if one exists
 - review comments, meta-review, or editor letter
+- the six-field `evaluation_summary` blocks from recent main experiments and analysis slices
 - recent main and analysis experiment results
 - prior decision and writing memory
 - existing figures, tables, and claim-evidence maps
 
 If the current paper/result state is still unclear, open `intake-audit` first before continuing the rebuttal workflow.
+Before launching any new supplementary experiment, read those structured `evaluation_summary` blocks first so the rebuttal plan starts from the already-recorded evidence state rather than from raw narrative memory.
 
 ## Core outputs
 

package/src/skills/review/SKILL.md

@@ -19,11 +19,11 @@ It is also not the same as `rebuttal`.
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing the review pass.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of the review pass, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of the review pass, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - When the review report, revision plan, or follow-up experiment TODO list becomes durable, send a richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` update that says what the main risks are, what should be fixed next, and whether the next route is writing, experiment, or claim downgrade.
 
 ## Purpose
@@ -77,12 +77,14 @@ Use, in roughly this order:
 - the current paper or report draft
 - the selected outline if one exists
 - the claim-evidence map if one exists
+- the six-field `evaluation_summary` blocks from recent main experiments and analysis slices
 - recent main and analysis experiment results
 - figures, tables, and captions
 - prior self-review or reviewer-first notes as low-trust auxiliary input
 - nearby papers when novelty or comparison is unclear
 
 If the draft/result state is still unclear, open `intake-audit` first before continuing the review workflow.
+Before proposing extra experiments, read those structured `evaluation_summary` blocks first so you do not request work that the recorded evidence already resolved.
 
 ## Core outputs
 

package/src/skills/scout/SKILL.md

@@ -12,12 +12,12 @@ Use this skill when the quest does not yet have a stable research frame.
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing scouting.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest scout progress update before assuming the task changed completely.
 - When scouting actually resolves the framing ambiguity, locks the evaluation contract, or makes the next anchor obvious, send one richer `artifact.interact(kind='milestone', reply_mode='threaded', ...)` update that says what is now clear, why it matters, and which stage should come next.
 

package/src/skills/write/SKILL.md

@@ -22,7 +22,7 @@ This skill intentionally absorbs the strongest old DeepScientist writing discipl
 - Treat `artifact.interact(...)` as the main long-lived communication thread across TUI, web, and bound connectors.
 - If `artifact.interact(...)` returns queued user requirements, treat them as the highest-priority user instruction bundle before continuing drafting or revision.
 - Immediately follow any non-empty mailbox poll with another `artifact.interact(...)` update that confirms receipt; if the request is directly answerable, answer there, otherwise say the current subtask is paused, give a short plan plus nearest report-back point, and handle that request first.
-- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` only when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or an occasional keepalive during truly long work. Do not update by tool-call cadence.
+- Emit `artifact.interact(kind='progress', reply_mode='threaded', ...)` when there is real user-visible progress: the first meaningful signal of long work, a meaningful checkpoint, or a concise keepalive if active work has drifted beyond roughly 10 to 30 tool calls without a user-visible update.
 - Prefer `bash_exec` for durable document-build commands such as LaTeX compilation, figure regeneration, and scripted export steps so logs remain quest-local and reviewable.
 - Keep progress updates chat-like and easy to understand: say what changed, what it means, and what happens next.
 - Default to plain-language summaries. Do not mention file paths, artifact ids, branch/worktree ids, session ids, raw commands, or raw logs unless the user asks or needs them to act.
@@ -56,7 +56,7 @@ This skill intentionally absorbs the strongest old DeepScientist writing discipl
 - If the runtime starts an auto-continue turn with no new user message, keep drafting or verifying from the durable state and active requirements instead of replaying the previous user turn.
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel respectful, human, and non-robotic.
 - Use `reply_mode='blocking'` only for real user decisions that cannot be resolved from local evidence.
-- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, wait up to 1 day when feasible, then choose the best option yourself and notify the user of the chosen option if the timeout expires.
+- For any blocking decision request, provide 1 to 3 concrete options, put the recommended option first, explain each option's actual content plus pros and cons, and wait up to 1 day when feasible. If the blocker is a missing external credential or secret that only the user can provide, keep the quest waiting, ask the user to supply it or choose an alternative, and do not self-resolve; if resumed without that credential and no other work is possible, a long low-frequency wait such as `bash_exec(command='sleep 3600', mode='await', timeout_seconds=3700)` is acceptable. Otherwise choose the best option yourself and notify the user of the chosen option if the timeout expires.
 - If a threaded user reply arrives, interpret it relative to the latest writing progress update before assuming the task changed completely.
 - Use milestone updates deliberately when outline selection, claim downgrades, proofing completion, bundle readiness, or route-back-to-experiment decisions become durably true.
 

package/src/tui/package.json

@@ -1,6 +1,6 @@
 {
   "name": "deepscientist-tui",
-  "version": "1.5.1",
+  "version": "1.5.3",
   "private": true,
   "type": "module",
   "main": "dist/index.js",