@researai/deepscientist 1.5.17 → 1.6.0

package/src/skills/analysis-campaign/references/artifact-flow-examples.md

@@ -0,0 +1,102 @@
+# Analysis Campaign Artifact Flow Examples
+
+Use this reference when the evidence question is clear but the exact artifact sequence is not.
+These are examples, not the only legal routes.
+The hard rule is: once a supplementary slice is launched as campaign work, its state must be recorded through `artifact.record_analysis_slice(...)`.
+
+## 1. One launched supplementary slice
+
+Use when one extra experiment is enough but lineage or reviewability matters.
+
+Typical sequence:
+
+1. `artifact.resolve_runtime_refs(...)` if ids are unclear
+2. `artifact.create_analysis_campaign(...)` with one slice
+3. run the returned slice in its returned workspace
+4. `artifact.record_analysis_slice(...)`
+5. record the route implication
+
+Good:
+
+- one-slice campaign still gives durable lineage and Canvas visibility
+
+Bad:
+
+- running the slice locally and only mentioning the result in chat
+
+## 2. Multi-slice evidence package
+
+Use when several slices together answer one evidence question.
+
+Typical sequence:
+
+1. define the currently justified slice frontier
+2. `artifact.create_analysis_campaign(...)`
+3. execute slices one by one in returned workspaces
+4. after each launched slice, call `artifact.record_analysis_slice(...)`
+5. aggregate only after slice-level evidence exists
+
+Good:
+
+- summary comes after slice records, not before
+
+Bad:
+
+- pretending planned slices count as evidence before they run
+
+## 3. Writing-facing slice
+
+Use when the slice directly supports the paper contract.
+
+Typical sequence:
+
+1. recover or inspect `selected_outline_ref` and paper matrix state when relevant
+2. `artifact.create_analysis_campaign(...)` with available paper-mapping fields
+3. run the slice
+4. `artifact.record_analysis_slice(...)`
+5. write back to matrix / ledger / section notes
+
+Good:
+
+- slice is durable and also write-backable
+
+Bad:
+
+- slice is completed but the paper contract still looks missing
+
+## 4. Failed or infeasible slice
+
+Use when the slice cannot complete honestly.
+
+Typical sequence:
+
+1. stop or mark the slice when blocked
+2. keep the real blocker visible
+3. `artifact.record_analysis_slice(...)` with non-success status
+4. route to redesign, `decision`, `experiment`, or stop
+
+Good:
+
+- non-success still becomes durable evidence
+
+Bad:
+
+- silently replacing a failed slice with a different slice and only reporting the later success
+
+## 5. Read-only bounded audit
+
+Use when no slice is launched and the answer comes from existing outputs, tables, logs, artifacts, or files.
+
+Typical sequence:
+
+1. inspect the existing evidence
+2. leave a durable report or decision
+3. do not pretend this was a launched campaign slice
+
+Good:
+
+- this avoids unnecessary campaign overhead
+
+Bad:
+
+- skipping `artifact.create_analysis_campaign(...)` for a real launched supplementary run and calling it a read-only audit afterward

package/src/skills/analysis-campaign/references/boundary-cases.md

@@ -0,0 +1,98 @@
+# Analysis Campaign Boundary Cases
+
+Use this reference when the evidence route is plausible but the success boundary, comparability boundary, or stage boundary is still fuzzy.
+
+## 1. This is really a new main experiment, not analysis
+
+Do not hide a new main experiment inside analysis-campaign when:
+
+- the proposed change defines a new main method
+- the comparison target itself is being replaced
+- the result would become the new primary measured line rather than a follow-up check
+
+Route back to `experiment` or `idea` instead.
+
+## 2. Non-comparable but still useful
+
+A slice can still be useful when it is not apples-to-apples comparable.
+
+Examples:
+
+- external dataset stress test
+- different metric family used only for failure analysis
+- altered protocol used to expose a limitation boundary
+
+Required behavior:
+
+- label the slice as non-comparable
+- do not mix it into the main comparison table as if it were direct support
+
+## 3. Qualitative or reviewer-example evidence
+
+This can be valid when:
+
+- the claim is about failure buckets, behavior, or explanation
+- the sample is concrete and scoped
+- the rubric or inspection basis is explicit
+- the evidence is used honestly as supporting or boundary evidence rather than fake objective measurement
+
+This is not valid when:
+
+- subjective inspection is presented as if it were a benchmark metric
+
+## 4. One slice is enough
+
+Stop after one slice when:
+
+- the claim boundary is already clear
+- the next route is obvious
+- extra slices would only add polish, not change the decision
+
+Do not widen the campaign just because more follow-ups are imaginable.
+
+## 5. Repeated failure with no new evidence
+
+Stop widening when:
+
+- the same failure class appears again
+- no route changed
+- no evidence changed
+- no execution environment changed
+
+At that point:
+
+- record the blocker
+- redesign the slice
+- route through `decision`
+- or return to `experiment`
+
+## 6. Writing-facing but pre-outline
+
+This can still be legitimate when:
+
+- the evidence question determines whether writing is even worth pursuing
+- no paper-ready claim is being finalized yet
+
+This becomes paper-ready only after it is mapped back to outline / matrix / ledger / section / claim artifacts.
+
+## 7. Extra comparator baseline inside analysis
+
+This is allowed when:
+
+- a slice genuinely needs an extra comparator
+- that comparator is analysis-local support rather than the new canonical quest baseline
+
+Required behavior:
+
+- keep the canonical quest baseline gate unchanged
+- record the extra comparator through `comparison_baselines`
+
+## 8. Stable support vs contradiction vs ambiguity
+
+Use these rough meanings:
+
+- stable support: slice materially strengthens the parent claim
+- contradiction: slice materially weakens or breaks the parent claim
+- unresolved ambiguity: slice leaves the decision unclear
+
+Do not blur these together in one optimistic summary.

package/src/skills/analysis-campaign/references/campaign-checklist-template.md

@@ -1,41 +1,56 @@
-# Analysis Campaign Checklist Template
+# Analysis Evidence Gate Checklist
 
-Use this as the living execution checklist for the campaign.
+Use this as a compact acceptance-boundary checklist when it helps.
+It is optional; the hard requirement is that launched slices, evidence boundaries, blockers, and next routes are durable and unambiguous.
 
 ## Identity
 
+- parent object:
+- parent claim or gap:
+- route:
 - campaign id:
-- parent run / idea:
 
-## Launch
+## Current Frontier
 
-- [ ] claim under test is explicit
-- [ ] selected outline is present when writing-facing
-- [ ] `PLAN.md` created
-- [ ] `CHECKLIST.md` created
-- [ ] slices prioritized by decision value
+- [ ] next slice, aggregation, blocker, or route decision is explicit
+- [ ] active uncertainty is written as a concrete question
+- [ ] next route is known if this gate clears or fails
 
-## Assets And Comparators
+## Resource Gate
 
-- [ ] required quest-local assets confirmed
-- [ ] extra baselines or comparators identified
-- [ ] fallback asset routes documented
+- [ ] current device, memory, storage, and wall-clock limits are explicit when they affect campaign design
+- [ ] planned slices have been screened as runnable-now, runnable-with-downscope, or blocked-by-resources
+- [ ] blocked high-value slices are recorded explicitly rather than silently dropped
+- [ ] the current frontier is prioritized by soundness gain under the real resource budget
 
-## Slice Execution
+## Evidence Gate
 
-- [ ] first claim-critical slice launched
-- [ ] smoke checks passed for long slices
-- [ ] monitoring cadence active
-- [ ] any slice redesign reflected in `PLAN.md`
-- [ ] each finished slice recorded durably
+- [ ] parent claim, paper gap, reviewer item, or decision being tested is explicit
+- [ ] each launched slice has a durable outcome or active monitoring path
+- [ ] evidence-bearing slices record question, intervention or inspection target, fixed conditions, metric or observable, and evidence path
+- [ ] claim update and comparability verdict are explicit
+- [ ] null, negative, partial, failed, blocked, or contradictory findings are visible
+- [ ] campaign-level interpretation is backed by per-slice evidence
 
-## Aggregation
+## Comparability Gate
 
-- [ ] stable support vs contradiction vs ambiguity classified
-- [ ] highest-impact slices summarized first
-- [ ] campaign report written
+- [ ] baseline or main comparison contract is preserved, or deviation is recorded
+- [ ] new dataset / split / metric / protocol changes are labeled as generalization, stress-test, boundary, or non-comparable
+- [ ] additional comparators do not overwrite the canonical quest baseline gate
+
+## Paper / Review Gate
+
+- [ ] paper-ready slices map to outline, paper matrix, evidence ledger, section, claim, table, reviewer item, or rebuttal item
+- [ ] if a paper-facing slice is complete, the write-back target is updated or the stale contract is recorded as a blocker
+
+## Blocked Boundary
+
+- [ ] if blocked, the failure class is explicit
+- [ ] if blocked, tried steps and evidence paths are recorded
+- [ ] if blocked, next best move is continue, redesign, return to experiment, return to idea, write, decision, stop, or reset
 
 ## Closeout
 
-- [ ] campaign outcome summarized in `1-2` sentences
+- [ ] strongest evidence boundary is summarized
+- [ ] main claim is classified as strengthened, weakened, narrowed, abandoned, or still ambiguous
 - [ ] next route recorded explicitly

package/src/skills/analysis-campaign/references/campaign-design.md

@@ -10,6 +10,8 @@ A strong campaign should move the evidence boundary from:
 - minimum -> solid
 - solid -> broader confidence
 
+It should do so with the highest soundness gain that still fits the current execution envelope.
+
 Do not treat every available follow-up as equally valuable.
 
 ## Priority order
@@ -21,6 +23,8 @@ Prefer this order:
 3. failure-mode explanation
 4. efficiency or secondary support
 
+Within the same priority tier, prefer the slice that is both runnable now and most likely to change the claim boundary.
+
 ## Slice classes
 
 - `auxiliary`
@@ -31,16 +35,6 @@ Prefer this order:
 - `supporting`
   - broadens confidence or interpretability after the main claim is already credible
 
-## Campaign question
-
-Each slice should answer one of these:
-
-- does the main claim still hold when X changes?
-- which component actually causes the gain?
-- where does the method fail?
-- how sensitive is the method to a controllable factor?
-- what boundary should be written into the paper?
-
 ## Writing-facing policy
 
 If the campaign is tied to a selected outline:
@@ -48,3 +42,25 @@ If the campaign is tied to a selected outline:
 - run the claim-carrying slices first
 - only then run supporting slices that deepen interpretation
 - route back to `write` once the evidence is strong enough for the selected narrative
+
+## Resource-aware design gate
+
+Before you expand a slice set, write down the current practical limits:
+
+- available machine class or devices
+- expected wall-clock budget
+- memory and storage limits
+- concurrency limits
+- environment or dependency risk
+
+Then tag each candidate slice as one of:
+
+- `runnable-now`
+- `runnable-with-downscope`
+- `blocked-by-resources`
+
+When resources are tight, optimize for soundness-per-cost:
+
+- prefer one decisive, runnable contradiction or robustness slice over several speculative expensive slices
+- use narrower sweeps, fewer seeds, shorter horizons, smaller held-out subsets, or cheaper diagnostics when they still answer the question honestly
+- record blocked high-value slices explicitly instead of letting them disappear

package/src/skills/analysis-campaign/references/campaign-plan-template.md

@@ -1,81 +1,80 @@
-# Analysis Campaign Plan Template
+# Analysis Evidence Record Template
 
-Use this for the campaign-level plan.
-Treat it as the durable version of the charter, not a separate optional memo.
+Use this only when a durable campaign record would reduce ambiguity.
+It is a reference template, not a required path or required filename.
+For `analysis-lite`, keep only the objective, first slice, evidence boundary, and next route.
+Expand only when the analysis is multi-slice, writing-facing, route-changing, expensive, unstable, or long-running.
 
 ## 1. Objective
 
-- campaign id:
-- parent run or idea:
-- main claim under test:
-- user's core requirements:
-- campaign outcome needed:
-- selected outline ref:
-- paper experiment matrix path:
-- current matrix execution frontier:
+- parent object:
+- parent claim or gap:
+- evidence question:
+- success condition:
+- stop or abandonment condition:
+- next route on support:
+- next route on contradiction / blocker:
 
-## 2. Boundary And Comparability
+## 2. Comparability Boundary
 
-- baseline comparison contract:
+- baseline or main comparison contract:
 - fixed conditions:
 - variables that may change:
-- non-comparable slices to watch for:
-
-## 3. Slice Plan
+- direct-comparison limits:
+- non-comparable slices to label explicitly:
 
-| Exp id | Slice id | Tier | Slice class | Experiment type | Research question | Expected value | Priority | Paper placement | Needs code change? | Needs extra baseline? |
-|---|---|---|---|---|---|---|---|---|---|---|
-| | | main_required / main_optional / appendix / optional | auxiliary / claim-carrying / supporting | ablation / sensitivity / robustness / efficiency / highlight / boundary / case-study | | | | main_text / appendix / maybe / omit | yes / no | yes / no |
+## 3. Slice Frontier
 
-## 4. Highlight Hypotheses
+| Slice id | Slice class | Evidence question | Intervention / inspection target | Metric / observable | Priority | Paper role |
+|---|---|---|---|---|---|---|
+| | auxiliary / claim-carrying / supporting / failure-analysis | | | | | main_text / appendix / reference_only / none |
 
-- highlight id:
-- one-line claim:
-- why it is plausible:
-- which slices validate or falsify it:
-- what happens if it fails:
-
-## 5. Assets And Dependencies
+## 4. Assets And Dependencies
 
 - quest-local assets already available:
 - checkpoints / baselines already available:
-- downloads or services still needed:
-- fallback options if external assets are blocked:
+- required comparators:
+- missing dependencies or credentials:
+- fallback if unavailable:
 
-## 6. Execution Strategy
+## 5. Execution Envelope
 
-- first slices to run:
-- smoke-test policy:
-- main run policy:
-- expected outputs:
+- available devices or machine class:
+- available memory:
+- available storage:
+- wall-clock budget:
+- concurrency or queue limits:
+- environment or dependency risk:
+- slices that are infeasible unless downscoped:
 
-Monitoring and sleep plan:
+## 6. Execution Choice
 
-- wait cadence:
-  - `60s`
-  - `120s`
-  - `300s`
-  - `600s`
-  - `1800s`
-- health signals that justify continued monitoring:
-- conditions that trigger slice redesign, kill, or campaign revision:
+- artifact-backed campaign needed:
+- working location:
+- environment or service route:
+- smoke / direct verification / real-run decision:
+- expected outputs or evidence paths:
+- monitoring plan if long-running:
 
-## 7. Reporting Plan
+## 7. Interpretation Boundary
 
 - what will count as stable support:
+- what will count as partial support:
 - what will count as contradiction:
 - what will count as unresolved ambiguity:
-- campaign summary should say in `1-2` sentences:
-- matrix refresh rule after every slice:
-- main-text gating rule:
+- what result would stop further slices:
 
-## 8. Checklist Link
+## 8. Paper Or Review Mapping
 
-- checklist path:
-- next unchecked item:
+- selected outline ref:
+- paper experiment matrix path:
+- evidence ledger path:
+- reviewer / rebuttal item:
+- write-back target:
 
-## 9. Revision Log
+## 9. Current Frontier
 
-| Time | What changed | Why it changed | Impact on slices or interpretation |
-|---|---|---|---|
-| | | | |
+- next action:
+- active blocker:
+- latest evidence:
+- next route:

package/src/skills/analysis-campaign/references/operational-guidance.md

@@ -0,0 +1,97 @@
+# Operational Guidance
+
+Use this reference when the analysis route needs the longer operational notes rather than the main control surface.
+
+## Artifact tactics
+
+Use `artifact.create_analysis_campaign(...)` when durable lineage or slice-level branch/worktree state matters.
+Even one extra experiment can still be represented as a one-slice campaign when durable lineage matters.
+Use a one-slice campaign when the slice should appear as a real child node in Git or Canvas, or when review, rebuttal, or paper traceability benefits from the campaign object.
+
+If `artifact.create_analysis_campaign(...)` returns slice worktrees, run each returned slice in its returned workspace unless there is a concrete reason to switch and record that reason.
+Branch that campaign from the current workspace or result node rather than mutating the completed parent node in place when lineage matters.
+Only create the campaign after you have verified that the listed slices are executable with the current quest assets and runtime, or explicitly mark infeasible slices as such.
+
+When the campaign is writing-facing, the create call should carry available paper-mapping fields such as `selected_outline_ref`, `research_questions`, `experimental_designs`, and `todo_items` when they exist and matter.
+If ids or refs are unclear, recover them first with `artifact.resolve_runtime_refs(...)`, `artifact.get_analysis_campaign(...)`, `artifact.get_quest_state(...)`, or `artifact.list_paper_outlines(...)` instead of guessing.
+Treat `campaign_id` as system-owned, and treat `slice_id` or `todo_id` as agent-authored semantic ids.
+
+After each launched slice finishes, fails, or becomes infeasible, call `artifact.record_analysis_slice(...)` or otherwise record the same durable truth through the artifact surface immediately.
+If a slice fails or becomes infeasible, still record an honest non-success status plus the real blocker and next recommendation; do not leave the campaign state ambiguous.
+
+For slice recording, `deviations` and `evidence_paths` are context fields, not mandatory ceremony; include them when they materially help explanation or auditability.
+An `evaluation_summary` is the preferred stable routing summary for UI, Canvas, review, and rebuttal.
+When useful, include these fields:
+
+- `takeaway`
+- `claim_update`
+- `baseline_relation`
+- `comparability`
+- `failure_mode`
+- `next_action`
+
+The longer prose still matters, but the summary should make the slice readable at a glance.
+
+## Resource gate
+
+Before launching a multi-slice campaign or any expensive slice, record the current execution envelope in the durable route record:
+
+- available GPUs, CPUs, memory, and storage
+- expected wall-clock budget
+- concurrency or queue limits
+- services, credentials, or dependencies that may block execution
+
+For each planned slice, decide explicitly:
+
+- runnable as written
+- runnable only after downscoping
+- infeasible in the current environment
+
+Do not keep an infeasible slice as if it were still on equal footing with runnable slices.
+Either replace it with a lower-cost proxy, defer it with a blocker note, or drop it.
+
+## Execution tactics
+
+Use whatever execution route is most faithful, observable, and efficient while preserving the hard gates.
+
+- A bounded smoke test is useful when the slice command, outputs, metric path, or evaluator wiring is uncertain.
+- Treat smoke work as a `0-2` default budget, not as an automatic mandatory phase.
+- If the path is already concrete, go straight to direct verification or the real slice.
+- If two candidate slices answer the same evidence question, prefer the one that preserves the most soundness under the current hardware and time budget.
+- If runtime is uncertain or likely long, prefer `bash_exec(mode='detach', ...)` plus managed monitoring.
+- `bash_exec(mode='read', id=...)` returns the full rendered log when it is 2000 lines or fewer; for longer logs it returns the first 500 lines plus the last 1500 lines and a hint to inspect omitted sections with `start` and `tail`.
+- If you need a middle section that was omitted from that default preview, use `bash_exec(mode='read', id=..., start=..., tail=...)`.
+- Monitor with `bash_exec(mode='read', id=..., tail_limit=..., order='desc')`.
+- After the first read, prefer `bash_exec(mode='read', id=..., after_seq=last_seen_seq, tail_limit=..., order='asc')` for incremental monitoring.
+- If ids become unclear, recover them through `bash_exec(mode='history')`.
+- Use `silent_seconds`, `progress_age_seconds`, `signal_age_seconds`, and `watchdog_overdue` as stall checks when they are available.
+- If a slice is invalid, wedged, or superseded, stop it with `bash_exec(mode='kill', id=..., wait=true, timeout_seconds=...)`.
+- If you only need wall-clock waiting between checks, use the canonical sleep choice:
+  - `bash_exec(command='sleep N', mode='await', timeout_seconds=N+buffer, ...)`
+  - do not set `timeout_seconds` exactly equal to `N`
+  - if you are waiting on an already running session, prefer `bash_exec(mode='await', id=..., wait_timeout_seconds=1800)` instead of starting a new sleep command
+  - if that bounded await returns while the session is still `running`, read the log and inspect real progress before deciding whether another `1800s` wait is justified
+- when you control the slice code, prefer a throttled `tqdm` progress reporter and concise structured progress markers when feasible
+- if the same failure class appears again without a real route or evidence change, stop widening the campaign and route through `decision`
+- if the same slice repeatedly fails because the environment cannot support it, stop retrying and redesign the slice set around the real resource envelope
+
+## Memory note
+
+Use memory only to avoid repeating known failures or to preserve reusable campaign lessons, not as a required step before every slice.
+
+Stage-start requirement:
+
+- begin an analysis campaign pass with `memory.list_recent(scope='quest', limit=5)` when resuming, reopening old command paths, or prior campaign lessons are likely to matter
+- run targeted `memory.search(...)` before launching or resuming slices when repeated failures, prior slice outcomes, or comparability caveats may affect the route
+
+Stage-end requirement:
+
+- if the campaign produced a durable cross-slice lesson, failure pattern, or comparability caveat, write at least one `memory.write(...)` before leaving the stage
+
+## Connector-facing campaign chart requirements
+
+- When a campaign result is promoted into a connector-facing chart, prefer restrained palettes such as `sage-clay` and `mist-stone`.
+- A useful `sage-clay` anchor for campaign visuals is `#7F8F84`.
+- Use color to separate campaign-critical slices from background slices, not to decorate every slice equally.
+- Keep the palette consistent with the system prompt instead of improvising a fresh theme per campaign.
+- Campaign visuals should make the main boundary change obvious even in compressed connector previews.

package/src/skills/analysis-campaign/references/writing-facing-slice-examples.md

@@ -1,6 +1,7 @@
 # Writing-Facing Slice Examples
 
 Use this reference when an analysis campaign is supporting a paper-like deliverable and each slice must bind to the paper contract.
+This is the fuller paper-ready form, not the minimum required shape for every non-paper or analysis-lite slice.
 
 ## Good writing-facing todo item
 
@@ -18,6 +19,11 @@ Use this reference when an analysis campaign is supporting a paper-like delivera
   "section_id": "analysis-mechanism",
   "item_id": "AN-ABL-001",
   "claim_links": ["C2"],
+  "analysis_role": "component ablation",
+  "reviewer_question": "Is the gain caused by the proposed component or by incidental extra computation?",
+  "target_display": "Main-text ablation table",
+  "main_or_appendix": "main_text",
+  "failure_interpretation": "If the gain remains unchanged, the paper should weaken the mechanism claim and treat the component as non-essential.",
   "completion_condition": "Show whether the central gain survives removal of the core component.",
   "why_now": "The draft cannot support the mechanism claim without this slice.",
   "success_criteria": "Produce a fair ablation under the accepted metric contract.",
@@ -38,28 +44,12 @@ Use this reference when an analysis campaign is supporting a paper-like delivera
 
 Why it is bad:
 
+- no `paper_role`
 - no `section_id`
 - no `item_id`
 - no `claim_links`
+- no `analysis_role`
+- no reviewer-facing question
+- no target display or failure interpretation
 - no paper placement
 - impossible to write back into the outline cleanly later
-
-## Case guide
-
-- Main claim support:
-  use `paper_role=main_text` and make the item part of `required_items`
-- Supporting but non-blocking evidence:
-  use `paper_role=appendix` and make the item part of `optional_items`
-- Useful but paper-excluded result:
-  keep the slice durable, but mark it `reference_only` or exclude it with a written reason in the matrix
-
-## Completion rule
-
-After `artifact.record_analysis_slice(...)`:
-
-1. the slice result must exist under the analysis worktree
-2. the mirror must exist under `experiments/analysis-results/`
-3. the evidence ledger must contain the corresponding `item_id`
-4. the selected outline section must show the updated row in `result_table`
-
-If step 3 or 4 is missing, the slice is not paper-ready yet.