@researai/deepscientist 1.5.17 → 1.6.0

package/src/skills/finalize/SKILL.md

@@ -7,6 +7,7 @@ skill_role: stage
 # Finalize
 
 Use this skill to close or pause a quest responsibly.
+The goal is to leave a stable stopping point and a clean resume path, not to keep rediscovering work that is already done.
 
 ## Interaction discipline
 
@@ -20,6 +21,33 @@ Use this skill to close or pause a quest responsibly.
 - Rechecking that the same bundle files still exist, or re-aligning status surfaces without changing the closure judgment, does not by itself count as a fresh milestone.
 - Hard execution rule: if this stage needs terminal work such as Git inspection, packaging checks, document builds, or file inspection, every such command must go through `bash_exec`.
 
+## Three-layer todo contract
+
+- keep quest-root `plan.md` as the top-level research map and closure map
+- keep workspace `PLAN.md` as the active finalize-node contract when closure work is multi-step
+- keep workspace `CHECKLIST.md` as the finalize execution frontier with one real in-progress item and a short `Next` list
+- if the frontier stops changing, stop nesting closure chores and make the route judgment explicit
+
+## Research-map role
+
+- `finalize` is the loop-closure controller, not automatic quest death
+- the finalize outcome should update quest-root `plan.md` to one of: `stop`, `park_and_continue_later`, or `publish_and_continue_from_new_incumbent`
+- every finalize handoff should leave explicit reopen conditions and, when relevant, the next loop candidate
+
+## Current-node plan and checklist
+
+When finalize work is multi-step, create or refresh:
+
+- workspace `PLAN.md` as the finalize-node contract
+- workspace `CHECKLIST.md` as the finalize execution frontier
+
+The finalize node should make explicit:
+
+- which closure state is currently most justified
+- what evidence still blocks closure
+- what reopen condition still matters
+- whether the next edge is stop, park, or a new loop
+
 ## Stage purpose
 
 The finalize stage should not pretend every line succeeded.
@@ -66,8 +94,16 @@ For paper-like deliverables, do not finalize while any of these remain true:
 - required main-text outline items are still unresolved
 - completed analysis remains unmapped into the paper contract
 - the active paper line still reports open supplementary work that is expected to block the manuscript
+- the current paper contract rows still fail to expose the main experiment or required analysis rows that the manuscript depends on
+- `artifact.validate_manuscript_coverage(detail='full')` does not report `submission_ready=true`
+- `artifact.validate_academic_outline(detail='full')` does not pass for the selected outline
+- `artifact.validate_manuscript_language(detail='full')` reports main-text wording blockers
+- the latest bundle is only a `draft_checkpoint` or `review_package`
 
 If the current paper-state blocker is not obvious from the existing files, call `artifact.get_paper_contract_health(detail='full')` before deciding whether finalize is legitimate.
+If the exact section rows, evidence rows, or experiment-matrix rows matter, call `artifact.get_paper_contract(detail='full')` before deciding whether finalize is legitimate.
+If a paper bundle exists, call `artifact.validate_manuscript_coverage(detail='full')` before treating the paper line as final.
+If a selected outline or draft exists, call `artifact.validate_academic_outline(detail='full')` and `artifact.validate_manuscript_language(detail='full')` before treating the paper line as final.
 If the active quest/runtime state is unclear after restart or long pause, call `artifact.get_quest_state(detail='summary')` first.
 If the exact latest `SUMMARY.md`, `status.md`, or active user requirement wording matters for closure, call `artifact.read_quest_documents(...)`.
 If earlier user/assistant continuity matters for whether the quest should really stop, call `artifact.get_conversation_context(...)` instead of guessing from prompt context alone.
@@ -290,9 +326,32 @@ Write to memory only when the lesson is reusable across quests, such as:
 - robust baseline lessons
 - durable writing or evaluation lessons
 
+If later continuation remains plausible, also write or refresh one compact checkpoint-style quest memory card that mirrors the live resume packet.
+That checkpoint-style memory should usually state:
+
+- current route
+- current active node such as the live paper line, accepted bundle pair, or active continue-later checkpoint
+- node history: which earlier node(s), report/decision pairs, or closure paths were superseded
+- strongest retained result or blocker
+- what not to reopen by default
+- next resume step
+- first files to read
+- reopen conditions
+
+Preferred tags usually include:
+
+- `stage:finalize`
+- `type:checkpoint-memory`
+- `type:status-clarification`
+- `route:<current_route>`
+- `node:<current_active_node>` when the node label is stable enough to reuse
+
+Use `references/checkpoint-memory-template.md` when helpful so the current node history stays explicit.
+
 Stage-end requirement:
 
 - if finalize produced a durable cross-quest lesson worth reusing later, write at least one `memory.write(...)` before leaving the stage
+- if the quest is stopping at continue-later / pause-ready rather than true completion, write or refresh one checkpoint-style quest memory card before leaving the stage
 
 Quest-specific closure state belongs in files and artifacts first, not only memory.
 
@@ -333,6 +392,7 @@ Use these references when you need a denser closure checklist:
 
 - `references/finalization-checklist.md`
 - `references/resume-packet-template.md`
+- `references/checkpoint-memory-template.md`
 
 ## Exit criteria
 
@@ -343,3 +403,5 @@ Exit the finalize stage once one of the following is durably true:
 - the limitations and recommendation are explicit
 - the stopping point is recorded through artifact
 - the claim ledger and package inventory are clear enough for later resumption or publication handoff
+
+A good finalize pass leaves the quest easier to reopen correctly than it was before finalization began.

package/src/skills/finalize/references/checkpoint-memory-template.md

@@ -0,0 +1,49 @@
+# Checkpoint Memory Template
+
+Use this when the quest is pausing, entering continue-later, or otherwise needs one compact memory card that says exactly where later turns should resume.
+
+## Recommended structure
+
+### 1. Current route
+
+- current route label
+- current stage / anchor
+- short closure or continue-later judgment
+
+### 2. Current active node
+
+- active paper line / run node / report-decision pair
+- why this is the authoritative checkpoint now
+
+### 3. Node history
+
+- predecessor node(s)
+- superseded completion, experiment, or writing paths
+- why the current checkpoint replaced them
+
+### 4. Strongest retained result or blocker
+
+- strongest still-valid result
+- or the dominant blocker
+- why it matters
+
+### 5. Do-not-reopen by default
+
+- experiments, branches, or closure paths that should stay closed unless new evidence appears
+- any explicit do-not-repeat rule
+
+### 6. Next resume step
+
+- the single best next step
+- why it dominates alternatives
+
+### 7. First-read files
+
+- `status.md`
+- `SUMMARY.md`
+- active resume / handoff packet
+- current manifest, report, or decision file that defines the live checkpoint
+
+### 8. Reopen condition
+
+- exact condition that would justify reopening an older node or switching away from the current checkpoint

package/src/skills/finalize/references/resume-packet-template.md

@@ -10,6 +10,13 @@ Use this when the quest is pausing rather than ending permanently.
 - present stage
 - final recommendation at pause time
 
+### 1A. Current node history
+
+- current active node
+- predecessor node(s)
+- superseded node(s) or closure paths
+- why the current node is the authoritative resume point
+
 ### 2. Accepted baseline
 
 - baseline id or path

package/src/skills/idea/SKILL.md

@@ -6,12 +6,162 @@ skill_role: stage
 
 # Idea
 
-Use this skill to turn the current baseline and problem frame into concrete, literature-grounded, testable directions.
+Use this skill to turn the current baseline and problem frame into concrete, literature-grounded, frontier-aware, testable directions.
+The goal is to choose the next executable research route, not to maximize brainstorming volume or reward shallow novelty.
 
 When `startup_contract.need_research_paper = false` and the quest already has a concrete optimization handle, `idea` may stop after selecting or seeding a direction and then hand off into `optimize` instead of insisting on the full paper-oriented ideation loop.
 In that algorithm-first case, `idea` should usually produce a small method-brief frontier and then defer candidate ranking, promotion, and bounded search to `optimize`.
 When doing that handoff, prefer the brief-shaping discipline later used by `optimize`: clarify the bottleneck and constraints, keep only a small differentiated `2-3` option slate, and hand off a recommended brief rather than a pile of loose intuitions.
 
+## Match signals
+
+Use `idea` when:
+
+- the accepted baseline and metric contract already exist, but the next route is still unresolved
+- the current line failed and the quest needs a new falsifiable direction
+- the problem is not "build a new module" but "decide what kind of route should be tried next"
+- the current bottleneck might be a mechanism problem, an objective mismatch, a measurement/evaluator problem, or an infrastructure constraint that changes what should be tested next
+
+Do not use `idea` when:
+
+- the baseline gate is still unresolved
+- the current board state is too stale or conflicting to say what the mainline actually is
+- the next step is already obviously `write`, `review`, or `finalize`
+
+If the current board cannot be compressed cleanly, route through `decision` or `intake-audit` before widening the frontier.
+
+## One-sentence summary
+
+Turn the current objective, board state, and bottleneck into a small differentiated frontier, then select the next falsifiable route.
+
+## Control workflow
+
+1. Write the objective contract.
+   Use `references/objective-contract-template.md`.
+   Make the real target, trusted proxies, false-progress signals, and hard constraints explicit before generating ideas.
+   Default durable path: `artifacts/idea/objective_contract.md`.
+2. Write the current board packet.
+   Use `references/current-board-packet-template.md`.
+   Compress the incumbent, latest decisive result, active blocker, and stale routes-to-ignore into one current state surface.
+   Default durable path: `artifacts/idea/current_board_packet.md`.
+3. Identify the important contradiction and plausible novelty source.
+   Use `references/high-value-idea-sourcing.md`.
+   Start from the most important unresolved contradiction, anomaly, bottleneck, or failure region rather than from a preferred mechanism.
+4. Run a broad, history-aware literature search before proposing serious ideas.
+   Use `references/related-work-playbook.md`, `references/research-history-playbook.md`, and `references/literature-survey-template.md`.
+   Cover direct in-domain frontier papers, foundational papers, strongest nearby competitors, and cross-domain papers whose mechanisms may translate into the current task.
+   If the runtime prompt explicitly enables cross-quest recall, follow that injected policy before going outside; otherwise stay inside the current quest's memory/artifacts and explicit user-provided files. See playbook §2.1 for the full source-order protocol.
+   If DeepXiv is available, use it for broad paper-centric discovery and citation expansion; otherwise use search engines and citation chaining directly.
+   Do not promote or even seriously shortlist a new idea until the durable survey and closest-prior-work comparison are updated enough to judge novelty and feasibility honestly.
+5. Extract the limitation pattern and novelty opportunity from the survey.
+   Distinguish what is already saturated, what is only a decorative tweak, and what could still support a differentiated route.
+   Default against small local edits unless they are explicitly shown to be the highest-value surviving route.
+6. Choose the idea family mix.
+   At minimum, decide whether the current pass should consider some mix of:
+   - mechanism-family routes
+   - objective-family routes
+   - measurement-family routes
+   - infrastructure-family routes
+7. Run bounded brainstorming.
+   Use `references/controlled-brainstorming-playbook.md` when the route is not already obvious.
+   Generate a small, meaningfully different slate rather than a pile of micro-variants.
+   Prefer candidate families that could change the conclusion, capability boundary, or paper value materially, not just move a knob.
+8. Write a compact pre-idea draft for the serious surviving candidates.
+   Use `references/pre-idea-draft-template.md`.
+   Normally write drafts for the top `1-3` candidates, not for the whole raw slate.
+   The draft must surface hidden assumptions, local-optimum lock-in risk, strongest outside-family alternative, strongest rejection case, and the cheapest falsification path before any formal idea submission.
+   Default durable path per candidate: `artifacts/idea/pre_idea_drafts/<candidate_id>.md`.
+9. Filter aggressively.
+   Use `references/selection-gate.md`.
+   Remove candidates that only improve a surrogate, reopen a stale route without new evidence, violate leakage or submission-time boundaries, or lack a cheap falsification path.
+10. Select and hand off.
+   The selected package must include the route, why now, novelty type, main risk, anti-win condition, core hypothesis, mechanism sketch, strongest falsification experiment, minimal validation, abandonment condition, and the next stage.
+
+## Draft-before-submit SOP
+
+Before a direction is formally submitted as the selected idea, write a compact pre-idea draft or equivalent durable challenge memo for each serious surviving candidate.
+
+The default rule is:
+
+- raw brainstorming can widen the frontier
+- pre-idea drafts narrow and stress-test the frontier
+- only then can a final selected idea be submitted
+
+The pre-idea draft exists to stop three failure modes:
+
+- local-optimum lock-in around the current mainline
+- hidden assumptions staying implicit
+- attractive ideas being promoted before the strongest rejection case is written down
+
+Unless there is already an up-to-date equivalent artifact, do not formally submit the final idea until at least one pre-idea draft has:
+
+- been written for the likely winner
+- been compared against the incumbent and at least one outside-family or assumption-reversal alternative
+- been revised, rejected, or promoted based on that comparison
+
+Default durable path rules:
+
+- objective contract: `artifacts/idea/objective_contract.md`
+- current board packet: `artifacts/idea/current_board_packet.md`
+- candidate frontier summary: `artifacts/idea/candidates.md`
+- pre-idea draft per serious candidate: `artifacts/idea/pre_idea_drafts/<candidate_id>.md`
+- final selected idea: `artifacts/idea/selected_idea.md`
+
+When a candidate is promoted, `artifacts/idea/selected_idea.md` should point back to the winning pre-idea draft path instead of losing that lineage in prose only.
+
+## AVOID / pitfalls
+
+- Do not start from swapping method A for method B before naming the important contradiction or bottleneck.
+- Do not brainstorm before the real objective and false-progress signals are explicit.
+- Do not treat lower loss, better average surrogate, or cleaner intermediate metrics as route health if the real target is unchanged.
+- Do not reopen stale routes unless new evidence explicitly weakens the current mainline.
+- Do not generate a large within-family variant swarm before the mechanism family itself is chosen.
+- Do not propose an idea as "new" before the direct-field and adjacent transferable literature have both been checked.
+- Do not default to cosmetic modifications, parameter nudges, or tiny architecture swaps unless the survey and bottleneck analysis show they are genuinely the best surviving route.
+- Do not let the current mainline, favorite mechanism, or easiest implementation path lock the search into a local optimum while the hidden assumptions behind that line remain unchallenged.
+- Do not jump from brainstorming notes straight to final idea submission without a compact draft that forces the hidden assumptions, strongest rejection case, and falsification path into the open.
+- Do not treat novelty as “totally unprecedented”; it may come from a new problem, view, mechanism, method, setting, evaluation, or boundary condition.
+- Do not promote a direction that fails a value/feasibility screen simply because it sounds exciting.
+- Do not promote a direction without a cheap falsification path and a visible anti-win condition.
+
+## Constraints
+
+- Keep the accepted dataset, metric, and evaluation contract fixed unless scope explicitly changed.
+- Do not propose routes that depend on submit-time unavailable features.
+- Do not propose routes that introduce leakage-prone targets or labels into training.
+- Do not let implementation convenience outrank target alignment.
+- Search should be broad enough to map the main paradigms, history, and strongest overlaps, not just skim a few recent papers.
+- Search must cover both the current field's strongest direct papers and adjacent or cross-domain papers whose mechanisms may translate into the current task, evaluator, or systems setting.
+- If DeepXiv is available, prefer it for broad paper-centric discovery; otherwise use search engines, citation chaining, and open-web search directly.
+- Serious idea generation should happen only after the survey is broad enough to rule out obvious rediscoveries and to reveal non-trivial opportunity gaps.
+- A serious candidate should be explainable in terms of importance, novelty type, feasibility, verification path, and failure value.
+- By default, prefer routes with step-change or boundary-changing potential over small local refinements.
+- Small refinements are allowed only when the literature and current evidence indicate they are still the highest-value route and the reason is stated explicitly.
+- In system optimization work, a valid idea may be a mechanism change, an objective/evaluator correction, a measurement fix, or an infrastructure change if that is what best improves the real target.
+
+## Validation
+
+Before the idea pass can end, the durable selected idea package should make explicit:
+
+- the important contradiction, gap, anomaly, or bottleneck it is targeting
+- the literature coverage used to justify the route, including direct-field papers and adjacent transferable papers
+- the dominant novelty type
+- the targeted limitation
+- the real objective and the strongest false-progress signal
+- the pre-idea draft or equivalent challenge memo that preceded promotion
+- the selected direction and why it won now
+- why the selected route is not merely a decorative tweak relative to the closest prior work or current baseline
+- the value/feasibility screen or equivalent judgment
+- the core hypothesis
+- the mechanism sketch
+- the strongest falsification experiment
+- the anti-win condition
+- the minimal validation
+- the abandonment condition
+- the next stage
+
+If those fields are still fuzzy, continue ideation or route back through `decision` rather than pretending the route is ready.
+
 ## Interaction discipline
 
 - Follow the shared interaction contract injected by the system prompt.
@@ -23,6 +173,38 @@ When doing that handoff, prefer the brief-shaping discipline later used by `opti
 - Message templates are references only. Adapt to the actual context and vary wording so updates feel natural and non-robotic.
 - If a threaded user reply arrives, interpret it relative to the latest idea progress update before assuming the task changed completely.
 
+## Three-layer todo contract
+
+- keep quest-root `plan.md` as the research map for the whole quest loop
+- keep workspace `PLAN.md` as the active idea-node contract when ideation is multi-step, literature-heavy, or route-sensitive
+- keep workspace `CHECKLIST.md` as the active ideation frontier with one real in-progress item and a short `Next` list
+- if the execution frontier stops changing across repeated passes, revise the node contract or the research map instead of nesting more substeps
+
+## Research-map role
+
+- `idea` selects or refreshes the next route within the current loop; it does not replace the whole quest roadmap
+- when an idea is selected, rejected, or downgraded, update quest-root `plan.md` so the next experiment node or fallback decision node is explicit
+- when a strong result later becomes the new incumbent, the next idea pass should open a new loop entry in quest-root `plan.md` rather than drifting into ad hoc brainstorming
+
+## Current-node plan and checklist
+
+When ideation becomes multi-step, create or refresh:
+
+- workspace `PLAN.md` as the current idea-node contract
+- workspace `CHECKLIST.md` as the ideation frontier
+
+The idea node should make explicit:
+
+- which bottleneck is being attacked now
+- which candidate families are still live
+- what selection gate must be cleared before experiment
+
+Before widening the frontier, the node should also make explicit:
+
+- the current objective contract
+- the current board packet
+- which candidate-family mix is actually being explored in this pass
+
 ## Stage purpose
 
 The idea stage should not generate vague inspiration.
@@ -34,9 +216,10 @@ It should produce executable hypotheses tied to:
 - the strongest relevant prior work
 
 This stage is not just "brainstorming".
-It is the research-direction selection stage.
+It is a controlled brainstorming plus route-selection stage.
 It still needs a bounded creative-divergence phase before convergence.
 Do not collapse onto the first plausible route just because it sounds implementable.
+Do not settle for a low-amplitude tweak when a broader, still-feasible route remains live.
 It should normally create a new candidate direction branch and node; it does not by itself decide the next optimization round.
 The output must survive three checks at once:
 
@@ -44,11 +227,13 @@ The output must survive three checks at once:
 - feasibility in the current repo and resource budget
 - manuscript defensibility if the line later becomes a paper claim
 
+When multiple routes survive, prefer the most differentiated route that is still falsifiable and executable in the current repo, rather than the easiest tiny patch.
+
 When the route already looks likely to become a paper-facing line, seed one lightweight structured outline candidate during idea work.
 Use `artifact.submit_paper_outline(mode='candidate', ...)` for that seed instead of leaving the future paper structure only in prose.
 Use `references/outline-seeding-example.md` for the minimum acceptable shape.
-The idea-stage outline candidate is not the full paper line yet, but it should already name the likely `research_questions`, `experimental_designs`, and the first section-level evidence needs that later supplementary slices must satisfy.
-Keep that seed minimal and executable: a small section skeleton plus expected evidence items is better than a long narrative outline with no concrete evidence hooks.
+The idea-stage outline candidate is not the full paper line yet, but it should already name the likely one-sentence paper idea, scoped claims, `research_questions`, `experimental_designs`, and the first section-level evidence needs that later supplementary slices must satisfy.
+Keep that seed minimal and executable: a short `paper_view` plus expected evidence items is better than a long narrative outline with no concrete evidence hooks.
 If the current research head, strongest measured branch, or active runtime refs are unclear after resume, call `artifact.get_quest_state(detail='summary')` and `artifact.list_research_branches(...)` before choosing a foundation.
 If the current brief / plan / status wording matters for direction choice, call `artifact.read_quest_documents(...)`.
 If earlier user conversation materially changes the direction-selection target, call `artifact.get_conversation_context(...)` before locking the next idea.
@@ -81,7 +266,7 @@ At the direction level, prefer elegant algorithmic or theoretical improvements o
 This stage should preserve the strongest old DeepScientist direction-selection logic:
 
 - understand the baseline and its failure modes
-- search related work broadly before claiming an idea is good
+- search related work broadly before claiming an idea is good, including adjacent fields with translatable mechanisms
 - derive limitations
 - produce a compact set of candidate ideas from an explicit direction set
 - rank them with explicit tradeoffs
@@ -101,6 +286,8 @@ Candidate sets should usually cover some mix of:
 - a strong local refinement of the incumbent
 - an orthogonal alternative that addresses the same bottleneck differently
 - a cleaner or more defensible route with lower conceptual complexity
+- an objective/evaluator fix when the current route may be optimizing the wrong thing
+- an infrastructure or throughput fix when measurement cost itself is blocking useful iteration
 
 Do not default to “run a small experiment and see” as the way to break ties.
 Break ties primarily through careful reasoning over:
@@ -117,26 +304,30 @@ Break ties primarily through careful reasoning over:
 - Do not select an idea before checking whether close prior work already did it.
 - Do not confuse "I can implement this" with "this is a publishable or useful research direction".
 - Do not treat a weak literature search as sufficient because the idea sounds elegant.
-- Do not write, promote, or submit a final idea until the durable survey covers at least `5` and usually `5-10` task-modeling-related, mechanism-relevant, or otherwise directly usable papers.
-- Treat that literature floor as a hard gate, not a suggestion.
-  If the direct task-modeling neighborhood truly contains fewer than `5` usable papers, record that evidence explicitly and fill the remaining slots with the closest adjacent papers whose mechanism can be translated into the current task and codebase.
+- Do not start serious ideation from memory or taste alone; refresh the external literature unless the existing survey already covers the needed frontier and that reuse is recorded explicitly.
+- Do not treat the current field as the only search space; check cross-domain mechanism transfer whenever the bottleneck might admit it.
+- Do not promote a small tweak by default; if an incremental route wins, record why broader routes failed on novelty, feasibility, or claim value.
+- For paper-ready idea packages, aim for a durable survey that usually covers at least `5` and often `5-10` task-modeling-related, mechanism-relevant, or otherwise directly usable papers.
+- If the direct task-modeling neighborhood truly contains fewer than `5` usable papers, record that evidence explicitly and fill the remaining coverage with the closest adjacent papers whose mechanism can still be translated into the current task and codebase.
 - Algorithm-first exception:
   - when `startup_contract.need_research_paper = false` and a concrete optimization handle already exists, you may stop after a memory sweep plus a small targeted paper check instead of satisfying the full `5-10` paper floor
   - use that exception only when the immediate goal is method-brief selection for `optimize`, not paper-level novelty claims
   - if you use the exception, say explicitly that the output is an optimization brief frontier rather than a paper-ready idea package
   - still shape that frontier deliberately: clarify the bottleneck and comparability boundary first, keep a differentiated `2-3` candidate slate, and explain why one brief is recommended now
-- Every fresh idea build or idea-refinement pass must begin with:
+- Every fresh idea build or idea-refinement pass should begin with:
   - a memory sweep, and
-  - an external literature sweep.
-- Every fresh or resumed idea pass must update `artifacts/idea/literature_survey.md` or an equivalent durable survey report before a direction is promoted.
+  - an external literature sweep or a clear reason why the existing survey is already sufficient.
+- For paper-ready promotion, refresh `artifacts/idea/literature_survey.md` or an equivalent durable survey report before the direction is promoted.
 - Every survey update must explicitly separate:
   - reused prior survey coverage
   - newly added papers or comparisons from this pass
   - still-missing or unresolved overlaps
 - When a web/search tool is available, actively use it.
   Prefer web search for paper discovery, usually targeting arXiv first, then expand with citation and open-web search for neighborhood coverage.
+- If DeepXiv is declared available by the system prompt, prefer the DeepXiv route for paper-centric discovery and shortlist paper triage before broad open-web search.
+- If DeepXiv is declared unavailable, do not try to force it; stay on the legacy route.
 - When a concrete arXiv paper needs to be read, compared, or summarized, use `artifact.arxiv(paper_id=..., full_text=False)`.
-  Keep search in web discovery; use `artifact.arxiv(...)` for reading shortlisted papers, and set `full_text=True` only when needed.
+  Keep search in web discovery by default; use `artifact.arxiv(...)` for reading shortlisted papers, and set `full_text=True` only when needed.
 - Before opening a broad new search, check quest and global memory with `memory.search(...)` and reuse existing paper notes, idea notes, and knowledge cards.
 - Search for genuinely missing, newly relevant, or more recent papers whenever possible.
   Do not rerun the same broad search without stating what gap the new search is meant to close.
@@ -212,6 +403,7 @@ Do not skip the `scout` pass just because the quest is already in the `idea` sta
 
 Use `references/idea-thinking-flow.md` when the main need is better reasoning hygiene.
 Use `references/idea-generation-playbook.md` when the main need is to create a new idea slate and select one clear next research object.
+Use `references/high-value-idea-sourcing.md` when the main need is to identify a truly important contradiction or bottleneck before widening.
 
 Default creation flow for a fresh idea pass:
 
@@ -283,10 +475,20 @@ If durable quest memory already contains a recent and explicit survey, reuse it
 For a normal selected-idea decision, the durable sweep must end with at least `5` and usually `5-10` papers that are close enough to the task-modeling problem, failure mode, mechanism, or codebase translation question to inform the actual design.
 This floor exists to prevent thin novelty claims and under-motivated ideas, not to reward quota chasing.
 
+Do not treat “recent papers” as a substitute for “the field history”.
+At minimum, map:
+
+- seminal or foundational papers
+- turning-point or paradigm-shift papers
+- current mainstream or SOTA papers
+
+Then use citation chaining to reconstruct how the question evolved and where the real breakpoints still are.
+
 When tools allow it, combine:
 
 - `memory.search(...)` and recent memory reads
-- web search for arXiv and adjacent sources
+- DeepXiv for broad paper-centric discovery and citation expansion when available
+- otherwise web search for arXiv and adjacent sources
 - `artifact.arxiv(paper_id=..., full_text=False)` for actually reading shortlisted papers
 - citation expansion or open-web search for follow-up papers, code, and comparisons
 
@@ -338,6 +540,8 @@ If the search is still too thin to support a novelty or value judgment, the idea
 
 The idea stage should usually leave behind:
 
+- an objective contract
+- a current board packet
 - a limitations analysis
 - a literature survey report
 - a survey-delta section that marks:
@@ -349,6 +553,7 @@ The idea stage should usually leave behind:
 - `2-5` candidate ideas, with the final serious frontier usually narrowed to `2-3`
 - a selected idea or explicit rejection of the current line
 - a durable Markdown idea draft that is finalized before the accepted idea is submitted
+- one pre-idea draft per serious surviving candidate, usually `1-3`
 - one or more memory cards for reusable rationale
 - one or more quest `papers` cards for the strongest papers or search clusters
 - an idea artifact and a decision artifact
@@ -380,14 +585,20 @@ Do not force this structure for every tiny ideation turn, but use it when the qu
 
 Recommended durable files:
 
+- `artifacts/idea/objective_contract.md`
+- `artifacts/idea/current_board_packet.md`
 - `artifacts/idea/literature_survey.md`
 - `artifacts/idea/related_work.md`
 - `artifacts/idea/limitations.md`
 - `artifacts/idea/candidates.md`
+- `artifacts/idea/pre_idea_drafts/<candidate_id>.md`
 - `artifacts/idea/selected_idea.md`
 - `artifacts/idea/research_outline.md`
 
 When producing the literature survey report, prefer the structure in `references/literature-survey-template.md`.
+When writing the objective contract, prefer `references/objective-contract-template.md`.
+When writing the current board packet, prefer `references/current-board-packet-template.md`.
+When the route needs a bounded but real creative-divergence pass, prefer `references/controlled-brainstorming-playbook.md`.
 
 When producing a full research-outline style note, prefer the detailed structure in `references/research-outline-template.md`.
 
@@ -1225,9 +1436,9 @@ Preferred artifact choices:
 - use `approval` when the user explicitly confirms a preference-sensitive choice
 - use `milestone` when ideation hits a meaningful user-visible checkpoint
 
-If the idea is selected and becomes the active route, immediately call `artifact.submit_idea(mode='create', lineage_intent='continue_line'|'branch_alternative', ...)`.
+If the idea is selected and becomes the active durable route, normally call `artifact.submit_idea(mode='create', lineage_intent='continue_line'|'branch_alternative', ...)`.
 Before that call, first finalize a concise but durable Markdown draft for the chosen route.
-Do not start writing that final draft until the literature survey has already met the hard minimum of at least `5` and usually `5-10` usable papers.
+For a paper-ready idea package, do not finalize that draft until the literature survey is broad enough to support the route credibly; for an execution-brief handoff, a smaller targeted survey can be enough.
 That draft should usually cover:
 
 - executive summary
@@ -1284,3 +1495,5 @@ Do not exit this stage with a "selected idea" if:
 - the novelty / value verdict is still hand-wavy
 - the falsification path is unclear
 - the experiment handoff contract is incomplete
+
+A good idea pass ends with one route the next stage can actually run, or one explicit reason why no route is ready yet.

package/src/skills/idea/references/controlled-brainstorming-playbook.md

@@ -0,0 +1,78 @@
+# Controlled Brainstorming Playbook
+
+Use this reference when the current route is not already obvious and `idea` needs a real divergence pass.
+
+The goal is not open-ended ideation.
+The goal is a bounded, differentiated slate that is broad enough to avoid premature convergence and narrow enough to remain auditable.
+
+## 1. Enter only after framing
+
+Before brainstorming, make sure both of these already exist:
+
+- an objective contract
+- a current board packet
+
+If either is still fuzzy, do not widen yet.
+
+## 2. Choose the family mix first
+
+Decide which route families are allowed in this pass:
+
+- `mechanism_family`
+  - new algorithmic or modeling idea
+- `objective_family`
+  - change the training target, ranking target, or optimization target
+- `measurement_family`
+  - change the evaluator, validation lens, or what is being trusted
+- `infrastructure_family`
+  - change throughput, batching, staging, or other system constraints that affect useful iteration
+
+Do not default to mechanism-family routes only.
+
+## 3. Generate a bounded slate
+
+Default target:
+
+- `6-12` raw ideas
+- collapse to a serious frontier of `2-3`, and at most `5`
+
+Require visible differentiation:
+
+- one local refinement of the incumbent
+- one orthogonal alternative
+- one route that changes the objective or measurement layer when that layer may be wrong
+- one infrastructure or iteration-speed route when throughput itself is blocking progress
+
+## 4. Filter aggressively
+
+Discard or downgrade candidates that:
+
+- only improve a surrogate without changing the real objective
+- violate hard constraints
+- are just within-family micro-variants
+- reopen stale routes without new evidence
+- have no cheap falsification path
+
+## 5. Force `why now`
+
+Every serious candidate must answer:
+
+- what changed?
+- why now?
+- why this family instead of the current mainline?
+
+If the answer is weak, the candidate should usually not survive.
+
+## 6. End with a structured selection
+
+The final serious candidates should each include:
+
+- family type
+- targeted limitation
+- why now
+- strongest prior-work overlap
+- anti-win condition
+- minimal validation
+- abandonment condition
+
+The selected route should beat the others on evidence-per-run, not just novelty theater.